Consider the below scenario:

You are a staff of a company called ABCD Ltd which has its HQ in in Sydney, Australia. Part of the call center system and support services have been outsourced to a company called XYZ Ltd in Singapore. This means L2 support services are being offered by staff out of Singapore. Being a different company, XYZ has setup its own s3 bucket where all L2 incident reports are stored. Because this s3 bucket is in a different region, you want all objects to be also replicated to your s3 bucket, established at the HQ in Sydney. This means you need cross region replication across different accounts (root account for ABCD Ltd is different than XYZ Ltd). In addition, only authorized IAM users from your company (ABCD) are granted permission to view the content of s3 bucket in Singapore as well as copy the objects to their local machine.

In this post, I will address this specific s3 bucket requirements.

To make this system work, we need to carry out the below steps.

1. Create a source s3 bucket – this will be the s3 bucket in Singapore where new objects will be added
2. Create a destination s3 bucket – this will be s3 bucket in Sydney where new objects from Singapore will be replicated
3. Create a IAM role under the XYZ Ltd account – this role will be granted permission to do the replication from source [in Singapore] to destination [in Sydney] bucket; without this role, replication across different region cannot work
4. Create replication rules for the source s3 bucket in Singapore – this should setup the replication process.
5. Update the source s3 bucket policy to allow the IAM user permission to copy objects from source s3 bucket in Singapore to local machine in Sydney

Step 1: Create source s3 bucket

Create a source s3 bucket in Singapore region [ap-southeast-1] under the root account for XYZ Ltd. Leave Block Public Access as Blocked. No other changes required at the moment.

Step 2: Create destination s3 bucket

Create a destination s3 bucket in Sydney region [ap-southeast-2] under any account for ABCD Ltd. Leave Block Public Access as Blocked. No other changes required at the moment.

Step 3: Create an IAM role in the Sydney Region

The purpose of this role called s3_replication_role (created under XYZ Ltd in Singapore region) is to do the replication (copy objects) from source s3 bucket in Singapore to destination s3 bucket in Sydney. Without this role, the replication cannot proceed. When creating the IAM role, administrator access needs to be provided on the s3 services. Since replication will happen from Singapore s3 bucket to Sydney s3 bucket, the IAM role needs permission on the Singapore s3 bucket policy.

Step 4: Create Replication Rules for the Source S3 bucket

Open the source s3 bucket rubes-s3-bucket, go to Management and scroll down to Replication rule and click Create replication rule and setup replication rule as below.

Points to note:

• The replication rule has been set to all objects in the source s3 bucket
• For the destination bucket, account ID used for Sydney region has been provided
• Correct IAM role has been provided
• Replication Time Control (RTC) has been checked so that replication of objects [including delete markers] can happen within 15 min of the action

Step 5: Update Source S3 bucket policy

We need to update the bucket policy for rubes-s3-bucket so that IAM user “tom-cruise” is given access to certain actions. Using the JSON Policy Generator, create the below bucket policy and paste it in the Bucket policy section.

Note that this step is not required if there is no plan for the IAM user to copy the objects from the source s3 bucket in Singapore to their local machine in Sydney. The IAM user will need to use CLI commands to copy the objects from source bucket to local machine. This cannot be done over the AWS console.

As you can see from the JSON script, the policy allows the IAM user “tom-cruise” (shown here by the user ARN: arn:aws:iam::271089868719:user/tom-cruise) to execute actions such as GetObject and ListBucket on the source bucket (shown here by the ARN: arn:aws:s3:::rubes-s3-bucket).

Now login as any IAM user for XYZ Ltd in Singapore region and add an object to the rubes-s3-bucket. In a different browser AWS console, login as “tom-cruise” and open the destination bucket clint-s3-bucket and refresh the Object list. Within a very short period [within 15 min], the object should be replicated in the clint-s3-bucket. Look at the last modified column!

Now open the command prompt in administrator mode and type “aws configure”. The purpose of this is to establish a connection between your local machine in Sydney and the default region (ap-southeast-2). As “tom-cruise”, you would need your AWS Access ID, AWS Secret Access Key, default region and default output format to establish the connection.

As can be seen, the user “tom-cruise” is able to view the content of the source s3 bucket in Singapore as well as copy the “test.txt” object to the download folder of his local machine in Sydney.

Now repeat the same process but this time establish a connection for the user “clint-eastwood” to AWS Sydney region. Try to view the content of the rubes-s3-bucket. You cannot! ACCESS DENIED. Why is that? Because the user “clint-eastwood” has not been allowed to view objects [GetObject] in the bucket policy. Access was granted to the IAM user “tom-cruise” only.

To test this solution, consider the below scenarios:

• When replication happens, the objects are stored in the destination bucket as encrypted [at rest]. Are you able to open the objects to view?
• Only certain prefix objects in the source bucket will be replicated across the regions, not all objects.
• Only certain prefix objects that are uploaded by certain IAM users in XYZ Ltd will trigger the replication process.
• When CloudWatch saves application logs in a s3 bucket, replication rule makes those logs get saved in a backup s3 bucket [either same or different region]