Configuring S3 for Cross-Account File Transfer

< All Topics

Summary

BLAM3 can leverage S3’s bucket and user policies to allow cross-account S3 file transfers. A common scenario requiring this functionality is when delivering content to a 3rd party as part of a BLAM Work Order or other automated workflows. The following document details the minimum policies required to successfully transfer files to a 3rd party AWS account outside of the configured BLAM S3 buckets (BLAM storage locations).

Usage

In general, AWS S3 controls access to a bucket through the application of policies applied to users and/or the bucket directly. In this way, access can be granularly applied to the owning AWS account’s users but also extended to 3rd party AWS account’s users. To allow the transfer of a file into a bucket from another account, two sets of policies are needed to in place:

1. User Policy – required to allow the transfer of files from the source bucket to the defined destination bucket
2. Bucket Policy – required on the destination bucket to allow the receipt of files from the sending account or user

For transferring files from a BLAM S3 bucket as the source, the user policy needs to be applied to the AWS user account that was configured for BLAM’s access to the bucket. The bucket policy is applied to the bucket on the 3rd party AWS account. Files transferred to S3 buckets outside of BLAM’s configured S3 buckets are unmanaged by BLAM.
The same principals can be applied in reverse to allow 3rd party AWS accounts to upload directly to BLAM S3 buckets and when coupled with SNS Notifications, the automatic ingesting of new assets into BLAM.

User Policies

The user policy is used to give the assigned users the permission to transfer files from the source bucket on the same AWS account to the destination bucket on the 3rd party AWS account. Either the AWS account owner or user with the correct permissions can create the policy in AWS Console by navigating to Services > IAM, selecting Policies located under ‘Access Management’ and then ‘Create policy’.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:ListBucket",
"s3:PutObject",
"s3:PutObjectAcl"
],
"Resource": [
"arn:aws:s3:::destination-bucket",
"arn:aws:s3:::destination-bucket/*"
]
}
]
}

The example JSON configures the policy to allow the assigned user the S3 actions for uploading S3 objects to a bucket “s3:PutObject”, “s3:PutObjectAcl” and listing S3 objects in a bucket “s3:ListBucket”. The JSON can be copied into the JSON tab in AWS Console and replacing destination-bucket with the name of the destination bucket.
Once the policy is created it must be applied to the correct S3 user by first selecting Users under ‘Access management’ and opening the desired user. Click Add permissions, then Attach existing policies directly and select the newly created policy from the list of policies in the table.

Bucket Policies

The bucket policy needs to be set directly on the bucket and AWS account the file is being transferred to and needs to be set by the account owner or a user on the account with permission to set S3 bucket policies. Bucket policies can be accessed using the AWS Console by navigating to S3 > bucket-name, opening the Permissions tab and clicking on Bucket Policy.
The example JSON detailed in the following sections assigns the same S3 actions defined for the user s3:PutObject and s3:ListBucket but in addition defines the Principal (User, Account or Service ARN) that is allowed to perform the actions.
When setting the bucket policy, permission can be provided to upload to the whole bucket or restricted to a folder or sub-folders on the bucket:

Uploading to a Bucket

{
"Version": "2012-10-17",
"Id": "Policy1541018284691",
"Statement": [
{
"Sid": "DelegateS3UploadAccess",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam:: 222222222222:root"
},
"Action": [
"s3:PutObject",
"s3:PutObjectAcl"
],
"Resource": "arn:aws:s3:::destination-bucket/*",
"Condition": {
"StringEquals": {
"s3:x-amz-acl": "bucket-owner-full-control"
}
}
},
{
"Sid": "DelegateS3ListAccess",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::222222222222:root"
},
"Action": "s3:ListBucket",
"Resource": "arn:aws:s3:::destination-bucket"
}
]
}

The example JSON allows all users and services from AWS account id 222222222222 to upload S3 objects to the bucket and list its contents. It does not give them permission to download or delete S3 objects from the bucket. This can be restricted further to a specific user on the external account by replacing root with user/username in the principal:
“arn:aws:iam:: 222222222222:user/username”
Finally, the name of the destination bucket should be replaced by the name of the bucket the policy is being applied to.

UPLOADING TO A FOLDER
{
"Version": "2012-10-17",
"Id": "Policy1541018284691",
"Statement": [
{
"Sid": "DelegateS3UploadAccess",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::222222222222:root"
},
"Action": [
"s3:PutObject",
"s3:PutObjectAcl"
],
"Resource": "arn:aws:s3:::destination-bucket/destination-folder/*",
"Condition": {
"StringEquals": {
"s3:x-amz-acl": "bucket-owner-full-control"
}
}
},
{
"Sid": "DelegateS3ListAccess",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::222222222222:root"
},
"Action": "s3:ListBucket",
"Resource": "arn:aws:s3:::destination-bucket",
"Condition": {
"StringLike": {
"s3:prefix": [
"destination-folder",
"destination-folder/*"
]
}
}
}
]
}

This policy restricts upload access to a specific folder. destination-folder should be replaced with the desired folder located on the bucket. The second statement in the policy returns only the S3 objects with the prefix destination-folder when the prefix is provided in the list objects request. The incoming account can only list objects within this folder and will return a not authorised error when trying to list anything without the folder prefix. This prevents the incoming user from listing the entire contents of the bucket from the root of the bucket.