Transfering files from AWS S3 bucket to another AWS S3 bucket from another account.

Transfering files from AWS S3 bucket to another AWS S3 bucket from another account.

Scenario requirements:

  • Environment A: S3 bucket
  • Environment B: S3 bucket and EC2 : The EC2 is not necessary, aws cli can be installed locally.
  • AWS IAM user creation on both accounts with permissions: PowerUserAccess and AmazonS3FullAccess

Create environment A S3 bucket

Update S3 bucket permissions > Bucket policy

{
    "Version": "2012-10-17",
    "Id": "<autogenerated>",
    "Statement": [
        {
            "Sid": "<id>",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::<iam-user-id-B>:user/<username-B>"
            },
            "Action": "s3:*",
            "Resource": [
                "arn:aws:s3:::<s3-bucket-name-A>",
                "arn:aws:s3:::<s3-bucket-name-A>/*"
            ]
        }
    ]
}

Scenario B

  • Create a role so that EC2 can access S3 bucket with the policy

AmazonS3FullAccess

  • Create an S3 bucket
  • Creating EC2

Assign IAM role

Assign to the same subnet as S3 bucket.

change EC2 security to open ssh(port 22) and https(port 443) to your IP

Security group rule ID 	Port Protocol    Source 	groups
sgr-<id>		22	 TCP	  <IP>  	default
sgr-<id>		443	 TCP	  <IP>          default

Connect to the EC2 via ssh

ssh -v -i "./<cert-name>.cer" ubuntu@<server-public-ip>

Inside the EC2 install AWS cli

apt update
apt install unzip
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install
aws configure

Insert your AWS account B: Access Key ID and Secret access key

To test the connection to the S3 bucket from Account A and/or account be use amazon cli

aws s3 ls s3://<S3 bucket B name>
aws s3 ls s3://<s3 bucket A name>

Once you have the right to acces both buckets then you can start copying over files using two methods:
AWS cli

aws s3 cp s3://<Bucket A name> /<filename+extension> s3://<bucket B name/<filename+extension>

Python (BOTO3)

pip install boto3

Create a file

import boto3

#Creating Session With Boto3.
session = boto3.Session(
aws_access_key_id='account B',
aws_secret_access_key='account B'
)

s3 = boto3.resource('s3')
copy_source = {
    'Bucket': '<bucket A name> ',
    'Key': '<filename + extension>'
}
s3.meta.client.copy(copy_source, '<bucket B name>', '<filename + extension>')

then you can execute the file

python3 <filename>.py
Collect recurring payments with Subscribie - Try Now