Working with the AWS S3 CLI in Linux

---

Introduction

AWS S3 (Simple Storage Service) is a reliable and scalable object storage service that is commonly used for storing and retrieving data across various industries. AWS S3 CLI (Command Line Interface) is a powerful tool that provides developers and system administrators with a command-line interface to interact with S3. In this article, we will explore the various AWS S3 CLI commands that can be used to perform operations such as creating and deleting S3 buckets, downloading files, and copying files from an S3 bucket. By leveraging these commands, users can streamline their workflow and manage their S3 resources more efficiently. With the AWS S3 CLI, users can also automate repetitive tasks and easily integrate S3 storage with other applications, providing a more seamless and integrated experience.

Setting up the AWS CLI

Before we can start using the AWS S3 CLI, we need to install the AWS CLI first. The AWS CLI can be installed on Linux using the following command −

$ sudo apt-get install awscli

Once the installation is complete, we need to configure the AWS CLI by running the following command and following the prompts −

$ aws configure

This will prompt you to enter your AWS access key ID, secret access key, default region name, and default output format. You can obtain the access key ID and secret access key from the AWS Management Console.

Once the AWS CLI is configured, we can start using the AWS S3 CLI to interact with S3. Here are some of the most commonly used AWS S3 CLI commands −

Creating an S3 Bucket

We can create an S3 bucket using the following command −

$ aws s3 mb s3://bucket-name

Replace "bucket-name" with the name of your desired bucket.

Listing All the Available S3 Buckets

To list all available S3 buckets in your AWS account, use the following command −

$ aws s3 ls

This command will display the names of all available S3 buckets in your AWS account.

Listing contents of a Specific S3 Bucket

We can list the contents of an S3 bucket using the following command −

$ aws s3 ls s3://bucket-name

Replace "bucket-name" with the name of the bucket you want to list.

Listing All the Items in a Specific Bucket

If you want to display more information about the objects in the bucket, you can use the following command −

$ aws s3 ls s3://<bucket-name> --recursive --human-readable 

The --recursive option is used to recursively list all objects in the bucket, including those in subdirectories.

The --human-readable option is used to display the object sizes in a human-readable format (e.g., KB, MB, GB) instead of bytes.

$ aws s3 ls s3://<bucket-name> --recursive --human-readable --summarize

This command will display a summary of the total size and number of objects in the bucket.

Deleting an S3 Bucket

We can delete an S3 bucket using the following command −

$ aws s3 rb s3://bucket-name

Substitute the placeholder "bucket-name" with the actual name of the bucket that you wish to remove.

The --force flag ensure that the bucket is removed along with all of its objects.

$ aws s3 rb s3://bucket-name --force

Copying Files to an S3 Bucket

Once we have the prerequisites set up, we can proceed with copying files to an S3 bucket using the AWS S3 CLI. The command to copy a file to an S3 bucket is as follows −

$ aws s3 cp /path/to/local/file s3://bucket-name/path/to/s3/object

Replace "/path/to/local/file" with the path to the file on your local machine, "bucket-name/path/to/s3/object" with the desired path to the object in your S3 bucket.

For example, if we have a file named "example.txt" located in the "/home/user" directory and we want to copy it to a bucket named "my-bucket" with the path "my-folder/example.txt", we can use the following command −

$ aws s3 cp /home/user/example.txt s3://my-bucket/my-folder/example.txt

This will upload the file "example.txt" to the "my-bucket" bucket with the path "my-folder/example.txt". If the specified bucket or folder is not present, it will be created.

If we want to copy all the files in a directory to an S3 bucket, we can use the recursive flag "-r" in the command. If we want to copy all the files in the "/home/user/my-folder" directory to the "my-bucket" bucket with the same directory structure, we can use the following command −

$ aws s3 cp /home/user/my-folder s3://my-bucket/my-folder –recursive

This will upload all the files in the "/home/user/my-folder" directory to the "my-bucket" bucket with the same directory structure.

Download a file from an S3 Bucket

To download a file from an S3 bucket using the AWS S3 CLI, we need to use the aws s3 cp command, which copies a file or object from a source to a destination. In this case, we will be copying the file from the S3 bucket to local system.

$ aws s3 cp s3://bucket-name/file-path local-file-path

Where bucket-name is the name of the S3 bucket, file-path is the path to the file in the bucket, and local-file-path is the path to where we want to download the file on our local machine.

For example, to download a file named example.txt from an S3 bucket named my-bucket, we would use the following command −

$ aws s3 cp s3://my-bucket/example.txt /path/to/local/folder/

Deleting Files from an S3 Bucket

We can delete files from an S3 bucket using the following command −

$ aws s3 rm s3://bucket-name/path/to/s3/object

Replace "bucket-name/path/to/s3/object" with the path to the object you want to delete.

Conclusion

The AWS S3 CLI can be a useful tool for developers and system administrators who prefer a terminal-based interface. This article covered how to install and configure the AWS CLI, and use the AWS S3 CLI to create and manage buckets, download files, delete files and list bucket contents. The AWS S3 CLI can simplify working with S3 in Linux.