Don’t be afraid to learn new things.

As I have stated many times, there is no shortage of things to learn. No matter where you are or how experienced you are, you can always learn new technologies. In fact, to survive in the new world of evolving technologies, you are expected to learn new things so that you remain relevant.

Now that majority of private and public organizations are considering moving applications from on-premises to hybrid cloud, as a tester, you are expected to verify the solution design. Doing it manually is cumbersome and takes a lot of effort. So why not automate tasks when it is possible?

Earlier this year I completed my AWS Solution Architect – Associate certification. The idea was that I would work towards AWS Solution Architect – Professional certification. However that path got a bit derailed because I wanted to know how to automate tasks in AWS, especially if I wanted to verify the infrastructure design, as specified in the solution document.

• Are the EC2 instances being provisioned in the correct region?
• Are the number of EC2 instances correct?
• Do EC2 instances have the correct tag?
• What security group is assigned to each EC2 instance? etc.

So I searched for a solution online and came across Boto3.

So what is Boto3?

Boto3 is the Amazon Web Services (AWS) Software Development Kit (SDK) for Python, which allows Python developers to write software that makes use of services like Amazon S3 and Amazon EC2. Seems like boto3 has been around for a while, since Nov, 2014 so I am late to the party.

Apparently using boto3, you are able to create python scripts which can retrieve a range of information about your AWS infrastructure especially your region, number of EC2 instances in the region, their states etc.

This is great, right? Here is a SDK that allows you to automate things in python that you would have to do manually over CLI.

There is a small problem though. I have very basic understanding of Python.

So I decided to quickly learn python. Looked up youtube and other sites for a good tutorial for beginners and came across below video by Mosh Hamedani. In my opinion, this is a great first time video to learn python. In my case, at least it gave me necessary knowledge on python to create python scripts using boto3 packages. [NOTE: there is no shortcut to learning. You have to put in the hard yards to learn.]

Now creating python scripts with boto3 means that you should have a very good grasp of AWS services and AWS design principles. That is why it is highly recommended that you complete the AWS Cloud Practitioner certification or if possible complete the AWS Solution Architect – Associate certification.

Running python scripts with boto3 has some pre-requisites:

1. Boto3 needs a host, so one of the EC2 instance has to be used as the host where boto3 will be installed and python scripts executed
2. When you provision an EC2 instance with Linux, by default it will come with python. But the version is quite old ~ 2.7.18. So you need to install the latest python
3. Once you install the latest python such as python 3.8.5 (at the time of writing this post), you will have two different python versions. So instead of using python3.8 for every interpreter command execution, use alias.
4. Then install boto3
5. Then enter the python interpreter and import boto3 and botocore

The screenshot below shows the steps (first install pip, then install latest python, then install boto3 for the latest python, create alias for python3.8, enter python3.8 interpreter and import boto3 and botocore). NOTE that the boto3 host EC2 instance needs to be running. Using the putty private key, I SSHed to the EC2 instance from my personal windows 10 laptop. [How to do that should be available in the AWS Cloud Practitioner or AWS Solution Architect – Associate courses.]

You also need to setup your AWS credentials and AWS config with your IAM access key details and other details. In the new EC2 instance, this will not be available. So use below to configure AWS credentials and AWS config file.

Once done, please open ~/.aws/credentials and ~/.aws/config to ensure that correct credentials have been used to configure these files.

NOW you are ready to create a python scripts and run it on AWS.

As I said, you need to know how to write python scripts to do the below tasks. So learn python programming first.

Now that I have boto3 and python ready, what do I do next?

I want the confirm that the region where the EC2 instances have been created is actually Asia Pacific Sydney (apsoutheast-2) , total number of EC2 instances in that region under my account is 3, their state and be able to start and stop a EC2 instance.

YES, I can do all of this through a single python script and with a single command.

As can be seen, I have 3 EC2 instance running in Sydney region (ap-southeast-2). The python script below will retrieve all that information for me. For privacy reasons, I had to mask some of the data below.

Running this script is as simple as writing the command <python myfirst.py> where myfirst.py is the python script I created. As can be seen, the script has correctly identified the region, the number of EC2 instances =3.

The script was also able to stop one of the EC2 instances.

Boto3 package allows a significant number of actions to be done through python scripts.

While this article introduces you to running a basic python script using boto3 SDK, there is a lot more to learn before you can get to a level to run extensive automation in AWS.