Load Testing using Kubernetes and AWS SQS

I wanted to perform a load test of AWS SQS and my application. So, I created a sample python program which will publish a message to SQS and check when the AWS SQS fails.

Prerequisites

• Docker
• Kubernetes
• SQS Queue

Step 1: Code

sqs-script.py
import json
import boto3
import time
num = 1
while (num <= 5):
try:
sqs = boto3.client('sqs', region_name='us-east-1', aws_access_key_id="<ACCESS_KEY>",aws_secret_access_key="<SECRET_KEY")
response = sqs.send_message(QueueUrl="QUEUE_URL",MessageBody="Hello")
print(response)
num = num + 1
except Exception as error:
print (error)
time.sleep(3600)

Step 2: Creating requirements.txt

requirements.txt
boto3==1.10.5

Step 3: Creating Dockerfile

FROM python:3
WORKDIR /usr/src/app
COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
CMD [ "python", "./sqs-script.py" ]

Step 4: Create Docker Image

docker build -t my-python-app .
docker run -d -p 5000:5000 --restart=always --name registry registry:2
docker tag my-python-app:latest localhost:5000/my-python-app
docker push localhost:5000/my-python-app

The above commands create a local registry and uploads the image to that registry and the Kubernetes yaml file use this image to start the containers. You can use DockerHub, your own repository such as nexus repo, etc. Please make the necessary changes in the yaml file.

Step 5: Create Kubernetes yaml file

apiVersion: apps/v1
kind: Deployment
metadata:
name: my-running-app
namespace: default
spec:
replicas: 50
selector:
matchLabels:
bb: my-running-app
template:
metadata:
labels:
bb: my-running-app
spec:
containers:
- name: my-running-app
image: localhost:5000/my-python-app
imagePullPolicy: IfNotPresent

Step 6: Deploy the yaml file

kubectl apply -f load-test.yaml

This start 50 containers of the application and publish 250 messages to the SQS queue. I have tested it for 50 ,75, 100 containers in my local environment. After that my local environment gave up as all the memory was utilized. If you have development Kubernetes cluster then you can do a load test on larger scale to replicate real life scenarios and mitigate the issues such as throttling, rate exceeded limit, etc.