MLOps deployment in to AWS Fargate: II

Welcome to 3 Part Tutorial for end to end MLOps, starting from training, tracking, deploying, inferencing.

Part 1: Setup MLflow on AWS EC2

Part 2: MLOps deployment on AWS Fargate: I

Part 3: MLOps deployment on AWS Fargate: II

Getting on with Part 3…

You can find the Github Repository for all the used code here!

In the previous part we had set up all the necessary infrastructure needed for deploying the model into cloud environment. Now that the training has been completed and we have the artefacts, the next challenging task is to create APIs and create a Docker image with all the necessities and set it into AWS.

To avoid the tedious task of creating a Docker image manually with all its dependencies we are going to use a Python framework called as BentoML.

BentoML makes developing and deploying neural models everyone’s cup of tea. Let’s see how we can setup the python script to get the desired results.

Two python scripts need to be written for creating the APIs. The first one to package all the necessary artefacts and the other to declare all the necessary APIs and dependencies.

Pre-requisites:

1. AWS account configured on the system:
   CLI installation: https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html
   Account configuration: https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html
2. Docker installed on the system
3. AWS ecs-cli installed and configured
   CLI installation: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ECS_CLI_installation.html

I. Developing the necessary scripts using BentoML guidelines.

1. Bento packaging script needs to be given the path to the artefacts. For now we downloaded the artefacts from S3 onto our local system and have given the corresponding path

2. Bento predicting script needs to incorporate all the requirements and the API structure that we expect to host into AWS.

Let’s take a look at the script

The decorator @bentoml.api helps in creating the API function with the name predict_image. The API has been built to receive image files in byte format in its request and in return it would send the label.

As you can see the necessary artefacts are a Tensorflow based model and a Json file containing the labels.

The decorator @bentoml.env helps in auto generating the requirements.txt file for building the Docker image based on the packages imported in the script.

3. Run Bento packaging script using the command

python3 bento_package.py

4. This would create a local repository with a UUID as it’s folder name and it would incorporate all the data necessary to build the docker image and test its functionality.

5. In order to test the results on local we need to run the following command:

bentoml serve ImageClassifier:latest

II. Containerise the BentoML model server for deployment

1. In order to create ECS deployment, the model server need to be containerised and pushed to a container registry (ECR)
2. We need to login to Docker

aws ecr get-login-password --region us-east-2
# Sample output (Authentication Token)

eyJ.................OOH

3. Use the output in the following command and add your account ID.

docker login -u AWS -p eyJ.................OOH https://account_id.dkr.ecr.us-west-2.amazonaws.com

4. Create AWS ECR repository

aws ecr create-repository --repository-name irisclassifier-ecs

5. Build the Docker image

saved_path=$(bentoml get IrisClassifier:latest --print-location --quiet)

docker build --tag=account_id.dkr.ecr.us-west-2.amazonaws.com/irisclassifier-ecs $saved_path

6. Push the docker image

docker push account_id.dkr.ecr.us-west-2.amazonaws.com/irisclassifier-ecs

7. Prepare AWS for ECR deployment

$ cat task-execution-assume-role.json

# Sample output

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "",
      "Effect": "Allow",
      "Principal": {
        "Service": "ecs-tasks.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

8. Create IAM Role

aws iam --region us-west-2 create-role --role-name ecsTaskExecutionRole \
  --assume-role-policy-document file://task-execution-assume-role.json

# Sample output

{
    "Role": {
        "Path": "/",
        "RoleName": "ecsTaskExecutionRole",
        "RoleId": "AROASZNL76Z7C7Q7SZJ4D",
        "Arn": "arn:aws:iam::192023623294:role/ecsTaskExecutionRole",
        "CreateDate": "2019-12-17T01:04:08Z",
        "AssumeRolePolicyDocument": {
            "Version": "2012-10-17",
            "Statement": [
                {
                    "Sid": "",
                    "Effect": "Allow",
                    "Principal": {
                        "Service": "ecs-tasks.amazonaws.com"
                    },
                    "Action": "sts:AssumeRole"
                }
            ]
        }
    }
}

9. Adding policy AmazonECSTaskExecutionRolePolicy to role ecsTaskExecutionRole

aws iam --region us-west-2 attach-role-policy --role-name ecsTaskExecutionRole \
  --policy-arn arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy

III. Configure ECR Profile

1. Create ECR CLI profile

ecs-cli configure profile --access-key $AWS_ACCESS_KEY_ID --secret-key $AWS_SECRET_ACCESS_KEY --profile-name docuedge-bentoml-profile

2. Create ECR Cluster profile config

ecs-cli configure --cluster docuedge-bentoml-modelzoo --default-launch-type FARGATE --config-name docuedge-bentoml-config --region us-east-2

3. Create a cluster using the profile created above

ecs-cli up --vpc vpc-01b4edaf92e19af08 --subnets subnet-0345b051535c9625d, subnet-00e7bff093931a167 --cluster-config docuedge-bentoml-config --cluster docuedge-bentoml-modelzoo

IV. Create the necessary .yaml files

1. Create docker-compose.yaml file, use the image tag from previous steps

2. Create ecs-params.yaml using the security groups and subnets that you used in the previous Part 1 of this set.

As we were using Tensorflow and the docker image was greater than 4GB in size, so the alloted size to the task is 8GB and CPU limit is respective to the same.

EFS is optional, if you have it set in your system you can add it to the ecs-params.yaml as mentioned above or else remove it from the file.

V. Deploying BentoService into Fargate

We set the target groups, cluster profile and project name as per what we have set in the previous chapters.

ecs-cli compose --project-name docuedge-bentoml-modelzoo service up --target-groups targetGroupArn=arn:aws:elasticloadbalancing:us-east-2:142339138776:targetgroup/docuedgedev-bentoml/5743cdb7a5630ff9,containerName=web,containerPort=5000 --create-log-groups --cluster-config docuedge-bentoml-config --ecs-profile docuedge-bentoml-profile

Congratulations you have successfully deployed the model into your AWS cloud environment. To view the Swagger docs or API suite, navigate to your AWS dashboard and use the DNS of your ALB.
Or if you would have attached the SSL certificate then open the HTTPS link that you had mentioned in your certificate in ACM

You should see something like this !!!

To use the API using Python just copy the link and replace it in the below code snippet

Hope you found the tutorial useful.😁😁