MLOps Basics Part 4

Containers

Docker is a virtual machine-like technology that allows the engineer to create a snapshot image of the container (think of a virtual machine). Containers are stateless or ephemeral technologies, meaning every time you restart the container your data saved inside the container is lost. Docker images are created from Dockerfiles, and Docker containers are running instances of Docker images. We will be building our images and then saving them in AWS ECR.

  • Listing running containers
docker ps
docker ps -a #include stopped containers
  • remove unused images from your computer
docker system prune
  • Turn your Dockerfile into a Docker image
docker build -t <tag> <path to Dockerfile>
  • Running a Docker container from an image
docker run -it <tag> <local commands to run>
  • Executing commands on a running Docker container
docker exec -it <id of running container> bash
FROM amazon/aws-lambda-pythonCOPY app/lambda_function.py   ./
COPY model ./
COPY Pipfile ./
COPY startup.sh ./
RUN yum install -y gcc \
&& pip3 install pipenv \
&& pipenv install \
&& pipenv run pip3 freeze > requirements.txt \
&& pip3 install -r requirements.txt --no-cache-dir
CMD ["lambda_function.handler"]
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"
[packages]
awslambdaric = "*"
pylint = "*"
sklearn = "0.24.2"
pickle5 = "*"
[dev-packages]
import pickle5 as pickle
import numpy as np
def handler(event, _):
"""
The main entry function
:param event:
:param _:
:return:
"""
loaded_model = pickle.load(open("model.pkl", 'rb'))
X = np.array(event["data"])
prediction = loaded_model.predict([X])
message = {
'prediction': float(prediction)
}
return message

AWS Lambda

The heart of our deployment will be the AWS serverless technology Lambda. The term Lambda comes from Calculus, and generally speaking, can be understood as ephemeral groups of code. When triggered AWS Lambda will take our image and run a container that will make a prediction using our ML Model. A Lambda allows us to run code without forcing us to manage any infrastructure.

AWS CLI

We will be using the AWS CLI to interact with AWS. The CLI will allow us to make changes to our AWS account, without using the GUI. In a future article, we will automate this process.

curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install

Creating the execution role

When our lambda is triggered, it will need an AWS role, which will be used to grant access to the lambda. The role will need a policy document attached to it, in our case, this will be minimalistic.

aws iam create-role --role-name lambda-ex --assume-role-policy-document '{"Version": "2012-10-17","Statement": [{ "Effect": "Allow", "Principal": {"Service": "lambda.amazonaws.com"}, "Action": "sts:AssumeRole"}]}'
{
"Role": {
"CreateDate": "..",
"RoleName": "lambda-ex",
"Arn": "...",
"Path": "/",
"RoleId": "...",
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Action": "sts:AssumeRole",
"Effect": "Allow",
"Principal": {
"Service": "lambda.amazonaws.com"
}
}
]
}
}
}
$ aws iam attach-role-policy --role-name lambda-ex --policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole

AWS ECR

Once you have built your Docker image you must send it to a registry for central access. AWS provides a paid service that we will be using called Elastic Container Registry, ECR.

aws ecr create-repository --repository-name my_tests/lambda-mlops-model
aws ecr get-login-password --region us-east-2 | docker login --username AWS --password-stdin <URI from ECR>
aws ecr get-login-password --region us-east-2 | docker login --username AWS --password-stdin 112437402463.dkr.ecr.us-east-2.amazonaws.com
docker build -t dev/lambda-mlops-model .
docker run -it dev/lambda-mlops-model bash
docker ps
docker exec -it <id> bash
docker tag my_tests/lambda-mlops-model:latest <URI from ECR>:latestdocker push <URI from ECR>:latest
docker tag my_tests/lambda-mlops-model:latest 112437402463.dkr.ecr.us-east-2.amazonaws.com/dev/lambda-mlops-model:latestdocker push 112437402463.dkr.ecr.us-east-2.amazonaws.com/dev/lambda-mlops-model:latest
aws lambda create-function --function-name mlops-model\
--runtime python3.8 --role ....lambda-ex\
--code=<URI from ECR>
aws lambda create-function --function-name mlops-model\
--runtime python3.8 --role arn:aws:iam::112437402463:role/lambda-ex\
--code=112437402463.dkr.ecr.us-east-2.amazonaws.com/my_tests/lambda-mlops-model:latest
aws lambda invoke --function-name mlops-model --payload '{
"data": [
" 4.38600006e+01",
"7.31360000e+06",
"2.59917778e+00",
"-9.93224908e-01"
]
}' output.txt

lambda without docker

Although we focused on AWS Lamda deployment with Docker, let's cover some key points with deploying our model without Docker.

  • Use Amazon Linux
docker run -d -v '/home/blipp':'/srv' amazonlinux tail -f  /dev/null
docke psdocker exec -it <id> bash
cd /srv/....

yum install -y gcc gcc-c++ make git patch openssl-devel zlib-devel readline-devel sqlite-devel bzip2-devel libffi-devel zip
curl https://pyenv.run | bash
export PYENV_ROOT="$HOME/.pyenv"
export PATH="$PYENV_ROOT/bin:$PATH"
eval "$(pyenv init --path)"
pyenv install 3.8.8
pyenv global 3.8.8
pip install pipenv
pipenv install
pip freeze > requirements.txt
mkdir lambda_deployment
cp -r app lambda_deployment/
cp -r model lambda_deployment/

pip3 install -t ./package -r requirements.txt
cd package
zip -r9 ../../lambda_deployment_function.zip .
cd ..
rm -rf package/
zip -gr ../lambda_deployment_function.zip *
  • Include your shared dependencies in a layer.
curl https://pyenv.run | bash
export PYENV_ROOT="$HOME/.pyenv"
export PATH="$PYENV_ROOT/bin:$PATH"
eval "$(pyenv init --path)"
pyenv install 3.8.8
pyenv global 3.8.8
pip install pipenv
pipenv install
pip freeze > requirements.txt
mkdir -p layer_deployment/python/lib/python3.7/site-packages
pip3 install -t layer_deployment/python/lib/python3.7/site-packages -r requirements.txt
cd layer_deployment
zip -r ../layer_deployment *

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store