In this article, I will guide you to create a Lambda Function that fetches data from MongoDB atlas and returns the result as a response. To use Pymongo in the Lambda function we will create a Lambda Layer. There are a few benefits to using Layers to use Python packages instead of Uploading everything to function. First, You can use the inline editing option as your code size will not be more than a few kilobytes. Second, you can use the same layer in different functions thus increasing re-usability. I will use Docker to create the packages because I am using Windows. In some stack overflow posts, I have seen they are having issues when uploading Python packages installed in Windows. If you are using Linux you can skip the docker. Just use the commands mentioned in the Dockerfile. I would still ask you to follow the article if you do not have high Linux skills.

Create a Docker Image

AWS Lambda supports Python 3.11. But in this article, I will create an application in Python 3.9 just to show how you can run older Python versions in Lambda.

Pull python:3.9 from docker hub.

docker image pull python:3.9

Create a directory for this project. Open the directory in VS code. Create a file named Dockerfile.

Copy the contents of Dockerfile.

FROM python:3.9

RUN apt update -y

WORKDIR /usr/src/app

COPY . .

RUN python3 -m venv python

RUN . python/bin/activate

RUN find . -type d -name __pycache__ -exec rm -r {} \+

RUN pip3 install -r requirements.txt

CMD ["python3", "/usr/src/app/app.py"]

Now create a requirements.txt file. And copy the contexts.

pymongo[srv]==4.6

Pymongo 4.6 is supported by Python 3.9. Make sure to use the package version supported by the Python version you intend to use.

Now create a .env file to store the MongoDB connection URL. I am using MongoDB Atlas.

Now write Python code to check everything is working. Copy the code.

import os
from pymongo import MongoClient

def connect():
    client = MongoClient(os.getenv('MONGO_URL'))
    return client

def getDb():
    client = connect()
    return client.test_db

def getDataFromDB():
    db = getDb()
    result = list(db.users.find({}))
    return result

if __name__ == "__main__":
    print(getDataFromDB())

Now create the docker image. Make sure to use a unique name.

 docker build -t lambda .

Once the image is created, create and run the container. Do not remove the container. We need to copy the Python packages from the container.

docker run --env-file .env --name lambda-py lambda

Create a Layer in AWS Lambda

To make everything work in Lambda we need to make some changes. First change the docker file content.

FROM python:3.9

RUN apt update -y && apt -y install zip

WORKDIR /usr/src/app

COPY . .

RUN python3 -m venv python

RUN . python/bin/activate

RUN find . -type d -name __pycache__ -exec rm -r {} \+

RUN pip3 install -r requirements.txt -t python/lib/python3.9/site-packages

RUN zip -r9 python.zip python

CMD ["ls", "-al"]

This code will install the packages in a specific location and then zip the entire virtual environment folder. To use Pymongo in Lambda packages need to be installed this way. We will upload the python.zip to lambda layers. Now create the container again by running docker run --name lambda-py lambda command. If the container already exists, run docker rm -f lambda-py and run the previous command again.

Now we need to copy the python.zip file from the container.

 docker cp lambda-py:\usr\src\app\python.zip python.zip

Login to your AWS Account and go to Lambda > Layers page.

Click on Create Layer button. Then Layer configuration page will open. Give proper values to each field. Upload the python.zip file copied from the container. As runtime select Python 3.9. Finally click on create.

Create Lambda Function

To create a function go to Lambda > Functions page. Click on Create function button. Give a proper name and select Python 3.9 as runtime.

Open the function. Go to the code section and copy-paste the below code.

import json
import os
from pymongo import MongoClient

def connect():
    client = MongoClient(os.getenv('MONGO_URL'))
    return client

def getDb():
    client = connect()
    return client.test_db

def getDataFromDB():
    db = getDb()
    result = list(db.users.find({}))
    return result

def lambda_handler(event, context):
    # TODO implement
    print(getDataFromDB())
    return {
        'statusCode': 200,
        'body': json.dumps('Hello from Lambda!')
    }

Click on deploy.

Now go to Configuration, and click on Environment variables. Click on Edit button. Click on Add environment variable. Set the key-value pair. Save the environment variable.

Attach Layer to Function

Click on Layers label in the Function Overview section.

At the end of the page, You will see Layers section. Click on Add a Layer button.

Select Custom Layer option. Then select the custom layer you want to use. And choose the highest available version. Click on Add.

Now test the function.

It is working as expected. It is time to return the result. To do that we need to make some changes in the function code. Change the lambda handler function to below code. And again click on Deploy button.

def lambda_handler(event, context):
    # TODO implement
    data = getDataFromDB()
    return {
        'statusCode': 200,
        'body': json.loads(json.dumps(getDataFromDB(), default=str))
    }

json.loads(json.dumps(getDataFromDB(), default=str)) this code will convert the Mongodb ObjectId values to string. In some cases, the request time can exceed 3 seconds which is default for Lambda Fucntions. To do increase the time go to Configuration > General COnfiguration section. Click on Edit button.

Now change the Timeout from 3 to 10 seconds. Save the configuration.

Now test the code again.