last article. It’s an image recognition deep learning model based on TensorFlow Inception-v3 model. Although our data is not so compressed. The overall file size is about 150 MB which is much beyond the specified limit of 50 MB.
This model was created specifically for this project. However, Machine Learning models can be downloaded from the following sources.
TensorFlow: official release, performance models, tensornets
2 Lets call our package as MachineLearning.zip.
zip MachineLearning.zip MachineLearning
3 Now check whether we could compress the file or not.
$ ls -lhtr | grep zip -rw-r--r-- 1 john staff 123M Nov 4 13:05 MachineLearning.zip
Even after compressing and zipping the overall package size is about 132 MB.
4 In order to create a lambda function, we need to create IAM role. Since our primary objective is to test the limits we’ll skip over the role creation process. Login to IAM Management Console with your credentials and create a Test-role and attach AWSLambdaRole policy.
5 Next, we’ll create a lambda function via AWS CLI and upload our deployment package directly to the function.
aws lambda create-function --function-name mlearn-test --runtime nodejs6.10 --role arn:aws:iam::XXXXXXXXXXXX:role/Test-role --handler tensorml --region ap-south-1 --zip-file fileb://./MachineLearning.zip
Replace XXXXXXXXXXXX with your AWS Account id. Since our package size is greater than 50 MB specified limit it throws an error.
An error occurred (RequestEntityTooLargeException) when calling the UpdateFunctionCode operation: Request must be smaller than 69905067 bytes for the UpdateFunctionCode operation
6 Since our deployment package is quite large we will load it again during AWS Lambda inference execution from Amazon S3. For this we need to create AWS S3 bucket from AWS CLI:
aws s3 mb s3://mlearn-test --region ap-south-1
This will create S3 bucket for us. Now we’ll upload our package to this bucket and update our lambda function with the S3 object key.
aws s3 cp ./ s3://mlearn-test/ --recursive --exclude "*" --include "MachineLearning.zip"
Once our package is uploaded into the bucket we’ll update our lambda function with the package’s object key.
aws lambda update-function-code --function-name mlearn-test --region ap-south-1 --s3-bucket mlearn-test --s3-key MachineLearning.zip
This time it shows no error even after updating our lambda function and we’re able to upload our package successfully. Which means that the package size can be greater than 50 MB if uploaded through S3 instead of uploading directly. Since our package size is about 132 MB after compression we are still not clear what is the maximum limit of the package to be uploaded.
In order to get the maximum limit we’ll create a random data of about 300 MB and upload it through S3 and update our lambda function.
fsutil file createnew sample300.txt 350000000
This will create a sample file of about 300 MB. We’ll zip the file and upload it again through S3.
aws s3 cp ./ s3://mlearn-test/ --recursive --exclude "*" --include "sample300.zip"
aws lambda update-function-code --function-name mlearn-test --region ap-south-1 --s3-bucket mlearn-test --s3-key sample300.zip
After updating our lambda function we get the following error:
An error occurred (InvalidParameterValueException) when calling the UpdateFunctionCode operation: Unzipped size must be smaller than 262144000 bytes
The error describes that the size of the unzipped package should be smaller than 262144000 bytes which is about 262 MB. We can notice here that this size is just a little greater than the specified limit of 250 MB size of code/dependences that can be zip into a deployment package (uncompressed .zip/.jar size). So we discovered that the maximum limit of the size of uncompressed deployment package is 250 MB when uploaded via S3. However we can’t upload more than 50 MB package while uploading directly into lambda function.
The important thing to notice here is that your code and its dependencies should be within 250 MB size limit when in uncompressed state. Even if we consider a larger package size it may seriously affect lambda function’s cold start time. Consequently, the lambda function will take longer time to execute with larger package size.
Failure detection, analytics and visibility for serverless applications in under 5 minutes.