On my AWS Lambda Python 3.6 function I'd like to use Google Firestore (Cloud Firestore BETA) for caching purposes, but as soon as I add
from google.cloud import firestore
to my Python script and upload ZIP to AWS Lambda function, Lambda test come back with error
Unable to import module 'MyLambdaFunction': cannot import name 'cygrpc'.
AWS CloudWatch log doesn't contain any details on the error, just that same error message.
Lambda function works great on my local dev machine (Windows 10), and I can write to Firestore fine. It also works on AWS if I comment out the import and all Firestore related lines.
Any tips how I could go about solving this?
The python client for Firestore relies on the C-based implementation of GRPC. This appears not to work by default in AWS Lambda.
Node.js users have reported similar problems and they've documented a workaround of building a docker image.
This should be similar to any getting any other python package that requires native code to work. Perhaps something like this method for getting scikit to work?
I hope this is enough to get you going in the right direction, but unfortunately I don't know anything about AWS Lambda :-(.
Ran into same issue, i solved it by using the plugin serverless-python-requirements for serverless framework and passing:
pythonRequirements:
dockerizePip: true
Essentially this installs your c-based packages (and all other packages) in a docker container where it would work and then symlinks them to your lambda fn.
A helpful guide can be found on: https://serverless.com/blog/serverless-python-packaging/
Plugin: https://github.com/UnitedIncome/serverless-python-requirements
Related
I've recently taken over a project that has AWS Serverless Image Handler version 5 implemented. And I've been sent various emails from the client they received about Node.js v12 hitting end of life and the Lambda functions needing to run on a new Node.js runtime.
Having a looking through the AWS account, I've seen I can just switch the runtime to Node.js v14 or v16, but do I need to do any code updates?
Sorry a complete noob to Lambda, and the CloudFormation stacks etc
Thought I would ask before I jump down the rabbit hole and look at setting up my own copy of Serverless Image Handler and doing some testing, or even trying to implement version 6.
This depends of the code / imported libraries in the Serverless Image Handler Lambda.
I would check if there are breaking changes in Node14, that could affect the code.
This does not have anything to do with Lambda / CF itself.
I am using matplotlib library in my python project which in turn uses numpy. I have deployed the libraries in AWS Lambda Layers and I am importing them in my AWS lambda function. When I test my AWS Lambda function it throws the following error:
Importing the numpy C-extensions failed. This error can happen for many reasons, often due to issues with your setup or how NumPy was installed. We have compiled some common reasons and troubleshooting tips at: numpy.org/devdocs/user/troubleshooting-importerror.html Please note and check the following: * The Python version is: Python3.8 from "/var/lang/bin/python3.8" * The NumPy version is: "1.18.5" Original error was: No module named 'numpy.core._multiarray_umath'
Any idea what could be the possible reason and how to resolve it?
I am answering the question so that if anyone in future also faces this issue so the below solution would might work for them as well.
The problem was that I compiled the required packages in windows 10 enviroment and then I deployed them on layers to be used by AWS Lambda function. AWS Lambda function and Layers use Linux in
the background so the packages compiled in the Windows enviroment were not compatible with AWS Lambda function. When I compiled the required packages again in Linux enviroment and deployed them on layers and used them again with lambda function then it worked like a charm!
This Medium article helped me in solving my issue.
We are using CloudFormation template to deploy some intermediate code on Lambda function.
We are using ZipFile function to add inline code through CloudFormation.
Current runtime for lambda function is node.js 8.10.
We need to update node version to 10.x.
While updating Lambda using cloudformation we are getting below error:
ZipFile can only be used when Runtime is set to either of nodejs,
nodejs4.3, nodejs6.10, nodejs8.10, python2.7, python3.6, python3.7.
I believe that this is a known issue. https://forums.aws.amazon.com/thread.jspa?threadID=303166&tstart=0
As of this writing, it is still an issue. My suggestion is to have super basic code in an S3 bucket and reference that instead of using a zip file and deploy your actually code after the lambda function is created. Alternatively, you can just upload your zip artifact to an S3 bucket. If your code is proprietary be careful about S3.
There are other similar question like mine, but I think no one looks complete or fits/answer my case.
I'm deploying a Python 3.6 application on AWS lambda via serverless framework.
With this application I'm using diskcache to perform some small file caching (not using at all sqlite actually)
I'm using "serverless-python-requirements" plugin in order to have all my dependencies (defined in requirements.txt file) packed up and uploaded (diskcache in this case)
When application is live on AWS and I request it, I'll get back a 500 error. And in my logs I can read:
Unable to import module 'handler': No module named '_sqlite3'
Then from answer below I get that sqlite module should not be needed to be installed.
Python: sqlite no matching distribution found for sqlite
So no need (and it wont work) to add sqlite as a requirement...
Then I wonder why AWS lambda is unable to find sqlite once deployed.
Any hint pls?
Thanks
I need to run CRUD operations on my bucket (database) in couchbase which is deployed ec2 instance. And the code which I have is running on aws lambda. However, when I try to test this code on lambda by passing details in the body I get the error as : "errorMessage": "/usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /var/task/node_modules/couchbase/build/Release/couchbase_impl.node)". This error is because in my function I require an npm module called "couchbase" which is used for executing CRUD operations on my couchbase bucket.
So can you guys help me as to what might be the problem here? Is the file missing on nodejs environment running on lambda or do I need to implement in a different way so as to get it working?
Thanks in advance.
I was able to solve this issue by locally compiling node_modules with the same nodejs version (v0.10.36) which lambda uses and uploading the zip file to lambda.