Unable to import module 'lambda_function': No module named 'aws_xray_sdk' - python-3.x

I am trying to implement this AWS Lambda Rest API Handler in my lambda code to handle proper response code. For this I needed to repackage the library aws_lambda_powertools and add as a layer in lambda function.
All the import related to this lib below are working.
from aws_lambda_powertools import Logger, Tracer
from aws_lambda_powertools.event_handler import APIGatewayRestResolver
from aws_lambda_powertools.logging import correlation_paths
from aws_lambda_powertools.utilities.typing import LambdaContext
But When I am creating object of above Tracer class below its giving error(Rest two commented object logger and app are working fine.
tracer = Tracer()
# logger = Logger()
# app = APIGatewayRestResolver()
Error I am getting while declaring tracer object is below:
Response
{
"errorMessage": "Unable to import module 'lambda_function': No module named 'aws_xray_sdk'",
"errorType": "Runtime.ImportModuleError",
"stackTrace": []
}
Function Logs
OpenBLAS WARNING - could not determine the L2 cache size on this system, assuming 256k
START RequestId: ae8b006b-e7f7-495b-99a0-eb5231c3f81c Version: $LATEST
[ERROR] Runtime.ImportModuleError: Unable to import module 'lambda_function': No module named 'aws_xray_sdk'
Traceback (most recent call last):
I tried to install pip install aws_xray_sdk and repackaged it and re-added to layer still its giving the same error.
Can anyone help me with this? I am new to lambda. Thanks in advance.

Fixed the error by using AWS Arn arn:aws:lambda:{region}:017000801446:layer:AWSLambdaPowertoolsPythonV2:18📋 instead of using my own custom repackaged library layer.
Reference Link:
https://awslabs.github.io/aws-lambda-powertools-python/2.6.0/

Tracing (like validation and parsing) requires additional dependencies. They're not included by default in an attempt to keep the resulting package as small as possible.
When packaging with AWS SAM, I'm using this in my my_code/requirements.txt, which I then pip install in my local virtual env:
aws-lambda-powertools[parser, validation, tracer]==2.6.0
Additionally, I'm including this in tests/requirements.txt, which I also pip install locally but which is not picked up by SAM (keeping the image small again, and it's not required at runtime anyhow).
aws-lambda-powertools[aws-sdk]==2.6.0
pytest==7.2.1

In version 2.0 the xray SDK is not included by default, as this would introduce a size overhead even for those who would not be using Tracer.
To solve this problem, just put in your requirements.txt the tracer dependency using aws-lambda-powertools[tracer] or all dependencies using aws-lambda-powertools[all].
Refer This link:
https://github.com/awslabs/aws-lambda-powertools-python/issues/1872

Related

Unable to Import module to EFS for Lambda

I am using AWS Lambda python3.8 runtime, and for the filesystem, I am using EFS.
On Lambda I am using openCV, which is returning such kind of error upon Lambda function invoke:
{
"errorMessage": "Unable to import module 'lambda_function': libpng16.so.16: cannot
open shared object file: No such file or directory"
}
I guess libpng16 dependencies were needed in order to make it work, but couldn't find exact steps where to put this lib on the mounted directory of EFS?

Unable to import module 'lambda_function': No module named 'bcrypt'

Unable to import module 'lambda_function': No module named 'bcrypt'
I get the above error when I import bcrypt in my python code on AWS Lambda
My Python version is 3.8 and I've added python bcrypt 3.2.0 as a layer to my lambda function. There tutorial I referred to add Layers to Lambda is here
The file structure for my bcrypt layer is
./python/bcryptfiles (bcryptfiles is my bcrypt package)
I initially included the bcrypt package as a folder adjecent to my lambda_function.py file, which solved bcrypt import issue. But when I invoked checkpw, I got the error
module 'bcrypt' has no attribute 'checkpw'
Can someone please suggest a solution? I couldn't find questions related to python bcrypt in AWS Lambda, hence raise this issue.
Source of my python bcrypt package download here
As this bcrypt should be a library installed as a layer, you shouldn't need to add it adjacent to your lambda_handler.py file.
Also, you would probably need an amazon linux build (amazon linux 2 for python 3.8) to run it on this runtime.
One method that has helped me build unavailable packages in the past was this:
https://github.com/lambci/docker-lambda

Unable to install PySpark Module Error No Module Found

I'm trying to work with Microsofts Hyperspace application.
In order to make it work with Python I need to install the module called Hyperspace.
When I implement the code from hyperspace import * I get the following error:
ModuleNotFoundError: No module named 'hyperspace'
I tried the following but still no luck
from pyspark hyperspace import *
Can someone let me know what it will take to successfully install the module?
I
Thanks
The module isn't supported on Databricks

Unable to import numpy 1.19.1 in AWS Lambda No module named 'numpy.core._multiarray_umath'

I am unable to import numpy 1.19.1 in AWS Lambda with python3.8 on AWS Lambda
I am using the following dependencies:
pandas 1.1.0
pyarrow 1.0.0
numpy 1.19.1
psycopg2 2.8.5
Because I work on a windows environment, I created an EC2 Linux instance installed python3.8 and downloaded all required libraries, then I added them into the project, but the moment I try to import pandas I get the following:
[ERROR] ImportError: Unable to import required dependencies:
numpy:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the numpy c-extensions failed.
- Try uninstalling and reinstalling numpy.
- If you have already done that, then:
1. Check that you expected to use Python3.8 from "/var/lang/bin/python3.8",
and that you have no directories in your PATH or PYTHONPATH that can
interfere with the Python and numpy version "1.18.2" you're trying to use.
2. If (1) looks fine, you can open a new issue at
https://github.com/numpy/numpy/issues. Please include details on:
- how you installed Python
- how you installed numpy
- your operating system
- whether or not you have multiple versions of Python installed
- if you built from source, your compiler versions and ideally a build log
- If you're working with a numpy git repository, try `git clean -xdf`
(removes all files not under version control) and rebuild numpy.
Note: this error has many possible causes, so please don't comment on
an existing issue about this - open a new one instead.
Original error was: No module named 'numpy.core._multiarray_umath'
Traceback (most recent call last):
  File "/var/task/src/py38-lib-test.py", line 28, in py38test
    import pandas
  File "/tmp/lib/pandas/__init__.py", line 16, in <module>
    raise ImportError(END RequestId: 07762380-1fc4)
Lastly, I noticed AWS Lambda provides a layer with numpy and sci-kit, I tried removing my numpy version but kept the rest and added the layer to the function, but the same error occurs.
Thanks in advance your comments.
I use the layer provided by Klayers to solve the problem.
Suppose you're running python 3.8 in us-east-1 region, according to this Klayers document, you can use arn:aws:lambda:us-east-1:770693421928:layer:Klayers-p38-numpy:9 as your layer so that you can run import numpy in the lambda function.
AWS Lambda function don't work this way. If you open the Pandas package it'll be having the Numpy package with them but they would not work.
The easy solution is to first download the required packages separately depending upon your python version and work enviroment from this site, unzip them and add them to your project directory. Create a .zip of your project and deploy it on AWS Lambda function. It'll work this way.
You can refer to this site in order to follow the complete procedure.
Is your ec2 instance an amazon linux2 machine? You could also try building and run a docker image for amazon linux 2 and get the python libs compatible to the environment you need in your Lambda, by volume mounting to your host.
Something similat to docker lambda:
https://github.com/lambci/docker-lambda/tree/master/python3.8
I had the same issue, tried packaging all libs with my base code, tried custom lambda layer by separating numpy and pandas libs. Nothing worked.
Used default AWS Layers. In the default layers, AWS provides layers like AWSSDKPandas, CodeGuru, Lambda Insights, etc. AWSSDKPandas layer is packaged with pandas libs and other dependencies like numpy, etc.
So I removed numpy dependency from my base package and added AWSSDKPandas as Lambda layer. Worked well.

How to import pygit2 library into AWS lambda python3.8 runtime

I am facing an issue with AWS lambda function. I am unable to import pygit2 library into my function. I tried it on 4 different runtime environments and each gives a different error, which makes me think that there is something wrong within the library rather than my code. These are the errors I get:
github_pull_source is the name of my function.
Python 2.7 - Unable to import module 'github_pull_source': No module named _pygit2
Python 3.6 - Unable to import module 'github_pull_source': /var/task/_cffi_backend.so: undefined symbol: _Py_ZeroStruct
Python 3.7 - Unable to import module 'github_pull_source': No module named 'pygit2._pygit2'
Python 3.8 - Unable to import module 'github_pull_source': No module named 'pygit2._pygit2'
I import the library like so:
from pygit2 import Keypair, discover_repository, Repository, clone_repository, RemoteCallbacks
Structure of the deployment package
deployment_package.zip
- source_code.py
- __init__.py
- pygit2
- libgit2-f9f42b17.so.1.0.0
- libgit2.tar.gz
- libgit2.so.26
- _pygit2.so
- and some other libraries like requests etc
Have anyone experienced anything similar?
Thanks
I have now managed to solve the problem by creating a lambda layer. The problem seems to be with the python packages that are in the lambda container by default.
See details in this issue I raised here.
Hopefully this will help others too.

Resources