I am using Pychromeless repo with success at AWS lambda.
But now I need to use pycrypto dependency, but I am getting
configure: error: no acceptable C compiler found in $PATH
when running make docker-build
(after placing pycrypto==2.6.1 at requirements.txt file).
There's this thread and someone said about the same problem:
"The gcc compiler is not in your $PATH. It means either you dont have gcc installed or it's not in your $PATH variable".
So tried placing apt-get install build-essential at Dockerfile, but I got
/bin/sh: apt-get: command not found
Then, I tried with yum install gcc
only to get
The command '/bin/sh -c yum install gcc' returned a non-zero code: 1
Docker-lambda [info page] (https://hub.docker.com/r/lambci/lambda/) says:
This project consists of a set of Docker images for each of the supported Lambda runtimes.
There are also a set of build images that include packages like gcc-c++, git, zip and the aws-cli for compiling and deploying.
So I guess I shouldn't be needing to install gcc. Maybe the gcc compiler is not in $PATH, but I don't know what to do to fix that.
Here is the dockerfile
FROM lambci/lambda:python3.6
MAINTAINER tech#21buttons.com
USER root
ENV APP_DIR /var/task
WORKDIR $APP_DIR
COPY requirements.txt .
COPY bin ./bin
COPY lib ./lib
RUN mkdir -p $APP_DIR/lib
RUN pip3 install -r requirements.txt -t /var/task/lib
Any help on solving this?
Well, well, well...today was a lucky day for me.
So simple: all I had to do was replace
pycrypto==2.6.1
by
pycryptodome
on my requirements.txt file.
This thread says: "Highly recommend NOT to use pycrypto. It is old and not maintained and contains many vulnerabilities. Use pycryptodome instead - it is compatible and up to date".
And that's it! Docker builds just fine with pycryptodome.
Related
I'm trying to build a docker with python 3 and google-cloud-bigquery with the following docker file:
FROM python:3.10-alpine
RUN pip3 install google-cloud-bigquery
WORKDIR /home
COPY *.py /home/
ENTRYPOINT ["python3", "-u", "myscript.py"]
But getting errors on the pip3 install google-cloud-bigquery (too long for here)..
What's missing for installing this on python-alpine?
Looks like an incompatibility issue with the latest version of google-cloud-bigquery (>3) and numpy:
ERROR: Could not build wheels for numpy, which is required to install pyproject.toml-based projects
Try specifying a previous version, this works for me:
RUN pip3 install google-cloud-bigquery==2.34.4
Actually it seems like not a problem with numpy, which builds smoothly with all the dependency libs install, but rather with pyarrow, which does not support alpine+pip build. I've found a workaround by using alpine pre-built version of pyarrow. It is much easier than building pyarrow from source. This build works for me just fine:
FROM python:3.10.6-alpine3.16
RUN apk add --no-cache build-base linux-headers \
py3-apache-arrow=8.0.0-r0
# Copying pyarrow to site-package of actual python path. Alpine python path
# and python's docker hub path are different.
RUN mv /usr/lib/python3.10/site-packages/* \
/usr/local/lib/python3.10/site-packages/
RUN rm -rf /usr/lib/python3.10
RUN --mount=type=cache,target=/root/.cache/pip \
pip install google-cloud-bigquery==3.3.2
Update python version, alpine version and py3-apache-arrow version to install later versions. This is the latest one on the time of writing.
And make sure to remove build dependencies (build-base, linux-headers) for your release docker. I prefer multistage dockers for this.
I run through the following steps to attempt to start up an app for production:
-Setup a virtualenv for the python dependencies: virtualenv -p /usr/bin/python3.8 ~/app_env
-Install pip dependencies: . ~/app_env/bin/activate && pip install -r ~/app/requirements.txt
-Un-comment the lines for privilege dropping in uwsgi.ini and change the uid and gid to your account name
-Login to root with sudo -s and re-source the env with source /home/usr/app_env/bin/activate
-Set the courthouse to production mode by setting the environment variable with export PRODUCTION=1
-Start the app: cd /home/usr/app && ./start_script.sh
And I get the following error:
(app_env) root#usr-Spin-SP314-53N:/home/usr/Desktop/app# ./start.sh
uwsgi: error while loading shared libraries: libpcre.so.1: cannot open shared object file: No such file or directory
I tried a few things such as installing a newer libpcre version like mentioned here, tried also the steps mentioned here but that didn't work. Also the environment I'm setting up doesn't use anaconda but regular python. I even tried pip install uwsgi in my virtual env but it said the requirement was already satisfied. I'm not much of an expert when it comes to somewhat complex package management like this, help with how to solve this would be greatly appreciated. Thanks. I'm on Ubuntu 20.04, using python 3.8.
What solved it for me was apparently just reinstalling UWSGI, like in this thread, in my virtual env while forcing it to ignore the cache so it could know to use the pcre library I installed.
In order, doing this
uwsgi --version
Was giving me this
uwsgi: error while loading shared libraries: libpcre.so.1: cannot open shared object file: No such file or directory
So I made sure I had the latest libpcre installed
sudo apt-get install libpcre3-dev
And then what linked it all together was this
pip install uwsgi -I --no-cache-dir
I tried to solve this error but it did not work no matter what I did and then reinstalled uwsgi, but the following 2 lines solved my problem
sudo find / -name libpcre.so.*
#change the path of the /home/anaconda3/lib/libpcre.so.1 with the one appears after above one.
sudo ln -s /home/anaconda3/lib/libpcre.so.1 /lib
which python
I am trying to set up a mysql connection that will work with SqlAlchemy in Python 3.6.5 . I have the following in my Dockerfile:
RUN pip3 install -r /event_git/requirements.txt
I also have, in requirements.txt:
mysql-connector-python==8.0.15
However, I am not able to connect to the DB. Is there anything else that I need to do to set this up?
Update:
I got 8.0.5 working but not 8.0.15 . Apparently, a protobuff dependency was added; does anyone know how to handle that?
docker file is:
RUN apt-get -y update && apt-get install -y python3 python3-pip fontconfig wget nodejs nodejs-legacy npm
RUN pip3 install --upgrade pip
# Copy contents of this directory (i.e. full source) to image
COPY . /my_project
# Install Python dependencies
RUN pip3 install -r /event_git/requirements.txt
# Set event_git folder as working directory
WORKDIR /my_project
ENV LANG C.UTF-8
I am running it via
docker build -t event_git .;docker run -t -i event_git /bin/bash
and then executing a script; the db is on my local machine. This is working on mysql-connector-python==8.0.5 but not 8.0.15, so the setup is ok; I think I just need to fulfill the protobuff dependency that was added (see https://github.com/pypa/warehouse/issues/5537 for mention of protobuff dependency).
The mysql-connector-python has the Python Protobuf as an installation requirement, this means that protobuf will be installed along mysql-connector-python.
If this doesn't work, try to add protobuf==3.6.1 in your requirements.txt.
Figured out the issue. The key is that import mysql.connector needs to be at the top of the file where the create_engine is. Still not sure of the exact reason, but at the very least that seems to define _CONNECTION_POOLS = {}. If anyone knows why, please do give your thoughts.
I'm trying to build pybind11 tests on a linux box. I downloaded the source and do the following -
cd pybind11-master
cd tests
mkdir build
cd build
cmake ..
I get the errors -
` Could not find a package configuration file provided by "pybind11" with any of
the following names:
pybind11Config.cmake
pybind11-config.cmake
Add the installation prefix of "pybind11" to CMAKE_PREFIX_PATH or set "pybind11_DIR" to a directory containing one of the above files. If "pybind11" provides a separate development package or SGK, be sure it has been installed
`
I followed this link - http://pybind11.readthedocs.io/en/master/basics.html and did as per the instruction in section 'Compiling Test Cases for linux/mac'
I am not sure how to proceed. Any pointers are helpful.
I already hit the same problem after installing pybind11 by pip install pybind11.
I'll post here my solution in case someone ends up here.
I installed following this link, everything went ok, and the file needed was there.
Basically:
$ git clone https://github.com/pybind/pybind11.git
$ cd pybind11
$ mkdir build
$ cd build
$ cmake ..
$ make -j`nproc`
$ make check
$ make -j`nproc` install
I have a new Linux install (Ubuntu Mini Remix 16.04) and I did the following:
sudo apt-get install build-essential
sudo apt-get install git
git clone https://github.com/openssl/openssl.git
cd openssl
./config --openssldir=/usr/local/ssl
make
make test
sudo make install
I then typed in openssl in the CLI and got this message:
openssl: error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory
Maybe /usr/local/lib is not in /etc/ld.so.conf?? If it is just try to run 'ldconfig' as root with no params, if it's not just add it and run 'ldconfig'.
try this:
do "which openssl" to confirm you're looking at the correct copy. do "ldd openssl" to get a list of what dynamic libraries it's expecting. Given the error you got there should be an error in the LDD output showing it can't find libssl.so.
Your "make install" may have simply left out "ldconfig". Do "ldconfig" to get the system to refresh it's information on where shared libraries are (it gets that from /etc/ld.so.conf and /etc/ld.so.conf.d/.
You can use the LD_LIBRARY_PATH environment variable to force it to use a certain directory to search for shared libraries, that often fixes this error.