I have a flask application which I'm trying to deploy to heroku. I'm trying to do this through my virtual box where I'm running ubuntu 18.04 .
Up on command : git push heroku master
I see a warning :
WARNING: The Python installation you are using does not appear to have
been installed with a shared library, or in the case of MacOS X, as a
framework. Where these are not present, the compilation of mod_wsgi may
fail, or if it does succeed, will result in extra memory being used by all
processes at run time as a result of the static library needing to
be loaded in its entirety to every process. It is highly recommended
that you reinstall the Python installation being used from source code,
supplying the '--enable-shared' option to the 'configure' script when
configuring the source code prior to building and installing it.
and it fails with an error message:
/usr/bin/ld: final link failed: Bad value
remote: collect2: error: ld returned 1 exit status
remote: error: command 'gcc' failed with exit status 1
I understand that I need to install python with --enable-shared flag in the "configure" script. but I don't seem to find the configure script at all in my python installation under bin or python folder. This is where I'm stuck since 3 days and its frustrating; I don't really have an idea of how to re-install python with such a flag .. After going through many posts I tried the following steps:
1) Downloaded python from the url: https://www.python.org/downloads/source/
2) Ran the command: ./configure --prefix=/opt/python --enable-shared
followed by make
and make install
3) Copied the shared object files to my original python directory
The output to the command: ldd ** path **/anaconda3/bin/python gives me:
libpython3.6m.so.1.0 => /usr/anaconda3/bin/../lib/libpython3.6m.so.1.0
(0x00007f902dd2e000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0
(0x00007f902db0e000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f902d906000)
libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007f902d6fe000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f902d4f6000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f902d156000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f902cd5e000)
/lib64/ld-linux-x86-64.so.2 (0x00007f902e26e000)
Yet, when I tried to deploy the app, I see the same warning of --enable-shared and the same error message of gcc. I can't seem to figure out how to properly enable my python installation to have shared object. Please note that I'm a total newbie to ubuntu, I really appreciate any leads into solving this issue.
P.S: My mod_wsgi installation was successful and I was able to run the app locally through mod_wsgi.
If you don't need to use custom build of python3, you can try to install python3.6-dev package through apt, it added shared libs for me.
Related
Edit 1
I forgot to mention that I want to use the theano library.
After I reached out to the admin, the cudnn modules were provided by them. However, I am still getting the same error.
$ module load devel/cudnn/10.2
$ python -c "import theano"
Can not use cuDNN on context None: cannot compile with cuDNN. We got this error:
/pfs/work7/workspace/scratch/ul_dco32-conda-0/conda/envs/my_env/bin/../lib/
gcc/x86_64-conda_cos6-linux-gnu/7.3.0/../../../../
x86_64-conda_cos6-linux-gnu/bin/ld: cannot find -lcudnn
collect2: error: ld returned 1 exit status
Mapped name None to device cuda: Tesla V100-SXM2-32GB (0000:3A:00.0)
$ vi $HOME/.theanorc
[global]
floatX = float32
device = cuda
[cuda]
root=$CUDA_HOME/bin
[dnn]
include_path=$CUDA_HOME/include
library_path=$CUDA_HOME/lib64
[lib]
cnmem=1
So, most probably, I have a link problem, however, I can't find it where.
Problem
I am struggling with ld: cannot find -lcudnn.
Description
I want to use Cuda and CuDnn for my project. I am working in a cluster, Cuda is installed in the cluster already (p.s. I don't have root permissions). So I copy the Cuda files into my local folder and install Cudnn as instructed on the official website. However, ld cannot find libcudnn.so.
$ lsb_release -a
Description: Red Hat Enterprise Linux Server release 7.7 (Maipo)
Release: 7.7
What I tried
$ ld -lcudnn --verbose
attempt to open //usr/x86_64-redhat-linux/lib64/libcudnn.so failed
attempt to open //usr/x86_64-redhat-linux/lib64/libcudnn.a failed
attempt to open //usr/lib64/libcudnn.so failed
attempt to open //usr/lib64/libcudnn.a failed
attempt to open //usr/local/lib64/libcudnn.so failed
attempt to open //usr/local/lib64/libcudnn.a failed
attempt to open //lib64/libcudnn.so failed
attempt to open //lib64/libcudnn.a failed
attempt to open //usr/x86_64-redhat-linux/lib/libcudnn.so failed
attempt to open //usr/x86_64-redhat-linux/lib/libcudnn.a failed
attempt to open //usr/local/lib/libcudnn.so failed
attempt to open //usr/local/lib/libcudnn.a failed
attempt to open //lib/libcudnn.so failed
attempt to open //lib/libcudnn.a failed
attempt to open //usr/lib/libcudnn.so failed
attempt to open //usr/lib/libcudnn.a failed
ld: cannot find -lcudnn
If I add the library path of libcudnn.so manually
$ ld -L "$CUDA_HOME/lib64" -lcudnn
attempt to open /home/ul/ul_student/ul_dco32/pkg/cuda/10.2/lib64/libcudnn.so succeeded
-lcudnn (/home/ul/ul_student/ul_dco32/pkg/cuda/10.2/lib64/libcudnn.so)
librt.so.1 needed by /home/ul/ul_student/ul_dco32/pkg/cuda/10.2/lib64/libcudnn.so
found librt.so.1 at /usr/lib64/librt.so.1
libpthread.so.0 needed by /home/ul/ul_student/ul_dco32/pkg/cuda/10.2/lib64/libcudnn.so
found libpthread.so.0 at /usr/lib64/libpthread.so.0
libdl.so.2 needed by /home/ul/ul_student/ul_dco32/pkg/cuda/10.2/lib64/libcudnn.so
found libdl.so.2 at /usr/lib64/libdl.so.2
libstdc++.so.6 needed by /home/ul/ul_student/ul_dco32/pkg/cuda/10.2/lib64/libcudnn.so
found libstdc++.so.6 at /usr/lib64/libstdc++.so.6
libm.so.6 needed by /home/ul/ul_student/ul_dco32/pkg/cuda/10.2/lib64/libcudnn.so
found libm.so.6 at /usr/lib64/libm.so.6
libgcc_s.so.1 needed by /home/ul/ul_student/ul_dco32/pkg/cuda/10.2/lib64/libcudnn.so
found libgcc_s.so.1 at /usr/lib64/libgcc_s.so.1
libc.so.6 needed by /home/ul/ul_student/ul_dco32/pkg/cuda/10.2/lib64/libcudnn.so
found libc.so.6 at /usr/lib64/libc.so.6
ld-linux-x86-64.so.2 needed by /home/ul/ul_student/ul_dco32/pkg/cuda/10.2/lib64/libcudnn.so
found ld-linux-x86-64.so.2 at /usr/lib64/ld-linux-x86-64.so.2
ld: warning: cannot find entry symbol _start; not setting start address
Here is my LD_LIBRARY_PATH
$ echo $LD_LIBRARY_PATH
/home/ul/ul_student/ul_dco32/pkg/cuda/10.2/lib64
ld somehow ignores LD_LIBRARY_PATH. Since I do not have the root permissions, I cannot either create a symbolic link or make changes in ldconfig. So, setting LD_LIBRARY_PATH is the only way I can fix it.
Thanks for your help.
(This could have been a comment but I don't have enough reputation for that; I am talking only about the general linker etc usage, I don't know about the cuda or cudnn projects and what build tools they use, etc)
If I add the library path of libcudnn.so manually
$ ld -L "$CUDA_HOME/lib64" -lcudnn
This is a link line with no object files to link and create an executable with.
attempt to open /home/ul/ul_student/ul_dco32/pkg/cuda/10.2/lib64/libcudnn.so succeeded
-lcudnn (/home/ul/ul_student/ul_dco32/pkg/cuda/10.2/lib64/libcudnn.so)
:
This shows you got the right path into -L.
ld: warning: cannot find entry symbol _start; not setting start address
This will be fixed once you use a proper link line perhaps by incorporating the -L path you have correctly specified above into the build process e.g. the Makefile or whatever else is appropriate.
Here is my LD_LIBRARY_PATH
$ echo $LD_LIBRARY_PATH
/home/ul/ul_student/ul_dco32/pkg/cuda/10.2/lib64
ld somehow ignores LD_LIBRARY_PATH. Since I do not have the root
permissions, I cannot either create a symbolic link or make changes in
ldconfig. So, setting LD_LIBRARY_PATH is the only way I can fix it.
LD_LIBRARY_PATH is not for use by ld at link time, it is for telling ld.so the runtime linker (dynamic loader) where to look when searching for shared libraries to load into the running program.
It turned out to be the problem was the theano configuration. Declaring the full path of CUDA_HOME (or CUDNN_HOME) than using $CUDA_HOME (or $CUDNN_HOME) solved my problem.
[cuda]
root=path/to/cuda/bin
[dnn]
include_path=path/to/cudnn/include
library_path=path/to/cudnn/lib64
P.S. The Cudnn and Cuda modules were installed by the admin, so one big hassle was gone.
The problem is that I'm trying to use nodejs for building some css files from scss. On the local computer everything is ok.
When I run npm run production it shows me the error:
ERROR in ./resources/assets/sass/images/promo.jpg
Module build failed: Error: Command failed: /var/www/vhosts/site.ru/shared/node_modules/mozjpeg/vendor/cjpeg
/var/www/vhosts/site.ru/shared/node_modules/mozjpeg/vendor/cjpeg: error while loading
shared libraries: libpng16.so.16: cannot open shared object file: No such file or
directory
But when I run the same command from another directory(For example: /var/www/vhosts/site2.ru/) everything is ok(script works with jpg files too).
If I run ldconfig -p | grep libpng I see the next:
libpng12.so.0 (libc6,x86-64) => /lib/x86_64-linux-gnu/libpng12.so.0
libpng12.so.0 (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/libpng12.so.0
So, my Libng12 is working correctly, why does the system needs in libpng16?
And what should I try do in this situation?
My system is Ubuntu 16.04
Find the solution:
apt-get install libpng16-dev
so, I just installed the required library.
I've just compiled an executable with stack build and then just copying over the executable to a different machine, however running the application results in:
./app: error while loading shared libraries: libcrypto.so.1.1: cannot open shared object file: No such file or directory
I'm guessing I need to install some packages from the appropriate Linux distro's packages - but is there more efficient way then googling for each error that comes up and trying to identify the package?
I've tried running executable within some docker images:
fpco/haskell-scratch, debian:stretch-slim which all result in the same behaviour:
sudo docker run --rm -it -v $(pwd):/abcxyz fpco/haskell-scratch sh
# /abcxyz/app
/abcxyz/app: error while loading shared libraries: libcrypto.so.1.1: cannot open shared object file: No such file or directory
Update
With the debian image I installed openssl:
apt-get install openssl
New error:
/abcxyz/app: error while loading shared libraries: libgmp.so.10: cannot open shared object file: No such file or directory
apt-get install libgmp-dev
And it works...
This is actually not specific to Haskell, but rather to the way Linux dynamically links shared libraries. You can use ldd <file>:
% ldd ./StateSort
linux-vdso.so.1 (0x00007ffd29385000)
libm.so.6 => /usr/lib/libm.so.6 (0x00007fd986bd1000)
libgmp.so.10 => /usr/lib/libgmp.so.10 (0x00007fd98693e000)
librt.so.1 => /usr/lib/librt.so.1 (0x00007fd986736000)
libdl.so.2 => /usr/lib/libdl.so.2 (0x00007fd986532000)
libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007fd986314000)
libc.so.6 => /usr/lib/libc.so.6 (0x00007fd985f5c000)
/lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007fd986f1d000)
If there are missing libraries, you will notice them right away: the right hand side with the path to the so file will be absent.
You will then have to refer to your Linux distro's package management system to find out what package may contain the desired so file. For Ubuntu you would use apt-file, for ArchLinux − pkgfile, and so on.
I must warn you that ldd actually executes some of the code found in a file, so a malicious program may take over and wreak havoc. If you want to dissect an executable of unknown origin this way, you must do so in a disposable forensic containment. As I understand, this is not applicable in the case at hand, but nevertheless I must warn you.
Problem
I'm trying to install Python 3 with the --enable-shared option. Installation "succeeds" but the resulting Python is not runnable. Trying to run Python after installation gives the following error:
$ /opt/python3/bin/python3.5
/opt/python3/bin/python3.5: error while loading shared libraries: libpython3.5m.so.1.0: cannot open shared object file: No such file or directory
Background
The OS is Debian (squeeze), and has a previous installation of Python 2.6, which is necessary to retain because other code relies on it, and Apache 2.2. Ultimately what I'm trying to do is set up Django to run on Apache, meaning I'm trying to install mod_wsgi (or mod_wsgi-express), which requires shared libraries. I have already tried to install mod_wsgi without using --enable-shared in the Python installation, and have gotten... well, the same thing, but this time from the mod_wsgi installer (and from pip install mod_wsgi, which I also tried): /opt/python3/bin/python3.5: error while loading shared libraries: libpython3.5m.so.1.0: cannot open shared object file: No such file or directory.
Trace
Starting from an installation as described in Background above, here is the minimum list of commands I've executed that produce the error above (with verbosity removed).
user#server:~$ wget https://www.python.org/ftp/python/3.5.1/Python-3.5.1.tgz
user#server:~$ tar -zxvf Python-3.5.1.tgz
user#server:~$ cd Python-3.5.1
user#server:~/Python-3.5.1$ ./configure --prefix=/opt/python3 --enable-shared
user#server:~/Python-3.5.1$ make && sudo make install
(... appears to install correctly)
user#server:~/Python-3.5.1$ /opt/python3/bin/python3.5
/opt/python3/bin/python3.5: error while loading shared libraries: libpython3.5m.so.1.0: cannot open shared object file: No such file or directory
I have also tried this with LD_RUN_PATH set as described in the solution to this other question, with the same results:
user#server:~/Python-3.5.1$ sudo make distclean
user#server:~/Python-3.5.1$ ./configure --prefix=/opt/python3 --enable-shared
user#server:~/Python-3.5.1$ LD_RUN_PATH=/usr/local/lib make
user#server:~/Python-3.5.1$ sudo make install
user#server:~/Python-3.5.1$ /opt/python3/bin/python3.5
/opt/python3/bin/python3.5: error while loading shared libraries: libpython3.5m.so.1.0: cannot open shared object file: No such file or directory
I have also tried this with Python 3.4, with the same results. I have not tried this with Python 2, because I do not want future development to be limited to Python 2.7 (therefore even a successful installation would not satisfy my requirements). I'm also assuming the attempt would not provide any new or useful information.
I've repeated your steps on CentOS7 and get something similar:
$ /tmp/py3/bin/python3
/tmp/py3/bin/python3: error while loading shared libraries: libpython3.5m.so.1.0: cannot open shared object file: No such file or directory
If you look at the linking of python, it isn't providing a full path to the library:
$ ldd /tmp/py3/bin/python3
linux-vdso.so.1 => (0x00007fff47ba5000)
libpython3.5m.so.1.0 => not found
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fdfaa32e000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007fdfaa12a000)
libutil.so.1 => /lib64/libutil.so.1 (0x00007fdfa9f27000)
libm.so.6 => /lib64/libm.so.6 (0x00007fdfa9c24000)
libc.so.6 => /lib64/libc.so.6 (0x00007fdfa9862000)
/lib64/ld-linux-x86-64.so.2 (0x000055e85eac5000)
For some reason, the Python build process isn't adding -rpath to the link line, which would "add a directory to the runtime library search path."
If you explicitly set your library path, it will work:
$ LD_LIBRARY_PATH=/tmp/py3/lib/ /tmp/py3/bin/python3
Python 3.5.1 (default, Jun 10 2016, 14:54:59)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
So now it becomes a question as to whether you want to:
set LD_LIBRARY_PATH globally on your system
Edit /etc/ld.so.conf (or /etc/ld.so.conf.d/*)
Use chrpath to change the embedded path
export LD_RUN_PATH={prefix}/lib before you run configure and make
(Where {prefix} is what you passed to --prefix. You used the wrong path.)
As explained in the mod_wsgi documentation, set LD_RUN_PATH at the time of installing mod_wsgi.
http://modwsgi.readthedocs.io/en/develop/user-guides/installation-issues.html?highlight=ld_run_path#unable-to-find-python-shared-library
I am using ft232r library provided by ftdi for programming an LPC11C14 micro-controller through Linux Mint. To initialize the software, I need to run the following command:
./ft232r_prog --manufacturer Sunswift --product $(PROJECT_NAME) --invert_rts --invert_dtr
When I run the code, I get the following issue:
Error while loading shared libraries: libftdi.so.1: cannot open shared object file: No such file or directory
on running ldd ft232r_prog, I get:
linux-gate.so.1 => (0xf77b8000)
libusb-0.1.so.4 => /lib/i386-linux-gnu/libusb-0.1.so.4 (0xf7790000)
libftdi.so.1 => not found
libc.so.6 => /lib/i386-linux-gnu/libc.so.6 (0xf75e5000)
/lib/ld-linux.so.2 (0xf77b9000)
The file libftdi.so.1 is located in /usr/lib/x86_64-linux-gnu. Since the executable ft232r_prog is unable to find the .so file, I have tried the following:
Updated the path environment variable to contain /usr/lib/x86_64-linux-gnu -- Failed
Updated the $LD_LIBRARY_PATH environment variable to contain /usr/lib/x86_64-linux-gnu -- Failed
Ran ldconfig in /usr/lib/x86_64-linux-gnu -- Failed
This appears to be a common issue with including shared libraries. Any ideas on how I can resolve it?
Thanks
To install 32-bit version of libftdi on Ubuntu 12.04 x64 try following:
$ sudo apt-get install libftdi1:i386
I suspect your system is 64-bit and the program is 32-bit. In this case, you need to install the 32-bit version of the library.