EC2 + Apache + Miniconda + mod_wsgi import errors - python-3.x

I'm trying to get apache and mod_wsgi to work on an amazon ec2 instance, using a miniconda3 virtual environment and I am unable to get the WSGIDaemonProcess to correctly load the python venv modules (getting the 'encodings' module not found errors). I have run mod_wsgi-express module-config to get the proper path settings.
The relevant lines from my http.conf:
LoadModule wsgi_module "/home/ec2-user/miniconda3/envs/ebenv/lib/python3.6/site-packages/mod_wsgi/server/mod_wsgi-py36.cpython-36m-x86_64-linux-gnu.so"
WSGIPythonHome "/home/ec2-user/miniconda3/envs/ebenv"
WSGIDaemonProcess django python-home=/home/ec2-user/miniconda3/envs/ebenv python-path=/home/ec2-user/enerbits
WSGIProcessGroup django
WSGIScriptAlias / /home/ec2-user/repo/website/wsgi.py
When I perform a sudo strace httpd -X I can see that it is trying to load files from a /lib64 dir that doesn't exist in the venv. Here is the output:
stat("/home/ec2-user/miniconda3/envs/ebenv/lib64/python3.6", 0x7ffe873e1860) = -1 ENOENT (No such file or directory)
stat("/home/ec2-user/miniconda3/envs/ebenv/lib64", 0x7ffe873e1860) = -1 ENOENT (No such file or directory)
stat("/home/ec2-user/miniconda3/envs/ebenv", {st_mode=S_IFDIR|0775, st_size=4096, ...}) = 0
stat("/home/ec2-user/miniconda3/envs/ebenv/lib64/python3.6", 0x7ffe873e0ec0) = -1 ENOENT (No such file or directory)
stat("/home/ec2-user/miniconda3/envs/ebenv/lib64/python3.6/lib-dynload", 0x7ffe873e1860) = -1 ENOENT (No such file or directory)
stat("/home/ec2-user/miniconda3/envs/ebenv/lib64/python3.6", 0x7ffe873e1860) = -1 ENOENT (No such file or directory)
stat("/home/ec2-user/miniconda3/envs/ebenv/lib64", 0x7ffe873e1860) = -1 ENOENT (No such file or directory)
stat("/home/ec2-user/miniconda3/envs/ebenv", {st_mode=S_IFDIR|0775, st_size=4096, ...}) = 0
stat("/home/ec2-user/miniconda3/envs/ebenv/lib64/python3.6/lib-dynload", 0x7ffe873e0ec0) = -1 ENOENT (No such file or directory)
I have tried reinstalling miniconda, mod_wsgi (both using pip and building from source) to no avail. I've searched around and found a few posts that detailed a solution to point the python-path to the site-packages dir or even moving libs into a different dir which seem like a dodgy option. The miniconda3 install is 64-bit.
To note, previously I had been using virtualenv to make a python env and it was working fine (I suppose because the /lib64 dir is used) but I need to use conda for a pytorch installation. Any help is greatly appreciated. I am not particularly well versed in sysadmin stuff of this type so I am somewhat muddling my way through it all.

Related

ERROR: In /vtk/IO/Import/vtkOBJImporter.cxx

I've tried to run tutorial example of kuka robot from website https://drake.guzhaoyuan.com/ but it failed while reading .obj files.
command: ./bazel-bin/manipulation/util/geometry_inspector ./manipulation/models/iiwa_description/sdf/iiwa14_no_collision.sdf
error: ERROR: In /vtk/IO/Import/vtkOBJImporter.cxx, line 495 vtkOBJPolyDataProcessor (0x5620bd39e440): Error reading 'v' at line 3
Things I've tried so far:
Open .obj files with Mashlab and save with character "v" (3 floats / 6 floats)
Open .obj files with Mashlab and save with character "vn" (3 floats)
Software:
Ubuntu 18.04.4 LTS
VTK version 7.1.1
The "VTK version 7.1.1" is a red flag for me. Drake explicitly uses version 8.2. So, the error may simply come down to the fact that you're connecting to an archaic vtk.
Try:
strace -o trace_output.txt python ./bazel-bin/manipulation/util/geometry_inspector ./manipulation/models/iiwa_description/sdf/iiwa14_no_collision.sdf
And then search te file trace_output.txt for libvtkIOGeometry. My results include:
stat("/home/seancurtis/code/drake/bazel-bin/manipulation/util/geometry_inspector.runfiles/drake/bindings/pydrake/../../_solib_k8/_U#vtk_S_S_CvtkIOGeometry___Ulib/haswell", 0x7ffd1a5cb790) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/home/seancurtis/code/drake/bazel-bin/manipulation/util/geometry_inspector.runfiles/drake/bindings/pydrake/../../_solib_k8/_U#vtk_S_S_CvtkIOGeometry___Ulib/x86_64/libvtkIOGeometry-8.2.so.1", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/home/seancurtis/code/drake/bazel-bin/manipulation/util/geometry_inspector.runfiles/drake/bindings/pydrake/../../_solib_k8/_U#vtk_S_S_CvtkIOGeometry___Ulib/x86_64", 0x7ffd1a5cb790) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/home/seancurtis/code/drake/bazel-bin/manipulation/util/geometry_inspector.runfiles/drake/bindings/pydrake/../../_solib_k8/_U#vtk_S_S_CvtkIOGeometry___Ulib/libvtkIOGeometry-8.2.so.1", O_RDONLY|O_CLOEXEC) = 3
I've only included two lines: THere are multiple instances that finish with = -1 ENOENT (No such file or directory) and one that ends in = 3. In all cases, the library its looking for is: libvtkIOGeometry-8.2.so.1.
What does your show? I suspect the wrong one. If it does, we'll have to drag one of the build experts in to figure out why you're getting the wrong vtk. But we can prime that with the answers to the following question:
How do you have drake on your machine? Install? Build from source?

Cannot open a python file in Ubuntu Terminal for Windows 10

I'd like to open a python file (main.py) in my Ubuntu terminal for windows 10, how would I go about doing that? I've tried simply opening terminal and entering "python main.py" or "python3 main.py" but I get errno2 (no such file or directory).
I've tried changing the directory to C: to access the file, and I've tried doing python (filepath) of the file.
python3: can't open file 'main.py': [Errno 2] No such file or directory
I expected for main.py to be open and ran in Ubuntu terminal
It sounds like main.py isn't in the same directory you're currently in.
You may want to familiarize yourself with Bash, the shell which Ubuntu (and most Linux distributions) uses by default, and what you're likely using in your terminal.
To see the directory you're currently in, use pwd:
cball#cball:~$ pwd
/home/cball
To see what is in your current directory, use ls:
cball#cball:~$ ls
Desktop
Documents
Downloads
...
You can move into another directory cd directoryName:
cball#cball:~$ cd Desktop/
cball#cball:~/Desktop$ ls
cats.jpg
Here is a fairly gentle intro to linux: https://www.digitalocean.com/community/tutorials/an-introduction-to-linux-basics
Once you know where your Python file is, you can run python /path/to/main.py or simply python main.py if you are already in that directory (or if main.py is in your path variable).
Good luck learning bash, linux, and python! :) Please let me know if you have any questions.

open("../data/../data/ in terminal ubuntu

I have a code which compiles. When I run the executable file in terminal the following error appears:
terminate called after throwing an instance of 'chrono::ChException'
what(): Cannot open stream
Aborted (core dumped)
I then run:
strace -e trace=open ./myexe
And I see that the program fails at:
open("../data/../data/_template_POV.pov", O_RDONLY) = -1 ENOENT (No such file or directory)
I have a file called "_template_POV.pov" but I don't know where to paste the file since do not know what "../data/../data/" means.
Can anyone help me?
Regards,
Anders
open("../data/../data/_template_POV.pov", O_RDONLY)
means that you should have your file in a directory ../data/ with a relative path of your executable. Could you check this ?

Can't load mod_wsgi compiled for Python 3

I'm on CentOS and trying to configure Apache to use mod_wsgi compiled against Anaconda Python 3.5. Compiling mod_wsgi seems to go OK:
sudo yum install httpd-devel
sudo ./configure --with-python=/opt/anaconda/anaconda3/bin/python
sudo make
ls -l /etc/httpd/modules/mod_wsgi.so
-rwxr-xr-x. 1 root root 702205 Mar 2 23:12 /etc/httpd/modules/mod_wsgi.so
But when I start the web server it can't seem to find its libraries:
sudo service httpd start
Starting httpd: httpd: Syntax error on line 221 of /etc/httpd/conf/httpd.conf:
Syntax error on line 1 of /etc/httpd/conf.d/wsgi.conf: Cannot load
/etc/httpd/modules/mod_wsgi.so into server: libpython3.5m.so.1.0: cannot
open shared object file: No such file or directory
libpython3.5m.so.1.0 is definitely in /opt/anaconda/anaconda3/lib, so I suppose the mod_wsgi.so module doesn't know where to look for it. Yet the mod_wsgi config.status seems to confirm that
S["LDFLAGS"]=" -L/opt/anaconda/anaconda3/lib -L/opt/anaconda/anaconda3/lib/python3.5/config-3.5m "
so I'm at a loss... Can anyone point me in the right direction?
EDIT: OK, thanks to #mata, I think I've compiled mod_wsgi against the correct libraries. But now the server logs just fill up with the same repeated message:
Current thread 0x00007f60d68d07e0 (most recent call first):
Could not find platform independent libraries <prefix>
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
Fatal Python error: Py_Initialize: Unable to get the locale encoding
ImportError: No module named 'encodings'
So I'm guessing apache is looking in the wrong place for the python libraries. The 00-wsgi.conf file giving all the trouble reads:
WSGIPythonHome /opt/anaconda/anaconda3
WSGIPythonPath /opt/anaconda/anaconda3/lib/python3.5/site-packages
LoadModule wsgi_module modules/mod_wsgi.so
so I thought (from what I've read on other threads about this) that I'd told it where to look for my Python install.
That means that the library libpython3.5m.so.1.0 can't be found at runtime because the directory /opt/anaconda/anaconda3/lib is not a place where the dynamic linker would look for it.
You can try to rebuild mod_wsgi using:
./configure LDFLAGS='-Wl,-rpath=/opt/anaconda/anaconda3/lib' --with-python=/opt/anaconda/anaconda3/bin/python
That will save the library path within the generated binary.
The other option would be to set the LD_LIBRARY_PATH environment variable for the apache process, which is not really a good method.
Or add the directory /opt/anaconda/anaconda3/lib to the library search path using a conf file in /etc/ld.so.conf.d/, that would be a global setting tough. See man ld-linux for more info.
Also, don't forget to correctly set the WSGIPythonHome directive in your config file.
edit:
I've done some experimenting and I could reproduce your second error message when the python3 binary is not found on the PATH.
In that case it seems setting the WSGIPythonHome directive is not enough, you need to set the PYTHONHOME environment variable before apache is started, or change PATH so the interpreter can be found. On CentOS changing /etc/sysconfig/httpd should do the trick, just add:
export PYTHONHOME=/opt/anaconda/anaconda3
# alternatively this should also work:
export PATH="$PATH:/opt/anaconda/anaconda3/bin"
Or create a symlink to the interpreter in a directory on the path, e.g. /usr/local/bin...
For reference, an extended explanation why this is needed can be found here

Could not load SWT library error

What I want to achieve:
1) To install IBM Installation Manager in Ubuntu Server 14.04 LTS (HVM)
What I've done:
1) Installed "agent.installer.linux.gtk.x86_64_1.8.4000.20151125_0201.zip"
2) Run ./userinst command
Problems:
ubuntu#ip-172-31-20-133:~/wkMacDownloads$ ./userinst
00:00.42 ERROR [main] com.ibm.cic.agent.internal.ui.AgentUI
reportStatus
Could not load SWT library. Reasons:
/home/ubuntu/wkMacDownloads/configuration/org.eclipse.osgi/bundles/784/1/.cp/libswt-pi-gtk-4239.so
(libgtk-x11-2.0.so.0: cannot open shared object file: No such file or
directory)
swt-pi-gtk (Not found in java.library.path)
/home/ubuntu/.swt/lib/linux/x86_64/libswt-pi-gtk-4239.so
(/home/ubuntu/.swt/lib/linux/x86_64/liblibswt-pi-gtk-4239.so.so:
cannot open shared object file: No such file or directory)
/home/ubuntu/.swt/lib/linux/x86_64/libswt-pi-gtk.so
(/home/ubuntu/.swt/lib/linux/x86_64/liblibswt-pi-gtk.so.so: cannot
open shared object file: No such file or directory)
/home/ubuntu/.swt/lib/linux/x86_64/libswt-pi-gtk-4239.so
(libgtk-x11-2.0.so.0: cannot open shared object file: No such file or
directory)
java.lang.UnsatisfiedLinkError: Could not load SWT library. Reasons:
/home/ubuntu/wkMacDownloads/configuration/org.eclipse.osgi/bundles/784/1/.cp/libswt-pi-gtk-4239.so
(libgtk-x11-2.0.so.0: cannot open shared object file: No such file or
directory)
swt-pi-gtk (Not found in java.library.path)
/home/ubuntu/.swt/lib/linux/x86_64/libswt-pi-gtk-4239.so
(/home/ubuntu/.swt/lib/linux/x86_64/liblibswt-pi-gtk-4239.so.so:
cannot open shared object file: No such file or directory)
/home/ubuntu/.swt/lib/linux/x86_64/libswt-pi-gtk.so
(/home/ubuntu/.swt/lib/linux/x86_64/liblibswt-pi-gtk.so.so: cannot
open shared object file: No such file or directory)
/home/ubuntu/.swt/lib/linux/x86_64/libswt-pi-gtk-4239.so
(libgtk-x11-2.0.so.0: cannot open shared object file: No such file or
directory)
at org.eclipse.swt.internal.Library.loadLibrary(Library.java:331)
at org.eclipse.swt.internal.Library.loadLibrary(Library.java:240)
at org.eclipse.swt.internal.gtk.OS.<clinit>(OS.java:23)
at java.lang.J9VMInternals.initializeImpl(Native Method)
...
The displayed failed to initialize. See the log
Question:
Is it possible to use IBM Installation Manager GUI mode in Ubuntu Server? Or is there a command line alternative?
Use ./install instead.
To resolve your issue,
make sure the gtk libraries are installed to support the graphical user interface (GUI).refer
You could try to install IM silently.
You can run the command ./userinstc -log log_file -acceptLicense

Resources