I have two versions of python installed on centos 6.4. It came with python 2.6.6. I installed 2.6.2 in /home/python.
My PATH variable has 2.6.6 in the path(/usr/bin/python). So in the console when I type 'python' it loads 2.6.6.
but, if I change the LD_LIBRARY_PATH to add /home/python/lib and no change to PATH variable, when I type 'python' at the console, it loads 2.6.2.
My understanding of executable search order in linux is 'looks at PATH' to find it. So how come the change in LD_LIBRARY_PATH loads 2.6.2? May be understanding is wrong. Could somebody help me understand this behavior?
"python" is a stub executable which loads the Python interpreter from its library. You've told it to look at a different set of libraries first, and so it uses the one it finds there rather than its normal library.
Related
I am trying to manually install a python3.8 module under conda. My module is a python wrapper of a C++ library. It comes with CMakeLists.txt. By specifying -DCMAKE_INSTALL_PREFIX to miniconda3 directory, I achieve that the corresponding .so module appears in miniconda3/lib/python3.8/dist-packages/.
Then, I run python in the same terminal and I can successfully import the module. But if I open another terminal, then the module cannot be found.
I have checked that both terminals run the same python (which python), and have identical environment (by comparing printenv outputs). I also have checked that sys.path values are identical and include miniconda3/lib/python3.8/. Still, the behavior is different.
So where does python look for modules?
The official documentation says that sys.path is the place, but it seems not the only place.
EDIT: This answer is most likely not related to the current problem.
Try running this command in your terminal before you open up python:
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/miniconda3/lib/python3.8/dist-packages
This isn't python looking for .so files, but rather ld (linker).
EDIT: The issue stemmed from different working directories in the used terminal windows. Python searches within sys.path and cwd (current working directory) for imported modules. More in the discussion under the question.
How do I setup a Python virtual environment with the FreeCAD library embedded as to enable import as a module into scripts?
I would like to avoid using the FreeCAD GUI as well as being dependent on having FreeCAD installed on the system, when working with Python scripts that use FreeCAD libraries to create and modify 3D geometry. I hope a Python virtual environment can make that possible.
I am working in PyCharm 2021.1.1 with Python 3.8 in virtualenv on Debian 10.
I started out with FreeCAD documentation for embedding in scripts as a basis:
https://wiki.freecadweb.org/Embedding_FreeCAD
In one attempt, I downloaded and unpacked .deb packages from Debian, taking care to get the correct versions required by each dependency. In another attempt, I copied the contents of a FreeCAD flatpak install, as it should contain all the libraries that FreeCAD depends on.
After placing the libraries to be imported in the virtual maching folder, I have pointed to them with sys.path.append() as well as PyCharm's Project Structure tool in various attempts. In both cases the virtual environment detects where FreeCAD.so is located, but fails to find any of its dependencies, even when located in the same folder. When importing these dependencies explicitly, each of them have the same issue. This leads to a dead end when an import fails because it does not define a module export function according to Python:
ImportError: dynamic module does not define module export function (PyInit_libnghttp2)
I seem to be looking at a very long chain of broken dependencies, even though I make the required libraries available and inform Python where they are located.
I would appreciate either straight up instructions for how to do this or pointers to documentation that describes importing FreeCAD libraries in Python virtual environments, as I have not come across anything that specific yet.
I came across a few prior questions which seemed to have similar intent, but no answers:
Embedding FreeCAD in python script
Is it possible to embed Blender/Freecad in a python program?
Similar questions for Conda focus on importing libraries from the host system rather than embedding them in the virtual environment:
Incude FreeCAD in system path for just one conda virtual environment
Other people's questions on FreeCAD forums went unanswered:
https://forum.freecadweb.org/viewtopic.php?t=27929
EDIT:
Figuring this out was a great learning experience. The problem with piecing dependencies together is that for that approach to work out, everything from the FreeCAD and its dependencies to the Python interpreter and its dependencies seems to need to be built on the same versions of the libraries that they depend on to avoid causing segmentation faults that brings everything to a crashing halt. This means that the idea of grabbing FreeCAD modules and libraries it depends on from a Flatpak installation is in theory not horrible, as all parts are built together using the same library versions. I just couldn't make it work out, presumably due to how the included libraries are located and difficulty identifying an executable for the included Python interpreter. In the end, I looked into the contents of the FreeCAD AppImage, and that turned out to have everything needed in a folder structure that appears to be very friendly to what PyCharm and Python expects from modules and libraries.
This is what I did to get FreeCAD to work with PyCharm and virtualenv:
Download FreeCAD AppImage
https://www.freecadweb.org/downloads.php
Make AppImage executable
chmod -v +x ~/Downloads/FreeCAD_*.AppImage
Create folder for extracting AppImage
mkdir -v ~/Documents/freecad_appimage
Extract AppImage from folder (note: this expands to close to 30000 files requiring in excess of 2 GB disk space)
cd ~/Documents/freecad_appimage
~/Downloads/./FreeCAD_*.AppImage --appimage-extract
Create folder for PyCharm project
mkdir -v ~/Documents/pycharm_freecad_project
Create pycharm project using Python interpreter from extracted AppImage
Location: ~/Documents/pycharm_freecad_project
New environment using: Virtualenv
Location: ~/Documents/pycharm_freecad_project/venv
Base interpreter: ~/Documents/freecad_appimage/squashfs-root/usr/bin/python
Inherit global site-packages: False
Make available to all projects: False
Add folder containing FreeCAD.so library as Content Root to PyCharm Project Structure and mark as Sources (by doing so, you shouldn't have to set PYTHONPATH or sys.path values, as PyCharm provides module location information to the interpreter)
File: Settings: Project: Project Structure: Add Content Root
~/Documents/freecad_appimage/squashfs-root/usr/lib
After this PyCharm is busy indexing files for a while.
Open Python Console in PyCharm and run command to check basic functioning
import FreeCAD
Create python script with example functionality
import FreeCAD
vec = FreeCAD.Base.Vector(0, 0, 0)
print(vec)
Run script
Debug script
All FreeCAD functionality I have used in my scripts so far has worked. However, one kink seems to be that the FreeCAD module needs to be imported before the Path module. Otherwise the Python interpreter exits with code 139 (interrupted by signal 11: SIGSEGV).
There are a couple of issues with PyCharm: It is showing a red squiggly line under the import name and claiming that an error has happened because "No module named 'FreeCAD'", even though the script is running perfectly. Also, PyCharm fails to provide code completion in the code view, even though it manages to do so in it's Python Console. I am creating new questions to address those issues and will update info here if I find a solution.
What is the difference between $PATH variable, sys.path and os.environ? I understand that they both serve as paths where python searches packages. But it'd be nice to have more elaborate response.
Just a working case from my practice is when I used the script with only os.environ before import on Ubuntu 16.04 I got ImportError: No module named XXX. At the same time on MacOS it worked well. After I added sys.path on Ubuntu I could get import module well.
This is actually more complicated than it would seem. It's unclear by the question if you understand the Linux/MacOS $PATH environment variable. Lets start there. The $PATH variable (in Python you're able to access the system environement variables from os.environ) denotes the current users $PATH variable as defined in various shell profile and environment files. It typically contains things like "/usr/bin" and other places where programs are installed. For example when you type "ls" into the system shell, the underlying system searches the $PATH for programs named "ls". So what actually gets executed is probably something like "/usr/bin/ls" I've included additional reading below.
sys.path on the other hand is constructed by Python when the interpreter is started, based on a number of things. The first sentence in the help page is as follows. "A list of strings that specifies the search path for modules. Initialized from the environment variable $PYTHONPATH, plus an installation-dependent default." The installation-dependent portion typically defines the installation location of Python site packages. $PYTHONPATH is another environment variable (like $PATH) which can be added to facilitate the module search location and can be set the same way the system $PATH can
Typically if you have non-installed sources (ie you have Python files that you want to run outside the site-packages directory) you typically need to manipulate sys.path either directly in your scripts or add the location to the $PYTHONPATH environment variable so the interpreter knows where to find your modules. Alternatively, you could use .pth files to manipulate the module search path as well
This is just a basic overview, I hope you read the docs for better understanding
Sources
Linux $PATH variable information
Python sys.path
Python site.py
sys.path
Is a list of strings that specifies the search path for modules. Initialized from the environment variable PYTHONPATH, plus an installation-dependent default.
os.environ
Is a mapping object representing the string environment. For example, environ['HOME'] is the pathname of your home directory (on some platforms), and is equivalent to getenv("HOME") in C.
Environment variable PATH
Specifies a set of directories where executable programs are located. In general, each executing process or user session has its own PATH setting.
I am working with multiple versions of Python, so I don't want to put any one of them in my windows path. I discovered that I can create a virtualenv (under Settings -> Project: X -> Project Interpreter and that will allow me to specify which python exe to use, but also where to get other useful programs, like pip. The end result is that using Virtualenv I can type "pip" or "python" in the terminal in Pycharm and use it as a regular terminal.
But now I'm working with Anaconda so I don't want to use virtualenv and I want to use conda instead. This works fine, but it doesn't seem to put anything into the path. (This is particularly important because where I intend to keep my project is not near my path to the python environment conda created.)
So if I type "pip" or "python" in the pycharm terminal, it doesn't recognize them. I have to specify a huge path to get it to work right.
I've been googling around and trying to find a solution to this problem. I've found a number of supposed solutions, but so far none have worked.
So, for example, I've already tried Add Content Root under project structure. This does nothing. I've tried add local, which seems to just screw everything up. (Presumably this is a way to create a local environment that I shouldn't be touching in conda.) I've tried adding a PathConfig.pth file. So far nothing works.
Remember, I don't want to add anything into the windows path (which obviously would solve my problem) because then it won't know which specific environment I want at the moment.
Background: I am trying to run a windows-installed version of python from within cygwin, rather than the cygwin version (some packages I am using prefer the non-cygwin version). The following post got me up and running: Using Windows Python from Cygwin. However, I am now having an issue.
Problem: The PYTHONPATH variable does not seem to be working for me. I try "import module1" when I am using python in an arbitrary directory. module1 is located here: c:\cygwin\home\mcsefl\testFolder\module1.py. And my PYTHONPATH variable is set in my .bashrc file in the following way: "export PYTHONPATH=$PYTHONPATH:/home/mcsefl/testFolder". However I still get a "No module named..." error.
Alternate tries: Since I am running the Windows Python, I thought perhaps python did not care about the cygwin environment variables (not sure if this is true..?) So, I also tried to set the windows PYTHONPATH env variable in the control panel of windows with no luck as well.
Thanks in advance
I deleted the line in my .bashrc file that tried to set the PYTHONPATH variable and just let it be set by Windows. This corrects the problem.
Side note: The title for this Question is a misnomer now that I re-read it. My issue was resolved without addressing that overall question