I'm trying out Freeling's API for python. The installation and test were ok, they provide a sample.py file that works perfectly (I've played around a little bit with it and it works).
So I was trying to use it on some other python code I have, in a different folder (I'm kind of guessing this is a path issue), but whenever I import freeling (like it shows on the sample.py):
import freeling
FREELINGDIR = "/usr/local";
DATA = FREELINGDIR+"/share/freeling/";
LANG="es";
freeling.util_init_locale("default");
I get this error:
ModuleNotFoundError: No module named 'freeling'.
The sample.py is located on the ~/Freeling-4.0/APIs/Python/ folder, while my other file is located in ~/project/, I dont know if that can be an issue.
Thank you!
A simple solution is to have a copy of freeling.py in the same directory as your code, since python will look there.
A better solution is to either paste it in one of the locations where it usually checks (like the lib folder in its install directory), or to tell it that the path where your file is should be scanned for a module.
You can check out this question to see how it can be done on Windows. You are basically just setting the PYTHONPATH environment variable, and there will only be minor differences in how to do so for other OSes. This page gives instructions that should work on Linux systems.
I like this answer since it adds the path at runtime in the script itself, doesn't make persistent changes, and is largely independent of the underlying OS (apart from the fact that you need to use the appropriate module path of course).
You need to set PYTHONPATH so python can find the modules if they are not in the same folder.
Related
I've come across some code recently that uses a trick that makes me rather nervous. The system I'm looking at has a Python extension Moo.so file stored outside the path and the developer wants to import it with just import Moo. For various reasons neither the file location nor sys.path can be changed, and the extension must be loaded with ExtensionFileLoader anyway.
So what has been done is to have a Moo.py in the path that loads the extension module and then replaces itself in sys.modules with the extension module, along the following lines:
' Moo.py '
from importlib.machinery import ExtensionFileLoader
loader = ExtensionFileLoader('AnotherNameForMoo', '/path/to/Moo.so')
module = loader.load_module()
sys.modules['Moo'] = module
Now this does actually work. (I have some tests of it in rather gory detail in this repo if you want to have a look.) It appears that, at least in CPython 3.4 through 3.7, import Moo does not bind to Moo the module that it loaded and put into sys.modules['Moo'], but instead binds the current value of sys.modules['Moo'] after the module's top-level script returns, regardless of whether or not that's what it originally put in there.
I can't find anything in any Python documentation that indicates that this is required behaviour rather than just an accident of implementation.
How safe is this? What are other ways that one might try to achieve a similar "bootstrap" effect?
I need some third-party c library to be imported into a low-level module. I'm following these instructions. It says that find_library() should help me find such a lib, excluding any lib prefix and .so suffix.
#next 2 lines just to test
test =find_library('spcm_linux')
print(test)
#this line below is the actual code
spcmDll = cdll.LoadLibrary("libspcm_linux.so")
Returns:
None
OSError: spcm_linux.so: cannot open shared object file: No such file or directory
My library lives at:
/usr/lib/gcc/x86_64-linux-gnu/libspcm_linux.so
Reading about find_library() from the docs, it tells me that I can set an environment variable to add an environement variable (LD_LIBRARY_PATH). So in /etc/environement I have:
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:"
PYTHONPATH="/home/fv/.local/lib/python3.6/site-packages"
LD_LIBRARY_PATH="/usr/lib/gcc/x86_64-linux-gnu"
I restart, try and it still doesn't work (same error message).
Any other way to "find" my library? Could put it in my program.py directory directly & import it some other way?
EDIT:
However, it does find libgomp.so.1, which is in the exact same folder as the library I'm trying to load. If that helps...
Turns out the root of the issue was rights.
Rights to the libspcm_linux.so lib weren't set properly and thus the script had no access rights to it.
However, the error that popped up was still about No such file or directory. I presume this is because it's a C library, which ctypes tried to load. It couldn't load it (because it didn't have the rights to it) however apparently it didn't see it fit to propagate the error (or couldn't because of some technicalities of how C code is loaded, perhaps).
Therefore from it's point of view, it couldn't load the requested library, and told me so - "No such file or..." even if this hide the real cause of the problem.
I've been working on a few scripts lately and I found that it would be really useful to separate some common functionalities on other files like 'utils' and the import them into my main scripts. For this, I used source ./utils.sh. However, It seems that this approach depends on the current path from where I'm calling my main script.
Let's say I have this folder structure:
scripts/
|-tools/
| |-utils.sh
|-main.sh
On main.sh:
source ./tools/utils.sh
some_function_defined_on_utils_sh
...
If I run main.sh from scripts/ folder everything works fine. But if I run it from a different directory ./scripts/main.sh the script fails because it can't find ./tools/utils.sh, which makes sense.
I understand why this doesn't work. My question is if there is any other mechanism to import 'modules' into my script, and make everything current-dir agnostic (just like python scripts from utils import some_function_defined_on_utils_sh))
Thanks!
Define an environment variable with a suitable default setting that is where you'll store your modules, and then read the module using the . (dot) command.
One serious option is simply to place the files in a directory already on your PATH — $HOME/bin is a plausible candidate for private material; /usr/local/bin for material to be shared. Then you won't even need to specify the path to the files; you can write . file-found-on-path to read it. Of course, the downside is that if someone switches their PATH setting, or puts a file with the same name in another directory earlier on their PATH, then you end up with a mess.
The files do not even need to be executable — they just need to be readable to be usable with the dot (.) command (or in Bash/C shells, the source command).
Or you could specify the location more exactly, such as:
. ${SHELL_UTILITY_DIR:-/opt/shell/bin}/tools/utils.sh
or something along those general lines. The choice of environment variable name and default location is up to you, and how much substructure you use on the directory is infinitely variable. Simplicity and consistency are paramount. (Also consider whether versioning will be a problem; how will you manage updates to your utilities?)
I have written a module for teaching Python. I'd like to make it difficult for the smarter ones to view the source code as a short-cut. Does not need to be fully secure - disabling the inspect module might be enough - if this is possible.
In case this is useful to anyone else using Python3 for class tests etc here's what I've ended up doing (with thanks to wbwlkr).
python3 -OO -m py_compile testmod.py creates a file __pycache__/testmod.cpython-34.pyo
Creating a symbolic link to this file named testmod.pyc means the code can't easily be inspected.
One other consideration is that sensitive local variables should be overwritten when not needed or they can be queried by locals()
What you are searching for is a solution to "obfuscate" the source code of your module.
You could compile your module to byte-code, as suggested here :
https://stackoverflow.com/a/7418341/8714367
I'm confused with some specific behaviour and can't find some informations that help me understand the error.
The situation is as follows: I made a small pyqt4 app that at some point dumps an OrderedDict to a yaml string using pyyaml or ruamel.yaml (tried both) and writes this to a file, or reads from this file. This goes very well executing the code as normal. Now I want to distribute my app by bundling it into a single file windows exe using pyinstaller.
Now if I directly use yaml.dump() or ruamel.yaml.dump() in a method of my pyqt4 form class to generate the yaml-string and write to a file (the standard way using with open ...), I am able to bundle the app using pyinstaller and the exe runs fine.
However, if I write a small function in a sub-folder/module that uses the exact same call to pyyaml (yaml.dump(dict)) or ruamel.yaml (ruamel.yaml.dump(dict, Dumper=ruamel.yaml.RoundTripDumper)) to generate the yaml string and save to a file using with open ... and use this in my pyqt4 method (I just wanted to make things more readable), pyinstaller starts to load a bunch of modules and does a lot more stuff (according to console output), resulting in the exe file beeing almost 5 times larger plus unusable throwing a fatal error pyi_rth_pkgres returned -1 at start.
Unfortunately, I don't understand much from either console output or warnings log, viewable in this gist. Maybe I am searching for the wrong terms. I also tried renaming the module to prevent shadowing.
Now my question is, does anybody know whats going on and can explain this behaviour?
After doing a lot of trial and error, I finally got it working.
I created a new module and build the dumping functions inside it. pyinstaller and the bundled exe work flawless. However, if i do the exact same thing in the previous module, even refactoring the name, it does not work. I even copied the complete code to the old module and it doesn't work. I have no idea why and at this point I am too afraid to ask :|
I am just glad it works now.