Use 32 bit Python in 64 bit Redhawk - redhawksdr

I am attempting to develop a component for 64 bit Redhawk that uses a 32 bit Python binary, but am not sure how to configure Redhawk use 32 bit Python instead of 64 bit Python when running this particular component. The reason I need to use 32 bit Python is because I am using a library (PyVisa, http://pyvisa.readthedocs.org/en/latest/) that only works with 32 bit Python (see https://decibel.ni.com/content/docs/DOC-2928).
I am using PyVisa to control lab equipment (i.e. Agilent signal generators) in order to a calibrate a receiver. I understand this may seem like a odd use of Redhawk, but tuning sig gens to a certain freq and power level is very similar to operations that need to be performed when controlling a receiver/transmitter as well.
I would rather not install 32 bit CentOS just to develop and test this 1 component and I'm assuming it will be equally troublesome to simultaneously install 32 bit and 64 bit Redhawk rpms on my system.
I have already installed 32 bit Python simultaneously with 64 bit Python on my system and have manually renamed the 32 bit Python binary to be "python_32" and placed it on the path (/usr/bin/). I tried to force Redhawk to use the 32 bit Python executable by changing the shebang at the top of my implementation file to
#!/usr/bin/env python_32
However, it didn't work, so then I tried
#!/usr/bin/python_32
which also didn't work.
See example below for a skeleton implementation, with all the boilerplate imports and function defs removed for clarity.
#!/usr/bin/python_32
import visa
def process(self):
sg = visa.instrument("TCPIP::10.2.8.236")
print sg.ask("*IDN?")
return NORMAL
Each time I attempt to launch a simple waveform containing this component, I receive the error below. This is the same error I receive when trying to use the PyVisa libary in 64 bit Python, which is why I think Redhawk is ignoring my shebang statements specifying the path to 32 bit Python.
OSError: /usr/local/vxipnp/linux/bin/libvisa.so.7: wrong ELF class: ELFCLASS32
Traceback (most recent call last):
File "/var/redhawk/sdr/dev/.DevMgr_orion/GPP_orion/components/sendSCPI/python/sendSCPI.py", line 9, in <module>
import visa
File "/usr/lib/python2.6/site-packages/PyVISA-1.5.dev0.dev-py2.6.egg/visa.py", line 237, in <module>
resource_manager = ResourceManager()
File "/usr/lib/python2.6/site-packages/PyVISA-1.5.dev0.dev-py2.6.egg/vpp43.py", line 85, in __new__
it.init(*args, **kwds)
File "/usr/lib/python2.6/site-packages/PyVISA-1.5.dev0.dev-py2.6.egg/visa.py", line 230, in init
self.session = self.vi = vpp43.open_default_resource_manager()
File "/usr/lib/python2.6/site-packages/PyVISA-1.5.dev0.dev-py2.6.egg/vpp43.py", line 788, in open_default_resource_manager
visa_library().viOpenDefaultRM(byref(session))
File "/usr/lib/python2.6/site-packages/PyVISA-1.5.dev0.dev-py2.6.egg/vpp43.py", line 162, in __call__
self.load_library()
File "/usr/lib/python2.6/site-packages/PyVISA-1.5.dev0.dev-py2.6.egg/vpp43.py", line 131, in load_library
self.__lib = self.__cdecl_lib = cdll.LoadLibrary(path)
File "/usr/lib64/python2.6/ctypes/__init__.py", line 431, in LoadLibrary
return self._dlltype(name)
File "/usr/lib64/python2.6/ctypes/__init__.py", line 353, in __init__
self._handle = _dlopen(self._name, mode)
OSError: /usr/local/vxipnp/linux/bin/libvisa.so.7: wrong ELF class: ELFCLASS32
For reference I am running Redhawk version 1.9.0. x64 on CentOS 6.4 x64.
If there is a better way to control my lab equipment through Redhawk, I'm open to that too.

The <entrypoint> in the component.spd.xml file can be changed to point to a script instead of the component's .py file. This is useful for doing things like setting environment variables for a single component or redirecting to a different program.
Consider this:
Change <entrypoint>python/myComponent.py</entrypoint> to <entrypoint>python/myScript.sh</entrypoint>
Create a script called myScript.sh in the python folder.
Populate myScript.sh with something like:
export PYTHONPATH=$PYTHONPATH # add/change PYTHONPATH (or other environment variables) here
/usr/bin/python_32 python/myComponent.py $*
chmod +x myScript.sh
Modify your autotools files so that the script gets installed correctly (or just copy it by hand)
This approach gives you a little more control over how myComponent.py gets run.

Related

VirtualEnv and python-embed

I have a Windows machine running many apps using Python 2. I want to add a new program I wrote using Python 3. To make sure I don't screw anything up, I wanted to use a Virtualenv with Python 3 embedded (Python 3 embed amd64).
So I extracted the embedded python 3 and tried running virualenv with the --python option enabled.
I tried running the virtualenv
python -m virtualenv --python E:\Projects\python3-embed\python.exe E:\Projects\Virtual\
And I got this error:
Running virtualenv with interpreter E:\Projects\python3-embed\python.exe
Using base prefix 'E:\\Projects\\python3-embed'
Traceback (most recent call last):
File "C:\Python27\lib\site-packages\virtualenv.py", line 2328, in <module>
main()
File "C:\Python27\lib\site-packages\virtualenv.py", line 713, in main
symlink=options.symlink)
File "C:\Python27\lib\site-packages\virtualenv.py", line 925, in create_environment
site_packages=site_packages, clear=clear, symlink=symlink))
File "C:\Python27\lib\site-packages\virtualenv.py", line 1147, in install_python
writefile(site_filename_dst, SITE_PY)
File "C:\Python27\lib\site-packages\virtualenv.py", line 362, in writefile
with open(dest, 'wb') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'E:\\Projects\\Virtual\\python36.zip\\site.py'
I am not entirely sure how can I fix this error. There is a site.pyd file in python36.zip but it seems that virtualenv can't use it.
Sorry in advance
I knew this is not the answer you meant to ask for, but I tried hard that (our) way, and got this solution at last.
TL;DR
Update on 17 December 2021
The download page has been moved to https://winpython.github.io/
, for my purpose, I will download Winpython64-3.10.0.1dot.exe for now. Please don't forget to verify the checksum for the executable package.
End of the update on 17 December 2021
Use the portable installer with postfix "Zero" from WinPython
WinPython64-3.7.0.2Zero.exe(64 bit version, YOU want this)
WinPython32-3.7.0.2Zero.exe(32 bit version)
The Too Long version
What I tried
Yes, I downloaded the python-3.7.0-embed-amd64.zip from this page
Yes, I set it up according to instructions in this page
Yes, it worked as a python interpreter
D:\temp\test>python
Python 3.7.0 (v3.7.0:1bf9cc5093, Jun 27 2018, 04:59:51) [MSC v.1914 64 bit (AMD64)] on win32
>>>
Yes, I installed the package virtualenv
D:\temp\test>virtualenv --version
16.0.0
Then, boom (where you asked)
D:\temp\test>virtualenv ENV
Using base prefix 'd:\\portable\\python-3.7.0-embed-amd64'
Traceback (most recent call last):
File "runpy.py", line 193, in _run_module_as_main
File "runpy.py", line 85, in _run_code
File "D:\portable\python-3.7.0-embed-amd64\Scripts\virtualenv.exe\__main__.py", line 9, in <module>
File "D:\portable\python-3.7.0-embed-amd64\lib\site-packages\virtualenv.py", line 712, in main
symlink=options.symlink)
File "D:\portable\python-3.7.0-embed-amd64\lib\site-packages\virtualenv.py", line 927, in create_environment
site_packages=site_packages, clear=clear, symlink=symlink))
File "D:\portable\python-3.7.0-embed-amd64\lib\site-packages\virtualenv.py", line 1149, in install_python
writefile(site_filename_dst, SITE_PY)
File "D:\portable\python-3.7.0-embed-amd64\lib\site-packages\virtualenv.py", line 363, in writefile
with open(dest, 'wb') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'D:\\temp\\test\\ENV\\python37.zip\\site.py'
Seems a problem with virtualenv, searched, got an open issue, and I don't like that long solution.
This is the end of virtualenv.
Then, I tried venv:
D:\temp\test>python -m venv venv
D:\portable\python-3.7.0-embed-amd64\python.exe: No module named venv
Yes, the "built-in module" was lost, and the document for Embedded Distribution doesn't even mention it, just something about Tcl/tk (...), pip and the Python documentation are not included.
This is the end of venv.
Well, start again from my original point
The purpose to choose Embedded Distribution is (just like SOMEBODY):
To make sure I don't screw anything up
this include "don't install anything into my unstable and oversized Windows"
but doesn't necessarily "must use the Official Embedded Distribution"
something portable and trusted is enough, yes Portable Python
but the main page says "Portable Python is not being developed anymore", and provide some choice
after some reading, I figured out, what I want is WinPython, the version with a Zero
ta-da ... That's it.
Something about Embedded Distribution (from official document)
The embedded distribution is a ZIP file containing a minimal Python environment. It is intended for acting as part of another application, rather than being directly accessed by end-users.
The embedded distribution is meant to be lightweight. To install a virtual environment, follow these steps.
https://packaging.python.org/guides/installing-using-pip-and-virtual-environments/
Once these steps are done, proceed with the recommended approach and create a virtual environment using the command below. The --system-site-packages is typically used to bring a non embedded distribution packages already in the python install into the virtual environment, but since this is embedded, you can omit. But it won't hurt to have it.
python -m venv --system-site-packages C:\Python\python-3.7.9-embed-amd64\py37venv

Problems using ambhas package: statistics module has no cpdf attribute

I am trying to use the ambhas package for copulas and the method
foo.generate_xy()
is not working.
This is error I get:
Traceback (most recent call last):
File "<pyshell#2>", line 1, in <module>
generate()
File "C:/Users/Mypc/Desktop/sfsdf.py", line 19, in genera
x1,y1 = foo.generate_xy()
File "C:\Python33\lib\site-packages\ambhas\copula.py", line 179, in generate_xy
self._inverse_cdf()
File "C:\Python33\lib\site-packages\ambhas\copula.py", line 256, in _inverse_cdf
x2, x1 = st.cpdf(self.X, kernel = 'Epanechnikov', n = 100)
AttributeError: 'module' object has no attribute 'cpdf'
Basically I checked the code in the ambhas module and it turns out it is trying to use a method from the statistics module (imported as st), st.cpdf() which I do not have in my statistics module.
How can I fix this problem?
Here is an example of the code working. This is the very same code I am trying to run through the generate() function:
https://code.google.com/p/ambhas/wiki/Cookbook
The AMBHAS code depends on this statistics module, not the "official" one that is now included in Python 3.4. Unfortunately, the module doesn't appear to have been updated in a while, and the latest Windows installer is for Python 3.2, so you'll need to build it from source. You can do this with Cygwin, or by installing Visual C++ 2010 Express (here is a blog post describing the process, but I haven't tried it myself so I don't know if it's completely accurate - YMMV).
Once you've compiled the extension, make sure you remove the statistics module you installed previously before running python setup.py install.

ImportError with cx_freeze and pywin32: Module 'pythoncom' isn't in frozen sys.path

I am attempting to create a .exe on Windows 7 from a python3 script using cx_freeze. the Script involves using pywin32 to manipulate Excel files. I can build the .exe successfully from my setup.py file; however, when I run the .exe, the following error is thrown:
Traceback (most recent call last):
File
"C:\Python33\lib\site-packages\cx_Freeze\initscripts\Console3.py",
line 27, in exec(code,m_dict_)
File "MyScript.py", line
12, in < module >
File
"C:\Python\64-bit\3.3\lib\importlib_bootstrap.py", line 1558, in
_find_and_load
File "C:\Python\64-bit\3.3\lib\importlib_bootstrap.py", line 1505, in
_find_and_load_unlocked
File "C:\Python\64-bit\3.3\lib\importlib_bootstrap.py", line 313, in
_call_with_frames_removed
File "C:\Python\64-bit\3.3\lib\importlib_bootstrap.py", line 1558, in
_find_and_load
File "C:\Python\64-bit\3.3\lib\importlib_bootstrap.py", line 1525, in
_find_and_load_unlocked
File "C:\Python33\lib\site-packages\win32com__init__.py", line 6, in
< module>
import pythoncom
File
"C:\Python\64-bit\3.3\lib\importlib_bootstrap.py", line 1558, in
_find_and_load
File "C:\Python\64-bit\3.3\lib\importlib_bootstrap.py", line 1525, in
_find_and_load_unlocked
File "C:\Python33\lib\site-packages\pythoncom.py", line 3, in
pywintypes._import_pywin32_system_module_("pythoncom", globals())
File "C:\Python33\lib\site-packages\win32\lib\pywintypes.py", line 61,
in _import_pywin32_system_module_
raise ImportError("Module '%s'
isn't in frozen sys.path %s" % (modname, sys.path))
ImportError: Module 'pythoncom' isn't in frozen sys.path
['C:\Python33\build\exe.win-amd64\3.3\MyScript.exe',
'C:\Python33\build\exe.win-amd64\3.3',
'C:\Python33\build\exe.win-amd64\3.3\MyScript.zip',
'C:\Python33\build\exe.win-amd64\3.3\library.zip']
Here is what my setup.py file currently looks like:
import sys
from cx_Freeze import setup, Executable
base = None
if sys.platform == 'win32':
base = 'Win32GUI'
includes = []
packages = []
executables = [Executable('MyScript.py', base=base)]
include_files = ['MyFolder1/','MyFolder2/Spreadsheet.xls']
setup(name='My Script',
version='0.1',
description='My Script',
executables=executables,
options = {'build_exe': {'includes':includes,
'packages':packages,
'include_msvcr':True,
'include_files':include_files}})
So far, I have tried listing both 'pythoncom' and 'win32com' in both the includes and the packages lists. Any help is greatly appreciated!
I had exactly the same issue when using pyinstaller to pack py into executable. Tried everything, but find out the issue is hidden in pyinstaller version. I was using pyinstaller 4.7 but when downgrade into 3.6, the exe works flawlessly.
Looking at the code, it looks like you need to ensure that a file called something like pythoncom33.dll is copied into the build directory.
So the whole problem actually stemmed from having the 32-bit version of pywin32 installed while running 64-bit versions of Python-3.3.2 and Windows 7. After adding pythoncom33.dll to the include_files of my setup.py as Thomas K had suggested, I got another error:
ImportError: DLL load failed: %1 is not a valid Win32 application
After some research, I found that this error is typical when mixing 32-bit and 64-bit. So I uninstalled pywin32 32-bit and installed pywin32 64-bit but my script threw yet another error:
import win32api, sys, os
ImportError: DLL load failed: The specified module could not be found.
As suggested in this post, I copied the 28 win32*.pyd files from the Lib/site-packages/win32 folder to the Python33 folder right next to python.exe , and everything worked.
Add this to the top of your main.py:
import pywintypes
import win32api
see:
PyWin32 and Python 3.8.0
So I ran into a similar problem where cx_Freeze was failing to pull in the pywintypes DLL, but frustratingly enough it was only on some machines which compelled me to explore the issue a bit further (I have a bad habit of chasing rabbits down holes). It seems that pywin32 tries to install its DLLs into a Windows system directory, but if that fails due to a lack of privileges it will fallback to placing them in your Python directory instead. The behavior I was seeing was a result of the fact that cx_Freeze applies a blacklist of well-known system folder paths against the collection of dependencies it has discovered in order to avoid pulling in operating system DLLs.
Copying these DLLs as needed by adding a path to them to the "include_files" parameter will indeed address the issue (just remember that you can't guarantee that they will be placed in the system folder). An alternative solution is to override the blacklist with a whitelist which you can configure using the "bin_includes" / "bin_path_includes" parameters. Here is a quick example that shows how to configure this parameter using a setup.py file (for Python 2.7, but the only difference should be the pywintypes DLL filename):
from cx_Freeze import setup, Executable
# This can also be an absolute path to the file if you wish to be very specific
bin_includes = ["pywintypes27.dll"]
options = {
"build_exe": {
"bin_includes": bin_includes
}
}
executable = Executable(script='main.py')
setup(version='x.x.x',
options=options,
executables=[executable])
Update pyinstaller related vision.
python -m pip install pyinstaller==3.6

Pyro4 connecting python2 to python3

I have the following setup: Pyro nameserver running under python2.7, Pyro daemon running under python2.7, and (ideally) a Pyro client running python3.3.
If I connect to the server using a client running under python2.7 everything works fine. When using python3.3 for the client I can create the proxy connection just fine, but I get:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/project_path/lib/python3.3/site-packages/Pyro4/core.py", line 149,
return self.__send(self.__name, args, kwargs)
File "/project_path/lib/python3.3/site-packages/Pyro4/core.py", line 271,
self.__pyroCreateConnection()
File "/project_path/lib/python3.3/site-packages/Pyro4/core.py", line 322,
uri=resolve(self._pyroUri)
File "/project_path/lib/python3.3/site-packages/Pyro4/naming.py", line 336
nameserver=locateNS(uri.host, uri.port)
File "/project_path/lib/python3.3/site-packages/Pyro4/naming.py", line 274
proxy.ping()
File "/project_path/lib/python3.3/site-packages/Pyro4/core.py", line 149,
return self.__send(self.__name, args, kwargs)
File "/project_path/lib/python3.3/site-packages/Pyro4/core.py", line 290,
data=self._pyroSerializer.deserialize(data, compressed=flags & MessageFactory.FLAGS_COMPRESSED)
File "/project_path/lib/python3.3/site-packages/Pyro4/util.py", line 146,
return self.pickle.loads(data)
ImportError: No module named 'exceptions'
when trying to use any remote methods. The pyro docs seem to imply that I should be able to interconnect python2.7 instances with python3.3, is this not the case?
Both the 2.7 and 3.3 instance of python are using Pyro4 version 4.16
EDIT: Here's some actual code that's not working for me:
(With a python2 ns started)
In a python2.7 virtualenv:
import Pyro4
class TestProxy(object):
def foo(self):
return "bar"
if __name__ == "__main__":
print "* Starting test proxy"
daemon=Pyro4.Daemon()
tproxy_uri=daemon.register(TestProxy())
ns=Pyro4.locateNS()
ns.register("foo",tproxy_uri)
print "* Proxy started"
daemon.requestLoop()
and in a python3.3 virtualenv:
import Pyro4
rtest = Pyro4.Proxy("PYRONAME:foo")
print(rtest.foo())
I get that exact exception I put above.
The reason is pickle. It is the „protocol” used for exchanging object over the network.
That's why you cannot pass classes, but only objects and so on. Pickle for py3 has major changes, but 2.7 vs. 2.6 does not. So, in theory, pyro will work between 2.6 > 2.7, but not 2.7 > 3.3.
(it works, I have a VPS with python 2.6 and I am force to move client app on my comp on py2.7 too.) Probably a hack at pickle module may avoid the error, but I am sure that will open a pandora box. The only „real” alternative is to use a python server, over http/socket/https (not necesarly a web server) and send/recive serialized strings/commands.
P.S.: I see that the post is old!
What cox said is right. Pickle are different between python2,3. But serpent seems to be compatible.
You can try
PYRO_SERIALIZERS_ACCEPTED=serpent /usr/bin/python3 -Wignore -m Pyro4.naming

System Error when running PyQt4's loadUi()

I'm pretty new to Qt, Python and their combinations. I'm currently writing a QGIS plugin in python (I used QtCreator 2.1 (Qt Designer 4.7) to generate a .ui-file and am now trying to use it for a Quantum GIS plugin that's written in Python 2.5 (and running in the Quantum GIS Python 2.5 console)).
I am running into trouble when loading the ui-file dynamically when the program runs the loadUi() function. What throws me off is that the error occurs outside my script. Does that mean, I'm passing something wrong into it? Where does the error come in? Any hints on what could be wrong?
code_dir = os.path.dirname(os.path.abspath(__file__))
self.ui = loadUi(os.path.join(code_dir, "Ui_myfile.ui"), self)
This is the Error Code I am getting (minus the first paragraph):
File "C:/Dokumente und Einstellungen/name.name/.qgis/python/plugins\myfile\myfile_gui.py", line 42, in __ init __
self.ui = loadUi(os.path.join(code_dir, "Ui_myfile.ui"), self)
File "C:\PROGRA~1\QUANTU~1\apps\Python25\lib\site-packages\PyQt4\uic__init__.py", line 112, in loadUi
return DynamicUILoader().loadUi(uifile, baseinstance)
File "C:\PROGRA~1\QUANTU~1\apps\Python25\lib\site-packages\PyQt4\uic\Loader\loader.py", line 21, in loadUi
return self.parse(filename)
File "C:\PROGRA~1\QUANTU~1\apps\Python25\lib\site-packages\PyQt4\uic\uiparser.py", line 768, in parse
actor(elem)
File "C:\PROGRA~1\QUANTU~1\apps\Python25\lib\site-packages\PyQt4\uic\uiparser.py", line 616, in createUserInterface
self.traverseWidgetTree(elem)
File "C:\PROGRA~1\QUANTU~1\apps\Python25\lib\site-packages\PyQt4\uic\uiparser.py", line 594, in traverseWidgetTree
handler(self, child)
File "C:\PROGRA~1\QUANTU~1\apps\Python25\lib\site-packages\PyQt4\uic\uiparser.py", line 233, in createWidget
topwidget.setCentralWidget(widget)
SystemError: error return without exception set
I'm not sure of what could be causing this precise problem, but using .ui files directly has never worked well for me - instead I compile them to python code using pyuic4 (should be in your path if your PyQt4 site-packages are correctly configured. The syntax is along the lines of :-
pyuic4 -o <python output> -x <uic input>
-: resource files can similarly be converted to (not so human-readable) python using :-
pyrrc4 -o <python output> <qrc input>
-: You can then import that python file as a module, not to mention that reading its code can give you clues on how to fiddle with the layout at runtime.
The other upside of this is that you are not having to parse xml at runtime - importing a python module is far quicker and you are not likely to change the ui anywhere near as often as you run the script. If you do find yourself in this situation you can just create a batch to run these before your script/application.
This is the method I use in conjunction with pyInstaller to deploy scripts that will 'just run' on XP, Vista and Windows 7 without modification, and I generate the python modules just before compiling with pyInstaller, so it can be a real time saver.

Resources