Pyro4, receiving: not enough data - python-3.x

I'm trying to run a function on the server with Pyro4.
Have a look at the code:
==========================Client======================
import Pyro4
def square(x):
return x**2
remoteServer = Pyro4.Proxy('PYRONAME:server')
print(remoteServer.evaluate(square, 4))
===========================Server==========================
import Pyro4
class Server(object):
def evaluate(self, func, args):
return func(*args)
def main():
server = Server()
Pyro4.Daemon.serveSimple({server: "server"},ns=True)
if __name__ == '__main__':
main()
========================Naming Server========================
import Pyro4
Pyro4.naming.startNSloop()
===========================================================
And got an error: "Pyro4.errors.ConnectionClosedError: receiving: not enough data". Look at the full stack trace below:
Traceback (most recent call last):
File "C:\Users\Alex\Desktop\2.py", line 9, in <module>
print(remoteServer.evaluate(square, 4))
File "C:\Users\Alex\AppData\Roaming\Python\Python34\site-packages\Pyro4\core.py", line 169, in __call__
return self.__send(self.__name, args, kwargs)
File "C:\Users\Alex\AppData\Roaming\Python\Python34\site-packages\Pyro4\core.py", line 380, in _pyroInvoke
msg = message.Message.recv(self._pyroConnection, [message.MSG_RESULT], hmac_key=self._pyroHmacKey)
File "C:\Users\Alex\AppData\Roaming\Python\Python34\site-packages\Pyro4\message.py", line 161, in recv
msg = cls.from_header(connection.recv(cls.header_size))
File "C:\Users\Alex\AppData\Roaming\Python\Python34\site-packages\Pyro4\socketutil.py", line 460, in recv
return receiveData(self.sock, size)
File "C:\Users\Alex\AppData\Roaming\Python\Python34\site-packages\Pyro4\socketutil.py", line 195, in receiveData
raise err
Pyro4.errors.ConnectionClosedError: receiving: not enough data
I want the server executes the function that is passed to a method (function).
PS: Windows7, Python 3.4.1
I've found similar question (How to send a function to a remote Pyro object), but didn't work for me.
Tried to use callback decorator #pyro4.callback, but even official example (https://github.com/delmic/Pyro4/tree/master/examples/callback : server2.py, client2.py) didn't work (with python 3.4 and python 2.7) => only message: "server: doing callback 1 to client" is showed and nothing happens.
Thanks

The github repo you're linking to is not the official one. It's a clone (from quite an old version). This is the official repo: https://github.com/irmen/Pyro4
As to your problem: it is not possible to serialize functions and send them over the wire. If you tell Pyro to use pickle as a serialization protocol, you can do more tricks, but pickle still needs to have the module source or bytecodes available on both sides of the connetion.
The reason your client crashes is because the server aborts the connection due to a Pyro protocol error. If you enable the logging it will show up in the server's logfile, something like: "Pyro4.errors.ProtocolError: unsupported serialized class: builtins.function".
Also a tip: you don't have to start the name server using a separate source code file. You can start it directly from the command line.
Lastly, the #pyro4.callback decorator is unrelated, please read http://pythonhosted.org/Pyro4/clientcode.html?highlight=callback#pyro-callbacks for what it is intended for. It influences the way Pyro handles exceptions, nothing more.
edit: note that since Pyro 4.35 this error will be correctly reported back to the client and it will no longer get an aborted connection.

Related

PJSUA transfer call not working in python37 how ever it is working in python27

current_call.transfer("sip:1001#xx.xx.xx.xx") is working python2.7 How ever it is not working in python3.7
below is the error
Original exception was:
Traceback (most recent call last):
File "trycall.py", line 151, in <module>
current_call.transfer("sip:1001#xx.xx.xx.xx")
File "/usr/local/lib/python3.7/dist-packages/pjsua.py", line 1734, in transfer
Lib._create_msg_data(hdr_list))
SystemError: <built-in function call_xfer> returned NULL without setting an error
I am using this branch of PJSIP: Link
for PJSUA implementation with Python 3.6 and have encountered same problem with transfer.
Removing this check from function py_pjsua_call_xfer (pjproject/pjsip-apps/src/python/_pjsua.c) solved my problem:
if (!PyBytes_Check(pDstUri))
return NULL;
This check always returns NULL in all my tests. I was not able to solved this with Python. Removing code mentioned above solved the issue and for now it haven't created any new problems. I have tested this modification with Asterisk and 3 SIP endpoints and transfer was correctly processed.
(note: I am not a C/C++ programmer, so I cannot provide detail explenation why this code doesn't work. This approach is based on trial and error.)

How to add a new method to an existing import in python? Specifically moviepy

For whatever reason, Python is not allowing me to access a custom method I created in moviepy's preview.py file. I just want to know how to correctly implement it into the file. For reference, before I changed the name of the method, it was working correctly.
I checked at least two __init.py__ files and they were effectively empty. I couldn't find if methods are initialized anywhere, and is probably what I'm missing.
I also tried restarting Git Bash and that didn't work either (another solution I saw).
Original:
#convert_masks_to_RGB
def preview(clip, fps=15, audio=True, audio_fps=22050, audio_buffersize=3000,
audio_nbytes=2, fullscreen=False):
Changed:
#requires_duration
#convert_masks_to_RGB
def preview_custom(clip, marker_overlay="marker_overlay.png", fps=15, audio=True, audio_fps=22050, audio_buffersize=3000,
audio_nbytes=2, fullscreen=False):
There are more than a few differences between the changed and original method, however at the moment the only result I expect is having the method be called correctly. Error is below:
Traceback (most recent call last):
File "T3AJM.py", line 249, in <module>
main()
File "T3AJM.py", line 34, in main
GUI_main_menu()
File "T3AJM.py", line 85, in GUI_main_menu
GUI_play_markers()
File "T3AJM.py", line 125, in GUI_play_markers
video.preview_custom(marker_overlay=TEMP_OVERLAY_FILE)
AttributeError: 'VideoFileClip' object has no attribute 'preview_custom'
Thank you for your time.
I'm not even sure if this technically fixes the problem, but just doing:
from moviepy.video.io.preview import *
and
preview_custom(video, marker_overlay=TEMP_OVERLAY_FILE)
fixed the problem. I have no idea why I had to change the way it was called, as doing clip.preview(), or in this case video.preview() worked perfectly fine before, but whatever.

Pywinauto and Installshield

My application is packaged using InstallShield. The first thing that happens when you execute the exe is that IS extracts the msi file; this takes about 10 seconds.
I can't look for my main window until the msi extraction is done, so I use time.sleep. Then I look for and find the handle of my window. But the call to app.window_(handle=1234).Wait("enabled", timeout=25, retry_interval=0.5) blows up with these error messages.
*C:\python32>python test.py
| [2558122] |
Traceback (most recent call last):
File "test.py", line 16, in <module>
dlg = app.Window_(handle=hwnd).Wait("enabled", timeout=25, retry_interval=0.
5)
File "C:\python32\lib\site-packages\pywinauto\application.py", line 380, in Wa
it
WaitUntil(timeout, retry_interval, lambda: self.__check_all_conditions(check
_method_names))
File "C:\python32\lib\site-packages\pywinauto\timings.py", line 292, in WaitUn
til
func_val = func(*args)
File "C:\python32\lib\site-packages\pywinauto\application.py", line 380, in <l
ambda>
WaitUntil(timeout, retry_interval, lambda: self.__check_all_conditions(check
_method_names))
File "C:\python32\lib\site-packages\pywinauto\application.py", line 337, in __
check_all_conditions
check = getattr(self, check_name)
File "C:\python32\lib\site-packages\pywinauto\application.py", line 252, in __
getattr__
ctrls = _resolve_control(self.criteria)
File "C:\python32\lib\site-packages\pywinauto\application.py", line 755, in _r
esolve_control
criteria)
File "C:\python32\lib\site-packages\pywinauto\timings.py", line 356, in WaitUn
tilPasses
func_val = func(*args)
File "C:\python32\lib\site-packages\pywinauto\application.py", line 522, in _g
et_ctrl
findwindows.find_window(**criteria[0]))
File "C:\python32\lib\site-packages\pywinauto\controls\HwndWrapper.py", line 1
80, in __new__
new_class = cls.FindWrapper(handle)
File "C:\python32\lib\site-packages\pywinauto\controls\HwndWrapper.py", line 1
12, in FindWrapper
class_name = handleprops.classname(handle)
File "C:\python32\lib\site-packages\pywinauto\handleprops.py", line 94, in cla
ssname
win32functions.GetClassName (handle, ctypes.byref(class_name), 256)
ctypes.ArgumentError: argument 1: <class 'TypeError'>: Don't know how to convert
parameter 1*
This is the code. I am an experienced programmer but a Python newbie. I have worked on this all day, googled my brains out, and have tried everything I can think of. Thanks in advance.
from pywinauto import application
from pywinauto import findwindows
app = application.Application()
app.start("MyInstallShieldApp.exe")
time.sleep(15)
hwnd=findwindows.find_windows(title=u"InstallShield Wizard", class_name="MsiDialogCloseClass")
print ("|", str(hwnd), "|")
dlg = app.Window_(handle=hwnd).Wait("enabled", timeout=25, retry_interval=0.5)
The problem in hwnd, it is a list, but handle=hwnd requires an int.
You should use find_window instead of find_windows or use only the first handle from hwnd, e.g.
hwnd = findwindows.find_windows(...
dlg = app.Window_(handle=hwnd[0])...
Also it may be useful for you to use SWAPY - UI inspector & code generator for pywinauto.
This question is a year old, but hopefully this will help anyone else in the same position. I initially had something similar to OP and was completely lost because I figured it would be as simple as the 7zip installation example provided in the pywinauto github repo. What I found, however, is that the actual installation opens up in a second process, defeating the point of opening the application with Application().start() to launch and hook onto the program simultaneously.
What I did was I used the Inspect program from the Windows SDK to locate the names of the elements and the window that held them once InstallShield was done extracting the .msi file. Instead of opening the InstallShield executable with start(), I used the subprocess module's popen() function to open the installer and then used Application().connect() to hook into the installer once it was extracted via InstallShield. Below is a rough version of what I did to get this functional, mixing my findings with the examples, documentation, and the OP:
import time
import subprocess
from pywinauto import application
# Open the module via Popen
# I tried using run() first but that prevented the click for reasons unknown
subprocess.Popen(r"C:\Users\fixer446\My Documents\InstallationFiles\Setup.exe")
# Let InstallShield extract the files and the installer load completely
time.sleep(15)
# Hook onto the newly opened setup window
app = application.Application().connect(title="Setup.WindowName", class_name="MsiDialogCloseClass")
# Work with the window and the controls within as advertised in pywinauto
# You may have to play around with this a little
installer = app["Setup.WindowName"]
installer["Next >"].Wait("enabled")
installer["Next >"].Click()
Hope this helps.
app.Window_(...) can take the same arguments as find_windows(...). Yeah, it's not highlighted in the docs now. But the code might be shorter:
from pywinauto import application
from pywinauto import findwindows
app = application.Application()
app.start("MyInstallShieldApp.exe")
# the dialog may not exist yet here
dlg_spec = app.Window_(title=u"InstallShield Wizard", class_name="MsiDialogCloseClass")
# after this line the dialog exists or exception is raised
dlg = dlg_spec.Wait("enabled", timeout=25, retry_interval=0.5)
# dlg is a DialogWrapper object (for really existing dialog)
You may read more detailed description of 2-level WindowSpecification concept in this answer.
I will move it into the docs with coming pywinauto 0.6.0 (a bit later).

Python Boto3 OpsWorks KeyError by getting CustomJson

I try to get the custom json from my OpsWorks stacks with python and boto3. Getting the name is ok but if I want to get the CustomJson - KeyError. Don't have a clue why.
import boto3
import traceback
client = boto3.client('opsworks')
response = client.describe_stacks()
max_elements = len(response['Stacks'])
for i in range(max_elements):
stack_Name = response['Stacks'][i]['Name'] # works
try:
stack_CustomJson = response['Stacks'][i]['CustomJson'] # KeyError
except:
traceback.print_exc()
That's the console output:
$ python3 get_custom_json.py
Traceback (most recent call last):
File "get_custom_json.py", line 27, in get_opsworks_details
stack_CustomJson = response['Stacks'][i]['CustomJson']
KeyError: 'CustomJson'
Reading the docs from http://boto3.readthedocs.org/en/latest/reference/services/opsworks.html#OpsWorks.Client.describe_stacks I don't see a difference between 'Name' and 'CustomJson' except that CustomJson is a JSON object. Do I have to transform it?
Thx in advance
You are getting a KeyError occasionally because the CustomStack element in the response is optional. If a custom stack is specified for the stack, it will be returned. Otherwise, the CustomStack key will not be there at all. You should do something like:
if 'CustomStack' in stack:
# do your thing
Had a quick chat with a developer in my company. Got some basic introductions for getting better in coding and python and whatever (must loose some of my admin thinking).
Don't iterate about max_elements, better iterate above 'stack in stacks'.
for stack in response['Stacks']:
print(stack['CustomJson'])
Now it works - I'll get the custom json from the OpsWorks stacks. But still there is the KeyError.
Traceback (most recent call last):
File "get_custom_json.py", line 22, in <module>
get_opsworks_details()
File "get_custom_json.py", line 18, in get_opsworks_details
print(stack['CustomJson'])
KeyError: 'CustomJson'
I'll check if I can fetch him again to see why that's happening.
[EDIT] Blind spot - if a stack has no custom json then the KeyError will occur.

Randomly hitting Joblib exception in Sklearn on parallel Grid Search

I am running gridSearchCV in parallel with n_jobs > 1, but randomly hit the following crash in joblib:
TypeError: Cannot create a consistent method resolution
order (MRO) for bases JoblibException, Exception
Here is the complete stack trace:
Traceback (most recent call last):
File "example_sklearn.py", line 92, in <module>
main()
File "example_sklearn.py", line 76, in main
).fit(X_train, y_train)
File "/usr/local/lib/python2.7/dist-packages/sklearn/grid_search.py",
line 372, in fit for clf_params in grid for train, test in cv)
File "/usr/local/lib/python2.7/dist-packages/sklearn/externals/joblib/parallel.py",
line 516, in __call__self.retrieve()
File "/usr/local/lib/python2.7/dist-packages/sklearn/externals/joblib/parallel.py",
line 448, in retrieve exception_type = _mk_exception(exception.etype)[0]
File "/usr/local/lib/python2.7/dist-packages/sklearn/externals/joblib/my_exceptions.py",
line 61, in _mk_exception__str__=JoblibException.__str__),
TypeError: Cannot create a consistent method resolution
order (MRO) for bases JoblibException, Exception
Any pointers on what this really is, and how I can debug this. Is this a known issue with sklearn
I had the exact same exception, exactly while using the GridSearchCV.
If you look at the exception, it is complaining about not being able to understand how exactly it should choose between two parent classes JoblibException and Exception. This is a bug in the joblib package, that the inheritance is improper.
But other than than, there exist another problem, which is the source of the exception itself. It's getting an exception while retrieve()ing, and while passing the exception, you get the error.
The second problem (which is the source of the exception), seems to be fixed in later versions of joblib. But scikit-learn is still using an old version (I will submit a pull request with the changed file soon).
A temporary workaround would be to install your own version of joblib using
easy_install joblib
and then go to the sklearn/exterlan folder, remove/rename the joblib folder, and create a symbolic link to your own joblib using:
ln -s /path/to/joblib joblib
EDIT: Seems somebody has had already fixed the problem. My version was also old.

Resources