I'm using asyncio with websockets and aiohttp for a Twitch API bot. Everything was fine, but at some point asyncio starts to hide errors and makes the code a brick.
After a while dealing with that, i realize that only happens on functions (and nested ones) awaited on asyncio.gather().
If you see in the images, I try print an unresolved reference, at that point the thread is dead and no errors are throw.
https://cdn.discordapp.com/attachments/532653651689472024/570191503587147776/Screenshot_2.png
Also, is weird this part of the asyncio code (asyncio/events.py) depending of the type error, asyncio loops over that part a lot of times. I dont know if that matters.
https://cdn.discordapp.com/attachments/532653651689472024/570191514634944522/Screenshot_4.png
Furthermore, i tried to uninstall asyncio with pycharm and with pip. The code still accessing to asyncio i dont know where. i dont have virtual environments set. is there some way to uninstall manually and try to reinstall asyncio cleanly?
By the way, my app still working, is just the fact of no having clues when I miss something, making the progress freaking hard
If you see in the images, I try print an unresolved reference, at that
point the thread is dead and no errors are throw.
asyncio.gather runs several tasks simultaneously, and by default propagates first exception any of tasks raised. Even if this exception didn't finish your program, it at least changed position where your interpreter is. And while other gathered tasks continue to run in background their exceptions can't be raised from the new point where your interpreter is currently on.
Exceptions occurred in tasks but not raised anywhere are printed as warnings "Task exception was never retrieved" when event loop is closed (usually at the very end of your program).
It's hard to say if this is exactly what happens in your case since you didn't provide minimal reproducible example. Feel free to provide one to receive more precise answer.
Furthermore, i tried to uninstall asyncio with pycharm and with pip.
The code still accessing to asyncio i dont know where. i dont have
virtual environments set. is there some way to uninstall manually and
try to reinstall asyncio cleanly?
asyncio is a part of the Python's standard library. This module can't be (or at least shouldn't be) (re/un)installed without (re/un)installing Python interpreter itself.
Related
I have a code running in Python 3.7.4 which forks off multiple processes. I believe I'm hitting a known issue (issue6721: https://github.com/python/cpython/issues/50970). I setup the child process to send "progress report" through a pipe to the parent process and noticed that sometimes a log statement doesn't get printed and that the code gets stuck in a deadlock situation.
After reading issue6721, I'm not sure I'm still understanding why parent might hold logger Handler lock after a log statement is done execution (i.e the line that logs is executed and the execution has moved to the next line of code). I totally get it that in the context of C++, the compiler might re-arrange instructions. Not fully understand it in context of Python. In C++ I can have barrier instructions to stop the compiler moving instructions beyond a point. Is there something similar that can be done in Python to avoid having a lock getting copied to child process?
I have seen solutions using "atfork" which is a library that seems not supported (so I can't really use it).
Does anyone know a reliable and standard solution to this problem?
So I'm writing a script/application that uses pythons multiprocessing BaseManager class. Now for the most part it works great, the only issue I have is that I am using the serve_forever as a blocking statement and then continue onwards however when I want to terminate or exit out of the serve_forever() function(ality) it automatically exits out and terminates the application, but like I mentioned I have some more things I want to take care of before I completely exit out.
I can exit out of serve_forever() by setting a stop event with stop_event.set(). Now this is all well and dandy however according to the source (https://github.com/python/cpython/blob/3.6/Lib/multiprocessing/managers.py#L147) serve_forever explicitly states sys.exit(0) and is part of the Server class that BaseManager uses within it's definition. Essentially I would like to remove that line (sys.exit(0)). How would I accompolish this?
When I search I'm coming up with results such as monkey patching? Can I just Subclass the Server class, explicitly define serve_forever to be the exact same code but without the sys.exit(0) line and call it a day? Something tells me that is not going to work. Do I subclass Server AND BaseManager?
Thanks!
Attempting to monkey-patch or inherit internal classes will result in code that will not be compatible across Python releases, not even patches.
Atop of that, these solutions will be unnecessarily complex and complicated, and are overall frowned upon.
I highly suggest re-implementing serve_forever() by using the start() method together with an event. Waiting for the event to be called or, if impossible, a loop checking if the manager is still alive, will be much easier and a better solution in almost all aspects that I can think of.
After discussing in chat, we realised the easiest approach is to just suppress the SystemExit being thrown from sys.exit(). I'm opening a bug report on CPython bug tracker accordingly to prevent sys.exit(). Do keep in mind the server will not actually shut down as it is run on a different thread. The whole recommendation of using .server().serve_forever() in the stdlib looks dubious at best.
If you wish to immediately shut down the server, call Server.listener.close() after catcing the exception.
I am using the spyder interface with a python script that works through many time steps in succession. At a random point in my code, the process will terminate and the console will say "kernel died, restarting...". I tried running the same script in pycharm and the process also terminates, seemingly at a random point, with some exit code which I assume means the same thing.
Anyone have any tips on how to get rid of this problem? Even a workaround so I can get some work done. This is incredibly frustrating.
Note: I recently moved and got a new router and internet service, not sure if that might affect things.
In my application I run wglGetCurrentDC() and wglGetCurrentContext() from onThread function
(this function should be called as declared here - EVT_THREAD(wxID_ANY,MyCanvas::onThread))
and I get NULL in both cases. When I run it not from onThread it is ok…
What is work around in order to solve the problem – (I have to run them when getting event from the thread!)
As Alex suggested I changed to wxPostEvent to redirect the event to main thread, which catches the event in its onThread function.In this onThread function I have wglGetCurrentDC() and wglGetCurrentContext() calls ...They still return null.Please explain me what I am doing wrong. And how to solve he problem.
Maybe I'm misunderstanding, but should you not be using wxGLCanvas and wxGLContext rather than the windows-specific code? At the very least it's probably more compatible with other wxWidget code.
Anyway, from the wglGetCurrentDC documentation, the function returns NULL if a DC for the current window doesn't exist. This suggests that either the context was destroyed somehow or you're not calling it from the window you think you're calling it from (perhaps because of your threading?). I would reiterate what Alex said; don't call UI code from any thread besides the main one.
If you could post some code showing how you're returning from the thread it might help identify the problem. It seems likely that you're doing UI stuff from the thread and just not realizing it. (Hard to tell without seeing any code, though.)
Don't touch any UI-related stuff from a worker thread. This is general requirement for all UI frameworks. Use wxPostEvent to redirect a work to the main application thread.
OK, so I am writing an app, which plays music with the pyGST bindings.
This requires the use of threads to handle playback. The bindings library handles most of the thread control for me, which is nice(and what I was looking for in them).
Now, I don't have a full grasp on this concept, so I would be eager for some references. But the way I understand it, is I have to basically inform the app that it can use multiple threads.
I gathered this from the examples on the gstreamer site, where they use this call:
gtk.gdk.threads_init()
gtk.main()
according to here, this tells the app it can use multiple threads(more or less), which is where my above assumption came from.
That is the background. Now get this. I have placed those lines in my code, and they work fine. My app plays music rather than crashing whenever it tries. But something doesn't feel right.
In the examples that I got those lines from, they use gtk for the whole GUI, but I want to use wxWidgets, so it feels wrong calling this gtk function to do this.
Is there a wx equivalent to this? or is it ok to use this, and will it still work cross platform?
Also, I have to figure out how to kill all these threads on exit(which it does not do right now) I see how they do it in the example using a gtk method again, so again, looking for a wx equivalent.
PS: I think this(or the solution) may be related to the wx.App.MainLoop() function, but I am lost in trying to understand how this loop works, so again, good references about this would be appreciated, but I suppose not necessary as long as I have a good solution.
Try using this instead:
import gobject
gobject.threads_init()
I wonder how come it is not written in large print at the start of every python gstreamer plugin piece of documentation: it only took me several hours to find it.
A bit more details here.
I have no experience with pyGST, but the general advice for using threads and wxPython is to only update the GUI from the main thread (i.e. the thread that starts the MainLoop). See http://wiki.wxpython.org/LongRunningTasks for more information.
I have no experience with the python bindings, but I have had success using wxWidgets and GStreamer together on Windows. The problem is that wxWidgets runs a Windows event loop while GStreamer uses a GLib event loop. If you don't care about any of the GStreamer events, you shouldn't need to do anything. However, if you care to receive any of the GStreamer events, you will have to run your own GLib event loop (GMainLoop) in a separate thread with a separate GMainContext. Use gst_bus_create_watch to create a GST event source, add a callback to the source with g_source_set_callback, and then attach it to the main context of your GLib event loop with g_source_attach. You can then handle the GST in the callback, for example, to forward the events to the wx main event loop.