I freshly installed Faust and ran a basic program to send and receive messages over Kafka.I used the sample code mentioned in (Faust example of publishing to a kafka topic) While running the program initially it connects to Kafka(which is also running on my machine). But then while trying to consume Kafka gets disconnected and the app crashes with the below exception
[2020-11-11 07:08:26,623] [76392] [ERROR] [^---Fetcher]: Crashed reason=ConsumerStoppedError()
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/mode/services.py", line 779, in _execute_task
await task
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/faust/transport/consumer.py", line 176, in _fetcher
await self._drainer
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/faust/transport/consumer.py", line 1039, in _drain_messages
async for tp, message in ait:
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/faust/transport/consumer.py", line 640, in getmany
records, active_partitions = await self._wait_next_records(timeout)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/faust/transport/consumer.py", line 676, in _wait_next_records
records = await self._getmany(
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/faust/transport/consumer.py", line 1269, in _getmany
return await self._thread.getmany(active_partitions, timeout)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/faust/transport/drivers/aiokafka.py", line 805, in getmany
return await self.call_thread(
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/mode/threads.py", line 436, in call_thread
result = await promise
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/mode/threads.py", line 383, in _process_enqueued
result = await maybe_async(method(*args, **kwargs))
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/mode/utils/futures.py", line 134, in maybe_async
return await res
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/faust/transport/drivers/aiokafka.py", line 822, in _fetch_records
raise ConsumerStoppedError()
On debugging the reason for the consumer getting disconnected I see that in fetcher.py of alokafka consumer is the connection getting closed due to the below exception
Unable to display children:Error resolving variables Traceback (most recent call last):
File "/Applications/PyCharm CE.app/Contents/plugins/python-ce/helpers/pydev/_pydevd_bundle/pydevd_resolver.py", line 205, in resolve
def resolve(self, dict, key):
KeyError: 'Exception'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Applications/PyCharm CE.app/Contents/plugins/python-ce/helpers/pydev/_pydevd_bundle/pydevd_comm.py", line 1227, in do_it
def do_it(self, dbg):
File "/Applications/PyCharm CE.app/Contents/plugins/python-ce/helpers/pydev/_pydevd_bundle/pydevd_vars.py", line 262, in resolve_compound_variable_fields
def resolve_compound_variable_fields(thread_id, frame_id, scope, attrs):
File "/Applications/PyCharm CE.app/Contents/plugins/python-ce/helpers/pydev/_pydevd_bundle/pydevd_vars.py", line 169, in getVariable
def getVariable(thread_id, frame_id, scope, attrs):
File "/Applications/PyCharm CE.app/Contents/plugins/python-ce/helpers/pydev/_pydevd_bundle/pydevd_resolver.py", line 205, in resolve
def resolve(self, dict, key):
AttributeError: 'dict' object has no attribute 'Exception'
The software versions are as given below
Mac OS : 10.15.4
Kafka : 2_12.2.1.1
Aiokafka: 1.1.6
Python : 3.9.0
Faust : 1.10.4
Please help here.
I think your problem is the same mine I was with python 3.9 and I changed to 3.8 now It works.
Related
I am a noob in Python multiprocessing and trying to learn it.
While trying to run a class method on a separate Multiprocessing-Process. The Python class is initialized in the main process and I am trying to start a class method in a separate process.
def __initialise_strat_class_obj(self, *, strat_name: str, strat_config: StratInput):
try:
strat_instance = strat_class_dict[strat_name](strat_args=self.strat_args, strat_input=strat_config) # dynamically fetching & calling the class
strat_instance.init()
except Exception as e:
self._logger.error(f"unable to init strategy `{strat_config.id}`: {e}")
return False
strat_process = Process(target=strat_instance.run)
strat_process.start()# starting the class method in a separate process
def start(self):
self.init()
if self.is_active():
self.run()
else:
self._logger.warning(f"{self.id} -> strat is inactive, returning from thread")
return
However, I encountered the cannot pickle '_io.TextIOWrapper' object error. Below is the stack trace of the error.
Traceback (most recent call last):
File "D:\repo\StockTradeBotV2\src\strategy\__init__.py", line 237, in __init__
self.__initialise_strat_class_obj(strat_name=strat_name, strat_config=strat_inp)
File "D:\repo\StockTradeBotV2\src\strategy\__init__.py", line 263, in __initialise_strat_class_obj
strat_process.start()
File "C:\Program Files\Python311\Lib\multiprocessing\process.py", line 121, in start
self._popen = self._Popen(self)
^^^^^^^^^^^^^^^^^
File "C:\Program Files\Python311\Lib\multiprocessing\context.py", line 224, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\Python311\Lib\multiprocessing\context.py", line 336, in _Popen
return Popen(process_obj)
^^^^^^^^^^^^^^^^^^
File "C:\Program Files\Python311\Lib\multiprocessing\popen_spawn_win32.py", line 94, in __init__
reduction.dump(process_obj, to_child)
File "C:\Program Files\Python311\Lib\multiprocessing\reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
TypeError: cannot pickle '_io.TextIOWrapper' object
To overcome the serialization issue, I made use of the multiprocess package which utilizes dill for serializing objects. This time I got a completely different error:
Traceback (most recent call last):
File "D:\repo\StockTradeBotV2\src\strategy\__init__.py", line 237, in __init__
self.__initialise_strat_class_obj(strat_name=strat_name, strat_config=strat_inp)
File "D:\repo\StockTradeBotV2\src\strategy\__init__.py", line 263, in __initialise_strat_class_obj
strat_process.start()
File "C:\PythonEnvironments\StockTradeBotV2\Lib\site-packages\multiprocess\process.py", line 121, in start
self._popen = self._Popen(self)
^^^^^^^^^^^^^^^^^
TypeError: BaseProcess._Popen() takes 1 positional argument but 2 were given
I have tried calling the raw class without initializing and passing the self-argument which didn’t help. On changing the Process to Thread, the code works as intended without any errors, but I want this to run on its own process instead of parallelism so that the additional cores on my PC are used.
I am doing all this on Python-3.11.1 with multiprocess-0.70.14 package.
Edit: I have to use multiprocess.Process rather than from multiprocess.process import BaseProcess as Process which solved the issue
{python.py:175} INFO - Done. Returned value was: <pyodbc.Cursor object at 0x7f55f4a02a30>
[2022-03-18, 17:56:17 UTC] {taskinstance.py:1700} ERROR - Task failed with exception
Traceback (most recent call last):
File "/opt/airflow_test/az/venv/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1329, in _run_raw_task
self._execute_task_with_callbacks(context)
File "/opt/airflow_test/az/venv/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1455, in _execute_task_with_callbacks
result = self._execute_task(context, self.task)
File "/opt/airflow_test/az/venv/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 1514, in _execute_task
self.xcom_push(key=XCOM_RETURN_KEY, value=result)
File "/opt/airflow_test/az/venv/lib/python3.9/site-packages/airflow/utils/session.py", line 70, in wrapper
return func(*args, session=session, **kwargs)
File "/opt/airflow_test/az/venv/lib/python3.9/site-packages/airflow/models/taskinstance.py", line 2135, in xcom_push
XCom.set(
File "/opt/airflow_test/az/venv/lib/python3.9/site-packages/airflow/utils/session.py", line 67, in wrapper
return func(*args, **kwargs)
File "/opt/airflow_test/az/venv/lib/python3.9/site-packages/airflow/models/xcom.py", line 100, in set
value = XCom.serialize_value(value)
File "/opt/airflow_test/az/venv/lib/python3.9/site-packages/airflow/models/xcom.py", line 329, in serialize_value
return pickle.dumps(value)
TypeError: cannot pickle 'pyodbc.Cursor' object
I am using pickle in my ML model loading, Not sure how to troubleshoot this error, how the pickle loading is connected to X_com, I stopped using Xcomm in code as well, even though I am getting the pyodbc.Cursor error. I am using pyodbc for my database connectivity check
When I try to make the HTTP client a class variable of my tcp server class I see the following error:
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "cisco-dial-out-mdt-async-client.py", line 208, in <module>
tcp_server.start(0)
File "/home/lab/venv3.7/lib/python3.7/site-packages/tornado/tcpserver.py", line 244, in start
self.add_sockets(sockets)
File "/home/lab/venv3.7/lib/python3.7/site-packages/tornado/tcpserver.py", line 166, in add_sockets
sock, self._handle_connection
File "/home/lab/venv3.7/lib/python3.7/site-packages/tornado/netutil.py", line 279, in add_accept_handler
io_loop.add_handler(sock, accept_handler, IOLoop.READ)
File "/home/lab/venv3.7/lib/python3.7/site-packages/tornado/platform/asyncio.py", line 99, in add_handler
self.asyncio_loop.add_reader(fd, self._handle_events, fd, IOLoop.READ)
File "/usr/lib/python3.7/asyncio/selector_events.py", line 324, in add_reader
return self._add_reader(fd, callback, *args)
File "/usr/lib/python3.7/asyncio/selector_events.py", line 254, in _add_reader
child 13 (pid 3260) killed by signal 2, restarting
(handle, None))
File "/usr/lib/python3.7/selectors.py", line 359, in register
Traceback (most recent call last):
File "cisco-dial-out-mdt-async-client.py", line 208, in <module>
self._selector.register(key.fd, poller_events)
Traceback (most recent call last):
FileExistsError: [Errno 17] File exists
File "/usr/lib/python3.7/asyncio/selector_events.py", line 251, in _add_reader
tcp_server.start(0)
File "/home/lab/venv3.7/lib/python3.7/site-packages/tornado/tcpserver.py", line 241, in start
process.fork_processes(num_processes, max_restarts)
File "/home/lab/venv3.7/lib/python3.7/site-packages/tornado/process.py", line 175, in fork_processes
key = self._selector.get_key(fd)
raise RuntimeError("Too many child restarts, giving up")
File "/usr/lib/python3.7/selectors.py", line 192, in get_key
RuntimeError: Too many child restarts, giving up
raise KeyError("{!r} is not registered".format(fileobj)) from None
KeyError: '9 is not registered'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "cisco-dial-out-mdt-async-client.py", line 208, in <module>
tcp_server.start(0)
Traceback (most recent call last):
File "/home/lab/venv3.7/lib/python3.7/site-packages/tornado/tcpserver.py", line 244, in start
File "/usr/lib/python3.7/asyncio/selector_events.py", line 251, in _add_reader
self.add_sockets(sockets)
File "/home/lab/venv3.7/lib/python3.7/site-packages/tornado/tcpserver.py", line 166, in add_sockets
key = self._selector.get_key(fd)
File "/usr/lib/python3.7/selectors.py", line 192, in get_key
sock, self._handle_connection
File "/home/lab/venv3.7/lib/python3.7/site-packages/tornado/netutil.py", line 279, in add_accept_handler
raise KeyError("{!r} is not registered".format(fileobj)) from None
KeyError: '9 is not registered'
When I remove the line that creates the object it works fine
class TelemetryTCPDialOutServer(TCPServer):
def __init__(self, elasticsearch_server):
super().__init__()
self.elastic_server = elasticsearch_server
self.lock = Lock()
log_name = __file__.strip('.py')
self.log = self.init_log(f"{log_name}.log")
self.lock = Lock()
self.http_client = AsyncHTTPClient() <== This line
Is it possible to put the http client in the class or make it a local object to some of the class functions. (This way works too).
Thansk
It looks like something has really messed up your stack trace, but I think the problem is that you're calling tcp_server.start(0) too late. start(0) forks multiple child processes, and it's invalid to fork multiple processes once the event loop has been created.
If you can, move this start(0) call earlier (typically it's the first thing you do after processing your command line or config file). If you can't move it earlier, you'll have to split up the process creation from starting the server. This is the "advanced multi-process" pattern from https://www.tornadoweb.org/en/stable/tcpserver.html:
# This part comes very early, just after processing configs
sockets = bind_sockets(8888)
tornado.process.fork_processes(0)
# This part can come later
server = TCPServer()
server.add_sockets(sockets)
IOLoop.current().start()
Trying to run the most basic test of add.delay(1,2) using celery 4.1.0 with Python 3.6.4 and getting the following error:
[2018-02-27 13:58:50,194: INFO/MainProcess] Received task:
exb.tasks.test_tasks.add[52c3fb33-ce00-4165-ad18-15026eca55e9]
[2018-02-27 13:58:50,194: CRITICAL/MainProcess] Unrecoverable error:
SystemError(' returned a result with an error set',) Traceback (most
recent call last): File
"/opt/myapp/lib/python3.6/site-packages/kombu/messaging.py", line 624,
in _receive_callback
return on_m(message) if on_m else self.receive(decoded, message) File
"/opt/myapp/lib/python3.6/site-packages/celery/worker/consumer/consumer.py",
line 570, in on_task_received
callbacks, File "/opt/myapp/lib/python3.6/site-packages/celery/worker/strategy.py",
line 145, in task_message_handler
handle(req) File "/opt/myapp/lib/python3.6/site-packages/celery/worker/worker.py", line
221, in _process_task_sem
return self._quick_acquire(self._process_task, req) File "/opt/myapp/lib/python3.6/site-packages/kombu/async/semaphore.py",
line 62, in acquire
callback(*partial_args, **partial_kwargs) File "/opt/myapp/lib/python3.6/site-packages/celery/worker/worker.py", line
226, in _process_task
req.execute_using_pool(self.pool) File "/opt/myapp/lib/python3.6/site-packages/celery/worker/request.py",
line 531, in execute_using_pool
correlation_id=task_id, File "/opt/myapp/lib/python3.6/site-packages/celery/concurrency/base.py",
line 155, in apply_async
**options) File "/opt/myapp/lib/python3.6/site-packages/billiard/pool.py", line 1486,
in apply_async
self._quick_put((TASK, (result._job, None, func, args, kwds))) File
"/opt/myapp/lib/python3.6/site-packages/celery/concurrency/asynpool.py",
line 813, in send_job
body = dumps(tup, protocol=protocol) TypeError: can't pickle memoryview objects
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File
"/opt/myapp/lib/python3.6/site-packages/celery/worker/worker.py", line
203, in start
self.blueprint.start(self) File "/opt/myapp/lib/python3.6/site-packages/celery/bootsteps.py", line
119, in start
step.start(parent) File "/opt/myapp/lib/python3.6/site-packages/celery/bootsteps.py", line
370, in start
return self.obj.start() File "/opt/myapp/lib/python3.6/site-packages/celery/worker/consumer/consumer.py",
line 320, in start
blueprint.start(self) File "/opt/myapp/lib/python3.6/site-packages/celery/bootsteps.py", line
119, in start
step.start(parent) File "/opt/myapp/lib/python3.6/site-packages/celery/worker/consumer/consumer.py",
line 596, in start
c.loop(*c.loop_args()) File "/opt/myapp/lib/python3.6/site-packages/celery/worker/loops.py", line
88, in asynloop
next(loop) File "/opt/myapp/lib/python3.6/site-packages/kombu/async/hub.py", line 354,
in create_loop
cb(*cbargs) File "/opt/myapp/lib/python3.6/site-packages/kombu/transport/base.py", line
236, in on_readable
reader(loop) File "/opt/myapp/lib/python3.6/site-packages/kombu/transport/base.py", line
218, in _read
drain_events(timeout=0) File "/opt/myapp/lib/python3.6/site-packages/librabbitmq-2.0.0-py3.6-linux-x86_64.egg/librabbitmq/init.py",
line 227, in drain_events
self._basic_recv(timeout) SystemError: returned a result with an error set
I cannot find any previous evidence of anyone hitting this error. I noticed from the celery site that only python 3.5 is mentioned as supported, is that the issue or is this something I am missing?
Any help would be much appreciated!
UPDATE: Tried with Python 3.5.5 and the problem persists. Tried with Django 4.0.2 and the problem persists.
UPDATE: Uninstalled librabbitmq and the problem stopped. This was seen after migration from Python 2.7.5, Django 1.7.7 to Python 3.6.4, Django 2.0.2.
After uninstalling librabbitmq, the problem was resolved.
I am trying to make a code that will ask for a user input and if that input is not given in a set amount of time will assume a default value and continue through the rest of the code without requiring the user to hit enter. I am running in Python 3.5.1 on Windows 10.
I have looked through: Keyboard input with timeout in Python, How to set time limit on raw_input, Timeout on a function call, and Python 3 Timed Input black boxing the answers but none of the answers are suitable as they are not usable on Windows (principally use of signal.SIGALRM which is only available on linux), or require a user to hit enter in order to exit the input.
Based upon the above answers however i have attempted to scrap together a solution using multiprocessing which (as i think it should work) creates one process to ask for the input and creates another process to terminate the first process after the timeout period.
import multiprocessing
from time import time,sleep
def wait(secs):
if secs == 0:
return
end = time()+secs
current = time()
while end>current:
current = time()
sleep(.1)
return
def delay_terminate_process(process,delay):
wait(delay)
process.terminate()
process.join()
def ask_input(prompt,term_queue,out_queue):
command = input(prompt)
process = term_queue.get()
process.terminate()
process.join()
out_queue.put(command)
##### this doesn't even remotly work.....
def input_with_timeout(prompt,timeout=15.0):
print(prompt)
astring = 'no input'
out_queue = multiprocessing.Queue()
term_queue = multiprocessing.Queue()
worker1 = multiprocessing.Process(target=ask_input,args=(prompt,term_queue,out_queue))
worker2 = multiprocessing.Process(target=delay_terminate_process,args=(worker1,timeout))
worker1.daemon = True
worker2.daemon = True
term_queue.put(worker2)
print('Through overhead')
if __name__ == '__main__':
print('I am in if statement')
worker2.start()
worker1.start()
astring = out_queue.get()
else:
print('I have no clue what happened that would cause this to print....')
return
print('returning')
return astring
please = input_with_timeout('Does this work?',timeout=10)
But this fails miserably and yields:
Does this work?
Through overhead
I am in if statement
Traceback (most recent call last):
File "C:\Anaconda3\lib\multiprocessing\queues.py", line 241, in _feed
obj = ForkingPickler.dumps(obj)
File "C:\Anaconda3\lib\multiprocessing\reduction.py", line 50, in dumps
cls(buf, protocol).dump(obj)
File "C:\Anaconda3\lib\multiprocessing\queues.py", line 58, in __getstate__
context.assert_spawning(self)
File "C:\Anaconda3\lib\multiprocessing\context.py", line 347, in assert_spawning
' through inheritance' % type(obj).__name__
RuntimeError: Queue objects should only be shared between processes through inheritance
Does this work?
Through overhead
I have no clue what happened that would cause this to print....
Does this work?Process Process-1:
Traceback (most recent call last):
File "C:\Anaconda3\lib\multiprocessing\queues.py", line 241, in _feed
obj = ForkingPickler.dumps(obj)
File "C:\Anaconda3\lib\multiprocessing\reduction.py", line 50, in dumps
cls(buf, protocol).dump(obj)
File "C:\Anaconda3\lib\multiprocessing\process.py", line 287, in __reduce__
'Pickling an AuthenticationString object is '
TypeError: Pickling an AuthenticationString object is disallowed for security reasons
Traceback (most recent call last):
File "C:\Anaconda3\lib\multiprocessing\process.py", line 254, in _bootstrap
self.run()
File "C:\Anaconda3\lib\multiprocessing\process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "C:\Anaconda3\saved_programs\a_open_file4.py", line 20, in ask_input
command = input(prompt)
EOFError: EOF when reading a line
Does this work?
Through overhead
I have no clue what happened that would cause this to print....
Traceback (most recent call last):
File "C:\Anaconda3\lib\multiprocessing\queues.py", line 241, in _feed
obj = ForkingPickler.dumps(obj)
File "C:\Anaconda3\lib\multiprocessing\reduction.py", line 50, in dumps
cls(buf, protocol).dump(obj)
File "C:\Anaconda3\lib\multiprocessing\queues.py", line 58, in __getstate__
context.assert_spawning(self)
File "C:\Anaconda3\lib\multiprocessing\context.py", line 347, in assert_spawning
' through inheritance' % type(obj).__name__
RuntimeError: Queue objects should only be shared between processes through inheritance
Process Process-2:
Traceback (most recent call last):
File "C:\Anaconda3\lib\multiprocessing\process.py", line 254, in _bootstrap
self.run()
File "C:\Anaconda3\lib\multiprocessing\process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "C:\Anaconda3\saved_programs\a_open_file4.py", line 16, in delay_terminate_process
process.terminate()
File "C:\Anaconda3\lib\multiprocessing\process.py", line 113, in terminate
self._popen.terminate()
AttributeError: 'NoneType' object has no attribute 'terminate'
I really don't understand the multiprocessing module well and although I have read the official docs am unsure why this error occurred or why it appears to have ran through the function call 3 times in the process. Any help on how to either resolve the error or achieve an optional user input in a cleaner manner will be much appreciated by a noob programmer. Thanks!