Why does django-q throw exception with arrow time - python-3.x

I'm trying to create a Django-q schedule and following the documents to use arrow for the next run I get the following error with the schedule:
schedule(
func='test.tasks.test_task',
name='test_task_nightly',
schedule_type=Schedule.DAILY,
next_run=arrow.utcnow().replace(hour=23, minute=30),
q_options={'timeout': 10800, 'max_attempts': 1},
)
Traceback (most recent call last):
File "/usr/lib/python3.8/code.py", line 90, in runcode
exec(code, self.locals)
File "<console>", line 1, in <module>
schedule(
File "/home/user/PycharmProjects/app/venv/lib/python3.8/site-packages/django_q/tasks.py", line 122, in schedule
s.full_clean()
File "/home/user/PycharmProjects/app/venv/lib/python3.8/site-packages/django/db/models/base.py", line 1209, in full_clean
self.clean_fields(exclude=exclude)
File "/home/user/PycharmProjects/app/venv/lib/python3.8/site-packages/django/db/models/base.py", line 1251, in clean_fields
setattr(self, f.attname, f.clean(raw_value, self))
File "/home/user/PycharmProjects/app/venv/lib/python3.8/site-packages/django/db/models/fields/__init__.py", line 650, in clean
value = self.to_python(value)
File "/home/user/PycharmProjects/app/venv/lib/python3.8/site-packages/django/db/models/fields/__init__.py", line 1318, in to_python
parsed = parse_datetime(value)
File "/home/user/PycharmProjects/app/venv/lib/python3.8/site-packages/django/utils/dateparse.py", line 107, in parse_datetime
match = datetime_re.match(value)
TypeError: expected string or bytes-like object
Not sure why it's not accepting the time format similar to the example given in the django-q documentation page.
EDIT:
The task being scheduled:
def test_task():
print('Executed test task')
Nothing too complex just for testing purposes

The Django ORM (verion 3.2 at time of writing) won't accept an Arrow object in any DateTimeField.
Arrow objects emulate Python's datetime object interface, but they are not real datetime objects. So any code receiving an Arrow object will fail if it explicitly checks if your value is an honest-to-goodness datetime. Which is exactly what the code in
django.db.models.fields.DateTimeField.to_python appears to be doing:
def to_python(self, value):
if value is None:
return value
if isinstance(value, datetime.datetime):
return value
if isinstance(value, datetime.date):
value = datetime.datetime(value.year, value.month, value.day)
...
try:
parsed = parse_datetime(value)
As you can see, when it doesn't match a datetime or date instance, Django hands it off to the parse_datetime() function to deal with, which expects a string. Which explains your error: TypeError: expected string or bytes-like object
You can get around this by getting the .datetime property, which will return a plain old python datetime, i.e.
schedule(
func='test.tasks.test_task',
name='test_task_nightly',
schedule_type=Schedule.DAILY,
next_run=arrow.utcnow().replace(hour=23, minute=30).datetime,
q_options={'timeout': 10800, 'max_attempts': 1},
)

Related

Application building using tkinter

I am trying to create an application that allow students to see the list of mandatory and elective courses using def key word in tkinter. This is the error i get after running the code.
Exception in Tkinter callback
Traceback (most recent call last):
File "C:\Users\mhm01\anaconda3\lib\tkinter\__init__.py", line 1892, in __call__
return self.func(*args)
File "C:\Users\mhm01\AppData\Local\Temp\ipykernel_13464\560191512.py", line 25, in courses
label3.configure(courses)
File "C:\Users\mhm01\anaconda3\lib\tkinter\__init__.py", line 1646, in configure
return self._configure('configure', cnf, kw)
File "C:\Users\mhm01\anaconda3\lib\tkinter\__init__.py", line 1635, in _configure
return self._getconfigure1(_flatten((self._w, cmd, '-'+cnf)))
File "C:\Users\mhm01\anaconda3\lib\tkinter\__init__.py", line 1623, in _getconfigure1
x = self.tk.splitlist(self.tk.call(*args))
_tkinter.TclError: unknown option "-"
You have a key in the courses dictionary variable which is an invalid input to the .configure() method of the label class of tkinter - according to the error it looks like you have a key in courses called -. Try removing that (and ensure you only have valid keys in the dictionary you pass to configure)

pymongo - bson.errors.InvalidDocument raised only sometimes for no apparent reason

my documents look like this : { "_id" : 5, "hunger" : 5, "energy" : 50 }
I'm calling this function..
def getEnergy(_id) -> int:
record = db.systems.find({"_id":_id}) # systems is the collection
return record[0]['energy']
and getting this error..
(...)
File "C:\Users\mateo\AppData\Roaming\Python\Python39\site-packages\pymongo\cursor.py", line 692, in __getitem__
for doc in clone:
File "C:\Users\mateo\AppData\Roaming\Python\Python39\site-packages\pymongo\cursor.py", line 1238, in next
if len(self.__data) or self._refresh():
File "C:\Users\mateo\AppData\Roaming\Python\Python39\site-packages\pymongo\cursor.py", line 1155, in _refresh
self.__send_message(q)
File "C:\Users\mateo\AppData\Roaming\Python\Python39\site-packages\pymongo\cursor.py", line 1044, in __send_message
response = client._run_operation(
File "C:\Users\mateo\AppData\Roaming\Python\Python39\site-packages\pymongo\mongo_client.py", line 1424, in _run_operation
return self._retryable_read(
File "C:\Users\mateo\AppData\Roaming\Python\Python39\site-packages\pymongo\mongo_client.py", line 1525, in _retryable_read
return func(session, server, sock_info, secondary_ok)
File "C:\Users\mateo\AppData\Roaming\Python\Python39\site-packages\pymongo\mongo_client.py", line 1420, in _cmd
return server.run_operation(
File "C:\Users\mateo\AppData\Roaming\Python\Python39\site-packages\pymongo\server.py", line 98, in run_operation
message = operation.get_message(
File "C:\Users\mateo\AppData\Roaming\Python\Python39\site-packages\pymongo\message.py", line 351, in get_message
request_id, msg, size, _ = _op_msg(
File "C:\Users\mateo\AppData\Roaming\Python\Python39\site-packages\pymongo\message.py", line 743, in _op_msg
return _op_msg_uncompressed(
bson.errors.InvalidDocument: cannot encode object: <pymongo.cursor.Cursor object at 0x0000021E52535670>, of type: <class 'pymongo.cursor.Cursor'>
Sometimes the function works just fine, and sometimes it throws an error. It seems to be a problem with the server but I can't figure out what exactly the problem is.
If the error is in that function (which we can't 100% tell as you've only posted half the stack trace), you will get this error is you are passing a cursor into the function; this snippet reproduces the error:
from pymongo import MongoClient
db = MongoClient()['mydatabase']
def getEnergy(_id) -> int:
record = db.systems.find({"_id":_id}) # systems is the collection
return record[0]['energy']
foo = db.somecollection.find()
getEnergy(foo)
error:
bson.errors.InvalidDocument: cannot encode object: <pymongo.cursor.Cursor object at 0x0000019DB0A0B070>, of type: <class 'pymongo.cursor.Cursor'>
You need to examine where you are calling getExport() and check the parameter you are passing in isn't a cursor object.

TypeError: not all arguments converted during string formatting in python connecting with postgresql

It seems like all no error in the code but no idea why I'm getting this.
I was creating a simple GUI app which store user details to a database(postgresql) and also they will be able to search for entries in database. This particular error ocures in this search() function so I haven't added the rest of the code. If necessary I can add them.
Hope I will get some solutions from this community.
def search(id):
conn = psycopg2.connect(dbname="postgres",user="postgres",password="1018",host="localhost",port="5432")
mycursor = conn.cursor()
query = '''select * from demotab where id=%s '''
mycursor.execute(query, (id))
row = mycursor.fetchone()
print(row)
conn.commit()
conn.close()
Getting this error below
Exception in Tkinter callback
Traceback (most recent call last):
File "c:\programdata\anaconda3\lib\tkinter\__init__.py", line 1702, in __call__
return self.func(*args)
File "appwithDB.py", line 51, in <lambda>
search_button = Button(newframe, text="Search", command=lambda : search(entry_search.get()))
File "appwithDB.py", line 71, in search
mycursor.execute(query, (id))
TypeError: not all arguments converted during string formatting
The second argument to mycursor.execute must be an iterable containing the values to insert in the query
You can use a list: mycursor.execute(query, [id])
or a one-element tuple: mycursor.execute(query, (id,))
Notice the comma. (id) is the same than id. In python, the comma is making the tuple, not the parenthesis.

pig_hook in airflow doesn't work for python3

Using Python 3.5.2, airflow 1.9.0
Trying to set up a pig hook, looking at the documentation here:
https://github.com/apache/incubator-airflow/blob/master/airflow/hooks/pig_hook.py
following the example in lines 50-52
>>> ph = PigCliHook()
>>> result = ph.run_cli("ls /;")
>>> ("hdfs://" in result)
Gives the following error:
File "python3.5/site-packages/airflow/hooks/pig_hook.py", line 53, in run_cli
f.write(pig)
File "python3.5/tempfile.py", line 622, in func_wrapper
return func(*args, **kwargs)
TypeError: a bytes-like object is required, not 'str'
If updated to run as:
>>> ph = PigCliHook()
>>> result = ph.run_cli("ls /;".encode('utf-8'))
>>> ("hdfs://" in result)
The error becomes:
File "python3.5/site-packages/airflow/hooks/pig_hook.py", line 74, in run_cli
stdout += line
TypeError: Can't convert 'bytes' object to str implicitly
And later in the same pig_hook.py file it does assume a string type for this field, so I don't think passing the input as a bytes object is correct.
I believe the object causing the problem is NamedTemporaryFile (from line 52 in pig_hook.py) which is opened by default in mode 'w+b' as described in the following post:
https://bugs.python.org/issue29245
But if I change line 53 in pig_hook.py to read:
with NamedTemporaryFile(dir=tmp_dir,'w') as f:
or
with NamedTemporaryFile(dir=tmp_dir, mode='w') as f:
it still expects a byte array resulting in the same error:
File "python3.5/site-packages/airflow/hooks/pig_hook.py", line 53, in run_cli
f.write(pig)
File "python3.5/tempfile.py", line 622, in func_wrapper
return func(*args, **kwargs)
TypeError: a bytes-like object is required, not 'str'
Does anyone know how I can solve this issue? I can't seem to get NamedTemporaryFile to open in a mode that uses string, and the rest of the code assumes a string.
It turns out that the error after calling:
>>> ph = PigCliHook()
>>> result = ph.run_cli("ls /;".encode('utf-8'))
>>> ("hdfs://" in result)
which was
File "python3.5/site-packages/airflow/hooks/pig_hook.py", line 74, in run_cli
stdout += line
TypeError: Can't convert 'bytes' object to str implicitly
Was not coming from my own input but from the logging. So I added:
line = line.decode('utf-8')
to line 75 of the pig_hook and it seems to be working just fine now.

Specify a one or many field using marshmallow

I would like to specify a schema field wichi accepts one or many resources. However I seem only able to specify one behavior or the other.
>>> class Resource(marshmallow.Schema):
... data = marshmallow.fields.Dict()
...
>>> class ContainerSchema(marshmallow.Schema):
... resource = marshmallow.fields.Nested(ResourceSchema, many=True)
...
>>> ContainerSchema().dump({'resource': [{'data': 'DATA'}]})
MarshalResult(data={'resource': [{'data': 'DATA'}]}, errors={})
In the above example a list must be defined. However I would prefer not to:
>>> ContainerSchema().dump({'resource': {'data': 'DATA'}})
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/lib64/python3.6/site-packages/marshmallow/schema.py", line 513, in dump
**kwargs
File "/lib64/python3.6/site-packages/marshmallow/marshalling.py", line 147, in serialize
index=(index if index_errors else None)
File "/lib64/python3.6/site-packages/marshmallow/marshalling.py", line 68, in call_and_store
value = getter_func(data)
File "/lib64/python3.6/site-packages/marshmallow/marshalling.py", line 141, in <lambda>
getter = lambda d: field_obj.serialize(attr_name, d, accessor=accessor)
File "/lib64/python3.6/site-packages/marshmallow/fields.py", line 252, in serialize
return self._serialize(value, attr, obj)
File "/lib64/python3.6/site-packages/marshmallow/fields.py", line 448, in _serialize
schema._update_fields(obj=nested_obj, many=self.many)
File "/lib64/python3.6/site-packages/marshmallow/schema.py", line 760, in _update_fields
ret = self.__filter_fields(field_names, obj, many=many)
File "/lib64/python3.6/site-packages/marshmallow/schema.py", line 810, in __filter_fields
obj_prototype = obj[0]
KeyError: 0
Can I have a schema allowing both a single item or many of it?
The point with giving the arguments as a list - whether it's one or many - is so the schema knows how to handle it in either case. For the schema to process arguments of a different format, like not in a list, you need to add a preprocessor to the schema, like this:
class ContainerSchema(marshmallow.Schema):
resource = marshmallow.fields.Nested(ResourceSchema, many=True)
#pre_dump
def wrap_indata(self, indata):
if type(indata['resource']) is dict:
indata['resource'] = [indata['resource']]

Resources