pymodm can't find an object, while pymongo successfully finds it - python-3.x

I have a problem getting an object from the mongodb instance. If I search for this object with pymongo interface, everything is fine - object can be found. If try to do the very same thing with pymodm - it fails with error.
Here is what I'm doing:
from pymodm import connect, MongoModel, fields
from pymongo import MongoClient
class detection_object(MongoModel):
legacy_id = fields.IntegerField()
client = MongoClient(MONGODB_URI)
db = client[MONGODB_DEFAULT_SCHEME]
collection = db['detection_object']
do = collection.find_one({'legacy_id': 1437424})
print(do)
connect(MONGODB_URI)
do = detection_object.objects.raw({'legacy_id': 1437424}).first()
print(do)
The first print outputs this: {'_id': ObjectId('5c4099dcffa4fb11494d983d'), 'legacy_id': 1437424}. However, during the execution of this command: do = detection_object.objects.raw({'legacy_id': 1437424}).first() interpreter fails with the following error:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/pymodm/queryset.py", line 127, in first
return next(iter(self.limit(-1)))
StopIteration
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/konsof01/PycharmProjects/testthisfuckingshit/settings.py", line 29, in <module>
do = detection_object.objects.raw({'legacy_id': 1437424}).first()
File "/usr/local/lib/python3.7/site-packages/pymodm/queryset.py", line 129, in first
raise self._model.DoesNotExist()
__main__.DoesNotExist
How can this be? I'm trying to query the very same object, with the same connection and collection. Any ideas, please?

you could try it as follows:
detection_object.objects.raw({'legacy_id': "1437424"} ).first()
probably the legacy_id is stored as string.
Othewise, make sure the db name is present at the end of the MONGO_URI as it is underlined in the docs.

Each document in your 'detection_object' collection requires to have '_cls' attribute. The string value stored in this attribute should be
__main__.classname
(class name according to your code is detection_object).
For example a document in your database needs to look like this:
{'_id': ObjectId('5c4099dcffa4fb11494d983d'), 'legacy_id': 1437424, '_cls': '__ main __.detection_object'}

Related

Django model object as parameter for celery task raises EncodeError - 'object of type someModelName is not JSON serializable'

Im working with a django project(im pretty new to django) and running into an issue passing a model object between my view and a celery task.
I am taking input from a form which contains several ModelChoiceField fields and using the selected object in a celery task. When I queue the task(from the post method in the view) using someTask.delay(x, y, z) where x, y and z are various objects from the form ModelChoiceFields I get the error object of type <someModelName> is not JSON serializable.
That said, if I create a simple test function and pass any of the same objects from the form into the function I get the expected behavior and the name of the object selected in the form is logged.
def test(object):
logger.debug(object.name)
I have done some poking based on the above error and found django serializers which allows for a workaround by serializing the object using serializers.serialize('json', [template]), in the view before passing it to the celery task.
I can then access the object in the celery task by using template = json.loads(template)[0].get('fields') to access its required bits as a dictionary -- while this works, it does seem a bit inelegant and I wanted to see if there is something I am missing here.
Im obviously open to any feedback/guidance here however my main questions are:
Why do I get the object...is not JSON serializable error when passing a model object into a celery task but not when passing to my simple test function?
Is the approach using django serializers before queueing the celery task considered acceptable/correct or is there a cleaner way to achieve this goal?
Any suggestions would be greatly appreciated.
Traceback:
I tried to post the full traceback here as well however including that caused the post to get flagged as 'this looks like spam'
Internal Server Error: /build/
Traceback (most recent call last):
File "/home/tech/sandbox_project/venv/lib/python3.8/site-packages/kombu/serialization.py", line 49, in _reraise_errors
yield
File "/home/tech/sandbox_project/venv/lib/python3.8/site-packages/kombu/serialization.py", line 220, in dumps
payload = encoder(data)
File "/home/tech/sandbox_project/venv/lib/python3.8/site-packages/kombu/utils/json.py", line 65, in dumps
return _dumps(s, cls=cls or _default_encoder,
File "/usr/lib/python3.8/json/__init__.py", line 234, in dumps
return cls(
File "/usr/lib/python3.8/json/encoder.py", line 199, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/usr/lib/python3.8/json/encoder.py", line 257, in iterencode
return _iterencode(o, 0)
File "/home/tech/sandbox_project/venv/lib/python3.8/site-packages/kombu/utils/json.py", line 55, in default
return super().default(o)
File "/usr/lib/python3.8/json/encoder.py", line 179, in default
raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type Template is not JSON serializable
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/tech/sandbox_project/venv/lib/python3.8/site-packages/django/core/handlers/exception.py", line 47, in inner
response = get_response(request)
Add this lines to settings.py
# Project/settings.py
CELERY_ACCEPT_CONTENT = ['json']
CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'
Then instead of passing object, send JSON with id/pk if you're using a model instance call the task like this..
test.delay({'pk': 1})
Django model instance is not available in celery environment, as it runs in a different process
How you can get the model instance inside task then? Well, you can do something like below -
def import_django_instance():
"""
Makes django environment available
to tasks!!
"""
import django
import os
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'Project.settings')
django.setup()
# task
#shared_task(name="simple_task")
def simple_task(data):
import_django_instance()
from app.models import AppModel
pk = data.get('pk')
instance = AppModel.objects.get(pk=pk)
# your operation

KeyError: 'document' occures when I try to use execute_script command in Python Selenium

I have simple code. Firstly I get an input from user. In target page, there is an editable div. I focus on it and insert the message that I got from user. When I focusing and inserting element I use Javascript so for using Javascript with Python Selenium I use execute_script command.
Code:
message = input()
message = message.replace("'","\\'")
message = message.replace('"','\\"')
browser.execute_script("setTimeout(function() {document.querySelector('._3ee1T ._3uMse ._2FVVk ._3FRCZ').focus();}, 0); document.querySelector('._3ee1T ._3uMse ._2FVVk ._3FRCZ').textContent = '{}'".format(message))
Error: Traceback (most recent call last): File "main.py", line 171, in <module> browser.execute_script("setTimeout(function() {document.querySelector('._3ee1T ._3uMse ._2FVVk ._3FRCZ').focus();}, 0); document.querySelector('._3ee1T ._3uMse ._2FVVk ._3FRCZ').textContent = '{}'".format(message)) KeyError: 'document'

Method createIndex() not callable on a collection

In the doc of the createIndex they say db.collection.createIndex(keys, options) so i call createIndex() with the code below. The name of my database is articles and the name of the collection is bce. Inside bce there is already a document with the field article.
class Stockage():
def __init__(self):
self.connexion()
def connexion(self):
client = pymongo.MongoClient("localhost", 27017)
self.bce = client.articles.bce
sql = Stockage()
sql.bce.createIndex({article : "text"})
And i have the following error :
Traceback (most recent call last):
File "<ipython-input-1132-fc8762d315d1>", line 1, in <module>
sql.bce.createIndex({article : "text"})
File "C:\ProgramData\Anaconda3\lib\site-packages\pymongo\collection.py", line 3321, in __call__
self.__name.split(".")[-1])
TypeError: 'Collection' object is not callable. If you meant to call the 'createIndex' method on a 'Collection' object it is failing because no such method exists.
Is not bce a collection ?
This is because in Pymongo the method is called create_index() instead of createIndex() as it is named in the mongo shell.
It also has a different format for the parameter compared to its mongo shell counterpart. Instead of accepting a document/dict as index specification, it accepts a list of tuples instead. This is because you cannot guarantee the ordering of dict keys in Python.
So the correct line should be:
sql.bce.create_index([("article", pymongo.TEXT)])
More details are available in the Pymongo Collection page.

CherryPy encoding: bool object not iterable

Hello,
I am using CherryPy to host the gui of an application that takes json files from qualtrics and drops them in a mysql server.
The code seems to work for most surveys but for some I get the following error:
Traceback (most recent call last):
File "C:\Users\jam66\AppData\Local\Programs\Python\Python37-32\lib\site-packages\cherrypy\_cprequest.py", line 627, in respond
self._do_respond(path_info)
File "C:\Users\jam66\AppData\Local\Programs\Python\Python37-32\lib\site-packages\cherrypy\_cprequest.py", line 686, in _do_respond
response.body = self.handler()
File "C:\Users\jam66\AppData\Local\Programs\Python\Python37-32\lib\site- packages\cherrypy\lib\encoding.py", line 264,
in __call__ct.params['charset'] = self.find_acceptable_charset()
File "C:\Users\jam66\AppData\Local\Programs\Python\Python37-32\lib\site- packages\cherrypy\lib\encoding.py", line 173, in find_acceptable_charset
if encoder(self.default_encoding):
File "C:\Users\jam66\AppData\Local\Programs\Python\Python37-32\lib\site-packages\cherrypy\lib\encoding.py", line 114, in encode_string
for chunk in self.body:
TypeError: 'bool' object is not iterable
Any help on beginning to understand this issues is appreciated
My guess is that some of your exposed methods are returning a boolean. You have to return a string or an iterable. Unless you are using the json tool, in that case the dictionary to string is handled by the tool.
As a way to debug it, just print or log the value that would will be returned, verify the type with the type function.

ldap3/python3 Connection modify() function error(malformed change) with active directory

i have an active directory running and would like to change an attribute of an user. Im using the python3 ldap3 library and the modify() method.
My code below trows an exception every time i try to modify the user.
from ldap3 import Server, Connection, MODIFY_REPLACE
server = Server('ldaps://domain.local:636', port = 636, use_ssl = True)
try:
conn = Connection(server, user="philipp#domain.local", password="MYPW", auto_bind=True)
print("User found")
except:
print("User not found or connection error")
result = conn.modify('CN=phil stone,OU=Firewall,DC=domain,DC=local',{'rules': [(MODIFY_REPLACE, ['replaced-text'])]})
print(result)
Traceback (most recent call last):
File "/mnt/hgfs/shared/eclipse/osfipa/ldap/ldap_write.py", line 28, in <module>
result = conn.modify('CN=phil stone,OU=Firewall,DC=domain,DC=local',{'rules': [(MODIFY_REPLACE, ['replaced-text'])]})
File "/usr/local/lib/python3.5/dist-packages/ldap3/core/connection.py", line 763, in modify
raise LDAPChangesError(self.last_error)
ldap3.core.exceptions.LDAPChangesError: malformed change
In the AD GUI i have checked the Attribute "distinguishedName" of the User(Object) and its Value is "CN=philipp stone,OU=Firewall,DC=domain,DC=local".
So the DN for the modify function should be correct. And i want to change the "rules" attribute.
Any Ideas what im doing wrong here?
Thx in advance.
https://github.com/cannatag/ldap3/issues/42
There is a simple syntax error. In the above link one can finde the answer.
The simple solution was to change it like the code below.
result = conn.modify('CN=phil stone,OU=Firewall,DC=domain,DC=local',{'rules': (MODIFY_REPLACE, ['replaced-text'])})
Look at the part "MODIFY_REPLACE replaced-text", there is no list([]) around it;)

Resources