I want to encrypt the data in the PostgreSQL. I am using the below two methods to insert the data, one using ORM, other without ORM
db = sql.create_engine(connection_string)
metadata = sql.schema.MetaData(bind=db, reflect=True)
inputStringtable = sql.Table('person_info', metadata, autoload=True)
######Using ORM########
class RowInputString(object):
pass
orm.Mapper(RowInputString, inputStringtable)
Sess = orm.sessionmaker(bind=db)
session = Sess()
inputTable = RowInputString()
inputTable.person_id = personId
inputTable.person_name = personName
session.add(inputTable)
session.commit()
################################
######not using ORM
def inserting_data(personId, personName):
insertData = inputStringtable.insert().values(person_id=personId, person_name=personName)
conn = db.connect()
conn.execute(ins)
inserting_data(personId, personName)
I came across the below snippet to the encrypt and send it to database:
INSERT INTO users(login, passwd)
VALUES('my_login', crypt('my_password', gen_salt('md5')));
I find it little difficult how I can use this snippet in my code?
For general encryption, you can use the EncryptedType SQLAlchemy type.
For password hashing you can define a custom type in SQLAlchemy:
https://github.com/sqlalchemy/sqlalchemy/wiki/DatabaseCrypt
This uses bind_expression of the TypeDecorator API to map the passed-in column value to an expression involving built-in database functions (gen_salt and crypt).
Related
I'm going to use SQLAlchemy ORM for my model of user in Python.
class User(Base):
__tablename__ = "user"
id = Column(Integer, primary_key=True)
user_id = Column(Integer, ForeignKey('user_id'))
Additionally I want to bind to each User data that shouldn't be saved in database, because it has meaning only during runtime. Such as link to some objects (asyncio.lock(), asyncio.Task()).
Ofcourse when I request data from database new object will be returned. So my question is what is best practice to relate such User(Base) objects with additional data that is existing while my app is working.
So I want something like this, but I wonder about proper pattern for that or more elegant solution. Here I need to save some key such as user_id, to relate some User extracted from DB to the instance of UserRuntimeData.
class UserRuntimeData():
def __init__(self, ...):
self.user_id = ...
self._lock = ...
self._current_task = ...
def compare(self, user: User) -> bool:
if user.user_id == self.user_id:
return True
return False
UPD. I have thought that I could also save runtime data by serializing in byte code and then just extract whole User info from DB, but I think it's kind of mess.
I manage to make a connection to a Google Cloud Datastore databased. Now I want to get some entities given their Key/Id. Right now I am doing the following:
from google.cloud import datastore
client = datastore.Client()
query = client.query(kind='City')
query.key_filter("325899977574122") -> Exception here
I get "Invalid key: '325899977574122'".
What could be the cause of error? That Id exist, a city does have that key/Id.
It looks like it needs to be of type google.cloud.datastore.key.Key
https://googleapis.dev/python/datastore/latest/queries.html#google.cloud.datastore.query.Query.key_filter
Also, 325899977574122 is probably supposed to be cast to a long
So something like this:
client = datastore.Client()
query = client.query(kind='City')
query.key_filter(Key('City', 325899977574122L, project=project))
EDIT:
Also if youre trying to retrieve a single id, you should probably use this:
https://googleapis.dev/python/datastore/latest/client.html#google.cloud.datastore.client.Client.get
client = datastore.Client()
client.get(Key('City', 325899977574122L, project=project))
Fetching by ID is faster than doing a query
I am using Python to conect to a firestore database from a client.
The problem is that I don't know how to see what collections he has in the database:
from google.cloud import firestore
import firebase_admin
from firebase_admin import credentials
from firebase_admin import firestore
cred = credentials.Certificate('credentials/credentials.json')
app = firebase_admin.initialize_app(cred)
db = firestore.client()
users_ref = db.collection(u'name_of_colection')
docs = users_ref.stream()
for doc in docs:
print(u'{} => {}'.format(doc.id, doc.to_dict()))
I have been looking how to get the name of the collections that he has but I didn't find anything that it was useful to me. I have also tried this:
cols = db.collections()
list_col = []
for col in cols:
list_col.append(col)
len(list_col)
I have obtained len = 6
Then I have done this for the different col in the list I have generated:
docs = list_col[5].stream()
data = []
for doc in docs:
data.append(doc.to_dict())
print(data)
This data print a dictionary with keys and values, but I don't know only get a list with the name of collections,
I think that you have to get the id from each collection (which is the collection name you are talking about
list_col = []
for col in collections:
list_col.append(col.id) // <-- add this please
print(list_col)
I hope it helps you
Any collection you see in fire base depends on your rights.
you can use
query = client.collection_group('mygroup')
or
query = client.collections()
It gives top hierarchy and you have to run multiple times to find the lowest document level.
query = client.collection_group('mygroup')
#param {string} collectionId Identifies the collections to query over. Every collection or subcollection with this ID as the last segment of its path will be included. Cannot contain a slash. #returns {Query} The created Query.
collections()[source]
List top-level collections of the client’s database.
Returns
iterator of subcollections of the current document.
Return type
Sequence[CollectionReference]
Simple solution:
import firebase_admin
from firebase_admin import firestore
app_options = {'projectId': 'test-project'}
default_app = firebase_admin.initialize_app(options=app_options)
db = firestore.client()
collection = db.collections()
list_col = []
for col in collection:
list_col.append(col.id)
print(list_col)
I'm rather new to the whole ORM topic, and I've already searched forums and docs.
The question is about a flask application with SQLAlchemy as ORM for the PostgreSQL.
The __init__.py contains the following line:
db = SQLAlchemy()
the created object is referenced in the other files to access the DB.
There is a save function for the model:
def save(self):
db.session.add(self)
db.session.commit()
and also an update function:
def update(self):
for var_name in self.__dict__.keys():
if var_name is not ('_sa_instance_state' or 'id' or 'foreign_id'):
# Workaround for JSON update problem
flag_modified(self, var_name)
db.session.merge(self)
db.session.commit()
The problem occurs when I'm trying to save a new object. The save function writes it to DB, it's visible when querying the DB directly (psql, etc.), but a following ORM query like:
model_list = db.session.query(MyModel).filter(MyModel.foreign_id == this_id).all()
gives an empty response.
A call of the update function does work as expected, new data is visible when requesting with the ORM.
I'm always using the same session object for example this:
<sqlalchemy.orm.scoping.scoped_session object at 0x7f0cff68fda0>
If the application is restarted everything works fine until a new object was created and tried to get with the ORM.
An unhandsome workaround is using raw SQL like:
model_list = db.session.execute('SELECT * FROM models_table WHERE
foreign_id = ' + str(this_id))
which gives a ResultProxy with latest data like this:
<sqlalchemy.engine.result.ResultProxy object at 0x7f0cf74d0390>
I think my problem is a misunderstanding of the session. Can anyone help me?
It figured out that the problem has nothing to do with the session, but the filter() method:
# Neccessary import for string input into filter() function
from sqlalchemy import text
# Solution or workaround
model_list = db.session.query(MyModel).filter(text('foreign_key = ' + str(this_id))).all()
I could not figure out the problem with:
filter(MyModel.foreign_id == this_id) but that's another problem.
I think this way is better than executing raw SQL.
My main intention is to dynamically change the Employees collection while using pymongo, and i was able to do it for insert commands, I am facing problems with the find command, no matter what i do exec() always returns None. but if i copy the string and run it value gets assigned to the variable.
can someone throw some light on why the exec is unable to return a resultset or assign a the resultset to a variable?
db.Employees.update_one(
{"id": criteria},
{
"$set": {
"name":name,
"age":age,
"country":country
}
}
)
from pymongo import MongoClient
import ast
client = MongoClient('localhost:27017')
db = client.TextClassifier
insert works
def mongo_insert_one(COLLECTION_NAME, JSON):
QUERY = """db.%(COLLECTION_NAME)s.insert_one( %(JSON)s )""" % locals();
exec(QUERY)
def mongo_retrive(COLLECTION_NAME, JSON):
resultset = None
query = """resultset = db.%(COLLECTION_NAME)s.find( %(JSON)s )""" % locals();
return resultset
print(mongo_retrive('hungry_intent', "{'Intent':'Hungry'}"))
neither this would work
resultset = exec(""" db.%(COLLECTION_NAME)s.find( %(JSON)s )""" % locals();)
this would not work for an entirely different reason,it says If you meant to call the 'locals' method on a 'Database' object it is failing because no such method exists.
resultset = db.locals()[COLLECTION_NAME].find()
PyMongo Database objects support bracket notation to access a named collection, and PyMongo's included bson module provides a much better JSON decoder than "eval":
from bson import json_util
COLLECTION_NAME = 'hungry_intent'
JSON = "{'Intent':'Hungry'}"
print(list(db[COLLECTION_NAME].find(json_util.loads(JSON))))
This will be faster and more reliable than your "eval" code, and also prevents the injection attack that your "eval" code is vulnerable to.
If you can avoid using JSON at all it could be preferable:
COLLECTION_NAME = 'hungry_intent'
QUERY = {'Intent':'Hungry'}
print(list(db[COLLECTION_NAME].find(QUERY)))