Problem accessing MySQL connection out of Flask methods - python-3.x

I'm developing a Flask app, with MySQL (flask-mysqldb) and MQTT (flask-mqtt) integrations. I can perform any DB operation from a Flask method (e.g. #app.route('/')), but if I try to do it from a MQTT method when I receive a message (e.g. #mqtt.on_message()) it does nothing. This last method works perfectly because it receives and shows in log the message received.
I have a method that performs DB operations, and depending on where I call it from, it works or not. I guess it should be because of the MySQL object, but I don't know exactly.
Here is the code I'm using (just the problem):
#mqtt.on_message()
def handle_mqtt_message(client, userdata, message):
print('New message {}'.format(message.payload.decode()))
storeDB('test') #Here it doesn't work
################## Methods ###########################
def storeDB(param_text):
cur = mysql.connection.cursor()
cur.execute(
'INSERT INTO contacts (fullname, phone, email) VALUES (%s,%s,%s)', (param_text, param_text, param_text))
mysql.connection.commit()
###################### FLASK #########################
#app.route('/')
def index():
storeDB('temp') #Here it works
return 'Hello World'
If I access to localhost it shows the "Hello World" text in browser and updates the DB; otherwise, if I receive a MQTT message, it is shown on terminal but not updated the DB.
Thanks.

This is how I have it working using the MySQLdbpackage:
import MySQLdb
try:
db = MySQLdb.connect("HOST", "USER", "PASS", "DB")
except:
print("Problem creating DB connection")
sys.exit()
cursor = db.cursor()
def storeDB(param_text1, param_text2, param_text3):
query = """INSERT INTO `DB`.`TABLE` (`fullname`, `phone`, `email`) VALUES ('""" + \
param_text1+"""','"""+param_text2+"""','"""+param_text3+"""');"""
try:
cursor.execute(query)
db.commit()
print('DB updated')
except:
db.rollback()
print("Problem updating DB :(")

Related

How to send data in infinite loop from server to client throught python socket.io?

I want to send data (like endless stream) from one vds machine to another. I've read that it's possible to do with python-socket.io. First of all, I try to do it on my laptop (server script runs in one terminal, client - in another). And send numbers from "server" to "client" in infinite loop. I want to get data on client side in real-time. (Server sent "1", client got "1", etc) But, when I run both scripts I see that server is sending data and client gets nothing. Only when I stop (ctrl+c) server, all sent data is printing on client's terminal.
How to fix code to get real-time connection?
server.py
import eventlet
import socketio
import time
sio = socketio.Server()
app = socketio.WSGIApp(sio)
#sio.event
def connect(sid, environ):
print('connect ', sid)
my_message(sid, "Client connected")
# f(sid)
#sio.event
def my_message(sid, data):
sio.send(data)
print('Send message ', data)
#sio.event
def disconnect(sid):
print('disconnect ', sid)
#sio.on('subscribe_to_data')
def subscribe(sid, data):
counter = 0
while True:
sio.send(counter)
print('Send message from server ', counter)
counter += 1
# my_message(sid, i)
time.sleep(1)
eventlet.wsgi.server(eventlet.listen(('', 5000)), app)
client.py
import socketio
sio = socketio.Client()
#sio.event
def connect():
print('connection established')
sio.emit('subscribe_to_data', "I want to subscribe")
#sio.event
def message(data):
print('message received with ', data)
#sio.event
def disconnect():
print('disconnected from server')
sio.connect('http://localhost:5000')
sio.wait()
A better implementation will be to maintain a list of connected clients and do something like
while sid in CONNECTED_CLIENTS:
pass
Also instead of time.sleep() use sio.sleep()

How can I get global pyramid request in socketio methods?

I want to use python-socketio and I want to query my database from socketio methods. My db settings are saved in the pyramid request. But I don't understand how to get these settings without http-requests.
#sio.event
def connect(sid, environ):
print('connect ', sid)
#sio.event
def message(sid, data):
# I want to query my database at this location
sio.send(data)
print('Server send', data)
#sio.event
def disconnect(sid):
print('disconnect ', sid)
I was thinking of using ' pyramid.threadlocal.get_current_request ' but this method returns None.
I add database settings to the pyramid request by this code.
__init__.py
def db(request):
session = session_maker(request)
return session
...
config.add_request_method(db, reify=True)
session_maker
def session_returner(request, connect_line):
engine = create_engine(connect_line, echo=debug)
Base.metadata.bind = engine
Session = orm.sessionmaker(bind=engine)
session = Session()
def cleanup(request):
if request.exception is not None:
session.rollback()
session.close()
request.add_finished_callback(cleanup)
return session
def session_maker(request, settings=None):
if settings is None:
settings = get_settings(request)
connect_line = 'postgresql://{user}:{password}#{postgre_server}:{bd_port}/{bd_name}'.format(user=settings['bd_user'], postgre_server=settings['postgre_server'], bd_port=settings['bd_port'], password=settings['bd_password'], bd_name=settings['bd_name'])
return session_returner(request, connect_line)
You can make a request object, but obviously there isn't one by default when not serving a request. This can be done via:
request = pyramid.request.Request.blank('/')
request.registry = registry
pyramid.request.apply_request_extensions(request)
request.db.query(...)
This assumes you have access to the registry which was defined by the wsgi app. Something like config.registry or app = config.make_wsgi_app(); registry = app.registry or request.registry from another request.

Simple Flask app to insert data to database works locally but wouldn't work when deployed

I have built a simple app that grabs the data from POST request and puts it into a database (Postgesql) using Flask. I have tested it locally and everything works as it should. But when I deploy it to pythonAnywhere it gives me the 500 error back when I POST the data to my app. It works though when I don't use psycopg2 and just return the fetched result back.
Please see my code below.
Also, I am relatively new to web development
import psycopg2
from flask import Flask, request
app = Flask(__name__)
#app.route('/', methods=['POST'])
def hello_world():
req_data = request.get_json()
info = req_data['info']
conn1 = psycopg2.connect(
user = "some_user",
password = "some_password",
host = "some_host",
port = "5432",
database = "some_db"
)
conn1.autocommit = True
cursor1 = conn1.cursor()
sql = "INSERT INTO amber_list (user_id, description) VALUES ('{}', '{}')".format(str(info), str(info))
cursor1.execute(sql)
conn1.close()
return '''
Database was successfully updated with "{}"
'''.format(info)
Also this is the sample string I am fetching
{
"info" : "Seems to be working :)"
}

Unexpected AssertionError: single test not using logged in user from previous step

I am following the tutorial by http://www.patricksoftwareblog.com/flask-tutorial/, which I believe is based on https://blog.miguelgrinberg.com/post/the-flask-mega-tutorial-part-i-hello-world. Great stuff for a beginner.
I am getting different results when testing my code through frontend manually (which works fine) v.s. through pytest.
My test tries to show the "groups" endpoint which requires a login (standard #login_required decorator).
I initially test the user getting a login page ("Knock knock") when trying to get the endpoint without a login. This works manually and through pytest.
I login a user. If I inspect the response from the login I can clearly see a "Welcome back Pete!" success message.
My second assert receives a response from URL /login?next=%2Fgroups indicating the /groups endpoint is called without a login/authentication preceding it and the assert fails. Testing this manually works as expected. Why is that single test not using the same user/session combination in the next step(s)?
Test with the problem is the first snippet below:
def test_groups(app):
assert b'Knock knock' in get(app, "/groups").data
login(app, "pete#testmail.com", "pete123")
assert b'Test group 1' in get(app, "/groups").data
My "get" function for reference:
def get(app, endpoint: str):
return app.test_client().get(endpoint, follow_redirects=True)
My "login" function for reference:
def login(app, email="testuser#testmail.com", password="testing"):
return app.test_client().post('/login', data=dict(email=email, password=password), follow_redirects=True)
The app (from a conftest fixture imported in the test module by #pytest.mark.usefixtures('app')) for reference:
#pytest.fixture
def app():
"""An application for the tests."""
_app = create_app(DevConfig)
ctx = _app.test_request_context()
ctx.push()
yield _app
ctx.pop()
The login route for reference:
#app.route('/login', methods=['GET', 'POST'])
def login():
form = LoginForm(request.form)
if request.method == 'POST':
if form.validate_on_submit():
user = User.query.filter_by(email=form.email.data).first()
if user is not None and user.is_correct_password(form.password.data):
user.authenticated = True
user.last_login = user.current_login
user.current_login = datetime.now()
user.insert_user()
login_user(user)
flash(f'Welcome back {user.name}!', 'success')
return redirect(url_for('our_awesome_group.index'))
else:
flash('Incorrect credentials! Did you already register?', 'error')
else:
flash_errors(form)
return render_template('login.html', form=form)
The groups route for reference:
#app.route('/groups')
#login_required
def groups():
groups_and_users = dict()
my_group_uuids = Membership.list_groups_per_user(current_user)
my_groups = [Group.query.filter_by(uuid=group).first() for group in my_group_uuids]
for group in my_groups:
user_uuids_in_group = Membership.list_users_per_group(group)
users_in_group = [User.query.filter_by(uuid=user).first() for user in user_uuids_in_group]
groups_and_users[group] = users_in_group
return render_template('groups.html', groups_and_users=groups_and_users)
Im going to sum up the comments I made that gave the answer on how to solve this issue.
When creating a test app using Pytest and Flask there are a few different ways to go about it.
The suggested way to create a test client with proper app context is to use something like:
#pytest.fixture
def client():
""" Creates the app from testconfig, activates test client and context then makes the db and allows the test client
to be used """
app = create_app(TestConfig)
client = app.test_client()
ctx = app.app_context()
ctx.push()
db.create_all()
yield client
db.session.close()
db.drop_all()
ctx.pop()
That creates the client while pushing the app context so you can register things like your database and create the tables to the test client.
The second way is show in OP's question where use app.test_request context
#pytest.fixture
def app():
"""An application for the tests."""
_app = create_app(DevConfig)
ctx = _app.test_request_context()
ctx.push()
yield _app
ctx.pop()
and then create the test client in another pytest fixture
#pytest.fixture
def client(app):
return app.test_client()
Creating a test client allows you to use various testing features and gives access to flask requests with the proper app context.

Can't write to MySQL DB

I'm a Flask newbie trying to create a simple app. I'm currently stuck at user registration where I'm trying to save data in database but it's not happening. However, the logging I'm doing indicates that the operation was a success. Could someone tell me what I'm doing wrong?
Here are portions of code that'll help you understand what I'm trying to do:
from flask import Flask, request, session, g, redirect, url_for, abort, render_template, flash
from flask.ext.mysqldb import MySQL
# Configuration
MYSQL_HOST = 'localhost'
MYSQL_USER = 'root'
MYSQL_PASSWORD = 'root'
MYSQL_DB = 'up2date'
DEBUG = True
SECRET_KEY =
'\xc6)\x0f\\\xc5\x86*\xd7[\x92\x89[\x95\xcfD\xfd\xc1\x18\x8e\xf1P\xf7_\r'
# Create the flask app
app = Flask(__name__)
app.config.from_object(__name__)
# Create instance for working with MySQL
mysql = MySQL(app)
# Function to connect to DB
def connect_db():
return mysql.connection.cursor()
# define functions that will make DB available automatically on each request
#app.before_request
def before_request():
g.db = connect_db()
#app.teardown_request
def teardown_request(exception):
g.db.close()
And finally, the code that performs user registration:
#app.route('/register', methods=['GET', 'POST'])
def register():
if request.method == 'POST':
email = request.form['email']
password = request.form['password']
result = g.db.execute('INSERT INTO users (email, password) VALUES (%s, %s)', [email, password])
print(email, password)
print(result, " rows affected")
flash('Registration successful! You may log in now.')
return redirect(url_for('show_home'))
The two print statements confirm that the email address and password were captured correctly, and the result variable contains 1, indicating 1 row affected. But still there's no row in the DB. I earlier thought this had something to do with committing, but g.db.commit() throws error: AttributeError: 'Cursor' object has no attribute 'commit'
I assume you use MySQL-python.
connect_db() returns the cursor and not the connection. The cursor does not have a commit() function, as the exception says, however the connection has the commit function you need. I think you need to do this:
def connect_db():
return mysql.connection
For more info you can take a look at the code.

Resources