I am using Flask and SqlAlchemy, the case is that I have a main database and then I have several databases where I get other information. But the credentials are not defined as fixed, so I have to obtain them from a table in the main database, depending on the plant where the user is. For this I use SQLALCHEMY_BINDS. the problem is that when I try to pass the connection string to the Bind I told myself that the function that returns it is out of context. Here a portion of the code
def scadaConnect():
idplanta_session=str(session['idPlanta'])
usernamequery = db.session.query(Scada.usernameScada).filter(Scada.idPlanta=='5')
hostquery = db.session.query(Scada.hostScada).filter(Scada.idPlanta=='5')
passwordquery = db.session.query(Scada.passScada).filter(Scada.idPlanta=='5')
nombredbquery = db.session.query(Scada.nombrebdScada).filter(Scada.idPlanta=='5')
nombredb = str(nombredbquery[0])[2:len(nombredbquery[0])-4]
host = str(hostquery[0])[2:len(hostquery[0])-4]
password = str(passwordquery[0])[2:len(passwordquery[0])-4]
username = str(usernamequery[0])[2:len(usernamequery[0])-4]
connexion = 'mysql+pymysql://'+username+":"+password+"#"+host+"/"+nombredb+"'"
def retorno():
return str(connexion)
from config import SQLALCHEMY_BINDS
SQLALCHEMY_BINDS['scada']= scadaConnect()
The error is as follows
RuntimeError: Working outside of request context.
This typically means that you attempted to use functionality that needed an active HTTP request. Consult the documentation on testing for information about how to avoid this problem.
session (first line in scadaConnect()) is only available in a request context a.k.a. view. It lives in the session cookie on the client and is sent to the server (and available to your view) only when the browser makes a request.
You will have to move the call to scadaConnect() to a view if you want to use session.
Related
I have Angular 8 web app. It needs to send some data for analysis to python flask App. This flask App will send it to 3rd party service and get response through webhook.
My need is to provide a clean interface to the client so that there is no need to provide webhook from client.
Hence I am trying to initiate a request from Flask app, wait until I get data from webhook and then return.
Here is the code.
#In autoscaled micro-service this will not work. Right now, the scaling is manual and set to 1 instance
#Hence keeping this sliding window list in RAM is okay.
reqSlidingWindow =[]
#app.route('/handlerequest',methods = ['POST'])
def Process_client_request_and_respond(param1,param2,param3):
#payload code removed
#CORS header part removed
querystring={APIKey, 'https://mysvc.mycloud.com/postthirdpartyresponsehook'}
response = requests.post(thirdpartyurl, json=payload, headers=headers, params=querystring)
if(response.status_code == SUCCESS):
respdata = response.json()
requestId = respdata["request_id"]
requestobject = {}
requestobject['requestId'] = requestId
reqSlidingWindow.append(requestobject)
#Now wait for the response to arrive through webhook
#Keep checking the list if reqRecord["AllAnalysisDoneFlag"] is set to True.
#If set, read reqRecord["AnalysisResult"] = jsondata
jsondata = None
while jsondata is None:
time.sleep(2)
for reqRecord in reqSlidingWindow:
if(reqRecord["requestId"] == da_analysis_req_Id ):
print("Found matching req id in req list. Checking if analysis is done.")
print(reqRecord)
if(reqRecord["AllAnalysisDoneFlag"] == True):
jsondata = reqRecord["AnalysisResult"]
return jsonify({"AnalysisResult": "Success", "AnalysisData":jsondata})
#app.route('/webhookforresponse',methods = ['POST'])
def process_thirdparty_svc_response(reqIdinResp):
print(request.data)
print("response receieved at")
data = request.data.decode('utf-8')
jsondata = json.loads(data)
#
for reqRecord in reqSlidingWindow:
#print("In loop. current analysis type" + reqRecord["AnalysisType"] )
if(reqRecord["requestId"] == reqIdinResp ):
reqRecord["AllAnalysisDoneFlag"] = True
reqRecord["AnalysisResult"] = jsondata
return
I'm trying to maintain sliding window of requests in list. Upon the
Observations so far:
First, this does not work. The function of 'webhookforresponse' does not seem to run until my request function comes out. i.e. my while() loop appears to block everything though I have a time.sleep(). My assumption is that flask framework would ensure that callback is called since sleep in another 'route' gives it adequate time and internally flask would be multithreaded?
I tried running python threads for the sliding window data structure and also used RLocks. This does not alter behavior. I have not tried flask specific threading.
Questions:
What is the right architecture of the above need with flask? I need clean REST interface without callback for angular client. Everything else can change.
If the above code to be used, what changes should I make? Is threading required at all?
Though I can use database and then pick from there, it still requires polling the sliding window.
Should I use multithreading specific to flask? Is there any specific example with skeletal design, threading library choices?
Please suggest the right way to proceed to achieve abstract REST API for angular client which hides the back end callbacks and other complexities.
I'm new to flask and currently converting an existing WSGI application to run through flask as long term it'll make life easier.
All requests are POST to specific routes however the current application inspects the post data prior to executing the route to see if the request needs to be run at all or not (i.e. if an identifier supplied in the post data already exists in our database or not).
If it does exist a 200 code and json is returned "early" and no other action is taken; if not the application continues to route as normal.
I think I can replicate the activity at the right point by calling before_request() but I'm not sure if returning a flask Response object from before_request() would terminate the request adequately at that point? Or if there's a better way of doing this?
NB: I must return this as a 200 - other examples I've seen result in a redirect or 4xx error handling (as a close parallel to this activity is authentication) so ultimately I'm doing this at the end of before_request():
if check_request_in_progress(post_data) is True:
response = jsonify({'request_status': 'already_running'})
response.status_code = 200
return response
else:
add_to_requests_in_progress(post_data)
Should this work (return and prevent further routing)?
If not how can I prevent further routing after calling before_request()?
Is there a better way?
Based on what they have said in the documents, it should do what you want it to do.
The function will be called without any arguments. If the function returns a non-None value, it’s handled as if it was the return value from the view and further request handling is stopped.
(source)
#app.route("/<name>")
def index(name):
return f"hello {name}"
#app.before_request
def thing():
if "john" in request.path:
return "before ran"
with the above code, if there is a "john" in the url_path, we will see the before ran in the output, not the actual intended view. you will see hello X for other string.
so yes, using before_request and returning something, anything other than None will stop flask from serving your actual view. you can redirect the user or send them a proper response.
Background: Apache server using mod_wsgi to serve a Flask app using Flask_Sqlalchemy connecting to MySQL. This is a full stack application so it is nearly impossible to create a minimal example but I have tried.
My problem is that when I make some change that should modify the database subsequent requests don't always seem to reflect that change. For example if I create an object, then try to edit that same object, the edit will sometimes fail.
Most of the time if I create an object then go to the page listing all the objects, it will not show up on the list. Sometimes it will show up until I refresh, when it will disappear, and with another refresh it shows up again.
The same happens with edits. Example code:
bp = Blueprint('api_region', __name__, url_prefix='/app/region')
#bp.route('/rename/<int:region_id>/<string:name>', methods=['POST'])
def change_name(region_id, name):
region = Region.query.get(region_id)
try:
region.name = name
except AttributeError:
abort(404)
db.session.add(region)
db.session.commit()
return "Success"
#bp.route('/name/<int:region_id>/', methods=['GET'])
def get_name(region_id):
region = Region.query.get(region_id)
try:
name = region.name
except AttributeError:
abort(404)
return name
After object is created send a POST
curl -X POST https://example.com/app/region/rename/5/Europe
Then several GETs
curl -X GET https://example.com/app/region/name/5/
Sometimes, the GET will return the correct info, but every now and then it will return whatever it was before. Further example output https://pastebin.com/s8mqRHSR it happens at varying frequency but about one in 25 will fail, and it isn't always the "last" value either, when testing it seems to get 'stuck' at a certain value no matter how many times I change it up.
I am using the "dynamically bound" example of Flask_Sqlalchemy
db = SQLAlchemy()
def create_app():
app = Flask(__name__)
db.init_app(app)
... snip ...
return app
Which creates a scoped_session accessible in db.session.
Apache config is long and complicated but includes the line
WSGIDaemonProcess pixel processes=5 threads=5 display-name='%{GROUP}'
I can post more information if required.
For reference if anyone finds this thread with the same issue, I fixed my problem.
My Flask App factory function had the line app.app_context().push() leftover from the early days when it was based off a Flask tutorial. Unfortunately snipped out of the example code otherwise it might have been spotted by someone. During a restructuring of the project this line was left out and the problem fixed itself. Not sure why or how this line would cause this issue, and only for some but not all requests.
I am working to convert example code from Flask to twisted. The flask program is storing data in a session like so:
session['samlUserdata'] = self.auth.get_attributes()
session['samlNameId'] = self.auth.get_nameid()
session['samlSessionIndex'] = self.auth.get_session_index()
session['samlExpiration'] = datetime.now() + timedelta(minutes=SESSION_LENGTH)
In this case session is a flask global, but I want to accomplish the same thing in twisted. Basically I want to store values in my session so I can use that data in other requests.
I know I can access the session data in request.getSession() and have seen some examples of counters but the idea is not translating to what I am trying to do.
Can anyone explain how I would set and retrieve data in a twisted session? As I have said I have seen the counter example and need a more concrete example of how this would be done.
Thanks!
Twisted's session code was conceived long ago and a lot has changed in the web server landscape. I'm not sure if this is the "optimal way" of doing it but you can store session info in a cookie using JWT. Here's an example using klein
import uuid
from klein import Klein
import jwt
router = Klein()
#router.route('/jwt')
def cookie_magic(request):
# set cookie if none
if request.getCookie(b'session_token') is None:
session_id = uuid.uuid4().hex
session_token = jwt.encode({'foo': session_id}, '#TODO Secret', algorithm='HS256')
request.addCookie(b'session_token', session_token)
return 'session_id set to {0}'.format(session_id)
# get the cookie
session_token = jwt.decode(request.getCookie(b'session_token'), '#TODO Secret', algorithm='HS256')
return 'hello {0}'.format(session_token['foo'])
router.run('0.0.0.0', 7777)
This allows you to be more flexible in terms of sessions. For instance if you have another non-twisted web app, you can easily get the session token from the cookie.
In my website I have a user panel which displays info about user (or a custom message if he is not logged in). I'd like to know how can I do that a user object is accessible within every template? (like Django's CONTEXT_PROCESSORS).
I know I can add_request_method() to my config object but as far as I understand it will only make user's object available in request but I will have to add it to returned context manually each time - not good.
Maybe I should add user's object to session?
What is the proper way to do it?
What you need in Pyramid is an authentication policy. This can be as simple as adding these lines to your __init__.py:
from pyramid.authentication import AuthTktAuthenticationPolicy
def main(global_config, **settings):
...
authentication_policy = AuthTktAuthenticationPolicy('some_key')
config = Configurator(settings=settings,
authentication_policy=authentication_policy)
...
Once an authentication policy is in place, you can use the 'forget' and 'remember' functions from pyramid.security to allow users to log in and out. Here's a simplified view function that handles logins:
from pyramid.security import remember, forget
def sign_in(request):
username = request.POST.get('username')
password = request.POST.get('password')
if username and User.verify_password(username, password):
headers = remember(request, username)
else:
headers = forget(request)
return HTTPFound(location=request.route_url('index'),
headers=headers)
There are a few things you'll have to change there - maybe your User object doesn't have a 'verify_password' method; maybe you want to redirect to a different page on successful login, etc. But it's something to get started with.
Then in my base template I can add:
<%
from pyramid.security import authenticated_userid
user_id = authenticated_userid(request)
%>
to make the user_id available for use in the template and then:
% if user_id:
<p>Welcome ${user_id}</p>
% else:
<p>Please login here</p>
% endif
Note I'm using Mako syntax - if you're using a different templating engine you should check their syntax for imports, conditionals, etc.
I've given a basic outline of the different pieces and how they slot together, but Pyramid has powerful authentication features, and I encourage you to read a little about how it works:
The Pyramid documentation: http://docs.pylonsproject.org/projects/pyramid/en/latest/api/authentication.html
This excellent demo of Pyramid's authentication policies:
https://github.com/mmerickel/pyramid_auth_demo
I know you already mentioned the add_request_method() solution, but that's what I ended up with after following this tutorial on adding a user object to the request.
I might not be doing anything that out there with my pyramid apps, but basically every view and template I have has a request object as part of it, so within any template I can do <% request.user %> and have my user object and everything part of that available. I even have some helper methods and relationships that just come through and are available automagically as request.user.
What example case do you have where the request object isn't available?