I'm new to flask and currently converting an existing WSGI application to run through flask as long term it'll make life easier.
All requests are POST to specific routes however the current application inspects the post data prior to executing the route to see if the request needs to be run at all or not (i.e. if an identifier supplied in the post data already exists in our database or not).
If it does exist a 200 code and json is returned "early" and no other action is taken; if not the application continues to route as normal.
I think I can replicate the activity at the right point by calling before_request() but I'm not sure if returning a flask Response object from before_request() would terminate the request adequately at that point? Or if there's a better way of doing this?
NB: I must return this as a 200 - other examples I've seen result in a redirect or 4xx error handling (as a close parallel to this activity is authentication) so ultimately I'm doing this at the end of before_request():
if check_request_in_progress(post_data) is True:
response = jsonify({'request_status': 'already_running'})
response.status_code = 200
return response
else:
add_to_requests_in_progress(post_data)
Should this work (return and prevent further routing)?
If not how can I prevent further routing after calling before_request()?
Is there a better way?
Based on what they have said in the documents, it should do what you want it to do.
The function will be called without any arguments. If the function returns a non-None value, it’s handled as if it was the return value from the view and further request handling is stopped.
(source)
#app.route("/<name>")
def index(name):
return f"hello {name}"
#app.before_request
def thing():
if "john" in request.path:
return "before ran"
with the above code, if there is a "john" in the url_path, we will see the before ran in the output, not the actual intended view. you will see hello X for other string.
so yes, using before_request and returning something, anything other than None will stop flask from serving your actual view. you can redirect the user or send them a proper response.
Related
I have studied the code as follow in Cherrypy web development,
if returnpage != '':
raise cherrypy.InternalRedirect(returnpage)
else:
raise cherrypy.HTTPRedirect("/hqc")
Google doesn't help much in this case after I did some research.
I've checked from cherrypy's __doc__, but the documentation there is very terse.
>>>print(cherrypy.InternalRedirect.__doc__)
Exception raised to switch to the handler for a different URL.
This exception will redirect processing to another path within the site
(without informing the client). Provide the new path as an argument when
raising the exception. Provide any params in the querystring for the new URL.
>>> print(cherrypy.HTTPRedirect.__doc__)
Exception raised when the request should be redirected.
This exception will force a HTTP redirect to the URL or URL's you give it.
The new URL must be passed as the first argument to the Exception,
e.g., HTTPRedirect(newUrl). Multiple URLs are allowed in a list.
If a URL is absolute, it will be used as-is. If it is relative, it is
assumed to be relative to the current cherrypy.request.path_info.
If one of the provided URL is a unicode object, it will be encoded
using the default encoding or the one passed in parameter.
There are multiple types of redirect, from which you can select via the
``status`` argument. If you do not provide a ``status`` arg, it defaults to
303 (or 302 if responding with HTTP/1.0).
Examples::
raise cherrypy.HTTPRedirect("")
raise cherrypy.HTTPRedirect("/abs/path", 307)
raise cherrypy.HTTPRedirect(["path1", "path2?a=1&b=2"], 301)
See :ref:`redirectingpost` for additional caveats.
My questions are:
- Why bother with redirect when you can simply invoke another handler?
- What are some practical senarios for the two redirect exception respectively?
InternalRedirect is only handled in the server side, this means that the client would not be aware of that redirection, because in terms of the HTTP protocol that is mediating the session between the client and the server, nothing changed. By server side I mean ONLY CherryPy will be aware of the rediction, if you have some intermediate server (like an nginx reverse proxy) it would not see anything different.
For example if the client visited a url /page_one and then you used raise InternalRedirect('/page_two'), the client (browser) will receive the content from the /page_two handler in the /page_one url. If you raised a regular HTTPRedirect the server would end the first request with an HTTP status code of 303 (or any other status that you passed to the exception) and a Location header to /page_two. Then is the client who will initiate another request to /page_two, basically everybody will be aware of the redirection (more info about HTTP redirection). Most of the time this is the better alternative.
Additionally you could detect if the request came from a previous InternalRedirect by verifying the cherrypy.request.prev property. It will have the previous cherrypy.request object as its value or None.
For the sake of a possible (maybe not the best example) use of an InternalRedirect, checkout this production/beta example page, in addition I added a tool to prohibit the client to reach to handlers directly.
The client will see a different content in the same page /. Note that the access log that CherryPy generates will log the url of the handler that end up handling the request, in this case you will see /_beta or /_production.
import random
import cherrypy
#cherrypy.tools.register('before_handler')
def private_handler():
"""End the request with HTTP 404 not found if the client
tries to reach the handler directly instead of being
internally redirected from other handler.
"""
if cherrypy.request.prev is None:
raise cherrypy.NotFound()
class MainApp:
#cherrypy.expose
def index(self):
# 50/50 change of receiving production or the new SHINY beta page
use_beta = random.randint(0, 1)
if use_beta:
raise cherrypy.InternalRedirect('/_beta')
else:
raise cherrypy.InternalRedirect('/_production')
#cherrypy.tools.private_handler()
#cherrypy.expose
def _production(self):
return (
"<html>"
"<h2>{}</h2>"
"</html>"
).format(
"Welcome to our awesome site!"
)
#cherrypy.tools.private_handler()
#cherrypy.expose
def _beta(self):
return (
"<html>"
'<h1 style="color: blue">{}</h1>'
"<p>{}</p>"
"</html>"
).format(
"Welcome to our awesome site!",
"Here is our new beta content..."
)
cherrypy.quickstart(MainApp())
I have Angular 8 web app. It needs to send some data for analysis to python flask App. This flask App will send it to 3rd party service and get response through webhook.
My need is to provide a clean interface to the client so that there is no need to provide webhook from client.
Hence I am trying to initiate a request from Flask app, wait until I get data from webhook and then return.
Here is the code.
#In autoscaled micro-service this will not work. Right now, the scaling is manual and set to 1 instance
#Hence keeping this sliding window list in RAM is okay.
reqSlidingWindow =[]
#app.route('/handlerequest',methods = ['POST'])
def Process_client_request_and_respond(param1,param2,param3):
#payload code removed
#CORS header part removed
querystring={APIKey, 'https://mysvc.mycloud.com/postthirdpartyresponsehook'}
response = requests.post(thirdpartyurl, json=payload, headers=headers, params=querystring)
if(response.status_code == SUCCESS):
respdata = response.json()
requestId = respdata["request_id"]
requestobject = {}
requestobject['requestId'] = requestId
reqSlidingWindow.append(requestobject)
#Now wait for the response to arrive through webhook
#Keep checking the list if reqRecord["AllAnalysisDoneFlag"] is set to True.
#If set, read reqRecord["AnalysisResult"] = jsondata
jsondata = None
while jsondata is None:
time.sleep(2)
for reqRecord in reqSlidingWindow:
if(reqRecord["requestId"] == da_analysis_req_Id ):
print("Found matching req id in req list. Checking if analysis is done.")
print(reqRecord)
if(reqRecord["AllAnalysisDoneFlag"] == True):
jsondata = reqRecord["AnalysisResult"]
return jsonify({"AnalysisResult": "Success", "AnalysisData":jsondata})
#app.route('/webhookforresponse',methods = ['POST'])
def process_thirdparty_svc_response(reqIdinResp):
print(request.data)
print("response receieved at")
data = request.data.decode('utf-8')
jsondata = json.loads(data)
#
for reqRecord in reqSlidingWindow:
#print("In loop. current analysis type" + reqRecord["AnalysisType"] )
if(reqRecord["requestId"] == reqIdinResp ):
reqRecord["AllAnalysisDoneFlag"] = True
reqRecord["AnalysisResult"] = jsondata
return
I'm trying to maintain sliding window of requests in list. Upon the
Observations so far:
First, this does not work. The function of 'webhookforresponse' does not seem to run until my request function comes out. i.e. my while() loop appears to block everything though I have a time.sleep(). My assumption is that flask framework would ensure that callback is called since sleep in another 'route' gives it adequate time and internally flask would be multithreaded?
I tried running python threads for the sliding window data structure and also used RLocks. This does not alter behavior. I have not tried flask specific threading.
Questions:
What is the right architecture of the above need with flask? I need clean REST interface without callback for angular client. Everything else can change.
If the above code to be used, what changes should I make? Is threading required at all?
Though I can use database and then pick from there, it still requires polling the sliding window.
Should I use multithreading specific to flask? Is there any specific example with skeletal design, threading library choices?
Please suggest the right way to proceed to achieve abstract REST API for angular client which hides the back end callbacks and other complexities.
Background: Apache server using mod_wsgi to serve a Flask app using Flask_Sqlalchemy connecting to MySQL. This is a full stack application so it is nearly impossible to create a minimal example but I have tried.
My problem is that when I make some change that should modify the database subsequent requests don't always seem to reflect that change. For example if I create an object, then try to edit that same object, the edit will sometimes fail.
Most of the time if I create an object then go to the page listing all the objects, it will not show up on the list. Sometimes it will show up until I refresh, when it will disappear, and with another refresh it shows up again.
The same happens with edits. Example code:
bp = Blueprint('api_region', __name__, url_prefix='/app/region')
#bp.route('/rename/<int:region_id>/<string:name>', methods=['POST'])
def change_name(region_id, name):
region = Region.query.get(region_id)
try:
region.name = name
except AttributeError:
abort(404)
db.session.add(region)
db.session.commit()
return "Success"
#bp.route('/name/<int:region_id>/', methods=['GET'])
def get_name(region_id):
region = Region.query.get(region_id)
try:
name = region.name
except AttributeError:
abort(404)
return name
After object is created send a POST
curl -X POST https://example.com/app/region/rename/5/Europe
Then several GETs
curl -X GET https://example.com/app/region/name/5/
Sometimes, the GET will return the correct info, but every now and then it will return whatever it was before. Further example output https://pastebin.com/s8mqRHSR it happens at varying frequency but about one in 25 will fail, and it isn't always the "last" value either, when testing it seems to get 'stuck' at a certain value no matter how many times I change it up.
I am using the "dynamically bound" example of Flask_Sqlalchemy
db = SQLAlchemy()
def create_app():
app = Flask(__name__)
db.init_app(app)
... snip ...
return app
Which creates a scoped_session accessible in db.session.
Apache config is long and complicated but includes the line
WSGIDaemonProcess pixel processes=5 threads=5 display-name='%{GROUP}'
I can post more information if required.
For reference if anyone finds this thread with the same issue, I fixed my problem.
My Flask App factory function had the line app.app_context().push() leftover from the early days when it was based off a Flask tutorial. Unfortunately snipped out of the example code otherwise it might have been spotted by someone. During a restructuring of the project this line was left out and the problem fixed itself. Not sure why or how this line would cause this issue, and only for some but not all requests.
Just yesterday got that I can't use one function that would return one template depending on cookies, and wich would be called by different methods of different routes. The reason is all the response.set_cookie() are not applied to main Bottle object HTTPResponse before the method finishes serving current route. So the question is there a way to explicitly cast application of all the changes to HTTPResponse object, so i could avoid passing complex structures to subroutines and etc.
Thank you for help!
UPD: there is a response.set_cookie('temp', 'sampletext') line for example. And then i am calling subroutine and it calls another one and so on. So In each of them i won't have request.get_cookie('temp') returning 'sampletext'. Because changes to cookies wasn't applied yet. There are in the Bottle.py code you can find following code:
class HTTPResponse(Response, BottleException):
def __init__(self, body='', status=None, headers=None, **more_headers):
super(HTTPResponse, self).__init__(body, status, headers, **more_headers)
def apply(self, response):
response._status_code = self._status_code
response._status_line = self._status_line
response._headers = self._headers
response._cookies = self._cookies
response.body = self.body
that seems to be executing once per request and changes from httpresponse objects are appliied to the response object only on finish of the rout's serving method termination.
I am asking if there is a way to apply changes to response object manually, during the route's serving method evaluation.
I am asking if there is a way to apply changes to response object manually, during the route's serving method evaluation.
No.
I'm not completely sure, but it sounds like you just want to store some state during a request.
The mechanism for this is to simply set attributes on the request object. E.g.
#route(...)
def index_page():
...
request.mydata = 'hello'
...
This is completely independent of cookies. If you need both, just set both; one on the request object and the other on the response.
I know for a PreparedRequest, we specifiy the timeout in session.send call, where session can be an instance of requests.Session class, as I've already seen here: http://docs.python-requests.org/en/latest/user/advanced/
But I need to put the timeout before I do the send. Somewhat inherently, to the PreparedRequest object. Because I'm using the session.send method as map function, to map to PreparedRequest instances.
def async_req2resp(reqs, session):
responses = []
try:
with futures.ThreadPoolExecutor(max_workers=workers) as executor:
for response in executor.map(session.send, requests):
responses.append(response)
except Exception as e:
print('async_requests2responses exception: ' + str(e))
return responses
Now I need to specify the timeout somehow. How can I do it?
The above code is wrapped inside a method. I am getting the session object as an argument. Is there a way I can set the timeout for all the requests that will be sent with the session object, before the parallel execution of session.send happens?
Thank you.
I'm waiting for your answer about sendit but until then, let me give you some information that should help you get the rest of the way there.
If you have a PreparedRequest object as request then you can do session.send(request, timeout=timeout_val) where timeout_val is whatever the value of the timeout is that you want.
With that in mind, this question becomes, "How do I pass that to sendit with each request?" and that question I don't have the answer for.
Side note:
You've tagged this with concurrent.futures which makes me suspect you're trying to optimize your application that uses this method. With that in mind, you should avoid appending to a list at all costs. You should, instead, try to do the following:
responses = []
try:
responses = list(sendit.map(session.send, requests))
# etc.
If you figure out how to pass the timeout parameter to sendit, feel free to suggest an edit to this answer to update it to be complete.
With the updated information in the question, I can more accurately answer your question.
Before you call executor.map you should do the following:
import functools
timedout_send = functools.partial(session.send, timeout=my_timeout)
try:
list(executor.map(timedout_send, requests))
# ...