Read data in chunks from large request_body - python-3.x

I am new to tornado . Currently I want to read from a post request_body . But data in post request is large so I want to implement it through stream_body_request in Tornado. But I am not able to implement it. How to read this 'img_data' in chunks?
#stream_request_body
class MyReportPDF(BaseHandler):
async def post(self):
data = escape.json_decode(self.get_body_argument('img_data'))#This data is an base_64_array
for i in range(len(data)):
# Clean the base_64 Image
data[i].replace('data:image/jpeg;base64,', '')
decode_image.append(base64.b64decode(data[i]))

When you use #stream_request_body, your RequestHandler class should define a method data_received which will be called with data as it comes in. The post method is called at the end after all of the data is received.
Note that it is not currently possible to use the argument-related methods with #stream_request_body; you'll need to parse the data in data_received as it comes in. This means that if possible you'll want to structure your API to receive the image as a plain HTTP PUT instead of being wrapped in a form-style POST.

Related

What happens if I do not use body parser or express.json()?

I am new to the whole backend stuff I understood that both bodyparser and express.json() will parse the incoming request(body from the client) into the request object.
But what happens if I do not parse the incoming request from the client ?
without middleware parsing your requests, your req.body will not be populated. You will then need to manually go research on the req variable and find out how to get the values you want.
Your bodyParser acts as an interpreter, transforming http request, in to an easily accessible format base on your needs.
You may read more on HTTP request here ( You can even write your own http server )
https://nodejs.org/api/http.html#http_class_http_incomingmessage
You will just lose the data, and request.body field will be empty.
Though the data is still sent to you, so it is transferred to the server, but you have not processed it so you won't have access to the data.
You can parse it yourself, by the way. The request is a Readable stream, so you can listen data and end events to collect and then parse the data.
You shall receive what you asked for in scenarios where you do not convert the data you get the raw data that looks somewhat like this username=scott&password=secret&website=stackabuse.com, Now this ain't that bad but you will manually have to filter out which is params, what is a query and inside of those 2 where is the data..
unless it is a project requirement all that heavy lifting is taken care of by express and you get a nicely formatted object looking like this
{
username: 'scott',
password: 'secret',
website: 'stackabuse.com'
}
For Situation where you DO need to use the raw data express gives you a convenient way of accessing that as well all you need to do is use this line of code
express.raw( [options] ) along with express.json( [options] )

How to read GTFS Real-time feed using Python

This is not a repeat question. I am desgning a routing algorithm that works in realtime. I know all about GTFS static feed but cant figure out my cities realtime. I want to understand how to parse and GTFS realtime feed using Python.
Link to Realtime set of my city is https://opendata.iiitd.edu.in/data/realtime/
I know a little about Protocol buffers and request library. Documentaion does not give the link to which API request should be placed.
What is the Url for this set which goes into requests.get?
There is a python library called gtfs_realtime_pb2, which can be used to parse the response of a request to the realtime feed and receive usefull output.
The main page for this library is here. There are also binding for other languages.
The main workflow is to initialize the
feed = gtfs_realtime_pb2.FeedMessage()
get the response from the feed with the package requests
response = requests.get(<url>, allow-redirects = True)
and parse it for example:
feed.ParseFromString(response.content)
feed.entity[int]

Displaying a form and saving the submited data

I have a simple app that saves and displays records off of a database,
#user_endpoints.get("/user/<id>")
def get_user(request, id):
dct = User.by_id(id)
if not dct:
return response.json({"Error": "Not Found"}, status=404)
return response.json(dct.to_dict(), status=200)
When it comes to displaying a user list something like the code below was sufficient,
#user_endpoints.get("/users")
def list_users(request):
dct = User.all()
template = template_env.get_template('user_list.html')
content = template.render(title='User List', users=dct)
return response.html(content)
The above uses Jinja2 but that is not important to me (this is not a real app). I am not clear on how to display a form for creating a new user and saving the submitted data, can someone provide a simple example for that?
There is no "one way" with Sanic to do that. Sanic does not know about forms. It completely depends on what you do in the frontend, how you encode the data. Are you sending JSON or is it "form-data" encoded? Maybe something completely different?
You would certainly use "POST" instead of "GET" like you did above. You can inspect request and find your data that's been sent from the frontend and then act on it. (Although nowadays you'd start with designing and implementing a proper (REST) API - usually based on JSON - and then use that. The word "form" does not appear here. It's just data.)
You could use Sanic-WTF
There is an example with Sanic-WTF + Jinja2 - https://bitbucket.org/voron-raven/synergy/src/1b8172a0bc61c5239c5ad2a2b9f064ed50ff81dd/views.py#lines-88

Reading POST data - CherryPy web-framework

I'm new with CherryPy web-framework. I would like to know how can
I read POST data that was delivered via HTTP request's body .
Thanks,
Gabriel.
You can either read the body of the request with cherrypy.request.body, cherrypy.request.params or if you use the default handler then, the distinction between GET and POST values is abstracted by cherrypy and you can get the values directly from the arguments:
#cherrypy.expose
def index(self, name, age):
return "My name is %s and age is %s" % (name, age)
Your POST request must have to provide the values for name and age on the traditional forms submission.
If you are planning on using json, then use the cherrypy.tools.json_in decorator and read the cherrypy.request.json property. http://docs.cherrypy.org/en/latest/basics.html#dealing-with-json
If you are using the MethodDispatcher, then there is another way to do it: http://docs.cherrypy.org/en/latest/tutorials.html#tutorial-7-give-us-a-rest
Maybe these posts might help you:
http://blog.joel.mx/posts/cherrypy-101
http://blog.joel.mx/posts/cherrypy-101-method-dispatcher
Cherrypy handles both get and post in a similar way by passing them into the function or method that is handling the request. A good example is located at tut03_get_and_post.py in the tutorial folder within the cherrypy package.
Here is a small portion that specifically speaks to your question...
#cherrypy.expose
def greetUser(self, name=None):
# CherryPy passes all GET and POST variables as method parameters.
# It doesn't make a difference where the variables come from, how
# large their contents are, and so on.

bottle httprequest object manual update

Just yesterday got that I can't use one function that would return one template depending on cookies, and wich would be called by different methods of different routes. The reason is all the response.set_cookie() are not applied to main Bottle object HTTPResponse before the method finishes serving current route. So the question is there a way to explicitly cast application of all the changes to HTTPResponse object, so i could avoid passing complex structures to subroutines and etc.
Thank you for help!
UPD: there is a response.set_cookie('temp', 'sampletext') line for example. And then i am calling subroutine and it calls another one and so on. So In each of them i won't have request.get_cookie('temp') returning 'sampletext'. Because changes to cookies wasn't applied yet. There are in the Bottle.py code you can find following code:
class HTTPResponse(Response, BottleException):
def __init__(self, body='', status=None, headers=None, **more_headers):
super(HTTPResponse, self).__init__(body, status, headers, **more_headers)
def apply(self, response):
response._status_code = self._status_code
response._status_line = self._status_line
response._headers = self._headers
response._cookies = self._cookies
response.body = self.body
that seems to be executing once per request and changes from httpresponse objects are appliied to the response object only on finish of the rout's serving method termination.
I am asking if there is a way to apply changes to response object manually, during the route's serving method evaluation.
I am asking if there is a way to apply changes to response object manually, during the route's serving method evaluation.
No.
I'm not completely sure, but it sounds like you just want to store some state during a request.
The mechanism for this is to simply set attributes on the request object. E.g.
#route(...)
def index_page():
...
request.mydata = 'hello'
...
This is completely independent of cookies. If you need both, just set both; one on the request object and the other on the response.

Resources