I know for a PreparedRequest, we specifiy the timeout in session.send call, where session can be an instance of requests.Session class, as I've already seen here: http://docs.python-requests.org/en/latest/user/advanced/
But I need to put the timeout before I do the send. Somewhat inherently, to the PreparedRequest object. Because I'm using the session.send method as map function, to map to PreparedRequest instances.
def async_req2resp(reqs, session):
responses = []
try:
with futures.ThreadPoolExecutor(max_workers=workers) as executor:
for response in executor.map(session.send, requests):
responses.append(response)
except Exception as e:
print('async_requests2responses exception: ' + str(e))
return responses
Now I need to specify the timeout somehow. How can I do it?
The above code is wrapped inside a method. I am getting the session object as an argument. Is there a way I can set the timeout for all the requests that will be sent with the session object, before the parallel execution of session.send happens?
Thank you.
I'm waiting for your answer about sendit but until then, let me give you some information that should help you get the rest of the way there.
If you have a PreparedRequest object as request then you can do session.send(request, timeout=timeout_val) where timeout_val is whatever the value of the timeout is that you want.
With that in mind, this question becomes, "How do I pass that to sendit with each request?" and that question I don't have the answer for.
Side note:
You've tagged this with concurrent.futures which makes me suspect you're trying to optimize your application that uses this method. With that in mind, you should avoid appending to a list at all costs. You should, instead, try to do the following:
responses = []
try:
responses = list(sendit.map(session.send, requests))
# etc.
If you figure out how to pass the timeout parameter to sendit, feel free to suggest an edit to this answer to update it to be complete.
With the updated information in the question, I can more accurately answer your question.
Before you call executor.map you should do the following:
import functools
timedout_send = functools.partial(session.send, timeout=my_timeout)
try:
list(executor.map(timedout_send, requests))
# ...
Related
I am trying to write an API client for Telegram using Telethon.
https://github.com/LonamiWebs/Telethon
If you create a TelegramClient(session) it prompts for input upon initialization if your session isn’t authorized.
This is great when manually running the program from the terminal, but what if I want to run it inside a daemon or cron job?
They are using the default Input method from python3 to gather the input. I don’t see any way in the library to specify a session file and check if it’s logged in that can be run before initializing a TelegramClient, and it’s the initializer that will prompt for input if not logged in.
This feels like a catch 22! Does anybody know if this might produce an error that could be caught? Or what happens when input() is run with no tty? Would it just hang? Could I add a timeout in that case?
Thanks in advance for helping me understand better!
You are affirming that the initialization of TelegramClient invokes the input function as default, but this is done inside the TelegramClient.start method (docs).
Taking the solution that you give at the end of your question is a fair aproach, so let's use a timeout when invoking input.
from asyncio import get_event_loop, wait_for, TimeoutError
from functools import partial
from telethon import TelegramClient
async def ainput(prompt):
"""Reads input from stdin in an async way"""
loop = get_event_loop()
await loop.run_in_executor(None, print, prompt)
return await loop.run_in_executor(None, input)
async def get_code(timeout):
"""Waits for the code from stdin with a timeout"""
try:
return await wait_for(
ainput("Please, type the code you received: "),
timeout=timeout
)
except TimeoutError:
pass
client = TelegramClient(session, api_id, api_hash).start(
phone=phone,
code_callback=partial(get_code, 30)
)
You should keep in mind that when you call start the arguments phone, and password also reads from stdin if it isn't provided a callable or default value, so you can handle them like in this example with code_callback.
In your case you can get the code from a POST to your API or in other way, just get creative and write the callable that fits your needs.
I'm coding a script that connects to the Binance websocket and uses the .run_forever() method to constantly get live data from the site. I want to be able to debug my code and watch the values of variables as the change but I'm not sure how to do this as the script basically hangs on the line with the .run_forever() method, because it is an infinite event loop. This is by design as I want to continuously get live data (it receives a message approximately every second), but I can't think of a way a good way to debug it.
I'm using VSCode and here are some snippets of my code to help understand my issue. The message function for the websocket is just a bunch of technical analysis and trade logic, but it is also the function that contains all the changing variables that I want to watch.
socket = f"wss://stream.binance.com:9443/ws/{Symbol}#kline_{interval}"
def on_open(ws):
print("open connection")
def on_message(ws, message):
global trade_list
global in_position
json_message = json.loads(message)
candle = json_message['k'] # Accesses candle data
...[trade logic code here]...
def on_close(ws):
print("Websocket connection close")
# ------------------------- Define a websocket object ------------------------ #
ws = websocket.WebSocketApp(socket, on_open=on_open, on_message=on_message, on_close=on_close)
ws.run_forever()
If more code is required to answer the question, then I can edit this question to include it (I'm thinking if you would like to have an idea of what variables I want to look at, I just thought it would be easier and simpler to show these parts).
Also, I know using global isn't great, once I've finished (or am close to finishing) the script, I want to go and tidy it up, I'll deal with it then.
I'm a little late to the party but the statement
websocket.enableTrace(True)
worked for me. Place it just before you define your websocket object and it will print all traffic in and out of the websocket including any exceptions that you might get as you process the messages.
I'm new to flask and currently converting an existing WSGI application to run through flask as long term it'll make life easier.
All requests are POST to specific routes however the current application inspects the post data prior to executing the route to see if the request needs to be run at all or not (i.e. if an identifier supplied in the post data already exists in our database or not).
If it does exist a 200 code and json is returned "early" and no other action is taken; if not the application continues to route as normal.
I think I can replicate the activity at the right point by calling before_request() but I'm not sure if returning a flask Response object from before_request() would terminate the request adequately at that point? Or if there's a better way of doing this?
NB: I must return this as a 200 - other examples I've seen result in a redirect or 4xx error handling (as a close parallel to this activity is authentication) so ultimately I'm doing this at the end of before_request():
if check_request_in_progress(post_data) is True:
response = jsonify({'request_status': 'already_running'})
response.status_code = 200
return response
else:
add_to_requests_in_progress(post_data)
Should this work (return and prevent further routing)?
If not how can I prevent further routing after calling before_request()?
Is there a better way?
Based on what they have said in the documents, it should do what you want it to do.
The function will be called without any arguments. If the function returns a non-None value, it’s handled as if it was the return value from the view and further request handling is stopped.
(source)
#app.route("/<name>")
def index(name):
return f"hello {name}"
#app.before_request
def thing():
if "john" in request.path:
return "before ran"
with the above code, if there is a "john" in the url_path, we will see the before ran in the output, not the actual intended view. you will see hello X for other string.
so yes, using before_request and returning something, anything other than None will stop flask from serving your actual view. you can redirect the user or send them a proper response.
I've been stuck on this same issue for short of a week now:
the program should add widgets based on a http request. However, that request may take some time depending on user's internet connection, so I decided to thread that request and add a spinner to indicate that something is being done.
Here lies the issue. Some piece of code:
#mainthread
def add_w(self, parent, widget):
parent.add_widget(widget)
def add_course():
# HTTP Request I mentioned
course = course_manager.get_course(textfield_text)
courses_stack_layout = constructor_screen.ids.added_courses_stack_layout
course_information_widget = CourseInformation(coursename_label=course.name)
self.add_w(courses_stack_layout, course_information_widget)
constructor_screen.ids.spinner.active = False
add_course is being called from a thread, and spinner.active is being set True before calling this function. Here's the result, sometimes: messed up graphical interface
I also tried solving this with clock.schedule_once and clock.schedule_interval with a queue. The results were the same. Sometimes it works, sometimes it doesn't. The spinner does spin while getting the request, which is great.
Quite frankly, I would've never thought that implementing a spinner would be so hard.
How to implement that spinner? Maybe another alternative to threading? Maybe another alternative to urllib to make a request?
edit: any feedback on how I should've posted this so I can get more help? Is is too long? Maybe I could've been more clear?
The problem here was simply that widgets must also be created within the mainthread.
Creating another function marqued with #mainthread and calling that from the threaded one solved the issue.
Thanks for those who contributed.
Just yesterday got that I can't use one function that would return one template depending on cookies, and wich would be called by different methods of different routes. The reason is all the response.set_cookie() are not applied to main Bottle object HTTPResponse before the method finishes serving current route. So the question is there a way to explicitly cast application of all the changes to HTTPResponse object, so i could avoid passing complex structures to subroutines and etc.
Thank you for help!
UPD: there is a response.set_cookie('temp', 'sampletext') line for example. And then i am calling subroutine and it calls another one and so on. So In each of them i won't have request.get_cookie('temp') returning 'sampletext'. Because changes to cookies wasn't applied yet. There are in the Bottle.py code you can find following code:
class HTTPResponse(Response, BottleException):
def __init__(self, body='', status=None, headers=None, **more_headers):
super(HTTPResponse, self).__init__(body, status, headers, **more_headers)
def apply(self, response):
response._status_code = self._status_code
response._status_line = self._status_line
response._headers = self._headers
response._cookies = self._cookies
response.body = self.body
that seems to be executing once per request and changes from httpresponse objects are appliied to the response object only on finish of the rout's serving method termination.
I am asking if there is a way to apply changes to response object manually, during the route's serving method evaluation.
I am asking if there is a way to apply changes to response object manually, during the route's serving method evaluation.
No.
I'm not completely sure, but it sounds like you just want to store some state during a request.
The mechanism for this is to simply set attributes on the request object. E.g.
#route(...)
def index_page():
...
request.mydata = 'hello'
...
This is completely independent of cookies. If you need both, just set both; one on the request object and the other on the response.