I wrote a FastAPI and try to perform load tests using different tools. I have found that the performance from Locust is vastly different from time() python function:
Locust shows min=17ms, max=2469ms, 99%=2000ms
time() function shows min()=3ms, max=1739ms
Can someone please shed a light on why is that? Which one is more accurate?
Below are my programs:
Fast API Function:
app = FastAPI()
#app.post('/predict/')
def predict(request: PredictRequest):
logger.info('Invocation triggered')
start_time = time.time()
response = adapter.predict(request.dict())
latency_time = (time.time() - start_time) * 1000
latency_logger.info(f'Predict call latency: {latency_time} ms')
return response
Locust parameters:
-u 500 -t 10 -r 500
Locust File:
class User(HttpUser):
wait_time = between(1, 2.5)
host = "http://0.0.0.0:80"
#task
def generate_predict(self):
self.client.post("/predict/",
json={"cid": [],
"user_id": 5768586,
"store_ids": [2725, 2757],
"device_type": "ios"},
name='predict')
Locust Output:
Locust and time are measuring two different things. time is measuring how long it takes to run only your adapter.predict function, server side. Locust measures the time it takes a client to get a response from your server route, which includes not only your adapter.predict call but also who knows what all else before and after that. "Which is more accurate" depends on what it is you're trying to measure. If you just want to know how long it takes to call adapter.predict, then time will be more accurate. If you want to know how long it takes a client to get the results of your /predict route, Locust is more accurate.
Related
I was trying to execute the following code:
import requests as rq
def google_translation(text, model='nmt', source_language=''):
temptime = time.time()
URL = f"https://translation.googleapis.com/language/translate/v2?q={text}&target=en&source={source_language}&model={model}&
try:
time_check = time.time()
result = rq.get(URL,timeout=0.5,headers = headers)
print('api_call time ' + str((time.time()-time_check)*1000))
print('elapsed time in api call ' + str(result.elapsed.total_seconds()*1000))
except:
result = {}
The two print statements have 500 ms of difference.
API call time is approx 750 ms
Elapsed time in API call is somewhere around 251 ms
The problem is encountered when the code is executed with uwsgi.
Has anyone faced similar scenario? If yes, please let me know the possible reason and resolution for the same.
Thanks in advance
I am doing an assignment to send requests to 1000 specific websites (some of which seem to no longer exist) in Python (3) with the HEAD method and report statistics about their response headers. The script has to finish in five minutes. Obviously you can make requests take less time by reducing the timeout, but the more you reduce the timeout the more timeout errors there are, and catching them seems to be very expensive. For example, when the timeout was 0.3 seconds there were 700 good requests and 300 timeout errors, and the total time spent catching the timeout errors was by itself greater than five minutes. Reducing the timeout does reduce the time to catch each timeout error, because requests has to wait for the timeout before throwing the error, but the number of timeouts also increases. I was only able to get the total time spent catching timeout errors below five minutes at timeout=0.05 and timeout=0.03, but the total time including the time spent on requests was still greater than five minutes. timeout=0.02 resulted in only 20 sites being reachable with a total error handling time of 5:17, and timeout=0.01 resulted in no sites reachable. The person who gave the assignment insists that it is possible, so I must be doing something wrong. I tried using a requests.Session object but that didn't result in any noticeable speedup. What else can I do to speed things up?
The real answer is to use asynchronous HTTP requests. But in order to ethically answer this question I must insist on a low limit per domain for simultaneous requests, otherwise you can overload servers (and get blacklisted).
Below is an (untested) example implementation using aiohttp that supports a configurable number of maximum parallelism as well as maximum parallelism per domain.
import aiohttp
import asyncio
from collections import Counter
NUM_PARALLEL = 64
MAX_PARALLEL_PER_DOMAIN = 4
TIMEOUT = aiohttp.ClientTimeout(total=60)
async def fetch_url(url, session):
try:
async with session.get(url) as response:
# Whatever you want.
return {
"url": url,
"status": response.status,
"content-type": response.headers["content-type"]
}
except aiohttp.ServerTimeoutError:
return {"url": url, "status": "timeout"}
except Exception as e:
return {"url": url, "status": "uncaught_exception", "exception": e}
domain_num_inflight = Counter()
domain_semaphore = {}
async def worker(urls, results):
async with aiohttp.ClientSession(timeout=TIMEOUT) as session:
while urls:
url = urls.pop()
domain = urlparse(url).netloc
if domain_num_inflight[domain] == 0:
domain_semaphore[domain] = asyncio.Semaphore(MAX_PARALLEL_PER_DOMAIN)
domain_num_inflight[domain] += 1
async with domain_semaphore[domain]:
results.append(await fetch_url(url, session))
domain_num_inflight[domain] -= 1
if domain_num_inflight[domain] == 0: # Prevent memory leak.
del domain_semaphore[domain]
del domain_num_inflight[domain]
urls = [...]
worklist = urls[:]
results = []
workers = [worker(worklist, results) for _ in range(NUM_PARALLEL)]
loop = asyncio.get_event_loop()
loop.run_until_complete(asyncio.gather(*workers))
print(results)
I am using flask, and have a route that sends emails to people. I am using threading to send them faster. When i run it on my local machine it takes about 12 seconds to send 300 emails. But when I run it on lambda thorough API Gateway it times out.
here's my code:
import threading
def async_mail(app, msg):
with app.app_context():
mail.send(msg)
def mass_mail_sender(order, user, header):
html = render_template('emails/pickup_mail.html', bruger_order=order.ordre, produkt=order.produkt)
msg = Message(recipients=[user],
sender=('Sender', 'infor#example.com'),
html=html,
subject=header)
thread = threading.Thread(target=async_mail, args=[create_app(), msg])
thread.start()
return thread
#admin.route('/lager/<url_id>/opdater', methods=['POST'])
def update_stock(url_id):
start = time.time()
if current_user.navn != 'Admin':
abort(403)
if request.method == 'POST':
produkt = Produkt.query.filter_by(url_id=url_id)
nyt_antal = int(request.form['bestilt_hjem'])
produkt.balance = nyt_antal
produkt.bestilt_hjem = nyt_antal
db.session.commit()
orders = OrdreBog.query.filter(OrdreBog.produkt.has(func.lower(Produkt.url_id == url_id))) \
.filter(OrdreBog.produkt_status == 'Ikke klar').all()
threads = []
for order in orders:
if order.antal <= nyt_antal:
nyt_antal -= order.antal
new_thread = mass_mail_sender(order, order.ordre.bruger.email, f'Din bog {order.produkt.titel} er klar til afhentning')
threads.append(new_thread)
order.produkt_status = 'Klar til afhentning'
db.session.commit()
for thread in threads:
try:
thread.join()
except Exception:
pass
end = time.time()
print(end - start)
return 'Emails sendt'
return ''
AWS lambda functions designed to run functions within these constraints:
Memory– The amount of memory available to the function during execution. Choose an amount between 128 MB and 3,008 MB in 64-MB increments.
Lambda allocates CPU power linearly in proportion to the amount of memory configured. At 1,792 MB, a function has the equivalent of one full vCPU (one vCPU-second of credits per second).
Timeout – The amount of time that Lambda allows a function to run before stopping it. The default is 3 seconds. The maximum allowed value is 900 seconds.
To put this in your mail sending multi threaded python code. The function will terminate automatically either when your function execution completes successfully or it reaches configured timeout.
I understand you want single python function to send n number of emails "concurrently". To achieve this with lambda try the "Concurrency" setting and trigger your lambda function through a local script, S3 hosted html/js triggered by cloud watch or EC2 instance.
Concurrency – Reserve concurrency for a function to set the maximum number of simultaneous executions for a function. Provision concurrency to ensure that a function can scale without fluctuations in latency.
https://docs.aws.amazon.com/lambda/latest/dg/configuration-console.html
Important: All above settings will affect your lambda function execution cost significantly. So plan and compare before applying.
If you need any more help, let me know.
Thank you.
I'm trying to make a countdown timer in appJar and have a label that shows how much time is remaining until the end of the allotted amount of time. I've looked at the guides on appJar's website a fair amount and know of the two ways that they say you can create loops with their library. You can use .registerevent(function) or .after(delay_ms, function, *args). I've tried both of these ways and can't get either to work. I haven't managed to figure out how to get the .after function to work and every time I try to use the .registerevent function something doesn't work. My current issue is that I can get the function to run but it isn't actually working. That is, it says the block of code is running but the GUI ins't updating
Here are the specific lines of code in question
def introduction_bill(x):
global time_remaining, time_allotted
time_remaining = 120
time_allotted = 120
app.removeAllWidgets()
print("ran 'introduction_bill'")
app.addLabel('timer', 'Time remaining: 2:00')
app.addButtons(['start timer','update'], [start_timer, update_timer])
app.addButton('next introduction', next_introduction)
....
def update_timer():
global time_remaining, current_function
current_function = 'update timer'
time_remaining = end_time - t.time()
minutes = int(time_remaining // 60)
seconds = round(time_remaining % 60, 2)
app.setLabel('timer', 'Timer remaining: ' + str(minutes) + ':' + str(seconds))
print("ran 'update_timer'")
if time_remaining > -10:
app.registerEvent(update_timer)
def start_timer(x):
print("ran 'start_timer'")
global start_time, end_time
start_time = t.time()
end_time = start_time + time_allotted
update_timer()
And here is the code in its entirety
Keep in mind that I am still a beginner in coding so this code is both incomplete and very rough.
You can achieve what you want with both of the methods you mention.
With registerEvent() you should only call it once, it then takes care of scheduling your function. In your code, you are calling it repeatedly, which won't work.
With after() you have to take care of calling it again and again.
So, with your current code, you're better off using after():
In update_timer() try changing the call to app.registerEvent(update_timer) to be app.after(100, update_timer)
The timer should then update every 0.1 seconds.
However, if you click the Start Timer button again, you'll run into problems, so you might want to disable the button until the timer finishes.
I am running grinder to test a POST URI with 10 different json bodies. The response times given by grinder are not proper.Individual json body tests are giving a reasonable response time though 10 the script with with 10 different json bodies is giving a very high response time and a very low tps. I am using 1 agent with 5 worker processes and 15 threads.Can someone help me figure out where the problem might be?
The script I am using are :-
`from net.grinder.script.Grinder import grinder
from net.grinder.script import Test
from net.grinder.plugin.http import HTTPRequest
from HTTPClient import NVPair
from java.io import FileInputStream
test1 = Test(1, "Request resource")
request1 = HTTPRequest()
#response1 = HTTPResponse()
test1.record(request1)
log = grinder.logger.info
class TestRunner:
def __call__(self):
#request1.setDataFromFile("ReqBody.txt")
payload1 = FileInputStream("/home/ubuntu/grinder-3.11/scripts/Req.txt")
payload2 = FileInputStream("/home/ubuntu/grinder-3.11/scripts/Req2.txt")
payload3 = FileInputStream("/home/ubuntu/grinder-3.11/scripts/Req3.txt")
payload4 = FileInputStream("/home/ubuntu/grinder-3.11/scripts/Req4.txt")
payload5 = FileInputStream("/home/ubuntu/grinder-3.11/scripts/Req5.txt")
payload6 = FileInputStream("/home/ubuntu/grinder-3.11/scripts/Req6.txt")
payload7 = FileInputStream("/home/ubuntu/grinder-3.11/scripts/Req7.txt")
payload8 = FileInputStream("/home/ubuntu/grinder-3.11/scripts/Req8.txt")
payload9 = FileInputStream("/home/ubuntu/grinder-3.11/scripts/Req9.txt")
payload10 = FileInputStream("/home/ubuntu/grinder-3.11/scripts/Req10.txt")
headersPost = [NVPair('Content-Type', ' application/json')]
#request1.setHeaders(headersPost)
#request1.setHeaders
myload = [payload1, payload2, payload3, payload4, payload5, payload6, payload7, payload8, payload9, payload10]
for f in myload:
result1 = request1.POST("http://XX.XX.XX.XX:8080/api/USblocks/ORG101/1/N/",f,headersPost)
log(result1.toString())`
First step for you is to run it with 1 thread 1 process and 1 agent . I hope it will run properly.
It looks since for loop is used all the scripts are going to run for each and every thread.
I think what you want /what should be done is that each thread should be sending one request .
You can move the request out to a global method and take some random number like Grinder.threadNo and use it to return the script to be executed . This then also demands that you remove the for loop since you can control the number of times the script is executed or leave it running for a duration .
Running 10 threads in parallel is a good number to start with then slowly you can see how many requests are actually getting accepted.