For an async web_url Loadrunner has added the web_reg_async_attributes. Within the Poll_0_ResponseCB I'm waiting until the aHttpStatusCode == 200. This is needed because it's an XHR request where the server transfers all the data async to the browser. The transaction is considered finished AFTER all the data has been received in the GET request.
The request looks as following:
web_reg_async_attributes("ID=Poll_0",
"Pattern=Poll",
"URL=https://[URL]/api2/notifications/GetUnreadNotifications",
"PollIntervalMs=500",
"RequestCB=Poll_0_RequestCB",
"ResponseCB=Poll_0_ResponseCB",
LAST);
web_url("GetUnreadNotifications_2",
"URL=https://[URL]/api2/notifications/GetUnreadNotifications",
"Resource=0",
"RecContentType=application/json",
"Referer=https://[URL]/",
"Snapshot=t14.inf",
"Mode=HTML",
LAST);
web_sync("ParamCreated=stopAsync", "RetryIntervalMs=500", "RetryTimeoutMs=120000", LAST);
web_stop_async("ID=Poll_0",LAST);
Loadrunner sees the polling mechanism as Wasted Time, but in reality it's polling untill all data is received and I need to include this in the actual Duration.
How can I include the web_sync polling part inside Duration instead of Wasted Time?
ended with a "Pass" status (Duration: 33,9532 Wasted Time: 33,3178).
Yes, this API is that slow...
I believe wasted time is the time to execute the loadrunner API itself. This came about in the late 1990s because a competing tool laid marketing landmines at customers for the Mercury team for not tracking the time to execute the APIs like their "superior tool" did at the time.
Well, you place a barrier in front of Mercury sales, then engineering gets involved. And "Boom!" we get tracked wasted time, along with tracked think time. You could reduce your polling window from 500ms to 750 milliseconds to have less overhead on your polling execution.
Related
I am working on a application that uses the Spotify Web API to build and maintain playlists for the user based on a given recipe (just a JSON that represents a logic-scheme basically). Currently the application is in development mode. I use delays between each API call I make, currently about 400ms. And I also had delays of 7.5s when I got the occasional 429 error (too many requests).
Anyway, I recently made it so that all of the playlist recipes get rebuilt in an infinite loop. So the process is just always running and making API calls about every 100ms, in order to keep all of the playlists up-to-date based on the recipes. However after letting this loop run for about 10 minutes, I started persistently getting 429s even after retrying after 7.5s and longer.
Apparently the 429 responses contain a header called 'retry-after' which is how long Spotify suggests waiting before making another call (as I said, before I was just using a fixed 7.5s delay on 429s). I am seeing that the value I am receiving for 'retry-after' is on the order of about 76,000s (21 hours).
But I thought that the rate limits are enforced over a 30s window...
(see https://developer.spotify.com/documentation/web-api/guides/rate-limits/) So why is my 'retry-after' header so high?
This is mostly a design philosophy question so the code itself I think is mostly irrelevant but if you'd like to take a look it's available here: https://github.com/jakefoglia/Smart-Playlist-Manager
site/SPM-core/maintainer.js : contains the 'infinite loop'
site/SPM-core/spotify_api_hook.js : contains most of the API calls
The 30s window is presented in the documentation only as an example, not as an actual way in which the API works. As you correctly say, Retry-After header (value is seconds) is all the information you need to decide how long to wait before doing the next call.
Each time your app "violates" the rate limit by making an early request, it gets "punished" by an increased delay period, — and since the app apparently never even consulted the header, and repeatedly violated the limit, the delay got this high. This however did not result in shutdown, or blocking, or rejection, or something similar, because the header only suggests the duration of a delay, rather than enforcing it.
Let's say you have a parking meter app. User selects an amount of time and pays. 20 minutes before their time is up you want to send them a text via Twilio that their time is almost up. I'm not concerned about the payment or text part. What's the best way to do the timing aspect in Node that triggers the function that sends the text 20min before their time is up? Im aware of setTimeout, but is this a scalable method of handling this? IIRC, setTimeout doesn't execute at exactly the end of it's timer, but is dependant on when it can execute within the event loop. Let's assume you may need a couple hundred timers running at once and your server is realtively busy with other users triggering other callbacks and async functions. Also, the text doesn't necessarily have to be sent at exactly 20min before their time is up, a couple minutes margin of error would be acceptable. Thanks for any help!
I have a Python project with a server that distributes work to one or more clients. Each client is given a number of assignments which contain parameters for querying a target API. This includes a maximum number of requests per second they can make with a given API key. The clients process the response and send the results back to the server to store into a database.
Both the server and clients use Tornado for asynchronous networking. My initial implementation for the clients relied on the PeriodicCallback to ensure that n-number of calls to the API would occur. I thought that this was working properly as my tests would last 1-2 minutes.
I added some telemetry to collect statistics on performance and noticed that the clients were actually having issues after almost exactly 2 minutes of runtime. I had set the API requests to 20 per second (the maximum allowed by the API itself) which the clients could reliably hit. However, after 2 minutes performance would fluctuate between 12 and 18 requests per second. The number of active tasks steadily increased until it hit the maximum amount of active assignments (100) given from the server and the HTTP request time to the API was reported by Tornado to go from 0.2-0.5 seconds to 6-10 seconds. Performance is steady if I only do 14 requests per second. Anything higher than 15 requests will experience issues 2-3 minutes after starting. Logs can be seen here. Notice how the column of "Active Queries" is steady until 01:19:26. I've truncated the log to demonstrate
I believed the issue was the use of a single process on the client to handle both communication to the server and the API. I proceeded to split the primary process into several different processes. One handles all communication to the server, one (or more) handles queries to the API, another processes API responses into a flattened class, and finally a multiprocessing Manager for Queues. The performance issues were still present.
I thought that, perhaps, Tornado was the bottleneck and decided to refactor. I chose aiohttp and uvloop. I split the primary process in a similar manner to that in the previous attempt. Unfortunately, performance issues are unchanged.
I took both refactors and enabled them to split work into several querying processes. However, no matter how much you split the work, you still encounter problems after 2-3 minutes.
I am using both Python 3.7 and 3.8 on MacOS and Linux.
At this point, it does not appear to be a limitation of a single package. I've thought about the following:
Python's asyncio library cannot handle more than 15 coroutines/tasks being generated per second
I doubt that this is true given that different libraries claim to be able to handle several thousand messages per second simultaneously. Also, we can hit 20 requests per second just fine at the start with very consistent results.
The API is unable to handle more than 15 requests from a single client IP
This is unlikely as I am not the only user of the API and I can request 20 times per second fairly consistently over an extended period of time if I over-subscribe processes to query from the API.
There is a system configuration causing the limitation
I've tried both MacOS and Debian which yield the same results. It's possible that's it a *nix problem.
Variations in responses cause a backlog which grows linearly until it cannot be tackled fast enough
Sometimes responses from the API grow and shrink between 0.2 and 1.2 seconds. The number of active tasks returned by asyncio.all_tasks remains consistent in the telemetry data. If this were true, we wouldn't be consistently encountering the issue at the same time every time.
We're overtaxing the hardware with the number of tasks generated per second and causing thermal throttling
Although CPU temperatures spike, neither MacOS nor Linux report any thermal throttling in the logs. We are not hitting more than 80% CPU utilization on a single core.
At this point, I'm not sure what's causing it and have considered refactoring the clients into a different language (perhaps C++ with Boost libraries). Before I dive into something so foolish, I wanted to ask if I'm missing something simple.
Conclusion
Performance appears to vary wildly depending on time of day. It's likely to be the API.
How this conclusion was made
I created a new project to demonstrate the capabilities of asyncio and determine if it's the bottleneck. This project takes two websites, one to act as the baseline and the other is the target API, and runs through different methods of testing:
Spawn one process per core, pass a semaphore, and query up to n-times per second
Create a single event loop and create n-number of tasks per second
Create multiple processes with an event loop each to distribute the work, with each loop performing (n-number / processes) tasks per second
(Note that spawning processes is incredibly slow and often commented out unless using high-end desktop processors with 12 or more cores)
The baseline website would be queried up to 50 times per second. asyncio could complete 30 tasks per second reliably for an extended period, with each task completing their run in 0.01 to 0.02 seconds. Responses were very consistent.
The target website would be queried up to 20 times per second. Sometimes asyncio would struggle despite circumstances being identical (JSON handling, dumping response data to queue, returning immediately, no CPU-bound processing). However, results varied between tests and could not always be reproduced. Responses would be under 0.4 seconds initially but quickly increase to 4-10 seconds per request. 10-20 requests would return as complete per second.
As an alternative method, I chose a parent URI for the target website. This URI wouldn't have a large query to their database but instead be served back with a static JSON response. Responses bounced between 0.06 seconds to 2.5-4.5 seconds. However, 30-40 responses would be completed per second.
Splitting requests across processes with their own event loop would decrease response time in the upper-bound range by almost half, but still took more than one second each to complete.
The inability to reproduce consistent results every time from the target website would indicate that it's a performance issue on their end.
I'm working on a crawler and I've noticed that by setting the length of time for waiting 1 minute per request has made the application more reliable and I now get fewer connection resets. Can you recommend a reasonable amount of time to wait? I think 1 minute is quite the belts and braces approach and I would like to reduce this ideally.
In this Node app I'm working on, it's possible for users to book appointments.
When an appointment is booked, the users will later get a reminder by mail X hours before the actual appointment.
I'm thinking about using Node-schedule for this task.
For each appointment: Set up a future Date, send the reminder mail once and the delete this particular scheduled job
But... there might be ALOT of appointments booked when the app grows, and this means there will be ALOT of Node-schedule processes simultaneously sleeping and waiting to fire...
On a regular day, lets pretend we have 180 appointments booked for the future per clients, and lets pretend the app right now has 50 clients. This gives us around 9000 scheduled tasks sleeping and waiting to fire.
Question: Is this perfectly OK? ... or will all these simultaneously scheduled task be to much/many?
Short answer: 9000 is not a lot, you're good to go. However, I would advise you to write a benchmark to see for yourself.
I checked node-schedule's source and sure enough, it delegates scheduling to setTimeout for date-based tasks. This means that your jobs are scheduled using node.js internal event loop.
This code is very efficient, as node.js is tailored to handle several thousands of requests simultaneously.
Regardless of the number of pending jobs, node.js will only care about the next task and sleep until its date (unless an async I/O interrupts it), so scheduling a task is essentially O(1).