In a .net-Application I work with several requests against a Web-Api, using HttpRequest. Since these request are not run on the main thread (async, await...), it is possible that some requests are launched at almost the same time, so a later one is started without a former one being finished. This, in the end, leads to an inconvienient behavior - for example when a first request runs into refreshing access tokens, a second one shall not start a similar request as well, but should wait for the first one to finish and run afterwards.
My idea is to somehow schedule the request in a FiFo-Way, using Arrays/List which are updated everytime one request is finished. A starting request then would check in this list if it is the one in line, or would wait for a start signal by some Array/List watcher.
However, I have strong doubts that this approach is the best/correct way for doing this. Any Help/Hint/Heads Up would be great!
Related
I've just started using MSW (https://mswjs.io/) in the tests. I came however across a case that I have no clue how to deal with.
Let's say we are testing component A (render <A /> in a test) which makes three different API requests. All of them are done independently in the useEffect section of component A.
The first test checks the influence of the first request (e.g. request returns a list and the list is rendered). The second test checks something related to second request and so forth. I'd like that any test is as independent as it could be and verifies one thing only.
Let's see what's happening in the first test:
render <A /> triggers three requests.
waitFor waits for the data from first requests and it's influence in the UI.
If A renders correctly - the tests passes and waitFor is over.
Second and third request is under way and the first tests is not waiting for it (since it's not related to things checked in the first test). This situation causes warnings Can't perform a React state update on an unmounted component.
What is the approach I should follow to get rid of the warning?
Should the first test has direct indication to wait for second and third request to be finished? If so, it means that I'm gonna end up with not independent tests. Is that correct?
I read some artirle talk about nodejs work as single thread. It talked nodejs handle second request after finished first request. So i try to test it. I have some code like
this. Then i send three request sequentially:
http://localhost:3000/?wait=1 : start at **14:29:14**.469Z
http://localhost:3000/?wait=1 : .........**14:29:19**.496Z
http://localhost:3000/ : .........**14:29:15**.725Z
I dont understand. I though third request should be started at 14:29:19, right ? or i misunderstood. Please explain for me, thanks everyone!
Javascript (not just node.js but even your web browser regardless if it's Chrome or Safari or IE or Edge) is single threaded. The structure of the javascript interpreter can generally be described by the following pseudocode:
script_to_exec = compile_js_from_files()
do {
event_queue = execute_javascript(script_to_exec)
completed_events = wait_for_events()
for_each (event from completed_events) {
this_event = event_queue.find_one_matching(event)
script_to_exec = this_event.callback
}
} while (there_is_pending_events)
You may notice that there is no threading code at all in the above code. All we are doing in wait_for_events() is to use the async I/O API provided by the OS. Depending on the OS this can be something like the POSIX select() or the Linux and OSX poll() or BSD epoll() or Windows overlapped I/O functions. Node uses the libuv library which will automatically select the best API to use at compile time.
Loop round 1
Ok. So first round of the loop it executes your entire code. This sets up a TCP socket listener when you set up express. Then it gets stuck at wait_for_events()
The first HTTP request causes wait_for_events() to return. This causes Express to go through your list of middlewares and routes to find a route to execute. It finds your route and calls wait() which calls setTimeout() which adds its callback to the list of timers.
Loop round 2
Since there is no more javascript to execute until await returns we go round the loop again and get stuck at wait_for_events() again.
The second HTTP request causes wait_for_events() to return. This causes Express to go through your list of middlewares and routes and repeat what I described previously.
Loop round 3
Since there is no more javascript to execute until await returns we go round the loop again and get stuck at wait_for_events() again.
The third HTTP request causes wait_for_events() to return. This causes Express to go through your list of middlewares and routes and repeat what I described previously.
Loop round 4
Again, we go round the loop again and get stuck at wait_for_events().
The first await wait() timer expires. This causes wait_for_events() to return and javascript figures out that the event is the timer for that wait therefore it continues processing the first HTTP request. This causes Express to send a response.
Loop round 5
We go round the loop again and wait for wait_for_events().
The second await wait() timer expires. This causes wait_for_events() to return and javascript figures out that the event is the timer for that wait therefore it continues processing the second HTTP request. This causes Express to send a response.
Loop round 6
We go round the loop again and wait for wait_for_events().
The third await wait() timer expires. This causes wait_for_events() to return and javascript figures out that the event is the timer for that wait therefore it continues processing the third HTTP request. This causes Express to send a response.
Loop round 7
We go round the loop again and get stuck at wait_for_events() again waiting for more HTTP requests...
Summary
Basically javascript can wait for multiple things in parallel. This evolved out of the fact that it started as a GUI scripting language that had to have the ability to wait for things like mouse click, keyboard input etc. But it can only execute one instruction at a time (unless of course you deliberately use features to execute additional processes/threads such as web workers or worker threads).
If you're wondering why people use node.js for high performance web servers I have written an answer to this related question explaining why node is fast: Node js architecture and performance
Here's some of my other answers going into different levels of details about asynchronous execution:
I know that callback function runs asynchronously, but why?
Is my understanding of asynchronous operations correct?
Is there any other way to implement a "listening" function without an infinite while loop?
node js - what happens to incoming events during callback excution
I know that callback function runs asynchronously, but why?
I've joined a legacy project, where there's virtually no logging. Few days ago we had a production release that failed massively, and we had no clear idea what's going on. That's why improving logging is one of the priorities now.
I'd like to introduce something like "correlation id", but I'm not sure what approach to take. Googling almost always brings me to the solutions that are suitable for "Microservices talking via REST" architecture, which is not my case.
Architecture is a mix of Spring Framework and NodeJS running on the same Unix box - it looks like this:
Spring receives a Request (first thread is started) and does minor processing.
Processing goes to a thread from ThreadPool (second thread is started).
Mentioned second thread starts a separate process of NodeJS that does some HTML processing.
Process ends, second thread ends, first thread ends.
Options that come to my mind are:
Generate UUID and pass it around as argument.
Generate UUID and store it in ThreadLocal, pass it when necessary when changing threads or when starting a process.
Any other ideas how it can be done correctly?
You are on the right track. Generate a UUID and pass it as a header into the request. For any of the request that do not have this header add a filter thats checks for it and add it.
Your filter will pick such a header and can put it in thread local where MDC can pick it from. There after any logging you do will have the correlation id. When making a call to any other process/request you need to make sure you pass this id as an argument/header. And the cycle repeats.
Your thread doing the task should just be aware of this ID. Its upto you to decide how you want to pass it. Try to just separate out such concerns from your biz logic (Using Aspects or any other way you see fit) and more you can keep this under the hood easier it would be for you.
You can refer to this example
I'm pretty new to Node.js, though I've been writing javascript for years. I'm more than open to any node advice for best-practices that I'm not following, or other rethinks. That said:
I'm building a system in which a user creates a reservation, and simultaneously submits a task for my firebase-queue to pick up. This queue has multiple specs associated with it. In turn, it's supposed to:
check availability, and in response confirm/throw an alert on the reservation and update the firebase data accordingly.
Update the users reservations, which is an index of reservation object keys, and removing any redundant ones.
Use node-schedule to create dated functions to send notifications about the pending expiration of their reservation.
However, when I run my script, only one of the firebase-queues that I instantiate runs. I can look in the dashboard and see that the progress is at 100, the _state is the new finished_state (which is the next spec's start_state), but that next queue won't pick up the task and process it.
If I quit my script and rerun it, that next queue will work fine. And then the queue after that won't work, until I repeat the act of quitting and rerunning the script. I can continue this until the entire task sequence completes, so I don't think the specs or the code being executed itself are blocking. I don't see any error states spring up, anyway.
From the documentation it looks like I should be able to write the script this way, with multiple calls to 'new Queue(queueRef, options, function(data, progress, resolve, reject)...' and they'll just run each task as I set them in their options (all of which are basically:
var options = {
'specId': 'process_reservation',
'numWorkers': 5,
'sanitize': true,
'suppressStack': false
};
but, nope. I guess I can spawn child-processes for each of the queue instances, but I'm not sure if that's an extreme reaction to the issues I'm seeing, and I'm not sure if it would complicate the node structure in terms of shared module exports. Also, I'm not sure if it'll start eating into my concurrent connection count.
Thanks!
I've got a Web application that continuously polls for data from the server using Ajax requests. I would like to implement an integration test for it using zombie.js.
What I am trying to do is to wait until the Ajax poll loop receives data from the server. The data should be received after 20 seconds, so I use browser.wait(done, callback) to check if the data is there, and set waitFor to a maximum timeout of one minute.
However, browser.wait() always returns almost immediately, even if my done callback returns false.
In the zombie API documentation, I read the following about browser.wait():
... it can't wait forever, especially not for timers that may fire repeatedly (e.g. checking page state, long polling).
I guess that's the reason for the behavior I see, but I don't really understand what's going on. Why can't I wait for one minute until my poll loop receives data from the server? Why can't browser.wait() wait for timers that may fire repeatedly? What do I need to do to implement my test?
Zombie.js will by default wait untill all the scripts on your page have loaded and executed if they are waiting for document ready.
If I understand you correctly, your script will not execute til after 20 seconds of document ready. In that case Zombie has a function which will let you evaluate javascript in the context of the browser, so you can kick off your ajax code quicker if it is on a timer, and you do not want to wait for it.
Look at browser.evaluate(expr)
Another option would be to simply use a normal JavaScript timeout to wait 20 seconds, and then look at the DOM for the changes you are expecting.
setTimeout(function(){
browser.document.query("#interestingElement")
}, 20*1000);