I have a normal NestJs application. I have a couple of APIs in it. Lately, I noticed that when I am testing them using Postman, the responses are being cached.
For eg.
I hit API-1 ... it gives me a response.
I hit API-2 ... it gives me the response plus the previous response.
Is there a tweak to disable this?
Thanks in advance.
This is API-1 which returns a normal response.
This is API-2 which returns the response of the previous API-1
#Controller({path: "product",scope: Scope.REQUEST})
I think this is the thing. Since node is single threaded it was not disposing my earlier instance and was using the same response variable over and over again. So my API-1 saved response to a variable called RESPONSE.
On the next request API-2 was using the same variable RESPONSE.
The RESPONSE variable structure was like this -
{
status: bool,
message: string,
data: object
}
API-1 modified it to
{
status: true,
message: "Done",
data: {"key":"value"}
}
However, API-2 was only using the first two keys.
{
status: true,
message: "Not done",
data: {"key":"value"}
}
And hence got the third data key of the previous request.
Ideally, this should not happen. But after applying the scope thing it got solved.
https://docs.nestjs.com/fundamentals/injection-scopes
Controllers can also have scope, which applies to all request method
handlers declared in that controller. Like provider scope, the scope
of a controller declares its lifetime. For a request-scoped
controller, a new instance is created for each inbound request, and
garbage-collected when the request has completed processing.
Related
I'm always reluctant to claim that a bug that I'm seeing is actually a .Net Core bug, but after spending 8+ hours investigating the following bug, it looks like a .Net Core SignalR bug to me. I need techniques for tracking this down further and for fixing it.
The first rule of honing in on a bug is to try to create a minimal amount of code that can consistently repro the bug. While I can't reproduce it in a small stand along project, I have worked hard try to zero in on what's happening.
I have a controller with the following action method
[HttpPost]
[Route("/hack/ajax/start")]
public JsonResult AjaxStart([FromBody] JObject data) {
//A call to some method that does some work
return Json(new {
started = true
});
}
Calling this code via a jquery ajax call or Postman works flawlessly every time if I do not have any SignalR Core 1.0 hubs registered in the startup.cs method. However, when I register the following in the startup.cs file I have intermittent issues.
namespace App.Site.Home {
public class HackHub : Hub {
public async Task SendMessage(string status, string progress) {
await Clients.All.SendAsync("serverMsg", status, progress);
}
}
}
Startup.cs ConfigureServices contains
services.AddSignalR();
Startup.cs Configure contains
app.UseSignalR(routes => {
routes.MapHub<App.Site.Home.HackHub>("/hub/hack");
});
If I were to comment out the one line above routes.MapHub<App.Site.Home.HackHub>("/hub/hack"); everything works fine every time. However with this line present, (I.e. some SignalR hub registered) then that's when the fun starts for me, even if I have no code executing on the client or server that makes use of the hub!
The issue is that sometimes when a HTTP POST request is made for the action method above, something in .Net Core (SignalR??) is converting the POST method to Post, and then because Post is not a valid HTTP Method it converts it to a blank method. And since My action method requires an HTTP POST a 404 status code is returned. Many of the HTTP POSTS for that endpoint work fine, but often the issue I just described occurres.
To ensure that my client code was not part of the problem, I was able to reproduce my issue using Postman to make the requests. Further to ensure that POST was actually being sent and not Post, I used Fiddler to watch what was going over the wire. All this is documented below.
Here is the first request (which always works) done via Postman:
Here is the second (identical!) request done via Postman, this one resulted in a 404:
Here is what the first request (the one that worked properly) looked like in fiddler:
Here is what the second request looked like in fiddler:
As you can see, the requests are identical. But the response certainly is not.
So to get a better idea what the server was seeing, I added the following code to the beginning of the startup.cs Configure method. Due to it's placement, for the request this code gets to run before any other application code or middleware.
public void Configure(IApplicationBuilder app, IHostingEnvironment env) {
//for debugging
app.Use(async (context, next) => {
if(context.Request.Method == "") {
string method = context.Request.Method;
string path = context.Request.Path;
IHttpRequestFeature requestFeature = context.Features.Get<IHttpRequestFeature>();
string kestralHttpMethod = requestFeature.Method;
string stop = path;
}
await next();
});
//more code here...
}
For the first request, the request.Method was POST as one would expect:
But for the second request request.Method was blank!!
To investigate this further, I accessed the requestFeature and checked the Http Method Method there. This is where things get really interesting. If I just hover over the property in the debuggger, it's blank too.
But, If I expand the requestFeature object and look at the Method property there, is it Post!!!
That alone seems like craziness. How can two views of the SAME property in the debugger have different values???! It would seem that some code converted the POST to Post, and at some level the system knows that Post is not a valid http method so in some views of that variable it's converted to a blank string. But that is so weird!
Also, we clearly saw via Postman and Fiddler that POST was sent, so how did it get changed to Post? What code did that? I'd like to claim that it can't be my code since I'm checking the value of the RequestFeature before any of my other code related to the request gets a chance to run. Also, if I comment out the one line of code that registers that SignalR hub, then POST is never converted to Post and I never get a 404. But with that SignalR hub registered I periodically get this behavior.
Are there any SignalR or other .net Core switches I can turn on to get better trace or logging info to see when the POST is getting changed to Post? Is there a way to fix this?
This question was looked into via this GitHub issue https://github.com/aspnet/KestrelHttpServer/issues/2591 which was originally opened up when someone else also observed random 404 errors
I want to especially thank #ben-adams for his help in understanding what was going on.
Let me start by saying that this did not turn out to be a bug in the framework. It was a bug in my code. How can that be given what I was observing?
Well, it's like this...
In some parts of the HttpRequest the method is a string, but it in other parts it's an enum. The enum value for POST is Post. So that's why the case conversion was happening.
The reason that one part of the request was saying Post while the other part showed a Method value of a blank string was because the request object was gummed up because I had accessed it at a time when it was in between requests.
How did I do THAT? you may wonder. Well let me tell you, because the plot thickens...
I turns out that I have some logging code that that gathers context information when it's called and one of the pieces of context info it gathers is the current request.Method. When this logging code is called from a main thread, there is no issue.
However, my system does have some code that runs on background threads that are either started via a Timer or via a ThreadPool.QueueUserWorkItem. If this code hits an exception it will call the same logger code.
When my logger code, running on a background thread, checks for the current httpContext via IHttpContextAccessor I fully expected it to receive null. And certainly this same code in the same situation when accessing the current HttpContext via HttpContext.Current in a non .Net Core website does receive null. But as it turns out, under .Net core, it wasn't receiving null, it was receiving an object. But that object was for a request that had already finished and who's request object had already been reset!!!
Starting in .Net Core 2.0 the HttpContext, and it's child objects like request, gets reset after the connection for a request closes. So the HttpContext object (and it's request object) the logger code was getting when running on a background thread was an object that had been reset. It's request.Path for example was null.
It turns out that a request in this state does not expects it's request.Method property to be accessed. And doing so gums up the works for the next request that comes in. Ultimately this is the source of why the next request that came in ended up returning a 404 error.
So how do we fix this? Why does IHttpContextAccessor return an object rather than null in this out of context situation especially given that the object may very possibly be between requests? The answer is that when I was used Timer or ThreadPool.QueueUserWorkItem to create a background task, the Execution Context was being flowed to the new thread. This is just what happens by default when you use these API methods. But, internally the IHttpContextAccessor uses an AsyncLocal to keep track of the current HttpContext and since my new thread received Execution Context from the main thread it had access to the same AsyncLocal. And so IHttpContextAccessor provided an object rather than the null I was expecting when called from a background thread.
The fix? (Thank you #Ben-Adams!) Instead of calling ThreadPool.QueueUserWorkItem I needed to call ThreadPool.UnsafeQueueUserWorkItem instead. This method DOES NOT flow the current Execution Context to the new thread, and therefore the new thread won't have access to those AsyncLocals from the main thread. Once I did this, IHttpContextAccessor then returned null when called from the background thread instead of returning a object that was in between requests and untouchable. Yea!
When creating a `Timer' I also needed to change my code to do it in a way that would not flow Execution Context. Here is the code I use (which was inspired by some #Ben-Adams suggested):
public static Timer GetNewTimer(TimerCallback callback, object state, int dueTime, int interval) {
bool didSuppress = false;
try {
if (!ExecutionContext.IsFlowSuppressed()) {
//We need to suppress the flow of the execution context so that it does not flow to our
//new asynchronous thread. This is important so that AsyncLocals (like the one used by
//IHttpaccessor) do not flow to the new thread we are pushing our work to. By not flowing the
//execution context, IHttpAccessor wil return null rather than bogusly returning a context for
//a request that is in between requests.
//Related info: https://github.com/aspnet/KestrelHttpServer/issues/2591#issuecomment-399978206
//Info on Execution Context: https://blogs.msdn.microsoft.com/pfxteam/2012/06/15/executioncontext-vs-synchronizationcontext/
ExecutionContext.SuppressFlow();
didSuppress = true;
}
return new Timer(callback, state, dueTime, interval);
} finally {
// Restore the current ExecutionContext
if (didSuppress) {
ExecutionContext.RestoreFlow();
}
}
}
This only leaves one remaining question unanswered. My original question noted that registering a SignalR hub was causing the system to exhibit this random 404 behavior but the system did not exhibit this behavior when no SignalR hub was registered (or so I thought). Why was this? I truly don't know. Perhaps it was putting more resource pressure on some part of the system and thus causing the issue to show up more easily. Not sure. All I know is that the root issue was that I was flowing Execution Context to my background threads without realizing it and that was causing the IHttpContextAccessor's AsyncLocal to be in scope. Not flowing the Execution Context to the background threads fixes that issue.
I am attempting to retrieve a file I have stored within a node within an Azure Batch pool. Passing a GET request using the URL:
https://ResourceName.southcentralus.batch.azure.com/jobs/adfv2-ResourceName/tasks/adaa9831-fca7-4562-8a7b-8aed60de151f/files/wd/filename.dat
Returns the error:
{
"odata.metadata":"https://ResourceName.southcentralus.batch.azure.com/$metadata#Microsoft.Azure.Batch.Protocol.Entities.Container.errors/#Element","code":"MissingRequiredQueryParameter","message":{
"lang":"en-US","value":"A query parameter that's mandatory for this request is not specified.\nRequestId:f72f00f7-1549-4dc4-b374-abaf3bd30b58\nTime:2018-05-18T13:59:17.0275742Z"
},"values":[
{
"key":"QueryParameterName","value":"api-version"
}
]
}
However, there is no indication as to the parameter to pass...
How can I retrieve the file?
The error response shows you what is needed in the request:
"values":[
{
"key":"QueryParameterName","value":"api-version"
}
]
An api-version is required on all requests. So for your example, try issuing a GET using:
https://ResourceName.southcentralus.batch.azure.com/jobs/adfv2-ResourceName/tasks/adaa9831-fca7-4562-8a7b-8aed60de151f/files/wd/filename.dat?api-version=2018-03-01.6.1
You can see the full REST API documentation for this action, which gives you an example.
For more information on available versions, see this doc.
I've been working for the last couple of weeks on a small Couch/NodeJS project where I need some field validation (required, type...). I solved this by adding validate_doc_update to the design doc.
After doing some test I realized that I was receiving 403 forbidden HTTP code instead of 400 bad request, which I think is more appropriate in this use case.
CouchDB Documentation says that I can just throw one of these error objects:
Throws: forbidden error to gracefully prevent document storing. (403)
Throws: unauthorized error to prevent storage and allow the user to re-auth. (401)
Any ideas on what's the best approach to manage this kind of validation issues?
Update:
content of validate_doc_update:
function (newDoc, oldDoc, userCtx){
function require(field, message){
message = message || "Block must have a " + field;
if (!newDoc[field]) throw({forbidden : message});
};
if(newDoc.type == "block"){
require("name")
}
}
I know that throw({forbidden: message}) will give 403... Is there any way to change that behaviour on CouchDB or should Ibuild my own validation middleware on express?
Yeah, this is an ugly corner of CouchDB actually. I'm surprised there's still not a provision to respond with a 422 (which is much more appropriate for an invalid document than a 403), but yes - what we did was add our own middleware to do the right thing.
I am trying to get security working with my jersey2 web app.
I register RolesAllowedDynamicFeature and my Request filter with AUTHENTICATION priority in my ResourceConfig
packages("example.jersey");
register(MyRequestFilter.class, Priorities.AUTHENTICATION);
register(RolesAllowedDynamicFeature.class);
I added #RolesAllowed to the method
#RolesAllowed("quinn")
#GET
#Path("/")
public Response getIt(#Context UriInfo uriInfo) {
return Response.ok().entity(service.get()).build();
}
In my request filter I set my security context
SecurityContext securityContext = containerRequestContext.getSecurityContext();
containerRequestContext.setSecurityContext(new MySecurityContext("gary", securityContext));
When I call the method from postman I get a 403 - Forbidden
I added logging to my request filter to see when it is called. It is NOT called.
If I remove the #RolesAllowed from the web method it does call the request filter.
It seems the Priorities.AUTHENTICATION is not making a difference.
Is there anything I'm missing?
Your filter is implemented as a post-matching filter. It means that the filters would be applied only after a suitable resource method has been selected to process the actual request i.e. after request matching happens. Request matching is the process of finding a resource method that should be executed based on the request path and other request parameters.
#RolesAllowed blocks the selection of the particular resource method giving you the 'not executing' behavior you mentioned.
You have two options... using #PreMatching as explained here.
Or, use custom annotations as explained on a similar question.
My nodejs application uses a basic REST communication style to allow an HTML web ui to pass commands.
For instance:
http://address/api/config/cmd1
http://address/api/config/cmd2
http://address/api/network/cmd3
...
In return, my web ui gets a JSON result of the form:
{
"success": true
}
or
{
"success": false,
"errorMsg": "Wrong parameter blabla"
}
My problem is, I now need to translate error messages on client-side (in many languages), and the english "errorMsg" is too variadic and too long to be a translation key.
So I need something like an "errorCode" (an integer, probably) and I'm searching for a strategy into my nodejs application to manage error codes. I don't really know what is usually done for that, considering I usually use throw new Error("message") to return the message directly to the web ui.
I don't know if it's better to make a list of uniq error codes for all my REST API of a contextual error list for every subset of this API.
UPDATE: finally, I opted for a string error id. For instance, "wrong argument for this command" becomes "WrongArgument" and will be used to identify the error on GUI side and thus, perform the localization process. And finally, I don't need to make the error id uniq.
Using a custom Error class in the back-end with error identifier will allow the front-end to directly use its translation module, without modifications.
From your Server, one can process the standard error by creating a custom class, throw the class, catch it in your controller and send to the front-end the response with correct http status and error id.
The reasons are:
1 - Front-end code maintanability.
2 - Keeping errors systems data on back-end logs because they might be sensitive. It should not be return to the client side.
I agree with naming identifier code instead of using a code number. This because it s hard to pre-define a range for each error type (SQL, API, Authentication...). And it s harder when the error may be found in different service.
Scope: In the front-end service, i define a scope when calling the server, so that if there is need for a global error to be more specific, it is there. The scope is just a string with the name of the page where the ressources is used, also found in JSON lang files.
If 'ld like to check this code out and give comments, it will be great:
'https://codepen.io/Aymer-El/pen/OJoRVgZ'
Also leaving place for a debug message in the response may help front-end devs. Tho, this is optional.