Handling parallel REST post requests - xpages

I have created my own REST service based on the examples from "Domino Sample REST Service Feature" from 901v00_11.20141217-1000 version of XPages Extension Library.
As far as I understand the design of the library each REST request will be run in its own thread on the server. This approach does not allow to handle parallel POST requests to the same document.
I have not found any examples in XPages Extension Library which would handle post requests as transactions on the server, i.e. which would block the server resource for the whole request processing time and would put put next requests in the queue?
Can anybody point to the source code of the service which would allow to handle parallel requests?
The skeleton for my post request processing function is this
#POST
#Path(PATH_SEPARATOR + MyURL)
#Consumes(MediaType.APPLICATION_JSON)
#Produces(MediaType.APPLICATION_JSON)
public Response myPost(
String requestEntity,
#Context final UriInfo uriInfo)
{
LogMgr.traceEntry(this, "myPost");
RestContext.verifyUserContext();
String myJson = ... // Process post
Response response = buildResponse(myJson);
LogMgr.traceExit(this, "myPost", "OK");
return response;
}
And I would like to implement something like this
// Start transaction
String myJson = ... // Process post
// Stop transaction
Is there a way to do it in Java?

I suppose you could use document locking in traditional Notes/Domino context - and synchronized in Java :-)
Have you tried any of these? I cannot see why they should not work.
/John

I agree with John. You can use document locking to prevent simultaneous updates to the same document. You might also want to consider some changes to the definition of your REST API.
First, you imply you are using POST to update an existing document. Usually, POST is used to create a new resource. Consider using PUT instead of POST.
Second, even with document locking, you still might want to check for version conflicts. For example, let's say a client reads version 2 of a document and then attempts to update it. Meanwhile, another client has already updated the document to version 3. Many REST APIs use HTTP ETags to handle such version conflicts.

Related

SignalR Core 1.0 intermittently changes the case of http method for non signalR POST, need fix (AKA Random 404 Errors)

I'm always reluctant to claim that a bug that I'm seeing is actually a .Net Core bug, but after spending 8+ hours investigating the following bug, it looks like a .Net Core SignalR bug to me. I need techniques for tracking this down further and for fixing it.
The first rule of honing in on a bug is to try to create a minimal amount of code that can consistently repro the bug. While I can't reproduce it in a small stand along project, I have worked hard try to zero in on what's happening.
I have a controller with the following action method
[HttpPost]
[Route("/hack/ajax/start")]
public JsonResult AjaxStart([FromBody] JObject data) {
//A call to some method that does some work
return Json(new {
started = true
});
}
Calling this code via a jquery ajax call or Postman works flawlessly every time if I do not have any SignalR Core 1.0 hubs registered in the startup.cs method. However, when I register the following in the startup.cs file I have intermittent issues.
namespace App.Site.Home {
public class HackHub : Hub {
public async Task SendMessage(string status, string progress) {
await Clients.All.SendAsync("serverMsg", status, progress);
}
}
}
Startup.cs ConfigureServices contains
services.AddSignalR();
Startup.cs Configure contains
app.UseSignalR(routes => {
routes.MapHub<App.Site.Home.HackHub>("/hub/hack");
});
If I were to comment out the one line above routes.MapHub<App.Site.Home.HackHub>("/hub/hack"); everything works fine every time. However with this line present, (I.e. some SignalR hub registered) then that's when the fun starts for me, even if I have no code executing on the client or server that makes use of the hub!
The issue is that sometimes when a HTTP POST request is made for the action method above, something in .Net Core (SignalR??) is converting the POST method to Post, and then because Post is not a valid HTTP Method it converts it to a blank method. And since My action method requires an HTTP POST a 404 status code is returned. Many of the HTTP POSTS for that endpoint work fine, but often the issue I just described occurres.
To ensure that my client code was not part of the problem, I was able to reproduce my issue using Postman to make the requests. Further to ensure that POST was actually being sent and not Post, I used Fiddler to watch what was going over the wire. All this is documented below.
Here is the first request (which always works) done via Postman:
Here is the second (identical!) request done via Postman, this one resulted in a 404:
Here is what the first request (the one that worked properly) looked like in fiddler:
Here is what the second request looked like in fiddler:
As you can see, the requests are identical. But the response certainly is not.
So to get a better idea what the server was seeing, I added the following code to the beginning of the startup.cs Configure method. Due to it's placement, for the request this code gets to run before any other application code or middleware.
public void Configure(IApplicationBuilder app, IHostingEnvironment env) {
//for debugging
app.Use(async (context, next) => {
if(context.Request.Method == "") {
string method = context.Request.Method;
string path = context.Request.Path;
IHttpRequestFeature requestFeature = context.Features.Get<IHttpRequestFeature>();
string kestralHttpMethod = requestFeature.Method;
string stop = path;
}
await next();
});
//more code here...
}
For the first request, the request.Method was POST as one would expect:
But for the second request request.Method was blank!!
To investigate this further, I accessed the requestFeature and checked the Http Method Method there. This is where things get really interesting. If I just hover over the property in the debuggger, it's blank too.
But, If I expand the requestFeature object and look at the Method property there, is it Post!!!
That alone seems like craziness. How can two views of the SAME property in the debugger have different values???! It would seem that some code converted the POST to Post, and at some level the system knows that Post is not a valid http method so in some views of that variable it's converted to a blank string. But that is so weird!
Also, we clearly saw via Postman and Fiddler that POST was sent, so how did it get changed to Post? What code did that? I'd like to claim that it can't be my code since I'm checking the value of the RequestFeature before any of my other code related to the request gets a chance to run. Also, if I comment out the one line of code that registers that SignalR hub, then POST is never converted to Post and I never get a 404. But with that SignalR hub registered I periodically get this behavior.
Are there any SignalR or other .net Core switches I can turn on to get better trace or logging info to see when the POST is getting changed to Post? Is there a way to fix this?
This question was looked into via this GitHub issue https://github.com/aspnet/KestrelHttpServer/issues/2591 which was originally opened up when someone else also observed random 404 errors
I want to especially thank #ben-adams for his help in understanding what was going on.
Let me start by saying that this did not turn out to be a bug in the framework. It was a bug in my code. How can that be given what I was observing?
Well, it's like this...
In some parts of the HttpRequest the method is a string, but it in other parts it's an enum. The enum value for POST is Post. So that's why the case conversion was happening.
The reason that one part of the request was saying Post while the other part showed a Method value of a blank string was because the request object was gummed up because I had accessed it at a time when it was in between requests.
How did I do THAT? you may wonder. Well let me tell you, because the plot thickens...
I turns out that I have some logging code that that gathers context information when it's called and one of the pieces of context info it gathers is the current request.Method. When this logging code is called from a main thread, there is no issue.
However, my system does have some code that runs on background threads that are either started via a Timer or via a ThreadPool.QueueUserWorkItem. If this code hits an exception it will call the same logger code.
When my logger code, running on a background thread, checks for the current httpContext via IHttpContextAccessor I fully expected it to receive null. And certainly this same code in the same situation when accessing the current HttpContext via HttpContext.Current in a non .Net Core website does receive null. But as it turns out, under .Net core, it wasn't receiving null, it was receiving an object. But that object was for a request that had already finished and who's request object had already been reset!!!
Starting in .Net Core 2.0 the HttpContext, and it's child objects like request, gets reset after the connection for a request closes. So the HttpContext object (and it's request object) the logger code was getting when running on a background thread was an object that had been reset. It's request.Path for example was null.
It turns out that a request in this state does not expects it's request.Method property to be accessed. And doing so gums up the works for the next request that comes in. Ultimately this is the source of why the next request that came in ended up returning a 404 error.
So how do we fix this? Why does IHttpContextAccessor return an object rather than null in this out of context situation especially given that the object may very possibly be between requests? The answer is that when I was used Timer or ThreadPool.QueueUserWorkItem to create a background task, the Execution Context was being flowed to the new thread. This is just what happens by default when you use these API methods. But, internally the IHttpContextAccessor uses an AsyncLocal to keep track of the current HttpContext and since my new thread received Execution Context from the main thread it had access to the same AsyncLocal. And so IHttpContextAccessor provided an object rather than the null I was expecting when called from a background thread.
The fix? (Thank you #Ben-Adams!) Instead of calling ThreadPool.QueueUserWorkItem I needed to call ThreadPool.UnsafeQueueUserWorkItem instead. This method DOES NOT flow the current Execution Context to the new thread, and therefore the new thread won't have access to those AsyncLocals from the main thread. Once I did this, IHttpContextAccessor then returned null when called from the background thread instead of returning a object that was in between requests and untouchable. Yea!
When creating a `Timer' I also needed to change my code to do it in a way that would not flow Execution Context. Here is the code I use (which was inspired by some #Ben-Adams suggested):
public static Timer GetNewTimer(TimerCallback callback, object state, int dueTime, int interval) {
bool didSuppress = false;
try {
if (!ExecutionContext.IsFlowSuppressed()) {
//We need to suppress the flow of the execution context so that it does not flow to our
//new asynchronous thread. This is important so that AsyncLocals (like the one used by
//IHttpaccessor) do not flow to the new thread we are pushing our work to. By not flowing the
//execution context, IHttpAccessor wil return null rather than bogusly returning a context for
//a request that is in between requests.
//Related info: https://github.com/aspnet/KestrelHttpServer/issues/2591#issuecomment-399978206
//Info on Execution Context: https://blogs.msdn.microsoft.com/pfxteam/2012/06/15/executioncontext-vs-synchronizationcontext/
ExecutionContext.SuppressFlow();
didSuppress = true;
}
return new Timer(callback, state, dueTime, interval);
} finally {
// Restore the current ExecutionContext
if (didSuppress) {
ExecutionContext.RestoreFlow();
}
}
}
This only leaves one remaining question unanswered. My original question noted that registering a SignalR hub was causing the system to exhibit this random 404 behavior but the system did not exhibit this behavior when no SignalR hub was registered (or so I thought). Why was this? I truly don't know. Perhaps it was putting more resource pressure on some part of the system and thus causing the issue to show up more easily. Not sure. All I know is that the root issue was that I was flowing Execution Context to my background threads without realizing it and that was causing the IHttpContextAccessor's AsyncLocal to be in scope. Not flowing the Execution Context to the background threads fixes that issue.

node-red wait for async query for http-response

I have REST-endpoint done with node-red which do some async tasks, for example odbc-query to the database.
When I'm resulting query to the http-response. It's just hangs on browser.
Looks like I need forward req and res object to the http-response. But how I do it?
Here is the flow:
[{"id":"a35e6901.509a1","type":"http in","z":"ec714e97.d73b58","name":"","url":"/get","method":"get","upload":false,"swaggerDoc":"","x":287.8571472167969,"y":1660.357177734375,"wires":[["7c116e0a.f8d67"]]},{"id":"98e22918.79bc4","type":"sqldbs","z":"ec714e97.d73b58","mydb":"8df7f8b0.bd44c","querytype":"select","name":"","x":723.5714111328125,"y":1644.6428833007812,"wires":[["de837ef9.5c1cf8"]]},{"id":"7c116e0a.f8d67","type":"function","z":"ec714e97.d73b58","name":"","func":"msq.topic = \"Select * from User\"\nreturn msg;","outputs":1,"noerr":0,"x":469.28570556640625,"y":1613.9286499023438,"wires":[["98e22918.79bc4"]]},{"id":"de837ef9.5c1cf8","type":"http response","z":"ec714e97.d73b58","name":"","statusCode":"","headers":{},"x":922.1428833007812,"y":1677.8570861816406,"wires":[]},{"id":"8df7f8b0.bd44c","type":"sqldbsdatabase","z":"","host":"127.0.0.1","port":"1433","db":"test","dialect":"mysql"}]
You need to make sure that it's the same message object that flows from the http-in node all the way to the http-out node.
This message object will have fields of msg.req and msg.res and the msg.res field will be used by the http-out node to send the response back to the browser.
Any node that takes an input and then forwards it on should preserve these field and not just create a new object for the output. All the built in nodes do this correctly and as should all the published 3rd party nodes. If you find one that does not follow this pattern you should raise a bug with the maintainer.
Your flow looks ok, but I've never used the DB nodes you have so can't comment on those.

What happens when a single NHibernate method called through multiple web api requests?

I have a method for authentication and it will be called for all the requests made by UI. When a multiple requests sent from client same time I am getting an exception
HResult=-2146233079
Message=There is already an open DataReader associated with this Command which must be closed first.
We are using NHibernate. And here I'm trying to get a single row data for all the requests.
And one more thing while mapping tables and in Ioc we are using lifeStyle as Transient.
Can some one please tell how to handle this in NHibernate ?
It seems that you're using the same session across several client requests that are processed in different threads (sorry, it's not clear from your question). Check your design - you should use different session instances in different threads
Thanks for the reply.
While opening a session will check 3 conditions.
ISession GetSession()
{
if (_session == null)
_session = _sessionFactory.OpenSession();
if (_session.IsOpen == false)
_session = _sessionFactory.OpenSession();
if (_session.Connection.State != System.Data.ConnectionState.Open)
{
_session.Connection.Close();
_session.Connection.Open();
}
return _session;
} code here
Based on this I'll get the session.
For single api request it works fine. But when multiple web api requests were sent then I am facing a problem in getting data from DB. getting error as explained above.

Does netsuite have REST ful API? [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 2 years ago.
Improve this question
I want to know does Netsuite provides REST ful api? Currently i am doing integration with my application(java) with soap based web services.i have done some research but didn't get useful information.IF it does where can i found api?
Avoid the SuiteTalk SOAP web services API like the plague; it will do nothing but waste your time to. Usage of Netsuite SOAP API is viable only when you are okay with the SOAP API being non performant, don't mind interacting with gross buggy SOAP API, have much time to implement robust error handling to account for the random SOAP errors, concurrency errors. You'll need much time to develop robust fault tolerance. All that time will be wasted time; because no amount of time will make the SOAP API performance acceptable.
RESTlet's are preferred over SOAP API usage for writing data; RESTlets tend to be slightly more performant for writes (although responses are still extremely slow and not suitable for a customer facing app).
RESTlet's are a viable short term solution for writing data to Netsuite. Its essentially a JS script that allows you to set up a token based auth poor man's JSON endpoint; in which you can send JSON request bodies and get back JSON response bodies. Usage is reasonable in cases in which not much data needs to be written via the Restlet's (for instance for SalesOrders). A queue based system and background jobs with retry capabilities will mitigate the random Netsuite error issues (concurrency errors, timeouts etc).
If you must write to a bunch of Netsuite entities frequently and are using Netsuite as the source of truth for your data rather than attempting to build an entire REST like JSON API on top of Netsuite; I'd recommend implementing a pub/sub service in which Netsuite publishes events to an external service subscribed to by your app/API. Your app could also publish mutations to a channel subscribed to by Netsuite. This way data mutations sent to Netsuite can occur in a middle layer with reduced complexity.
To fetch Netsuite data for outside apps the most efficient means available appears to be the Netsuite ODBC database driver; it provides a direct connection to Netsuite database read only table views. Simple select queries for a set of Items that with same schema in Postgres or MySQL typically take 0.5 ms or less; typically take between 15 seconds to slightly over 100 seconds to return.
Connection timeouts and other errors from Netsuite are still common using NS ODBC driver. Despite slow query results retrieval of all data needed for a set of 5000 items in 14 seconds is far better than the hours it would take to get the same via Netsuite's SOAP API.
Yes. That is in Customization/Scripts section. You will find "RestLet" there. Doc is here.
However you said your application is soap based, I suggest you take a look Netsuite's WebServices aka SuiteTalk.
The SuiteTalk Platform provides programmatic access to your NetSuite data and business processes through an XML-based application programming interface (API).
I think you do need to access to your Netsuite data, right?
You can download their sample for test and learning.
In NetSuite, you can build RESTlet scripts which provide a REST-based interface. You can essentially use them to build your own JSON API. Recommend researching RESTlets in the NetSuite Help.
SOAP is easier to configure and use, but only allows 1 connection per
Netsuite account (you use your login credentials as authentication)
and is relatively slow.
That's not quite true, as you can extend it with suite cloud plus program. Check help for:
- Understanding Web Services Governance
- Enabling Web Services Concurrent Users with SuiteCloud Plus
UPDATE: There are two types of governance in NetSuite since approx July 2016 - user governance (also known as a legacy governance model, implicitly used when sessions / SOAP Login method are utilized) and account governance. In the account governance there is a shared pool for all incoming concurrent requests (no sessions should be used, authentication via user credentials or Token-Based Authentication).
This is the proper REST API provided by NetSuite for integration purposes.
https://system.netsuite.com/help/helpcenter/en_US/APIs/REST_API_Browser/record/v1/2020.1/index.html
The REST API can be invoked either via Token-based authentication or OAuth 2.0 enabled HTTP client.
First you need to login to NetSuite account and enable the SuiteTalk Webservice features of the account (Setup->Company->Enable Features).
Then obtain the SuiteTalk Base URL, which contains the account ID under the company URLs (Setup->Company->Company Information). E.g., https://<ACCOUNT_ID>.suitetalk.api.netsuite.com
After that create an integration application (Setup->Integration->New), enable OAuth 2.0 or TBA. This blog contains the process of enabling features and obtaining tokens.
Then use the BaseUrl + API resource path to as the HTTP client path to invoke each record API. Operations such as CRUD, search and filter can be done via this REST API. For more information See NetSuite Documentation
Yes, Netsuite supports REST web services.
Here's a working Java example, that uses the open source scribe library.
Note that an Accept (and for Posts, a Content-Type) header of application/json is needed for Netsuite to accept the requests, otherwise you'll get a "Request media type is not valid." error. Also getSignatureType method must be implemented for API class (NetSuiteApi.java).
Change all the string constants to suit your setup. Note that this code will also work with Netsuite RESTlets.
REST documentation is available here:
https://[your-netsuite-ID].app.netsuite.com/help/helpcenter/en_US/PDF/REST_Web_Services.pdf
File #1: NetSuiteApi.java
package com.scribe.api;
import com.github.scribejava.core.builder.api.DefaultApi10a;
import com.github.scribejava.core.model.OAuth1RequestToken;
public class NetSuiteApi extends DefaultApi10a {
private static class InstanceHolder {
private static final NetSuiteApi INSTANCE = new NetSuiteApi();
}
public static NetSuiteApi instance() {
return InstanceHolder.INSTANCE;
}
#Override
public String getAccessTokenEndpoint() {
return null;
}
#Override
public String getRequestTokenEndpoint() {
return null;
}
#Override
public String getAuthorizationUrl(OAuth1RequestToken requestToken) {
return null;
}
#Override
protected String getAuthorizationBaseUrl() {
return null;
}
#Override
public OAuth1SignatureType getSignatureType() {
return OAuth1SignatureType.HEADER;
}
}
File #2: NetSuiteApiCallExample.java
package com.scribe.api;
import com.github.scribejava.core.builder.ServiceBuilder;
import com.github.scribejava.core.model.OAuth1AccessToken;
import com.github.scribejava.core.model.OAuthRequest;
import com.github.scribejava.core.model.Response;
import com.github.scribejava.core.model.Verb;
import com.github.scribejava.core.oauth.OAuth10aService;
public final class NetSuiteRestExample {
private String CONSUMER_KEY = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx";
private String CONSUMER_SECRET = "yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy";
private String TOKEN_ID = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa";
private String TOKEN_SECRET = "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb";
private String REST_URL = "https://1234567-sb1.suitetalk.api.netsuite.com/rest/platform/v1/record/inventoryitem/";
private String REALM = "1234567_SB1";
private String POSTBODY = "{\"type\": \"SIMPLE\",\"authorId\": -5}";
public static void main(String[] args) {
final OAuth10aService service = new ServiceBuilder(CONSUMER_KEY).apiSecret(CONSUMER_SECRET))
.build(NetSuiteApi.instance());
OAuth1AccessToken accessToken = new OAuth1AccessToken(TOKEN_ID, TOKEN_SECRET);
// This is POST method call
// OAuthRequest request = new OAuthRequest(Verb.POST, REST_URL);
// request.addHeader("Content-Type", "application/json");
// // Without next line, you'll get a "Request media type is not valid." error, even though this is not needed with Postman
// request.addHeader("Accept", "application/json");
// request.setRealm(REALM);
// request.setPayload(POSTBODY);
// This is GET method call
OAuthRequest request = new OAuthRequest(Verb.GET, params.get("REST_URL"));
// Without next line, you'll get a "Request media type is not valid." error, even though this is not needed with Postman
request.addHeader("Accept", "application/json");
request.setRealm(params.get("REALM"));
service.signRequest(accessToken, request);
System.out.println("Sending this request...");
System.out.println(request.getHeaders());
System.out.println(request.getCompleteUrl());
// System.out.println(request.getPayload());
final Response response = service.execute(request);
System.out.println("Got this response...");
System.out.println(response.getCode() + "\n" + response.getHeaders());
System.out.println(response.getBody());
return response.getBody();
}
}
Add this to you Maven dependencies (pom.xml):
...
<dependencies>
...
<dependency>
<groupId>com.github.scribejava</groupId>
<artifactId>scribejava-apis</artifactId>
<version>6.9.0</version>
</dependency>
</dependencies>

Accessing Request, Response, Service and Db Context, etc. in ServiceStack

In my previous project, I use a framework (Agatha RRSL) similar to ServiceStack, in that everything is made of Request, Response and Handler. It also has Interceptors that can attach to handler and you can inject other interfaces to the handler as well. I can use this to open a transaction BeforeHandling, access to both request and response in AfterHandling, create audit, save to database and close transaction if needed.
I try to experiment similar with SerivceStack. But seems with Filters, I can't grab request and response together?
With custom ServiceRunner. When I try to debug OnAfterExecute(...), I can see the name of my request dto in IRequestContext {ServiceStack.ServiceHost.HttpRequestContext}. But just the name, I couldn't figure out how to retrieve the actual request object to work with the response object.
Another thing I haven't figure out is if it's possible to inject the auto wired service interface into it, like a db context or audit service. Maybe this one is too far ahead in the pipeline?
The final thing is, it seems I can only register one custom service runner? With Interceptor, I can drop a bunch of them, and they will wrap around each other.
Any thoughts? Thanks
The RequestContext also contains the HttpRequest and HttpResponse which you can get access with:
var httpReq = RequestContext.Get<IHttpRequest>();
var httpRes = RequestContext.Get<IHttpResponse>();
See the docs on Accessing HTTP Specific features for more info.

Resources