completely failed to read post data in vertx route handler chain - tried all ways no success - groovy

i'm running a gradle build with vertx web. my library dependecies include
// for mock API serving https://mvnrepository.com/artifact/io.vertx/vertx-core
compile group: 'io.vertx', name: 'vertx-core', version: '3.5.3'
compile group: 'io.vertx', name: 'vertx-web', version: '3.5.3'
so in my groovy script code i do this
Vertx vertx = Vertx.vertx()
HttpServer server = vertx.createHttpServer()
...
I then declare a route to catch all requests on resource and process the request and form a response - trying to keep it simple - just return simple string as response
Router allRouter = Router.router(vertx)
allRouter.route ( "/api/now/table/incident")
.handler(BodyHandler.create())
.blockingHandler { routingContext ->
def request = routingContext.request()
HttpMethod method = request.method()
def response = routingContext.response()
response.putHeader ("content-type", "text/plain")
def uri = routingContext.request().absoluteURI()
switch (method) {
case HttpMethod.GET:
println "processing a resource GET on uri : $uri "
response.end ("(GET) howdi will")
break
case HttpMethod.POST:
String bodyEnc = routingContext.getBodyAsJson().encodePrettily()
println "processing a resource POST on uri : $uri"
println "post request received post data : " + bodyEnc
response.end ("(POST) howdi will")
break
}
}
I create a handler for BodyHandling, before the general handler in the route.
Then I start the server with the route
server.requestHandler(allRouter.&accept)
server.listen(8081, "localhost")
works all fine for a get request from postman.
when i use a post request with a request body data - the service hangs and i have to cancel - it never gets to the switch statement. All that happens in the console when i issue the post is
23:30:16.455 [vert.x-eventloop-thread-1] DEBUG io.netty.util.Recycler - -Dio.netty.recycler.maxCapacityPerThread: 32768
23:30:16.455 [vert.x-eventloop-thread-1] DEBUG io.netty.util.Recycler - -Dio.netty.recycler.maxSharedCapacityFactor: 2
23:30:16.455 [vert.x-eventloop-thread-1] DEBUG io.netty.util.Recycler - -Dio.netty.recycler.linkCapacity: 16
23:30:16.455 [vert.x-eventloop-thread-1] DEBUG io.netty.util.Recycler - -Dio.netty.recycler.ratio: 8
23:30:16.478 [vert.x-eventloop-thread-1] DEBUG io.netty.buffer.AbstractByteBuf - -Dio.netty.buffer.bytebuf.checkAccessible: true
23:30:16.482 [vert.x-eventloop-thread-1] DEBUG io.netty.util.ResourceLeakDetectorFactory - Loaded default ResourceLeakDetector: io.netty.util.ResourceLeakDetector#51d486
This is the nearest to being a related topic enter link description here
I have tried this countless ways now, set bodyHandler inside the request when i get it etc - can't get it to work.
my postman post looks like this - where the headers are set to Content-Type application/json, Content-Length is 296 bytes (length of bytes in utf16), and Accept is text/plain to receive simple response
The documentation is just not clear. Blown 12 hours trying to crack this.
Does any one know exactly how one should get the post data on a request when using vertx web please

It appears as though my problem was the size of bytes being sent. I had calculated the bytes of the reqBody string as 296 bytes (using UTF16 charset), and set this into the Postman request.
So the vertx server was trying to read that many when processing the body and hanged. When I just did reqBody.getBytes().size - effectively UTF8, this returned 147 bytes. When I changed the content-length to this in postman, it started to work.
There is a timing issue in where you setup the BodyHandler. I tried this is tin the switch case, and it fails with request has already been read.
However neither of these seem to work successfully.
option 1 : ' create body handler before the route (path) call'
allRouter.route().handler(BodyHandler.create())
//now try and setup the route for API path interception
allRouter.route ( "/api/now/table/incident")
.blockingHandler { routingContext -> ...
option 2 : 'chain the handlers but put the Body Handler first'
//now try and setup the route for API path interception, followed by body handler
allRouter.route ( "/api/now/table/incident")
.handler(BodyHandler.create())
.blockingHandler { routingContext ->...
This was setting the content-length, assuming UTF16 for internal byte size that was the real problem. Now back to trying to complete what I set out to do.

Related

Spring Integration HTTP Outbound Gateway header not forwarder on a consecutive request

I'm struggling with the following flow:
.enrichHeaders(h -> h.headerFunction("ocp-apim-subscription-key", m ->
"xxx"))
.handle(Http.outboundGateway("https://northeurope.api.cognitive.microsoft.com/vision/v3" +
".0/read/analyzeResults/abc")
.mappedRequestHeaders("ocp-apim-subscription-key")
.httpMethod(HttpMethod.GET))
.enrichHeaders(h -> h.headerFunction("ocp-apim-subscription-key", m ->
"xxx"))
.handle(Http.outboundGateway("https://northeurope.api.cognitive.microsoft.com/vision/v3" +
".0/read/analyzeResults/def")
.mappedRequestHeaders("ocp-apim-subscription-key")
.httpMethod(HttpMethod.GET))
The first request is submitted correctly and I get the result, for the second one I get 401 UNAUTHORIZED which means, the ocp-apim-subscription-key is not included. I've tried without the second enrichment step as I thought that the headers won't be cleared but it also didn't change anything.
Any idea what I might be doing wrong? Do I need to configure the header mapper somehow differently?
Here is the output of the debug which clearly shows that the header is included:
17:45:31.468 [main] DEBUG org.springframework.integration.http.outbound.HttpRequestExecutingMessageHandler - bean 'ocrDocument.http:outbound-gateway#2' for component 'ocrDocument.org.springframework.integration.config.ConsumerEndpointFactoryBean#3'; defined in: 'processing.OCRIntegrationFlow'; from source: 'bean method ocrDocument' received message: GenericMessage [payload=<200,[Transfer-Encoding:"chunked", Content-Type:"application/json; charset=utf-8", x-envoy-upstream-service-time:"25", CSP-Billing-Usage:"CognitiveServices.ComputerVision.Transaction=1", Strict-Transport-Security:"max-age=31536000; includeSubDomains; preload", x-content-type-options:"nosniff", Date:"Mon, 31 Aug 2020 15:45:31 GMT"]>, headers={Transfer-Encoding=chunked, ocp-apim-subscription-key=xxx, id=11fa4a77-d97a-772b-69b6-059de29ef808, contentType=application/json;charset=utf-8, http_statusCode=200 OK, Date=1598888731000, timestamp=1598888731467}]
UPDATE
I've recorded a session with wireshark (switched to http instead of https as I couldn't get it to work). It seems that in the second request the subscription-key isn't propagated. For some reason in the second one more headers are included.
First one
Second one
OK. I see where is the problem:
private HttpEntity<?> createHttpEntityFromPayload(Message<?> message, HttpMethod httpMethod) {
Object payload = message.getPayload();
if (payload instanceof HttpEntity<?>) {
// payload is already an HttpEntity, just return it as-is
return (HttpEntity<?>) payload;
}
HttpHeaders httpHeaders = mapHeaders(message);
Since you propagate a ResponseEntity from the first call to the second one, there is indeed no any headers mapping since we just don't do that logic in the AbstractHttpRequestExecutingMessageHandler and use the provided HttpEntity as is.
We can't make an assumption what you would like to do with that, but since you have provided the whole entity, we just don't mutate it and perform request against it as is.
To fix the problem I suggest to include before second call some simple .transform((p) -> "") to avoid some HTTP entity assumption.
And yes, you don't need the second header enricher if the value for the ocp-apim-subscription-key is the same.
We probably need to improve docs on the matter and explain how request message is handled in this component. Feel free to raise a GH issue!

Request processed on POST but fails on PATCH

I am using this code to upload an image (direct streaming into the database):
request.multipart.file('file', {}, async function(file) {
const fileContent = await getStream.buffer(file.stream)
photo.filecontents = fileContent
photo.type = `${file.type}/${file.subtype}`
})
await request.multipart.process()
This works perfectly fine when on a POST HTTP request (MyController.store() method).
However, on a PATCH HTTP request (MyController.update() ), the request always fails to process (I get a 500 HTTP code on the client side)
Both POST and PATH concern the same route which I added in processManually.
I spent one hour debugging this, the PATCH request reaches the server, everything works except when I un-comment await request.multipart.process().
If I leave only this in the update() method of my controller, the request fails to process too:
request.multipart.field((name, value) => {
product.name= value
})
await request.multipart.process()
Did I set the correct headers on my PATCH request? Yes, the same as I did for POST.
The request also fails to process on PATCH but runs smoothly on POST
Any hints as why this is not working?

Pact-js: POST body is garbled by VerifyProvider

I'm running into an odd issue with Pact-js and POST bodies.
The background:
Consumer Side
- I have a NodeJs app which I'm trying to test
- I configured Pact and set up the appropriate framework
- All test run successfully and generate contract
Provider Side:
- Again, I have a NodeJs app which I'm trying to test
- Pact has been set up and framework in place
- When i run the test, all GET requests run successfully, however all POSTs report a fail.
The Issue:
- When I echo out the POST body being passed to the service from Pact (veryifyProvider), i can see that its wrapped the body (JSON) inside another 'Key: value' pairing, where the body i want to parse is the Key and the value is empty. Its also added escape chars ( \ ) to all the double quotes inside the Body.
EX:
{"{\"Account\":\"123\",\"Orbit\":\"5.00\",\"Date\":\"2016-06-22\",\"Client\":\"1234\",\"Spring\":\"1234\"}":""}
When i look in my Pact contract json, everything looks correct. Somewhere between VerifyProvider reading in the JSON and passing it to the REST endpoint, the body is mangled. This only seam to happen during tests, normal invocation works appropriately.
Here is the code I'm using Provider side to execute the Verify:
const verifier = require('pact').Verifier;
const path = require('path');
let contract = path.join(__dirname, 'pactContract.json');
let opts = {
providerBaseUrl: "http://localhost:3001",
pactUrls: [contract],
};
verifier.verifyProvider(opts)
.then((res) => {
console.log('pact veryify complete, !!!');
console.log(res);
process.exit(0);
}).catch((error) => {
console.log(error);
process.exit(1);
});
I'm unable to pull down my exact consumer codebase, but its nearly identical in structure shown here Pact-js.
Any help is VERY much appreciated!
Please check the Content-Type header and ensure it is application/json. This can happen if the service thinks it's matching text messages.

Video seeking in Google Chrome: How to handle continuous Partial Content requests?

This question may be a little overwhelming, but I feel close to understanding the way video seeking works in Google Chrome, but it's still very confusing to me and support is difficult to find.
If I am not mistaken, Chrome initially sends a request header with Range bytes=0- to test if the server understands Partial Content requests, and expects the server to respond with status code 206.
I have read the following answers to get a better understanding:
Need more rep to link them, their topics are:
can't seek html5 video or audio in chrome
HTML5 video will not loop
HTTP Range header
My server is powered by Node.js and I am having trouble with getting continuous range requests out of chrome during playback. When a video is requested, the server receives a bytes=0-, the server then responds with status code 206 and
then the media player breaks.
My confusion is with with the response header, because I am not sure how to
construct my response header and handle eventual range requests:
Do I respond with a status code 200 or 206 initially?
When I respond with 206 I only receive bytes=0-, but when I respond with
200 I receive bytes=0- and after that bytes=355856504-.
If I were to subtract 355856504 of the total Content-Length of the video file, the result is 58, and bytes=0-58 seems like a valid Content-Range?
But after those two requests, I receive no more range requests from Chrome.
I am also unsure if the Content-Range in the response header should looks like "bytes=0-58" or like "bytes=0-58/355856562" for example.
Here is the code
if(req.headers.range) console.info(req.headers.range); // prints bytes=0-
const type = rc.sync(media, 0, 32); // determines mime type
const size = fs.statSync(media)["size"]; // determines content length
// String range, initially "bytes=0-" according to Chrome
var Strange = req.headers.range;
res.set({
"Accept-Ranges": "bytes",
"Content-Type": ft(type).mime,
"Content-Length": size-Strange.replace(/bytes=/, "").split("-")[0],
"Content-Range": Strange+size+"/"+size
});
//res.status(206); // one request from chrome, then breaks
res.status(200); // two requests from chrome, then breaks
// this prints 35585604-58, whereas i expect something like 0-58
console.log("should serve range: "+
parseInt(Strange.replace(/bytes=/, "").split("-")[0]) +"-"+
parseInt(size-Strange.replace(/bytes=/, "").split("-")[0])
);
// this function reads some bytes from 'media', and then streams it:
fs.createReadStream(media, {
start: 0,
end: parseInt(size-Strange.replace(/bytes=/, "").split("-")[0]) // 58
}).pipe(res);
Screenshots of the request and response headers when status code is 200:
first response and request headers
second response and request headers
Screenshot of the request and response header when status code is 206:
Need more rep, to show another screenshot
Essentially the request is:
"Range: bytes=0-"
and the Content-Range response is:
"bytes=0-355856562/355856562"
One apparent error is that you are returning an invalid value in the Range header. See the sepcification - it should be 0-355856561/355856562 since the second value after the dash is the last byte position not length.

Get headers from a request using Nodejs

I'd like to get the headers form a request (ex: status code, content-lenght, content-type...). My code :
options = {
method:'HEAD'
host:"123.30.xxx.xxx"
port:80
}
http.request(options,(res)->
res.send JSON.stringify(res.headers)
)
but this is not working
Please help me :(
Your JSON is not valid and it appears that you are not instantiating options as a variable prior to it's use. ev0lutions code resolves these issues as well as ending the request.
For info on how to create valid JSON check out this tutorial:
http://www.w3schools.com/json/
You need to call .end() on your http.request() object in order to make your request - see the docs:
With http.request() one must always call req.end() to signify that you're done with the request - even if there is no data being written to the request body.
For example:
var options = {
method:"HEAD",
host:"google.com",
port:80
};
var req = http.request(options,function(res) {
console.log(JSON.stringify(res.headers));
});
req.end();
Another issue in your code is that res doesn't have a .send() method - if you're referring to another res variable (for example, containing the code that you have posted) then your variables will be conflicting. If not, you should double check what you're trying to do here.

Resources