I'm new to using groovy and have started to use it to test some REST services. I'm having an issue parsing my XML response from our service due to 'Content not allowed in prolog.' After awhile searching I came across a post saying there might be a Byte Order Marker at the beginning. To compensate I followed their approach to trim the characters before the first < and then parse the response. While this works, I was also told the issue is that the response is coming back as 'Transfer-Encoding: chunked'.
Using HTTPBuilder, is there a way to handle chunked responses without trimming characters off?
If I try:
def http = new HTTPBuilder('url')
http.request( Method.valueOf("GET"), XML )
I get the 'Content not allowed in prolog message. However:
http.request( Method.valueOf("GET"), TEXT )
Works, but requires trimming the text until the first < before sending the response to XmlParser.
I had the same issue when I needed to interact with an IIS server. The XML returned had a bogus character in front of the actual XML returned by the web server. I worked around it like this:
StringReader reader = builder.get( path: 'rcserver/systeminfo.xml', contentType: ContentType.TEXT )
def text = reader.getText()
def xml = new XmlSlurper().parseText(text.substring(1));
The HTTPBuilder class has a setContentEncoding() method that allows you to specify the response's content-type.
Maybe something like:
http.contentEncoding = ContentEncoding.Type.GZIP
http.request( Method.GET, XML)
Hope this helps.
I was having this problem as well hitting an IIS server over https. Here is a little addition to Wim Deblauwe's answer for a POST request. You have to send a different type in the request than you expect in the response.
Send a POST with XML as the request type and TEXT as the response type. Then, parse the text response into XML. This worked for me.
In Groovy:
def reader = http.request(Method.POST, ContentType.TEXT){
uri.path = "myPath.api"
send ContentType.XML, postBodyXml
}
def text = reader.getText()
def resultxml = new XmlSlurper().parseText(text.substring(1));
Related
I created a NodeJS application which should get some data from an external API-Server. That server provides its data only as 'Content-Type: text/plain;charset=ISO-8859-1'. I have got that information through the Header-Data of the server.
Now the problem for me is that special characters like 'ä', 'ö' or 'ü' are shown as �.
I tried to convert them with Iconv to UTF-8, but then I got these things '�'...
My question is, what am I doing wrong?
For testing I use Postman. These are the steps I do to test everything:
Use Postman to trigger my NodeJS application
The App requests data from the API-Server
API-Server sends Data to NodeJS App
My App prints out the raw response-data of the API, which already has those strange characters �
The App then tries to convert them with Iconv to UTF-8, where it shows me now this '�' characters
Another strange thing:
When I connect Postman directly to the API-Server, the special characters get shown as they have too without problems. Therefore i guess my application causes the problem but I cannot see where or why...
// Javascript Code:
try {
const response = await axios.get(
URL
{
params: params,
headers: headers
}
);
var iconv = new Iconv('ISO-8859-1', 'UTF-8');
var converted = await iconv.convert(response.data);
return converted.toString('UTF-8');
} catch (error) {
throw new Error(error);
}
So after some deeper research I came up with the solution to my problem.
The cause of all trouble seems to lie within the post-process of axios or something similar. It is the step close after data is received and convertet to text and shortly before the response is generated for my nodejs-application.
What I did was to define the "responseType" of the GET-method of axios as an "ArrayBuffer". Therefore an adjustment in axios was necessary like so:
var resArBuffer = await axios.get(
URL,
{
responseType: 'arraybuffer',
params: params,
headers: headers
}
);
Since JavaScript is awesome, the ArrayBuffer provides a toString() method itself to convert the data from ArrayBuffer to String by own definitions:
var response = resArBuffer.data.toString("latin1");
Another thing worth mentioning is the fact that I used "latin1" instead of "ISO-8859-1". Don't ask me why, some sources even recommended to use "cp1252" instead, but "latin1" workend for me here.
Unfortunately that was not enough yet since I needed the text in UTF-8 format. Using "toString('utf-8')" itself was the wrong way too since it would still print the "�"-Symbols. The workaround was simple. I used "Buffer.from(...)" to convert the "latin1" defined text into a "utf-8" text:
var text = Buffer.from(response, 'utf-8').toString();
Now I get the desired UTF-8 converted text I needed. I hope this thread helps anyone else outhere since thse informations hwere spread in many different threads for me.
I am trying to get data from the Bing search API, and since the existing libraries seem to be based on old discontinued APIs I though I'd try myself using the request library, which appears to be the most common library for this.
My code looks like
var SKEY = "myKey...." ,
ServiceRootURL = 'https://api.datamarket.azure.com/Bing/Search/v1/Composite';
function getBingData(query, top, skip, cb) {
var params = {
Sources: "'web'",
Query: "'"+query+"'",
'$format': "JSON",
'$top': top, '$skip': skip
},
req = request.get(ServiceRootURL).auth(SKEY, SKEY, false).qs(params);
request(req, cb)
}
getBingData("bookline.hu", 50, 0, someCallbackWhichParsesTheBody)
Bing returns some JSON and I can work with it sometimes but if the response body contains a large amount of non ASCII characters JSON.parse complains that the string is malformed. I tried switching to an ATOM content type, but there was no difference, the xml was invalid. Inspecting the response body as available in the request() callback actually shows bad code.
So I tried the same request with some python code, and that appears to work fine all the time. For reference:
r = requests.get(
'https://api.datamarket.azure.com/Bing/Search/v1/Composite?Sources=%27web%27&Query=%27sexy%20cosplay%20girls%27&$format=json',
auth=HTTPBasicAuth(SKEY,SKEY))
stuffWithResponse(r.json())
I am unable to reproduce the problem with smaller responses (e.g. limiting the number of results) and unable to identify a single result which causes the issue (by stepping up the offset).
My impression is that the response gets read in chunks, transcoded somehow and reassembled back in a bad way, which means the json/atom data becomes invalid if some multibyte character gets split, which happens on larger responses but not small ones.
Being new to node, I am not sure if there is something I should be doing (setting the encoding somewhere? Bing returns UTF-8, so this doesn't seem needed).
Anyone has any idea of what is going on?
FWIW, I'm on OSX 10.8, node is v0.8.20 installed via macports, request is v2.14.0 installed via npm.
i'm not sure about the request library but the default nodejs one works well for me. It also seems a lot easier to read than your library and does indeed come back in chunks.
http://nodejs.org/api/http.html#http_http_request_options_callback
or for https (like your req) http://nodejs.org/api/https.html#https_https_request_options_callback (the same really though)
For the options a little tip: use url parse
var url = require('url');
var params = '{}'
var dataURL = url.parse(ServiceRootURL);
var post_options = {
hostname: dataURL.hostname,
port: dataURL.port || 80,
path: dataURL.path,
method: 'GET',
headers: {
'Content-Type': 'application/json; charset=utf-8',
'Content-Length': params.length
}
};
obviously params needs to be the data you want to send
I think your request authentication is incorrect. Authentication has to be provided before request.get.
See the documentation for request HTTP authentication. qs is an object that has to be passed to request options just like url and auth.
Also you are using same req for second request. You should know that request.get returns a stream for the GET of url given. Your next request using req will go wrong.
If you only need HTTPBasicAuth, this should also work
//remove req = request.get and subsequent request
request.get('http://some.server.com/', {
'auth': {
'user': 'username',
'pass': 'password',
'sendImmediately': false
}
},function (error, response, body) {
});
The callback argument gets 3 arguments. The first is an error when applicable (usually from the http.Client option not the http.ClientRequest object). The second is an http.ClientResponse object. The third is the response body String or Buffer.
The second object is the response stream. To use it you must use events 'data', 'end', 'error' and 'close'.
Be sure to use the arguments correctly.
You have to pass the option {json:true} to enable json parsing of the response
I need to implement a HTTP server in node.js without using http module. How fun!
I'm having trouble with sending the response socket.
I'm trying to fetch a file and so my code looks as follows:
fileStream = fs.createReadStream('example.jpg');
fileStream.on("end", function (close) {
var str = "HTTP/1.1 200 OK\r\Content-Type: image/jpeg\r\n" //and some more headers.
socket.write(str);
socket.pipe(fileStream);
socket.end("\r\n");
});
What am I missing?
I am of course using net module to get the socket and fs as well.
There are two main issues with the code you have.
Responses as formatted as headers followed by two sets of "\r\n", and then the content. The "\r\n" in your end() call should be in str. You also are missing an 'n' from the first "\r\n".
You are trying to pipe the contents of the readstream, which is great, but you are doing all of this inside of the steam's end(), so the pipe as nothing left to send because the data was all already emitted.
Try something like this instead. Create the read stream, send the response and pipe then rest.
var str = "HTTP/1.1 200 OK\r\nContent-Type: image/jpeg\r\n\r\n";
socket.write(str);
var fileStream = fs.createReadStream('example.jpg');
fileStream.pipe(socket);
have created REST service using servicestack and in post request I have return object in following way
return new HttpResult(request)
{
StatusCode = HttpStatusCode.Created,
};
request: object which i have posted in database
When i check it in fiddler it render whole HTML Page of servicestack in response body, instead of that i would like to return Status code only, so please tell me how can i do?
Thanks
There was a bug in versions before < v3.05 that did not respect the HttpResult ContentType in some scenarios, it should be fixed now with the latest version of ServiceStack on NuGet or available from:
https://github.com/ServiceStack/ServiceStack/downloads
Prior to this you can still force the desired ContentType by changing the Accept:application/json Request Header on HttpClient or by appending ?format=json on the querystring of your url.
So now if you don't want to have any DTO serialized, you don't add it to the HttpResult:
return new HttpResult() { StatusCode = HttpStatusCode.Created };
Note you still might get an empty Html response back if calling this service in the browser (or any Rest Client that Accepts:text/html). You can force a ContentType that won't output any response if it has empty payload (e.g JSON/JSV) by specifying it in the result as well, e.g;
return new HttpResult() {
StatusCode = HttpStatusCode.Created,
ContentType = ContentType.Json
};
I am testing an API call to the server using Cucumber + Capybara with Selenium WebDriver.
I managed to get the response obj, but how do you assert that the response body contains
certain String? For example if the response body contains "Hello World" I want to assert that
this response body (Which is string) contains a pattern "World"
ex. Something like:
response = http.request(request)
response.body.should
have_text("World")
Alternatively is there a way to get "application/json" from the response and assert the contents using
Capybara?
Thanks!
I think what your looking for is have_content.
response.body.should have_content("World")
For more info, check out the README: Check out the documentation: https://github.com/jnicklas/capybara