I am now using logstash https filter plugin
my filter part is as below
filter {
http {
url => "someUrl"
verb => "GET"
target_body => "apiResponse"
target_headers => "apiResponseHeader"
}
}
with http filter, I can't get HTTP response status codes
is there anyway to get HTTP status code from http filter?
target_headers give me back only this info
"responseHeader" => {
"transfer-encoding" => "chunked",
"content-type" => "application/json",
"date" => "Thu, 09 Feb 2023 09:07:18 GMT",
"keep-alive" => "timeout=60",
"connection" => "keep-alive"
},
No, you cannot access the status code. When the underlying Manticore client is called, the status code is returned separately from the response headers. The http client receives and tests the code, but does not store it in the event.
Related
In my React app I sending requests to my backend node / express app using axios. In my local environment, everything works well when I call it using a function that looks like this:
await axios.post('/createproduct', createProductBody).then(res => console.log('Data send')).catch(err => console.log(err.data))
However, this after I push my code to production, this line of code returns a status code 405 (see screenshot)
Could it be because I have added the "proxy": "http://localhost:3001" line in my package.jsonfile? I don't quite understand why this would work in locally but not in prod.
Thanks
According to the output of the below, your AWS S3 bucket is not configured to allow POST:
axios = require('axios')
function createProductBody() {
return {}
}
async function stackOverflowQuestionNumber67929785() {
await axios.post('https://editor.blankt.io/createproduct', createProductBody()).then(res => console.log('Data send')).catch(err => console.log(err))
}
stackOverflowQuestionNumber67929785();
Prints:
response: {
status: 405,
statusText: 'Method Not Allowed',
headers: {
'content-type': 'application/xml',
'transfer-encoding': 'chunked',
connection: 'close',
allow: 'HEAD, DELETE, GET, PUT',
date: 'Fri, 11 Jun 2021 00:58:55 GMT',
server: 'AmazonS3',
'x-cache': 'Error from cloudfront',
via: '1.1 cf2a58a1ade01b9796df7d87fe311e64.cloudfront.net (CloudFront)'
}
See the allow response header.
I have the code below which is supposed to send a POST request but with an empty JSON body:
request.post({
url: url,
headers: {
"data_digest": 'muZ7EcapQZVb77RF',
"partner_code": "ebfebc241db9b9c1a",
"msg_type": "GET_BASEDATA_TRANSPORT",
"msg_id": '1590464383047',
"Accept-Language": "zh-cn"
},
json: true,
body: {}
}, function(error, response, body) {
console.log(body);
});
However this keeps returning
'System Exception:\r\ntype org.springframework.web.HttpMediaTypeNotAcceptableException\r\nmessage:Could not find acceptable representation'
But using Postman I'm able to correctly send POST request with the exact same headers but just an empty {} in the Body parameter with Raw format.
How can I send a POST request with an empty JSON body in Node JS?
Your way of sending the body is fine. If you'd look at the actual request sent (for example using Fiddler or Wireshark) you'd see the body is sent correctly. The problem is something else - instead you'd see that the headers are not the exact same.
Using json (either with json: true and body, or as you do it, with json as object) also automatically sets the Accept header to application/json and attempts to parse the response as JSON.
It seems that this server chiguotest.ytoglobal.com has a bug, where it returns JSON but does not handle the Accept header correctly (I tested it and it seems the server "thinks" it is returning text/plain). So request is (correctly) telling the server "I expect you to return JSON", but the server says "What, JSON? No I don't know how to return JSON, only text, sorry.". Yet, it in fact does return JSON.
You can circumvent this server bug by explicitely sending an Accept: */* header:
request.post({
url: url,
headers: {
"data_digest": 'muZ7EcapQZV',
"partner_code": "ebfebc241db9b9c",
"msg_type": "GET_BASEDATA_TRANSPORT",
"msg_id": '1590464383047',
"Accept-Language": "zh-cn",
"Accept": "*/*"
},
json: true,
body: {}
}, function(error, response, body) {
console.log(body);
});
My output:
{
"data": {
"productKinds": [
{
"productCnname": "(美国)测试用-不拉G单号",
"productCode": "1",
"productEnname": "1",
"productServerhawbcodeNeed": "Y"
},
{
"productCnname": "散客可见产品",
"productCode": "111",
"productEnname": "内部产品",
"productServerhawbcodeNeed": "N"
},
... many more entries ...
]
},
"errorCode": "SUCCESS",
"errorMsg": "成功",
"status": true
}
I have a fetch request like
fetch(`http://localhost:4000/test`, {
method: 'GET',
mode: 'cors',
cache: 'no-cache',
headers: {
'Content-Type': 'application/json'
}
})
.then(result => console.log('success'))
.catch(error => console.log('error', error));
And a nodejs express router (express-promise-router) for the request
this.router.get('/test', (request: Request, response: Response) => {
response.status(200).send({'status': 'ok'});
});
When the fetch gets called the success is logged. So that's good.
But when I change the nodejs router function to async like
this.router.get('/test', (request: Request, response: Response) => {
return asyncFunction().then(result => response.status(200).send({'status': 'ok'}));
});
The fetch gets canceled and the console log displays TypeError: Failed to fetch.
When I call the /test api with the browser URL or Postman I get a successful result.
I can't figure out why the fetch is failing. (Also tried it with Axios with the same result).
I was on the wrong track with the fetch issue.
The issue wasn't related to the fetch but to the event in which the fetch is called.
At the end of the method I needed an event.preventDefault() so the event was stopped and the fetch was handled correctly.
I am writing a client program to connect to a website over HTTP/HTTPS. The program first tries to connect to the server using HTTPS. However, after receiving a response status code of 301, I tried handling the request with HTTP whenever there is a 301 by making a new request to the HTTP server. As is commonly done to consume data, I added a listener callback on the 'data' event of the http.get() using the on method of http.clientRequest. However, there is no data in the console output. I am suspecting this is due to the following CSP header that I have been receiving with requests:
Message Headers:
content-type: text/html; charset=UTF-8
location: http://www.whoscored.com/
server: Microsoft-IIS/8.0
strict-transport-security: max-age=16070400
content-security-policy: frame-ancestors *.whoscored.com; upgrade-insecure-requests;
x-content-security-policy: frame-ancestors *.whoscored.com; upgrade-insecure-requests;
date: Sun, 29 Oct 2017 02:44:33 GMT
connection: close
content-length: 148
Logging the data:
The code is provided below:
Https.get(options, (res: Http.IncomingMessage): void => {
logger.log('HTTPS Client for ScrapeX');
logger.log('-------------------------');
logger.logHeaders(res.headers);
switch(res.statusCode) {
case 200:
(() => {
//
logger.log('The connection was established successfully');
})();
break;
case 301:
(() => {
// fallback to http
let buf = '';
httpClient((res1) => {
logger.log('HTTP Client');
logger.log('----------------');
logger.logHeaders(res1.headers);
}, n_options)
.on('error', (err) => {
logger.log('Error: ' + err.message);
logger.printStack(err);
})
.on('data', (chunk: string): void => {
buf += chunk;
})
.on('close', () => {
logger.log('Logging the data: ');
logger.log(buf);
});
})();
break;
}
})
.on('error', (err) => {
logger.log(err.message);
logger.log(err.stack);
})
.on('close', () => {
logger.log('Connection closed');
});
You can't connect to that server with HTTP, that's your problem; it's using both upgrade-insecure-requests and more importantly strict-transport-security. Any browser that respects strict-transport-security will simply refuse to connect over unsecured HTTP.
Not sure what to tell you beyond in this case you simply can't retry in HTTP, or at least, it'll always give either an error or just redirect to HTTPS.
The CSP shouldn't really be an issue, that's for loading other resources while on that page, it doesn't block people from downloading the page in the first place.
I use request to check whether a given URL is broken or not. However, I encountered a strange situation that one given URL keeps redirecting to itself, and the request fails to return any response. But when I open the url with browser, status code 200 is returned.
Anyone knows why request falls into the redirect loop and cannot get the response while the url works find in the browser? How to deal with this problem? Thanks!
request({
uri: 'http://testurl'
}, function (error, response, body) {
......
}
})
The following is the output after setting "request.debug = true"
REQUEST { uri: 'http://testurl',
callback: [Function],
tunnel: false }
REQUEST make request http://testurl
REQUEST onResponse http://testurl 302 { 'x-cnection': 'close',
date: 'Wed, 12 Nov 2014 23:59:22 GMT',
'transfer-encoding': 'chunked',
location: 'http://testurl',
......,
'x-powered-by': 'Servlet/2.5 JSP/2.1' }
REQUEST redirect http://testurl
REQUEST redirect to http://testurl
REQUEST {}
REQUEST make request http://testurl
REQUEST response end http://testurl 302 { 'x-cnection': 'close',
date: 'Wed, 12 Nov 2014 23:59:22 GMT',
'transfer-encoding': 'chunked',
location: 'http://testurl',
......,
'x-powered-by': 'Servlet/2.5 JSP/2.1' }
REQUEST onResponse http://testurl 302 { 'x-cnection': 'close',
......
UPDATE:
After reading request documentation, I realized it may have something to do with cookies. So I add option jar: true to the request, finally it works.
Try using these request options as explained in their documentation:
- followAllRedirects - follow non-GET HTTP 3xx responses as redirects (default: false)
- maxRedirects - the maximum number of redirects to follow (default: 10)
request({
uri: 'http://testurl',
followAllRedirects: true,
maxRedirects: 50 // some arbitrary value greater than 10
}, function (error, response, body) {
......
}
})
Is the example a public url? I would like to take a look if this is the case.