KrakenJS: perform POST request over a controller ends with error - node.js

I'm using KrakenJS to build a web app. Being it MVC, I'm implenting a REST service by a controller, here's a sample code:
//users can get data
app.get('myRoute', function (req, res) {
readData();
});
//users can send data
app.post('myRoute', function (req, res) {
writeData();
});
I can read data with no problems. But when I try dummy data insertion with POST requests, it ends up with this error:
Error:Forbidden
127.0.0.1 - - [Thu, 06 Feb 2014 00:11:30 GMT] "POST /myRoute HTTP/1.1" 500 374 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/32.0.1700.102 Chrome/32.0.1700.102 Safari/537.36"
How can I overcome this?

One thing is to make sure you're sending the correct CSRF Headers (http://krakenjs.com/#Security). If I remember correctly, by default Kraken expects those headers to be specified.
You can disable CSRF too and see if that fixes your problem. Since Kraken uses the Lusca module for CSRF, you can get information on how to disable/configure from here: https://github.com/paypal/lusca

I used a trick earlier in which you don't have to turn off csrf...
In your "index.dust" ->
<input id="csrfid" type="hidden" name="_csrf" value="{_csrf}">
In your "script.js" ->
var csrf = document.getElementById('csrfid').value;
$http({ method: 'POST',
url: 'http://localhost:8000/myRoute/',
data: { '_csrf': csrf, 'object': myObject }
}).success(function(result) {
//success handler
}).error(function(result) {
//error handler
});
i was using angularjs btw

As Dan said you can turn csrf off, but you may also want to consider using it, for the added security it brings.
Check out the shopping cart example for more info: https://github.com/lmarkus/Kraken_Example_Shopping_Cart

If you do not need csrf:
By placing this in middleware in your config.json and setting the values to false, you are disabling the use of the csrf middlware, and your app will function as expected.
"middleware": {
"appsec": {
"priority": 110,
"module": {
"name": "lusca",
"arguments": [
{
"csrf": false,
"xframe": "SAMEORIGIN",
"p3p": false,
"csp": false
}
]
}
},

Related

Why does http.get take so long in NodeJS?

Using NodeJS and http.get, I am trying to see if a website uses a redirect. I tried a few URLs which all worked great. However, when I ran the code with washingtonpost.com it took over 5 seconds. In my browser the website works just fine. What could be the issue?
console.time("Done. Script executed in");
const http = require("http");
function checkRedirectHttp(input){
return new Promise((resolve) => {
http.get(input, {method: 'HEAD'}, (res) => { resolve([res.headers.location, res.statusCode]) })
.on('error', (e) => { throw {Error: `Cannot reach website ${input}`} });
});
};
checkRedirectHttp("http://www.washingtonpost.com/").then(result => {
console.log(result);
console.timeEnd("Done. Script executed in");
})
Output:
[
'http://www.washingtonpost.com/gdpr-consent/?next_url=https%3a%2f%2fwww.washingtonpost.com%2f',
302
]
Done. Script executed in: 8.101s
I ran your code, enhanced it some and slowly added back the actual headers that are sent from my browser when I go to the same link in the browser. When I changed the request to a "GET" (no longer a "HEAD") and added the following headers from my browser:
"user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.89 Safari/537.36",
"accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
"accept-encoding": "gzip, deflate, br",
"accept-language": "en-US,en;q=0.9",
"cookie": "a very long cookie here"
then the response went from 9 seconds to 71ms.
So, apparently the server doesn't like the HEAD request and doesn't like that a bunch of headers it expects to be there are missing. Probably, it is detecting that this isn't a real browser and it's either analyzing something for 8 seconds or it's just purposely delaying a response to a "fake client".
Also, if you use the http://www.washingtonpost.com URL instead of https://www.washingtonpost.com, it redirects to https every time for me. So, you may as well just start with the https:// form of the URL.

Node Js(Express) - POST req.body is empty (unusual)

I didn't realize how common and tricky this problem is. I have spent many hour reviewing all the previous situations and answers. Needless to say, none apply.
I am making a httpClient POST call from Angular 5 to a nodejs/express url. The application makes many of these calls and all works except this one:
Angular component
this.ezzy.post(this.config.api.createConnectAccount, this.AuthCode, true, true, true)
.subscribe((data) => {
if (data.code === '0') {
angular http call
ngOnInit() {
........
createConnectAccount(url, body, loadingIcon, withCredentials, showErrorToast) {
console.log(`CREATE CONNECT ACCOUNT....${url}...${JSON.stringify(body)}`);
const headers = this.ezzy.preAjax(loadingIcon, withCredentials);
return this.http.post(url, body, { withCredentials, headers })
.map((res) => this.ezzy.postAjax(res, showErrorToast))
.catch((err) => this.ezzy.handleError(err));
}
I can confirm that both the url and the authCode/body are correct and present up tho this point.
router.post (Nodejs)
router.post('/users/createConnectAccount', async(req, res, next) => {
// console.log (`REQ BODY FROM PAYOUT DASH: ${JSON.stringify(req)}`);
console.log(`ENTER CREATE CONNECT ACCOUNT...code......${req.body.code}`);
console.log(`ENTER CREATE CONNECT ACCOUNT..body......${JSON.stringify(req.body)}`);
console.log(`REQ HEADERS: ${JSON.stringify(req.headers)}`);
Here are the differences with other similar calls:
1. The angular component was activated from an external call to its endpoint (localhost:3000/dealer?code='1234'. The code was retrieved succesfully in the component's constructor and assigned to authCode.
2. The angular http call orginated inside the ngOnInit. I am trying to get some info and update the db before rendering the component page.
I am using
app.use(json());
app.use(urlencoded({
extended: true
}));
and a console.log of the req.header before the call is this:
ENTER CREATE CONNECT ACCOUNT...code......undefined
ENTER CREATE CONNECT ACCOUNT..body......{}
REQ HEADERS: {"host":"localhost:3000","connection":"keep-alive","content-length":"35","accept":"application/json,
text/plain, */*","sec-fetch-dest":"empty","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.163 Safari/537.36","content-type":"text/plain","origin":"http://localhost:3000","sec-fetch-site":"same-origin","sec-fetch-mode":"cors","referer":"http://localhost:3000/payout-dashboard?code=ac_H5nP4MUbEbp94K13jkA5h1DRG6f6pgOn&state=2lt8v9le8a5","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9","cookie":"connect.sid=s%3AsWLHYTY02P2EvYZy1FIVQzZLC6M0vR5p.GnU%2BU20RcjPYeG3lAUEDV9q1vmLceBPAfEE488ej5M4; _ga=GA1.1.695338957.1586021131; _gid=GA1.1.1793736642.1586291730; PDToken=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJlbWFpbCI6InNlYWthaG1haWxAZ21haWwuY29tIiwibmlja25hbWUiOiIiLCJjYXRlZ29yeSI6IiIsImlhdCI6MTU4NjgyMDYyMSwiZXhwIjoxNjE4MzU2NjIxfQ.09gx1F_YJPxAs7BiiYToetdJhjd5DsUUkdoo3leFscU; io=yysQe40_plBblVuSAAAA"}
If you notice that the content-type is:
"content-type":"text/plain"
and the accepted is:
"accept":"application/json,
text/plain, */*"
and the code is present:
code=ac_H5nP4MUbEbp94K13jkA5h1DRG6f6pgOn&state=2lt8v9le8a5"
YET...I get empty req.body.
BTW....it works from postman
ENTER CREATE CONNECT ACCOUNT...code......ac_H5ikfuYleQromTeP5LnHGEmfEWaYD3he
ENTER CREATE CONNECT ACCOUNT..body......{"code":"ac_H5ikfuYleQromTeP5LnHGEmfEWaYD3he"}
REQ HEADERS: {"user-agent":"PostmanRuntime/7.24.1",
"accept":"*/*","postman-token":"0d5faea6-4684-408e-9235-c5e14b306918",
"host":"localhost:3000",
"accept-encoding":"gzip,
deflate, br","connection":"keep-alive",
"content-type":"application/x-www-form-urlencoded",
"content-length":"40","cookie":"connect.sid=s%3ASahJY3VqXVjTjXF1X-SlU_9Shexa59Tm.Q0SRM1h%2FxJnoEnjS3u3I3x%2F%2FnLs%2FLzyiHGoJNuo0U7M"}
Sorry to be so long...but I am baffled
The urlencoded express middleware only parses the body when the Content-Type of the request matches the type option. By default, the type option is application/x-www-form-urlencoded. Either set the Content-Type of your request from text/plain to application/x-www-form-urlencoded, or pass {"type": "text/plain"} to urlencoded(...) to overwrite the default behavior.

TOO_MANY_REDIRECTS error when iFraming Kibana dashboard using cookies

I'm attempting to embed password protected Kibana dashboards inside an iFrame to my Node powered Express application, based on information in this thread. Kibana is protected using X-Pack and requires users to login in order to see their visualised data.
This currently requires the user to log in twice, once to login into the application and again to access Kibana dashboards, which is not the goal.
Following information in this thread, I implemented some code that makes a pre-flight POST request to https://elk-stack.my.domain:5601/api/security/v1/login to obtain a cookie 🍪
This client side request...
function preFlightKibanaAuth () {
...
$.ajax({
type: 'POST',
url: '/kibana-auth',
datatype: 'json',
success: function (response) {
if (response && response.authenticated) {
$('iframe#kibana-dashboard').prop('src', 'https://elk-stack.my.domain:5601/s/spacename/app/kibana#/dashboards?_g=()')
}
},
error: function (err) {
console.log(err)
}
})
}
Is routed to this route...
router
.route('/kibana-auth')
.post((req, res, next) => {
...
if (authorised) {
...
authenticateKibana(req)
.then(cookie => {
if (cookie && cookie.name && cookie.value) {
res.set('Set-Cookie', `${cookie.name}=${cookie.value}; Domain=my.domain; Path=/; Secure; HttpOnly`)
res.send({ 'authenticated': true })
} else {
res.send({ 'authenticated': false })
}
})
.catch((err) => {
logger.error(err)
res.send({ 'authenticated': false })
})
}
...
})
Which makes it's way to this function, where the cookie is actually obtained and parsed...
authenticateKibana () {
return new Promise((resolve, reject) => {
...
request({
method: 'POST',
uri: `https://elk-stack.my.domain:5601/api/security/v1/login`,
headers: {
'kibana-version': '6.5.4',
'kibana-xsrf': 'xsrftoken',
},
type: 'JSON',
form: {
password: 'validPassword',
username: 'validUsername'
}
}, function (error, res, body) {
if (!error) {
let cookies = cookieParser.parse(res)
cookies.forEach(function (cookie) {
if (cookie.name.startsWith('kibana')) {
// Got Kibana Cookie
return resolve(cookie)
}
})
}
...
})
})
}
This works great and I can successfully authenticate with Kibana, obtain the cookie and set in the clients browser (see below screenshot).
The issue I'm seeing is when the src of the iFrame is updated in the success callback of the preFlightKibanaAuth() request. I can see the authenticated Kibana dashboard load in briefly (so the cookie is allowing the client to view their authenticated dashboards), however, I then see multiple GET requests to /login?next=%2Fs%2Fspacename%2Fapp%2Fkibana that results in a TOO_MANY_REDIRECTS error.
I've found the below comment in the GitHub issues page, which I think maybe the issue I'm having in some way because I'm seeing this in the logs (see bottom): "message":"Found 2 auth sessions when we were only expecting 1.". I just can't figure it out!
Usually what causes this is having multiple cookies for the same
"domain" and "name", but with different values for "path". If you open
the developer tools in Chrome, then click on the Application tab, then
expand the Cookies section, and click on the domain, do you have
multiple cookies with the name "sid"? If so, you can fix this issue by
clearing all of them.
I changed the cookie name from "sid" to "kibana" but don't have two of them visible in Applications tab, just the one I set following the call to /kibana-auth.
The iFrame then loads in the https://elk-stack.my.domain:5601/s/spacename/app/kibana#/dashboards?_g=() and the issue arises. Clearing my cookies just resulted in fetching and setting another one (if we don't already have one), which is what is required, so this didn't solve the problem.
When I send the Set-Cookie header back to the client, I am setting the Domain to the main domain: my.domain, which ends up as .my.domain. The Kibana instance is on a subdomain: elk-stack.my.domain and if I login to the Kibana front end, I can see that the Domain of the cookie it returns is set to elk-stack.my.domain. I'm not sure that should matter though.
Can anyone please shed any light on this or point me in the direction?
Thanks in advance
Here's a glimpse at the logging info from /var/log/kibana/kibana.stdout when a request is made. There's a bit of junk in there still but you can still see what's happening.
{"type":"log","#timestamp":"2019-02-12T19:47:44Z","tags":["debug","security","basic"],"pid":7857,"message":"Trying to authenticate user request to /api/security/v1/login."}
{"type":"log","#timestamp":"2019-02-12T19:47:44Z","tags":["debug","security","basic"],"pid":7857,"message":"Trying to authenticate via header."}
{"type":"log","#timestamp":"2019-02-12T19:47:44Z","tags":["debug","security","basic"],"pid":7857,"message":"Request has been authenticated via header."}
{"type":"response","#timestamp":"2019-02-12T19:47:44Z","tags":[],"pid":7857,"method":"post","statusCode":204,"req":{"url":"/api/security/v1/login","method":"post","headers":{"kibana-version":"6.5.4","kbn-xsrf":"6.5.4","host":"10.30.10.30:5601","content-type":"application/
x-www-form-urlencoded","content-length":"35","connection":"close"},"remoteAddress":"192.168.56.102","userAgent":"192.168.56.102"},"res":{"statusCode":204,"responseTime":109,"contentLength":9},"message":"POST /api/security/v1/login 204 109ms - 9.0B"}
{"type":"log","#timestamp":"2019-02-12T19:47:44Z","tags":["debug","legacy","proxy"],"pid":7857,"message":"Event is being forwarded: connection"}
{"type":"log","#timestamp":"2019-02-12T19:47:44Z","tags":["trace","legacy","service"],"pid":7857,"message":"Request will be handled by proxy GET:/s/spacename/app/kibana."}
{"type":"log","#timestamp":"2019-02-12T19:47:44Z","tags":["warning","security","auth","session"],"pid":7857,"message":"Found 2 auth sessions when we were only expecting 1."}
{"type":"log","#timestamp":"2019-02-12T19:47:44Z","tags":["debug","security","basic"],"pid":7857,"message":"Trying to authenticate user request to /app/kibana."}
{"type":"log","#timestamp":"2019-02-12T19:47:44Z","tags":["debug","security","basic"],"pid":7857,"message":"Trying to authenticate via header."}
{"type":"log","#timestamp":"2019-02-12T19:47:44Z","tags":["debug","security","basic"],"pid":7857,"message":"Authorization header is not presented."}
{"type":"response","#timestamp":"2019-02-12T19:47:44Z","tags":[],"pid":7857,"method":"get","statusCode":302,"req":{"url":"/app/kibana","method":"get","headers":{"host":"elk-stack.my.domain:5601","connection":"keep-alive","upgrade-insecure-requests":"1","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8","referer":"https://local.local.my.domain/fortigate/reporting/dashboard","accept-encoding":"gzip, deflate, br","accept-language":"en-GB,en;q=0.9,en-US;q=0.8,la;q=0.7,fr;q=0.6"},"remoteAddress":"192.168.56.102","userAgent":"192.168.56.102","referer":"https://local.local.my.domain/fortigate/reporting/dashboard"},"res":{"statusCode":302,"responseTime":3,"contentLength":9},"message":"GET /app/kibana 302 3ms - 9.0B"}
{"type":"log","#timestamp":"2019-02-12T19:47:45Z","tags":["debug","legacy","proxy"],"pid":7857,"message":"Event is being forwarded: connection"}
{"type":"log","#timestamp":"2019-02-12T19:47:45Z","tags":["debug","legacy","proxy"],"pid":7857,"message":"\"getConnections\" has been called."}
{"type":"ops","#timestamp":"2019-02-12T19:47:45Z","tags":[],"pid":7857,"os":{"load":[0.2568359375,0.31640625,0.3173828125],"mem":{"total":33567580160,"free":346796032},"uptime":1585351},"proc":{"uptime":33636.577,"mem":{"rss":322772992,"heapTotal":225566720,"heapUsed":184707176,"external":2052484},"delay":6.417333126068115},"load":{"requests":{"5601":{"total":2,"disconnects":0,"statusCodes":{"204":1,"302":1}}},"concurrents":{"5601":1},"responseTimes":{"5601":{"avg":56,"max":109}},"sockets":{"http":{"total":0},"https":{"total":0}}},"message":"memory: 176.2MB uptime: 9:20:37 load: [0.26 0.32 0.32] delay: 6.417"}
{"type":"log","#timestamp":"2019-02-12T19:47:45Z","tags":["debug","monitoring-ui","kibana-monitoring"],"pid":7857,"message":"Received Kibana Ops event data"}
{"type":"log","#timestamp":"2019-02-12T19:47:45Z","tags":["debug","monitoring-ui","kibana-monitoring"],"pid":7857,"message":"Received Kibana Ops event data"}
{"type":"log","#timestamp":"2019-02-12T19:47:45Z","tags":["trace","legacy","service"],"pid":7857,"message":"Request will be handled by proxy GET:/login?next=%2Fs%2Fspacename%2Fapp%2Fkibana."}
{"type":"response","#timestamp":"2019-02-12T19:47:45Z","tags":[],"pid":7857,"method":"get","statusCode":302,"req":{"url":"/login?next=%2Fs%2Fspacename%2Fapp%2Fkibana","method":"get","headers":{"host":"elk-stack.my.domain:5601","connection":"keep-alive","upgrade-insecure-requests":"1","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8","referer":"https://local.local.my.domain/fortigate/reporting/dashboard","accept-encoding":"gzip, deflate, br","accept-language":"en-GB,en;q=0.9,en-US;q=0.8,la;q=0.7,fr;q=0.6"},"remoteAddress":"192.168.56.102","userAgent":"192.168.56.102","referer":"https://local.local.my.domain/fortigate/reporting/dashboard"},"res":{"statusCode":302,"responseTime":2,"contentLength":9},"message":"GET /login?next=%2Fs%2Fspacename%2Fapp%2Fkibana 302 2ms - 9.0B"}
{"type":"log","#timestamp":"2019-02-12T19:47:45Z","tags":["debug","legacy","proxy"],"pid":7857,"message":"Event is being forwarded: connection"}
The below then repeats over and over...
{"type":"log","#timestamp":"2019-02-12T19:47:45Z","tags":["trace","legacy","service"],"pid":7857,"message":"Request will be handled by proxy GET:/s/spacename/app/kibana."}
{"type":"log","#timestamp":"2019-02-12T19:47:45Z","tags":["warning","security","auth","session"],"pid":7857,"message":"Found 2 auth sessions when we were only expecting 1."}
{"type":"log","#timestamp":"2019-02-12T19:47:45Z","tags":["debug","security","basic"],"pid":7857,"message":"Trying to authenticate user request to /app/kibana."}
{"type":"log","#timestamp":"2019-02-12T19:47:45Z","tags":["debug","security","basic"],"pid":7857,"message":"Trying to authenticate via header."}
{"type":"log","#timestamp":"2019-02-12T19:47:45Z","tags":["debug","security","basic"],"pid":7857,"message":"Authorization header is not presented."}
{"type":"response","#timestamp":"2019-02-12T19:47:45Z","tags":[],"pid":7857,"method":"get","statusCode":302,"req":{"url":"/app/kibana","method":"get","headers":{"host":"elk-stack.my.domain:5601","connection":"keep-alive","upgrade-insecure-requests":"1","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8","referer":"https://local.local.my.domain/fortigate/reporting/dashboard","accept-encoding":"gzip, deflate, br","accept-language":"en-GB,en;q=0.9,en-US;q=0.8,la;q=0.7,fr;q=0.6"},"remoteAddress":"192.168.56.102","userAgent":"192.168.56.102","referer":"https://local.local.my.domain/fortigate/reporting/dashboard"},"res":{"statusCode":302,"responseTime":2,"contentLength":9},"message":"GET /app/kibana 302 2ms - 9.0B"}
{"type":"log","#timestamp":"2019-02-12T19:47:45Z","tags":["debug","legacy","proxy"],"pid":7857,"message":"Event is being forwarded: connection"}
{"type":"log","#timestamp":"2019-02-12T19:47:45Z","tags":["trace","legacy","service"],"pid":7857,"message":"Request will be handled by proxy GET:/login?next=%2Fs%2Fspacename%2Fapp%2Fkibana."}
{"type":"response","#timestamp":"2019-02-12T19:47:45Z","tags":[],"pid":7857,"method":"get","statusCode":302,"req":{"url":"/login?next=%2Fs%2Fspacename%2Fapp%2Fkibana","method":"get","headers":{"host":"elk-stack.my.domain:5601","connection":"keep-alive","upgrade-insecure-requests":"1","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8","referer":"https://local.local.my.domain/fortigate/reporting/dashboard","accept-encoding":"gzip, deflate, br","accept-language":"en-GB,en;q=0.9,en-US;q=0.8,la;q=0.7,fr;q=0.6"},"remoteAddress":"192.168.56.102","userAgent":"192.168.56.102","referer":"https://local.local.my.domain/fortigate/reporting/dashboard"},"res":{"statusCode":302,"responseTime":2,"contentLength":9},"message":"GET /login?next=%2Fs%2Fspacename%2Fapp%2Fkibana 302 2ms - 9.0B"}
{"type":"log","#timestamp":"2019-02-12T19:47:45Z","tags":["debug","legacy","proxy"],"pid":7857,"message":"Event is being forwarded: connection"}
{"type":"log","#timestamp":"2019-02-12T19:47:45Z","tags":["plugin","debug"],"pid":7857,"message":"Checking Elasticsearch version"}
{"type":"log","#timestamp":"2019-02-12T19:47:45Z","tags":["trace","legacy","service"],"pid":7857,"message":"Request will be handled by proxy GET:/s/spacename/app/kibana."}
{"type":"log","#timestamp":"2019-02-12T19:47:45Z","tags":["warning","security","auth","session"],"pid":7857,"message":"Found 2 auth sessions when we were only expecting 1."}
{"type":"log","#timestamp":"2019-02-12T19:47:45Z","tags":["debug","security","basic"],"pid":7857,"message":"Trying to authenticate user request to /app/kibana."}
{"type":"log","#timestamp":"2019-02-12T19:47:45Z","tags":["debug","security","basic"],"pid":7857,"message":"Trying to authenticate via header."}
{"type":"log","#timestamp":"2019-02-12T19:47:45Z","tags":["debug","security","basic"],"pid":7857,"message":"Authorization header is not presented."}
{"type":"response","#timestamp":"2019-02-12T19:47:45Z","tags":[],"pid":7857,"method":"get","statusCode":302,"req":{"url":"/app/kibana","method":"get","headers":{"host":"elk-stack.my.domain:5601","connection":"keep-alive","upgrade-insecure-requests":"1","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8","referer":"https://local.local.my.domain/fortigate/reporting/dashboard","accept-encoding":"gzip, deflate, br","accept-language":"en-GB,en;q=0.9,en-US;q=0.8,la;q=0.7,fr;q=0.6"},"remoteAddress":"192.168.56.102","userAgent":"192.168.56.102","referer":"https://local.local.my.domain/fortigate/reporting/dashboard"},"res":{"statusCode":302,"responseTime":2,"contentLength":9},"message":"GET /app/kibana 302 2ms - 9.0B"}
{"type":"log","#timestamp":"2019-02-12T19:47:45Z","tags":["debug","legacy","proxy"],"pid":7857,"message":"Event is being forwarded: connection"}
{"type":"log","#timestamp":"2019-02-12T19:47:45Z","tags":["trace","legacy","service"],"pid":7857,"message":"Request will be handled by proxy GET:/login?next=%2Fs%2Fspacename%2Fapp%2Fkibana."}
{"type":"response","#timestamp":"2019-02-12T19:47:45Z","tags":[],"pid":7857,"method":"get","statusCode":302,"req":{"url":"/login?next=%2Fs%2Fspacename%2Fapp%2Fkibana","method":"get","headers":{"host":"elk-stack.my.domain:5601","connection":"keep-alive","upgrade-insecure-requests":"1","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8","referer":"https://local.local.my.domain/fortigate/reporting/dashboard","accept-encoding":"gzip, deflate, br","accept-language":"en-GB,en;q=0.9,en-US;q=0.8,la;q=0.7,fr;q=0.6"},"remoteAddress":"192.168.56.102","userAgent":"192.168.56.102","referer":"https://local.local.my.domain/fortigate/reporting/dashboard"},"res":{"statusCode":302,"responseTime":2,"contentLength":9},"message":"GET /login?next=%2Fs%2Fspacename%2Fapp%2Fkibana 302 2ms - 9.0B"}
{"type":"log","#timestamp":"2019-02-12T19:47:45Z","tags":["debug","legacy","proxy"],"pid":7857,"message":"Event is being forwarded: connection"}
{"type":"log","#timestamp":"2019-02-12T19:47:45Z","tags":["plugin","debug"],"pid":7857,"message":"Checking Elasticsearch version"}
Kibana Version: 6.5.4
Elasticsearch: 6.5.4
At first, I thought this all turned out to be a mismatch in the Cookie attributes, alas, it wasn't!
Received some info from the Elastic team...
The cookie which Kibana replies with generally sets the httpOnly flag,
and the secure flag (when hosted over https), in addition to the
domain. If any of the settings differ for the cookie which you're
trying to force Kibana to use, you'll see 2 cookies being submitted
and behaviour similar to what you're seeing.
Thought I was setting the cookie with different attributes, but wasn't... ended up using a plugin to get this off the ground: https://readonlyrest.com/

Is there a programmatic way to change user agent in Cypress.io?

I have some ad calls that are only made on mobile devices. In Chrome, I can use Device Mode and simulate a mobile device, and the resulting ad call from the server is correctly tailored to mobile. I'm not sure how Chrome does this, except possibly by sending a different user agent.
In the Cypress.io documentation, it says the user agent can be changed in the configuration file (Cypress.json). But, I need to run a test for a desktop viewport and then a mobile viewport with a mobile user agent. Is there a way to change the user agent programmatically?
Update: According to https://github.com/cypress-io/cypress/issues/3873 it is possible since Cypress 3.3.0 use user-agent property in a cy.request() and cy.visit().
If you need, for example, set userAgent as Googlebot:
cy.visit(url, {
headers: {
'user-agent': 'Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)',
}
});
Original answer before Cypress 3.3.0
before(() => {
cy.visit(url, {
onBeforeLoad: win => {
Object.defineProperty(win.navigator, 'userAgent', {
value: 'Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)',
});
},
});
});
Now cypress supports passing user agent in the header for cy.visit as well as cy.request:
it('Verify Social Sharing Meta Tags', () => {
cy.visit(portalURL + '/whats_new/140', {
headers: {
'user-agent': 'LinkedInBot/1.0 (compatible; Mozilla/5.0; Apache-HttpClient +http://www.linkedin.com)',
}
});
cy.document().get('head meta[name="og:type"]')
.should('have.attr', 'content', 'website');
});
https://on.cypress.io/changelog#3-3-0
Update as on Aug 12, 2021:
It seems you can't change the user agent anymore, reference https://docs.cypress.io/api/cypress-api/config#Notes
The other answers do not set the User-Agent header of the underlying HTTP request, just the userAgent property of win.navigator. To set the User-Agent header to a custom value for all HTTP requests, you can set the userAgent configuration option:
{
// rest of your cypress.json...
"userAgent": "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
}

Does "request" follow redirects from meta refresh tags?

In my nodejs program, I'm using require(request). It doesn't seem to be following redirects even though it should be by default.
I even explicitly set the redirect flag (even though this should be set by default)
var options = {
url:url
, followRedirect: true
, headers: {
'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.110 Safari/537.36'
}
}
request(options, function (err, res, body) {...
For example, check out the site http://www.fanniemae.com/
which redirects to http://www.fanniemae.com/portal/index.html
Inside the /index.html, the html contains this
<meta http-equiv="REFRESH" content="0;url=/portal/index.html">
request doesn't seem to be following this meta tag redirect.
Is this normal? And how can I make it follow that redirect?
Request doesn't render pages as browsers do, it's just a way to make simple http calls (redirects would work if there were server-side redirects on external services). So that's why it can't understand this kind of redirect.
As a solution you could try to use something like PhantomJS (http://phantomjs.org/) to make it worked with some workaround mentioned here.
Or probably scripts written for Selenium server might help you to solve your problem.
Came across this post while having the same problem. I extracted the refresh URL and made another request to get the page contnt like this:
var regex = /<meta http-equiv="Refresh" CONTENT="1; URL=([^"]+)[^>]+>/;
var match = regex.exec(response.body);
if (match[1] !== undefined) {
request.get({
url: host + match[1],
}, function(error, response, body) {
console.log(error, response, body);
});
} else {
console.log('no meta redirect found :(');
}

Resources