From this example I've use this code:
chrome.webRequest.onHeadersReceived.addListener(
function (details) {
details.responseHeaders.forEach(function(responseHeader){
console.log(responseHeader.name + "===" + responseHeader.value);
if (responseHeader.name.toLowerCase() === "set-cookie") {
responseHeader.value = processSetCookieStr(responseHeader.value);
}
});
return {
responseHeaders: details.responseHeaders
};
}, {
urls: ["*://*/*"]
}, ['blocking','responseHeaders']
);
But I'm not able to see any cookie in the headers.
For main_frame type I've printed the values but cookie isn't there:
Cache-Control===private
Content-Type===text/html; charset=utf-8
Server===Microsoft-IIS/7.5
X-AspNet-Version===4.0.30319
X-Powered-By===ASP.NET
Date===Sat, 29 Apr 2017 08:51:45 GMT
Content-Length===29880
Though I'm able to fetch the desired cookie using chrome.cookies.get
Why am I unable to access cookie info in the onHeadersReceived?
You should add "extraHeaders" to the third parameter of the webRequest listener and it should be ['blocking','responseHeaders', 'extraHeaders'] for your example.
Starting from Chrome 72, the Set-Cookie response header is not provided and cannot be modified or removed without specifying 'extraHeaders' in opt_extraInfoSpec.
Refer https://developer.chrome.com/docs/extensions/reference/webRequest/
Related
I will start off by saying this has nothing to do with authentication of JWT tokens.
I am trying to pull data from a public api. Lets call it www.abc.com/WeatherAPI.
When I do this in node with axios using axios.get(url), I am able to console.log(res.headers) and they show perfectly.( I need them for pagination and a recursive function)
When I use the EXACT SAME code in react, I get empty headers returned....
eg: content-type: "application/json"
date: "Sun, 08 Mar 2020 09:23:03 GMT"
Code:
return axios
.get(
'https://api.xxxxxxxx' +
(cursor ? '&cursor=' + cursor : '')
)
.then(res => {
console.log(res);
console.log(res.headers['cursor']);
// If there is no data, return...
if (res.data.length < 1) return;
// Push current data into OB state
setOB({ ...oB, ...res.data });
//te
//If there is no cursor... ie there is no more data, return
if (!res.headers['cursor']) return;
return OB(res.headers['cursor']);
});
};
// I dont know if use effect is right here... we will see..
useEffect(() => {
OB();
}, []);`
as the API is public, it could be that the response header differs based on the agent. it is not likely to be the case but it can be. I would suggest overriding the headers object including the User-Agent
axios.get(
`https://api.xxxxxxxx${cursor ? '&cursor=' + cursor : ''}`,
{ headers: { 'User-Agent': 'YOUR-SERVICE-NAME' } },
).then()...
I have a dead simple http nodejs server:
require('http').createServer(function(req, res) {
res.statusCode = 200;
res.setHeader('Set-Cookie', 'age=44; name=ok; something=else; path=/');
res.end('ok ok ok', 'utf8');
}).listen(9999);
Upon visiting the page in the latest version of Chrome, these are the Response Headers sent from the server:
So, the server sends the correct cookies. However, the browser only seems to store the first one (age=44). If I refresh the page again, these are the Request Headers being sent to the server:
Only the first cookie (age=44) is sent. Logging document.cookie in the console also returns just 'age=44'.
When inspecting the cookies from the Chrome's UI, I can also see that only the first one is saved:
What's the problem here?
If you're just using vanilla Node then you can simply pass an array as the second parameter to res.setHeader(field, value):
res.setHeader('Set-Cookie', [
'age=44; path=/',
'name=ok; path=/',
'something=else; path=/'
])
If using Express 4.11.0+ you can use res.append(field [, value]):
res.append('Set-Cookie', 'age=44; path=/');
res.append('Set-Cookie', 'name=ok; path=/');
res.append('Set-Cookie', 'something=else; path=/');
When I make requests to a server with Postman(an api service), chrome automatically makes a cookie. However, when I make a request with my nodejs server, the cookie is not being made even thought the request is successful.
//Headers
var options = {
method: 'GET'
};
options.headers = {};
options.headers.Authorization = auth;
options.url = urlm;
console.log(options);
request(options, function(error,response,body) {
res.status(200).send(response.headers);
});
The response header is
{"date":"Tue, 23 Feb 2016 20:06:57 GMT","server":"Jetty(9.2.1.v20140609)","x-csrf-header":"X-CSRF-TOKEN","expires":"Thu, 01 Jan 1970 00:00:00 GMT","x-csrf-token":"xxxxxxxxxxx","cache-control":"no-store","content-type":"audio/mpeg","set-cookie":["JSESSIONID=uiqwnksadbohqjkq675d;Path=/;HttpOnly"],"connection":"close","transfer-encoding":"chunked"}
Pass { jar: true } in your request options.
From the documentation:
jar - If true, remember cookies for future use (or define your custom cookie jar; see examples section)
I have built a API using node.js and express.
But i need to be able to proxy some requests on a specific route to a external server and show the response from the external server to the clint doing the request.
But i also need to forward the basic auth that the client is send along with the request.
I have tried using the request module like:
app.get('/c/users/', function(req,res) {
//modify the url in any way you want
var newurl = 'https://www.external.com'
request(newurl).pipe(res),
})
But it seems to not send the basic auth header because i get "403 Forbidden" back form the external server(www.external.com)
The request im making is looking like:
GET http://example.se:4000/c/users/ HTTP/1.1
Accept-Encoding: gzip,deflate
X-version: 1
Authorization: Basic bmR4ZHpzNWZweWFpdjdxfG1vcmV1c2*******=
Accept: application/json
Host: example.se:4000
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.1.1 (java 1.5)
And it works if i do the exact same request but against www.external.com directly so there is some issue when doing the proxy in node.
The request module is totally unaware of anything that you don't pass explicitly to it. To set the request headers and copy the response headers as well do the following:
// copy headers, except host header
var headers = {}
for (var key in req.headers) {
if (req.headers.hasOwnProperty(key)) {
headers[key] = req.get(key)
}
}
headers['host'] = 'final-host'
var newurl = 'http://final-host/...'
request.get({url:newurl, headers: headers }, function (error, response, body) {
// debug response headers
console.log(response.headers)
// debug response body
console.log(body)
// copy response headers
for (var key in response.headers) {
if (response.headers.hasOwnProperty(key)) {
res.setHeader(key, response.headers[key])
}
}
res.send(response.statusCode, body)
})
Try explicitly passing in the auth details like so
request(newurl).auth(username, password).pipe(res);
https://github.com/mikeal/request#http-authentication
I am using nodejs and the REST API to interact with bigquery. I am using the google-oauth-jwt module for JWT signing.
I granted a service account write permission. So far I can list projects, list datasets, create a table and delete a table. But when it comes to upload a file via multipart POST, I ran into two problems:
gzipped json file doesn't work, I get an error saying "end boundary missing"
when I use uncompressed json file, I get a 401 unauthorized error
I don't think this is related to my machine's time being out of sync since other REST api calls worked as expected.
var url = 'https://www.googleapis.com/upload/bigquery/v2/projects/' + projectId + '/jobs';
var request = googleOauthJWT.requestWithJWT();
var jobResource = {
jobReference: {
projectId: projectId,
jobId: jobId
},
configuration: {
load: {
sourceFormat: 'NEWLINE_DELIMITED_JSON',
destinationTable: {
projectId: projectId,
datasetId: datasetId,
tableId: tableId
},
createDisposition: '',
writeDisposition: ''
}
}
};
request(
{
url: url,
method: 'POST',
jwt: jwtParams,
headers: {
'Content-Type': 'multipart/related'
},
qs: {
uploadType: 'multipart'
},
multipart: [
{
'Content-Type':'application/json; charset=UTF-8',
body: JSON.stringify(jobResource)
},
{
'Content-Type':'application/octet-stream',
body: fileBuffer.toString()
}
]
},
function(err, response, body) {
console.log(JSON.parse(body).selfLink);
}
);
Can anyone shine some light on this?
P.S. the documentation on bigquery REST api is not up to date on many things, wish the google guys can keep it updated
Update 1:
Here is the full HTTP request:
POST /upload/bigquery/v2/projects/239525534299/jobs?uploadType=multipart HTTP/1.1
content-type: multipart/related; boundary=71e00bd1-1c17-4892-8784-2facc6998699
authorization: Bearer ya29.AHES6ZRYyfSUpQz7xt-xwEgUfelmCvwi0RL3ztHDwC4vnBI
host: www.googleapis.com
content-length: 876
Connection: keep-alive
--71e00bd1-1c17-4892-8784-2facc6998699
Content-Type: application/json
{"jobReference":{"projectId":"239525534299","jobId":"test-upload-2013-08-07_2300"},"configuration":{"load":{"sourceFormat":"NEWLINE_DELIMITED_JSON","destinationTable":{"projectId":"239525534299","datasetId":"performance","tableId":"test_table"},"createDisposition":"CREATE_NEVER","writeDisposition":"WRITE_APPEND"}}}
--71e00bd1-1c17-4892-8784-2facc6998699
Content-Type: application/octet-stream
{"practiceId":2,"fanCount":5,"mvp":"Hello"}
{"practiceId":3,"fanCount":33,"mvp":"Hello"}
{"practiceId":4,"fanCount":71,"mvp":"Hello"}
{"practiceId":5,"fanCount":93,"mvp":"Hello"}
{"practiceId":6,"fanCount":92,"mvp":"Hello"}
{"practiceId":7,"fanCount":74,"mvp":"Hello"}
{"practiceId":8,"fanCount":100,"mvp":"Hello"}
{"practiceId":9,"fanCount":27,"mvp":"Hello"}
--71e00bd1-1c17-4892-8784-2facc6998699--
You are most likely sending duplicate content-type headers to the Google API.
I don't have the capability to effortlessly make a request to Google BigQuery to test, but I'd start with removing the headers property of your options object to request().
Remove this:
headers: {
'Content-Type': 'multipart/related'
},
The Node.js request module automatically detects that you have passed in a multipart array, and it adds the appropriate content-type header. If you provide your own content-type header, you most likely end up with a "duplicate" one, which does not contain the multipart boundary.
If you modify your code slightly to print out the actual headers sent:
var req = request({...}, function(..) {...});
console.log(req.headers);
You should see something like this for your original code above (I'm using the Node REPL):
> req.headers
{ 'Content-Type': 'multipart/related',
'content-type': 'multipart/related; boundary=af5ed508-5655-48e4-b43c-ae5be91b5ae9',
'content-length': 271 }
And the following if you remove the explicit headers option:
> req.headers
{ 'content-type': 'multipart/related; boundary=49d2371f-1baf-4526-b140-0d4d3f80bb75',
'content-length': 271 }
Some servers don't deal well with multiple headers having the same name. Hopefully this solves the end boundary missing error from the API!
I figured this out myself. This is one of those silly mistakes that would have you stuck for the whole day and at the end when you found the solution you would really knock on your own head.
I got the 401 by typing the selfLink URL in the browser. Of course it's not authorized.