Is it possible to use the request module to look at the entire redirect chain, like how puppeteer does it?
I'd like to be able to see each of the status codes / urls / how many redirects happen when I go a site
for example, if i request 'http://apple.com'
the url is set up to redirect to
https://www.apple.com (in this case, the chain is 1)
I'd like to know that (1) that redirect happened and (2) how many redirects it took to get to that
If this isn't possible with request, are there any other libraries? (I'm not using puppeteer anymore because puppeteer doesn't work well with testing attachments)
Figured it out, yes, it is completely possible.
const request = require('request')
request.get({
uri: 'http://apple.com',
followAllRedirects: true
}, function (err, res, body) {
console.log(res.request._redirect.redirectsFollowed)
console.log(res.request._redirect.redirects) // this gives the full chain of redirects
});
Not only is it possible, it can be even easier to use:
Redirect Object: https://github.com/request/request/blob/master/lib/redirect.js
request.get (
{
uri: `http://somesite.com/somepage`,
followAllRedirects: true
},
(err, res, body) => {
if (err) {
// there's an error
}
if (!res) {
// there isn't a response
}
if (res) {
const status = res.statusCode; // 404 , 200, 301, etc
const chain = res.request._redirect.redirects; // each redirect has some info too, see the redirect link above
const contentType = res.headers["content-type"] // yep, you can do this too
}
}
)
Related
I am rather new with express together with the request-promise module,
and need to create a service S
that is called from serverA
and after S has asked ServerB for some additional info,
it redirects the request of serverA to ServerC.
Since I get a
Error: Can't set headers after they are sent.
even though I do not add something by myself, I wonder someone could help me to get this workflow straight.
This is the code:
`
const express = require('express')
const rp = require('request-promise')
...
app.get('/dispatch', cors(), (req, res, next) => {
var options = {
uri: 'https://ServerB/calc-something..',
headers: {
'User-Agent': 'its-me',
'Data': data_from_serverA
},
resolveWithFullResponse: true, // Get statuscode
json: true // Parse the JSON string in the response
};
rp(options) // Do request to serverB
.then(function (response) {
console.log(`ServerB responded with statuscode ${response.statusCode}`)
// No error, so redirect original res
res.redirect('https://serverC/...') // error occurs here
return next(response)
})
.catch(function (err) {
console.log(`ServerB responded with error ${err}`)
return next(err) // send 500 to serverA
})
})
`
Your cors() middleware is setting CORS headers. This is causing the headers to be sent while your promise is resolving.
A redirect ALSO sends headers, and this is the issue. A redirect sets a location header, but you've already sent the headers so that won't work.
The solution is to split your final middleware into two. First, check to see if a redirect is needed and if so, do that. Otherwise, set whatever data you need on the req object and handle this AFTER the cors call.
Your final route will look something like:
app.get('/dispatch', checkRedirect, cors(), (req, res, next) => {
//do something useful, or send your error
})
The contents of your checkRedirect function will be pretty similar to what you have above. However, you do not pass data to the next() function. That just passes control to the next middleware. Instead, put any data you need on the req object and handle it in the final middleware, AFTER cors. If all you are doing is setting a 500 error, you don't even need CORS for that.
According to #Rampant 's answer,
this is how I did it with request-promise (rp):
function checkPrecondition(req, res, next){
req.precondition = false
rp({ method: 'POST',
...
})
.then((data) => {
...
req.precondition = true
next()
})
.catch((data) => {
...
next()
})
}
and in the express handler:
app.post('/query', checkPrecondition, cors(), (req, res, next) => {
if(!req.precondition){
res.status(400).send(JSON.stringify({status: 'insufficient'}))
return
}
res.redirect('target.host')
})
Thanks for clearifying the CORS issue.
I have to do request from node to Yii2 api. It doesn't throw any errors, but doesn't return anything either. When I do request to Yii2 api method directly in browser, value is returned. Here is my request in route in node:
router.get('', function (req, res) {
var parameter = 20;
request({
url: 'http://**.**.**.***:8000/web/index.php?r=api/get-value',
parameter: parameter,
method: 'GET'
}, function(error, response, body) {
if(error || response.statusCode != 200)
throw error;
res.send(body);
});
});
module.exports = router;
And here is method/endpoint in Yii2 controllers/apiController.php:
public function actionGetValue($inverterId) {
return $inverterId * 2;
}
Any suggestions what could be wrong/missing?
You can use the following
var http = require('http');
var client = http.createClient(8000, 'localhost');
var request = client.request('GET', '/web/index.php?r=api/get-value');
request.write("stuff");
request.end();
request.on("response", function (response) {
// handle the response
});
Resource Link:
Http request with node?
Sending http request in node.js
or Another full example:
Get requests
Now we’ll set up a super simple test to make sure it’s working. If it’s not still running, run your simple Node server so that it’s listening on http://localhost:8000. In a separate file in the same directory as your http-request.js where your new module lives, add a file called test-http.js with the following contents:
// test-http.js
'use strict';
const
request = require('./http-request'),
config = {
method: 'GET',
hostname: 'localhost',
path: '/',
port: 8000
};
request(config).then(res => {
console.log('success');
console.log(res);
}, err => {
console.log('error');
console.log(err);
});
This will import our module, run a request according to the configured options, and console log either the response, or an error if one is thrown. You can run that file by navigating to its directory in the command line, and typing the following:
$ node test-http.js
You should see the following response:
success
{ data: 'Cake, and grief counseling, will be available at the conclusion of the test.' }
Resource Link:
https://webcake.co/sending-http-requests-from-a-node-application/
Okay, shame on me, I did not check, what's going on in public function beforeAction($action) in apiController.php - since request to endpoint getValue() is done from the "outside", it falls under a condition, that does not allow further actions and returns false - that's why response wasn't changing no matter what was done/set in getValue().
STUDENT QUESTION!
I'm learning about Node.js/Express and MySQL databases using the Sequelize ORM. Traditionally in our simple applications, after querying a MySQL database with Sequelize, we will issue a res.redirect('/') within Express PUT route's .then promise, similar to this:
app.post("/", function (req, res) {
db.Burgers.create({
burger_name: req.body.burger_name
}).then(function () {
res.redirect('/');
});
});
I'm running into a problem when creating a sequelize query using the findOrCreate() method. Namely, I'm struggling to find where to place the res.redirect statement on an AJAX PUT request. For some reason, when I have the res.redirect('/') attached within the express route for the PUT statement, I will see duplicate PUT requests in the Chrome Network inspector. The first PUT request is displayed as (localhost:3000/devour, type:text/plain, status:302).
The PUT request is received by the Express server and the sequelize query succeeds, updating the proper tables in the MySQL database. However, the redirect on the Express route does not succeed and Chrome Inpector shows an error " PUT http://localhost:3000/ 404 (Not Found)" and when I look at the Network tab I see a second PUT request (localhost:3000/, type:xhr, status:404).
This is the Express PUT route:
app.put("/devour", function (req, res) {
var customerId;
// Check to see if the customer name entered already exists, if not create
db.Customers.findOrCreate({
where: {
customer_name: req.body.customer_name
}
})
.spread((user, created) => {
if (created) console.log("User created");
customerId = user.id;
var update = {
"devoured": req.body.devoured,
"CustomerId": customerId
};
db.Burgers.update(update, {
where: {
id: req.body.id
}
}).then(function () {
res.redirect('/');
});
})
});
What is generating this second PUT request? Is it a response rather than request?
This is my first Sequelize query using the findOrCreate() method so perhaps I'm misunderstading the use of .spread().
If I comment out the res.redirect on the PUT Express route, the error does not occur but I have to manually refresh the page to see the updated data from the MySQL database.
= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =
I added the 303 status code to my Express PUT route: res.redirect(303, '/'). This elmininated the 404 error on the redirect, however the HTML page was not refreshing with GET request to reload the page with updates from the PUT request.
Then I looked at my Ajax call and realized that, perhaps, I needed to add code there to handle the response from the server:
$.ajax({
url: URL,
type: 'PUT',
contentType: 'application/json',
data: JSON.stringify(dataObject)
})
So I added a .done promise callback and the page successfully refreshes following the PUT request:
$.ajax({
url: URL,
type: 'PUT',
contentType: 'application/json',
data: JSON.stringify(dataObject)
}).done(function(){
window.location.href = window.location.origin + '/'
})
I guess I'm a bit confused about why the server-side res.redirect(303, '/'), alone, doesn't result in the page refresh on the client-side. What is the point of providing the '/' path as an argument?
Thank you!
You can read more about what a redirect header does when included in PUT requests in Why POST redirects to GET and PUT redirects to PUT?
Long story short your PUT request remains a PUT request, just gets redirected to /.
Your app should work properly if you provide 303 status code.
res.redirect( 303, '/' );
Im doing a node app, which has a html form doing an action to /users
This url calls this method on post
exports.create = function(req, res, next) {
const user = new User(req.body);
user.save((err) => {
if (err) {
return next(err);
} else {
res.status(302).json(user).redirect('/chat');
}
});
};
However i'm unable to do a redirect after storing the data in my db, and get the error Can't set headers after they are sent.
I've tried placing the redirect different places in the form, but i keep getting the same error.
Try removing .json(user) from the redirect. You cannot both send a JSON and a redirect at the same time.
Or if you want to send the JSON, maybe as a response to an Ajax request, don't send .status(302) but do the redirect on client side JavaScript.
I'm making a testbench with Test'em and Mocha (that run on node.js) in order to test a PHP website.
What I want is to request some URL (e.g http://www.my-website/test.php) and get the http status code as well as the content returned.
I'm doing it with the node.js Request module.
The problem is:
I need to be authenticated to access this page, otherwise I'm
redirected to the login page.
So, does it exist a way to log in my application through Node.js and keep the session open to be able to chain tests on any pages I want?
I was thinking on get the PHPSESSID on login request if it is possible. Do you thing it is a good direction ?
Any help would be much appreciated.
Thank you, have a nice day :)
Michaël
If you set jar: true in your options or use your own custom cookie jar, then request will remember cookies set by the server so that you can keep your session between requests.
mscdex thanks for your answer! But unfortunately it did not work for me :/
hyubs thanks to you too.
Finally I carried on to use Mocha + Request.
Basically what I did is:
Connect through a POST request to the login page and get the PHPSESSID cookie that is returned in the response header.
Pass the cookie in the header in the next requests that target a URL where you have to be logged.
Here is my code :
var params = {
email: 'your_username',
password: 'your_password'
};
var paramsString = JSON.stringify(params);
// Login to the application
request.post('http://localhost/biings/front-end/rest/auth',
{
headers: {
"Content-Type" : "application/json",
'Content-Length' : paramsString.length
},
body: paramsString,
},function (error, response, body) {
// get the PHPSESSID (the last one) that is returned in the header. Sometimes more than one is returned
var sessionCookie = response.headers['set-cookie'][response.headers['set-cookie'].length - 1];
sessionCookie = sessionCookie.split(';');
sessionCookie = sessionCookie[0];
// Write it in a file (this is a quick trick to access it globally)
// e.g.: PHPSESSID=ao9a1j0timv9nmuj2ntt363d92 (write it simply as a string)
fs.writeFile('sessionCookie.txt', sessionCookie, function (err)
{
if(err)
{
return console.log(err);
}
});
});
// don't care about this it() function (it's for Mocha)
it("test 1", function(done)
{
// Get the cookie
fs.readFile('sessionCookie.txt','utf8', function (err, data)
{
if(err)
{
throw err;
}
else
{
// Launch a request that includes the cookie in the header
request.get('http://localhost/biings/front-end/rest/group',
{
headers: {"Cookie" : data},
}, function (error, response, body) {
// Check your request reaches the right page
expect(response.statusCode).equals(200);
console.log(body);
done();
});
}
});
});
It works like a charm for me.
Tell me if you see something wrong or which could be optimized :)
Michaël
Instead of using the request module, use headless browsers like PhantomJS and zombie.js. You can even emulate user interaction with these.