on a fastify route like that:
app.get('/users/:id', async (req, resp) => {
// pseudo code
// copy request just tweak url
const tweaked = { ...req, url: `http://somehost/customers/${req.params.id}` };
const response = fetch(tweaked);
// in case there was a 200, modify the body
resp.send(response)
});
What is the safest way to create a new Request - based on the incoming one - such that all headers, cookies and any other data is preserved?
Related
Main Question:
Do I need to pass the header in every internal microservice request to have it in the other microservice if it is created by common proxy server?
Example:
Structure:
In Nginx.conf:
proxy_set_header X-Request-ID $request_id;
and when I request to any of these microservices, I can get the header value by using req.headers["x-request-id"]
But, when I do like:
// calling ms2 from ms1
const response = await fetch("http://ms2/");
In ms1 (the 1st hop) I can get the header as I show before, but in ms2 I can't.
I know express server processes a bunch of requests concurrently and it because of that. But anyways it feels weird when you need to pass the header every time with the internal requests:
// calling ms2 from ms1
const response = await fetch("http://ms2/", {
headers: {
"X-Request-ID": req.headers["x-request-id"],
},
});
// when do like this I can get the value in ms2.
Is there any way, or it is what it is?
If you need to perform the request to ms2 for every request to (a) specific endpoint(s) on ms1, you could create an Express middleware that would at least save you the trouble of having to perform the request each time manually:
const makeMs2Request = async (req, res, next) => {
const response = await fetch("http://ms2/", {
headers: {
"X-Request-ID": req.headers["x-request-id"],
}
);
req.ms2data = await response.json();
next();
});
app.get('/my/endpoint/on/ms1', makeMs2Request, (req, res) => {
// here you can use `req.ms2data`
…
});
My app (using vue) allows users to upload files with some info about the data to my node backend. When the user submits the form, this function is triggered:
methods: {
buttonOK () {
const formData = new FormData()
formData.append('name', this.detailFirm.name)
formData.append('description', this.detailFirm.description)
formData.append('version', this.detailFirm.version)
formData.append('date', this.detailFirm.date)
formData.append('file', this.file)
for (var [key, value] of formData.entries()) {
console.log(key, value)
}
let headers = {
'Content-Type': 'multipart/form-data',
'Accept': 'multipart/form-data'
}
this.$http.put('/firmware', formData, {headers: headers})
this.visible = false
}
The log statement shows everything that it ought to, and when this request is made, the network tab in the chrome dev tools shows the post data going through, and it has all the values it should:
name: test
description: test
version: 1
date: 0555-05-05
file: (binary)
My multer middleware looks like this:
const multer = require('multer')
const mult = multer({
dest: '/firmware'
})
module.exports = function (req, res, next) {
/* --Convert multipart/form-data to useable format within express-- */
if (req.path === '/firmware') {
mult.single('file')
console.log('MULTER MIDDLEWARE')
}
next()
}
The log statement there works, leading me to believe that multer is working.
I can't seem to access this information in back end though. Here I have tried both file and formData as the file name in mult.single('').
Here is my controller function:
let firmware = {
name: req.body.name,
version: req.body.version,
description: req.body.description,
date: req.body.date,
file: req.body.file
}
firmwareRepo.create(firmware, (err, create) => {
.............
I've read some other questions, and have made a few adjustments, but I always get an empty object when I log req.body in the controller. Please advise.
https://github.com/expressjs/multer#diskstorage
Note that req.body might not have been fully populated yet. It depends on the order that the client transmits fields and files to the server.
EDIT:
Firstly, I remember I had one problem on the frontend (React), by adding headers, which are not needed (somehow by adding formdata headers u **** up everything), here is the example:
data append stuff goes here
const data = new FormData()
data.append('id', values.id)
......
return async (dispatch) => {
const respond = await fetch('/api/postdata', {
method: 'post',
headers: {
//SEE? THIS IS EMPTY
},
body: data
})
// send form to backend
dispatch(dataSend())
}
}
Second issue could be on the backend. The thing is, that you can't just simply access file info through the req.body. You need to access it through the req.file
.post('/api/post', (req, res, next)=> {
const photo = {}
const newData = {}
uploadData(req, res, (err) => {
if(err){
console.log('error')
}
else {
Object.assign(photo, {file: req.file})
Object.assign(newData, {newData: req.body})
Then pass the photo to where you want to do something with it
const addDataController = new AddDataController(req, res, next, newAdvertData, photo)
addAdvertController.postAdvert()
}
})
Basically what I did is I separated regular data with file, and passed them further to combine and conclude the form. Sorry if this won't help, you're very close anyways!
I don't know why this worked, but everything started functioning as it should when I stopped using multer as an imported middleware, like this:
module.exports = function (req, res, next) {
/* --Convert multipart/form-data to useable format within express-- */
if (req.path === '/firmware') {
mult.single('formData')
console.log('MULTER MIDDLEWARE')
}
next()
}
and instead applied it directly to the route function, like this:
router.put('/firmware', upload.single('formData'), firmware.create) // doesn't work as standalone middleware
If anyone knows why that would be the case, please let me know.
I am rather new with express together with the request-promise module,
and need to create a service S
that is called from serverA
and after S has asked ServerB for some additional info,
it redirects the request of serverA to ServerC.
Since I get a
Error: Can't set headers after they are sent.
even though I do not add something by myself, I wonder someone could help me to get this workflow straight.
This is the code:
`
const express = require('express')
const rp = require('request-promise')
...
app.get('/dispatch', cors(), (req, res, next) => {
var options = {
uri: 'https://ServerB/calc-something..',
headers: {
'User-Agent': 'its-me',
'Data': data_from_serverA
},
resolveWithFullResponse: true, // Get statuscode
json: true // Parse the JSON string in the response
};
rp(options) // Do request to serverB
.then(function (response) {
console.log(`ServerB responded with statuscode ${response.statusCode}`)
// No error, so redirect original res
res.redirect('https://serverC/...') // error occurs here
return next(response)
})
.catch(function (err) {
console.log(`ServerB responded with error ${err}`)
return next(err) // send 500 to serverA
})
})
`
Your cors() middleware is setting CORS headers. This is causing the headers to be sent while your promise is resolving.
A redirect ALSO sends headers, and this is the issue. A redirect sets a location header, but you've already sent the headers so that won't work.
The solution is to split your final middleware into two. First, check to see if a redirect is needed and if so, do that. Otherwise, set whatever data you need on the req object and handle this AFTER the cors call.
Your final route will look something like:
app.get('/dispatch', checkRedirect, cors(), (req, res, next) => {
//do something useful, or send your error
})
The contents of your checkRedirect function will be pretty similar to what you have above. However, you do not pass data to the next() function. That just passes control to the next middleware. Instead, put any data you need on the req object and handle it in the final middleware, AFTER cors. If all you are doing is setting a 500 error, you don't even need CORS for that.
According to #Rampant 's answer,
this is how I did it with request-promise (rp):
function checkPrecondition(req, res, next){
req.precondition = false
rp({ method: 'POST',
...
})
.then((data) => {
...
req.precondition = true
next()
})
.catch((data) => {
...
next()
})
}
and in the express handler:
app.post('/query', checkPrecondition, cors(), (req, res, next) => {
if(!req.precondition){
res.status(400).send(JSON.stringify({status: 'insufficient'}))
return
}
res.redirect('target.host')
})
Thanks for clearifying the CORS issue.
Brothers and sisters, I am building an Express API Endpoint that needs to consume an external API, perform some changing of keys and values, and return to the result to the client. Here is what I have thus far:
const external_endpoint = <external_api_end_point>;
app.get('/', function (req, res, next) {
request({ url: external_endpoint}).pipe(res);
});
This returns the exact payload you would get from hitting the external_endpoint directly.
Isn't there something I can do to change res before it gets sent to the client? I tried a few things but nothings has worked. Any ideas or best practices associated with doing a transform on the incoming payload?
For the sake of simplicity. Lets say this is the payload obj.json:
{
"sad": {
"userid": 5,
"username": "jsmith",
"isAdmin": true
}
}
and I am wanting to change sad to happy.
I know outside of the request I could do something like this:
obj = JSON.parse(JSON.stringify(obj).split('"sad":').join('"happy":'));
but throwing obj in place of res will not work. I have tried assigning the value of this res and res.body but no dice.
Thanks for you help in advance!
If you're using request-promise, you can simply make a new response and send it, or modify the response you got back:
app.get('/', function (req, res, next) {
request({ url: external_endpoint, json: true})
.then(response => res.json({ happy: response.sad })))
.catch(next);
});
(of course, you need to handle errors appropriately)
If you want to process it as a stream (which makes sense if you have a massive amount of data), you can use the original request module, and use event-stream to create your pipe:
const es = require('event-stream');
const swapper = es.through(
function write(data) {
this.emit("data", data.replace("sad", "happy"));
},
function end() {
this.emit("end");
}
);
request({ url: external_endpoint})
.pipe(es.stringify())
.pipe(swapper)
.pipe(es.parse())
.pipe(res);
Here's a sandbox to test the stream processing: https://codesandbox.io/s/3wqx67pq6
I would like to capture an async call made by "request".
The call I am looking to intercept is "https://api.ap.org/v2/yada/yada" .
I want to intercept this third party call to api.ap.org and redirect it to another service, say 127.0.0.1:3001.
I would also like to add headers during this intercept process.
I know how to intercept all calls made by the express js route via http-proxy, but this does not intercept calls made within nodejs itself.
router.get('/', function(req, res, next) {
request("https://api.ap.org/v2/yada/yada", {}, (err, data) => {
console.log('---- call made')
console.log(data);
});
res.render('index', { title: 'Express' });
});
UPDATE - from Estus
function patchedRequest(url, options, ...args) {
let newUrl = 'https://www.google.com/' // replace url with another one;
console.log('------ args');
console.log(url);
console.log(options);
if(url.match(/api\.ap\.org/).length){
options = {};
newUrl = 'http://127.0.0.1:3000/api'
}
return originalRequest(newUrl, options, ...args);
}
This allows me to intercept the call to the third party API and send it the service of my choosing.
Thanks Estus!
This can be done by mocking original request module.
This is roughly how cache-mangling libraries like proxyquire work:
patch-request.js
const originalRequest = require('request');
function patchedRequest(url, ...args) {
const newUrl = 'https://www.google.com/' // replace url with another one;
return originalRequest(newUrl, ...args);
}
Object.assign(patchedRequest, originalRequest);
for (const verb of 'get,head,options,post,put,patch,del,delete'.split(',')) {
patchedRequest[verb] = function (url, ...args) {
const newUrl = 'https://www.google.com/' // replace url with another one;
return originalRequest[verb](newUrl, ...args);
};
}
module.exports = require.cache[require.resolve('request')].exports = patchedRequest;
index.js
// patch request before it's required anywhere else
require('./patch-request');
// load the app that uses request