Main Question:
Do I need to pass the header in every internal microservice request to have it in the other microservice if it is created by common proxy server?
Example:
Structure:
In Nginx.conf:
proxy_set_header X-Request-ID $request_id;
and when I request to any of these microservices, I can get the header value by using req.headers["x-request-id"]
But, when I do like:
// calling ms2 from ms1
const response = await fetch("http://ms2/");
In ms1 (the 1st hop) I can get the header as I show before, but in ms2 I can't.
I know express server processes a bunch of requests concurrently and it because of that. But anyways it feels weird when you need to pass the header every time with the internal requests:
// calling ms2 from ms1
const response = await fetch("http://ms2/", {
headers: {
"X-Request-ID": req.headers["x-request-id"],
},
});
// when do like this I can get the value in ms2.
Is there any way, or it is what it is?
If you need to perform the request to ms2 for every request to (a) specific endpoint(s) on ms1, you could create an Express middleware that would at least save you the trouble of having to perform the request each time manually:
const makeMs2Request = async (req, res, next) => {
const response = await fetch("http://ms2/", {
headers: {
"X-Request-ID": req.headers["x-request-id"],
}
);
req.ms2data = await response.json();
next();
});
app.get('/my/endpoint/on/ms1', makeMs2Request, (req, res) => {
// here you can use `req.ms2data`
…
});
Related
I have an issue where I want to call an external API to verify a certain token coming from my frontend.
i.e
router.post('/api/verifyToken', async (ctx, next) => {
router.post('https://external-site.com/verify').then((response) => {
if (response.success) { console.log('Do some stuff') };
})
})
How would I go about doing this with koa?
You misunderstood with the router itself.
In your router, you define a route where your clients can send HTTP requests and according to your business logic, you return the answers to them.
You can simply imagine router.post('/api/verifyToken' as an event listener. When a request comes in you run whatever is inside of it. It is not an HTTP client though.
If you want to send an external request you have to use an HTTP client for it.
There are a bunch of options:
Built-in http/https package
Axios
Node-fetch
GOT
Request
And many others
One simple example how to do with Axios would be
import axios from 'axios';
router.post('/api/verifyToken', async (ctx, next) => {
try{
const response = await axios.post('https://external-site.com/verify');
// Do your stuff here
console.log(response.data);
ctx.body = response.data;
}catch(e){
ctx.status = 422;
ctx.body = e.message;
}
})
on a fastify route like that:
app.get('/users/:id', async (req, resp) => {
// pseudo code
// copy request just tweak url
const tweaked = { ...req, url: `http://somehost/customers/${req.params.id}` };
const response = fetch(tweaked);
// in case there was a 200, modify the body
resp.send(response)
});
What is the safest way to create a new Request - based on the incoming one - such that all headers, cookies and any other data is preserved?
I know what is wrong with my code and I have looked into the best way of solving it, however with my lack of experience, I am having a hard time finding a good answer.
I need my first route(/data) to be fully completed before the second(/logo) express route sends the data. In short, I just need the variable symbolUrl to be completed before it goes into the second fetch call. Here is the code down below to explain
app.use(express.static('public'));
const url =
'https://pro-api.coinmarketcap.com/v1/cryptocurrency/listings/latest';
const qString =
'?CMC_PRO_API_KEY=' + process.env.apiKey + '&start=1&limit=10&convert=USD';
let symbol = [];
app.get('/data', async (req, res) => {
const fetch_res = await fetch(url + qString);
const coinData = await fetch_res.json();
for (let i = 0; i < 9; i++) {
symbol.push(coinData.data[i]['symbol']);
};
res.json(coinData);
});
app.get('/logo', async (req, res) => {
const symbolUrl = symbol.join(',');
const url2 = 'https://pro-api.coinmarketcap.com/v1/cryptocurrency/info';
const qString2 = `?CMC_PRO_API_KEY=${apiKey}%symbol=${symbolUrl}`;
const fetch_res2 = await fetch(url2 + qString2);
const coinLogo = await fetch_res2.json();
res.json(coinLogo);
});
The issue I am trying to solve with this project is that I want to send the data(/data) to be sent to the front end first because this API call will load the majority of the page. Then my second call will load images and other larger files afterward. HOWEVER, the API I am working with to get the logos(images) of the specific crypto coins I want, I need a different endpoint as well as use %symbol=${symbolUrl} in the API call to get the correct tokens I want to call.
client code:
fetch('http://localhost:2000/data')
.then(async (response) => {
return response.json();
})
.then(async (data) => {
const parsedData = data['data'];
// console.log(data['data'][0]['name'])
await parsedData.forEach((element) => {
// this just has about 20 lines of code generating the the look of the page. It works as intended
});
fetch('http://localhost:2000/logo')
.then(async (response) => {
return response.json();
})
.then(async (logo) => {
console.log(logo)});
***I have tried putting this in an async function and awaiting the first fetch call
All I need to be done is for app.get(/data) to be fully complete before doing my second app.get. I have done testing and I know that is the issue. I apologize if it is something easy, but I couldn't find anything on making an app.get synchronous and I have tried putting both in a async function, however that did not work.
You cannot send responses in fragments like you're trying to do, it would throw an error saying Can't set headers after they are sent to client
The proper method to implement what you are trying to do is to define the first layer as middleware, and then allow the second layer to return the response. Here layer basically means a function handler.
In order to control when the execution passes to the next layer / next function handler, express has a third parameter (request, response, next). You're only using request and response, researching about next will solve your concern.
Express next function, what is it really for?
First handler
app.get('something_unique', async (req, res, next) => {
// do whatever you want to do first
// save data into res.locals
res.locals.foo = {...}
next()
})
Second Handler
app.get('something_unique', (req, res) => {
const data = res.locals.foo;
// whatever you want
return res.json({ anything })
})
More:
Express next function, what is it really for?
Error: Can't set headers after they are sent to the client
Passing variables to the next middleware using next() in Express.js
I'm not sure what client code you're really running as it sounds like you've bee trying several things, but this should work to sequence the /data request and the /logo request so that the /logo request is not run until the response from the /data request has been received.:
async function run() {
const r1 = await fetch('http://localhost:2000/data');
const data = await r1.json();
const parsedData = data.data;
parsedData.forEach((element) => {
// this just has about 20 lines of code generating
// the the look of the page. It works as intended
});
const r2 = await fetch('http://localhost:2000/logo');
const logo = await r2.json();
return logo;
}
run().then(logo => {
console.log(logo);
}).catch(err => {
// handle errors here
console.log(err);
});
If there is any asynchronous code inside the .forEach(), then we will have to see that also to properly sequence that.
As I've said in my comments, stuffing the data from the first request into a server-side variable is probably the wrong design on the server because two separate clients both issuing /data requests will conflict with one another, creating race conditions. But, you haven't explained what this data is really for or why you're stuffing it into a variable on the server for us to suggest an alternate design.
What are the consequences of making request global (or singleton), so that it is accessible all over the server and does not have to be passed in each function call? For example:
index.js:
const http = require('http');
const { saveReq } = require('./shared');
const {
doSomethingWithReqPassingItAsParameter,
doSomethingWithReqPassingItAsGlobal
} = require('./lib');
const requestListener = function (req, res) {
// approach 1
doSomethingWithReqPassingItAsParameter(req);
// approach 2
saveReq(req);
doSomethingWithReqPassingItAsGlobal();
res.writeHead(200);
res.end('Hello, World!');
}
const server = http.createServer(requestListener);
server.listen(8080);
lib.js:
const { loadReq } = require('./shared');
const doSomethingWithReqPassingItAsParameter = (req) => {
console.log('req as parameter', req.url);
};
const doSomethingWithReqPassingItAsGlobal = () => {
console.log('req as global', loadReq().url);
};
module.exports = {
doSomethingWithReqPassingItAsParameter,
doSomethingWithReqPassingItAsGlobal,
};
shared.js
var request;
const saveReq = (r) => request = r;
const loadReq = () => request;
module.exports = {
saveReq,
loadReq,
}
This is very convenient for large projects with many levels of function calls, but how parallel requests will be handled? I know that nodejs is single-threaded, does it mean than each http request will be run from end to finish separately or they can overlap, thus using a global request object would make a mess?
The consequences are that your server will only work for one request at a time and as soon as you have more than one request in flight at the same time, data will be mixed up between requests dealing to bugs, crashes, security issues and incorrect results.
Simply put, you cannot program a server that way. Pass the req object or data from it to any functions that need it. That keeps the appropriate req object associated with the right execution to avoid all the problems of trying to store a req object in some sort of global location where multiple requests in flight at the same time will/can conflict.
There is a relatively new thing in nodejs called "async local storage" that could perhaps be used for this. You can read a little about it here, though it's my personal opinion that it's still better to pass your request data to the functions that want to use it rather than the async local storage for this.
so I have the following Scenario; I have a private API key that Angular will show in XHR request. To combat this, I decided to use Express as a proxy and make server side requests. However, I cannot seem to find documentation on how to make my own get requests.
Architecture:
Angular makes request to /api/external-api --> Express handles the route and makes request to externalURL with params in req.body.params and attaches API key from config.apiKey. The following is pseudocode to imitate what I'm trying to accomplish:
router.get('/external-api', (req, res) => {
externalRestGetRequest(externalURL, req.body.params, config.apiKey)
res.send({ /* get response here */})
}
You are half way there! You need something to make that request for you. Such as the npm library request.
In your route something like
var request = require('request');
router.get('/external-api', function(req, res){
request('http://www.google.com', function (error, response, body) {
console.log('error:', error); // Print the error if one occurred and handle it
console.log('statusCode:', response && response.statusCode); // Print the response status code if a response was received
res.send(body)
});
})
This allows you to make any type of request using whatever URL or API keys you need. However it's important to note you also need to handle any errors or bad response codes.
The accepted answer is good, but in case anyone comes across this question later, let's keep in mind that as of February, 2020, request is now deprecated.
So what can we do? We can use another library. I would suggest Axios.
Install it and do something like:
const axios = require('axios')
const url = "https://example.com"
const getData = async (url) => {
try {
const response = await axios.get(url)
const data = response.data
console.log(data)
} catch (error) {
console.log(error)
}
}
getData(url)