Let's say that I'm using a GET request on https://example.com and the response is this:
This is a response message.
How would I modify it in a way so that in my code, so that it can change the response to say something like this:
This is a MODIFIED response message.
For example, if my Electron app were to navigate to https://example.com, the screen would show me the modified content instead of the original content.
Essentially, I am trying to literally modify the request.
I have based my code off of this question but it only shows a proof of concept with a pre-typed Buffer, as in my situation I'd like modify the response instead of outright replacing it. So, my code looks like this:
protocol.interceptBufferProtocol("http", (req, CALLBACK) => {
if(req.url.includes("special url here")) {
var request = net.request({
method: req.method,
headers: req.headers,
url: req.url
});
request.on("response", (rp) => {
var d = [];
rp.on("data", c => d.push(c));
rp.on("end", () => {
var e = Buffer.concat(d);
console.log(e.toString());
// do SOMETHING with 'e', the response, then callback it.
CALLBACK(e);
});
});
request.end();
} else {
// Is supposedly going to carry out the request without interception
protocol.uninterceptProtocol("http");
}
}
This is supposed to manually request the URL, grab the response and return it. Without the protocol event, it works and gives me a response, but after some debugging, this piece of code consistently calls the same URL over and over with no response.
There is also the WebRequest API, but there is no way of modifying the response body, you can only modify the request headers & related content.
I haven't looked fully into Chromium-based solutions, but after looking at this, I'm not sure if it is possible to modify the response so it appears on my app's end in the first place. Additionally, I'm not familiar with the Chromium/Puppeteer messages that get sent all over the place.
Is there an elegant way to have Electron to get a URL response/request, call the URL using the headers/body/etc., then save & modify the response to appear different in Electron?
Related
Non-English country, please forgive my spelling mistakes.
For example, I want to first redirect url1(http://localhost:3000/api/song/167278) to url2(http://localhost:4000/api/song/167278) to use url2's api. And url2 will reponse a json file, which can be seen in the postman's panel.
(postman's pannel)
But there maybe a lot of elements, I only want an element in the file, such as data[0].url. How can I return just return the url value (data[0].url in this json) when people access http://localhost:3000/api/song/167278.
I am using express.js now, how can I edit it? Or is there any other methods?
app.get('api/song/:id', async (req, res) => {
try {
const { id } = req.params
url = "http://localhost:4000/api/song/" + id
res.redirect(url)
}
catch (e) {
console.log(e)
}
}
You could either proxy the entire request there or fetch localhost:4000/api/song/1 in your request handler (with something like node-fetch or axios or with node's APIs and send the fields that you want back to the client as json.
Нello! I am looking to call a function which has been passed to an expressRouter.post(...) call.
This expressRouter.post(...) call is occurring in a file which I am unable to modify. The code has already been distributed to many clients and there is no procedure for me to modify their versions of the file. While I have no ability to update this file for remote clients, other developers are able to. I therefore face the issue of this POST endpoint's behaviour changing in the future.
I am also dealing with performance concerns. This POST endpoint expects req.body to be a parsed JSON object, and that JSON object can be excessively large.
My goal is to write a GET endpoint which internally activates this POST endpoint. The GET endpoint will need to call the POST endpoint with a very large JSON value, which has had URL query params inserted into it. The GET's functionality should always mirror the POST's functionality, including if the POST's functionality is updated in the future. For this reason I cannot copy/paste the POST's logic. Note also that the JSON format will never change.
I understand that the issue of calling an expressjs endpoint internally has conventionally been solved by either 1) extracting the router function into an accessible scope, or 2) generating an HTTP request to localhost.
Unfortunately in my case neither of these options are viable:
I can't move the function into an accessible scope as I can't modify the source, nor can I copy-paste the function as the original version may change
Avoiding the HTTP request is a high priority due to performance considerations. The HTTP request will require serializing+deserializing an excessively large JSON body, re-visiting a number of authentication middlewares (which require waiting for further HTTP requests + database queries to complete), etc
Here is my (contrived) POST endpoint:
expressRouter.post('/my/post/endpoint', (req, res) => {
if (!req.body.hasOwnProperty('val'))
return res.status(400).send('Missing "val"');
return res.status(200).send(`Your val: ${req.body.val}`);
});
If I make a POST request to localhost:<port>/my/post/endpoint I get the expected error or response based on whether I included "val" in the JSON body.
Now, I want to have exactly the same functionality available, but via GET, and with "val" supplied in the URL instead of in any JSON body. I have attempted the following:
expressRouter.get('/my/get/endpoint/:val', (req, res) => {
// Make it seem as if "val" occurred inside the JSON body
let fakeReq = {
body: {
val: req.params.val
}
};
// Now call the POST endpoint
// Pass the fake request, and the real response
// This should enable the POST endpoint to write data to the
// response, and it will seem like THIS endpoint wrote to the
// response.
manuallyCallExpressEndpoint(expressRouter, 'POST', '/my/post/endpoint', fakeReq, res);
});
Unfortunately I don't know how to implement manuallyCallExpressEndpoint.
Is there a solution to this problem which excludes both extracting the function into an accessible scope, and generating an HTTP request?
This seems possible, but it may make more sense to modify req and pass it, rather than create a whole new fakeReq object. The thing which enables this looks to be the router.handle(req, res, next) function. I'm not sure this is the smartest way to go about this, but it will certainly avoid the large overhead of a separate http request!
app.get('/my/get/endpoint/:val', (req, res) => {
// Modify `req`, don't create a whole new `fakeReq`
req.body = {
val: req.params.val
};
manuallyCallExpressEndpoint(app, 'POST', '/my/post/endpoint', req, res);
});
let manuallyCallExpressEndpoint = (router, method, url, req, res) => {
req.method = method;
req.url = url;
router.handle(req, res, () => {});
};
How about a simple middleware?
function checkVal(req, res, next) {
const val = req.params.val || req.body.val
if (!val) {
return res.status(400).send('Missing "val"');
}
return res.status(200).send(`Your val: ${val}`);
}
app.get('/my/get/endpoint/:val', checkVal)
app.post('/my/post/endpoint', checkVal)
This code isn't tested but gives you rough idea on how you can have the same code run in both places.
The checkVal function serves as a Express handler, with request, response and next. It checks for params first then the body.
I'm still new enough with Node that HTTP requests trip me up. I have checked all the answers to similar questions but none seem to address my issue.
I have been dealt a hand in the Wild of having to go after JSON files in an API. I then parse those JSON files to separate them out into rows that populate a SQL database. The API has one JSON file with an ID of 'keys.json' that looks like this:
{
"keys":["5sM5YLnnNMN_1540338527220.json","5sM5YLnnNMN_1540389571029.json","6tN6ZMooONO_1540389269289.json"]
}
Each array element in the keys property holds the value of one of the JSON data files in the API.
I am having problems getting either type of file returned to me, but I figure if I can learn what is wrong with the way I am trying to get 'keys.json', I can leverage that knowledge to get the individual JSON data files represented in the keys array.
I am using the npm modules 'request' and 'request-promise-native' as follows:
const request = require('request');
const rp = require('request-promise-native');
My URL is constructed with the following elements, as follows (I have used the ... to keep my client anonymous, but other than that it is a direct copy:
let baseURL = 'http://localhost:3000/Users/doug5solas/sandbox/.../server/.quizzes/'; // this is the development value only
let keysID = 'keys.json';
Clearly the localhost aspect will have to go away when we deploy but I am just testing now.
Here is my HTTP call:
let options = {
method: 'GET',
uri: baseURL + keysID,
headers: {
'User-Agent': 'Request-Promise'
},
json: true // Automatically parses the JSON string in the response
};
rp(options)
.then(function (res) {
jsonKeysList = res.keys;
console.log('Fetched', jsonKeysList);
})
.catch(function (err) {
// API call failed
let errMessage = err.options.uri + ' ' + err.statusCode + ' Not Found';
console.log(errMessage);
return errMessage;
});
Here is my console output:
http://localhost:3000/Users/doug5solas/sandbox/.../server/.quizzes/keys.json 404 Not Found
It is clear to me that the .catch() clause is being taken and not the .then() clause. But I do not know why that is because the data is there at that spot. I know it is because I placed it there manually.
Thanks to #Kevin B for the tip regarding serving of static files. I revamped the logic using express.static and served the file using that capability and everything worked as expected.
With the Node JS request module it is possible to get the response, but, is there any way of getting the request headers sent?
I'm not sure what the official way of doing this is but there are several things that seem to work.
If you haven't bound the callback to another this value then it will just be the request, e.g.:
request.get(options, function() {
console.log(this.getHeader('... header name ...'));
console.log(this.headers);
});
You could also access the request using response.request:
request.get(options, function(err, response) {
console.log(response.request.getHeader('... header name ...'));
console.log(response.request.headers);
});
That second approach should work anywhere that you have access to the response.
I believe these are the relevant lines in the source code:
https://github.com/request/request/blob/253c5e507ddb95dd88622087b6387655bd0ff935/request.js#L940
https://github.com/request/request/blob/253c5e507ddb95dd88622087b6387655bd0ff935/request.js#L1314
Trying to understand how to work properly with :
1. Express
2. request
3. middleware
It's a follow up question from here where the discussion wad fruitful and helpfull (thanks #BlazeSahlzen , you are great!) but I realize that I tried at one point to put too much issues (although they are all related) into the same question.
So, this one is a focused question... I hope :-)
Case: I want to build POST() that recives parameter via path (/:param1),
uses it to request() #1 an external API,
gets the result from the external API,
Uses the result to do somwething and send ANOTHER request() #2 to a 2nd external API,
get's the outcome of the 2nd APi request(),
decide if the POST is statusCode = 200 with message="ok" or statusCode = something_else and message = "problem"
and res.send() it properly.
for that, here is my pseudo code -
var middle_1 = function(req, res, next) {
param1 = req.params.param1; //trying to access the param1 from the path, not sure it will work in middleware
req.middle_1_output = {
statusCode: 404,
message: "param1"
}
var options = {
method: 'PUT',
url: `EXTERNAL_API_1`,
headers: {
'cache-control': 'no-cache',
'content-type': 'application/x-www-form-urlencoded',
apikey: `KEY`
}
};
request(options, function(error, response, body) {
if (error) throw new Error(error);
// CODE THAT DO SOMETHING AND GET INFORMATION
req.request_1_output.statusCode = 200;
req.request_1_output.message = "hello world";
next(); // not sure what happens here - will it "jump" outside of the middle_1() or go to the next request() down the code??
});
var options = {
method: 'PUT',
url: `EXTERNAL_API_2`,
headers: {
'cache-control': 'no-cache',
'content-type': 'application/x-www-form-urlencoded',
apikey: `KEY`
}
};
request(options, function(error, response, body) {
if (error) throw new Error(error);
//Can I use here the req.request_1_output.message ???
//How can I use here ALSO some of the EXTERNAL_API_1 outcome????
// Some more CODE THAT DO SOMETHING AND GET INFORMATION
req.request_2_output.statusCode = 201;
req.request_2_output.message = "hello world";
next(); // not sure what happens here
});
}
//This middleware is only used to send success response
var response_success = function(req, res) {
sum_statusCode = req.request_1_output.statusCode + req.request_2_output.statusCode;
if (req.request_2_output.message == req.request_1_output.message) {
meassge = "hello world";
} else {
message = "goodbye world!";
}
res.json({
"statusCode": sum_statusCode,
"message": message
});
}
app.post('/test', middle_1, response_success);
I am not sure how to connect the different requests (request #1 and request #2) in this case - should they all become middleware? how should I write it? (connect => make them run one only after the other is done.)
How can I get also infomation from the request #1 outcome and use it in the request #2 ?
look at my code at response_success() -> will this work? can I access like this data from req that originated within the request #1 and request #2?
How am I suppose to access inside the response_success() data which is the OUTCOME of the request #1 and request #2?
// EDITED - question #5 and #6 are a late edition of mine but should be a stand alone questions. I leave them here but I will be opening a new thread just for them.
Let's say my middle_1 needs to get information as an outcome from the request_1 , calculate something, and move it forward to a middle_2... how do I take the request_1 information into something that can be transffered into a middle_2? I think I am suppose to create a property inside "req" , something like req.middle_1_outcome = DATA , but I am not sure how to "get the DATA" from the request_1 outcome...
How do I "monitor and wait" for request_1 to be done before my middle_1 moves forward to calculate things? is there a requestSync() funciton for Synced requests?
Thanks in advance to all the helpers :-)
A given middleware function should call next() only once when it is done with all its processing.
A given request handler should only submit one response per request. Anything more will be considered an error by Express.
I am not sure how to connect the different requests (request #1 and
request #2) in this case - should they all become middleware? how
should I write it? (connect => make them run one only after the other
is done.)
If your two request() calls can run in parallel (the second one does not depend upon the first results), then you can run them both and then monitor when they are both done, collect the results, do what you need to do with the request and then once and only once call next().
If they must be run in sequence (use the results from the first in the second), then you can nest them.
How can I get also information from the request #1 outcome and use it
in the request #2 ?
There are a variety of ways to solve that issue. If the requests are being run in sequence, then the usual way is to just put request #2 inside the completion callback for request #1 where the results from #1 are known.
Look at my code at response_success() -> will this work? can I access like this data from req that originated within the request #1 and request #2?
You can't quite do it like that because you can't call next() multiple times from the same middleware.
How am I suppose to access inside the response_success() data which is the OUTCOME of the request #1 and request #2?
If you nest the two operations and run request #2 from inside the completion of
request #1, then inside the completion for request #2, you can access both results. There is no need to a completely separate request handler to process the results. That just makes more complication that is necessary.
If you need to serialize your two requests because you want to use the result from the first request in the second request, then you can use this structure where you nest the second request inside the completion of the first one:
function middle_1(req, res, next) {
var param1 = req.params.param1; //trying to access the param1 from the path, not sure it will work in middleware
var options = {
method: 'PUT',
url: `EXTERNAL_API_1`,
headers: {
'cache-control': 'no-cache',
'content-type': 'application/x-www-form-urlencoded',
apikey: `KEY`
}
};
request(options, function (error, response, body) {
if (error) return next(error);
// CODE THAT DO SOMETHING AND GET INFORMATION
var options = {
method: 'PUT',
url: `EXTERNAL_API_2`,
headers: {
'cache-control': 'no-cache',
'content-type': 'application/x-www-form-urlencoded',
apikey: `KEY`
}
};
// this second request is nested inside the completion callback
// of the first request. This allows it to use the results from
// from the first request when sending the second request.
request(options, function (error2, response2, body2) {
if (error2) return next(error2);
// right here, you can access all the results from both requests as
// they are all in scope
// response, body from the first request and
// response2, body2 from the second request
// When you are done with all your processing, then you
// can send the response here
res.json(....);
});
});
}
app.post('/test', middle_1);
Note several things about the structure of this code:
To use the results of the first request in the second one, just nest the two.
When nesting like this, the results from both requests will be available in the completion callback for request #2 as long as you give the arguments unique names so they don't accidentally hide parent scoped variables of the same name.
It does you no good to throw from an async callback, since there's no way for your code to ever catch an exception throw from a plain async callback. The request will likely just sit there forever until it eventually times out if you throw. You need to actually handle the error. Since I didn't know what error handling you wanted, I called next(err) to at least give you default error handling.
I would not suggest using multiple middleware functions that are really just one operation. You may as well just put the one operation in one request handler function as I've shown.