Error trying to render new page after fetch request in Express - node.js

Problem
I have set up an HTTP fetch request from the client side with some JSON data (a dataURL from a canvas), and I want to be able to access it from the server side and then render a new page that displays that dataURL. This is an Express only app (no client-side frameworks).
However, I keep getting an error. It would seem to me that there's something wrong with the JSON data or the way I handle it, but look at the code.
In Safari, it's Unhandled Promise Rejection: SyntaxError: The string did not match the expected pattern.
In Chrome, it's Uncaught (in promise) SyntaxError: Unexpected token < in JSON at position 0
In Firefox, it's SyntaxError: JSON.parse: unexpected character at line 1 column 1 of the JSON data
Code
Client Side
I have an HTML canvas that I've converted into a dataURL and saved in a JSON object. That object gets sent to sendData.
If I get rid of the line response.json(), then I don't get the error. But I want to render a new page with the dataURL displayed in a paragraph tag.
function sendData(input) {
fetch('/', {
method: 'POST',
headers: {
'Content-Type': 'application/json'
},
body: JSON.stringify(input)
})
.then((response) => {
response.json()
})
}
//Saves dataURL as string
function saveData(input){
var dataURL = input.toDataURL()
data = { imageData: dataURL}
sendData(data)
}
Server Side
If I get rid of the line res.render('images'), I don't get the error, but, once again, I can't render the new page then. Also, this might be another issue, but Express is already not rendering the new page. Logging the req.body.imageData works.
exports.submit_image = function (req, res, next) {
console.log(req.body.imageData)
res.render('images')
}
Routes
And just in case this helps, here's the index.js in the routes directory
router.get('/', index.get_index)
router.post('/', index.submit_image)
How can I resolve this issue?

Node.js automatically handles rejected promises by logging them. You can suppress it using the following code. Take a look at this page for more information about promises.
window.addEventListener('unhandledrejection', event => {
event.preventDefault()
}, false)

Related

How to modify a response in Electron

Let's say that I'm using a GET request on https://example.com and the response is this:
This is a response message.
How would I modify it in a way so that in my code, so that it can change the response to say something like this:
This is a MODIFIED response message.
For example, if my Electron app were to navigate to https://example.com, the screen would show me the modified content instead of the original content.
Essentially, I am trying to literally modify the request.
I have based my code off of this question but it only shows a proof of concept with a pre-typed Buffer, as in my situation I'd like modify the response instead of outright replacing it. So, my code looks like this:
protocol.interceptBufferProtocol("http", (req, CALLBACK) => {
if(req.url.includes("special url here")) {
var request = net.request({
method: req.method,
headers: req.headers,
url: req.url
});
request.on("response", (rp) => {
var d = [];
rp.on("data", c => d.push(c));
rp.on("end", () => {
var e = Buffer.concat(d);
console.log(e.toString());
// do SOMETHING with 'e', the response, then callback it.
CALLBACK(e);
});
});
request.end();
} else {
// Is supposedly going to carry out the request without interception
protocol.uninterceptProtocol("http");
}
}
This is supposed to manually request the URL, grab the response and return it. Without the protocol event, it works and gives me a response, but after some debugging, this piece of code consistently calls the same URL over and over with no response.
There is also the WebRequest API, but there is no way of modifying the response body, you can only modify the request headers & related content.
I haven't looked fully into Chromium-based solutions, but after looking at this, I'm not sure if it is possible to modify the response so it appears on my app's end in the first place. Additionally, I'm not familiar with the Chromium/Puppeteer messages that get sent all over the place.
Is there an elegant way to have Electron to get a URL response/request, call the URL using the headers/body/etc., then save & modify the response to appear different in Electron?

Log request that caused error in node modules

I run a NodeJS server with two new error types in the logs:
[2021-05-21T09:11:33.891Z] SyntaxError: Unexpected token h in JSON at position 0
at JSON.parse (<anonymous>)
at createStrictSyntaxError (~/server/node_modules/body-parser/lib/types/json.js:158:10)
at parse (~/server/node_modules/body-parser/lib/types/json.js:83:15)
at ~/server/node_modules/body-parser/lib/read.js:121:18
at invokeCallback (~/server/node_modules/raw-body/index.js:224:16)
at done (~/server/node_modules/raw-body/index.js:213:7)
at IncomingMessage.onEnd (~/server/node_modules/raw-body/index.js:273:7)
at IncomingMessage.emit (events.js:323:22)
at IncomingMessage.EventEmitter.emit (domain.js:482:12)
at endReadableNT (_stream_readable.js:1204:12)
at processTicksAndRejections (internal/process/task_queues.js:84:21)
The stacktrace shows only node_modules paths, not where in my code this error may have started. The stdout logs do not show what could have originated this error around that time either.
The server code that handles JSON objects is:
// Use JSON parser (required to parse POST forms)
app.use((req, res, next) => {
bodyParser.json()(req, res, next);
});
app.use(bodyParser.urlencoded({extended: true}));
I added logging inside this function in case I have the same error in the future.
In general, how can I log information about the request that caused an error in the node modules?
update with client-side code
This error originated from a user and I am unable to replicate it. The client-side code sending JSON data is:
// `id` indicates the ID of the video
var body = {
percent: percent,
videoId: id,
eventLabel: eventLabel
}
async function view() {
return await fetch("/viewership", {
method: "POST",
credentials: "include",
headers: {
Accept: "application/json",
"Content-Type": "application/json"
},
body: JSON.stringify(body)
});
};
The Network tab of the Chrome debugger shows this request payload:
{percent: 0, videoId: ..., eventLabel: "play"}
Well, here's what we know.
The code is processing an IncomingMessage (and incoming http request)
The error comes from the body-parser module
The error comes from JSON.parse() on what is apparently supposed to be a JSON body
The error appears to come from this particular section of code in the body-parser module.
That code is this:
if (strict) {
var first = firstchar(body)
if (first !== '{' && first !== '[') {
debug('strict violation')
throw createStrictSyntaxError(body, first)
}
}
So, it is apparently failing to find a leading { or [ on the JSON and is instead finding an h.
We can deduce from that information that an incoming http request (probably a POST) is supposed to have a JSON body, but the data is not legal JSON.
Your first point of debugging is to see exactly the JSON body data is in the request. If this request is coming from a browser, you can look in the Chrome network tab of the debugger and see exactly what the browser is sending your server.
So, this is most likely a client-caused error. Either the content-type is set wrongly to JSON when the data is not JSON or the client is supposed to be sending JSON, but is not sending proper JSON.
If you can show us the client-side code for this, we may be able to spot the error in that code.
Do you know a way to log any request that throws an error, e.g. for other bad requests in the future unrelated to JSON?
When the body-parser gets bad JSON, it calls the Express error handler with the exception. If you go the the "Writing Error Handlers" on this Express doc page, it will show you how to catch these errors and handle them with some error page back to the client and as much logging as you want.

Axios request with chunked response stream from Node

I have a client app in React, and a server in Node (with Express).
At server side, I have an endpoint like the following (is not the real endpoint, just an idea of what i'm doing):
function endpoint(req, res) {
res.writeHead(200, {
'Content-Type': 'text/plain',
'Transfer-Encoding': 'chunked'
});
for(x < 1000){
res.write(some_string + '\n');
wait(a_couple_of_seconds); // just to make process slower for testing purposes
}
res.end();
}
This is working perfect, i mean, when I call this endpoint, I receive the whole stream with all the 1.000 rows.
The thing is that I cannot manage to get this data by chunks (for each 'write' or a bunch of 'writes') in order to show that on the frontend as soon as i'm receiving them..(think of a table that shows the rows as soon as i get them from the endpoint call)
In the frontend I'm using Axios to call the API with the following code:
async function getDataFromStream(_data): Promise<any> {
const { data, headers } = await Axios({
url: `http://the.api.url/endpoint`,
method: 'GET',
responseType: 'stream',
timeout: 0,
});
// this next line doesn't work. it says that 'on' is not a function
data.on('data', chunk => console.log('chunk', chunk));
// data has actually the whole response data (all the rows)
return Promise.resolve();
}
The thing is that the Axios call returns the whole data object after the 'res.end()' on the server is called, but I need to get data as soon as the server will start sending the chunks with the rows (on each res.write or whenever the server thinks is ready to send some bunch of chunks).
I have also tried not to use an await and get the value of the promise at the 'then()' of the axios call but it is the same behavior, the 'data' value comes with all the 'writes' together once the server does the 'res.end()'
So, what I doing wrong here ? maybe this is not possible with Axios or Node and I should use something like websockets to solve it.
Any help will be very appreciate it because I read a lot but couldn't get a working solution yet.
For anyone interested in this, what I ended up doing is the following:
At the Client side, I used the Axios onDownloadProgress handler that allows handling of progress events for downloads.
So, I implemented something like this:
function getDataFromStream(_data): Promise<any> {
return Axios({
url: `http://the.api.url/endpoint`,
method: 'GET',
onDownloadProgress: progressEvent => {
const dataChunk = progressEvent.currentTarget.response;
// dataChunk contains the data that have been obtained so far (the whole data so far)..
// So here we do whatever we want with this partial data..
// In my case I'm storing that on a redux store that is used to
// render a table, so now, table rows are rendered as soon as
// they are obtained from the endpoint.
}
}).then(({ data }) => Promise.resolve(data));
}

How to open web brower by using AWS post lambda

I have written the piece of code below:
static async postSearchResult(httpContext: HttpContext, injector: Injector) {
const log = injector.get(Log);
const service = injector.get(Service);
try {
let result = await service.redirectToUI(JSON.parse(httpContext.getRequestBody()));
httpContext.ok(result, 200, {'Content-Type': 'application/json'});
} catch (e) {
httpContext.fail(e, 500);
}
}
protected redirectToUI(response: any) {
// If any post api call happened then it should open web browser and pass some field as query parameter
window.open("https://www.google.com?abc=response.abc");
return response ? response : "failed";
}
Here I am getting the following error :
Execution failed ReferenceError: Window is not defined
What am I doing wrong?
What you are trying to accomplish doesn't make much of a sense. Lambda is a back-end service. To open new browser window, you need to use front-end JavaScript, not back-end Node (on the back-end, you have no access to the front-end window object).
If you want to open a new browser window as a reaction to some back-end response, then you can send some indicator in the HTTP response (i.e shouldOpenNewWindow: true as a part of the response object), parse that response on the front-end and it the indicator is present, then you can issue window.open command. But it has to be done on front-end.

Node.js - Why does my HTTP GET Request return a 404 when I know the data is there # the URL I am using

I'm still new enough with Node that HTTP requests trip me up. I have checked all the answers to similar questions but none seem to address my issue.
I have been dealt a hand in the Wild of having to go after JSON files in an API. I then parse those JSON files to separate them out into rows that populate a SQL database. The API has one JSON file with an ID of 'keys.json' that looks like this:
{
"keys":["5sM5YLnnNMN_1540338527220.json","5sM5YLnnNMN_1540389571029.json","6tN6ZMooONO_1540389269289.json"]
}
Each array element in the keys property holds the value of one of the JSON data files in the API.
I am having problems getting either type of file returned to me, but I figure if I can learn what is wrong with the way I am trying to get 'keys.json', I can leverage that knowledge to get the individual JSON data files represented in the keys array.
I am using the npm modules 'request' and 'request-promise-native' as follows:
const request = require('request');
const rp = require('request-promise-native');
My URL is constructed with the following elements, as follows (I have used the ... to keep my client anonymous, but other than that it is a direct copy:
let baseURL = 'http://localhost:3000/Users/doug5solas/sandbox/.../server/.quizzes/'; // this is the development value only
let keysID = 'keys.json';
Clearly the localhost aspect will have to go away when we deploy but I am just testing now.
Here is my HTTP call:
let options = {
method: 'GET',
uri: baseURL + keysID,
headers: {
'User-Agent': 'Request-Promise'
},
json: true // Automatically parses the JSON string in the response
};
rp(options)
.then(function (res) {
jsonKeysList = res.keys;
console.log('Fetched', jsonKeysList);
})
.catch(function (err) {
// API call failed
let errMessage = err.options.uri + ' ' + err.statusCode + ' Not Found';
console.log(errMessage);
return errMessage;
});
Here is my console output:
http://localhost:3000/Users/doug5solas/sandbox/.../server/.quizzes/keys.json 404 Not Found
It is clear to me that the .catch() clause is being taken and not the .then() clause. But I do not know why that is because the data is there at that spot. I know it is because I placed it there manually.
Thanks to #Kevin B for the tip regarding serving of static files. I revamped the logic using express.static and served the file using that capability and everything worked as expected.

Resources