How do I make server-side fetch calls? - node.js

I have a React web application which currently does fetch calls client-side to update a dashboard with live information (let's say current weather, as an example), meaning that with an increase in users it will cause unnecessary traffic calls and could potentially crash this weather website.
What I am trying to understand is how can I make those fetch calls be server-side? I have looked into creating a Node.js Express server, but I am unsure if it has the functionality to make fetch calls to a remote host.
Here is my code with request-weather which does not really work, unfortunately.
const { response } = require('express');
const express = require('express');
const app = express();
var fetch = require('node-fetch');
const port = process.env.PORT || 5000;
app.use(express.json());
// This displays message that the server running and listening to specified port
app.listen(port, () => console.log(`Listening on port ${port}`));
// create a GET route
app.get('/request-info', (req, res) => {
res.send({ information: 'information call successful' });
});
app.get('/request-weather', (req, res) => {
fetch('http://thisotherwebsite.com/weather-query-that-returns-json',
{method: 'GET',
headers: {' Accept': 'application/json'}})
.then(res => {
return res;
})
});

Couple things:
Your /request-weather handler makes the request to thisotherwebsite but doesn't do anything with the response.
Your .then(res => { return res; }) doesn't actually do anything. You're just taking what fetch already returns and returning it.
If you want to send the response back to the browser you might do something like this:
fetch(...) // make the request
.then(result => result.json()) // extract the data
.then(data => {
res.json(data); // send it to the browser
})
If you want to do additional processing you could await the fetch call and then do whatever else you need to do with it:
app.get('/request-weather', async (req, res) => { // make handler async
// get data from the other site
const data = await fetch(...)
.then(response => response.json());
// package it up with some other stuff
responseData = {
fromOtherSite: data,
myExpressStuff: {
foo: 1,
bar: 2,
}
}
// return it to the browser
res.json(responseData);
Reference:
fetch: response.json() - Extracting data from a fetch response
express response.json() - Sending json to the response (usually to the browser)

Related

How to make nodejs server act like a proxy and get img from cloudinary then send it to browser

for storage space issues i cannot save images in server so i had to store it in cloudinary
and for seo purposes I had to serve it from my domain not cloudinary's
so i thought to get img files from cloudinary and send it directly to browser (to be served from my domain name )
what i am missing is converting the img file i got from cloudinary api into the right form so i can send it using response object in nodejs
here is the code
app.get('/uploads/img/:imgName', (req, res) => {
axios.get('https://res.cloudinary.com/dkhccaa25/image/upload/blog_img/${req.params.imgName}')
.then(response => {
console.log(response);
/* how to convert response into the right format so it can be sent */
//
//
//
})
.then (response => {
/*converted response */
res.sendFile(response)
})
.catch(error => {
console.log(error);
});
how I can be able to send the file from node server to browser so it can be displayed using
<img src="img url...">
You do not have to use res.sendFile, this will require saving it to the filesystem. Basically - accept the response and pass it directly with the correct content-type header send by the upstream response to the client.
Minimal example:
const express = require('express');
const axios = require('axios');
const app = express();
app.get('/', (req, res) => {
axios.get('https://static.pexels.com/photos/45201/kitty-cat-kitten-pet-45201.jpeg').then((axiosResp) => {
res.header('content-type', axiosResp.headers['content-type']).send(axiosResp.data);
});
});
app.listen(3000);
finally the problem solved by editing on #madflow answer (thanks for him )
const express = require('express');
const axios = require('axios');
const app = express();
app.get('/', (req, res) => {
axios.get('https://static.pexels.com/photos/45201/kitty-cat-kitten-pet-45201.jpeg', {responseType: 'stream'})
.then((axiosResp) => {
res.set({
'Content-Type': axiosResp.headers['content-type']
})
axiosResp.data.pipe(res)
});
});
app.listen(3000);

Cannot run .JSON on response from the Fetch API

I am working with React, Express, PostgreSQL, Node, and the Fetch API. When I try to run a "GET" request (within a try block) to get data from my database, the request fails (and enters the catch block) with the following error:
Unexpected token < in JSON at position 0
Here is the failing code that I have on the front end:
const getRequests = async () => {
try {
const responseInfo = await fetch("/api/advice-requests", {
headers: { "Accept": "application/json" },
});
if (responseInfo.status === 200) {
console.log("200 running"); // This is logged to the console.
const data = await responseInfo.json();
console.log("data :", data); // This is NOT logged to the console. It fails.
setAdviceRequests(data.requests);
setResponses(data.responses);
return;
}
} catch (error_o) {
// The UI is updated with the text of the below error
setError(
"Something went wrong on our end. We are looking into how we can improve..."
);
return;
}
};
Here is some of my server code (there is more, but it is not relevant), including some changes I made that worked to solve this problem for other people.
const adviceRequests = require("./controllers/adviceRequests");
const express = require("express");
const app = express();
const cors = require("cors");
app.use(express.json());
app.use(cors());
app.options("*", cors());
if (process.env.NODE_ENV === "production") {
app.use(express.static(path.join(__dirname, "../build")));
app.get("/*", (req, res) => {
res.sendFile(path.join(__dirname, "../build", "index.html"));
});
}
app.get("/api/advice-requests", adviceRequests.getAll);
app.listen(PORT, () => {
console.log(`SERVER RUNNING ON ${PORT}.`);
});
Lastly, here is the adviceRequests.getAll function:
getAll: async (req, res) => {
const db = req.app.get("db");
try {
let requests = await db.requests.getAll(req.session.user.id);
let responses = await db.responses.getAll([req.session.user.id]);
return res.status(200).send([requests, responses]);
} catch (error) {
return res.status(500).send({
message:
"Something went wrong on our end. We are looking into how we can improve.",
error,
});
}
},
A bit more information:
The code works just fine when I run it locally
Even on the live server, I can successfully run several POST requests for authentication and adding requests. I just cannot get them.
I have done quite a bit of research into this and am posting my own as a last resort. Nothing that has worked for other people has worked for me so far.
Everytime I have had this "Unexpected token < in JSON at position 0" it was because i was trying to parse an html plain response as if it was a json. Note that every html file starts with a <.
I suggest you change this console.log("200 running"); with a console.log(responseInfo);, that way you'll notice if the response is a json or not.
From what I see, the problem might be the order in which the app.get are defined. Note that express serves first come first served, so since you have already defined an app.get("/*"), everything will be served by that route. Also note that you are sending back an index.html, which matches the issue shown in the frontend.

Send params from frontend to be used in a NodeJS Get request

I am trying to pass data to my Node.js backend in order to use it in a get request to an API.
For example:
Server.js
const PORT = 8000
const axios = require('axios').default
const express = require('express')
const app = express()
const cors = require('cors')
app.use(cors())
require('dotenv').config()
app.use(express.json())
app.get('/convertedAmount', (req, res) => {
const contentBody = req.body
const options = {
method: 'GET',
url: 'https://currency-converter5.p.rapidapi.com/currency/convert',
params: {format: 'json', from: contentBody.primaryCurrency, to:
contentBody.secondaryCurrency, amount: contentBody.primaryCurrencyAmount},
headers: {
'x-rapidapi-host': process.env.RAPIDAPI_HOST,
'x-rapidapi-key': process.env.RAPIDAPI_KEY,
},
}
axios
.request(options)
.then((response) => {
res.json(response.data)
})
.catch((error) => {
console.error(error)
})
})
app.listen(PORT, () => console.log(`server running on PORT ${PORT}`))
The issue is, I don't know how to pass data to the backend from the frontend, without making it a POST request. But that doesn't work if I make a POST request to the rapidAPI url. So the backend needs to stay the same.
My question is, how would I write the frontend part for this?
Using a POST request in the frontend sends the data with the req.body, but I cant get the data to display in my browser on localhost:8000/convertedAmount.
Thank you
My attempt is:
Frontend.js
...
axios.post('/convertedAmount', {
primaryCurrency: 'USD',
secondaryCurrency: 'GBP',
primaryCurrencyAmount: 1
})
.then((response) => {
console.log(response);
}, (error) => {
console.log(error);
})
...
You shouldn't be trying to send data in the body (such as in req.body) through a GET request. That is what a POST request is for.
With a GET request, you should pass data (such as a user id) as a parameter. Basically, in the url string.
FRONTEND
axios.get("/convertedAmount/USD/GBP/1")
BACKEND
app.get("/convertedAmount/:primaryCurrency/:secondaryCurrency/:primaryCurrencyAmount", (req, res)=>{
console.log(req.params.primaryCurrency);
console.log(req.params.secondaryCurrency);
console.log(req.params.primaryCurrencyAmount);
});
Alternatively, you could use query strings, which would look like this:
FRONTEND
axios.get("/convertedAmount?primaryCurrency=USD&secondaryCurrency=GBP&primaryCurrencyAmount=1")
BACKEND
app.get("/convertedAmount*", (req, res)=>{
console.log(req.query.primaryCurrency);
console.log(req.query.secondaryCurrency);
console.log(req.query.primaryCurrencyAmount);
});

Retrieving <title> of a page with URL in NodeJS

I am trying to get <title> of the urls without using third party packages. Is it possible to get the title of a page from the url without using third party packages.
Routes can be single or multiple
/I/want/title/?address=http://yahoo.com
/I/want/title/?address=google.com&address=www.dawn.com/events/
app.js
const http = require('http');
const url = require('url');
const app = require('express')();
app.get('/I/want/title/', (req, res, next) => {
const urls = url.parse(req.url,true).query;
const queryObject = url.parse('https://nodejs.org/en/', true).query;
res.send('Sucess');
next(); // Allows the request to continue to the next middleware in line
});
const server = http.createServer(app);
server.listen(3000);
You need to fetch the requested page, parse the code to extract it's <title>.
The url is just for passing the url :-) Like parsing that 'https://nodejs.org/en/' uses https protocol etc., not for fetching its remote content.
Use something like this.
const fetch = require('node-fetch'),
express = require('express'),
app = express()
const parseTitle = (body) => {
let match = body.match(/<title>([^<]*)<\/title>/) // regular expression to parse contents of the <title> tag
if (!match || typeof match[1] !== 'string')
throw new Error('Unable to parse the title tag')
return match[1]
}
app.get('/', (req, res) => {
const { url } = req.query
if (!url)
return res.status(400).end('Missing url query parameter')
fetch(url)
.then(res => res.text()) // parse response's body as text
.then(body => parseTitle(body)) // extract <title> from body
.then(title => res.send(title)) // send the result back
.catch(e => res.status(500).end(e.message)) // catch possible errors
})
app.listen(3000)
Your server will listen on http://localhost:3000, just run this thru your browser:
http://localhost:3000/?url=https://google.com gives Google
http://localhost:3000/?url=https://stackoverflow.com/questions/64051968/retrieving-title-of-a-page-with-url-in-nodejs gives node.js - Retrieving <title> of a page with URL in NodeJS - Stack Overflow
If you insist on using no 3rd-party libraries (bad idea?? Also... express is a 3rd party library...), you can use the following:
const http = require('http');
let app = require('express')();
app.get('/I/want/title', async (req, res, next) => {
try {
// The same "http" module can help us make a request
let html = await new Promise((resolve, reject) => {
let [ , protocol, host, port, path ] = req.query.address.match(/(http|https):[/][/]([^:/]*)(:[0-9]*)?([/].*)?/) || [];
if (![ 'http', 'https' ].includes(protocol)) throw new Error('Supplied url is invalid');
port = port || (protocol === 'http' ? 80 : 443);
let request = require(protocol).request(`${protocol}://${host}`, { port, method: 'GET', path }, response => {
// Collect the http body in `chunks`
let chunks = [];
response.setEncoding('utf8');
response.on('data', chunk => chunks.push(chunk));
// Either reject with error, or resolve with full http body
response.on('error', err => reject(err));
response.on('end', () => resolve(chunks.join('')));
});
request.on('error', err => reject(err));
request.end();
});
// A regex parse is subject to errors, but should do here:
let [ ,, title=null ] = html.match(/<title( [^>]*)?>(.*)<[/]title>/i) || [];
if (!title) throw new Error(`Response contained no title`);
res.send(title.split('\n').map(ln => ln.trim()).filter(Boolean).join(' '));
} catch(err) {
res.status(400).send(`Couldn't get title: ${err.stack}`.replace(/\n/g, '<br/>'));
}
next(); // Allows the request to continue to the next middleware in line
});
http.createServer(app).listen(3000);
I haven't tested this myself but I expect it to work and handle some more obvious edge-cases.
You can see that specifically making the http request is a real headache; you need to handle http and https somewhat separately. This also expects the "address" param to be a fully qualified domain name with protocol, and optional port and path.
You can try running the server and requesting http://localhost:3000/I/want/title?address=https://stackoverflow.com; should produce "Stack Overflow - Where Developers Learn, Share, & Build Careers". You could also try http://localhost:3000/I/want/title?address=http://chess2.fun, and you should see "CHESS2".
a simple req.originalUrl will get what you are looking for if I understand your question. The you can you write javascript string functions to get the part you want
app.get('/I/want/title/', (req, res, next) => {
const url = req.originalUrl;
res.send('Sucess');
next(); // Allows the request to continue to the next middleware in line
});

How to know the url that I will be redirected to? [nodejs] [node-fetch]

I am trying to load a JSON file from a url in google cloud. I am using the node-fetch package and it works fine for a couple of hours. The problem is that google changes the redirected url frequently. How can I make a get request to the url I will be forwarded to? Or at least know what url I will be forwarded to? I see there is also a package called request, but its deprecated.
This is the code
var express = require('express');
var router = express.Router();
var fetch = require('node-fetch');
router.get('/', async (req, res) => {
const url = 'https://storage.cloud.google.com/blablabla/config.json';
fetch(url)
.then((res) => {
if (res.ok) {
return res.json();
}
})
.then((data) => res.send({ data }))
.catch((err) => res.send(err));
});
module.exports = router;
You can look up the final URL in the response headers. In your case res.headers.get('location') should do the trick.
The Response object has an undocumented url property. So, let's say you call
const response = await fetch(url, {
redirect: 'follow',
follow: 10,
});
response.url will be the URL of the last redirect that was followed.

Resources