Retrieving <title> of a page with URL in NodeJS - node.js

Question

I am trying to get <title> of the urls without using third party packages. Is it possible to get the title of a page from the url without using third party packages.
Routes can be single or multiple
/I/want/title/?address=http://yahoo.com
/I/want/title/?address=google.com&address=www.dawn.com/events/
app.js
const http = require('http');
const url = require('url');
const app = require('express')();
app.get('/I/want/title/', (req, res, next) => {
const urls = url.parse(req.url,true).query;
const queryObject = url.parse('https://nodejs.org/en/', true).query;
res.send('Sucess');
next(); // Allows the request to continue to the next middleware in line
});
const server = http.createServer(app);
server.listen(3000);

Answer 1

You need to fetch the requested page, parse the code to extract it's <title>.
The url is just for passing the url :-) Like parsing that 'https://nodejs.org/en/' uses https protocol etc., not for fetching its remote content.
Use something like this.
const fetch = require('node-fetch'),
express = require('express'),
app = express()
const parseTitle = (body) => {
let match = body.match(/<title>([^<]*)<\/title>/) // regular expression to parse contents of the <title> tag
if (!match || typeof match[1] !== 'string')
throw new Error('Unable to parse the title tag')
return match[1]
}
app.get('/', (req, res) => {
const { url } = req.query
if (!url)
return res.status(400).end('Missing url query parameter')
fetch(url)
.then(res => res.text()) // parse response's body as text
.then(body => parseTitle(body)) // extract <title> from body
.then(title => res.send(title)) // send the result back
.catch(e => res.status(500).end(e.message)) // catch possible errors
})
app.listen(3000)
Your server will listen on http://localhost:3000, just run this thru your browser:
http://localhost:3000/?url=https://google.com gives Google
http://localhost:3000/?url=https://stackoverflow.com/questions/64051968/retrieving-title-of-a-page-with-url-in-nodejs gives node.js - Retrieving <title> of a page with URL in NodeJS - Stack Overflow

Answer 2

If you insist on using no 3rd-party libraries (bad idea?? Also... express is a 3rd party library...), you can use the following:
const http = require('http');
let app = require('express')();
app.get('/I/want/title', async (req, res, next) => {
try {
// The same "http" module can help us make a request
let html = await new Promise((resolve, reject) => {
let [ , protocol, host, port, path ] = req.query.address.match(/(http|https):[/][/]([^:/]*)(:[0-9]*)?([/].*)?/) || [];
if (![ 'http', 'https' ].includes(protocol)) throw new Error('Supplied url is invalid');
port = port || (protocol === 'http' ? 80 : 443);
let request = require(protocol).request(`${protocol}://${host}`, { port, method: 'GET', path }, response => {
// Collect the http body in `chunks`
let chunks = [];
response.setEncoding('utf8');
response.on('data', chunk => chunks.push(chunk));
// Either reject with error, or resolve with full http body
response.on('error', err => reject(err));
response.on('end', () => resolve(chunks.join('')));
});
request.on('error', err => reject(err));
request.end();
});
// A regex parse is subject to errors, but should do here:
let [ ,, title=null ] = html.match(/<title( [^>]*)?>(.*)<[/]title>/i) || [];
if (!title) throw new Error(`Response contained no title`);
res.send(title.split('\n').map(ln => ln.trim()).filter(Boolean).join(' '));
} catch(err) {
res.status(400).send(`Couldn't get title: ${err.stack}`.replace(/\n/g, '<br/>'));
}
next(); // Allows the request to continue to the next middleware in line
});
http.createServer(app).listen(3000);
I haven't tested this myself but I expect it to work and handle some more obvious edge-cases.
You can see that specifically making the http request is a real headache; you need to handle http and https somewhat separately. This also expects the "address" param to be a fully qualified domain name with protocol, and optional port and path.
You can try running the server and requesting http://localhost:3000/I/want/title?address=https://stackoverflow.com; should produce "Stack Overflow - Where Developers Learn, Share, & Build Careers". You could also try http://localhost:3000/I/want/title?address=http://chess2.fun, and you should see "CHESS2".

Answer 3

a simple req.originalUrl will get what you are looking for if I understand your question. The you can you write javascript string functions to get the part you want
app.get('/I/want/title/', (req, res, next) => {
const url = req.originalUrl;
res.send('Sucess');
next(); // Allows the request to continue to the next middleware in line
});