Is it possible to clear node.js require cache using express.js? - node.js

I wrote node.js small app for myself which was using small js code to clear require cache before handling request:
let clearCache
if (process.env.NODE_ENV === 'development') {
const {cache} = require
clearCache = (except) => {
for (let key in cache)
if (!except.includes(key) && key.indexOf('/node_modules/') === -1)
delete cache[key]
}
} else {
clearCache = () => {}
}
const { createServer } = require('http')
const persistentPaths = ['/path/to/my/db/file']
createServer((req, res) => {
clearCache(persistentPaths)
require('./handleRequest')(req, res)
}).listen(3000, () => {})
I believe this is the most efficient way to working with app in development.
When I change code it applies immediately and this worked fine to me.
Now I would like to work with express.js, and it seems to impossible to use this approach.
Okay, seems like it is self-answering question and I need to use nodemon.
But I really don't want to use tool which will be watch all files and will relaunch whole server. I'm afraid that it would be much slower.
In my example you can see that I reload files except db file, so my app in development connects to database only once and keeps that connection. In case of using nodemon app need to load also all node_modules code, make new connection to db each time when code changed, maybe it will connect each time to postgres and to redis.
Question is: I can't be the one who doing it this way, are there solutions to reload modules for express.js?

Did it! Would be glad if you will share your thoughts on it.
Because I can google only nodemon and chokidar solutions, maybe I'm doing something wierd, here is my solution:
server.ts:
import express from 'express'
const app = express()
const port = 3000
app.listen(port, () =>
console.log(`Example app listening at http://localhost:${port}`)
)
if (process.env.NODE_ENV === 'development') {
const {cache} = require
const persistentFiles: string[] = []
const clearCache = (except: string[]) => {
for (let key in cache)
if (!except.includes(key) && key.indexOf('/node_modules/') === -1)
delete cache[key]
}
app.use((req, res, next) => {
clearCache(persistentFiles)
const {router} = require('config/routes')
router.handle(req, res, next)
})
} else {
const router = require('config/routes')
app.use(router.handle)
}
And routes.ts:
import {Router, Request, Response} from 'express'
export const router = Router()
router.get('/', function (req: Request, res: Response) {
res.send('Hello world')
})
Usage: NODE_ENV=development NODE_PATH=src node src/server.js
(my IDE compiles ts to js while editing and put it right there, if you are using VS code it might be more complicated, I guess)

You don't need to write your one util. You can use module like chokidar for so. You can specify directory and ignore the same time.
You can reload on callback the watch.
Sample from there wiki:: https://github.com/paulmillr/chokidar
const chokidar = require("chokidar");
// Full list of options. See below for descriptions.
// Do not use this example!
chokidar.watch("file", {
persistent: true,
ignored: "*.txt",
ignoreInitial: false,
followSymlinks: true,
cwd: ".",
disableGlobbing: false,
awaitWriteFinish: {
stabilityThreshold: 2000,
pollInterval: 100,
},
ignorePermissionErrors: false,
atomic: true, // or a custom 'atomicity delay', in milliseconds (default 100)
});
// One-liner for current directory
chokidar.watch(".").on("all", (event, path) => {
// Reload module here.
});

Related

Why is my Heroku Express API data persistent even though it's only coming from a variable

I created a simple API using express, and deployed it to Heroku, this is the code for it:
const express = require("express");
const app = express();
const cors = require("cors");
app.use(express.json());
app.use(cors());
app.use(express.static("build"));
let notes = [
{
id: 1,
content: "HTML is easy",
date: "2022-05-30T17:30:31.098Z",
important: true,
},
{
id: 2,
content: "Browser can execute only Javascript",
date: "2022-05-30T18:39:34.091Z",
important: false,
},
{
id: 3,
content: "GET and POST are the most important methods of HTTP protocol",
date: "2022-05-30T19:20:14.298Z",
important: true,
},
];
const generateId = (arr) => {
const maxId = arr.length < 0 ? 0 : Math.max(...arr.map((item) => item.id));
return maxId + 1;
};
app.get("/", (req, res) => {
res.send(`<h1>Hello World!</h1>`);
});
app.get("/api/notes", (req, res) => {
res.json(notes);
});
app.get("/api/notes/:id", (req, res) => {
const id = Number(req.params.id);
const note = notes.find((note) => note.id === id);
if (note) {
res.json(note);
} else {
res.status(404).end();
}
});
app.delete("/api/notes/:id", (req, res) => {
const { id } = Number(req.params);
notes = notes.filter((note) => note.id !== id);
res.status(204).end();
});
app.post("/api/notes", (req, res) => {
const body = req.body;
if (!body.content) {
return res.status(400).json({
error: "Content Missing",
});
}
const note = {
content: body.content,
important: body.important || false,
date: new Date(),
id: generateId(notes),
};
notes = notes.concat(note);
res.json(note);
});
app.put("/api/notes/:id", (req, res) => {
const newNote = req.body;
notes = notes.map((note) => (note.id !== newNote.id ? note : newNote));
res.json(newNote);
});
const PORT = process.env.PORT || 3001;
app.listen(PORT, () => {
console.log(`Server running on port ${PORT}`);
});
as you can see, the data served to the frontend (A React app) comes from the '/api/notes' endpoint, this endpoint returns a response with the notes array.
After deploying to Heroku (https://fierce-chamber-07494.herokuapp.com/) the functionality of adding notes, and setting importance all work perfectly normal, but what I wasn't expecting was for the data to be persistent even after refreshing the page, visiting it in another device, etc. The data only comes from a variable, not a database, nothing. So why is it persistent? does Heroku modify the variable itself? how does this work?
The top-level code of an Express server often runs once, when you start up the server. Variables declared at that top level are then persistent if there are any handlers that reference them.
Consider how a client-side page with JavaScript works - the page loads, and then the JavaScript runs. If you keep the tab open for a couple hours and then come back to it, you'll see that variables declared on pageload still exist when you come back. The same sort of thing is happening here, except that the persistent environment is on your server, rather than on a client's page.
The code that starts up the Express server - that is, your
const express = require("express");
const app = express();
const cors = require("cors");
app.use(express.json());
app.use(cors());
...
and everything below it - doesn't run every time a request is made to the server. Rather, it runs once, when the server starts up, and then when requests are made, request handlers get called - such as the callback inside
app.get("/", (req, res) => {
res.send(`<h1>Hello World!</h1>`);
});
So, the variables declared at the top-level are persistent (even across different requests) because that server environment is persistent.
That said - something to keep in mind with Heroku is that with their free and cheap tiers, if no request is made to your app for a period of time (maybe 30 minutes), Heroku will essentially turn your server off by spinning down the dyno until another request is made, at which point they'll start your server up again, which will run the top-level code again. So while you'll sometimes see a top-level variable that appears to have its mutated values persist over multiple requests, that's not something to count on if your Heroku plan doesn't guarantee 100% uptime for your server.

'TypeError a.map is not a function' in Npm run build, but not npm start

I have a basic MERN stack application. On the app when I do npm start, it runs fine and functions as expected. However, after doing a npm run build, and running serve -s build, the app fails with the following message -
TypeError: a.map is not a function
at He (Contacts.js:27:16)
at wl (react-dom.production.min.js:166:137)
at Pi (react-dom.production.min.js:215:270)
at wu (react-dom.production.min.js:291:202)
at ys (react-dom.production.min.js:279:389)
at vs (react-dom.production.min.js:279:320)
at ms (react-dom.production.min.js:279:180)
at as (react-dom.production.min.js:270:88)
at rs (react-dom.production.min.js:267:429)
at x (scheduler.production.min.js:13:203)
Here is my express main server.js file -
const express = require('express')
const cors = require('cors')
const app = express()
const connectDB = require('./config/db')
app.use(cors())
//Connect database
connectDB()
//Init Middleware
app.use(express.json({ extended: false }))
app.get('/', (req, res) =>
res.json({ msg: 'Welcome to the Digital RoloDex API'}))
//Define Routes
app.use('/api/users', require('./routes/users'))
app.use('/api/auth', require('./routes/auth'))
app.use('/api/contacts', require('./routes/contacts'))
const PORT = process.env.PORT || 4444
app.listen(PORT, () => console.log(`app running on ${PORT}`))
Here is the file where the error is coming from -
import React, { useEffect } from 'react'
import ContactItem from './ContactItem'
import Spinner from '../layout/Spinner'
import { getContacts, useContacts } from '../context/contact/ContactState'
const Contacts = () => {
const [ contactState, contactDispatch ] = useContacts()
const { contacts, filtered, loading } = contactState
useEffect(() => {
getContacts(contactDispatch)
}, [contactDispatch])
if (contacts === 0 && !loading) {
return <h4>Please add a contact</h4>
}
return (
<div>
{ contacts !== null && !loading ?
filtered.length !== 0 ?
filtered.map((contact) => (
<ContactItem key={contact._id} contact={contact} />
))
: contacts.map((contact) => (
<ContactItem key={contact._id} contact={contact} />
))
:
<Spinner />
}
</div>
)
}
export default Contacts
I found two stackoverflow questions with the same problem. I applied their tips, but it is not helping either. Any help would be greatly appreciated.
https://stackoverflow.com/questions/65069848/typeerror-a-map-is-not-a-function
https://stackoverflow.com/questions/64362803/map-is-not-a-function-when-getting-react-files-from-build-folder
I want to answer my own question just in case someone runs into the same error. Foremost, in a static build, any variables, components, functions, etc. are going to be renamed with abbreviated letters. Which is why it is stating "a".map is not a function.
In addition, my backend and client were in two separate folders. So my front end wasn't calling to my back end properly. I uninstalled cors since this would make my users' information vulnerable across the web. Instead, I put my backend folder and files in the root and kept the client in its own folder. From there, I cd. into my client folder and did an npm run build. The error went away and was able to call back to the express api.

React routes working locally but not through Heroku

I’ve built an app with Express and React which has GET and POST routes which work perfectly locally. I’ve deployed through Heroku and nothing is working anymore! I’m just getting a 404 error. I’ve tried to create a static.json file which hasn’t worked, although I didn't use Create-React-App to set it up anyway.
This is my index.js:
const mongoose = require('mongoose')
const bodyParser = require('body-parser')
const app = express()
const router = require('./config/router')
const { port, dbURI } = require('./config/environment')
const errorHandler = require('./lib/errorHandler')
const logger = require('./lib/logger')
app.use(express.static(`${__dirname}/public`))
app.use(express.static(`${__dirname}/dist`))
mongoose.connect(dbURI, { useNewURLParser: true, useCreateIndex: true})
app.use(bodyParser.json())
app.use(logger)
app.use('/api', router)
app.use(errorHandler)
app.listen(port, () => console.log(`Up and running on port ${port}`))
module.exports = app
router.js
const vacancies = require('../controllers/vacancies')
router.route('/vacancies')
.get(vacancies.index)
.post(vacancies.create)
router.route('/vacancies/:id')
.get(vacancies.show)
module.exports = router
controller:
//tested in insomnia - works
function indexRoute(req, res, next) {
Vacancy
.find(req.query)
.then(vacancies => res.status(200).json(vacancies))
.catch(next)
}
//tested in insomnia - works
function showRoute(req, res, next) {
Vacancy
.findById(req.params.id)
.then(vacancy => res.status(200).json(vacancy))
.catch(next)
}
//tested in insomnia - works
function createRoute(req, res) {
Vacancy
.create(req.body)
.then(vacancy => res.status(201).json(vacancy))
.catch(err => res.status(422).json(err))
}
module.exports = {
index: indexRoute,
show: showRoute,
create: createRoute
}
and lastly, environment file:
const port = process.env.PORT || 4000
const dbURI = process.env.MONGODB_URI || `mongodb://localhost/dxw-job-board-${process.env.NODE_ENV || 'dev'}`
module.exports = { port, dbURI }
This is for a code test for a job I really want to super anxious about it not working properly - any help would be greatly appreciated!
I ran into this and pulled my hair out when I first tried to deploy an app on Heroku (was my first experience with any sort of deployment actually). On the production server (Heroku) you should be serving the static files from the build directory, not the /public directory.
Something like this:
if (process.env.NODE_ENV === "production") {
// react build directory /public
} else {
// Local production env react /public dir
}
This should tell Node that it's on Heroku (or whatever platform you use) and needs to use the production build and not the development version.
Hope that solves the issue for you.

"window is not defined" in Angular service yet code works perfectly

I'm using indexeddb in an Angular 8 service and need window. The code builds without errors and the app creates the db objectstore flawlessly. But at runtime in production mode (with an actual node server instead of ng serve where this error does not occur), I get this error in the terminal running angular:
ERROR ReferenceError: window is not defined
at IndexedDBService.isSupported (D:\MartijnFiles\Documents\Programming\Fenego\fenego-labs-angular\dist\server.js:71199:9)
at IndexedDBService.openDB (D:\MartijnFiles\Documents\Programming\Fenego\fenego-labs-angular\dist\server.js:71203:18)
at Promise (D:\MartijnFiles\Documents\Programming\Fenego\fenego-labs-angular\dist\server.js:72026:46)
Again, it all works and the isSupported() function would stop openDB() from being run if window was actually undefined. There is also no error in the browser console.
Here is the relevant part of my service.
#Injectable()
export class IndexedDBService {
isSupported(): boolean {
return !!window.indexedDB;
}
openDB(dbName: string,
version: number,
onUpgradeNeededCallback: OnUpgradeNeededCallback,
onSuccessCallback: OnOpenSuccessCallback,
onErrorCallback: OnOpenErrorCallback,
onBlockedCallback: OnOpenBlockedCallback): Observable<IDBOpenDBRequest> {
let openDBRequest: IDBOpenDBRequest = null;
if (this.isSupported()) {
openDBRequest = window.indexedDB.open(dbName, version);
openDBRequest.onupgradeneeded = onUpgradeNeededCallback;
openDBRequest.onsuccess = onSuccessCallback;
openDBRequest.onerror = onErrorCallback;
openDBRequest.onblocked = onBlockedCallback;
}
return of(openDBRequest);
}
There are many suggest "solutions" out there that mostly boil down to providing it via a service or plain injection (eg. point 1 in this blog https://willtaylor.blog/angular-universal-gotchas/) but all it does is pass window from some other service via injection to mine. But my code works so it clearly has access to window...
Update:
The following line in a component's ngOnInit() has the same problem with Worker being "not defined" yet the worker is loaded and runs perfectly:
const offlineProductsWorker = new Worker('webworkers/offline-products-worker.js');
Update2:
I have found a solution (posted below) but checking for server side rendering seems more like a workaround than solving the fact that server side rendering is happening (not sure if that is supposed to be the case).
I will include my server.ts script that I use with webpack below. It is a modification of one from another project and I don't understand most of it. If anyone can point out to me what I could change to stop the server side rendering, that would be great. Or, if it is supposed to do that then why?
// tslint:disable:ish-ordered-imports no-console
import 'reflect-metadata';
import 'zone.js/dist/zone-node';
import { enableProdMode } from '#angular/core';
import * as express from 'express';
import { join } from 'path';
import * as https from 'https';
import * as fs from 'fs';
/*
* Load config from .env file
*/
require('dotenv').config({ path: './ng-exp/.env' });
const IS_HTTPS = process.env.IS_HTTPS === 'true';
const SSL_PATH = process.env.SSL_PATH;
const ENVIRONMENT = process.env.ENVIRONMENT;
// Faster server renders w/ Prod mode (dev mode never needed)
enableProdMode();
const logging = !!process.env.LOGGING;
// Express server
const app = express();
const PORT = process.env.PORT || 4200;
const DIST_FOLDER = process.cwd();
// * NOTE :: leave this as require() since this file is built Dynamically from webpack
const { AppServerModuleNgFactory, LAZY_MODULE_MAP } = require('./dist/server/main');
// Express Engine
import { ngExpressEngine } from '#nguniversal/express-engine';
// Import module map for lazy loading
import { provideModuleMap } from '#nguniversal/module-map-ngfactory-loader';
// Our Universal express-engine (found # https://github.com/angular/universal/tree/master/modules/express-engine)
app.engine(
'html',
ngExpressEngine({
bootstrap: AppServerModuleNgFactory,
providers: [provideModuleMap(LAZY_MODULE_MAP)],
})
);
app.set('view engine', 'html');
app.set('views', join(DIST_FOLDER, 'ng-exp'));
// Server static files from /browser
app.get(
'*.*',
express.static(join(DIST_FOLDER, 'ng-exp'), {
setHeaders: (res, path) => {
if (/\.[0-9a-f]{20,}\./.test(path)) {
// file was output-hashed -> 1y
res.set('Cache-Control', 'public, max-age=31557600');
} else {
// file should be re-checked more frequently -> 5m
res.set('Cache-Control', 'public, max-age=300');
}
},
})
);
// ALl regular routes use the Universal engine
app.get('*', (req: express.Request, res: express.Response) => {
if (logging) {
console.log(`GET ${req.url}`);
}
res.render(
'index',
{
req,
res,
},
(err: Error, html: string) => {
res.status(html ? res.statusCode : 500).send(html || err.message);
if (logging) {
console.log(`RES ${res.statusCode} ${req.url}`);
if (err) {
console.log(err);
}
}
}
);
});
const sslOptions = {
key: fs.readFileSync(`${SSL_PATH}/${ENVIRONMENT}/server.key`),
cert: fs.readFileSync(`${SSL_PATH}/${ENVIRONMENT}/server.crt`),
};
// Start up the Node server
let server;
if (IS_HTTPS) {
server = https.createServer(sslOptions, app);
} else {
server = app;
}
server.listen(PORT, () => {
console.log(`Node Express server listening on http${IS_HTTPS ? 's' : ''}://localhost:${PORT}`);
const icmBaseUrl = process.env.ICM_BASE_URL;
if (icmBaseUrl) {
console.log('ICM_BASE_URL is', icmBaseUrl);
}
});
There is a related issue here:
https://github.com/hellosign/hellosign-embedded/issues/107
Basically, to avoid the error you can declare somewhere globally the window.
if (typeof window === 'undefined') {
global.window = {}
}
I also found React JS Server side issue - window not found which explains the issue better and why it works on the client side.
I found the solution thanks to some input from ChrisY
I deploy my code using webpack and run it using node. It seems that node somehow renders it server side and then the browser renders it too. The server site portion has no effect on the storefront but does cause the (seemingly harmless) error. In isSupported() I added console.log(isPlatformBrowser(this.platformId))and it printed false in the server terminal but true in the browser. Thus, I changed the code as follows:
constructor(#Inject(PLATFORM_ID) private platformId: any) {}
isSupported(): boolean {
return isPlatformBrowser(this.platformId) && !!indexedDB;
}
Now it still works in the browser as it did before but there is no server error.
Update:
I have also found the cause for the server side rendering. The server.ts file in the description has a block with res.render(. This first renders the page on the server and if it does not receive html, it returns status code 500. Otherwise it allows the client to render it. Seeing as this is a realistic scenario, I have decided to keep the extra isPlatformBrowser(this.platformId) check in my code. This should then be done for anything that can only be performed by the client (window, dom, workers, etc.).
Not not have server side rendering, an alternative to the res.render( block is
res.status(200).sendFile(`/`, {root: join(DIST_FOLDER, 'ng-exp')});

Nodejs Api call in background

I'm developing an app with node JS, the app generates a report calling the endpoint api.example.com/generate-report
But this report takes around 1 minute on be generated, then I want to implement something like this:
User click on generate report
System return response {generating:"ok"}
After the system generate the report send a notification (this I what I know how to do)
User get the report
Is this possible with nodejs?
After I do some research, this can be easily done using Promises.
To run the following code it's necessary to install express and node uuid
npm install --save express
npm install --save uuid
node index.js
The source code of index is:
//index.js
const express = require("express");
const app = express();
const PORT = process.env.PORT || 5000;
const uuidV1 = require('uuid/v1');
// this is where we'll store the results of our jobs by uuid once they're done
const JOBS = {};
app.get("/", (req, res) => {
res.send("It works!");
});
app.get("/startjob", (req, res) => {
let times = [100, 1000, 10000, 20000];
let promises = [];
for (let time of times) {
promises.push(new Promise((resolve, reject) => {
setTimeout(resolve, time, `${time} is done.`);
}));
}
// obviously, you'd want to generate a real uuid here to avoid collisions
let uuid = uuidV1();
console.log(uuid);
Promise.all(promises).then(values => { JOBS[uuid] = values; });
res.redirect(`progress/${uuid}`);
});
app.get("/progress/:uuid", (req, res) => {
if (JOBS[req.params.uuid] === undefined) {
res.send("Still processing your request.");
} else {
res.send(`Here's your result: ${JOBS[req.params.uuid]}.`);
// instead of immediately deleting the result of the job (and making it impossible for the user
// to fetch it a second time if they e.g. accidentally cancel the download), it would be better
// to run a periodic cleanup task on `JOBS`
delete JOBS[req.params.uuid];
}
});
app.listen(PORT, () => {
console.log(`Listening on localhost:${PORT}.`);
});
When the code runs you will be redirected to /process/uuid and I get the status of the process.
This needs some improvements because I want the response like "{process:uuid}" and I can store this on my Local Storage to use after.
Well, I hope this help to someone.

Resources