I've a project with the following folder structure:
enter image description here
It has a firebase cloud functions folder as well as a react app.
Here's the code for the index.js file in the functions folder:
const functions = require("firebase-functions");
const { exec } = require("child_process");
const util = require("util");
const execPromise = util.promisify(exec);
exports.installModules = functions.https.onCall((data, context) => {
return new Promise((resolve, reject) => {
console.log("Starting install modules process");
exec(
"npm i",
{
cwd: "../react-app",
},
(err, stdout, stderr) => {
if (err) {
console.error("Error running build command:", err);
reject(err);
}
console.log("Modules installed successfully with output:", stdout);
resolve("Modules installed successfully");
}
);
});
});
exports.build = functions.https.onCall((data, context) => {
return new Promise((resolve, reject) => {
console.log("Starting build process...");
exec(
"npm run build",
{
cwd: "../react-app",
},
(err, stdout, stderr) => {
if (err) {
console.error("Error running build command:", err);
reject(err);
}
console.log("Build completed successfully with output:", stdout);
resolve("Build completed successfully");
}
);
});
});
The first function (installModules) installs modules inside the react app. The second function (build) makes a build folder for the react app. Both of these functions work fine when testing them with the firebase functions shell (with the command firebase functions:shell, and then nameOfFunction({}).
However, when running deploying these to the cloud I get the following error when calling them from a frontend.
**severity: "ERROR"
textPayload: "Unhandled error Error: spawn /bin/bash ENOENT
at Process.ChildProcess._handle.onexit (node:internal/child_process:285:19)
at onErrorNT (node:internal/child_process:485:16)
at processTicksAndRejections (node:internal/process/task_queues:83:21) {
errno: -2,
code: 'ENOENT',
syscall: 'spawn /bin/bash',
path: '/bin/bash',
spawnargs: [ '-c', 'npm i' ],
cmd: 'npm i'
}**
I've tried running these in an express server deployed to both Google Cloud and Heroku but was running into too many issues which is why I decided to give firebase cloud functions a try. According to this post it is possible to run npm commands inside of a google function.
Thanks, any help is appreciated!
What you're trying to do isn't possible. You can't dynamically install modules or do anything at all to modify the filesystem that was created for your function (it is a read-only docker image). From the documentation:
The function execution environment includes an in-memory file system that contains the source files and directories deployed with your function (see Structuring source code). The directory containing your source files is read-only, but the rest of the file system is writeable (except for files used by the operating system). Use of the file system counts towards a function's memory usage.
The question you linked does not actually say that it's possible to run npm commands in Cloud Functions. It just says that it's possible to spawn commands (typically that you provide in your own deployment).
If you want to run arbitrary commands to build a filesystem that you can use to execute more programs, Cloud Functions is not the right product for your use case. If you are just trying to build some software and deploy it somewhere, maybe Cloud Build is what you need as part of your deployment pipeline.
The issue I see is you are trying to execute /bin/bash shell scripting through exec within a Firebase function. Although it may be possible, I think you've outgrown Firebase in this regard and may want to look into using Cloud Run which has full access to shell without restriction and can still communicate with your GCFs (Firebase functions).
Creating a Cloud Run service is really easy with the pre-made container images you can use in the Google Cloud Console GUI so you can be up and running quickly.
Related
I am working on an app wherein I would like to be able to install NPM modules via the frontend. I have no idea, though, how to do so. That is, I know how to do CRUD actions via the front end, but I don't know how to either interact with the command line or run command line functions via the front end.
Are there packages that can help with this or is this built into Node.js somehow?
In short, how can I connect my front-end to my backend in such a way that I can install an NPM package?
What you want is the child_process module. It's built-in so you don't need to install any additional module.
Mostly what you're looking for is either spawn() or exec().
For example, if you want to run npm install some_module you can do:
const { exec } = require('child_process');
let command = 'npm install some_module';
let options = { cwd: '/path/to/node/project' };
exec(command, options, (error, stdout, stderr) => {
// Do anything you want with program output here:
console.log('output:', stdout, stderr);
});
You may check the documentation for child_process in Node JS:
Child Process
const { spawn } = require('child_process');
const ls = spawn('ls', ['-lh', '/usr']);
ls.stdout.on('data', (data) => {
console.log(`stdout: ${data}`);
});
ls.on('close', (code) => {
console.log(`child process exited with code ${code}`);
});
The key difference between exec() and spawn() is how they return the data. As exec() stores all the output in a buffer, it is more memory intensive than spawn(), which streams the output as it comes.
Generally, if you are not expecting large amounts of data to be returned, you can use exec() for simplicity. Good examples of use-cases are creating a folder or getting the status of a file. However, if you are expecting a large amount of output from your command, then you should use spawn()
The easiest way to install an npm package via "the front end" is to have node spawn npm as a child process based off of the package name that the client provides.
var child = require('child_process').exec(`npm i ${package_name}`);
child.on('exit',function(){
//npm finished
});
This should install the module given that package_name is the name of the npm package, in the same directory that the script is running in. In terms of getting the package name from the front end to the back end, there are several different ways to do that.
You cannot run the your backend application without NPM modules installed, one thing that I think you can do is to make a plain nodejs file without any modules, which will receive the args when invoked, and you can use that args to the required modules, because that file will run with just core modules
If I try to run my sequelize migrations and then run my Node server in the same command, I run into the issue of my server never starting up. If the migrations have already been run before, the sequelize db:migrate command doesn't go past the "No migrations were executed, database schema was already up to date." message, and my second command is never able to run. If the migration has not run before, everything runs properly in sequence.
This is my npm start command: sequelize db:migrate && node index.js
I assume that internally sequelize db:migrate is not resolving anything in the case where this log message is shown, so is there a way I can "terminate" this command after some time and proceed to my node command?
For anyone else running into this issue, this is how I ended up solving it.
1) Create a new file that you will run in your npm script.
2) I ended up wrapping the process call in a child_process exec, and then terminated the process when I received the above console.log message since the library itself does not resolve anything at this point.
// myRuntimeFile.js --> Make sure this file is in the same directory where your .sequelizerc file lives
(async()=> {
const { exec } = require('child_process');
await new Promise((resolve, reject) => {
const migrate = exec(
'sequelize db:migrate',
{ env: process.env },
(err, stdout, stderr) => {
resolve();
}
);
// Listen for the console.log message and kill the process to proceed to the next step in the npm script
migrate.stdout.on('data', (data) => {
console.log(data);
if (data.indexOf('No migrations were executed, database schema was already up to date.') !== -1) {
migrate.kill();
}
});
});
})();
Obviously the above code is not ideal, but hopefully this is just temporary until the internals of this edge case are resolved properly in a promise.
3) Update your npm script with the following:
"start": "node myRuntimeFile.js && node index.js"
Or if you are running on a Windows machine and cannot use &&, you can use the npm-run-all library.
I have a Node Express server that works on localhost. It uses child_process to run a C++ standalone executable.
The code that uses child_process is the following (the application creates output.txt):
app.post('/generate', async function(req, res)
{
var input1 = req.body.input1;
var input2 = req.body.input2;
var execFile = require('child_process').execFile;
var program = "path/to/executable";
var args = [input1, input2];
var child = execFile(program, args,
function (error, stdout, stderr){
console.log(error);
console.log(stdout);
console.log(stderr);
const file = __dirname + "/output.txt"
app.get('/output.txt', function (req, res) {
res.sendFile(path.join(__dirname + '/output.txt'));
});
res.send("/output.txt");
})
})
This works locally.
I'm now trying to deploy it on Google Cloud Platform with App Engine.
However, when I go the website that I host, and launch this POST /generate request, I don't get the expected output. In the Google Cloud Platform logs of my project I can see the following error:
textPayload: "{ Error: spawn cpp/web_cpp_app/x64/Debug/web_cpp_app ENOENT
at Process.ChildProcess._handle.onexit (internal/child_process.js:240:19)
at onErrorNT (internal/child_process.js:415:16)
at process._tickCallback (internal/process/next_tick.js:63:19)
"
At first I didn't understand the error, but now I can see that if I locally run the same project, but set the path of the standalone executable to an invalid path, I get the same error. I'm guessing that when I deploy, my executable is somehow not included?
Is there something specific I need to add in package.json or app.yaml files, to include the executable?
EDIT:
Could it be that the app engine runs on Linux, and my executable is for Windows?
ENOENT means "no such file or directory", so your path could be wrong, or the container doesn't recognize the program as executable.
But either way, you will need to build and include a linux-compatible binary of your child_process program in your project directory when you deploy. You could build this manually, or use something like Cloud Build to compile it in a container that's identical to that of App Engine.
You are right about the OS, per this Doc Appengine standard for NodeJS uses Ubuntu OS, and flexible uses Debian
About the executable compatibility I found this post
I use vue-cli in my dockerized project, where port mapping looks like this: "4180:8080", and the actual message after compiling my SPA looks like:
App running at:
- Local: http://localhost:8080/app/
It seems you are running Vue CLI inside a container.
Access the dev server via http://localhost:<your container's external mapped port>/app/
App works fine, I could access at via http://localhost:4180/app/ as conceived, but I'm not able to find a proper way to change the message above to show this link instead of "It seems you are running Vue CLI inside a container...". I could use webpack hooks to insert link before the message but actually wanna find the way to change the message, generated by cli. Is it possible somehow?
I came to this question - as I was looking to do the same thing with bash, running inside a Docker container (possibly what you're already doing).
You could achieve this by invoking Vue CLI commands through spawning a child node process, from within your Docker container (assuming your container is running node). You can then modify the output of stdout and stderr accordingly.
You can call a Javascript file in one of two ways:
use a shell script (bash, for example) to call node and run this script
set the entrypoint of your Dockerfile to use this script (assuming you're running node by default)
// index.js
const { spawn } = require('child_process')
const replacePort = string => {
return string.replace(`<your container's external mapped port>`, 8000)
}
const vueCLI = (appLocation, args) => {
return new Promise((resolve, reject) => {
const vue = spawn('vue', args, {cwd: appLocation})
vue.stdout.on('data', (data) => {
console.log(replacePort(data.toString('utf8', 0, data.length)))
})
vue.stderr.on('data', (error) => {
console.log(replacePort(error.toString('utf8', 0, error.length)))
})
vue.on('close', (exitCode) => {
if (exitCode === 0) {
resolve()
} else {
reject(new Error('Vue CLI exited with a non-zero exit code'))
}
})
})
}
vueCLI('path/to/app', CLI_Options).then(() => resolve()).catch(error => console.error(error))
This approach does have drawbacks, not limited to:
performance being slower - due to this being less efficient
potential danger of memory leaks, subject to implementation
risk of zombies, should the parent process die
For the reasons above and several others, this is a route that was found to be unsuitable in my specific case.
Instead of changing the message, it's better to change the port Vue is listening on.
. npm run serve -- --port 4180
This automatically updates your message to say the new port, and after you updated your docker port forward for the new port, it it will work again.
I have a large project which contains multiple node application endpoints, each with their own package.json file.
I have a main build script (written in jake) which sets up a given environment, runs tests, packages apps etc.
So is there a way for the root build script to run "npm install" on the given directories.
I expect psudo code would be:
var npm = require("npm");
var package1Directory = "some-directory";
npm.install(packageDirectory);
Cannot find any documentation around this though so not sure if it is possible... so is it?
Yes, have a look at the docs:
var npm = require("npm")
npm.load(myConfigObject, function (er) {
if (er) return handlError(er)
npm.commands.install(["some", "args"], function (er, data) {
if (er) return commandFailed(er)
// command succeeded, and data might have some info
})
npm.on("log", function (message) { .... })
})
Also have a look at this example, which gives some more insights on how to use npm programmatically.