I've written a Casperjs script to do some scraping, however, after running into memory exhaustion issues, I've now written a node script to turn Phantom off and on via exec. I can run this with no issues locally, however when I deploy to heroku I get the following error
Error: Command failed: casperjs turnitoffandon.js
ERROR: stderr was:/bin/sh 1: casperjs: not found
I used the nodejs buildpack and have Phantom and Casper defined in my dependencies. In the heroku bash, running phantomjs --version returns 2.1.1 and casperjs --version returns 1.1.4.
Do I need to define where Casper is? If so how? I have set my PATH variable as /usr/local/bin:/usr/bin:/bin:/app/vendor/phantomjs/bin:/app/vendor/casperjs/bin:/node_module/casperjs/bin , like in this SO question
The issue actually had nothing to do with Heroku. As noted in this SO answer, using exec and providing any environmental variables in the options parameter replaces the entire set of environment variables. This includes the path, effectively overwriting any paths already specified to Heroku in the buildpack and npm modules.
You can create a copy of process.env and pass that along in the parameters in addition to other environmental parameters needed to be passed.
Related
Can I find a way to show the parameters of node.js interpreter like max-old-space-size in the node app itself? So that I can show them when I am running the program
Either check process.argv if passed as arguments or process.config:
The process.config property returns an Object containing the
JavaScript representation of the configure options used to compile the
current Node.js executable. This is the same as the config.gypi file
that was produced when running the ./configure script.
Situation
I have an Alpine/NodeJS Docker image running my app (Alpine Linux 3.11, nodeJS v12.15.0), and I recently needed to internationalize currencies in this app.
I noticed that my container was missing locales, so I needed to install full-ICU. Consequently I modified my Alpine-based Docker image to add two lines to install full-ICU:
RUN npm i -g full-icu
ENV NODE_ICU_DATA=“/home/node/.npm/lib/node_modules/full-icu”
The installation went smoothly, the console output said:
Step 10/15 : RUN npm i -g full-icu
---> Running in b14d826c8689
/home/node/.npm/bin/node-full-icu-path -> /home/node/.npm/lib/node_modules/full-icu/node-icu-data.js
> full-icu#1.3.1 postinstall /home/node/.npm/lib/node_modules/full-icu
> node postinstall.js
npm install icu4c-data#64l (Node 12.15.0 and small-icu 64.2) -> icudt64l.dat
full-icu$ /usr/bin/node /home/node/.npm/lib/node_modules/npm/bin/npm-cli.js install icu4c-data#64l
+ icu4c-data#0.64.2
added 1 package from 1 contributor in 62.073s
√ icudt64l.dat (link)
Node will use this ICU datafile if the environment variable NODE_ICU_DATA is set to “/home/node/.npm/lib/node_modules/full-icu”
or with node --icu-data-dir=/home/node/.npm/lib/node_modules/full-icu YOURAPP.js
For package.json:
{"scripts":{"start":"node --icu-data-dir=/home/node/.npm/lib/node_modules/full-icu YOURAPP.js"}}
By the way, if you have full data, running this in node:
> new Intl.DateTimeFormat('es',{month:'long'}).format(new Date(9E8));
... will show “enero”. If it shows “January” you don't have full data.
News: Please see https://github.com/icu-project/full-icu-npm/issues/6
+ full-icu#1.3.1
added 1 package from 1 contributor in 63.276s
It seems fine, it recognized my NodeJS version, there were no errors. I could check and see that the ICU data files were at the right place.
Problem
But when opening a shell inside this container (running docker run -ti myimage sh), and playing with Intl, I noticed that the locales were working properly only when running node with the --icu-data-dir option, not when using the NODE_ICU_DATA environment variable.
However, my preference definitely goes to the environment variable, for various reasons, so I would have liked it to work.
Tests so far
Here are my tests with node:
node --icu-data-dir=/home/node/.npm/lib/node_modules/full-icu
Welcome to Node.js v12.15.0.
Type ".help" for more information.
> new Intl.DateTimeFormat('es',{month:'long'}).format(new Date(9E8));
'enero'
It's saying "enero", so it's working. It means that full-ICU is properly installed and reachable.
export NODE_ICU_DATA=“/home/node/.npm/lib/node_modules/full-icu”
node
Welcome to Node.js v12.15.0.
Type ".help" for more information.
> new Intl.DateTimeFormat('es',{month:'long'}).format(new Date(9E8));
'January'
It doesn't care about my environment variable (also tried putting the environment variable in the Dockerfile, as shown above)
env NODE_ICU_DATA=“/home/node/.npm/lib/node_modules/full-icu” node
Welcome to Node.js v12.15.0.
Type ".help" for more information.
> new Intl.DateTimeFormat('es',{month:'long'}).format(new Date(9E8));
'January'
It also doesn't care when the environment variable is inlined.
I also tried with .js scripts by the way, not just the NodeJS console, and also various ways to pass the environment variable.
And just to be sure, I tried to install system ICU packages, with RUN apk --update add --no-cache icu icu-libs icu-dev.
So...
Would anyone have an idea about the reason why specifying the path in an environment variable doesn't work, and what I should check?
I have the same issue with on a VPS with Plesk.
I can't update the Node version and the installed Node version is the, v12.4.0.
In my case I can't also install full-icu as global module, and the process manager start my app without run the start script in the package.json.
In this situation for me, the only way for load the full-icu support is to use the environment vars.
I tried first via command line:
export NODE_ICU_DATA=/full/path/of/my/app/node_modules/full-icu
and then
-bash-4.2$ /opt/plesk/node/12/bin/node
Welcome to Node.js v12.4.0.
Type ".help" for more information.
> new Intl.DateTimeFormat('it',{month:'long'}).format(new Date(9E8));
'gennaio'
>
it works fine.
It works too if I run this command in the root folder of my app:
NODE_ICU_DATA=node_modules/full-icu /opt/plesk/node/12/bin/node
So I added an ENV var on the app node setting (in Plesk)
NODE_ICU_DATA
with a value
node_modules/full-icu
Restarting the app the i18n support rightly works.
I hope this could help others people there are in my same situation.
For anyone falling on this issue: the problem was version-specific.
Deploying a more recent NodeJS version fixed these ICU bugs.
I'm trying to access items in my PATH environment variable from my electron instance, when I run it with npm start while developing it through node.js I get all the expected variables, but when I run the electron application with my resources inside I'm left with only usr/bin
This is how it looks like when I run it from npm:
And this is how it looks when run from the electron mac application precompiled:
Does anyone know why this could be the case? And if I can do anything to reach my normal PATH variables
UPDATE:
After a lot of research I found out that GUI applications ran from finder or docker in Mac OSX use different environment variables compared to if they are ran from the terminal:
This can be edited through plist files, either globally or application specific
You can use fix-path package. Works perfectly!
const fixPath = require('fix-path');
console.log(process.env.PATH);
//=> '/usr/bin'
fixPath();
console.log(process.env.PATH);
//=> '/usr/local/bin:/usr/bin...'
Node.js and the express generator are really handy and simple to understand. I cannot however get my server to start by running c:\my-application-root>DEBUG=my-application ./bin/www
Windows doesn’t seem to understand that command. I can however run it by calling node ./bin/www
Am i missing something?
Did you try set DEBUG=my-application followed by node ./bin/www? Windows does not allow setting an environment variable in the way that Linux and others do.
1st command 'set DEBUG=my-application'.2nd command 'npm start'
First you need to go the cmd that is supported by node (search for node cmd when you click in windows icon if you're using windows 7)
type
set DEBUG=my-application
Second
simply cd c:\my-application\bin\
Then type
node www
www is the file that contains the script needed by node to run your server, DEBUG if set as an environment variable will also help you run node against it since the path will be known
I use this buildpack in order to use casperjs on heroku.
Once my app created, I check my PATH
$ heroku config:get PATH
/usr/local/bin:/usr/bin:/bin:/app/vendor/phantomjs/bin:/app/vendor/casperjs/bin
OK, casperjs is in here!
--
Procfile
scraper: node scraper.js
scraper.js
var exec = require('child_process').exec;
exec('casperjs casper-script.js');
--
But when I launch the scraper process, it crashes with the following logs:
2012-10-09T02:23:38+00:00 heroku[scraper.1]: Starting process with command `node scraper.js`
2012-10-09T02:23:39+00:00 app[scraper.1]: bash: node: command not found
Why exec does not find casperjs which is in the PATH ?
PS: I tried with spawn but no more luck...
[EDIT]: a testable gist here https://gist.github.com/3856317
Got it!
#vinayr: you were right, neither exec nor casperjs command was involved in this issue, it was node which was the command not found!
I thought installing a buildpack with phantomjs and casperjs (eg: heroku-buildpack-casperjs) would have kept node capabilities for my app, but it does not seem so.
To make it work (node+phantomjs+casperjs) I rather had to fork the heroku-buildpack-nodejs buildpack and add phantomjs/casperjs binaries to it, which I could have called: heroku-buildpack-nodejs-casperjs.
In other words, when using a custom buildpack, you have to include the node support yourself .