Let's say we have a .env file with some variables specified:
AWS_PROFILE=hsz
ENVIRONMENT=development
There is also a simple npm task defined:
{
"name": "project",
"version": "0.0.1",
"scripts": {
"deploy": "sls deploy"
}
}
But runnning npm run deploy ignores our .env definition.
It can be resolved with better-npm-run like:
{
"name": "project",
"version": "0.0.2",
"scripts": {
"deploy": "bnr deploy"
},
"betterScripts": {
"deploy": "sls deploy"
},
"devDependencies": {
"better-npm-run": "^0.1.1",
}
}
but this looks like an overhead - especially when we have 10+ tasks.
Is there a better way to always load .env without proxying all tasks via better-npm-run?
A bit ugly, but you could try something like this:
"scripts": {
"deploy": "export $(cat .env | xargs) && sls deploy"
}
This will export all environment variables from the .env file before running sls deploy.
There are some variations to this tehnique in this answer.
Not very clean but it avoids usage of an extra module.
You can use env-cmd npm package to set environment variables loaded from .env file before executing a npm script.
Add package to your package.json devDependencies:
npm i env-cmd -D
Prefix your npm script with env-cmd program in package.json:
{
"scripts": {
"deploy": "env-cmd sls deploy"
}
}
Maintain and load all your environment specific configuration in project itself.
dev.js
module.exports = {
"host":"dev.com"
}
prod.js
module.exports = {
"host":"prod.com"
}
config.js - main file that will resolve configuration based on process.env.ENV variable.
const dev = require('./dev');
const prod = require('./prod');
let envObject = {};
const env = process.env.ENV || "dev";
switch(env) {
case 'prod':
envObject = prod;
break;
default:
envObject = dev;
}
envObject['ENV'] = env;
process.env = Object.assign(process.env,envObject); // Optional if you prefer to add them into process environment otherwise `require('./config')` where you need configuration.
module.exports = envObject;
index.js - node project root file call every time when project start
const config = require('./config');
console.log('config object => ',config.host);
package.json
{
"name": "project",
"version": "0.0.2",
"scripts": {
"deploy": "sls deploy"
}
}
Running you node.js code
Prod environment ENV=prod npm run deploy;
Development environment - npm run deploy;
Default environment is set to dev in ./config.js
Using this simple practice you don't need any npm module to manage your environment configurations.
I was having the same issue while trying to syncing the DB using an external command and fixed the issue by requiring dotenv package which will load the variables
"scripts": {
"db-sync": "node --require dotenv/config ./src/sequelize/sync.js"}
then just call npm run db-sync
Related
I have an NX workspace with a single application called my-app. I would like to run Playwright tests for my-app application by using NX console. Currently NX doesn't support Playwright plugin, so I've created a custom NX executor according to this tutorial. I've created necessary files for executor. After, I registered custom e2e command in application's project.json file. The playwright configuration file stays in the my-app folder.
When I run nx run my-app:e2e, the executor is been executed, however for some reason, playwright doesn't start. Instead, I see an error.
When I run manually in the console the command triggered by nx run my-app:e2e which is npx playwright test --config=apps/my-app/playwright.config.ts the playwright starts and does necessary testing.
project.json
...
...
...
"e2e": {
"executor": "./tools/executors/playwright:playwright",
"options": {
"path": "apps/my-app/playwright.config.ts"
}
}
executor.json
{
"executors": {
"playwright": {
"implementation": "./impl",
"schema": "./schema.json",
"description": "Runs Playwright Test "
}
}
}
impl.ts
export default async function echoExecutor(
options: PlaywrightExecutorOptions,
context: ExecutorContext
) {
console.info(`Executing "Playwright"...`);
console.info(`Options: ${JSON.stringify(options, null, 2)}`);
const { stdout, stderr } = await promisify(exec)(
`npx playwright test --config=${options.path}`,
);
console.log(stdout);
console.error(stderr);
const success = !stderr;
return { success };
}
schema.json
{
"$schema": "http://json-schema.org/schema",
"type": "object",
"cli": "nx",
"properties": {
"path": {
"type": "string",
"description": "Path to the project"
}
}
}
package.json
{
"executors": "./executor.json"
}
I'm not sure but maybe the problem is in promisify? I'm trying to call npx with it. Maybe there is a different way to call npx in this context?
const { stdout, stderr } = await promisify(exec)(
`npx playwright test --config=${options.path}`,
);
I will recommend https://github.com/marksandspencer/nx-plugins/tree/main/packages/nx-playwright
You will need to install the plugin using
yarn add --dev #mands/nx-playwright
yarn playwright install --with-deps
Remove existing e2e app nx generate remove <APP-NAME>-e2e before generating a new one
Generate new e2e app
yarn nx generate #mands/nx-playwright:project <APP-NAME>-e2e --project <APP-NAME>
PS: I am also author of the plugin
I would suggest to use https://github.com/marksandspencer/nx-plugins/tree/main/packages/nx-playwright
It should work out of the box
Try to use pnpm nx ...
Firstly, in package.json define:
"scripts": {
"test": "playwright test --output build --workers 2",
"test:debug": "playwright test --output build --debug",
"test:headed": "playwright test --output build --headed",
"test:codegen": "playwright codegen https://xxx.xxx.com/ -o records.test.ts",
"test:codegen-json": "playwright codegen https://xxx.xxx.com/ --save-storage=storage/auth_1.json",
"test:api": "playwright test src/e2e-api/ --retries 0 --output build "
}
Then, in common line:
pnpm nx test e2e-tests --skip-nx-cache
or
pnpm nx test:debug e2e-tests --skip-nx-cache
or
pnpm nx test:headed e2e-tests --skip-nx-cache
e2e-tests is a e2e destination folder where the package.json is located
--skip-nx-cache - to skip nx cache output
This command line you can use in any place, for example in gitlab-ci.yaml
In my Express/React app, I am using Webpack to handle server-side rendering. However, I am experiencing a build error related to environment variables that I'm trying to access in my Express server script.
In the server script, index.js, I am setting a few variables like so:
const gitCommit = process.env.GIT_COMMIT || require("./gitignore/git_commit.js"),
buildDate = process.env.BUILD_DATE || require("./gitignore/build_date.js")
And since I am running a test production build on my local machine, I delete the gitignore/ directory and set those environment variables:
$ export GIT_COMMIT="test commit hash"
$ export BUILD_DATE="test build date"
Then I npm run build, which executes the following scripts:
"build:client": "webpack --config webpack.config.js",
"build:server": "webpack --config webpack.server.config.js",
"build": "npm run build:client && npm run build:server"
build:client executes with no problem, but build:server throws errors...
ERROR in ./index.js
Module not found: Error: Can't resolve './gitignore/git_commit.js' in '/Users/filepath'
# ./index.js 12:38-74
ERROR in ./index.js
Module not found: Error: Can't resolve './gitignore/build_date.js' in '/Users/filepath'
# ./index.js 13:42-78
implying that the two environment variables referenced in index.js can't be found, and so it's looking for the gitignore/ instead, which shouldn't exist (I mean, it does exist locally, but I've deleted since I'm simulating a production build).
Here is the complete webpack.server.config.js:
const fs = require("fs"),
path = require("path")// ,
// ExtractTextPlugin = require("extract-text-webpack-plugin")
module.exports = {
"entry": path.resolve(__dirname, "index.js"),
// keep node_module paths out of the bundle
"externals": fs.readdirSync(path.resolve(__dirname, "node_modules")).concat(["react-dom/server", "react/addons"]).reduce((ext, mod) => {
ext[mod] = `commonjs ${mod}`
return ext
}, {}),
"module": {
"loaders": [
{
"exclude": /node_modules/,
"loader": "babel-loader",
"query": { "presets": ["react", "es2015", "stage-2"] },
"test": /\.jsx$/
},
{
"exclude": /node_modules/,
"loader": "babel-loader",
"query": { "presets": ["react", "es2015", "stage-2"] },
"test": /\.js$/
}
]
},
"node": {
"__dirname": true,
"__filename": true
},
"output": {
"filename": "server.bundle.js"
},
"target": "node"
}
Now I expect that gitignore/ would not be found, but what I don't understand is why the two environment variables that I set are not being detected by index.js - they are definitely set in the console before I even run the build command. If I console.log() them in the beginning of webpack.server.config.js, it logs them correctly, and if I run my development version instead (which doesn't use the server config), I can log them correctly in index.js. What gives?
Node version 6.11.1, NPM version 3.10.10, Webpack version 2.6.0.
Your environment variables are only available when Webpack runs, but not when you execute your index.js.
You will need to use the EnvironmentPlugin in your Webpack config like that:
plugins: [new webpack.EnvironmentPlugin(['GIT_COMMIT ', 'BUILD_DATE'])]
That plugin will replace the variables by their actual values.
HINT: Do not use ||. Webpack does not know how to optimize it. Try the ternary operator:
const gitCommit = (process.env.GIT_COMMIT) ? (
process.env.GIT_COMMIT
) : (
require('./gitignore/git_commit.js')
);
Webpack will bundle this to:
const gitCommit = (true) ? (
"test commit hash"
) : (
require('./gitignore/git_commit.js')
);
No IgnorePlugin is needed. Even better, with the UglifyJSPlugin, your code will be optimized to const gitCommit = "test commit hash";. In some cases gitCommit is removed completely as a variable. Its string value will be used instead anywhere where you applied gitCommit.
I'm very new to nodejs.
In my dockerized environment, I want to provide appdynamics support to nodejs apps. This mandates every app to require the following as the first line in their app.
require("appdynamics").profile({
controllerHostName: '<controller host name>',
controllerPort: <controller port number>,
controllerSslEnabled: false, // Set to true if controllerPort is SSL
accountName: '<AppDynamics_account_name>',
accountAccessKey: '<AppDynamics_account_key>', //required
applicationName: 'your_app_name',
tierName: 'choose_a_tier_name',
nodeName: 'choose_a_node_name',
});
I plan to do that by providing a wrapper called appdynamics.js around the app's entry file. Details:
I run a script in my nodejs docker image to replace the entry file name in the app's package.json with "appdynamics.js", where appdynamics.js has the above appdynamics related require statement.
Ex : {scripts { "start" : "node server.js" }} will be replaced with
{scripts { "start" : "node appdynamics.js"}}
Then, i "require" the "server.js" inside appdynamics.js.
Invoke npm start.
My only concern is this:
If the package.json had something like scripts { "start" : "coffee server.coffee" }, my script will replace it to { "start" : "coffee appdynamics.js" }. and then my script will invoke npm start, which will error out.
What is the best way to solve this?
This is a follow up question to Use "coffee" instead of "node" command in production
Write a wrapper called appdynamics.coffee
Compile this wrapper to .js
Replace server.js with appdynamics.js and server.coffee with appdynamics.coffee
After this operations
{
"scripts": {
"start": "node server.js"
}
}
will be
{
"scripts": {
"start": "node appdynamics.js"
}
}
and
{
"scripts": {
"start": "coffee server.coffee"
}
}
will be
{
"scripts": {
"start": "coffee appdynamics.coffee"
}
}
How to use npm scripts and a postinstall hook to display the license of an npm package. Right now I'm doing it with:
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1",
"postinstall": "cat ./MIT-license.txt"
},
on the package.json. But this fails on windows because, well, cat. I know that we can use type on windows to output the contents of a file over the console, but how to do that in an npm script (without failing cat on windows and type on unix/mac)?
If i understand correctly, you need a cross-platform mechanism for logging the contents of a file to the console. I think the easiest way to do this is via a custom Node script, since you know the user will have Node installed, whatever their operating system.
Just write a script like this:
// print-license.js
'use strict';
const fs = require('fs');
fs.readFile('./MIT-license.txt', 'utf8', (err, content) => {
console.log(content);
});
And then, in your package.json:
// package.json
"scripts": {
"postinstall": "node ./print-license.js"
},
Or, if you don't want a serparate script hanging around, this is just about short enough to do inline, like so:
// package.json
"scripts": {
"postinstall": "node -e \"require('fs').readFile('./MIT-license.txt', 'utf8', function(err, contents) { console.log(contents); });\""
},
Update
And now that I think about it, you might be better off with a reusable executable that would allow you to specify a file as a command line argument. That's also very simple:
// bin/printfile
#!/usr/bin/env node
'use strict';
const FILE = process.argv[2];
require('fs').readFile(FILE, 'utf8', (err, contents) => {
console.log(contents);
});
And add the following to your package.json:
// package.json
"bin": {
"printfile": "./bin/printfile"
},
"scripts": {
"postinstall": "printfile ./MIT-license.txt"
}
Because we don't want sensitive data in the project code, including the package.json file, using environment variables would be a logical choice in my opinion.
Example package.json:
"dependencies": {
"accounting": "~0.4.0",
"async": "~1.4.2",
"my-private-module":"git+https://${BB_USER}:${BB_PASS}#bitbucket.org/foo/bar.git"
Is this possible?
The question is not if this is wise or not good, just if it's possible.
In case you use .env file, let's use grep or eval to get a value environment variable from the .env file.
Updated start2 as #Paul suggested:
"scripts": {
"start": "NODE_ENV=$(grep NODE_ENV .env | cut -d '=' -f2) some_script",
"start2": "eval $(grep '^NODE_ENV' .env) && some_script"
}
I have similar but different requirement. For me, I want to use environment variables in the scripts.
Instead of using the environment variables directly in package.json, I do:
"some-script": "./scripts/some-script.sh",
And in some-script.sh:
#!/bin/sh
npm run some-other-script -- --prop=$SOME_ENV_VAR
Here's how I managed to work around package.json to achieve the same purpose. It uses a script that reads from a custom section of package.json for URL modules, interpolates environment variables in them, and installs them with npm install --no-save (the --no-save could be omitted, depending on the usecase).
As a bonus: it tries to read the env variable from .env.json, which can be gitignore'd, and very useful for development.
Create a script that will read from a custom section of package.json
env-dependencies.js
const execSync = require('child_process').execSync
const pkg = require('./package.json')
if (!pkg.envDependencies) {
return process.exit(0)
}
let env = Object.assign({}, process.env)
if (typeof pkg.envDependencies.localJSON === 'string') {
try {
Object.assign(env, require(pkg.envDependencies.localJSON))
} catch (err) {
console.log(`Could not read or parse pkg.envDependencies.localJSON. Processing with env only.`)
}
}
if (typeof pkg.envDependencies.urls === 'undefined') {
console.log(`pkg.envDependencies.urls not found or empty. Passing.`)
process.exit(0)
}
if (
!Array.isArray(pkg.envDependencies.urls) ||
!(pkg.envDependencies.urls.every(url => typeof url === 'string'))
) {
throw new Error(`pkg.envDependencies.urls should have a signature of String[]`)
}
const parsed = pkg.envDependencies.urls
.map(url => url.replace(/\${([0-9a-zA-Z_]*)}/g, (_, varName) => {
if (typeof env[varName] === 'string') {
return env[varName]
} else {
throw new Error(`Could not read env variable ${varName} in url ${url}`)
}
}))
.join(' ')
try {
execSync('npm install --no-save ' + parsed, { stdio: [0, 1, 2] })
process.exit(0)
} catch (err) {
throw new Error('Could not install pkg.envDependencies. Are you sure the remote URLs all have a package.json?')
}
Add a "postinstall": "node env-dependencies.js" to your package.json, that way it will be run on every npm install
Add your private git repos to package.json using the URLs you want (note: they all must have a package.json at root!):
"envDependencies": {
"localJSON": "./.env.json",
"urls": [
"git+https://${GITHUB_PERSONAL_ACCESS_TOKEN}#github.com/user/repo#semver:^2.0.0"
]
},
(the semver specifier #semver:^2.0.0 can be omitted, but refers to a git tag, which can be very useful, as it makes your git server a fully-fledge package manager)
npm install
No, it's not possible. You should access the repo using git+ssh, and store a private key in ~/.ssh.
Your line then looks like:
"my-private-module":"git+ssh://git#bitbucket.org/foo/bar.git"
Which doesn't contain anything sensitive.
No it isn't possible as npm does not treat any string values as any kind of templates.
It may be better to just use git+ssh (if your provider supports it) with an ssh agent.
You can use environment values to inject in your package.json like this:
Any environment variables that start with npm_config_ will be interpreted as a configuration parameter. For example, putting npm_config_foo=bar in your environment will set the foo configuration parameter to bar. Any environment configurations that are not given a value will be given the value of true. Config values are case-insensitive, so NPM_CONFIG_FOO=bar will work the same.
https://docs.npmjs.com/misc/config#environment-variables
I had the same need and my solution was based on #Long Nguyen's response. This way, I can only rely on what's defined on the .env file.
.env
...
SKIP_PREFLIGHT_CHECK=true
...
package.json
...
"scripts": {
"test": "yarn cross-env $(grep SKIP_PREFLIGHT_CHECK ../../.env) react-app-rewired test --watchAll=false"
}
...
You can install package https://www.npmjs.com/package/env-cmd
and all your envs from .env file will be visible
ie:
./.env:
ENV1=THANKS
ENV2=FOR ALL
ENV3=THE FISH
Package.json:
"scripts": {
"test": "env-cmd pact-broker can-i-deploy --broker-token=${ENV1}"
}
or another example from your question:
"my-private-module":"env-cmd git+https://${BB_USER}:${BB_PASS}#bitbucket.org/foo/bar.git"
For complicated environment variables, you can use
https://stedolan.github.io/jq/
to access JSON file (env file at your case)
JSON file could be something like
{
"env" :
{
"username" : "1345345",
"Groups" : [],
"arraytest" : [
{
"yes" : "1",
"no" : "0"
}
]
}
}
so the script could be something like this to access yes value
"scripts": {
"yes": "jq [].arraytest[0].yes?"
}
If you're running node inside a Docker container
Use Docker Compose to inject the env variable
app:
environment:
- NODE_ENV=staging
Run your package.json script from your Dockerfile
CMD [ "npm", "run", "start" ]
Use echo or printenv
"scripts": {
"start": "node -r dotenv/config app.js dotenv_config_path=/run/secrets/$(echo $NODE_ENV)"
"start": "node -r dotenv/config app.js dotenv_config_path=/run/secrets/$(printenv NODE_ENV)"
}
Don't use this for sensitive env variables. It's a really good way to point to a Docker secrets file (like this example shows).