Adding .env vairables to run in for a given command - linux

I have a .env file with variables in it like this:
HELLO=world
SOMETHING=nothing
I found this awesome script the other day, that puts these variables into the current session so when I run something like this
$(cat ./.env | grep -v ^# | xargs) && node test
Then I can access the variables in the test.js node file.
console.log(process.env.HELLO)
console.log(process.env.SOMETHING)
The problem with this is that that command puts the .env variables in the entire session so when I run node test without $(cat ./.env | grep -v ^# | xargs) after I run it, it will still have access to those variables, I'm writing a node.js test that accounts for these variables and I'd love to be able to run the same command with and without these .env variables without worrying if they are still in the session. Ideally I want to run
put-env-variables-for-this-command-first-command node test && node test
And have the env variables only be available in the test script the first time it runs.

This will give you access to the variables during the invocation of node test only
env $(<.env) node test

I started using foreman for this which used to be bundled with the heroku toolbelt. It's not included anymore, however there is a stand-alone version called forego that works really well.
brew install forego on the mac.
Then you can use this to prop-up the .env variables into the current process.
forego run ${ANY COMMAND}
Examples
forego run babel-node index.js
forego run node index.js
forego run npm start

Related

View .env variables of Node.js process

I use package dotenv to set env variables in .env file. Here is the content of .env:
NODE_ENV=development
PORT=4000
Then, I run nodemon --exec babel-node -- src/index.js to start the project.
How to check value of env variables by command line?
I don't want to use console.log in the JS source code, I want something like so (in terminal):
node
> process.env
But it doesn't show the values I have set in .env file.
Could you give me some resolution?
From your console, type following for mac/ubuntu:
echo $NODE_ENV
and following for windows:
echo %NODE_ENV%
If you want to avoid the use of dotenv as sometimes it creates confusion that where to call the dotenv. So, you can use foreman node module and can start your project by command nf start. It will automatically load .env variables into your project without calling dotenv.

What does running node with a module name do?

I'm new to node.js so I'm pretty sure that I am doing something wrong however I would like to understand what is going on.
I tried to run this:
$ node tslint
$
Which output nothing. It didn't fail either.
Checking the help:
$ node --help
Usage: node [options] [ -e script | script.js | - ] [arguments]
node inspect script.js [arguments]
I suspect that I should have passed in the path to the js file instead.
$ node node_modules/tslint/bin/tslint
No files specified. Use --project to lint a project folder.
Sure enough that printed an error since I didn't pass a file name which was expected.
This also works:
$ npx tslint
No files specified. Use --project to lint a project folder.
So my question is, what does node tslint do and why does it neither to find the script nor actually run the script?

Run `source` from package.json to set Enviroment-Variables for an script-run

I have some script-Entries in my package.json.
For deploying an App via Fastlane to an appstore, I need to set enviroment-Variables for APPLE_ID or for the JSON-File of google-services.
This settings are individual per User (and maybe per Project).
I know, how to add an hardcoded ENV-Variable via package.json:
"ANDROID_closedBeta": "FASTLANE_JSON_KEY_FILE=`find $(pwd) -name release_manager.json` bundle exec fastlane android deploy_closedBeta",
But thats doesn't solve my Problem. I have created an File: ios_env and linux_env which exports some sensitive credentials.
If I run source linux_env, the variables are set and I can run fastlane without any issue via CLI-Command: fastlane android deploy_closedBeta
But if I try to run it via yarn or npm, this ENV will not found.
So I've tried to run source in front of the falstlane-command, but also didn't work:
"ANDROID_closedBeta": "`source ./linux_env` FASTLANE_JSON_KEY_FILE=`find $(pwd) -name release_manager.json` bundle exec fastlane android deploy_closedBeta",
How can I get the package.json-script to recognise my enviroment-Variables without the need to hardcode it into package.json?
I've found an ugly workaround... but would be glad if somebody could show me a better way to to:
In package.json I declare two settings. One which has the main Script which need the still received ENV-Variables, and one Script-entry that loads first the Enviroment and then run the main-Script-Task within this enviroment.
Hard to explain, easy to show... look at this:
"scripts": {
...
"IOS_internalTest": "source ./fastlane/ios_env && IOS_DEPLOY_TYPE=deploy_internalTest npm run IOS_runDeployment",
"IOS_betaTest": "source ./fastlane/ios_env && IOS_DEPLOY_TYPE=deploy_betaTest npm run IOS_runDeployment",
"IOS_runDeployment": "bundle exec fastlane ios $IOS_DEPLOY_TYPE",
...
}
First entry source my file, where all my ENV Exports are (like export BIMBOM=FooBar)
After finish, it executes npm run IOS_doInternalTest. Because it's run in the same shell-context, the ENV-Variables I've sourceed before, are still available.
Kudos to you, if you got an solution in an one-liner

NPM - Failed to replace env in config: ${NPM_TOKEN}

I am trying to build a react app, but when I execute the command npm -i it gives me the following error:
Error: Failed to replace env in config: ${NPM_TOKEN}
at /usr/local/lib/node_modules/npm/lib/config/core.js:415:13
at String.replace (<anonymous>)
at envReplace (/usr/local/lib/node_modules/npm/lib/config/core.js:411:12)
at parseField (/usr/local/lib/node_modules/npm/lib/config/core.js:389:7)
at /usr/local/lib/node_modules/npm/lib/config/core.js:330:24
at Array.forEach (<anonymous>)
at Conf.add (/usr/local/lib/node_modules/npm/lib/config/core.js:328:23)
at ConfigChain.addString (/usr/local/lib/node_modules/npm/node_modules/config-chain/index.js:244:8)
at Conf.<anonymous> (/usr/local/lib/node_modules/npm/lib/config/core.js:316:10)
at /usr/local/lib/node_modules/npm/node_modules/graceful-fs/graceful-fs.js:78:16
/usr/local/lib/node_modules/npm/lib/npm.js:61
throw new Error('npm.load() required')
^
Error: npm.load() required
at Object.get (/usr/local/lib/node_modules/npm/lib/npm.js:61:13)
at process.errorHandler (/usr/local/lib/node_modules/npm/lib/utils/error-handler.js:205:18)
at process.emit (events.js:182:13)
at process._fatalException (internal/bootstrap/node.js:448:27)
I am using MacOS High Sierra. I tried to set the NPM_TOKEN as an environment variable with following command:
set -x NPM_TOKEN = xyz
but it doesn't work.
What is the problem?
Actually proper solution
Update your CI deployment configuration:
npm config set '//registry.npmjs.org/:_authToken' "${NPM_TOKEN}"
npm publish
Remove this line from the .npmrc file:
//registry.npmjs.org/:_authToken=${NPM_TOKEN}
Example build config
You can see this solution used in practice in one of my GitHub repositories: https://github.com/Jezorko/lambda-simulator/blob/5882a5d738060c027b830bcc2d58824c5d27942b/.github/workflows/deploy.yml#L26-L27
The encrypted environment variable is an NPM token.
Why the other "solutions" are mere workarounds
I've seen answers here and under this question that recommend simply removing the variable setting line or .npmrc file entirely.
Thing is, the .npmrc file might not be ignored by your VCS system and modifying it might lead to accidental pushes to your project's repository. Additionally, the file may contain other important settings.
The problem here is that .npmrc does not allow defaults when setting up environment variables. For example, if the following syntax was allowed, the issue would be non-existent:
//registry.npmjs.org/:_authToken=${NPM_TOKEN:-undefined}
First Possible Solution:
Simple Solution: rm -f ./.npmrc (Deleting a .npmrc file)
Second Possible Solution:
However if you don't want to delete the file, you can simply remove this line of code in the .npmrc file.
Line of Code: //registry.npmjs.org/:_authToken=${NPM_TOKEN} (Remove this code)
Third Possible Solution
Worst case scenario:
nano ~/.bash_aliases or nano ~/.bash_profile
add export NPM_TOKEN="XXXXX-XXXXX-XXXXX-XXXXX"
CTRL + X to exit
Y to save
I have an easy solution to this issue. After you set your NPM_TOKEN globally into your environment then replace
//registry.npmjs.org/:_authToken=${NPM_TOKEN}
with
//registry.npmjs.org/:_authToken=$NPM_TOKEN
It's worked well for me on macOS Catalina.
If you just set your ~/.profile for the first time (OSX, Ubuntu) and added this line: export NPM_TOKEN="XXXXX-XXXXX-XXXXX-XXXXX".
Then you must enter this line to the terminal afterward:
source ~/.profile
Running npm install in an IDE (like WebStorm) was my problem. I added the NPM_TOKEN environment variable to .bash_profile and restarted my Terminal, but not my IDE! The IDE did not pick up the changes to the environment until I restarted it as well.
The following worked for me. I had to place
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" # This loads nvm
[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion" # This loads nvm bash_completion
AFTER the line where i specify
export NPM_TOKEN='mytoken'
Im my case moving the export of the token inside my .zsh (or .bash_profile) to the top of the file fixed the problem because it has been initialised too late before.
I fixed it by setting NPM_TOKEN=""
In github action, i set the env:
jobs:
build:
runs-on: ubuntu-latest
env:
NPM_TOKEN: ""
# ....
I fixed this via passing a docker build arg as
ARG NPM_TOKEN
ENV NPM_TOKEN=$NPM_TOKEN
ENV NPM_CONFIG_TOKEN=$NPM_TOKEN
My scoped registry line with authToken wasn't being read properly inside my docker container. When running inside the container and invoking npm config list is didn't register.
So setting this variable : NPM_CONFIG_TOKEN and running npm config list registers it under a token key in the config.
https://www.runoob.com/linux/linux-shell-variable.html
replace
'//registry.npmjs.org/:_authToken=${NPM_TOKEN}'
with
'//registry.npmjs.org/:_authToken='${NPM_TOKEN}
I got this issue while trying to setup a CI/CD job in Gitlab. I eventually found out that the error was caused because the variable that was throwing the error was set to a protected variable.
I changed it under Settings > CI / CD > Variables.
For people on Ubuntu coming from google:
nano ~/.bash_aliases
export NPM_TOKEN="PUT_YOUR_TOKEN_HERE"
CTRL+X to exit
Y to save
I am also getting this problem but I find a solution when I am pushing my repo on Heroku so I notice that Heroku run the command react-script start or build
//registry.npmjs.org/:_authToken=${NPM_TOKEN}
so this syntax didn't give the error but when I use the same syntax in my system and run the command it gives me.
Because usually when we run in our system we use cmd npm or yarn but if you use react-script then it will not gives an error
On Windows while using git bash, setting a regular Windows environment variable worked for me. This answer helped setting an environment variable in Git Bash
In case of windows and visual studio code - just restart your visual studio, it helps.
Also, how to set this environment variable on windows?
open Registry Editor, and follow \HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Environment, and create there another one "string value" with your token or whatever you need.
For mac
vim ~/.bash_profile
add export NPM_TOKEN=XXXXX-XXXXX-XXXXX-XXXXX
source ~/.bash_profile
also, add the below entry in the .zshrc file to apply the profile when a new terminal tab/window is opened.
if [ -f ~/.bash_profile ]; then
. ~/.bash_profile
fi
Using AWS CODEARTIFACT
If you use docker, you need to add this to your Dockerfile
...
ARG CODEARTIFACT_AUTH_TOKEN
...
RUN export CODEARTIFACT_AUTH_TOKEN=$CODEARTIFACT_AUTH_TOKEN
RUN npm i
...
This is the .npmrc file
registry=https://sidanmor-codeartifact-main-112233.d.codeartifact.eu-west-1.amazonaws.com/npm/js-utils/
//https://sidanmor-codeartifact-main-112233.d.codeartifact.eu-west-1.amazonaws.com/npm/js-utils/:always-auth=true
//https://sidanmor-codeartifact-main-112233.d.codeartifact.eu-west-1.amazonaws.com/npm/js-utils/:_authToken=${CODEARTIFACT_AUTH_TOKEN}
registry=http://registry.npmjs.org
And the build command will be:
docker build --build-arg CODEARTIFACT_AUTH_TOKEN=xxxyyyzzz . --tag my-tag
For anyone using npm-run-all running into this issue that:
Doesn't want to remove or modify their .npmrc file (because the syntax is actually ok)
Doesn't want to add the variable in question into their shell profile
Doesn't want to set the variable before every single script as "build": "NPM_TOKEN=... ..."
I found that when using npm-run-all any scripts using run-p or run-s that use other scripts (either directly or indirectly) that use the form npm run <SCRIPT> will parse the .npmrc configuration file, which in turn will cause errors if variables are not set.
To fix this, all you need to do is convert any scripts that are used, directly or indirectly by scripts using run-s or run-p to use run-s
Let's take this as an example:
...
"scripts": {
"dev": "run-s build*",
"build:js": "...",
"build:css": "...",
"predev": "npm run clean",
"clean": "rm -rf ./dist ./build"
}
...
Although this script does not call predev directly, it will be run before dev and this will cause configuration to be read, throwing the error
Instead, change it as follows:
...
"scripts": {
"dev": "run-s build*",
"build:js": "...",
"build:css": "...",
"predev": "run-s clean",
"clean": "rm -rf ./dist ./build"
}
...
where predev now uses run-s instead of npm run
I had the same issue on Macbook pro. Actually the issue is with the naming. Previously i added my token to my env file as below
export GITHUB_TOKEN="xxxx-xxxx-xxxx"
Solved
Solved by changing the name to
export GITHUB_ACCESS_TOKEN="xxxx-xxxx-xxxx"
Conclution
When you define access tokens it is better to name it as **_ACCESS_TOKEN so the npm can identify it correctly.
you can also replace the ${NPM_TOKEN} with your own GitHub generated personal token

How to run TypeScript files from command line?

I'm having a surprisingly hard time finding an answer to this. With plain Node.JS, you can run any js file with node path/to/file.js, with CoffeeScript it's coffee hello.coffee and ES6 has babel-node hello.js. How do I do the same with Typescript?
My project has a tsconfig.json which is used by Webpack/ts-loader to build a nice little bundle for the browser. I have a need for a build step run from the console before that, though, that would use some of the .ts files used in the project to generate a schema, but I can't seem to be able to run a single Typescript file without compiling the whole project.
How do I do the same with Typescript
You can leave tsc running in watch mode using tsc -w -p . and it will generate .js files for you in a live fashion, so you can run node foo.js like normal
TS Node
There is ts-node : https://github.com/TypeStrong/ts-node that will compile the code on the fly and run it through node 🌹
npx ts-node src/foo.ts
Run the below commands and install the required packages globally:
npm install -g ts-node typescript '#types/node'
Now run the following command to execute a typescript file:
ts-node typescript-file.ts
We have following steps:
First you need to install typescript
npm install -g typescript
Create one file helloworld.ts
function hello(person){
return "Hello, " + person;
}
let user = "Aamod Tiwari";
const result = hello(user);
console.log("Result", result)
Open command prompt and type the following command
tsc helloworld.ts
Again run the command
node helloworld.js
Result will display on console
To add to #Aamod answer above, If you want to use one command line to compile and run your code, you can use the following:
Windows:
tsc main.ts | node main.js
Linux / macOS:
tsc main.ts && node main.js
Edit: May 2022
ts-node now has an --esm flag use it.
Old Answer:
None of the other answers discuss how to run a TypeScript script that uses modules, and especially modern ES Modules.
First off, ts-node doesn't work in that scenario, as of March 2020. So we'll settle for tsc followed by node.
Second, TypeScript still can't output .mjs files. So we'll settle for .js files and "type": "module" in package.json.
Third, you want clean import lines, without specifying the .js extension (which would be confusing in .ts files):
import { Lib } from './Lib';
Well, that's non-trivial. Node requires specifying extensions on imports, unless you use the experimental-specifier-resolution=node flag. In this case, it would enable Node to look for Lib.js or Lib/index.js when you only specify ./Lib on the import line.
Fourth, there's still a snag: if you have a different main filename than index.js in your package, Node won't find it.
Transpiling makes things a lot messier than running vanilla Node.
Here's a sample repo with a modern TypeScript project structure, generating ES Module code.
I created #digitak/esrun, a thin wrapper around esbuild and that executes a TypeScript file almost instantly. esrun was made because I was disappointed with ts-node: too slow, and just didn't work most of the time.
Advantages of esrun over ts-node include:
very fast (uses esbuild),
can import ESM as well as CJS (just use the libraries of your choice and esrun will work out of the box),
there is an included watch mode, run your script with the --watch option and any change to your entry file or any of its dependencies will re-trigger the result
you can use esrun in inspect mode to use the DevTools console instead of your terminal console.
After installing, just run:
npx #digitak/esrun file.ts
Just helpful information - here is newest TypeScript / JavaScript runtime Deno.
It was created by the creator of node Ryan Dahl, based on what he would do differently if he could start fresh.
You can also try tsx.
tsx is a CLI command (alternative to node) for seamlessly running TypeScript, its build upon esbuild so its very fast.
https://github.com/esbuild-kit/tsx
Example:
npx tsx ./script.ts
As of May 2022 ts-node does support es modules
npx ts-node --esm file.ts
you will likely need to add "type": "module", to your package.json. And some of the imports might be wonky unless you turn on experimental-specifier-resolution=node
npmjs.com/package/ts-node#commonjs-vs-native-ecmascript-modules
For linux / mac you can add the ts-node-script shebang.
Install typescript / ts-node globally (see 1 below for non global install):
npm install ts-node typescript --save-dev --global
Add this as the first line in your .ts file:
#!/usr/bin/env ts-node-script
Then make the file executable:
$ chmod +x ./your-file.ts
You can then run the file directly from the command line:
$ ./your-file.ts
Notes:
1 For non global install you can install local to your project
npm install ts-node typescript --save-dev
and add the relative path to the shebang script eg:
#!/usr/bin/env ./node_modules/.bin/ts-node-script
2 Support for shebangs was officially added in ts-node v8.9.0.
Like Zeeshan Ahmad's answer, I also think ts-node is the way to go. I would also add a shebang and make it executable, so you can just run it directly.
Install typescript and ts-node globally:
npm install -g ts-node typescript
or
yarn global add ts-node typescript
Create a file hello with this content:
#!/usr/bin/env ts-node-script
import * as os from 'os'
function hello(name: string) {
return 'Hello, ' + name
}
const user = os.userInfo().username
console.log(`Result: ${hello(user)}`)
As you can see, line one has the shebang for ts-node
Run directly by just executing the file
$ ./hello
Result: Hello, root
Some notes:
This does not seem to work with ES modules, as Dan Dascalescu has pointed out.
See this issue discussing the best way to make a command line script with package linking, provided by Kaspar Etter. I have improved the shebang accordingly
Update 2020-04-06: Some changes after great input in the comments: Update shebang to use ts-node-script instead of ts-node, link to issues in ts-node.
Write yourself a simple bash wrapper may helps.
#!/bin/bash
npx tsc $1 && node ${1%%.ts}
For environments such as Webstorm where the node command cannot be changed to ts-node or npx:
npm install ts-node typescript (Install dependencies)
node --require ts-node/register src/foo.ts (Add --require ts-node/register to "Node parameters")
This answer may be premature, but deno supports running both TS and JS out of the box.
Based on your development environment, moving to Deno (and learning about it) might be too much, but hopefully this answer helps someone in the future.
Just in case anyone is insane like me and wants to just run typescript script as though it was a .js script, you can try this. I've written a hacky script that appears to execute the .ts script using node.
#!/usr/bin/env bash
NODEPATH="$HOME/.nvm/versions/node/v8.11.3/bin" # set path to your node/tsc
export TSC="$NODEPATH/tsc"
export NODE="$NODEPATH/node"
TSCFILE=$1 # only parameter is the name of the ts file you created.
function show_usage() {
echo "ts2node [ts file]"
exit 0
}
if [ "$TSCFILE" == "" ]
then
show_usage;
fi
JSFILE="$(echo $TSCFILE|cut -d"." -f 1).js"
$TSC $TSCFILE && $NODE $JSFILE
You can do this or write your own but essentially, it creates the .js file and then uses node to run it like so:
# tsrun myscript.ts
Simple. Just make sure your script only has one "." else you'll need to change your JSFILE in a different way than what I've shown.
Install ts-node node module globally.
Create node runtime configuration (for IDE) or use node in command line to run below file js file (The path is for windows, but you can do it for linux as well)
~\AppData\Roaming\npm\node_modules\ts-node\dist\bin.js
Give your ts file path as a command line argument.
Run Or Debug as you like.
Create your TypeScript file (ex. app.ts)
npm i -D typescript ts-node -> to install the dev dependencies local
npx nodemon app.ts
Using nodemon, automatically recompile app.ts every time you change the file
Here is the command
tsc index.ts --outDir .temp && node .temp/index.js && rm -rf .temp
<<<<<<<<< Compile >>>>>>>>> <<<<<<< Run >>>>>>> << Clean >>
There is also an option to run code directly from the CLI, not the *.ts file itself.
It's perfectly described in the ts-node manual.
As a first step, install ts-node globally via npm, yarn, or whatever you like.
...and now just use ts-node -e 'console.log("Hello, world!")' (you may also add the -p flag for printing code)
This little command is perfect for checking, does everything installed fine. And for finding some other error, relevant with tsconfig.json options.
We can run it using nodemon as well
nodemon ./filepath/filename.ts
This question was posted in 2015. In 2018, node recognizes both .js and .ts. So, running node file.ts will also run.

Resources