I'm deploying a node.js app on Heroku dyno and using config module that requires me to define a system variable NODE_CONFIG_DIR with the location of the config folder.
The config folder is located on my project's root.
I tried to define the system variable NODE_CONFIG_DIR using the following values, all failed:
./config
~/config
app/config
~/app/config
./app/config
$HOME/config
$HOME/app/config
I keep getting this error:
WARNING: No configurations found in configuration directory:app/config
(replace app/config with any of the values above)
I manage to set a system variable, but its value is not pointing the right place.
What is the correct way to refer to the root of my tree when using a system variable in Heroku?
Based on documentation - if config folder is in the root of your application you should not need to specify $NODE_CONFIG_DIR env variable.
From node-config documentation:
Node-config reads configuration files in the './config' directory for the running process, typically the application root. This can be overridden by setting the $NODE_CONFIG_DIR environment variable to the directory containing your configuration files. It can also be set from node, before loading Node-config:
process.env["NODE_CONFIG_DIR"] = __dirname + "/configDir/";
const config = require("config");
$NODE_CONFIG_DIR can be a full path from your root directory, or a relative path from the process if the value begins with ./ or ../.
You could use above code to set it from your node code.
You were close: /app is the correct path. You can verify it by running heroku run bash.
It was my bad...
Both answers are correct but didn't solve my issue.
The problem was that I used lowercase for my configuration file name while the NODE_ENV value was uppercase.
Related
Currently I'm trying to breakdown my serverless service into multiple services to get over the cloudFormation resource limit.
My current project structure is as follows:
aws-backend
functions
workers
serverless.yml // workers service
.env.local
.env.dev
serverless.yml // Rest of the functions in here
In my workers service, I'm trying to reference the .env.* files in the root folder using variables.
My issue is when i use the following syntax
${env:SLS_AWS_REGION}
I get a
Error:Cannot resolve serverless.yml: Variables resolution errored with:
- Cannot resolve variable at "provider.region": Value not found at "env" source
but when I use the following syntax:
${../../env:SLS_AWS_REGION}
It works but I get a warning:
Warning: Invalid configuration encountered
at 'package.individually': must be boolean
at 'provider.region': must be equal to one of the allowed values [use-east-1, etc...]
How can I get rid of this error? Am I even using the correct syntax?
Thanks
as for this error
Error:Cannot resolve serverless.yml: Variables resolution errored with:
- Cannot resolve variable at "provider.region": Value not found at "env" source
You get this error because the Framework cannot find SLS_AWS_REGION environment variable. The env variable source doesn't read from .env files, but from environment variables from the process.
As for this syntax:
${../../env:SLS_AWS_REGION}
This does not work because env is a correct variable source, not ../../env.
You have two options here:
Ensure that content of the .env file(s) is exported so the variables defined in these files are actually exported as environment variables before running serverless commands.
Set useDotenv: true in your serverless.yml file so the .env files will be loaded automatically with dotenv. Please see the docs for reference on how it works: https://www.serverless.com/framework/docs/environment-variables
According to the plugin documentation, you should run sls deploy with --stage or --env (deprecated in Serverless >=3.0.0) or NODE_ENV corresponding to the .env file name. If you run it without any of those flags, it will default to development, and the plugin will look for the plugin will look for files named .env, .env.development, .env.development.local
Note that Serverless Framework .env files resolution works differently to the plugin.
The framework looks for .env and .env.{stage} files in service directory and then tries to load them using dotenv. If .env.{stage} is found, .env will not be loaded. If stage is not explicitly defined, it defaults to dev.
I believe the plugin takes precedence here.
I want to deploy my Node.js application in Pivotal Cloud Foundry using manifest.yml. I need to update the PATH variable of the container before the application starts, to include the path of a directory in my application's src directory. Can this be achieved?
manifest.yml:
---
applications:
- name: node-apollo-graphql-server
command: npm start
instances: 1
memory: 512M
buildpack: dicf_nodejs_buildpack_rc
stack: cflinuxfs3
You cannot do this by setting env variables with cf push -e or the env: block in manifest.yml. If you set path using one of these methods, you'll override path when what you likely want to do is append to it.
To append to $PATH, add a file .profile to the root of your project (directory from which you're running cf push). In that file, put one line export PATH=$PATH:<new loc> where <new loc> is the path you want to append to the $PATH env variable.
The .profile file is sourced before your application starts so you can use this to dynamically set environment variables or apply configuration before your application starts up.
The only caveat is that this happens before your application starts so it blocks the starting of your application. As such, you should avoid running expensive/time-consuming processes here. Otherwise, you will delay the start of your application, or possibly even cause app failures if you exceed the startup timeout (cf push -t).
I have .env file at my project root directory.
How should I handle .env file for dev, qa, stage and prod?
Should include them in git repo? if not where I put them? different folder on external drive for example?
What is the correct extensions? .env.qa or .qa.env?
If I want to build my bundle using webpack to the dist folder (server side), should I include the env file or manually copy it to the dist folder?
You should not check-in your env files into any source control. Any of those secrets will be forever available to anyone having access to the repo until the history is rewritten to remove them.
If you use AWS services, for example, I would suggest using the Secrets Manager.
Any environment variables introduced to Webpack should not be secrets but be configuration values. Anyone who can view source can read those values. If you need to have environment-specific configurations, the Webpack DefinePlugin will replace vars like MY_API_HOST with their values with the following config:
const plugins = [
new webpack.DefinePlugin({
MY_API_HOST: JSON.stringify('https://my-domain.com/api/'),
MY_API_VERSION: JSON.stringify('v2')
})
]
Config module is a easy way to address the different env specific values. Read about config module at - https://www.npmjs.com/package/config. You will have a config folder in the repository with env specific files and I like this approach as the files are in the repository but very well separated.This provides a really easy way to set default values, override the environment specific values etc. It is also very convenient to use the different environment specific files by setting the appropriate node environment variable(export NODE_ENV=development or acceptance or production).
I have a node application that I run on a Linux Server (CentOS 6.5) by setting my environment with a bash script
env_nodejs.sh
#!/bin/bash
PATH=$PATH:/opt/nodejs/node-v6.9.4-linux-x64/bin
export PATH
So that I can
# . ./env_nodejs.sh
# node /var/www/html/application/app.js
That all works fine but if I do the following in a separate script
run_app.sh
#!/bin/bash
$COMMAND=/opt/nodejs/node-v6.9.4-linux-x64/bin/node
$SITE=/var/www/html/application/app.js
nohup $COMMAND $SITE > /tmp/nodeapp.log &
This runs the node server and app but with errors that seem to be related to npm
Error: Failed to lookup view "control/users" in views directory "/views"
I have a feeling this is because the environment is not set but is there a way to run it correctly without the environment or to pass additional parameters for the npm location?
It seems like it's trying to find the files in "/views" which is a top level directory in your file system.
If that is the case then it's not PATH but PWD that's in fault here.
Make sure that when you define where to look for the views in your app, instead of saying 'views' or './views' you use path.join(__dirname, 'views') instead.
You first need to require path with: var path = require(path);
Of course this is just my guess as you didn't include any part of your source code that you have problem with.
We have a nodejs project running on Amazon Elastic Beanstalk that uses private modules that we host using nodejitsu's private npm registry.
However getting access to the private npm registry from the elastic instances hasn't been straightforward and is not documented well.
What is the best way to get this access set up?
None of the other answers were working for me. After hours of hair pulling, we finally figured it out. The solution that worked is almost the same as the other answers but with a very minor tweak.
Set an NPM_TOKEN environment variable on Elastic Beanstalk under Configuration > Software Configuration > Environment Properties.
Create a .ebextensions/npm.config file. (The name does not have to be 'npm'.)
Put this content into the file:
files:
"/tmp/.npmrc":
content: |
//registry.npmjs.org/:_authToken=${NPM_TOKEN}
Note that it uses ${NPM_TOKEN} and not $NPM_TOKEN. This is vital. Using $NPM_TOKEN will not work; it must have the curly braces: ${NPM_TOKEN}.
Why are the curly braces needed? No idea. In shell/POSIX languages, ${VAR} and $VAR are synonymous. However, in .npmrc files (at the time of this writing), variables without the curly brackets are not recognized as variables, so npm must be using a slightly different syntax standard.
UPDATE
Also, this has worked for us only on new or cloned environments. For whatever reason, environments which were not initialized with a /tmp/.npmrc will not read it in any future deployments before running npm install --production. We've tried countless methods on 4 different apps, but cloning and replacing an environment has been the only method which has worked.
So, we managed to get this working by using the npm userconfig file. See the doc page for npmrc for more info.
When a nodejs application is being deployed to Elastic Beanstalk, the root user runs npm install. So you will need to write the root's npm userconfig file, which is at /tmp/.npmrc.
So if you add a file called private_npm.config (or whatever name you choose) to your .ebextensions folder with all the information needed, you will be good to go. See Customizing and Configuring AWS Elastic Beanstalk Environments for more info.
So here is what my file looks like to use nodejitsu private registry.
.ebextensions/private_npm.config:
files:
#this is the npm user config file path
"/tmp/.npmrc":
mode: "000777"
owner: root
group: root
content: |
_auth = <MY_AUTH_KEY>
always-auth = true
registry = <PATH_TO_MY_REGISTRY>
strict-ssl = true
email = <NPM_USER_EMAIL>
Using an .npmrc within the project also works. For example...
.npmrc
registry=https://npm.mydomain.com
You may want to .gitignore this file if you include an _authToken line but make sure you don't .ebignore it so it's correctly bundled up with each deployment. After trying a few things unsuccessfully, I came across this post which made me realize specifying it locally in a project is possible.
The answer above as a step in the right direction, but the permissions and owner did not work for me. Managed to get it to work with the following combination:
files:
#this is the npm user config file path
"/tmp/.npmrc":
mode: "000600"
owner: nodejs
group: nodejs
content: |
_auth = <MY_AUTH_KEY>
always-auth = true
registry = <PATH_TO_MY_REGISTRY>
strict-ssl = true
email = <NPM_USER_EMAIL>
Place the below within your .ebextensions/app.config.
files:
"/tmp/.npmrc":
mode: "000777"
owner: root
group: root
content: |
//registry.npmjs.org/:_authToken=$NPM_TOKEN
Where NPM_TOKEN is an environment variable with the value of your actual npmjs auth token.
Note that environment variables within elasticbeanstalk can and should be set from within the AWS console Elasticbeanstalk software configuration tab.
AWS Elasticbeanstalk Configuration
In new Elastic Beanstalk Linux 2 Platforms, none of these solutions work (apart from the .npmrc file solution that works but has its issues when using them in development evironments due to the requirements that all developers have their ${NPM_TOKEN} Env Var defined in their own environments).
The reason is that the /tmp/.npmrc location no longer works.
Option 1
You have to change the .ebextensions/npm.config file to this new format:
files:
#this is the npm user config file path
"/root/.npmrc":
mode: "000777"
owner: root
group: root
content: |
_auth= ${NPM_TOKEN}
registry = https://{yourprivatenpmrepository.com}/
Option 2
Add a custom .npmrc_{any-suffix} to the root of your app and create a prebuild hook to rename it before Beanstalk executes the npm install so that it can use your private repository configuration:
Add the following file (path from your app root) .platform/hooks/prebuild/01_set_npmrc.sh with the following content:
#!/bin/bash
#Copy and rename .npmrc_beanstalk to .npmrc
mv .npmrc_beanstalk .npmrc
Create an .npmrc_beanstalk file in your root with the following content (modify it depending on your private npm config):
_auth= ${NPM_TOKEN}
registry = https://{yourprivatenpmrepository.com}/
Chmod the hook file so that it has the necessary exec permissions when uploaded to EB: chmod +x .platform/hooks/prebuild/01_set_npmrc.sh
Re-deploy using EB CLI and you are done!
With modern platforms, you no longer need to do this via .ebextensions
You can simply create a .npmrc file at the root of your deployment package, alongside your package.json with the following line:
//registry.npmjs.org/:_authToken=${NPM_TOKEN}
Using this method, you can create an environment variable named NPM_TOKEN in your AWS console so you don't have to store the token in your repo.
Structure:
~/your-app/
|-- package.json
|-- .npmrc