I am deploying a Node.js app to EC2 using CodeDeploy. I am storing credentials within AWS Systems Manager, Parameter Store however cannot find a method to expose these to my application.
I am using PM2 for process management. I can successfully retrieve the parameter from the Parameter Store on the target machine, so there are no permission issues. For example:
aws ssm get-parameters --region us-east-1 --names LOCAL_CACHE_PATH --with-decryption --query Parameters[0].Value`
...successfully returns the correct string. I attempt to use this in my applicationStart.sh CodeDeploy file and start the app:
#!/bin/bash
export LOCAL_CACHE_PATH=$(aws ssm get-parameters --region us-east-1 --names LOCAL_CACHE_PATH --with-decryption --query Parameters[0].Value)
pm2 start ecosystem.config.js --env production
LOCAL_CACHE_PATH returns undefined in my app when accessing process.env.LOCAL_CACHE_PATH.
So the environment variable is available within the applicationStart.sh script and yet undefined when the app starts from that script.
I am looking for a recommended approach to use environment variables from the Parameter Store with CodeDeploy.
I have read literally dozens of posts on similar topics but cannot resolve it. Very much appreciate any guidance.
The solution I am using is to write the environment variables to a .env file and use that in my app:
afterInstall.sh:
echo LOCAL_CACHE_PATH=$(aws ssm get-parameters --output text --region us-east-1 --names LOCAL_CACHE_PATH --with-decryption --query Parameters[0].Value) >> /home/ubuntu/foo/.env
Related
Wanted to learn AWS and found the tutorial Build a Serverless Web Application. In my research the closest Q&A I could find for my issue was Unable to locate credentials aws cli.
My process has been:
Created a repo in Github
Navigated to IAM and created a user trainer. Tutorial didn't specify policies so chose AdministratorAccess. Per instructions went the Security credentials and Create access key. Downloaded the file locally.
Went to Configuration basics and did Importing a key pair via .CSV file with the command of:
aws configure import --csv file:///Users/path/to/file/aws-training.csv
params:
User name: trainer
Access key ID: ****57
Secret access key: *****1b
but then found that the file didn't contain region or format so did:
aws configure --profile trainer
and re-did all values based on the CSV (Quick Setup):
AWS Access Key ID: ****57
AWS Secret Access Key: *****1b
Default region name: us-east-1
Default output format: json
Made sure to reboot my terminal and locally in a directory I run the command:
aws s3 cp s3://wildrydes-us-east-1/WebApplication/1_StaticWebHosting/website ./ --recursive
The terminal has a delay then throws:
fatal error: Unable to locate credentials
Research
Q&As I've read through to try and see if I could diagnose the problem:
aws cli with shell script: upload failed: Unable to locate credentials
Bash with AWS CLI - unable to locate credentials
Unable to locate credentials aws cli
Unable to locate credentials in boto3 AWS
Get "fatal error: Unable to locate credentials" when I'm copying file from S3 to EC2 using aws cli
Unable to locate credentials when trying to copy files from s3-bucket to my ec2-instance
How can I resolve my error of Unable to locate credentials and what am I doing wrong or misunderstanding?
Per the comment:
Check the content of ~/.aws/credentials and ~/.aws/config
credentials
command:
nano ~/.aws/credentials
renders:
[training]
aws_access_key_id = *****57
aws_secret_access_key = ***1b
[trainer]
aws_access_key_id = *****57
aws_secret_access_key = ***1b
config
command:
nano ~/.aws/config
renders:
[profile training]
region = us-east-1
output = json
[profile trainer]
region = us-east-1
output = json
You've configured the profile with the name trainer. You didn't create a default profile, you created a named profile. You're getting the current error because the CLI tool is looking for a default profile, and you don't have one configured.
In order to use the trainer profile you either have to add --profile trainer to every aws command you run in the command line, or you need to set the AWS_PROFILE environment variable inside your command line environment:
export AWS_PROFILE=trainer
It looks like you also tagged this with nodejs, so I recommend going the environment variable route, which will also work with the nodeJS AWS SDK.
We traditionally stored our env vars in Environment->Configuration->Software in Elastic Beanstalk console and access it in our NodeJS app by typing process.env.VariableName. There's a hard limit for the number of vars stored there, so we moved our variables to AWS Parameter Store.
I have a .ebextention script that downloads the parameters from the parameter store and saves them to a file in /home/ec2-user directory named .env.local, but I can't reference them in the app with process.env.
.ebextention script:
commands:
01_command2:
command: aws ssm get-parameters-by-path --path /latest/ --recursive --with-decryption --output text --query "Parameters[].[Name,Value]" --region us-west-2 | sed -E 's#latest/([^[:space:]]*)[[:space:]]*#export \1=#' > /home/ec2-user/.env.local
How can I access these vars in NodeJS without modifying the way that we correctly use env vars (process.env.VarName)?
How can I create an Azure container instance and configure it with an environment variables file?
Something that'd be equivalent to Docker's --env-file flag for the run command. I couldn't find a way to do that but I'm new to both Azure and Docker.
So it'd look something like: az container create <...> --env-file myEnvFile where myEnvFile is stored somewhere on Azure so I could grab it, like how Docker can grab such a file locally.
You can find what you want here https://learn.microsoft.com/en-us/cli/azure/container?view=azure-cli-latest#az-container-create
i.e.
az container create -g MyResourceGroup --name myapp --image myimage:latest --environment-variables key1=value1 key2=value2
Apologies, realised you want it from a file, if running in a script can you not have the file set local environment variables or parse the file to set them and then run the command above?
I'm really sure there is no parameter to set the environment variables of the Azure container instance from a file only through one command.
You can take a look at the parameter --environment-variables in the command az container create:
A list of environment variables for the container. Space-separated
values in 'key=value' format.
It requires the value of a list. So you can read from the file to create a list and then use the list as the value of the parameter --environment-variables in the create command.
As far as I'm aware, from answers and my research, this is currently not supported.
I have a Cloud9 environment spun up and have modified my ~/.bash_profile to export a value at the end of the file.
export foo="hello world"
I run . ~/.bash_profile and then echo $foo and I see hello world output in the terminal.
I then created a NodeJS Lambda with API Gateway. I run the API Gateway locally in Cloud 9 and attempt to read the environment variables
console.log(process.env)
I see a list of variables available to me that AWS has defined. My export is not listed there however. Since I will be using environment variables when my Lambda is deployed, I want to test it with environment variables defined in the Cloud9 environment.
Is there something specific I have to do in order to get the Lambda to read my .bash_profile exports?
AWS Cloud9's Lambda plugin is backed by SAM Local, which uses Docker: https://github.com/awslabs/aws-sam-cli . By default, this means that the ~/.bash_profile file is not used by Lambda; you'll want to load this in manually.
Please see Using the AWS Serverless Application Model (AWS SAM) article that describes how to work with environment variables in SAM (so also in cloud9).
In summary - put environment variables into the template.yaml file (present in the root folder of your app) like below:
Properties:
.... # tons of other properties here, add yours at the end
Environment:
Variables:
MY_ENV_VARIABLE: 'This is my awesome env variable value'
I'm deploying a Django based project on AWS Elastic Beanstalk.
I have been following the Amazon example, where I add my credentials (ACCESS_KEY/SECRET) to my app.config under the .ebextentions directory.
The same config file has:
container_commands:
01_syncdb:
command: "django-admin.py migrate --noinput"
leader_only: true
02_collectstatic:
command: "django-admin.py collectstatic --noinput"
leader_only: true
Problem is that this is forcing me to store my credentials under Version Control, and I will like to avoid that.
I tried to remove the credentials and then add them with eb setenv, but the problem is that the two django commands require the these settings to be set on the environment.
I'm using the v3 cli:
eb create -db -c foo bar --profile foobar
where foobar is the name of the profile under ~/.aws/credentials, and where I want to keep my secret credentials.
What is the best security practices for the AWS credentials using EB?
One solution is to keep the AWS credentials, but create a policy that ONLY allows them to POST objects on the one bucket used for /static.
I ended up removing the collecstatic step from the config file, and simply take care of uploading statics on the build side.
After that, all credentials can be removed and all other boto commands will grab the credentials from the security role on the EC2 instance.