I deployed a node js app on AWS EC2 amazon linux 2 so now I'm trying to update my project how can I push new changes of node app to EC2
The simplest way I've found for EC2 is to SSH to the machine, then do a git pull from your repo, onto the machine.
Background
I have a node app that essentially needs to use aws-sdk to access S3 bucket and perform other operations. In my local machine I have a .aws config file that sits in ~/.aws/config. I want to run this node app in a docker container (so that I can deploy it to EC2 later).
The problem
How do I configure my docker container to use the aws config file so that the node app running in it can use aws-sdk?
Clarification
I do not want to use IAM. I specifically want to use aws config file which has the secret access key etc.
You can do what AWS is doing when they explain how to use their containers on local machines. For example, for local AWS Glue they simply share the ~/.aws/ with the docker container using:
-v ~/.aws:/root/.aws:ro
Obviously you would have to adjust the paths above to match your local and docker setup.
The other way is to pass the AWS credentials using docker environment variables.
How can I set the environment variables that I'm using in my Node.js application on AWS EC2? I tried to export them inside the instance console but it didn't work. It's running on an ubuntu server 20.04 btw.(p.s. my application is coming from a git repository, so there isn't a .env file there).
Manual Start:
Variable=ThisIsTheValue node index
System D:
Environment=Variable=ThisisTheValue
ExecStart=/usr/bin/node /path/to/index.js
I have developed my website on aws platform using aws s3 buckets.How do I run php files on my domain?
S3 doesn't run any sort of CGI script (PHP, Perl, Ruby, etc). Think of it as a static html and image repository.
If you want to host your PHP application on AWS, consider using AWS Beanstalk. It will launch an environment (server, IP, etc) where you can deploy and run your PHP application easily.
I have already taken a look at How to install mongodb in Elastic Beanstalk? dated 2014, which no longer works. as well as https://docs.mongodb.org/ecosystem/platforms/amazon-ec2/#manually-deploy-mongodb-on-ec2
I have set up a new elastic beanstalk environment running on node.js with 1 ec2 micro instance '64bit Amazon Linux 2016.03 v2.1.0 running Node.js'
I have already tried using ssh to connect into my instance and install the mongodb packages using yum command:
$ sudo yum install -y mongodb-org-server mongodb-org-shell mongodb-org-tools
and received this call back:
Loaded plugins: priorities, update-motd, upgrade-helper
No package mongodb-org-server available.
No package mongodb-org-shell available.
No package mongodb-org-tools available.
Error: Nothing to do
When I first ssh 'd into my instance, I received this error warning:
This EC2 instance is managed by AWS Elastic Beanstalk. Changes made via SSH
WILL BE LOST if the instance is replaced by auto-scaling. For more information
on customizing your Elastic Beanstalk environment, see our documentation here:
http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/customize-containers-ec2.html
Currently my environment is set up as a single instance environment, to save on costs. However, in the future I will upgrade to an auto-scaling environment.
Because of this, I am asking is it recommendable to make any changes via ssh in ec2, or should I only be using EB CLI?
I have both EC2 and EB CLI installed locally, however I have never used EB CLI before. If I should be using EB, does anyone have a recommended way to install mongodb?
In case anyone is looking for an answer, here is the advice I received from aws business support.
All code deployed to Elastic Beanstalk needs to be "stateless" I.E. Never make changes directly to a running beanstalk instance using SSH or FTP.... As this will cause inconsistencies and or data lose!
- Elastic Beanstalk is not designed for application that are not stateless.
The environment is designed to scale up and down pending on your Network / CPU load and build new instances from a base AMI. If an instance has issues or the underlying hardware, Elastic Beanstalk will terminate these running instances and replace with new instances. Hence, why no code modification must be applied or done "directly" to an existing instance as new instances will not be aware of these direct changes. ALL changes / code needs to be either uploaded to Elastic Beanstalk console or the CLI tools and the pushed to all the running instances.
More information on Elastic Beanstalk design concepts can be read at the following link
http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/concepts.concepts.design.html
Suggested Solution:
With the above in mind, if using MongoDB to store application data our recommendation would be to DE-couple the MongoDB environment from your Node.js application.
I.E Create a MongoDB Server outside of Elastic Beanstalk, example launching MongoDB directly on a EC2 instance and have your Elastic Beanstalk Node.js application connect to MongoDB Server using connection settings in your app.
-Creating MongoDB
Below is some example links that may be of use for your scenario for creating a MongoDB Server.
Deploy MongoDB on EC2,
https://docs.mongodb.org/ecosystem/platforms/amazon-ec2/
MongoDB node client
https://docs.mongodb.org/getting-started/node/client/
MongoDB on the AWS Cloud quick start guide
http://docs.aws.amazon.com/quickstart/latest/mongodb/architecture.html
-Adding environment variables to Elastic Beanstalk to reference your MongoDB server
Once you have created your MongoDB Server you can pass the needed connection settings to your Elastic Beanstalk environment using environment variables.
Example using .ebextensions .config which you can add Mongo URL / ports / users etc..
option_settings:
- option_name: MONGO_DB_URL
value: "Your MongoDB EC2 internal IP address"
Information on how to use environment properties and read them from within your application can be seen below.
http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/create_deploy_nodejs.container.html#create_deploy_nodejs_custom_container-envprop
And information using .ebextensions .config can be found at the following link
http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/ebextensions.html
Alternatively you can also set environment variable using the cli or via the AWS Console
eb cli set environment variables can be read per the below link.
http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/eb3-setenv.html
Using AWS Console
To set system properties (AWS Management Console)
Open the Elastic Beanstalk console.
Navigate to the management console for your environment.
Choose Configuration.
In the Software Configuration section, choose Edit.
Under Environment Properties, create your name / values ...
Accessing Environment Configuration Settings
Inside the Node.js environment running in AWS Elastic Beanstalk, you can access the environment variables using process.env.ENV_VARIABLE similar to the following example.
process.env.MONGO_DB_URL
process.env.PARAM2
http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/create_deploy_nodejs.container.html#create_deploy_nodejs_custom_container-envprop
Summary:
In summary I would recommend the following steps to integrate MongoDB with Elastic Beanstalk environments.
Step 1) Create a MongoDB Server outside of Elastic Beanstalk
Step 2) Create your Node.js application in Elastic Beanstalk that connect to your MongoDB server
3 options:
1) SSH into an eb instance and install the mongo CLI manually:
sudo yum-config-manager --add-repo https://repo.mongodb.org/yum/amazon/2013.03/mongodb-org/4.0/x86_64/
sudo yum install --nogpgcheck -y mongodb-org-shell
Disadvantage is that if EB scales down its number of instances and the instance you are currently on gets terminated, you get kicked out of the SSH session:
The system is going down for halt NOW!
Connection to 1.2.3.4 closed by remote host.
Connection to 1.2.3.4 closed.
ERROR: CommandError - An error occurred while running: ssh.
You then need to start all over again: connect to instance, install mongo CLI...
2) Pre-install mongo CLI on instances by using a .config file:
container_commands:
01-mongocli:
command: "sudo yum-config-manager --add-repo https://repo.mongodb.org/yum/amazon/2013.03/mongodb-org/4.0/x86_64/;sudo yum install --nogpgcheck -y mongodb-org-shell"
ignoreErrors: true //use this the ensure instance deployment even if mongo CLI installation fails
Again, if the instance gets terminated by auto-scaler you'd have to connect again, but you don't have to the install mongo CLI manually.
3) Create a separate instance that hosts your mongo CLI, as described in #amyloula's answer. If your mongodb is within an VPC you need to create that separate instance also within the VPC. You will then need to create a Gateway to access the instance publicly, as you cannot connect directly to an instance in a VPC.