I'm building a personal blog website with express.I hosted it on ec2 and also hosted another version at digital ocean .But when it comes to updating my code (like i am changing a few lines or adding or removing a new feature) i have to remove all the files and re-upload and again run it.When i was using www.heroku.com it was easy like git push heroku master.How can i do similar with ec2 or digital ocean server?.(I'm using pem file to login to server)
I recommend looking at Code Pipeline tutorials. You can use this AWS Service to build a CI/CD use case. See:
CodePipeline tutorials
As SMAC mentioned in the comments, you're looking for a CI/CD solution essentially. Heroku does a nice job of automating that for you, as do several other products out in the market. Depending on what Git provider you're using, GitHub and GitLab both provide a native solution (GitHub actions vs GitLab CI).
I'd recommend you combine that with something like AWS's ElasticBeanstalk to get a simple change/push workflow like you're looking for.
Related
I'm writing this post after exhausting all the options I could try.
Scenario:
I want to deploy a lightweight backend logic written in Node.js (using JavaScript, Express.js, MongoDB).
Platforms considered:
AWS (preferred)
Heroku
Bitbucket Pipelines (least preferred)
Currently it's deployed on Heroku (and running as expected), but I want to deploy it on our preferred platform, i.e., AWS. Also, I want the whole deployment process to be automated (using Continuous Delivery/Deployment).
To achieve that, I created:
A fresh Node.js environment (v16.0, platform v5.6.0) on Elastic Beanstalk
A new CodePipeline:
Source: Bitbucket
Repository (cloned from here)
Change detection options: Start the pipeline on source code change
Output artifact format: CodePipeline default
Build stage: skipped
Deploy provider: AWS Elastic Beanstalk (with correct region, app and env name)
Outcome on every try:
On the first try when the environment gets created, we get greeted with the default screen:
After subsequent commits, things like this start to show up:
Error pages with error codes in the series of 400s and 500s:
To overcome those problems, I scratched all the blog posts and support articles available regarding those issues and followed the steps thoroughly gaining no success in the end.
Some of the (many) solutions I tried:
Create a Procfile with the the params
Write app.js/index.js outside the src/ (dumb thing, I know)
Switch to immutable deployments (as mentioned here)
I seriously need help resolving this issue. Any help or guidance towards some solution would be greatly appreciated.
As you understand from the title, that I have to deploy an nodejs application build on Nextjs on the existing instance.
The situation is:
can we deploy nodejs with the instance that already have php environment? (I think it is possible as we normally work on our local computer on daily basis).
how to create a basic pipeline for that? (I go through some tutorials on the Elastic Beanstalk in AWS, but it creates a new instance for that which I don't want, is there is any alternative ways to do that.)
how to connect pipeline with instance and also with the project folder.
Also, I've heard some rumors that, when the pipeline builds our projects, then it will delete all the existing projects in the instance. (is it really true).
I only have the book knowledge, but never done a practical on that.
Please help me out.
Thanks.
I need help with someone familiar with AWS and web servers. Currently I'm walking through this tutorial trying to get started with NodeJS and AWS. https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/create_deploy_nodejs.html
I'm trying to figure out how to essentially do a "git clone" of a traditional project but do whatever equivalent that is for an AWS project (ex: If I wanted to work on my existing AWS project on a different machine)
I read some EB CLI documentation (https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/eb3-cmd-commands.html). I tried the "eb clone env-name". However, this actually created a separate environment on AWS within my application, which isn't what I wanted. It also only added a .gitignore and a .elasticbeanstalk folder to my directory, none of my source code for my AWS application.
I'm confused on what the standard process is for working with AWS projects. Particularly, how can I start working on my existing AWS project from another machine? (Is there anyway to pull my source code from AWS project?) Is there anyway I can view my code on AWS?
Side note: In the past I worked with Google Apps Scripts on the cloud, which used Clasp CLI for pushing and pulling code to the cloud. This was very intuitive because it was literally clasp pull to pull code from cloud and clasp push to push code to it.
Elastic Beanstalk isn't a code repo. It's a way to host applications in a simplified way, without having to configure the compute resources. Compare this to something like EC2 where all the networking and web server configuration is manual.
You can still use git to manage your source code, and there's git CLI integration with Elastic Beanstalk too. Once you've got your source code working, you bundle it up into a .zip file and upload it to EB. You can also use AWS CodeBuild to watch git repos, build source code into bundles, and automatically deploy it to Elastic Beanstalk.
If you are looking for a way to host source code on AWS, AWS CodeCommit is the managed git solution.
You should take a look at the Amplify Framework by AWS: https://aws-amplify.github.io/docs/ – here's a walkthrough that will get you were you are heading faster – sure, it mentions teams but, the result can be applied to single developers too: https://aws-amplify.github.io/docs/cli/multienv?sdk=js
Since you mentioned "view my code on AWS", you should have a look here: https://aws.amazon.com/cloud9/ – this will walk you through setting up an account, repos and working with your code on the cloud.
Good luck!
I've been using AWS for a while now but am wondering about how to go about developing with Lambda. I'm a big fan of having server-less functions and letting Amazon handle the maintenance and have been using it for a while. My question: Is there a recommended workflow for version control and development?
I understand there's the ability to publish a new version in Lambda. And that you can point to specific versions in a service that calls it, such as API Gateway. I see API Gateway also has some nice abilities to partition who calls which version. i.e. Having a test API and also slowly rolling updates to say 10% of production API calls and scaling up slowly.
However, this feels a bit clunky for an actual version control system. Perhaps the functions are coded locally and uploaded using the AWS CLI and then everything is managed through a third party version control system (Github, Bitbucket, etc)? Can I deploy to new or existing versions of the function this way? That way I can maintain a separation of test and production functions.
Development also doesn't feel as nice through the editor in Lambda. Not to mention using custom packages require to upload anyways. Seems local development is the better solution. Trying to understand others workflows so I can improve mine.
How have you approached this issue in your experience?
I wrote roughly a dozen lambda functions that trigger based on S3 file write event or time, and make a HTTP req to an API to kickstart data processing jobs.
I don't think there's any gold standard. From my research, there are various approaches and frameworks out there. I decided that I didn't want to depend on any kind of frameworks like Serverless nor Apex because I didn't want to learn how to use those things on top of learning about Lambda. Instead I built out improvements organically based on my needs as I was developing a function.
To answer your question, here's my workflow.
Develop locally and git commit changes.
Mock test data and test locally using mocha and chai.
Run a bash script that creates a zip file compressing files to be deployed to AWS lambda.
Upload the zip file to AWS lambda.
You can have version control on your lambda using aws CodeCommit (much simpler than using an external git repository system, although you can do either). Here is a tutorial for setting up a CodePipeline for commit/build/deploy stages: https://docs.aws.amazon.com/codepipeline/latest/userguide/tutorials-simple-codecommit.html
This example deploys an EC2 instance, so for the deploy portion for a lambda, see here
If you set up a pipeline you can have an initial commit stage, then a build stage that runs your unit tests and packages the code, and then a deploy stage (and potentially more stages if required). It's a very organized way of deploying lambda changes.
I would suggest you to have a look at SAM. SAM is a command line tool and a framework to help you to develop your serverless application. Using SAM, you can test your applications locally before to upload them to the cloud. It also support blue / green deployment and CI/CD workflows, starting automatically from github.
https://github.com/awslabs/aws-sam-cli
I want to build web ui for StrongLoop. It would let a user build and deploy process with that UI like StrongLoop Arc.
There are simple node applications(Web Services) without created with StrongLoop tools. Need to deploy these applications via web ui. Solution in my mind is some server-side processes, listed steps below:
Upload zip folder(node application) to server
Extract zip and build to tar.gz by shell command (slc build) through node.js child_process API
Deploy tar.gz file to relevant StrongLoop host by shell command(slc deploy..) through API which is mentioned on previous step.
I wonder is there any alternative way to deploy node application(without created with StrongLoop tools) to StrongLoop host via web ui using some StrongLoop API?
I have looked API could not find specific solution.
What you require is a CDP (Continuous delivery pipeline) setup, there seem to be many ways in which you can achieve this (easiest way is using Codeship or similar platforms), but if you want to know how it works it requires a bit of orchestration tools to help you. To describe the steps I'll be using the following tools:
Docker (what is docker?)
Ansible (Use Cases and How it works?)
Jenkins (What is it and Why to use it?)
"There are many other combination of tools that you can look at, but this should give you an idea"
Now that we have the tools, I'll try to describe the deployment pipeline with a very basic use-case.
Step I "Ideally" - Creating a docker image for your nodejs application.
What generally everyone suggests is that you create a docker image of your application. Then save this image on docker-hub. How this will help you is that, now your nodejs application is contained inside a docker image which makes it independent of the Host and can be deployed anywhere you want.
To create this image all you need to do is create a Dockerfile, which is described in the in the link I've shared.
Step II "Ideally" - Creating an Ansible playbook to mimic the setup steps of your application.
Ansible playbooks are basically used to automate every manual process that you would need to do in order to setup-deploy-run your application. This decreases the need to run even trivial tasks like "slc build".
Step III "Ideally" - This is where we get to the UI stuff
By using Jenkins, you are given a UI which will help you configure tasks that can be combined with Github hooks and trigger the deployment as soon as you make a commit. This is explained in more details in the link shared.
So to summarize, This is what goes on at back to some extent, in order to automate the build and deployment of your application using UI. I hope this serves as a good starting point to achieve your requirements, and also in case you want skip these steps in the start, you could always go with Codeship or similar other tools to help you with the steps that you've mentioned.