I have setup travis to aws codeDeploy for a node app. Now that the latest code can be deployed to EC2 correctly, but I need to manually restart the node app again to make the changes take effect.
How to auto restart the node app after codeDeploy? I guess I can do it by setting afterInstall in appspec.yml, but I find that many tutorial/walkthrough didn't mention about this, so I wonder is this the only/best way to restart the node app.
You should use the ApplicationStart hook in appspec.yml. From the AWS docs:
AfterInstall – You can use this deployment lifecycle event for tasks such as configuring your application or changing file permissions.
ApplicationStart – You typically use this deployment lifecycle event to restart services that were stopped during ApplicationStop.
You can use pm2 or forever and you can restart your node after code deploy. Your appspec.yml should be as below
version: 0.0
os: linux
files:
- source: /deployment.zip
destination: /home/ubuntu/PorjDir/
hooks:
BeforeInstall:
- location: stop_server.sh
timeout: 300
runas: ubuntu
AfterInstall:
- location: start_server.sh
timeout: 300
runas: ubuntu
Note: You need to create stop_server.sh and start_server.sh into your project and need to export it as artifact so code deploy agent can use it.
start_server.sh should contains below:
sudo /usr/bin/pm2 restart appname
Hope this will help you!!
Related
I'm learning how to use CodePipeline and have problem with CodeDeploy for small testing node app. My target is to implement CD for large express + react app and I need to use hooks from AppSpec.yml.
For now everything else is working, files are copied etc, it just doesn't fire script. I started with BeforeInstall (delete process from pm2) and ApplicationStart (start app with pm2) hooks, but now I switched to using ApplicationStart with script to remove process from pm2 just to see if it works.
My AppSpec.yml:
version: 0.0
os: linux
files:
- source: /
destination: /home/ubuntu/api
permissions:
- object: /home/ubuntu/api/
owner: ubuntu
group: ubuntu
mode: "777"
# I use appStop.sh just to check if this works:
ApplicationStart:
- location: scripts/appStop.sh
runas: ubuntu
# I tried also running as root, still nothing
timeout: 60
appStop.sh:
#!/bin/bash
cd /home/ubuntu/api
pm2 delete 0
I tried many things, also running everything as root (though I prefer to use ubuntu user).
There are no ERRORs in log file in /var/log/aws/codedeploy-agent.
I can also see all files and scripts dir in reviev in /opt/codedeploy-agent/deployment-root/...
When I manually run appStop script in home dir it works.
It looks like CodeDeploy agent is just not running script.
Ok it seems I made it work.
First I cleaned codedeploy-agent data by removing /opt/deployment-root/<deployment droup id> dir and /opt/deployment-root/deployment-instructions
I also changed location, don't know if this helped, but had to do it since I decided to go with root user to make things easier. App is now in /var/www/api.
I also reinstalled all js software (node, pm2, npm) using sudo
My working AppSpec.yml:
version: 0.0
os: linux
files:
- source: /
destination: /var/www/api
permissions:
- object: /var/www/api/
mode: 775
type:
- file
- directory
hooks:
ApplicationStop:
- location: scripts/appStop.sh
runas: root
ApplicationStart:
- location: scripts/appStart.sh
runas: root
and working scripts:
appStop.sh:
#!/bin/bash
cd /var/www/api
sudo pm2 delete 0
appStart.sh:
#!/bin/bash
cd /var/www/api
sudo pm2 start server.js
I'm wondering how to do log task customization in the new Elastic Beanstalk platform (the one based on Amazon Linux 2). Specifically, I'm comparing:
Old: Single-container Docker running on 64bit Amazon Linux/2.14.3
New: Single-container Docker running on 64bit Amazon Linux 2/3.0.0
(My question actually has nothing to do with Docker as such, I'm speculating the problem exist for any of the new Elastic Beanstalk platforms).
Previously I could follow Amazon's recipe, meaning put a file into /opt/elasticbeanstalk/tasks/bundlelogs.d/ and it would then be acted upon. This is no longer true.
Has this changed? I can't find it documented. Anyone been successful in doing log task customization on the newer Elastic Beanstalk platform? If so, how?
Minimal working example
I've created a minimal working example and deployed on both platforms.
Dockerfile:
FROM ubuntu
COPY daemon-run.sh /daemon-run.sh
RUN chmod +x /daemon-run.sh
EXPOSE 80
ENTRYPOINT ["/daemon-run.sh"]
Dockerrun.aws.json:
{
"AWSEBDockerrunVersion": "1",
"Logging": "/var/mydaemon"
}
daemon-run.sh:
#!/bin/bash
echo "Starting daemon" # output to stdout
mkdir -p /var/mydaemon/deeperlogs
while true; do
echo "$(date '+%Y-%m-%dT%H:%M:%S%:z') Hello World" >> /var/mydaemon/deeperlogs/app_$$.log
sleep 5
done
.ebextensions/mydaemon-logfiles.config:
files:
"/opt/elasticbeanstalk/tasks/bundlelogs.d/mydaemon-logs.conf" :
mode: "000755"
owner: root
group: root
content: |
/var/log/eb-docker/containers/eb-current-app/deeperlogs/*.log
If I do "Full Logs" action on the old platform I would get a ZIP with my deeperlogs included
inside var/log/eb-docker/containers/eb-current-app. On the new platform I don't.
Investigation
If you look on the disk you'll see that the new Elastic Beanstalk doesn't have a /opt/elasticbeanstalk/tasks folder at all, unlike the old one. Hmm.
On Amazon Linux 2 the folder is:
/opt/elasticbeanstalk/config/private/logtasks/bundle
The .ebextensions/mydaemon-logfiles.config should be:
files:
"/opt/elasticbeanstalk/config/private/logtasks/bundle/mydaemon-logs.conf":
mode: "000644"
owner: root
group: root
content: |
/var/mydaemon/deeperlogs/*.log
container_commands:
append_deeperlogs_to_applogs:
command: echo -e "\n/var/log/eb-docker/containers/eb-current-app/deeperlogs/*" >> /opt/elasticbeanstalk/config/private/logtasks/bundle/applogs
The mydaemon-logfiles.config also adds deeperlogs into applogs file. Without it deeperlogs will not be included in the download log zip bundle. Which is intresting, because the folder will be in the correct location, i.e., /var/log/eb-docker/containers/eb-current-app/deeperlogs/. But without being explicitly listed in applogs, it will be skipped when zip bundle is being generated.
I tested it with single docker environment (3.0.1).
The full log bundle successful contained deeperlogs with correct log data:
Hope that this will help. I haven't found any references for that. AWS documentaiton does not document this, as it is mostly based on Amazon Linux 1, not Amazon Linux 2.
Amazon has fixed this problem in version of the Elastic Beanstalk AL2 platforms released on 04-AUG-2020.
It has been fixed so that log task customization on AL2-based platforms now works the way it has always worked (i.e. on the prevision generation AL2018 platforms) and you can therefore follow the official documentation in order to make this happen.
Succesfully tested with platform "Docker running on 64bit Amazon Linux 2/3.1.0". If you (still) use "Docker running on 64bit Amazon Linux 2/3.0.x" then you must use the undocumented workaround described in Marcin's answer but you are probably better off by upgrading your platform version.
As of 2021/11/05, I tried the accepted answer and various other examples including the latest official documentation on using the .ebextensions folder with *.config files without success.
Most likely something I was doing wrong but here's what worked for me.
The version I'm using: Docker running on 64bit Amazon Linux 2/3.4.8
Simply, add a volume to your docker-compose.yml file to share your application logs to the Elastic Beanstalk log directory.
Example docker-compose.yml:
version: "3.9"
services:
app:
build: .
ports:
- "80:80"
user: root
volumes:
- ./:/var/www/html
# "${EB_LOG_BASE_DIR}/<service name>:<log directory inside container>
- "${EB_LOG_BASE_DIR}/app:/var/www/html/application/logs" # ADD THIS LINE
env_file:
- .env
For more info, here's the documentation I followed.
Hopefully, this helps future readers like myself 👍
I am trying to deploy a Node.js application on windows EC2 instances. Deployment finishes successfully but node server is not started automatically on those instances. I've to login to each instance to run command node app.js to start node server. I tried to include this command in appspec.yml file but then I got below error,
LifecycleEvent - ApplicationStart
Script - node_start.bat
[stdout]
[stdout]C:\Windows\system32>cd C:/host/
[stdout]
[stdout]C:\host>npm start
[stderr]'npm' is not recognized as an internal or external command,
[stderr]operable program or batch file.
My appspec.yml file is as below,
version: 0.0
os: windows
files:
- source: app.js
destination: c:\host
- source: package.json
destination: c:\host
- source: \node_modules
destination: c:\host\node_modules
- source: node_start.bat
destination: c:\host
- source: before_install.bat
destination: c:\host
hooks:
AfterInstall:
- location: before_install.bat
timeout: 300
ApplicationStart:
- location: node_start.bat
timeout: 300
Node is already installed on those two instances and Path variable is also properly set. Logging manually to those servers and running command npm start works perfectly fine. It fails only though AWS Code deploy.
I want to introduce AWS Lambda function also after this step to do health check.
Any help to resolve this issue would be greatly appreciated.
The issue isn't really related to CodeDeploy. CodeDeploy simply runs the script that you give it; in your case, node_start.bat. Perhaps, this question answers the issue that you're really having. CodeDeploy has no knowledge of your application unless you tell it.
You will likely either need to edit node_start.bat or your environment so that npm is a valid command.
Here are a couple of suggestions to help your case:
Test your appspec and scripts
You can test your appspec.yml and related scripts using the CodeDeploy local CLI, which comes with the agent.
Validate that your service is running
Obviously, it's not awesome if your deployment succeeds, but your application is not actually running. However, you can use the ValidateService lifecycle hook to confirm that your application is actually running or any other validation. You can include a script that can see that the process is running, confirm that no errors are getting logged, run tests, or whatever else you might want.
I have downloaded node.js executable. How can I run that executable as windows service?
I cannot use standard node.js installer, since I need to run multiple version of node.js concurrently.
Late to the party, but node-windows will do the trick too.
It also has system logging built in.
There is an API to create scripts from code, i.e.
var Service = require('node-windows').Service;
// Create a new service object
var svc = new Service({
name:'Hello World',
description: 'The nodejs.org example web server.',
script: 'C:\\path\\to\\helloworld.js'
});
// Listen for the "install" event, which indicates the
// process is available as a service.
svc.on('install',function(){
svc.start();
});
svc.install();
FD: I'm the author of this module.
I found the thing so useful that I built an even easier to use wrapper around it (npm, github).
Installing it:
npm install -g qckwinsvc
Installing your service:
qckwinsvc
prompt: Service name: [name for your service]
prompt: Service description: [description for it]
prompt: Node script path: [path of your node script]
Service installed
Uninstalling your service:
qckwinsvc --uninstall
prompt: Service name: [name of your service]
prompt: Node script path: [path of your node script]
Service stopped
Service uninstalled
WinSer is a node.js friendly wrapper around the popular NSSM (Non-Sucking Service Manager)
From this blog
Next up, I wanted to host node as a service, just like IIS. This way
it’d start up with my machine, run in the background, restart
automatically if it crashes and so forth.
This is where nssm, the non-sucking service manager, enters the
picture. This tool lets you host a normal .exe as a Windows service.
Here are the commands I used to setup an instance of the your node
application as a service, open your cmd like administrator and type
following commands:
nssm.exe install service_name c:\your_nodejs_directory\node.exe c:\your_application_directory\server.js
net start service_name
I'm not addressing the question directly, but providing an alternative that might also meet your requirement in a more node.js fashion way.
Functionally the requirements are:
Have the logic (app) running in the background
Be able to start/stop the logic
Automatically start the logic when system boots up
These requirements can be satisfied by using a process manager (PM) and making the process manager start on system startup. Two good PMs that are Windows-friendly are:
PM2
forever
To make the PM start automatically, the most simple way is to create a scheduled task with a "At Startup" trigger:
Since qckwinsvc has not been updated for a while there's a new version called qckwinsvc2 (npm, github)
It now supports args passed to the service. It also keeps a local cache so you don't have to provide a path every time you want to perform an action
Use the now arg to start the service as soon as it's installed
qckwinsvc2 install name="Hello" description="Hello World" path="C:\index.js" args="--y" now
qckwinsvc2 uninstall name="Hello"
qckwinsvc2 list
The process manager + task scheduler approach I posted a year ago works well with some one-off service installations. But recently I started to design system in a micro-service fashion, with many small services talking to each other via IPC. So manually configuring each service has become unbearable.
Towards the goal of installing services without manual configuration, I created serman, a command line tool (install with npm i -g serman) to install an executable as a service. All you need to write (and only write once) is a simple service configuration file along with your executable. Run
serman install <path_to_config_file>
will install the service. stdout and stderr are all logged. For more info, take a look at the project website.
A working configuration file is very simple, as demonstrated below. But it also has many useful features such as <env> and <persistent_env> below.
<service>
<id>hello</id>
<name>hello</name>
<description>This service runs the hello application</description>
<executable>node.exe</executable>
<!--
{{dir}} will be expanded to the containing directory of your
config file, which is normally where your executable locates
-->
<arguments>"{{dir}}\hello.js"</arguments>
<logmode>rotate</logmode>
<!-- OPTIONAL FEATURE:
NODE_ENV=production will be an environment variable
available to your application, but not visible outside
of your application
-->
<env name="NODE_ENV" value="production"/>
<!-- OPTIONAL FEATURE:
FOO_SERVICE_PORT=8989 will be persisted as an environment
variable machine-wide.
-->
<persistent_env name="FOO_SERVICE_PORT" value="8989" />
</service>
https://nssm.cc/ service helper good for create windows service by batch file
i use from nssm & good working for any app & any file
I have downloaded node.js executable. How can I run that executable as windows service?
I cannot use standard node.js installer, since I need to run multiple version of node.js concurrently.
Late to the party, but node-windows will do the trick too.
It also has system logging built in.
There is an API to create scripts from code, i.e.
var Service = require('node-windows').Service;
// Create a new service object
var svc = new Service({
name:'Hello World',
description: 'The nodejs.org example web server.',
script: 'C:\\path\\to\\helloworld.js'
});
// Listen for the "install" event, which indicates the
// process is available as a service.
svc.on('install',function(){
svc.start();
});
svc.install();
FD: I'm the author of this module.
I found the thing so useful that I built an even easier to use wrapper around it (npm, github).
Installing it:
npm install -g qckwinsvc
Installing your service:
qckwinsvc
prompt: Service name: [name for your service]
prompt: Service description: [description for it]
prompt: Node script path: [path of your node script]
Service installed
Uninstalling your service:
qckwinsvc --uninstall
prompt: Service name: [name of your service]
prompt: Node script path: [path of your node script]
Service stopped
Service uninstalled
WinSer is a node.js friendly wrapper around the popular NSSM (Non-Sucking Service Manager)
From this blog
Next up, I wanted to host node as a service, just like IIS. This way
it’d start up with my machine, run in the background, restart
automatically if it crashes and so forth.
This is where nssm, the non-sucking service manager, enters the
picture. This tool lets you host a normal .exe as a Windows service.
Here are the commands I used to setup an instance of the your node
application as a service, open your cmd like administrator and type
following commands:
nssm.exe install service_name c:\your_nodejs_directory\node.exe c:\your_application_directory\server.js
net start service_name
I'm not addressing the question directly, but providing an alternative that might also meet your requirement in a more node.js fashion way.
Functionally the requirements are:
Have the logic (app) running in the background
Be able to start/stop the logic
Automatically start the logic when system boots up
These requirements can be satisfied by using a process manager (PM) and making the process manager start on system startup. Two good PMs that are Windows-friendly are:
PM2
forever
To make the PM start automatically, the most simple way is to create a scheduled task with a "At Startup" trigger:
Since qckwinsvc has not been updated for a while there's a new version called qckwinsvc2 (npm, github)
It now supports args passed to the service. It also keeps a local cache so you don't have to provide a path every time you want to perform an action
Use the now arg to start the service as soon as it's installed
qckwinsvc2 install name="Hello" description="Hello World" path="C:\index.js" args="--y" now
qckwinsvc2 uninstall name="Hello"
qckwinsvc2 list
The process manager + task scheduler approach I posted a year ago works well with some one-off service installations. But recently I started to design system in a micro-service fashion, with many small services talking to each other via IPC. So manually configuring each service has become unbearable.
Towards the goal of installing services without manual configuration, I created serman, a command line tool (install with npm i -g serman) to install an executable as a service. All you need to write (and only write once) is a simple service configuration file along with your executable. Run
serman install <path_to_config_file>
will install the service. stdout and stderr are all logged. For more info, take a look at the project website.
A working configuration file is very simple, as demonstrated below. But it also has many useful features such as <env> and <persistent_env> below.
<service>
<id>hello</id>
<name>hello</name>
<description>This service runs the hello application</description>
<executable>node.exe</executable>
<!--
{{dir}} will be expanded to the containing directory of your
config file, which is normally where your executable locates
-->
<arguments>"{{dir}}\hello.js"</arguments>
<logmode>rotate</logmode>
<!-- OPTIONAL FEATURE:
NODE_ENV=production will be an environment variable
available to your application, but not visible outside
of your application
-->
<env name="NODE_ENV" value="production"/>
<!-- OPTIONAL FEATURE:
FOO_SERVICE_PORT=8989 will be persisted as an environment
variable machine-wide.
-->
<persistent_env name="FOO_SERVICE_PORT" value="8989" />
</service>
https://nssm.cc/ service helper good for create windows service by batch file
i use from nssm & good working for any app & any file