I have been trying to create CI/CD using code deploy and bitbucket repo.
The pipeline is successful but I am not seeking any codes into ec2. I can only see the node module in ec2.
If anyone came through the same issues or could help me to solve them that would be great.
appspec.yml
version: 0.0
os: linux
files:
- source: /
destination: /home/ubuntu/gt
hooks:
ApplicationStart:
- location: scripts/start_server.sh
runas: root
start_server.sh
sudo apt-get update
# install the application using npm
# we need to traverse to where the application bundle is copied too.
#some comments
#added commets
sudo su
rm -rf /home/ubuntu/gt
mkdir /home/ubuntu/gt
echo installing application with npm
cd /home/ubuntu/gt
sudo apt-get install -y npm
echo installing pm2
npm install pm2 -g
sudo yarn
pm2 delete gt
pm2 start npm --name 'gt' -- start
I am not seeking any codes into ec2
This could be because you are removing all content of your folder:
rm -rf /home/ubuntu/gt
since your ApplicationStart runs after files. So whatever you copy in files, gets deleted in ApplicationStart. For the order of execution, please have a look here.
Hi I am building docker image from node js project.
docker files contents are like below
fetch code from git repo.
install npm
install grunt cli
grunt build
please see below docker file
FROM ubuntu:latest
# Install Node.js and npm
RUN apt-get -y update
RUN apt-get -y install nodejs npm git wget curl
RUN git config --global http.sslverify false
WORKDIR /somerepo/
RUN git clone -somerepo
WORKDIR /somerepo/src/
RUN npm cache clean -f
RUN npm install -g n
RUN n stable
RUN npm install
RUN npm install -g grunt-cli
RUN grunt build
EXPOSE 3000
CMD ["node", "web.js"]
but its grunt build step is failing below error in logs
Step 14/15 : RUN grunt build
---> Running in d0f8863081f1
Local Npm module "jshint-stylish" not found. Is it installed?
Running "clean:dist" (clean) task
...
...
...
Running "githooks:all" (githooks) task
Binding jsbeautifier:pregit to pre-commit Git hook.
OK
Running "jshint:dev" (jshint) task
Warning: The "path" argument must be of type string. Received type object Use --force to continue.
Do any have any idea why grunt build is not ending successufully ?
I have already tried --save-dev option and also tried manually installing jshint with below command
#RUN npm install --save-dev jshint-stylish
Also i tried to print the files/directories having pattern "jshint" with below command in dockerfile
RUN find . -name "*jshint*"
and its printing below output in logs
Step 13/15 : RUN find . -name "*jshint*"
---> Running in db355c19f75a
./node_modules/.bin/jshint
./node_modules/ascii-table/.jshintrc
./node_modules/bl/.jshintrc
./node_modules/errno/.jshintrc
./node_modules/escodegen/.jshintrc
./node_modules/estraverse/.jshintrc
./node_modules/exit/.jshintrc
./node_modules/findup-sync/.jshintrc
./node_modules/gaze/.jshintrc
./node_modules/getobject/.jshintrc
./node_modules/globule/.jshintrc
./node_modules/grunt-contrib-clean/.jshintrc
./node_modules/grunt-contrib-compress/.jshintrc
./node_modules/grunt-contrib-concat/.jshintrc
./node_modules/grunt-contrib-cssmin/.jshintrc
./node_modules/grunt-contrib-jshint
./node_modules/grunt-contrib-jshint/tasks/jshint.js
./node_modules/grunt-contrib-jshint/tasks/lib/jshint.js
./node_modules/grunt-contrib-uglify/.jshintrc
./node_modules/grunt-contrib-watch/.jshintrc
./node_modules/grunt-express-server/.jshintrc
./node_modules/grunt-git-describe/.jshintrc
./node_modules/grunt-githooks/.jshintrc
./node_modules/grunt-img/node_modules/.bin/jshint
./node_modules/grunt-img/node_modules/grunt-contrib-jshint
./node_modules/grunt-img/node_modules/grunt-contrib-jshint/.jshintrc
./node_modules/grunt-img/node_modules/grunt-contrib-jshint/docs/jshint-examples.md
./node_modules/grunt-img/node_modules/grunt-contrib-jshint/docs/jshint-options.md
./node_modules/grunt-img/node_modules/grunt-contrib-jshint/docs/jshint-overview.md
./node_modules/grunt-img/node_modules/grunt-contrib-jshint/tasks/jshint.js
./node_modules/grunt-img/node_modules/grunt-contrib-jshint/tasks/lib/jshint.js
./node_modules/grunt-img/node_modules/grunt-contrib-jshint/test/jshint_test.js
./node_modules/grunt-img/node_modules/jshint
./node_modules/grunt-img/node_modules/jshint/packages/jshint
./node_modules/grunt-img/node_modules/jshint/packages/jshint/jshint.js
./node_modules/grunt-jsbeautifier/.jshintrc
./node_modules/grunt-json-replace/.jshintrc
./node_modules/grunt-legacy-log/.jshintrc
./node_modules/grunt-legacy-log-utils/.jshintrc
./node_modules/grunt-legacy-util/.jshintrc
./node_modules/grunt-lib-contrib/.jshintrc
./node_modules/grunt-manifest/.jshintrc
./node_modules/grunt-prettify/.jshintrc
./node_modules/grunt-sloc/.jshintrc
./node_modules/grunt-sloccount/.jshintrc
./node_modules/grunt-text-replace/.jshintrc
./node_modules/handlebars/.jshintrc
./node_modules/ibrik/node_modules/estraverse/.jshintrc
./node_modules/isstream/.jshintrc
./node_modules/istanbul/node_modules/estraverse/.jshintrc
./node_modules/jshint
./node_modules/jshint/bin/jshint
./node_modules/jshint/src/jshint.js
./node_modules/jshint-stylish
./node_modules/keep-alive-agent/.jshintrc
./node_modules/less/.idea/jsLinters/jshint.xml
./node_modules/less/.jshintrc
./node_modules/log4js/.jshintrc
./node_modules/nodemon/.jshintrc
./node_modules/phantomjs/.jshintrc
./node_modules/promise/.jshintrc
./node_modules/prr/.jshintrc
./node_modules/request-progress/.jshintrc
./node_modules/resolve-url/.jshintrc
./node_modules/restify/node_modules/qs/.jshintignore
./node_modules/restify/node_modules/qs/.jshintrc
./node_modules/save/.jshintignore
./node_modules/save/.jshintrc
./node_modules/shelljs/jshint.json
./node_modules/source-map-resolve/.jshintrc
./node_modules/source-map-url/.jshintrc
./node_modules/undefsafe/.jshintrc
./node_modules/urix/.jshintrc
./node_modules/xtend/.jshintrc
if anyone have any idea what is wrong please let me know
Thanks in advance !!!
Isit possible to configure EBS to install my NodeJS application using yarn package manager instead of NPM?
I've figured out a way, but it is a little hacky.
Create a .ebextensions/yarn.config file. (The name does not have to be 'yarn'.)
Put this content into the file:
files:
# Runs right before `npm install` in '.../50npm.sh'
"/opt/elasticbeanstalk/hooks/appdeploy/pre/49yarn.sh" :
mode: "000775"
owner: root
group: users
content: |
#!/bin/bash
app="$(/opt/elasticbeanstalk/bin/get-config container -k app_staging_dir)";
# install node
curl --silent --location https://rpm.nodesource.com/setup_8.x | bash -;
# install yarn
curl --silent --location https://dl.yarnpkg.com/rpm/yarn.repo | tee /etc/yum.repos.d/yarn.repo;
yum -y install yarn;
# install node_modules with yarn
cd "${app}";
yarn --production;
This ebextension creates a file which does 3 things:
Installs node.
Installs yarn.
Installs node_modules with yarn.
In order to make Elastic Beanstalk run yarn install before it runs npm install, the file is created under /opt/elasticbeanstalk/hooks/appdeploy/pre. This turns the file into a pre-deployment hook, which means that Elastic Beanstalk will run it during the first phase of deployment. By default, there is another file in this directory called 50npm.sh, which runs npm install. Since Elastic Beanstalk runs the files in this directory alphabetically, 49yarn.sh (our file) will run before 50npm.sh (the default file), resulting in yarn install running before npm install.
One potential problem is that the environment variables set in the Elastic Beanstalk UI (under Configuration > Software Configuration) are not available at this point of the deployment phase. This is a big problem if you have an npm auth token there which you use to install private npm modules.
Another potential problem is that this installs node manually, so the "Node version" you specify in the Elastic Beanstalk UI (under Configuration > Software Configuration) will have no effect on the version of node your application uses; you need to specify it in this ebextension. Elastic Beanstalk's 50npm.sh both installs node and runs npm install. Since we have to run yarn install before that file runs, we also have to install node manually. Then, when Elastic Beanstalk goes to install node, it detects that node is already installed but does not verify that it is the correct version, so it skips the node installation.
For reference, the yarn installation instructions came from here: https://yarnpkg.com/docs/install#linux-tab
I did this following instructions on https://yarnpkg.com/lang/en/docs/install/
commands:
01_install_yarn:
command: "sudo wget https://dl.yarnpkg.com/rpm/yarn.repo -O /etc/yum.repos.d/yarn.repo && curl --silent --location https://rpm.nodesource.com/setup_6.x | sudo bash - && sudo yum install yarn -y"
This way that i came up with lets you still control the node version via the Elastic Beanstalks Dashboard.
Thanks for this question! couldn't have come to this solution without it :D
"/opt/elasticbeanstalk/hooks/appdeploy/pre/50npm.sh":
mode: "000755"
owner: root
group: users
content: |
#!/usr/bin/env bash
#
# Prevent installing or rebuilding like Elastic Beanstalk tries to do by
# default.
#
# Note that this *overwrites* Elastic Beanstalk's default 50npm.sh script
# (https://gist.github.com/wearhere/de51bb799f5099cec0ed28b9d0eb3663).
"/opt/elasticbeanstalk/hooks/configdeploy/pre/50npm.sh":
mode: "000755"
owner: root
group: users
content: |
#!/usr/bin/env bash
#
# Prevent installing or rebuilding like Elastic Beanstalk tries to do by
# default.
#
# Note that this *overwrites* Elastic Beanstalk's default 50npm.sh script.
# But their default script actually doesn't work at all, since the app
# staging dir, where they try to run `npm install`, doesn't exist during
# config deploys, so ebnode.py just aborts:
# https://gist.github.com/wearhere/de51bb799f5099cec0ed28b9d0eb3663#file-ebnode-py-L140
"/opt/elasticbeanstalk/hooks/appdeploy/pre/49yarn.sh" :
mode: "000775"
owner: root
group: users
content: |
tmp="$(mktemp || bail)";
app="$(/opt/elasticbeanstalk/bin/get-config container -k app_staging_dir)";
version="$(/opt/elasticbeanstalk/bin/get-config optionsettings -n aws:elasticbeanstalk:container:nodejs -o NodeVersion)";
echo $version
major="$(cut -d'.' -f1 <<<${version})"
yum -y install python26 python26-libs
wget https://dl.yarnpkg.com/rpm/yarn.repo -O /etc/yum.repos.d/yarn.repo;
wget "https://rpm.nodesource.com/pub_${major}.x/el/7/x86_64/nodejs-${version}-1nodesource.x86_64.rpm" -O "${tmp}";
rpm -i --nosignature --force "${tmp}";
rm -f "${tmp}";
yum -y install yarn;
cd "${app}";
yarn --production;
Had to revisit this as we couldn't figure out why we were stuck on node 8 even though we set it to node 12 in the EB UI. Seems that if you install a global node it overrides the version setting. Instead of installing a global node, this uses the Elastic Beanstalk node install and adds it to the path. You have to add the PATH in again at the start of your yarn install script but it seems to be the least invasive way to use yarn.
content: |
#!/usr/bin/env bash
set -euxo pipefail
EB_NODE_VERSION=$(/opt/elasticbeanstalk/bin/get-config optionsettings -n aws:elasticbeanstalk:container:nodejs -o NodeVersion)
echo "EB node version: $(EB_NODE_VERSION)"
# Make sure Node binaries can be found (required to run npm).
# And this lets us invoke npm more simply too.
export PATH=/opt/elasticbeanstalk/node-install/node-v$EB_NODE_VERSION-linux-x64/bin:$PATH
if yarn -v; then
echo 'Yarn already installed.'
else
echo 'Installing yarn...'
npm install yarn -g
fi
An easy way to prevent EB from running npm install is to create an empty node_modules folder in a prebuild hook:
Edit your_project/.platform/hooks/prebuild/prevent-npm.sh:
#!/bin/bash
# EB build scripts will not install using npm if node_modules folder exists
mkdir node_modules
This way, EB will still install Node.js, so you don't have to install that yourself. But it will skip npm install when node_modules exists.
Then inside the predeploy hook, you can run yarn. If you are running Node.js 16 or newer, you can use corepack yarn, and it will just work (you don't have to install yarn from yum, npm or anything else.)
Edit your_project/.platform/hooks/predeploy/yarn.sh:
#!/bin/bash
corepack yarn
Be sure to also create symlinks under confighooks in your project, or else it will fail to build when changing config (e.g. updating environment values):
mkdir -p .platform/confighooks/{prebuild,predeploy}
ln -s ../../hooks/predeploy/yarn.sh .platform/confighooks/predeploy/yarn.sh
ln -s ../../hooks/prebuild/prevent-npm.sh .platform/confighooks/prebuild/prevent-npm.sh
Since get-config is no longer present in the new Amazon Linux 2 platform, we had to figure another clean way to do this, and came up with the following :
container_commands:
01_npm_install_yarn:
command: "npm install -g yarn"
10_yarn_install:
command: 'PATH="$PATH:$(dirname $(readlink $(which node)))" yarn install'
You may want to put the PATH= logic in a script and call it before every yarn command, to have clean command: instructions in your extentions.
Also, note that if you install yarn using the yum package manager, you completely break the NodeJS version management provided by Beanstalk (since it the black magic running behind make some symlinks in /bin and /usr/bin).
I am trying to deploy an application on AWS Elastic Beanstalk. I have the following file in .ebextensions:
commands:
01-install-git:
command: "yum install -y git &>> /tmp/deploy.log"
02-install-nodejs-npm:
command: "yum install -y --enablerepo=epel nodejs npm &>> /tmp/deploy.log"
03-install-grunt:
command: "npm install -g grunt-cli &>> /tmp/deploy.log"
04-install-coffee:
command: "npm install -g coffee-script &>> /tmp/deploy.log"
05-install-bower:
command: "npm install -g bower &>> /tmp/deploy.log"
container_commands:
01_grunt:
command: "export PATH=/usr/local/bin:/bin:/usr/bin; grunt prod &>> /tmp/deploy.log"
Basically, I want to run grunt prod and that will download bower dependencies, compile my coffeescript, minify/concat my js and some other stuff. The git, nodejs, grunt, coffee, and bower installation works fine (I can ssh and confirm that the commands are available and on the path). However, if I don't include the export PATH=/usr/local/bin:/bin:/usr/bin; part when calling bower, I get:
Running "bower:install" (bower) task
Fatal error: git is not installed or not in the PATH
I tried to debug and add which git &>> /tmp/deploy.log but got which: no git in ((null)). However if I do echo $PATH &>> /tmp/deploy.log I get a good looking path: /usr/local/bin:/bin:/usr/bin
Question is: why do I need to specify the path when calling bower?
After a lot of debugging, it seems like the PATH is set but not exported. I only needed to add export PATH=$PATH; before calling grunt:
container_commands:
01_grunt:
command: "export PATH=$PATH; grunt prod &>> /tmp/deploy.log"
Note that you must execute the PATH=$PATH within the same command:.
This does not work...
container_commands:
01_path:
command: "export PATH=$PATH; echo $PATH &>> /tmp/01_path.log"
ignoreErrors: true
02_bower_install:
command: "$NODE_HOME/bin/node ./node_modules/bower/bin/bower install --allow-root &>> /tmp/02_bower_install.log"
ignoreErrors: true
Fails with...
bower no-home HOME environment variable not set. User config will not be loaded.
ENOGIT git is not installed or not in the PATH
NB: As container_command is executed as root, you must use bower install --allow-root
I'm trying to install an npm package globally on elastic beanstalk. This is what my config file looks like which I wrote based on this documentation.
container_commands:
install_phantom:
command: "npm install phantomjs -g"
And when I deploy to Elastic Beanstalk I get this error
Command failed on instance. Return code: 1 Output: Error occurred
during build: Command install_phantom failed .
Based on the answer given here, have you tried:
container_commands:
install_phantom:
command: "export PATH=$PATH; npm install phantomjs -g"
The environment variable for the node installation is NODE_HOME, so you should do this, to run npm or node in a container command in your config files:
container_commands:
install_phantom:
command: bash -c "PATH=$PATH:$NODE_HOME/bin npm install phantomjs -g"