I am trying to deploy an application on AWS Elastic Beanstalk. I have the following file in .ebextensions:
commands:
01-install-git:
command: "yum install -y git &>> /tmp/deploy.log"
02-install-nodejs-npm:
command: "yum install -y --enablerepo=epel nodejs npm &>> /tmp/deploy.log"
03-install-grunt:
command: "npm install -g grunt-cli &>> /tmp/deploy.log"
04-install-coffee:
command: "npm install -g coffee-script &>> /tmp/deploy.log"
05-install-bower:
command: "npm install -g bower &>> /tmp/deploy.log"
container_commands:
01_grunt:
command: "export PATH=/usr/local/bin:/bin:/usr/bin; grunt prod &>> /tmp/deploy.log"
Basically, I want to run grunt prod and that will download bower dependencies, compile my coffeescript, minify/concat my js and some other stuff. The git, nodejs, grunt, coffee, and bower installation works fine (I can ssh and confirm that the commands are available and on the path). However, if I don't include the export PATH=/usr/local/bin:/bin:/usr/bin; part when calling bower, I get:
Running "bower:install" (bower) task
Fatal error: git is not installed or not in the PATH
I tried to debug and add which git &>> /tmp/deploy.log but got which: no git in ((null)). However if I do echo $PATH &>> /tmp/deploy.log I get a good looking path: /usr/local/bin:/bin:/usr/bin
Question is: why do I need to specify the path when calling bower?
After a lot of debugging, it seems like the PATH is set but not exported. I only needed to add export PATH=$PATH; before calling grunt:
container_commands:
01_grunt:
command: "export PATH=$PATH; grunt prod &>> /tmp/deploy.log"
Note that you must execute the PATH=$PATH within the same command:.
This does not work...
container_commands:
01_path:
command: "export PATH=$PATH; echo $PATH &>> /tmp/01_path.log"
ignoreErrors: true
02_bower_install:
command: "$NODE_HOME/bin/node ./node_modules/bower/bin/bower install --allow-root &>> /tmp/02_bower_install.log"
ignoreErrors: true
Fails with...
bower no-home HOME environment variable not set. User config will not be loaded.
ENOGIT git is not installed or not in the PATH
NB: As container_command is executed as root, you must use bower install --allow-root
Related
I try to deploy a Laravel repo on AWS Elastic Beanstalk. For this I've created a PHP environment on a t2.medium instance and initialized a deployment with AWS CodePipeline.
I need to run npm and gulp on production for creating my static assets, but unfortunately I'm not able to run NPM because of an old NodeJS version installed. It's 0.10.46-1nodesource.el7.centos.
At the moment I'm using two config scripts to install NodeJS 10.X, run the migrations and gulp:
First:
commands:
01getNodeRepo:
command: "curl --silent --location https://rpm.nodesource.com/setup_10.x | bash -"
02installNode:
command: "yum install -y nodejs"
03updateNpm:
command: "npm install npm -g"
04enableSudo:
command: "echo Defaults:root \\!requiretty >> /etc/sudoers"
Second:
container_commands:
01artisanMigrate:
command: "php artisan migrate --force"
02showNodeVersion:
command: "node -v"
03showNpmVersion:
command: "npm -v"
04npmInstall:
command: "sudo npm install"
05gulp:
command: "sudo ./node_modules/.bin/gulp --production"
Here is an extract from my log files which shows that AWS EB found two nodesource repos and will be using the old version:
[2019-01-01T19:32:23.008Z] INFO [1535] - [Application update code-pipeline-xx-xxx#17/AppDeployStage0/EbExtensionPreBuild/Infra-EmbeddedPreBuild/xxxx/Command 02installNode] : Starting activity...
[2019-01-01T19:32:25.314Z] INFO [1535] - [Application update code-pipeline-xx-xxx#17/AppDeployStage0/EbExtensionPreBuild/Infra-EmbeddedPreBuild/xxxx/Command 02installNode] : Completed activity. Result:
Loaded plugins: priorities, update-motd, upgrade-helper
Repository nodesource is listed more than once in the configuration
Repository nodesource-source is listed more than once in the configuration
Resolving Dependencies
--> Running transaction check
---> Package nodejs.x86_64 0:0.10.46-1nodesource.el7.centos will be installed
--> Finished Dependency Resolution
Any ideas how I can force the environment to use the new version? Thanks!
Got it. This was the solution: https://github.com/nodesource/distributions/issues/421#issuecomment-318560799
rm -f /etc/yum.repos.d/nodesource-el.repo
yum clean all
yum -y remove nodejs
yum -y install nodejs
I'm running a nodejs application's build on jenkins. I run node as shell script step, because I have some limitations in terms of the plugins I can install in this jenkins instance.
This is what the npm install step looks like:
#!/bin/bash +x
source ~/.bashrc
cd $WORKSPACE/ && \
nvm use 7.8.0 && node --version && npm install
The problem I have is, when npm install finishes, it doesn't install everything. If I ssh into the box where jenkins is installed and run npm install inside that project's workspace, with the same user jenkins uses, it works ok. Any ideas?
EDIT
I just realized the dependencies that it's not installing are devDependencies
The problem was I had the node env set to production, and of course, it wouldn't get the devDependencies...
Isit possible to configure EBS to install my NodeJS application using yarn package manager instead of NPM?
I've figured out a way, but it is a little hacky.
Create a .ebextensions/yarn.config file. (The name does not have to be 'yarn'.)
Put this content into the file:
files:
# Runs right before `npm install` in '.../50npm.sh'
"/opt/elasticbeanstalk/hooks/appdeploy/pre/49yarn.sh" :
mode: "000775"
owner: root
group: users
content: |
#!/bin/bash
app="$(/opt/elasticbeanstalk/bin/get-config container -k app_staging_dir)";
# install node
curl --silent --location https://rpm.nodesource.com/setup_8.x | bash -;
# install yarn
curl --silent --location https://dl.yarnpkg.com/rpm/yarn.repo | tee /etc/yum.repos.d/yarn.repo;
yum -y install yarn;
# install node_modules with yarn
cd "${app}";
yarn --production;
This ebextension creates a file which does 3 things:
Installs node.
Installs yarn.
Installs node_modules with yarn.
In order to make Elastic Beanstalk run yarn install before it runs npm install, the file is created under /opt/elasticbeanstalk/hooks/appdeploy/pre. This turns the file into a pre-deployment hook, which means that Elastic Beanstalk will run it during the first phase of deployment. By default, there is another file in this directory called 50npm.sh, which runs npm install. Since Elastic Beanstalk runs the files in this directory alphabetically, 49yarn.sh (our file) will run before 50npm.sh (the default file), resulting in yarn install running before npm install.
One potential problem is that the environment variables set in the Elastic Beanstalk UI (under Configuration > Software Configuration) are not available at this point of the deployment phase. This is a big problem if you have an npm auth token there which you use to install private npm modules.
Another potential problem is that this installs node manually, so the "Node version" you specify in the Elastic Beanstalk UI (under Configuration > Software Configuration) will have no effect on the version of node your application uses; you need to specify it in this ebextension. Elastic Beanstalk's 50npm.sh both installs node and runs npm install. Since we have to run yarn install before that file runs, we also have to install node manually. Then, when Elastic Beanstalk goes to install node, it detects that node is already installed but does not verify that it is the correct version, so it skips the node installation.
For reference, the yarn installation instructions came from here: https://yarnpkg.com/docs/install#linux-tab
I did this following instructions on https://yarnpkg.com/lang/en/docs/install/
commands:
01_install_yarn:
command: "sudo wget https://dl.yarnpkg.com/rpm/yarn.repo -O /etc/yum.repos.d/yarn.repo && curl --silent --location https://rpm.nodesource.com/setup_6.x | sudo bash - && sudo yum install yarn -y"
This way that i came up with lets you still control the node version via the Elastic Beanstalks Dashboard.
Thanks for this question! couldn't have come to this solution without it :D
"/opt/elasticbeanstalk/hooks/appdeploy/pre/50npm.sh":
mode: "000755"
owner: root
group: users
content: |
#!/usr/bin/env bash
#
# Prevent installing or rebuilding like Elastic Beanstalk tries to do by
# default.
#
# Note that this *overwrites* Elastic Beanstalk's default 50npm.sh script
# (https://gist.github.com/wearhere/de51bb799f5099cec0ed28b9d0eb3663).
"/opt/elasticbeanstalk/hooks/configdeploy/pre/50npm.sh":
mode: "000755"
owner: root
group: users
content: |
#!/usr/bin/env bash
#
# Prevent installing or rebuilding like Elastic Beanstalk tries to do by
# default.
#
# Note that this *overwrites* Elastic Beanstalk's default 50npm.sh script.
# But their default script actually doesn't work at all, since the app
# staging dir, where they try to run `npm install`, doesn't exist during
# config deploys, so ebnode.py just aborts:
# https://gist.github.com/wearhere/de51bb799f5099cec0ed28b9d0eb3663#file-ebnode-py-L140
"/opt/elasticbeanstalk/hooks/appdeploy/pre/49yarn.sh" :
mode: "000775"
owner: root
group: users
content: |
tmp="$(mktemp || bail)";
app="$(/opt/elasticbeanstalk/bin/get-config container -k app_staging_dir)";
version="$(/opt/elasticbeanstalk/bin/get-config optionsettings -n aws:elasticbeanstalk:container:nodejs -o NodeVersion)";
echo $version
major="$(cut -d'.' -f1 <<<${version})"
yum -y install python26 python26-libs
wget https://dl.yarnpkg.com/rpm/yarn.repo -O /etc/yum.repos.d/yarn.repo;
wget "https://rpm.nodesource.com/pub_${major}.x/el/7/x86_64/nodejs-${version}-1nodesource.x86_64.rpm" -O "${tmp}";
rpm -i --nosignature --force "${tmp}";
rm -f "${tmp}";
yum -y install yarn;
cd "${app}";
yarn --production;
Had to revisit this as we couldn't figure out why we were stuck on node 8 even though we set it to node 12 in the EB UI. Seems that if you install a global node it overrides the version setting. Instead of installing a global node, this uses the Elastic Beanstalk node install and adds it to the path. You have to add the PATH in again at the start of your yarn install script but it seems to be the least invasive way to use yarn.
content: |
#!/usr/bin/env bash
set -euxo pipefail
EB_NODE_VERSION=$(/opt/elasticbeanstalk/bin/get-config optionsettings -n aws:elasticbeanstalk:container:nodejs -o NodeVersion)
echo "EB node version: $(EB_NODE_VERSION)"
# Make sure Node binaries can be found (required to run npm).
# And this lets us invoke npm more simply too.
export PATH=/opt/elasticbeanstalk/node-install/node-v$EB_NODE_VERSION-linux-x64/bin:$PATH
if yarn -v; then
echo 'Yarn already installed.'
else
echo 'Installing yarn...'
npm install yarn -g
fi
An easy way to prevent EB from running npm install is to create an empty node_modules folder in a prebuild hook:
Edit your_project/.platform/hooks/prebuild/prevent-npm.sh:
#!/bin/bash
# EB build scripts will not install using npm if node_modules folder exists
mkdir node_modules
This way, EB will still install Node.js, so you don't have to install that yourself. But it will skip npm install when node_modules exists.
Then inside the predeploy hook, you can run yarn. If you are running Node.js 16 or newer, you can use corepack yarn, and it will just work (you don't have to install yarn from yum, npm or anything else.)
Edit your_project/.platform/hooks/predeploy/yarn.sh:
#!/bin/bash
corepack yarn
Be sure to also create symlinks under confighooks in your project, or else it will fail to build when changing config (e.g. updating environment values):
mkdir -p .platform/confighooks/{prebuild,predeploy}
ln -s ../../hooks/predeploy/yarn.sh .platform/confighooks/predeploy/yarn.sh
ln -s ../../hooks/prebuild/prevent-npm.sh .platform/confighooks/prebuild/prevent-npm.sh
Since get-config is no longer present in the new Amazon Linux 2 platform, we had to figure another clean way to do this, and came up with the following :
container_commands:
01_npm_install_yarn:
command: "npm install -g yarn"
10_yarn_install:
command: 'PATH="$PATH:$(dirname $(readlink $(which node)))" yarn install'
You may want to put the PATH= logic in a script and call it before every yarn command, to have clean command: instructions in your extentions.
Also, note that if you install yarn using the yum package manager, you completely break the NodeJS version management provided by Beanstalk (since it the black magic running behind make some symlinks in /bin and /usr/bin).
I have a local development server where I test a lot of things, now I'm playing with bower to manage the libraries' dependencies in my Symfony2 project. After getting NodeJS (v0.10.31) installed and bower (1.3.9), I tried to run the command sp:bower:install which belongs to Symfony2 SpBowerBundle from console as root:
Symfony > sp:bower:install
Installing bower dependencies for "TemplateBundle" into "/var/www/html/tanane/src/Tanane/TemplateBundle/Resources/config/bower/../../public/components"
bower ESUDO Cannot be run with sudo
Additional error details:
Since bower is a user command, there is no need to execute it with superuser permissions.
If you're having permission errors when using bower without sudo, please spend a few minutes learning more about how your system should work and make any necessary repairs.
http://www.joyent.com/blog/installing-node-and-npm
https://gist.github.com/isaacs/579814
You can however run a command with sudo using --allow-root option
I know that adding --allow-root works since I tested directly from bash but it apparently isn't allowed from the bundle command line. Now, is the only way to run bower as root to add --allow-root or does it exist another way ?
below answer is for symfony framework's bundle,
but if you come here from google using phrase "bower root" you have two options to solve that:
add --allow-root to command
set global bower config that will allow
running bower as root
Option 1: you can run bower as root by typing:
bower install --allow-root
root is allowed by setting --allow-root command parameter
Option 2: is using global setting that allows root, by creating file:
/root/.bowerrc
which have inside following configuration:
{ "allow_root": true }
how to do this in SpBowerBundle symfony bundle:
probably you haven't set sp_bower.allow_root to true in SpBowerBundle config
in bundle config, by default you have set something like this:
allow_root: false # optional
but you should have:
allow_root: true
so in app/config/config.yml add this bundle config
sp_bower:
allow_root: false # optional
bundle config reference (all settings):
https://github.com/Spea/SpBowerBundle/blob/master/Resources/doc/configuration_reference.md
If you are encountering this issue on Docker containers just add this line in your Dockerfile:
RUN echo '{ "allow_root": true }' > /root/.bowerrc
I fixed a similar problem by changing the directory permissions:
sudo chown -R $USER:$GROUP ~/.npm
sudo chown -R $USER:$GROUP ~/.config
This might be stupid but for me bower install --allow-root did not work but bower --allow-root install did, using grunt-bower-install version 1.6.0
This was on a docker running with root user, perhaps will save someone some time :)
Faced similar issue when installing swagger-editor.
Changed the following line in package.json from
"bower-install": "bower install"
to
"bower-install": "bower install --allow-root"
This works for me (add -u parameter on docker run)
bash
docker run -it -v ${PWD}:/www -w /www -u node node ./node_modules/bower/bin/bower install
For My case it is in Pom.xml where i have added as an argument as below:
<executable>bower</executable>
<arguments>
<argument>install</argument>
<argument>--allow-root</argument>
</arguments>
If need to avoid this --allow--root parameter we can do a compile out of root user
remover / delete bower_components
and reinstall bower
bower install
Yeoman and bower do not work at all. I installed node using the Mac node package downloaded from the nodejs site.
npm: 1.4.21 / node: v0.10.30
.bash_profile:
export PATH=$HOME/local/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin
export M2_HOME=/Users/cmorrow/apache-maven-3.1.1
export PATH=$PATH:$M2_HOME/bin
export JAVA_HOME=/Library/Java/Home
[[ -s "$HOME/.rvm/scripts/rvm" ]] && source "$HOME/.rvm/scripts/rvm" # Load RVM into a shell session *as a function*
export PATH=/usr/local/bin:$PATH
export PATH=/Users/cmorrow/npm/lib/node_modules:$PATH
I installed yo with:
npm install -g yo
The post install reads:
yo#1.2.1 postinstall /Users/cmorrow/npm/lib/node_modules/yo
node ./scripts/doctor
[Yeoman Doctor] Everything looks alright!
If you install nodejs by HomeBrew he manually create a symbolic link in the folder
/usr/local/bin
if the manually installation don't generate that symbolic link you can doing manually with this command (with sudo):
ln -s /Users/cmorrow/npm/lib/node_modules/yo/cli.js /usr/local/bin/yo
chmod +x /usr/local/bin/yo
Not sure the exact fix, but I made a few changes and everything now works.
Things I did were:
Upgraded node js to v0.10.31
sudo npm cache clean -f
sudo npm install -g n
sudo n stable
Reinstalled yeoman
sudo npm install -g yo