I have a Node/Angular app which works locally in a Docker instance. For deployment to a Google VM, we have a script which checks out the Git files, compiles them, copies the outputs into an nginx folder and restarts the nginx.
I have (what I thought was) identical setups on two different VMs. I test any changes on the "staging" VM and then if all is working correctly, I run them on the "production" VM.
Yesterday, this system broke. My staging VM pulls, compiles and runs the code without any issues or even warnings. The production VM gives errors during the compilation
The compilation line is
sudo $(npm bin)/ng build --prod --output-path=dist
and the first error I see is
ERROR in : Unexpected value 'AngularFireDatabaseModule in /home/redacted/ng-app/node_modules/#angular/fire/database/angular-fire-database.d.ts' imported by the module 'AppModule in /home/redacted/ng-app/src/app/app.module.ts'. Please add a #NgModule annotation.
So, I saw this question which said I should move import to the declarations section. I did this for AngularFireDatabaseModule and then it would not compile on my (working) staging server, giving me an error
Please add a #Pipe/#Directive/#Component annotation
Which this answer says means I need to move the declaration back to the imports!
My assumption is that my two supposedly identical VMs have something different about them, but I am having difficulty figuring out what.
Is there some kind soul who can help me figure out what the difference is?
Both VMs: Running Ubuntu 20.04.3 LTS (GNU/Linux 5.11.0-1028-gcp x86_64)
Both VMs up to date
Both VMs: node --version
v17.4.0
Both VMs: sudo node --version
v13.14.0
Both VMs: npm -v
8.3.1
Both VMs: sudo npm -v
6.14.4
What else should I be checking for?
OK, I can't be sure what exactly fixed it, but in the end, I had to go into the folder where the compilation was occurring and run
sudo npm update
which seemd to fix the problems. Could've sworn I did that a few times before, but ¯\(ツ)/¯
Leaving this for anyone who is despairing - the simple answers seem to work best
Related
Environment
I do use CI/CD of gitlab to bundle my application.
I do use node:14-alpine as image and do run yarn to build my app.
After build is finished, I do deploy my app via rsync to the target-server, which run's ubuntu 20.04.
On this server, I do use pm2 to start the app and keep it running.
Issue
If I look into the logs, I do see an error like this:
I've searched a bit, and found that the issue might be caused of musl-dev is missing.
I've installed it at my server, and into the docker-container, but with same result.
BUT, if I do delete the node_modules directory from server, and run yarn install right at the Server, the app run like expected
Question
So why does this issue happens here? Must I have the same distribution & version of linux in my docker-container to fit all dependencies?
Don't use an Alpine image if you're deploying on Ubuntu.
So why does this issue happens here?
The fundamental C standard library implementation is different on the two (Alpine uses musl libc; Ubuntu and more or less all other distros use GNU C Library (glibc)).
Trying to move binaries (such as those that might appear in node_modules for native modules) built against one libc implementation to a system using the other will likely be painful or not work at all (as you noticed).
Must I have the same distribution & version of linux in my docker-container to fit all dependencies?
If none of the dependencies use native code, then you should be able to just move things over without issues, but otherwise it'll be easiest (e.g. considering the versions of other libraries your dependencies may link against) to just use the same version as your target OS – or, if you don't want to think about that, just deploy your application as a Docker container.
Even if the suggestion from #AKX is a good answer, I've played a bit around to figure out how to solve this special case.
Here is my solution:
install musl-dev at the server
link it to /lib
apt-get install musl-dev
ln -s /usr/lib/x86_64-linux-musl/libc.so /lib/libc.musl-x86_64.so.1
In my case it's only this single dependency which cause the trouble. If I got more of this, I will follow AKX's suggestion and choose a debian/ubuntu-like distribution to bundle it.
For a college assignment I had to configure gitlab on my virtual machine that’s hosted on google cloud engine and is currently running Ubuntu 20.04.
I tried to install gitlab twice but the install fails (first it got stuck for at least 5 minutes on unpacking github-ce (13.10.2-ce.0) then it failed.
Reconfiguring gets me the same message but without any context, I don’t know where the error is, what is causing it and how to fix it.
I did research this error but the only thing that I found out is that it’s probably related with the config file. Only line in my config line that’s not commented out is the external url and it has a value so I have no idea what to do.
I guess something on the machine is fup. Try the same on a known-good new machine / fresh install.
Ubunut 20.04
Sounds like a fun OS.
First of all, I think this is more of a Linux issue as the problem seems to be on a linux-flavoured Docker container, but I'm happy to accept that I can do something to the team city config to overcome this.
I'm also not very experienced with Linux, Docker or node/npm, though I do have a lot of development experience and am very comfortable with command line interfaces in general.
Background
We currently have Team City set up as a build server, for building a variety of projects:
.Net Framework,
.Net Core
Angular CLI
A couple of simple websites which use node packages to generate HTML from Markdown.
The server is running as a Docker container using Docker for Windows on a Windows Server box, and this is working well.
We have one Windows 10 Build agent (a VM) which is also working fine, and builds all the .Net and .Net Core stuff fine.
The simple docs site stuff primarily uses the markdown-to-html node package, so its build steps simply get all the source .md files and compile to html with markdown-to-html, plus use some other npm packages for SASS compilation and minification of js etc. No actual node code as such, just some jQuery. In order to not tie up the other agent, and because this stuff can run happily on Linux, I want to have this running on a small docker image rather than a full VM build agent somewhere.
I previously successfully used a node.js team city agent docker image (either jacobpeddk/teamcity-agent-nodejs or omez/teamcity-agent-nodejs - can't recall) which did work for a time, though I had issues with being able to install some npm packages globally in build scripts, which meant I had to get a bash terminal into the container and run some manual npm commands. I also I think had to run apt-get install zip to get a zipping step to work. This worked fine for a while (weeks).
I added some extra JS stuff to one of these simple projects, and suddenly I was getting errors when trying to build. I (perhaps stupidly) decided that this was probably due to the container having older versions of node and/or npm etc, so I attempted to update this by getting a bash shell into the container, installing nvm and updating node.js & npm.
This ended up with a rather broken container (node errors), so I thought I'd instead start again, but actually start with the jetbrains/minimal-build-agent Docker image instead, with the aim of ending up with a nice bespoke image for our needs specifically (as I couldn't find a very up-to-date pre-existing one)
I've running a Bash shell directly on the build agent container by executing this on the host:
docker exec -it basicagent /bin/bash
then from there I've installed nvm, Python (required for node install step) and node:
curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.2/install.sh | bash
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" # This loads nvm
[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion" # This loads nvm bash_completion
apt-get update
apt-get install python 3.6
nvm install v8.11.1 (matching version on my dev machine)
npm install -g markdown-folder-to-html (npm package I previously found I had to install globally)
apt-get install zip (just used for a build step to zip up artifacts)
If I now run (via the bash shell) npm -version I get back 5.6.
If I try to get a build to run that uses npm in a command line step, then I get this error in the build log:
/opt/buildagent/temp/agentTmp/custom_script2764770419520852926: npm: not found
I wondered if it was an issue with the user/path that the team city agent process is using vs. the one I'm using in Bash, so I added the following to the build script:
echo PATH = $PATH
echo user var = $USER
echo user via 'id':
id -u -n
the output of which is:
PATH = /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
user var =
user via id:
root
So it's running the agent as root, and doesn't appear to have node in the $PATH at all.
If I run the above directly from Bash however, I can see that I am root, but my $PATH is different:
PATH = /root/.nvm/versions/node/v8.11.1/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
root
So I'm now confused: I've re-started the container and this has had no effect - it seems that when I'm logged in as root manually I have a certain path set, but when the build agent service is running as root it's different.
I have no idea why this happens, but I've basically worked around the problem by adding:
export PATH=$PATH:/root/.nvm/versions/node/v8.11.1/bin
to the top of every build step that uses npm in a script. To my mind this seems a rather daft thing to have to do - considering this used to work without this, and the only real difference is possibly a slightly different flavour of linux container. AFAIK the original build agent container was based on the jetbrains minimal-build-agent one, so unless they've changed what they base that on it should be roughly the same...
I also had to change the compressor being used in a node-minify build step from gcc (google closure compiler) to babel-minify as the former was basically hanging indefinitely, but that is a separate problem (though also something that was fine and now isn't...)
Thanks to anyone who took the time to read... though I do wonder if one-day I'll exhaust my own research options, and finally go ask the internet and actually get someone respond - for some reason whenever I get to the point where I have to ask, it always seems no-one else has the answer either and I end up having to work it out myself. It's probably character-building though I suppose.. (this isn't just SO - I've found this be the case for over 15 years on various forums about various things...)
Before I start, I want to say that I already checked these answers:
Jenkins build step fails on 'npm install <whatever>'
Jenkin's build failing on npm install
Now, I'm dealing with this issue for a while already and thus I tried a bunch of stuff.
Firstly, I installed node + npm via homebrew. A simple $ node -v and $ npm -v echoed the version v0.10.36 for node and v2.3.* for npm, which also means I HAVE THEM IN THE PATH and they work while called in the terminal.
Simply adding node -v; npm -v to the execute shell in Jenkins didn't do it. After a bit of tinkering I copied what $: which node yielded in the terminal to the above mentioned script, which now looked like this: /usr/local/bin/node and apparently that worked. The Jenkins build succeeded and 'node-v0.10.36' was proudly displayed in the console output.
When doing the same for 'npm' which happened to be /usr/local/bin/npm --version the computing gods weren't so merciful anymore. A big 'env: node: No such file or directory' error was thrown this time and the whole build failed.
The actual command that fails is
$ /bin/sh -xe /var/folders/wr/g_dl81tn5_x0t_yz3jw602cr0000gn/T/hudson8770480548136671253.sh and "surprisingly" when I run the same command in the terminal it succeeds.
I also uninstalled the homebrew node & npm versions and installed them afterwards via the package manager. Same results.
Ultimately I also did this: https://gist.github.com/DanHerbert/9520689, with no luck.
Notes:
I'm running Jenkins 1.613 and tried with 1.5**
I didn't create a "Jenkins" specific user but instead I'm using the admin. This happens to be the same user that Jenkins runs, since the who am i command inside the executable script yields the admin's user name.
sudo'ing doens't help
I'm also running the whole thing in a Virtual Environment - vagrant
I'm not running Jenkins as a deamon, as it's conflicting with xtools, but as a simple process
I also tried out jenkins-node plugin with various configs (can detail if needed)
Thanks a lot for your help, and let me know if you need any other info, screenshots, logs, etc.
I found my own solution. The problem was that the PATH although visible in shell was not exported for the Jenkins job, and so, the first workaround, as found here, was to export it in the actual script like so:
but this feels like a hack!
The right and elegant solution is to use Jenkins EnvInject Plugin and export the path in the added Properties content textarea on the configuration page, like so:
Manage Jenkins -> Configure System -> Global properties -> Environment variables
I have followed these instructions.
And as far as I can tell I have successfully installed node.js azure tools. No error - nothing to suggest it failed.
However, I cannot, and the documentation says, simple run "azure"...
Maybe there is something I am missing with node.js?
There are a few problems you may be experiencing.
First of all, I would ensure you are running Node.js v0.6.20. You can do this by opening the command prompt and running:
node -v
You should have v0.6.20 echoed back.
If this doesn't work, you may be missing a path variable to Node.js or the NPM cache. Verify the Environment variables exist by running [in the command prompt]:
path
you should see two paths:
%appdata%\npm
[x64 Machine]
%programfiles(x86)%\nodejs\
[x86 Machine]
%programfiles%\nodejs\
If this doesn't work, I would check to ensure that the azure module was loaded into the %appdata%\npm\node_modules directory.
It could be the PATH issue. In my case, the azure program is located at ~/.npm-global/bin.
run "export PATH=$PATH:~/.npm-global/bin". Or just add to bash source file