ENOSPC error when deploying Meteor 1.3 app to Bluemix - node.js

Meteor 1.3 was released a few days ago.
Deployment using cf push of Meteor 1.3 apps to IBM Bluemix may fail with ENOSPC error. It appears that only simple Meteor 1.3 apps deploy successfully, as only after removing many packages or files (or creating a new app without adding many packages) it works.
ENOSPC means no space left or the number of files watched has reached the maximum allowed.
I think the latter may be the case. A solution for Node.JS apps is to increase the limit as described in Node.JS Error: ENOSPC:
echo fs.inotify.max_user_watches=524288 | sudo tee -a /etc/sysctl.conf && sudo sysctl -p
However an attempt to use sudo in Bluemix returns
sudo: no tty present and no askpass program specified
A GitHub issue was created in the cf-meteor-buildpack repository that show a number of other people experiencing the same problem:
https://github.com/cloudfoundry-community/cf-meteor-buildpack/issues/20
How can Meteor 1.3 apps be successfully deployed to Bluemix?
Update:
I have found the point at which just adding one more package to the app and deploying it causes the ENOSPC error. Without that package, the app deploys and runs successfully and the total disk usage as shown in the Bluemix Dashboard is just 220.1 MB. The default disk quota is 1 GB, so there is actually plenty of free space left. Therefore it is likely that the error is caused by exceeding a maximum number of files, not the disk quota being reached.
Just to be sure, I increased the disk quota to 2 GB via Cloud Foundry's manifest.yml and the ENOSPC error still occurred when I added the extra package. Without the package, the disk usage is "220.1 MB / 2 GB" in the dashboard. That package (kadira:flow-router) is very unlikely to need 1.8 GB of space.
Meteor 1.3 creates many more files than previous versions because there is a new node-modules directory into which many individual Node.js modules are installed.
To check the value of max_user_watches, I added this line to the buildpack script:
cat /proc/sys/fs/inotify/max_user_watches
The result shown is 8192.
I then added this line to attempt to change the value:
echo 32768 > /proc/sys/fs/inotify/max_user_watches
The result is /proc/sys/fs/inotify/max_user_watches: Permission denied.
Is it possible to increase the maximum number of allowed file watches via Cloud Foundry? The operating system is Linux.

Related

System limit for number of file watchers reached on Heroku

I deployed a node application with following libreoffice buildpack: https://elements.heroku.com/buildpacks/bluetealondon/heroku-buildpack-libreoffice-for-heroku-18
But while my application tries to start on Heroku, it spits out following error:
Internal watch failed: ENOSPC: System limit for number of file watchers reached, watch '/app/vendor/libreoffice/usr/share/icons/hicolor/32x32/mimetypes/libreoffice6.4-presentation-template.svg'
Obviously I found that I need to increase the limit, but on Heroku I don't have sudo access.
My question is now, how can I fix this? Because I need the Libreoffice buildpack for my application.

Docker Memory usage and Server stuck

We are getting an issue on Ubuntu EC2 instance, we are running 8 docker container on Amazon EC2 instance which is having 30 GB memory and 16 cores. What we observed our EC2 being stuck after 3-4 days of reboot and in the logs we are getting Memory issues. When we monitored to docker stats command it shows every Docker is running on less than 2GB (Total used 16G) on the EC2 when we ran the free –g command it shows the used memory is around 16-17GB and other memory is in buff / cache and Free is having 0.
Please let us know There is any issue or we have to do any configuration. I tried to drop the cache but it got filled in 10 minutes.
Versions –
Ubuntu – 16.04 ( xenial)
Docker - 17.09.0-ce
Please let me know if required more details for troubleshooting.
Thanks,

openshift gear using 100% inodes

I'm trying to git push my app up to Openshift. I get the following error:
Stopping Node.js application...
Warning: Gear 575efb722d5271dec00000f4 is using 100.0% of inodes allowed
Failed to execute: 'control stop' for /var/lib/openshift/575efb722d5271dec00000f4/nodej
I don't know what this means. What are inodes? I found no help on Openshift help. I checked the space quota for my gear and I'm at 60%. What does this mean?
If this is any clue, I recently downloaded a few babel files so I can start to use async/await in my node app.
Anybody?

Docker nodejs build works locally but hangs on server

I have built a node app inside docker and it builds and runs perfectly on my local machine (mint 18). But when I upload the same to Digital ocean's Docker droplet (ubuntu 16.04) it hangs mid way while building and eventually throws an error. This occurs at exactly the same place each time.
Here is the last line & the error message I can see when building ..
npm info lifecycle app#0.0.1~preinstall: app#0.0.1
Killed
The command '/bin/sh -c npm install' returned a non-zero code: 137
PS:I am new to docker and only been using it a few days so this might be something very obvious.
If you look at issue 1554, it could be a resource issue.
Either a low memory or low disk would cause such an error message.
This Digital Ocean tutorial mentions the basic Droplet has only 512MB disk space. Maybe the combined images of your Dockerfile project are too important.
Details
I tried to deploy NodeJS app via docker-compose to Digital Ocean droplet. My app hanged every time on building step. But when I executed docker-compose up --build locally, I had no problems.
P.S. I have 1 Gb RAM memory on my DO droplet.
Solution
So, I just added .dockerignore (source) to the NodeJS project
# Logs
logs
*.log
# Runtime data
pids
*.pid
*.seed
# Directory for instrumented libs generated by jscoverage/JSCover
lib-cov
# Coverage directory used by tools like istanbul
coverage
# Grunt intermediate storage (http://gruntjs.com/creating-plugins#storing-task-files)
.grunt
# node-waf configuration
.lock-wscript
# Compiled binary addons (http://nodejs.org/api/addons.html)
build/Release
# Dependency directory
# https://www.npmjs.org/doc/misc/npm-faq.html#should-i-check-my-node_modules-folder-into-git
node_modules
server/*.spec.js
kubernetes
You probably is lacking some swap space!
I use docker's node to build react applications and the server requirements after build are pretty low, a 512MB or 1G are enough for a testing environments and even for some small production production environments.
Although, node requires much more memory during building time and digital ocean droplets comes with no swap space, but it is easy to work around.

Docker + Cassandra ulimit error

I am trying to start a cassandra (not dsc) server on Docker (ubuntu 14.04). When I run service cassandra start (as root), I get
/etc/init.d/cassandra: 82: ulimit: error setting limit (Operation not permitted)
line 82 of that file is
ulimit -l unlimited
I'm not really sure what I need to change it to.
I would expect you would get that warning but that Cassandra would continue to start up and run correctly. As pointed out in the other answer, Docker restricts certain operations for safety reasons. In this case, the Casssandra init script is trying to allow unlimited locked memory. Assuming you are running with swap disabled (as it is a Cassandra best practice) then you can safely ignore this error.
I run Cassandra in Docker for my development environment and also get this warning, but Cassandra starts and runs just fine. If it is not starting up, check the cassandra log files for another problem.
A short intro into ulimit: RESOURCE LIMITS ON UNIX SYSTEMS (ULIMIT).
The command this init script is trying to issue is supposed to set the max locked memory limit to, well, unlimited. Should succeed for root. Does whoami print root?
UPD: further research led me to this Google Groups discussion. Hopefully it will clarify things a bit.
/etc/init.d/cassandra start/restart/status will not work because init system is not running inside the container so the available option is to restart the container
docker restart "container id or container name"

Resources