How to increase the file limit of GitHub Actions? - node.js

Question

I have the follwing error:
Error: ENOSPC: System limit for number of file watchers reached, watch '/home/ runner/work...
I tried all ways to increase the limit (like ulimit -S -n unlimited, sysctl, etc) but seems to not work, neither with sudo
screnshot
My website has a lot of markdown files (~ 80k) used by gatsby to build the final .htmls.
On my machine I need to increase the file limit, of couse, then works. But in the github actions I can't figure out a way to do this.
My github action workflow.yml
name: Build
on: [push, repository_dispatch]
jobs:
update:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v1
- name: Increase file limit
run: sudo sysctl -w fs.file-max=65536
- name: Debug
run: ulimit -a
- name: Set Node.js
uses: actions/setup-node#master
with:
node-version: 12.x
- name: Install dependencies
run: npm install
- name: Build
run: npm run build
I think this could be related to this issue: https://github.com/gatsbyjs/gatsby/issues/17321

Answer 1

It sounds like these GitHub/Expo issues might be the problem:
https://github.com/expo/expo-github-action/issues/20
ENOSPC: System limit for number of file watchers reached
https://github.com/expo/expo-cli/issues/277
Handle ENOSPC error (fs.inotify.max_user_watches reached)
Thanks for testing!
I'm afraid this seems to be a GitHub Action
limitation. That docker image is forcing the
fs.inotify.max_user_watches limit to 524288, but apparently GHA is
overwriting this back to 8192. You can see this happen in a fork of
your repo (when we are done, I'll remove the fork ofc, let me know if
you want to have it removed earlier).
Continuing...
Yes, it's related to a limitation of the environment you are running
Expo CLI in. The metro bundler requires a high amount of listeners
apparently. This fails if the host environment is limiting this. So
technically its an environment issue, but I'm not sure if the CLI can
change anything about this.
I find the limit in GitHub Action personally a little low. Like I
tried to outline in an earlier comment on that CLI issue, the
limitation in other CI vendors is actually set to the default max
listeners. Why they did not do this in GH Actions is unclear, that's
what I try to find out. Might be a configurational issue on their
hands, or an intentional limitation.
... And ...
So, there exists a fix, that seemed to work for me when I tried. What
I did was to follow this guys tip: “Increasing the number of
watchers” — #JNYBGR https://link.medium.com/9Zbt3B4pM0
I then did this in my main action.yml With all the specifics
underneath the dev release
steps:
- uses: actions/checkout#v1
- name: Setup kernel for react native, increase watchers
run: echo fs.inotify.max_user_watches=524288 | sudo tee -a /etc/sysctl.conf && sudo sysctl -p
- name: Run dev release fastlane inside docker action
Please let us know if any of this matches your environment/scenario, and if you find a viable workaround.
UPDATE:
The OP tried fs.inotify.max_user_watches=524288 in his .yaml, and now Gatsby is failing with Error: EMFILE: too many open files open '/home/runner/work/virtualizedfy.gatsby, and NodeJS subsequently crashes with an assertion error:
node[3007]: ../src/spawn_sync.cc:460:v8:Maybe<bool> node:SycProcessRunner::TryInitializeAndRunLoop(v8:Local<v8::Value>): Assertion `{uv_loop_init(vu_loop_ == (0)' failed.
ADDITIONAL SUGGESTION:
https://github.com/gatsbyjs/gatsby/issues/12011
Google seems to suggest https://github.com/isaacs/node-graceful-fs as
a drop in replacement for fs, I might also experiment with that to see
if it makes a difference.
EDIT: I can confirm that monkeypatching fs with graceful-fs at the top
of gatsby-node.js as in the snippet below fixes the issue for me.
const realFs = require('fs')
const gracefulFs = require('graceful-fs')
gracefulFs.gracefulify(realFs)
EDIT2: Actually after upgrading from Node 10 to Node 11 everything
seems to be fine without having to patch fs... So all is well!