Github actions caching for Node.js application is not working - node.js

I'm working on configuration of CI/CD in github and faced a problem with caching of dependencies.
My github actions lint config for my Node.js app attached.
As you can see I have additional step called build which is used to cache dependencies using actions/cache#v2. Then on eslint and Prettier steps I extract cached data using restore-keys.
The script fails on eslint step with error:
sh: 1: eslint: not found
I have eslint is my devDependencies section in package.json.
name: test
on: [push]
jobs:
build:
name: Build
runs-on: ubuntu-latest
container:
image: node:14.17.0-stretch-slim
steps:
- uses: actions/checkout#v2
- name: Cache dependencies
uses: actions/cache#v2
with:
path: ~/.npm
key: ${{ runner.OS }}-cache-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
${{ runner.OS }}-cache-
- name: Install dependencies
run: npm ci --ignore-scripts
eslint:
needs: build
name: ESLint
runs-on: ubuntu-latest
container:
image: node:14.17.0-stretch-slim
steps:
- uses: actions/checkout#v2
- name: Cache dependencies
uses: actions/cache#v2
with:
path: ~/.npm
key: ${{ runner.OS }}-cache-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
${{ runner.OS }}-cache-
- name: Lint source code with ESLint
run: npm run lint
prettier:
needs: build
name: Prettier
runs-on: ubuntu-latest
container:
image: node:14.17.0-stretch-slim
steps:
- uses: actions/checkout#v2
- name: Cache dependencies
uses: actions/cache#v2
with:
path: ~/.npm
key: ${{ runner.OS }}-cache-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
${{ runner.OS }}-cache-
- name: Lint source code with Prettier
run: npm run check:format

The problem was that I didn't run dependencies installation on eslint and prettier steps. It still needs to be done to create node_modules.

Related

Why do I always fail when I run jobs on github actions? even though I have installed the latest version of Node on my machine

I have installed the latest version of NodeJs (v18.14.0), but it still fails to do jobs, what should I do? this is the code from my workflow and the screenshot of the error.
# This workflow will do a clean installation of node dependencies, cache/restore them, build the source code and run tests across different versions of node
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-nodejs
name: Node.js CI
on:
push:
branches: ['main']
pull_request:
branches: ['*']
jobs:
quality:
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [14.x, 16.x, 18.x]
steps:
- uses: actions/checkout#v3
- uses: actions/setup-node#v3
with:
node-version: ${{ matrix.node-version }}
- run: npm install
- run: npm test
publish:
runs-on: ubuntu-latest
if: ${{ github.ref == 'refs/heads/main' }}
needs: [quality]
steps:
- uses: actions/checkout#v3
- uses: actions/setup-node#v3
with:
node-version: ${{ matrix.node-version }}
- run: npm install
- run: npm run semantic-release
env:
NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
I've tried installing everything from scratch by removing node_modules and package.lock.json then I did an npm install, but the result is still the same. When pushed to the repository it always fails to run the job
The strategy is missing for the publish job. You have to define it under publish too.
Using ${{ matrix.node-version }} is invalid here without strategy:
publish:
runs-on: ubuntu-latest
# ...
steps:
# ...
- uses: actions/setup-node#v3
with:
node-version: ${{ matrix.node-version }} # invalid
You need to verify which version actions/setup-node installs under publish with that invalid value. Apparently, only the default preinstalled Node.js 16.19.0 is there.
NOTE: The default preinstalled NodeJS version will be set to 18 from tomorrow (Monday - February 13th, 2023).
Apart from that, the error in that image:
[semantic-release]: node version >=18 is required. Found v16.19.0.
means that this step can only run on NodeJS v18+. Installing other lower versions i.e. 14.x and 16.x would result in failures. This makes strategy completely redundant because you only need v18:
- uses: actions/setup-node#v3
with:
node-version: 18
You might want to lint your workflows with https://rhysd.github.io/actionlint/ to identify potential issues.

Deploying to GH pages with Custom Jekyll Plugins

I am trying to deploy a site to gh-pages. I am using this template. The side is a node built site with custom jekyll plugins.
My .yml is below.
name: Build and Deploy to Github Pages
permissions:
id-token: write
pages: write
on:
push:
branches:
- main
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Checkout 🛎️
uses: actions/checkout#v3
- name: Setting up Node
uses: actions/setup-node#v3
with:
node-version: '16'
cache: 'npm'
- run: npm ci
- name: Setting up Ruby
uses: actions/setup-ruby#v1
with: { ruby-version: '3.1.2' }
- name: Caching Gems
uses: actions/cache#v2
with:
path: vendor/bundle
key: ${{ runner.os }}-gems-${{ hashFiles('**/Gemfile.lock') }}
restore-keys: |
${{ runner.os }}-gems-
- name: Build Site 🏗️
run: npm run install-jekyll
env:
NODE_ENV: production
- name: Upload artifact
uses: actions/upload-pages-artifact#v1
with:
path: "./site/"
deploy:
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
runs-on: ubuntu-latest
needs: build
steps:
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages#v1
Everything appears to run fine - no errors and GH tells me the site was deployed to zakraicik.github.io fine. However, when I visit the site- it shows 404.
Can anyone help me figure out where my mistake is? I assume it's somewhere in the .yml file.
The full repo is available here.

Github action with azure web app application setting

I am trying to deploy a vuejs app with the azure web app and github action. Here is my yml:
name: 'test'
on:
push:
branches:
- release
workflow_dispatch:
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: azure/login#v1
with:
creds: '${{ secrets.AZURE_CREDENTIALS_DEV }}'
- uses: azure/appservice-settings#v1
with:
app-name: 'test'
app-settings-json: '${{ secrets.APP_SETTINGS_DEV }}'
general-settings-json: '{"alwaysOn": "false", "webSocketsEnabled": "true"}' #'General configuration settings as Key Value pairs'
id: settings
- run: echo "The webapp-url is ${{ steps.settings.outputs.webapp-url }}"
- run: |
az logout
- uses: actions/checkout#v2
- name: Set up Node.js version
uses: actions/setup-node#v1
with:
node-version: '14.x'
- name: npm install, build
run: |
npm install
npm run build
- name: Upload artifact for deployment job
uses: actions/upload-artifact#v2
with:
name: node-app
path: .
deploy:
runs-on: ubuntu-latest
needs: build
environment:
name: 'test'
url: ${{ steps.deploy-to-webapp.outputs.webapp-url }}
steps:
- name: Download artifact from build job
uses: actions/download-artifact#v2
with:
name: node-app
- name: 'Deploy to Azure Web App'
id: deploy-to-webapp
uses: azure/webapps-deploy#v2
with:
app-name: 'test'
slot-name: 'Production'
publish-profile: ${{ secrets.AZUREAPPSERVICE_PUBLISHPR }}
package: .
And I followed this tutorial to retrieve the application settings from web app:
https://github.com/Azure/appservice-settings
So I got the variable and secrets in pipeline, but it seems like when building the app, it doesn't build with those secrets, the environment variables turned to be undefined in the app:(
Does anyone know a solution for it?
So yes, I figured out how I can solve this pain.
So all I had to do is create .env file before the build, see the full yml below:
name: 'test'
on:
push:
branches:
- release
workflow_dispatch:
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Set up Node.js version
uses: actions/setup-node#v1
with:
node-version: '14.x'
- name: create .env file
run: |
touch .env
echo VUE_APP_CLIENT_ID =${{ secrets.VUE_APP_CLIENT_ID_DEV }} >> .env
echo VUE_APP_CLIENT_SECRET =${{ secrets.VUE_APP_CLIENT_SECRET_DEV }} >> .env
- name: npm install, build
run: |
npm install
npm run build
- name: Zip artifact for deployment
run: zip release.zip ./* -r
- name: Upload artifact for deployment job
uses: actions/upload-artifact#v2
with:
name: node-app
path: release.zip
deploy:
runs-on: ubuntu-latest
needs: build
environment:
name: 'test'
url: ${{ steps.deploy-to-webapp.outputs.webapp-url }}
steps:
- name: Download artifact from build job
uses: actions/download-artifact#v2
with:
name: node-app
- name: unzip artifact for deployment
run: unzip release.zip
- name: 'Deploy to Azure Web App'
id: deploy-to-webapp
uses: azure/webapps-deploy#v2
with:
app-name: 'test'
slot-name: 'Production'
publish-profile: ${{ secrets.AZUREAPPSERVICE_PUBLISHPR }}
package: .
So the imports piece is to create the .env file in the container and get secrets(VUE_APP_CLIENT_ID_DEV) which was stored in GitHub secrets:
- name: create .env file
run: |
touch .env
echo VUE_APP_CLIENT_ID =${{ secrets.VUE_APP_CLIENT_ID_DEV }} >> .env
echo VUE_APP_CLIENT_SECRET =${{ secrets.VUE_APP_CLIENT_SECRET_DEV }} >> .env
Below another piece worth look, it zip the artifacts and unzip when deploy, which helped improve the performance big time:
- name: Upload artifact for deployment job
uses: actions/upload-artifact#v2
with:
name: node-app
path: release.zip
- name: unzip artifact for deployment
run: unzip release.zip
It is working perfectly for me, hope this helps you!

Cache dependencies in GitHub Actions on Windows

I am having a Github action which runs on Windows OS. Then, for caching the dependencies, I use the actions/cache#v2. But, it is not working as expected. When I saw the debug logs, the size of the cache is only about 30 B.
Here is the code I am using:
name: Build
on: push
jobs:
build_on_win:
runs-on: windows-latest
if: "!contains(github.event.head_commit.message, 'skip-publish')"
steps:
- uses: actions/checkout#v2
- uses: actions/setup-node#master
with:
node-version: 15
- name: Cache NPM dependencies
uses: actions/cache#v2
with:
path: ~/.npm
key: ${{ runner.OS }}-npm-cache-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
${{ runner.OS }}-npm-cache-
- name: install dependencies
run: npm install
rest of the code..
Any help is greatly appreciated !
Caching package dependencies is a feature that is already available in the actions/setup-node#v2. You don't have to configure actions/cache#v2 and instead of that what you can do is use the setup-node#v2 action like below and you just have to add cache: 'npm' and it will automatically detect and will cache the dependencies.
steps:
- uses: actions/checkout#v2
- uses: actions/setup-node#v2
with:
node-version: '14'
cache: 'npm'
- run: npm install
- run: npm test
If you have monorepo then you can also define the path
steps:
- uses: actions/checkout#v2
- uses: actions/setup-node#v2
with:
node-version: '14'
cache: 'npm'
cache-dependency-path: subdir/package-lock.json
- run: npm install
- run: npm test
You can read more about it in Caching packages dependencies.

How to specifiy path for actions/setup-node in Github

Note: I have already seen these two:
How do I run my CI steps in a specific folder in github action
How to specify node's path in Github action?
But I still cant get it to work, thats why I am asking how I am able to set the working directory for a uses command. My yaml currently looks as follows:
# This workflow will build a Java project with Maven
# For more information see: https://help.github.com/actions/language-and-framework-guides/building-and-testing-java-with-maven
name: Java CI with Maven
on:
push:
branches: [ main, Create-.yml-file ]
pull_request:
branches: [ main, Create-.yml-file ]
jobs:
javatest:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- uses: actions/setup-java#v2
with:
java-version: '16'
distribution: 'adopt'
- name: Cache Maven packages
uses: actions/cache#v2
with:
path: ~/.m2
key: ${{ runner.os }}-m2-${{ hashFiles('**/pom.xml') }}
restore-keys: ${{ runner.os }}-m2
- name: Build with Maven
run: |
mvn -f ./backend/pom.xml -B test
#mvn -f ./notification/pom.xml -B test
- name: Generate JaCoCo Badge
uses: cicirello/jacoco-badge-generator#v2
with:
generate-branches-badge: true
on-missing-report: quiet
jacoco-csv-file: >
-backend/target/site/jacoco/jacoco.csv
- name: Log coverage percentage
run: |
echo "coverage = ${{ steps.jacoco.outputs.coverage }}"
echo "branch coverage = ${{ steps.jacoco.outputs.branches }}"
- name: Commit the badge (if it changed)
run: |
if [[ `git status --porcelain` ]]; then
git config --global user.name 'myusername'
git config --global user.email 'myId#users.noreply.github.com'
git add -A
git commit -m "Autogenerated JaCoCo coverage badge"
git push
fi
- name: Upload JaCoCo coverage report
uses: actions/upload-artifact#v2
with:
name: jacoco-report
path: target/site/jacoco/
nodejstest:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node#v2
with:
node-version: ${{ matrix.node-version }}
cache: 'npm'
- run: npm ci
working-directory: ./frontend
- run: npm run build --if-present
working-directory: ./frontend
- run: npm test
working-directory: ./frontend
with the error occuring here:
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node#v2
with:
node-version: ${{ matrix.node-version }}
cache: 'npm'
And looking like this:
Run actions/setup-node#v2
/usr/local/bin/npm config get cache
/home/runner/.npm
Error: Dependencies lock file is not found in /home/runner/work/path/to/main/directory. Supported file patterns: package-lock.json,yarn.lock
My package-lock ist located in the .../path/to/main/directory/frontend so it is obvious that it can not be found but according to the other two solutions this snippet should work shouldn't it? I also already tried combining the last three run statements into as well as move the working-directory setting to different places. All with varying amounts of failure
The support for custom path (relative to repository root) was added in version 2.4:
https://github.com/actions/setup-node/releases/tag/v2.4.0
You can try specifying '**/package-lock.json' in the
cache-dependency-path (it didn't work for me with other patterns).
Also you can try setting the working directory (either for all jobs or your nodejstest specific job) to point to your frontend folder.
nodejstest:
runs-on: ubuntu-latest
defaults:
run:
working-directory: 'frontend' # Here the path to the folder where package-lock.json is located.
strategy:
matrix:
node-version: [16.x] # Are you are missing this specification?
steps:
- uses: actions/checkout#v2
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node#v2
with:
node-version: ${{ matrix.node-version }}
cache: 'npm'
cache-dependency-path: '**/package-lock.json' # THIS PATTERN did the trick for me.
setup_build_environment:
runs-on: ubuntu-latest
defaults:
run:
shell: bash
strategy:
matrix:
node-version: [14.x]
# See supported Node.js release schedule at https://nodejs.org/en/about/releases/
steps:
- uses: actions/checkout#v2
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node#v2
with:
node-version: ${{ matrix.node-version }}
cache: 'yarn'
cache-dependency-path: ./node-app/yarn.lock
- name: run npm commands
working-directory: node-app
run: |
yarn && CI=true
# yarn build --if-present
# yarn test
node test.js
Reference : https://github.com/actions/setup-node#caching-packages-dependencies
The action defaults to search for the dependency file (package-lock.json or yarn.lock) in the repository root, and uses its hash as a part of the cache key. Use cache-dependency-path for cases when multiple dependency files are used, or they are located in different subdirectories.
I have used yarn, and the node app resides in /node-app directory, so in cache-dependency-path I use the lock file from yarn. And using working-directory for running yarn commands in node-app directory.
You can break down Node setup and cache into two steps like below
nodejstest:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node#v2
with:
node-version: ${{ matrix.node-version }}
- name : Cache
uses: actions/cache#v2
with:
path: "frontend/node_modules"
key: node-modules-${{ hashFiles('frontend/package.json') }}
- run: npm ci
working-directory: ./frontend
- run: npm run build --if-present
working-directory: ./frontend
- run: npm test
working-directory: ./frontend

Resources