I have the following yml that I have updated from a manual run:
name: run scrapper.py
# Controls when the workflow will run
on: [workflow_dispatch]
to the below (including cron schedule):
name: run scrapper.py
on:
schedule:
- cron: "40 16 * * *"
jobs:
build:
runs-on: ubuntu-latest
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
- uses: actions/checkout#v3
- uses: actions/setup-python#v4
with:
python-version: '3.10'
- name: Install dependencies
run: pip install -r requirements.txt || pip install --editable . || pip install .
- run: pip install safety
- run: safety check
- name: execute py script
run: python scrapper.py
env:
DB: ${{ secrets.DB }}
DB_HOST: ${{ secrets.DB_HOST }}
DB_PORT: ${{ secrets.DB_PORT }}
DB_USER: ${{ secrets.DB_USER }}
DB_PASS: ${{ secrets.DB_PASS }}
It is looking (to me) as it should according to docs, but it's not triggering the python script. Any ideas?
Related
I have installed the latest version of NodeJs (v18.14.0), but it still fails to do jobs, what should I do? this is the code from my workflow and the screenshot of the error.
# This workflow will do a clean installation of node dependencies, cache/restore them, build the source code and run tests across different versions of node
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-nodejs
name: Node.js CI
on:
push:
branches: ['main']
pull_request:
branches: ['*']
jobs:
quality:
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [14.x, 16.x, 18.x]
steps:
- uses: actions/checkout#v3
- uses: actions/setup-node#v3
with:
node-version: ${{ matrix.node-version }}
- run: npm install
- run: npm test
publish:
runs-on: ubuntu-latest
if: ${{ github.ref == 'refs/heads/main' }}
needs: [quality]
steps:
- uses: actions/checkout#v3
- uses: actions/setup-node#v3
with:
node-version: ${{ matrix.node-version }}
- run: npm install
- run: npm run semantic-release
env:
NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
I've tried installing everything from scratch by removing node_modules and package.lock.json then I did an npm install, but the result is still the same. When pushed to the repository it always fails to run the job
The strategy is missing for the publish job. You have to define it under publish too.
Using ${{ matrix.node-version }} is invalid here without strategy:
publish:
runs-on: ubuntu-latest
# ...
steps:
# ...
- uses: actions/setup-node#v3
with:
node-version: ${{ matrix.node-version }} # invalid
You need to verify which version actions/setup-node installs under publish with that invalid value. Apparently, only the default preinstalled Node.js 16.19.0 is there.
NOTE: The default preinstalled NodeJS version will be set to 18 from tomorrow (Monday - February 13th, 2023).
Apart from that, the error in that image:
[semantic-release]: node version >=18 is required. Found v16.19.0.
means that this step can only run on NodeJS v18+. Installing other lower versions i.e. 14.x and 16.x would result in failures. This makes strategy completely redundant because you only need v18:
- uses: actions/setup-node#v3
with:
node-version: 18
You might want to lint your workflows with https://rhysd.github.io/actionlint/ to identify potential issues.
I need to reuse a workflow in another workflow instead of repeating things.
The build workflow already performs pip install and I just need the installed package to be reused during the deploy workflow instead of re-running pip install
build:
runs-on: ubuntu-latest
needs: create-envfile
steps:
- uses: actions/checkout#v2
- name: Set up Python version
uses: actions/setup-python#v1
with:
python-version: '3.8'
- name: Create and start virtual environment
run: |
python -m venv venv
source venv/bin/activate
- name: Install dependencies
run: |
eval `ssh-agent -s`
cat ./.github/workflows/id_rsa | ssh-add -
pip install -r requirements.txt
- name: Upload artifact for deployment jobs
uses: actions/upload-artifact#v2
with:
name: python-app
path: |
.
!venv/
deploy:
runs-on: ubuntu-latest
needs: build
environment:
name: 'Production'
url: ${{ steps.somesecret }}
steps:
- name: Download artifact from build job
uses: actions/download-artifact#v2
with:
name: python-app
path: .
- name: 'Deploy to Azure Web App'
uses: azure/webapps-deploy#v2
id: deploy-to-webapp
with:
app-name: 'project-prod'
slot-name: 'Production'
publish-profile: ${{ secrets.somesecrethere }}
What happens here, is as long as the deploy workflow finishes downloading artifacts from build, it will restart running pip install the packages from requirements.txt, which of course a redundant. How to prevent this and reuse the already installed package instead
I'm having Github workflow to run a python file. Before running the python file I want to swap user so that no sudo is required to execute the python file.
I tried with 'su' but it's says su must be run from command line. How to do that through Github Workflow. Thank you
.yaml
name: CI
on:
push:
branches: [ main ]
workflow_dispatch:
jobs:
build:
runs-on: self-hosted
strategy:
matrix:
python: [3.8.1]
steps:
- uses: actions/checkout#v2
- name: Set up Python
uses: actions/setup-python#v2
with:
python-version: ${{ matrix.python }}
- name: run multiple commands
run: |
echo $PWD
whoami
su - user
python helloworld.py
Note: I have already seen these two:
How do I run my CI steps in a specific folder in github action
How to specify node's path in Github action?
But I still cant get it to work, thats why I am asking how I am able to set the working directory for a uses command. My yaml currently looks as follows:
# This workflow will build a Java project with Maven
# For more information see: https://help.github.com/actions/language-and-framework-guides/building-and-testing-java-with-maven
name: Java CI with Maven
on:
push:
branches: [ main, Create-.yml-file ]
pull_request:
branches: [ main, Create-.yml-file ]
jobs:
javatest:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- uses: actions/setup-java#v2
with:
java-version: '16'
distribution: 'adopt'
- name: Cache Maven packages
uses: actions/cache#v2
with:
path: ~/.m2
key: ${{ runner.os }}-m2-${{ hashFiles('**/pom.xml') }}
restore-keys: ${{ runner.os }}-m2
- name: Build with Maven
run: |
mvn -f ./backend/pom.xml -B test
#mvn -f ./notification/pom.xml -B test
- name: Generate JaCoCo Badge
uses: cicirello/jacoco-badge-generator#v2
with:
generate-branches-badge: true
on-missing-report: quiet
jacoco-csv-file: >
-backend/target/site/jacoco/jacoco.csv
- name: Log coverage percentage
run: |
echo "coverage = ${{ steps.jacoco.outputs.coverage }}"
echo "branch coverage = ${{ steps.jacoco.outputs.branches }}"
- name: Commit the badge (if it changed)
run: |
if [[ `git status --porcelain` ]]; then
git config --global user.name 'myusername'
git config --global user.email 'myId#users.noreply.github.com'
git add -A
git commit -m "Autogenerated JaCoCo coverage badge"
git push
fi
- name: Upload JaCoCo coverage report
uses: actions/upload-artifact#v2
with:
name: jacoco-report
path: target/site/jacoco/
nodejstest:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node#v2
with:
node-version: ${{ matrix.node-version }}
cache: 'npm'
- run: npm ci
working-directory: ./frontend
- run: npm run build --if-present
working-directory: ./frontend
- run: npm test
working-directory: ./frontend
with the error occuring here:
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node#v2
with:
node-version: ${{ matrix.node-version }}
cache: 'npm'
And looking like this:
Run actions/setup-node#v2
/usr/local/bin/npm config get cache
/home/runner/.npm
Error: Dependencies lock file is not found in /home/runner/work/path/to/main/directory. Supported file patterns: package-lock.json,yarn.lock
My package-lock ist located in the .../path/to/main/directory/frontend so it is obvious that it can not be found but according to the other two solutions this snippet should work shouldn't it? I also already tried combining the last three run statements into as well as move the working-directory setting to different places. All with varying amounts of failure
The support for custom path (relative to repository root) was added in version 2.4:
https://github.com/actions/setup-node/releases/tag/v2.4.0
You can try specifying '**/package-lock.json' in the
cache-dependency-path (it didn't work for me with other patterns).
Also you can try setting the working directory (either for all jobs or your nodejstest specific job) to point to your frontend folder.
nodejstest:
runs-on: ubuntu-latest
defaults:
run:
working-directory: 'frontend' # Here the path to the folder where package-lock.json is located.
strategy:
matrix:
node-version: [16.x] # Are you are missing this specification?
steps:
- uses: actions/checkout#v2
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node#v2
with:
node-version: ${{ matrix.node-version }}
cache: 'npm'
cache-dependency-path: '**/package-lock.json' # THIS PATTERN did the trick for me.
setup_build_environment:
runs-on: ubuntu-latest
defaults:
run:
shell: bash
strategy:
matrix:
node-version: [14.x]
# See supported Node.js release schedule at https://nodejs.org/en/about/releases/
steps:
- uses: actions/checkout#v2
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node#v2
with:
node-version: ${{ matrix.node-version }}
cache: 'yarn'
cache-dependency-path: ./node-app/yarn.lock
- name: run npm commands
working-directory: node-app
run: |
yarn && CI=true
# yarn build --if-present
# yarn test
node test.js
Reference : https://github.com/actions/setup-node#caching-packages-dependencies
The action defaults to search for the dependency file (package-lock.json or yarn.lock) in the repository root, and uses its hash as a part of the cache key. Use cache-dependency-path for cases when multiple dependency files are used, or they are located in different subdirectories.
I have used yarn, and the node app resides in /node-app directory, so in cache-dependency-path I use the lock file from yarn. And using working-directory for running yarn commands in node-app directory.
You can break down Node setup and cache into two steps like below
nodejstest:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node#v2
with:
node-version: ${{ matrix.node-version }}
- name : Cache
uses: actions/cache#v2
with:
path: "frontend/node_modules"
key: node-modules-${{ hashFiles('frontend/package.json') }}
- run: npm ci
working-directory: ./frontend
- run: npm run build --if-present
working-directory: ./frontend
- run: npm test
working-directory: ./frontend
I have 2 workflows inside my repository - one that is a CI that runs on each push and one that is supposed to execute a script on a scheduled time.
The scheduled workflow looks like that:
name: scheduled-run
on:
schedule:
# UTC time
- cron: "45 14 1,15,30 * *"
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Set up Python 3.7
uses: actions/setup-python#v2
with:
python-version: 3.7
- name: Install dependencies
run: |
python -m pip install --upgrade pip
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
- name: Execute python script
env:
TELEGRAM_API_TOKEN: ${{ secrets.TELEGRAM_API_TOKEN }}
TELEGRAM_CHAT_ID: ${{ secrets.TELEGRAM_CHAT_ID }}
AWS_KEY_ID: ${{ secrets.AWS_KEY_ID }}
AWS_SECRET_KEY: ${{ secrets.AWS_SECRET_KEY }}
run: |
python cci.py -r
No matter how and to when I set the cron schedule - the workflow does not start! Sometimes it would start way after the supposed time but mostly it does not start. I have tried to recreate the workflow from scratch but it doesn't seem to help. What's going on here?
I tested your workflow with slightly changed cron
name: scheduled-run
on:
schedule:
# UTC time
- cron: "55 13,14,15 1,4,15,30 * *"
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Set up Python 3.7
uses: actions/setup-python#v2
with:
python-version: 3.7
And it works as expected:
Please try with sth smaller (like above) and with slightly different cron so you would not wait long to verify your changes.