Im trying to find out if there is a way to exclude certain files from being sent over github actions, for example, i have a server and a client in the same repository. right now, both the server (node.js) and the client (its a react.js application) are being hosted together on azure app services. once the / is hit, it serves up the index.html file from the build folder.
however I am finding that hosting these two things together is taking its toll on the overall application, for example, it sometimes takes up to 10 seconds for the server to respond and return the index file to the client. I remember in my training some of my more senior devs didnt like to host the server and client together, and im starting to see why..
so I likely will need to split these up to improve performance, but before i go through a daunting task of splitting the repositories up. is there a way to specify in github actions in a workflow to ignore certain files/folders etc..
the only modification i've made to this is that i added an action to zip the application for faster upload to azure to improve workload performance.
here is my workflow:
# Docs for the Azure Web Apps Deploy action: https://github.com/Azure/webapps-deploy
# More GitHub Actions for Azure: https://github.com/Azure/actions
name: Build and deploy Node.js app to Azure Web App
on:
push:
branches:
- main
workflow_dispatch:
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Set up Node.js version
uses: actions/setup-node#v1
with:
node-version: '14.x'
- name: npm install, build, and test
run: |
npm install
npm run build --if-present
npm run test --if-present
- name: Zip artifact for deployment
run: zip release.zip ./* -r
- name: Upload artifact for deployment job
uses: actions/upload-artifact#v2
with:
name: node-app
path: release.zip
deploy:
runs-on: ubuntu-latest
needs: build
environment:
name: 'Production'
url: ${{ steps.deploy-to-webapp.outputs.webapp-url }}
steps:
- name: Download artifact from build job
uses: actions/download-artifact#v2
with:
name: node-app
- name: unzip artifact for deployment
run: unzip release.zip
- name: 'Deploy to Azure Web App'
id: deploy-to-webapp
uses: azure/webapps-deploy#v2
with:
app-name: 'Omitted'
slot-name: 'Production'
publish-profile: ${{SECRET}}
package: .
You could create a shell script that excludes the files you don't want.
In .github, create a new folder scripts. Inside the scripts folder, create a new file named exclude.sh.
In the exclude.sh, add the following:
zip -r [file_name.zip] [files/folder to zip] -x [file path/name to exclude]
In your workflow:
- name: unzip artifact for deployment
run: unzip file_name.zip
Related
I'm using a GitHub Action to build and deploy a Vue Azure Static Web App. When using the default template, my staticwebapp.config.json file which is at the root of the Vue app gets applied correctly and I see Copying 'staticwebapp.config.json' to build output logged.
When using a customized GitHub workflow (shown below) to separate the build and deploy steps which has skip_app_build set to true, the artifact that gets uploaded/downloaded does not contain the staticwebapp.config.json file.
How can I modify the GitHub action to make sure the staticwebapp.config.json file gets copied to the output directory so that it gets deployed?
jobs:
build:
if: github.event_name == 'push'
runs-on: ubuntu-latest
steps:
- name: Checkout repo
uses: actions/checkout#v2
- name: Setup Node.js
uses: actions/setup-node#v3
- name: npm install and run build
run: npm install && npm run build
- name: Upload artifact
uses: actions/upload-artifact#v3.1.0
with:
name: app
path: dist/
deploy:
runs-on: ubuntu-latest
needs: build
steps:
- name: Download artifact
uses: actions/download-artifact#v3.0.0
with:
name: app
- name: Deploy to Azure
id: deploy
uses: Azure/static-web-apps-deploy#v1
with:
azure_static_web_apps_api_token: ${{ secrets.AZURE_STATIC_WEB_APPS_API_TOKEN_BLUE_STONE_0BAB0F910 }}
repo_token: ${{ secrets.GITHUB_TOKEN }} # Used for GitHub integrations (i.e. PR comments)
action: "upload"
###### Repository/Build Configurations ######
app_location: "" # App source code path relative to repository root
api_location: "" # Api source code path relative to repository root - optional
skip_app_build: true
###### End of Repository/Build Configurations ######
I was able to solve this by moving the staticwebapp.config.json file to the public directory of the Vue app. This made it so that file was in the published artifact.
After doing that, I was able to see Using staticwebapp.config.json file for configuration information in the logs during the static-web-apps-deploy step.
So I have this this small Angular project of mine and every time I try to deploy it to Azure, it uploads ~43k files as an artifact. I'm not any good at deployment to Azure, so this may as well be a really stupid question, but still.
So, here is my GitHub Actions workflow file
name: Build and deploy Node.js app to Azure Web App - minesweeper
on:
release:
branches:
- main
workflow_dispatch:
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Set up Node.js version
uses: actions/setup-node#v1
with:
node-version: '16.x'
- name: npm install, build, and test
run: |
npm install
npm run build --prod
working-directory: .
- name: Upload artifact for deployment job
uses: actions/upload-artifact#v2
with:
name: node-app
path: .
deploy:
runs-on: ubuntu-latest
needs: build
environment:
name: 'Production'
url: ${{ steps.deploy-to-webapp.outputs.webapp-url }}
steps:
- name: Download artifact from build job
uses: actions/download-artifact#v2
with:
name: node-app
- name: 'Deploy to Azure Web App'
id: deploy-to-webapp
uses: azure/webapps-deploy#v2
with:
app-name: 'minesweeper'
publish-profile: $
package: ./dist/minesweeper_
So, here I have a path, that matches my project's name: minesweeper_ and app name is from azure
What am I doing wrong here
https://github.com/yan14171/Minesweeper - here is the repo itselff
There are over 10,000 files in this artifact, consider creating an archive before upload to improve the upload performance.
As per documentation:
During upload, each file is uploaded concurrently in 4MB chunks using a separate HTTPS connection per file. Chunked uploads are used so that in the event of a failure, the upload can be retried. If there is an error, a retry will be attempted after a certain period of time.
Alternatively, you can try zip and unzip steps as mentioned by Steve.
You can refer to React Deployment on App Service Linux, and Deploying Node.js to Azure App Service with GitHub Actions
I have a GitHub action doing .net solution build, test and deploy to Azure:
name: Build and deploy ASP.Net Core app to Azure Web App - project-test-api
on:
push:
branches:
- main
workflow_dispatch:
env:
# Stop wasting time caching packages
DOTNET_SKIP_FIRST_TIME_EXPERIENCE: true
# Disable sending usage data to Microsoft
DOTNET_CLI_TELEMETRY_OPTOUT: true
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: AutoModality/action-clean#v1
- uses: actions/checkout#v2
- name: Set up .NET Core
uses: actions/setup-dotnet#v1
with:
dotnet-version: '6.0.x'
include-prerelease: true
- name: Manually restore
working-directory: SolutionDir
run: dotnet restore --force
- name: Build with dotnet
working-directory: SolutionDir
run: dotnet build --configuration Release --no-restore
- name: Test
working-directory: SolutionDir
run: dotnet test --no-restore --no-build --configuration Release
- name: dotnet publish
working-directory: SolutionDir
run: dotnet publish -c Release -o ${{env.DOTNET_ROOT}}/myapp
- name: Upload artifact for deployment job
uses: actions/upload-artifact#v2
with:
name: .net-app
path: ${{env.DOTNET_ROOT}}/myapp
deploy:
runs-on: ubuntu-latest
needs: build
environment:
name: 'Production'
url: ${{ steps.deploy-to-webapp.outputs.webapp-url }}
steps:
- name: Download artifact from build job
uses: actions/download-artifact#v2
with:
name: .net-app
- name: Deploy to Azure Web App
id: deploy-to-webapp
uses: azure/webapps-deploy#v2
with:
app-name: 'quickplanner-test-api'
slot-name: 'Production'
publish-profile: ${{ secrets.AZUREAPPSERVICE_PUBLISHPROFILE_SECRET }}
package: .
Recently I added a test project to the solution. From what I can see one of the packages used in the test project uses Newtonsoft.Json v9.0 when the rest of the solution uses v13.0. The solution can be built locally, tested and everything is ok. The GitHub action also finishes successfully building the solution, runs tests and deploys it to Azure. The problem occurs on Azure - somewhere along the way GitHub action uses an older version of Newtonsoft.Json. All projects expect newer ones, so the whole website breaks because of this. I'm not sure how to fix this - I've tried adding manually correct version of Newtonsoft.Json to the test project, to all projects, clearing caches in GitHub actions but without luck. What works is just removing test project from the solution, but obviously I want tests to be working. Does anyone has idea why this breaks and how to fix it?
I managed to fix this problem by adding this code to my Tests project csproj file:
<PropertyGroup>
<AutoGenerateBindingRedirects>true</AutoGenerateBindingRedirects>
<GenerateBindingRedirectsOutputType>true</GenerateBindingRedirectsOutputType>
</PropertyGroup>
I'm not entirely sure what was the root cause behind this problem
I have I'm trying to deploy my app on Azure Web apps. I have a Github Actions I was given by default when connecting my GitHub repository in Azure. The problem is that deploying a single thing takes about 45 minutes to be deployed!! It's so insane the amount of time is giving to be deployed. I see a message like this:
Any idea of why is this happening or a better way to optimize this?
(By the way, I'm actually deploying a Next.js app)
This is my Github Actions File:
# Docs for the Azure Web Apps Deploy action: https://github.com/Azure/webapps-deploy
# More GitHub Actions for Azure: https://github.com/Azure/actions
name: Build and deploy Node.js app to Azure Web App - app-admin
on:
push:
branches:
- develop
workflow_dispatch:
jobs:
build:
runs-on: ubuntu-latest
env:
AUTH0_CLIENT_ID: ${{ secrets.AUTH0_CLIENT_ID }}
AUTH0_DOMAIN: ${{secrets.AUTH0_DOMAIN}}
AUTH0_MANAGEMENT_CLIENT_ID: ${{secrets.AUTH0_MANAGEMENT_CLIENT_ID}}
AUTH0_MANAGEMENT_CLIENT_SECRET: ${{secrets.AUTH0_MANAGEMENT_CLIENT_SECRET}}
NEXT_PUBLIC_HASURA_GRAPHQL_ENDPOINT: ${{secrets.NEXT_PUBLIC_HASURA_GRAPHQL_ENDPOINT}}
NEXT_PUBLIC_HASURA_GRAPHQL_API_KEY: ${{secrets.NEXT_PUBLIC_HASURA_GRAPHQL_API_KEY}}
NEXT_PUBLIC_AUTH0_CLIENT_ID: ${{secrets.NEXT_PUBLIC_AUTH0_CLIENT_ID}}
NEXT_PUBLIC_AUTH0_DOMAIN: ${{secrets.NEXT_PUBLIC_AUTH0_DOMAIN}}
steps:
- uses: actions/checkout#v2
- name: Set up Node.js version
uses: actions/setup-node#v1
with:
node-version: '14.x'
- name: npm install, build, and test
run: |
npm install
npm run build --if-present
npm run test --if-present
- name: Upload artifact for deployment job
uses: actions/upload-artifact#v2
with:
name: node-app
path: .
deploy:
runs-on: ubuntu-latest
needs: build
environment:
name: 'Production'
url: ${{ steps.deploy-to-webapp.outputs.webapp-url }}
steps:
- name: Download artifact from build job
uses: actions/download-artifact#v2
with:
name: node-app
- name: 'Deploy to Azure Web App'
id: deploy-to-webapp
uses: azure/webapps-deploy#v2
with:
app-name: 'app-admin'
slot-name: 'Production'
publish-profile: ${{ secrets.AZUREAPPSERVICE_PUBLISHPROFILE_667E58BB348E475EA5F1141747DD1CA9 }}
package: .
The upload of 38000 files will take forever. As the logs indicate you can speed up this tremendously by zipping up the contents, uploading that and unzipping the contents on the other end.
- run: |
zip -r node-app.zip .
- name: Upload artifact for deployment job
uses: actions/upload-artifact#v2
with:
name: node-app
path: node-app.zip
Then unzip it after downloading the artefact in the 2nd job:
- name: Download artifact from build job
uses: actions/download-artifact#v2
with:
name: node-app
- run: |
unzip node-app.zip
rm node-app.zip
Zip or Tar or any other tool will work and, in some cases, simply storing without compression may be faster depending on the size of the archive (time spent compressing vs time spent uploading/downloading).
Thank you Lex Li supporting your answer adding the process on how to deploy docker image
The better way is to deploy docker image which is very fast in uploading a single file.
Below is the github link as well as the example code of few docker image deployments.
on: [push]
name: Linux_Container_Node_Workflow
jobs:
build-and-deploy:
runs-on: ubuntu-latest
steps:
# checkout the repo
- name: 'Checkout Github Action'
uses: actions/checkout#master
- uses: azure/docker-login#v1
with:
login-server: contoso.azurecr.io
username: ${{ secrets.REGISTRY_USERNAME }}
password: ${{ secrets.REGISTRY_PASSWORD }}
- run: |
docker build . -t contoso.azurecr.io/nodejssampleapp:${{ github.sha }}
docker push contoso.azurecr.io/nodejssampleapp:${{ github.sha }}
- uses: azure/webapps-deploy#v2
with:
app-name: 'node-rnc'
publish-profile: ${{ secrets.azureWebAppPublishProfile }}
images: 'contoso.azurecr.io/nodejssampleapp:${{ github.sha }}'
It's because npm install step, it will take time to upload all files in node_modules folder to the target.
I removed that step and it run faster then.
But you need to setup node_module folders on the server first by npm install, and the action will only copy js file.
build:
runs-on: windows-latest
steps:
- uses: actions/checkout#v2
- name: Set up Node.js version
uses: actions/setup-node#v1
with:
node-version: '14.x'
# - name: npm install, build, and test
# run: |
# npm install
# npm run build --if-present
# npm run test --if-present
- name: Upload artifact for deployment job
uses: actions/upload-artifact#v2
with:
name: node-app
path: .
on:
push:
branches:
- master
jobs:
build:
name: Build
runs-on: ubuntu-latest
steps:
- name: Checkout Repo
uses: actions/checkout#master
- name: Install Dependencies
run: npm install
- name: Build
run: npm run build
- name: Archive Production Artifact
uses: actions/upload-artifact#master
with:
name: build
path: build
deploy:
name: Deploy
needs: build
runs-on: ubuntu-latest
steps:
- name: Checkout Repo
uses: actions/checkout#master
- name: Download Artifact
uses: actions/download-artifact#master
with:
name: build
- name: Deploy to Firebase
uses: w9jds/firebase-action#master
with:
args: deploy --only hosting
env:
FIREBASE_TOKEN: ${{ secrets.FIREBASE_TOKEN }}
now this is the gtihub actions workflow it is executing build job without errors but in deployment there comes an error
this is the error image
the error its shows is Error: Specified public directory 'build' does not exist, can't deploy hosting to site landing-page-design-1 i have followed the blog from where the workflow is copied i did everything same except some of my project details which is obvious please help me out why is this error occuring and how can i fix it
You're probably unpacking artifact to root directory instead of build/.
I'm guessing article was written for download-artifact#v1 while you are using download-artifact#v2 (as that's where master points currently). Difference between both is discussed here.
I'd verify first what is going on after artifact is downloaded
- name: Display directory structure
run: ls -R
shell: bash
If files are indeed in root directory, adding path should fix that.
- name: Download Artifact
uses: actions/download-artifact#v2
with:
name: build
path: build
PS: Using actions/<name>#master is not recommended, as it can always lead to issues if same action behaves differently between versions... for example actions/download-artifact ;)
You can also try to use firebase-publish-react to simplify your workflow file
This particular action plugin takes care of building the application internally and also can reuse the build directory from previous steps.
- name: Deploy to Firebase
uses: mohammed-atif/firebase-publish-react#v1.0
with:
firebase-token: ${{ secrets.FIREBASE_TOKEN }}