Use powershell script to customize azure devOps pipeline? - azure

I have an azure function app that is running on Azure. I am now trying to configure the application based on a powershell script. This powershell script is being used to create azure resources (Ex: KeyVault, ApplicationInsights...) that will be then used by the function app. I created the powershell script and my question here is how can I add the powershell script in the yaml file which is responsible to deploy the application. The powershell script is located in the repository under the name functionapp.ps1 . The idea here is to run the powershell script once the build is finished so in the deploy stage. What I did so far is the following:
stages:
- stage: Build
displayName: Build stage
jobs:
- job: Build
displayName: Build
pool:
vmImage: $(vmImageName)
steps:
- task: DotNetCoreCLI#2
displayName: Build
inputs:
command: 'build'
projects: |
$(workingDirectory)/*.csproj
arguments: --output $(System.DefaultWorkingDirectory)/publish_output --configuration Release
- task: ArchiveFiles#2
displayName: 'Archive files'
inputs:
rootFolderOrFile: '$(System.DefaultWorkingDirectory)/publish_output'
includeRootFolder: false
archiveType: zip
archiveFile: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
replaceExistingArchive: true
- publish: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
artifact: drop
- stage: Deploy
displayName: Deploy stage
dependsOn: Build
condition: succeeded()
jobs:
- deployment: Deploy
displayName: Deploy
environment: 'development'
pool:
vmImage: $(vmImageName)
strategy:
runOnce:
deploy:
steps:
- task: CmdLine#2
inputs:
script: |
echo '$(System.DefaultWorkingDirectory)'
dir
- task: PowerShell#2
inputs:
targetType: 'filePath'
filePath: $(System.DefaultWorkingDirectory)/functionapp.ps1
- task: AzureFunctionApp#1
displayName: 'Azure functions app deploy'
inputs:
azureSubscription: '$(azureSubscription)'
appType: functionApp
appName: $(functionAppName)
package: '$(Pipeline.Workspace)/drop/$(Build.BuildId).zip'
When the stages are being run and it arrives to this task I get the following error:
##[error]Invalid file path 'D:\a\1\s\functionapp.ps1'. A path to a .ps1 file is required.
I tried using the command line task to check the default working directory and I got the following result: D:\a\1\s
I still don't know what I am missing here and why I am getting this error. Can someone please give me a concrete solution to this problem ?
I ended up having the same error. Can someone please tell me what is the issue here ?

Related

Azure FunctionApp without functions after successful CI/CD over Azure DevOps

I've created an Azure Function App (Linux) running on an App Service Plan Y1 and have my sources in Azure DevOps Git. The functions are written in C# on DOTNET 6.
Below you can see my YAML definitions for the CI and a separated CD pipeline. When I execute the pipeline everything works well (both are green). After the deployment however this is how my Azure Portal looks in the Functions blade:
Using the VS Code Azure Extension and looking into the files of the function app I get:
When I look in the artifact of the CI pipeline everything looks good (explorer view of the downloaded zip):
The bin-folder is populated there.
Some key points:
I have no slots in the function app.
The deployments are visible in the associated App Insights.
My reference point is Microsoft Learn
So is it normal to get the 404 when searching in VS code and did somebody experience something similar or even know the solution?
Sidenote:
I used to deploy my function using VS Code with extensions install. Today I now get a weired error message after the deployment:
CI YAML
pool:
vmImage: 'ubuntu-latest'
trigger: none
variables:
- name: 'Solution'
value: '**/MyProject.sln'
- name: 'ProjectFilter'
value: '**/*.csproj'
- name: 'BuildConfiguration'
value: 'Release'
steps:
- task: UseDotNet#2
displayName: Use DotNet 6
inputs:
version: '6.0.x'
- task: DotNetCoreCLI#2
displayName: Restore
inputs:
command: restore
projects: '$(ProjectFilter)'
- task: DotNetCoreCLI#2
displayName: Build
inputs:
projects: '$(ProjectFilter)'
arguments: '--no-restore --configuration $(BuildConfiguration)'
- task: DotNetCoreCLI#2
displayName: Publish Image Converter
inputs:
command: publish
projects: src/Functions/**/MyProject.csproj
publishWebProjects: false
arguments: '--no-restore --no-build --configuration $(BuildConfiguration) --output $(Build.DefaultWorkingDirectory)/function_publish_output'
zipAfterPublish: false
- task: ArchiveFiles#2
displayName: Archive Image Converter
inputs:
rootFolderOrFile: '$(Build.DefaultWorkingDirectory)/function_publish_output'
includeRootFolder: false
archiveFile: '$(Build.ArtifactStagingDirectory)/MyProject.zip'
- task: PublishBuildArtifacts#1
displayName: 'Publish Artifact'
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)'
condition: succeededOrFailed()
CD YAML
variables:
- name: 'vmImageName'
value: 'ubuntu-latest'
- name: 'serviceConnectionName'
value: 'MYCONN'
- name: 'project'
value: 'MYPROJECT'
resources:
pipelines:
- pipeline: ci
source: 'NAMEOFCI'
trigger: true
pool:
vmImage: $(vmImageName)
trigger: none
stages:
- stage: Production
displayName: Production
jobs:
- deployment: Deploy
displayName: 'Deploy'
environment: 'Production'
pool:
vmImage: $(vmImageName)
strategy:
runOnce:
deploy:
steps:
- download: ci
displayName: 'Download Artifact'
artifact: drop
- task: AzureFunctionApp#1
displayName: 'Deploy Image Converter Function'
inputs:
azureSubscription: '$(serviceConnectionName)'
appType: functionAppLinux
appName: 'fapp-**********-prod'
package: '$(Pipeline.Workspace)/ci/drop/MyProject.zip'
runtimeStack: 'DOTNET|6.0'
Can you check what exactly you see on the file system? You can use console or Advanced tools:

Azure Pipeline refuses to deploy from zip instead calling from blob which doesn't exist

Very much still new to YAML, but i've showed this to a colleague who's equally confused as me. I've built a CI/CD pipeline, and the CI end seems to work just fine. The pipeline claims its deploying a function app successfully, but when I go to check, there's nothing in the function app, and the output code from the pipeline seems to be calling from a blob storage folder that doesn't exist. We can't work out how to change the behaviour of it.
Is this something anyone has seen before?
This is the YAML:
# Python Function App to Linux on Azure
# Build a Python function app and deploy it to Azure as a Linux function app.
# Add steps that analyze code, save build artifacts, deploy, and more:
# https://learn.microsoft.com/azure/devops/pipelines/languages/python
trigger:
- master
variables:
# Azure Resource Manager connection created during pipeline creation
azureSubscription: 'ab1c86d0-0d0c-4029-913b-e5483df967b2'
# Function app name
functionAppName: 'categoriserfunctionapp'
# Agent VM image name
vmImageName: 'ubuntu-latest'
# Working Directory
workingDirectory: ''
stages:
- stage: Build
displayName: Build stage
jobs:
- job: Build
displayName: Build
pool:
vmImage: $(vmImageName)
steps:
- bash: |
if [ -f extensions.csproj ]
then
dotnet build extensions.csproj --runtime ubuntu.16.04-x64 --output ./bin
fi
workingDirectory: $(workingDirectory)
displayName: 'Build extensions'
- task: UsePythonVersion#0
displayName: 'Use Python 3.6'
inputs:
versionSpec: 3.6 # Functions V2 supports Python 3.6 as of today
architecture: 'x64'
- bash: |
pip install --target="./.python_packages/lib/site-packages" -r ./requirements.txt
workingDirectory: $(workingDirectory)
displayName: 'Install application dependencies'
- task: ArchiveFiles#2
displayName: 'Archive files'
inputs:
rootFolderOrFile: "$(System.DefaultWorkingDirectory)"
includeRootFolder: false
archiveType: zip
archiveFile: "$(System.DefaultWorkingDirectory)/$(Build.BuildId).zip"
replaceExistingArchive: true
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: '$(System.DefaultWorkingDirectory)/$(Build.BuildId).zip'
artifactName: 'drop'
- stage: Deploy
displayName: Deploy stage
dependsOn: Build
condition: succeeded()
jobs:
- deployment: Deploy
displayName: Deploy
environment: 'test'
pool:
vmImage: 'windows-latest'
strategy:
runOnce:
deploy:
steps:
- task: DownloadPipelineArtifact#2
displayName: 'Download Pipeline Artifact'
inputs:
buildType: 'current'
artifactName: 'drop'
targetPath: '$(Pipeline.Workspace)/drop/'
- task: AzureFunctionApp#1
inputs:
azureSubscription: 'Visual Studio Enterprise Subscription – MPN (f1f3e234-557b-4acd-b353-2ff89c547e49)'
appType: 'functionAppLinux'
appName: 'categoriserfunctionapp'
package: '$(Pipeline.Workspace)/drop/$(Build.BuildId).zip'
runtimeStack: 'PYTHON|3.9'
And this is the output from the CD end:
2022-09-27T15:59:26.6330661Z ##[section]Starting: AzureFunctionApp
2022-09-27T15:59:26.6442433Z ==============================================================================
2022-09-27T15:59:26.6442718Z Task : Azure Functions
2022-09-27T15:59:26.6443009Z Description : Update a function app with .NET, Python, JavaScript, PowerShell, Java based web applications
2022-09-27T15:59:26.6443282Z Version : 1.208.2
2022-09-27T15:59:26.6443454Z Author : Microsoft Corporation
2022-09-27T15:59:26.6443686Z Help : https://aka.ms/azurefunctiontroubleshooting
2022-09-27T15:59:26.6443957Z ==============================================================================
2022-09-27T15:59:27.4092341Z Got service connection details for Azure App Service:'categoriserfunctionapp'
2022-09-27T15:59:30.9793711Z Trying to update App Service Application settings. Data: {"WEBSITE_RUN_FROM_PACKAGE":"https://categoriserfunc9c44.blob.core.windows.net/azure-pipelines-deploy/package_1664294369833.zip?***"}
2022-09-27T15:59:32.4412565Z Updated App Service Application settings.
2022-09-27T15:59:32.4414182Z Updated WEBSITE_RUN_FROM_PACKAGE Application setting to https://categoriserfunc9c44.blob.core.windows.net/azure-pipelines-deploy/package_1664294369833.zip?***
2022-09-27T15:59:37.4527980Z Syncing triggers for function app
2022-09-27T15:59:39.3782043Z Sync triggers for function app completed successfully
2022-09-27T15:59:41.0691225Z Successfully added release annotation to the Application Insight : categoriserfunctionapp
2022-09-27T15:59:41.3968439Z App Service Application URL: https://categoriserfunctionapp.azurewebsites.net
2022-09-27T15:59:41.4063927Z ##[section]Finishing: AzureFunctionApp
We tried changing the hardcode of WEBSITE_RUN_FROM_PACKAGE but it seems to have changed itself back since I refreshed the function app.
Does anyone have any ideas for fixing this?
I can successfully deploy to azure function app with a slight modification based on the YAML you provided.
trigger:
- none
variables:
# Azure Resource Manager connection created during pipeline creation
azureSubscription: '<ARM Service Connection Name>'
resourceGroupName: '<Resource Group Name of storage>'
# Function app name
functionAppName: '<Your Function App Name>'
# Agent VM image name
vmImageName: 'ubuntu-latest'
# Working Directory
workingDirectory: ''
storage_str: 'DefaultEndpointsProtocol=https;AccountName=xxx;AccountKey=xxx;EndpointSuffix=core.windows.net'
stages:
- stage: Build
displayName: Build stage
jobs:
- job: Build
displayName: Build
pool:
vmImage: $(vmImageName)
steps:
- task: UsePythonVersion#0
displayName: 'Use Python 3.9'
inputs:
versionSpec: 3.9 # Functions V2 supports Python 3.6 as of today
architecture: 'x64'
- bash: |
pip install --target="./.python_packages/lib/site-packages" -r ./requirements.txt
workingDirectory: $(workingDirectory)
displayName: 'Install application dependencies'
- task: ArchiveFiles#2
displayName: 'Archive files'
inputs:
rootFolderOrFile: "$(System.DefaultWorkingDirectory)"
includeRootFolder: false
archiveType: zip
archiveFile: "$(System.DefaultWorkingDirectory)/$(Build.BuildId).zip"
replaceExistingArchive: true
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: '$(System.DefaultWorkingDirectory)/$(Build.BuildId).zip'
artifactName: 'drop'
- stage: Deploy
displayName: Deploy stage
dependsOn: Build
condition: succeeded()
jobs:
- deployment: Deploy
displayName: Deploy
environment: 'test'
pool:
vmImage: 'windows-latest'
strategy:
runOnce:
deploy:
steps:
- task: DownloadPipelineArtifact#2
displayName: 'Download Pipeline Artifact'
inputs:
buildType: 'current'
artifactName: 'drop'
targetPath: '$(Pipeline.Workspace)/drop/'
- task: AzureAppServiceSettings#1
inputs:
azureSubscription: '$(azureSubscription)'
appName: '$(functionAppName)'
resourceGroupName: '$(resourceGroupName)'
appSettings: |
[
{
"name": "AzureWebJobsStorage",
"value": "$(storage_str)",
"slotSetting": false
}
]
- task: AzureFunctionApp#1
inputs:
azureSubscription: '$(azureSubscription)'
appType: 'functionAppLinux'
appName: '$(functionAppName)'
package: '$(Pipeline.Workspace)/drop/$(Build.BuildId).zip'
runtimeStack: 'PYTHON|3.9'
From your description, the blob file doesn't exist? If so, the function will not execute successfully.
You must make sure there is a file to use for 'run from package'.
There are several problems with your YAML which may cause the issue:
Variables in YAML are not fully used. The service connection and variable definitions in the Deployment stage are different (I have changed to the same).
The python version used is different from the azure function app version (I have changed to the same).
By the way, In order to rule out the problem that the storage is controlled by some policy or program, you can create a new storage to test and provide the connection string in the above YAML file (the location of the package your function app is based on is determined by AzureWebJobsStorage, the above YAML can be Change settings before actual deployment.).
And additional, if you can deploy the function app with no problem on local(such as VS Code), then you can use something like below to deploy the function app.
trigger:
- none
variables:
- name: System.debug
value: true
pool:
VMAS #agent based on your local machine.
steps:
- task: AzurePowerShell#5
inputs:
azureSubscription: 'xxx'
ScriptType: 'InlineScript'
Inline: 'func azure functionapp publish xxx --build remote'
azurePowerShellVersion: 'LatestVersion'
My repo structure likes below:

Is it possible to run a code from Azure repo, as a function app with scheduler?

So I've finished my backend and frontend part of the project.
Now big aspect of my project is scraper function, which is implemented in the backend side of the code. Right now, I need to open VS code every day, and run a function which will trigger the scrapers. Now I've researched about, and Azure has a function apps which has a scheduled function.
Now what I want is: I want just to call a file inside my Azure repo. My backend and frontend are in the different repos, and I want to run file scraping-service.js inside scraping folder in order to scrape data and insert the data into the db.
Now normally I run pipeline with azure-service.yml which has its own configuration for running my project. Is any way to implement this function to run just scraping-service.js at certain time of the day?
Thanks!
Is any way to implement this function to run just scraping-service.js at certain time of the day?
In Azure Pipeline, you could set schedules for pipelines.
Here is a doc about the detailed info: Configure schedules for pipelines.
For example: In yaml pipeline , you could set cron .
trigger:
- main
variables:
# Azure Resource Manager connection created during pipeline creation
azureSubscription: 'xxx'
# Web app name
webAppName: 'stanbackapp'
# Environment name
environmentName: 'stanbackapp'
# Agent VM image name
vmImageName: 'ubuntu-latest'
schedules:
- cron: "0 0 * * *"
displayName: Daily midnight build
branches:
include:
- main
stages:
- stage: Build
displayName: Build stage
condition: ne(variables['Build.Reason'], 'Schedule')
jobs:
- job: Build
displayName: Build
pool:
vmImage: $(vmImageName)
steps:
- task: NodeTool#0
inputs:
versionSpec: '10.x'
displayName: 'Install Node.js'
- script: |
npm install
npm run build --if-present
npm run test --if-present
displayName: 'npm install, build and test'
- task: ArchiveFiles#2
displayName: 'Archive files'
inputs:
rootFolderOrFile: '$(System.DefaultWorkingDirectory)'
includeRootFolder: false
archiveType: zip
archiveFile: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
replaceExistingArchive: true
- upload: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
artifact: drop
- stage: Build1
displayName: Build1 stage
condition: eq(variables['Build.Reason'], 'Schedule')
jobs:
- job: Build1
displayName: Build1
pool:
vmImage: $(vmImageName)
steps:
- task: NodeTool#0
inputs:
versionSpec: '10.x'
displayName: 'Install Node.js'
- task: CmdLine#2
inputs:
script: ' node $(build.sourcesdirectory)/scraping/scraping-service.js'
- stage: Deploy
displayName: Deploy stage
dependsOn: Build
condition: succeeded()
jobs:
- deployment: Deploy
displayName: Deploy
environment: $(environmentName)
pool:
vmImage: $(vmImageName)
strategy:
runOnce:
deploy:
steps:
- task: AzureWebApp#1
displayName: 'Azure Web App Deploy: stanbackapp'
inputs:
azureSubscription: $(azureSubscription)
appType: webAppLinux
appName: $(webAppName)
runtimeStack: 'NODE|10.10'
package: $(Pipeline.Workspace)/drop/$(Build.BuildId).zip
startUpCommand: 'npm run dev'
In this case, when the pipeline is triggered by schedules, it will run Single file, or it will run the whole pipeline.
You can also add condition to task level.
The normal way is to create an azure devops pipeline that deploys the code you want to run to an azure function when the code changes. So put the code of scraping-service.js in the function or have the function call the method in scraping-service.js. See the docs
Although there might be ways to run the code in an Azure Devops pipeline I don't think it is meant to run application code. You won't have the monitoring capabilities an azure function gives you, nor the availability of scaling, configuration and all the thing Azure Functions provides.

No package found with specified pattern: d:\a\r1\a\**\*.zip<br/>Check if the package mentioned in the task is published as an artifact

I am creating a Build and release pipeline for my .Net core FunctionApp in Azure. I am getting the following error
2020-07-13T07:59:10.6443361Z ##[error]Error: No package found with specified pattern: d:\a\r1\a\**\*.zip<br/>Check if the package mentioned in the task is published as an artifact in the build or a previous stage and downloaded in the current job.
Below is the yaml file of the pipeline
YAML file
# .NET Core Function App to Windows on Azure
# Build a .NET Core function app and deploy it to Azure as a Windows function App.
# Add steps that analyze code, save build artifacts, deploy, and more:
# https://learn.microsoft.com/en-us/azure/devops/pipelines/languages/dotnet-core
trigger:
- master
variables:
# Azure Resource Manager connection created during pipeline creation
azureSubscription: 'f296911a-0481-4fed-ba93-a30ef6a5b0f2'
# Function app name
functionAppName: 'srlcustomermanagerapp'
# Agent VM image name
vmImageName: 'vs2017-win2016'
# Working Directory
workingDirectory: '$(System.DefaultWorkingDirectory)/CustomerOrderApi'
stages:
- stage: Build
displayName: Build stage
jobs:
- job: Build
displayName: Build
pool:
vmImage: $(vmImageName)
steps:
- task: DotNetCoreCLI#2
displayName: Build
inputs:
command: 'build'
projects: |
$(workingDirectory)/*.csproj
arguments: --output $(System.DefaultWorkingDirectory)/publish_output --configuration Release
- task: ArchiveFiles#2
displayName: 'Archive files'
inputs:
rootFolderOrFile: '$(System.DefaultWorkingDirectory)/publish_output'
includeRootFolder: false
archiveType: zip
archiveFile: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
replaceExistingArchive: true
- publish: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
artifact: drop
- stage: Deploy
displayName: Deploy stage
dependsOn: Build
condition: succeeded()
jobs:
- deployment: Deploy
displayName: Deploy
environment: 'development'
pool:
vmImage: $(vmImageName)
strategy:
runOnce:
deploy:
steps:
- task: AzureFunctionApp#1
displayName: 'Azure functions app deploy'
inputs:
azureSubscription: '$(azureSubscription)'
appType: functionApp
appName: $(functionAppName)
package: '$(Pipeline.Workspace)/drop/$(Build.BuildId).zip'
Release configuration
1.Make sure you've set the Build Pipeline as artifact source, and the build do provide the artifacts:
2.If the issue persists, try to specify the path via Browse Package or folder option:
Then you can check if your release can get the required xx.zip file via this option. Also you can choose to specify the file path using this option.

Deploying and running .exe files using Azure Pipelines

I´m struggling to make my MultiStage pipelines to run a .exe file in a hosted agent running in a Azure VM.
My .yaml file is:
trigger:
- develop
stages:
- stage: build
displayName: Build
jobs:
- job: buildJob
pool:
vmImage: 'ubuntu-16.04'
variables:
buildConfiguration: 'Release'
steps:
- task: NuGetToolInstaller#1
inputs:
versionSpec: '5.5.0'
- task: DotNetCoreCLI#2
displayName: 'Dotnet Build $(buildConfiguration)'
inputs:
command: 'build'
arguments: '--configuration $(buildConfiguration)'
projects: '**/TestProj.csproj'
- task: DotNetCoreCLI#2
displayName: "Publish"
inputs:
command: 'publish'
publishWebProjects: false
projects: '**/TestProj.csproj'
arguments: '--no-restore --configuration $(BuildConfiguration) --output $(Build.ArtifactStagingDirectory)'
zipAfterPublish: false
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: '$(Build.ArtifactStagingDirectory)'
ArtifactName: 'drop'
publishLocation: Container
- stage: Release
displayName: Release
dependsOn: build
jobs:
- deployment: AzureVMDeploy
displayName: agentDeploy
environment:
name: AzureDeploy
resourceName: vmName
resourceType: VirtualMachine
tags: develop
This VM is on the azure pipelines Environment. After I run this pipeline, the folder is downloaded into the VM, but I cannot find how to automate the execution of the output .exe file in this folder.
I think the way is to create a job with a task to do it, but I cannot figure out how to set the agent installed on the VM to run this task.
How can I do that?
If I understood you correctly, you want to execute your artifact file which was deployed to VM.
I think that PowerShell on Target Machines task should do the job for you. Yoy can write simple inline script to exeute your file. However, you need to have remoting confogured on VM. This article may help you with this.
You could specify tasks in strategy: Deployment job For example:
YAML
stages:
- stage: build
jobs:
- job: buildJob
pool:
vmImage: 'Ubuntu-16.04'
steps:
- task: PublishPipelineArtifact#1
inputs:
targetPath: '$(Pipeline.Workspace)'
publishLocation: 'pipeline'
- stage: deploy
dependsOn: build
jobs:
- deployment: DeployWeb
displayName: deploy Web App
environment:
name: vm1
resourceType: virtualmachine
strategy:
runOnce:
deploy:
steps:
- script: echo my first deployment
- task: CmdLine#2
inputs:
script: 'more README.md'
workingDirectory: '$(Pipeline.Workspace)/build.buildJob/s'
For this YAML pipeline, I publish all files in pipeline workspace to artifact in build stage, then this artifact will be download to target virtual machine of vm1 environment in deploy stage (folder name will be {stage name}.{job name}), then run command line task to get a file's content. (The script and command line tasks will be run on that virtual machine)

Resources