uninstall packages in azure function - python-3.x

My azure function app(python) is throwing an exception: module typing has no attribute '_classVar'. A fix for this would be to uninstall the dataclasses package. How do I uninstall this package on a python azure function using pip?
If I run pip uninstall dataclasses, will this reflect on deployment?

If you are using python version 3.7 or greater you need to uninstall the dataclass library using the same pip uninstall dataclasses.
As The dataclasses package is a backport of the Python 3.7 dataclass functionality.
Or, if still you want to exist dataclasses you can downgrade your python version to 3.6.
For more information please refer the below links:
Blog|AttributeError: module ‘typing’ has no attribute ‘_ClassVar’ with Tune
Similar GitHub Issue

I was also having a lot of trouble trying to deploy azure functions from an Azure Devops pipeline with a Python 3.7 environment, so I decided to place this here as it might help someone else with the same problem.
You need to prepare the following yaml file with your respective variables.
trigger:
- {{ branch }}
variables:
# Azure Resource Manager connection created during pipeline creation
azureSubscription: '{{ azureRmConnection.Id }}'
# Function app name
functionAppName: '{{ functionAppName }}'
# Agent VM image name
vmImageName: 'ubuntu-latest'
# Working Directory
workingDirectory: '{{ workingDirectory }}'
stages:
- stage: Build
displayName: Build stage
jobs:
- job: Build
displayName: Build
pool:
vmImage: $(vmImageName)
steps:
- bash: |
if [ -f extensions.csproj ]
then
dotnet build extensions.csproj --runtime ubuntu.16.04-x64 --output ./bin
fi
workingDirectory: $(workingDirectory)
displayName: 'Build extensions'
- task: UsePythonVersion#0
displayName: 'Use Python 3.6'
inputs:
versionSpec: 3.6 # Functions V2 supports Python 3.6 as of today
- bash: |
pip install --target="./.python_packages/lib/site-packages" -r ./requirements.txt
rm -rf ./.python_packages/lib/site-packages/dataclasses-0.6*
rm ./.python_packages/lib/site-packages/dataclasses.py
workingDirectory: $(workingDirectory)
displayName: 'Install application dependencies'
- task: ArchiveFiles#2
displayName: 'Archive files'
inputs:
rootFolderOrFile: '$(workingDirectory)'
includeRootFolder: false
archiveType: zip
archiveFile: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
replaceExistingArchive: true
- publish: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
artifact: drop
- stage: Deploy
displayName: Deploy stage
dependsOn: Build
condition: succeeded()
jobs:
- deployment: Deploy
displayName: Deploy
environment: 'development'
pool:
vmImage: $(vmImageName)
strategy:
runOnce:
deploy:
steps:
- task: AzureFunctionApp#1
displayName: 'Azure functions app deploy'
inputs:
azureSubscription: '$(azureSubscription)'
appType: functionAppLinux
appName: $(functionAppName)
package: '$(Pipeline.Workspace)/drop/$(Build.BuildId).zip'
These are the key lines after installing the requirements.txt. These will remove the package from the site-packages folder.
rm -rf ./.python_packages/lib/site-packages/dataclasses-0.6*
rm ./.python_packages/lib/site-packages/dataclasses.py
pip uninstall dataclasses will not work because you are not in the right folder.
Hope this helps!

Related

azure function app - linux consumption plan - unable to import modules

The deployment with vscode run 100% fine,
in the log I see it uses oryx.
header:
import datetime
import logging
import adal
import requests
import json
I want to upload the code using Azure Pipelines though, for the sake of automation.
Here is my code
steps:
- bash: |
if [ -f extensions.csproj ]
then
dotnet build extensions.csproj --output ./bin
fi
displayName: 'Build extensions'
- task: UsePythonVersion#0
displayName: 'Use Python 3.9'
inputs:
versionSpec: '3.9'
- bash: |
python3.9 -m venv worker_venv
source worker_venv/bin/activate
pip3.9 install setuptools
pip3.9 install -r requirements.txt
displayName: 'Install application dependencies'
- task: ArchiveFiles#2
displayName: "Archive files"
inputs:
rootFolderOrFile: "$(System.DefaultWorkingDirectory)/functions"
includeRootFolder: false
archiveType: zip
archiveFile: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
replaceExistingArchive: true
- publish: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
artifact: drop
- task: AzureFunctionApp#1
displayName: 'Deploy functions to Function App'
inputs:
azureSubscription: Service-Conn
appType: functionAppLinux
appName: 'pythontest'
package: '$(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip'
runtimeStack: 'Python|3.9'
deploymentMethod: 'zipDeploy'
resourceGroupName: $(resourcegroup_name_app)
But I end up with module not found error (in the monitor of function in azure portal).
Result: Failure Exception: ModuleNotFoundError: No module named 'adal'.
the uploaded zip have site packages
there is no error in pipeline
What am I missing? Any ideas guys?
pip install --target="./.python_packages/lib/site-packages" -r ./requirements.txt is the command you want to run if you need to ship your libraries in the deployment zip file instead of running pip on function app service.
Your pipeline code installed libraries into .venv and function app runtime will not use that folder.
refs:
https://learn.microsoft.com/en-us/azure/azure-functions/functions-how-to-azure-devops?tabs=dotnet-core%2Cyaml%2Cpython#build-your-app
https://learn.microsoft.com/en-us/azure/azure-functions/deployment-zip-push#deployment-customization
I believe you have to run a task to install pip in pipeline console. See link below for better clarity: https://learn.microsoft.com/en-us/azure/devops/pipelines/ecosystems/python?view=azure-devops

Azure Pipeline refuses to deploy from zip instead calling from blob which doesn't exist

Very much still new to YAML, but i've showed this to a colleague who's equally confused as me. I've built a CI/CD pipeline, and the CI end seems to work just fine. The pipeline claims its deploying a function app successfully, but when I go to check, there's nothing in the function app, and the output code from the pipeline seems to be calling from a blob storage folder that doesn't exist. We can't work out how to change the behaviour of it.
Is this something anyone has seen before?
This is the YAML:
# Python Function App to Linux on Azure
# Build a Python function app and deploy it to Azure as a Linux function app.
# Add steps that analyze code, save build artifacts, deploy, and more:
# https://learn.microsoft.com/azure/devops/pipelines/languages/python
trigger:
- master
variables:
# Azure Resource Manager connection created during pipeline creation
azureSubscription: 'ab1c86d0-0d0c-4029-913b-e5483df967b2'
# Function app name
functionAppName: 'categoriserfunctionapp'
# Agent VM image name
vmImageName: 'ubuntu-latest'
# Working Directory
workingDirectory: ''
stages:
- stage: Build
displayName: Build stage
jobs:
- job: Build
displayName: Build
pool:
vmImage: $(vmImageName)
steps:
- bash: |
if [ -f extensions.csproj ]
then
dotnet build extensions.csproj --runtime ubuntu.16.04-x64 --output ./bin
fi
workingDirectory: $(workingDirectory)
displayName: 'Build extensions'
- task: UsePythonVersion#0
displayName: 'Use Python 3.6'
inputs:
versionSpec: 3.6 # Functions V2 supports Python 3.6 as of today
architecture: 'x64'
- bash: |
pip install --target="./.python_packages/lib/site-packages" -r ./requirements.txt
workingDirectory: $(workingDirectory)
displayName: 'Install application dependencies'
- task: ArchiveFiles#2
displayName: 'Archive files'
inputs:
rootFolderOrFile: "$(System.DefaultWorkingDirectory)"
includeRootFolder: false
archiveType: zip
archiveFile: "$(System.DefaultWorkingDirectory)/$(Build.BuildId).zip"
replaceExistingArchive: true
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: '$(System.DefaultWorkingDirectory)/$(Build.BuildId).zip'
artifactName: 'drop'
- stage: Deploy
displayName: Deploy stage
dependsOn: Build
condition: succeeded()
jobs:
- deployment: Deploy
displayName: Deploy
environment: 'test'
pool:
vmImage: 'windows-latest'
strategy:
runOnce:
deploy:
steps:
- task: DownloadPipelineArtifact#2
displayName: 'Download Pipeline Artifact'
inputs:
buildType: 'current'
artifactName: 'drop'
targetPath: '$(Pipeline.Workspace)/drop/'
- task: AzureFunctionApp#1
inputs:
azureSubscription: 'Visual Studio Enterprise Subscription – MPN (f1f3e234-557b-4acd-b353-2ff89c547e49)'
appType: 'functionAppLinux'
appName: 'categoriserfunctionapp'
package: '$(Pipeline.Workspace)/drop/$(Build.BuildId).zip'
runtimeStack: 'PYTHON|3.9'
And this is the output from the CD end:
2022-09-27T15:59:26.6330661Z ##[section]Starting: AzureFunctionApp
2022-09-27T15:59:26.6442433Z ==============================================================================
2022-09-27T15:59:26.6442718Z Task : Azure Functions
2022-09-27T15:59:26.6443009Z Description : Update a function app with .NET, Python, JavaScript, PowerShell, Java based web applications
2022-09-27T15:59:26.6443282Z Version : 1.208.2
2022-09-27T15:59:26.6443454Z Author : Microsoft Corporation
2022-09-27T15:59:26.6443686Z Help : https://aka.ms/azurefunctiontroubleshooting
2022-09-27T15:59:26.6443957Z ==============================================================================
2022-09-27T15:59:27.4092341Z Got service connection details for Azure App Service:'categoriserfunctionapp'
2022-09-27T15:59:30.9793711Z Trying to update App Service Application settings. Data: {"WEBSITE_RUN_FROM_PACKAGE":"https://categoriserfunc9c44.blob.core.windows.net/azure-pipelines-deploy/package_1664294369833.zip?***"}
2022-09-27T15:59:32.4412565Z Updated App Service Application settings.
2022-09-27T15:59:32.4414182Z Updated WEBSITE_RUN_FROM_PACKAGE Application setting to https://categoriserfunc9c44.blob.core.windows.net/azure-pipelines-deploy/package_1664294369833.zip?***
2022-09-27T15:59:37.4527980Z Syncing triggers for function app
2022-09-27T15:59:39.3782043Z Sync triggers for function app completed successfully
2022-09-27T15:59:41.0691225Z Successfully added release annotation to the Application Insight : categoriserfunctionapp
2022-09-27T15:59:41.3968439Z App Service Application URL: https://categoriserfunctionapp.azurewebsites.net
2022-09-27T15:59:41.4063927Z ##[section]Finishing: AzureFunctionApp
We tried changing the hardcode of WEBSITE_RUN_FROM_PACKAGE but it seems to have changed itself back since I refreshed the function app.
Does anyone have any ideas for fixing this?
I can successfully deploy to azure function app with a slight modification based on the YAML you provided.
trigger:
- none
variables:
# Azure Resource Manager connection created during pipeline creation
azureSubscription: '<ARM Service Connection Name>'
resourceGroupName: '<Resource Group Name of storage>'
# Function app name
functionAppName: '<Your Function App Name>'
# Agent VM image name
vmImageName: 'ubuntu-latest'
# Working Directory
workingDirectory: ''
storage_str: 'DefaultEndpointsProtocol=https;AccountName=xxx;AccountKey=xxx;EndpointSuffix=core.windows.net'
stages:
- stage: Build
displayName: Build stage
jobs:
- job: Build
displayName: Build
pool:
vmImage: $(vmImageName)
steps:
- task: UsePythonVersion#0
displayName: 'Use Python 3.9'
inputs:
versionSpec: 3.9 # Functions V2 supports Python 3.6 as of today
architecture: 'x64'
- bash: |
pip install --target="./.python_packages/lib/site-packages" -r ./requirements.txt
workingDirectory: $(workingDirectory)
displayName: 'Install application dependencies'
- task: ArchiveFiles#2
displayName: 'Archive files'
inputs:
rootFolderOrFile: "$(System.DefaultWorkingDirectory)"
includeRootFolder: false
archiveType: zip
archiveFile: "$(System.DefaultWorkingDirectory)/$(Build.BuildId).zip"
replaceExistingArchive: true
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: '$(System.DefaultWorkingDirectory)/$(Build.BuildId).zip'
artifactName: 'drop'
- stage: Deploy
displayName: Deploy stage
dependsOn: Build
condition: succeeded()
jobs:
- deployment: Deploy
displayName: Deploy
environment: 'test'
pool:
vmImage: 'windows-latest'
strategy:
runOnce:
deploy:
steps:
- task: DownloadPipelineArtifact#2
displayName: 'Download Pipeline Artifact'
inputs:
buildType: 'current'
artifactName: 'drop'
targetPath: '$(Pipeline.Workspace)/drop/'
- task: AzureAppServiceSettings#1
inputs:
azureSubscription: '$(azureSubscription)'
appName: '$(functionAppName)'
resourceGroupName: '$(resourceGroupName)'
appSettings: |
[
{
"name": "AzureWebJobsStorage",
"value": "$(storage_str)",
"slotSetting": false
}
]
- task: AzureFunctionApp#1
inputs:
azureSubscription: '$(azureSubscription)'
appType: 'functionAppLinux'
appName: '$(functionAppName)'
package: '$(Pipeline.Workspace)/drop/$(Build.BuildId).zip'
runtimeStack: 'PYTHON|3.9'
From your description, the blob file doesn't exist? If so, the function will not execute successfully.
You must make sure there is a file to use for 'run from package'.
There are several problems with your YAML which may cause the issue:
Variables in YAML are not fully used. The service connection and variable definitions in the Deployment stage are different (I have changed to the same).
The python version used is different from the azure function app version (I have changed to the same).
By the way, In order to rule out the problem that the storage is controlled by some policy or program, you can create a new storage to test and provide the connection string in the above YAML file (the location of the package your function app is based on is determined by AzureWebJobsStorage, the above YAML can be Change settings before actual deployment.).
And additional, if you can deploy the function app with no problem on local(such as VS Code), then you can use something like below to deploy the function app.
trigger:
- none
variables:
- name: System.debug
value: true
pool:
VMAS #agent based on your local machine.
steps:
- task: AzurePowerShell#5
inputs:
azureSubscription: 'xxx'
ScriptType: 'InlineScript'
Inline: 'func azure functionapp publish xxx --build remote'
azurePowerShellVersion: 'LatestVersion'
My repo structure likes below:

Use python environment as an Artifact in Azure pipelines

so I´m trying to create a python environment in az pipelines and publish as an artifact in order to use it across jobs. I can publish the artifact, also it is downloaded in next job, problem I have is that seems that "environment/bin/activate" doesn´t work, what am I doing wrong? is there another way to achieve that?
build and publish artifact
jobs:
- job: BuildPythonArtifact
workspace:
clean: all
pool:
vmImage: $(UBUNTU_DEFAULT_VERSION)
steps:
- task: UsePythonVersion#0
inputs:
versionSpec: "3.9"
displayName: "Use Python 3.9"
- script: |
python -m venv env
source env/bin/activate
python -m pip install --upgrade pip
pip install -r requirements.txt
workingDirectory: $(Build.SourcesDirectory)
displayName: "Install requirements"
- task: PublishPipelineArtifact#1
inputs:
targetPath: $(Build.SourcesDirectory)/env
artifactName: pythonenv
download artifact
steps:
- task: UsePythonVersion#0
inputs:
versionSpec: "3.9"
displayName: "Use Python 3.9"
- task: DownloadPipelineArtifact#2
inputs:
artifact: pythonenv
- script: |
source $(Pipeline.Workspace)/bin/activate
files are there, bit seems that it ignores the env activate
What I expect is to can activate or reuse python env to avoid installing requirements in every job

Use powershell script to customize azure devOps pipeline?

I have an azure function app that is running on Azure. I am now trying to configure the application based on a powershell script. This powershell script is being used to create azure resources (Ex: KeyVault, ApplicationInsights...) that will be then used by the function app. I created the powershell script and my question here is how can I add the powershell script in the yaml file which is responsible to deploy the application. The powershell script is located in the repository under the name functionapp.ps1 . The idea here is to run the powershell script once the build is finished so in the deploy stage. What I did so far is the following:
stages:
- stage: Build
displayName: Build stage
jobs:
- job: Build
displayName: Build
pool:
vmImage: $(vmImageName)
steps:
- task: DotNetCoreCLI#2
displayName: Build
inputs:
command: 'build'
projects: |
$(workingDirectory)/*.csproj
arguments: --output $(System.DefaultWorkingDirectory)/publish_output --configuration Release
- task: ArchiveFiles#2
displayName: 'Archive files'
inputs:
rootFolderOrFile: '$(System.DefaultWorkingDirectory)/publish_output'
includeRootFolder: false
archiveType: zip
archiveFile: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
replaceExistingArchive: true
- publish: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
artifact: drop
- stage: Deploy
displayName: Deploy stage
dependsOn: Build
condition: succeeded()
jobs:
- deployment: Deploy
displayName: Deploy
environment: 'development'
pool:
vmImage: $(vmImageName)
strategy:
runOnce:
deploy:
steps:
- task: CmdLine#2
inputs:
script: |
echo '$(System.DefaultWorkingDirectory)'
dir
- task: PowerShell#2
inputs:
targetType: 'filePath'
filePath: $(System.DefaultWorkingDirectory)/functionapp.ps1
- task: AzureFunctionApp#1
displayName: 'Azure functions app deploy'
inputs:
azureSubscription: '$(azureSubscription)'
appType: functionApp
appName: $(functionAppName)
package: '$(Pipeline.Workspace)/drop/$(Build.BuildId).zip'
When the stages are being run and it arrives to this task I get the following error:
##[error]Invalid file path 'D:\a\1\s\functionapp.ps1'. A path to a .ps1 file is required.
I tried using the command line task to check the default working directory and I got the following result: D:\a\1\s
I still don't know what I am missing here and why I am getting this error. Can someone please give me a concrete solution to this problem ?
I ended up having the same error. Can someone please tell me what is the issue here ?

Deployment of Python web app fails when all packages are not included in the file requirements.txt (even though they are installed in the yaml tasks))

I deploy Azure functions using yaml scripts.
For some reasons, I do not put all my package requirements in the file requirements.txt, as this is used by some other processes.
Yet, when deploying my web app through a YAML pipeline, I want to install additional python packages. However, the resulting app that is deployed crashes with errors saying that it does not have those packages installed.
my pipeline:
# Python to Linux Web App on Azure
# Build your Python project and deploy it to Azure as a Linux Web App.
# Change python version to one thats appropriate for your application.
# https://learn.microsoft.com/azure/devops/pipelines/languages/python
trigger:
- master
variables:
# Azure Resource Manager connection created during pipeline creation
azureServiceConnectionId: .........................
# Web app name
webAppName: .........................
# Agent VM image name
vmImageName: 'ubuntu-latest'
# Environment name
environmentName: .........................
# Project root folder. Point to the folder containing manage.py file.
projectRoot: $(System.DefaultWorkingDirectory)
# Python version: 3.8
pythonVersion: '3.8'
stages:
- stage: Build
displayName: Build stage
jobs:
- job: BuildJob
pool:
vmImage: $(vmImageName)
steps:
- task: UsePythonVersion#0
inputs:
versionSpec: '$(pythonVersion)'
displayName: 'Use Python $(pythonVersion)'
- script: |
python -m venv antenv
source antenv/bin/activate
python -m pip install --upgrade pip
pip install setup
pip install -r requirements.txt
pip install pyodbc
workingDirectory: $(projectRoot)
displayName: "Install requirements"
- task: ArchiveFiles#2
displayName: 'Archive files'
inputs:
rootFolderOrFile: '$(projectRoot)'
includeRootFolder: false
archiveType: zip
archiveFile: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
replaceExistingArchive: true
- upload: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
displayName: 'Upload package'
artifact: drop
- stage: Deploy
displayName: 'Deploy Web App'
dependsOn: Build
condition: succeeded()
jobs:
- deployment: DeploymentJob
pool:
vmImage: $(vmImageName)
environment: $(environmentName)
strategy:
runOnce:
deploy:
steps:
- task: UsePythonVersion#0
inputs:
versionSpec: '$(pythonVersion)'
displayName: 'Use Python version'
- task: AzureWebApp#1
displayName: 'Deploy Azure Web App : .........................
inputs:
azureSubscription: $(azureServiceConnectionId)
appName: $(webAppName)
package: $(Pipeline.Workspace)/drop/$(Build.BuildId).zip
startUpCommand: 'startup-nocontainer.sh'
Here, pyodbc is installed separately in the pipeline, and not present in the file requirements.txt.
Checking the logs of the pipeline, I can see that the installation was performed successfully.
Yet when the app starts, it crashes at first import of pyodbc, as if the app that is eventually deployed only relyed on requirements.txt.
Any idea?
I think pyodbc is not being installed in the venv you create. The venv contains only the packages you specify in requirements.txt
Check the solutions proposed in this old post

Resources