Pipeline Artifact not Downloading with Stages - azure

Question

I have recently upgraded my YAML pipeline to include Stages, since this has happened the build artifacts are not downloading in a later stage task. For the life of me I cant figure out why.
Please see my YAML Code Bellow with an explanation on how my pipeline works.
First I have the main pipeline that calls the template yaml files.
mainpipeline.yml
pool:
vmImage: 'ubuntu-latest'
resources:
repositories:
- repository: Terraform
name: VALUE/Terraform
path:
- include: /Terraform
type: git
ref: main #branch name
- repository: Website
name: VALUE/Website
path:
- include: /Website
type: git
ref: newartifactpipeline #branch name
- repository: AuthenticationServer
name: VALUE/AuthenticationServer
path:
- include: /AuthenticationServer
type: git
ref: VALUE #branch name
trigger:
branches:
include:
- master
variables:
buildConfiguration: 'Release'
stages:
- stage: build_website_files
displayName: Building Main Website Files
jobs:
- job: build_main_website
steps:
- checkout: Website
- template: buildartifact.yml#website
parameters:
#vmImage: 'windows-latest'
buildConFiguration: $(buildConfiguration)
project: Website.csproj
artifactName: Website
- stage: build_authenticationserver_files
displayName: Building AuthenticationServer Website Files
jobs:
- job: build_authenticationserver_website
steps:
- checkout: AuthenticationServer
- template: buildartifact.yml#AuthenticationServer
parameters:
# vmImage: 'windows-latest'
buildConFiguration: $(buildConfiguration)
project: AuthenticationServer.csproj
artifactName: AuthenticationServer
- stage: run_terraform_pre_build
displayName: Building Terraform Applications and Deploying Web Apps
jobs:
- job: building_terraform_applications
steps:
- checkout: Terraform
- template: /VALUE/runterraform.yml#Terraform
parameters:
terraformWorkingDirectory: '$(System.DefaultWorkingDirectory)/VALUE'
serviceConnection: 'VALUE'
azureSubscription: 'VALUE'
appconnectionname: 'VALUE'
backendresourcegroupname: 'VALUE'
backendstorageaccountname: 'VALUE'
backendcontainername: 'VALUE'
RG: 'RG_Example'
azureLocation: 'UK South'
terraformVersion: '1.0.4'
artifactName: 'Website'
- stage: run_terraform_post_build
displayName: Apply Post Build Settings
jobs:
- job: apply_post_build_settings
steps:
- checkout: Terraform
- template: /Terraform/PostBuild/runterraformpostbuild.yml#Terraform
parameters:
terraformWorkingDirectory: '$(System.DefaultWorkingDirectory)/VALUE/PostBuild'
serviceConnection: 'VALUE'
azureSubscription: 'VALUE'
appconnectionname: 'VALUE'
backendresourcegroupname: ''VALUE''
backendstorageaccountname: 'VALUE'
backendcontainername: 'VALUE'
RG: 'RG_Example'
azureLocation: 'UK South'
terraformVersion: '1.0.4'
artifactName: 'Website'
The first stage builds and calls this build artifact template yaml file, that does successfully publish the artifact file, yaml code below:
Please see screenshot bellow as proof:
buildartifact.yml
parameters:
- name: buildConfiguration
type: string
default: 'Release'
- name: project
type: string
default: 'Website.csproj'
- name: artifactName
type: string
default: 'Website'
- name: vmImage
type: string
default: 'windows-latest'
jobs:
- job: build_website
pool:
vmImage: ${{ parameters.vmImage }}
steps:
- checkout: Website
- task: CmdLine#2
inputs:
script: |
echo '$(System.DefaultWorkingDirectory)'
dir
- task: DotNetCoreCLI#2
displayName: dotnet restore
inputs:
command: restore
projects: '**/${{ parameters.project }}'
# Node.js tool installer
# Finds or downloads and caches the specified version spec of Node.js and adds it to the PATH
- task: NodeTool#0
displayName: 'Install Node .js'
inputs:
versionSpec: '14.17.3'
force32bit: false # Optional
checkLatest: false # Optional
- script: |
npm install -g #angular/cli#12.1.3
npm install
displayName: 'npm install'
- task: Npm#1
displayName: 'npm run build'
inputs:
command: 'custom'
workingDir: ClientApp
customCommand: 'build'
- task: DotNetCoreCLI#2
displayName: 'Build'
inputs:
command: 'build'
projects: '**/${{ parameters.project }}'
arguments: '--configuration ${{ parameters.buildConfiguration }}'
- task: DotNetCoreCLI#2
displayName: dotnet restore unit tests
inputs:
command: restore
projects: 'UnitTests/UnitTests.csproj'
- task: DotNetCoreCLI#2
displayName: dotnet Test
inputs:
command: test
projects: 'UnitTests/UnitTests.csproj'
arguments: '--configuration Release'
- task: DotNetCoreCLI#2
displayName: 'Publish Application'
inputs:
command: 'publish'
publishWebProjects: false
projects: '**/${{ parameters.project }}'
arguments: '--configuration ${{ parameters.buildConfiguration }} --output $(Pipeline.Workspace)/website/'
- task: PublishPipelineArtifact#1
displayName: 'Publish Artifacts'
inputs:
targetPath: '$(Pipeline.Workspace)/website/'
artifact: ${{ parameters.artifactName }}
publishLocation: 'pipeline'
With this yaml for the PublishPipelineArtifact task I have tried the following env variables for pipleines: pipeline.workspace and System.DefaultWorkingDirectory
Yet both have not worked in the later stage where the final yaml file tries to download the pipeline artifact see yaml code bellow:
runterraformanddownloadartifact.yml
parameters:
- name: terraformWorkingDirectory
type: string
default: $(System.DefaultWorkingDirectory)/Terraform
- name: serviceConnection
type: string
default: value
- name: azureSubscription
type: string
default: value
- name: appconnectionname
type: string
default: value
- name: backendresourcegroupname
type: string
default: DevOpsTerraform
- name: backendstorageaccountname
type: string
default: value
- name: backendcontainername
type: string
default: value
- name: RG
type: string
default: RG_Example
- name: azureLocation
type: string
default: UK South
- name: terraformVersion
type: string
default: 1.0.4
- name: artifactName
type: string
default: Website
jobs:
- job: Run_Terraform
displayName: Installing and Running Terraform
steps:
- checkout: Terraform
- task: TerraformInstaller#0
displayName: install
inputs:
terraformVersion: '${{ parameters.terraformVersion }}'
- task: CmdLine#2
inputs:
script: |
echo '$(System.DefaultWorkingDirectory)'
dir
- task: TerraformTaskV2#2
displayName: init
inputs:
provider: azurerm
command: init
backendServiceArm: '${{ parameters.serviceConnection }}'
backendAzureRmResourceGroupName: '${{ parameters.backendresourcegroupname }}'
backendAzureRmStorageAccountName: '${{ parameters.backendstorageaccountname }}'
backendAzureRmContainerName: '${{ parameters.backendcontainername }}'
backendAzureRmKey: terraform.tfstate
workingDirectory: '${{ parameters.terraformWorkingDirectory }}'
- task: TerraformTaskV1#0
displayName: plan
inputs:
provider: azurerm
command: plan
commandOptions: '-input=false'
environmentServiceNameAzureRM: '${{ parameters.serviceConnection }}'
workingDirectory: '${{ parameters.terraformWorkingDirectory }}'
- task: TerraformTaskV1#0
displayName: apply
inputs:
provider: azurerm
command: apply
commandOptions: '-input=false -auto-approve'
environmentServiceNameAzureRM: '${{ parameters.serviceConnection }}'
workingDirectory: '${{ parameters.terraformWorkingDirectory }}'
- job: Put_artifacts_into_place
displayName: Putting_artifacts_into_place
dependsOn: Run_Terraform
steps:
- checkout: Website
- checkout: AuthenticationServer
- task: DownloadPipelineArtifact#2
displayName: Download Build Artifacts
inputs:
artifact: '${{ parameters.artifactName }}'
patterns: /website/**/*.zip
path: $(Pipeline.Workspace)/website/
- task: AzureWebApp#1
displayName: 'Azure Web App Deploy: VALUE'
inputs:
package: $(Pipeline.Workspace)/website/**/*.zip
azureSubscription: '${{ parameters.azureSubscription }}'
ConnectedServiceName: '${{ parameters.appconnectionname}}'
appName: VALUE
ResourceGroupName: '${{ parameters.RG}}'
- task: DownloadPipelineArtifact#2
displayName: Download Build Artifacts
inputs:
artifact: '${{ parameters.artifactName}}'
patterns: /authsrv/**/*.zip
path: $(Pipeline.Workspace)/authsrv/
- task: AzureWebApp#1
displayName: 'Azure Web App Deploy: VALUE'
inputs:
package: $(Pipeline.Workspace)/authsrv/**/*.zip
azureSubscription: '${{ parameters.azureSubscription }}'
ConnectedServiceName: '${{ parameters.appconnectionname}}'
appName: VALUE
ResourceGroupName: '${{ parameters.RG}}'
Essentially the first pipeline calls these two pipelines which are wrapped in Stages. Before they were not wrapped in stages and this pipeline worked well. Since moving it to stages I have had the issue where the task: DownloadPipelineArtifact#2 completes but downloads nothing. Please see screen shot bellow:
The error I am getting at the end of the pipeline is:
##[error]Error: No package found with specified pattern: /home/vsts/work/1/website/**/*.zip<br/>Check if the package mentioned in the task is published as an artifact in the build or a previous stage and downloaded in the current job.
I have tried the following solutions without success:
File pattern for Publish Pipeline Artifact in Azure DevOps
how to use PublishPipelineArtifact#1 with build script
And consulted MS Doc: https://learn.microsoft.com/en-us/azure/devops/pipelines/artifacts/pipeline-artifacts?view=azure-devops&tabs=yaml