Dependencies getting copied to private feed in Azure DevOps - azure

I have set up a build pipeline for a model library that's shared between several of my projects. I'm accessing it through a private feed in Azure DevOps, and it works just fine. I can retrieve the library in Visual Studio and my projects all get the most up-to-date version. However, in the feed are all the dependency libraries used within the model library (e.g. Microsoft.Azure.Storage.Blob, System.Threading, Microsoft.AspNetCore, etc.). I haven't been able to find any guidance on why this is happening, if it's the expected behavior, or if I'm screwing something up. My YAML file for the build pipeline is below:
Also, does anyone know a better to handle package versioning? This seems really hacky, but it was the only way I could get auto-incrementing versions to work.
# Starter pipeline
# Start with a minimal pipeline that you can customize to build and deploy your code.
# Add steps that build, run tests, deploy, and more:
# https://aka.ms/yaml
name: $(projectName)-$(majorMinorVersion).$(semanticVersion)
pool:
vmImage: 'windows-latest'
# pipeline variables
variables:
majorMinorVersion: 1.1
# semanticVersion counter is automatically incremented by one in each execution of pipeline
# second parameter is seed value to reset to every time the referenced majorMinorVersion is changed
semanticVersion: $[counter(variables['majorMinorVersion'], 0)]
projectName: 'MyProject.Models'
buildConfiguration: 'Release'
projectPath: 'Shared/MyProject.Models.csproj'
fullVersion: '$(majorMinorVersion).$(semanticVersion)'
steps:
# show version number on start
- task: Bash#3
inputs:
targetType: 'inline'
script: |
echo Building $(projectName)-$(fullVersion)
- task: DotNetCoreCLI#2
inputs:
command: 'pack'
packagesToPack: $(projectPath)
versioningScheme: 'byEnvVar'
versionEnvVar: 'fullVersion'
- task: DotNetCoreCLI#2
inputs:
command: 'push'
packagesToPush: '$(Build.ArtifactStagingDirectory)/MyProject.Models*.nupkg'
nuGetFeedType: 'internal'
publishVstsFeed: '<feed GUID>'

Dependencies getting copied to private feed in Azure DevOps
That because your private Nuget Feed set nuget.org as an Upstream source by default if you set Package from public sources enable when you create the this feed:
Then go to Setting->Upstream source, you will find there are three public sources listed:
When we download any packages from the Upstream sources, it will been cached in the Artifacts, you will see it next time. They are cached packages from upstream sources, so we do not need to download them again from upstream sources next time we use them, and the included upstream sources are all approved by MS, so you do not need to worry about them.
Besides, if you still worry about them, you can disable the Upstream sources, but in this case, you need publish all the dependencies to your private feed, otherwise, your model library package will throw the error could not found the dependencies.
Hope this helps.

Related

Can we reference a YAML template from a Nuget package?

I need to update a couple of pipelines to use the same logic and thought about using templates to split the common YAML steps.
All the projects share the same dependency with an in-house Nuget package that is used for deployment. My initial idea is to use that package to hold the template YAML and reference it on each project but when I read the documentation I am not entirely sure this is supported.
To give you a better idea I'm going to exemplify with the same sample code on the documentation.
This would be our YAML in the Nuget package, to be inserted as a template reference on each project.
# File: templates/include-npm-steps.yml
steps:
- script: npm install
- script: yarn install
- script: npm run compile
This would be the YAML on each project that needed to consume the above YAML from the Nuget package.
# File: azure-pipelines.yml
jobs:
- job: Linux
pool:
vmImage: 'ubuntu-latest'
steps:
- template: templates/include-npm-steps.yml # Template reference
- job: Windows
pool:
vmImage: 'windows-latest'
steps:
- template: templates/include-npm-steps.yml # Template reference
Probably at run time the pipeline checks for the referenced templates and won't run since they need to be restored first. Any ideas?
As suspected you cannot consume a YAML in a Nuget package but have to reference the YAML from a repo instead.

AzureDevops Task name NuGetCommand is ambiguous

I am getting the following error when trying to run my pipeline
Job Job: Step task reference is invalid. The task name NuGetCommand is ambiguous.
Specify one of the following identifiers to resolve the ambiguity:
.NuGetCommand, .NuGetCommand
Below is my .yml file for my .Net Standard library
trigger:
- main
pool:
vmImage: 'windows-latest'
variables:
solution: '**/MyLibrary.sln'
buildPlatform: 'Any CPU'
buildConfiguration: 'Release'
steps:
- task: NuGetToolInstaller#1
- task: NuGetCommand#2
inputs:
command: 'restore'
restoreSolution: '**/MyLibrary.sln'
feedsToUse: 'config'
nugetConfigPath: './nuget.config'
- task: VSBuild#1
inputs:
solution: '**\MyLibrary.sln'
vsVersion: '15.0'
restoreNugetPackages: true
It seems to be complaining about NugetCommand#2 which doesn't make sense when it comes built in?
Edit
Have also tried using - task: 333b11bd-d341-40d9-afcf-b32d5ce6f23b#2 instead of - task: NuGetCommand#2
One may have been pushed straight into the account using tfx build tasks upload, run tfx build tasks list to find out which ones are installed and if needed delete the non-official one using tfx build tasks delete.
C:\Users\jesse>npm install tfx-cli -g
C:\Users\jesse>tfx build tasks list
TFS Cross Platform Command Line Interface v0.8.3
Copyright Microsoft Corporation
> Service URL: https://dev.azure.com/jessehouwing-dev
> Personal access token:
The one you want to keep is:
id : 333b11bd-d341-40d9-afcf-b32d5ce6f23b
name : NuGetCommand
friendly name : NuGet
visibility :
description : Restore, pack, or push NuGet packages, or run a NuGet command. Supports NuGet.org and authenticated feeds like Azure Artifacts and MyGet. Uses NuGet.exe and works with .NET Framework apps. For .NET Core and .NET Standard apps, use the .NET Core task.
version : 2.179.0
If there is one with a different guid, delete is with:
C:\Users\jesse>tfx build tasks delete --task-id the-task-id-guid-to-delete
It may also have been pushed as part of a privately shared custom extension. The marketplace will block tasks with the rame GUID, but it will allow installing a task with the same name just fine through an extension. Check your installed extensions, especially privately shared ones.
AzureDevops Task name NuGetCommand is ambiguous
According to the error message, it seems that error can happen when two tasks\extensions exist in your Azure DevOps organization with the same name.
You could use this REST API:
https://dev.azure.com/{organisationName}/_apis/distributedtask/tasks?visibility%5B%5D=Build
to check here for possible duplicates.
To resolve this issue, you could use command line to invoke nuget.exe to restore the solution:
nuget.exe restore a.sln -source "xx" -PackagesDirectory

How to create a YAML build pipeline in azure devops for xamarin projects with referenced dot net projects

I recently moved my sources to azure devOps to use cd/ci and all the other cool stuff.
Now i created my first build pipeline to build the android part of my Xamarin project. But I end up getting an error message, that a resource of a referenced project could not be found and i shall do a package restore and try again.
Now, since i have azure hosted build agents and not self hosted, i have no ways of setting the agent up properly before doing the build.
But i guess there should be some way to properly configure the build pipeline to do all the necessary stuff.
Its just that i have no clue what i should add to my yaml file in order to fix this stuff.
This is the error message i got:
##[error]C:\Program Files\dotnet\sdk\2.2.105\Sdks\Microsoft.NET.Sdk\targets\Microsoft.PackageDependencyResolution.targets(208,5): Error NETSDK1004: Assets file 'd:\a\1\s\*****\*****\*****\*****\obj\project.assets.json' not found. Run a NuGet package restore to generate this file.
The problem is, that this file should be generated by compiling a referenced project and is not part of a nuget package.
Here is my build pipeline as far as i figured it out by myself.
# Xamarin.Android
# Build a Xamarin.Android project.
# Add steps that test, sign, and distribute an app, save build artifacts, and more:
# https://learn.microsoft.com/azure/devops/pipelines/languages/xamarin
trigger:
- Share/main
pool:
vmImage: 'VS2017-Win2016'
variables:
buildConfiguration: 'Debug'
outputDirectory: '$(build.binariesDirectory)/$(buildConfiguration)'
steps:
- task: NuGetToolInstaller#1
inputs:
versionSpec: 5.1.0
- task: NuGetCommand#2
displayName: 'Restore NuGet Packages'
inputs:
command: restore
restoreSolution: '**/*.sln'
- task: XamarinAndroid#1
inputs:
projectFile: 'Mobile4/Droid/Mobile4.Droid.csproj'
outputDirectory: '$(outputDirectory)'
configuration: '$(buildConfiguration)'
- task: AndroidSigning#3
inputs:
apksign: false
zipalign: false
apkFiles: '$(outputDirectory)/*.apk'
- task: PublishBuildArtifacts#1
inputs:
pathtoPublish: '$(outputDirectory)'
The build always breaks on step XamarinAndroid
I hope you can help me.
The solution must be out there somewhere, i just cannot see it right now.
Thx in Advance.
Mav
[error]C:\Program Files\dotnet\sdk\2.2.105\Sdks\Microsoft.NET.Sdk\targets\Microsoft.PackageDependencyResolution.targets(208,5):
Error NETSDK1004: Assets file
'd:\a\1\s********************\obj\project.assets.json' not found.
Run a NuGet package restore to generate this file.
According to this error message, the project is .NetCore and its SDK used is 2.2.105. For the file "....\obj\project.assets.json", whether the project.assets.json exists is determined by package restore step. Now, it prompt this could not be found, it means the package restore does not restore this file successfully.
As I mentioned previously, it is a .NetCore project. So you should use dotnet restore instead of nuget restore. For .NetCore project, the obj folder restored by nuget restore does not contain project.assets.json in it.
So, to solve the issue you meet, you should replace the task Nuget restore as dotnet restore: dotnet.
- task: DotNetCoreCLI#2
displayName: 'dotnet restore'
inputs:
command: restore
projects: '**/*.csproj'
vstsFeed: 'e157d03d-******-fc06f9e13177'

nuget push timeouts after 5 minutes in Azure DevOps

I'm trying to publish my node application using NuGet and Azure DevOps.
Client requires to pack node_modules together with source code, so that application can be used without installing dependencies on host.
Here is part of azure-pipelines.yml responsible for pushing package:
- task: NuGetCommand#2
displayName: "Publish the artifacts"
inputs:
command: push
packagesToPush: "path-to-package.nupkg"
nuGetFeedType: internal
publishVstsFeed: "SomeFeed"
versioningScheme: byEnvVar
versionEnvVar: BuildNumber
verbosityPush: "Detailed"
arguments: "-Timeout 901"
Docs say that i can use arguments field to pass parameters to the command, but it seems it it ignored and push command always times out after 5 minutes (or 300 seconds - that is default timeout)
There is no information in the docs about format of arguments field, I just assumed it should be a string.
What am I missing?

Azure DevOps pipeline build locally with YAML

how can I simulate the build process of Azure Devops pipeline on the local machine before pushing it to branch to test the possible errors.
the solution gets build locally correct with no errors and warnings. also from the VS command line MSBuild builds the solution with no errors but on some push tries the pipeline build throws many errors mostly related to preprocessor defenition and precompiled header.
I wanted to know how can test the same process locally on my machine without pushing to repo.
azure-pipelines.yml
-------------------
pool:
vmImage: 'vs2017-win2016'
steps:
- task: MSBuild#1
displayName: 'Build solution'
inputs:
platform: 'Win32'
configuration: 'release'
solution: 'mysolution.sln'
- task: VSTest#2
displayName: 'Run Test'
inputs:
platform: 'Win32'
Configuration: 'release'
testAssemblyVer2: |
**\*.Test.dll
!**\*TestAdapter.dll
!**\obj\**
runSettingsFile: project.Test/test.runsettings
codeCoverageEnabled: true
If you are using a git repsotiory you can create another branch and make a pull request. As long as the pull request is not set to auto complete the code will not get committed to the repository.
If you are using a TFVC respository you can setup a gated build that is configured to fail. The pipeline should be a copy of your original pipeline but add a PowerShell task at the end of the build pipeline that throws a terminating error. Be sure to setup this gated build on a separate branch so it does not block normal development.
Write-Error "Fail here" -ErrorAction 'Stop'
You can now make pull requests or trigger a gated build without the code actually being commited.
You can use AzurePipelinesPS to install an agent on your local machine with the Install-APAgent command if you need another agent.
I'm only a few hours into to development with Azure, but I think I found a solution that would work for you. I happen to already have the solution in place. Use gradle, then the default YML just runs gradle and you don't have to worry too much about it after the first run. In the gradle file you could also spin up a docker image if you want and build on that.
The issue you have is most likely related to the difference between your local environment and the one on build agent where this YAML pipeline execute the build. Testing it locally (even if it would be possible) will not help as it will be executed in your environment, where you know already that every component required for the successful build are already installed. On the other side on the environment where build agent is running the build there seems to be missed components (or different versions) which cause your build to fail. Try to compare list of installed components and environment variables (like PATH) on your local machine and on build agent - there might be some difference between them.

Resources