Renovate bot Azure DevOps npm feed auth error - azure

I get an 401 error if I try to use my private npm registry in Azure DevOps. My configuration looks like this:
# pipeline.yaml (repo root folder)
steps:
- task: npmAuthenticate#0
inputs:
workingFile: .npmrc
- script: |
git config --global user.email 'bot#renovateapp.com'
git config --global user.name 'Renovate Bot'
npx --userconfig .npmrc renovate
env:
TOKEN: $(System.AccessToken)
PAT: $(PAT)
# config.js (repo root folder)
module.exports = {
platform: 'azure',
endpoint: 'https://devops.<url>.de/.../',
logLevel: 'debug',
token: process.env.TOKEN,
repositories: ['...'],
enabledManagers: ["npm"],
hostRules: [
{
enabled: true,
hostType: 'npm',
matchHost: 'devops.<url>.de',
token: process.env.PAT,
},
],
};
# .npmrc (repo root folder)
registry=https://devops.<url>.de/Collaboration/_packaging/.../npm/registry/
always-auth=true
The installation of renovate works and my registry get used for it. But renovate itself runs into a 401. How can I tell renovate to use the .npmrc generated from the `npmAuthenticate#0` task?
Error stack:
ERROR: Repository has unknown error (repository=...)
"err": {
"statusCode": 401,
"message": "Failed request: (401)",
"stack": "Error: Failed request: (401)\n at RestClient.<anonymous> (/root/.npm/_npx/05eeecd92f4e18e0/node_modules/typed-rest-client/RestClient.js:202:31)\n at Generator.next (<anonymous>)\n at fulfilled (/root/.npm/_npx/05eeecd92f4e18e0/node_modules/typed-rest-client/RestClient.js:6:58)\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5)"
}

The renovate command will modify the repo you defined in the config.js file(e.g. repositories: ['...']).
Since you are using the $(System.AccessToken) as authentication method, you need to grant Contribute permissions (e.g. Contribute,Contribute to pull requests,Create branch ) of target repo to the corresponding build service account.
Project Level Build Service Account Name: Your-project-name Build Service (your-collection-name)
Organization Level Build Service Account Name: Project Collection Build Service (your-collection-name)
You can navigate to Project Settings -> Repositories -> Target Repo -> Security and grant the Contribute permission to the two build service account.
For example:
For more detailed info, you can refer to this doc: Manage build service account permissions
On the other hand, if you need to update the repo from another project. You need to disable the option: Limit job authorization scope to current project for non-release pipelines in Project Settings -> Settings.

It seems that the official renovate docs for azure devops with a private feed isn't correct. This works for me:
Give the pipeline "Build User" contribute permissions on the feed:
Azure Devops Artifacts -> Settings -> Permissions -> Add the user/service that runs the pipeline with contributor.
azure-pipelines.yml
schedules:
- cron: '0 3 * * *'
displayName: 'Every day at 3am'
branches:
include: [main]
always: true
trigger: none
pool:
vmImage: ubuntu-latest
steps:
- task: npmAuthenticate#0
inputs:
workingFile: .npmrc
- bash: |
git config --global user.email 'bot#renovateapp.com'
git config --global user.name 'Renovate Bot'
npx --userconfig .npmrc renovate
env:
LOG_LEVEL: DEBUG
TOKEN: $(System.AccessToken)
RENOVATE_TOKEN: AZURE_DEVOPS_PAT_TOKEN_HERE
GITHUB_COM_TOKEN: REPLACEME
config.js
The important part here is to not use "pkgs.dev.azure.com" as the matchHost value, instead you can see in the debug logs if the feed is different on the 401'd requests, in my case it's "ORG_NAME_LOWERCASED.pkgs.visualstudio.com".
const repositories = require("./repositories");
// Security token used by the running build
const pipelineToken = process.env.TOKEN;
const patTokenForFeed = process.env.RENOVATE_TOKEN;
module.exports = {
platform: "azure",
endpoint: "https://dev.azure.com/ORG_NAME/",
token: pipelineToken,
hostRules: [
{
hostType: "npm",
matchHost: "ORG_NAME_LOWERCASED.pkgs.visualstudio.com",
username: "apikey",
password: patTokenForFeed,
},
],
repositories
};
.npmrc
registry=https://pkgs.dev.azure.com/ORG_NAME/PROJECT_NAME/_packaging/FEED_NAME/npm/registry/
always-auth=true

Related

Give Azure DevOps Pipelines access to publish gradle package to Azure DevOps Artifact feed

I am setting up a shared code library and have managed to build artifacts and publish them to a feed using a personal access token, but I want to move the publishing to an Azure Pipeline. I am trying to use the build account with the access token available at build time (System.AccessToken) but I get a 403 error from Azure Artifacts. The [Project] Build Service ([Org]) account is added as a contributor to the feed.
I have a build.gradle.kts file that has the following config (sensitive info replaced):
publishing {
publications {
create<MavenPublication>("private") {
groupId = "[groupId]"
artifactId = "shared"
version = "0.0.1"
artifact("./build/libs/lib.jar")
}
}
repositories {
maven(url = "https://pkgs.dev.azure.com/[Org]/[Project]/_packaging/maven-private/maven/v1") {
name = "maven-private"
credentials {
username = System.getenv("AZURE_DEVOPS_USER")
password = System.getenv("AZURE_DEVOPS_ACCESS_TOKEN")
}
}
}
}
And the Azure pipelines config:
trigger:
- main
pr: none
resources:
- repo: self
stages:
- stage: Build
displayName: Build stage
jobs:
- job: Build
displayName: Build
pool:
vmImage: 'ubuntu-latest'
steps:
- task: Gradle#2
inputs:
gradleWrapperFile: 'gradlew'
workingDirectory: '$(Build.SourcesDirectory)'
tasks: 'build'
javaHomeOption: 'JDKVersion'
- task: Gradle#2
env:
AZURE_DEVOPS_USER: "[Project] Build Service ([Org])"
AZURE_DEVOPS_ACCESS_TOKEN: $(System.AccessToken)
inputs:
gradleWrapperFile: 'gradlew'
workingDirectory: '$(Build.SourcesDirectory)'
tasks: 'publish'
javaHomeOption: 'JDKVersion'
The publish step fails with the following error:
Execution failed for task ':lib:publishPrivatePublicationToMaven-privateRepository'.
> Failed to publish publication 'private' to repository 'maven-private'
> Could not PUT 'https://pkgs.dev.azure.com/[Org]/[Project]/_packaging/maven-private/maven/v1/[groupId]/shared/0.0.1/shared-0.0.1.jar'. Received status code 403 from server: Forbidden - User '[userId]' lacks permission to complete this action. You need to have 'ReadPackages'.
I have no idea how to proceed with debugging, does anyone have any suggestions?
For the error ' You need to have 'ReadPackages'', you should enter your Azure Artifacts–>your feed–>feed settings–>Permissions。
Depending on your choice scope(organization scope or project scope),
You should click Add Users/groups and search for Project Collection Build Service ({OrgName}) and add as Contributor for organization-level scoped feed.
Or click Add Users/groups and search for {Project Name} Build Service ({Org Name}) and add as Contributor for project-level scoped feed.
Please see the details in the Configure permissions doc.
I hope this could do some help.

dir: cannot access in azure devops pipeline

Everytime I run my devops pipeline, I keep getting dir: cannot access "..." error messages. I have no idea why it happens.
Here is my pipeline code:
---
resources:
repositories:
- repository: testrepo
type: github
endpoint: testendp
name: testrepo/data
trigger:
- none
pool:
name: Hosted Ubuntu 1604
steps:
- checkout: testrepo
- script: dir $(Build.SourcesDirectory)
data=$(jq 'to_entries | map(select(.value.isdata=="true")) | from_entries' datafiles.json )
echo "$data"
But I am always getting the dir: cannot access "..." error.
Open project settings->Settings->check the option Limit job authorization scope to current project for non-release pipelines and Limit job authorization scope to referenced Azure DevOps repositories.
You can refer this doc for more details.

HTTP Error: 401, Request had invalid authentication credentials when deploying my webapp to firebase via bitbucket pipeline

I'm trying to deploy my webapp to firebase via a bitbucket pipeline and I'm encountering this error "HTTP Error: 401, Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential"
In my pipeline I've specified three variables: $KEY_FILE $FIREBASE_TOKEN $FIREBASE_PROJECT. The Keyfile is supposed to be a base64 encrypted json key. I used https://www.base64encode.org/ to encode it but I don't know if that works. I also have in the webapp a firebase json file which looks like this atm
{
"hosting": {
"public": "src",
"ignore": ["firebase.json", "**/.*", "**/node_modules/**"]
}
}
Maybe there is some authentication that needs to be present there? Any help is appreciated. I've been using google but haven't stumbled upon a solution yet.
Pipeline yaml:
image: node:10.15.3
pipelines:
default:
- parallel:
- step:
name: Build and Test
caches:
- node
script:
- npm install
- step:
name: Build
script:
- npm install && npm run build
artifacts:
- build/**
- step:
name: Deploy to Firebase
deployment: production
script:
- pipe: atlassian/firebase-deploy:0.2.1
variables:
KEY_FILE: $KEY_FILE
FIREBASE_TOKEN: $FIREBASE_TOKEN
PROJECT_ID: $FIREBASE_PROJECT

GitHub Actions: automatically push NuGet package

I'm trying to configure my Github repository in order to automatically have a NuGet package built and pushed to both nuget.org and github.com. So what I want is that each time a commit is made on the master branch, or another branch is merged into the master, github publishes a new Nuget package of the head of the master to both Nuget and Github.
NuGet
On my nuget organization account, I generated an access token (username - API keys - Create)
On Github (select your organization - View organization - Settings tab - Secrets) I added a secret with the name PUBLISH_TO_NUGET_ORG and my nuget access token
Github
On my personal account, I generated an access token (Account - Settings - Developer settings - Personal access tokens - generate)
On Github I added a secret with the name PUBLISH_TO_GITHUB_COM and my github access token
These are the scopes for my Github access token:
Setup
In my github repository I've setup an action to restore, build, test, pack and publish:
name: .NET Core
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Setup .NET Core
uses: actions/setup-dotnet#v1
with:
dotnet-version: 3.1.301
- name: Install dependencies
run: dotnet restore
- name: Build
run: dotnet build --configuration Release --no-restore
- name: Test
run: dotnet test --no-restore --verbosity normal
# - name: Publish
# uses: brandedoutcast/publish-nuget#v2.5.2
# with:
# PROJECT_FILE_PATH: MintPlayer.SeasonChecker/MintPlayer.SeasonChecker.csproj
# NUGET_KEY: ${{secrets.PUBLISH_TO_NUGET_ORG}}
# INCLUDE_SYMBOLS: true
- name: Pack
run: dotnet pack --no-build --configuration Release MintPlayer.SeasonChecker/MintPlayer.SeasonChecker.csproj --output .
- name: PushNuget
run: dotnet nuget push *.nupkg --source https://api.nuget.org/v3/index.json --api-key ${{secrets.PUBLISH_TO_NUGET_ORG}} --skip-duplicate
- name: AddGithubSource
run: dotnet nuget add source --username PieterjanDeClippel --password ${{secrets.PUBLISH_TO_GITHUB_COM}} --name github https://nuget.pkg.github.com/MintPlayer/index.json
- name: PushGithub
run: dotnet nuget push *.nupkg --source github --skip-duplicate
The push to nuget.org works fine, but the push to my GitHub feed fails with an Unauthorized error.
I've taken a look at some plugins like this one, and I want to embed this into my action in order not to build my project multiple times.
First take:
dotnet nuget push *.nupkg --source https://nuget.pkg.github.com/MintPlayer/index.json --api-key ${{secrets.PUBLISH_TO_GITHUB_COM}} --skip-duplicate
Result:
warn : Your request could not be authenticated by the GitHub Packages service. Please ensure your access token is valid and has the appropriate scopes configured.
Second take with multiple commands:
dotnet nuget add source --username PieterjanDeClippel --password ${{secrets.PUBLISH_TO_GITHUB_COM}} --name github https://nuget.pkg.github.com/MintPlayer/index.json
dotnet nuget push *.nupkg --source github --skip-duplicate
This one fails with the following (obvious) message:
error: Password encryption is not supported on .NET Core for this platform. The following feed try to use an encrypted password: 'github'. You can use a clear text password as a workaround.
error: Encryption is not supported on non-Windows platforms.
Does anyone have any experience with automated publishing of Nuget packages to Github?
Link to action configuration file
Edit
I tried sending a POST request:
Url: https://api.github.com/user
Authorization: Basic Auth
Username:
Password: <my-api-key>
And I'm getting my user information, so my access token definitely works.
Edit
I also tried running the command on my computer, replacing the token with my own and that as well does work.
Turns out I was missing a nuget.config file in my Solution
https://github.community/t/github-actions-automatically-push-nuget-package/128242/4
nuget.config
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<packageSources>
<add key="nuget.org" value="https://api.nuget.org/v3/index.json" protocolVersion="3" />
</packageSources>
</configuration>
And my workflow file:
name: .NET Core
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout#v2
- name: Setup .NET Core
uses: actions/setup-dotnet#v1.5.0
with:
dotnet-version: 3.1.301
# Authenticates packages to push to GPR
source-url: https://nuget.pkg.github.com/${{ github.repository_owner }}/index.json
env:
NUGET_AUTH_TOKEN: '%NUGET_AUTH_TOKEN%'
- name: Install dependencies
run: dotnet restore
- name: Build
run: dotnet build --configuration Release --no-restore
- name: Test
run: dotnet test --no-restore --verbosity normal
- name: Pack
run: dotnet pack --no-build --configuration Release
- name: PushNuget
run: dotnet nuget push **/*.nupkg --source https://api.nuget.org/v3/index.json --api-key ${{ secrets.PUBLISH_TO_NUGET_ORG }} --skip-duplicate
- name: PushGithub
# The github token is automatically being pulled from the workflow
run: dotnet nuget push **/*.nupkg --no-symbols --skip-duplicate
env:
NUGET_AUTH_TOKEN: ${{ github.token }}
Per the github actions docs
<packageSourceCredentials>
<github>
<add key="Username" value="USERNAME" />
<add key="ClearTextPassword" value="TOKEN" />
</github>
</packageSourceCredentials>
So I think you just need to set -StorePasswordInClearText in your nuget add source command as you are currently encrypting the token
References:
Github Actions - https://docs.github.com/en/packages/using-github-packages-with-your-projects-ecosystem/configuring-dotnet-cli-for-use-with-github-packages#authenticating-to-github-packages
Nuget Config Docs - https://learn.microsoft.com/en-us/nuget/reference/nuget-config-file#packagesourcecredentials

Jenkins pipeline does not deploy to azure

im trying to deploy azure web app to azure form git through jenkins pipeline
code looks like this
pipeline {
agent any
stages {
stage ('Checkout') {
steps {
checkout([$class: 'GitSCM', branches: [[name: '*/develop']], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: 'creds', url: 'https://xx.git']]])
}
}
stage ('Development - NuGet restore') {
steps {
bat """
C:\\nuget\\nuget.exe restore "%WORKSPACE%\\src\\xx.sln"
"""
}
}
stage ('Development - MSBuild') {
steps {
bat """
"C:\\Program Files\\dotnet\\dotnet.exe" msbuild "%WORKSPACE%\\src\\xx.sln" /p:VisualStudioVersion=15.0 /p:BuildInParallel=true /m:8 /p:Configuration=Release /p:DeployOnBuild=true /t:Clean,Build
"""
}
}
stage ('Development - Deploy') {
steps {
azureWebAppPublish appName: "xx",
azureCredentialsId: 'xx',
resourceGroup: "xx",
filePath: 'xx'
}
}
}
post {
failure {
xxx....;
}
}
}
Output form azure deployment plugin is:
Starting Azure Web App Deployment
Cloning repository https://xx.scm.azurewebsites.net:443/gitfile.git
c:\Program Files\Git\cmd\git.exe init C:\Program Files (x86)\Jenkins\workspace\xx.azure-deploy # timeout=10
Fetching upstream changes from https://xx.scm.azurewebsites.net:443/gitfile.git
c:\Program Files\Git\cmd\git.exe --version # timeout=10
using GIT_ASKPASS to set credentials
c:\Program Files\Git\cmd\git.exe fetch --tags --progress https://xx.scm.azurewebsites.net:443/gitfile.git +refs/heads/:refs/remotes/origin/ # timeout=10
c:\Program Files\Git\cmd\git.exe config remote.origin.url https://xx.scm.azurewebsites.net:443/gitfile.git # timeout=10
c:\Program Files\Git\cmd\git.exe config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10
Seen 0 remote branches
c:\Program Files\Git\cmd\git.exe add -A # timeout=10
Deploy repository is up-to-date. Nothing to commit.
Done Azure Web App deployment.
Plugin is trying to fetch changes form actual azure webapp url which is obviously wrong, using correct git file name.
How is this possible? Is there any way to supply git repo url as a parameter to azure plugin?
Thanks!
The comments part from this Azure Function Plugin link talks regarding similar issue (i.e., regarding issue of 'Deploy repository is up-to-date. Nothing to commit.') and the comment provided is to explicitly have a process to make sure the changed files are included in the 'Files' list. For more information and comparison with Azure App Service plugin, please refer it. Hope this helps!!

Resources