sharing NPM script/app through azure devops organizations using azure artifacts - node.js

I have a git repository containing a node.js "app" (only 1 file) with its package.json.
I need to use that "app" from azure devops pipeline of other organizations, so I made a pipeline in that nodejs project that makes a versioning+publish of that script on a NPM based azure artifact feed.
On another organization, in the pipeline where I wanted to use that app, I used an "npm" task configured like that :
steps:
- task: Npm#1
displayName: 'npm install MyPackage'
inputs:
command: custom
verbose: false
customCommand: 'install MyPackage'
customRegistry: useFeed
customFeed: '...some-guid....'
The installs seems ok (I created a feed in the 2nd organization with the feed of 1st organization as upstream source as indicated in the doc).
Now, how am I supposed to exec that script in the next pipeline task ?
I tried to connect to the agent, but when I run "MyPackage" command, I get "command not found".
Where is the script installed ? How am I supposed to execute it ?
Is there something to put in the PATH ?
Should I use the "-g" option ?
Thanks for your help :)
Updating with the requested logs :
Task log :
Starting: npm custom
==============================================================================
Task : npm
Description : Install and publish npm packages, or run an npm command. Supports npmjs.com and authenticated registries like Azure Artifacts.
Version : 1.182.0
Author : Microsoft Corporation
Help : https://learn.microsoft.com/azure/devops/pipelines/tasks/package/npm
==============================================================================
/usr/local/bin/npm --version
6.13.7
/usr/local/bin/npm config list
; cli configs
metrics-registry = "https://pkgs.dev.azure.com/org/_packaging/....some-guid..../npm/registry/"
scope = ""
user-agent = "npm/6.13.7 node/v13.11.0 darwin x64"
; environment configs
userconfig = "/Users/.../AGENTS/vsts-agent-osx-x64-2.179.0-AZURE-01/_work/10/npm/33366.npmrc"
; userconfig /Users/.../AGENTS/vsts-agent-osx-x64-2.179.0-AZURE-01/_work/10/npm/33366.npmrc
registry = "https://pkgs.dev.azure.com/org/_packaging/....some-guid..../npm/registry/"
; builtin config undefined
prefix = "/usr/local"
; node bin location = /usr/local/Cellar/node/13.11.0/bin/node
; cwd = /Users/.../AGENTS/vsts-agent-osx-x64-2.179.0-AZURE-01/_work/10/s
; HOME = /Users/...
; "npm config ls -l" to show all defaults.
/usr/local/bin/npm install -g MyPackage
+ MyPackage#1.0.20210216-1
added 198 packages from 99 contributors in 18.319s
Finishing: npm custom
Here is the 2nd definition :
steps:
- script: |
echo "1"
node MyPackage
echo "2"
MyPackage
displayName: 'Command Line Script'
And the associated execution log :
Starting: Command Line Script
==============================================================================
Task : Command line
Description : Run a command line script using Bash on Linux and macOS and cmd.exe on Windows
Version : 2.182.0
Author : Microsoft Corporation
Help : https://learn.microsoft.com/azure/devops/pipelines/tasks/utility/command-line
==============================================================================
Generating script.
========================== Starting Command Output ===========================
/bin/bash --noprofile --norc /Users/.../AGENTS/vsts-agent-osx-x64-2.179.0-AZURE-01/_work/_temp/7fe7833a-ce7d-48af-ab8f-1fcf2e740c36.sh
1
internal/modules/cjs/loader.js:979
throw err;
^
Error: Cannot find module '/Users/.../AGENTS/vsts-agent-osx-x64-2.179.0-AZURE-01/_work/10/s/MyPackage'
at Function.Module._resolveFilename (internal/modules/cjs/loader.js:976:15)
at Function.Module._load (internal/modules/cjs/loader.js:859:27)
at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
at internal/main/run_main_module.js:17:47 {
code: 'MODULE_NOT_FOUND',
requireStack: []
}
2
/Users/.../AGENTS/vsts-agent-osx-x64-2.179.0-AZURE-01/_work/_temp/7fe7833a-ce7d-48af-ab8f-1fcf2e740c36.sh: line 5: MyPackage: command not found
##[error]Bash exited with code '127'.
Finishing: Command Line Script

We could add an Azure Artifacts feed in a different organization within your Azure AD tenant as an upstream source.
For example, you create a feed in the first organization, and add npm package to the feed, now you want to use the package in the second organization, we need to set the upstream source in the second organization feed, such as azure-feed://myOrg/myProject/myFeed#local, then we could install and use the package in the second organization pipeline.
You could also check this blog for more details.
Update1
I need to find the path where it is installed
We could add task Command line and run npm list to see the installed non-global libraries for your current location. Check this ticket and blog for more details.
The install path is $(Build.SourcesDirectory)/node_modules, we could add task bash and run ls '$(Build.SourcesDirectory)/node_modules' to check it.

Finally found the issues :
1st I had to install the package globally with the -g option
2nd I had to use npx to
run it.
3rd I had to update the package.json with :
"bin": {
"NAME_OF_THE_BINARY": "./FILE_TO_EXECUTE.js"
},

Related

An unhandled exception occurred: Cannot find module './features/colr-v1'

I am building an Angular 8.5.5 project in Azure DevOps, which deployed successfully in the past. Recently I tried publishing it again and now getting
An unhandled exception occurred: Cannot find module './features/colr-v1'
Npm failed with return code: 127
Except, of course, there is no features/colr module (I even tried adding it, but that made no difference)
It fails on the step "Prod Build" but this step works fine locally. Any clues?
run ng build --prod
So far I've tried the suggestions found here: How to resolve Azure Pipeline Error: Npm failed with return code: 1, npm install task. SyntaxError: Unexpected end of JSON input while parsing near
(delete packages.lock, forced the cache clear locally, dropped the modules folder)
works locally, but dies on the build server...
2022-05-26T21:23:43.4078903Z ##[section]Starting: Prod build
2022-05-26T21:23:43.4200582Z ==============================================================================
2022-05-26T21:23:43.4200910Z Task : npm
2022-05-26T21:23:43.4201263Z Description : Install and publish npm packages, or run an npm command. Supports npmjs.com and authenticated registries like Azure Artifacts.
2022-05-26T21:23:43.4201635Z Version : 1.202.0
2022-05-26T21:23:43.4201841Z Author : Microsoft Corporation
2022-05-26T21:23:43.4202130Z Help : https://learn.microsoft.com/azure/devops/pipelines/tasks/package/npm
2022-05-26T21:23:43.4202490Z ==============================================================================
2022-05-26T21:23:44.4762030Z [command]C:\Windows\system32\cmd.exe /D /S /C "C:\hostedtoolcache\windows\node\16.15.0\x64\npm.cmd --version"
2022-05-26T21:23:45.5190169Z 8.5.5
2022-05-26T21:23:47.5735442Z [command]C:\Windows\system32\cmd.exe /D /S /C "C:\hostedtoolcache\windows\node\16.15.0\x64\npm.cmd config list"
2022-05-26T21:23:48.5878213Z ; "global" config from C:\npm\prefix\etc\npmrc
2022-05-26T21:23:48.5878806Z
2022-05-26T21:23:48.5879227Z cache = "C:\\npm\\cache"
2022-05-26T21:23:48.5879455Z
2022-05-26T21:23:48.5879831Z ; "env" config from environment
2022-05-26T21:23:48.5880055Z
2022-05-26T21:23:48.5880412Z prefix = "C:\\npm\\prefix"
2022-05-26T21:23:48.5880833Z userconfig = "D:\\a\\1\\npm\\140.npmrc"
2022-05-26T21:23:48.5881068Z
2022-05-26T21:23:48.5881511Z ; node bin location = C:\hostedtoolcache\windows\node\16.15.0\x64\node.exe
2022-05-26T21:23:48.5881972Z ; cwd = D:\a\1\s
2022-05-26T21:23:48.5882348Z ; HOME = C:\Users\VssAdministrator
2022-05-26T21:23:48.5882787Z ; Run `npm config ls -l` to show all defaults.
2022-05-26T21:23:48.5886667Z [command]C:\Windows\system32\cmd.exe /D /S /C "C:\hostedtoolcache\windows\node\16.15.0\x64\npm.cmd run ng build --prod"
2022-05-26T21:23:52.2011491Z An unhandled exception occurred: Cannot find module './features/colr-v1'
2022-05-26T21:23:52.2011799Z
2022-05-26T21:23:52.2012498Z Require stack:
2022-05-26T21:23:52.2012829Z > ms-identity-angular-c7s1#1.0.0 ng
2022-05-26T21:23:52.2013380Z - D:\a\1\s\node_modules\caniuse-lite\data\features.js
2022-05-26T21:23:52.2013697Z > ng "build"
2022-05-26T21:23:52.2014190Z - D:\a\1\s\node_modules\caniuse-lite\dist\unpacker\features.js
2022-05-26T21:23:52.2014375Z
2022-05-26T21:23:52.2014980Z - D:\a\1\s\node_modules\caniuse-lite\dist\unpacker\index.js
2022-05-26T21:23:52.2016272Z - D:\a\1\s\node_modules\#angular-devkit\build-angular\src\utils\build-browser-features.js
2022-05-26T21:23:52.2017336Z - D:\a\1\s\node_modules\#angular-devkit\build-angular\src\utils\index.js
2022-05-26T21:23:52.2018363Z - D:\a\1\s\node_modules\#angular-devkit\build-angular\src\browser\index.js
2022-05-26T21:23:52.2019402Z - D:\a\1\s\node_modules\#angular-devkit\architect\node\node-modules-architect-host.js
2022-05-26T21:23:52.2020422Z - D:\a\1\s\node_modules\#angular-devkit\architect\node\index.js
2022-05-26T21:23:52.2021382Z - D:\a\1\s\node_modules\#angular\cli\models\architect-command.js
2022-05-26T21:23:52.2022552Z - D:\a\1\s\node_modules\#angular\cli\commands\build-impl.js
2022-05-26T21:23:52.2023515Z - D:\a\1\s\node_modules\#angular-devkit\schematics\tools\export-ref.js
2022-05-26T21:23:52.2024504Z - D:\a\1\s\node_modules\#angular-devkit\schematics\tools\index.js
2022-05-26T21:23:52.2025476Z - D:\a\1\s\node_modules\#angular\cli\utilities\json-schema.js
2022-05-26T21:23:52.2026423Z - D:\a\1\s\node_modules\#angular\cli\models\command-runner.js
2022-05-26T21:23:52.2027362Z - D:\a\1\s\node_modules\#angular\cli\lib\cli\index.js
2022-05-26T21:23:52.2028297Z - D:\a\1\s\node_modules\#angular\cli\lib\init.js
2022-05-26T21:23:52.2029191Z - D:\a\1\s\node_modules\#angular\cli\bin\ng
2022-05-26T21:23:52.2030184Z See "C:\Users\VSSADM~1\AppData\Local\Temp\ng-2wWjsm\angular-errors.log" for further details.
2022-05-26T21:23:52.2445104Z ##[warning]Couldn't find a debug log in the cache or working directory
2022-05-26T21:23:52.2457516Z ##[error]Error: Npm failed with return code: 127
2022-05-26T21:23:52.2468571Z ##[section]Finishing: Prod b
The solution was arrived at by completely blowing away my local environment and pulling the code from the repo anew. I was then able to see the errors in the dependency chain. Hope this helps someone
this worked for now: I'll have to update some libraries later.
npm install --legacy-peer-deps
I think you have to install colr-v1 module using npm install

Gitlab job failure: Cannot overwrite variable Host because it is read-only or constant

Hi people of the internet.
Basically I am unable to run even the simplest job and I keep getting the same error no matter what I put in the .gitlab-ci.yml file. See example below:
Here is the .gitlab-ci.yml file:
stages:
- test
job1:
stage: test
tags:
- testing
script:
- echo "Hello world!"
Here is the output ("?" corresponds to intentionally blacked out information):
Running with gitlab-runner 14.10.0 (c6bb62f6)
on runner_test ????????
Preparing the "shell" executor
00:00
Using Shell executor...
Preparing environment
00:00
Running on LAPTOP-????????...
Getting source from Git repository
00:01
WriteError:
Line |
219 | $HOST="[MASKED]"
| ~~~~~~~~~~~~~~~~~~~~~~
| Cannot overwrite variable Host because it is read-only or constant.
ERROR: Job failed: exit status 1
I know that $HOST is a reserved variable in powershell but I don't see the link between the error and the code. It may have something to do with the configuration of the runner on Windows. Has anyone encountered this error on Gitlab before? Or any suggestions on how to debug?
Here are the steps that I took to install the runner on Gitlab for Windows (see https://docs.gitlab.com/runner/install/windows.html):
Create a folder somewhere in the system: C:\GitLab-Runner.
Download the binary for 64-bit and put it into the folder (see https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-windows-amd64.exe).
Run prompt as an administrator
Run the following command:
cd C:\GitLab-Runner
gitlab-runner.exe register
Enter your GitLab instance URL (see Gitlab > Settings > CI/CD > Runners > Specific runners)
Enter the token to register the runner (see Gitlab > Settings > CI/CD > Runners > Specific runners)
Enter a description for the runner: runner_test for instance
Enter the tags associated with the runner, separated by commas: testing, windows for instance
Provide the runner executor: shell
Install GitLab Runner as a service and start it
cd C:\GitLab-Runner
gitlab-runner.exe install
gitlab-runner.exe start
I also had to install the latest version of pwsh in Windows (see gitlab-runner: prepare environment failed to start process pwsh in windows):
Run prompt as an administrator
Install the newer pwsh.exe:
winget install Microsoft.PowerShell
Restart the runner
cd C:\GitLab-Runner
gitlab-runner.exe restart
This issue was due to my choice of shell for some reason. A Gitlab runner can choose a shell among the following: bash, sh, powershell, pwsh, and cmd (the last one being deprecated now).
As I stated above I had been using pwsh. So, I went after the config.toml file inside of the C:\GitLab-Runner directory to manually make the change from pwsh to powershell.
...
[[runners]]
name = "runner_test"
executor = "shell"
shell = "powershell"
...
I then restarted the runner and got the job to complete properly:
cd C:\GitLab-Runner
gitlab-runner restart
I still get the error (more like a warning now) but it does not prevent the job from finishing anymore. If anyone has a better answer with a proper explanation I would gladly accept it as the answer to this question.
Note that pwsh to powershell are both powershell scripts (see https://docs.gitlab.com/runner/shells/index.html):
powershell Fully Supported PowerShell script. All commands are executed in PowerShell Desktop context. In GitLab Runner 12.0-13.12, this is the default when registering a new runner.
pwsh Fully Supported PowerShell script. All commands are executed in PowerShell Core context. In GitLab Runner 14.0 and later, this is the default when registering a new runner.

Selfhosted Azure DevOps Agent as NetworkService npm Install problem

we have a selfhosted Azure DevOps Server with selfhosted Build Agents. The Agents are installed as a Windows Service with user NetworkService.
Our project contains a .NET Application with public and private (selfhosted Repository) NuGet Packages and an angular WebApplication.
For the NuGet Repositories we configured global environment variables HTTP_PROXY and HTTPS_PROXY. The .NET part works fine.
The npm install step does not work with the error message:
npm ERR! Cannot read property 'startsWith' of null
Seems like npm does not like the proxy settings in environment variables and no proxy settings in .npmrc file.
My question is, how can i configure npm to use proxy settings? How can i configure .npmrc file for user NetworkService?
The build log contains following messages:
metrics-registry = "https://registry.npmjs.org/" scope = "" user-agent
= "npm/6.14.13 node/v14.17.0 win32 x64"
; environment configs userconfig =
"E:\Agent_02\_work\3\npm\15146.npmrc"
; builtin config undefined prefix =
"C:\WINDOWS\ServiceProfiles\NetworkService\AppData\Roaming\npm"
; node bin location = E:\Program Files\nodejs\node.exe ; cwd =
E:\Agent_02_work\3\s\Merida.UI.Web ; HOME =
C:\WINDOWS\ServiceProfiles\NetworkService ; "npm config ls -l" to show
all defaults.
[command]C:\WINDOWS\system32\cmd.exe /D /S /C ""E:\Program
Files\nodejs\npm.cmd" install" npm ERR! Cannot read property
'startsWith' of null
As you can see the build takes a .npmrc file from path: E:\\Agent_02\\_work\\3\\npm\\15146.npmrc. Each build the file path is different.
How can i solve this problem?
Thanks in advance
I finally found out how to set global config values for user NetworkService.
I put my proxy settings and private registry config values in the following file:
C:\WINDOWS\ServiceProfiles\NetworkService\AppData\Roaming\npm\etc\npmrc
Now the build succeeds and the npm packages are installed without errors.

Unable to add web part to catalog pipeline azure - access denied

I have created a pipeline to bundle and add a webpart to the app catalog. I am getting this access error below. Any ideas?
2019-10-17T16:28:44.6630984Z ##[section]Starting: add package to catalog
2019-10-17T16:28:44.6740356Z ==============================================================================
2019-10-17T16:28:44.6740433Z Task : Command line
2019-10-17T16:28:44.6740486Z Description : Run a command line script using Bash on Linux and macOS and cmd.exe on Windows
2019-10-17T16:28:44.6740546Z Version : 2.151.2
2019-10-17T16:28:44.6740587Z Author : Microsoft Corporation
2019-10-17T16:28:44.6740630Z Help : https://learn.microsoft.com/azure/devops/pipelines/tasks/utility/command-line
2019-10-17T16:28:44.6740689Z ==============================================================================
2019-10-17T16:28:45.4965469Z Generating script.
2019-10-17T16:28:45.5061921Z Script contents:
2019-10-17T16:28:45.5070670Z o365 spo app add -p d:\a\r1\a/_ABFI-CI/drop/sharepoint/solution/json-feed.sppkg --overwrite
2019-10-17T16:28:45.5392405Z ========================== Starting Command Output ===========================
2019-10-17T16:28:45.5641516Z ##[command]"C:\windows\system32\cmd.exe" /D /E:ON /V:OFF /S /C "CALL "d:\a\_temp\6ca1b36f-40ef-444b-ae13-9e5ba3ba5d98.cmd""
2019-10-17T16:28:49.1097084Z Error: Access denied. You do not have permission to perform this action or access t**strong text**his resource.
2019-10-17T16:28:49.2305193Z ##[error]Cmd.exe exited with code '1'.
2019-10-17T16:28:49.2676400Z ##[section]Finishing: add package to catalog
I just needed to be added as an admin to the site!

Using Bower in a VM with puppet to run python-nvd3

I'm trying to set up a VM to use python-nvd3 and using puppet to provision the machine. I've got node.js/npm to go in ok but then I get an error trying to install packages using bower:
package { 'bower':
provider => npm
}
exec { 'install d3':
command => '/usr/local/bin/bower install d3#3.3.8',
require => Package["bower"]
}
exec { 'install nvd3':
command => '/usr/local/bin/bower install nvd3#1.1.12-beta',
require => Package["bower"]
}
err: /Stage[main]/Infinite_interns::Box::Tm351d001r/Exec[install d3]/returns: change from notrun to 0 failed: /usr/local/bin/bower install d3#3.3.8 returned 1 instead of one of [0] at /vagrant/modules/infinite_interns/manifests/box/tm351d001r.pp:39
If I ssh in to the VM, and try to run a bower command I get a y/n prompt to ask if I want to allow error logging. In this case I don't want to and I don't want my provisioner to fall over because it can't answer the question (if this is the problem that's causing the above puppet error).
You have to set the CI environment variable to true or use the config.interactive=false bower flag to avoid interactive operations. Example:
exec { 'install d3':
command => '/usr/local/bin/bower install d3#3.3.8 --config.interactive=false',
require => Package["bower"]
}
Reference:
https://github.com/bower/bower#running-on-a-continuous-integration-server

Resources