we have a selfhosted Azure DevOps Server with selfhosted Build Agents. The Agents are installed as a Windows Service with user NetworkService.
Our project contains a .NET Application with public and private (selfhosted Repository) NuGet Packages and an angular WebApplication.
For the NuGet Repositories we configured global environment variables HTTP_PROXY and HTTPS_PROXY. The .NET part works fine.
The npm install step does not work with the error message:
npm ERR! Cannot read property 'startsWith' of null
Seems like npm does not like the proxy settings in environment variables and no proxy settings in .npmrc file.
My question is, how can i configure npm to use proxy settings? How can i configure .npmrc file for user NetworkService?
The build log contains following messages:
metrics-registry = "https://registry.npmjs.org/" scope = "" user-agent
= "npm/6.14.13 node/v14.17.0 win32 x64"
; environment configs userconfig =
"E:\Agent_02\_work\3\npm\15146.npmrc"
; builtin config undefined prefix =
"C:\WINDOWS\ServiceProfiles\NetworkService\AppData\Roaming\npm"
; node bin location = E:\Program Files\nodejs\node.exe ; cwd =
E:\Agent_02_work\3\s\Merida.UI.Web ; HOME =
C:\WINDOWS\ServiceProfiles\NetworkService ; "npm config ls -l" to show
all defaults.
[command]C:\WINDOWS\system32\cmd.exe /D /S /C ""E:\Program
Files\nodejs\npm.cmd" install" npm ERR! Cannot read property
'startsWith' of null
As you can see the build takes a .npmrc file from path: E:\\Agent_02\\_work\\3\\npm\\15146.npmrc. Each build the file path is different.
How can i solve this problem?
Thanks in advance
I finally found out how to set global config values for user NetworkService.
I put my proxy settings and private registry config values in the following file:
C:\WINDOWS\ServiceProfiles\NetworkService\AppData\Roaming\npm\etc\npmrc
Now the build succeeds and the npm packages are installed without errors.
Related
I am building an Angular 8.5.5 project in Azure DevOps, which deployed successfully in the past. Recently I tried publishing it again and now getting
An unhandled exception occurred: Cannot find module './features/colr-v1'
Npm failed with return code: 127
Except, of course, there is no features/colr module (I even tried adding it, but that made no difference)
It fails on the step "Prod Build" but this step works fine locally. Any clues?
run ng build --prod
So far I've tried the suggestions found here: How to resolve Azure Pipeline Error: Npm failed with return code: 1, npm install task. SyntaxError: Unexpected end of JSON input while parsing near
(delete packages.lock, forced the cache clear locally, dropped the modules folder)
works locally, but dies on the build server...
2022-05-26T21:23:43.4078903Z ##[section]Starting: Prod build
2022-05-26T21:23:43.4200582Z ==============================================================================
2022-05-26T21:23:43.4200910Z Task : npm
2022-05-26T21:23:43.4201263Z Description : Install and publish npm packages, or run an npm command. Supports npmjs.com and authenticated registries like Azure Artifacts.
2022-05-26T21:23:43.4201635Z Version : 1.202.0
2022-05-26T21:23:43.4201841Z Author : Microsoft Corporation
2022-05-26T21:23:43.4202130Z Help : https://learn.microsoft.com/azure/devops/pipelines/tasks/package/npm
2022-05-26T21:23:43.4202490Z ==============================================================================
2022-05-26T21:23:44.4762030Z [command]C:\Windows\system32\cmd.exe /D /S /C "C:\hostedtoolcache\windows\node\16.15.0\x64\npm.cmd --version"
2022-05-26T21:23:45.5190169Z 8.5.5
2022-05-26T21:23:47.5735442Z [command]C:\Windows\system32\cmd.exe /D /S /C "C:\hostedtoolcache\windows\node\16.15.0\x64\npm.cmd config list"
2022-05-26T21:23:48.5878213Z ; "global" config from C:\npm\prefix\etc\npmrc
2022-05-26T21:23:48.5878806Z
2022-05-26T21:23:48.5879227Z cache = "C:\\npm\\cache"
2022-05-26T21:23:48.5879455Z
2022-05-26T21:23:48.5879831Z ; "env" config from environment
2022-05-26T21:23:48.5880055Z
2022-05-26T21:23:48.5880412Z prefix = "C:\\npm\\prefix"
2022-05-26T21:23:48.5880833Z userconfig = "D:\\a\\1\\npm\\140.npmrc"
2022-05-26T21:23:48.5881068Z
2022-05-26T21:23:48.5881511Z ; node bin location = C:\hostedtoolcache\windows\node\16.15.0\x64\node.exe
2022-05-26T21:23:48.5881972Z ; cwd = D:\a\1\s
2022-05-26T21:23:48.5882348Z ; HOME = C:\Users\VssAdministrator
2022-05-26T21:23:48.5882787Z ; Run `npm config ls -l` to show all defaults.
2022-05-26T21:23:48.5886667Z [command]C:\Windows\system32\cmd.exe /D /S /C "C:\hostedtoolcache\windows\node\16.15.0\x64\npm.cmd run ng build --prod"
2022-05-26T21:23:52.2011491Z An unhandled exception occurred: Cannot find module './features/colr-v1'
2022-05-26T21:23:52.2011799Z
2022-05-26T21:23:52.2012498Z Require stack:
2022-05-26T21:23:52.2012829Z > ms-identity-angular-c7s1#1.0.0 ng
2022-05-26T21:23:52.2013380Z - D:\a\1\s\node_modules\caniuse-lite\data\features.js
2022-05-26T21:23:52.2013697Z > ng "build"
2022-05-26T21:23:52.2014190Z - D:\a\1\s\node_modules\caniuse-lite\dist\unpacker\features.js
2022-05-26T21:23:52.2014375Z
2022-05-26T21:23:52.2014980Z - D:\a\1\s\node_modules\caniuse-lite\dist\unpacker\index.js
2022-05-26T21:23:52.2016272Z - D:\a\1\s\node_modules\#angular-devkit\build-angular\src\utils\build-browser-features.js
2022-05-26T21:23:52.2017336Z - D:\a\1\s\node_modules\#angular-devkit\build-angular\src\utils\index.js
2022-05-26T21:23:52.2018363Z - D:\a\1\s\node_modules\#angular-devkit\build-angular\src\browser\index.js
2022-05-26T21:23:52.2019402Z - D:\a\1\s\node_modules\#angular-devkit\architect\node\node-modules-architect-host.js
2022-05-26T21:23:52.2020422Z - D:\a\1\s\node_modules\#angular-devkit\architect\node\index.js
2022-05-26T21:23:52.2021382Z - D:\a\1\s\node_modules\#angular\cli\models\architect-command.js
2022-05-26T21:23:52.2022552Z - D:\a\1\s\node_modules\#angular\cli\commands\build-impl.js
2022-05-26T21:23:52.2023515Z - D:\a\1\s\node_modules\#angular-devkit\schematics\tools\export-ref.js
2022-05-26T21:23:52.2024504Z - D:\a\1\s\node_modules\#angular-devkit\schematics\tools\index.js
2022-05-26T21:23:52.2025476Z - D:\a\1\s\node_modules\#angular\cli\utilities\json-schema.js
2022-05-26T21:23:52.2026423Z - D:\a\1\s\node_modules\#angular\cli\models\command-runner.js
2022-05-26T21:23:52.2027362Z - D:\a\1\s\node_modules\#angular\cli\lib\cli\index.js
2022-05-26T21:23:52.2028297Z - D:\a\1\s\node_modules\#angular\cli\lib\init.js
2022-05-26T21:23:52.2029191Z - D:\a\1\s\node_modules\#angular\cli\bin\ng
2022-05-26T21:23:52.2030184Z See "C:\Users\VSSADM~1\AppData\Local\Temp\ng-2wWjsm\angular-errors.log" for further details.
2022-05-26T21:23:52.2445104Z ##[warning]Couldn't find a debug log in the cache or working directory
2022-05-26T21:23:52.2457516Z ##[error]Error: Npm failed with return code: 127
2022-05-26T21:23:52.2468571Z ##[section]Finishing: Prod b
The solution was arrived at by completely blowing away my local environment and pulling the code from the repo anew. I was then able to see the errors in the dependency chain. Hope this helps someone
this worked for now: I'll have to update some libraries later.
npm install --legacy-peer-deps
I think you have to install colr-v1 module using npm install
I have a git repository containing a node.js "app" (only 1 file) with its package.json.
I need to use that "app" from azure devops pipeline of other organizations, so I made a pipeline in that nodejs project that makes a versioning+publish of that script on a NPM based azure artifact feed.
On another organization, in the pipeline where I wanted to use that app, I used an "npm" task configured like that :
steps:
- task: Npm#1
displayName: 'npm install MyPackage'
inputs:
command: custom
verbose: false
customCommand: 'install MyPackage'
customRegistry: useFeed
customFeed: '...some-guid....'
The installs seems ok (I created a feed in the 2nd organization with the feed of 1st organization as upstream source as indicated in the doc).
Now, how am I supposed to exec that script in the next pipeline task ?
I tried to connect to the agent, but when I run "MyPackage" command, I get "command not found".
Where is the script installed ? How am I supposed to execute it ?
Is there something to put in the PATH ?
Should I use the "-g" option ?
Thanks for your help :)
Updating with the requested logs :
Task log :
Starting: npm custom
==============================================================================
Task : npm
Description : Install and publish npm packages, or run an npm command. Supports npmjs.com and authenticated registries like Azure Artifacts.
Version : 1.182.0
Author : Microsoft Corporation
Help : https://learn.microsoft.com/azure/devops/pipelines/tasks/package/npm
==============================================================================
/usr/local/bin/npm --version
6.13.7
/usr/local/bin/npm config list
; cli configs
metrics-registry = "https://pkgs.dev.azure.com/org/_packaging/....some-guid..../npm/registry/"
scope = ""
user-agent = "npm/6.13.7 node/v13.11.0 darwin x64"
; environment configs
userconfig = "/Users/.../AGENTS/vsts-agent-osx-x64-2.179.0-AZURE-01/_work/10/npm/33366.npmrc"
; userconfig /Users/.../AGENTS/vsts-agent-osx-x64-2.179.0-AZURE-01/_work/10/npm/33366.npmrc
registry = "https://pkgs.dev.azure.com/org/_packaging/....some-guid..../npm/registry/"
; builtin config undefined
prefix = "/usr/local"
; node bin location = /usr/local/Cellar/node/13.11.0/bin/node
; cwd = /Users/.../AGENTS/vsts-agent-osx-x64-2.179.0-AZURE-01/_work/10/s
; HOME = /Users/...
; "npm config ls -l" to show all defaults.
/usr/local/bin/npm install -g MyPackage
+ MyPackage#1.0.20210216-1
added 198 packages from 99 contributors in 18.319s
Finishing: npm custom
Here is the 2nd definition :
steps:
- script: |
echo "1"
node MyPackage
echo "2"
MyPackage
displayName: 'Command Line Script'
And the associated execution log :
Starting: Command Line Script
==============================================================================
Task : Command line
Description : Run a command line script using Bash on Linux and macOS and cmd.exe on Windows
Version : 2.182.0
Author : Microsoft Corporation
Help : https://learn.microsoft.com/azure/devops/pipelines/tasks/utility/command-line
==============================================================================
Generating script.
========================== Starting Command Output ===========================
/bin/bash --noprofile --norc /Users/.../AGENTS/vsts-agent-osx-x64-2.179.0-AZURE-01/_work/_temp/7fe7833a-ce7d-48af-ab8f-1fcf2e740c36.sh
1
internal/modules/cjs/loader.js:979
throw err;
^
Error: Cannot find module '/Users/.../AGENTS/vsts-agent-osx-x64-2.179.0-AZURE-01/_work/10/s/MyPackage'
at Function.Module._resolveFilename (internal/modules/cjs/loader.js:976:15)
at Function.Module._load (internal/modules/cjs/loader.js:859:27)
at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:71:12)
at internal/main/run_main_module.js:17:47 {
code: 'MODULE_NOT_FOUND',
requireStack: []
}
2
/Users/.../AGENTS/vsts-agent-osx-x64-2.179.0-AZURE-01/_work/_temp/7fe7833a-ce7d-48af-ab8f-1fcf2e740c36.sh: line 5: MyPackage: command not found
##[error]Bash exited with code '127'.
Finishing: Command Line Script
We could add an Azure Artifacts feed in a different organization within your Azure AD tenant as an upstream source.
For example, you create a feed in the first organization, and add npm package to the feed, now you want to use the package in the second organization, we need to set the upstream source in the second organization feed, such as azure-feed://myOrg/myProject/myFeed#local, then we could install and use the package in the second organization pipeline.
You could also check this blog for more details.
Update1
I need to find the path where it is installed
We could add task Command line and run npm list to see the installed non-global libraries for your current location. Check this ticket and blog for more details.
The install path is $(Build.SourcesDirectory)/node_modules, we could add task bash and run ls '$(Build.SourcesDirectory)/node_modules' to check it.
Finally found the issues :
1st I had to install the package globally with the -g option
2nd I had to use npx to
run it.
3rd I had to update the package.json with :
"bin": {
"NAME_OF_THE_BINARY": "./FILE_TO_EXECUTE.js"
},
I would like to fetch private npm from private repository using scopes.
I have set the .npmrc as such.
registry=https://registry.npmjs.org/
#myco:registry=https://nexus.myco.com/nexus/repository/
I have set the .yarnrc as such.
registry "https://registry.npmjs.org/"
"#myco:registry" "https://nexus.myco.com/nexus/repository/"
But when I do:
yarn --verbose add #myco/some-private-npm
It throws this error:
verbose 0.708 Performing "GET" request to "https://nexus.myco.com/nexus/repository/#myco%2fsome-private-npm".
verbose 0.792 Request "https://nexus.myco.com/nexus/repository/#myco%2fsome-private-npm" finished with status code 404.
When I do:
yarn --verbose add #myco:some-private-npm
It goes to this 400 url (Nexus: Invalid repository path):
verbose 0.957 Request "https://nexus.myco.com/nexus/repository/#myco:some-private-npm" finished with status code 400.
The actual npm is located at:
https://nexus.myco.com/nexus/repository/myco/some-private-npm
How do I make sure the url fetched does not have "#" and "%2f" is a "/"?
Thanks!
Per this github issue for yarn, try adding similar config to your .yarnrc:
registry "https://registry.npmjs.org/"
"#myco:registry" "https://nexus.myco.com/nexus/repository/"
Failing that, I recommend perusing the linked issue and trying the solutions provided.
I run into exact the same issue some time ago.
check at NPM if your repository team access level is read/write.
On my side the team i was in had only read access level. Switching solved the issue.
Another possible scenario is that the slash encoding isn't really the problem. You simply do not have access to that specific repository.
That would explain the 404, since if you don't have access, for security reasons, it's as if it doesn't exist, you don't get a 401.
I found this page when also trying to run npm publish of a .tgz file in Github Actions, and getting the following (replaced scope and package):
npm ERR! 404 Not Found - PUT https://registry.npmjs.org/#SCOPE%2fPACKAGE - Not found
npm ERR! 404
npm ERR! 404 '#SCOPE/PACKAGE#0.0.5' is not in the npm registry.
npm ERR! 404 You should bug the author to publish it (or use the name yourself!)
This package IS published so I thought the %2f was causing the issue. I was using NPM_TOKEN as an environment variable as well, as per npmjs documentation.
Solution:
- name: Setup Node.js environment
uses: actions/setup-node#v2.1.2
with:
node-version: '12'
registry-url: 'https://registry.npmjs.org'
- name: Publish to npmjs
run: |
npm install
npm run pack
npm publish <package tgz> --access public
env:
NPM_TOKEN: ${{ secrets.NPM_APIKEY }}
NODE_AUTH_TOKEN: ${{ secrets.NPM_APIKEY }}
via Github Documentation.
NOTE: I'm not sure NPM_TOKEN is still required here.
Make sure .yarnrc actually contains
registry "https://registry.npmjs.org/"
and not
registry "registry.npmjs.org"
That'll give the same error.
In my C:\Users\XXXXXX.npmrc file, I have the following settings;
https-proxy=http://proxy.server.com:0000
http-proxy=http://proxy.server.com:0000
strict-ssl=false
The proxy has been verified as correct and when I attempt to install an item from the Node Package Manager to a directory that contains a package.json file, I receive the following error:
npm REGISTRY json parsing error
When I attempt to see my npm configurations, I use the following command:
npm config list
and see the following settings:
;cli configs
user-agent = "npm/2.15.5 node/v4.4.5 win32 x64"
; environment configs
cache = "C:\\LOCATION"
prefix = "C:\\LOCATION"
registry = "http://another.proxy.server:0000"
; userconfig C:\Users\USERNAME\.npmrc
https-proxy = "https://proxy.server.com:0000
http = "http://proxy.server.com:0000
rejectUnauthorized=false
strict-ssl=false
;builtin config defined
;node bin location = C:\LOCATION
;cwd = C:\LOCATOIN
;HOME = C:\LOCATION
; 'npm config ls -l' to show all defaults.
I am able to change the registry using the following command:
npm config set registry http://registry.npmjs.org
However, when I run the following command:
npm config list
It still lists the other proxy as the default.
Any ideas?
UPDATE
After viewing other threads, I decided to see all of the NPM default configurations in detail by using the following command:
npm config ls -l
Once running this command, I see the following entry (which I could not see running the previous command) for the registry configuration:
; registry = "https://registry.npmjs.org/" (overridden)
Does anyone have any idea why the registry would be overridden or common applications/locations that would do such a thing?
The line that you are seeing from 'npm config ls -l' shows that the default value of registry is 'https://registry.npmjs.org/'. When you ran the command 'npm config set registry http://registry.npmjs.org', you overrode that value. You should see a line in the userconfig section of the ls output that shows that you set the 'registry' value in your user config. It also looks like the registry is being overridden by an environment variable that probably overrides both the default config and the user config (what you are setting with the 'config set' command).
We have a nodejs project running on Amazon Elastic Beanstalk that uses private modules that we host using nodejitsu's private npm registry.
However getting access to the private npm registry from the elastic instances hasn't been straightforward and is not documented well.
What is the best way to get this access set up?
None of the other answers were working for me. After hours of hair pulling, we finally figured it out. The solution that worked is almost the same as the other answers but with a very minor tweak.
Set an NPM_TOKEN environment variable on Elastic Beanstalk under Configuration > Software Configuration > Environment Properties.
Create a .ebextensions/npm.config file. (The name does not have to be 'npm'.)
Put this content into the file:
files:
"/tmp/.npmrc":
content: |
//registry.npmjs.org/:_authToken=${NPM_TOKEN}
Note that it uses ${NPM_TOKEN} and not $NPM_TOKEN. This is vital. Using $NPM_TOKEN will not work; it must have the curly braces: ${NPM_TOKEN}.
Why are the curly braces needed? No idea. In shell/POSIX languages, ${VAR} and $VAR are synonymous. However, in .npmrc files (at the time of this writing), variables without the curly brackets are not recognized as variables, so npm must be using a slightly different syntax standard.
UPDATE
Also, this has worked for us only on new or cloned environments. For whatever reason, environments which were not initialized with a /tmp/.npmrc will not read it in any future deployments before running npm install --production. We've tried countless methods on 4 different apps, but cloning and replacing an environment has been the only method which has worked.
So, we managed to get this working by using the npm userconfig file. See the doc page for npmrc for more info.
When a nodejs application is being deployed to Elastic Beanstalk, the root user runs npm install. So you will need to write the root's npm userconfig file, which is at /tmp/.npmrc.
So if you add a file called private_npm.config (or whatever name you choose) to your .ebextensions folder with all the information needed, you will be good to go. See Customizing and Configuring AWS Elastic Beanstalk Environments for more info.
So here is what my file looks like to use nodejitsu private registry.
.ebextensions/private_npm.config:
files:
#this is the npm user config file path
"/tmp/.npmrc":
mode: "000777"
owner: root
group: root
content: |
_auth = <MY_AUTH_KEY>
always-auth = true
registry = <PATH_TO_MY_REGISTRY>
strict-ssl = true
email = <NPM_USER_EMAIL>
Using an .npmrc within the project also works. For example...
.npmrc
registry=https://npm.mydomain.com
You may want to .gitignore this file if you include an _authToken line but make sure you don't .ebignore it so it's correctly bundled up with each deployment. After trying a few things unsuccessfully, I came across this post which made me realize specifying it locally in a project is possible.
The answer above as a step in the right direction, but the permissions and owner did not work for me. Managed to get it to work with the following combination:
files:
#this is the npm user config file path
"/tmp/.npmrc":
mode: "000600"
owner: nodejs
group: nodejs
content: |
_auth = <MY_AUTH_KEY>
always-auth = true
registry = <PATH_TO_MY_REGISTRY>
strict-ssl = true
email = <NPM_USER_EMAIL>
Place the below within your .ebextensions/app.config.
files:
"/tmp/.npmrc":
mode: "000777"
owner: root
group: root
content: |
//registry.npmjs.org/:_authToken=$NPM_TOKEN
Where NPM_TOKEN is an environment variable with the value of your actual npmjs auth token.
Note that environment variables within elasticbeanstalk can and should be set from within the AWS console Elasticbeanstalk software configuration tab.
AWS Elasticbeanstalk Configuration
In new Elastic Beanstalk Linux 2 Platforms, none of these solutions work (apart from the .npmrc file solution that works but has its issues when using them in development evironments due to the requirements that all developers have their ${NPM_TOKEN} Env Var defined in their own environments).
The reason is that the /tmp/.npmrc location no longer works.
Option 1
You have to change the .ebextensions/npm.config file to this new format:
files:
#this is the npm user config file path
"/root/.npmrc":
mode: "000777"
owner: root
group: root
content: |
_auth= ${NPM_TOKEN}
registry = https://{yourprivatenpmrepository.com}/
Option 2
Add a custom .npmrc_{any-suffix} to the root of your app and create a prebuild hook to rename it before Beanstalk executes the npm install so that it can use your private repository configuration:
Add the following file (path from your app root) .platform/hooks/prebuild/01_set_npmrc.sh with the following content:
#!/bin/bash
#Copy and rename .npmrc_beanstalk to .npmrc
mv .npmrc_beanstalk .npmrc
Create an .npmrc_beanstalk file in your root with the following content (modify it depending on your private npm config):
_auth= ${NPM_TOKEN}
registry = https://{yourprivatenpmrepository.com}/
Chmod the hook file so that it has the necessary exec permissions when uploaded to EB: chmod +x .platform/hooks/prebuild/01_set_npmrc.sh
Re-deploy using EB CLI and you are done!
With modern platforms, you no longer need to do this via .ebextensions
You can simply create a .npmrc file at the root of your deployment package, alongside your package.json with the following line:
//registry.npmjs.org/:_authToken=${NPM_TOKEN}
Using this method, you can create an environment variable named NPM_TOKEN in your AWS console so you don't have to store the token in your repo.
Structure:
~/your-app/
|-- package.json
|-- .npmrc