We have a private npm repository based on Sinopia
What should I define in package.json that some packages will be installed from Synopia rather then from global npm repository?
If I install it from command line I can run: npm install <package_name> --registry <http://<server:port>
P.S. tried to google and looked in official NPM documentation but have found nothing.
One of the method i know that is by .npmrc
You can also use .npmrc also inside the project
set configuration like this
registry = http://10.197.142.28:8081/repository/npm-internal/
init.author.name = Himanshu sharma
init.author.email = rmail#email.com
init.author.url = http://blog.example.com
# an email is required to publish npm packages
email=youremail#email.com
always-auth=true
_auth=YWRtaW46YWRtaW4xMjM=
auth can be generate by
username:password
echo -n 'admin:admin123' | openssl base64
output YWRtaW46YWRtaW4xMjM=
The whole point of sinopia is a private registry and a proxy at the same time. You can use uplinks install all your packages from one registry entry point. Sinopia is able to route to any registry if the local storage is not able to resolve the dependency. By default, he points to npmjs .
So, if you set your configuration like
# a list of other known repositories we can talk to
uplinks:
npmjs:
url: https://registry.npmjs.org/
packages:
'#*/*':
# scoped packages
access: $all
publish: $authenticated
proxy: npmjs
'**':
# allow all users (including non-authenticated users) to read and
# publish all packages
#
# you can specify usernames/groupnames (depending on your auth plugin)
# and three keywords: "$all", "$anonymous", "$authenticated"
access: $all
# allow all known users to publish packages
# (anyone can register by default, remember?)
publish: $authenticated
# if package is not available locally, proxy requests to 'npmjs' registry
proxy: npmjs
You should be able to resolve all your dependencies independently of the source of each of them
btw: sinopia has no longer maintained.
Related
I'm using circle ci to deploy a serverless built in nodejs. And I added as dependency of the main repo,a private github repo. E.g:
// package.json
.....
"dependencies": {
"my-private-github-repo": "git+ssh://git#github.com:company-name/my-private-github-repo.git",
.....
},
.....
The problem is that I need to give access the deploy process to read and clone the private repo when npm install runs
I have configured my ssh user-keys in circle ci,I followed the steps in this documentation: creating-a-bitbucket-user-key, and I m also adding it in my config.yml like this:
// .circleci/config.xml
....
steps:
- add_ssh_keys:
fingerprints:
- "My fingerprint"
....
But during the cicd it throws this message: 'There are no configured ssh keys to install'
There are no configured ssh keys to install
and, of course, npm install fails because can not access to the repo
Any clue? Thanks anyway
This thread mentions:
When this error appears, it typically means that the ssh keys have not been configured in all locations.
SSH keys will need to be set in both the project setting's page and within the config.yml.
Just in case, double-check the URL https://app.circleci.com/settings/project/github/<your organization name>/<project name>/ssh and see if it matches Checkout SSH Keys page mentioned in the official documentation
I would like to fetch private npm from private repository using scopes.
I have set the .npmrc as such.
registry=https://registry.npmjs.org/
#myco:registry=https://nexus.myco.com/nexus/repository/
I have set the .yarnrc as such.
registry "https://registry.npmjs.org/"
"#myco:registry" "https://nexus.myco.com/nexus/repository/"
But when I do:
yarn --verbose add #myco/some-private-npm
It throws this error:
verbose 0.708 Performing "GET" request to "https://nexus.myco.com/nexus/repository/#myco%2fsome-private-npm".
verbose 0.792 Request "https://nexus.myco.com/nexus/repository/#myco%2fsome-private-npm" finished with status code 404.
When I do:
yarn --verbose add #myco:some-private-npm
It goes to this 400 url (Nexus: Invalid repository path):
verbose 0.957 Request "https://nexus.myco.com/nexus/repository/#myco:some-private-npm" finished with status code 400.
The actual npm is located at:
https://nexus.myco.com/nexus/repository/myco/some-private-npm
How do I make sure the url fetched does not have "#" and "%2f" is a "/"?
Thanks!
Per this github issue for yarn, try adding similar config to your .yarnrc:
registry "https://registry.npmjs.org/"
"#myco:registry" "https://nexus.myco.com/nexus/repository/"
Failing that, I recommend perusing the linked issue and trying the solutions provided.
I run into exact the same issue some time ago.
check at NPM if your repository team access level is read/write.
On my side the team i was in had only read access level. Switching solved the issue.
Another possible scenario is that the slash encoding isn't really the problem. You simply do not have access to that specific repository.
That would explain the 404, since if you don't have access, for security reasons, it's as if it doesn't exist, you don't get a 401.
I found this page when also trying to run npm publish of a .tgz file in Github Actions, and getting the following (replaced scope and package):
npm ERR! 404 Not Found - PUT https://registry.npmjs.org/#SCOPE%2fPACKAGE - Not found
npm ERR! 404
npm ERR! 404 '#SCOPE/PACKAGE#0.0.5' is not in the npm registry.
npm ERR! 404 You should bug the author to publish it (or use the name yourself!)
This package IS published so I thought the %2f was causing the issue. I was using NPM_TOKEN as an environment variable as well, as per npmjs documentation.
Solution:
- name: Setup Node.js environment
uses: actions/setup-node#v2.1.2
with:
node-version: '12'
registry-url: 'https://registry.npmjs.org'
- name: Publish to npmjs
run: |
npm install
npm run pack
npm publish <package tgz> --access public
env:
NPM_TOKEN: ${{ secrets.NPM_APIKEY }}
NODE_AUTH_TOKEN: ${{ secrets.NPM_APIKEY }}
via Github Documentation.
NOTE: I'm not sure NPM_TOKEN is still required here.
Make sure .yarnrc actually contains
registry "https://registry.npmjs.org/"
and not
registry "registry.npmjs.org"
That'll give the same error.
I am wondering how to configure the .npmrc file so that I can have a default registry and a different scoped registry with authentication.
I am using Nexus for the private repository and I am not sure how to set authentication for the scoped registry, only the default registry.
For example my ~/.npmrc file is:
registry=https://registry.npmjs.org/
#test-scope:registry=http://nexus:8081/nexus/content/repositories/npm-test/
email=test#user.com
_auth="…"
If I do npm publish for a package scoped to test-scope, I get an authentication error.
AFAIK, the _auth only applies to the registry=... section. Is there a way of specifying an auth key for the #test-scope:registry=... section?
Thanks,
So, after some digging through the NPM source code, it turns out there is a way to do this.
My solution is below:
registry=https://registry.npmjs.org/
#test-scope:registry=http://nexus:8081/nexus/content/repositories/npm-test/
//nexus:8081/nexus/content/repositories/npm-test/:username=admin
//nexus:8081/nexus/content/repositories/npm-test/:_password=YWRtaW4xMjM=
email=…
Explanation:
The scope #test-scope specifies that packages with the scope should be published to a different registry than the default registry= when executing the npm publish command.
The two lines starting with //nexus:8081/... are used to specify the credentials to the scoped repository for both username and _password where _password is the base64 encoded password component from the previously used _auth credentials.
Using this approach, only scoped packages will be published and installed from the private registry and all other packages will be installed from the default registry.
Edit:
Additional to this, the password can be specified as an environment variable so that it is not stored in plaintext in the file.
For example:
registry=https://registry.npmjs.org/
#test-scope:registry=http://nexus:8081/nexus/content/repositories/npm-test/
//nexus:8081/nexus/content/repositories/npm-test/:username=admin
//nexus:8081/nexus/content/repositories/npm-test/:_password=${BASE64_PASSWORD}
email=…
Also, when using Nexus, the email= line must be specified.
for some strange reason the _auth is called _authToken when used with scoped packages. If you are using this you don't have to store your plain text password in your .npmrc
registry=https://registry.npmjs.org/
#test-scope:registry=http://nexus:8081/nexus/content/repositories/npm-test/
//nexus:8081/nexus/content/repositories/npm-test/:_authToken=...
email=…
Run the following command, replacing #company-scope with the scope, and company-registry with the name of your company’s npm Enterprise registry:
npm login --scope=#company-scope --registry=https://registry.company-registry.npme.io/
This information is available on the npm documention.
We have a nodejs project running on Amazon Elastic Beanstalk that uses private modules that we host using nodejitsu's private npm registry.
However getting access to the private npm registry from the elastic instances hasn't been straightforward and is not documented well.
What is the best way to get this access set up?
None of the other answers were working for me. After hours of hair pulling, we finally figured it out. The solution that worked is almost the same as the other answers but with a very minor tweak.
Set an NPM_TOKEN environment variable on Elastic Beanstalk under Configuration > Software Configuration > Environment Properties.
Create a .ebextensions/npm.config file. (The name does not have to be 'npm'.)
Put this content into the file:
files:
"/tmp/.npmrc":
content: |
//registry.npmjs.org/:_authToken=${NPM_TOKEN}
Note that it uses ${NPM_TOKEN} and not $NPM_TOKEN. This is vital. Using $NPM_TOKEN will not work; it must have the curly braces: ${NPM_TOKEN}.
Why are the curly braces needed? No idea. In shell/POSIX languages, ${VAR} and $VAR are synonymous. However, in .npmrc files (at the time of this writing), variables without the curly brackets are not recognized as variables, so npm must be using a slightly different syntax standard.
UPDATE
Also, this has worked for us only on new or cloned environments. For whatever reason, environments which were not initialized with a /tmp/.npmrc will not read it in any future deployments before running npm install --production. We've tried countless methods on 4 different apps, but cloning and replacing an environment has been the only method which has worked.
So, we managed to get this working by using the npm userconfig file. See the doc page for npmrc for more info.
When a nodejs application is being deployed to Elastic Beanstalk, the root user runs npm install. So you will need to write the root's npm userconfig file, which is at /tmp/.npmrc.
So if you add a file called private_npm.config (or whatever name you choose) to your .ebextensions folder with all the information needed, you will be good to go. See Customizing and Configuring AWS Elastic Beanstalk Environments for more info.
So here is what my file looks like to use nodejitsu private registry.
.ebextensions/private_npm.config:
files:
#this is the npm user config file path
"/tmp/.npmrc":
mode: "000777"
owner: root
group: root
content: |
_auth = <MY_AUTH_KEY>
always-auth = true
registry = <PATH_TO_MY_REGISTRY>
strict-ssl = true
email = <NPM_USER_EMAIL>
Using an .npmrc within the project also works. For example...
.npmrc
registry=https://npm.mydomain.com
You may want to .gitignore this file if you include an _authToken line but make sure you don't .ebignore it so it's correctly bundled up with each deployment. After trying a few things unsuccessfully, I came across this post which made me realize specifying it locally in a project is possible.
The answer above as a step in the right direction, but the permissions and owner did not work for me. Managed to get it to work with the following combination:
files:
#this is the npm user config file path
"/tmp/.npmrc":
mode: "000600"
owner: nodejs
group: nodejs
content: |
_auth = <MY_AUTH_KEY>
always-auth = true
registry = <PATH_TO_MY_REGISTRY>
strict-ssl = true
email = <NPM_USER_EMAIL>
Place the below within your .ebextensions/app.config.
files:
"/tmp/.npmrc":
mode: "000777"
owner: root
group: root
content: |
//registry.npmjs.org/:_authToken=$NPM_TOKEN
Where NPM_TOKEN is an environment variable with the value of your actual npmjs auth token.
Note that environment variables within elasticbeanstalk can and should be set from within the AWS console Elasticbeanstalk software configuration tab.
AWS Elasticbeanstalk Configuration
In new Elastic Beanstalk Linux 2 Platforms, none of these solutions work (apart from the .npmrc file solution that works but has its issues when using them in development evironments due to the requirements that all developers have their ${NPM_TOKEN} Env Var defined in their own environments).
The reason is that the /tmp/.npmrc location no longer works.
Option 1
You have to change the .ebextensions/npm.config file to this new format:
files:
#this is the npm user config file path
"/root/.npmrc":
mode: "000777"
owner: root
group: root
content: |
_auth= ${NPM_TOKEN}
registry = https://{yourprivatenpmrepository.com}/
Option 2
Add a custom .npmrc_{any-suffix} to the root of your app and create a prebuild hook to rename it before Beanstalk executes the npm install so that it can use your private repository configuration:
Add the following file (path from your app root) .platform/hooks/prebuild/01_set_npmrc.sh with the following content:
#!/bin/bash
#Copy and rename .npmrc_beanstalk to .npmrc
mv .npmrc_beanstalk .npmrc
Create an .npmrc_beanstalk file in your root with the following content (modify it depending on your private npm config):
_auth= ${NPM_TOKEN}
registry = https://{yourprivatenpmrepository.com}/
Chmod the hook file so that it has the necessary exec permissions when uploaded to EB: chmod +x .platform/hooks/prebuild/01_set_npmrc.sh
Re-deploy using EB CLI and you are done!
With modern platforms, you no longer need to do this via .ebextensions
You can simply create a .npmrc file at the root of your deployment package, alongside your package.json with the following line:
//registry.npmjs.org/:_authToken=${NPM_TOKEN}
Using this method, you can create an environment variable named NPM_TOKEN in your AWS console so you don't have to store the token in your repo.
Structure:
~/your-app/
|-- package.json
|-- .npmrc
I'm trying to install github private repository by npm that includes other private github repositories as dependency.
Have tried a lot of ways and posts but none is working. Here is what i'm doing :
npm install git+https://github.com/myusername/mygitrepository.git
in package.json is like :
"dependencies": {
"repository1name": "git+https://github.com/myusername/repository1.git",
"repository2name": "git+https://github.com/myusername/repository2.git"
}
What is the the right way to do it?
Try this:
"dependencies" : {
"name1" : "git://github.com/user/project.git#commit-ish",
"name2" : "git://github.com/user/project.git#commit-ish"
}
You could also try this, where visionmedia/express is name/repo:
"dependencies" : {
"express" : "visionmedia/express"
}
Or (if the npm package module exists):
"dependencies" : {
"name": "*"
}
Taken from NPM docs
The following worked just fine in all scenarios i needed :
"dependencies": {
"GitRepo": "git+https://<token-from-github>:x-oauth-basic#github.com/<user>/<GitRepo>.git"
}
For those of you who came here for public directories, from the npm docs: https://docs.npmjs.com/files/package.json#git-urls-as-dependencies
Git URLs as Dependencies
Git urls can be of the form:
git://github.com/user/project.git#commit-ish
git+ssh://user#hostname:project.git#commit-ish
git+ssh://user#hostname/project.git#commit-ish
git+http://user#hostname/project/blah.git#commit-ish
git+https://user#hostname/project/blah.git#commit-ish
The commit-ish can be any tag, sha, or branch which can be supplied as an argument to git checkout. The default is master.
The accepted answer works, but I don't like much the idea to paste secure tokens into the package.json
I have found it elsewhere, just run this one-time command as documented in the git-config manpage.
git config --global url."https://${GITHUB_TOKEN}#github.com/".insteadOf git#github.com:
GITHUB_TOKEN may be setup as environmnet variable or pasted directly
and then I install private github repos like: npm install user/repo --save
works also in Heroku, just setup the above git config ... command as heroku-prebuild script in package.json and setup GITHUB_TOKEN as Heroku config variable.
There are multiple ways to do it as people point out, but the shortest versions are:
// from master
"depName": "user/repo",
// specific branch
"depName": "user/repo#branch",
// specific commit
"depName": "user/repo#commit",
// private repo
"depName": "git+https://[TOKEN]:x-oauth-basic#github.com/user/repo.git"
e.g.
"dependencies" : {
"hexo-renderer-marked": "amejiarosario/dsa.jsd#book",
"hexo-renderer-marked": "amejiarosario/dsa.js#8ea61ce",
"hexo-renderer-marked": "amejiarosario/dsa.js",
}
"dependencies": {
"some-package": "github:github_username/some-package"
}
or just
"dependencies": {
"some-package": "github_username/some-package"
}
https://docs.npmjs.com/files/package.json#github-urls
Since Git uses curl under the hood, you can use ~/.netrc file with the credentials. For GitHub it would look something like this:
machine github.com
login <github username>
password <password OR github access token>
If you choose to use access tokens, it can be generated from:
Settings -> Developer settings -> Personal access tokens
This should also work if you are using Github Enterprise in your own corporation. just put your enterprise github url in the machine field.
Here is a more detailed version of how to use the Github token without publishing in the package.json file.
Create personal github access token
Setup url rewrite in ~/.gitconfig
git config --global url."https://<TOKEN HERE>:x-oauth-basic#github.com/".insteadOf https://x-oauth-basic#github.com/
Install private repository. Verbose log level for debugging access errors.
npm install --loglevel verbose --save git+https://x-oauth-basic#github.com/<USERNAME HERE>/<REPOSITORY HERE>.git#v0.1.27
In case access to Github fails, try running the git ls-remote ... command that the npm install will print
Further, in order to make key's access secure
Create .env file at the same directory level where package.json resides.
Mention PERSONAL_ACCESS_TOKEN=******************************* into .env file
Dont forget to add '.env' into .gitingore list which will prevent exposing key to outside world while you make git commit to your repo.
Now you can add your dependency in package.json as below,
Package.json
"dependencies": {
...
"my-private-github-repo": "git+https://${ENV.PERSONAL_ACCESS_TOKEN}#github.com/USER/abcd-repo-3.4.0.git",
...
}
There are other ways using 'DOTENV' npm package, but it could not do much when we are trying to resolve "Github" package dependency. Above seems to be straight forward solution.
There's also SSH Key - Still asking for password and passphrase
Using ssh-add ~/.ssh/id_rsa without a local keychain.
This avoids having to mess with tokens.
If you want to add the dependency that is not anchored to master nor to a particular commit, you can do it by using semver. Like that:
"dependencies": {
"some-package": "github:github_username/some-package#semver:^1.0.0"
}
For my private repository reference I didn't want to include a secure token, and none of the other simple (i.e. specifying only in package.json) worked. Here's what did work:
Went to GitHub.com
Navigated to Private Repository
Clicked "Clone or Download" and Copied URL (which didn't match the examples above)
Added #commit-sha
Ran npm install
Note that the github repos that you try to add as a dependency to your package.json file needs to have its own package.json file defined.