I need to have my common npm module as git dependency and use it in many modules.
I'm trying to use access token for authorize in git. Bit I don't want to store the token in package.json (at least, it's hard to change it in few modules if it change).
Can I do something like this:
package.json
...
"scripts": {
"preinstall": "read-credentials.sh"
}
...
"dependencies": {
...
"my-module": "git+https://$$$token-from-variable$$$:x-oauth-basic#some-repo-url"
...
}
I know about options with private npm modules or make ssh keys for git access, but it isn't a good fit for me. The main point of the question is to figure out if there is a way to set variables from some preinstall script (where I can have access to access tokens) and use them in package.json.
You can have private modules on npm. It costs $7/month and is created for exactly the use case that you have here.
If you don't want to do that for some reason, then you can rely on git-specific environment variables to resolve your dependencies. For example if you use git+ssh://git#SERVER:USER/MODULE.git as a dependency then npm will try to get it with git using the default credentials used by git (which in turn uses the credentials used by ssh). You can even change GIT_SSH environment variable to completely control the SSH session however you want.
For more options see Install node module from own gitlab server.
As I understood, there is no way to create such kind of variables.
My solution for my problem:
Creating other module, which can take a token from external place and which I can put in public git or npm repo.
Add this module in dependency.
Call it in postinstall script.
Get token, build url and install my private-git-module from JS using npm.commands.install.
Related
I have created a private typings npm module that I am using for my firebase functions and app projects. When I went to deploy firebase functions, I get a big error for every function that basically says ERR! remote: Invalid username or password.
For what I have read, it looks like I need to create a .npmrc file and put it in the /functions directory. (https://cloud.google.com/functions/docs/writing/specifying-dependencies-nodejs#using_private_modules)
I cannot however find proper instructions on how to do this anywhere. From what I found, I have done the following:
ran npm login
ran npm token create --read-only
This then gave me a token that looks like this: XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX.
I then create a file called .npmrc in my functions directory, and placed //registry.npmjs.org/:_authToken=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX in it.
Additionally, I saw that the error message was trying to use ssh to install my private repo package, I have not setup ssh and am using https instead, because of this I changed my package file to git+https://github.com/accounts-name/repo#commit-num so that it uses HTTPS.
However, I still get the same error message. What am I missing? The above is what I have strung together from lots of google searching.
It seems that you have made too many different changes while trying to make it work, so let's just go through the whole process step by step.
Check the package.json of your npm module and publish it:
Remove "private" property or set it to false because private modules in the npm are meant to be never published. That is not obvious but that is true.
Next step is to apply restricted access to the package. In order to do that, add such property in the package.json file:
"publishConfig": {
"access": "restricted"
},
Make sure that npm account you use for publishing supports private packages.
Now open the terminal in the root directory of your package, type npm login then sign in to npm. Check if you put the proper version in the package.json.
Run npm publish. The package should be published in few seconds. No worries, thanks to publishConfig property nobody can access it.
Now it is time to allow package installation in your project
Go to the directory of the project and open package.json file
Check that you have the name and version of your package in the dependencies list
Open browser, navigate to https://npmjs.com, login to your account, navigate to settings page of your account and open the tokens tab
Create new token and copy it
Now again go to the directory of your project, on same level where package.json file is situated (that is important!) and create .npmrc file there.
Put such string in the .npmrc file:
//registry.npmjs.org/:_authToken=TOKEN_HERE
You are done!
Deployment with remote CI/CD services
The easiest approach is not add .npmrc into .gitignore. In such case the file will be always in repository, so npm install will run smoothly on any machine where project was cloned
If you don't want to have token string in the repository, you can move it to the environment variable of your CI/CD service and then link .npmrc file to that variable. For example, you can put generated token into the NPM_TOKEN env variable (Just token from npmjs, not the whole string from .npmrc!)
And then change the .npmrc file in the next way:
//registry.npmjs.org/:_authToken=${NPM_TOKEN}.
So, with those steps you should be able to install your restricted packages without any issues. Good luck!
If you are trying to deploy you functions with firebase deploy from a CI and your .npmrc file looks like this.
#acmecorp:registry=https://npm.pkg.github.com/
//npm.pkg.github.com/:_authToken=${NPM_REGISTRY_TOKEN}
You will run into the problem even if you have the env var set.
Build failed: Error: Failed to replace env in config: ${NPM_REGISTRY_TOKEN}
Firebase for some reason needs access to that private repo. But the env var is not sent over to firebase.
Solution I've implemented was to replace ${NPM_REGISTRY_TOKEN} in the .npmrc file on every run of the CI pipeline.
sed -i.bak "s/\${NPM_REGISTRY_TOKEN}/${NPM_REGISTRY_TOKEN}/g" .npmrc
This breaks if you use Yarn. Took me a while to find a thread pointing to npm install in the firebase cli predeploy step. If there's no package-lock.json and you only use yarn, this will break. Remove yarn.lock and install using npm to resolve the issue.
I have to work with some packages in the private registry. So, in my package.json in the dependencies section I have a lines like this one:
...
"dependencies": {
"#myco/my-awesome-package": "^0.4.5",
...
}
...
There is authentication required for the private registry, so I have to create the .npmrc file in my project:
registry=https://registry.npmjs.org/
#myco:registry=https://myco-registry-path/
//myco-registry-path/:username=${MYCO_REGISTRY_USER}
//myco-registry-path/:_password=${MYCO_REGISTRY_PASSWORD_BASE64}
Yes, I know about _authToken, but in my case it is easier to use user and password.
Anyway, here you can see two env variables: ${MYCO_REGISTRY_USER} and ${MYCO_REGISTRY_PASSWORD_BASE64} which I have to replace before npm install.
I know the very simple solution for this problem: put them to the "global" env variables for example to my .bash_profile (or any terminal profile of your choice).
But I do not want to keep variables like this in the "global" scope because the are important only for the current project. What I want to do is to use dotenv. I want to create a .env file in the root of my project:
MYCO_REGISTRY_USER=myco-registry-username-value
MYCO_REGISTRY_PASSWORD_BASE64=myco-registry-password-value-base64
I want that this values replace env variables in my .npmrc on the install action. But when I try npm install I get an error: Error: Failed to replace env in config: ${MYCO_REGISTRY_USER}. I can understand why it happens. Possibly because npm reads .npmrc values first and try to replace env variables and fails, because in this moment it know nothing about dotenv.
My question is how to deal with it?
Short summary:
I do not want to keep env variables in the terminal profile, instead I want to put it in the .env file inside my project.
I have to replace env variables in the .npmrc file with dotenv before npm install
I know this answer might come too late, but in case anyone else is looking for answers, here's a solution:
You need to prepend your scripts with dotenv-cli as so:
dotenv npm install
or in my case where the file was not .env:
dotenv -e .env.local npm install
The problem is that you cannot save this anywhere so that someone can use it with "npm install" somehow. Definitely npm preinstall is run after reading .npmrc so it fails too.
You will need to either document it well or just include a small shell script, but if you're supporting different OSs then it can get funny really fast...
Happily so, CD platforms like Netlify allow you to set environment variables manually.
But I guess this must not be the nicest of starts if someone clones your repo and the first they've got is a failing npm install 🤷♂️
Also, check this one out: locking-the-vault-on-font-awesome-npm-tokens
I have a Node project in a private GitHub repo that includes an NPM dependency on another private GitHub repo, e.g.:
{
"dependencies": {
"my-secret-module": "git+ssh://github.com/user/repo-name.git#sha"
}
}
The project is being run on Heroku, where the npm install step fails during deployment while attempting to install from the private repo.
I'm aware of the possibility of generating OAuth tokens on GitHub and using them in the above URL like so: git+ssh://<token>#github.com/user/repo-name.git, but I don't want to store the token in version control.
The most ideal situation would be to simply store the token in an environment variable, so my question is: is it possible to authenticate with GitHub for this situation using environment variables? Or something else similarly bound to the Heroku app?
You could take a look at this buildpack that allows you to add your ssh key to be used as environment variable. Also, take a look at this SO answer that will do basically the same but using pre-build scripts.
Does anyone know how to configure location of global repository?
My global repo is somewhere under $HOMEDRIVE/$HOMEPATH/blahblahblah
and all my packages are being installed under that place fopr global reference
but I want to park it somewhere specific and secret like the docroot of my appserver ? so I can operate demos and proof-of-concepts and prototypes ands show them off
can you tell me how I can configure the path to my global repository? I am on windows7 which is thoroughly supported and chmod chown issues are not as prevalent on linux
is this directory anchor controlled by a designated variable within NPM?
is this variable ever referenced by javascript modules indiscriminantly? i would hope not
I assume this variable is within the NPM tool itself.
what about bower... would bower operate the same configurable? or is bower a different animal and place.
is bower a subset of npm? anmd of so does it operate the same configuration as npm?
thank you
See the npm docs about the folders. It states that the global modules are installed under a configured prefix. You can get it from the npm config comand:
npm config get prefix
And you may change it with a similar command:
npm config set prefix /path/to/my/global/folder
However, modules are usually installed globally if want to use some command line command they provide. For using in some node.js application, prefer to install them locally. If you still want to use the globally installed modules inside the application, you should use the link command (though I'm not sure if it works in a Windows environment).
Bower is another thing completely. Looking at the api documentation, you will see that there is no option to install modules globally (which makes sense, as Bower is intended for front-end dependencies).
You could change the default folder using the directory parameter of your .bowerrc file (see the documentation). This way you would be able to set all projects to use the same folder, but notice that's not the way it's intended to use and you would need to set it in all projects.
npm config set registry <registry url>
once this command is run, check in ~/.npmrc, it must show your changes.
Basically the thing is I'm working on a project that uses grunt for build tasks and as I have a few dependencies here and there I thought it was a good idea to declare those on a package.json so that my co-workers can npm install without being required to manually install every package at the correct version.
Now the thing is, what if someone "accidentally" runs npm publish? Is there a way to have the package.json while keeping my stuff private?
Yes, set private to true.
If you set "private": true in your package.json, then npm will refuse
to publish it.
This is a way to prevent accidental publication of private
repositories. If you would like to ensure that a given package is only
ever published to a specific registry (for example, an internal
registry), then use the publishConfig hash described below to override
the registry config param at publish-time.
You can set "private" : true in your package.json file
Your CoWorkers will get an error if they try to publish it