Uploading node_modules to Nexus hosted NPM repository - node.js

We are building our project in offline build server. Thus we have to store and maintain all our project dependencies in local network Nexus.
I have created hosted NPM registry in Nexus following this guide. I skipped the proxy and group part. Now I want to upload (and maintain) all the project NPM packages (node_modules) from my local workstation to this repository. I don't see upload button in the Nexus repository settings. How to properly upload and maintain NPM packages in Nexus NPM registry? Can this be done by the Nexus GUI, or do I have to use command line? Note the nexus is disconnected from the internet.

The UI does not handle transitive dependencies.
You can also use the rest API to manage components directly, and upload all your .tgz packages.
POST ​/v1​/components
For instance, to upload the package my-npm-package-0.0.0.tgz to the repository npm-private use the following:
curl -u user:password -X POST "http://localhost:8081/service/rest/v1/components?repository=npm-private" -H "accept: application/json" -H "Content-Type: multipart/form-data" -F "npm.asset=#my-npm-package-0.0.0.tgz;type=application/x-compressed"
The complete live API specs can be found at endpoint /#admin/system/api
The official nexus documentation can be found at https://help.sonatype.com/repomanager3/rest-and-integration-api/components-api

UI upload for npm packages was added in version 3.7.0.
https://help.sonatype.com/display/NXRM3/Uploading+Components

You don't need to upload the node_modules.
You need to create a proxy and a group repository.
Then you can delete the node_modules and package-lock.json in your project and run npm install.
Because your hosted don't have those packages,it will download them from proxy.
Then you can see those packages in the group.
When you run npm install again,it will download form Nexus ,not NPM.
The download speed will be very fast.

Related

Is it mandatory for each deployment to production for remove node modules and run npm install?

I use vuetify (vue)
Is it mandatory for each deployment to production for remove node modules and run npm install? Or just run npm run build?
I have two option :
Option 1 : Every deployment, I run the npm run build directly
Option 2 :
Delete the contents of dist folder
Delete node_modules folder
npm install
npm run build
Which is the best option?
npm install
This command installs a package, and any packages that it depends on. If the package has a package-lock or shrinkwrap file, the installation of dependencies will be driven by that, with an npm-shrinkwrap.json taking precedence if both files exist. See package-lock.json and npm-shrinkwrap.
If you did not install or update the package before releasing the project, you do not need to execute npm install, otherwise, you need to execute it to ensure that dependent packages on the production environment is consistent with your local dependent package version.
If you are using an automatic build deployment tool like jenkins, for convenience you can execute the install command before each build. It's okay.
Imagine more environments, not just a production:
development
testing1
staging
uat
production
Can we upload the npm run build result (compressed js) or node_modules to our git repository? ANSWER IS NOT!!. So if you need to have a version of your app running in any of these environments, you must to execute npm run build. And this command needs the classic npm run install. I think this last sentence, answer your question.
(ADVICE) Docker to the rescue
assumption 1 your client-side app (vue) is not complex(no login, no session, no logout, etc ), you could publish it using a basic nginx, apache, basic-nodejs.
assumption 2 you are able to have one more server for docker private repository. Also if you are in google, amazon o azure, this service is ready to use, of course a payment is required
In one line, with docker you must execute just one time npm install and npm run build. Complete flow is:
developer push some changes to the git repository
manually or automatically a docker build in launched.
inside Dockerfile, npm install and npm run build is executed. Also a minimal server with nodejs (example) is configured pointing to your builded assets
your new docker image is uploaded to your docker private repository
that's all
If your quality assurance team needs to perform some tests to your new app, just a docker image download is required. If everything is ok, you pass to the next stage (staging or uat) or production. Steps will be the same: just download the docker image.
Optimizations
Use docker stages to split build and start steps
If your app does not have complex flows(no login, no session, no logout, etc ), replace node basic server with a simple nginx
I need login and logout
In this case, nginx or apache does not helps you because they are a simple static servers.
You could use a minimal nodejs code like this:
https://github.com/jrichardsz/nodejs-static-pages/blob/master/server.js
Adding /login , /logout, etc
Or use my server:
https://github.com/utec/geofrontend-server
which has a /login, /logout and other cool features for example: How are you planning to pass your backend api urls to your vue app in any of your environments?.

How to publish npm package to private nexus repository

I have created a sample react app which I have published to npm repository so I can install them into some other react project. Now I want to publish the same package to Nexus repository so it would be private and I am not able to figure out how to do this. I have tried the following command
npm config set registry "https://my-custom-registry.registry.nodesource.io/"
but when I am trying to publish my npm repo there It giving me an authentication error. what are the other steps I need to perform?
if Nexus 3 use token bearer token, if use nexus 2 you have to use basic authentication
registry=http://.......
always-auth=true
email=mail#mail.test
_auth=ZGV2ZWxvcGVyOmRldmVsb3BlcjEyMw==
this is file .npmrc
i had same error and with this i works like a charm

Deploy React Project to an external server

I want to upload my React project to an external server. I already have a project and I use node server locally to run it using webpack.
I don't have any deployment experience prior to this. So any insights regarding deployment would be helpful.
You can follow these steps:
SSH to the server using server credentials.
sudo su
git clone repository address [Only if the repository hasn't been cloned yet]
cd projectFolder
git checkout <branchYouWantToDeploy>
git pull origin <branchYouWantToDeploy>
npm install
npm run build:production
apachectl graceful [only for apache servers]
service nginx reload [only for nginx servers]
exit
If git and npm are not installed in the server, you can follow these links respectively:
GIT
https://git-scm.com/book/en/v2/Getting-Started-Installing-Git
NODE
https://nodejs.org/en/download/package-manager/
npm is installed with Node.js npm is distributed with Node.js- which
means that when you download Node.js, you automatically get npm
installed on your computer.
You may/may not need to follow all the steps depending upon the type and complexity of your project and server. However those should be good enough to start. You can always add more details regarding your project and deployment server in your question.
Here are some medium blog post that might you to go in-depth:
Understanding React deployment
Deploy react-webpack to Digital Ocean with Nginx and Github

npm install package from local folder

I am behind a firewall and I believe git connection at 9418 are getting blocked. So I want to install a repository via npm (mysam https://github.com/mysamai/mysam). I tried installing directly with
npm --proxy <proxy-details> install -g mysam
Which fails saying -
fatal: Unable to look up github.com (port 9418) (No such host is known. )
So to avoid that I downloaded zip of the git repository and found on internet to use
cd package-folder
npm link
However that also fails stating same error. Is there any option to install the locally downloaded repository.
Thanks
Using npm without an internet connection just does not work.
This is how we worked around this for a fully isolated node project:
You need a machine with internect connection. On this machine you do all the npm install calls.
Getting the installed files to other machines is easy if you are on linux or mac. With windows you are likely to have problems with long path names. Windows can handle them internally but almos all tools including windows explorer, winzip, 7-zip and friends cannot handle them. You have to use either robocopy or put everything into a git repo to move them arround. For GIT check https://stackoverflow.com/a/26111092/671639 to make it handle long path names on Windows. Also add the node_modles/**/bin/** folder to .gitignore.
We used GIT and moved around the repo. Attaching it as a remote allows to pull in new installed modules to the closed system.
Once the new module is available in your closed system call npm rebuild to make sure you have all the tooling availbable.
Of course this means to add node_modules to GIT or whatever SCM you are using.

Is there any way to configure multiple registries in a single npmrc file

Here is my problem. We have a private NPM registry which only works in VPN. I would like to have a fallback registry https://registry.npmjs.org so that when I am out of VPN it works seamlessly.
P.S. Currently I am using npmrc which does a good job in switching between .npmrc files as a workaround
You can have multiple registries for scoped packages in your .npmrc file. For example:
#polymer:registry=<url register A>
registry=http://localhost:4873/
Packages under #polymer scope will be received from https://registry.npmjs.org, but the rest will be received from your local NPM.
On version 4.4.1, if you can change package name, use:
npm config set #myco:registry http://reg.example.com
Where #myco is your package scope.
You can install package in this way:
npm install #myco/my-package
For more info: https://docs.npmjs.com/misc/scope
I believe the top-voted answer might be outdated. As of June 2021, there is a much easier way to do this using npmrc.
Refer to npm Docs.
1. Install npmrc
To install npmrc, on the command line, run
npm i npmrc -g
2. Create your first npm profile
After installing npmrc, you can create a profile to access your custom (maybe company's) registry.
To create an npm Enterprise profile, on the command line, run npmrc -c name-of-profile.
For example, to create a profile called "work", run the following command:
npmrc -c work
To set an npm Enterprise registry for the profile, run the following command, replacing your-company-registry with the name of your company's npm Enterprise registry:
npm config set registry https://registry.your-company-registry.npme.io/
3. Create a profile for the public npm registry
After you have created your npm Enterprise profile, you can create a second profile for a different registry, such as the public npm registry.
To create a profile for the public registry, on the command line, run npmrc -c name-of-profile. For example, to create a profile called "open-source", run npmrc -c open-source.
To set the public registry for your open source profile, run the following command:
npm config set registry https://registry.npmjs.org/
4. Switch profiles with npmrc
To switch profiles, on the command line, run the following command, replacing profile-name with the name of your profile:
npmrc profile-name
Not the best way but If you are using mac or linux even in windows you can set alias for different registries.
##############NPM ALIASES######################
alias npm-default='npm config set registry https://registry.npmjs.org'
alias npm-sinopia='npm config set registry http://localhost:4873/'
For anyone looking also for a solution for authentication, I would add on the scoped packages solution that you can have multiple lines in your .npmrc file:
//internal-npm.example.com:8080/:_authToken=xxxxxxxxxxxxxxx
//registry.npmjs.org/:_authToken=yyyyyyyyyy
Each line represents a different NPM registry
Since it has been a couple years and it doesn't seem possible to do this (using npm alone), a solution to this problem is to use the Nexus Repository Manager (from Sonatype). Nexus supports multiple repositories, lets you order them, and also proxies/caches to improve speed.
A free version and pro/paid version exist. The feature that supports this is described at:
https://help.sonatype.com/repomanager3/node-packaged-modules-and-npm-registries
The relevant information is duplicated below so if/when the above URL/link stops working the information is still here.
A repository group is the recommended way to expose all your npm registries repositories from the repository manager to your users, without needing any further client side configuration. A repository group allows you to expose the aggregated content of multiple proxy and hosted repositories with one URL to npm and other tools.
It lets you create private npm registries
A private npm registry can be used to upload your own packages as well as third-party packages.
And
To reduce duplicate downloads and improve download speeds for your developers and CI servers, you should proxy the registry hosted at https://registry.npmjs.org. By default npm accesses this registry directly. You can also proxy any other registries you require.
So a quick bulleted list of things you do to get this working is:
Install Nexus
Create a local/private repo (or point to your private repo on another server)
Create a GROUP that lists your private repo, and the public repo.
Configure your $HOME/.npmrc file to point to the "GROUP" just created.
Publish your private npm packages to the local repo.
Users now can run a one time setup.
npm config set registry https://nexus/content/groups/GROUP
Then users can install both public or private packages via npm install.
npm install my-private-package
npm install lodash any-other-public-package
And both your public and private packages can be installed via a simple npm install command. Nexus finds the package searching each repo configured in the group and returns the results. So npm still thinks there is just one registry but behind the curtain there are multiple repos being used.
IMPORTANT NOTE: When you publish your components, you'll need to specify the npm publish --registry https://nexus/content/repositories/private-repo my-private-package command so your package is published to the correct repo.
You can use multiple repositories syntax for the registry entry in your .npmrc file:
registry=http://serverA.url/repository-uri/
//serverB.url/repository-uri/
//serverC.url/repository-uri/:_authToken=00000000-0000-0000-0000-0000000000000
//registry.npmjs.org/
That would make your npm look for packages in different servers.
Some steps you can try. (its how we do it at my workplace)
Create a registry group with two (or more) repository source address. One would be your internal private and the other a proxy to npmjs giving priority to the internal one.
Make this group your registry in the .npmrc file. This way npm will always try to get it from the internal one, if not found get it from the proxy
Hope that helps.
I encounter the same problem when my company set up its own registry, so I heavily rework on proxy-registry into proxy-multi-registries to solve this problem. Hope it will also helps you.
As of 13 April 2020 there is no such functionality unless you are able to use different scopes, but you may use the postinstall script as a workaround. It is always executed, well, after each npm install:
Say you have your .npmrc configured to install #foo-org/foo-pack-private from your private github repo, but the #foo-org/foo-pack-public public package is on npm (under the same scope: foo-org).
Your postinstall might look like this:
"scripts": {
...
"postinstall": "mv .npmrc .npmrcc && npm i #foo-org/foo-pack --dry-run && mv .npmrcc .npmrc".
}
Don't forget to remove #foo-pack/foo-org from the dependencies array to make sure npm install does not try and get it from github and to add the --dry-run flag that makes sure package.json and package-lock.json stay unchanged after npm install.
My approach was to make a slight command line variant that adds the registry switch.
I created these files in the nodejs folder where the npm executable is found:
npm-.cmd:
#ECHO OFF
npm --registry https://registry.npmjs.org %*
npm-:
#!/bin/sh
"npm" --registry https://registry.npmjs.org "$#"
Now, if I want to do an operation against the normal npm registry (while I am not connected to the VPN), I just type npm- where I would usually type npm.
To test this command and see the registry for a package, use this example:
npm- view lodash
PS. I am in windows and have tested this in Bash, CMD, and Powershell. I also
I use Strongloop's cli tools for that; see https://strongloop.com/strongblog/switch-between-configure-public-and-private-npm-registry/ for more information
Switching between repositories is as easy as : slc registry use <name>
I had the same issue and I've tried many solutions that didn't work, now i encountered that by using different scoops for my npm private packages, in that way in can use multiple registries with same .npmrc file like that :
.npmrc : (You can put as many packages you want)
#scop1:registry=https://gitlab.example.com/api/v4/projects/<project1_id>/packages/npm/
#scop2:registry=https://gitlab.example.com/api/v4/projects/<project2_id>/packages/npm/
//gitlab.example.com/api/v4/projects/<project1_id>/packages/npm/:_authToken=${GITLAB_TOKEN}
//gitlab.example.com/api/v4/projects/<project2_id>/packages/npm/:_authToken=${GITLAB_TOKEN}
No, NPM does not support multiple registry except for scoped one.
Refer to: https://docs.npmjs.com/cli/v7/using-npm/scope

Resources