I am working in a environment where build env doesn't have access to internet and my application uses npm packages during the build. Till now we have copied the node_modules to source and used them to build. We recently moved to gitlab premium and want to use gitlab registry to store our npm packages. So our issue is gitlab allows to publish only the registry and not its dependencies. But that will not help use because when we do npm install it will to get the some package dependencies from internet(registry.npm.org.js) which will fail as it will not have internet access.
My requirement is i should be able to publish all the dependencies also along with package as tar files in the npm registry itself so that during the build when npm install happens it should download packages from gitlab registry itself.
My package.json looks like this:
{
"name": "#npm-project/aws-sdk-v3-iam-examples",
"version": "1.2.0",
"main": "index.js",
"repository": "git#github.com/awsdocs/aws-doc-sdk-examples/tree/master/javascriptv3/example_code/iam.git",
"author": "Brian Murray <brmur#amazon.com>, Alex Forsyth <alex-git#amazon.com>",
"license": "Apache 2.0",
"dependencies": {
"#aws-sdk/client-sqs": "^3.32.0",
"#aws-sdk/node-http-handler": "^3.32.0",
"#aws-sdk/types": "^3.32.0",
"ts-node": "^9.0.0"
},
"devDependencies": {
"#types/node": "^14.0.23",
"typescript": "^4.0.2"
},
"publishConfig": {
"#npm-project:registry": "https://gitlab.com/api/v4/projects/xxxxx/packages/npm/"
}
}
i want #aws-sdk/client-sqs,#aws-sdk/node-http-handler,#aws-sdk/types packages to be available in my npm registry itself.
it might look something like this,
What you are searching for are mirror/proxy Registries like Nexus, jFrog or Verdaccio. Maintaining all your packages manually will be incredibly tedious.
In the GitLab Docs I can't find such a feature, they only support publishing scoped private packages so packages like ts-node could not be published at all. So it might be an option to setup such a mirror/proxy Registry yourself which could update automatically. Especially Verdaccio is OpenSource, free and easy to setup. Also jFrog+GitLab is a common combination I think but I think it's a paid product.
Related
Is it possible to get NPM to install a dependency from its repository or a custom repo depending on if its meant to install production dependencies only, i.e. --production. and install as a symlink when installing dev dependencies? I want developers to be able to run and debug from local source while production pulls from the github repo. I'm also afraid if the package.json only indicates the repo as a source, a developer will change the package.json to debug and forget to change it back when committing.
I have my package.json like this but with this it will only install the package when I allow dev dependencies and installs nothing when I don't
"dependencies": {
"myPackage": "github:mygithub/mypackage",
},
"devDependencies": {
"myPackage": "file:./../mypackage"
}
I'm encountering a problem with the npm install step during my CI/CD build in Azure DevOps.
The specific package that is causing this problem appears to be node-sass. This package has given me a lot of trouble in the past as well but I did get it working and builds have been running fine for a while. Now for some reason they are failing again but I cannot seem to reproduce the problem on my machine and the error doesn't make any sense to me.
Here is the output from the failed build: https://pastebin.com/w4aK4dEh.
The error message is "Error: Cannot find module './version'"
I have tried to modify the step and changed it from a simple npm install to npm install --save-dev --unsafe-perm but it didn't seem to have any effect.
package.json
{
"version": "1.0.0",
"name": "myproject",
"private": true,
"devDependencies": {
"gulp": "4.0.2",
"gulp-concat": "2.6.1",
"gulp-cssmin": "0.2.0",
"gulp-rename": "2.0.0",
"gulp-sass": "4.1.0",
"gulp-uglify": "3.0.2",
"rimraf": "3.0.2",
"node-sass": "^4.8.3"
},
"dependencies": {}
}
I have also tried to remove node-sass from the dependencies entirely because I thought gulp-sass already includes it by default, but then I ran into "module node-sass not found" errors.
I'm really stumped on this one. Any suggestions would be appreciated.
I am not entirely sure which of the following was the key to success but here's what I did to resolve this:
Delete the node_modules folder from my project, commit and then modify my .gitignore to exclude it from the repository permanently (yes, I know this one is a bit controversial as some people say the folder should be checked in)
Update the versions of the dependencies in the package.json file
Specify the specific version of NodeJs (and thus npm) to use. I did this by simply adding a step in the Azure build pipeline. Interestingly, the very latest version (15.2.1 at the time of writing this) did not work, however the LTS version 14.15.1 did work.
Some combination of those three changes did resolve the problem for me. I am leaving this information here in case someone else faces this same error message.
In my package.json I have
"dependencies": {
"components": "link:../components",
"react": "^16.9.0",
"react-dom": "^16.9.0"
}
This works fine during development, however when trying to publish the packages to npm the "link:../components" is published into the package.
Is there a way to take the "link:../components" and replace it with the version of the package.json in the file it's being linked with?
basically
"dependencies": {
"components": "link:../components",
"react": "^16.9.0",
"react-dom": "^16.9.0"
}
would convert to
"dependencies": {
"components": "1.2.3",
"react": "^16.9.0",
"react-dom": "^16.9.0"
}
before being published to npm. The linked dependency would remain locally, though.
npm link is a development utility to test your npm packages before you publish them.
https://docs.npmjs.com/cli/link
Using your example names, say you are developing a npm package components. While you are developing it, you want to test it in your main project (The consumer of your package).
The way to go about this is, to go in your components package dir, and run npm link, this will create a global link in your system that will make it available to other projects.
So now you can go in your main project dir, and run npm link components (components is the name of your package, not to be confused with dir name)
This is fine during development, but of course won't work when you publish your packages.
Is there a way to take the "link:../components" and replace it with the version of >the package.json in the file it's being linked with?
First publish components package to npm (A good
article
that explains it)
In your main project dir, run npm unlink
components.
This will remove the global link to components in your
project dir.
In your main project dir, run npm install components.
This will fetch components from npm.
I am creating a nodejs project and wish to have same library versions used across dev, qa, staging and production environment.
Tried package.json but was unable to find good help. Need some suggestion here.
EDITED
{
"dependencies": {
"ioredis": "1.7.5",
"redis": "0.12.1",
"redis-hash": "0.0.4",
"debug": "*"
}
}
Specify your dependencies using the exact versions in the package.json file, and when you run npm install it will get the proper versions of all your dependencies.
If you don't specifiy an exact version for the libraries, there's a chance that there's a new version since the last time you ran the command, you will end up with different versions of your libraries on your servers, which might not be what you want.
NPM has an outdated command which will list all the dependencies that have fallen behind and need to be updated. You can then run npm install.
I have a module called 'sails-mongo' and I want to update it to the newest version using the following command:
npm update sails-mongo --save
I also tried uninstall then install again. I tried sails-mongo#latest and sails-mongo#beta.
Problem:
The current version (master) on GitHub the package.json (https://github.com/balderdashy/sails-mongo/blob/master/package.json) file has:
"dependencies": {
"async": "~0.2.9",
"lodash": "~2.4.1",
"mongodb": "1.4.2",
"waterline-errors": "~0.10.0"
},
And in the one being updated
"dependencies": {
"async": "0.2.10",
"underscore": "1.5.2",
"underscore.string": "2.3.3",
"mongodb": "~1.3.23"
},
The only way I get the master branch is using the command npm install git+https://github.com/balderdashy/sails-mongo
Why doesn't sails-mongo#latest install the master branch?
By default, NPM dependencies are pulled from the NPM repository. Authors must manually upload new versions of their software to the NPM repository, so the "#latest" version of the code hosted on NPM is different from the latest version of the code that exists anywhere (e.g., on GitHub).
According to the NPM repository's info page on Sails, the latest NPM-hosted version is 0.9.16 while the current GitHub version is 0.10.0-rc3.
If you want to have your project depend upon a particular branch or commit of a particular Git repo (instead of the version(s) hosted on the NPM repository), the NPM developers have included an explicit mechanism to allow this, detailed in "Git URLs as Dependencies" in the package.json docs:
Git URLs as Dependencies
Git urls can be of the form:
git://github.com/user/project.git#commit-ish
git+ssh://user#hostname:project.git#commit-ish
git+ssh://user#hostname/project.git#commit-ish
git+http://user#hostname/project/blah.git#commit-ish
git+https://user#hostname/project/blah.git#commit-ish
The commit-ish can be any tag, sha, or branch which can be supplied as an argument to git checkout. The default is master.
In fact, it's easier still to use a Github.com repo as a dependency:
As of version 1.1.65, you can refer to GitHub urls as just "foo": "user/foo-project". For example:
{
"name": "foo",
"version": "0.0.0",
"dependencies": {
"express": "visionmedia/express"
}
}
So, to use the Sails GitHub repo, simply use:
"dependencies": {
"sails": "balderdashy/sails-mongo",
...
}
And to use the exact state of Sails as it exists on GitHub as of April 28, 2014, use:
"dependencies": {
"sails": "git://github.com/balderdashy/sails-mongo#b9cdce9a48",
...
}
I had a similar issue. Via the NPM Registry I was trying to get the latest from a project I saw in in GitHub, like this:
//package.json
"devDependencies": {
"foo-package": "^3.3.0",
}
But the code I got back from npm install (as observed in the node_modules/ folder) was not what I saw in GitHub repository's master branch. I was confused; as the two didn't match.
I eventually found: https://docs.npmjs.com/cli/view, which reveals some information (versions and dates) of what the NPM Registry is aware of for a particular repository.
// Console example
npm view foo-package
After confirming that what I wanted from GitHub repository's master branch wasn't in the NPM Registry, I eventually changed my approach Git URLs as Dependencies, just as #apsillers answers.