I don't understand how node packages are managed in nodester. When I run nodester npm install <package-name> from CLI I don't see any packages in my app's source folder. Without these packages visible in my folder, can I use them in usual way (as if I had I installed them in my apps folder directly).
I am advised against storing packages directly in the folder since Nodester offers Node PaaS for free and it would be unkind to not optimize my app and make it use minimal space.
Secondly is there a way through which I can run the app both locally and on nodester. How can I tell git not to push the locally installed git modules. I have heard something like git ignore. How do I manage git ignore so that my local packages are not pushed on nodester?
I might not have been eloquent in framing the question as I am a newbie to node so anyone who can put my question in a better way, feel free to Edit this.
Generally the best way is to add the node_modules dir to your .gitignore file. My .gitignore looks like this for my node projects:
*.sw*
.DS_Store
node_modules
The first line ignores any Vim temp files, the second to ignore OSX .DS_Store files and the last one ignores my node_modules dir. You will need to delete your node_modules dir from your repo first using git rm if its already committed.
More explination re. gitignore files is here from GitHub.
So that will make Git disregard your node_modules, awesome. Secondly, you will need to create a package.json file. This is what tells npm (and Nodester) what your app depends on.
{
"author": "Mr Awesome", // This is your name :)
"name": "my_awesome_app", // This is your apps name
"description": "More awesome than most other apps.", // What your app does
"version" : "0.0.1", // Your apps version (increment this when you deploy)
"node": "0.6.12", // The version of node you want Nodester to run your app on
"dependencies": {
"connect" : "2.0.3", // depend on version 2.0.3 of connect
"express" : "*" // depend on the latest version of express
}
}
More information about package.json formats can be found here:
When you push to nodester should read the package.json and install your dependencies.
Hope that helps!
Related
I have some local modules that I want to include in my package to be sent to a server. Since these local packages are not on the npm registry, they need to be part of the package.
I have tried some ways to get the node_modules folder included in the package using npm pack, however it seems this is not possible at all?
Secondly I tried to list the local modules in the bundledDepencies in the package.json file and use npm pack, however this also does not include the local modules, no matter what;
{
"name": "dev",
"version": "1.0.0",
"main": "main.js",
"dependencies": {
"local-module": "file:../../local-module"
},
"bundledDependencies": [
"local-module"
]
}
How can I get these local modules included in the dev package?
The local module does contain dependencies itself, not sure if that makes things more complicated?
I had a similar issue a while back, and a good and simple solution, is just to put your local modules into private git repos (which are now free on GitHub, thanks Microsoft 🙌)
Then, in your package.json, just add:
"dependencies" : {
"name1" : "git://github.com/user/project.git#commit-ish",
"name2" : "git://github.com/user/project.git#commit-ish"
}
Source, npm docs
I believe Alicia's approach is the cleanest one. However for someone that runs into the same issue as I did, whereby a server requires a tarball, but does not have git installed, I added my local packages to a seperate folder in the project called repo and referenced them in the package.json as;
"dependencies": {
"my-local-package": "file:./repo/my-local-package"
}
There's also yalc, which creates a local store for local packages. After adding a local package to the store, you can use yalc to copy the local package into another package. You can also use it to do updates of local packages.
We're starting to adopt a monorepo setup using yarn workspaces and we'd like to have our firebase functions inside it. The repo structure is something like:
repo
node_modules <- all dependencies
packages
core
commom
functions <- firebase functions
So, I have 2 problems with this setup:
The dependencies of the functions don't live on the same folder as the entry file from functions
The functions depends on other packages such as core and commom that are in the repo so yarn symlinks from node_modules to the packages in the repo.
Is there anyway I can handle this?
With Yarn 2 node_modules aren't fetched and placed into in the respective functions directory (as it would be the case with calling npm i in the functions directory). So when calling firebase deploy --project default --only function the node_modules folder is missing and firebase will complain about this and abort the deployment process with the following error (or similar):
Error parsing triggers: Cannot find module [...]
Try running "npm install" in your functions directory before deploying.
There are two github issues that are tracking this issue at the moment:
Support mono-repos in deployment
Functions deployment fails when firebase-functions has been hoisted by a monorepo manager like yarn/workspaces or lerna
In the two issues above, several clever workarounds are presented by firebase users, e.g. using webpack to create a build that contains all the local packages in the release or using rsync or other tools that rewire the packages before release.
Another solution is not hoisting your project packages, if that is possible. You can do this, be adding the following two directives to your .yarnrc.yml file.
# yarnrc.yml
# disables yarn's plugnplay style and uses node_modules instead
nodeLinker: node-modules
# makes sure the node_modules are not hoisted to the (monorepo) project root
nmHoistingLimits: "dependencies"
The two directives above are explained in the yarnrc configuration docs as follows:
nmHoistingLimits Defines the highest point where packages can be hoisted. One of workspaces (don't hoist packages past the workspace that depends on them), dependencies (packages aren't hoisted past the direct dependencies for each workspace), or none (the default, packages are hoisted as much as possible). This setting can be overriden per-workspace through the installConfig.hoistingLimits field.
nodeLinker Defines what linker should be used for installing Node packages (useful to enable the node-modules plugin), one of: pnp, node-modules.
The solution I found for this is Yarn's nohoist option in your root package.json file.
By default Yarn hoists dependencies to the root directory so they can be shared between your packages. Unfortunately this will not work with Firebase. This means you need to tell Yarn not to hoist the dependencies used by your Firebase functions.
The documentation for nohoist is less than ideal, but here is an official blog post about it here:
https://yarnpkg.com/blog/2018/02/15/nohoist/
You probably want something like this:
{
"workspaces": {
"packages": [
"packages/*"
],
"nohoist": [
"functions/core",
"functions/common",
"functions/**"
]
}
}
Keep in mind that this uses the name field used in the package.json files of each workspace package. So in this example, it is assume that the functions directory has a package.json with "functions" as it's name.
functions/** tells yarn not to hoist any of the dependencies specified in packages/functions/package.json. This doesn't work for your shared yarn packages though, so functions/core and functions/common need to be specified separately.
You also need to include your workspaces as dependencies in your functions project, so add them to your package.json:
{
"name": "functions",
"dependencies": {
"core": "*",
"common": "*",
}
}
Once you have added all that, you should delete your packages/functions/node_modules directory and run yarn install. After doing this, you should see all your dependencies included in packages/functions/node_modules (not symlinks).
I am not sure I understand the question exactly, but I could give you my two cents on yarn workspaces based on whatever I understood from your question and from my experience using it.
Yarn workspaces consolidate all your dependencies into the node_modules present in project root as well as in a single package-lock.json to reduce conflicts and enables yarn to optimize the installation process giving you a faster yarn install. And also another advantage of it is, with a single pass yarn install can install dependencies of all packages under the workspace.
Edit: I think for some reason yarn link is not being called and instead only yarn install is being run, which will search the npm registries and throws the error mentioned in comment since it can't find the mentioned package on npm registry. So for a solution try creating an entry in the firebase's package.json like
"dependencies": {
"a": "file:../dependency-package-name/",
}
I'm setting up a gulp build file for a node.js project and I really don't have any experience with either of them.
So basically what I'm doing is simply copying all the code to a deployment directory, but I'm unsure of how to handle all the dependencies that are stored in node_modules. Do you simply copy all of them as well, or is there a more preferred way of doing it?
gulp.task('deliver', function() {
gulp.src('src/*.html').pipe(gulp.dest('deployment/'));
gulp.src('src/*.js').pipe(gulp.dest('deployment/'));
gulp.src('src/games/').pipe(gulp.dest('deployment/'));
});
The standard way would be to have a package.json file that lists all your dependencies. Then as part of your deployment process run npm install which will go through your package.json and install any necessary packages in the node_modules folder.
I am trying to figure out the best way to handle node_modules in git. From what I read, there are two options:
A. Keep all the node_modules in the git repository, together with my project. This way, a person cloning my project does not have to install any modules.
B. Don't keep any node_modules in the git repository, i.e., have a ".gitignore" file that contains "node_modules".
However, in some projects, I don't see any of these two options. For example, in this node.js project, there are no node_modules, but also no .gitignore file...
When I fork this repo, and do npm install, the folder is filled with node_modules, and since there is no .gitignore, git tries to commit them...
What am I doing wrong?
You are not doing anything wrong, npm install will download and install all the dependencies of the project, which are defined in package.json:
"dependencies": {
"underscore" : ">=1.3.3"
},
"devDependencies" : {
"mocha" : ">=1.0.0",
"canvas" : ">=0.10.0",
"cradle" : ">=0.2.0",
"should" : ">=0.6.0",
"async" : ">=0.1.18"
}
There are many possible explanations as to how these do not appear in the source tree:
One possibility is that they are installed globally.
One other possibility is that they are actually added in .gitignore, but that .gitignore itself is never committed (this is done by adding .gitignore in the .git/info/exclude file of the project.
In any case, the only way to know why no .gitignore exists is by asking the project's owner :).
Am not an expert for this node modules stuff but one things for sure. If there is no .gitignore then no files are being ignored. This clearly means that the committer is taking care of it manually not to commit these modules.
Assumed that I have written a module for Node.js which I would like to keep private. I know that I can (should) add the line:
"private": "true"
to the package.json file, and I also know that I can npm install this module using a file system path or a link to a git repository, including GitHub.
I also know that I can put such a file system path or a link to a git repo into package.json, so that the dependencies part may look somewhat like this:
"dependencies": {
"myprivatemodule": "git#github.com:..."
}
What I now want is not to link to the latest version, but to a specific one. The only possibility I know of is to link to a specific commit using its ID. But this is way less readable and worse maintainable than using a version number such as 0.3.1.
So my question is: Is it possible to specify such a version number anyway and make npm search the git repository for the latest commit that includes this version?
If not, how do you resolve this issue in your projects? Do you live with commit IDs or is there a better solution to this?
The accepted answer did not work for me.
Here's what I'm doing to pull a package from github:
npm install --save "git://github.com/username/package.git#commit"
Or adding it manually on package.json:
"dependencies": {
"package": "git://github.com/username/package.git#commit"
}
Here's the full npm documentation:
https://docs.npmjs.com/cli/v9/configuring-npm/package-json?v=true#git-urls-as-dependencies
A dependency has to be available from the registry to be installed just by specifying a version descriptor.
You can certainly create and use your own registry instead of registry.npmjs.org if your projects shouldn't be shared publicly.
But, if it's not in a registry, it'll have to be referenced by URL or Git URL. To specify a version with a Git URL, include an appropriate <commit-ish>, such as a tag, at the end as a URL fragment.
Example, for a tag named 0.3.1:
"dependencies": {
"myprivatemodule": "git#github.com:...#0.3.1"
}
Note: The above snippet shows the base URL the same as it was posted in the question.
The snipped portion (...) should be filled in:
"myprivatemodule": "git#github.com:{owner}/{project}.git#0.3.1"
And, a different address format will be needed when SSH access isn't available:
"myprivatemodule": "git://github.com/{owner}/{project}.git#0.3.1"
Depending on your OS, you may also be able to link to the dependency in another folder where you have it cloned from Github.
If by version you mean a tag or a release, then github provides download links for those. For example, if I want to install fetch version 0.3.2 (it is not available on npm), then I add to my package.json under dependencies:
"fetch": "https://github.com/github/fetch/archive/v0.3.2.tar.gz",
The only disadvantage when compared with the commit hash approach is that a hash is guaranteed not to represent changed code, whereas a tag could be replaced. Thankfully this rarely happens.
Update:
These days the approach I use is the compact notation for a GitHub served dependency:
"dependencies": {
"package": "github:username/package#commit"
}
Where commit can be anything commitish, like a tag. In the case of GitHub you can even drop the initial github: since it's the default.
This command installs npm package username/package from specific git commit:
npm install https://github.com/username/package#3d0a21cc
Here 3d0a21cc is first 8 characters of commit hash.
My example comment to #qubyte above got chopped, so here's something that's easier to read...
The method #surjikal described above works for branch commits, but it didn't work for a tree commit I was trying include.
The archive mode also works for commits. For example, fetch # a2fbf83
npm:
npm install https://github.com/github/fetch/archive/a2fbf834773b8dc20eef83bb53d081863d3fc87f.tar.gz
yarn:
yarn add https://github.com/github/fetch/archive/a2fbf834773b8dc20eef83bb53d081863d3fc87f.tar.gz
format:
https://github.com/<owner>/<repo>/archive/<commit-id>.tar.gz
Here's the tree commit that required the /archive/ mode:
yarn add https://github.com/vuejs/vuex/archive/c3626f779b8ea902789dd1c4417cb7d7ef09b557.tar.gz
for the related vuex commit
I needed to run two versions of tfjs-core and found that both needed to be built after being installed.
package.json:
"dependencies": {
"tfjs-core-0.14.3": "git://github.com/tensorflow/tfjs-core#bb0a830b3bda1461327f083ceb3f889117209db2",
"tfjs-core-1.1.0": "git://github.com/tensorflow/tfjs-core#220660ed8b9a252f9d0847a4f4e3c76ba5188669"
}
Then:
cd node_modules/tfjs-core-0.14.3 && yarn install && yarn build-npm && cd ../../
cd node_modules/tfjs-core-1.1.0 && yarn install && yarn build-npm && cd ../../
And finally, to use the libraries:
import * as tf0143 from '../node_modules/tfjs-core-0.14.3/dist/tf-core.min.js';
import * as tf110 from '../node_modules/tfjs-core-1.1.0/dist/tf-core.min.js';
This worked great but is most certainly #hoodrat
I describe here a problem that I faced when run npm install - the package does not appear in node_modules.
The issue was that the name value in package.json of installed package was different than the name of imported package (key in package.json of my project).
So if your installed project name is some-package (name value in its package.json) then
in package.json of your project write: "some-package": "owner/some-repo#tag".
If you're doing this with more than one module and want to have more control over versions, you should look into having your own private npm registry.
This way you can npm publish your modules to your private npm registry and use package.json entries the same way you would for public modules.
https://docs.npmjs.com/files/package.json#dependencies