I am working on a React.js website that uses webpack and some shared code in a module to which I also have the source code. I'd like to make some tweaks to the shared library but currently it's installed from npm via a node_module. How can I set up my development environment to be able to work on the shared module but still be able to see my changes to the module live with hot reloading?
Use a local dependency, pointing to the local copy of your package:
{
"dependencies": {
"bar": "file:../foo"
}
Then build foo in watch mode.
The main app will recognize that something has changed and will hot-reload the relevant parts.
Update: Here is a more descriptive answer of how to use yarn link: https://stackoverflow.com/a/48688156/2748017
It looks like there is a built in solution to handle this with yarn.
docs: https://yarnpkg.com/cli/link
$> yarn link <destination>
I believe npm might handle this, too, just with a bit more lifting.
As far as hot reloading, I think that more-so depends on the app consuming the local dependency, but npm-watch seems like a decent solution.
Related
I run into this problem pretty frequently while developing react applications. The latest is hwid. I am using yarn to manage dependencies.
I added the module using
yarn add hwid
It added it to the package.json file and gave me no errors. When I run the application, it says it is unable to find the module. The module is there in node_modules and everything seems to be correct and in place. So I tried deleting node_modules and running yarn install. I've done this several times. I tried force cleaning the npm cache. I have run yarn remove and yarn add several times.
I am using the WebStorm IDE. It gives me no errors, and in fact, if I let it resolve the import, it finds it just fine. This seems to only happen to me in react projects. I think, but I'm not sure, that it is usually typescript modules that give me problems.
Is there a magic bullet for this? The module is a pretty critical part of my app, so if I can't resolve it using node and react's import system, I'm going to have to just copy the files into my project. I would really rather not do that for obvious reasons.
Any help is appreciated.
If it's about typescript modules, have you tries also installing types of that modulea?
E.g.yarn add #types/hwid
I'm trying to write an npm package that will be published and used as a framework in other projects. The problem is -- I can't figure out a solid workflow for working on it at the same time as working on projects that depend on it.
I know this seems super basic and that npm link solves the issue, but this is a bigger one than just being able to import one local package from another.
I have my framework package scaffolded out; let's call it gumby, It exports a function that does console.log('hello from gumby'). That's all that matters for right now.
Now I'm ready to create a project that will use gumby. Let's call this one client. I set that up too and npm link gumby so client can import from it, etc. OK cool, it's working as expected.
So now it's time to publish gumby. I run npm publish and it goes out to npm as version 0.0.1.
At this point, how do I get the published, npm-hosted version of gumby into the package.json for client? I mean, I could just delete the symlinked copy from my node_modules and then yarn add gumby, but what if I want to go back and work on it locally again? And then run it against the npm version again? And then work on it some more? And then...
You get the point, I imagine. There's no obvious way to switch between the npm copy of a package that you're working on, and the local one. There's the additional problem of how to do that without messing with your package.json too much, e.g. what if I accidentally commit to it version control with some weird file:// dependency path. Any suggestions would be much appreciated.
For local development, having the package symlinked is definitely the way to go, the idea of constantly publishing / re-installing the package sounds like a total pain.
The real issue sounds more like you’re concerned about committing a dev configuration to prod - you could address that problem with something as simple as a pre-commit hook on your VCS e.g. block if it detects any local file references in the package.json.
I've been playing around with webpack and npm registry. I created my own little project. which I published to npm registry. Everything seems to work fine, but there is one thing that worries me.
The file which is built by webpack (lib/index.js) contains almost 2k lines of code while my library is really tiny. I see that it includes two dependencies that I use which are prop-types and detect-browser. They make most of the code.
Hover after downloading these two libraries and using them in a test project I would expect them to be somehow deduped by tree-shaking or something like that, but the size of my package is still the same.
Basically, all I'm asking is: does my library includes more code than necessary? It seems like so to me.
I also tried to additionally install prop-types and detect-browser in the test project to see if bundle size will remain the same. Hower the size of my library is still the same and bundle size increased (I expected that redundant libraries will be removed or something).
How did you generate the dependency graph?
BTW your package looks fine except for the fact that your are including in you distributed code both the packages you are depending on (prop-types and detect-browser).
In order to avoid that you should mark as external those libraries in your webpack config file. Like:
externals: {
react: 'react',
prop-types: 'prop-types',
detect-browser: 'detect-browser',
}
Just a final (personal) note: give Rollup a try to bundle your libraries. :)
I've converted to npm for my build system: no gulp etc. Also no webpack, rollup etc, it's an es6 system based on modules & no bundling. Sure is simple!
Obviously I don't want to drag around a node_modules hierarchy for my run-time, front end modules. And don't want to import foo from './node_modules/god/awful/path.js'. So I'd like to have a top level directory for the run-time, front-end dependencies.
So how do I copy my "dependencies", not "devDependencies", to a top level directory for deployment?
I've got a run script that can do it but it's pretty messy and the location of the package under node_modules is not always obvious. Maybe there's a package for doing this automatically? Or a nifty npm trick of some sort?
OK, I'm starting to need this even more than before so thought I'd nag and be clearer about what I seem to be forced to do.
First of all, I use no workflow task managers, just npm run scripts in package.json.
My dependencies (npm --save .. not --save-dev) are:
"dependencies": {
"lzma": "^2.3.0",
"pako": "^1.0.0",
"three": "*"
},
.. and my scripts cli for hoisting the dependencies into a top level libs/ dir is simply a huge cp:
"build-deps": "cp
node_modules/lzma/src/lzma.js
node_modules/lzma/src/lzma_worker.js
node_modules/pako/dist/pako.min.js
node_modules/three/build/three.js
node_modules/three/build/three.min.js
node_modules/three/examples/js/controls/OrbitControls.js
node_modules/three/examples/js/controls/FlyControls.js
node_modules/three/examples/js/controls/FirstPersonControls.js
node_modules/three/examples/js/libs/stats.min.js
node_modules/three/examples/js/libs/dat.gui.min.js
libs/",
This is pretty primitive: I have to find the dependencies in node_modules (not always obvious) and add them to the list by hand. Sure makes me want fewer dependencies! :)
I know that bower is designed for "front end" dependencies (--save in npm speak). But it seems like npm would be perfect for dependencies, and yet it appears I need to be this primitive.
What am I missing here? What do you do?
This is something I wrestled with in my earliest forays into development with Node packages. I had the exact same idea you had: "I can use npm to pull in whatever I need for my project, great!" and then trying to manage how I reference each of my dependent libraries. I didn't properly grasp at the time that npm wasn't just there to help me fetch my dependencies, it's also there to help me manage them.
Here's the long & short of it: you do NOT want to be copying anything out of your node_modules folder to some other, more human-convenient location. There's a lot of reasons why, but the biggest is that you don't need to copy anything out of node_modules -- everything your project needs is right there.
When you're developing in ECMAScript 2015+ you should only ever have to do the following (apologies for the overly-simplistic code):
/* N.B. These all reside under node_modules, yet I don't
* have to spell out their paths under node_modules: */
import $ from 'jquery';
import _ from 'lodash';
import moment from 'moment';
import NiftyLibrary from 'niftywhatever';
// ... code ...
let $name = $('#name');
let now = moment();
// ... other code ...
In other words, your development environment setup should just handle the module resolution for you. You should never have to specify that the jQuery library you want to use is at "node_modules/jquery/dist/jquery.min.js". If you're doing this you should take a minute to figure out why you're doing this -- it's unnecessary, it's a time- and brain-suck and you'd rather be writing your application code than managing your dependencies ... NOT managing the node_modules tree.
You mentioned you're developing with ES6 modules, but not using webpack, gulp, Grunt, rollup or any other build or bundling tool. Is your project intended to run entirely in Node? I ask because the last I heard most browsers aren't quite ready to run ES6 modules natively. So how are your modules getting transpiled to ES5? Maybe you're approaching this in some novel way I haven't heard of yet, but in my experience a build or bundling tool is necessary. (Also, it's a heck of a lot of fun.)
I've used Grunt with RequireJS in the past, but am now using webpack 3 with Babel and a few additional loaders (depending on the type of project I'm working on). I use npm scripts to handle my top-level tasks (running a development server, building a finished distribution package, running tests, etc.), but I let webpack handle all the business of transpiling ES6 into ES5, translating Sass styles, precompiling Vue components, etc. It's a bit of work to wrap your mind around the webpack approach, but it's well worth the effort.
Maybe webpack doesn't fit your style -- fair enough. But there are a number of other tools you could use instead. They all require a little time to get acclimated to their approaches, but they should all be able to take care of module resolution for your dependencies. Once you get a build environment set up correctly it should cease being a visible part of your development workflow; you'll just reference your dependent libraries by their names, map them to a module-local variable and use them.
Is this helpful?
EDIT: This is webpack-specific, but there should be similar options available with other bundlers or build tools.
In webpack you can use the copy-webpack-plugin to copy npm-sourced dependencies to a separate folder. This can be useful within a service worker, for example, where the execution context is a little different.
What I would suggest is scoping your dependencies (#frontend/...) and symlinking the scoped folder to your top-level dir in a post install script (similar to how bower handled the transition to yarn: see https://github.com/sheerun/bower-away)
Example:
...
"dependencies": {
"#frontend/jquery": "npm:jquery#~3.4.1",
},
"engines": {
"npm": ">=6.9.0",
"node": ">=12.10.0"
},
"scripts": {
"postinstall": "node -e \"try { require('fs').symlinkSync(require('path').resolve('node_modules/#frontend'), 'static/bower', 'junction') } catch (e) { }\""
}
Requires NPM >= 6.9 for alias support.
FYI: aliases break audits
npm allows us to specify bundledDependencies, but what are the advantages of doing so? I guess if we want to make absolutely sure we get the right version even if the module we reference gets deleted, or perhaps there is a speed benefit with bundling?
Anyone know the advantages of bundledDependencies over normal dependencies?
For the quick reader : this QA is about the package.json bundledDependencies field, not about the package.
What bundledDependencies do
"bundledDependencies" are exactly what their name implies. Dependencies that should be inside your project. So the functionality is basically the same as normal dependencies. They will also be packed when running npm pack.
When to use them
Normal dependencies are usually installed from the npm registry.
Thus bundled dependencies are useful when:
you want to re-use a third party library that doesn't come from the npm registry or that was modified
you want to re-use your own projects as modules
you want to distribute some files with your module
This way, you don't have to create (and maintain) your own npm repository, but get the same benefits that you get from npm packages.
When not to use bundled dependencies
When developing, I don't think that the main point is to prevent accidental updates though. We have better tools for that, namely code repositories (git, mercurial, svn...) or now lock files.
To pin your package versions, you can use:
Option1: Use the newer NPM version 5 that comes with node 8. It uses a package-lock.json file (see the node blog and the node 8 release)
Option2: use yarn instead of npm.
It is a package manager from facebook, faster than npm and it uses a yarn.lock file. It uses the same package.json otherwise.
This is comparable to lockfiles in other package managers like Bundler
or Cargo. It’s similar to npm’s npm-shrinkwrap.json, however it’s not
lossy and it creates reproducible results.
npm actually copied that feature from yarn, amongst other things.
Option3: this was the previously recommended approach, which I do not recommend anymore. The idea was to use npm shrinkwrap most of the time, and sometimes put the whole thing, including the node_module folder, into your code repository. Or possibly use shrinkpack. The best practices at the time were discussed on the node.js blog and on the joyent developer websites.
See also
This is a bit outside the scope of the question, but I'd like to mention the last kind of dependencies (that I know of): peer dependencies. Also see this related SO question and possibly the docs of yarn on bundledDependencies.
One of the biggest problems right now with Node is how fast it is changing. This means that production systems can be very fragile and an npm update can easily break things.
Using bundledDependencies is a way to get round this issue by ensuring, as you correctly surmise, that you will always deliver the correct dependencies no matter what else may be changing.
You can also use this to bundle up your own, private bundles and deliver them with the install.
Other advantage is that you can put your internal dependencies (application components) there and then just require them in your app as if they were independent modules instead of cluttering your lib/ and publishing them to npm.
If/when they are matured to the point they could live as separate modules, you can put them on npm easily, without modifying your code.
I'm surprised I didn't see this here already, but when carefully selected, bundledDependencies can be used to produce a distributable package from npm pack that will run on a system where npm is not configured. This is helpful if you have e.g. a system that's not networked / not on the internet: bring your package over on a thumb drive (or whatever) and unpack the tarball, then npm run or node index.js and it Just Works.
Maybe there's a better way to bundle up your application to run "offline", but if there is I haven't found it.
Operationally, I look at bundledDependencies as a module's private module store, where dependencies is more public, resolved among your module and its dependencies (and sub-dependencies). Your module may rely on an older version of, say, react, but a dependency requires latest-and-greatest. Your package/install will result in your pinned version in node_modules/$yourmodule/node_modules/react, while your dependency will get their version in node_modules/react (or node_modules/$dependency/node_modules/react if they're so inclined).
A caveat: I recently ran into a dependency that did not properly configure its dependency on react, and having react in bundledDependencies caused that dependent module to fail at runtime.