The situation:
client and server both share a folder shared
when we change shared in our development flow, we want the corresponding references to change in customer and server
server works because somehow with npm it seems to work, shared and server use npm
customer doesn't work and uses yarn
mixed typescript and js project
Code Structure:
root/
|- client/
|- package.json
|- src/
|- ...
|- server/
|- package.json
|- src/
|- ...
|- shared/
|- package.json // we don't want to change version every change
|- src/
|- ...
What's been tried
3 solutions proposed here
create a folder common under root and just require the files you need from your files. But you could end up with "long" require such as require("../../../../../common/file")
long require doesn't work with resolution with webpack - and isn't a nice solution
use module-alias to avoid that problem: https://github.com/ilearnio/module-alias
module-alias seems to be the same solution as the next one, in how it behaves.
you could make common a local module (using file:) and install it in package.json https://docs.npmjs.com/files/package.json#local-paths
we currently do this, but we still have to reinstall the shared folder on a change, we currently use yarn upgrade shared which takes a long time and still requires us to know when we need to run it.
in addition we've attempted to get yarn link working but doesn't seem to link properly
we've attempted to use a post-pull hook with husky to run yarn upgrade shared but it's too slow to run every pull especially since it's not needed often
we've considered a true mono-repo 'package' like Lerna but still don't think that it's necessary for the cost to migrate
link-module-alias to sym link folders on post install script, but this fails with typescript files
Goal
either
find a way to automatically sync these in dev environement
find a solution that installs/updates manually - but is fast and can be run on every pull
find a way to automate running yarn upgrade shared that runs (roughly) only when needed not all the time
I guess we could find a way to automate the version increment on any change of shared's version key, and then it's tracked, we could run yarn install and that work work.
The find solution was that we were using React-Native and therefore the normal steps for syncing would work for the IDE but not the React-Native app.
Here is a great article describing how to get metro bundler working with it - however for Typescript we added it to the TSConfig and for the IDE we still needed to add it to our package using the file:../shared directive.
Related
I would like to use a parent NPM package from inside a subfolder, as part of the development process. Let's call my library "logary".
Structure:
.
|- package.json
|- tsconfig.json
|- src
|- ... all the source code
|- dist
|- ... compiled TS as .js, .js.map and .d.ts
|- examples
|- with-nextjs
|- package.json
|- tsconfig.json
The library is written in TypeScript and so is the example. The parent compiles with outDir: ./dist in tsconfig.json. I have multiple entry points (one for the main lib, one for each peer dependency as for example: import withLogary from 'logary/plugins/nextjs'.
I would like to be able to write import Logary from 'logary'; inside ./examples/with-nextjs and have Logary, the constructor, have types attached to it be fully "resolvable" by the TypeScript compiler. I want changes in . to be reflected when ./examples/with-nextjs is reloaded.
It's not a monorepo (in the common sense). I've tried yarn link in . and then yarn link logary inside ./examples/with-nextjs, which doesn't seem to solve it (cannot import module). Furthermore, as I have bumped the version in . to 6.0.0, as I yarn install inside ./examples/with-nextjs, yarn complains that version is not published to npmjs.com and asks me to select a previous version, which it then downloads.
I know there's Lerna ("Splitting up large codebases into separate independently versioned packages is extremely useful for code sharing." — this is a tiny code-base and only has a single package, I just want to reference it in a single direction from an example) and Yarn Workspaces ("It allows you to setup multiple packages in such a way that you only need to run yarn install once to install all of them in a single pass." — which is not my aim)
I'm not interested in adding any special config (like paths) to tsconfig.json in ./examples/with-nextjs because then it's not just to copy the example to as your own app.
➡ How do I best structure a TypeScript library project with multiple peer dependencies and with an examples folder?
Maybe related?
Use yarn workspaces and typescript's project references to reference another package subdirectory — use exports in package.json to solve the logary/plugins/nextjs problem (I'm happy doing that, but I can't get the parent project properly linked with peers correctly installed, so just import Logary from 'logary' in the examples uses the old published version, despite me having run yarn link)
Peer Dependencies In A Monorepo — this is probably a good solution to the peer dependencies problem (e.g. with https://www.npmjs.com/package/rollup-plugin-peer-deps-external)
Using Lerna with unpublished packages — has a similar question, but doesn't ask about consume-only app packages like my with-nextjs. Also, my root is at . not at packages/logary; but maybe I must conform?
Using composite: true in tsconfig.json in . and also references in the child project's tsconfig.json changes how the outDir works; it adds the src/ folder as prefix to all output paths, which is not what I want
i have a project with two nodeJS projects inside, a backend project with inside a folder "client" for the frontend.
Now i would like to make one common node_moduels folder both.
I found that PNPM could do that, but for me the documentation is not so clear. How doest it work?
How can i run pnpm i on my root folder and then make my node_modules accessable for my "client" folder?
Thank you!
On Linux or Mac, i usually symlink the module (ln -s) into my other projects' libs. Then i import or require the module from there. It's located directly in your codebase, not in node_modules, but it works just fine.
Just don't update the common module without keeping in mind all the dependant projects.
For easy versioning, you may choose to publish your module to npm (even as private), and update it with each push for each of the dependants.
My higher goal is to create a .zip file with the server bundle and all needed assets.
That file can be then moved to another device and executed there (the other device has the same NodeJS version installed). Building this on the device is not an option, because it's offline.
build/
|- node_modules //this is where I would like to have my externals
|- server.bundle.js
|- public/
My current solution is to pick and copy the external modules by hand, which is neither scalable nor convenient.
Would there be a possiblity, to let webpack handle this for me? Or even bundling externals like express into the bundle itself.
You could have just express in the dependencyarray on package.json, then run npm install --productionto just install that, so your node_modules is going to be small.
In the webpack config, you have to be sure to add externals (webpack_externals). Due to some binaries in some node dependencies, webpack is not able to bundle them together. You are going to have a require call for just those dependencies.
Recently started working with Gulp and I can't figure out is it really necessary to have a copy of node_modules directly in folder with current project?
E.g. I have this structure:
mysite
└─builder
└──node_modules
└─work
└─work2
How can I access node_modules in folder 'builder' from folder 'work' or 'work2' without copying it? It is quite large, about 100mb, and seems to me it has no sense to have a copy of it for every new project.
I tried this line export NODE_PATH='D:\OpenServer\domains\mysite\build' in file package.json and then tried command gulp but it replied[10:24:27] Local gulp not found in d:\OpenServer\domains\mysite\work
[10:24:27] Try running: npm install gulp
Short answer
Don't do it. Let NPM work the way it's designed to. However, to save space, you can delete the node_modules folder on projects that are currently dormant, and recreate it with a single shot of npm install when you switch back to them.
Justification
Even if you share your node_modules, you'll probably have redundancies in it anyway. What will you do about them next ?
It is the essence of NPM to replicate modules per project. If you dig into the node_modules folder tree, you may notice that it can even contain several replications of a same library under one given dependencies tree. Say you requested two modules explicitely, and both these modules themselves pulled a dependency that takes care of a lot of things, and is therefore called lib_DADDYMUMMY :
node_modules
+ a_module_you_use v0.5
+ lib_DADDYMUMMY v0.1 (pulled as a dependency of this module)
+ another_module_that_you_requested v0.3
+ lib_DADDYMUMMY v0.1 (again ! pulled as a dependency of this other module)
This comes in handy when your two module start needing different versions of lib_DADDYMUMMY. This comes in handy when you maintain long-lived projects ! And hell knows that in the JavaScript world, with fast changing APIs, you can consider most any decent project as long-lived. :)
One could imagine having all dependencies being shared by everyone, living in a flat structure, with several versions of a library living next to each other and every one finding what he needs there. That repository could be called, say, .m2. But that's just not the way NPM works unfortunately.
NPM considers that storage space is cheap. That's its price for helping you manage versions in dependencies, dependencies of dependencies, and dependencies of dependencies of dependencies. I consider that it's an affordable price for taking care of the dirty jobs the day when work and work2, as their lives go on, take diverging maintenance paths. I wouldn't try getting in its way by forcing a half-Maven-like folder model.
Maybe you should put your package.json into your root directory(mysite/package.json),
then try to install node_modules on the root.
In addition, you write gulpfile on the same dir.
eg.
mysite
|- package.json
|- node_modules
|- gulpfile.js
└─builder
└─work
└─work2
However, I recommend that you write one single gulpfile for each project.
One problem why you shouldn't do this is because of versioning. If your modules require different versions of the same package, you're going to run into problems. One package is going to win, and it might break another package.
Further, you get into the problem of having to merge the dependency lists in some way - meaning, you'll have to get the dependencies from work/package.json, work2/package.json, etc. and then install all of them at once.
Merging node_modules/ won't solve your problem, either - believe me, don't try.
Paste the node_modules folder inside your mySite directory.
All npm packages such as gulp will work in your work or work2 directory.
But, now(your folder structure) work folders can't find node_modules in their parent directory.
I have a project in which I use node-webkit. node-webkit allows npm packages to be used for developing desktop applications. I make use of grunt to build my application.
My folder structure looks like this at the moment:
project root
node_modules/ (1)
package.json (1)
App/
node_modules/ (2)
package.json (2)
bower.json
bower_components/
...
controllers/
filters/
...
app.js
The npm dependencies for the application itself are kept within the App folder, but the dev dependencies for building the project are not related to the application source code, so i keep them in node_modules (1) inside the root folder. I also know that in a package.json file one can express dependencies and dev dependencies, exactly for this reason. I would rather have one package.json file in the root expressing ALL dependencies, including dev dependencies, but i would rather have a separation of those dependencies on folder level.
Two questions arise:
Is this a good way to organize my npm dependencies? If yes, awesome? If no, which I expect:
What is a better way to organize my dependencies? Is it possible to specify that dev dependencies go into folder a, and 'regular' dependencies go into folder b? If so, how do I do this?
In case anyone is wondering, this is the project i am talking about:
https://github.com/michahell/pinbored-webkit
[updated folder structure to include app.js for clarity]
It is perfectly fine to keep more than one package.json file and multiple node_module directories for a project. If you consider the parts as separate components.
An example might be if, you have one directory containing a node server, another containing a react app, and a third containing some kind of deployment script written in javascript.
#Michael package.json file contains all the dependencies related to that project.There is no need for multiple package files and multiple node_modules folders..
But you need to check where is your App.js file!!
your App.js , package.json must be in same folder unless configured.