I am trying to package up some modules that I have been working on. I have five modules, split in to five files. Four of them are the actual outward-facing modules that I want the user to be able to install. The other one is a support module that they all need to function correctly. They are all stored in the same directory. I want to be able to specify each as a separate module in the same directory. But as far as I can tell, one can only define a single module in package.json.
Is there a way to specify multiple modules? If not, that means this must be a bad practice. How should I structure my module's exports to move it in to one main module?
Currently there's not a supported way of having a separate package.json file for each module you'll be publishing within the same directory. And really, this makes sense, as each package you deploy may have issues, feature requests, bugs, etc that need to be handled separately and don't force updates of the others. Separating these out will allow you to focus on the maintenance of each independently, and also allow the consumers of these modules to include them separately. A lot of larger scale projects who have started by creating something they think people will like, end up having the thing that everyone actually use be the random sub-project that was created separately.
So separate directories, and separate package.json files, then include dependencies within the package.json for each. If you haven't already seen there's a couple good writeups to help development of node packages here:
https://docs.npmjs.com/about-packages-and-modules
https://docs.npmjs.com/creating-a-package-json-file
https://docs.npmjs.com/using-npm-packages-in-your-projects
Related
I am creating my first open source project, and I am making some plugins for it. These plugins will be published as npm packages, and they will have identical dependencies.
My question is, what I the best way to deliver them and avoid code repetition? I know I can use something like Rollup.js to pack all dependencies used by that module in the final distribution js file, but if the user is using multiple modules, the inlined dependencies will be repeated and make the file bloat.
I know end user can use a bundler to remove those repeated codes, but is there anything more I can do to reduce the size of my distribution js files?
I have a simple local command-line module that I want to use in 2 different ways (basically different defaults), but it uses the same core logic, so I want to extract that logic into a third entity and use that from the two entry points.
I have everything working with two bin command scripts, but each file has its own copy of the logic to run, and I am not sure how to pull this duplicated code out into a third file within the same module. I figure I could do it by creating an entire separate module and loading it with require(), but I would rather just keep it together since it's tightly coupled.
The structure is like this:
bin\
cmdone.js
cmdtwo.js
core.js
package.json
I would like to move the logic, which currently exists in both cmdone.js and cmdtwo.js, into core.js and reference it from the two files in bin. Is this possible?
If i understand your question correct, then what you need is require function of nodejs
Well, after some more poking around, I discovered that this works:
const test = require('../core.js');
I suppose I misunderstood the distinction between Node modules and NPM packages. I was basically equating the two, but it seems that you can create and use modules entirely within packages, they don't have to be one-to-one.
I have made a module for Node.js, which is part native, a C++ compiled library and half-JavaScript - about 10 *.js files. How should I distribute that? As single module or separately?
Depends on the use case I suppose, but as one module is probably fine. Unless you'd like to maintain them in separate repositories, you've got other modules that might prefer depending on them separately, or other otherwise and then it is just a matter of adding one to the other's package.json dependency list.
I need to obfuscate my source code as best as possible so I decided to use uglifyjs2.. Now I have the project structure that has nested directories, how can I run it through uglifyjs2 to do the whole project instead of giving it all the input files?
I wouldn't mind if it minified the whole project into a single file or something
I've done something very similar to this in a project I worked on. You have two options:
Leave the files in their directory structure.
This is by far the easier option, but provides a much lower level of obfuscation since someone interested enough in your code basically has a copy of the logical organization of files.
An attacker can simply pretty-print all the files and rename the obfuscated variable names in each file until they have an understanding of what is going on.
To do this, use fs.readdir and fs.stat to recursively go through folders, read in every .js file and output the mangled code.
Compile everything into a single JS file.
This is much more difficult for you to implement, but does make life harder on an attacker since they no longer have the benefit of your project's organization.
Your main problem is reconciling your require calls with files that no longer exist (since everything is now in the same file).
I did this by using Uglify to perform static analysis of my source code by analyzing the AST for calls to require. I then loaded the source code of the required file and repeated.
Once all code was loaded, I replaced the require calls with calls to a custom function, wrapped each file's source code in a function that emulates how node's module system works, and then mangled everything and compiled it into a single file.
My custom require function does most of what node's require does except that rather than searching the disk for a module, it searches the wrapper functions.
Unfortunately, I can't really share any code for #2 since it was part of a proprietary project, but the gist is:
Parse the source text into an AST using UglifyJS.parse.
Use the TreeWalker to visit every node of the AST and check if
node instanceof UglifyJS.AST_Call && node.start.value == 'require'
As I have just completed a huge pure Nodejs project in 80+ files I had the same problem as OP. I needed at least a minimal protection for my hard work, but it seems this very basic need had not been covered by the NPMjs OS community. Add salt to injury the JXCore package encryption system was cracked last week in a few hours so back to obfuscation...
So I created the complete solution, that handles file merging, uglifying. You have the option of leaving out specified files/folders as well from merging. These files are then copied to the new output location of the merged file and references to them are rewritten auto.
NPMjs link of node-uglifier
Github repo of of node-uglifier
PS: I would be glad if people would contribute to make it even better. This is a war between thieves and hard working coders like yourself. Lets join our forces, increase the pain of reverse engineering!
This isn't supported natively by uglifyjs2.
Consider using webpack to package up your entire app into a single minified .js file, excluding node_modules:
http://jlongster.com/Backend-Apps-with-Webpack--Part-I
I had the same need - for which I created node-optimize and grunt-node-optimize.
https://www.npmjs.com/package/grunt-node-optimize
Is there any sort of notion of packaging typescript files currently?
One thing im finding a pain at the moment when trying to migrate a pure javascript project over to typescript is the references, in some cases where I have complex objects I am having to write several reference statements pulling files from all over the place.
Part of this is down to my project layout, as its a pretty big and modular one, so I have a system like this:
- modules
|- module1
|- models
|- services
|- controllers
|- module2
|- models
|- services
|- controllers
|- core
|- models
|- services
|- data
|- validation
There is much more, but you get the point, now currently core is used by every module, but with javascript I just expect it to be loaded in at runtime, which will still need to happen, however as the typescript concerns are only really at compile time I was wondering if there was some notion of packaging all typescript files up into some typescript library or something, and then that could be referenced from projects rather than having module1 models referencing core models etc.
The problem currently revolves around the directory structures, as the namespaces work fine but if I move a file I need to go to every file which references that moved file and update it. Which is tiresome, whereas if there is some sort of package idea then I could just reference that once its output, so im no longer worrying about file systems and directories, im just worrying about a package and namespaces.
I think a lot of this is very similar to how C# works, you have a project which has references. Then every file within that project can use any of the classes within the references, so the code exposure is managed by references and namespaces.
I am thinking about having my build script just make a local references.ts file and just loop through every *.ts file in the relavant module and put them into one big file:
///<reference path="core/models/some-model.ts"/>
///<reference path="core/models/some-model-2.ts"/>
///<reference path="core/services/some-service.ts"/>
like shown above, then using this reference file in all typescript files which require core files, so this acts as a kind of project level reference, this may mean some files have references they dont need, but its compile time so I dont really care...
I dont want to go hand rolling my own solution to this problem if a good way already exists, Hope that makes sense...
== EDIT ==
I just wanted to post this up here as for my scenario has saved me TONS of time and has also reduced my reference guff by like 99%, this wont be applicable for people who don't have build scripts though.
Right now assuming you do have a build script I took the path of having a step in my script which went through every single file within a root level directory (module1, module2 etc in this case) and it would then output a local.references.ts into a references folder within that directory. Then I manually have written an external.references.ts which references external descriptors or other modules references wherever needed.
After this part was done when I am compiling my typescript I basically point it again at the root directories and tell it to compile them all (*/.ts) into one big js file (i.e module1.js). Now because this will automatically include the local and external references, I dont need to put ANY reference declarations in the individual clases.
So this way providing the local and external reference files (local.reference.ts, external.reference.ts) are included within the bulk processing of the files you just have to worry about namespaces, making it pretty much the same as how C# would operate.
If however you do not have a build script which is able to do your local reference generation and compilation of typescript then the comment link given would be a good option.
Currently there is no formal process for packaging your source typescript files as a pre built library as you are describing.
There are a few different solutions currently used much like the one linked in the comments which will allow you to put all your references into central typescript files then just reference them from your individual scripts, or the approach you put forward where you do the same sort of approach but rather than manually writing it you get your build script to generate the references for you and get the compilation process to inject the references in rather than explicitly referencing them in each file.
As Typescript gets more mature there may be more formal ways of doing this, but for the moment just take whichever solution works best given your tooling and appraoch to developing with Typescript.