I have an emscripten/webassembly wrapped C++ class that launches and uses pthreads internally. It generally works fine when used in javascript on Chrome in its stand-alone development sandbox. However, I can't really get it to work with create-react-app and webpack (i.e. I used react-app-rewired to be able to configure webpack).
The problem seems to be from the complexity of the javascript and wasm files that a threaded emscripten build creates, and the confusion those scripts have due to the renaming of files that webpack produces. The files produced for a multi-threaded emscipten build are:
MyModule.js
MyModule.wasm
MyModule.worker.js
I can (I think) get it to work in the non-threaded mode where there is just a MyModule.js and a MyModule.wasm file through the use of the Module's locateFile method. However, the system gets very confused due to the introduction of the new MyModule.worker.js file which the MyModule.js file launches on its own within a WebWorker. And of course the MyModule.worker.js file needs to also find and get access to MyModule.wasm, and that gets lost too (i.e. can't find the right file name due to the webpack renaming).
Anyway, I've spent hours searching the web for any successful use cases like this, and trying may things, including manually editing the MyModule.js and MyModule.worker.js files, but without any luck so far.
Has anyone successfully tried something like this, or any other advice on getting a multi-threaded emscripten/wasm build to run properly with webpack (or create-react-app)? Any advice would be greatly appreciated.
NOTE: This question isn't about running a emscripten/webassembly module in a web-worker thread (although in the end I'm going to run all this in a web-worker). Its specifically about a multi-threaded emscripten build (i.e. C++ with pthreads) launching and running properly with webpack.
In Module's locateFile, you should return the correct path for MyModule.wasm and MyModule.worker.js.
Besides, In Module's mainScriptUrlOrBlob, you should assign the path for MyModule.js.
The mainScriptUrlOrBlob is used in worker for loading the wasm.
Related
I'm just starting to learn Angular, I installed in my Ubuntu Linux:
Angular CLI: 7.1.4. and
Node: 10.14.2.
My question is, why one project is too large? I mean a simple "helloworld" is 334MB, I don't know why, Is it possible resize it? or delete any folder that is unnecessary? Or Is there something wrong with my installation? (how can I detect that?)
The bigger folder is node_modules, and when I create my projects, generates a lot of folders.
I was reading about "angular lazy loading", but I'm new.. It is not totally clear for me.
this is the folder spaces:
Please... I hope somebody can help me...
best regards
You might be using big bundles which are not needed, so you can break them up:
https://medium.com/#julienetienne/javascript-bundles-are-too-big-break-up-the-libraries-3be87d5ef487
In modern JavaScript, projects require modules that themselves require modules that require modules... resulting in node_modules directory with hundreds of dependencies. A 100Kb library might be included because someone needed one function from it. JavaScript is not compiled, so all that source tends to be rather big. This is an unfortunate, but unavoidable truth; your Angular project directories will be big, and there's nothing you can do about it. It is perfectly normal.
The good part: modern JavaScript deployment typically includes packing up those libraries with Webpack or Parcel or similar code bundlers. Most of them implement "tree shaking", which analyses the code to find only the functions that are potentially utilised starting from the entry point, and only bundle those, ignoring the rest. This means that 100Kb library whose one function is used is transformed into just that one function in the final distribution bundle.
Not all of the bundlers are equally good at it at this point. For example, Webpack cannot tree-shake the CommonJS modules, only ES6 ones.
You can remove node_modules folder when you are not going to use the app.
And, when you need work on the application, you can re-generate node_modules using the command: npm install
Those are just node-modules, they are needed for building the project, but not necessarily everything inside of them will be deployed to production. As Amadan said, there is a process of tree-shaking (filtering only used modules) and also in production you use the minified version of the same JS code (where for example whitespace is missing and variable-names are shortened), among other optimizations. A production-optimized Angular project should not be more than a 100KB for a hello-world application.
In the provided image I see packages like
selenium-webdriver
protractor
Those belong to dev-dependencies (see your package.json file) because they are used for testing. When building for production, no code from dev-dependencies should be included. The typescript package (which is nr.2 in size in your screenshot) will also not be present in production because types like string are only used for writing Typescript code, but the browser receives Javascript, which it is compiled to.
I am working on node js application and it is now ready to use. I want to make exe of this application so that it can be used for commercial usage.
Up to now I have used enclose module using which I have compiled the code of application but I have found some issues in that (app got crash on idle condition). App is running good without enclose or compiled code.
I have searched on google and found some alternate modules like JXcore, Node webkit and Electron etc. but JX core giving error same as in SO question.
In node web-kit, it's functionality is not looking suitable as we need its executable and some dll's along with our code, which makes our package bulky.
I have also tried jxcore. The main problem with the exe's and with modules that we use is their ability to work with native modules, in my case the Kinect.node module. This module cannot be compiled. We need a workaround to package only this along with our .exe file. Enclose provides this workaround in its inbuilt functionality.
Also looking a response from EncloseJS, which is actually run by just one person who gives further instructions upon purchase. A purchase is needed for commercial usage.
In case of Electron, It is supporting only Electron-based application source code. So If I choose this then I have to modify my application code.
So can any one suggest me what can I do to make exe file from node js code there?
Thank you!
I had the same issue before, the node js application close when running in background. now i am using process manager2 (pm2), it is working fine and if the application is crash due to any other reason it is automatically started again.
I have gotten my answer:
First, reason was DiskDB database, it was not compatible with the node webkit so that is why I was getting error of native modules.
Now I am using sqlite3 module for local database. It is better than DiskDB.
Second, One reason was free version of enclose, Paid version of Enclose JS module ignores the timeout issue which I was getting.
This way I have resolved my question.
Can someone give me some advice or links for discussion on whether I should bundle JS for backend?
I tried to Google with this title (and similar words) and I can't get any useful links.
Just want to know, say I am using latest Node.JS (es6-ready), should I bundle/compile the JS? If not, how am I suppose to use typescript/flow?
Thank you.
I feel like you are asking two different questions. I'll try to answer both.
How can I just run TypeScript code?
This is the one your question's title seems to ask ("How to use typescript/flow in nodejs without compiling it"). For this, you can use the ts-node package on npm. But it's usually not a good idea to use ts-node over just compiling when running in production because it tends not to be as fast.
How should TypeScript code get distributed to be run?
Any TypeScript code will need to get compiled from .ts files to .js files to eventually be run. Basically something like the same thing applies to Flow code.
If you plan to distribute a package written in TypeScript, you should be publishing the .js and .d.ts files together. This is so that
Your package consumers don't have to recompile your package. (they already get .js files.
Your non-TypeScript consumers don't need to install TypeScript to use your package. (they already have runnable .js files)
Your TypeScript consumers can get good type safety and completions. (they get your .d.ts files)
For more information, see the TypeScript documentation on Publishing Declaration Files.
I'm trying to compile an express web API into a binary so that my source code can be protected.
I'm trying nexe. I noticed a few pitfalls such as __dirname and __filename's behavior will be modified. I certainly can modify my own code to avoid using these two variables. However, I cannot control node_modules I referenced, or I don't want to either as there're too many of them and modifying them will cause issues in subsequent version upgrade.
Do anyone has any advice on how to handle issues like this? I'm looking for possible directions:
Compile my own code into binary, not the referenced node_modules, but still keep the reference when running the binary.
A complete different way of source code protection?
I'm using windows.
You can use pkg to compile to executable.
This question already has answers here:
Is it possible to create desktop applications with node.js? [duplicate]
(5 answers)
Closed 7 years ago.
Is there a way to compile a node.js application?
I maybe very late but you can use "nexe" module that compile nodejs + your script in one executable: https://github.com/crcn/nexe
EDIT 2021: Nexe's latest release is from 2017 and it appears that development has otherwise slowed, so the more-widely-used alternative from Vercel should also be considered these days: pkg
Node.js runs on top of the V8 Javascript engine, which itself optimizes performance by compiling javascript code into native code... so no reason really for compiling then, is there?
https://developers.google.com/v8/design#mach_code
EncloseJS.
You get a fully functional binary without sources.
Native modules also supported. (must be placed in the same folder)
JavaScript code is transformed into native code at compile-time using V8 internal compiler. Hence, your sources are not required to execute the binary, and they are not packaged.
Perfectly optimized native code can be generated only at run-time based on the client's machine. Without that info EncloseJS can generate only "unoptimized" code. It runs about 2x slower than NodeJS.
Also, node.js runtime code is put inside the executable (along with your code) to support node API for your application at run-time.
Use cases:
Make a commercial version of your application without sources.
Make a demo/evaluation/trial version of your app without sources.
Make some kind of self-extracting archive or installer.
Make a closed source GUI application using node-thrust.
No need to install node and npm to deploy the compiled application.
No need to download hundreds of files via npm install to deploy your application. Deploy it as a single independent file.
Put your assets inside the executable to make it even more portable.
Test your app against new node version without installing it.
There was an answer here: Secure distribution of NodeJS applications. Raynos said: V8 allows you to pre-compile JavaScript.
You can use the Closure compiler to compile your javascript.
You can also use CoffeeScript to compile your coffeescript to javascript.
What do you want to achieve with compiling?
The task of compiling arbitrary non-blocking JavaScript down to say, C sounds very daunting.
There really isn't that much speed to be gained by compiling to C or ASM. If you want speed gain offload computation to a C program through a sub process.
Now this may include more than you need (and may not even work for command line applications in a non-graphical environment, I don't know), but there is nw.js.
It's Blink (i.e. Chromium/Webkit) + io.js (i.e. Node.js).
You can use node-webkit-builder to build native executable binaries for Linux, OS X and Windows.
If you want a GUI, that's a huge plus. You can build one with web technologies.
If you don't, specify "node-main" in the package.json (and probably "window": {"show": false} although maybe it works to just have a node-main and not a main)
I haven't tried to use it in exactly this way, just throwing it out there as a possibility. I can say it's certainly not an ideal solution for non-graphical Node.js applications.
javascript does not not have a compiler like for example Java/C(You can compare it more to languages like PHP for example). If you want to write compiled code you should read the section about addons and learn C. Although this is rather complex and I don't think you need to do this but instead just write javascript.