Setting max-old-space-size in Azure DevOps pipeline - node.js

My "IIS Web App Deploy" starts failing to deploy zip package when its size exceeded 200MB. The error is FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
The solution must be to increase node memory with max-old-space-size option e.g. [Environment]::SetEnvironmentVariable("NODE_OPTIONS", "--max-old-space-size=16384", "Machine")
However, whatever I do does not help. Always the same error. I tried setting env variable and can confirm that it changes (I check env variables via My computer on Windows). I also tried set NODE_OPTIONS=--max_old_space_size=8192 but IIS Web App Deploy task always fails with the same error message.
I added a Power Shell script to check node memory size with simple js script
const v8 = require('v8');
const totalHeapSize = v8.getHeapStatistics().total_available_size;
const totalHeapSizeGb = (totalHeapSize / 1024 / 1024 / 1024).toFixed(2);
console.log('totalHeapSizeGb: ', totalHeapSizeGb);
and if I run this script locally it shows amended size but if I run it as a task in the pipeline it ways shows default totalHeapSizeGb: 1.39
My question is How to change max-old-space-size for node so IIS Web App Deploy task use it instead of default value and stops failing?

increase-memory-limit is a workaround to fix heap out of memory when running node binaries. It's a common issue when using TypeScript 2.1+ and webpack.
How to use
npm install -g increase-memory-limit
Run from the root location of your project:
increase-memory-limit
Running from NPM task
Alternatively, you can configure a npm task to run the fix
// ...
"scripts": {
"fix-memory-limit": "cross-env LIMIT=2048 increase-memory-limit"
},
"devDependencies": {
"increase-memory-limit": "^1.0.3",
"cross-env": "^5.0.5"
}
// ...
npm run fix-memory-limit
Here is the document you can refer to.

Related

Cannot connect to my databse addon on heroku

I attempted to use the command Heroku pg:psql to connect to my database addon in heroku but got a response below
--> Connecting to postgresql-regular-61345
unrecognized win32 error code: 123could not find a "psql" to execute
unrecognized win32 error code: 123could not find a "psql" to execute
psql: fatal: could not find own program executable
! psql exited with code 1
After using the heroku logs --tail command i got the following errors
sh: 1: nodemon: not found
Process exited with status 127
State changed from starting to crashed
I can also see all processes stopping with SIGTERM and the process exiting with status 143
Resolution steps I have taken
Verified that the environment variables have the path for installed postgress14 on my PC
Added a procfile to the root file in my backend code and spcified "web: node matthewfaceappback/server.js in the file"
Changed my set port to a variable port using process.env.PORT || 3000
Set all environment variable including my database url(set by default) on config variable in heroku
Verified there is a start up script
Updated all my packages using "npm update". after doing this i started expereincing the issue of processes stopping with SIGTERM and the process exiting with status 143
I moved nodemon from devDependencies to dependencies. nodemon version is 2.0.15
In package.json i inputed an engines parameter using the version of node in my case
{"engines": {
"node": "14.17.4"
}}
I restarted heroku using "heroku restart"
Below are links to the screenshots of the error
https://www.dropbox.com/s/5bdbyi9e99lbxhu/pic1.PNG?dl=0
https://www.dropbox.com/s/41euniaes5q68c9/pic2.PNG?dl=0
https://www.dropbox.com/s/50oqzbwmwrqogax/pic3.PNG?dl=0
Put nodemon back in the devDependencies and add it as a second node script in package.json:
"scripts": {
"start": "node matthewfaceappback/server.js",
"dev": "nodemon matthewfaceappback/server.js"
},
These two errors are completely unrelated.
The database connection error
The first issue, which I believe is the one you actually care about at the moment based on the title of the question, indicates that the Heroku CLI can't find a PostgreSQL client on your local machine.
The documentation makes the following recommendation
Set up Postgres on Windows
Install Postgres on Windows by using the Windows installer.
Remember to update your PATH environment variable to add the bin directory of your Postgres installation. The directory is similar to: C:\Program Files\PostgreSQL\<VERSION>\bin. Commands like heroku pg:psql depend on the PATH and do not work if the PATH is incorrect.
If you haven't already installed Postgres locally, do so. (This is a good idea anyway as you should be developing locally and you'll probably need a database.)
Then make sure to add its bin/ directory to your PATH environment variable.
The Nodemon error
The second issue is because you are trying to use nodemon in production. Heroku strips development dependencies out of Node.js applications after building them, which normally makes sense. Nodemon is a development tool, not something that should be used for production hosting.
Depending on the contents of your package.json, this might be as simple as changing your start script from nodemon some-script.js to node some-script.js. Alternatively, you can add a Procfile with the command you actually want to run on Heroku:
web: node some-script.js
See also Need help deploying a RESTful API created with MongoDB Atlas and Express

ENOMEM error when starting Next.js app on shared hosting server

TL;DR: I got an spawn ENOMEM error when trying to build and run my Next.js app on a shared hosting server, which had over 900MB RAM and 70+ processes available at the time this happened. The log showed that the RSS size was 50814976 when the error was caught.
I am not quite sure if the error is simply caused by insufficient memory or it occurs because of incorrect settings or configs. Could you please give me some advice?
==========Details below============
I am building a Next.js app with Node.js, and it’s built with a custom server (my entry point: server.js). I can run my app in a local environment on localhost:3000. Then I try deploying it to check if it’s ok on the network.
I have subscribed to A2 Hosting’s DRIVE Web Shared Hosting Plan, which is optimised to support Node.js environment. What they offer in the plan are 1GB of physical memory and 75 available processes.
My application was created through the Setup Node.js App function on the cPanel, and cloned my project into the server with git via SSH. NPM packages were also installed on the server with the command npm install. The node version was v.12.9.1 and npm version was 6.14.8.
My npm scripts were defined to run the custom server. Here were the npm scripts defined in my package.json file:
"scripts": {
"dev": "node server.js"
"build": "next build",
"start": "NODE_ENV=production node server.js"
}
Then I used the command npm run build to create the production application in the .next folder, but an error spawn ENOMEM came up immediately.
I googled that it was a memory usage related issue. Some said this error occurred when memory was not enough, and the workaround to bypass this was to build the production folder locally and upload it to the server. So I copied the steps.
However, the result was frustrating when I ran the command npm run start. The error ENOMEM was still here, coming up in less than a second after I entered the command.
Cleaning the npm cache and reinstalling the npm modules didn’t seem to work either.
I tried increasing the memory limit by adding option --max-old-space-size in the command and ran NODE_ENV=production node --max-old-space-size=1024 server.js; but unfortunately this didn’t seem to work and the ENOMEM still popped up.
I added console.log(process.memoryUsage()) to show the usage when an error was caught and this was the result:
{
rss: 50814976,
heapTotal: 34107392,
heapUsed: 23076064,
external: 1632450
}
The total rss size was far less than the limit. Or did I use a wrong method to inspect the memory consumption?
How can I solve the ENOMEM problem? What exactly the error is caused by? Is it really just because the available RAM doesn't meet the requirement of a next.js app?
I am not sure if I have applied incorrect settings, overlooked some important configs, or miswritten any codes that bring about this error. I want to figure out what is going on underneath. Upgrading the plan impulsively without adequate understanding isn’t good for me as a newbie developer, and it's my responsibility to make good use of the budget.
Could you please give me some advice?
Thank you for your attention.

How do you increase the memory limit of Vue UI [duplicate]

Today I ran my script for filesystem indexing to refresh RAID files index and after 4h it crashed with following error:
[md5:] 241613/241627 97.5%
[md5:] 241614/241627 97.5%
[md5:] 241625/241627 98.1%
Creating missing list... (79570 files missing)
Creating new files list... (241627 new files)
<--- Last few GCs --->
11629672 ms: Mark-sweep 1174.6 (1426.5) -> 1172.4 (1418.3) MB, 659.9 / 0 ms [allocation failure] [GC in old space requested].
11630371 ms: Mark-sweep 1172.4 (1418.3) -> 1172.4 (1411.3) MB, 698.9 / 0 ms [allocation failure] [GC in old space requested].
11631105 ms: Mark-sweep 1172.4 (1411.3) -> 1172.4 (1389.3) MB, 733.5 / 0 ms [last resort gc].
11631778 ms: Mark-sweep 1172.4 (1389.3) -> 1172.4 (1368.3) MB, 673.6 / 0 ms [last resort gc].
<--- JS stacktrace --->
==== JS stack trace =========================================
Security context: 0x3d1d329c9e59 <JS Object>
1: SparseJoinWithSeparatorJS(aka SparseJoinWithSeparatorJS) [native array.js:~84] [pc=0x3629ef689ad0] (this=0x3d1d32904189 <undefined>,w=0x2b690ce91071 <JS Array[241627]>,L=241627,M=0x3d1d329b4a11 <JS Function ConvertToString (SharedFunctionInfo 0x3d1d3294ef79)>,N=0x7c953bf4d49 <String[4]\: ,\n >)
2: Join(aka Join) [native array.js:143] [pc=0x3629ef616696] (this=0x3d1d32904189 <undefin...
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
1: node::Abort() [/usr/bin/node]
2: 0xe2c5fc [/usr/bin/node]
3: v8::Utils::ReportApiFailure(char const*, char const*) [/usr/bin/node]
4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [/usr/bin/node]
5: v8::internal::Factory::NewRawTwoByteString(int, v8::internal::PretenureFlag) [/usr/bin/node]
6: v8::internal::Runtime_SparseJoinWithSeparator(int, v8::internal::Object**, v8::internal::Isolate*) [/usr/bin/node]
7: 0x3629ef50961b
Server is equipped with 16gb RAM and 24gb SSD swap. I highly doubt my script exceeded 36gb of memory. At least it shouldn't
Script creates index of files stored as Array of Objects with files metadata (modification dates, permissions, etc, no big data)
Here's full script code:
http://pastebin.com/mjaD76c3
I've already experiend weird node issues in the past with this script what forced me eg. split index into multiple files as node was glitching when working on such big files as String. Is there any way to improve nodejs memory management with huge datasets?
If I remember correctly, there is a strict standard limit for the memory usage in V8 of around 1.7 GB, if you do not increase it manually.
In one of our products we followed this solution in our deploy script:
node --max-old-space-size=4096 yourFile.js
There would also be a new space command but as I read here: a-tour-of-v8-garbage-collection the new space only collects the newly created short-term data and the old space contains all referenced data structures which should be in your case the best option.
If you want to increase the memory usage of the node globally - not only single script, you can export environment variable, like this:
export NODE_OPTIONS=--max_old_space_size=4096
Then you do not need to play with files when running builds like
npm run build.
Just in case anyone runs into this in an environment where they cannot set node properties directly (in my case a build tool):
NODE_OPTIONS="--max-old-space-size=4096" node ...
You can set the node options using an environment variable if you cannot pass them on the command line.
Here are some flag values to add some additional info on how to allow more memory when you start up your node server.
1GB - 8GB
#increase to 1gb
node --max-old-space-size=1024 index.js
#increase to 2gb
node --max-old-space-size=2048 index.js
#increase to 3gb
node --max-old-space-size=3072 index.js
#increase to 4gb
node --max-old-space-size=4096 index.js
#increase to 5gb
node --max-old-space-size=5120 index.js
#increase to 6gb
node --max-old-space-size=6144 index.js
#increase to 7gb
node --max-old-space-size=7168 index.js
#increase to 8gb
node --max-old-space-size=8192 index.js
I just faced same problem with my EC2 instance t2.micro which has 1 GB memory.
I resolved the problem by creating swap file using this url and set following environment variable.
export NODE_OPTIONS=--max_old_space_size=4096
Finally the problem has gone.
I hope that would be helpful for future.
i was struggling with this even after setting --max-old-space-size.
Then i realised need to put options --max-old-space-size before the karma script.
also best to specify both syntaxes --max-old-space-size and --max_old_space_size my script for karma :
node --max-old-space-size=8192 --optimize-for-size --max-executable-size=8192 --max_old_space_size=8192 --optimize_for_size --max_executable_size=8192 node_modules/karma/bin/karma start --single-run --max_new_space_size=8192 --prod --aot
reference https://github.com/angular/angular-cli/issues/1652
I encountered this issue when trying to debug with VSCode, so just wanted to add this is how you can add the argument to your debug setup.
You can add it to the runtimeArgs property of your config in launch.json.
See example below.
{
"version": "0.2.0",
"configurations": [{
"type": "node",
"request": "launch",
"name": "Launch Program",
"program": "${workspaceRoot}\\server.js"
},
{
"type": "node",
"request": "launch",
"name": "Launch Training Script",
"program": "${workspaceRoot}\\training-script.js",
"runtimeArgs": [
"--max-old-space-size=4096"
]
}
]}
I had a similar issue while doing AOT angular build. Following commands helped me.
npm install -g increase-memory-limit
increase-memory-limit
Source: https://geeklearning.io/angular-aot-webpack-memory-trick/
I just want to add that in some systems, even increasing the node memory limit with --max-old-space-size, it's not enough and there is an OS error like this:
terminate called after throwing an instance of 'std::bad_alloc'
what(): std::bad_alloc
Aborted (core dumped)
In this case, probably is because you reached the max mmap per process.
You can check the max_map_count by running
sysctl vm.max_map_count
and increas it by running
sysctl -w vm.max_map_count=655300
and fix it to not be reset after a reboot by adding this line
vm.max_map_count=655300
in /etc/sysctl.conf file.
Check here for more info.
A good method to analyse the error is by run the process with strace
strace node --max-old-space-size=128000 my_memory_consuming_process.js
I've faced this same problem recently and came across to this thread but my problem was with React App. Below changes in the node start command solved my issues.
Syntax
node --max-old-space-size=<size> path-to/fileName.js
Example
node --max-old-space-size=16000 scripts/build.js
Why size is 16000 in max-old-space-size?
Basically, it varies depends on the allocated memory to that thread and your node settings.
How to verify and give right size?
This is basically stay in our engine v8. below code helps you to understand the Heap Size of your local node v8 engine.
const v8 = require('v8');
const totalHeapSize = v8.getHeapStatistics().total_available_size;
const totalHeapSizeGb = (totalHeapSize / 1024 / 1024 / 1024).toFixed(2);
console.log('totalHeapSizeGb: ', totalHeapSizeGb);
Steps to fix this issue (In Windows) -
Open command prompt and type %appdata% press enter
Navigate to %appdata% > npm folder
Open or Edit ng.cmd in your favorite editor
Add --max_old_space_size=8192 to the IF and ELSE block
Your node.cmd file looks like this after the change:
#IF EXIST "%~dp0\node.exe" (
"%~dp0\node.exe" "--max_old_space_size=8192" "%~dp0\node_modules\#angular\cli\bin\ng" %*
) ELSE (
#SETLOCAL
#SET PATHEXT=%PATHEXT:;.JS;=;%
node "--max_old_space_size=8192" "%~dp0\node_modules\#angular\cli\bin\ng" %*
)
Recently, in one of my project ran into same problem. Tried couple of things which anyone can try as a debugging to identify the root cause:
As everyone suggested , increase the memory limit in node by adding this command:
{
"scripts":{
"server":"node --max-old-space-size={size-value} server/index.js"
}
}
Here size-value i have defined for my application was 1536 (as my kubernetes pod memory was 2 GB limit , request 1.5 GB)
So always define the size-value based on your frontend infrastructure/architecture limit (little lesser than limit)
One strict callout here in the above command, use --max-old-space-size after node command not after the filename server/index.js.
If you have ngnix config file then check following things:
worker_connections: 16384 (for heavy frontend applications)
[nginx default is 512 connections per worker, which is too low for modern applications]
use: epoll (efficient method) [nginx supports a variety of connection processing methods]
http: add following things to free your worker from getting busy in handling some unwanted task. (client_body_timeout , reset_timeout_connection , client_header_timeout,keepalive_timeout ,send_timeout).
Remove all logging/tracking tools like APM , Kafka , UTM tracking, Prerender (SEO) etc middlewares or turn off.
Now code level debugging: In your main server file , remove unwanted console.log which is just printing a message.
Now check for every server route i.e app.get() , app.post() ... below scenarios:
data => if(data) res.send(data) // do you really need to wait for data or that api returns something in response which i have to wait for?? , If not then modify like this:
data => res.send(data) // this will not block your thread, apply everywhere where it's needed
else part: if there is no error coming then simply return res.send({}) , NO console.log here.
error part: some people define as error or err which creates confusion and mistakes. like this:
`error => { next(err) } // here err is undefined`
`err => {next(error) } // here error is undefined`
`app.get(API , (re,res) =>{
error => next(error) // here next is not defined
})`
remove winston , elastic-epm-node other unused libraries using npx depcheck command.
In the axios service file , check the methods and logging properly or not like :
if(successCB) console.log("success") successCB(response.data) // here it's wrong statement, because on success you are just logging and then `successCB` sending outside the if block which return in failure case also.
Save yourself from using stringify , parse etc on accessive large dataset. (which i can see in your above shown logs too.
Last but not least , for every time when your application crashes or pods restarted check the logs. In log specifically look for this section: Security context
This will give you why , where and who is the culprit behind the crash.
I will mention 2 types of solution.
My solution : In my case I add this to my environment variables :
export NODE_OPTIONS=--max_old_space_size=20480
But even if I restart my computer it still does not work. My project folder is in d:\ disk. So I remove my project to c:\ disk and it worked.
My team mate's solution : package.json configuration is worked also.
"start": "rimraf ./build && react-scripts --expose-gc --max_old_space_size=4096 start",
For other beginners like me, who didn't find any suitable solution for this error, check the node version installed (x32, x64, x86). I have a 64-bit CPU and I've installed x86 node version, which caused the CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory error.
if you want to change the memory globally for node (windows) go to advanced system settings -> environment variables -> new user variable
variable name = NODE_OPTIONS
variable value = --max-old-space-size=4096
You can also change Window's environment variables with:
$env:NODE_OPTIONS="--max-old-space-size=8192"
Unix (Mac OS)
Open a terminal and open our .zshrc file using nano like so (this will create one, if one doesn't exist):
nano ~/.zshrc
Update our NODE_OPTIONS environment variable by adding the following line into our currently open .zshrc file:
export NODE_OPTIONS=--max-old-space-size=8192 # increase node memory limit
Please note that we can set the number of megabytes passed in to whatever we like, provided our system has enough memory (here we are passing in 8192 megabytes which is roughly 8 GB).
Save and exit nano by pressing: ctrl + x, then y to agree and finally enter to save the changes.
Close and reopen the terminal to make sure our changes have been recognised.
We can print out the contents of our .zshrc file to see if our changes were saved like so: cat ~/.zshrc.
Linux (Ubuntu)
Open a terminal and open the .bashrc file using nano like so:
nano ~/.bashrc
The remaining steps are similar with the Mac steps from above, except we would most likely be using ~/.bashrc by default (as opposed to ~/.zshrc). So these values would need to be substituted!
Link to Nodejs Docs
Use the option --optimize-for-size. It's going to focus on using less ram.
I had this error on AWS Elastic Beanstalk, upgrading instance type from t3.micro (Free tier) to t3.small fixed the error
In my case, I upgraded node.js version to latest (version 12.8.0) and it worked like a charm.
Upgrade node to the latest version. I was on node 6.6 with this error and upgraded to 8.9.4 and the problem went away.
For Angular, this is how I fixed
In Package.json, inside script tag add this
"scripts": {
"build-prod": "node --max_old_space_size=5048 ./node_modules/#angular/cli/bin/ng build --prod",
},
Now in terminal/cmd instead of using ng build --prod just use
npm run build-prod
If you want to use this configuration for build only just remove --prod from all the 3 places
I experienced the same problem today. The problem for me was, I was trying to import lot of data to the database in my NextJS project.
So what I did is, I installed win-node-env package like this:
yarn add win-node-env
Because my development machine was Windows. I installed it locally than globally. You can install it globally also like this: yarn global add win-node-env
And then in the package.json file of my NextJS project, I added another startup script like this:
"dev_more_mem": "NODE_OPTIONS=\"--max_old_space_size=8192\" next dev"
Here, am passing the node option, ie. setting 8GB as the limit.
So my package.json file somewhat looks like this:
{
"name": "my_project_name_here",
"version": "1.0.0",
"private": true,
"scripts": {
"dev": "next dev",
"dev_more_mem": "NODE_OPTIONS=\"--max_old_space_size=8192\" next dev",
"build": "next build",
"lint": "next lint"
},
......
}
And then I run it like this:
yarn dev_more_mem
For me, I was facing the issue only on my development machine (because I was doing the importing of large data). Hence this solution. Thought to share this as it might come in handy for others.
I had the same issue in a windows machine and I noticed that for some reason it didn't work in git bash, but it was working in power shell
Just in case it may help people having this issue while using nodejs apps that produce heavy logging, a colleague solved this issue by piping the standard output(s) to a file.
If you are trying to launch not node itself, but some other soft, for example webpack you can use the environment variable and cross-env package:
$ cross-env NODE_OPTIONS='--max-old-space-size=4096' \
webpack --progress --config build/webpack.config.dev.js
For angular project bundling, I've added the below line to my pakage.json file in the scripts section.
"build-prod": "node --max_old_space_size=5120 ./node_modules/#angular/cli/bin/ng build --prod --base-href /"
Now, to bundle my code, I use npm run build-prod instead of ng build --requiredFlagsHere
hope this helps!
If any of the given answers are not working for you, check your installed node if it compatible (i.e 32bit or 64bit) to your system. Usually this type of error occurs because of incompatible node and OS versions and terminal/system will not tell you about that but will keep you giving out of memory error.
None of all these every single answers worked for me (I didn't try to update npm tho).
Here's what worked: My program was using two arrays. One that was parsed on JSON, the other that was generated from datas on the first one. Just before the second loop, I just had to set my first JSON parsed array back to [].
That way a loooooot of memory is freed, allowing the program to continue execution without failing memory allocation at some point.
Cheers !
You can fix a "heap out of memory" error in Node.js by below approaches.
Increase the amount of memory allocated to the Node.js process by using the --max-old-space-size flag when starting the application. For example, you can increase the limit to 4GB by running node --max-old-space-size=4096 index.js.
Use a memory leak detection tool, such as the Node.js heap dump module, to identify and fix memory leaks in your application. You can also use the node inspector and use chrome://inspect to check memory usage.
Optimize your code to reduce the amount of memory needed. This might involve reducing the size of data structures, reusing objects instead of creating new ones, or using more efficient algorithms.
Use a garbage collector (GC) algorithm to manage memory automatically. Node.js uses the V8 engine's garbage collector by default, but you can also use other GC algorithms such as the Garbage Collection in Node.js
Use a containerization technology like Docker which limits the amount of memory available to the container.
Use a process manager like pm2 which allows to automatically restart the node application if it goes out of memory.

Setting up Angular Universal App for development

I have created a project with Angular-CLI. (using command: ng new my-angular-universal).
Then I carefully followed all the instructions from https://github.com/angular/angular-cli/wiki/stories-universal-rendering
It builds for --prod and works fine. But there are no instructions on how I can set up a --dev build and have it served with --watch flag.
I tried removing --prod flags from npm "scripts", and it doesn't even run in dev mode. It builds fine but when I open it in browser this is what I see (directly printed to response):
TypeError: Cannot read property 'moduleType' of undefined
at C:\Users\Mikser\documents\git\my-angular-universal\dist\server.js:7069:134
at ZoneDelegate.invoke (C:\Users\Mikser\documents\git\my-angular-universal\dist\server.js:105076:26)
at Object.onInvoke (C:\Users\Mikser\documents\git\my-angular-universal\dist\server.js:6328:33)
at ZoneDelegate.invoke (C:\Users\Mikser\documents\git\my-angular-universal\dist\server.js:105075:32)
at Zone.run (C:\Users\Mikser\documents\git\my-angular-universal\dist\server.js:104826:43)
at NgZone.run (C:\Users\Mikser\documents\git\my-angular-universal\dist\server.js:6145:69)
at PlatformRef.bootstrapModuleFactory (C:\Users\Mikser\documents\git\my-angular-universal\dist\server.js:7068:23)
at Object.renderModuleFactory (C:\Users\Mikser\documents\git\my-angular-universal\dist\server.js:52132:39)
at View.engine (C:\Users\Mikser\documents\git\my-angular-universal\dist\server.js:104656:23)
at View.render (C:\Users\Mikser\documents\git\my-angular-universal\dist\server.js:130741:8)
the versions of npm packages that I use are currently the latest:
#angular/* - #5.2.*
#angular/cli #1.7.3
except for ts-loader, had to downgrade it because it wasn't working:
ts-loader #3.5.0
So if anyone has any info on how to make this work, it would be very appreciated! Or maybe you know some project templates with Angular Universal App configured for both --dev and --prod builds and ability to --watch?
For development, run npm run start which triggers ng serve. The current setup has hot module reloading so it will watch for your changes and update your dev view. I used the same instructions and got it working here https://github.com/ariellephan/angular5-universal-template
In short, for development, run npm run start and look at http://localhost:4200.
For production, run npm run build:ssr and npm run serve:ssrand look at http://localhost:4000
As contributors have pointed out, it might not be the most efficient and fastest way to develop, but nevertheless I did not want to accept workarounds. Besides, hosting front and back on separate servers brings up CORS issues, and I never planned my app to run on separate hosts, I wanted it all on the same host together with API methods.
The problem with --dev build was this:
when building with the following command:
ng build --app 1 --output-hashing=false (note that there is no --prod flag)
AppServerModuleNgFactory turned out missing in the ./dist-server/main.bundle
I imagine that this relates to the ahead of time(--aot) compilation which is the default behavior if you are building for --prod. So the instructions from https://github.com/angular/angular-cli/wiki/stories-universal-rendering included instructions to configure express server for production build only. And since there is no need for server to be able to dynamically render html templates the working --dev build command would be:
ng build --app 1 --output-hashing=false --aot
and this gets rid of the TypeError: Cannot read property 'moduleType' of undefined
Now to watch this whole mess:
run these in separate command windows:
ng build --watch
ng build --app 1 --output-hashing=false --aot --watch
webpack --config webpack.server.config.js --progress --colors --watch
And for the server to restart on change, you have to install nodemon package and run it like this:
nodemon --inspect dist/server (--inspect if you wish to debug server with chrome)
Some other important stuff:
Angular/CLI has a command to generate necessary scaffolding for a universal app:
ng generate universal
and it generates a fixed version of main.ts that avoids client angular bootstrap issue:
document.addEventListener('DOMContentLoaded', () => {
platformBrowserDynamic().bootstrapModule(AppModule)
.catch(err => console.log(err));
});
a problem that I stumbled upon once I implemented TransferState
There are basically two parts - the server and the UI. While developing the UI, I simply use ng serve. That means when I make changes in my code in the IDE, the browser refreshes automatically. And, here the server part is not used.
I do prod build and run the server only for final testing to see if everything works as expected (No error due to any 3PP library DOM manipulation or AOT related issues, etc.)
Here, I have created a skeleton structure of an Angular Universal project. As I extensively use Vagrant and Docker in my projects, I run the server in a Docker container within the Vagrant guest system. And for development of the UI, I don't run the server. Simply, the ng serve is used.
If you look into my structure in the above Github link, you'll find the details as to how to run it for development and production in the Readme file.
The web server handler server.ts uses the server bundle
const { AppServerModuleNgFactory, LAZY_MODULE_MAP } = require('./dist/server/main.bundle');
That's why the server bundle needs to be compiled before you can compile the server.ts file.
So having a watch system would mean
watching/recompiling the client bundle
watching/recompiling the server bundle
recompiling the server.ts once the server bundle is created
All of them take some time (especially if you do it with aot)
I'd recommend, like Saptarshi Basu mentionned, to develop as best as you can with ng serve and check with angular universal every so often.
Otherwise, it should be possible do achieve what you want with some kind of tasks (grunt/gulp/...) which triggers sequentially ng build ... and recompilation of server.ts file.
It is a bit messy no doubt, as we preferably wish for one command to rule them all.
I came up with a somewhat OK solution where my output will be:
dist/browser
dist/ng-server
Using the executable npm-run-all package (I find it working a lot better on windows machines than concurrently does) I run the three watch tasks: browser, ng-server and nodeJS. Watching node has a pre-task defined that simply runs a small utility/helper/file that watches for the existence of a dist/ng-server folder and terminate itself once found.
For all of this to work (based on the universal-starter repo as of november 2018) there's a couple of modifications to package.json required. Primarily, to support the --watch flag on ng run commands we need to update the compiler-cli (if memory serves), ng update --all should take care of that, giving you the latest angular/cli version in the process (assuming you have a recent cli version installed globally).
package.json
ng update --all
angular 6+
angular/cli 7+
yarn add/npm install the following
chokidar
npm-run-all
(runs our tasks in parallel with the -p flag. -p kills all processes, -l gives each running task a specific color and name in the console)
ts-node (runs nodejs in it's ts-format)
nodemon // for restarting ts-node
add something similar to my util/await-file.js (after some consideration I added my own file-watcher code below even though it wasn't exactly written with the intentions to be put up on display...)
modify your package.json scripts like below
modify your angular.json to match your folder names, following my examples, mainly the "server"'s outputPath should be changed from dist/server to dist/ng-server.
package.json scripts
"dev": "npm-run-all -p -r -l watch:ng-server watch:browser watch:node",
"watch:browser": "ng build --prod --progress --watch --delete-output-path",
"watch:ng-server": "ng run ng-universal-demo:server --watch --delete-output-path",
"watch:node": "yarn run watch:file-exist && yarn run ts-node",
"ts-node": "nodemon --exec ts-node server.ts -e ts,js",
"watch:file-exist": "node utils/await-file.js",
util/await-file.js
const chokidar = require('chokidar');
const fs = require('fs');
const path = require('path');
const DIR_NAME = 'ng-server';
const DIST_PATH = './dist';
// creates dist folder if it doesn't exist - prior to adding it to the watcher.
if (!fs.existsSync(DIST_PATH)) {
fs.mkdirSync(DIST_PATH);
}
const watcher = chokidar.watch('file, dir', {
ignored: '*.map',
persistent: true,
awaitWriteFinish: {
stabilityThreshold: 5000,
pollInterval: 100
}
});
const FOLDER_PATH = path.join(process.cwd(), 'dist');
watcher.add(FOLDER_PATH);
console.log(`file-watcher running, waiting for ${DIST_PATH}/${DIR_NAME}`);
function fileFound() {
console.log(`${DIR_NAME} folder found - closing`);
watcher.close();
process.exit();
}
watcher
.on('add', function (filePath) {
const matchWith = path.join('dist', DIR_NAME);
const paths = filePath.split(path.sep);
const fileName = paths[paths.length - 1];
if ((filePath.indexOf(matchWith) >= 0)
&& fileName.indexOf('.js') > fileName.length - 4) {
fileFound();
}
})
.on('error', error => console.log(`Watcher error: ${error}`));
"npm run start" and using "http://localhost:4200" works for me. Even with Angular 10

Redeploying NAR (node.js) archive with PM2

There is node.js app built with Typescript, so it needs to be first "compiled" to JS before it gets run. I'm planning to use NAR (https://github.com/h2non/nar) to build ready-to-deploy package to avoid fiddling with npm install and compiling it on production. I also use PM2 as process manager for node apps.
However as far as I know PM2 can only deploy from git (fetching sources and calling npm install etc. later on), but I couldnt find a way to easily deploy application that is already pre-built.
This is my deploy.yml file contained within archive that I extract with nar extract <package>:
apps:
- script: dist/app.js
merge_logs: true
name: server
instances: 1 # 0 => max, depending on CPU cores
exec_mode: cluster
node_args: --harmony --harmony_destructuring --harmony_default_parameters
log_file: deploy/logs/server.log
pid_file: deploy/pids/server.pid
source_map_support: true
env:
NODE_ENV: production
It works fine when run for the first time, but then when I try to redeploy it (replacing application content with new version) and call pm2 reload all I get errored processes saying they either cannot load ProcessManager from PM2 or cannot find my .env file (which is in place).
As soon as I kill PM2 daemon with pm2 kill and start apps again with pm2 start all deploy.yml it clicks. But this is probably not how PM2 should be used, right?
Do you have any experience with such setup and had similar issues? Or maybe can you point me to another way of running my deployment?

Resources