How to load a large geojson file in leaflet - node.js

I am having a large Geojson file about 9mb that i need to load in angular 5 using leaflet but the application crash when i run ng serve giving this error:
<--- Last few GCs --->
[18416:0215BF30] 248777 ms: Mark-sweep 734.9 (778.9) -> 734.7 (783.9) MB, 946.3 / 0.0 ms allocation failure GC in old space requested
[18416:0215BF30] 249725 ms: Mark-sweep 734.7 (783.9) -> 734.6 (766.9) MB, 947.1 / 0.0 ms last resort GC in old space requested
[18416:0215BF30] 250669 ms: Mark-sweep 734.6 (766.9) -> 734.6 (766.4) MB, 943.8 / 0.0 ms last resort GC in old space requested
<--- JS stacktrace --->
==== JS stack trace =========================================
Security context: 35816169 <JSObject>
1: addMappingWithCode [C:\Users\Mhuss\Desktop\modz\programming\Angular\leaflet-geojson\node_modules\webpack-sources\node_modules\source-map\lib\source-node.js:~150] [pc=085B361A](this=04A85729 <JSGlobal Object>,mapping=1621A345 <Object map = 088F842D>,code=3816C225 <String[3]: 0.0>)
2: /* anonymous */ [C:\Users\Mhuss\Desktop\modz\programming\Angular\leaflet-geojson\node_modules\webpack-sources...
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
1: node_module_register
2: v8::internal::Factory::NewUninitializedFixedArray
3: v8::internal::WasmDebugInfo::SetupForTesting

Related

Heap out of memory - increasing max-old-space-size didn't solve the issue

I'm working on a quite huge react application and have recently encountered the following problem:
When I start my app with npm start it works just fine and nothing seems wrong, even though I get
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
From time to time in my console output, which doesn't really prevent my app from running. However, when I try to actually build my app, the very same error appears and build folder doesn't contain all the files. I've tried increasing --max-old-space-size up to 32GB, though it didn't really help.
Full output:
<--- Last few GCs --->
[360:000001C14A7A3D80] 58555 ms: Mark-sweep (reduce) 1993.9 (2033.6) -> 1993.9 (2033.1) MB, 615.8 / 0.0 ms (average mu = 0.685, current mu = 0.000) last resort GC in old space requested
[360:000001C14A7A3D80] 59180 ms: Mark-sweep (reduce) 1993.9 (2005.1) -> 1993.7 (2005.1) MB, 625.3 / 0.0 ms (average mu = 0.510, current mu = 0.000) last resort GC in old space requested
<--- JS stacktrace --->
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
1: 00007FF7F282052F napi_wrap+109311
2: 00007FF7F27C5256 v8::internal::OrderedHashTable<v8::internal::OrderedHashSet,1>::NumberOfElementsOffset+33302
3: 00007FF7F27C6026 node::OnFatalError+294
4: 00007FF7F309163E v8::Isolate::ReportExternalAllocationLimitReached+94
5: 00007FF7F30764BD v8::SharedArrayBuffer::Externalize+781
6: 00007FF7F2F2094C v8::internal::Heap::EphemeronKeyWriteBarrierFromCode+1516
7: 00007FF7F2F1BD61 v8::internal::Heap::AllocateExternalBackingStore+1569
8: 00007FF7F2F35F06 v8::internal::Factory::AllocateRaw+166
9: 00007FF7F2F49924 v8::internal::FactoryBase<v8::internal::Factory>::NewFixedArrayWithFiller+84
10: 00007FF7F2F49C23 v8::internal::FactoryBase<v8::internal::Factory>::NewFixedArrayWithMap+35
11: 00007FF7F2D50D6C v8::internal::OrderedHashTable<v8::internal::OrderedHashMap,2>::Allocate+124
12: 00007FF7F2D5356B v8::internal::OrderedHashTable<v8::internal::OrderedHashMap,2>::Rehash+59
13: 00007FF7F2D5234A v8::internal::OrderedHashTable<v8::internal::OrderedHashMap,2>::EnsureGrowable+90
14: 00007FF7F2C89837 v8::internal::interpreter::JumpTableTargetOffsets::iterator::operator=+122119
15: 00007FF7F3119FED v8::internal::SetupIsolateDelegate::SetupHeap+463949
Moreover, it happens in few environments. Both on my local machine and when I try to build it through Gitlab CI/CD

ng build --prod command fails with the message: "FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory"

While building Angular 4 application with the command:
ng build --prod
I am receiving the error message: "FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory"
The full error message is:
92% chunk asset optimization
<--- Last few GCs --->
118862 ms: Mark-sweep 636.4 (717.1) -> 636.0 (717.1) MB, 949.7 / 0.0 ms [allocation failure] [GC in old space requested].
119770 ms: Mark-sweep 636.0 (717.1) -> 636.0 (717.1) MB, 908.3 / 0.0 ms [allocation failure] [GC in old space requested].
120673 ms: Mark-sweep 636.0 (717.1) -> 639.2 (705.1) MB, 902.1 / 0.0 ms [lastresort gc].
121592 ms: Mark-sweep 639.2 (705.1) -> 643.0 (705.1) MB, 919.1 / 0.0 ms [lastresort gc].
<--- JS stacktrace --->
==== JS stack trace =========================================
Security context: 0427B80D <JS Object>
1: reduce_vars [042081D9 <undefined>:~8085] [pc=33877059] (this=0D827DD5 <an AST_SymbolRef with map 127AC9C9>,tw=2A3BE0B1 <a TreeWalker with map 12716AED>,descend=0C1795B5 <JS Function noop (SharedFunctionInfo 08B51CAD)>,compressor=1066
FAE5 <a Compressor with map 117D74D1>)
2: visit [042081D9 <undefined>:~8175] [pc=376ADA83] (this=2A3BE0B1 <a TreeWalker with map 12716AED>,node=0D827DD5 <an AS...
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
This application has been built before many times with no problems. Only today have I started getting this error. No updates to node.js have been done since the last time the app has been built.
Node.js version is 6.11.0
Many people say it can be the memory allocation issue. I have tried this suggestion but it did not fix the issue.
After digging through many web pages with people having similar issues, I tried the following:
ng build --prod --aot false
and it worked. aot stands for ahead-of-time compilation in this case I believe. I still have no clue why this fix worked but it did.

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - process out of memory - mongodb

I have a admin portal where all the documents from database are configured and manipulated.
We have a collection for language translation which contains a lot of document.
And admin can modify all this document.
If admin opens any other collection it works fine. But when he opens this language translation collection the systems gets slower and after few mins I found this error.
<--- Last few GCs --->
513530251 ms: Mark-sweep 1397.7 (1458.0) -> 1397.7 (1458.0) MB, 2719.4 / 2 ms [allocation failure] [
GC in old space requested].
513533054 ms: Mark-sweep 1397.7 (1458.0) -> 1397.7 (1458.0) MB, 2802.9 / 2 ms [last resort gc].
513535773 ms: Mark-sweep 1397.7 (1458.0) -> 1397.6 (1458.0) MB, 2718.9 / 2 ms [last resort gc].
<--- JS stacktrace --->
==== JS stack trace =========================================
Security context: 000002D0BF1B4639 <JS Object>
1: new constructor(aka WritableState) [_stream_writable.js:88] [pc=0000036F0D0CA7F9] (this=00000
153740AD191 <a WritableState with map 0000017D64825C01>,options=00000065E299D0F1 ,stream=00000153740ACFA1 )
3: Writable [_stream_writable.js:143] [pc=0000036F0D0CA0C2] (this=00000153740ACFA1 <a Socket with map 0000017D...
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - process out of memory
Can anyone help me what can solve this issue???
I start my node with the following syntax.
set node_debug=foo&& node --max-old-space-size=8192 server.js
I had the same problem with Node installed through Homebrew.
Try to run vim `which npm`
and change:
#!/usr/bin/env node
to:
#!/usr/bin/env node --max-old-space-size=2048
Update: By the way, I have fixed this error by following these simple steps
Add an environment variable:
TOOL_NODE_FLAGS="--max-old-space-size=4096"

node not allowing custom heap limit (--max-old-space-size)

I'm on Ubuntu 15.10 and am running node v6.2.1.
My machine has 15GB of RAM:
>> sudo lshw -class memory
*-memory
description: System memory
physical id: 4
size: 15GiB
But, when I try to start node with an increased heap limit:
node --max-old-space-size=2048
...it immediately runs out of memory:
<--- Last few GCs --->
25 ms: Mark-sweep 1.9 (19.5) -> 1.9 (19.5) MB, 0.7 / 0 ms [allocation failure] [GC in old space requested].
26 ms: Mark-sweep 1.9 (19.5) -> 1.9 (19.5) MB, 0.8 / 0 ms [allocation failure] [GC in old space requested].
27 ms: Mark-sweep 1.9 (19.5) -> 1.9 (19.5) MB, 0.9 / 0 ms [allocation failure] [GC in old space requested].
28 ms: Mark-sweep 1.9 (19.5) -> 1.9 (19.5) MB, 0.7 / 0 ms [last resort gc].
29 ms: Mark-sweep 1.9 (19.5) -> 1.9 (19.5) MB, 0.8 / 0 ms [last resort gc].
<--- JS stacktrace --->
==== JS stack trace =========================================
Security context: 0x3d67857d <JS Object>
2: replace [native string.js:134] [pc=0x4bb3f0c7] (this=0xb523c015 <Very long string[2051]>,N=0xb523d05d <JS RegExp>,O=0xb520b269 <String[2]: \">)
3: setupConfig [internal/process.js:112] [pc=0x4bb3d146] (this=0xb523727d <an Object with map 0x2ea0bc25>,_source=0x454086c1 <an Object with map 0x2ea0deb1>)
4: startup(aka startup) [node.js:51] [pc=0x4bb3713e] (this=0x3d6080c9 <undefined>)
...
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
Aborted (core dumped)
Any advice on how I can start a node process with a higher heap limit?
As far as I know, there was an issue with (old-space-)memory on 6.2.1. Update to 6.4 and see what happens. I had a similar issue while using gulp watchers. On first sight, 4GB were not enough, so I tried pushing up to 11.5 (which seemed to be the max limit). After all, the problem was with gulp.run() as it's deprecated now.
What I wanted to say is that it's not always about memory where the bug resides :)
That doesn't quite answer your question, but perhaps it's of help.

CircleCI is timing out, and it's related to node and eslint

CircleCI is timing out while running eslint using node.
I get the following error message:
command ... took more than 10 minutes since last output
On my local machine, it only takes 17 seconds.
(Answer below...)
I logged into CircleCI using "Debug via SSH". I confirmed that eslint was hanging. Then, I figured out how to get more debugging information:
DEBUG=eslint:cli-engine eslint .
After a long time, Node actually crashed:
<--- Last few GCs --->
345472 ms: Scavenge 1399.8 (1457.3) -> 1399.8 (1457.3) MB, 38.0 / 0 ms (+ 6.8 ms in 1 steps since last GC) [allocation failure] [incremental marking delaying mark-sweep].
348177 ms: Mark-sweep 1399.8 (1457.3) -> 1399.8 (1457.3) MB, 2705.8 / 0 ms (+ 8.7 ms in 2 steps since start of marking, biggest step 6.8 ms) [last resort gc].
350927 ms: Mark-sweep 1399.8 (1457.3) -> 1399.5 (1457.3) MB, 2749.7 / 0 ms [last resort gc].
<--- JS stacktrace --->
==== JS stack trace =========================================
Security context: 0xd2a8c0b4629 <JS Object>
1: /* anonymous */ [/home/ubuntu/website-django/static/node_modules/babel-eslint/babylon-to-espree/toToken.js:~1] [pc=0x33a525e2adb9] (this=0x1e91da709851 <JS Global Object>,token=0x349f83a2fc01 <a Token with map 0x3b6a9d8c2e31>,tt=0x2c0cfbd85ee1 <an Object with map 0x3b6a9d898959>,source=0x3314aa504101 <Very long string[1177579]>)
2: toTokens [/home/ubuntu/website-django/static/node_mod...
FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - process out of memory
Aborted (core dumped)
Finally, I realized that it was trying to lint my build directory which contained a bunch of third-party libraries, including Highchart, which are known to cause eslint problems because they're so big.
I added this to my .eslintignore:
build/**
Then, the problem went away.
The take home message is: make sure you're only linting the things you need to lint.

Resources