I'm building a web crawler in Node.js using the npm crawler package. My program right now creates 5 child processes which each instantiate a new Crawler, which crawls a list of URLS which the parent provides.
When it runs for about 15-20 minutes, it slows down to a halt and the processes' STATE column from the output of the top command reads stuck for all the children. [see below]
I have little knowledge of the top command, and the columns provided, but I want to know is there a way to find out what is causing the processes to slow down by looking at the output of top? I realize that it is probably my code that has a bug in it, but I want to know where I should start debugging: memory leak, caching issue, not enough children, too many children, etc.
Below is the entire output of top
PID COMMAND %CPU TIME #TH #WQ #PORT #MREG MEM RPRVT PURG CMPRS VPRVT VSIZE PGRP PPID STATE UID FAULTS COW MSGSENT MSGRECV SYSBSD SYSMACH
11615 node 2.0 17:16.43 8 0 42 2519 94M- 94M- 0B 1347M+ 1538M 4150M 11610 11610 stuck 541697072 14789409+ 218 168 21 6481040 63691
11614 node 2.0 16:57.66 8 0 42 2448 47M- 47M- 0B 1360M+ 1498M- 4123M 11610 11610 stuck 541697072 14956093+ 217 151 21 5707766 64937
11613 node 4.4 17:17.37 8 0 44 2415 100M+ 100M+ 0B 1292M- 1485M 4114M 11610 11610 sleeping 541697072 14896418+ 215 181 22 6881669+ 66098+
11612 node 10.3 17:37.81 8 0 42 2478 24M+ 24M+ 0B 1400M- 1512M 4129M 11610 11610 stuck 541697072 14386703+ 215 171 21 7083645+ 65551
11611 node 2.0 17:09.52 8 0 42 2424 68M- 68M- 0B 1321M+ 1483M 4111M 11610 11610 sleeping 541697072 14504735+ 220 168 21 6355162 63701
11610 node 0.0 00:04.63 8 0 42 208 4096B 0B 0B 126M 227M 3107M 11610 11446 sleeping 541697072 45184 410 52 21 36376 6939
Here are the dependencies:
├── colors#0.6.2
├── crawler#0.2.6
├── log-symbols#1.0.0
├── robots#0.9.4
└── sitemapper#0.0.1
Sitemapper is one I wrote myself which could be a source for bugs.
Related
I am encountering the messages below after creating an UBIFS filesystem in
a new UBI volume on an existing UBI partition.
This happens while reading all the files on the UBI fs via tar -cf /dev/null *:
UBIFS error (pid 1810): ubifs_read_node: bad node type (255 but expected 1)
UBIFS error (pid 1810): ubifs_read_node: bad node at LEB 33:967120, LEB mapping status 0
Not a node, first 24 bytes:
00000000: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
CPU: 0 PID: 1810 Comm: tar Tainted: P O 3.18.80 #3
Backtrace:
[<8001b664>] (dump_backtrace) from [<8001b880>] (show_stack+0x18/0x1c)
r7:00000000 r6:00000000 r5:80000013 r4:80566cbc
[<8001b868>] (show_stack) from [<801d77e0>] (dump_stack+0x88/0xa4)
[<801d7758>] (dump_stack) from [<80171608>] (ubifs_read_node+0x1fc/0x2b8)
r7:00000021 r6:00000712 r5:000ec1d0 r4:bd59d000
[<8017140c>] (ubifs_read_node) from [<8018b654>](ubifs_tnc_read_node+0x88/0x124)
r10:00000000 r9:b4f9fdb0 r8:bd59d264 r7:00000001 r6:b71e6000 r5:bd59d000
r4:b6126148
[<8018b5cc>] (ubifs_tnc_read_node) from [<80174690>](ubifs_tnc_locate+0x108/0x1e0)
r7:00000001 r6:b71e6000 r5:00000001 r4:bd59d000
[<80174588>] (ubifs_tnc_locate) from [<80168200>] (do_readpage+0x1c0/0x39c)
r10:bd59d000 r9:000005b1 r8:00000000 r7:bd258710 r6:a5914000 r5:b71e6000
r4:be09e280
[<80168040>] (do_readpage) from [<80169398>] (ubifs_readpage+0x44/0x424)
r10:00000000 r9:00000000 r8:bd59d000 r7:be09e280 r6:00000000 r5:bd258710
r4:00000000
[<80169354>] (ubifs_readpage) from [<80089654>](generic_file_read_iter+0x48c/0x5d8)
r10:00000000 r9:00000000 r8:00000000 r7:be09e280 r6:bd2587d4 r5:bd5379e0
r4:00000000
[<800891c8>] (generic_file_read_iter) from [<800bbd60>](new_sync_read+0x84/0xa8)
r10:00000000 r9:b4f9e000 r8:80008e24 r7:bd6437a0 r6:bd5379e0 r5:b4f9ff80
r4:00001000
[<800bbcdc>] (new_sync_read) from [<800bc85c>] (__vfs_read+0x20/0x54)
r7:b4f9ff80 r6:7ec2d800 r5:00001000 r4:800bbcdc
[<800bc83c>] (__vfs_read) from [<800bc91c>] (vfs_read+0x8c/0xf4)
r5:00001000 r4:bd5379e0
[<800bc890>] (vfs_read) from [<800bc9cc>] (SyS_read+0x48/0x80)
r9:b4f9e000 r8:80008e24 r7:00001000 r6:7ec2d800 r5:bd5379e0 r4:bd5379e0
[<800bc984>] (SyS_read) from [<80008c80>] (ret_fast_syscall+0x0/0x3c)
r7:00000003 r6:7ec2d800 r5:00000008 r4:00082a08
UBIFS error (pid 1811): do_readpage: cannot read page 0 of inode 1457, error -22
Mounting looks like this:
UBI-0: ubi_attach_mtd_dev:default fastmap pool size: 190
UBI-0: ubi_attach_mtd_dev:default fastmap WL pool size: 25
UBI-0: ubi_attach_mtd_dev:attaching mtd3 to ubi0
UBI-0: scan_all:scanning is finished
UBI-0: ubi_attach_mtd_dev:attached mtd3 (name "data", size 3824 MiB)
UBI-0: ubi_attach_mtd_dev:PEB size: 1048576 bytes (1024 KiB), LEB size: 1040384 bytes
UBI-0: ubi_attach_mtd_dev:min./max. I/O unit sizes: 4096/4096, sub-page size 4096
UBI-0: ubi_attach_mtd_dev:VID header offset: 4096 (aligned 4096), data offset: 8192
UBI-0: ubi_attach_mtd_dev:good PEBs: 3816, bad PEBs: 8, corrupted PEBs: 0
UBI-0: ubi_attach_mtd_dev:user volume: 6, internal volumes: 1, max. volumes count: 128
UBI-0: ubi_attach_mtd_dev:max/mean erase counter: 4/2, WL threshold: 4096, image sequence number: 1362948729
UBI-0: ubi_attach_mtd_dev:available PEBs: 2313, total reserved PEBs: 1503, PEBs reserved for bad PEB handling: 72
UBI-0: ubi_thread:background thread "ubi_bgt0d" started, PID 419
UBIFS: mounted UBI device 0, volume 6, name "slot1", R/O mode
UBIFS: LEB size: 1040384 bytes (1016 KiB), min./max. I/O unit sizes: 4096 bytes/4096 bytes
UBIFS: FS size: 259055616 bytes (247 MiB, 249 LEBs), journal size 12484608 bytes (11 MiB, 12 LEBs)
UBIFS: reserved for root: 4952683 bytes (4836 KiB)
UBIFS: media format: w4/r0 (latest is w4/r0), UUID 92C7B251-2666-4717-B735-5539900FE749, small LPT model
The rate of reproduction is one in about 20 sequences of: ubirmvol,
ubimkvol (256MB), unpack a rootfs.tgz (20MB, 40MB unpacked) in the
mounted UBIFS, umount, ubidetach, sync, reboot. There are no xattrs
created. No power cuts happen during the tests.
The board runs a 3.18.80 kernel on an ARM board. I tested other
identical test boards where only the NAND chip manufacturer was
different.
I had CONFIG_MTD_UBI_FASTMAP enabled when the error occured, but I
used no explicit fastmap boot cmdline option. I tried mounting the
ubifs with a rebuilt kernel that had the config option disabled after
the issue reproduced, but the read error still occured.
I also tried mounting with a kernel that had "ubifs: Fix journal
replay w.r.t. xattr nodes" patch [1] applied, after reading older similar error reports, but once reproduced on a NAND, the read error
persisted.
[1] https://patchwork.ozlabs.org/patch/713213/
Is there some special requirement except just umount, ubidetach and
sync before running reboot, or regarding free space fixup ? Would
creating the fs image offline with mkfs.ubifs instead of unpacking it
via tar make a difference here ? Would it help to test with
cherry picked commits from linux v4.14 that touch
fs/ubifs ?
I have to debug a crash. But everytime my system crash it failed to dump the crashdump into the swap memory. The erorr i am seeing is:-
**Uptime: 7m32s
Dumping 3735 out of 131037 MB:..1%(ada0:ahcich0:0:0:0): WRITE_DMA48. ACB: 35 00 16 c9 c3 40 39 00 00 00 08 00
(ada0:ahcich0:0:0:0): CAM status: CCB request was invalid
(ada0:ahcich0:0:0:0): Error 22, Unretryable error
Aborting dump due to I/O error.
** DUMP FAILED (ERROR 22) **
**
In my rc.conf i have set the dumpdev to AUTO and my swap memory is 4GB.
Here is the ouput of fstab:-
# Device Mountpoint FStype Options Dump Pass#
/dev/ada0p2 / ufs rw 1 1
/dev/ada0p3 none swap sw 0 0
Thanks
Your swap partition is smaller than your memory.
How do you expect 12 Gig to fit into 4Gig?
I want exactly what this page do
http://codepen.io/netsi1964/full/vNoemp/
I have the path and need to know it's bounding box as a rect element, it's x,y,width and height
given code
<path d="M147.5 55.8c-5.8-7.2-13.6-14.4-25.5-14.4-8.4 0-15.4 8.2-27 8.2-9 0-13-7.8-23-7.8C51.4 41.8 31 60.4 31 84.5c0 12.8 4.2 32.5 13.6 49.7C51 146.7 59.4 155 69 155c6.7 0 14.7-6.3 24.2-6.3 8.4 0 16.2 5.6 23.8 5.6 18 0 35-23.5 35-39.3 0-.8-.3-1.4-.3-2v-1c-11.8-6.3-18.2-15.7-18.2-29.3 0-11 4.8-20.5 13.6-26.7l.5-.2zm-53-8.8c13.7-4.2 26.3-14.4 26.3-32 0-1.5-.2-3.3-.4-5.3l-.2-.8C106.4 12.6 94 23.4 94 40.3c0 1.6.2 3.6.6 5.8v.8z" style="translate(0px,-212.47488403320312px) scale(1,1)" >
and know the rect properties
With pure JavaScript: give your path an ID and get its bounding box using getBBox().
var myPathBox = document.getElementById("myPath").getBBox();
console.log(myPathBox);
Here is a demo:
var myPathBox = document.getElementById("myPath").getBBox();
console.log(myPathBox);
<svg width="400" height="400">
<path id="myPath" d="M147.5 55.8c-5.8-7.2-13.6-14.4-25.5-14.4-8.4 0-15.4 8.2-27 8.2-9 0-13-7.8-23-7.8C51.4 41.8 31 60.4 31 84.5c0 12.8 4.2 32.5 13.6 49.7C51 146.7 59.4 155 69 155c6.7 0 14.7-6.3 24.2-6.3 8.4 0 16.2 5.6 23.8 5.6 18 0 35-23.5 35-39.3 0-.8-.3-1.4-.3-2v-1c-11.8-6.3-18.2-15.7-18.2-29.3 0-11 4.8-20.5 13.6-26.7l.5-.2zm-53-8.8c13.7-4.2 26.3-14.4 26.3-32 0-1.5-.2-3.3-.4-5.3l-.2-.8C106.4 12.6 94 23.4 94 40.3c0 1.6.2 3.6.6 5.8v.8z" style="translate(0px,-212.47488403320312px) scale(1,1)" >
</svg>
I know this is an old issue, but thought I would put this variant to Furtado's answer for reference here.
An easy way to get the bounding box for the path.
Make sure the path (or any other SVG element has an id on it.
Open the svg file in Chrome (or FF or probably IE).
Inspect the image
Open the console in the inspection tool.
Enter the JS: document.getElementById("myPath").getBBox(); (where
myPath is the id)
The bounding box info will be displayed in the console.
Same method, just no need for custom code.
v0.10.4
Here's the simple loop that results in an ever-increasing memory usage:
function redx(){
setTimeout(function(){ redx() },1000);
console.log('loop');
}
redx();
What am I doing wrong ??
EDIT
OK, just tried the suggestion to reference the timeout object in the scope and it seems that garbage collection does kick in after about 40 seconds, here's abbreviated logs from TOP:
3941 root 20 0 32944 7284 4084 S 4.587 3.406 0:01.32 node
3941 root 20 0 32944 7460 4084 S 2.948 3.489 0:01.59 node
3941 root 20 0 32944 7516 4084 S 2.948 3.515 0:01.68 node
3941 root 20 0 33968 8400 4112 S 2.948 3.928 0:02.15 node
3941 root 20 0 33968 8920 4112 S 3.275 4.171 0:02.98 node
3941 root 20 0 33968 8964 4112 S 2.948 4.192 0:03.07 node
3941 root 20 0 33968 9212 4112 S 2.953 4.308 0:03.16 node
3941 root 20 0 33968 9212 4112 S 2.953 4.308 0:03.25 node
3941 root 20 0 33968 9212 4112 S 3.276 4.308 0:03.35 node
3941 root 20 0 33968 9212 4112 S 2.950 4.308 0:03.44 node
No idea why but apparently if you reference the timeout object in the scope of the function nodejs will do the garbage collect that correctly.
function redx(){
var t = setTimeout(function(){ redx() },50);
console.log('hi');
}
redx();
Actually, I think it might be just the way the V8 garbage collector works.
On my system, node heap tends to increase up to 48 MB and then stabilize, so I think if you keep your program running for a long time, the memory consumption will eventually stabilize.
You can have information about when/how the GC kicks in by launching node with one of the V8 command line option: the --trace_gc flag.
In your first tries with Redis, you were systematically connecting/disconnecting from Redis at each call. This tends to generate garbage. You are supposed to open a connection once and use it many times. Nevertheless, even when I do this, memory consumption tends to stabilize. Here is the evolution of memory consumption on this example with Redis:
// something close to your initial function (when Redis was still in the picture)
function redx(){
var client = redis.createClient();
client.get("tally", function(err, reply) {
client.quit();
});
setTimeout(function(){ redx() }, 50 );
}
Here, the stabilization after 60 MB seems to be quite obvious.
I'm dealing with this issue: We have recently upgraded to VS2010 and I am working on recompiling all of our software tools in '10. One of these tools in VC++ was created by an outside vendor. We have the source code for this tool (fairly old) and its required dll (also fairly old), although we don't have the source for the dll. The problem arises when I build the project and get several unresolved externals (LNK2019). I've used a dll export viewer and verified that the dll is in fact exporting the right functions, and I've experimented with a few other workarounds to no avail. The only explanation I can come up with is that the dll, likely built in VS2005, also needs to be rebuilt in VS2010 - although my experience (which I admit is limited) tells me that this should not be necessary.
My question is: Is this really an issue? Does a dll built in an older version of VS need to be rebuilt in the version that the project that uses it is currently using?
I can provide more details if necessary.
Thank you,
Andy
Dump of file usbcan32.dll
File Type: DLL
Section contains the following imports:
SETUPAPI.dll
10012188 Import Address Table
10016E8C Import Name Table
0 time date stamp
0 Index of first forwarder reference
12D SetupDiGetClassDevsA
11F SetupDiEnumDeviceInterfaces
143 SetupDiGetDeviceInterfaceDetailA
VERSION.dll
100121D4 Import Address Table
10016ED8 Import Name Table
0 time date stamp
0 Index of first forwarder reference
0 GetFileVersionInfoA
A VerQueryValueA
1 GetFileVersionInfoSizeA
KERNEL32.dll
10012020 Import Address Table
10016D24 Import Name Table
0 time date stamp
0 Index of first forwarder reference
347 Sleep
169 GetLastError
21E InterlockedDecrement
247 LeaveCriticalSection
8F EnterCriticalSection
309 SetEvent
383 WaitForSingleObject
49 CreateEventA
1F5 GlobalFree
200 GlobalUnlock
1F9 GlobalLock
1EE GlobalAlloc
1E9 GetWindowsDirectoryA
219 InitializeCriticalSection
7A DeleteCriticalSection
2E CloseHandle
24 CancelIo
2B6 ReleaseMutex
381 WaitForMultipleObjects
271 OpenEventA
278 OpenMutexA
5A CreateMutexA
7C DeleteFileA
1DF GetVersionExA
1BE GetSystemTime
175 GetModuleFileNameA
177 GetModuleHandleA
378 VirtualLock
373 VirtualAlloc
18B GetOEMCP
376 VirtualFree
2C5 ResumeThread
222 InterlockedIncrement
25E MapViewOfFile
4E CreateFileMappingA
13B GetCurrentProcessId
363 UnmapViewOfFile
83 DeviceIoControl
252 LocalFree
EA FormatMessageA
4D CreateFileA
2A9 ReadFile
394 WriteFile
18C GetOverlappedResult
336 SetThreadPriority
69 CreateThread
F5 GetACP
32A SetStdHandle
FC GetCPInfo
16C GetLocaleInfoA
21F InterlockedExchange
2CA RtlUnwind
1B1 GetStdHandle
248 LoadLibraryA
360 UnhandledExceptionFilter
14F GetEnvironmentStringsW
EE FreeEnvironmentStringsW
14D GetEnvironmentStrings
ED FreeEnvironmentStringsA
212 HeapSize
303 SetEndOfFile
1B2 GetStringTypeA
15E GetFileType
1AF GetStartupInfoA
20A HeapDestroy
1D5 GetTickCount
1B5 GetStringTypeW
210 HeapReAlloc
AF ExitProcess
20C HeapFree
206 HeapAlloc
108 GetCommandLineA
297 QueryPerformanceCounter
13E GetCurrentThreadId
1C0 GetSystemTimeAsFileTime
379 VirtualProtect
1BB GetSystemInfo
37B VirtualQuery
23A LCMapStringA
387 WideCharToMultiByte
26B MultiByteToWideChar
23B LCMapStringW
198 GetProcAddress
34F TerminateProcess
13A GetCurrentProcess
30E SetFilePointer
E5 FlushFileBuffers
317 SetHandleCount
208 HeapCreate
USER32.dll
100121A0 Import Address Table
10016EA4 Import Name Table
0 time date stamp
0 Index of first forwarder reference
1DE MessageBoxA
216 RegisterClassA
60 CreateWindowExA
13A GetMessageA
2AA TranslateMessage
2B5 UnregisterDeviceNotification
99 DestroyWindow
203 PostQuitMessage
21C RegisterDeviceNotificationA
8E DefWindowProcA
201 PostMessageA
A1 DispatchMessageA
ADVAPI32.dll
10012000 Import Address Table
10016D04 Import Name Table
0 time date stamp
0 Index of first forwarder reference
1E2 RegOpenKeyExA
1CD RegCreateKeyExA
1F9 RegSetValueExA
1E1 RegOpenKeyA
1EC RegQueryValueExA
1C9 RegCloseKey
55 ConvertStringSecurityDescriptorToSecurityDescriptorA
SHELL32.dll
10012198 Import Address Table
10016E9C Import Name Table
0 time date stamp
0 Index of first forwarder reference
AF SHGetFolderPathA
Summary
3000 .data
6000 .rdata
2000 .reloc
1000 .rsrc
11000 .text