dma_alloc_coherent() page allocation failure with CMA enabled - linux

I am receiving a page allocation failure when calling dma_alloc_coherent() after a number of runs. I have plenty of CMA setup on boot (3GB). I am also making sure that dma_free_coherent() is called on every DMA before the next run. I am at a loss figuring out what could be causing this failure. Any insight or ideas is welcome. Thanks!
[84060.080230] Main: page allocation failure: order:10, mode:0xcc4(GFP_KERNEL|GFP_DMA32), nodemask=(null),cpuset=/,mems_allowed=0
[84060.080248] CPU: 0 PID: 7462 Comm: Main Tainted: G OE 5.15.85evl-g0b08924b969c #3
[84060.080252] Hardware name: Supermicro Super Server/X10SDV-6C-TLN4F, BIOS 1.3 02/13/2018
[84060.080253] IRQ stage: Linux
[84060.080255] Call Trace:
[84060.080257] <TASK>
[84060.080260] dump_stack_lvl+0x81/0xbe
[84060.080268] dump_stack+0x10/0x16
[84060.080271] warn_alloc.cold.144+0x7b/0xdd
[84060.080279] ? __alloc_pages_direct_compact+0x169/0x180
[84060.080285] __alloc_pages_slowpath.constprop.133+0xca9/0xcd0
[84060.080290] ? inband_irq_restore+0x2b/0x40
[84060.080295] ? inband_irq_restore+0x2b/0x40
[84060.080298] __alloc_pages+0x2af/0x2d0
[84060.080303] __dma_direct_alloc_pages.constprop.24+0x181/0x220
[84060.080310] dma_direct_alloc+0x45/0x170
[84060.080314] dma_alloc_attrs+0x39/0x60
[84060.080318] alloc_dma_buf+0x9d/0xe0 [dma_buf_mgr]
[84060.080324] dma_buf_mgr_ioctl+0xa9/0x280 [dma_buf_mgr]
[84060.080329] __x64_sys_ioctl+0x91/0xd0
[84060.080334] do_syscall_64+0x4c/0xa0
[84060.080338] entry_SYSCALL_64_after_hwframe+0x61/0xcb
[84060.080344] RIP: 0033:0x7f4ac00d2307
[84060.080348] Code: 44 00 00 48 8b 05 69 1b 2d 00 64 c7 00 26 00 00 00 48 c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 39 1b 2d 00 f7 d8 64 89 01 48
[84060.080351] RSP: 002b:00007f4ad1e67568 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
[84060.080355] RAX: ffffffffffffffda RBX: 00000000ac0a0e00 RCX: 00007f4ac00d2307
[84060.080357] RDX: 00007f4ad1e67580 RSI: 00000000c0102500 RDI: 0000000000000070
[84060.080359] RBP: 00000000002dc6c0 R08: 00007f4ad1e78700 R09: 00007f4ad1e67ac8
[84060.080361] R10: 0000000000000000 R11: 0000000000000246 R12: 00000000002dc6c0
[84060.080363] R13: 00007f4ad1e67ae8 R14: 00007f4ad1e679e0 R15: 00000000ac0b5980
[84060.080366] </TASK>
[84060.080367] Mem-Info:
[84060.080370] active_anon:269 inactive_anon:1461195 isolated_anon:0
active_file:526596 inactive_file:641587 isolated_file:0
unevictable:1089001 dirty:7 writeback:0
slab_reclaimable:74489 slab_unreclaimable:24737
mapped:71474 shmem:4450 pagetables:9652 bounce:0
kernel_misc_reclaimable:0
free:88878 free_pcp:1012 free_cma:20953
[84060.080377] Node 0 active_anon:1076kB inactive_anon:5844780kB active_file:2106384kB inactive_file:2566348kB unevictable:4356004kB isolated(anon):0kB isolated(file):0kB mapped:285896kB dirty:28kB writeback:0kB shmem:17800kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 2019328kB writeback_tmp:0kB kernel_stack:7936kB pagetables:38608kB all_unreclaimable? no
[84060.080384] Node 0 DMA free:11264kB min:64kB low:80kB high:96kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15972kB managed:15360kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
[84060.080390] lowmem_reserve[]: 0 1800 15786 15786 15786
[84060.080397] Node 0 DMA32 free:90408kB min:7696kB low:9620kB high:11544kB reserved_highatomic:6144KB active_anon:64kB inactive_anon:418192kB active_file:311208kB inactive_file:8kB unevictable:968252kB writepending:8kB present:1975960kB managed:1910424kB mlocked:968252kB bounce:0kB free_pcp:520kB local_pcp:520kB free_cma:0kB
[84060.080404] lowmem_reserve[]: 0 0 13986 13986 13986
[84060.080410] Node 0 Normal free:253840kB min:59820kB low:74772kB high:89724kB reserved_highatomic:2048KB active_anon:1012kB inactive_anon:5426476kB active_file:1795176kB inactive_file:2566340kB unevictable:3387752kB writepending:20kB present:14680064kB managed:14326668kB mlocked:3387752kB bounce:0kB free_pcp:4156kB local_pcp:0kB free_cma:83812kB
[84060.080417] lowmem_reserve[]: 0 0 0 0 0
[84060.080424] Node 0 DMA: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 1*1024kB (U) 1*2048kB (M) 2*4096kB (M) = 11264kB
[84060.080443] Node 0 DMA32: 36*4kB (UM) 133*8kB (UME) 309*16kB (UME) 305*32kB (UME) 197*64kB (UME) 138*128kB (UME) 67*256kB (UME) 16*512kB (UME) 1*1024kB (E) 9*2048kB (ME) 0*4096kB = 90984kB
[84060.080468] Node 0 Normal: 6742*4kB (UEC) 5324*8kB (UEHC) 3012*16kB (UMEHC) 1297*32kB (UMEHC) 415*64kB (UMEHC) 192*128kB (MEC) 1*256kB (C) 0*512kB 1*1024kB (C) 11*2048kB (C) 5*4096kB (C) = 254680kB
[84060.080495] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB
[84060.080497] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
[84060.080500] 1201421 total pagecache pages
[84060.080501] 0 pages in swap cache
[84060.080502] Swap cache stats: add 0, delete 0, find 0/0
[84060.080504] Free swap = 0kB
[84060.080505] Total swap = 0kB
[84060.080507] 4167999 pages RAM
[84060.080508] 0 pages HighMem/MovableOnly
[84060.080509] 104886 pages reserved
[84060.080510] 786432 pages cma reserved
[84060.080511] 0 pages hwpoisoned
[84060.080513] dma_buf_mgr ALRM: alloc_dma_buf: 14: Failed to allocated DMA buffer
I increased the amount of CMA in order to try and resolve this problem and it did not solve it. It did however increase the number of runs until I see this problem.

Related

usb_control_msg returns -EAGAIN

I have a Customized board which interfaces over USB.. I am writing USB Linux driver.
Everything is working fine when I test it on my Virtual Machine.. But when I switch to the real hardware and use Yocto on hardware and run the following code.. It fails with -EAGAIN..
retval = usb_control_msg(serial->dev,
usb_rcvctrlpipe(serial->dev, 0),
CP210X_GET_MDMSTS,
USB_DIR_IN | USB_TYPE_VENDOR | USB_RECIP_INTERFACE,
0,
i,
(void *)&modem_status,
1,
USB_CTRL_SET_TIMEOUT);
retval returns with -11. I am not sure why this is not happening on the Virtual Machine, as the only difference is that the PIC Board gets connected to X86 Customized board and runs linux..
dmesg output:
transfer buffer not dma capable
------------[ cut here ]------------
WARNING: CPU: 1 PID: 754 at /usr/src/kernel/drivers/usb/core/hcd.c:1595 usb_hcd_map_urb_for_dma+0x3e6/0x5b0
Modules linked in: cp2108(O)
CPU: 1 PID: 754 Comm: test_quad Tainted: G O 4.14.68-intel-pk-standard #1
task: ffff9a33b7d4a4c0 task.stack: ffff9ce5c0130000
RIP: 0010:usb_hcd_map_urb_for_dma+0x3e6/0x5b0
RSP: 0018:ffff9ce5c0133b08 EFLAGS: 00010296
RAX: 000000000000001f RBX: ffff9a33b7d89780 RCX: 0000000000000000
RDX: ffff9a33bfc9d680 RSI: ffff9a33bfc95598 RDI: ffff9a33bfc95598
RBP: ffff9ce5c0133b28 R08: 0000000000000001 R09: 0000000000000328
R10: ffff9a33ba840068 R11: 0000000000000000 R12: ffff9a33ba2ea000
R13: 00000000fffffff5 R14: 0000000001400000 R15: 0000000000000200
FS: 00007fac9eeed4c0(0000) GS:ffff9a33bfc80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000003c334e3cb0 CR3: 0000000179498000 CR4: 00000000003406e0
Call Trace:
usb_hcd_submit_urb+0x420/0xa00
? tty_port_open+0xa7/0xd0
? tty_ldisc_unlock+0x1a/0x20
? tty_open_proc_set_tty+0x7f/0xb0
? tty_unlock+0x29/0x40
? tty_open+0x38e/0x450
usb_submit_urb+0x364/0x550
usb_start_wait_urb+0x5f/0xe0
usb_control_msg+0xc5/0x110
cp210x_ioctl+0x4d2/0xe20 [cp2108]
? filemap_map_pages+0x129/0x290
? do_filp_open+0xa0/0xf0
serial_ioctl+0x46/0x50
tty_ioctl+0xe7/0x870
do_vfs_ioctl+0x99/0x5e0
? putname+0x4c/0x60
SyS_ioctl+0x79/0x90
Can you guys please provide a hint for me to try.. Appreciate your time and efforts.
I got the solution Kernel >= 4.9 no longer accepts any statically allocated buffer.
Modified the code to use dynamic memory and it worked.
modem_status = kmalloc(sizeof(unsigned long), GFP_KERNEL);
if (!modem_status)
return -ENOMEM;
for (i = 0; i < MAX_CP210x_INTERFACE_NUM; i++) {
retval = usb_control_msg(serial->dev,
usb_rcvctrlpipe(serial->dev, 0),
CP210X_GET_MDMSTS,
USB_DIR_IN | USB_TYPE_VENDOR | USB_RECIP_INTERFACE,
0,
i,
(void *)modem_status,
1,
USB_CTRL_SET_TIMEOUT);

Windbg Native memory leak unable to find root cause

I have memory leak on .Net Web service application.Upon suggestion, I'm able to analyze the dump file. I guess it is a native memory leak. But, I'm unable to figure out the root cause of the issue. I have followed steps mentioned in the link
Here what I have so far
address summary
--- Usage Summary ---------------- RgnCount ----------- Total Size -------- %ofBusy %ofTotal
Heap 328 4a256000 ( 1.159 GB) 69.38% 57.93%
<unknown> 1253 1b64b000 ( 438.293 MB) 25.63% 21.40%
Free 246 151fa000 ( 337.977 MB) 16.50%
Native Heap
0:000> !heap -s
Heap Flags Reserv Commit Virt Free List UCR Virt Lock Fast
(k) (k) (k) (k) length blocks cont. heap
-----------------------------------------------------------------------------
001b0000 00000002 1036480 1024552 1036480 411 745 68 5 0 LFH
00010000 00008000 64 4 64 2 1 1 0 0
001b0000 is using more than 1GB
Allocation info
0:000> !heap -stat -h 001b0000
heap # 001b0000
group-by: TOTSIZE max-display: 20
size #blocks total ( %) (percent of total busy bytes)
4e24 3a24 - 11bf2510 (28.82)
1001f e89 - e8ac297 (23.61)
Filtering 4e24
0:000> !heap -flt s 4e24
_HEAP # 1b0000
HEAP_ENTRY Size Prev Flags UserPtr UserSize - state
01fa4810 09c6 0000 [00] 01fa4818 04e24 - (busy)
01fa9640 09c6 09c6 [00] 01fa9648 04e24 - (busy)
01fae470 09c6 09c6 [00] 01fae478 04e24 - (busy)
There are ton of busy
0:000> dc 01fb32a0 L 2000
01fb32a0 b87718ff 0c12a52e 6f4d3c00 656c7564 ..w......<Module
01fb32b0 6977003e 4d2e796c 4d6b636f 6c75646f >.wily.MockModul
01fb32c0 65440065 6c756166 79442074 696d616e e.Default Dynami
01fb32d0 6f4d2063 656c7564 6c697700 67412e79 c Module.wily.Ag
01fb32e0 00746e65 2e6d6f63 796c6977 6573692e ent.com.wily.ise
01fb32f0 7261676e 65722e64 74736967 49007972 ngard.registry.I
01fb3300 69676552 79727473 76726553 00656369 RegistryService.
01fb3310 6f63736d 62696c72 73795300 006d6574 mscorlib.System.
01fb3320 656a624f 50007463 79786f72 67655249 Object.ProxyIReg
01fb3330 72747369 72655379 65636976 6f4d4e00 istryService.NMo
01fb3340 49006b63 6f766e49 69746163 61486e6f ck.IInvocationHa
01fb3350 656c646e 695f0072 636f766e 6f697461 ndler._invocatio
01fb3360 6e61486e 72656c64 73795300 2e6d6574 nHandler.System.
01fb3370 6c666552 69746365 4d006e6f 6f687465 Reflection.Metho
01fb3380 666e4964 6d5f006f 6f687465 666e4964 dInfo._methodInf
01fb3390 70614d6f 766e4900 00656b6f 2e6d6f63 oMap.Invoke.com.
01fb33a0 796c6977 6573692e 7261676e 74752e64 wily.isengard.ut
01fb33b0 742e6c69 00656572 65726944 726f7463 il.tree.Director
01fb33c0 74615079 646e4168 72746e45 65520079 yPathAndEntry.Re
01fb33d0 74736967 6e457972 00797274 72657571 gistryEntry.quer
01fb33e0 746e4579 73656972 6d6f6300 6c69772e yEntries.com.wil
01fb33f0 73692e79 61676e65 6f2e6472 696f676e y.isengard.ongoi
01fb3400 7571676e 00797265 65755141 6f4e7972 ngquery.AQueryNo
01fb3410 69666974 69746163 72006e6f 73696765 tification.regis
01fb3420 4f726574 696f676e 7551676e 00797265 terOngoingQuery.
01fb3430 2e6d6f63 796c6977 6573692e 7261676e com.wily.isengar
01fb3440 6f702e64 666f7473 65636966 736f5000 d.postoffice.Pos
01fb3450 66664f74 53656369 69636570 72656966 tOfficeSpecifier
01fb3460 72694400 6f746365 61507972 61006874 .DirectoryPath.a
01fb3470 6e456464 00797274 45746567 7972746e ddEntry.getEntry
01fb3480 6c656400 45657465 7972746e 74656700 .deleteEntry.get
01fb3490 44627553 63657269 69726f74 2e007365 SubDirectories..
01fb34a0 726f7463 74632e00 0000726f 00000000 ctor..ctor......
01fb34b0 00000000 00000000 00000000 00000000 ...............
I'm not sure if I'm in the right path of casue analysis
That < Module > in the beginning is a sign of dynamically generated assembly.
Load SOS extension using .loadby sos clr (for current machine dump) or .cordll -ve -u -l if you debug someone else's dump (doesn't work well in old Windbg 6.x, but works well for WinDbg from Windows Development Kit 8 and above)
Execute !eeheap and check the Module Thunk heaps section. It should contain thousands of records:
--------------------------------------
Module Thunk heaps:
Module 736b1000: Size: 0x0 (0) bytes.
Module 004f2ed4: Size: 0x0 (0) bytes.
...
<thousands of similar lines>
...
Total size: Size: 0x0 (0) bytes.
In my case it was assemblies generated for serialization by MS XmlSerializer class that took all the memory:
00000000`264b7640 00 00 3e 00 ce 01 00 00 00 00 00 00 00 3c 4d 6f ..>..........<Mo
00000000`264b7650 64 75 6c 65 3e 00 4d 69 63 72 6f 73 6f 66 74 2e dule>.Microsoft.
00000000`264b7660 47 65 6e 65 72 61 74 65 64 43 6f 64 65 00 52 65 GeneratedCode.Re
00000000`264b7670 66 45 6d 69 74 5f 49 6e 4d 65 6d 6f 72 79 4d 61 fEmit_InMemoryMa
00000000`264b7680 6e 69 66 65 73 74 4d 6f 64 75 6c 65 00 6d 73 63 nifestModule.msc
00000000`264b7690 6f 72 6c 69 62 00 53 79 73 74 65 6d 2e 53 65 63 orlib.System.Sec
I could avoid this leak by using only a single instance of XmlSerializer for each type.
In your case it seems that some other thing (wily.MockModule) generates assemblies, and some other solution might be required

How to send a UDP packet from inside linux kernel

I'm modifying the UDP protocol such that when connect() is called on a UDP socket, in addition to finding the route, a "Hello" packet is also sent to the destination.
From the UDP proto structure, I figured out that the function ip4_datagram_connect does the job of finding the route to the destination. Now at the end of this function, I need to send the Hello packet.
I don't think I can use udp_sendmsg() as it's used for copying and sending data from the userspace.
I think udp_send_skb() should be used to sent the hello. My problem is I don't know how to create an appropriate skbuff to store the Hello message (it should be a proper udp datagram) to be passed on to udp_send_skb(). I've tried this
int quic_connect(struct sock *sk, struct flowi4 *fl4, struct rtable *rt){
struct sk_buff *skb;
char *hello;
int err = 0, exthdrlen, hh_len, datalen, trailerlen;
char *data;
hh_len = LL_RESERVED_SPACE(rt->dst.dev);
exthdrlen = rt->dst.header_len;
trailerlen = rt->dst.trailer_len;
datalen = 200;
//Create a buffer to be send without fragmentation
skb = sock_alloc_send_skb(sk,
exthdrlen + datalen + hh_len + trailerlen + 15,
MSG_DONTWAIT, &err);
if (skb == NULL)
goto out;
skb->ip_summed = CHECKSUM_PARTIAL; // Use hardware checksum
skb->csum = 0;
skb_reserve(skb, hh_len);
skb_shinfo(skb)->tx_flags = 1; //Time stamp the packet
/*
* Find where to start putting bytes.
*/
data = skb_put(skb, datalen + exthdrlen);
skb_set_network_header(skb, exthdrlen);
skb->transport_header = (skb->network_header +
sizeof(struct iphdr));
err = udp_send_skb(skb, fl4);
However, this gives me errors in the kernel log
BUG: unable to handle kernel NULL pointer dereference at 0000000000000018
IP: [<ffffffff81686555>] __ip_local_out+0x45/0x80
PGD 4f4dd067 PUD 4f4df067 PMD 0
Oops: 0000 [#1] SMP
Modules linked in:
CPU: 0 PID: 3019 Comm: client Not tainted 3.13.11-ckt39-test006 #28
Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
task: ffff8800598df6b0 ti: ffff880047022000 task.ti: ffff880047022000
RIP: 0010:[<ffffffff81686555>] [<ffffffff81686555>] __ip_local_out+0x45/0x80
RSP: 0018:ffff880047023d78 EFLAGS: 00010287
RAX: 0000000000000001 RBX: ffff880047008a00 RCX: 0000000020000000
RDX: 0000000000000000 RSI: ffff880047008a00 RDI: ffff8800666fde40
RBP: ffff880047023d88 R08: 0000000000003200 R09: 0000000000000001
R10: 0000000000000000 R11: 00000000000001f9 R12: ffff880047008a00
R13: ffff8800666fde80 R14: ffff880059aec380 R15: ffff880059aec690
FS: 00007f5508b04740(0000) GS:ffff88007fc00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 0000000000000018 CR3: 000000004f561000 CR4: 00000000000406f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Stack:
ffff880047023d80 ffff880047008a00 ffff880047023da0 ffffffff8168659d
ffffffff81c8f8c0 ffff880047023db8 ffffffff81687810 0000000000000000
ffff880047023df8 ffffffff816ac6be 0000000000000020 ffff880047008a00
Call Trace:
[<ffffffff8168659d>] ip_local_out+0xd/0x30
[<ffffffff81687810>] ip_send_skb+0x10/0x40
[<ffffffff816ac6be>] udp_send_skb+0x14e/0x3d0
[<ffffffff816b0e9e>] quic_connect+0x6e/0x80
[<ffffffff816aa3ff>] __ip4_datagram_connect+0x2bf/0x2d0
[<ffffffff816aa437>] ip4_datagram_connect+0x27/0x40
[<ffffffff816b8748>] inet_dgram_connect+0x38/0x80
[<ffffffff8161fd97>] SYSC_connect+0xc7/0x100
[<ffffffff817ed471>] ? __schedule+0x341/0x8c0
[<ffffffff816206e9>] SyS_connect+0x9/0x10
[<ffffffff817f8d42>] system_call_fastpath+0x16/0x1b
Code: c8 00 00 00 66 c1 c0 08 66 89 47 02 e8 d5 e0 ff ff 48 8b 53 58 b8 01 00 00 00 48 83 e2 fe 48 81 3d 9d 0e 64 00 f0 73 cc 81 74 26 <4c> 8b 42 18 49 c7 c1 f0 45 68 81 c7 04 24 00 00 00 80 31 c9 48
RIP [<ffffffff81686555>] __ip_local_out+0x45/0x80
RSP <ffff880047023d78>
CR2: 0000000000000018
---[ end trace 474c5db1b9b19a03 ]---
So my question is, what else do I need to fill in my skbuff before it can be handled properly by udp_send_skb. Or am I missing something else here?
There is a bug in your code.
if (skb_tailroom(hbuff) > 30) {
printk(" Enough room for QUIC connect message\n");
hello = kmalloc(30, GFP_ATOMIC); //You allocate slub memory
hello = "Hello from QUIC connect"; //You let 'hello' point to a string,
//which is stored somewhere else.
//At this point, your slub memory
//allocated is lost.
memcpy(__skb_put(hbuff, 30), hello, 30);
kfree(hello); //You try to free the memory pointed by
//hello as slub memory, I think this is
// why you get mm/slub.c bug message.
} else
You can change your code like this:
if (skb_tailroom(hbuff) > 30) {
printk(" Enough room for QUIC connect message\n");
memcpy(__skb_put(hbuff, 30), "Hello from QUIC connect", 30);
} else
I just used the functions ip_make_skb followed by ip_send_skb. Since ip_make_skb is used to copy data from user space and I didn't need to do that, I just used a dummy pointer and provided the length to be copied as Zero (+sizeof udphdr, as required by this function). It's a dirty way to do it, but it works for now.
My code:
int quic_connect(struct sock *sk, struct flowi4 *fl4, struct rtable *rt, int oif){
struct sk_buff *skb;
struct ipcm_cookie ipc;
void *hello = "Hello";
int err = 0;
ipc.opt = NULL;
ipc.tx_flags = 1;
ipc.ttl = 0;
ipc.tos = -1;
ipc.oif = oif;
ipc.addr = fl4->daddr;
skb = ip_make_skb(sk, fl4, ip_generic_getfrag, hello, sizeof(struct udphdr),
sizeof(struct udphdr), &ipc, &rt,
0);
err = PTR_ERR(skb);
if (!IS_ERR_OR_NULL(skb)){
err = udp_send_skb(skb, fl4);
}
return err;
}

Why hrtimer causing a deadlock of soft interrupt?

I want to use hrtimer, but the module can't work. the source codes is:
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/hrtimer.h>
#define US_TO_NS(usec) ((usec) * 1000)
#define MS_TO_US(msec) ((msec) * 1000)
#define MS_TO_NS(msec) ((msec) * 1000 * 1000)
DEFINE_PER_CPU(struct hrtimer, send_hr_timer);
enum hrtimer_restart hrtimer_send_skb(struct hrtimer *timer)
{
ktime_t ktime;
int cpu;
unsigned long delay_in_ms = 10000L; //unsigned long delay_in_us = 5000L;
ktime = ktime_set(0, MS_TO_NS(delay_in_ms));
//ktime = ktime_set(0, US_TO_NS(delay_in_us));
cpu = get_cpu(); printk(KERN_INFO "hrtimer is on:%d", cpu);
hrtimer_forward(&per_cpu(send_hr_timer, cpu), ktime_get(), ktime);
put_cpu();
return HRTIMER_RESTART;
//return HRTIMER_NORESTART;
}
void init_hr_timer(void)
{
ktime_t ktime;
int cpu;
unsigned long delay_in_ms = 100L;
ktime = ktime_set(0, MS_TO_NS(delay_in_ms));
for_each_online_cpu(cpu) {
hrtimer_init(&per_cpu(send_hr_timer, cpu), CLOCK_MONOTONIC, HRTIMER_MODE_REL);
per_cpu(send_hr_timer, cpu).function = &hrtimer_send_skb;
hrtimer_start(&per_cpu(send_hr_timer, cpu), ktime, HRTIMER_MODE_REL);
}
}
void del_hr_timer(void)
{
int cpu;
for_each_online_cpu(cpu) {
/* del hrtimer. */
hrtimer_cancel(&per_cpu(send_hr_timer, cpu));
}
}
static int minit(void)
{
int rc = 0;
printk("Start %s.\n", THIS_MODULE->name);
init_hr_timer();
return rc;
}
static void mexit(void)
{
printk("Exit %s.\n", THIS_MODULE->name);
del_hr_timer();
}
When I install the module, something is error, the messages is:
BUG: soft lockup - CPU#22 stuck for 67s! [migration/22:92]
Modules linked in: ws_hrtimer(U) wscubic(U) wsfasttcp(U) 8021q garp stp llc autofs4 cpufreq_ondemand acpi_cpufreq freq_table mperf be2iscsi iscsi_boot_sysfs ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp bnx2i cnic uio ipv6 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp libiscsi scsi_transport_iscsi ext3 jbd dm_mirror dm_multipath video output sbs sbshc power_meter wsrelay(U) wsgetsnd(U) parport sg acpi_pad ses enclosure igb dca sb_edac edac_core i2c_i801 i2c_core pata_acpi ata_generic iTCO_wdt iTCO_vendor_support dm_raid45 dm_memcache xor dm_region_hash dm_log dm_mod usb_storage ata_piix isci libsas shpchp mpt2sas scsi_transport_sas raid_class ext4 mbcache jbd2 [last unloaded: microcode]
CPU 22
Modules linked in: ws_hrtimer(U) wscubic(U) wsfasttcp(U) 8021q garp stp llc autofs4 cpufreq_ondemand acpi_cpufreq freq_table mperf be2iscsi iscsi_boot_sysfs ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr iscsi_tcp bnx2i cnic uio ipv6 cxgb3i libcxgbi cxgb3 mdio libiscsi_tcp libiscsi scsi_transport_iscsi ext3 jbd dm_mirror dm_multipath video output sbs sbshc power_meter wsrelay(U) wsgetsnd(U) parport sg acpi_pad ses enclosure igb dca sb_edac edac_core i2c_i801 i2c_core pata_acpi ata_generic iTCO_wdt iTCO_vendor_support dm_raid45 dm_memcache xor dm_region_hash dm_log dm_mod usb_storage ata_piix isci libsas shpchp mpt2sas scsi_transport_sas raid_class ext4 mbcache jbd2 [last unloaded: microcode]
Pid: 92, comm: migration/22 Not tainted 2.6.32-358.6.1.ws5.b.5.2.8 #1 Inspur SA5212H2/SA5212H2
RIP: 0010:[<ffffffff810c6709>] [<ffffffff810c6709>] stop_machine_cpu_stop+0x59/0x100
RSP: 0018:ffff880874733de0 EFLAGS: 00000293
RAX: 0000000000000001 RBX: ffff880874733e10 RCX: 0000000000000028
RDX: ffffffff81c059e0 RSI: 0000000000000018 RDI: ffffffff81c059e0
RBP: ffffffff8100ba8e R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
R13: 0000000000000000 R14: ffff881074f0e040 R15: 0000000000000082
FS: 0000000000000000(0000) GS:ffff88089c540000(0000) knlGS:0000000000000000
CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b
CR2: 000000373c89ac00 CR3: 0000000001a85000 CR4: 00000000000407e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process migration/22 (pid: 92, threadinfo ffff880874732000, task ffff880874731540)
Stack:
0000001674733e00 ffff88089c550840 ffff880874731540 ffffffff810c66b0
<d> ffff8806fa783e68 ffff8806fa783dd8 ffff880874733ee0 ffffffff810c68d5
<d> 0000000000000000 0000002165ed0d57 0000000000000082 ffff880874415ca0
Call Trace:
[<ffffffff810c66b0>] ? stop_machine_cpu_stop+0x0/0x100
[<ffffffff810c68d5>] ? cpu_stopper_thread+0xa5/0x170
[<ffffffff8105a992>] ? default_wake_function+0x12/0x20
[<ffffffff8104e1eb>] ? __wake_up_common+0x5b/0x90
[<ffffffff810c6830>] ? cpu_stopper_thread+0x0/0x170
[<ffffffff810c6830>] ? cpu_stopper_thread+0x0/0x170
[<ffffffff81090ce6>] ? kthread+0x96/0xa0
[<ffffffff8100bfca>] ? child_rip+0xa/0x20
[<ffffffff81090c50>] ? kthread+0x0/0xa0
[<ffffffff8100bfc0>] ? child_rip+0x0/0x20
Code: 18 48 85 c0 0f 84 9b 00 00 00 0f a3 18 19 c0 85 c0 41 0f 95 c6 31 c0 45 31 ff eb 0d 66 0f 1f 44 00 00 83 fb 04 74 3b 89 d8 f3 90 <41> 8b 5c 24 20 39 c3 74 ee 83 fb 02 74 49 83 fb 03 74 54 f0 41
Call Trace:
[<ffffffff810c66b0>] ? stop_machine_cpu_stop+0x0/0x100
[<ffffffff810c68d5>] ? cpu_stopper_thread+0xa5/0x170
[<ffffffff8105a992>] ? default_wake_function+0x12/0x20
[<ffffffff8104e1eb>] ? __wake_up_common+0x5b/0x90
[<ffffffff810c6830>] ? cpu_stopper_thread+0x0/0x170
[<ffffffff810c6830>] ? cpu_stopper_thread+0x0/0x170
[<ffffffff81090ce6>] ? kthread+0x96/0xa0
[<ffffffff8100bfca>] ? child_rip+0xa/0x20
[<ffffffff81090c50>] ? kthread+0x0/0xa0
[<ffffffff8100bfc0>] ? child_rip+0x0/0x20
please help me, thank you.

dispatch queue causing sigabrt after a loop ends

I am getting a strange crash. I am writing data on bluetooth device. when the loop ends the app gets SIGABRT. I am not able to figure out the crash till now. Could someone help me to fix this ?
The app crashes in this function if the loop is called 2 times and it does not crash if it called 1 time.
BOOL success = [strongSelf.CronoDeviceObject.deviceHandler writeSettingsScreenData:data module:strongSelf.CronoDeviceObject.moduleNumber];
#pragma mark Button Save and cancel
- (IBAction)OkTaped:(id)sender{
//for submit
if ([self validateData]) {
[[ProgressHUD defaultHUD] setLoadingText:CVLocalizedString(#"Updating",nil)];
[[ProgressHUD defaultHUD]showInView:self.view];
[self performSelector:#selector(saveAllSettings) withObject:nil afterDelay:1.0];
}
}
- (void)saveAllSettings {
__weak __typeof__(self) weakSelf = self;
dispatch_queue_t queue = dispatch_queue_create("writesettings",NULL);
dispatch_async(queue, ^{
__strong __typeof__(self) strongSelf = weakSelf;
if (strongSelf.CronoDeviceObject) {
[strongSelf.CronoDeviceObject.deviceHandler writeFastWindUp:strongSelf.CronoDeviceObject.moduleNumber];
[strongSelf.CronoDeviceObject.deviceHandler writeLightOnOff:strongSelf.CronoDeviceObject.moduleNumber state:strongSelf.btnCheckBox2.selected?0x01:0x00];
strongSelf.CronoDeviceObject.lightState = (strongSelf.btnCheckBox2.selected)?0:1;
NSLog(#"Button Light is %#",([strongSelf.CronoDeviceObject isEnergySavingMode])?#"ON":#"Off");
NSString* semicolon =[NSString stringWithFormat:#";%#;%#;%#",strongSelf.BrandNametxtfld.text,strongSelf.ModalNametxtfld.text,[NSString stringWithFormat:#"%#;", strongSelf.WatchWinderNametxtfld.text]];
NSMutableData* data = [[NSMutableData alloc] initWithData:[semicolon dataUsingEncoding:NSUTF8StringEncoding]];
if ([strongSelf.CronoDeviceObject.deviceHandler respondsToSelector:#selector(writeSettingsScreenData:module:)]) {
BOOL success = [strongSelf.CronoDeviceObject.deviceHandler writeSettingsScreenData:data module:strongSelf.CronoDeviceObject.moduleNumber];
**//The app does not get here if the above function loop called 2 times but if it called 1 time it's working fine.**
if (success) {
dispatch_async(dispatch_get_main_queue(), ^{
__strong __typeof__(self) strongSelf = weakSelf;
self.CronoDeviceObject.Devicemodal.Brandname = strongSelf.BrandNametxtfld.text;
self.CronoDeviceObject.Devicemodal.modalName = strongSelf.ModalNametxtfld.text;
self.CronoDeviceObject.Devicemodal.winderName = strongSelf.WatchWinderNametxtfld.text;
// Update the UI
[[ProgressHUD defaultHUD]hideActivityIndicator];
[strongSelf.navigationController popViewControllerAnimated:YES];
});
}
}
}
});
}
- (BOOL) writeSettingsScreenData:(NSMutableData*) payload module:(Byte) module {
if (payload == nil) {
return false;
}
int numwrites = 1;
numwrites += (payload.length - 1)/32;
numwrites = (numwrites>8)?8:numwrites;
for(int i = 0; i < numwrites; i++){
//#try {
NSUInteger bufferLength = (payload.length - i*32);
Byte buff[32];
memset(buff, 0, 32);
[payload getBytes:&buff range:NSMakeRange(i*32, bufferLength)];
self.finalString = #"";
//rcvbuff = new StringBuffer(32);
self.chronoVisionMessage = [ChronoVisionMessage new];
//Set Line Address And Dir(R)
// chronoVisionMessage.setLineAddr(ChronoVisionMessage.LINE_BROADCAST);
// chronoVisionMessage.setLineDir(ChronoVisionMessage.LINE_WRITE);
//Set Module (communicnMstr/Address)
[self.chronoVisionMessage setModRsvd:(MODULE_BUSMASTER)];//set to 0
[self.chronoVisionMessage setModCh:(MODULE_BUSMASTER)];//set to default zero
if (module == 0) {
[self.chronoVisionMessage setModAddr:(MODULE_BROADCAST)];
}
else{
[self.chronoVisionMessage setModAddr:module];
}
//Set Register Access(R)/Addresss
[self.chronoVisionMessage setRegRw:REIGSTER_WRITE];
[self.chronoVisionMessage setRegType:REIGSTER_ACCESSTYPE1];//Access standard reg
[self.chronoVisionMessage setRegAddr:REIGSTER_ACCESSTYPE0];//set to 0
//[self.chronoVisionMessage setValueHi:REIGSTER_ACCESSTYPE0];//set to 0
[self.chronoVisionMessage setValueLo:((Byte)i)];//set to 0 to 7
[self.chronoVisionMessage setNewData:[NSMutableData dataWithBytes:buff length:32]];//set to 0
#try {
NSMutableData* header = [self.chronoVisionMessage getMessage];
NSMutableData* final = [[NSMutableData alloc] initWithBytes:[header bytes] length:[header length]];
[final appendBytes:buff length:32];
[self writeData:final];
int retryCount = 0;
while (!self.allDataReceived && retryCount<kTimeout) {
[NSThread sleepForTimeInterval:0.1];
}
//String allData = read(5000);
if(self.finalString.length == 0){
//_peripheral.setListener(null);
return false;
}
if ([self.finalString length] >= 8 && [self.finalString rangeOfString:#"\r"].location > 0) {
self.finalString = [self.finalString substringWithRange:NSMakeRange(0, [self.finalString rangeOfString:#"\r"].location)];
}
NSData *finalData = [self.finalString dataFromHex];
NSLog(#"final readBuff is %#",finalData);
if (i==(numwrites-1) && finalData.length >= 36) {
return true;
}
//free(buff);
}
#catch (NSException *exception) {
NSLog(#"Exception occured when writing user data %#",[exception description]);
}
}
return false;
}
here is an image after the crash
here is the back trace log
bt
* thread #12: tid = 0x13cf99, 0x3611c9aa libsystem_c.dylib`__abort + 102, queue = 'com.apple.root.default-qos', stop reason = EXC_BREAKPOINT (code=EXC_ARM_BREAKPOINT, subcode=0xdefe)
* frame #0: 0x3611c9aa libsystem_c.dylib`__abort + 102
frame #1: 0x3611cf94 libsystem_c.dylib`__stack_chk_fail + 180
frame #2: 0x0007a2d4 Chronovision`-[DeviceHandler writeSettingsScreenData:module:](self=0x17081990, _cmd=0x001069c2, payload=0x00000000, module='\x02') + 3028 at DeviceHandler.m:435
frame #3: 0x000d1f2c Chronovision`__34-[CVDetailViewController OkTaped:]_block_invoke(.block_descriptor=<unavailable>) + 400 at CVDetailViewController.m:357
frame #4: 0x002b219a libdispatch.dylib`_dispatch_call_block_and_release + 10
frame #5: 0x002bcc48 libdispatch.dylib`_dispatch_root_queue_drain + 1596
frame #6: 0x002bde20 libdispatch.dylib`_dispatch_worker_thread3 + 108
frame #7: 0x361fada8 libsystem_pthread.dylib`_pthread_wqthread + 668
(lldb)
Update
I ran the app in xcode 7.1.1 and I got the following log in console when the app crashes.
Dec 7 16:55:09 Rahuls-iPad Chronovision[1650] <Error>: ================================================================
==1650==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x088e06a0 at pc 0x0036f4a9 bp 0x088e02c4 sp 0x088dfe94
WRITE of size 42 at 0x088e06a0 thread T23
#0 0x36f4a7 in wrap_memmove (/private/var/mobile/Containers/Bundle/Application/33337E25-C69D-49F5-ABA1-5AD65673725C/Chronovision.app/Frameworks/libclang_rt.asan_ios_dynamic.dylib+0x2a4a7)
#1 0x287936b3 in <redacted> (/System/Library/Frameworks/Foundation.framework/Foundation+0x236b3)
#2 0x8a971 in -[DeviceHandler writeSettingsScreenData:module:] (/private/var/mobile/Containers/Bundle/Application/33337E25-C69D-49F5-ABA1-5AD65673725C/Chronovision.app/Chronovision+0x2d971)
#3 0x139eaf in __41-[CVDetailViewController saveAllSettings]_block_invoke (/private/var/mobile/Containers/Bundle/Application/33337E25-C69D-49F5-ABA1-5AD65673725C/Chronovision.app/Chronovision+0xdceaf)
#4 0x374e15 in __wrap_dispatch_async_block_invoke (/private/var/mobile/Containers/Bundle/Application/33337E25-C69D-49F5-ABA1-5AD65673725C/Chronovision.app/Frameworks/libclang_rt.asan_ios_dynamic.dylib+0x2fe15)
#5 0x853199 in _dispatch_call_block_and_release (/usr/lib/system/introspection/libdispatch.dylib+0x1199)
#6 0x85bd87 in _dispatch_queue_drain (/usr/lib/system/introspection/libdispatch.dylib+0x9d87)
#7 0x855ac3 in _dispatch_queue_invoke (/usr/lib/system/introspection/libdispatch.dylib+0x3ac3)
#8 0x85db19 in _dispatch_root_queue_drain (/usr/lib/system/introspection/libdispatch.dylib+0xbb19)
#9 0x85ee1f in _dispatch_worker_thread3 (/usr/lib/system/introspection/libdispatch.dylib+0xce1f)
#10 0x361fada7 in _pthread_wqthread (/usr/lib/system/libsystem_pthread.dylib+0xda7)
#11 0x361faafb in start_wqthread (/usr/lib/system/libsystem_pthread.dylib+0xafb)
Address 0x088e06a0 is located in stack of thread T23 at offset 128 in frame
#0 0x8a427 in -[DeviceHandler writeSettingsScreenData:module:] (/private/var/mobile/Containers/Bundle/Application/33337E25-C69D-49F5-ABA1-5AD65673725C/Chronovision.app/Chronovision+0x2d427)
This frame has 12 object(s):
[16, 24) 'r.i1'
[48, 56) 'r.i'
[80, 84) ''
[96, 128) 'buff'
[160, 168) '' <== Memory access at offset 128 partially underflows this variable
[192, 196) 'header'
[208, 212) 'final'
[224, 232) ''
[256, 264) ''
[288, 296) ''
[320, 324) 'finalData'
[336, 340) 'exception'
HINT: this may be a false positive if your program uses some custom stack unwind mechanism or swapcontext
(longjmp and C++ exceptions *are* supported)
Thread T23 created by T0 here:
<empty stack>
SUMMARY: AddressSanitizer: stack-buffer-overflow ??:0 wrap_memmove
Shadow bytes around the buggy address:
0x4111c080: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x4111c090: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x4111c0a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x4111c0b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x4111c0c0: 00 00 00 00 f1 f1 00 f2 f2 f2 00 f2 f2 f2 04 f2
=>0x4111c0d0: 00 00 00 00[f2]f2 f2 f2 00 f2 f2 f2 04 f2 04 f2
0x4111c0e0: 00 f2 f2 f2 00 f2 f2 f2 00 f2 f2 f2 04 f2 04 f3
0x4111c0f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x4111c100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x4111c110: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x4111c120: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Heap right redzone: fb
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack partial redzone: f4
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
==1650==ABORTING
I got the problem with xcode 7.1.1.
I don't know why xcode 6.2 was unable to detect the crash location.
The problem was in the range I was using in the function
- (BOOL) writeSettingsScreenData:(NSMutableData*) payload module:(Byte) module {
if (payload == nil) {
return false;
}
int numwrites = 1;
numwrites += (payload.length - 1)/32;
numwrites = (numwrites>8)?8:numwrites;
for(int i = 0; i < numwrites; i++){
//#try {
//Changed this
NSUInteger bufferLength = (payload.length - i*32);
Byte buff[32];
memset(buff, 0, 32);
[payload getBytes:&buff range:NSMakeRange(i*32, bufferLength)];
//To this
NSUInteger bufferLength = ((payload.length - i*32)>32)?32:(payload.length-i*32);
Byte buff[32];
memset(buff, 0, 32);
if (payload && payload.length > bufferLength) {
[payload getBytes:&buff range:NSMakeRange(i*32, bufferLength)];
}
self.finalString = #"";
//rcvbuff = new StringBuffer(32);
self.chronoVisionMessage = [ChronoVisionMessage new];
//Set Line Address And Dir(R)
// chronoVisionMessage.setLineAddr(ChronoVisionMessage.LINE_BROADCAST);
// chronoVisionMessage.setLineDir(ChronoVisionMessage.LINE_WRITE);
//Set Module (communicnMstr/Address)
[self.chronoVisionMessage setModRsvd:(MODULE_BUSMASTER)];//set to 0
[self.chronoVisionMessage setModCh:(MODULE_BUSMASTER)];//set to default zero
if (module == 0) {
[self.chronoVisionMessage setModAddr:(MODULE_BROADCAST)];
}
else{
[self.chronoVisionMessage setModAddr:module];
}
//Set Register Access(R)/Addresss
[self.chronoVisionMessage setRegRw:REIGSTER_WRITE];
[self.chronoVisionMessage setRegType:REIGSTER_ACCESSTYPE1];//Access standard reg
[self.chronoVisionMessage setRegAddr:REIGSTER_ACCESSTYPE0];//set to 0
//[self.chronoVisionMessage setValueHi:REIGSTER_ACCESSTYPE0];//set to 0
[self.chronoVisionMessage setValueLo:((Byte)i)];//set to 0 to 7
[self.chronoVisionMessage setNewData:[NSMutableData dataWithBytes:buff length:32]];//set to 0
#try {
NSMutableData* header = [self.chronoVisionMessage getMessage];
NSMutableData* final = [[NSMutableData alloc] initWithBytes:[header bytes] length:[header length]];
[final appendBytes:buff length:32];
[self writeData:final];
int retryCount = 0;
while (!self.allDataReceived && retryCount<kTimeout) {
[NSThread sleepForTimeInterval:0.1];
}
//String allData = read(5000);
if(self.finalString.length == 0){
//_peripheral.setListener(null);
return false;
}
if ([self.finalString length] >= 8 && [self.finalString rangeOfString:#"\r"].location > 0) {
self.finalString = [self.finalString substringWithRange:NSMakeRange(0, [self.finalString rangeOfString:#"\r"].location)];
}
NSData *finalData = [self.finalString dataFromHex];
NSLog(#"final readBuff is %#",finalData);
if (i==(numwrites-1) && finalData.length >= 36) {
return true;
}
//free(buff);
}
#catch (NSException *exception) {
NSLog(#"Exception occured when writing user data %#",[exception description]);
}
}
return false;
}

Resources