How a corrupted double linked list error is detected in malloc? - malloc

I don't understand a condition for gettint the error - corrupted double-linked list (not small) -
There is a point in the program where it crashes due to corrupted double-linked list (not small).
At this moment, I don't want to know why it crashed, but I want to know how this conditions work
if (__builtin_expect (P->fd_nextsize->bk_nextsize != P, 0) || __builtin_expect(P->bk_nextsize->fd_nextsize != P, 0))
malloc_printerr (check_action,"corrupted double-linked list (not small)", P, AV);
Here we can see that the error double-linked list is observed when the P->fd_nextsize->bk_nextsize != P is not accomplished or P->bk_nextsize->fd_nextsize is not accomplished.
But my main problem is that I can't explain how this part of code - P->fd_nextsize->bk_nextsize - expects to be P. I am expecting that P->fd_nextsize->bk_nextsize is P->bk_nextsize.
Maybe I am confused with the structure of the double linked list in the bins.
At this moment, I understand the double linked list as this:
+------------------->+----------------+ <---------+
| | fd_nextsize | |
| | | |
| | | |
| +----------------+ |
| | | |
| |fd_nextsize | |
| | | |
| +----------------+ |
| | | |
| +bk_nextisize |--------------------+
| | | | |
| +----------------+ | |
| | |
| | |
| | |
| | |
| +---------------+ | |
| | P | | |
| | | | |
| +---------------+ | |
| | | | |
| |fd_nextsize +-----------+ |
| | | |
| +---------------+ |
| | | |
| +-----------+bk_nextsize | |
| | | | |
| | +---------------+ |
| | |
| | |
| | |
| +----------->+----------------+<-----------------+
| | bk_nextisize |
| | |
| | |
| +----------------+
| | |
+----------------------+ fd_nextsize |
| |
+----------------+
|bk_nextsize |
| |
+----------------+
So, if I am incorrect about my understanding of the double linked lists in the bins (Important information, according to the code and also due to the fact that we are using fd_nextsize and bk_nextsize this chunk is not in the smallbin) please correct me. Since at this moment, this condition does not make any sense to me.

Example of corrupted double list with correct diagram.
gef> x/200x victim
0x2c76330: 0x00000000 0x00000000 0x000004d1 0x00000000
0x2c76340: 0x140e8fb8 0x00007f12 0x140e8fb8 0x00007f12
0x2c76350: 0x02c15e48 0x00000000 0x02c76330 0x00000000
0x2c76360: 0x00000318 0x00000000 0x00000000 0x00000000
Here
0x02c15e48 >>> fd_nextsize
0x02c76330 >>> bk_nextsize
gef> x/100x 0x02c15e48
0x2c15e48: 0x02c76348 0x00000000 0x00000000 0x00000000
0x2c15e58: 0x00000019 0x00000000 0x09080003 0x0071db5c
0x2c15e68: 0xc0b51d4f 0x00000001 0x000000fa 0x00000000
0x2c15e78: 0x00000003 0x00007706 0x00007706 0x586a0001
0xc0b51d4f >>> victim->fd_nextisize->fd_nextsize
0x000000fa >>> vicitim->fd_nextsize->bk_nextsize >>>>>>>>>>>>>>>>>>>>>>>>>>>>> This is not valid.
gef> x/100x 0x02c76330
0x2c76330: 0x00000000 0x00000000 0x000004d1 0x00000000
0x2c76340: 0x140e8fb8 0x00007f12 0x140e8fb8 0x00007f12
0x2c76350: 0x02c15e48 0x00000000 0x02c76330 0x00000000
0x2c76360: 0x00000318 0x00000000 0x00000000 0x00000000
0x02c15e48 >>> victim->bk_nextsize->fd_nextsize
0x02c76330 >>> victim->bk_nextsize->bk_nextsize >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> This condition is valid.
Double Linked Lists in GLIBC
+----------------------+<------------------+
| ChunkA | |
+-----------------------+ |
|fd_nextsize | |
+----------------------+ |
| | |
+---+bk_nextsize | |
| +----------------------+ |
| |
| +------------------------+<---------+ |
| | ChunkB | | |
| +------------------------+ | |
| |fd_nextsize | | |
| | | | |
| +------------------------+ | |
+-----------+bk_nextsize | | |
| | +------------------------+ | |
| | | |
| +->+------------------------+<-----------------------------+
| | ChunkC | | | |
| +------------------------+ | | |
| |fd_nextsize +------------------+ |
| | | | |
| +------------------------+ | |
| +------+bk_nextsize | | |
| | +------------------------+ | |
| | | |
+---------->+------------------------+ | |
| | ChunkD | | |
| +------------------------+ | |
| |fd_nextsize +----------+ |
| +------------------------+ |
| | | |
| |bk_nextsize | |
| +------------------------+ |
| |
+----> +------------------------+ |
| ChunkE | |
+------------------------+ |
|fd_nextsize +------------------------------+
| |
+------------------------+
|bk_nextsize |
+------------------------+

Related

AVPlayer crash on fullscreen

On iOS 13.3.1 the AVPlayer fullscreen is crashing. Once I rotate the screen the Xcode console shows an error Make a symbolic breakpoint at UICollectionViewFlowLayoutBreakForInvalidSizes to catch this in the debugger.
I have added a symbolic breakpoint, and debugger catch the following error
UIWindow:0x108b99360
| UITransitionView:0x108c53a10
| | UIView:0x11be462b0
| | | UIView:0x11beab9d0
| | | UIView:0x11a7c9890
| | | UIView:0x11bebacc0
| | | UIView:0x11be10d00
| | UIDropShadowView:0x113c14b00
| | | UILayoutContainerView:0x108bf0790
| | | | UITransitionView:0x108bc1410
| | | | | UIViewControllerWrapperView:0x108ba1590
| | | | | | •UILayoutContainerView:0x108ba25c0
| | | | | | | +UINavigationTransitionView:0x108bc04b0
| | | | | | | | +UIViewControllerWrapperView:0x11ae32650
| | | | | | | | | •UIView:0x11a7c90f0
| | | | | | | | | | *<UILayoutGuide: 0x280700b60 - "UIViewSafeAreaLayoutGuide", layoutFrame = {{0, 20}, {667, 425}}, owningView = <UIView: 0x11a7c90f0; frame = (0 0; 667 375); autoresize = W+H; animations = { position=<CABasicAnimation: 0x283e654e0>; bounds.origin=<CABasicAnimation: 0x283e65420>; bounds.size=<CABasicAnimation: 0x283e65460>; position-2=<CABasicAnimation: 0x283e60a20>; bounds.origin-2=<CABasicAnimation: 0x283e60980>; bounds.size-2=<CABasicAnimation: 0x283e60920>; }; layer = <CALayer: 0x283ebb900>>>- AMBIGUOUS LAYOUT for UILayoutGuide:0x280700b60'UIViewSafeAreaLayoutGuide'.Height{id: 8167}
| | | | | | | | | | *UIView:0x11a7c8e10
| | | | | | | | | | | •UIView:0x11ae5b960
| | | | | | | | | | | | *<UILayoutGuide: 0x28071fc60 - "UIViewSafeAreaLayoutGuide", layoutFrame = {{0, 0}, {375, 375}}, owningView = <UIView: 0x11ae5b960; frame = (0 0; 375 375); autoresize = W+H; gestureRecognizers = <NSArray: 0x2830ff750>; layer = <CALayer: 0x283f1b6a0>>>
| | | | | | | | | | | | *UIView:0x11ae598b0
| | | | | | | | | | | | *UIImageView:0x108ba76e0
| | | | | | | | | | | | *UIButton:0x11aea33b0
| | | | | | | | | | | | | UIImageView:0x108cf2320
| | | | | | | | | | | | *UILabel:0x11aea3660'Tap to reload'
| | | | | | | | | | | | *NTV.GradientView:0x11ae77740
| | | | | | | | | | | | | *UIView:0x108ba8140
| | | | | | | | | | | | | | *UILabel:0x108b93db0'Now Playing- Swasthya Sar...'
| | | | | | | | | | | | | | *UILabel:0x108ba64a0'2020-02-14 09:30 - 10:00'
| | | | | | | | | | | | | *UILabel:0x114c062e0'11:59:23'
| | | | | | | | | | | | | *UISlider:0x108ba8870 alignmentRect = (16 300.5; 343 18.5) for frame = (14 300.5; 347 19.5);
| | | | | | | | | | | | | | *UIProgressView:0x108c3b7c0
| | | | | | | | | | | | | | | UIImageView:0x108c142a0
| | | | | | | | | | | | | | | UIImageView:0x108c06090
| | | | | | | | | | | | | | UIView:0x11bea2880
| | | | | | | | | | | | | | | UIImageView:0x11bea2b60
| | | | | | | | | | | | | | UIView:0x11bea29f0
| | | | | | | | | | | | | | | UIImageView:0x11bea4b20
| | | | | | | | | | | | | | UIImageView:0x11bea4cf0
| | | | | | | | | | | | | *UILabel:0x11ae01e00'00:00'
| | | | | | | | | | | | | *UILabel:0x108ba3e70'🔴LIVE'
| | | | | | | | | | | | | *UIView:0x113c7d1e0
| | | | | | | | | | | | | | *UIStackView:0x113c61b80
| | | | | | | | | | | | | | | *UIButton:0x108ba40e0
| | | | | | | | | | | | | | | | UIImageView:0x108cf24f0
| | | | | | | | | | | | | | | *UIActivityIndicatorView:0x114c16df0
| | | | | | | | | | | | | | | | UIImageView:0x114c0c0d0
| | | | | | | | | | | | | | | *UIButton:0x113c5f6d0
| | | | | | | | | | | | | | | | UIImageView:0x108cf26c0
| | | | | | | | | | | | | | | *UIButton:0x113c4e840
| | | | | | | | | | | | | | | | UIImageView:0x108c24720
| | | | | | | | | | | | | | | *UIButton:0x11ae31150
| | | | | | | | | | | | | | | | UIImageView:0x108c248f0
| | | | | | | | | | | | | | *UIButton:0x113c37f90'Auto'
| | | | | | | | | | | | | | | UIButtonLabel:0x11bea2170'Auto'
| | | | | | | | | | | | | | *UIStackView:0x11ae30d80
| | | | | | | | | | | | | | | *<_UIOLAGapGuide: 0x280208a00 - "UISV-distributing", layoutFrame = {{30, 0}, {12, 0}}, owningView = <UIStackView: 0x11ae30d80; frame = (265 10; 94 20); opaque = NO; autoresize = RM+BM; layer = <CATransformLayer: 0x283f1ad80>>>- AMBIGUOUS LAYOUT for _UIOLAGapGuide:0x280208a00'UISV-distributing'.minY{id: 9088}, _UIOLAGapGuide:0x280208a00'UISV-distributing'.Height{id: 9089}
| | | | | | | | | | | | | | | *<_UIOLAGapGuide: 0x28021c600 - "UISV-distributing", layoutFrame = {{62, 0}, {12, 0}}, owningView = <UIStackView: 0x11ae30d80; frame = (265 10; 94 20); opaque = NO; autoresize = RM+BM; layer = <CATransformLayer: 0x283f1ad80>>>- AMBIGUOUS LAYOUT for _UIOLAGapGuide:0x28021c600'UISV-distributing'.minY{id: 9090}, _UIOLAGapGuide:0x28021c600'UISV-distributing'.Height{id: 9091}
| | | | | | | | | | | | | | | *UIButton:0x11ae967d0
| | | | | | | | | | | | | | | | UIImageView:0x108c54740
| | | | | | | | | | | | | | | *UIButton:0x11aea1a50
| | | | | | | | | | | | | | | | UIImageView:0x11a757f70
| | | | | | | | | | | | | | | *AVRoutePickerView:0x108c4f1d0
| | | | | | | | | | | | | | | | *UIButton:0x108c227f0
| | | | | | | | | | | | | | | | | UIButtonLabel:0x11be29d30
| | | | | | | | | | *UIView:0x11a7c9260- AMBIGUOUS LAYOUT for UIView:0x11a7c9260.Height{id: 8163}
| | | | | | | | | | | *XLPagerTabStrip.ButtonBarView:0x109a18c00
| | | | | | | | | | | | •XLPagerTabStrip.ButtonBarViewCell:0x11be6b470
| | | | | | | | | | | | | *UIView:0x11be6b870
| | | | | | | | | | | | | | *<UILayoutGuide: 0x280701420 - "UIViewSafeAreaLayoutGuide", layoutFrame = {{0, 0}, {162.5, 40}}, owningView = <UIView: 0x11be6b870; frame = (0 0; 162.5 40); gestureRecognizers = <NSArray: 0x28312e910>; layer = <CALayer: 0x283e8db00>>>
| | | | | | | | | | | | | | *UILabel:0x11be6b9e0'LIVE CHANNEL'
| | | | | | | | | | | | | | *UIImageView:0x11be6be40
| | | | | | | | | | | | •XLPagerTabStrip.ButtonBarViewCell:0x11be6d2a0
| | | | | | | | | | | | | *UIView:0x11be6d4a0
| | | | | | | | | | | | | | *<UILayoutGuide: 0x280701500 - "UIViewSafeAreaLayoutGuide", layoutFrame = {{0, 0}, {162.5, 40}}, owningView = <UIView: 0x11be6d4a0; frame = (0 0; 162.5 40); gestureRecognizers = <NSArray: 0x28312c5a0>; layer = <CALayer: 0x283e8d500>>>
| | | | | | | | | | | | | | *UILabel:0x11be6d610'LIVE EVENTS'
| | | | | | | | | | | | | | *UIImageView:0x11be6d880
| | | | | | | | | | | | UIView:0x108c303b0
| | | | | | | | | | | | _UIScrollViewScrollIndicator:0x108bf1d10
| | | | | | | | | | | | | UIView:0x11ae577e0
| | | | | | | | | | | *UIView:0x11a7c8f80
| | | | | | | | | | | *UIScrollView:0x109a15000- AMBIGUOUS LAYOUT for UIScrollView:0x109a15000.Height{id: 8164}
| | | | | | | | | | | | •UIView:0x11be60eb0
| | | | | | | | | | | | | *<UILayoutGuide: 0x280701260 - "UIViewSafeAreaLayoutGuide", layoutFrame = {{0, 0}, {375, 173}}, owningView = <UIView: 0x11be60eb0; frame = (0 0; 375 173); autoresize = W+H; layer = <CALayer: 0x283eb6f20>>>
| | | | | | | | | | | | | *UICollectionView:0x109a48400
| | | | | | | | | | | | | | •NTV.LiveChannelCollectionViewCell:0x11ae5b750
| | | | | | | | | | | | | | | *UIView:0x11ae5bb60
| | | | | | | | | | | | | | | | *UIView:0x11ae5bcd0
| | | | | | | | | | | | | | | | | *UIImageView:0x11ae5be40
| | | | | | | | | | | | | | | | *UILabel:0x11ae5c190'NTV'
| | | | | | | | | | | | | | •NTV.LiveChannelCollectionViewCell:0x11ae5d820
| | | | | | | | | | | | | | | *UIView:0x11ae5da30
| | | | | | | | | | | | | | | | *UIView:0x11ae5dba0
| | | | | | | | | | | | | | | | | *UIImageView:0x11ae5dd10
| | | | | | | | | | | | | | | | *UILabel:0x11ae5e060'NTV Plus'
| | | | | | | | | | | | | | •NTV.LiveChannelCollectionViewCell:0x11ae5f3d0
| | | | | | | | | | | | | | | *UIView:0x11ae5f5e0
| | | | | | | | | | | | | | | | *UIView:0x11ae5f750
| | | | | | | | | | | | | | | | | *UIImageView:0x11ae5f8c0
| | | | | | | | | | | | | | | | *UILabel:0x11ae5fc10'NTV News'
| | | | | | | | | | | | | | •NTV.LiveChannelCollectionViewCell:0x11be64720
| | | | | | | | | | | | | | | *UIView:0x11be64930
| | | | | | | | | | | | | | | | *UIView:0x11be64aa0
| | | | | | | | | | | | | | | | | *UIImageView:0x11be64c10
| | | | | | | | | | | | | | | | *UILabel:0x11be64f60'NTV Kohalpur'
| | | | | | | | | | | | | | •NTV.LiveChannelCollectionViewCell:0x11be66400
| | | | | | | | | | | | | | | *UIView:0x11be66610
| | | | | | | | | | | | | | | | *UIView:0x11be66780
| | | | | | | | | | | | | | | | | *UIImageView:0x11be668f0
| | | | | | | | | | | | | | | | *UILabel:0x11be66c40'NTV Itahari'
| | | | | | | | | | | | | *UIView:0x11be61020
| | | | | | | | | | | | | | *UIView:0x11be61190
| | | | | | | | | | | | | | | *UIImageView:0x11be603e0
| | | | | | | | | | | | | | | *UILabel:0x11be605b0'NTV'
| | | | | | | | | | | | | | | *UIButton:0x11be60a10' MORE SCHEDULE '
| | | | | | | | | | | | | | | | UIButtonLabel:0x11ae5a230' MORE SCHEDULE '
| | | | | | | | | | | | | | | *UIView:0x11be61300
| | | | | | | | | | | | | | *UIView:0x11be61770
| | | | | | | | | | | | | | | *UIImageView:0x11a7c9a40
| | | | | | | | | | | | | | | *UIImageView:0x11a73d9c0
| | | | | | | | | | | | | | | *UICollectionView:0x1098e3400
| | | | | | | | | | | | | | | | •NTV.EPGCollectionViewCell:0x11be2aad0
| | | | | | | | | | | | | | | | | *UIView:0x11be2ace0
| | | | | | | | | | | | | | | | | | *UIImageView:0x11be2ae50
| | | | | | | | | | | | | | | | | | *UILabel:0x11bebbbf0'08:00 - 08:25'
| | | | | | | | | | | | | | | | | | *UILabel:0x11bebbe60'The News'
| | | | | | | | | | | | | | | | | | *UIView:0x11bebc0d0
| | | | | | | | | | | | | | | | | | *UILabel:0x11bebc240' On Air '
| | | | | | | | | | | | | | | | •NTV.EPGCollectionViewCell:0x11beb0cf0
| | | | | | | | | | | | | | | | | *UIView:0x11beae950
| | | | | | | | | | | | | | | | | | *UIImageView:0x11beb0220
| | | | | | | | | | | | | | | | | | *UILabel:0x11beb0f00'09:00 - 09:30'
| | | | | | | | | | | | | | | | | | *UILabel:0x11beb1170'Sirjanaka Sarathi'
| | | | | | | | | | | | | | | | | | *UIView:0x11beb13e0
| | | | | | | | | | | | | | | | | | *UILabel:0x11beb1550' On Air '
| | | | | | | | | | | | | | | | •NTV.EPGCollectionViewCell:0x11beb3850
| | | | | | | | | | | | | | | | | *UIView:0x11bebe860
| | | | | | | | | | | | | | | | | | *UIImageView:0x11bebeec0
| | | | | | | | | | | | | | | | | | *UILabel:0x11bebf090'Swasthya Sarokar'
| | | | | | | | | | | | | | | | | | *UILabel:0x11bebf4c0
| | | | | | | | | | | | | | | | | | *UIView:0x11bebe9d0
| | | | | | | | | | | | | | | | | | *UILabel:0x11bebf730' On Air '
| | | | | | | | | | | | | | | | •NTV.EPGCollectionViewCell:0x11aeaf4d0
| | | | | | | | | | | | | | | | | *UIView:0x11aeaf6e0
| | | | | | | | | | | | | | | | | | *UIImageView:0x11aeaf850
| | | | | | | | | | | | | | | | | | *UILabel:0x11aeafa20'10:00 - 10:15'
| | | | | | | | | | | | | | | | | | *UILabel:0x11aeafc90'Hamro Kathmandu'
| | | | | | | | | | | | | | | | | | *UIView:0x11aeaff00
| | | | | | | | | | | | | | | | | | *UILabel:0x11aeb0070' On Air '
| | | | | | | | | | | | | | | | •NTV.EPGCollectionViewCell:0x11aebe350
| | | | | | | | | | | | | | | | | *UIView:0x11aebe560
| | | | | | | | | | | | | | | | | | *UIImageView:0x11aebd270
| | | | | | | | | | | | | | | | | | *UILabel:0x11aebd440'09:00 - 09:30'
| | | | | | | | | | | | | | | | | | *UILabel:0x11aebebc0'Sirjanaka Sarathi'
| | | | | | | | | | | | | | | | | | *UIView:0x11aebe840
| | | | | | | | | | | | | | | | | | *UILabel:0x11aebee30' On Air '
| | | | | | | | | | | | | | | | •NTV.EPGCollectionViewCell:0x11aeb6230
| | | | | | | | | | | | | | | | | *UIView:0x113c2beb0
| | | | | | | | | | | | | | | | | | *UIImageView:0x11aea7020
| | | | | | | | | | | | | | | | | | *UILabel:0x108c40a30'Swasthya Sarokar'
| | | | | | | | | | | | | | | | | | *UILabel:0x11beb9460
| | | | | | | | | | | | | | | | | | *UIView:0x11bea74d0
| | | | | | | | | | | | | | | | | | *UILabel:0x11beab180' On Air '
| | | | | | | | | | | | | | | | •NTV.EPGCollectionViewCell:0x11aeaba80
| | | | | | | | | | | | | | | | | *UIView:0x11aeabe90
| | | | | | | | | | | | | | | | | | *UIImageView:0x11aeac000
| | | | | | | | | | | | | | | | | | *UILabel:0x11aeac1d0'10:00 - 10:15'
| | | | | | | | | | | | | | | | | | *UILabel:0x11aeac630'Hamro Kathmandu'
| | | | | | | | | | | | | | | | | | *UIView:0x11aeaca90
| | | | | | | | | | | | | | | | | | *UILabel:0x11aeacc00' On Air '
| | | | | | | •UINavigationBar:0x108c24220
| | | | | | | | _UIBarBackground:0x108c37e40
| | | | | | | | | UIImageView:0x11ae59160
| | | | | | | | •_UINavigationBarContentView:0x108c4c340
| | | | | | | | | *<UILayoutGuide: 0x280714c40 - "BackButtonGuide(0x108c50bc0)", layoutFrame = {{0, 0}, {16, 32}}, owningView = <_UINavigationBarContentView: 0x108c4c340; frame = (0 0; 375 44); layer = <CALayer: 0x283e4b0a0>> layout=0x108c50bc0>
| | | | | | | | | *<UILayoutGuide: 0x280714d20 - "LeadingBarGuide(0x108c50bc0)", layoutFrame = {{16, 0}, {70, 32}}, owningView = <_UINavigationBarContentView: 0x108c4c340; frame = (0 0; 375 44); layer = <CALayer: 0x283e4b0a0>> layout=0x108c50bc0>
| | | | | | | | | *<UILayoutGuide: 0x280714e00 - "TitleView(0x108c50bc0)", layoutFrame = {{92, 0}, {231, 32}}, owningView = <_UINavigationBarContentView: 0x108c4c340; frame = (0 0; 375 44); layer = <CALayer: 0x283e4b0a0>> layout=0x108c50bc0>
| | | | | | | | | *<UILayoutGuide: 0x280716300 - "TrailingBarGuide(0x108c50bc0)", layoutFrame = {{323, 0}, {44, 32}}, owningView = <_UINavigationBarContentView: 0x108c4c340; frame = (0 0; 375 44); layer = <CALayer: 0x283e4b0a0>> layout=0x108c50bc0>
| | | | | | | | | *<UILayoutGuide: 0x280716220 - "UIViewLayoutMarginsGuide", layoutFrame = {{16, 0}, {343, 44}}, owningView = <_UINavigationBarContentView: 0x108c4c340; frame = (0 0; 375 44); layer = <CALayer: 0x283e4b0a0>> layout=0x108c50bc0>
| | | | | | | | | *_UIButtonBarStackView:0x114cdfb30
| | | | | | | | | | *<UILayoutGuide: 0x28071f1e0 - "UIViewLayoutMarginsGuide", layoutFrame = {{0, 0}, {70, 32}}, owningView = <_UIButtonBarStackView: 0x114cdfb30; frame = (16 0; 70 44); layer = <CALayer: 0x283e81140>> buttonBar=0x28071ef40>
| | | | | | | | | | *<_UILayoutSpacer: 0x280155680 - "UISV-alignment-spanner", layoutFrame = {{0, 0}, {0, 32}}, owningView = <_UIButtonBarStackView: 0x114cdfb30; frame = (16 0; 70 44); layer = <CALayer: 0x283e81140>> buttonBar=0x28071ef40>- AMBIGUOUS LAYOUT for _UILayoutSpacer:0x280155680'UISV-alignment-spanner'.minX{id: 9224}, _UILayoutSpacer:0x280155680'UISV-alignment-spanner'.Width{id: 9225}
| | | | | | | | | | *_UITAMICAdaptorView:0x114cdf6b0
| | | | | | | | | | | UIButton:0x108bcc8d0
| | | | | | | | | | | | UIImageView:0x11ae57f00
| | | | | | | | | | *UIView:0x11aea4d00
| | | | | | | | | | *_UIButtonBarButton:0x11aea5ae0
| | | | | | | | | | | *<UILayoutGuide: 0x2807031e0 - "UIViewLayoutMarginsGuide", layoutFrame = {{0, 11}, {37, 10}}, owningView = <_UIButtonBarButton: 0x11aea5ae0; frame = (33 0; 37 44); tintColor = UIExtendedGrayColorSpace 1 1; layer = <CALayer: 0x283f1b620>>>
| | | | | | | | | | | *_UIModernBarButton:0x11aea5ec0'NTV'
| | | | | | | | | | | | *UIButtonLabel:0x11bea7910'NTV'
| | | | | | | | | *_UIButtonBarStackView:0x108ba78e0
| | | | | | | | | | *<UILayoutGuide: 0x28071eca0 - "UIViewLayoutMarginsGuide", layoutFrame = {{0, 0}, {44, 32}}, owningView = <_UIButtonBarStackView: 0x108ba78e0; frame = (323 0; 44 44); layer = <CALayer: 0x283e841c0>> buttonBar=0x28071ebc0>
| | | | | | | | | | *_UIButtonBarButton:0x114c16590
| | | | | | | | | | | UILabel:0x108c4ff70'0'
| | | | | | | | | | | *_UIModernBarButton:0x114c16770
| | | | | | | | | | | | UIImageView:0x114c14c00
| | | | UITabBar:0x108bf0990
| | | | | _UIBarBackground:0x108bf34f0
| | | | | | _UIBarBackgroundShadowView:0x108bf5d80
| | | | | | | _UIBarBackgroundShadowContentImageView:0x108bfb190
| | | | | | UIImageView:0x113c0bc10
| | | | | UITabBarButton:0x113c10f70
| | | | | | UITabBarSwappableImageView:0x113c11530
| | | | | | UITabBarButtonLabel:0x113c11290'Home'
| | | | | UITabBarButton:0x113c11ed0
| | | | | | UITabBarSwappableImageView:0x113c12490
| | | | | | UITabBarButtonLabel:0x113c121f0'Live'
| | | | | UITabBarButton:0x113c12e30
| | | | | | UITabBarSwappableImageView:0x113c0dea0
| | | | | | UITabBarButtonLabel:0x113c13150'TV Shows'
| | | | | UITabBarButton:0x113c13ba0
| | | | | | UITabBarSwappableImageView:0x113c14160
| | | | | | UITabBarButtonLabel:0x113c13ec0'News'
| | | | | UITabBarButton:0x108c51190
| | | | | | UITabBarSwappableImageView:0x108c51de0
| | | | | | UITabBarButtonLabel:0x108c51b40'More'
| UIView:0x108bf0bb0
The issue is on iOS 13.3.1 devices and it is working for 12.4.5 devices. I have tried to change the constrains of AMBIGUOUS view but could not resolve the issue. Any help would be appreciated. Thank you.
For me, this happened because of an infinite loop under layoutSubviews. Once I fix that it was solved. Thought the crash appears only on iOS 13, the previous version of iOS has no issue with it.

Pandas sort not sorting data properly

I am trying to sort the results of sklearn.ensemble.RandomForestRegressor's feature_importances_
I have the following function:
def get_feature_importances(cols, importances):
feats = {}
for feature, importance in zip(cols, importances):
feats[feature] = importance
importances = pd.DataFrame.from_dict(feats, orient='index').rename(columns={0: 'Gini-importance'})
importances.sort_values(by='Gini-importance')
return importances
I use it like so:
importances = get_feature_importances(X_test.columns, rf.feature_importances_)
print()
print(importances)
And I get the following results:
| PART | 0.035034 |
| MONTH1 | 0.02507 |
| YEAR1 | 0.020075 |
| MONTH2 | 0.02321 |
| YEAR2 | 0.017861 |
| MONTH3 | 0.042606 |
| YEAR3 | 0.028508 |
| DAYS | 0.047603 |
| MEDIANDIFF | 0.037696 |
| F2 | 0.008783 |
| F1 | 0.015764 |
| F6 | 0.017933 |
| F4 | 0.017511 |
| F5 | 0.017799 |
| SS22 | 0.010521 |
| SS21 | 0.003896 |
| SS19 | 0.003894 |
| SS23 | 0.005249 |
| SS20 | 0.005127 |
| RR | 0.021626 |
| HI_HOURS | 0.067584 |
| OI_HOURS | 0.054369 |
| MI_HOURS | 0.062121 |
| PERFORMANCE_FACTOR | 0.033572 |
| PERFORMANCE_INDEX | 0.073884 |
| NUMPA | 0.022445 |
| BUMPA | 0.024192 |
| ELOH | 0.04386 |
| FFX1 | 0.128367 |
| FFX2 | 0.083839 |
I thought the line importances.sort_values(by='Gini-importance') would sort them. But it is not. Why is this not performing correctly?
importances.sort_values(by='Gini-importance') returns the sorted dataframe, which is overlooked by your function.
You want return importances.sort_values(by='Gini-importance').
Or you could make sort_values inplace:
importances.sort_values(by='Gini-importance', inplace=True)
return importances

pdfjam: Merging multiple pdfs into one page without white space

which I want to get AB closer without space:
------ ------ ---------- ------------
| | | | | ———— | | —— —— |
| A | | B | | |A|B| | | |A| |B| |
| | | | | ———— | | —— —— |
| | | | ---> | |NOT | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
------ ------ ------ --------
which I use:
pdfjam figurea.pdf figureb.pdf --nup 2x1 --landscape --outfile figureout.pdf
Try specifying a negative value for delta:
pdfjam figurea.pdf figureb.pdf --delta '-10 0' --nup 2x1 --landscape --outfile figureout.pdf

Memsql is taking lot of time for a query to execute

I am having one problem with my memsql cluster when I run the query to fetch the 51M records it returns result in 5 minutes
but it used to take more than 15 min when data insertion is parallel to read.
I measured disk io and it is ok and the disk is hdd disk.
There are no other connections to the memsql and cpu is also 15% utilized with 64 core machine
Below are my varaiables
Variable_name | Value |
+----------------------------------------------+------------------------------------------------------------------------------+
| aggregator_failure_detection | ON |
| auto_replicate | OFF |
| autocommit | ON |
| basedir | /data/master-3306 |
| character_set_client | utf8 |
| character_set_connection | utf8 |
| character_set_filesystem | binary |
| character_set_results | utf8 |
| character_set_server | utf8 |
| character_sets_dir | /data/master-3306/share/charsets/ |
| collation_connection | utf8_general_ci |
| collation_database | utf8_general_ci |
| collation_server | utf8_general_ci |
| columnar_segment_rows | 102400 |
| columnstore_window_size | 2147483648 |
| compile_only | OFF |
| connect_timeout | 10 |
| core_file | ON |
| core_file_mode | PARTIAL |
| critical_diagnostics | ON |
| datadir | /data/master-3306/data |
| default_partitions_per_leaf | 16 |
| enable_experimental_metrics | OFF |
| error_count | 0 |
| explain_expression_limit | 500 |
| external_user | |
| flush_before_replicate | OFF |
| general_log | OFF |
| geo_sphere_radius | 6367444.657120 |
| hostname | **** |
| identity | 0 |
| kerberos_server_keytab | |
| lc_messages | en_US |
| lc_messages_dir | /data/master-3306/share |
| leaf_failure_detection | ON |
| load_data_max_buffer_size | 1073741823 |
| load_data_read_size | 8192 |
| load_data_write_size | 8192 |
| lock_wait_timeout | 60 |
| master_aggregator | self |
| max_allowed_packet | 104857600 |
| max_connection_threads | 192 |
| max_connections | 100000 |
| max_pooled_connections | 4096 |
| max_prefetch_threads | 1 |
| max_prepared_stmt_count | 16382 |
| max_user_connections | 0 |
| maximum_memory | 506602 |
| maximum_table_memory | 455941 |
| memsql_id | ** |
| memsql_version | 5.7.2 |
| memsql_version_date | Thu Jan 26 12:34:22 2017 -0800 |
| memsql_version_hash | 03e5e3581e96d65caa30756f191323437a3840f0 |
| minimal_disk_space | 100 |
| multi_insert_tuple_count | 20000 |
| net_buffer_length | 102400 |
| net_read_timeout | 3600 |
| net_write_timeout | 3600 |
| pid_file | /data/master-3306/data/memsqld.pid |
| pipelines_batches_metadata_to_keep | 1000 |
| pipelines_extractor_debug_logging | OFF |
| pipelines_kafka_version | 0.8.2.2 |
| pipelines_max_errors_per_partition | 1000 |
| pipelines_max_offsets_per_batch_partition | 1000000 |
| pipelines_max_retries_per_batch_partition | 4 |
| pipelines_stderr_bufsize | 65535 |
| pipelines_stop_on_error | ON |
| plan_expiration_minutes | 720 |
| port | 3306 |
| protocol_version | 10 |
| proxy_user | |
| query_parallelism | 0 |
| redundancy_level | 1 |
| reported_hostname | |
| secure_file_priv | |
| show_query_parameters | ON |
| skip_name_resolve | AUTO |
| snapshot_trigger_size | 268435456 |
| snapshots_to_keep | 2 |
| socket | /data/master-3306/data/memsql.sock |
| sql_quote_show_create | ON |
| ssl_ca | |
| ssl_capath | |
| ssl_cert | |
| ssl_cipher | |
| ssl_key | |
| sync_slave_timeout | 20000 |
| system_time_zone | UTC |
| thread_cache_size | 0 |
| thread_handling | one-thread-per-connection |
| thread_stack | 1048576 |
| time_zone | SYSTEM |
| timestamp | 1504799067.127069 |
| tls_version | TLSv1,TLSv1.1,TLSv1.2 |
| tmpdir | . |
| transaction_buffer | 67108864 |
| tx_isolation | READ-COMMITTED |
| use_join_bucket_bit_vector | ON |
| use_vectorized_join | ON |
| version | 5.5.8 |
| version_comment | MemSQL source distribution (compatible; MySQL Enterprise & MySQL Commercial) |
| version_compile_machine | x86_64 |
| version_compile_os | Linux |
| warn_level | WARNINGS |
| warning_count | 0 |
| workload_management | ON |
| workload_management_expected_aggregators | 1 |
| workload_management_max_connections_per_leaf | 1024 |
| workload_management_max_queue_depth | 100 |
| workload_management_max_threads_per_leaf | 8192 |
| workload_management_queue_time_warning_ratio | 0.500000 |
| workload_management_queue_timeout | 3600
Some thoughts:
Workload profiling (https://docs.memsql.com/concepts/v5.8/workload-profiling-overview/) can help you understand what resources are limiting the speed of this query - if it's not cpu and not disk io, maybe it's network io or something
Query profiling (https://docs.memsql.com/sql-reference/v5.8/profile/) will help indicate which operators within the query are expensive
You mentioned your machine has 64 cores - is your cluster properly numa-optimized? Run memsql-ops memsql-optimize to check.
Jack, I did the workload profiling and find out that lock_time_ms is quite high when NETWORK_LOGICAL_SEND_B is high
LAST_FINISHED_TIMESTAMP, LOCK_TIME_MS, LOCK_ROW_TIME_MS, ACTIVITY_NAME, NETWORK_LOGICAL_RECV_B, NETWORK_LOGICAL_SEND_B, activity_name, database_name, partition_id, left(q.query_text, 50)
'2017-09-11 10:41:31', '39988', '0', 'InsertSelect_AggregatedHourly_temp_7aug_KING__et_al_c44dc7ab56d56280', '0', '538154753', 'InsertSelect_AggregatedHourly_temp_7aug_KING__et_al_c44dc7ab56d56280', 'datawarehouse', '7', 'SELECT \n combined.day AS day,\n `lineitem'

Severe mysqldump performance degradation using Centos Linux, 8GB PAE and MySQL 5.0.77

We use MySQL 5.0.77 on CentOS 5.5 on VMWare:
Linux dev.ic.soschildrensvillages.org.uk 2.6.18-194.11.4.el5PAE #1 SMP Tue Sep 21 05:48:23 EDT 2010 i686 i686 i386 GNU/Linux
We have recently upgraded from 4GB RAM to 8GB. When we did this the time of our mysqldump overnight backup jumped from under 10 minutes to over 2 hours. It also caused unresponsiveness on our plone based web site due to database load. The dump is using the optimized mysqldump format and is spooled directly through a socket to another server.
Any ideas on what we could do to fix gratefully appreciated. Would a MySQL upgrade help? Anything we can do to MySQL config? Anything we can do to Linux config? Or do we have to add another server or go to 64-bit?
We ran a previous (non-virtual) server on 6GB PAE and didn't notice a similar issue. This was on same MySQL version, but Centos 4.4.
Server config file:
[mysqld]
port=3307
socket=/tmp/mysql_live.sock
wait_timeout=31536000
interactive_timeout=31536000
datadir=/var/mysql/live/data
user=mysql
max_connections = 200
max_allowed_packet = 64M
table_cache = 2048
binlog_cache_size = 128K
max_heap_table_size = 32M
sort_buffer_size = 2M
join_buffer_size = 2M
lower_case_table_names = 1
innodb_data_file_path = ibdata1:10M:autoextend
innodb_buffer_pool_size=1G
innodb_log_file_size=300M
innodb_log_buffer_size=8M
innodb_flush_log_at_trx_commit=1
innodb_file_per_table
[mysqldump]
# Do not buffer the whole result set in memory before writing it to
# file. Required for dumping very large tables
quick
max_allowed_packet = 64M
[mysqld_safe]
# Increase the amount of open files allowed per process. Warning: Make
# sure you have set the global system limit high enough! The high value
# is required for a large number of opened tables
open-files-limit = 8192
Server variables:
mysql> show variables;
+---------------------------------+------------------------------------------------------------------+
| Variable_name | Value |
+---------------------------------+------------------------------------------------------------------+
| auto_increment_increment | 1 |
| auto_increment_offset | 1 |
| automatic_sp_privileges | ON |
| back_log | 50 |
| basedir | /usr/local/mysql-5.0.77-linux-i686-glibc23/ |
| binlog_cache_size | 131072 |
| bulk_insert_buffer_size | 8388608 |
| character_set_client | latin1 |
| character_set_connection | latin1 |
| character_set_database | latin1 |
| character_set_filesystem | binary |
| character_set_results | latin1 |
| character_set_server | latin1 |
| character_set_system | utf8 |
| character_sets_dir | /usr/local/mysql-5.0.77-linux-i686-glibc23/share/mysql/charsets/ |
| collation_connection | latin1_swedish_ci |
| collation_database | latin1_swedish_ci |
| collation_server | latin1_swedish_ci |
| completion_type | 0 |
| concurrent_insert | 1 |
| connect_timeout | 10 |
| datadir | /var/mysql/live/data/ |
| date_format | %Y-%m-%d |
| datetime_format | %Y-%m-%d %H:%i:%s |
| default_week_format | 0 |
| delay_key_write | ON |
| delayed_insert_limit | 100 |
| delayed_insert_timeout | 300 |
| delayed_queue_size | 1000 |
| div_precision_increment | 4 |
| keep_files_on_create | OFF |
| engine_condition_pushdown | OFF |
| expire_logs_days | 0 |
| flush | OFF |
| flush_time | 0 |
| ft_boolean_syntax | + -><()~*:""&| |
| ft_max_word_len | 84 |
| ft_min_word_len | 4 |
| ft_query_expansion_limit | 20 |
| ft_stopword_file | (built-in) |
| group_concat_max_len | 1024 |
| have_archive | YES |
| have_bdb | NO |
| have_blackhole_engine | YES |
| have_compress | YES |
| have_crypt | YES |
| have_csv | YES |
| have_dynamic_loading | YES |
| have_example_engine | NO |
| have_federated_engine | YES |
| have_geometry | YES |
| have_innodb | YES |
| have_isam | NO |
| have_merge_engine | YES |
| have_ndbcluster | DISABLED |
| have_openssl | DISABLED |
| have_ssl | DISABLED |
| have_query_cache | YES |
| have_raid | NO |
| have_rtree_keys | YES |
| have_symlink | YES |
| hostname | app.ic.soschildrensvillages.org.uk |
| init_connect | |
| init_file | |
| init_slave | |
| innodb_additional_mem_pool_size | 1048576 |
| innodb_autoextend_increment | 8 |
| innodb_buffer_pool_awe_mem_mb | 0 |
| innodb_buffer_pool_size | 1073741824 |
| innodb_checksums | ON |
| innodb_commit_concurrency | 0 |
| innodb_concurrency_tickets | 500 |
| innodb_data_file_path | ibdata1:10M:autoextend |
| innodb_data_home_dir | |
| innodb_adaptive_hash_index | ON |
| innodb_doublewrite | ON |
| innodb_fast_shutdown | 1 |
| innodb_file_io_threads | 4 |
| innodb_file_per_table | ON |
| innodb_flush_log_at_trx_commit | 1 |
| innodb_flush_method | |
| innodb_force_recovery | 0 |
| innodb_lock_wait_timeout | 50 |
| innodb_locks_unsafe_for_binlog | OFF |
| innodb_log_arch_dir | |
| innodb_log_archive | OFF |
| innodb_log_buffer_size | 8388608 |
| innodb_log_file_size | 314572800 |
| innodb_log_files_in_group | 2 |
| innodb_log_group_home_dir | ./ |
| innodb_max_dirty_pages_pct | 90 |
| innodb_max_purge_lag | 0 |
| innodb_mirrored_log_groups | 1 |
| innodb_open_files | 300 |
| innodb_rollback_on_timeout | OFF |
| innodb_support_xa | ON |
| innodb_sync_spin_loops | 20 |
| innodb_table_locks | ON |
| innodb_thread_concurrency | 8 |
| innodb_thread_sleep_delay | 10000 |
| interactive_timeout | 31536000 |
| join_buffer_size | 2097152 |
| key_buffer_size | 8384512 |
| key_cache_age_threshold | 300 |
| key_cache_block_size | 1024 |
| key_cache_division_limit | 100 |
| language | /usr/local/mysql-5.0.77-linux-i686-glibc23/share/mysql/english/ |
| large_files_support | ON |
| large_page_size | 0 |
| large_pages | OFF |
| lc_time_names | en_US |
| license | GPL |
| local_infile | ON |
| locked_in_memory | OFF |
| log | OFF |
| log_bin | OFF |
| log_bin_trust_function_creators | OFF |
| log_error | |
| log_queries_not_using_indexes | OFF |
| log_slave_updates | OFF |
| log_slow_queries | OFF |
| log_warnings | 1 |
| long_query_time | 10 |
| low_priority_updates | OFF |
| lower_case_file_system | OFF |
| lower_case_table_names | 1 |
| max_allowed_packet | 67108864 |
| max_binlog_cache_size | 4294963200 |
| max_binlog_size | 1073741824 |
| max_connect_errors | 10 |
| max_connections | 200 |
| max_delayed_threads | 20 |
| max_error_count | 64 |
| max_heap_table_size | 33554432 |
| max_insert_delayed_threads | 20 |
| max_join_size | 18446744073709551615 |
| max_length_for_sort_data | 1024 |
| max_prepared_stmt_count | 16382 |
| max_relay_log_size | 0 |
| max_seeks_for_key | 4294967295 |
| max_sort_length | 1024 |
| max_sp_recursion_depth | 0 |
| max_tmp_tables | 32 |
| max_user_connections | 0 |
| max_write_lock_count | 4294967295 |
| multi_range_count | 256 |
| myisam_data_pointer_size | 6 |
| myisam_max_sort_file_size | 2146435072 |
| myisam_recover_options | OFF |
| myisam_repair_threads | 1 |
| myisam_sort_buffer_size | 8388608 |
| myisam_stats_method | nulls_unequal |
| ndb_autoincrement_prefetch_sz | 1 |
| ndb_force_send | ON |
| ndb_use_exact_count | ON |
| ndb_use_transactions | ON |
| ndb_cache_check_time | 0 |
| ndb_connectstring | |
| net_buffer_length | 16384 |
| net_read_timeout | 30 |
| net_retry_count | 10 |
| net_write_timeout | 60 |
| new | OFF |
| old_passwords | OFF |
| open_files_limit | 8192 |
| optimizer_prune_level | 1 |
| optimizer_search_depth | 62 |
| pid_file | /var/mysql/live/mysqld.pid |
| plugin_dir | |
| port | 3307 |
| preload_buffer_size | 32768 |
| profiling | OFF |
| profiling_history_size | 15 |
| protocol_version | 10 |
| query_alloc_block_size | 8192 |
| query_cache_limit | 1048576 |
| query_cache_min_res_unit | 4096 |
| query_cache_size | 0 |
| query_cache_type | ON |
| query_cache_wlock_invalidate | OFF |
| query_prealloc_size | 8192 |
| range_alloc_block_size | 4096 |
| read_buffer_size | 131072 |
| read_only | OFF |
| read_rnd_buffer_size | 262144 |
| relay_log | |
| relay_log_index | |
| relay_log_info_file | relay-log.info |
| relay_log_purge | ON |
| relay_log_space_limit | 0 |
| rpl_recovery_rank | 0 |
| secure_auth | OFF |
| secure_file_priv | |
| server_id | 0 |
| skip_external_locking | ON |
| skip_networking | OFF |
| skip_show_database | OFF |
| slave_compressed_protocol | OFF |
| slave_load_tmpdir | /tmp/ |
| slave_net_timeout | 3600 |
| slave_skip_errors | OFF |
| slave_transaction_retries | 10 |
| slow_launch_time | 2 |
| socket | /tmp/mysql_live.sock |
| sort_buffer_size | 2097152 |
| sql_big_selects | ON |
| sql_mode | |
| sql_notes | ON |
| sql_warnings | OFF |
| ssl_ca | |
| ssl_capath | |
| ssl_cert | |
| ssl_cipher | |
| ssl_key | |
| storage_engine | MyISAM |
| sync_binlog | 0 |
| sync_frm | ON |
| system_time_zone | GMT |
| table_cache | 2048 |
| table_lock_wait_timeout | 50 |
| table_type | MyISAM |
| thread_cache_size | 0 |
| thread_stack | 196608 |
| time_format | %H:%i:%s |
| time_zone | SYSTEM |
| timed_mutexes | OFF |
| tmp_table_size | 33554432 |
| tmpdir | /tmp/ |
| transaction_alloc_block_size | 8192 |
| transaction_prealloc_size | 4096 |
| tx_isolation | REPEATABLE-READ |
| updatable_views_with_limit | YES |
| version | 5.0.77 |
| version_comment | MySQL Community Server (GPL) |
| version_compile_machine | i686 |
| version_compile_os | pc-linux-gnu |
| wait_timeout | 31536000 |
+---------------------------------+------------------------------------------------------------------+
237 rows in set (0.00 sec)
If you're sending data over sockets, you might as well use MySQL Replication with the binlog. Use binlog_format=ROWS so that your slave(s) doesn't waste time with triggers and so on.
Failing that, you could try a "hotcopy" script such as http://dev.mysql.com/doc/refman/5.0/en/mysqlhotcopy.html
Above all, I think you are running outdated software. Upgrade to CentOS 5.5 64bit, and Percona Server (100% MySQL-compatible database on steroids) http://www.percona.com/software/percona-server/

Resources