Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

To analyze: 464.h264ref OOBs under ASan on Windows #50

Closed
ramosian-glider opened this issue Aug 31, 2015 · 14 comments
Closed

To analyze: 464.h264ref OOBs under ASan on Windows #50

ramosian-glider opened this issue Aug 31, 2015 · 14 comments
Labels
OpSys-Windows Priority-Medium ProjectAddressSanitizer Status-Verified Type-Other

Comments

@ramosian-glider
Copy link
Member

Originally reported on Google Code with ID 50

$ runspec --size=test --iterations=1 --loose --tune=base --config=asan-O2-win 464.h264ref

=================================================================
==9264== ERROR: AddressSanitizer heap-buffer-overflow on address 0x02e64de4 at pc 0x544743
bp 0xdeadbeef sp 0x20af3a8
READ of size 4 at 0x02e64de4 thread T0
  #0 0x544743  RDCost_for_macroblocks+0x0x000022d3
  #1 0x570763  encode_one_macroblock+0x0x0001ad33
  #2 0x5868e8  encode_one_slice+0x0x00002d38
  #3 0x4562eb  code_a_picture+0x0x0000029b
  #4 0x4608d1  frame_picture+0x0x00000cf1
  #5 0x45d974  encode_one_frame+0x0x00007214
  #6 0x46f1fe  main+0x0x0000169e
  #7 0x5c509d  __tmainCRTStartup f:\dd\vctools\crt_bld\self_x86\crt\src\crt0.c:266
  #8 0x76943677  BaseThreadInitThunk+0x0x00000012
  #9 0x77009f42  RtlInitializeExceptionChain+0x0x00000063
  #10 0x77009f15  RtlInitializeExceptionChain+0x0x00000036
0x02e64de4 is located 260 bytes to the right of 3168-byte region [0x02e64080,0x02e64ce0)
allocated by thread T0 here:
  #0 0x5b6e85  calloc c:\src\llvm\projects\compiler-rt\lib\asan\asan_malloc_win.cc:52
  #1 0x4d662c  get_mem2Dshort+0x0x0000008c
  #2 0x4d6827  get_mem3Dshort+0x0x000000a7
  #3 0x4c81b4  alloc_colocated+0x0x000000a4
  #4 0x50f304  GenerateSequenceParameterSet+0x0x00000914
  #5 0x50df93  GenerateParameterSets+0x0x00000023
  #6 0x46de87  main+0x0x00000327
  #7 0x5c509d  __tmainCRTStartup f:\dd\vctools\crt_bld\self_x86\crt\src\crt0.c:266
  #8 0x76943677  BaseThreadInitThunk+0x0x00000012
  #9 0x77009f42  RtlInitializeExceptionChain+0x0x00000063
  #10 0x77009f15  RtlInitializeExceptionChain+0x0x00000036
==9264== ABORTING
Stats: 12M malloced (11M for red zones) by 10362 calls
Stats: 0M realloced by 0 calls
Stats: 0M freed by 12 calls
Stats: 0M really freed by 0 calls
Stats: 60M (15371 full pages) mmaped in 15 calls
  mmaps   by size class: 8:16383; 9:8191; 10:4095; 11:2047; 12:1024; 13:2560; 14:256;
15:128; 16:64; 17:32; 19:8;
  mallocs by size class: 8:7201; 9:623; 10:10; 11:4; 12:10; 13:2474; 14:19; 15:1; 16:6;
17:13; 19:1;
  frees   by size class: 8:6; 12:1; 13:1; 14:1; 15:1; 16:2;
  rfrees  by size class:
Stats: malloc large: 14 small slow: 183
Shadow byte and word:
  0x205cc9bc: fa
  0x205cc9bc: fa fa fa fa
More shadow bytes:
  0x205cc9ac: fa fa fa fa
  0x205cc9b0: fa fa fa fa
  0x205cc9b4: fa fa fa fa
  0x205cc9b8: fa fa fa fa
=>0x205cc9bc: fa fa fa fa
  0x205cc9c0: fa fa fa fa
  0x205cc9c4: fa fa fa fa
  0x205cc9c8: fa fa fa fa
  0x205cc9cc: fa fa fa fa

Reported by timurrrr on 2012-03-16 16:15:13
@ramosian-glider
Copy link
Member Author 

FTR, the stack looks similar to the global buffer overflow mentioned on the FoundBugs
wiki page; but the top frames are slightly different.

Also, the report in this issue is marked as heap overflow, not global.

Reported by timurrrr on 2012-03-16 16:16:45

@ramosian-glider
Copy link
Member Author 

This is by no means an asan bug or feature request. (or is it?)
Please don't pollute our bug tracker.

Reported by konstantin.s.serebryany on 2012-03-16 16:39:51

  • Status changed: Invalid

@ramosian-glider
Copy link
Member Author 

asan prints "heap-buffer-overflow" because it checks the shadow bytes, and they clearly
indicate that this is a heap bug, has nothing to do with globals. 
Shadow byte and word:
  0x205cc9bc: fa
  0x205cc9bc: fa fa fa fa

Reported by konstantin.s.serebryany on 2012-03-16 17:05:14

@ramosian-glider
Copy link
Member Author 

> This is by no means an asan bug or feature request. (or is it?)
It is a task, related to asan.
Reopening until we/I have understanding what's going on.

Re-ran with `set ASAN_OPTIONS="redzone=256"` on Windows -> got the same stack and right-OOB
report:
0x02b44e64 is located 260 bytes to the right of 3168-byte region [0x02b44100,0x02b44d60)

Re-ran with `set ASAN_OPTIONS="redzone=512"` and got a left-OOB report:
0x02d96f64 is located 668 bytes to the left of 1584-byte region [0x02d97200,0x02d97830)
allocated by thread T0 here:
  #0 0x5b6e85  calloc c:\src\llvm\projects\compiler-rt\lib\asan\asan_malloc_win.cc:52
  #1 0x4d57ec  get_mem2D+0x0x0000008c
  #2 0x4c8224  alloc_colocated+0x0x00000114
  #3 0x50f304  GenerateSequenceParameterSet+0x0x00000914
  #4 0x50df93  GenerateParameterSets+0x0x00000023
  #5 0x46de87  main+0x0x00000327

FTR, the "fa fa fa fa" shadow bytes are kAsanHeapLeftRedzoneMagic.
Not clear why the first two reports say "right-OOB"

Reported by timurrrr on 2012-03-16 17:08:22

  • Status changed: Accepted

@ramosian-glider
Copy link
Member Author 

with redzone=2048 the allocation stack is different:
0x04ea3564 is located 668 bytes to the left of 260-byte region [0x04ea3800,0x04ea3904)
allocated by thread T0 here:
  #0 0x5b6e85  calloc c:\src\llvm\projects\compiler-rt\lib\asan\asan_malloc_win.cc:52
  #1 0x473d7f  get_mem_DCcoeff+0x0x0000031f
  #2 0x470fa0  init_img+0x0x00000730
  #3 0x46de8c  main+0x0x0000032c
  #4 0x5c509d  __tmainCRTStartup f:\dd\vctools\crt_bld\self_x86\crt\src\crt0.c:266
  #5 0x76943677  BaseThreadInitThunk+0x0x00000012
  #6 0x77009f42  RtlInitializeExceptionChain+0x0x00000063

Reported by timurrrr on 2012-03-16 17:10:21

@ramosian-glider
Copy link
Member Author 

Can't reproduce on linux, even with redzone=512
Ok, there is a chance this is an asan/win bug, reopening.

Reported by konstantin.s.serebryany on 2012-03-16 17:12:31

@ramosian-glider
Copy link
Member Author 

Iterestingly, this doesn't repro with -O0 on Windows too...

Reported by timurrrr on 2012-03-16 18:34:17

@ramosian-glider
Copy link
Member Author 

FTR, clang -O2 (without asan) also fails this test:
*** Miscompare of foreman_train_baseline_encodelog.out; for details see
    C:/src/spec2006/benchspec/CPU2006/464.h264ref/run/run_base_train_none.0001/foreman_train_baseline_encodelog.out.mis
0015:  0000(IDR)   21952 0 28  37.383  41.260  42.850        0       0     FRM    99
       0000(IDR)    6808 0 28  -1.#IO  40.796  42.637        0       0     FRM    99
                       ^
0016:  0001(P)      3024 0 28  36.868  41.036  42.720        0       0     FRM    
0
       0001(P)      5360 0 28  -1.#IO  40.027  41.179        0       0     FRM    99
                       ^
0017:  0002(P)      4120 0 28  36.875  40.936  42.683        0       0     FRM    
2
       0002(P)      5424 0 28  -1.#IO  40.024  40.980        0       0     FRM    99
                       ^
0018:  0003(P)      4552 0 28  36.851  40.950  42.670        0       0     FRM    
2
       0003(P)      5520 0 28  -1.#IO  40.026  40.936        0       0     FRM    99
                       ^
0019:  0004(P)      4952 0 28  36.848  40.815  42.407        0       0     FRM    
2
       0004(P)      5408 0 28  -1.#IO  39.977  41.178        0       0     FRM    99
                       ^
0020:  0005(P)      4448 0 28  36.775  40.691  42.386        0       0     FRM    
1
       0005(P)      5504 0 28  -1.#IO  39.938  41.115        0       0     FRM    99
                       ^
0021:  0006(P)      3944 0 28  36.689  40.740  42.206        0       0     FRM    
1
       0006(P)      5480 0 28  -1.#IO  39.815  41.027        0       0     FRM    99
                       ^
0022:  0007(P)      4040 0 28  36.698  40.821  42.262        0       0     FRM    
0
       0007(P)      5552 0 28  -1.#IO  39.820  41.052        0       0     FRM    99
                       ^
0023:  0008(P)      4104 0 28  36.751  40.824  42.114        0       0     FRM    
0
       0008(P)      5528 0 28  -1.#IO  39.863  40.818        0       0     FRM    99
                       ^
0024:  0009(P)      4576 0 28  36.724  40.773  41.954        0       0     FRM    
1
       0009(P)      5640 0 28  -1.#IO  40.015  40.742        0       0     FRM    99

Reported by timurrrr on 2012-03-19 12:26:03

@ramosian-glider
Copy link
Member Author 

and it passes with clang -O3...

Reported by timurrrr on 2012-03-19 12:26:14

@ramosian-glider
Copy link
Member Author 

Now that we have debug info and fixed quite a few ABI issues in Clang, I think it's
time to revisit.

Reported by [email protected] on 2014-05-08 13:36:17

@ramosian-glider
Copy link
Member Author 

Here's a fresh report with a symbolized stacktrace:
=================================================================
==5056==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x00c3e990 at pc
0xf5efc7 sp 0xc3e8d4
READ of size 4 at 0x00c3e990 thread T0
    #0 0xf5efc6 in SATD  mv-search.c:1093
    #1 0xf6209f in SubPelBlockMotionSearch  mv-search.c:1398
    #2 0xf78e79 in BlockMotionSearch  mv-search.c:2672
    #3 0xf80ab2 in PartitionMotionSearch  mv-search.c:3272
    #4 0xfd0aef in encode_one_macroblock  rdopt.c:3096
    #5 0xff319c in encode_one_slice  slice.c:253
    #6 0xed074e in code_a_picture  image.c:236
    #7 0xed7683 in frame_picture  image.c:798
    #8 0xed1ca7 in encode_one_frame  image.c:409
    #9 0xef12c9 in main  lencod.c:413

Address 0x00c3e990 is located in stack of thread T0 at offset 80 in frame
    #0 0xf5e6ef in SATD  mv-search.c:1018

  This frame has 1 object(s):
    [16, 80) 'd' <== Memory access at offset 80 overflows this variable
HINT: this may be a false positive if your program uses some custom stack unwind mechanism
or swapcontext
      (longjmp and C++ exceptions *are* supported)
SUMMARY: AddressSanitizer: stack-buffer-overflow  mv-search.c:1093 SATD
Shadow bytes around the buggy address:
  0x20187ce0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x20187cf0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x20187d00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x20187d10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x20187d20: 00 00 00 00 00 00 00 00 f1 f1 00 00 00 00 00 00
=>0x20187d30: 00 00[f3]f3 f3 f3 00 00 00 00 00 00 00 00 00 00
  0x20187d40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x20187d50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x20187d60: 00 00 00 00 f1 f1 00 00 00 00 00 00 00 00 00 00
  0x20187d70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x20187d80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

Reported by [email protected] on 2014-05-19 13:42:34

  • Status changed: Started

@ramosian-glider
Copy link
Member Author 

Aha, this is an obvious bug which is already documented in FoundBugs#Spec_CPU_2006

Reported by [email protected] on 2014-05-19 13:54:54

@ramosian-glider
Copy link
Member Author 

ASan WAI

Reported by [email protected] on 2014-05-19 13:55:15

  • Status changed: Verified

@ramosian-glider
Copy link
Member Author 

Adding Project:AddressSanitizer as part of GitHub migration.

Reported by ramosian.glider on 2015-07-30 09:12:58

  • Labels added: ProjectAddressSanitizer

@ramosian-glider ramosian-glider mentioned this issue Sep 1, 2015
Closed
@sarora5 sarora5 mentioned this issue Jun 12, 2019
Open
@oviano oviano mentioned this issue Mar 31, 2020
Open
@UjjwalChitransh UjjwalChitransh mentioned this issue Apr 3, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
OpSys-Windows Priority-Medium ProjectAddressSanitizer Status-Verified Type-Other
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@ramosian-glider