While parsing a specially crafted GIF file, an integer overflow vulnerability and result in out of bounds heap memory overwrite potentially leading to arbitrary code execution.
Oracle Outside In IX sdk 8.5.1
While parsing a GIF file with a ImageWidth in IMAGEDESCRIPTOR set to 0xFFFF an integer loop counter checking on truncated integer size results in an integer overflow which leading to unbounded memory write in two branches of the same function.
The first integer overflow occurs while parsing the supplied testcase and
following basic block in
libvs_gif.so is reached
(image base being 0xB7F81000):
` .text:B7F82832 loc_B7F82832: .text:B7F82832 movzx eax, byte ptr [edx]  .text:B7F82835 shl al, 4 .text:B7F82838 or al, [edx+1] .text:B7F8283B mov [edi], al  .text:B7F8283D add edi, 1 .text:B7F82840 add edx, 2  .text:B7F82843 mov eax, edx  .text:B7F82845 sub ax, word ptr [esp+10h+var_10]  .text:B7F82849 cmp ax, [ebp+60h]  .text:B7F8284D jb short loc_B7F8 `
In the above code, at ,
ebx points to a buffer on the heap.
At , each time the code loops,
edx is incremented by 2. At
 pointer from
edx is moved to
eax for further comparison.
At , value of
var_10 contains a saved pointer to the start
of the heap buffer (the original value of edx when entering the basic
block for the first time). Also at , lower two bytes of
are substracted from
ax (which effectively contains lower two bytes of current
buffer pointer) effectively calculating the offset into the buffer.
This offset is then compared at  to a value at
is actually the ImageWidth from the file.
Since value at
ebp+0x60 is always 0xFFFF, 16 bit integer arithmetic
is in place, and
edx is always incremented by 2, the jump condition
at the end of the block will always be satisfied. Thus the integer
overflow resulting from substraction at  turns this into an infinite loop.
In the supplied testcase (integer_overflow.gif), the overwrite eventually hits either an unmapped region of memory or a read-only page resulting in a crash.
The only change to the original file is ImageWidth (at offset 0x32) which is set to 0xFFFF. Added garbage at the end of the file is there to align the heap so the application crashes on WriteAV at  otherwise it crashes with ReadAV at 
A similar integer overflow can be triggered if the LocalColorTableFlag is set in the IMAGEDESCRIPTOR PackedFields (at offset 0x36 in integer_overflow2.gif). In this case, the other branch in the function at 0xB7F8280C is taken and code ends up in the following basic block:
` .text:B7F8285F loc_B7F8285F: .text:B7F8285F movzx ecx, byte ptr [esi] .text:B7F82862 shl cl, 7 .text:B7F82865 movzx eax, byte ptr [esi+1] .text:B7F82869 shl al, 6 .text:B7F8286C or cl, al .text:B7F8286E movzx eax, byte ptr [esi+4] .text:B7F82872 shl al, 3 .text:B7F82875 or cl, al .text:B7F82877 movzx eax, byte ptr [esi+6] .text:B7F8287B add al, al .text:B7F8287D or cl, al .text:B7F8287F movzx edx, byte ptr [esi+2] .text:B7F82883 shl dl, 5 .text:B7F82886 movzx eax, byte ptr [esi+3] .text:B7F8288A shl al, 4 .text:B7F8288D or dl, al .text:B7F8288F movzx eax, byte ptr [esi+5] .text:B7F82893 add al, al .text:B7F82895 add al, al .text:B7F82897 or dl, al .text:B7F82899 or dl, [esi+7] .text:B7F8289C or cl, dl .text:B7F8289E mov [edi], cl .text:B7F828A0 add edi, 1 .text:B7F828A3 add esi, 8 .text:B7F828A6 mov eax, esi .text:B7F828A8 sub ax, word ptr [esp+10h+var_10] .text:B7F828AC cmp [ebp+60h], ax  .text:B7F828B0 ja short loc_B7F8 `
At , the same comparison is made as in the previously described integer overflow. The supplied testcase (integer_overflow2.gif) has a few more bytes changed, to accommodate for added LocalColorTableFlag.
Both issues can be triggered with the supplied testcase in the
supplied with the SDK.
2016-04-12 - Vendor Notification
2016-07-19 – Public Disclosure
Discovered by Aleksandar Nikolic of Cisco Talos.