LIBTIFF-0003

Summary

JPEGEncodeRaw() packs the caller’s raw subsampled samples into libjpeg’s downsampled buffers. It reads the input as inptr = (TIFF_JSAMPLE *)buf + clumpoffset (tif_jpeg.c:2479) and consumes samples_per_clump * clumps_per_line samples per clumpline (reads at 2486 / 2496, inptr += samples_per_clump). In the 12-bit build TIFF_JSAMPLE is a 2-byte J12SAMPLE, so it reads 2 * samples_per_clump * clumps_per_line bytes per clumpline.

But 12-bit TIFF sample data is stored bit-packed (12 bits/sample, 1.5 bytes) — the decode side packs it (tif_jpeg.c:1811-1828) — and the per-clumpline stride bytesperclumpline (tif_jpeg.c:2447-2453) is correctly computed as ceil(clumps * (h*v+2) * data_precision / 8) ≈ 1.5×, matching TIFFStripSize. The encoder therefore reads 2 bytes/sample from a 1.5-byte/sample buffer and runs ~0.5 * samples_per_clumpline bytes past the end of the TIFFStripSize-sized input strip on the final clumpline:

• even width → heap-buffer-overflow READ (tif_jpeg.c:2496)
• odd width → buf += bytesperclumpline advances an odd number of bytes, so (J12SAMPLE*)buf is misaligned → UBSan misaligned-load UB

The over-read bytes are entropy-coded into the output JPEG, i.e. adjacent heap memory is disclosed into attacker-observable output.

Reproduction

pocs/libtiff/LIBTIFF-0003/encode_poc.c writes a 12-bit YCbCr 2×2 JPEG via TIFFWriteEncodedStrip, buffer sized to TIFFStripSize() (canonical usage), default RAW JPEGColorMode.

./enc 16 /tmp/out.tif
==ERROR: AddressSanitizer: heap-buffer-overflow READ of size 2
    #0 JPEGEncodeRaw        libtiff/tif_jpeg.c:2496
    #1 TIFFWriteEncodedStrip libtiff/tif_write.c:342
  allocated by: calloc(TIFFStripSize=576) in main
SUMMARY: heap-buffer-overflow tif_jpeg.c:2496 in JPEGEncodeRaw

Fires for every tested width (8/16/24/32/48). See repro.log.

Root cause

The 12-bit raw encode path treats a bit-packed 12-bit input buffer as an array of 2-byte J12SAMPLE. Either the input must be unpacked to 2-byte samples (and bytesperclumpline/TIFFStripSize sized to match), or the reads must honor the packed layout. As written, TIFFStripSize (1.5×, packed) disagrees with the encoder’s 2× read, so a caller trusting TIFFStripSize over-reads. This is the encode-side sibling of LIBTIFF-0002 (the 12-bit downsampled sizing mismatch).

Verification evidence

Reproduced end-to-end through the public TIFFWriteEncodedStrip API of libtiff 4.7.2 built against libjpeg-turbo 3.1.4.1 (dual 8/12-bit mode) under clang-20 ASan+UBSan: heap-buffer-overflow READ at tif_jpeg.c:2496 on a TIFFStripSize-sized buffer, plus misaligned-load UB for odd widths.

Impact

Apply impact-rubric: severity medium. Heap out-of-bounds read reachable from the public strip/tile write API when encoding 12-bit subsampled YCbCr JPEG in raw mode, with canonical buffer sizing (TIFFStripSize). Two consequences: (1) information disclosure — adjacent heap bytes are encoded into the output JPEG, observable by whoever receives the file; (2) potential crash on an unmapped page. Narrower than the decode-side write bug (LIBTIFF-0002): the raw-encode path requires an application that supplies pre-subsampled 12-bit data without JPEGCOLORMODE_RGB, and a 12-bit (turbo ≥3.0 dual-mode) libtiff. No OOB write and no integrity impact. The 12-bit raw-encode path is also functionally broken (mis-packs), so it is likely rarely exercised in the wild.