capa explorer: elf bug

[mr-tz]

sample: 16904

[mr-tz]

on error, cannot close plugin wait dialog, need to kill the entire IDA process

[mr-tz]

this appears to be a general bug with ELF files in IDA? see e.g. https://github.com/mandiant/capa-testfiles/blob/master/294b8db1f2702b60fb2e42fdc50c2cee6a5046112da9a5703a548a4fa50477bc.elf_

[mr-tz]

Debug output in IDA

Cannot read SHT in IDA like implemented, so subsequent parsing is faulty (get_bytes returns all 0xFFs 😞)

DEBUG:capa:reading 0x40 bytes at 0x0 (ea: 0x400000)
DEBUG:capa.features.extractors.elf:ei_class: 0x02 ei_data: 0x01
DEBUG:capa.features.extractors.elf:e_phoff: 0x40 e_phentsize: 0x38 e_phnum: 9
DEBUG:capa:reading 0x1f8 bytes at 0x40 (ea: 0x400040)
DEBUG:capa.features.extractors.elf:reading section header at 0x15608, size 0x740
DEBUG:capa:reading 0x740 bytes at 0x15608 (ea: 0x15608)
DEBUG:capa.features.extractors.elf:shbuf: b'ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff'
DEBUG:capa.features.extractors.elf:guess: osabi: None
DEBUG:capa.features.extractors.elf:ph:p_type: 0x0006
DEBUG:capa.features.extractors.elf:ph:p_offset: 0x40 p_filesz: 0x01f8
DEBUG:capa:reading 0x1f8 bytes at 0x40 (ea: 0x400040)
DEBUG:capa.features.extractors.elf:ph:p_type: 0x0003
DEBUG:capa.features.extractors.elf:ph:p_offset: 0x238 p_filesz: 0x001c
DEBUG:capa:reading 0x1c bytes at 0x238 (ea: 0x400238)
DEBUG:capa.features.extractors.elf:ph:p_type: 0x0001
DEBUG:capa.features.extractors.elf:ph:p_offset: 0x00 p_filesz: 0x14a76
DEBUG:capa:reading 0x14a76 bytes at 0x0 (ea: 0x400000)
DEBUG:capa.features.extractors.elf:ph:p_type: 0x0001
DEBUG:capa.features.extractors.elf:ph:p_offset: 0x14d58 p_filesz: 0x0748
DEBUG:capa:reading 0x748 bytes at 0x14d58 (ea: 0x614d58)
DEBUG:capa.features.extractors.elf:ph:p_type: 0x0002
DEBUG:capa.features.extractors.elf:ph:p_offset: 0x14d88 p_filesz: 0x0270
DEBUG:capa:reading 0x270 bytes at 0x14d88 (ea: 0x614d88)
DEBUG:capa.features.extractors.elf:ph:p_type: 0x0004
DEBUG:capa.features.extractors.elf:ph:p_offset: 0x254 p_filesz: 0x0044
DEBUG:capa:reading 0x44 bytes at 0x254 (ea: 0x400254)
DEBUG:capa.features.extractors.elf:ph:namesz: 0x04 descsz: 0x10 type: 0x0001
DEBUG:capa.features.extractors.elf:name: GNU
DEBUG:capa.features.extractors.elf:GNU_ABI_TAG: 0x00
DEBUG:capa.features.extractors.elf:abi tag: OS.LINUX earliest compatible kernel: 2.6.32
DEBUG:capa.features.extractors.elf:guess: ph notes: OS.LINUX
DEBUG:capa.features.extractors.elf:b'ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff'
DEBUG:capa.features.extractors.elf:64
DEBUG:capa.features.extractors.elf:sh:sh_offset: 0xffffffffffffffff sh_size: 0xffffffffffffffff

[mr-tz]

This data may only be available in IDA via get_segm_* and may not provide all we expect here (by default IDA does not load all segments, so note things, for example, unlikely will be there)?
Ping @williballenthin on what he thinks. Two options:

1. Adjust the handling
2. ignore guess_os_from_sh_notes (and other invalid reads) from the IDA extractor

[williballenthin]

probably should raise an exception from a wrapper around get_bytes rather than trying to process all those FFFFFFF bytes

[williballenthin]

such as here:

capa/capa/ida/helpers.py

Lines 183 to 185 in 64ebf14

	if ea == idc.BADADDR:
	# best guess, such as if file is mapped at address 0x0.
	ea = self.offset

seems like we're hiding bugs by not letting client code decide how to handle unmapped regions