crytic_compile.py | get_line_from_offset() pulling from incorrect file

[plotchy]

Ran into issues involving KeyError XXX similar to crytic/slither#1324

Did some basic debugging and noticed this is due to sourceMap iteration going out of bounds for particular contracts in the initialization stages of crytic_compile.

I added in some basic print statements to the get_line_from_offset() fn and noticed that changing unrelated contracts would adjust the KeyError OoB value for another unrelated contract.

Set Up

Two contract files:

// ./src/A.sol
pragma solidity ^0.8.4;

contract BaseContract{
    uint256 one;
    uint256[50] private __gap;
    uint256 two;
}

contract DerivedContract is BaseContract{
    uint256 three;
    uint256[50] private __gap;
    uint256 four;
}

// ./src/complex_func.sol
pragma solidity ^0.4.24;
// pragma solidity ^0.8.12;

contract Complex {
    uint i0 = 0;
    uint i1 = 0;
    uint i2 = 0;
    uint i3 = 0;
    uint i4 = 0;
    uint i5 = 0;
    uint i6 = 0;

    function complexStateVars() external {
        i0 = 1;
    }
}

Command:
slither .
As I'm in a foundry project, crytic_compile runs command
forge build --extra-output abi --extra-output userdoc --extra-output devdoc --extra-output evm.methodIdentifiers --force
under the hood

Debugging Output

FIrst, I added a print statement to show the file file offset, line, and char offset in the get_line_from_offset() fn

crytic-compile/crytic_compile/crytic_compile.py

Lines 273 to 292 in 6bb6e72

	def get_line_from_offset(self, filename: Union[Filename, str], offset: int) -> Tuple[int, int]:
	"""Return the line from a given offset
	Args:
	filename (Union[Filename, str]): filename
	offset (int): global offset
	Returns:
	Tuple[int, int]: (line, line offset)
	"""
	if isinstance(filename, str):
	file = self.filename_lookup(filename)
	else:
	file = filename
	if file not in self._cached_offset_to_line:
	self._get_cached_offset_to_line(file)
	lines_delimiters = self._cached_offset_to_line[file]
	return lines_delimiters[offset]

def get_line_from_offset(self, filename: Union[Filename, str], offset: int) -> Tuple[int, int]:
        ### snip ###

        lines_delimiters = self._cached_offset_to_line[file]

        # added this print:
        print("File offset: " + str(offset) + ", line: " + str(lines_delimiters[offset][0]) + ", char: " + str(lines_delimiters[offset][1]) + " from file: " + str(file.short))

        return lines_delimiters[offset]

Which provides this type of output:

File offset: 0, line: 1, char: 1 from file: src/complex_func.sol
File offset: 24, line: 1, char: 25 from file: src/complex_func.sol
File offset: 54, line: 4, char: 1 from file: src/complex_func.sol
File offset: 259, line: 16, char: 2 from file: src/complex_func.sol
File offset: 0, line: 1, char: 1 from file: src/A.sol
File offset: 23, line: 1, char: 24 from file: src/A.sol
File offset: 25, line: 3, char: 1 from file: src/A.sol
File offset: 114, line: 7, char: 2 from file: src/A.sol
File offset: 116, line: 9, char: 1 from file: src/A.sol
File offset: 227, line: 14, char: 0 from file: src/A.sol

Where two paths diverge

Running complex_func.sol as solidity ^0.4.24 causes a different output from running it as solidity ^0.8.12

Under 0.8.12:

File offset: 28, line: 2, char: 1 from file: src/complex_func.sol
File offset: 52, line: 2, char: 25 from file: src/complex_func.sol
File offset: 54, line: 4, char: 1 from file: src/complex_func.sol
File offset: 259, line: 16, char: 2 from file: src/complex_func.sol
File offset: 0, line: 1, char: 1 from file: src/A.sol
File offset: 23, line: 1, char: 24 from file: src/A.sol
File offset: 25, line: 3, char: 1 from file: src/A.sol
File offset: 114, line: 7, char: 2 from file: src/A.sol
File offset: 116, line: 9, char: 1 from file: src/A.sol
File offset: 227, line: 14, char: 0 from file: src/A.sol
File offset: 77, line: 5, char: 5 from file: src/complex_func.sol  ##Note: Scope has changed to complex_func for inner level contract nodes!
File offset: 88, line: 5, char: 16 from file: src/complex_func.sol
File offset: 94, line: 6, char: 5 from file: src/complex_func.sol
File offset: 105, line: 6, char: 16 from file: src/complex_func.sol
File offset: 111, line: 7, char: 5 from file: src/complex_func.sol
File offset: 122, line: 7, char: 16 from file: src/complex_func.sol
File offset: 128, line: 8, char: 5 from file: src/complex_func.sol
File offset: 139, line: 8, char: 16 from file: src/complex_func.sol
File offset: 145, line: 9, char: 5 from file: src/complex_func.sol
File offset: 156, line: 9, char: 16 from file: src/complex_func.sol
File offset: 162, line: 10, char: 5 from file: src/complex_func.sol
File offset: 173, line: 10, char: 16 from file: src/complex_func.sol
File offset: 179, line: 11, char: 5 from file: src/complex_func.sol
File offset: 190, line: 11, char: 16 from file: src/complex_func.sol
File offset: 197, line: 13, char: 5 from file: src/complex_func.sol
File offset: 257, line: 15, char: 6 from file: src/complex_func.sol

• Compilation succeeds
  Seems like sourceMapping goes from:
  complex_func - file level
  A - file level
  complex_func - contract level
  A - contract level
  ...

Under 0.4.24:

File offset: 0, line: 1, char: 1 from file: src/complex_func.sol
File offset: 24, line: 1, char: 25 from file: src/complex_func.sol
File offset: 54, line: 4, char: 1 from file: src/complex_func.sol
File offset: 259, line: 16, char: 2 from file: src/complex_func.sol
File offset: 0, line: 1, char: 1 from file: src/A.sol
File offset: 23, line: 1, char: 24 from file: src/A.sol
File offset: 25, line: 3, char: 1 from file: src/A.sol
File offset: 114, line: 7, char: 2 from file: src/A.sol
File offset: 116, line: 9, char: 1 from file: src/A.sol
File offset: 227, line: 14, char: 0 from file: src/A.sol
File offset: 77, line: 5, char: 13 from file: src/A.sol  # Note: Why does this stay in file A.sol? And why is it referring to complex_func's offset 77? This is incorrect
File offset: 88, line: 5, char: 24 from file: src/A.sol
File offset: 94, line: 5, char: 30 from file: src/A.sol
File offset: 105, line: 6, char: 10 from file: src/A.sol
File offset: 111, line: 6, char: 16 from file: src/A.sol
File offset: 122, line: 9, char: 7 from file: src/A.sol
File offset: 128, line: 9, char: 13 from file: src/A.sol
File offset: 139, line: 9, char: 24 from file: src/A.sol
File offset: 145, line: 9, char: 30 from file: src/A.sol
File offset: 156, line: 9, char: 41 from file: src/A.sol
File offset: 162, line: 10, char: 5 from file: src/A.sol
File offset: 173, line: 10, char: 16 from file: src/A.sol
File offset: 179, line: 11, char: 3 from file: src/A.sol
File offset: 190, line: 11, char: 14 from file: src/A.sol
File offset: 197, line: 11, char: 21 from file: src/A.sol

• Compilation fails
• Appears that after switching to context of A.sol, it remains there and attempts to map complex_func.sol's source to it

Eventually output continues to a traceback saying KeyError 257, which is OoB in A.sol source.
Offset 257 happens to coincide with complex_func.sol's ending curly brace of fn complexStateVars()

Adding lines/comments to complex_func will relate to a new KeyError offset within A.sol.

I'm not sure where the context switching logic is within crytic_compile, but it seems to be working incorrectly for mismatched solidity versions coming from foundry/hardhat repos.

[plotchy]

Likely root issue of both #269 and crytic/slither#1241 as well