big_mod_exp syscall: exclude inputs of numbers larger than 4096-bits

[valiksinev]

Problem

The changes related to Big integer modular exponentiation (EIP-198). The calculation is implemented as syscall.

The formula that was used to approximate the compute budgets for large number exponentiation seems to under-approximate the costs for exponentiation of very large numbers (i.e. >>1000 byte numbers). This could lead to a negative impact on the network if the syscall was to be invoked with very large numbers.
In practice, it seems like any normal/non-adversarial use of the syscall will be on numbers of size less than or equal to 4096-bits (i.e. for RSA-4096)

Summary of Changes

Inside syscall all inputs are limited by value 4069 bits (512 bytes).
If the length is exceeded, an error SyscallError::InvalidLength returned.

[t-nelson]

thanks! mind adding some test cases to cover this new logic?

[codecov]

Codecov Report

Merging #31859 (a2b73c2) into master (db8f118) will decrease coverage by 0.1%.
The diff coverage is 0.0%.

@@            Coverage Diff            @@
##           master   #31859     +/-   ##
=========================================
- Coverage    81.9%    81.9%   -0.1%     
=========================================
  Files         752      752             
  Lines      206751   206753      +2     
=========================================
- Hits       169427   169401     -26     
- Misses      37324    37352     +28     

[mvines]

I've moved #29882 off the activation schedule, and put it into https://github.com/solana-labs/solana/wiki/Feature-Gate-Activation-Schedule#features-are-blocked as any activation of that feature cannot happen until this PR is merged and backported to v1.16

[derrekr]

Ping @valiksinev do you have a test case yet?

[mvines]

The changes in this PR were merged via #32520