Improve performance of data service

[alchzh]

What is the use case for the feature?

The current data service and resource implementation in ofrak causes a substantial number of redundant copies of root resource data.

• When resources are patched, every patch causes a back and forth copy to/from bytearrays
• Fetching a resource's data causes copies due to slicing the root resource's data range
• Passing data to ctypes (such as with the magic and entropy components) causes multiple copies
• Components that call external tools often write out the data to disk, causing an additional slowdown on top of the above

This can severely degrade performance when resources have a lot of data.

Does the feature contain any proprietary information about another company's intellectual property?

No

How would you implement this feature?

• Changing the data service to maintain a bytearray for the root data instead of a readonly bytes object
• Adding methods to get memoryview objects for resource data for copy-free operation
• Consider implementing a mmap backed data service

Are you interested in implementing it yourself?

Yes. Some work on this is in progress.