Investigate use of lru_cache on methods

[DanielNoord]

Problem

I stumbled upon this video which explains problems with using lru_cache on methods as it leaks self. I haven't watched the full video yet but watched the previous video and saw the PR at pytest here: pytest-dev/pytest#9148

This is something we're doing ourselves as well and could be a potential source of memory leaks and other problems. I think this warrants an investigation and probably a refactor.

/CC @keichi As your PR #1242 also decorates methods, they might also be affected by this. Note that I haven't watched the full video yet so I don't know the exact implications but this is worthwhile to investigate.

[jacobtylerwalls]

We looked into this in #1521 and decided the two remaining uses of @lru_cache on methods were fine because of the extremely low cache sizes, 128 and ~13.