Make better use of i18n

[Gusted]

• Previously, we relied on performing numerous expensive map lookups and storing each parsed ini file in memory.
  As we add more translations and languages, this strategy won't scale and has become visible and measurable in memory utilization. All parsed ini currently uses about 20-30Mb of memory, which is nothing on modern computers. Every bit counts when running Gitea on low-powered devices like Raspberry Pis and Android phones. This PR fixes that by using a new data structure that don't require more memory than is necessary and is scalable for the future.
• Startup memory is now closer to 60Mb than 90Mb, making it easier to stay under 100Mb. The new implementation needs 7Mb to hold a single map for translation key -> index and for each locale a maps index -> translation value.
• The lookup for translation values may be greatly improved, leading to faster performance overall, even though I haven't performed any tests or benchmarks comparing the current and earlier implementations. Additionally, the language struct, which contains the important map for index -> translation value, so you don't have to do another map lookup to get the locale struct.

---

Before:

25Mb to initialize the locales(After Go's GC)

After:

7Mb to initialize the locales(After Go's GC)

---

As usual, goodluck reviewing this piece of beauty and mess.

[wxiaoguang]

I think it's on the correct direction for the i18n refactoring.

The only question in my mind is: is the perfect hash map necessary? If the simple map can still save the memory and works fine, I would prefer to use a simple map (like map[string][]string => map[key][offset]) to reduce maintenance work.

[Gusted]

is the perfect hash map necessary? If the simple map can still save the memory and works fine, I would prefer to use a simple map (like map[string][]string => map[key][offset]) to reduce maintenance work.

So, I'd preferred the perfect hash as it was made for these kind of situations, need fast query and the keys are known before-hand. Thereby there was no really good quality perfect hash library(so that's why I crafted one and included in the PR). I've pushed a commit(d498472) that utilized the same concept of the language offset and then instead use golang's Map feature. The memory is bit more on the heap(it says 10Mb) but the overall memory usage still seems to be around the same levels of what the previous implementation had.

6bfa1ba actually is the one that moves back the original data structure that is optimized for memory.

[wxiaoguang]

L-G-T-M (except CI failure is related)

And, could the translationKeys be set to nil to save some memory? It seems that it's not used after the loop IIRC.

[delvh]

--- FAIL: TestLDAPUserSigninFailed (0.30s)
testlogger.go:78: 2022/06/24 01:51:14 modules/git/git.go:133:checkInit() [W] git module has been initialized already, duplicate init should be fixed

That issue once again...

[wxiaoguang]

--- FAIL: TestLDAPUserSigninFailed (0.30s)
testlogger.go:78: 2022/06/24 01:51:14 modules/git/git.go:133:checkInit() [W] git module has been initialized already, duplicate init should be fixed

That issue once again...

Off-topic Unrelated .... just a warning, no harm. It might be resolved after another big git module refactoring.

[delvh]

I think, I got the gist of what you're doing.
If it works, LGTM.

[Gusted]

Pushed commits to separate dev and production localestores(so it's easier to implement live-reloading etc.). Provided by @zeripath

[wxiaoguang]

I think some code can be simplified, will try on it later.

[Gusted]

I think some code can be simplified, will try on it later.

Even more refactors 😄 I will mark this PR for 1.19 already

[wxiaoguang]

I believe 1.18 is fine (will try on it later means several hours later, I am going outside soon 😂) .

[wxiaoguang]

I proposed a PR about my idea. Gusted#5

Major benefits:

• more simple (+185 −509)
• no more go-routine, no need to close (in case it's forgotten ...)
• no duplicate code, only one Tr / reloadLocaleByIni function, more easier to be maintained
• works fine for dev mode, the reload delay is (at most) 1 second, it won't affect development in real life: change locale files -> refresh page to see the result -> more than 1 second has been elapsed.

[wxiaoguang]

(approve, although I also made some changes, open for suggestions)