[WIP] JSON Log Files #70830

ZephyrTFA · 2022-10-27T03:03:25Z

About The Pull Request

// TODO ZEPHYRTFA

Log Viewer pictures

This is a heavy WIP, and simply being put up early to convince me to actually finish this, (and early review)

Why It's Good For The Game

// TODO ZEPHYRTFA

Changelog

🆑
admin: New log viewer, crawling logs has never been easier!
admin: Hey guess what, a brand new log system, no more crawling through files!
refactor: The entire logging system just about
/:cl:

bobbah · 2022-10-27T03:09:37Z

what are the memory and performance implications of this

bobbah · 2022-10-27T03:12:59Z

also you will need to make sure that the logfiles are actually produced even if the server crashes; this is something that (while it is hard to ascertain from the current state of the pr) you may be about to forget about

under the current system logfile lines are written to disk as the round progresses, server crashes do not prevent us from seeing what was going on in the round

san7890

I know it's WIP, but here's two thoughts I had

code/controllers/subsystem/logging.dm

code/datums/logging/_log_entry.dm

code/modules/unit_tests/json_logging.dm

MrStonedOne · 2022-10-28T06:14:12Z

CONFIG_GET(string/server) gets the URL + port, which in our case hides the real IP.

This does not work. none of the servers have it set ever except by vv for a round.

Also its a stupid setting that has side effects.

Cross_coms_name server_name, sql_name, are all fallback names that could be used. world.port how ever should be the main fallback, its how /tg/ used to differentiate servers and since byond has no default port, supporting the same port on multiple servers by the same community when the config(s) exists seems silly. world.address is unreliable and trying to read it can actually sleep in edge cases.

Adding in a config for public hostname/url is actually something we should do, and rename server to reconnect_uri because all server really does is just forces a byond redirect to the configured address on round end (like i'll vv server to basil's url when i take down sybil) instead of letting byond keep the connections open while pretending they are new connections in game code (the better option for server load reasons).

code/modules/unit_tests/json_logging_category_unique.dm

regex supports match exact

code/modules/unit_tests/json_logging_semver.dm

Co-authored-by: Aleksej Komarov <[email protected]>

remove unused var recover entries_by_key

Mothblocks · 2022-10-30T19:21:50Z

BTW I don't know if oranges would disagree but something like this is never going to fully replace text logs, not as a matter of churn but as a fact that it would be a bad idea to impose too much work on something like logging.

This is going to be extremely useful still I just want to make sure we acknowledge that

stylemistake · 2022-10-30T21:12:39Z

tgui/packages/tgui/interfaces/LogViewer.tsx

        {entries.map((entry) => (
-          <Flex.Item key={entry.key}>
+          <Stack.Item key={entry.key}>


You can replace this Stack.Item with a Box and have a similar effect (minus the gaps). The Stack itself can be removed.

stylemistake · 2022-10-30T21:16:38Z

tgui/packages/tgui/interfaces/LogViewer.tsx

+                onTextChange={(text) => setFilterText(text)}
+                onFlagsChange={(flags) => setFilterFlags(flags)}
+              />
+            </Stack.Item>
          )}
          <LogCategoryBar


You need to wrap this in a Stack.Item with grow property (as I desribed it to you in Discord), and section (part of LogViewerContent) must have a "fill" property. Stack -> Stack.Item hierarchy is important to maintain. You also might want to split LogCategoryBar from LogViewerContent, and instead of nesting have them as individual Sections wrapped in individual Stack.Items.

optimumtact · 2022-10-31T01:00:20Z

BTW I don't know if oranges would disagree but something like this is never going to fully replace text logs, not as a matter of churn but as a fact that it would be a bad idea to impose too much work on something like logging.

This is going to be extremely useful still I just want to make sure we acknowledge that

Then why the heck are you encouraging them to keep going?

Why would we have two logging systems

ZephyrTFA · 2022-10-31T02:07:21Z

Well that was a lot of wasted work

optimumtact · 2022-10-31T04:11:19Z

you should PR it to a downstream

sorry that this happened btw, I never asked anyone to do my log overhaul because it was never a fully fleshed out idea. @ZephyrTFA who did you talk to that encouraged you to start this?

I always intended it to replace the entire existing logging system, yes it requires more developer overhead, but they can suck it up for server admins lives being easier.

optimumtact · 2022-10-31T04:26:03Z

@ninjanomnom you have to decide if you want two logging systems or not, mothblocks and I cannot agree

GoldenAlpharex · 2022-10-31T04:35:07Z

I think the little amount of developper overhead is worth it, even if it's mostly to enforce a certain format. We can make wrappers for stuff, I'm sure, with time it won't be much worse than it currently is, it'll just mean you'll have to think for more than two seconds about how you format your log entry.

Mothblocks · 2022-10-31T07:19:50Z

Maintainers have historically never enforced adding logs to things outside of very obvious cases because even to the ones that are regular admins, it's not obvious what you'll need until you need it

If this was required you would see people add logs far less

I'm not saying it isn't worth it, I'm saying people won't do it without being asked or if they personally benefit from it

GoldenAlpharex · 2023-02-11T22:20:53Z

I come back here after many months, bearing ninja's response to this, saying that he's fine with the JSON logging, so long as it is the only system, and not there alongside the old logging system.

ZephyrTFA · 2023-02-11T22:30:12Z

I have a different backend already in place, this just means I can begin JSON'ification of existing logs

ZephyrTFA · 2023-02-11T22:30:35Z

Which means my log viewer baby can become alive again, which is pog

Mothblocks · 2023-02-11T23:19:18Z

You are never going to see people adding logs without being asked again

optimumtact · 2023-02-12T19:44:34Z

you will have to make a new PR this one cannot be reopened

ZephyrTFA · 2023-02-12T20:32:10Z

you will have to make a new PR this one cannot be reopened

I need to make a new PR anyway; since I already have a json logging back end in place from a previous PR that I want to make use of

MrStonedOne · 2023-02-12T20:45:39Z

I am still holding firm that logs have to be human readable descriptive lines in files first and data/machine formated whatever second.

If any json system can't produce log files that look like our current logs (descriptive lines in files that state what event happened in human terms) its not gonna pass my mark.

optimumtact · 2023-02-12T23:59:26Z

they would be seperate logfiles are you high

ZephyrTFA added 3 commits October 26, 2022 23:00

base log entry

5fe0a67

asay

74a28cc

ooc and tick files

589fade

tgstation-server added Administration As generous gods, we have deigned to throw the jannies a bone Refactor Makes the code harder to read labels Oct 27, 2022

github-actions bot requested a review from dragomagol October 27, 2022 03:04

san7890 reviewed Oct 27, 2022

View reviewed changes

code/controllers/subsystem/logging.dm Outdated Show resolved Hide resolved

code/datums/logging/_log_entry.dm Outdated Show resolved Hide resolved

Mothblocks assigned optimumtact and Mothblocks Oct 27, 2022

Mothblocks reviewed Oct 27, 2022

View reviewed changes

code/datums/logging/_log_entry.dm Outdated Show resolved Hide resolved

Mothblocks reviewed Oct 27, 2022

View reviewed changes

code/datums/logging/_log_entry.dm Outdated Show resolved Hide resolved

Mothblocks reviewed Oct 27, 2022

View reviewed changes

code/datums/logging/_log_entry.dm Outdated Show resolved Hide resolved

dragomagol self-assigned this Oct 27, 2022

ZephyrTFA added 2 commits October 27, 2022 19:57

a little bit of doc cleanup and can_inspect striking

ffd3a96

unit test checking semver validity

95b82d7

github-actions bot requested a review from Mothblocks October 28, 2022 00:12

Mothblocks reviewed Oct 28, 2022

View reviewed changes

code/modules/unit_tests/json_logging.dm Outdated Show resolved Hide resolved

Mothblocks reviewed Oct 28, 2022

View reviewed changes

code/modules/unit_tests/json_logging.dm Outdated Show resolved Hide resolved

Mothblocks reviewed Oct 28, 2022

View reviewed changes

code/modules/unit_tests/json_logging.dm Outdated Show resolved Hide resolved

ZephyrTFA added 2 commits October 27, 2022 21:09

minor rename

b26ff98

snip

e511567

github-actions bot requested a review from Mothblocks October 28, 2022 01:13

forgot to check for not modifying the version

1d3cc74

ZephyrTFA mentioned this pull request Oct 28, 2022

add a unix_timestamp extern tgstation/rust-g#122

Merged

stuff

15d1bea

ZephyrTFA commented Oct 28, 2022

View reviewed changes

code/modules/unit_tests/json_logging_category_unique.dm Outdated Show resolved Hide resolved

Update code/modules/unit_tests/json_logging_category_unique.dm

b9a3d43

LogViewer tgui

316d8d3

tgstation-server added the UI We make the game less playable, but with round edges label Oct 29, 2022

github-actions bot requested a review from stylemistake October 29, 2022 15:11

ZephyrTFA added 4 commits October 29, 2022 11:19

minor qol changes

2d56cf1

regex supports match exact

add proc to get key

2b2cefa

some minor changes, recursive data reader

183baa6

progress

3c59d0b

Mothblocks reviewed Oct 29, 2022

View reviewed changes

code/modules/unit_tests/json_logging_semver.dm Outdated Show resolved Hide resolved

it looks nicer now

795124b

Co-authored-by: Aleksej Komarov <[email protected]>

github-actions bot requested a review from Mothblocks October 30, 2022 10:23

dont rely on SStimer

d82fbcc

remove unused var recover entries_by_key

stylemistake reviewed Oct 30, 2022

View reviewed changes

optimumtact closed this Oct 31, 2022

ZephyrTFA deleted the json_log branch October 31, 2022 02:07

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] JSON Log Files #70830

[WIP] JSON Log Files #70830

ZephyrTFA commented Oct 27, 2022 •

edited

Loading

bobbah commented Oct 27, 2022

bobbah commented Oct 27, 2022

san7890 left a comment

MrStonedOne commented Oct 28, 2022 •

edited

Loading

Mothblocks commented Oct 30, 2022

stylemistake Oct 30, 2022

stylemistake Oct 30, 2022 •

edited

Loading

optimumtact commented Oct 31, 2022

ZephyrTFA commented Oct 31, 2022

optimumtact commented Oct 31, 2022 •

edited

Loading

optimumtact commented Oct 31, 2022

GoldenAlpharex commented Oct 31, 2022

Mothblocks commented Oct 31, 2022 •

edited

Loading

GoldenAlpharex commented Feb 11, 2023

ZephyrTFA commented Feb 11, 2023

ZephyrTFA commented Feb 11, 2023

Mothblocks commented Feb 11, 2023

optimumtact commented Feb 12, 2023

ZephyrTFA commented Feb 12, 2023

MrStonedOne commented Feb 12, 2023 •

edited

Loading

optimumtact commented Feb 12, 2023

[WIP] JSON Log Files #70830

[WIP] JSON Log Files #70830

Conversation

ZephyrTFA commented Oct 27, 2022 • edited Loading

About The Pull Request

Why It's Good For The Game

Changelog

bobbah commented Oct 27, 2022

bobbah commented Oct 27, 2022

san7890 left a comment

Choose a reason for hiding this comment

MrStonedOne commented Oct 28, 2022 • edited Loading

Mothblocks commented Oct 30, 2022

stylemistake Oct 30, 2022

Choose a reason for hiding this comment

stylemistake Oct 30, 2022 • edited Loading

Choose a reason for hiding this comment

optimumtact commented Oct 31, 2022

ZephyrTFA commented Oct 31, 2022

optimumtact commented Oct 31, 2022 • edited Loading

optimumtact commented Oct 31, 2022

GoldenAlpharex commented Oct 31, 2022

Mothblocks commented Oct 31, 2022 • edited Loading

GoldenAlpharex commented Feb 11, 2023

ZephyrTFA commented Feb 11, 2023

ZephyrTFA commented Feb 11, 2023

Mothblocks commented Feb 11, 2023

optimumtact commented Feb 12, 2023

ZephyrTFA commented Feb 12, 2023

MrStonedOne commented Feb 12, 2023 • edited Loading

optimumtact commented Feb 12, 2023

ZephyrTFA commented Oct 27, 2022 •

edited

Loading

MrStonedOne commented Oct 28, 2022 •

edited

Loading

stylemistake Oct 30, 2022 •

edited

Loading

optimumtact commented Oct 31, 2022 •

edited

Loading

Mothblocks commented Oct 31, 2022 •

edited

Loading

MrStonedOne commented Feb 12, 2023 •

edited

Loading