Allow to store logs in files

[qwerty287]

#2278

[qwerty287]

In it's current state, this allows to store logs in files on the regular filesystem. Do we need native support for something like S3 or can we just ask users to mount it directly into the FS?

[zc-devs]

Do we need native support for something like S3 or can we just ask users to mount it directly into the FS?

in object storage

Native S3.

1. I believe, environments exist, where you cannot mount arbitrary FUSE (AWS EKS, Windows, I guess).
2. I'm not sure if FUSE S3 can patch a file. If can't, then this is not well from performance and maybe costs perspective.

---

Sadly, there is no description in the issue, but I suppose it is related to growing Database. If so, tiering across storage mediums could be implemented.

1. Some bunch of data stores in DB.
2. After some retention condition old data archives to S3 (Delete old pipeline logs after X days or Y new runs #1068).

---

That said, this PR adds a value, therefore it can be merged, but without closing the issue.

[qwerty287]

Sadly, there is no description in the issue, but I suppose it is related to growing Database.

Yes, I also think so.

Sadly, there is no description in the issue, but I suppose it is related to #3506 (comment). If so, tiering across storage mediums could be implemented.

1. Some bunch of data stores in DB.
2. After some retention condition old data archives to S3 (Delete old pipeline logs after X days or Y new runs #1068).

I get your point, but this looks pretty complicated instead of just deleting the pipelines/logs what is the idea in the linked issue.

That said, this PR adds a value, therefore it can be merged, but without closing the issue.

Probably it's best to just merge this how it's now, but keep the discussion open and ask there what the requirements are.

[woodpecker-bot]

Deployment of preview was torn down

[anbraten]

Could we try to merge #3722 first or do you think they wouldn't conflict that much?

[qwerty287]

I don't think there are many conflicts, this is only about the storing. I took a quick look at your changes and it looks like you didn't change it (the db methods)?

[codecov]

Codecov Report

Attention: Patch coverage is 0% with 62 lines in your changes missing coverage. Please review.

Project coverage is 25.81%. Comparing base (2c7edbf) to head (7e318d7).

Files	Patch %	Lines
server/services/log/file/file.go	0.00%	49 Missing ⚠️
cmd/server/setup.go	0.00%	6 Missing ⚠️
cmd/server/server.go	0.00%	4 Missing ⚠️
server/api/pipeline.go	0.00%	2 Missing ⚠️
server/grpc/rpc.go	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3568      +/-   ##
==========================================
- Coverage   25.87%   25.81%   -0.06%     
==========================================
  Files         361      362       +1     
  Lines       26667    26725      +58     
==========================================
  Hits         6899     6899              
- Misses      19235    19293      +58     
  Partials      533      533              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.