Initialize package and docs for log-collection library transfer #9328
Conversation
djaglowski
force-pushed
the
init-log-collection
branch
2 times, most recently
from
a58b980 to
da59a5a
Compare
djaglowski
force-pushed
the
init-log-collection
branch
from
da59a5a to
00f810a
Compare
pkg/stanza/docs/operators/README.md
Outdated
| ## What is an operator? | ||
| An operator is the most basic unit of log processing. Each operator fulfills a single responsibility, such as reading lines from a file, or parsing JSON from a field. Operators are then chained together in a pipeline to achieve a desired result. | ||
|
|
||
| For instance, a user may read lines from a file using the `file_input` operator. From there, the results of this operation may be sent to a `regex_parser` operator that creates fields based on a regex pattern. And then finally, these results may be sent to a `elastic_output` operator that writes each line to Elasticsearch. |
There was a problem hiding this comment.
| For instance, a user may read lines from a file using the `file_input` operator. From there, the results of this operation may be sent to a `regex_parser` operator that creates fields based on a regex pattern. And then finally, these results may be sent to a `elastic_output` operator that writes each line to Elasticsearch. | |
| For instance, a user may read lines from a file using the `file_input` operator. From there, the results of this operation may be sent to a `regex_parser` operator that creates fields based on a regex pattern. And then finally, these results may be sent to a `stdout` operator that writes each line to stdout. |
This is probably minor, but elastic is no longer included among the outputs. Also, I'm wondering if we should describe that the library is used in the receiver which means it does not need to use outputs normally
There was a problem hiding this comment.
Good feedback, this doc is quite out of date. I took a pass through the document and updated the contents to reflect the current state much more accurately.
| @@ -0,0 +1,46 @@ | |||
| ## What is an operator? | |||
There was a problem hiding this comment.
Perhaps we could add a status describing that the library move is in progress?
|
@djaglowski one approach we did when moved lots of core components to contrib, was to move everything using git subtree to save history, and explicitly remove that module from the "Makefile" build/test/checks until we were done with it. The beauty was that we saved the entire history of commits. |
This sounds like a better approach. I'll look into it. |
|
I'm converting this back to a draft until Bogdan's suggested approach is looked into. |
|
This PR was marked stale due to lack of activity. It will be closed in 14 days. |
|
Closing in favor of #9376 |
This PR is a first incremental step towards moving the log-collection library into this repo, as defined by #8850.
An all-in-one PR (#9227) was made, but proved to require many untracked changes. I am now taking an incremental approach as described in this comment on #9227.