Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add flushing controls #4

Open
MichaelXavier opened this issue Feb 29, 2016 · 2 comments
Open

Add flushing controls #4

MichaelXavier opened this issue Feb 29, 2016 · 2 comments

Comments

@MichaelXavier
Copy link
Collaborator

A good point was raised that we don't do any internal buffering of logs in handle scribe. We may want to provide the user with controls such as how many (if any) messages to buffer, when to fsync, etc. Our elasticsearch scribe actually does some buffering with a bounded queue but this is just a memory usage tactic. Scribes still eagerly send every message as soon as they get it. There may be a heuristic there where its preferable to wait and use the bulk indexing API in elasticsearch.

@ozataman
Copy link
Member

Seems like it should be on the backends to do this since I doubt a single buffering strategy will be well suited for all future backends.

@MichaelXavier
Copy link
Collaborator Author

Yeah, katip-elasticsearch sets a good precedence of this: gathers all configuration into a type, exports only the type and lenses to its fields and a default constructor for maximum backwards compatibility when new fields are added. The next time we have to add configuration to the handle scribe (or add a new scribe) we should pull the band-aid off and gather all of its configuration arguments into a type.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants