Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

100% sampling, aka firehose mode (mostly storage affecting) #1869

Open
codefromthecrypt opened this issue Jan 6, 2018 · 2 comments
Open

100% sampling, aka firehose mode (mostly storage affecting) #1869

codefromthecrypt opened this issue Jan 6, 2018 · 2 comments
Labels
collector enhancement storage Group label for Storage components

Comments

@codefromthecrypt
Copy link
Member

to be fleshed out: this would be a mode that has more impact on tracers and transports than the server. That said, a 100% mode likely means less or no indexing on storage side, with a short (like 1hr) retention. There could be UI support at least fetch by ID which could query both the normal sampled storage and a firehose one, or it could be considered different apps. These details will be added and refreshed into a revised description with a plan.
@codefromthecrypt codefromthecrypt mentioned this issue Jan 7, 2018
Closed
codefromthecrypt pushed a commit that referenced this issue Jan 9, 2018
Adds ability to disable indexing for in-memory and cassandra v2 storage
7452571 
This starts towards "firehose mode" by making it possible to disable
indexing, notably on cassandra where SASI amplifies writes into the
cluster.

When enabled, only retrieval by ID is allowed. That said, someone can
feel free to run the dependency linker if they like.. If 100% sampling
is on, the dependency links will relate to actual service calls made.

See #1869
@codefromthecrypt codefromthecrypt mentioned this issue Jan 9, 2018
Merged
@codefromthecrypt
Copy link
Member Author

here's a start #1876 cc @drolando

codefromthecrypt pushed a commit that referenced this issue Jan 10, 2018
Adds ability to disable indexing for in-memory and cassandra v2 storage
15e09de 
This starts towards "firehose mode" by making it possible to disable
indexing, notably on cassandra where SASI amplifies writes into the
cluster.

When enabled, only retrieval by ID is allowed. That said, someone can
feel free to run the dependency linker if they like.. If 100% sampling
is on, the dependency links will relate to actual service calls made.

See #1869
codefromthecrypt pushed a commit that referenced this issue Jan 12, 2018
Adds ability to disable indexing for in-memory and cassandra v2 storage
85fcb2d 
This starts towards "firehose mode" by making it possible to disable
indexing, notably on cassandra where SASI amplifies writes into the
cluster.

When enabled, only retrieval by ID is allowed. That said, someone can
feel free to run the dependency linker if they like.. If 100% sampling
is on, the dependency links will relate to actual service calls made.

See #1869
@codefromthecrypt
Copy link
Member Author

Current status:

As of latest version, SEARCH_ENABLED=false works for elasticsearch, cassandra3 and in-memory storage, disabling all indexing. A future change #1893 should hide the Find UI as it no longer would work when search is disabled.

@codefromthecrypt codefromthecrypt mentioned this issue Jan 30, 2018
Merged
@codefromthecrypt codefromthecrypt added enhancement collector storage Group label for Storage components labels Oct 23, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
collector enhancement storage Group label for Storage components
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@codefromthecrypt