Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Provide more insight into indexing failures #117700

Open
1 of 2 tasks
martijnvg opened this issue Nov 28, 2024 · 2 comments
Open
1 of 2 tasks

Provide more insight into indexing failures #117700

martijnvg opened this issue Nov 28, 2024 · 2 comments
Assignees
@martijnvg
Labels
:Distributed Indexing/Engine Anything around managing Lucene and the Translog in an open shard. >enhancement :StorageEngine/Logs You know, for Logs Team:Distributed Indexing Meta label for Distributed Indexing team Team:StorageEngine

Comments

@martijnvg
Copy link
Member

martijnvg commented Nov 28, 2024 •
edited
Loading

The goal of this issue to do the following improvements in order to improve visibility into indexing failures:

  • Start to keep track of indexing failures caused by a predefined exception classification in InternalIndexingStats. For example indexing failures caused by: VersionConflictEngineException, other client errors and internal server errors. These counter stats will then be exposes in IndexingStats and consumed by IndicesMetrics (which would stores indexing stats as apm metrics).
  • Add new debug logging for indexing failures. Add a new IndexingOperationListener implementation that logs indexing failure as debug using a new logger. The postIndex(ShardId, Engine.Index, Exception) method should be overwritten to log exceptions and the postIndex(ShardId, Engine.Index, Engine.IndexResult) method should delegate to it if the index result type is failure. (Add additional debug logging for indexing failure. #117728)
@martijnvg martijnvg added :Distributed Indexing/Engine Anything around managing Lucene and the Translog in an open shard. :StorageEngine/Logs You know, for Logs >enhancement labels Nov 28, 2024
@elasticsearchmachine elasticsearchmachine added Team:StorageEngine Team:Distributed Indexing Meta label for Distributed Indexing team labels Nov 28, 2024
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-storage-engine (Team:StorageEngine)

@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-distributed-indexing (Team:Distributed Indexing)

@martijnvg martijnvg self-assigned this Nov 28, 2024
martijnvg added a commit to martijnvg/elasticsearch that referenced this issue Nov 28, 2024
@martijnvg
Add additional debug logging for indexing failure.
c0e2099 
This change adds a new IndexingOperationListener that logs debug logging for indexing failures.

Today `IndexShard` logger needs to be set to TRACE in order to see indexing failures. This logger is very verbose and logs many other aspects of indexing at trace level. This new logger is targeted just for seeing indexing failures.

Relates elastic#117700
@martijnvg martijnvg mentioned this issue Nov 28, 2024
Merged
martijnvg added a commit that referenced this issue Dec 6, 2024
@martijnvg
Add additional debug logging for indexing failure. (#117728)
b09f1d7 
This change adds a new IndexingOperationListener that logs debug logging for indexing failures.

Today `IndexShard` logger needs to be set to TRACE in order to see indexing failures. This logger is very verbose and logs many other aspects of indexing at trace level. This new logger is targeted just for seeing indexing failures.

Relates #117700
martijnvg added a commit to martijnvg/elasticsearch that referenced this issue Dec 6, 2024
@martijnvg
Add additional debug logging for indexing failure. (elastic#117728)
2e89f47 
This change adds a new IndexingOperationListener that logs debug logging for indexing failures.

Today `IndexShard` logger needs to be set to TRACE in order to see indexing failures. This logger is very verbose and logs many other aspects of indexing at trace level. This new logger is targeted just for seeing indexing failures.

Relates elastic#117700
elasticsearchmachine pushed a commit that referenced this issue Dec 6, 2024
@martijnvg
Add additional debug logging for indexing failure. (#117728) (#118144)
d08c26b 
This change adds a new IndexingOperationListener that logs debug logging for indexing failures.

Today `IndexShard` logger needs to be set to TRACE in order to see indexing failures. This logger is very verbose and logs many other aspects of indexing at trace level. This new logger is targeted just for seeing indexing failures.

Relates #117700
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@martijnvg martijnvg

Labels
:Distributed Indexing/Engine Anything around managing Lucene and the Translog in an open shard. >enhancement :StorageEngine/Logs You know, for Logs Team:Distributed Indexing Meta label for Distributed Indexing team Team:StorageEngine
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@martijnvg @elasticsearchmachine