Drop processor #801

dapowers87 · 2022-01-03T16:09:03Z

Description

This creates a Drop Processor to drop messages

Issues Resolved

Check List

New functionality includes testing.
New functionality has been documented.
- New functionality has javadoc added
Commits are signed with a real name per the DCO

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: David Powers <[email protected]>

…pper into DropProcessor

Signed-off-by: David Powers <[email protected]>

codecov-commenter · 2022-01-03T16:49:25Z

Codecov Report

Merging #801 (95d3350) into main (1183a7a) will increase coverage by 0.10%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##               main     #801      +/-   ##
============================================
+ Coverage     91.53%   91.64%   +0.10%     
- Complexity      637      640       +3     
============================================
  Files            75       75              
  Lines          1902     1902              
  Branches        161      161              
============================================
+ Hits           1741     1743       +2     
  Misses          121      121              
+ Partials         40       38       -2

Impacted Files	Coverage Δ
...com/amazon/dataprepper/pipeline/ProcessWorker.java	`91.42% <0.00%> (+5.71%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1183a7a...95d3350. Read the comment docs.

dlvenable · 2022-01-03T16:51:13Z

data-prepper-plugins/drop-processor/build.gradle

@@ -0,0 +1,29 @@
+/*


This is not the correct copyright header (unless you copied this source code from another project).

It should be:

/* * Copyright OpenSearch Contributors * SPDX-License-Identifier: Apache-2.0 */

dlvenable · 2022-01-03T16:51:27Z

...rop-processor/src/main/java/com/amazon/dataprepper/plugins/processor/drop/DropProcessor.java

@@ -0,0 +1,59 @@
+/*


Incorrect copyright header - see message above.

dlvenable · 2022-01-03T16:51:41Z

...rocessor/src/test/java/com/amazon/dataprepper/plugins/processor/drop/DropProcessorTests.java

@@ -0,0 +1,49 @@
+package com.amazon.dataprepper.plugins.processor.drop;


Please add copyright header.

dlvenable · 2022-01-03T16:52:28Z

...rop-processor/src/main/java/com/amazon/dataprepper/plugins/processor/drop/DropProcessor.java

+import java.util.LinkedList;
+import java.util.List;
+
+@SingleThread


This processor does not need the @SingleThread annotation.

dlvenable · 2022-01-03T16:58:09Z

...rop-processor/src/main/java/com/amazon/dataprepper/plugins/processor/drop/DropProcessor.java

+
+    @Override
+    public Collection<Record<Event>> doExecute(Collection<Record<Event>> records) {
+        List<Record<Event>> emptySet = new LinkedList<>();


You can return Collections.emptyList() here instead.

dlvenable · 2022-01-03T16:58:56Z

...rocessor/src/test/java/com/amazon/dataprepper/plugins/processor/drop/DropProcessorTests.java

+
+        final List<Record<Event>> droppedRecords = (List<Record<Event>>) dropProcessor.doExecute(Collections.singletonList(record));
+
+        Assert.assertEquals(0, droppedRecords.size());


Please use Hamcrest:

assertThat(droppedRecords.size(), equalTo(0));

Changed to Hamcrest

dlvenable · 2022-01-03T17:01:41Z

...rocessor/src/test/java/com/amazon/dataprepper/plugins/processor/drop/DropProcessorTests.java

+    private final String PLUGIN_NAME = "drop_events";
+
+    @Test
+    void doExecute() {


It would be good to have a test which also has more than one item in the input list.

Added the second test

dlvenable · 2022-01-03T17:02:30Z

settings.gradle

@@ -31,6 +31,7 @@ include 'data-prepper-plugins:otel-trace-source'
 include 'data-prepper-plugins:peer-forwarder'
 include 'data-prepper-plugins:blocking-buffer'
 include 'data-prepper-plugins:http-source'
+include 'data-prepper-plugins:drop-processor'


I think we should try to keep the names consistent. You named the plugin drop_events. So, I think this project should be drop-events-processor.

Renamed to drop-events-processor

cmanning09

Let's document this plugin with a README.md file.

Signed-off-by: David Powers <[email protected]>

...or/src/test/java/com/amazon/dataprepper/plugins/processor/drop/DropEventsProcessorTests.java

Signed-off-by: David Powers <[email protected]>

David Powers added 2 commits December 27, 2021 14:37

Create Drop Processor and a simple test

cdc18a9

Signed-off-by: David Powers <[email protected]>

Merge branch 'main' of https://github.com/opensearch-project/data-pre…

a9cccb3

…pper into DropProcessor

dapowers87 requested a review from a team as a code owner January 3, 2022 16:09

David Powers added 3 commits January 3, 2022 10:15

Remove unused import

f44387a

Signed-off-by: David Powers <[email protected]>

Fix imports

f9465aa

Signed-off-by: David Powers <[email protected]>

Correcting an overzealous IDE.

22eb2c8

Signed-off-by: David Powers <[email protected]>

dlvenable requested changes Jan 3, 2022

View reviewed changes

cmanning09 reviewed Jan 3, 2022

View reviewed changes

Add README; Correct copyright headers; minor logic tweaks;

415a8b4

Signed-off-by: David Powers <[email protected]>

dlvenable previously approved these changes Jan 3, 2022

View reviewed changes

sbayer55 suggested changes Jan 3, 2022

View reviewed changes

Renamed tests; Removed unnecessary casts

95d3350

Signed-off-by: David Powers <[email protected]>

dapowers87 dismissed dlvenable’s stale review via 95d3350 January 3, 2022 20:19

sbayer55 approved these changes Jan 3, 2022

View reviewed changes

cmanning09 approved these changes Jan 4, 2022

View reviewed changes

dapowers87 merged commit a59d2d2 into opensearch-project:main Jan 4, 2022

dapowers87 deleted the DropProcessor branch January 4, 2022 16:06

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Drop processor #801

Drop processor #801

dapowers87 commented Jan 3, 2022

codecov-commenter commented Jan 3, 2022 •

edited

Loading

dlvenable Jan 3, 2022

dapowers87 Jan 3, 2022

dlvenable Jan 3, 2022

dapowers87 Jan 3, 2022

dlvenable Jan 3, 2022

dapowers87 Jan 3, 2022

dlvenable Jan 3, 2022

dapowers87 Jan 3, 2022

dlvenable Jan 3, 2022

dapowers87 Jan 3, 2022

dlvenable Jan 3, 2022

dapowers87 Jan 3, 2022

dlvenable Jan 3, 2022

dapowers87 Jan 3, 2022

dlvenable Jan 3, 2022

dapowers87 Jan 3, 2022

cmanning09 left a comment

		@@ -0,0 +1,49 @@
		package com.amazon.dataprepper.plugins.processor.drop;


		final List<Record<Event>> droppedRecords = (List<Record<Event>>) dropProcessor.doExecute(Collections.singletonList(record));

		Assert.assertEquals(0, droppedRecords.size());

Drop processor #801

Drop processor #801

Conversation

dapowers87 commented Jan 3, 2022

Description

Issues Resolved

Check List

codecov-commenter commented Jan 3, 2022 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cmanning09 left a comment

Choose a reason for hiding this comment

codecov-commenter commented Jan 3, 2022 •

edited

Loading