From 3cea274242a000b7ecdd16ee2ec6118b46ad6d97 Mon Sep 17 00:00:00 2001 From: David Medina Date: Sun, 3 Nov 2024 20:55:16 +0100 Subject: [PATCH 1/6] added idempotent consumer pattern --- idempotent-consumer/README.md | 78 +++++++++++ idempotent-consumer/pom.xml | 126 ++++++++++++++++++ .../com/iluwatar/idempotentconsumer/App.java | 74 ++++++++++ .../InvalidNextStateException.java | 36 +++++ .../iluwatar/idempotentconsumer/Request.java | 60 +++++++++ .../RequestNotFoundException.java | 39 ++++++ .../idempotentconsumer/RequestRepository.java | 40 ++++++ .../idempotentconsumer/RequestService.java | 91 +++++++++++++ .../RequestStateMachine.java | 63 +++++++++ .../RequestServiceTests.java | 115 ++++++++++++++++ pom.xml | 1 + 11 files changed, 723 insertions(+) create mode 100644 idempotent-consumer/README.md create mode 100644 idempotent-consumer/pom.xml create mode 100644 idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/App.java create mode 100644 idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/InvalidNextStateException.java create mode 100644 idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/Request.java create mode 100644 idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/RequestNotFoundException.java create mode 100644 idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/RequestRepository.java create mode 100644 idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/RequestService.java create mode 100644 idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/RequestStateMachine.java create mode 100644 idempotent-consumer/src/test/java/com/iluwatar/idempotentconsumer/RequestServiceTests.java diff --git a/idempotent-consumer/README.md b/idempotent-consumer/README.md new file mode 100644 index 000000000000..de3d0233676c --- /dev/null +++ b/idempotent-consumer/README.md @@ -0,0 +1,78 @@ +--- +title: "Idempotent Consumer Pattern in Java: Ensuring Reliable Message Processing" +shortTitle: Idempotent Consumer +description: "Learn about the Idempotent Consumer pattern in Java. Discover how it ensures reliable and consistent message processing, even in cases of duplicate messages." +category: Messaging +language: en +tag: + - Messaging + - Fault tolerance + - Event-driven + - Reliability +--- + +## Also known as + +* Idempotency Pattern + +## Intent of Idempotent Consumer Pattern + +The Idempotent Consumer pattern is used to handle duplicate messages in distributed systems, ensuring that multiple processing of the same message does not cause undesired side effects. This pattern guarantees that the same message can be processed repeatedly with the same outcome, which is critical in ensuring reliable communication and data consistency in systems where message duplicates are possible. + +## Detailed Explanation of Idempotent Consumer Pattern with Real-World Examples + +### Real-world Example + +> In a payment processing system, ensuring that payment messages are idempotent prevents duplicate transactions. For example, if a user’s payment message is accidentally processed twice, the system should recognize the second message as a duplicate and prevent it from executing a second time. By storing unique identifiers for each processed message, such as a transaction ID, the system can skip any duplicate messages. This ensures that a user is not charged twice for the same transaction, maintaining system integrity and customer satisfaction. + +### In Plain Words + +> The Idempotent Consumer pattern prevents duplicate messages from causing unintended side effects by ensuring that processing the same message multiple times results in the same outcome. This makes message processing safe in distributed systems where duplicates may occur. + +### Wikipedia says + +> In computing, idempotence is the property of certain operations in mathematics and computer science whereby they can be applied multiple times without changing the result beyond the initial application. + +## Detailed Explanation with Real-World Example Diagram + +![Idempotent Consumer Diagram](./etc/idempotent_consumer.png) + +The diagram shows the flow in which the consumer processes the first message and skips the duplicate using the unique transaction ID stored in memory or a persistent storage. + +## When to Use the Idempotent Consumer Pattern + +The Idempotent Consumer pattern is particularly useful in scenarios: + +* When messages can be duplicated due to network retries or communication issues. +* In distributed systems where message ordering is not guaranteed, making deduplication necessary to avoid repeated processing. +* In financial or critical systems, where duplicate processing would have significant side effects. + +## Real-World Applications of Idempotent Consumer Pattern + +* Payment processing systems that avoid duplicate transactions. +* E-commerce systems to prevent multiple entries of the same order. +* Inventory management systems to prevent multiple entries when updating stock levels. + +## Benefits and Trade-offs of the Idempotent Consumer Pattern + +### Benefits + +* **Reliability**: Ensures that messages can be processed without unwanted side effects from duplicates. +* **Consistency**: Maintains data integrity by ensuring that duplicate messages do not cause redundant updates or actions. +* **Fault Tolerance**: Handles message retries gracefully, preventing them from causing errors. + +### Trade-offs + +* **State Management**: Requires storing processed message IDs, which can add memory overhead. +* **Complexity**: Implementing deduplication mechanisms can increase the complexity of the system. +* **Scalability**: In high-throughput systems, maintaining a large set of processed messages can impact performance and resource usage. + +## Related Patterns in Java + +* [Retry Pattern](https://java-design-patterns.com/patterns/retry/): Works well with the Idempotent Consumer pattern to handle failed messages. +* [Circuit Breaker Pattern](https://java-design-patterns.com/patterns/circuitbreaker/): Often used alongside idempotent consumers to prevent repeated failures from causing overload. + +## References and Credits + +* [Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions](https://amzn.to/4dznP2Y) +* [Designing Data-Intensive Applications](https://amzn.to/3UADv7Q) diff --git a/idempotent-consumer/pom.xml b/idempotent-consumer/pom.xml new file mode 100644 index 000000000000..8090f1fc0708 --- /dev/null +++ b/idempotent-consumer/pom.xml @@ -0,0 +1,126 @@ + + + + 4.0.0 + + com.iluwatar + java-design-patterns + 1.26.0-SNAPSHOT + + + idempotent-consumer + + + + org.springframework.boot + spring-boot-dependencies + pom + 3.2.3 + import + + + org.hibernate + hibernate-core + 6.4.4.Final + + + + + + + + + org.springframework.boot + spring-boot-starter-data-jpa + + + + + org.springframework.boot + spring-boot-starter-test + test + + + + + org.junit.jupiter + junit-jupiter-engine + test + + + + + org.mockito + mockito-core + test + + + + + com.h2database + h2 + runtime + + + + + + + + + + org.apache.maven.plugins + maven-assembly-plugin + + + package + + single + + + + jar-with-dependencies + + + + + com.iluwatar.idempotentconsumer.App + + + + + + + + + + + + diff --git a/idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/App.java b/idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/App.java new file mode 100644 index 000000000000..4fcc2801673f --- /dev/null +++ b/idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/App.java @@ -0,0 +1,74 @@ +/* + * This project is licensed under the MIT license. Module model-view-viewmodel is using ZK framework licensed under LGPL (see lgpl-3.0.txt). + * + * The MIT License + * Copyright © 2014-2022 Ilkka Seppälä + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.iluwatar.idempotentconsumer; + +import java.util.UUID; +import lombok.extern.slf4j.Slf4j; +import org.springframework.boot.CommandLineRunner; +import org.springframework.boot.SpringApplication; +import org.springframework.boot.autoconfigure.SpringBootApplication; +import org.springframework.context.annotation.Bean; + +/** + * The main entry point for the idempotent-consumer application. + * This application demonstrates the use of the Idempotent Consumer + * pattern which ensures that a message is processed exactly once + * in scenarios where the same message can be delivered multiple times. + * + * @see Idempotence (Wikipedia) + * @see Idempotent Consumer Pattern (Apache Camel) + */ +@SpringBootApplication +@Slf4j +public class App { + public static void main(String[] args) { + SpringApplication.run(App.class, args); + } + /** + * The starting point of the CommandLineRunner + * where the main program is run. + * + * @param requestService idempotent request service + * @param requestRepository request jpa repository + */ + @Bean + public CommandLineRunner run(RequestService requestService, RequestRepository requestRepository) { + return args -> { + Request req = requestService.create(UUID.randomUUID()); + requestService.create(req.getUuid()); + requestService.create(req.getUuid()); + LOGGER.info("Nb of requests : {}", requestRepository.count()); // 1, processRequest is idempotent + req = requestService.start(req.getUuid()); + try { + req = requestService.start(req.getUuid()); + } catch (InvalidNextStateException ex) { + LOGGER.error("Cannot start request twice!"); + } + req = requestService.complete(req.getUuid()); + LOGGER.info("Request: {}", req); + }; + } +} + diff --git a/idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/InvalidNextStateException.java b/idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/InvalidNextStateException.java new file mode 100644 index 000000000000..c29e913aa33f --- /dev/null +++ b/idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/InvalidNextStateException.java @@ -0,0 +1,36 @@ +/* + * This project is licensed under the MIT license. Module model-view-viewmodel is using ZK framework licensed under LGPL (see lgpl-3.0.txt). + * + * The MIT License + * Copyright © 2014-2022 Ilkka Seppälä + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.iluwatar.idempotentconsumer; + +/** + * This exception is thrown when an invalid transition is attempted in the Statemachine + * for the request status. This can occur when attempting to move to a state that is not valid + * from the current state. + */ +public class InvalidNextStateException extends RuntimeException { + public InvalidNextStateException(String s) { + super(s); + } +} diff --git a/idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/Request.java b/idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/Request.java new file mode 100644 index 000000000000..484b79550823 --- /dev/null +++ b/idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/Request.java @@ -0,0 +1,60 @@ +/* + * This project is licensed under the MIT license. Module model-view-viewmodel is using ZK framework licensed under LGPL (see lgpl-3.0.txt). + * + * The MIT License + * Copyright © 2014-2022 Ilkka Seppälä + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.iluwatar.idempotentconsumer; + +import jakarta.persistence.Entity; +import jakarta.persistence.Id; +import java.util.UUID; +import lombok.Data; +import lombok.NoArgsConstructor; + +/** + * The {@code Request} class represents a request with a unique UUID and a status. + * The status of a request can be one of four values: PENDING, STARTED, COMPLETED, or INERROR. + */ +@Entity +@NoArgsConstructor +@Data +public class Request { + enum Status { + PENDING, + STARTED, + COMPLETED, + INERROR + } + + @Id + private UUID uuid; + private Status status; + + public Request(UUID uuid) { + this(uuid, Status.PENDING); + } + + public Request(UUID uuid, Status status) { + this.uuid = uuid; + this.status = status; + } +} diff --git a/idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/RequestNotFoundException.java b/idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/RequestNotFoundException.java new file mode 100644 index 000000000000..5294298d65f2 --- /dev/null +++ b/idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/RequestNotFoundException.java @@ -0,0 +1,39 @@ +/* + * This project is licensed under the MIT license. Module model-view-viewmodel is using ZK framework licensed under LGPL (see lgpl-3.0.txt). + * + * The MIT License + * Copyright © 2014-2022 Ilkka Seppälä + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.iluwatar.idempotentconsumer; + +import java.util.UUID; + +/** + * This class extends the RuntimeException class to handle scenarios where a Request is not found. + * It is intended to be used where you would like to have a custom exception that signals that a requested object or action + * was not found in the system, based on the UUID of the request. + * + */ +public class RequestNotFoundException extends RuntimeException { + RequestNotFoundException(UUID uuid) { + super(String.format("Request %s not found", uuid)); + } +} diff --git a/idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/RequestRepository.java b/idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/RequestRepository.java new file mode 100644 index 000000000000..0d8d3744b3dd --- /dev/null +++ b/idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/RequestRepository.java @@ -0,0 +1,40 @@ +/* + * This project is licensed under the MIT license. Module model-view-viewmodel is using ZK framework licensed under LGPL (see lgpl-3.0.txt). + * + * The MIT License + * Copyright © 2014-2022 Ilkka Seppälä + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.iluwatar.idempotentconsumer; + +import java.util.UUID; +import org.springframework.data.jpa.repository.JpaRepository; +import org.springframework.stereotype.Repository; + +/** + * This is a repository interface for the "Request" entity. It extends the JpaRepository interface from Spring Data JPA. + * JpaRepository comes with many operations out of the box, including standard CRUD operations. + * With JpaRepository, we are also able to leverage the power of Spring Data's query methods. + * The UUID parameter in JpaRepository refers to the type of the ID in the "Request" entity. + * + */ +@Repository +public interface RequestRepository extends JpaRepository { +} diff --git a/idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/RequestService.java b/idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/RequestService.java new file mode 100644 index 000000000000..c3cc7950b5f8 --- /dev/null +++ b/idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/RequestService.java @@ -0,0 +1,91 @@ +/* + * This project is licensed under the MIT license. Module model-view-viewmodel is using ZK framework licensed under LGPL (see lgpl-3.0.txt). + * + * The MIT License + * Copyright © 2014-2022 Ilkka Seppälä + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.iluwatar.idempotentconsumer; + +import java.util.Optional; +import java.util.UUID; +import org.springframework.stereotype.Service; + +/** + * This service is responsible for handling request operations including + * creation, start, and completion of requests. + */ +@Service +public class RequestService { + RequestRepository requestRepository; + RequestStateMachine requestStateMachine; + + public RequestService(RequestRepository requestRepository, + RequestStateMachine requestStateMachine) { + this.requestRepository = requestRepository; + this.requestStateMachine = requestStateMachine; + } + + /** + * Creates a new Request or returns an existing one by it's UUID. + * This operation is idempotent: performing it once or several times + * successively leads to an equivalent result. + * + * @param uuid The unique identifier for the Request. + * @return Return existing Request or save and return a new Request. + */ + public Request create(UUID uuid) { + Optional optReq = requestRepository.findById(UUID.randomUUID()); + if (!optReq.isEmpty()) { + return optReq.get(); + } + return requestRepository.save(new Request(uuid)); + } + + /** + * Starts the Request assigned with the given UUID. + * + * @param uuid The unique identifier for the Request. + * @return The started Request. + * @throws RequestNotFoundException if a Request with the given UUID is not found. + */ + public Request start(UUID uuid) { + Optional optReq = requestRepository.findById(uuid); + if (optReq.isEmpty()) { + throw new RequestNotFoundException(uuid); + } + return requestRepository.save(requestStateMachine.next(optReq.get(), Request.Status.STARTED)); + } + + /** + * Complete the Request assigned with the given UUID. + * + * @param uuid The unique identifier for the Request. + * @return The completed Request. + * @throws RequestNotFoundException if a Request with the given UUID is not found. + */ + public Request complete(UUID uuid) { + Optional optReq = requestRepository.findById(uuid); + if (optReq.isEmpty()) { + throw new RequestNotFoundException(uuid); + } + return requestRepository.save(requestStateMachine.next(optReq.get(), Request.Status.COMPLETED)); + } +} diff --git a/idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/RequestStateMachine.java b/idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/RequestStateMachine.java new file mode 100644 index 000000000000..5861276ca033 --- /dev/null +++ b/idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/RequestStateMachine.java @@ -0,0 +1,63 @@ +/* + * This project is licensed under the MIT license. Module model-view-viewmodel is using ZK framework licensed under LGPL (see lgpl-3.0.txt). + * + * The MIT License + * Copyright © 2014-2022 Ilkka Seppälä + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.iluwatar.idempotentconsumer; + +import org.springframework.stereotype.Component; + +/** + * This class represents a state machine for managing request transitions. + * It supports transitions to the statuses: PENDING, STARTED, and COMPLETED. + */ +@Component +public class RequestStateMachine { + + /** + * Provides the next possible state of the request based on the current and next status. + * + * @param req The actual request object. This object MUST NOT be null and SHOULD have a valid status. + * @param nextStatus Represents the next status that the request could transition to. MUST NOT be null. + * @return A new Request object with updated status if the transition is valid. + * @throws InvalidNextStateException If an invalid state transition is attempted. + */ + public Request next(Request req, Request.Status nextStatus) { + String transitionStr = String.format("Transition: %s -> %s", req.getStatus(), nextStatus); + switch (nextStatus) { + case PENDING -> throw new InvalidNextStateException(transitionStr); + case STARTED -> { + if (Request.Status.PENDING.equals(req.getStatus())) { + return new Request(req.getUuid(), Request.Status.STARTED); + } + throw new InvalidNextStateException(transitionStr); + } + case COMPLETED -> { + if (Request.Status.STARTED.equals(req.getStatus())) { + return new Request(req.getUuid(), Request.Status.COMPLETED); + } + throw new InvalidNextStateException(transitionStr); + } + default -> throw new InvalidNextStateException(transitionStr); + } + } +} diff --git a/idempotent-consumer/src/test/java/com/iluwatar/idempotentconsumer/RequestServiceTests.java b/idempotent-consumer/src/test/java/com/iluwatar/idempotentconsumer/RequestServiceTests.java new file mode 100644 index 000000000000..20b4d1883e75 --- /dev/null +++ b/idempotent-consumer/src/test/java/com/iluwatar/idempotentconsumer/RequestServiceTests.java @@ -0,0 +1,115 @@ +/* + * This project is licensed under the MIT license. Module model-view-viewmodel is using ZK framework licensed under LGPL (see lgpl-3.0.txt). + * + * The MIT License + * Copyright © 2014-2022 Ilkka Seppälä + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ +package com.iluwatar.idempotentconsumer; + +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.ExtendWith; +import org.mockito.Mock; +import org.mockito.junit.jupiter.MockitoExtension; + +import java.util.Optional; +import java.util.UUID; + +import static org.junit.jupiter.api.Assertions.*; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.Mockito.times; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.when; + +@ExtendWith(MockitoExtension.class) +class RequestServiceTests { + private RequestService requestService; + @Mock + private RequestRepository requestRepository; + private RequestStateMachine requestStateMachine; + @BeforeEach + void setUp() { + requestStateMachine = new RequestStateMachine(); + requestService = new RequestService(requestRepository, requestStateMachine); + } + + @Test + void createRequest_whenNotExists() { + UUID uuid = UUID.randomUUID(); + Request request = new Request(uuid); + when(requestRepository.findById(any())).thenReturn(Optional.empty()); + when(requestRepository.save(request)).thenReturn(request); + assertEquals(request, requestService.create(uuid)); + verify(requestRepository, times(1)).findById(uuid); + verify(requestRepository, times(1)).save(any()); + } + @Test + void createRequest_whenExists() { + UUID uuid = UUID.randomUUID(); + Request request = new Request(uuid); + when(requestRepository.findById(any())).thenReturn(Optional.of(request)); + assertEquals(request, requestService.create(uuid)); + verify(requestRepository, times(1)).findById(uuid); + verify(requestRepository, times(0)).save(any()); + } + + @Test + void startRequest_whenNotExists_shouldThrowError() { + UUID uuid = UUID.randomUUID(); + when(requestRepository.findById(any())).thenReturn(Optional.empty()); + assertThrows(RequestNotFoundException.class, ()->requestService.start(uuid)); + verify(requestRepository, times(1)).findById(uuid); + verify(requestRepository, times(0)).save(any()); + } + + @Test + void startRequest_whenIsPending() { + UUID uuid = UUID.randomUUID(); + Request request = new Request(uuid); + Request startedEntity = new Request(uuid, Request.Status.STARTED); + when(requestRepository.findById(any())).thenReturn(Optional.of(request)); + when(requestRepository.save(any())).thenReturn(startedEntity); + assertEquals(startedEntity, requestService.start(uuid)); + verify(requestRepository, times(1)).findById(uuid); + verify(requestRepository, times(1)).save(startedEntity); + } + + @Test + void completeRequest_whenStarted() { + UUID uuid = UUID.randomUUID(); + Request request = new Request(uuid, Request.Status.STARTED); + Request completedEntity = new Request(uuid, Request.Status.COMPLETED); + when(requestRepository.findById(any())).thenReturn(Optional.of(request)); + when(requestRepository.save(any())).thenReturn(completedEntity); + assertEquals(completedEntity, requestService.complete(uuid)); + verify(requestRepository, times(1)).findById(uuid); + verify(requestRepository, times(1)).save(completedEntity); + } + @Test + void completeRequest_whenNotInprogress() { + UUID uuid = UUID.randomUUID(); + Request request = new Request(uuid); + when(requestRepository.findById(any())).thenReturn(Optional.of(request)); + assertThrows(InvalidNextStateException.class, () -> requestService.complete(uuid)); + verify(requestRepository, times(1)).findById(uuid); + verify(requestRepository, times(0)).save(any()); + } +} \ No newline at end of file diff --git a/pom.xml b/pom.xml index 743c8ea30122..41a46a6c3038 100644 --- a/pom.xml +++ b/pom.xml @@ -217,6 +217,7 @@ virtual-proxy function-composition microservices-distributed-tracing + idempotent-consumer From f7612e73d2a30bfbb8f41d297819e7dee837320e Mon Sep 17 00:00:00 2001 From: David Medina Date: Sun, 3 Nov 2024 21:01:14 +0100 Subject: [PATCH 2/6] updated doc --- idempotent-consumer/README.md | 6 ------ 1 file changed, 6 deletions(-) diff --git a/idempotent-consumer/README.md b/idempotent-consumer/README.md index de3d0233676c..312fd4a27bf2 100644 --- a/idempotent-consumer/README.md +++ b/idempotent-consumer/README.md @@ -33,12 +33,6 @@ The Idempotent Consumer pattern is used to handle duplicate messages in distribu > In computing, idempotence is the property of certain operations in mathematics and computer science whereby they can be applied multiple times without changing the result beyond the initial application. -## Detailed Explanation with Real-World Example Diagram - -![Idempotent Consumer Diagram](./etc/idempotent_consumer.png) - -The diagram shows the flow in which the consumer processes the first message and skips the duplicate using the unique transaction ID stored in memory or a persistent storage. - ## When to Use the Idempotent Consumer Pattern The Idempotent Consumer pattern is particularly useful in scenarios: From d313ad38565a22f140452e291611580adebd4f0c Mon Sep 17 00:00:00 2001 From: David Medina Date: Mon, 4 Nov 2024 00:26:08 +0100 Subject: [PATCH 3/6] fixed bug in RequestService --- .../java/com/iluwatar/idempotentconsumer/RequestService.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/RequestService.java b/idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/RequestService.java index c3cc7950b5f8..066e52e66917 100644 --- a/idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/RequestService.java +++ b/idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/RequestService.java @@ -52,7 +52,7 @@ public RequestService(RequestRepository requestRepository, * @return Return existing Request or save and return a new Request. */ public Request create(UUID uuid) { - Optional optReq = requestRepository.findById(UUID.randomUUID()); + Optional optReq = requestRepository.findById(uuid); if (!optReq.isEmpty()) { return optReq.get(); } From 9a1fa5c55cdb7d0ac1657ca6de81897f7c9ebb90 Mon Sep 17 00:00:00 2001 From: David Medina Date: Mon, 4 Nov 2024 01:13:08 +0100 Subject: [PATCH 4/6] add test converage --- .../RequestServiceTests.java | 38 ++++++++++++++----- 1 file changed, 29 insertions(+), 9 deletions(-) diff --git a/idempotent-consumer/src/test/java/com/iluwatar/idempotentconsumer/RequestServiceTests.java b/idempotent-consumer/src/test/java/com/iluwatar/idempotentconsumer/RequestServiceTests.java index 20b4d1883e75..1d5eebd34bcd 100644 --- a/idempotent-consumer/src/test/java/com/iluwatar/idempotentconsumer/RequestServiceTests.java +++ b/idempotent-consumer/src/test/java/com/iluwatar/idempotentconsumer/RequestServiceTests.java @@ -24,21 +24,21 @@ */ package com.iluwatar.idempotentconsumer; +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertThrows; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.Mockito.times; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.when; + +import java.util.Optional; +import java.util.UUID; import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.Test; import org.junit.jupiter.api.extension.ExtendWith; import org.mockito.Mock; import org.mockito.junit.jupiter.MockitoExtension; -import java.util.Optional; -import java.util.UUID; - -import static org.junit.jupiter.api.Assertions.*; -import static org.mockito.ArgumentMatchers.any; -import static org.mockito.Mockito.times; -import static org.mockito.Mockito.verify; -import static org.mockito.Mockito.when; - @ExtendWith(MockitoExtension.class) class RequestServiceTests { private RequestService requestService; @@ -92,6 +92,26 @@ void startRequest_whenIsPending() { verify(requestRepository, times(1)).save(startedEntity); } + @Test + void startRequest_whenIsStarted_shouldThrowError() { + UUID uuid = UUID.randomUUID(); + Request requestStarted = new Request(uuid, Request.Status.STARTED); + when(requestRepository.findById(any())).thenReturn(Optional.of(requestStarted)); + assertThrows(InvalidNextStateException.class, ()->requestService.start(uuid)); + verify(requestRepository, times(1)).findById(uuid); + verify(requestRepository, times(0)).save(any()); + } + + @Test + void startRequest_whenIsCompleted_shouldThrowError() { + UUID uuid = UUID.randomUUID(); + Request requestStarted = new Request(uuid, Request.Status.COMPLETED); + when(requestRepository.findById(any())).thenReturn(Optional.of(requestStarted)); + assertThrows(InvalidNextStateException.class, ()->requestService.start(uuid)); + verify(requestRepository, times(1)).findById(uuid); + verify(requestRepository, times(0)).save(any()); + } + @Test void completeRequest_whenStarted() { UUID uuid = UUID.randomUUID(); From dc3200d15054c1efbcb0855f94556284b02b6a71 Mon Sep 17 00:00:00 2001 From: David Medina Date: Mon, 4 Nov 2024 22:30:45 +0100 Subject: [PATCH 5/6] Add test converage to state machine --- .../iluwatar/idempotentconsumer/Request.java | 3 +- .../iluwatar/idempotentconsumer/AppTest.java | 46 +++++++++++++++ .../RequestStateMachineTests.java | 59 +++++++++++++++++++ 3 files changed, 106 insertions(+), 2 deletions(-) create mode 100644 idempotent-consumer/src/test/java/com/iluwatar/idempotentconsumer/AppTest.java create mode 100644 idempotent-consumer/src/test/java/com/iluwatar/idempotentconsumer/RequestStateMachineTests.java diff --git a/idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/Request.java b/idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/Request.java index 484b79550823..29f5d6fba375 100644 --- a/idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/Request.java +++ b/idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/Request.java @@ -41,8 +41,7 @@ public class Request { enum Status { PENDING, STARTED, - COMPLETED, - INERROR + COMPLETED } @Id diff --git a/idempotent-consumer/src/test/java/com/iluwatar/idempotentconsumer/AppTest.java b/idempotent-consumer/src/test/java/com/iluwatar/idempotentconsumer/AppTest.java new file mode 100644 index 000000000000..e161b2edc30d --- /dev/null +++ b/idempotent-consumer/src/test/java/com/iluwatar/idempotentconsumer/AppTest.java @@ -0,0 +1,46 @@ +package com.iluwatar.idempotentconsumer; + +import org.junit.jupiter.api.Test; +import org.mockito.Mockito; +import org.springframework.boot.CommandLineRunner; + +import java.util.UUID; + +import static org.junit.jupiter.api.Assertions.assertDoesNotThrow; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.Mockito.times; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.when; + +/** + * Application test + */ +class AppTest { + + @Test + void main() { + assertDoesNotThrow(() -> App.main(new String[] {})); + } + + @Test + void run() throws Exception { + RequestService requestService = Mockito.mock(RequestService.class); + RequestRepository requestRepository = Mockito.mock(RequestRepository.class); + UUID uuid = UUID.randomUUID(); + Request requestPending = new Request(uuid); + Request requestStarted = new Request(uuid, Request.Status.STARTED); + Request requestCompleted = new Request(uuid, Request.Status.COMPLETED); + when(requestService.create(any())).thenReturn(requestPending); + when(requestService.start(any())).thenReturn(requestStarted); + when(requestService.complete(any())).thenReturn(requestCompleted); + + CommandLineRunner runner = new App().run(requestService, requestRepository); + + runner.run(); + + verify(requestService, times(3)).create(any()); + verify(requestService, times(2)).start(any()); + verify(requestService, times(1)).complete(any()); + verify(requestRepository, times(1)).count(); + } +} \ No newline at end of file diff --git a/idempotent-consumer/src/test/java/com/iluwatar/idempotentconsumer/RequestStateMachineTests.java b/idempotent-consumer/src/test/java/com/iluwatar/idempotentconsumer/RequestStateMachineTests.java new file mode 100644 index 000000000000..af779648c8a3 --- /dev/null +++ b/idempotent-consumer/src/test/java/com/iluwatar/idempotentconsumer/RequestStateMachineTests.java @@ -0,0 +1,59 @@ +package com.iluwatar.idempotentconsumer; + +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import java.util.UUID; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertThrows; + +class RequestStateMachineTests { + private RequestStateMachine requestStateMachine; + + @BeforeEach + public void setUp() { + requestStateMachine = new RequestStateMachine(); + } + + @Test + void transitionPendingToStarted() { + Request startedRequest = requestStateMachine.next(new Request(UUID.randomUUID(), Request.Status.PENDING), + Request.Status.STARTED); + assertEquals(Request.Status.STARTED, startedRequest.getStatus()); + } + + @Test + void transitionAnyToPending_shouldThrowError() { + assertThrows(InvalidNextStateException.class, + () -> requestStateMachine.next(new Request(UUID.randomUUID(), Request.Status.PENDING), + Request.Status.PENDING)); + assertThrows(InvalidNextStateException.class, + () -> requestStateMachine.next(new Request(UUID.randomUUID(), Request.Status.STARTED), + Request.Status.PENDING)); + assertThrows(InvalidNextStateException.class, + () -> requestStateMachine.next(new Request(UUID.randomUUID(), Request.Status.COMPLETED), + Request.Status.PENDING)); + } + + @Test + void transitionCompletedToAny_shouldThrowError() { + assertThrows(InvalidNextStateException.class, + () -> requestStateMachine.next(new Request(UUID.randomUUID(), Request.Status.COMPLETED), + Request.Status.PENDING)); + assertThrows(InvalidNextStateException.class, + () -> requestStateMachine.next(new Request(UUID.randomUUID(), Request.Status.COMPLETED), + Request.Status.STARTED)); + assertThrows(InvalidNextStateException.class, + () -> requestStateMachine.next(new Request(UUID.randomUUID(), Request.Status.COMPLETED), + Request.Status.COMPLETED)); + } + + @Test + void transitionStartedToCompleted() { + Request completedRequest = requestStateMachine.next(new Request(UUID.randomUUID(), Request.Status.STARTED), + Request.Status.COMPLETED); + assertEquals(Request.Status.COMPLETED, completedRequest.getStatus()); + } + + +} \ No newline at end of file From 9cc98d2ac0d0b7bb14f7fca548711096b7561512 Mon Sep 17 00:00:00 2001 From: David Medina Date: Sun, 10 Nov 2024 19:07:30 +0100 Subject: [PATCH 6/6] renamed module and added code example --- idempotent-consumer/README.md | 72 -------- microservices-idempotent-consumer/README.md | 161 ++++++++++++++++++ .../pom.xml | 2 +- .../com/iluwatar/idempotentconsumer/App.java | 0 .../InvalidNextStateException.java | 0 .../iluwatar/idempotentconsumer/Request.java | 0 .../RequestNotFoundException.java | 0 .../idempotentconsumer/RequestRepository.java | 0 .../idempotentconsumer/RequestService.java | 0 .../RequestStateMachine.java | 0 .../iluwatar/idempotentconsumer/AppTest.java | 0 .../RequestServiceTests.java | 0 .../RequestStateMachineTests.java | 0 pom.xml | 2 +- 14 files changed, 163 insertions(+), 74 deletions(-) delete mode 100644 idempotent-consumer/README.md create mode 100644 microservices-idempotent-consumer/README.md rename {idempotent-consumer => microservices-idempotent-consumer}/pom.xml (98%) rename {idempotent-consumer => microservices-idempotent-consumer}/src/main/java/com/iluwatar/idempotentconsumer/App.java (100%) rename {idempotent-consumer => microservices-idempotent-consumer}/src/main/java/com/iluwatar/idempotentconsumer/InvalidNextStateException.java (100%) rename {idempotent-consumer => microservices-idempotent-consumer}/src/main/java/com/iluwatar/idempotentconsumer/Request.java (100%) rename {idempotent-consumer => microservices-idempotent-consumer}/src/main/java/com/iluwatar/idempotentconsumer/RequestNotFoundException.java (100%) rename {idempotent-consumer => microservices-idempotent-consumer}/src/main/java/com/iluwatar/idempotentconsumer/RequestRepository.java (100%) rename {idempotent-consumer => microservices-idempotent-consumer}/src/main/java/com/iluwatar/idempotentconsumer/RequestService.java (100%) rename {idempotent-consumer => microservices-idempotent-consumer}/src/main/java/com/iluwatar/idempotentconsumer/RequestStateMachine.java (100%) rename {idempotent-consumer => microservices-idempotent-consumer}/src/test/java/com/iluwatar/idempotentconsumer/AppTest.java (100%) rename {idempotent-consumer => microservices-idempotent-consumer}/src/test/java/com/iluwatar/idempotentconsumer/RequestServiceTests.java (100%) rename {idempotent-consumer => microservices-idempotent-consumer}/src/test/java/com/iluwatar/idempotentconsumer/RequestStateMachineTests.java (100%) diff --git a/idempotent-consumer/README.md b/idempotent-consumer/README.md deleted file mode 100644 index 312fd4a27bf2..000000000000 --- a/idempotent-consumer/README.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -title: "Idempotent Consumer Pattern in Java: Ensuring Reliable Message Processing" -shortTitle: Idempotent Consumer -description: "Learn about the Idempotent Consumer pattern in Java. Discover how it ensures reliable and consistent message processing, even in cases of duplicate messages." -category: Messaging -language: en -tag: - - Messaging - - Fault tolerance - - Event-driven - - Reliability ---- - -## Also known as - -* Idempotency Pattern - -## Intent of Idempotent Consumer Pattern - -The Idempotent Consumer pattern is used to handle duplicate messages in distributed systems, ensuring that multiple processing of the same message does not cause undesired side effects. This pattern guarantees that the same message can be processed repeatedly with the same outcome, which is critical in ensuring reliable communication and data consistency in systems where message duplicates are possible. - -## Detailed Explanation of Idempotent Consumer Pattern with Real-World Examples - -### Real-world Example - -> In a payment processing system, ensuring that payment messages are idempotent prevents duplicate transactions. For example, if a user’s payment message is accidentally processed twice, the system should recognize the second message as a duplicate and prevent it from executing a second time. By storing unique identifiers for each processed message, such as a transaction ID, the system can skip any duplicate messages. This ensures that a user is not charged twice for the same transaction, maintaining system integrity and customer satisfaction. - -### In Plain Words - -> The Idempotent Consumer pattern prevents duplicate messages from causing unintended side effects by ensuring that processing the same message multiple times results in the same outcome. This makes message processing safe in distributed systems where duplicates may occur. - -### Wikipedia says - -> In computing, idempotence is the property of certain operations in mathematics and computer science whereby they can be applied multiple times without changing the result beyond the initial application. - -## When to Use the Idempotent Consumer Pattern - -The Idempotent Consumer pattern is particularly useful in scenarios: - -* When messages can be duplicated due to network retries or communication issues. -* In distributed systems where message ordering is not guaranteed, making deduplication necessary to avoid repeated processing. -* In financial or critical systems, where duplicate processing would have significant side effects. - -## Real-World Applications of Idempotent Consumer Pattern - -* Payment processing systems that avoid duplicate transactions. -* E-commerce systems to prevent multiple entries of the same order. -* Inventory management systems to prevent multiple entries when updating stock levels. - -## Benefits and Trade-offs of the Idempotent Consumer Pattern - -### Benefits - -* **Reliability**: Ensures that messages can be processed without unwanted side effects from duplicates. -* **Consistency**: Maintains data integrity by ensuring that duplicate messages do not cause redundant updates or actions. -* **Fault Tolerance**: Handles message retries gracefully, preventing them from causing errors. - -### Trade-offs - -* **State Management**: Requires storing processed message IDs, which can add memory overhead. -* **Complexity**: Implementing deduplication mechanisms can increase the complexity of the system. -* **Scalability**: In high-throughput systems, maintaining a large set of processed messages can impact performance and resource usage. - -## Related Patterns in Java - -* [Retry Pattern](https://java-design-patterns.com/patterns/retry/): Works well with the Idempotent Consumer pattern to handle failed messages. -* [Circuit Breaker Pattern](https://java-design-patterns.com/patterns/circuitbreaker/): Often used alongside idempotent consumers to prevent repeated failures from causing overload. - -## References and Credits - -* [Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions](https://amzn.to/4dznP2Y) -* [Designing Data-Intensive Applications](https://amzn.to/3UADv7Q) diff --git a/microservices-idempotent-consumer/README.md b/microservices-idempotent-consumer/README.md new file mode 100644 index 000000000000..794eb6e644aa --- /dev/null +++ b/microservices-idempotent-consumer/README.md @@ -0,0 +1,161 @@ +--- +title: "Idempotent Consumer Pattern in Java: Ensuring Reliable Message Processing" +shortTitle: Idempotent Consumer +description: "Learn about the Idempotent Consumer pattern in Java. Discover how it ensures reliable and consistent message processing, even in cases of duplicate messages." +category: Messaging +language: en +tag: + - Messaging + - Fault tolerance + - Event-driven + - Reliability +--- + +## Also known as + +* Idempotency Pattern + +## Intent of Idempotent Consumer Pattern + +The Idempotent Consumer pattern is used to handle duplicate messages in distributed systems, ensuring that multiple processing of the same message does not cause undesired side effects. This pattern guarantees that the same message can be processed repeatedly with the same outcome, which is critical in ensuring reliable communication and data consistency in systems where message duplicates are possible. + +## Detailed Explanation of Idempotent Consumer Pattern with Real-World Examples + +### Real-world Example + +> In a payment processing system, ensuring that payment messages are idempotent prevents duplicate transactions. For example, if a user’s payment message is accidentally processed twice, the system should recognize the second message as a duplicate and prevent it from executing a second time. By storing unique identifiers for each processed message, such as a transaction ID, the system can skip any duplicate messages. This ensures that a user is not charged twice for the same transaction, maintaining system integrity and customer satisfaction. + +### In Plain Words + +> The Idempotent Consumer pattern prevents duplicate messages from causing unintended side effects by ensuring that processing the same message multiple times results in the same outcome. This makes message processing safe in distributed systems where duplicates may occur. + +### Wikipedia says + +> In computing, idempotence is the property of certain operations in mathematics and computer science whereby they can be applied multiple times without changing the result beyond the initial application. + +## When to Use the Idempotent Consumer Pattern + +The Idempotent Consumer pattern is particularly useful in scenarios: + +* When messages can be duplicated due to network retries or communication issues. +* In distributed systems where message ordering is not guaranteed, making deduplication necessary to avoid repeated processing. +* In financial or critical systems, where duplicate processing would have significant side effects. + +## Real-World Applications of Idempotent Consumer Pattern + +* Payment processing systems that avoid duplicate transactions. +* E-commerce systems to prevent multiple entries of the same order. +* Inventory management systems to prevent multiple entries when updating stock levels. + +## Programmatic example of Idempotent Consumer Pattern +In this Java example, we have an idempotent service that offers functionality to create and update (start, complete, etc.) orders. The service ensures that the **create order** operation is idempotent, meaning that performing it multiple times with the same order ID will lead to the same result without creating duplicates. For state transitions (such as starting or completing an order), the service enforces valid state changes and throws exceptions if an invalid transition is attempted. The state machine governs the valid order status transitions, ensuring that statuses progress in a defined and consistent sequence. +### RequestService - Managing Idempotent Order Operations +The `RequestService` class is responsible for handling the creation and state transitions of orders. The `create` method is designed to be idempotent, ensuring that it either returns an existing order or creates a new one without any side effects if invoked multiple times with the same order ID. +```java +public class RequestService { + // Idempotent: ensures that the same request is returned if it already exists + public Request create(UUID uuid) { + Optional optReq = requestRepository.findById(uuid); + if (!optReq.isEmpty()) { + return optReq.get(); // Return existing request + } + return requestRepository.save(new Request(uuid)); // Save and return new request + } + + public Request start(UUID uuid) { + Optional optReq = requestRepository.findById(uuid); + if (optReq.isEmpty()) { + throw new RequestNotFoundException(uuid); + } + return requestRepository.save(requestStateMachine.next(optReq.get(), Request.Status.STARTED)); + } + + public Request complete(UUID uuid) { + Optional optReq = requestRepository.findById(uuid); + if (optReq.isEmpty()) { + throw new RequestNotFoundException(uuid); + } + return requestRepository.save(requestStateMachine.next(optReq.get(), Request.Status.COMPLETED)); + } +} +``` +### RequestStateMachine - Managing Order Transitions +The `RequestStateMachine` ensures that state transitions occur in a valid order. It handles the progression of an order's status, ensuring the correct sequence (e.g., from `PENDING` to `STARTED` to `COMPLETED`). +```java +public class RequestStateMachine { + + public Request next(Request req, Request.Status nextStatus) { + String transitionStr = String.format("Transition: %s -> %s", req.getStatus(), nextStatus); + switch (nextStatus) { + case PENDING -> throw new InvalidNextStateException(transitionStr); + case STARTED -> { + if (Request.Status.PENDING.equals(req.getStatus())) { + return new Request(req.getUuid(), Request.Status.STARTED); // Valid transition + } + throw new InvalidNextStateException(transitionStr); // Invalid transition + } + case COMPLETED -> { + if (Request.Status.STARTED.equals(req.getStatus())) { + return new Request(req.getUuid(), Request.Status.COMPLETED); // Valid transition + } + throw new InvalidNextStateException(transitionStr); // Invalid transition + } + default -> throw new InvalidNextStateException(transitionStr); // Invalid status + } + } +} +``` +### Main Application - Running the Idempotent Consumer Example + +In the main application, we demonstrate how the `RequestService` can be used to perform idempotent operations. Whether the order creation or state transition is invoked once or multiple times, the result is consistent and does not produce unexpected side effects. + +```java +Request req = requestService.create(UUID.randomUUID()); +// Try creating the same Request again with the same UUID (idempotent operation) +requestService.create(req.getUuid()); +// Again, try creating the same Request (idempotent operation, no new Request should be created) +requestService.create(req.getUuid()); +LOGGER.info("Nb of requests : {}", requestRepository.count()); // 1, processRequest is idempotent +// Attempt to start the Request (the first valid transition) +req = requestService.start(req.getUuid()); +// Try to start the Request again, which should throw an exception since it's already started +try { + req = requestService.start(req.getUuid()); +} catch (InvalidNextStateException ex) { + // Log an error message when trying to start a request twice + LOGGER.error("Cannot start request twice!"); +} +// Complete the Request (valid transition from STARTED to COMPLETED) +req = requestService.complete(req.getUuid()); +// Log the final status of the Request to confirm it's been completed +LOGGER.info("Request: {}", req); +``` +Program output: +``` +19:01:54.382 INFO [main] com.iluwatar.idempotentconsumer.App : Nb of requests : 1 +19:01:54.395 ERROR [main] com.iluwatar.idempotentconsumer.App : Cannot start request twice! +19:01:54.399 INFO [main] com.iluwatar.idempotentconsumer.App : Request: Request(uuid=2d5521ef-6b6b-4003-9ade-81e381fe9a63, status=COMPLETED) +``` +## Benefits and Trade-offs of the Idempotent Consumer Pattern + +### Benefits + +* **Reliability**: Ensures that messages can be processed without unwanted side effects from duplicates. +* **Consistency**: Maintains data integrity by ensuring that duplicate messages do not cause redundant updates or actions. +* **Fault Tolerance**: Handles message retries gracefully, preventing them from causing errors. + +### Trade-offs + +* **State Management**: Requires storing processed message IDs, which can add memory overhead. +* **Complexity**: Implementing deduplication mechanisms can increase the complexity of the system. +* **Scalability**: In high-throughput systems, maintaining a large set of processed messages can impact performance and resource usage. + +## Related Patterns in Java + +* [Retry Pattern](https://java-design-patterns.com/patterns/retry/): Works well with the Idempotent Consumer pattern to handle failed messages. +* [Circuit Breaker Pattern](https://java-design-patterns.com/patterns/circuitbreaker/): Often used alongside idempotent consumers to prevent repeated failures from causing overload. + +## References and Credits + +* [Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions](https://amzn.to/4dznP2Y) +* [Designing Data-Intensive Applications](https://amzn.to/3UADv7Q) diff --git a/idempotent-consumer/pom.xml b/microservices-idempotent-consumer/pom.xml similarity index 98% rename from idempotent-consumer/pom.xml rename to microservices-idempotent-consumer/pom.xml index 8090f1fc0708..453e4a1ec762 100644 --- a/idempotent-consumer/pom.xml +++ b/microservices-idempotent-consumer/pom.xml @@ -35,7 +35,7 @@ 1.26.0-SNAPSHOT - idempotent-consumer + microservices-idempotent-consumer diff --git a/idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/App.java b/microservices-idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/App.java similarity index 100% rename from idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/App.java rename to microservices-idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/App.java diff --git a/idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/InvalidNextStateException.java b/microservices-idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/InvalidNextStateException.java similarity index 100% rename from idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/InvalidNextStateException.java rename to microservices-idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/InvalidNextStateException.java diff --git a/idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/Request.java b/microservices-idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/Request.java similarity index 100% rename from idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/Request.java rename to microservices-idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/Request.java diff --git a/idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/RequestNotFoundException.java b/microservices-idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/RequestNotFoundException.java similarity index 100% rename from idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/RequestNotFoundException.java rename to microservices-idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/RequestNotFoundException.java diff --git a/idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/RequestRepository.java b/microservices-idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/RequestRepository.java similarity index 100% rename from idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/RequestRepository.java rename to microservices-idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/RequestRepository.java diff --git a/idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/RequestService.java b/microservices-idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/RequestService.java similarity index 100% rename from idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/RequestService.java rename to microservices-idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/RequestService.java diff --git a/idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/RequestStateMachine.java b/microservices-idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/RequestStateMachine.java similarity index 100% rename from idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/RequestStateMachine.java rename to microservices-idempotent-consumer/src/main/java/com/iluwatar/idempotentconsumer/RequestStateMachine.java diff --git a/idempotent-consumer/src/test/java/com/iluwatar/idempotentconsumer/AppTest.java b/microservices-idempotent-consumer/src/test/java/com/iluwatar/idempotentconsumer/AppTest.java similarity index 100% rename from idempotent-consumer/src/test/java/com/iluwatar/idempotentconsumer/AppTest.java rename to microservices-idempotent-consumer/src/test/java/com/iluwatar/idempotentconsumer/AppTest.java diff --git a/idempotent-consumer/src/test/java/com/iluwatar/idempotentconsumer/RequestServiceTests.java b/microservices-idempotent-consumer/src/test/java/com/iluwatar/idempotentconsumer/RequestServiceTests.java similarity index 100% rename from idempotent-consumer/src/test/java/com/iluwatar/idempotentconsumer/RequestServiceTests.java rename to microservices-idempotent-consumer/src/test/java/com/iluwatar/idempotentconsumer/RequestServiceTests.java diff --git a/idempotent-consumer/src/test/java/com/iluwatar/idempotentconsumer/RequestStateMachineTests.java b/microservices-idempotent-consumer/src/test/java/com/iluwatar/idempotentconsumer/RequestStateMachineTests.java similarity index 100% rename from idempotent-consumer/src/test/java/com/iluwatar/idempotentconsumer/RequestStateMachineTests.java rename to microservices-idempotent-consumer/src/test/java/com/iluwatar/idempotentconsumer/RequestStateMachineTests.java diff --git a/pom.xml b/pom.xml index 3a276e44c9d4..2c4ba190c021 100644 --- a/pom.xml +++ b/pom.xml @@ -217,7 +217,7 @@ virtual-proxy function-composition microservices-distributed-tracing - idempotent-consumer + microservices-idempotent-consumer