MDC

README.md

@@ -4,4 +4,4 @@ Make your Java application be truly observable! Let's dive into logs, traces and
 
 ## Workshop
 
-Go to ...
+Go to https://worldline.github.io/observability-workshop/

compose.yml

@@ -440,4 +440,4 @@ services:
       test: ["CMD-SHELL", "kafka-topics --bootstrap-server kafka:9092 --list"]
       interval: 30s
       timeout: 10s
-      retries: 5
+      retries: 5

docker/alloy/config.alloy

@@ -54,6 +54,12 @@ loki.process "jsonlogs" {
 		format = "RFC3339"
 		fallback_formats = ["UnixMs",]
 	}*/
+
+  /*stage.luhn {
+    		min_length  = 13
+    		replacement = "**MASKED**"
+    	}
+    	*/
 }
 
 // EXPORTER (LOKI)

docs/workshop.md

@@ -492,27 +492,30 @@ Go to the ``PaymentResource`` class and modify the method ``processPayment()`` t
 ```java
 public ResponseEntity<PaymentResponse> processPayment(PaymentRequest paymentRequest)
 
-MDC.put("context",paymentRequest);
+MDC.put("CardNumber",paymentRequest.cardNumber());
+MDC.put("POS",paymentRequest.posId());
+
 [...]
 MDC.clear();
 return httpResponse;
 
 ```
 
-Go to the MDC spring profile configuration file (``easypay-service/src/main/resources/application-mdc.properties``) and check the configuration got the ``context`` field.
+Go to the MDC spring profile configuration file (``easypay-service/src/main/resources/application-mdc.properties``) and check the configuration got both the ``CardNumber`` & ``POS``fields.
 
-Restart the application activating the ``mdc`` profile and see how the logs look like now.
+Activate the ``mdc`` profile in the ``compose.yml`` file:
 
-```bash
-./gradlew :easypay-service:bootRun -x test  --args='--spring.profiles.active=default,mdc'
+```yaml
+  easypay-service:
+    image: easypay-service:latest
+    [...]
+      SPRING_PROFILES_ACTIVE: default,docker,mdc
 ```
 
 > aside positive
->
-> You can verify the MDC profile is applied by checking the presence of this log message:
-> ``The following 2 profiles are active: "default", "mdc"``
+> To apply these modifications, we must restart the ``easypay-service``. 
+> It will be done later.
 > 
-
 ### Adding more content in our logs
 
 To have more logs, we will run several HTTP requests using [K6](https://k6.io/):
@@ -525,54 +528,31 @@ $ k6 run -u 5 -d 5s k6/01-payment-only.js
 
 Check then the logs to pinpoint some exceptions.
 
-### Personal Identifiable Information (PII)  bfuscation
+### Personal Identifiable Information (PII) obfuscation
 For compliance and preventing personal data loss, we will obfuscate the card number in the logs:
 
-In the Alloy configuration file (``docker/alloy/config.alloy``), add the [luhn stage](https://grafana.com/docs/alloy/latest/reference/components/loki.process/#stageluhn-block) into the ``jsonlogs`` loki process stage
-
-``
-stage.luhn {
-replacement= "**DELETED**"
-}
-``
-
-We will then have the following configuration for processing the JSON logs:
+In the Alloy configuration file (``docker/alloy/config.alloy``), uncomment the [luhn stage](https://grafana.com/docs/alloy/latest/reference/components/loki.process/#stageluhn-block).
 
 ```
-loki.process "jsonlogs" {
-	forward_to = [loki.write.endpoint.receiver]
-
-	stage.luhn {
-    	    replacement= "**DELETED**"
-    	}
-
-	stage.json {
-		expressions = {
-			// timestamp   = "timestamp",
-			application = "context.properties.applicationName",
-			instance    = "context.properties.instance",
-			trace_id    = "mdc.trace_id",
-		}
-	}
-
-	stage.labels {
-		values = {
-			application = "application",
-			instance    = "instance",
-			trace_id    = "trace_id",
-		}
-	}
-
+/*stage.luhn {
+min_length  = 13
+replacement = "**MASKED**"
 }
+*/
+``` 
 
+Rebuild/Restart then the whole platform: 
+
+```bash
+$ docker compose down
+$ docker compose up -d --build --remove-orphans
 ```
 
 
-Restart then Alloy: 
+> aside positive
+>
+> During this workshop, we will only obfuscate the card numbers in Loki. It will therefore be stored as is in the log files but obfuscated in Loki and by this way in the data exposed on Grafana.
 
-```bash
-$ docker restart collector
-```
 ### Logs Correlation  
 > aside positive
 >
@@ -581,10 +561,6 @@ $ docker restart collector
 > One approach would be to correlate all of your logs using a correlation Id.
 > If an incoming request has no correlation id header, the API creates it. If there is one, it uses it instead.
 
-> aside negative
->
-> TODO mettre la manipulation pour le correlation ID et un exemple d'utilisation
-
 ### Let's dive into our logs on Grafana!
 
 Logs are stored in the logs folder (``easypay-service/logs``).
@@ -826,6 +802,7 @@ The ``Metric`` field lists all the metrics available in Prometheus server: take
   * To split the memory usage per service, you can click on the ``By label`` button and select the label named ``application`` (do not forget to click on ``Run query`` afterthat).
 
 * You can also filter metrics to be displayed using ``Label filters``: try to create a filter to display only the metric related to the application named easypay-service.
+* Check the card numbers are now obfuscated with the ``**MASKED**`` content. 
 
 > aside positive
 >

easypay-service/src/main/java/com/worldline/easypay/payment/boundary/PaymentResource.java

@@ -75,7 +75,8 @@ public ResponseEntity<Payment> findById(
     @ApiResponse(responseCode = "201", description = "Payment processed", content = @Content(mediaType = "application/json"))
     public ResponseEntity<PaymentResponse> processPayment(
             @Parameter(description = "The payment to be processed", required = true) @Valid @NotNull @RequestBody PaymentRequest paymentRequest) {
-        // MDC.put("context", paymentRequest);
+//        MDC.put("CardNumber",paymentRequest.cardNumber());
+//        MDC.put("POS",paymentRequest.posId());
         // LOG.info("Processing new payment: {}", paymentRequest);
         PaymentProcessingContext paymentContext = new PaymentProcessingContext(paymentRequest);
 

easypay-service/src/main/resources/application-mdc.properties

@@ -1 +1 @@
-logging.pattern.console=%clr(%d{yyyy-MM-dd'T'HH:mm:ss.SSSXXX}){faint} %clr(%5p) %clr(${PID:- }){magenta} %clr(---){faint} %clr(%applicationName[%15.15t]){faint} %clr(${LOG_CORRELATION_PATTERN:-}){faint}%clr(%-40.40logger{39}){cyan} %clr(:){faint} %clr(CONTEXT=){faint}%clr(%X{context:-null}) %m%n%wEx
+logging.pattern.console=%clr(%d{yyyy-MM-dd'T'HH:mm:ss.SSSXXX}){faint} %clr(%5p) %clr(${PID:- }){magenta} %clr(---){faint} %clr(%applicationName[%15.15t]){faint} %clr(${LOG_CORRELATION_PATTERN:-}){faint}%clr(%-40.40logger{39}){cyan} %clr(:){faint} %clr(CardNumber=){faint}%clr(%X{CardNumber:-null}) %clr(POS=){faint}%clr(%X{POS:-null}) %m%n%wEx