Adding the Service Logs Streaming feature

README.md

@@ -157,6 +157,91 @@ xtf.foo.v2.version=1.0.3
 
 Retrieving an instance with this metadata: `Produts.resolve("product");`
 
+#### Service Logs Streaming (SLS)
+This feature allows for you to stream the services output while the test is running; this way you can see immediately 
+what is happening inside the cluster.
+This is of great help when debugging provisioning, specifically on Cloud environments, which instead would require for
+you to access your Pods.
+
+##### Kubernetes/OpenShift implementation
+The SLS OpenShift platform implementation relies upon the following fabric8 Kubernetes Client API features:
+
+- Watching Kubernetes events (see
+  [PodWatchEquivalent.java](https://github.com/fabric8io/kubernetes-client/blob/master/kubernetes-examples/src/main/java/io/fabric8/kubernetes/examples/kubectl/equivalents/PodWatchEquivalent.java))
+- Watching Pod logs (see
+  [PodLogExample.java](https://github.com/fabric8io/kubernetes-client/blob/master/kubernetes-examples/src/main/java/io/fabric8/kubernetes/examples/PodLogExample.java))
+
+The expected behavior is to stream the output of all the containers that are started or terminated in the selected 
+namespaces.
+
+##### Usage
+The SLS feature can be configured and enabled either via annotations or via properties.
+This behavior is provided by 
+[the ServiceLogsStreamingRunner JUnit 5 extension](./junit5/src/main/java/cz/xtf/junit5/extensions/ServiceLogsStreamingRunner.java).
+There are two different ways for enabling the SLS functionality, which are summarized in the following sections, please 
+refer to the [JUnit 5 submodule documentation](./junit5/README.md) in order to read about the extension implementation 
+details.
+
+###### The `@ServiceLogsStreaming` annotation (Developer perspective)
+Usage is as simple as annotating your test with `@ServiceLogsStreaming` e.g.:
+
+```java
+@ServiceLogsStreaming
+@Slf4j
+public class HelloWorldTest {
+  // ...
+}
+```
+
+###### The `xtf.log.streaming.enabled` and `xtf.log.streaming.config` property (Developer/Automation perspective)
+You can enable the SLS feature by setting the `xtf.log.streaming.enabled` property so that it would apply to 
+all the test classes being executed. 
+
+Conversely, if the above property is not set, you can set the `xtf.log.streaming.config` 
+property in order to provide multiple SLS configurations which could map to different test classes. 
+
+The `xtf.log.streaming.config` property value is expected to be a _comma_ (`,`) separated list of _configuration items_,
+each one formatted as a _semi-colon_ (`;`) separated list of _name_ and _value_ pairs for the above mentioned
+attributes, where the name/value separator is expected to be the _equals_ char (`=`). 
+A single _configuration item_ represents a valid source of configuration for a single SLS activation and exposes the 
+following information:
+
+* _**target**_: a regular expression which allows for the testing engine to check whether the current context test class
+    name matches the Service Logs Streaming configuration - **REQUIRED**
+
+* _**filter**_: a string representing a _regex_ to filter out the resources which the Service Logs Streaming activation
+    should be monitoring - **OPTIONAL**
+
+* _**output**_: the base path where the log stream files - one for each executed test class - will be created. 
+  **OPTIONAL**, if not assigned, logs will be streamed to `System.out`. When assigned, XTF will attempt to create the 
+  path in case it doesn't exist and default to `System.out` should any error occur. 
+
+###### Usage examples 
+Given what above, enabling SLS for all test classes is possible by executing the following command: 
+
+```shell
+mvn clean install -Dxtf.log.streaming.enabled=true
+```
+
+Similarly, in order to enable the feature for all test classes whose name is ending with "Test" should
+be as simple as executing something similar to the following command:
+
+```shell
+mvn clean install -Dxtf.log.streaming.config="target=.*Test"
+```
+
+which would differ in case the logs should be streamed to an output file:
+
+```shell
+mvn clean install -Dxtf.log.streaming.config="target=.*Test;output=/home/myuser/sls-logs"
+```
+
+or in case you'd want to provide multiple configuration items to map different test classes, e.g.:
+
+```shell
+mvn clean install -Dxtf.log.streaming.config="target=TestClassA,target=TestClassB.*;output=/home/myuser/sls-logs;filter=.*my-app.*"
+```
+
 ### JUnit5
 JUnit5 module provides a number of extensions and listeners designed to easy up OpenShift images test management. 
 See [JUnit5](https://github.com/xtf-cz/xtf/blob/master/core/src/main/java/cz/xtf/core/waiting/SimpleWaiter.java) for 

core/src/main/java/cz/xtf/core/service/logs/streaming/AnnotationBasedServiceLogsConfigurations.java

@@ -0,0 +1,39 @@
+package cz.xtf.core.service.logs.streaming;
+
+import java.util.HashMap;
+import java.util.Map;
+
+import lombok.extern.slf4j.Slf4j;
+
+/**
+ * Implements {@link ServiceLogsSettings} in order to provide the concrete logic to handle SLS configuration based
+ * on the {@link ServiceLogsStreaming} annotation.
+ */
+@Slf4j
+public class AnnotationBasedServiceLogsConfigurations {
+
+    private static final Map<String, ServiceLogsSettings> CONFIGURATIONS = new HashMap<>();
+
+    private ServiceLogsSettings loadConfiguration(Class<?> testClazz) {
+        ServiceLogsStreaming annotation = testClazz.getAnnotation(ServiceLogsStreaming.class);
+        if (annotation != null) {
+            return new ServiceLogsSettings.Builder()
+                    .withTarget(testClazz.getName())
+                    .withFilter(annotation.filter())
+                    .withOutputPath(annotation.output())
+                    .build();
+        }
+        return null;
+    }
+
+    public ServiceLogsSettings forClass(Class<?> testClazz) {
+        ServiceLogsSettings serviceLogsSettings = CONFIGURATIONS.get(testClazz.getName());
+        if (serviceLogsSettings == null) {
+            serviceLogsSettings = loadConfiguration(testClazz);
+            if (serviceLogsSettings != null) {
+                CONFIGURATIONS.put(testClazz.getName(), serviceLogsSettings);
+            }
+        }
+        return serviceLogsSettings;
+    }
+}

core/src/main/java/cz/xtf/core/service/logs/streaming/ServiceLogColor.java

@@ -0,0 +1,55 @@
+package cz.xtf.core.service.logs.streaming;
+
+import java.util.concurrent.locks.Lock;
+import java.util.concurrent.locks.ReentrantLock;
+
+/**
+ * This enum contains the colors that are used when logging container's log on the console; each container gets a
+ * different color in order to visually differentiate containers; the background is also colored in order to
+ * differentiate application log from containers log.
+ */
+public enum ServiceLogColor {
+    ANSI_RESET_FG("\u001B[0m"),
+    ANSI_RESET_BG("\u001b[0m"),
+    ANSI_POD_LOG_BG("\u001b[40m"),
+    ANSI_RED1("\u001B[31m"),
+    ANSI_GREEN1("\u001B[32m"),
+    ANSI_YELLOW1("\u001B[33m"),
+    ANSI_AZURE1("\u001B[34m"),
+    ANSI_VIOLET1("\u001B[35m"),
+    ANSI_WATER1("\u001B[36m"),
+    ANSI_BRIGHT_RED("\u001B[91m"),
+    ANSI_BRIGHT_GREEN("\u001B[92m"),
+    ANSI_BRIGHT_YELLOW("\u001B[93m"),
+    ANSI_BRIGHT_BLUE("\u001B[94m"),
+    ANSI_BRIGHT_PURPLE("\u001B[95m"),
+    ANSI_BRIGHT_CYAN("\u001B[96m"),
+    ANSI_BRIGHT_WHITE("\u001B[97m"),
+    ANSI_POD_NAME_BG("\u001b[40;3m");
+
+    public final String value;
+    private static final Lock lock = new ReentrantLock();
+    private static int idx = 0;
+
+    public static final ServiceLogColor[] COLORS = {
+            ANSI_BRIGHT_GREEN, ANSI_BRIGHT_RED, ANSI_BRIGHT_YELLOW,
+            ANSI_BRIGHT_BLUE, ANSI_BRIGHT_PURPLE, ANSI_BRIGHT_CYAN, ANSI_BRIGHT_WHITE,
+            ANSI_RED1, ANSI_GREEN1, ANSI_YELLOW1, ANSI_AZURE1, ANSI_VIOLET1, ANSI_WATER1
+    };
+
+    private ServiceLogColor(String value) {
+        this.value = value;
+    }
+
+    public static ServiceLogColor getNext() {
+        lock.lock();
+        try {
+            if (++idx >= COLORS.length) {
+                idx = 0;
+            }
+            return COLORS[idx];
+        } finally {
+            lock.unlock();
+        }
+    }
+}

core/src/main/java/cz/xtf/core/service/logs/streaming/ServiceLogColoredPrintStream.java

@@ -0,0 +1,199 @@
+package cz.xtf.core.service.logs.streaming;
+
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.io.InterruptedIOException;
+import java.io.OutputStream;
+import java.io.PrintStream;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.List;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public class ServiceLogColoredPrintStream extends PrintStream {
+    private static final Logger logger = LoggerFactory.getLogger(ServiceLogColoredPrintStream.class);
+    private final ServiceLogColor color;
+    private final String prefix;
+    private boolean needHeader = true;
+
+    /**
+     * Constructor has private access because you are supposed to used the
+     * {@link Builder}
+     */
+    private ServiceLogColoredPrintStream(OutputStream out, ServiceLogColor color, String prefix) {
+        // this class just writes to `out` so its life cycle, which must be handled at the outer scope, is not altered
+        // at all
+        super(out, true);
+        this.color = color;
+        this.prefix = prefix;
+    }
+
+    /**
+     * Splits the current buffer into tokens; the criteria for splitting consists in considering each new line as a
+     * token and everything else as another token.
+     *
+     * If we have e.g.: "AAAA\nBBBB", we would have 3 tokens: "AAAA","\n","BBBB";
+     * Note that a buffer not containing any new line is considered a single token e.g. buffer "AAAA" is considered as
+     * the single token "AAAA"
+     * 
+     * @param buf the data.
+     * @param off the start offset in the data.
+     * @param len the number of bytes to write.
+     * @return the list of tokens in the data; joining the tokens you get the original data.
+     * @throws IOException in case something goes wrong while managing streams internally
+     */
+    protected List<byte[]> getTokens(byte buf[], int off, int len) throws IOException {
+        final List<byte[]> tokens = new ArrayList<>();
+        try (ByteArrayOutputStream baos = new ByteArrayOutputStream()) {
+            for (int i = off; i < off + len; i++) {
+                if ((char) buf[i] == '\n') {
+                    if (baos.size() > 0) {
+                        tokens.add(baos.toByteArray());
+                        baos.reset();
+                    }
+                    tokens.add(Arrays.copyOfRange(buf, i, i + 1));
+                } else {
+                    baos.write(buf[i]);
+                }
+            }
+            if (baos.size() > 0)
+                tokens.add(baos.toByteArray());
+        }
+        if (tokens.isEmpty())
+            tokens.add(Arrays.copyOfRange(buf, off, len));
+        return tokens;
+    }
+
+    private boolean isNewLine(byte[] token) {
+        return token != null && token.length == 1 && (char) token[0] == '\n';
+    }
+
+    /**
+     * Prints the line header which usually consists of the name of the container the log comes from, preceded by the
+     * name of the pod and the name of the namespace where the container is running, with properly colored background
+     * and foreground
+     * 
+     * @throws IOException in case something goes wrong while writing to the underlying stream
+     */
+    private void printLineHeader() throws IOException {
+        // set line header foreground and background
+        out.write(color.value.getBytes(StandardCharsets.UTF_8));
+        out.write(ServiceLogColor.ANSI_POD_NAME_BG.value.getBytes(StandardCharsets.UTF_8));
+        // actual line header
+        out.write(ServiceLogUtils.formatStreamedLogLine(prefix).getBytes(StandardCharsets.UTF_8));
+        // reset foreground and background
+        out.write(ServiceLogColor.ANSI_RESET_FG.value.getBytes(StandardCharsets.UTF_8));
+        out.write(ServiceLogColor.ANSI_RESET_BG.value.getBytes(StandardCharsets.UTF_8));
+        out.flush();
+    }
+
+    /**
+     * Prints a token of the log line with properly colored background and foreground
+     * 
+     * @param buf the data.
+     * @param off the start offset in the data.
+     * @param len the number of bytes to write.
+     * @throws IOException in case something goes wrong writing to the underlying stream
+     */
+    private void printToken(byte buf[], int off, int len) throws IOException {
+        // set log token foreground and background
+        out.write(ServiceLogColor.ANSI_POD_LOG_BG.value.getBytes(StandardCharsets.UTF_8));
+        out.write(color.value.getBytes(StandardCharsets.UTF_8));
+        // actual log token
+        out.write(buf, off, len);
+        // reset foreground and background
+        out.write(ServiceLogColor.ANSI_RESET_FG.value.getBytes(StandardCharsets.UTF_8));
+        out.write(ServiceLogColor.ANSI_RESET_BG.value.getBytes(StandardCharsets.UTF_8));
+        out.flush();
+    }
+
+    @Override
+    public void write(int b) {
+        write(new byte[] { (byte) b }, 0, 1);
+    }
+
+    /**
+     * Prints the data with properly colored background and foreground; takes care of adding a line header to each line
+     * 
+     * @param buf the data.
+     * @param off the start offset in the data.
+     * @param len the number of bytes to write.
+     */
+    @Override
+    public void write(byte buf[], int off, int len) {
+        try {
+            synchronized (out) {
+                if (out == null)
+                    throw new IOException("Stream is closed");
+
+                logger.trace("TOKEN_FROM_CONTAINER: {}", new String(Arrays.copyOfRange(buf, off, len)));
+
+                // first line ever --> print header
+                if (needHeader) {
+                    printLineHeader();
+                    needHeader = false;
+                }
+                // split by newline
+                List<byte[]> tokens = getTokens(buf, off, len);
+                if (!tokens.isEmpty()) {
+                    for (int i = 0; i < tokens.size(); i++) {
+                        if (isNewLine(tokens.get(i))) {
+                            out.write('\n');
+                            // print the line header unless it's the last token: in that case set the reminder for the next
+                            if (i < tokens.size() - 1) {
+                                printLineHeader();
+                            } else {
+                                needHeader = true;
+                            }
+                        } else {
+                            printToken(tokens.get(i), 0, tokens.get(i).length);
+                        }
+                    }
+                }
+            }
+        } catch (InterruptedIOException x) {
+            Thread.currentThread().interrupt();
+        } catch (IOException x) {
+            setError();
+        }
+    }
+
+    public static class Builder {
+        private OutputStream outputStream;
+        private ServiceLogColor color;
+        private String prefix;
+
+        public Builder outputTo(final OutputStream outputStream) {
+            this.outputStream = outputStream;
+            return this;
+        }
+
+        public Builder withColor(final ServiceLogColor color) {
+            this.color = color;
+            return this;
+        }
+
+        public Builder withPrefix(final String prefix) {
+            this.prefix = prefix;
+            return this;
+        }
+
+        public ServiceLogColoredPrintStream build() {
+            if (outputStream == null) {
+                throw new IllegalStateException("OutputStream must be specified!");
+            }
+            if (color == null) {
+                throw new IllegalStateException("Color must be specified!");
+            }
+            if (prefix == null) {
+                throw new IllegalStateException("Prefix must be specified!");
+            }
+            return new ServiceLogColoredPrintStream(outputStream, color, prefix);
+        }
+
+    }
+
+}