Merge pull request #338 from adorsys/analyze-document-and-present-dat…

…asafe-storage-api

update: datasafe-storage-api and improve test-coverage

datasafe-storage/datasafe-storage-api/README.md

@@ -1,5 +1,124 @@
 # Storage API
 
-This module exposes storage API used by other modules. Use 
-[StorageService](src/main/java/de/adorsys/datasafe/storage/api/StorageService.java) interface, provided by this module,
-if you want to write your own adapter.
+This module provides an abstraction layer for various storage operations, enabling interaction with different types
+of storage backends through a unified interface. It allows for the reading, writing, listing, and removal of data stored in different locations such as S3 buckets, local file systems, based on URI schemes or patterns.
+It exposes storage API used by other modules. 
+
+- Use [StorageService](src/main/java/de/adorsys/datasafe/storage/api/StorageService.java) interface, provided by this module,
+if you want to write your own adapter.
+
+## Key Types and Interfaces
+
+#### StorageCheckService
+- **Purpose:**  To check if a specified resource exists at a given location.
+- **Key Method:**
+```java
+boolean objectExists(AbsoluteLocation location);
+```
+#### StorageListService
+- **Purpose:**  To list resources at a given location.
+- **Key Method:**
+```java
+Stream<AbsoluteLocation<ResolvedResource>> list(AbsoluteLocation location);
+```
+#### StorageReadService
+- **Purpose:**  To read data from a specified resource location.
+- **Key Method:**
+```java
+InputStream read(AbsoluteLocation location);
+```
+
+#### StorageRemoveService
+- **Purpose:**  To remove a specified resource location.
+- **Key Method:**
+```java
+void remove(AbsoluteLocation location);
+```
+#### StorageWriteService
+- **Purpose:**  To write data to a specified resource location
+- **Key Method:**
+```java
+OutputStream write(WithCallback<AbsoluteLocation, ? extends ResourceWriteCallback> locationWithCallback);
+```
+- **Additional Method:**
+```java
+default Optional<Integer> flushChunkSize(AbsoluteLocation location) { ... }
+```
+#### StorageService
+- **Purpose:**   Combines all storage operations into a single interface.
+- **Implements:**
+*   StorageCheckService
+*   StorageListService
+*   StorageReadService
+*   StorageRemoveService
+*   StorageWriteService
+
+## Key Classes
+
+#### BaseDelegatingStorage
+- **Purpose:**  Abstract base class that delegates storage operations to actual storage implementations.
+- **Method:** Implements methods from StorageService and delegates them to an abstract service method.
+```java
+protected abstract StorageService service(AbsoluteLocation location);
+```
+#### RegexDelegatingStorage
+- **Purpose:**  Delegates storage operations based on regex matching of URIs.
+
+- **Key Fields:** 
+```java
+private final Map<Pattern, StorageService> storageByPattern;
+```
+- **Implementation of service method**
+```java
+protected StorageService service(AbsoluteLocation location) { ... }
+```
+#### SchemeDelegatingStorage
+- **Purpose:**  Delegates storage operations based on URI schemes.
+
+- **Key Fields:**
+```java
+private final Map<String, StorageService> storageByScheme;
+```
+- **Implementation of service method**
+```java
+protected StorageService service(AbsoluteLocation location) { ... }
+```
+#### UriBasedAuthStorageService
+- **Purpose:**  Manages storage connections based on URIs containing credentials, such as S3 URIs.
+
+- **Key Fields:**
+```java
+private final Map<AccessId, StorageService> clientByItsAccessKey = new ConcurrentHashMap<>();
+private final Function<URI, String> bucketExtractor;
+private final Function<URI, String> regionExtractor;
+private final Function<URI, String> endpointExtractor;
+private final Function<AccessId, StorageService> storageServiceBuilder;
+```
+- **Key Inner Class: AccessId**
+```java
+private final String accessKey;
+private final String secretKey;
+private final String region;
+private final String bucketName;
+private final String endpoint;
+private final URI withoutCreds;
+private final URI onlyHostPart;
+```
+
+#### UserBasedDelegatingStorage
+- **Purpose:**  Delegates storage operations based on user-specific bucket mappings.
+
+- **Key Fields:**
+```java
+private final Map<String, StorageService> clientByBucket = new ConcurrentHashMap<>();
+private final List<String> amazonBuckets;
+private final Function<String, StorageService> storageServiceBuilder;
+```
+### URI Routing Flowchart:
+This flowchart depicts how storage operations are routed based on URIs and patterns:
+![URI Routing flowchart](http://www.plantuml.com/plantuml/dpng/dL1DImCn4BtlhnZtj0N1GtiKQTdMAXHR3D9Z6PFPDfWcaang_VTck-lgKWNn56RcVUIzSM3q7FSckz1McgW8hilHLJdQbCuo7VacorRaWxD53EGl8NzAJzwy0NX7C4L6WHL1OETnIp1PtUU3JBm7fdsXqZMaQtjCn0ullk7JVkNTGQiaYX2jhZGfq9R9LoW9AkUR2ILhkuKtpJiueDSkXixu6UKBMHKwzytio4KO9l79Me0OrZQbSL5rb1Jce2Nr6PKs54vZmY-SH0EtQGKD9E-MhKYVx58d_YljiXwxgAArIuSvMV9Q_l2Jx95Cs_PvUtNjDJsL1YKQKuUjyMV8K-mf6TeYDvJFJonVoIDhPt_bzWhufqQ_Xp-ePE9kkTuiPlFPmxGOP6EoAkxD1m00)
+## Conclusion
+This module provides a robust and flexible framework for managing data storage in a system with multiple storage backends. It offers several key benefits:
+- **Pluggability:**  Easily add new storage implementations without modifying existing code.
+- **Testability:** Simplify testing with a clear API and mocking capabilities.
+- **Maintainability:** Centralize storage logic and reduce code duplication.

datasafe-storage/datasafe-storage-api/src/main/java/de/adorsys/datasafe/storage/api/UriBasedAuthStorageService.java

@@ -19,9 +19,9 @@
 public class UriBasedAuthStorageService extends BaseDelegatingStorage {
 
     private final Map<AccessId, StorageService> clientByItsAccessKey = new ConcurrentHashMap<>();
-    private final Function<URI, String> bucketExtractor;
-    private final Function<URI, String> regionExtractor;
-    private final Function<URI, String> endpointExtractor;
+    final Function<URI, String> bucketExtractor;
+    final Function<URI, String> regionExtractor;
+    final Function<URI, String> endpointExtractor;
 
     // Builder to create S3 or other kind of Storage service
     private final Function<AccessId, StorageService> storageServiceBuilder;

datasafe-storage/datasafe-storage-api/src/test/java/de/adorsys/datasafe/storage/api/RegexDelegatingStorageTest.java

@@ -0,0 +1,62 @@
+package de.adorsys.datasafe.storage.api;
+import de.adorsys.datasafe.types.api.resource.AbsoluteLocation;
+import de.adorsys.datasafe.types.api.resource.BasePrivateResource;
+import de.adorsys.datasafe.types.api.resource.WithCallback;
+import de.adorsys.datasafe.types.api.shared.BaseMockitoTest;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.mockito.Mock;
+
+import java.util.Collections;
+import java.util.Map;
+import java.util.regex.Pattern;
+
+import static org.junit.jupiter.api.Assertions.assertThrows;
+import static org.mockito.ArgumentMatchers.any;
+import static org.mockito.Mockito.verify;
+
+public class RegexDelegatingStorageTest extends BaseMockitoTest{
+
+    @Mock
+    private StorageService service;
+    private RegexDelegatingStorage tested;
+    private AbsoluteLocation location;
+
+    @BeforeEach
+    void setUp() {
+        Map<Pattern, StorageService> storageByPattern = Collections.singletonMap(Pattern.compile("s3://.*"), service);
+        tested = new RegexDelegatingStorage(storageByPattern);
+        location = new AbsoluteLocation<>(BasePrivateResource.forPrivate("s3://bucket"));
+    }
+    @Test
+    void objectExists() {
+        tested.objectExists(location);
+        verify(service).objectExists(location);
+    }
+    @Test
+    void list() {
+        tested.list(location);
+        verify(service).list(location);
+    }
+    @Test
+    void read() {
+        tested.read(location);
+        verify(service).read(location);
+    }
+    @Test
+    void remove() {
+        tested.remove(location);
+        verify(service).remove(location);
+    }
+    @Test
+    void write() {
+        tested.write(WithCallback.noCallback(location));
+        verify(service).write(any(WithCallback.class));
+    }
+    @Test
+    void objectExistsWithNoMatch() {
+    AbsoluteLocation badlocation = new AbsoluteLocation<>(BasePrivateResource.forPrivate("file://bucket"));
+    assertThrows(IllegalArgumentException.class, () -> tested.objectExists(badlocation));
+    }
+
+}

datasafe-storage/datasafe-storage-api/src/test/java/de/adorsys/datasafe/storage/api/UriBasedAuthStorageServiceTest.java

@@ -2,12 +2,14 @@
 
 import de.adorsys.datasafe.types.api.resource.AbsoluteLocation;
 import de.adorsys.datasafe.types.api.resource.BasePrivateResource;
+import de.adorsys.datasafe.types.api.resource.PrivateResource;
 import de.adorsys.datasafe.types.api.resource.WithCallback;
 import de.adorsys.datasafe.types.api.shared.BaseMockitoTest;
 import lombok.Getter;
 import lombok.RequiredArgsConstructor;
 import lombok.ToString;
 import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
 import org.junit.jupiter.params.ParameterizedTest;
 import org.junit.jupiter.params.provider.MethodSource;
 import org.junit.jupiter.params.provider.ValueSource;
@@ -42,8 +44,49 @@ void init() {
         when(getService.apply(argumentCaptor.capture())).thenReturn(storage);
         tested = new UriBasedAuthStorageService(getService);
     }
+    @Test
+    void testDefaultConstructor() {
+        UriBasedAuthStorageService service = new UriBasedAuthStorageService(getService);
+        assertThat(service).isNotNull();
+    }
 
-    @MethodSource("fixture")
+    @Test
+    void testCustomConstructor() {
+        Function<URI, String[]> segmentator = uri -> new String[] {"region", "bucket"};
+        UriBasedAuthStorageService service = new UriBasedAuthStorageService(getService, segmentator);
+        assertThat(service).isNotNull();
+    }
+    @Test
+    void testRegionExtractor() {
+        Function<URI, String[]> segmentator = uri -> new String[] {"region", "bucket"};
+        UriBasedAuthStorageService service = new UriBasedAuthStorageService(getService, segmentator);
+
+        URI uri = URI.create("http://host.com/region/bucket");
+        String region = service.regionExtractor.apply(uri);
+        assertThat(region).isEqualTo("region");
+    }
+
+    @Test
+    void testBucketExtractor() {
+        Function<URI, String[]> segmentator = uri -> new String[] {"region", "bucket"};
+        UriBasedAuthStorageService service = new UriBasedAuthStorageService(getService, segmentator);
+
+        URI uri = URI.create("http://host.com/region/bucket");
+        String bucket = service.bucketExtractor.apply(uri);
+        assertThat(bucket).isEqualTo("bucket");
+    }
+
+    @Test
+    void testEndpointExtractor() {
+        Function<URI, String[]> segmentator = uri -> new String[] {"region", "bucket"};
+        UriBasedAuthStorageService service = new UriBasedAuthStorageService(getService, segmentator);
+
+        URI uri = URI.create("http://host.com:8080/region/bucket");
+        String endpoint = service.endpointExtractor.apply(uri);
+        assertThat(endpoint).isEqualTo("http://host.com:8080/");
+    }
+
+        @MethodSource("fixture")
     @ParameterizedTest
     void objectExists(MappedItem item) {
         tested.objectExists(item.getUri());