Skip to content

Commit

Permalink
In Memory implementation of the Caching API (zowe#904)
Browse files Browse the repository at this point in the history
In Memory implementation of the Caching API 

Signed-off-by: Jakub Balhar <[email protected]>
  • Loading branch information
balhar-jakub authored Oct 26, 2020
1 parent 85d9b0d commit 678d1a9
Show file tree
Hide file tree
Showing 31 changed files with 1,601 additions and 9 deletions.
69 changes: 69 additions & 0 deletions caching-service/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
# Caching Service

To support the High Availability of all components within Zowe, components either need to be stateless, or offload the state to a location accessible by all instances of the service, including those which just started. At the current time, some services are not, and cannot be stateless. For these services, we introduce the Caching service.

The Caching service aims to provide an API which offers the possibility to store, retrieve and delete data associated with keys. The service will be used only by internal Zowe applications and will not be exposed to the internet. The Caching service needs to support a hot-reload scenario in which a client service requests all available service data.

The initial implementation of the service will depend on VSAM to store the key/value pairs, as VSAM is a native z/OS solution to storing key/value pairs. Eventually, there will be other implementations for solutions such as MQs. As such, this needs to be taken into account for the initial design document.

## Architecture

Internal architecture needs to take into consideration, namely the fact that there will be multiple storage solutions. The API, on the other hand, remains the same throughout various storage implementations.

![Diagram](cachingServiceStructure.png "Architecture of the service")

## How to use

The Caching Service is built on top of the spring enabler, which means that it is dynamically registered to the API Mediation Layer. It appears in the API Catalog under the tile "Zowe Applications".

There are REST APIs available to create, delete, and update key-value pairs in the cache, as well as APIs to read a specific key-value pair or all key-value pairs in the cache.

## Storage

There are multiple storage solutions supported by the Caching Service with the option to
add custom implementation. [Additional Storage Support](#additional-storage-support) explains
what needs to be done to implement custom solution.

### In Memory

This storage is useful for testing and integration verification. Don't use it in production.
The key/value pairs are stored only in the memory of one instance of the service and therefore
won't persist.

### VSAM

TO BE DONE

### Additional Storage Support

To add a new implementation it is necessary to provide the library with the implementation
of the Storage.class and properly configure the Spring with the used implementation.

@ConditionalOnProperty(
value = "caching.storage",
havingValue = "custom"
)
@Bean
public Storage custom() {
return new CustomStorage();
}

The example above shows the Configuration within the library that will use different storage than the default InMemory one.

It is possible to provide the custom implementation via the -Dloader.path property provided on startup of the Caching service.

## How do you run for local development

The Caching Service is a Spring Boot application. You can either add it as a run configuration and run it together with other services, or the npm command to run API ML also runs the mock.

Command to run full set of api-layer: `npm run api-layer`. If you are looking for the Continuous Integration set up run: `npm run api-layer-ci`

In local usage, the Caching Service will run at `https://localhost:10016`. The API path is `/cachingservice/api/v1/cache/${path-params-as-needed}`.
For example, `https://localhost:10016/cachingservice/api/v1/cache/my-key` retrieves the cache entry using the key 'my-key'.

## Configuration properties

The Caching Service uses the standard `application.yml` structure for configuration.

`apiml.service.routes` only specifies one API route as there is no need for web socket or UI routes.
`caching.storage` this property is reserved for the setup of the proper storage within the Caching Service.
82 changes: 82 additions & 0 deletions caching-service/build.gradle
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
buildscript {
repositories mavenRepositories
dependencies {
classpath("org.springframework.boot:spring-boot-gradle-plugin:${springBootVersion}")
classpath("gradle.plugin.com.gorylenko.gradle-git-properties:gradle-git-properties:${gradleGitPropertiesVersion}")
}
}

normalization {
runtimeClasspath {
ignore("**/*git.properties*")
ignore("**/*build-info.properties*")
}
}

apply plugin: 'org.springframework.boot'
apply plugin: 'com.gorylenko.gradle-git-properties'

springBoot {
// This statement tells the Gradle Spring Boot plugin to generate a file
// build/resources/main/META-INF/build-info.properties that is picked up by Spring Boot to display
// via /info endpoint
buildInfo {
properties {
// Generate extra build info:
additionalProperties = [
by: System.properties['user.name'],
operatingSystem: "${System.properties['os.name']} (${System.properties['os.version']})",
number: System.getenv('BUILD_NUMBER') ? System.getenv('BUILD_NUMBER') : "n/a",
machine: InetAddress.localHost.hostName
]
}
}
}

gitProperties {
gitPropertiesDir = new File("${project.rootDir}/${name}/build/resources/main/META-INF")
}

dependencies {
compile project(':common-service-core')
compile project(':zaas-client')

implementation project(':onboarding-enabler-spring')

compile libraries.jjwt
compile libraries.jjwt_impl
compile libraries.jjwt_jackson

implementation libraries.springFox
implementation libraries.spring_boot_starter
implementation libraries.spring_boot_starter_actuator
implementation libraries.spring_boot_starter_web
implementation libraries.spring_boot_starter_websocket

implementation libraries.bootstrap
implementation libraries.jquery

implementation libraries.gson
compileOnly libraries.lombok
annotationProcessor libraries.lombok

testImplementation libraries.spring_boot_starter_test
}


bootJar.archiveName = "${bootJar.baseName}.jar"

jar {
enabled = true
archiveName = "${jar.baseName}-thin.jar"

def libClassPathEntries = configurations.runtimeClasspath.files.collect {
"lib/" + it.getName()
}
doFirst {
manifest {
attributes "Class-Path": libClassPathEntries.join(" "),
"Main-Class": "org.zowe.apiml.caching.CachingService"
}
}
}
Binary file added caching-service/cachingServiceStructure.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
/*
* This program and the accompanying materials are made available under the terms of the
* Eclipse Public License v2.0 which accompanies this distribution, and is available at
* https://www.eclipse.org/legal/epl-v20.html
*
* SPDX-License-Identifier: EPL-2.0
*
* Copyright Contributors to the Zowe Project.
*/
package org.zowe.apiml.caching;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.zowe.apiml.enable.EnableApiDiscovery;

@SpringBootApplication
@EnableApiDiscovery
public class CachingService {

public static void main(String[] args) {
SpringApplication app = new SpringApplication(CachingService.class);
app.setLogStartupInfo(false);
app.run(args);
}
}
Loading

0 comments on commit 678d1a9

Please sign in to comment.