Merge pull request #25 from nguptaopensds/master

[User-Guide]Added user guide for object lifecycle management

content/guides/developer-guides/object-lifecycle-management/_index.md

@@ -0,0 +1,87 @@
+---
+title: Object Lifecycle Management
+description: "Developer guide for Object Lifecycle Management."
+weight: 40
+disableToc: false
+tags: ["developer guide", "object lifecycle"] 
+---
+## Overview
+This guide is intended for Object Lifecycle feature developers. It provides detailed steps to write new API using OpenSDS. To understand the feature and requirements in detail, please refer to the Object Lifecycle Management user guide and Object Lifecycle design doc. 
+
+## APIs
+APIs are a set of functions and procedures that allow for the creation of applications that access data and features of other applications, services or operating system. In OpenSDS lifecycle management feature we are using APIs to communicate with different storage backend.
+
+## Write new API
+Before starting to write the APIs, it is required to have basic understanding of XML and JSON syntax.
+
+## How to write the APIs
+
+### There are three major steps involved:
+    a) All APIs goes into the 's3' module of the OpenSDS multi-cloud package, i,e. opensds/multi-cloud/api/pkg/s3 directory
+    b) The corresponding router for the API are defined same directory.
+    c) For every API there must be a function defined in proto file.
+Example of adding API router (router.go):
+```
+func RegisterRouter(ws *restful.WebService) {
+	handler := NewAPIService(client.DefaultClient)
+	ws.Route(ws.POST("/{tenantId}/backends/{id}").To(handler.postBackend)).
+		Doc("post backend details")
+```
+
+#### Points to remember:
+   a) After adding a router to API just create the API file in the same directory.   
+   b) Try to call the API using curl or postman to see if the API is been called successfully.          
+   c) If your API has XML struct in Request body then unmarshal it into json struct
+
+Example of xml (xmlstruct.go):
+```
+type Rule struct {
+	ID                             string                         `xml:"ID"`
+	param1                         int32                          `xml:"param1"`
+	param2                         string                         `xml:"param2"`
+}
+```
+Example of json struct (in protobuf):
+```
+type LifecycleRule struct {
+	Id                             string              `protobuf:"bytes,1,opt,name=id,proto3" json:"id,omitempty"`
+	param1                         int32               `protobuf:"bytes,2,opt,name=param1,proto3" json:"param1,omitempty"`
+	param2                         string              `protobuf:"bytes,3,opt,name=param2,proto3" json:"param2,omitempty"`
+}
+```
+### Points to remember:
+    a) After unmarshalling into json struct and populating the fields, call the respective function from the s3 client and also in database.
+    b) Validate all the functions using log messages.
+    c) The API should successfully return and all the changes to be reflected in database should be seen in database client(for lifecycle feature the database is MongoDB)
+
+## How to start multi-cloud into the Docker container
+To start testing multi-cloud lifecycle feature, you need to first start multi-cloud service using steps below in the docker container.
+
+```
+docker-compose up -d
+docker ps
+```
+#### Check if multi-cloud api service and mongo is running:
+```
+CONTAINER ID        IMAGE                             COMMAND                  CREATED             STATUS              PORTS                                                NAMES
+c245b39b9a7d        opensdsio/multi-cloud-dataflow    "/dataflow"              6 hours ago         Up 6 hours                                                               multi-cloud_dataflow_1_b4432dc97238
+6bff89fd48b6        opensdsio/multi-cloud-datamover   "/datamover"             6 hours ago         Up 6 hours                                                               multi-cloud_datamover_1_6aa7bb101f33
+561d334d2ca8        opensdsio/multi-cloud-s3          "/s3"                    6 hours ago         Up 6 hours                                                               multi-cloud_s3_1_84b4c3f135b2
+a7288d668842        opensdsio/multi-cloud-api         "/api"                   6 hours ago         Up 6 hours          0.0.0.0:8089->8089/tcp                               multi-cloud_api_1_783889b02933
+247d70f62574        wurstmeister/kafka:2.11-2.0.1     "start-kafka.sh"         12 days ago         Up 6 hours          0.0.0.0:9092->9092/tcp                               multi-cloud_kafka_1_e666cc1a0f3b
+a2c136ae7421        opensdsio/multi-cloud-backend     "/backend"               12 days ago         Up 6 hours                                                               multi-cloud_backend_1_1983cccd8ac9
+eea40a48de97        wurstmeister/zookeeper            "/bin/sh -c '/usr/sb…"   12 days ago         Up 6 hours          22/tcp, 2888/tcp, 3888/tcp, 0.0.0.0:2181->2181/tcp   multi-cloud_zookeeper_1_6643a83ddcdd
+5dbac9cca10e        mongo                             "docker-entrypoint.s…"   12 days ago         Up 6 hours          0.0.0.0:27017->27017/tcp                             multi-cloud_datastore_1_a0281d684ef3
+```
+
+To check if the API changes are reflected in database, install mongoDB compass and check the metadata of the buckets
+
+![mongo connection image  ](opensds_mongo.PNG?raw=true)
+
+#### The postman request for API call:
+
+Here is a sample API call using postman, it contains request body information as well as response.
+
+![postman api call image  ](postman.png?raw=true)
+
+Check log files of multi-cloud api service for more details and debug information

content/guides/user-guides/object-lifecycle-management/_index.md

@@ -0,0 +1,106 @@
+---
+title: Object Lifecycle Management
+description: "User guide for Object Lifecycle Management."
+weight: 40
+disableToc: false
+tags: ["user guide", "object lifecycle"] 
+---
+
+## Introduction to Object lifecycle
+OpenSDS is aimed at addressing the storage integration challenges of both the Cloud Native environment and traditional IT environment. OpenSDS multi-cloud allows user to manage distributed cloud environment on a single platform. Object lifecycle management feature gives freedom to user to manage buckets and objects in the cloud using OpenSDS interface.
+
+## Getting Started!
+This is a guide that shows how to install, configure, and use Object lifecycle management feature in a simple OpenSDS setup.
+By following this guide you will be able to install OpenSDS multi-cloud with creating backends and buckets, uploading object and setting up the lifecycle for those objects.
+
+## Installing Opensds multi-cloud
+
+### Download opensds-installer code
+
+```cassandraql
+git clone https://github.com/opensds/opensds-installer.git
+cd opensds-installer
+# Checkout the latest release, give master by default or specify the branch 
+git checkout master
+```
+
+### How to enable multi-cloud installation
+
+Install docker container to run multi-cloud in container environment.
+
+Update the file : opensds-installer/ansible/group_vars/gelato.yml to install multi-cloud.
+```cassandraql
+# repository branch
+gelato_repo_branch: master
+```
+Detailed instruction is in this link
+
+- [OpenSDS Local Cluster Installation through Ansible On Ubuntu](https://github.com/opensds/opensds/wiki/OpenSDS-Cluster-Installation-through-Ansible) (Recommended)
+
+#### Export required OpenSDS  environment variables
+```
+export HOST_IP={your_real_host_ip}
+export OPENSDS_ENDPOINT=http://{{ apiserver_cluster_ip }}:50040
+export OPENSDS_AUTH_STRATEGY=keystone
+export OS_AUTH_URL=http://{{ authchecker_cluster_ip }}/identity
+```
+#### Run OpenSDS multi-cloud in docker container
+```
+docker-compose up -d
+docker ps 
+   ```
+Note: check if multi-cloud services are running in the docker container.
+```cassandraql
+# docker ps
+CONTAINER ID        IMAGE                             COMMAND                  CREATED             STATUS              PORTS                                                NAMES
+3222f0b4c25d        opensdsio/multi-cloud-dataflow    "/dataflow"              2 minutes ago       Up 2 minutes                                                             multi-cloud_dataflow_1_4a975c39f8ff
+33de3d47b1ef        opensdsio/multi-cloud-datamover   "/datamover"             2 minutes ago       Up 2 minutes                                                             multi-cloud_datamover_1_8ccb3d34f551
+3d3661466b3d        wurstmeister/kafka:2.11-2.0.1     "start-kafka.sh"         2 minutes ago       Up 2 minutes        0.0.0.0:9092->9092/tcp                               multi-cloud_kafka_1_e399b2c2167a
+e370acd6c922        mongo                             "docker-entrypoint.s…"   2 minutes ago       Up 2 minutes        0.0.0.0:27017->27017/tcp                             multi-cloud_datastore_1_a8c5edcc4e62
+6d5239b01503        wurstmeister/zookeeper            "/bin/sh -c '/usr/sb…"   2 minutes ago       Up 2 minutes        22/tcp, 2888/tcp, 3888/tcp, 0.0.0.0:2181->2181/tcp   multi-cloud_zookeeper_1_82daeecd7252
+7d9f01911356        opensdsio/multi-cloud-backend     "/backend"               2 minutes ago       Up 2 minutes                                                             multi-cloud_backend_1_b9cbaa8bcf83
+f139d2e383ab        opensdsio/multi-cloud-s3          "/s3"                    2 minutes ago       Up 2 minutes                                                             multi-cloud_s3_1_3c7878089fa4
+```
+
+## Open OpenSDS Object lifecycle management UI
+
+OpenSDS Object lifecycle management feature specific UI can be accessed via
+
+`http://{your_host_ip}:8088/#/home`
+
+login  using the default admin credentials: `admin/opensds@123`. 
+
+![multi-cloud UI image  ](opensds_home.PNG?raw=true)
+
+### Create  new backend  using remote cloud bucket
+Click on (+) for registering a storage backend
+
+![multi-cloud backend image  ](opensds_backend.PNG?raw=true)
+
+### Create bucket and upload the object
+After creating new backend follow the steps given below to create a bucket and upload an object in that bucket:
+1. Launch Resource from OpenSDS home page
+
+2. Create a new bucket with appropriate backend
+
+	![multi-cloud bucket image  ](opensds_bucket.PNG?raw=true)
+3. Click on upload button to upload an object in the selected bucket
+
+	![multi-cloud object image  ](opensds_object.PNG?raw=true)
+4. Click on lifecycle tab and create a new lifecycle rule
+
+	![multi-cloud lifecycle image  ](opensds_lifecycle.PNG?raw=true)
+
+After creating the lifecycle rule user will be able to see the lifecycle configuration on bucket.
+
+![multi-cloud lifecycle config image  ](opensds_lifecycle_config.PNG?raw=true)
+
+Save dashboard and return.
+### Check lifecycle in remote cloud service provider
+After configuring lifecycle in dashboard, a routine scheduler will run and it will wait till the date lifecycle rule comes in-effect. Once the action is performed user can log in to remote cloud ( backend cloud service provider) and check the object status.
+
+### FAQ
+1. How to check if the lifecycle rule is applied on bucket/object ?
+
+Ans: 
+OpenSDS does not create the rule in the cloud backend (CSP). Once the bucket/object satisfies the rule, OpenSDS will automatically call cloud API and the action will be performed. If there are any errors or issues in rule, user will know at the time of lifecycle action.