-
Using the new
Rest
DSL andElasticsearch
component of Apache Camel, expose CRUD services for a repository of blog articles. -
Secure RESTful endpoints using
APIMan
andKeycloak
according to different scenario : basic authentication, Oauth2 and Oauth2 & roles. -
Populate and query an Elasticsearch database using the
Elasticsearch
component of Apache Camel. -
Analyze the Elasticsearch data using
Kibana
. -
Document the RESTful endpoints using Swagger API.
This JBoss Fuse demo will demonstrate how the new Apache Camel REST DSL syntax can be used to expose REST services. These REST services will use to add, search or delete blog articles in an Elasticsearch database. The use case has been enriched to also expose a file endpoint:
The file endpoint is responsible for polling a folder, consuming csv files in that folder and inserting all the articles into the database. The Apache Camel Elasticsearch component is called from different routes and will communicate with the ElasticSearch Database to perform the CRUD operations. The data can be visualized using a Kibana dashboard. Swagger API will be used to document the RESTFull services. Finally, we will secure the endpoints using Apiman & Keycloak projects using Basic authentication, Oauth2 or Oauth2 with roles.
The Elasticsearch database and the Apache Camel projects will be deployed into different Fuse Managed Containers operated by a JBoss Fuse Fabric Server.
A kibana dashboard will be used to analyze the Blog articles published into the database
The CRUD services can be accessed using the new Camel REST component by performing HTTP requests (GET/PUT/DELETE) while the file component will also allow to do a bulk insert of blog articles.
By example, to use the REST Service responsible to insert an article within the Elasticsearch database, a JSON article { "user": "cmoulliard" "postDate": "2015-12-12", "body": "Integration is hard.", "title": "On distributed search" }
message is issued against the REST endpoint /blog/article/id
using a HTTP GET operation.
The content of this HTTP request will be processed by an Apache Camel route as showed hereafter and transformed in order to access the elasticsearch component to call the Elasticsearch database.
rest("/blog/").id("rest-blog-service").produces("application/json").consumes("application/json")
.put("/article/{id}").id("rest-put-article").type(Blog.class)
.to("direct:add");
JacksonDataFormat jacksondf = new JacksonDataFormat(Blog.class);
from("direct:add").id("add-direct-route")
.log(LoggingLevel.INFO,"Add new Blog entry service called !")
.setHeader(ElasticsearchConfiguration.PARAM_INDEX_NAME).simple("{{indexname}}")
.setHeader(ElasticsearchConfiguration.PARAM_INDEX_TYPE).simple("{{indextype}}")
.setHeader(ElasticsearchConfiguration.PARAM_OPERATION).constant(ElasticsearchConfiguration.OPERATION_INDEX)
// Transform Java Object to JSON
.marshal(jacksondf)
// Call the add service of the elasticsearchService POJO to generate the IndexRequest object
.beanRef("elasticSearchService", "add")
// Call the elasticsearch Service to add/insert an entry within the index
.to("elasticsearch://{{clustername}}?ip={{address}}")
// Response received and returned to the REST endpoint
.log("Response received : ${body}");
The following table summaries the Routes used, REST Services exposed and the Bean Methods called like the Object type used to call the ElasticSearch database
Rest URl |
Operation |
Route |
Bean method |
ElasticSearch Class Type |
/blog/article/ |
PUT |
AddArticleToElasticRoute |
ElasticSearchService.add() |
IndexRequest |
blog/article/search/user/{user} |
GET |
SearchArticleToElasticRoute |
ElasticSearchService.getBlog() |
GetResponse |
blog/article/search/id/{id} |
GET |
SearchArticleToElasticRoute |
ElasticSearchService.getBlogs() |
SearchResponse |
blog/article/{id} |
DELETE |
DeleteArticleToElasticRoute |
ElasticSearchService.() |
DeleteRequest |
To do this bulk import, you will create a CSV file containing this record structure id,user,blog description,title
that you can define several times into the file.
All the records will be uploaded by the file endpoint, transformed using the Apache Camel Bindy
Dataformat to a collection of Blog objects.
Next, each Blog object will be used as input object to issue a request to insert a new record within the Elasticsearch database using the Bean ElasticSearch.
from("{{fileUri}}")
.log(LoggingLevel.DEBUG,"Records received : ${body}")
.unmarshal(csv)
.split(body())
.setHeader("id").simple("${body.id}")
.to("direct:add");
JacksonDataFormat jacksondf = new JacksonDataFormat(Blog.class);
from("direct:add").id("add-direct-route")
.log(LoggingLevel.INFO,"Add new Blog entry service called !")
.setHeader(ElasticsearchConfiguration.PARAM_INDEX_NAME).simple("{{indexname}}")
.setHeader(ElasticsearchConfiguration.PARAM_INDEX_TYPE).simple("{{indextype}}")
.setHeader(ElasticsearchConfiguration.PARAM_OPERATION).constant(ElasticsearchConfiguration.OPERATION_INDEX)
// Transform Java Object to JSON
.marshal(jacksondf)
// Call the add service of the elasticsearchService POJO to generate the IndexRequest object
.beanRef("elasticSearchService", "add")
// Call the elasticsearch Service to add/insert an entry within the index
.to("elasticsearch://{{clustername}}?ip={{address}}")
.log("Response received : ${body}");
-
JDK 1.7 or 1.8
-
Apache Maven 3.2.3
The following is an overview of the installation steps involved with this project :
-
Download and compile the project locally
-
Edit the hosts file to add an entry
-
Download and install the JBoss Fuse Server
-
Edit the etc/users.properties file of the server to define an admin user
-
Create a Fuse Fabric Server and 2 child containers
-
Deploy the Fabric project into the Fabric Server
-
Run a Web Container to provide access to the Kibana dashboard
-
Play with the demo
-
Install, configure Apiman & Keycloak servers to secure the REST Services and provide service governance rule
-
Open a Windows or Unix terminal.
-
Using the
git
utility, clone this project to your local workstation.NoteFor the purposes of these lab instructions, the directory of this cloned project on your local workstation will be referred to as: $DEMO_DIRECTORY
. -
Change directory into:
$DEMO_DIRECTORY
and build the maven project:cd rest-dsl-in-action mvn clean install
Ensure that your $HOSTNAME environment variable on your workstation maps to either:
-
Your local loopback address (127.0.01) when NOT connected to a network OR
-
The ip address of the network your workstation is currently connected to (wifi, ethernet, etc).
On Unix type operating systems, these networking modifications can be made in: /etc/hosts
.
The workstation’s hostname
is used by the following components:
-
The ElasticSearch server uses the
hostname
to bind its socket server upon start up. -
The Apache Camel Elasticsearch component uses the
hostname
to access the Elasticsearch server. -
The Fuse Fabric Server uses the
hostname
to assign an IP address to the socket server of the Apache Zookeeper server.
-
Download JBoss Fuse Server and unzip the archive file.
NoteFor the purposes of this lab, the directory created from having unzipped the JBoss Fuse
zip archive will be referred to as$FUSE_HOME
. -
Edit the
etc/user.properties
file found in the home directory ofjboss-fuse-6.2.0.redhat-xxx
Uncomment the line containing the
admin
user and save the file.admin=admin,admin,manager,viewer,Monitor, Operator, Maintainer, Deployer, Auditor, Administrator, SuperUser
-
Open a second Windows or Unix terminal and change directory to the root of the JBoss Fuse installation.
-
Execute the following command to make all Fuse shell scripts executable:
chmod 755 bin/*
-
Execute the following command to launch the JBoss Fuse Server:
./bin/fuse
-
Next, within the Fuse Karaf console, issue the following shell command:
shell:source mvn:org.jboss.fuse/deployment/1.0/script/install
NoteThe
install
shell script contains some Fabric subshell commands responsible for setup of a Fabric server and two containers (elasticsearch-node and demo). The first child container will be used as the Elasticsearch database server. The second Fuse managed container is used to run the Apache Camel Routes and expose the REST and File endpoints.$JBOSS_FUSE_INSTALL/bin/fuse Please wait while JBoss Fuse is loading... 100% [========================================================================] _ ____ ______ | | _ \ | ____| | | |_) | ___ ___ ___ | |__ _ _ ___ ___ _ | | _ < / _ \/ __/ __| | __| | | / __|/ _ \ | |__| | |_) | (_) \__ \__ \ | | | |_| \__ \ __/ \____/|____/ \___/|___/___/ |_| \__,_|___/\___| JBoss Fuse (6.2.0.redhat-133) http://www.redhat.com/products/jbossenterprisemiddleware/fuse/ Hit '<tab>' for a list of available commands and '[cmd] --help' for help on a specific command. Open a browser to http://localhost:8181 to access the management console Create a new Fabric via 'fabric:create' or join an existing Fabric via 'fabric:join [someUrls]' Hit '<ctrl-d>' or 'osgi:shutdown' to shutdown JBoss Fuse. JBossFuse:karaf@root>shell:source mvn:org.jboss.fuse/deployment/1.0/script/install ...
After a few moments, the server will report on the console that the Fabric Server and the 2 child containers have been created.
Waiting for container: root Waiting for container root to provision. Creating new instance on SSH port 8102 and RMI ports 1100/44445 at: /Users/chmoulli/Fuse/Fuse-servers/jboss-fuse-6.2.0.redhat-133/instances/elasticsearch-node The following containers have been created successfully: Container: elasticsearch-node. Creating new instance on SSH port 8103 and RMI ports 1101/44446 at: /Users/chmoulli/Fuse/Fuse-servers/jboss-fuse-6.2.0.redhat-133/instances/lab The following containers have been created successfully: Container: demo.
-
Verify that the 2 containers are running by issuing the command:
fabric:container-list
.Inspect the column
connected
which represents the state of the Fuse OSGI containers. If the status is equal toyes
, then the container has been created successfully. Theprovision status
column reports the status of the provisioning of the container. If the status is equal tosuccess
, that means that the server has been packaged with the required OSGI bundles, Config Properties files, …JBossFuse:karaf@root>fabric:container-list [id] [version] [type] [connected] [profiles] [provision status] root* 1.0 karaf yes fabric success fabric-ensemble-0000-1 jboss-fuse-full elasticsearch-node 1.0 karaf yes insight-elasticsearch.datastore success demo 1.0 karaf yes feature-camel success
-
In a terminal window, change to the
routing
directory of this project:rest-dsl-in-action/routing
-
Execute the following command:
mvn fabric8:deploy
Doing so will deploy the
demo
project configuration into thegpe-fuse
profile of the Fabric Server.-
If Fuse has not been previously run on your workstation, it’s likely that the following prompt will appear:
There is no <server> section in your ~/.m2/settings.xml file for the server id: fabric8.upload.repo You can enter the username/password now and have the settings.xml updated or you can do this by hand if you prefer. Would you like to update the settings.xml file now? (y/n): y Please let us know the login details for this server: fabric8.upload.repo Username: admin Password: Repeat Password: Copied original: /home/jbride/.m2/settings.xml to: /home/jbride/.m2/settings.xml.backup-1.xml Updated settings file: /home/jbride/.m2/settings.xml
If prompted as per above, respond with a
y
(to allow for a modification to your~/.m2/settings.xml
) and use username / password credentials ofadmin / admin
(as per the default in$FUSE_INSTALL/etc/user.properties
). -
Once deployment is complete, you should see output similar to the following:
$ mvn fabric8:deploy [INFO] Uploading file /Users/chmoulli/.m2/repository/com/redhat/gpe/routing/1.0/routing-1.0.jar Uploading: http://127.0.0.1:8181/maven/upload/com/redhat/gpe/routing/1.0/routing-1.0.jar Uploaded: http://127.0.0.1:8181/maven/upload/com/redhat/gpe/routing/1.0/routing-1.0.jar (18 KB at 575.9 KB/sec) Uploading: http://127.0.0.1:8181/maven/upload/com/redhat/gpe/routing/1.0/routing-1.0.pom Uploaded: http://127.0.0.1:8181/maven/upload/com/redhat/gpe/routing/1.0/routing-1.0.pom (7 KB at 275.5 KB/sec) [INFO] Updating profile: gpe-fuse with parent profile(s): [feature-camel] using OSGi resolver [INFO] About to invoke mbean io.fabric8:type=ProjectDeployer on jolokia URL: http://localhost:8181/jolokia with user: admin [INFO] [INFO] Profile page: http://127.0.0.1:8181/hawtio/index.html=/wiki/branch/1.0/view/fabric/profiles/gpe/fuse.profile [INFO] [INFO] Uploading file org.jboss.fuse.demo.properties to invoke mbean io.fabric8:type=Fabric on jolokia URL: http://localhost:8181/jolokia with user: admin [INFO] Uploading file fuse-lab1.png to invoke mbean io.fabric8:type=Fabric on jolokia URL: http://localhost:8181/jolokia with user: admin [INFO] Uploading file Readme.md to invoke mbean io.fabric8:type=Fabric on jolokia URL: http://localhost:8181/jolokia with user: admin [INFO] Performing profile refresh on mbean: io.fabric8:type=Fabric version: 1.0 profile: gpe-fuse [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------
-
-
To verify that the profile has been created and published on JBoss Fuse, use the following command within the JBoss Fuse Karaf Console:
JBossFuse:karaf@root> fabric:profile-display gpe-fuse Profile id: gpe-fuse Version : 1.0 Attributes: abstract: false parents: feature-camel Containers: Container settings ---------------------------- Features : camel-jetty camel-http4 camel-jackson camel-elasticsearch camel-bindy Bundles : mvn:org.apache.servicemix.bundles/org.apache.servicemix.bundles.json/20140107_1 mvn:org.jboss.fuse/routing/1.0 Agent Properties : lastRefresh.gpe-fuse = 1442824458002 Configuration details ---------------------------- PID: io.fabric8.web.contextPath org.jboss.fuse/routing routing PID: org.jboss.fuse.demo port 9200 indextype post address localhost clustername insight fileUri file://articles?noop=true indexname blog Other resources ---------------------------- Resource: Readme.md Resource: dependencies/org.jboss.fuse/routing-requirements.json Resource: fuse-lab1.png
-
Afterwards, install the
gpe-fuse
profile into thedemo
Fuse Demo Managed container-
Assign the
gpe-fuse
profile to ourdemo
container by executing the following fabric command within the JBoss Fuse Console:fabric:container-add-profile demo gpe-fuse
-
Verify that the container has been successfully provisioned and that the
demo
container contains thefeature-camel
andgpe-fuse
profiles. To do so, execute the following in the Fuse Console:fabric:container-list
JBossFuse:karaf@root> fabric:container-list [id] [version] [type] [connected] [profiles] [provision status] root* 1.0 karaf yes fabric success fabric-ensemble-0000-1 jboss-fuse-full elasticsearch-node 1.0 karaf no insight-elasticsearch.datastore success demo 1.0 karaf no feature-camel success gpe-fuse
NoteIf you change the code of this demo, then you can redeploy by executing a
mvn clean install
followed by themvn fabric8:deploy
command. Next, remove and add again the profile to the Fuse Container using these Fabric commandsfabric:container-remove-profile demo gpe-fuse fabric:container-add-profile demo gpe-fuse
-
In order to collect the data using the ElasticSearch No SQL database, an index with a name of blog
and type article
needs to be created.
In addition, the properties of the new type article
need to be defined.
To do so, the following JSON definition will be used:
{ "article": { "properties": { "user": { "type": "string" }, "title": { "type": "string" }, "postDate": { "type": "date", "format": "yyyy-MM-dd'T'HH:mm" }, "body": { "type": "string" }, "id": { "type": "integer" } } } }
The tasks in this section of the lab will be executed by invoking the RESTful services of the Elasticsearch
container managed by Fuse Fabric.
-
In a terminal window, change directory to
$DEMO_DIRECTORY
. -
Execute the following 3 commands using the either the
http
orcurl
utilities:-
http
:http PUT http://localhost:9200/blog http PUT http://localhost:9200/blog/_mapping/article < data/elasticsearch/mapping.json http http://localhost:9200/blog/_mapping/article
-
or
curl
:curl -X PUT http://localhost:9200/blog curl -X PUT http://localhost:9200/blog/_mapping/article -d @data/elasticsearch/mapping.json curl http://localhost:9200/blog/_mapping/article
NoteThe first command creates the index and the second command defines the properties of the new type article
. The third command confirms that the mapping between thearticle
type and its properties have been created.
-
The data inserted into the Elasticsearch Database can be analyzed using the Kibana dashboard. The dashboard is designed around modern HTML5 Web technologies and can be run locally with a Web Container.
This demo provides a maven goal that launches Kibana embedded in a Jetty
HTTP Server.
Kibana can then be accessed from a browser at the following address: http://localhost:9090/kibana3/index.html
-
To start Kibana in your local workstation, open a terminal window and change to the
$DEMO_DIRECTORY/kibana
directory. -
Execute the following maven command:
mvn jetty:run
mvn jetty:run ... [INFO] Configuring Jetty for project: FuseByExample :: REST DSL Demo :: Kibana 3 [INFO] Webapp source directory = /Users/chmoulli/Downloads/rest-dsl-in-action-master/kibana/src/main/webapp [INFO] Reload Mechanic: automatic [INFO] Classes directory /Users/chmoulli/Downloads/rest-dsl-in-action-master/kibana/target/classes does not exist [INFO] Context path = /kibana3 [INFO] Tmp directory = /Users/chmoulli/Downloads/rest-dsl-in-action-master/kibana/target/tmp [INFO] Web defaults = org/eclipse/jetty/webapp/webdefault.xml [INFO] Web overrides = none [INFO] web.xml file = file:///Users/chmoulli/Downloads/rest-dsl-in-action-master/kibana/src/main/webapp/WEB-INF/web.xml [INFO] Webapp directory = /Users/chmoulli/Downloads/rest-dsl-in-action-master/kibana/src/main/webapp 2015-09-21 11:46:46.627:INFO:oejs.Server:main: jetty-9.3.0.M1 2015-09-21 11:46:48.507:INFO:oejsh.ContextHandler:main: Started o.e.j.m.p.JettyWebAppContext@70325d20{/kibana3,file:///Users/chmoulli/Downloads/rest-dsl-in-action-master/kibana/src/main/webapp/,AVAILABLE}{file:///Users/chmoulli/Downloads/rest-dsl-in-action-master/kibana/src/main/webapp/} 2015-09-21 11:46:48.710:INFO:oejs.ServerConnector:main: Started ServerConnector@53afb8df{HTTP/1.1,[http/1.1]}{localhost:9090} 2015-09-21 11:46:48.711:INFO:oejs.Server:main: Started @4174ms [INFO] Started Jetty Server
-
Once the server is launched, open your browser and navigate to:
http://localhost:9090/kibana3/index.html
.NoteThe first time connecting to the Dashboard, the index HTML page will display general content about the kibana project, how to setup a dashboard and how to import data from an Elasticsearch Database.
A Kibana dashboard has been created and is provided in this project.
This dashboard is already configured with histogram
and list
panels.
The dashboard included in this project needs to be loaded into the Elasticsearch HTTP server.
This can be done via the kibana-int
REST service running in JBoss Fuse.
-
In a terminal window, change to the
$LAB_ASSETS
directory. -
Execute one of the following commands to post the new dashboard to the Elasticsearch HTTP server:
-
http
utility:http PUT http://localhost:9200/kibana-int/dashboard/fusedemo < data/elasticsearch/dashboard.json
-
curl
utility:curl -X PUT http://localhost:9200/kibana-int/dashboard/fusedemo -d @data/elasticsearch/dashboard.json
-
-
To view the new dashboard, refresh
Kibana
home page within your browser and select theload
button from the top menu bar: -
Select the
fuse-demo
dashboard:Currently, the
fuse-demo
dashboard contains panels without any data as we haven’t yet populated the database with records.
This section of the lab provides instructions on how to add a new blog and modify data maintained in Elasticsearch and exposed through Kibana. The commands in this section of the lab invoke RESTful camel services (exposed via port 9191) running in a Fuse container managed by Fabric.
-
In an open terminal window, change directory to
$DEMO_DIRECTORY
. -
Add a blog
-
Issue one of the following HTTP requests using either the
http
orcurl
utilities:http PUT http://localhost:9191/blog/article < data/elasticsearch/entry.json
curl -X PUT http://localhost:9191/blog/article -d @data/elasticsearch/entry.json
NoteBefore issuing the HTTP GET request, the content of the blog article can be modified by editing the file: data/elasticserch/entry.json
-
-
Search the user
cmoulliard
http http://localhost:9191/blog/article/search/user/cmoulliard
curl http://localhost:9191/blog/article/search/user/cmoulliard
-
Search the user which has been encoded with the id
0
http http://localhost:9191/blog/article/search/id/0
curl http://localhost:9191/blog/article/search/id/0
-
Delete a user.
http DELETE http://localhost:9191/blog/article/0
curl -X DELETE http://localhost:9191/blog/article/0
In order to perform a buk import of articles within the ElasticSearch database, copy/paste the records.csv file from the $DEMO_DIRECTORY/camel/src/data
directory to the articles
directory created under the local instance demo
.
-
Create the
FUSE_HOME
ENV variable to point to the home directory of the installation directory of JBoss Fuseexport FUSE_HOME=/Users/chmoulli/Fuse/Fuse-servers/jboss-fuse-6.2.0.redhat-133
-
Create the
DEMO_DIRECTORY
ENV variable to point to the folder containing the demo projectexport DEMO_DIRECTORY=/Users/chmoulli/Downloads/rest-dsl-in-action
-
Copy the file
cp $DEMO_DIRECTORY/routing/src/data/articles/records.csv $FUSE_HOME/instances/demo/articles/
NoteThe
articles
directory folder is scanned by the Apache Camel roteFileToAddServiceRoute
-
In your browser, refresh the
kibana
dashboard to see all of the newly imported data.
The Camel REST services included in this project have been documented using the Swagger API.
The documentation is defined within the swagger/src/main/resources/services.json
file of the swagger maven module.
Note
|
This swagger file has been produced manually. The camel-swagger component (version 2.15) packaged within JBoss Fuse 6.2 doesn’t yet parse the REST DSL syntax to automatically generate the REST documentation. This feature should be available in the next release of JBoss Fuse. |
To access the Swagger documentation, execute the following:
-
In a terminal window, change to the
$DEMO_DIRECTORY/swagger
directory. -
Launch the script ./run_jetty.sh to build the project locally and start a jetty instance with Swagger UI
./run_jetty.sh
-
When the jetty instance is started, open your browser at this address
http://localhost:8000/?url=services.json
-
You can now add a user by clicking on the service
Add a new Blog Article
. -
To select the user data, just click on the Model schema and the field (left part of the service
Add a new Blog Article
will be populated. -
Change the data to be used
{ "id": "21", "user": "cmoulliard", "body": "This is a blog article", "title": "Title of the blog article", "postDate": "2015-10-03T10:10" }
-
Click on
try it out!
button -
Search for a user using the service
Blog Search Id operation
and fill the parameter with the id21
-
Review the results using the
Kibana
dashboard and search for the id which is equal to21
During the first part of this demo, Apache Camel Routes exposing the REST services have been deployed.
In this section of the lab, ApiMan & Keycloak are used to secure these REST endpoints. In particular, incoming HTTP requests will be authenticated and authorization will be granted based on the role of the authenticated user.
An overview of the installation instructions for APIMan are as follows:
-
Download Wildfly and unzip
-
Download APIMan and unzip inside the wildfly directory
-
Start WildFly 8 using the
standalone-apiman.xml
configuration file -
Point your browser at the apiman UI and login using
admin
as user andadmin123!
as passwordNoteThe following are specific commands that can be used to install APIMan:
mkdir ~/apiman-1.1.9.Final cd ~/apiman-1.1.9.Final curl http://downloads.jboss.org/wildfly/8.2.0.Final/wildfly-8.2.0.Final.zip -o wildfly-8.2.0.Final.zip curl http://downloads.jboss.org/apiman/1.1.9.Final/apiman-distro-wildfly8-1.1.9.Final-overlay.zip -o apiman-distro-wildfly8-1.1.9.Final-overlay.zip unzip wildfly-8.2.0.Final.zip unzip -o apiman-distro-wildfly8-1.1.9.Final-overlay.zip -d wildfly-8.2.0.Final cd wildfly-8.2.0.Final export JBOSS_HOME=$(pwd) ./bin/standalone.sh -c standalone-apiman.xml
-
In your browser, navigate to the following URL and authenticate in using the credentials:
admin / admin123!
. -
Once authenticated, your browser will be redirected to the
Api Management screen
:
Now that the Apiman server has been launched, an organization
is needed to manage the different Camel REST services that need to be secured.
-
In the
Organizations
section of theAPIMan
home page, click on theCreate a New Organization
link. -
Enter
Fuse
in theOrganization Name
text box. -
Click on the
Create Organisation
button. -
The Web UI will refresh and a new organization called
Fuse
will appear in theHome
screen -
Click on the
Services
Tab of theFuse
organization to register the first REST Service to be secured.NoteAs we will not define different SLA (e.g. quotas, …) to manage the service, No plan will be defined for the demo
The first use case that we will deal with is very simple and can be compared to a proxy forwarding scenario: the incoming request is matched to a service and then forwarded to the real endpoint exposing the REST service. To design it, we will create a service, specify the address of the service and publish the service in order to let APiMan Gateway to accept the HTTP requests.
-
In the
Services
screen of theFuse
organization, click the button:New Service
. -
Within
New Service
screen, addblog-service
as Service Name and provide adescription
to mention that this service is not secured "Insecure service. No authentication will be required to access the service". When done, click on the Create Service` button -
You will be redirected to the
blog-service
, version1.0
screen where we will add the URL of the REST Service exposed by the Apache Camel REST DSL and specify that the service is public. -
Click on the
implementation
link and add the Api endpointhttp://localhost:9191/blog/article
, selectREST
from theApi Type
dropdown box. The API endpoint corresponds to our REST Service path as defined within the REST DSLblog/article
. -
Click on the
Save
button -
From the
blog-service
screen, select thePlans
and click within the checkboxMake this service public
as we haven’t defined any plans. -
Save the service definition by clicking again on the
save
button. -
Our service is ready and we will publish it.
NoteWhen a service is published, then the Api endpoint will become available and the Api Gateway will be able to process HTTP requests addressed to this service. -
Click on the
Publish
button of theblog-service
screen.NoteThe screen is refreshed and additional information appears under the list like Contracts, Endpoint, Activity, Metrics
-
Click on
Endpoint
to see the URL address of the endpoint exposed by the Api Gateway and select it -
Use one of the existing opened terminal and issue a HTTP REST request to search about a user
http --verify=no https://localhost:8443/apiman-gateway/Fuse/blog-service/1.0/search/user/cmoulliard curl -k https://localhost:8443/apiman-gateway/Fuse/blog-service/1.0/search/user/cmoulliard
NoteRemark that we have added at the end of the Api endpoint the path of search/user/${username}` service WarningWe will use the verify=no option of httpie to skip the host’s SSL certificate verification or -k of curl -
Check the response
http --verify=no https://localhost:8443/apiman-gateway/Fuse/blog-service/1.0/search/user/cmoulliard HTTP/1.1 200 OK Accept: */* Accept-Encoding: gzip, deflate Connection: keep-alive Content-Type: application/json Date: Thu, 03 Sep 2015 13:00:34 GMT Server: Jetty(8.1.17.v20150415) Transfer-Encoding: chunked User-Agent: HTTPie/0.9.2 X-Powered-By: Undertow/1 breadcrumbId: ID-dabou-local-62090-1441284856992-0-11 indexName: blog indexType: post user: cmoulliard [ { "body": "Integration is hard. Integration should be easy.", "id": "1", "postDate": "2015-08-10T10:10", "title": "On distributed search", "user": "cmoulliard" } ]
Based on the previous service created, we will define a new security policy to be able to authenticate the HTTP request using Basic authentication as the previous was too insecure. From the service version 1.0 created, we will create a new (= clone the previous and extend the features) service and add a Basic authentication policy. The credentials will be managed locally using a static list of users and passwords.
The steps needed to realize the use case are described hereafter
-
First, we will duplicate the existing service to create a new. So, from the blog-service web page, click on the button
New Version
-
Encode
2.0
as Version number and click on the buttoCreate Version
-
When the screen has been refreshed. Click on the
Add policy
button which is available under the policies tab. Click on it. -
Select
Basic Authentication Policy
within the dropdownPolicy Type
list -
Add
blog-basic-auth
as Authentication Realm. Click within the checkboxesTransport security required
andBasic Auth required
-
Choose as Identity Source,
static
within the dropdown list -
Add a user and password (example : charles/demo)
-
Save the policy by clikcing on the
Add policy
button. Then, you will return to the service definition screen where you will publish the service. -
Click on the
Publish
button to publish it.NoteCheck the url of the endpoint to be used to access the new service -
Issue a HTTP Request from a terminal to access the REST search service
http --verify=no https://localhost:8443/apiman-gateway/Fuse/blog-service/2.0/search/user/cmoulliard WWW-Authenticate:'BASIC realm=blog-basic-auth' HTTP/1.1 401 Unauthorized Connection: keep-alive Content-Length: 168 Content-Type: application/json Date: Wed, 09 Sep 2015 10:01:54 GMT Server: WildFly/8 WWW-Authenticate: BASIC realm="blog-basic-auth" X-Policy-Failure-Code: 10004 X-Policy-Failure-Message: BASIC authentication failed. X-Policy-Failure-Type: Authentication X-Powered-By: Undertow/1 { "failureCode": 10004, "headers": { "WWW-Authenticate": "BASIC realm=\"blog-basic-auth\"" }, "message": "BASIC authentication failed.", "responseCode": 0, "type": "Authentication" } curl -k https://localhost:8443/apiman-gateway/Fuse/blog-service/2.0/search/user/cmoulliard {"type":"Authentication","failureCode":10004,"responseCode":0,"message":"BASIC authentication failed.","headers":{"WWW-Authenticate":"BASIC realm=\"blog-basic-auth\""}}
WarningThe request fails as we haven’t passed the credentials o the user to be used. We receive as response Basic authentication failed
for the realmblog-basic-auth
-
Run a new HTTP request where we will pass as parameters the username and password to be authenticated.
http --verify=no -a charles:demo https://localhost:8443/apiman-gateway/Fuse/blog-service/2.0/search/user/cmoulliard curl --user charles:demo -k https://localhost:8443/apiman-gateway/Fuse/blog-service/2.0/search/user/cmoulliard
The last use case that we will cover Oauth2 & Openid-connect authentication & authorization. Basically, ApiMan will be used as the platform issuing a request against the OAuth2 authorization server to verify that the token send by an application is authorized to access the REST service. For that purpose, an OpenID JSON Token will be generated by apiman and returned to the application interested to access the real service. Based on the information provided within the token like the role, the OAuth2 platform will verify the request and determine if the application is authorized to access the service according to the credentials provided and role.
In order to use Oauth2 with Apiman, we will first install a new plugin as the Oauth2 plugin is not installed per default. Here are the steps required to install it
-
Return to the home page of the ApiMan Management Server and click on the link
Manager plugins
of the System Administration section. -
Next, click on the
Add Plugin
button in order to pass the information required to install the plugin -
Fill the fields using these values and click on the`Add plugin` button when this is done
Group Id : io.apiman.plugins Artifact ID : apiman-plugins-keycloak-oauth-policy Version : 1.1.8.Final
-
When we will return to the plugins screen, we can see that a new plugin entry has been added
Like we have done for the basic authentication with Apiman, we will first create a realm that the OAuth2 server will use to manage the credentials, the roles and the applications authorized to access the service. To achieve this goal, we will first configure Keycloak which is the Oauth2 server used by Apiman behind the scene to authorize the applications.
-
First, log on to the KeyCloak server usign the username
admin
and passwordadmin123!
at this addresshttp://localhost:8080/auth/admin
-
If the authentication succeeds, then you will access the Management console from where we will create a new Realm
-
Click on the button
add Realm
which is displayed at the top part of the screen -
Next add
Fuse
as name within the screenAdd Realm
and save the result by clicking on thesave
button -
From the settings Fuse screen, select (if not yet done) the
Direct Grant Api
by clicking on theon/off
button that we have within theLogin
tab and save the action done. -
Next, we will define the application which is a client which is authorized to access the fuse realm. So click on the tab
Clients
and click on the buttonCreate
-
Add within the fied
Client ID
,fuse
as name of the application that we will authorize -
Change next
Direct Grant Api
by clicking on theon/off
button. This step is required to allow REST Apiman to use the REST APi to access the Realm managed by the Oauth2 server. TheClient Protocol
that we will use isopenid-connect
and the access type ispublic
. Save the result by clicking on the buttonSave
. -
As an application (or client) will be authenticated by the Oauth2 Server, we will now define an
admin
user and setup its password to access the platform. So, select the tabUsers
and click on theAdd user
button -
Fill the fields
Username
andFirst Name
with the valueadmin
and save the modifications done by clicking on theSave
button -
As the platform generates a password, we will change it to use our own password. So, click on the
Credentials
tab and encode theadmin
password within the fieldsNew password
andPassword confirmation
. Disable the optionTemporary
by clicking on theon/off
button
We have finished to create the REALM within Keycloak Server as also the client ID auhorized to access the platform and finally we have define a admin user like its password. The next
step will consist to create a new version fo the service like we did for the Basic authentication but now using Oauth Policy.
So, from the home page of the Apiman Management Console, seclect the service version 2.0 of Blog-service
-
Duplicate the service
version 2
by clicking on the buttonNew Version
and when the screen is refreshed, seclectPolicies
tab to remove the basic authentication policy defined -
Add a new policy by clicking on the button
Add policy
. When the screenAdd policy
appears, selectKeycloak Oauth Policy
within the dropdown list of the policy type. -
Encode the endpoint of the REST api which is authorized to access the Oauth2 Server to request a Oauth2 Token. So, fill the field
Realm
with this urlhttp://localhost:8080/auth/realms/fuse
and add to the fieldKeycloak Realm Certificate
the certificate which has been generated for theFuse
realm. The certificate can be retrieved from the Oauth2 Server as explained just after. -
Copy / paste the certificate that you can get from the Keycloak server (
http://localhost:8080/auth/admin/master/console/#/realms/fuse/keys-settings
) under the screenSettings
,Keys
tab -
Finally, select
true
as the value of the field Forward Realm Roles` to send to the application, the roles defined within the realm. They will be used within the last use cases to control if the application calling the Oauth2 server is authorized according to its role to use a specific service. -
Save the policy and the service version
3.0
. -
Publish the service like we have done before
-
When the service has been published, you will use a bash script responsible to call the OAuth2 server to get a token that next the application will use to issue the CRUD requests against the Api Gateway
#!/usr/bin/env bash REALM=fuse USER=admin PASSWORD=admin CLIENT_ID=fuse HOST=localhost PORT_HTTP=8080 PORT_HTTPS=8443 auth_result=$(http --verify=no -f http://$HOST:$PORT_HTTP/auth/realms/$REALM/protocol/openid-connect/token username=$USER password=$PASSWORD grant_type=password client_id=$CLIENT_ID) access_token=$(echo -e "$auth_result" | awk -F"," '{print $1}' | awk -F":" '{print $2}' | sed s/\"//g | tr -d ' ') APIGATEWAY=https://$HOST:$PORT_HTTPS/apiman-gateway ORG=fuse SERVICE=blog-service VERSION=3.0 URL=$APIGATEWAY/$ORG/$SERVICE/$VERSION echo ">>> Token query" #echo "curl -X POST http://127.0.0.1:8080/auth/realms/$REALM/protocol/openid-connect/token -H 'Content-Type: application/x-www-form-urlencoded' -d 'username=$USER' -d 'password=$PASSWORD' -d 'grant_type=password' -d 'client_id=$USER'" echo "http --verify=no -f http://$HOST:$PORT_HTTP/auth/realms/$REALM/protocol/openid-connect/token username=$USER password=$PASSWORD grant_type=password client_id=$CLIENT_ID" echo ">>> TOKEN Received" echo $access_token echo ">>> Gateway Service URL" echo "$URL" echo ">>> GET Blog article : 1" http --verify=no GET $URL/search/id/1 "Authorization: Bearer $access_token" echo ">>> GET Blog articles of Charles Moulliard" http --verify=no GET $URL/search/user/cmoulliard "Authorization: Bearer $access_token" echo ">>> PUT Blog article n° 10" echo '{ "user": "cmoulliard", "postDate": "2015-09-15T10:10", "body": "Integration is hard - 10", "title": "On distributed search" }' | http --verify=no PUT $URL/10 "Authorization: Bearer $access_token" echo ">>> DELETE Blog Article n° 10" http --verify=no DELETE $URL/10 "Authorization: Bearer $access_token" echo ">>> GET Blog article : 10" http --verify=no GET $URL/search/id/10 "Authorization: Bearer $access_token"
-
To execute the script, use one of the existing terminal and run from the root of the project, this request
./script/src/main/resources/oauth2-all-requests.sh
-
If the requests are issued correctly, you should be able to see the token received and the CRUD requests.
./script/src/main/resources/oauth2-all-requests.sh >>> Token query http --verify=no -f http://localhost:8080/auth/realms/fuse/protocol/openid-connect/token username=admin password=admin grant_type=password client_id=fuse HTTP/1.1 200 OK Connection: keep-alive Content-Type: application/json Date: Mon, 21 Sep 2015 14:58:46 GMT Server: WildFly/8 Transfer-Encoding: chunked X-Powered-By: Undertow/1 { "access_token": "eyJhbGciOiJSUzI1NiJ9.eyJqdGkiOiIwY2M2M2M0NS1kMGU5LTRmOWYtYjhlYy00OTM1NWYxZjllZjEiLCJleHAiOjE0NDI4NDc4MjYsIm5iZiI6MCwiaWF0IjoxNDQyODQ3NTI2LCJpc3MiOiJodHRwOi8vbG9jYWxob3N0OjgwODAvYXV0aC9yZWFsbXMvZnVzZSIsImF1ZCI6ImZ1c2UiLCJzdWIiOiIzNTkxZTQxNy03YzYwLTQ0NjQtODcxNC05NjE5MGM3ZmFkOTIiLCJhenAiOiJmdXNlIiwic2Vzc2lvbl9zdGF0ZSI6ImVlNTU2NjUxLTc5MDctNDk1OC05NmYyLTVmNTFhYzI4NTU4NCIsImNsaWVudF9zZXNzaW9uIjoiOTY2MzAxMmEtYTRhZi00MjFmLWFkZDgtMDYzODYwNTNiYTMyIiwiYWxsb3dlZC1vcmlnaW5zIjpbXSwicmVzb3VyY2VfYWNjZXNzIjp7ImFjY291bnQiOnsicm9sZXMiOlsibWFuYWdlLWFjY291bnQiLCJ2aWV3LXByb2ZpbGUiXX19LCJuYW1lIjoiYWRtaW4gIiwicHJlZmVycmVkX3VzZXJuYW1lIjoiYWRtaW4iLCJnaXZlbl9uYW1lIjoiYWRtaW4ifQ.MnWNFgmiMl6StVLfVsCiubOaEa1ntf0XPCvlgJTt2l16J6wE_dDOx8Neu0KhwESVG0J7t1tBCpLyT1q60hHWy6alcYjDDhETe65Wy_g2DKBDtkcRB7IGFSXUZHbjqy-VHeHHypaVuQLUxagz8Z7H_9HiixrubkT3D7y7fleiXrI0mmPPJpvPzABoHEUP0iF6W-G4llUYEfxBX2qAC_eOgjallbpI3zt8cICcl_Yy4YAogYB43PTSF-GzvMIqrJFZM0j648C6dpCSOylJWUcsN0ytLFTJHiGzqAG9K2hO4aIHECqVuXJoCr-F-kmWUrsnhAWXANNfLLLuZfC5ke8qPg", "expires_in": 300, "id_token": "eyJhbGciOiJSUzI1NiJ9.eyJqdGkiOiI1ZDZmNzQ2ZS0xMDY2LTQ5YzEtODRhZS0xMjJhYTg4OTcwMzkiLCJleHAiOjE0NDI4NDc4MjYsIm5iZiI6MCwiaWF0IjoxNDQyODQ3NTI2LCJpc3MiOiJodHRwOi8vbG9jYWxob3N0OjgwODAvYXV0aC9yZWFsbXMvZnVzZSIsImF1ZCI6ImZ1c2UiLCJzdWIiOiIzNTkxZTQxNy03YzYwLTQ0NjQtODcxNC05NjE5MGM3ZmFkOTIiLCJhenAiOiJmdXNlIiwic2Vzc2lvbl9zdGF0ZSI6ImVlNTU2NjUxLTc5MDctNDk1OC05NmYyLTVmNTFhYzI4NTU4NCIsIm5hbWUiOiJhZG1pbiAiLCJwcmVmZXJyZWRfdXNlcm5hbWUiOiJhZG1pbiIsImdpdmVuX25hbWUiOiJhZG1pbiJ9.JzzhOj1c2XXrnr2r_s-83RsNPgHRP4Q_FGJVlEhUDycLnSgHAndrWj1GVsdTo-uGKeXJl3G-j0eUQwrSgtnr-Z_33iHroa58oEJ0w5u_Be53acWXInuJEvfsoN9VkimKW2sEx5VCo8nVru0TMZoNS3mHGcdkB0ZGF9VHxflbgOSWTagIV8wVJdd2-L6oujR7N7W39IJFvtHYQTi20xvihmHa5yrpDRfnpBOa0d_JBS_QznHnbQc8qKzeRP1gEfWuYCLp-g-16Trgl8acPAFsEKsBTswYw2B3sgxrn3sEz0NmsNxb5KNA7B3p7o2migXULktN-CjlcCNuIw3ZggJ9Lw", "not-before-policy": 0, "refresh_expires_in": 1800, "refresh_token": "eyJhbGciOiJSUzI1NiJ9.eyJqdGkiOiJkNmJhODBlNC01OTVmLTQzODItYmRiMy05N2Q5ZTAwN2U0NzkiLCJleHAiOjE0NDI4NDkzMjYsIm5iZiI6MCwiaWF0IjoxNDQyODQ3NTI2LCJpc3MiOiJodHRwOi8vbG9jYWxob3N0OjgwODAvYXV0aC9yZWFsbXMvZnVzZSIsInN1YiI6IjM1OTFlNDE3LTdjNjAtNDQ2NC04NzE0LTk2MTkwYzdmYWQ5MiIsInR5cCI6IlJFRlJFU0giLCJhenAiOiJmdXNlIiwic2Vzc2lvbl9zdGF0ZSI6ImVlNTU2NjUxLTc5MDctNDk1OC05NmYyLTVmNTFhYzI4NTU4NCIsImNsaWVudF9zZXNzaW9uIjoiOTY2MzAxMmEtYTRhZi00MjFmLWFkZDgtMDYzODYwNTNiYTMyIiwicmVzb3VyY2VfYWNjZXNzIjp7ImFjY291bnQiOnsicm9sZXMiOlsibWFuYWdlLWFjY291bnQiLCJ2aWV3LXByb2ZpbGUiXX19fQ.UqgCnsFdAajkmebdvwwA_HP8Hk2xXKZcODz8pDeBw9qL547dyI-VWcgZaFsnxFRvXg__JXyaPx5QeRGNA3_9PveW5cveGeav1DLIvuCjLBZaoWcxNK4QVzXXOBMzSqlfMpbH7PC2SRE401Nss9AyRI-54lECX4QAj6lY3r9_zsALpPd3pznuexSItvjcbyF24OBfw0uy6bZoneMs5aEQLltKc7S2F1kFNy9bSzM_xiSiDQJJtZHQUsym-TtWYPAQyv9gyD3qCD5-sAo3tHgtU60girhAztWFffRRktigUlopLtETs94QtJCyr-Ql0d9bui2pjSKZTOsug23hev4C7Q", "session-state": "ee556651-7907-4958-96f2-5f51ac285584", "token_type": "bearer" } >>> TOKEN Received eyJhbGciOiJSUzI1NiJ9.eyJqdGkiOiJhZjY4ZmFjNi1mZDUwLTRiNzMtYmQzNy01YzU1NWE4ZTU2MWUiLCJleHAiOjE0NDI4NDc4MjUsIm5iZiI6MCwiaWF0IjoxNDQyODQ3NTI1LCJpc3MiOiJodHRwOi8vbG9jYWxob3N0OjgwODAvYXV0aC9yZWFsbXMvZnVzZSIsImF1ZCI6ImZ1c2UiLCJzdWIiOiIzNTkxZTQxNy03YzYwLTQ0NjQtODcxNC05NjE5MGM3ZmFkOTIiLCJhenAiOiJmdXNlIiwic2Vzc2lvbl9zdGF0ZSI6ImY1OGQ1ZGZjLTZlNGMtNGFkMi1iZDJmLTcwNzEzZjZiOTQyZCIsImNsaWVudF9zZXNzaW9uIjoiZjA2YjY3M2YtZWNiZS00N2YyLWJhNzYtYjZhNTkwMWQ1YWZlIiwiYWxsb3dlZC1vcmlnaW5zIjpbXSwicmVzb3VyY2VfYWNjZXNzIjp7ImFjY291bnQiOnsicm9sZXMiOlsibWFuYWdlLWFjY291bnQiLCJ2aWV3LXByb2ZpbGUiXX19LCJuYW1lIjoiYWRtaW4gIiwicHJlZmVycmVkX3VzZXJuYW1lIjoiYWRtaW4iLCJnaXZlbl9uYW1lIjoiYWRtaW4ifQ.Ts82zYqhYl2K6MF2qCaSc_MQNJodqk_wggGYuFmcumnaYXFeK598p0WbBfAZyZ0nNuBcLiru_qH-Epk5HxjfP66rfdsi0vCWkWx80PYm41Yq0b1KQR_DvzTA6uaxdMJADba3xk9wykZn3Oe-1JN4nTku76q_4ks484gK3ErN9LwX4ZmfffF5pXZB2B7WL7fWDCbmLu3z2BP10qgScaf6LWybzuukLbFiKlJbym_NIc8inbTNDIJFEjRBTl5KPdU37pZTGYefpBB7eqIy1v1WPfC8Mbips7-fXlN6cxkKafDObrkiFCgN5AcwG4g9U5zhkLlGb3J9SbORbmgRVE0mOQ >>> Gateway Service URL https://localhost:8443/apiman-gateway/fuse/blog-service/3.0 >>> GET Blog article : 1 HTTP/1.1 200 OK Accept: */* Accept-Encoding: gzip, deflate Access-Control-Allow-Headers: Origin, Accept, X-Requested-With, Content-Type, Access-Control-Request-Method, Access-Control-Request-Headers Access-Control-Allow-Methods: GET, HEAD, POST, PUT, DELETE, TRACE, OPTIONS, CONNECT, PATCH Access-Control-Allow-Origin: * Access-Control-Max-Age: 3600 Authorization: Bearer eyJhbGciOiJSUzI1NiJ9.eyJqdGkiOiJhZjY4ZmFjNi1mZDUwLTRiNzMtYmQzNy01YzU1NWE4ZTU2MWUiLCJleHAiOjE0NDI4NDc4MjUsIm5iZiI6MCwiaWF0IjoxNDQyODQ3NTI1LCJpc3MiOiJodHRwOi8vbG9jYWxob3N0OjgwODAvYXV0aC9yZWFsbXMvZnVzZSIsImF1ZCI6ImZ1c2UiLCJzdWIiOiIzNTkxZTQxNy03YzYwLTQ0NjQtODcxNC05NjE5MGM3ZmFkOTIiLCJhenAiOiJmdXNlIiwic2Vzc2lvbl9zdGF0ZSI6ImY1OGQ1ZGZjLTZlNGMtNGFkMi1iZDJmLTcwNzEzZjZiOTQyZCIsImNsaWVudF9zZXNzaW9uIjoiZjA2YjY3M2YtZWNiZS00N2YyLWJhNzYtYjZhNTkwMWQ1YWZlIiwiYWxsb3dlZC1vcmlnaW5zIjpbXSwicmVzb3VyY2VfYWNjZXNzIjp7ImFjY291bnQiOnsicm9sZXMiOlsibWFuYWdlLWFjY291bnQiLCJ2aWV3LXByb2ZpbGUiXX19LCJuYW1lIjoiYWRtaW4gIiwicHJlZmVycmVkX3VzZXJuYW1lIjoiYWRtaW4iLCJnaXZlbl9uYW1lIjoiYWRtaW4ifQ.Ts82zYqhYl2K6MF2qCaSc_MQNJodqk_wggGYuFmcumnaYXFeK598p0WbBfAZyZ0nNuBcLiru_qH-Epk5HxjfP66rfdsi0vCWkWx80PYm41Yq0b1KQR_DvzTA6uaxdMJADba3xk9wykZn3Oe-1JN4nTku76q_4ks484gK3ErN9LwX4ZmfffF5pXZB2B7WL7fWDCbmLu3z2BP10qgScaf6LWybzuukLbFiKlJbym_NIc8inbTNDIJFEjRBTl5KPdU37pZTGYefpBB7eqIy1v1WPfC8Mbips7-fXlN6cxkKafDObrkiFCgN5AcwG4g9U5zhkLlGb3J9SbORbmgRVE0mOQ Connection: keep-alive Content-Type: application/json Date: Mon, 21 Sep 2015 14:58:47 GMT OkHttp-Received-Millis: 1442847527382 OkHttp-Response-Source: NETWORK 200 OkHttp-Selected-Protocol: http/1.1 OkHttp-Sent-Millis: 1442847527376 Server: Jetty(8.1.17.v20150415) Transfer-Encoding: chunked User-Agent: HTTPie/0.9.2 X-Powered-By: Undertow/1 breadcrumbId: ID-dabou-local-53720-1442828961817-0-135 id: 1 indexName: blog indexType: post operation: GET_BY_ID { "body": "Integration is hard 1.", "id": "1", "postDate": "2015-09-29T05:15", "title": "On distributed search", "user": "cmoulliard" }
-
When the ApiMan gateway will receive the HTTP requests, it will log the information (Number of requests, number of responses / type) into a local ElasticsSearch database. The
Metrics
tab that we have / service can allow us to check these data. -
You can view the responses / usage using the`Show` dropdown list and filter the services for the last hour, days …
-
Or Select
Response Type
to see the details according to the status (successfull, fail, error)
To be exhaustive, we will now extend the OAuth2 Fuse Realm to specify 2 roles (writer/reader) to restrict the access to the service according to them. Typically, we will create the writer role to let a user to PUT or DELETE a blog article while a reader role will only allow the users to query the REST services.
-
Open your web browser at this address to open the Oauth2 server
http:/localhost:8080/auth/admin
-
Select the
fuse
realm and add 2 roles by clicking on the tabRoles
-
Click on the button
Add Role
and fill the fieldRole name
with thereader
value -
Save the role by clicking on the button
Save
-
Repeat the operation to create a
writer
role -
When you return to the
Roles
screen, a table will display the 2 roles created -
Now, we will assign the roles to their respective users. So, let’s say that the
admin
user can has thewriter
andreader
roles while a lambda user will only be able to use thereader
role -
Create a new user as we did previously using as name
reader
and passwordreader
. Don’t forget to disable the temporary option. -
Save it and select the
Role Mappings
tab to assign thereader
role -
Assign the roles
writer
andreader
to the admin user -
We can leaver the OAuth2 server platform and return to Apiman in order to create a new service - version 4.0
NoteClone the Service - version 3.0 and specify 4.0 as the vesion name of this service -
Add a new policy by clicking on the button
Policy
. Within the dropdown list, select the valueAuthorization Policy
-
We will now define 3 rules to map a path to a REST verb (which corresponds to the HTTP operation) and a role by using the
Add Authorization Rule
section of the screen. Encode the information as defined within the following tablePath
Verb
Role required
.*
PUT
Writer
.*
DELETE
Writer
.*
GET
Reader
-
Save the result by clicking on the button
Update Policy
-
Publish the service version 4.0
-
Use respectively these bash scripts to issue HTTP requests against the ApiMan gateway but using a different username (admin or reader)
./script/src/main/resources/oauth2-role-reader.sh ./script/src/main/resources/oauth2-role-writer.sh
-
Verify the metrics
NoteYou can also change the login/password to test if it works or create new users within the Oauth2 Server. NOTE : The content of the JSon Web token replied to the application can be visualized using this web server - http://jwt.io/. Click on the Debugger link and add within the field the content of the token displayed in the terminal by the bash script
Hip hip hourra, you have finished !!!
If, for any reason, you would like to restart the demo from the beginning. Then, perform these steps to clean the JBoss Fuse Server
-
Exit from the JBoss Fuse Console using the command
CTRL-D
orosgi:shutdown
command -
Run this script
./bin/deletefabric8
.NoteIt will kill the jvm instances and delete the instances and some sub-folders defined under the data folder. -
Stop the jetty web container started using the command
mvn jetty:run
-
When the local Camel REST endpoints don’t work, you can query directly the elasticsearch database using these HTTPie requests to check if it works.
WARNING : The hostname must be changed depending if you run locally or remotely the JBoss Fuse Server
http http://localhost:9191/blog/post/1 pretty==true http http://localhost:9200/blog/post/_search q=="user:cmoulliard" pretty==true curl 'http://localhost:9200/blog/post/_search?q=user:cmoulliard&pretty=true'
-
Delete all articles
http DELETE http://localhost:9200/blog/post/_query q=="user:*"
-
Delete Index
http DELETE http://localhost:9200/blog
-
Create Index
http PUT http://localhost:9200/blog
-
Add mapping
http PUT http://localhost:9200/blog/_mapping/article < data/elasticsearch/mapping.json
-
Check mapping
http http://localhost:9200/blog/_mapping/article
-
Add user
http PUT http://localhost:9200/blog/article/1 < data/elasticsearch/entry.json
You can also run the project locally using mvn camel:run
at the condition that the Karaf feature insight-elasticsearch
or the profile insight-elasticsearch.datastore
has been deployed into JBoss Fuse 6.2.
-
Open a Windows or Unix terminal and move to the directory
rest-dsl-in-action-master/routing
-
Launch Apache Camel
mvn camel:run
-
Control that Apache Camel has been started
[ Blueprint Extender: 3] BlueprintCamelContext INFO Apache Camel 2.15.1.redhat-620133 (CamelContext: camel-1) is starting [ Blueprint Extender: 3] ManagedManagementStrategy INFO JMX is enabled [ Blueprint Extender: 3] ElasticsearchEndpoint INFO Joining ElasticSearch cluster insight [ Blueprint Extender: 3] ElasticsearchEndpoint INFO REMOTE ELASTICSEARCH: localhost [ Blueprint Extender: 3] plugins INFO [Sack] loaded [], sites [] [ Blueprint Extender: 3] JacksonDataFormat INFO Registering module: com.fasterxml.jackson.module.jaxb.JaxbAnnotationModule@5ce285fa [ Blueprint Extender: 3] JacksonDataFormat INFO Registering module: com.fasterxml.jackson.module.jaxb.JaxbAnnotationModule@5272a5f [ Blueprint Extender: 3] JacksonDataFormat INFO Registering module: com.fasterxml.jackson.module.jaxb.JaxbAnnotationModule@3fd6ac8d [ Blueprint Extender: 3] JacksonDataFormat INFO Registering module: com.fasterxml.jackson.module.jaxb.JaxbAnnotationModule@37ec4ba9 [ Blueprint Extender: 3] JacksonDataFormat INFO Registering module: com.fasterxml.jackson.module.jaxb.JaxbAnnotationModule@354718b0 [ Blueprint Extender: 3] JacksonDataFormat INFO Registering module: com.fasterxml.jackson.module.jaxb.JaxbAnnotationModule@5971c095 [ Blueprint Extender: 3] JacksonDataFormat INFO Registering module: com.fasterxml.jackson.module.jaxb.JaxbAnnotationModule@4ace6503 [ Blueprint Extender: 3] JacksonDataFormat INFO Registering module: com.fasterxml.jackson.module.jaxb.JaxbAnnotationModule@4683ea40 [ Blueprint Extender: 3] HttpComponent INFO Created ClientConnectionManager org.apache.http.impl.conn.PoolingHttpClientConnectionManager@79d51938 [ Blueprint Extender: 3] BlueprintCamelContext INFO AllowUseOriginalMessage is enabled. If access to the original message is not needed, then its recommended to turn this option off as it may improve performance. [ Blueprint Extender: 3] BlueprintCamelContext INFO StreamCaching is not in use. If using streams then its recommended to enable stream caching. See more details at http://camel.apache.org/stream-caching.html [ Blueprint Extender: 3] JacksonDataFormat INFO Registering module: com.fasterxml.jackson.module.jaxb.JaxbAnnotationModule@4576c19b [ Blueprint Extender: 3] FileEndpoint INFO Endpoint is configured with noop=true so forcing endpoint to be idempotent as well [ Blueprint Extender: 3] FileEndpoint INFO Using default memory based idempotent repository with cache max size: 1000 [ Blueprint Extender: 3] BlueprintCamelContext INFO Route: add-direct-route started and consuming from: Endpoint[direct://add] [ Blueprint Extender: 3] BlueprintCamelContext INFO Route: remove-direct-route started and consuming from: Endpoint[direct://remove] [ Blueprint Extender: 3] BlueprintCamelContext INFO Route: file-marshal-split-service started and consuming from: Endpoint[file://src/data/articles?noop=true] [ Blueprint Extender: 3] BlueprintCamelContext INFO Route: route1 started and consuming from: Endpoint[direct://error] [ Blueprint Extender: 3] JettyHttpComponent INFO Using default Jetty continuation timeout for: Endpoint[http://0.0.0.0:9191/blog/article/search/id/%7Bid%7D?httpMethodRestrict=GET] [ Blueprint Extender: 3] Server INFO jetty-8.1.17.v20150415 [ Blueprint Extender: 3] AbstractConnector INFO Started [email protected]:9191 [ Blueprint Extender: 3] BlueprintCamelContext INFO Route: rest-searchbyid started and consuming from: Endpoint[http://0.0.0.0:9191/blog/article/search/id/%7Bid%7D?httpMethodRestrict=GET] [ Blueprint Extender: 3] BlueprintCamelContext INFO Route: rest-searchbyuser started and consuming from: Endpoint[http://0.0.0.0:9191/blog/article/search/user/%7Buser%7D?httpMethodRestrict=GET] [ Blueprint Extender: 3] BlueprintCamelContext INFO Route: rest-put-article started and consuming from: Endpoint[http://0.0.0.0:9191/blog/article/%7Bid%7D?httpMethodRestrict=PUT] [ Blueprint Extender: 3] BlueprintCamelContext INFO Route: rest-deletearticle started and consuming from: Endpoint[http://0.0.0.0:9191/blog/article/%7Bid%7D?httpMethodRestrict=DELETE] [ Blueprint Extender: 3] BlueprintCamelContext INFO Route: searchbyid-direct-route started and consuming from: Endpoint[direct://searchById] [ Blueprint Extender: 3] BlueprintCamelContext INFO Route: searchbyuser-direct-route started and consuming from: Endpoint[direct://searchByUser] [ Blueprint Extender: 3] BlueprintCamelContext INFO Route: searchbyuser2-direct-route started and consuming from: Endpoint[direct://searchByUser2] [ Blueprint Extender: 3] BlueprintCamelContext INFO Total 11 routes, of which 11 is started. [ Blueprint Extender: 3] BlueprintCamelContext INFO Apache Camel 2.15.1.redhat-620133 (CamelContext: camel-1) started in 0.660 seconds [ #0 - file://src/data/articles] add-direct-route INFO Add new Blog entry service called ! [ #0 - file://src/data/articles] add-direct-route INFO Response received : 1 [ #0 - file://src/data/articles] add-direct-route INFO Add new Blog entry service called ! [ #0 - file://src/data/articles] add-direct-route INFO Response received : 2 [ #0 - file://src/data/articles] add-direct-route INFO Add new Blog entry service called ! [ #0 - file://src/data/articles] add-direct-route INFO Response received : 3 ...
NOTE : Additional parameters could be defined for the elasticsearch database using the io.fabric8.elasticsearch-insight.cfg
config file deployed into the etc
folder of JBoss Fuse.
That should also work if you deploy locally an Elasticsearch instance on your machine but this use case hasn’t been tested.
When you would like test your project and if the Elasticsearch server is running locally, you can copy/paste this list of HTTPie queries to play with the CRUD scenario It will delete the blog index, create a new index, insert articles, search using a user name or id and will delete a user.
http DELETE http://localhost:9200/blog http PUT http://localhost:9200/blog http PUT http://localhost:9200/blog/_mapping/article < data/elasticsearch/mapping.json http http://localhost:9200/blog/_mapping/article http PUT http://localhost:9191/blog/article < data/elasticsearch/entry.json http http://localhost:9191/blog/article/search/id/0 http http://localhost:9191/blog/article/search/user/cmoulliard http http://localhost:9191/blog/article/search/user/cmoullia http DELETE http://localhost:9191/blog/article/1 http http://localhost:9191/blog/article/search/id/1