- RADAR-Appserver
This is an app server which provides facilities to store app information (User related) and scheduling of push notifications using Firebase Cloud Messaging.
This is specifically developed to support the RADAR-Questionnaire application but can be easily extended or modified to suit the needs of different applications.
The app server provides REST endpoints to interact with the entities and data. For detailed info on the REST API please see the relevant section below.
-
First you will need to create a Firebase project for your application and add it to your app. This will give you access to all the Firebase services. Follow the instructions on the official docs according to your platform.
-
Configure the Server Key and Sender ID (obtained from FCM) in application.properties.
-
The AppServer needs a database to work. You can either use a
stand-aloneinstance of the database of use an in-memoryembeddedinstance-3.1. To use the standalone instance, run the database a docker service by
```bash docker-compose -f src/integrationTest/resources/docker/non_appserver/docker-compose.yml up -d postgres ``` This will start the database at `localhost:5432`3.2. To use as an embedded in-memory database instance (Not recommended for production deployments), set the
spring.datasource.url=jdbc:hsqldb:mem:/appserverinapplication-<profile>.properties. Also, change the properties insrc/main/resources/application.propertiesto dev or prod according to your requirements. -
Build the project using gradle wrapper and run using spring boot. Note: This project uses JAVA 17, please download and install it before building.
-
The build will need to create a logs directory. The default path is
/usr/local/var/lib/radar/appserver/logs. Either create the directory there usingsudo mkdir -p /usr/local/var/lib/radar/appserver/logsfollowed bysudo chown $USER /usr/local/var/lib/radar/appserver/logsor change logs file directory insrc/main/resources/logback-spring.xmlto local log directory like<property name="LOGS" value="logs" /> -
The appserver uses the Admin SDK to communicate with the Firebase Cloud Messaging. To configure this, please look at the FCM section.
-
To run the build, run the command below -
./gradlew bootRun
You can also run in an IDE (like IntelliJ Idea) by giving the
/src/main/java/org/radarbase/appserver/AppserverApplication.javaas the main class. -
The App-server is now running and is able to send FCM messages. You can make the request to create a project, a user and a notification using the REST API.
-
Voila!, you will now receive a notification at the schedule time (specified by
scheduledTimein the payload) on your device.
The full API specification and documentation is available via Swagger UI when you launch the app server. Please refer to the Documentation section below.
- Create a project. If using Management portal, this should be exactly same as the project name
in management portal.
POST http://localhost:8080/projects/p1 { "projectId": "p1" } - Create a user. Make sure to use the correct FCM token otherwise you will not receive the
notification on the device.
POST http://localhost:8080/projects/p1/users/u2 { "subjectId": "u2", "fcmToken" : "shdzdxcvc", "enrolmentDate": "2018-11-29T00:00:00Z", "timezone": "Australia/Sydney", "language": "en" } - Create a notification.
POST http://localhost:8080/projects/p1/users/u2/messaging/notifications { "title" : "Questionnaire Time", "body": "Urgent Questionnaire Pending. Please complete now.", "ttlSeconds": 0, "sourceId": "null", "type": "ers", "sourceType": "aRMT", "appPackage": "org.phidatalab.radar_armt", "scheduledTime": "2022-02-23T09:04:00Z", "additionalData": { "questionnaire":"{\"name\":\"ers\",\"questionnaire\":{\"avsc\":\"questionnaire\",\"name\":\"ers\",\"repository\":\"https://raw.githubusercontent.com/RADAR-CNS/RADAR-REDCap-aRMT-Definitions/master/questionnaires/\"},\"protocol\":{\"clinicalProtocol\":null,\"completionWindow\":{\"amount\":1440,\"unit\":\"minutes\"},\"notification\":{\"title\":{\"en\":\"Questionnaire Time\"},\"text\":{\"en\":\"Urgent Questionnaire Pending. Please complete now.\"}},\"reminders\":{\"repeat\":0,\"amount\":0,\"unit\":\"day\"},\"repeatProtocol\":{\"amount\":9999999999,\"unit\":\"minutes\"},\"repeatQuestionnaire\":{\"unitsFromZero\":[0],\"unit\":\"minutes\"}},\"referenceTimestamp\":1645607040.000000000,\"showInCalendar\":true,\"showIntroduction\":false,\"estimatedCompletionTime\":1,\"order\":0,\"isDemo\":false,\"startText\":{},\"endText\":{},\"warn\":{}}", "action":"QUESTIONNAIRE_TRIGGER", "metadata":"\"{}\"" } }
The same result as stated in Getting Started can be achieved using REST endpoints of the AppServer.
-
Run the AppServer by following the first 3 steps in the Getting Started section.
-
Create a new Project by making a
POSTrequest to the endpointhttp://localhost:8080/projectswith the following body-{ "projectId": "radar" } -
Create a new User in the Project by making a
POSTrequest to the endpointhttp://localhost:8080/project/test/userswith the following body-{ "subjectId": "sub-1", "fcmToken" : "get-this-from-the-device", "enrolmentDate": "2019-07-29T00:00:00Z", "timezone": 7200, "language": "en" }Note: You will need to get the FCM token from the device and the app. Please see the setup info for your platform.
-
Add (and schedule) a notification for the above user by making a
POSTrequest to the endpointhttp://localhost:8080/project/test/users/sub-1/notificationswith the following body-{ "title" : "Test Title", "body": "Test Body", "ttlSeconds": 86400, "sourceId": "z", "fcmMessageId": "12864132148", "type": "ESM", "sourceType": "aRMT", "appPackage": "aRMT", "scheduledTime": "2019-06-29T15:25:58.054Z" }Please update the
scheduledTimeto the desired time of notification delivery. -
You will now receive a notification at the
scheduledTimefor the App and device associated with the FCM token for the user. There are other features provided via the REST endpoints. These can be explored using swagger-ui. Please refer to Documentation section.
To configure AdminSDK, follow the official
Firebase documentation till you setup the environment
variable (GOOGLE_APPLICATION_CREDENTIALS). In the properties
file, you would need to set fcmserver.fcmsender to org.radarbase.fcm.downstream.AdminSdkFcmSender.
The AppServer is also available as a docker container. Its Dockerfile is provided with the project. It can be run as follows -
docker run -v /logs/:/var/log/radar/appserver/ \
-v etc/google-credentials.json:/etc/google-credentials.json \
-e "GOOGLE_APPLICATION_CREDENTIALS=/etc/google-credentials.json" \
radarbase/radar-appserver:1.1.0Make sure to have the correct path to the google-credentials.json file.
The same can be achieved by running as a docker-compose service. Just specify the following in docker-compose.yml
file -
services:
appserver:
image: radarbase/radar-appserver:1.1.0
restart: always
ports:
- 8080:8080
volumes:
- ./radar-is.yml:/resources/radar-is.yml
- ./logs/:/var/log/radar/appserver/
- ./etc/google-credentials.json:/etc/google-credentials.json
environment:
JDK_JAVA_OPTIONS: -Xmx4G -Djava.security.egd=file:/dev/./urandom
GOOGLE_APPLICATION_CREDENTIALS: /etc/google-credentials.json
RADAR_ADMIN_USER: "radar"
RADAR_ADMIN_PASSWORD: "radar"
SPRING_APPLICATION_JSON: '{"spring":{"boot":{"admin":{"client":{"url":"http://spring-boot-admin:1111","username":"radar","password":"appserver"}}}}}'
RADAR_IS_CONFIG_LOCATION: "/resources/radar-is.yml"
SPRING_BOOT_ADMIN_CLIENT_INSTANCE_NAME: radar-appserverAn example docker-compose file with all the other components is provided
in integrationTest resources.
Here is a high level architecture and data flow diagram for the AppServer and its example interaction with a Cordova application (hybrid) like the RADAR-Questionnaire.
┌───────────────────┐ Downstream
│Device (Google Play│◀─────────────────────────────────Message .───────────.
│ Services/Apple │ │ _.─' `──.
│ IPNS) │ └───────────────────────────────────────,' `.
└────────▲────┬─────┴─────────────────────────────────────┐ ,' `.
│ │ │ ╱ ╲
│ │ XMPP ; :
│ │ Upstream │ Firebase Cloud Messaging │
.┴────▼─. Message───────────────────────────────────▶│ Service │
,' `. : ;
; Native Code : ╲ ╱
:(IOS/Android); ╲ ╱
╲ ╱ `. ,'
`▲ ,' `. ,'
│`─────│ ▲`──. _.─'
│ │ │ `────────'▲
│ │ │
│ │ │ ┃
│ │ │
│ │ │ ┃
│──────▼. Send
,─' '─. downstream ┃
╱ Cordova FCM ╲ Message at
; Plugin : Scheduled ┃
: ; Time
╲ ╱ │ ┃
╲ ╱ │ FCM Admin
'─▲ ,─' │ SDK (Only
│`─────'│ │ downstream
│ │ │ messaging)
│ │ │
│ │ │ ┃
│ │ │
│ │ ┌────┴───────▼──────┻─────── ▼────┐
│ │ │ │
┌─────┴───────▼─────────┐ │ │
│ │ ┌───────────────────────────────────▶ │
│ │ Schedule message │ │
│ │ for future delivery │ New App Server │
│ │ using HTTP REST │ │
│ ├───────────────────────────────────┘ │ (HTTP Protocol) │
│ CORDOVA APPLICATION │ │ (REST API and FCM Admin SDK) │
│ │ Get confirmation of ─────────────────────────┤ │
│ ◀───────────────────────success for each request. │ │
│ │ │ │
│ │ ┌───────────────────────────────────▶ │
│ │ │ │ │
│ │ Get/Set user metrics, │ │
│ ├───────────────────schedule, notifications, etc │ │
│ │ │ │
│ │ More │ │
│ ├ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ functionality ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ▶ │
└───────────────────────┘ ..... └─────────────────────────────────┘
The Appserver manages the lifecycle of the Notifications through state change events. It uses Pub/Sub paradigm utilising Spring Events so other subscribers can also hook up to the Events as listeners. Currently, there are 10 possible states as follows -
// Database controlled
ADDED, UPDATED, CANCELLED
// Scheduler Controlled
SCHEDULED, EXECUTED
// Controlled by entities outside the appserver.
// These will need to be reported to the appserver.
DELIVERED, OPENED, DISMISSED
// Miscellaneous
ERRORED, UNKNOWN
REST Endpoints are provided to update and query the STATE. Update can only be made to any of the ones above that can be updated by external entities(i.e. DELIVERED, OPENED, DISMISSED, ERRORED and UNKNOWN ).
Here is a simple flow between the states --
┌───────────────────────────────────────────────────────┐
│ │
▼ │
.─────────────────. .─────────────────.
_.──' `───. _.──' `───.
╱ ╲ ╱ ╲
┌─────▶( SCHEDULED ) ─────────────────────▶( UPDATED )
│ `. ,────────────┐ `. ,'
│ `───. _.──' │ `───. _.──'
│ `───────────────' │ `───────────────'
┌─────────────────────────────────┐ │ │ │ │
│ │ │ │ │ │
│ ▼ │ │ │ │
┌───────────────┐ .──────────────────┘ │ │ │
│ │ _.──' `───. └────────┐ ┌───────────────│────────────────────────────┘
│ REST │ ╱ ╲ │ │ │
│ │────┐ ▲ ADDED )────┐ │ │ │
│ │ │ │`. ,' │ │ │ │
└───────────────┘ │ │ `───. _.──' │ ▼ ▼ │
▲ │ │ `───────────────' │ .─────────────────. │ .─────────────────.
┌─────────┘ │ │ │ _.──' `───. │ _.──' `───.
┌───┴─────┐ │ │ │ ╱ ╲ │ ╱ ╲
│ Request │ ┌────┼────┘ ├───▶( ERRORED ) └───────────▶( EXECUTED )
└─────────┴───┐ │ │ │ `. ,' `. ,'
▼ │ │ │ `───. ◀─────────┐ `───. _.──'
┌───────────────┤ │ │ `───────────────' │ `───────────────'
│ │ │ .─────────────────. │ ▲ │ │
│ │ │ _.──' `───. │ │ │ │
│ XMPP │ │ ╱ ╲ │ └──────────────────┼───────────────────────────┤
│ │ └────▶( CANCELLED )───┘ │ │
│ │ `. ,' │ │
└───────────────┘ `───. _.──' │ │
│ `───────────────' │ │
│ ▲ │ │
│ │ │ ▼
└──────────────────────────────────┘ │ .─────────────────.
│ _.──' `───.
│ ╱ ╲
└───────────( DELIVERED )
`. ,'
`───. _.──'
┌──────────────────────┐ `───────────────'
.───────────. │ │ │
,─' '─. ▼ │ │
,' `. .─────────────. └─────────────────────────┤
; : _.─' `──. ┌────┘
: UNKNOWN ; ,' `. │
╲ ╱ ; DISMISSED : │
`. ,' : ; │
'─. ,─' ╲ ╱ ▼
`─────────' `. ,' .─────────────────.
`──. _.─' _.──' `───.
`───────────' ╱ ╲
( OPENED )
`. ,'
`───. _.──'
`───────────────'
The AppServer has support for providing Protocols for the RADAR-Questionnaire application. Currently, one strategy for getting the protocols from Github(Take a look at RADAR-aRMT-protocols) is provided. The AppServer also caches the protocols, so they are still available if there are any issues with GitHub. Later, we intend to extend this functionality to add protocols directly in the AppServer possibly by a UI. You can host your own protocols and configure the following properties -
| Property | Description | Default | Required? |
|---|---|---|---|
| radar.questionnaire.protocol.github.repo.path | The Github repo where protocols are hosted. | RADAR-base/RADAR-aRMT-protocols |
No |
| radar.questionnaire.protocol.github.file.name | The filename containing the Protocol for each Project. | protocol.json |
No |
| radar.questionnaire.protocol.github.branch | The Branch of the Repository from which to fetch the protocols | master |
No |
By default, appserver sends push notifications to mobile devices. Optionally, Notifications can be sent via email in
addition. To enable email notifications, you need to set the following properties in the application.properties file:
- set the
radar.notification.email.enabledproperty totrue. - set the
radar.notification.email.fromproperty to the email address from which the notifications will be sent. - set Spring Boot mail properties to configure the email server.
Example:
radar.notification.email.enabled=true
radar.notification.email.from[email protected]
spring.mail.host=smtp.gmail.com
spring.mail.port=587
spring.mail.username=my_username
spring.mail.password=my_password
spring.mail.properties.mail.smtp.auth=true
spring.mail.properties.mail.smtp.starttls.enable=trueIn addition, in the notification scheduling request set the following fields:
- emailEnabled with value true.
- (optional) emailTitle: subject of the email (notification title will be used when not specified)
- (optional) emailBody: body of the email (notification text will be used when not specified)
Example of a request body (partial) to the notification scheduling endpoint:
{
...
"emailEnabled": true,
"emailTitle": "My email title",
"emailBody": "My email body",
...
}Note: HTML email is not supported at the moment of this writing.
Api docs are available through swagger open api 3 config.
The raw json is present at the <your-base-url/v3/api-docs>. By default this should
be http://localhost:8080/v3/api-docs. This will provide the specification in JSON format. If YAML format is
preferred, you can query http://localhost:8080/v3/api-docs.yaml
The Swagger UI is shown below.
It is present at <your-base-url/swagger-ui.html
The Swagger API docs are also available at Swagger Hub but may not be most up-to-date. Please check the version matches the app-server that you have deployed.
The Java docs are also available as static content when you build and deploy the app-server.
These are stored in the /src/main/resources/static/java-docs path automatically when building and spring picks this up
and exposes it on the path <your-base-url/java-docs/index.html> as shown below -
You can generate a client in 40 different languages for the api using Swagger Codegen tool. There is even a javascript library that is completely dynamic and does not require static code generation.
By Default, no OAuth 2.0 security is enabled for the endpoints. Only basic Auth is present on Admin endpoints. To enable security of specific provider, please read the sections below.
To enable security via the RADAR Management Portal, set the following property -
security.radar.managementportal.enabled=true
security.radar.managementportal.url=<your management portal base url>This will instantiate all the classes needed for security using the management portal. Per endpoint level auth is controlled using Pre and Post annotations for each permission. All the classes are located in /src/main/java/org/radarbase/appserver/auth/managementportal.
You can provide the Management Portal specific config in radar-is.yml file providing the public key
endpoint and the resource name. The path to this file should be specified in the env
variable RADAR_IS_CONFIG_LOCATION.
If security is enabled, please also make sure that the correct resources and scope are set in the OAuth Client
configurations in Management Portal.
The resource res_AppServer and scopes MEASUREMENT.CREATE,SUBJECT.UPDATE,SUBJECT.READ,PROJECT.READ must be added to
the aRMT client. Please check the /src/integrationTest/resources/docker/etc/config/oauth_client_details.csv file for
an example.
For using other type of security providers, set managementportal.security.enabled=false and configure the security
provider in the spring context and add any necessary classes. See Management Portal Security
section for an example.
Then you will need to change the Pre and Post Authorise annotations for each endpoint method according to the
semantics provided by your provider. Currently, these are configured to work with Management portal.
The App server has built in support for
the Spring Boot Actuator and
can be accessed via the default REST endpoints <your-base-url>/actuator/*. To see all the features provided please
refer to
the official docs of
Spring boot actuator.
It also has functionality to register itself as a client to the Spring Boot Admin Server for providing a beautiful UI for the Actuator and some other useful admin stuff. To make this work,
-
Run an instance of the Spring Boot Admin server (various examples on the internet and also via Docker) on the machine and then
-
configure the client to point to the Admin Server for registration by adding the following to your
application.propertiesfile -spring.boot.admin.client.url = http://localhost:8888In this case, the Spring Boot admin server was running on
http://localhost:8888. If http basic auth is enabled on the server also add the following to theapplication.propertiesfile -spring.boot.admin.client.url = http://localhost:8888 spring.boot.admin.client".username = admin-server-username spring.boot.admin.client".password = admin-server-password
The same can be achieved when deployed with the components as microservices in docker containers using docker-compose.
The file docker-compose.yml in this
project shows an example of how this is achieved.
Please note how the App server is configured in the container compared to the method of adding properties
in application.properties file shown above.
Just run -
cd src/integrationTest/resources/docker/appserver_dockerhub/
sudo docker-compose up -dThen go to the browser to the URL : https://localhost:8888 and login with credentials radar:appserver to see the
Spring Boot Admin Server in action.
Deploying the Spring Boot Admin server and the client as different components makes sure that the same Server can be used to register multiple client apps and the server's lifecycle is not associated with the client. This also means that our client app is lighter and production ready.
The app server supports performance testing using Gatling and Scala. The simulations are located in the
folder src/gatling/simulations.
First run the application and then run the gatling test using the command -
./gradlew gatlingRunTo run a particular simulation you can run it like follows -
./gradlew gatlingRun-org.radarbase.appserver.BatchingApiGatlingSimulationTestYou can modify the number of iterations of the requests by change the variables the the top of the
file src/gatling/simulations/org/radarbase/appserver/ApiGatlingSimulationTest.scala like -
val numOfProjects = 5
val numOfUsersPerProject = 300
val numOfNotificationsPerUser = 300
val numOfSimultaneousClients = 2You can also edit the base URL of your deployed instance of the server by changing the value of
val baseUrl = "http://localhost:8080"Ideally deploy the server on a remote instance using a persisted database instead of in-memory for real-world testing. Running on your local machine may not reflect the best results.
The reports will be generated at the path build/reports/gatling/ and the folders will be
named apigatlingsimulationtest-{date-time}. Open the folder with the correct date time and open the index.html file
in a browser to view the results. Example result is shown in the screengrab below-
Various tools are enabled to ensure code quality and styling while also doing static code analysis for bugs. PMD, CheckStyle is included and all of these can be run with the command -
./gradlew checkNote: This will also run the test and integrationTest.
The reports are generated in the build/reports folder. The config files for rules are present in the config folder.
A style template following the Google Java style guidelines is also provided for use with IntellJ
Idea (style plugin) in config/codestyles folder.
Unit Tests and Integration Tests are provided with the AppServer. These can be run as follows-
# For Unit tests
./gradlew test# For Integration Tests
./gradlew integrationTestThe integration tests are currently provided for Management Portal as the security provider and uses a running instance
of Management Portal to get a valid client token and provide access to resources.
To change the security provider implement the
interface OAuthHelper and provide a
valid Access Token in getAccessToken() using your security provider.
Then just use this instance in static Initialisation of the Tests. For more info take a look
at MPOAuthHelper.
You can also run all the checks and tests in a single command using -
./gradlew checkThis will run checkstyle, PMD, spot bugs, unit tests and integration tests.
- Provides a general purpose FCM library with facility to send messages with Admin SDK support.
- Provides functionality of scheduling notifications via FCM.
- Acts as a data store for important user and app related data (like FCM token to subject mapping, notifications, user metrics, etc).
- Can be easily extended for different apps.
- Uses Liquibase for easy evolution of database.
- Contains swagger integration for easy API documentation and generation of Java client.
- Uses lombok.data in most places to reduce boilerplate code and make it more readable.
- Has support for Auditing of database entities.