Skip to content

Commit

Permalink
Merge pull request #24 from instacart/kg/feat/ntp_sync
Browse files Browse the repository at this point in the history
feat: full ntp spec implementation
  • Loading branch information
kaushikgopal authored Oct 18, 2016
2 parents fb8c9f0 + 84032eb commit f5efdff
Show file tree
Hide file tree
Showing 10 changed files with 304 additions and 150 deletions.
24 changes: 11 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

*Make sure to check out our counterpart too: [TrueTime](https://github.com/instacart/TrueTime.swift), an SNTP library for Swift.*

SNTP client for Android. Calculate the date and time "now" impervious to manual changes to device clock time.
NTP client for Android. Calculate the date and time "now" impervious to manual changes to device clock time.

In certain applications it becomes important to get the real or "true" date and time. On most devices, if the clock has been changed manually, then a `new Date()` instance gives you a time impacted by local settings.

Expand All @@ -16,7 +16,7 @@ You can read more about the use case in our [blog post](https://tech.instacart.c

It's pretty simple actually. We make a request to an NTP server that gives us the actual time. We then establish the delta between device uptime and uptime at the time of the network response. Each time "now" is requested subsequently, we account for that offset and return a corrected `Date` object.

Also, once we have this information it's valid until the next time you boot your device. This means if you enable the disk caching feature, after a signle successfull SNTP request you can use the information on disk directly without ever making another network request. This applies even across application kills which can happen frequently if your users have a memory starved device.
Also, once we have this information it's valid until the next time you boot your device. This means if you enable the disk caching feature, after a single successfull NTP request you can use the information on disk directly without ever making another network request. This applies even across application kills which can happen frequently if your users have a memory starved device.

# Installation

Expand Down Expand Up @@ -62,13 +62,11 @@ Date noReallyThisIsTheTrueDateAndTime = TrueTime.now();

## Rx-ified Version

If you're down to using [RxJava](https://github.com/ReactiveX/RxJava) then there's a niftier `initialize()` api that takes in the pool of hosts you want to query.
If you're down to using [RxJava](https://github.com/ReactiveX/RxJava) then we go all the way and implement the full NTP. Use the nifty `initializeRx()` api which takes in an NTP pool server host.

```java
List<String> ntpHosts = Arrays.asList("0.north-america.pool.ntp.org",
"1.north-america.pool.ntp.org");
TrueTimeRx.build()
.initialize(ntpHosts)
.initializeRx("0.north-america.pool.ntp.org")
.subscribeOn(Schedulers.io())
.subscribe(date -> {
Log.v(TAG, "TrueTime was initialized and we have a time: " + date);
Expand All @@ -85,24 +83,24 @@ TrueTimeRx.now(); // return a Date object with the "true" time.

### What is nifty about the Rx version?

* it can take in multiple SNTP hosts to shoot out the UDP request
* those UDP requests are executed in parallel
* if one of the SNTP requests fail, we retry the failed request (alone) for a specified number of times
* as soon as we hear back from any of the hosts, we immediately take that and terminate the rest of the requests
* as against just SNTP, you get full NTP (read: far more accurate time)
* the NTP pool address you provide is resolved into multiple IP addresses
* we query each IP multiple times, guarding against checks, and taking the best response
* if any one of the requests fail, we retry that failed request (alone) for a specified number of times
* we collect all the responses and again filter for the best result as per the NTP spec

## Notes/tips:

* Each `initialize` call makes an SNTP network request. TrueTime needs to be `initialize`d only once ever, per device boot. Use TrueTime's `withSharedPreferences` option to make use of this feature and avoid repeated network request calls.
* Preferable use dependency injection (like [Dagger](http://square.github.io/dagger/)) and create a TrueTime @Singleton object
* TrueTime was built to be accurate "enough", hence the use of [SNTP](https://en.wikipedia.org/wiki/Network_Time_Protocol#SNTP). If you need exact millisecond accuracy then you probably want [NTP](https://www.meinbergglobal.com/english/faq/faq_37.htm) (i.e. SNTP + statistical analysis to ensure the reference time is exactly correct). TrueTime provides the building blocks for this. We welcome PRs if you think you can do this with TrueTime(Rx) pretty easily :).
* You can read up on Wikipedia the differences between [SNTP](https://en.wikipedia.org/wiki/Network_Time_Protocol#SNTP) and [NTP](https://www.meinbergglobal.com/english/faq/faq_37.htm).
* TrueTime is also [available for iOS/Swift](https://github.com/instacart/truetime.swift)

## Exception handling:

* an `InvalidNtpServerResponseException` is thrown every time the server gets an invalid response (this can happen with the SNTP calls).
* an `InvalidNtpServerResponseException` is thrown every time the server gets an invalid response (this can happen with the individual SNTP calls).
* If TrueTime fails to initialize (because of the above exception being throw), then an `IllegalStateException` is thrown if you try to call `TrueTime.now()` at a later point.


# License

```
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,10 @@
import butterknife.ButterKnife;
import butterknife.OnClick;
import com.instacart.library.truetime.TrueTime;
import com.instacart.library.truetime.extensionrx.TrueTimeRx;
import com.instacart.library.truetime.TrueTimeRx;
import java.text.DateFormat;
import java.text.SimpleDateFormat;
import java.util.Arrays;
import java.util.Date;
import java.util.List;
import java.util.Locale;
import java.util.TimeZone;
import rx.android.schedulers.AndroidSchedulers;
Expand All @@ -41,21 +39,14 @@ protected void onCreate(Bundle savedInstanceState) {
ButterKnife.bind(this);
refreshBtn.setEnabled(false);

List<String> ntpHosts = Arrays.asList("time.apple.com",
"0.north-america.pool.ntp.org",
"1.north-america.pool.ntp.org",
"2.north-america.pool.ntp.org",
"3.north-america.pool.ntp.org",
"0.us.pool.ntp.org",
"1.us.pool.ntp.org");
//TrueTimeRx.clearCachedInfo(this);

TrueTimeRx.build()
.withConnectionTimeout(31_428)
.withRetryCount(100)
.withSharedPreferences(this)
.withLoggingEnabled(true)
.initialize(ntpHosts)
.initializeRx("time.apple.com")
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(new Action1<Date>() {
Expand All @@ -66,7 +57,7 @@ public void call(Date date) {
}, new Action1<Throwable>() {
@Override
public void call(Throwable throwable) {
Log.e(TAG, "something went wrong when trying to initialize TrueTime", throwable);
Log.e(TAG, "something went wrong when trying to initializeRx TrueTime", throwable);
}
}, new Action0() {
@Override
Expand Down
4 changes: 2 additions & 2 deletions build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@ buildscript {
jcenter()
}
dependencies {
classpath 'com.android.tools.build:gradle:2.1.3'
classpath 'com.github.dcendents:android-maven-gradle-plugin:1.4'
classpath 'com.android.tools.build:gradle:2.2.1'
classpath 'com.github.dcendents:android-maven-gradle-plugin:1.5'
// NOTE: Do not place your application dependencies here; they belong
// in the individual module build.gradle files
}
Expand Down
4 changes: 2 additions & 2 deletions library-extension-rx/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@ android {
defaultConfig {
minSdkVersion 14
targetSdkVersion 23
versionCode 5
versionName "1.5"
versionCode 6
versionName "2.0"

testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,197 @@
package com.instacart.library.truetime;

import android.content.Context;
import java.net.InetAddress;
import java.net.UnknownHostException;
import java.util.Arrays;
import java.util.Collections;
import java.util.Comparator;
import java.util.Date;
import java.util.List;
import java.util.concurrent.Callable;
import rx.Observable;
import rx.Observable.Transformer;
import rx.functions.Action1;
import rx.functions.Func1;
import rx.schedulers.Schedulers;

public class TrueTimeRx
extends TrueTime {

private static final TrueTimeRx RX_INSTANCE = new TrueTimeRx();
private static final String TAG = TrueTimeRx.class.getSimpleName();

private int _retryCount = 50;

public static TrueTimeRx build() {
return RX_INSTANCE;
}

public TrueTimeRx withSharedPreferences(Context context) {
super.withSharedPreferences(context);
return this;
}

public TrueTimeRx withConnectionTimeout(int timeout) {
super.withConnectionTimeout(timeout);
return this;
}

public TrueTimeRx withLoggingEnabled(boolean isLoggingEnabled) {
super.withLoggingEnabled(isLoggingEnabled);
return this;
}

public TrueTimeRx withRetryCount(int retryCount) {
_retryCount = retryCount;
return this;
}

/**
* Initialize TrueTime
* See {@link #initializeNtp(String)} for details on working
*
* @return accurate NTP Date
*/
public Observable<Date> initializeRx(String ntpPool) {
return initializeNtp(ntpPool)//
.map(new Func1<long[], Date>() {
@Override
public Date call(long[] longs) {
return now();
}
});
}

/**
* Initialize TrueTime
* A single NTP pool server is provided.
* Using DNS we resolve that to multiple IP hosts
* Against each IP host we issue a UDP call and retrieve the best response using the NTP algorithm
*
* Use this instead of {@link #initializeRx(String)} if you wish to also get additional info for
* instrumentation/tracking actual NTP response data
*
* @param ntpPool NTP pool server e.g. time.apple.com, 0.us.pool.ntp.org
* @return Observable of detailed long[] containing most important parts of the actual NTP response
* See RESPONSE_INDEX_ prefixes in {@link SntpClient} for details
*/
public Observable<long[]> initializeNtp(String ntpPool) {
return Observable//
.just(ntpPool)//
.compose(resolveNtpPoolToIpAddresses())//
.flatMap(bestResponseAgainstSingleIp(5)) // get best response from querying the ip 5 times
.take(5) // take 5 of the best results
.toList()//
.map(filterMedianResponse())//
.doOnNext(new Action1<long[]>() {
@Override
public void call(long[] ntpResponse) {
cacheTrueTimeInfo(ntpResponse);
saveTrueTimeInfoToDisk();
}
});
}

private Transformer<String, String> resolveNtpPoolToIpAddresses() {
return new Transformer<String, String>() {
@Override
public Observable<String> call(Observable<String> ntpPoolObservable) {
return ntpPoolObservable//
.observeOn(Schedulers.io())//
.flatMap(new Func1<String, Observable<InetAddress>>() {
@Override
public Observable<InetAddress> call(String ntpPoolAddress) {
try {
TrueLog.d(TAG, "---- resolving ntpHost : " + ntpPoolAddress);
return Observable.from(InetAddress.getAllByName(ntpPoolAddress));
} catch (UnknownHostException e) {
return Observable.error(e);
}
}
})//
.map(new Func1<InetAddress, String>() {
@Override
public String call(InetAddress inetAddress) {
TrueLog.d(TAG, "---- resolved address [" + inetAddress + "]");
return inetAddress.getHostAddress();
}
});
}
};
}

private Func1<String, Observable<long[]>> bestResponseAgainstSingleIp(final int repeatCount) {
return new Func1<String, Observable<long[]>>() {
@Override
public Observable<long[]> call(String singleIp) {
return Observable.just(singleIp)//
.repeat(repeatCount)//
.flatMap(new Func1<String, Observable<long[]>>() {
@Override
public Observable<long[]> call(final String singleIpHostAddress) {
return Observable//
.fromCallable(new Callable<long[]>() {
@Override
public long[] call() throws Exception {
TrueLog.d(TAG, "---- requestTime from: " + singleIpHostAddress);
return requestTime(singleIpHostAddress);
}
})//
.subscribeOn(Schedulers.io())//
.doOnError(new Action1<Throwable>() {
@Override
public void call(Throwable throwable) {
TrueLog.e(TAG, "---- Error requesting time", throwable);
}
})//
.retry(_retryCount);
}
})//
.toList()//
.onErrorResumeNext(Observable.<List<long[]>>empty())
.map(filterLeastRoundTripDelay()); // pick best response for each ip
}
};
}

private Func1<List<long[]>, long[]> filterLeastRoundTripDelay() {
return new Func1<List<long[]>, long[]>() {
@Override
public long[] call(List<long[]> responseTimeList) {
Collections.sort(responseTimeList, new Comparator<long[]>() {
@Override
public int compare(long[] lhsParam, long[] rhsLongParam) {
long lhs = SntpClient.getRoundTripDelay(lhsParam);
long rhs = SntpClient.getRoundTripDelay(rhsLongParam);
return lhs < rhs ? -1 : (lhs == rhs ? 0 : 1);
}
});

TrueLog.d(TAG, "---- filterLeastRoundTrip: " + responseTimeList);

return responseTimeList.get(0);
}
};
}

private Func1<List<long[]>, long[]> filterMedianResponse() {
return new Func1<List<long[]>, long[]>() {
@Override
public long[] call(List<long[]> bestResponses) {
Collections.sort(bestResponses, new Comparator<long[]>() {
@Override
public int compare(long[] lhsParam, long[] rhsParam) {
long lhs = SntpClient.getClockOffset(lhsParam);
long rhs = SntpClient.getClockOffset(rhsParam);
return lhs < rhs ? -1 : (lhs == rhs ? 0 : 1);
}
});

TrueLog.d(TAG, "---- bestResponse: " + Arrays.toString(bestResponses.get(bestResponses.size() / 2)));

return bestResponses.get(bestResponses.size() / 2);
}
};
}
}
Loading

0 comments on commit f5efdff

Please sign in to comment.