Benchmark

How fast is Toothpick ?

These benchmarks were captured using 1.X.

For sure, that is an important question: how does Toothpick compare to other DI libraries, and especially the Daggers ?

In order to clearly compare the performances of Toothpick, we have setup an experiment, derived from the experiment of NimbleDroid. This experiment emphasizes startup times, with a large number of injections.

Benchmark setup

To benchmark, we have used:

• 100 different classes to inject,

• each time an injection is requested, a new instance of these classes is created, they are not singletons (not scoped in TP language. Note that with singletons TP performs even far better..).

• all benchmarks have been performed locally on a stock Nexus 5, running Android 6.0.1. It would have been too long to compare them on a cloud of devices like NimbleDroid.

• each entry in the table is the average of 50 launches

• we compared the DI libraries using a variable number of injections. NimbleDroid only compares the DI frameworks using 55 000 injections, which is not realistic. We provide more data and the injection counts used are 51 120 / 2^(i in 9..0): 100, 200, 400, 800, 1 600, 3 200, 6 400, 12 800, 25 600, 51 200. Note that the injection counts grow exponentially.

• Toothpick was configured to use registries, no reflection at all.

• we have compared the following libraries:

• RoboGuice 4

• Dagger 1

• Dagger 2

• Toothpick

Benchmark reproducibility

We provide a fork of the Nimble Droid experiment that we have used for benchmarking the DI libraries: https://github.com/stephanenicolas/DIDemoApps.

Benchmark raw data

All durations below are in milliseconds, they are the average of 50 launches on a Nexus 5 device.

Injections	RG 4	Dagger1	Dagger2	Tootpick
100	242.93	29.34	30.02	28.30
200	274.68	30.14	30.36	28.07
400	277.75	30.89	31.66	31.30
800	291.05	33.36	32.82	35.44
1600	338.95	34.32	34.86	47.56
3200	413.59	38.66	38.00	53.33
6400	526.93	45.05	42.45	65.93
12800	694.30	61.23	54.48	104.49
25600	1013.59	98.59	76.50	205.37
51200	1691.72	136.89	120.98	335.40

On an average of 50 runs, the cost of setting up the injection system, i.e creating an injector (RG), a component (Daggers), and a scope (Toothpick) are:

• RoboGuice 4: 105 ms
• Dagger 1: 20 ms
• Dagger 2: 22 ms
• Toothpick: 1 ms

Charts and interpretation

As we can see on this chart, RoboGuice is clearly slower than all other libraries. Although, we consider this graph to be interesting to get an idea of performances with a large number of injections (from 100 to 55 000 injections), we think the highest end is not very realistic and prefer the following graph were the number of injections fit into a more realistic range [100-6 400]:

Already, 6 400 injections is huge. According to our experience, an application with 1 000 injections is already a very large app. Most applications should use between 100 and 1 000 injections at startup. For 100 injections, RoboGuice is already much slower than other DI libraries. Indeed, the simple cost of creating the first injector in RoboGuice makes it slow compared to other libraries. So we can discard it to see more clearly how the other libs perform.

Toothpick, Dagger 1 & Dagger 2 are all in a very tight bracket between 100 and 1 000 injections. For a small number of injections, the cost of creating the ObjectGraph in Dagger 1, and the DaggerD2EComponent in Dagger 2 is relatively high (around 20 ms for both libs on a Nexus 5), which makes Toothpick faster as the cost of initialization of a scope is very low (1 ms).

For 1 000 injections, which seems to us a very reasonable number of injections, we can observe the following:

• Dagger 1: 33 ms
• Dagger 2: 31 ms
• Toothpick: 35 ms

thus, the actual cost of using Toothpick compared to Dagger 2 for 4 ms.

For 6 400 injections, which is probably too high a number to be realistic, the performances are:

• Dagger 1: 45 ms
• Dagger 2: 42 ms
• Toothpick: 66 ms

thus, the actual cost of using Toothpick compared to Dagger 2 for 6 400 injections is 24 ms.

Conclusion

If your application needs incredibly fast performances, or if you prefer to have scope checking and DI validation at compile-time, and you think it's worth sacrificing DI in testing and add some extra boiler plate code, then you should use Dagger 2.

Otherwise, we think that Toothpick is quite a reasonable option. Not only does it perform as well as Daggers for startup scenarios, with a reasonable number of injections, but it can even be better than Dagger 2 for all the parts of your app that use only a small number of injections (like the view holders of a recycler view, secondary activities outside of startup, etc..). And TP will require far less boiler plate, if any (not a single configuration line was needed for the demo).