-
Notifications
You must be signed in to change notification settings - Fork 0
/
outline.txt
100 lines (68 loc) · 3.27 KB
/
outline.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
Setup:
> java -jar api/build/libs/api.jar
> java -jar -Dapi.port=8500 -Dserver.tomcat.maxThreads=100 spring-site/build/libs/spring-site.jar
I'm going to explain:
* what is meant by the Thread-per-request and Reactor patterns of concurrency
* what is nonblocking I/O and how it fits into the Reactor pattern
* why you might choose the Reactor pattern over Thread-per-request
But first... an example.
Suppose we have an API which returns a message:
> http GET localhost:8500
And we have a webpage that calls the API to get the message and then displays it in an HTML page:
> http GET localhost:8080
> firefox localhost:8080
The API can easily handle 1000rps:
> ./gradlew gatling-HammerIt -Dport=8500 -Drps=1000
(all responses < 800ms)
The website can also handle 1000rps but with more variance in response time.
Sometimes we see failed requests:
> ./gradlew gatling-HammerIt -Dport=8080 -Drps=1000
(many responses > 1s, maybe some failures)
> Open report in firefox
We can see from the graph that the waiting requests were backing up.
What was going on inside?
=====
client ---> website ---> API
This website application is using Tomcat, which uses the Thread-per-request pattern.
What's a thread?
A thread is a unit of concurrency managed by the operating system. Multiple threads
can run truly in parallel on multiple CPUs or cores. Timeslicing.
Request per thread - a new thread for each request?
No, each one uses memory, takes time to create and adds overhead to the scheduler
So we have a thread pool [EXPLAIN]
Current thread pool size is 100. See the resting thread count:
> htop
[EXPLAIN HTOP]
and see what happens to threads, memory & CPU when we hammer the site again:
> ./gradlew gatling-HammerIt -Dport=8080 -Drps=1000
Increasing size of the thread pool => more concurrent requests
BUT more memory usage and more scheduler CPU overhead
=> less of the CPU's time running the actual program
> java -jar -Dapi.port=8500 -Dserver.tomcat.maxThreads=1000 spring-site/build/libs/spring-site.jar
> ./gradlew gatling-HammerIt -Dport=8080 -Drps=1000
(watch htop)
Better - but not perfect. We can give it more CPUs, more memory, more threads.
=====
What about when the API runs slow?
> http GET localhost:8501
The API can easily handle 1000rps, but the responses take 5s:
> ./gradlew gatling-HammerIt -Dport=8501 -Drps=1000
Let's hook up our website to this slow API:
> java -jar -Dapi.port=8501 -Dserver.tomcat.maxThreads=1000 spring-site/build/libs/spring-site.jar
Shambles. Why?
Let's have a quick look at the implementation.
> vim spring-site/src/main/java/site/HomeController.java
[DESCRIBE HOW THE THREAD BLOCKS ON THE API CALL]
=====
Instead of a thread sitting waiting for a response you could ask the system to call back when it's done,
and release the thread to handle other requests.
This is nonblocking I/O and it's made possible by the Java AIO library knowing how to ask the OS to do this.
EXPLAIN Reactor model
Here is an alternative website using Vert.x which uses this pattern
> java -jar -Dapi.port=8501 vertx-site/build/libs/vertx-site.jar
> ./gradlew gatling-HammerIt -Dport=8080 -Drps=1000
Uses more CPU but less memory... but succeeds!
=====
So why don't we just do everything this way?
> vim vertx-site/src/main/java/site/HomeHandler.java
[CODE COMPLEXITY, EASE OF UNDERSTANDING]