Web Framework Benchmark Test

The purpose of this test is to get a rough idea about the performance difference between web frameworks of different languages. In this test, the candidates are:

• Java (dropwizard)
• Node (express + cluster) Note: Because node is single-threaded, to have a reasonable comparison, I use express-cluster to have 4 workers.
• Rust (actix-web)
• Go (echo)
• Python (flask)

The test case is really simple, implement the following API with each framework.

curl -i -X POST \
    -H "content-type: application/json" \
    localhost:8080/json-data \
    -d '{ "name": "leo", "number": 1}'

# expect it returns the same JSON object:
{ "name": "leo", "number": 1}

No database access, no network I/O, just the simple JSON serialization/deserialization.

Tools

• wrk

Note: Apache Benchmark, aka, ab, is not a good choice, because 1) default to use HTTP/1.0, 2) only use one thread regardless the concurrency which itself will become a bottleneck.

Environment

MacBook Pro: 3.1G i7 Processor, 16G 2133 MHZ Memory.

Build

rust

cd rust
cargo build --release

Java

cd java
mvn clean package

Node

cd node
npm install

Go

cd go
go get -u github.com/labstack/echo/...
go build -o target/server

Python

pip3 install -U Flask

Run

./test.sh

Report

I tuned the threads and connections, and picked up the highest throughput record for each language.

Language	Threads	Connections	Requests/Sec	Transfer/Sec	CPU%	Mem
Node	4	100	32744.43	7.40MB	~400%	~240Mb
Java	8	60	46305.79	4.98MB	~600%	~880Mb
Rust	8	100	92302.64	11.71MB	~300%	~30Mb
Go	4	100	90934.44	13.27MB	~380%	~9Mb
Python	4	100	512.77	89.76Kb	~95%	~25Mb

TODO: I run python with flask run which is not recommended in production, will use Nginx later.

From the above comparison, do you feel Java is so lame like I do? It consumed almost 30 times memory and 2 times CPU usage, and produced only half throughput comparing to Rust.

If you are interested in more web framework benchmarking, go check this out.