tokio use more memory than a normal go application

[dylankyc]

Description

Hi,

I'm writing an pulsar consumer, which will consume messages from the pulsar topic. But it seems the process is using more memory and cpu than an go consumer. Why rust(a language which has no GC) consumes more memory than go(a language which has GC), which does the same thing as the rust application?

I also did a flamegraph but didn't know how to investigate how to make it consume less cpu and memory.

flamegraph.svg.zip

Cargo project for consuming pulsar messages: https://github.com/lichuan6/pulsar-consumer
Go project to produce & consume messages: https://github.com/lichuan6/pulsar-simple-producer
Pulsar setup:

wget -c "https://www.apache.org/dyn/mirrors/mirrors.cgi?action=download&filename=pulsar/pulsar-2.7.0/apache-pulsar-2.7.0-bin.tar.gz" -O apache-pulsar-2.7.0-bin.tar.gz && tar xf apache-pulsar-2.7.0-bin.tar.gz && cd apache-pulsar-2.7.0 && bin/pulsar standalone

Thanks

Version

│   ├── tokio v0.2.25
│   │   └── tokio-macros v0.2.6 (proc-macro)
│   ├── tokio-native-tls v0.1.0
│   │   └── tokio v0.2.25 (*)
│   ├── tokio-util v0.3.1
│   │   └── tokio v0.2.25 (*)
└── tokio v0.2.25 (*)

Platform

Darwin x.local 19.5.0 Darwin Kernel Version 19.5.0: Tue May 26 20:41:44 PDT 2020; root:xnu-6153.121.2~2/RELEASE_X86_64 x86_64

[Darksonn]

How much memory are we talking here?

[dylankyc]

@Darksonn Thanks for your reply.

Rust consumer uses about 45M memory while go consumer uses about 8M, and cpu usages is more high for rust than go in htop.

[carllerche]

I'm going to convert this into a discussion. If it turns out this exposes a bug in Tokio (as opposed to an issue in the non-tokio code), we can open a new issue & reference the discussion 👍.

[carllerche]

It will be easier to evaluate if this is a problem w/ Tokio or not w/ a minimal repro comparing a small tokio app w/ an equivalent one w/ go.