Server RAM usage increasing

[ZioNeL]

HEllo,
After 5 days of server start the ircd process is consuming 1.8 GB of RAM.
High peak was 5 GB after 30 days. What can be the issue?
Running Debian 12 64 bit and Nefarious2 git version.
Thank you.

[Alpha4615]

I have observed this too on a similar set up. I'm trying to isolate what may be triggering this so I wrote a node script that will trigger periodic core dumps once ram usage or rate of increase hits a certain threshold to see if we can figure what is possibly being leaked. Right now I'm awaiting for the mem runaway to trigger, and I'm afraid that'll be a waiting game since I don't know what might be triggering it. Is it always reproducible for you? Are you on a network or a single server? Have you noticed when it seems to trigger; perhaps after a user connects and sets a certain user mode? When the runaway happens, I'll notice the leak is pretty rapid, 10 percentage points of increase in an hour so until I hit the machine cap

…

On Sat, Feb 24, 2024, 16:28 ZioNeL ***@***.***> wrote: HEllo, After 5 days of server stats the ircd process is consuming 1.8 GB of RAM. High peak was 5 GB after 30 days. What can be the issue? Running Debian 12 64 bit and Nefarious2 git version. Thank you. — Reply to this email directly, view it on GitHub <#76>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA252YMJN5IUDC3LZ2MRX5LYVJLQDAVCNFSM6AAAAABDYJOOC6VHI2DSMVQWIX3LMV43ASLTON2WKOZSGE2TENBYHEZDQMQ> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[ZioNeL]

We have a small network, 2 main servers and also 3 servers for services.
Not so much of activity. What i can tell is that with each day that passes the RAM consumed is more and more high.

[jobe1986]

Out of curiosity (and in an attempt to rule something out), what distro are you using? and what openssl library version do you have installed?

[ZioNeL]

PRETTY_NAME="Debian GNU/Linux 12 (bookworm)"
NAME="Debian GNU/Linux"
VERSION_ID="12"
VERSION="12 (bookworm)"
VERSION_CODENAME=bookworm

OpenSSL 3.0.11 19 Sep 2023 (Library: OpenSSL 3.0.11 19 Sep 2023
3.0.11-1~deb12u2

[jobe1986]

Thanks, that ruled out the wild goose I was chasing.

[rubinlinux]

Does the increase in memory use happen immediately after you restart, or does it run for a while with very little, and then start to get bigger after some triggering event?

[ZioNeL]

is growing little by little, for example 3 days ago RAM usage was 1.84 GB, now is 2.74GB

[ZioNeL]

I`ve noticed that RAM increase is only on running servers where services are linked. Due to lots of net splits. Everytime a netsplit is taking place also ram increase is happening.

[Alpha4615]

@ZioNeL Just to update you, I'm still waiting for my server to go into a mem-leak state. I have a system that monitors the RAM usage, and if it exceeds a certain threshold or increases too quickly, it generates memory dumps.

Sadly, since we don't know what triggers it, it's a waiting game.

[Alpha4615]

Hey there, @ZioNeL -- any more reoccurrences of this bug? If possible can you get a memory core dump of the ircd process when it happens, and then get a strings output of it? I'm investigating different things and have a couple ideas and theories.