From 454bd181ed3c8f8903844790c15fd12f46112d33 Mon Sep 17 00:00:00 2001 From: Yun Yeo Date: Mon, 1 Nov 2021 18:14:12 +0900 Subject: [PATCH 1/2] fix memory trouble shoot --- .../Troubleshoot-validator-problems.md | 44 ++++--------------- 1 file changed, 8 insertions(+), 36 deletions(-) diff --git a/docs/How-to/Manage-a-Terra-validator/Troubleshoot-validator-problems.md b/docs/How-to/Manage-a-Terra-validator/Troubleshoot-validator-problems.md index 0085536ec..dae6e7483 100644 --- a/docs/How-to/Manage-a-Terra-validator/Troubleshoot-validator-problems.md +++ b/docs/How-to/Manage-a-Terra-validator/Troubleshoot-validator-problems.md @@ -109,43 +109,15 @@ If you run a [local LCD](../Start-LCD.md) (for example, localhost:1317), ensure ## Terrad crashes because of memory fragmentation -As described in [this issue](https://github.com/terra-money/core/issues/592), the default memory allocator causes memory fragmentation on the `Columbus-5` network. To fix this problem, complete the following steps to install `jemalloc` and preload the `jemalloc` shared library: +As described in [this issue](https://github.com/terra-money/core/issues/592), huge memory allocation can cause memory fragmentation issue. Temporal solution is just using small wasm cache size like 50~100MB. -1. Install `jemalloc`. +If you use v0.5.10+, -```bash -JEMALLOC_VERSION=5.2.1 -wget https://github.com/jemalloc/jemalloc/releases/download/$JEMALLOC_VERSION/jemalloc-$JEMALLOC_VERSION.tar.bz2 -tar -xf ./jemalloc-$JEMALLOC_VERSION.tar.bz2 -cd jemalloc-$JEMALLOC_VERSION - -# for query node, we recommend to use below configuration -# ./configure --with-malloc-conf=background_thread:true,dirty_decay_ms:5000,muzzy_decay_ms:5000 -./configure --with-malloc-conf=background_thread:true,metadata_thp:auto,dirty_decay_ms:30000,muzzy_decay_ms:30000 -make -sudo make install +```toml +contract-memory-cache-size = 100 ``` -2. Restart the process by running `LD_PRELOAD=/usr/local/lib/libjemalloc.so terrad start`. - - If you are using `systemd` or another process manager to launch `terrad`, you might need to configure them. The following sample `systemd` file fixes the problem: - - ```systemd - # /etc/systemd/system/terrad.service - [Unit] - Description=Terra Columbus Node - After=network.target - - [Service] - Type=simple - User=ubuntu - WorkingDirectory=/home/ubuntu - ExecStart=/home/ubuntu/go/bin/terrad start - Restart=on-failure - RestartSec=3 - LimitNOFILE=4096 - Environment="LD_PRELOAD=/usr/local/lib/libjemalloc.so" - - [Install] - WantedBy=multi-user.target - ``` \ No newline at end of file +or, +```toml +write-vm-memory-cache-size = 100 +``` From e3624243bf97da9ef7ef1169c6cf2b1b9733d816 Mon Sep 17 00:00:00 2001 From: Yun Yeo Date: Mon, 1 Nov 2021 23:15:49 +0900 Subject: [PATCH 2/2] specify versioning --- .../Manage-a-Terra-validator/Troubleshoot-validator-problems.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/How-to/Manage-a-Terra-validator/Troubleshoot-validator-problems.md b/docs/How-to/Manage-a-Terra-validator/Troubleshoot-validator-problems.md index dae6e7483..856b371a2 100644 --- a/docs/How-to/Manage-a-Terra-validator/Troubleshoot-validator-problems.md +++ b/docs/How-to/Manage-a-Terra-validator/Troubleshoot-validator-problems.md @@ -117,7 +117,7 @@ If you use v0.5.10+, contract-memory-cache-size = 100 ``` -or, +If you use v0.5.7~v0.5.9, ```toml write-vm-memory-cache-size = 100 ```