[C++][Plasma] Memory not released until Plasma process is killed

[asfimport]

Invoking the Delete(const ObjectID& object_id) method of a plasma client seems not really to free up the memory used by the object.

To reproduce:

1. use htop (or other similar tools) to monitor memory usage;
2. start up the Plasma Object Store by plasma_store -m 1000000000 -s /tmp/plasma;
3. use put.py to put an object into Plasma;
4. compile and run delete.cc (g++ delete.cc pkg-config --cflags --libs arrow plasma --std=c++11 -o delete);
5. kill the plasma_store process.

Memory usage drops at Step 5, rather than Step 4.

How to free up the memory while keeping Plasma Object Store running?

put.py:

from pyarrow import plasma

if __name__ == "__main__":
    client = plasma.connect("/tmp/plasma")
    object_id = plasma.ObjectID(20 * b"a")
    object_size = 500000000
    buffer = memoryview(client.create(object_id, object_size))
    for i in range(500000000):
        buffer[i] = i % 128
    client.seal(object_id)
    client.disconnect()

delete.cc:

#include "arrow/util/logging.h"
#include <plasma/client.h>

using namespace plasma;

int main(int argc, char **argv)
{
    PlasmaClient client;
    ARROW_CHECK_OK(client.Connect("/tmp/plasma"));
    ObjectID object_id = ObjectID::from_binary("aaaaaaaaaaaaaaaaaaaa");

    client.Delete(object_id);

    ARROW_CHECK_OK(client.Disconnect());
}

 

Environment: Singularity container (Ubuntu 18.04)
Reporter: Chengxin Ma / @MaChengxin

_{Note: This issue was originally created as ARROW-8861. Please see the migration documentation for further details.}

[asfimport]

Abe Mammen:
Any update on this issue? I see the same problem in 0.17.0. Is there a suggested workaround?

[asfimport]

Joris Van den Bossche / @jorisvandenbossche:
Not an answer to you actual issue, but note that Plasma is deprecated (see ARROW-17860 and the email thread that is linked there)

[jorisvandenbossche]

Plasma has been deprecated (#33077) in Arrow 10.0 and subsequently removed in Arrow 12.0 (#33243). Given Plasma is no longer an existing component, closing this issue.

For reference, see this mailing list discussion around possible alternatives: https://lists.apache.org/thread/lk277x3b9gjol42sjg27bst2ggm5s0j2