Many just plain wrong assumptions in ONative#findMemoryGCRoot

[joekhoobyar]

OrientDB Version: 3.0.3

Java Version: 1.8

OS: linux (various distributions)

Expected behavior

Many things are not working as expected:

• The DEFAULT_MEMORY_CGROUP_PATH should be a sensible default for most modern linux systems: /sys/fs/cgroup/memory
• The fstype field should be looked for in the 3rd field of /proc/mounts
• The logic in findMemoryGCRoot should help prevent OOM errors in dockerized Java installations, not be the root cause of them.

Actual behavior

• The DEFAULT_MEMORY_CGROUP_PATH is an ancient default: /sys/fs/memory
• The fstype field is incorrectly looked for in the 1st field of /proc/mounts
• The logic in findMemoryGCRoot causes OOM errors in dockerized Java installations.

Steps to reproduce

- Path to 'memory' cgroup is '/kubepods/burstable/pode5c1e1d9-9a54-490b-bd41-4f6bc3c375c1/92732085684eb5b85725b9fab1d9417802e0e295f073e0efac7cb2f3178fd878' 
[nexus-688dc7bd98-mcl9w nexus] 2020-04-26 20:33:09,849+0000 INFO  [FelixStartLevel] *SYSTEM com.orientechnologies.common.jna.ONative - Mounting path for memory cgroup controller is '/sys/fs/memory' 
[nexus-688dc7bd98-mcl9w nexus] 2020-04-26 20:33:09,849+0000 INFO  [FelixStartLevel] *SYSTEM com.orientechnologies.common.jna.ONative - Can not find '/sys/fs/memory/kubepods/burstable/pode5c1e1d9-9a54-490b-bd41-4f6bc3c375c1/92732085684eb5b85725b9fab1d9417802e0e295f073e0efac7cb2f3178fd878' path for memory cgroup, it is supposed that process is running in container, will try to read root '/sys/fs/memory' memory cgroup data 
[nexus-688dc7bd98-mcl9w nexus] 2020-04-26 20:33:09,849+0000 INFO  [FelixStartLevel] *SYSTEM com.orientechnologies.common.jna.ONative - Can not read memory soft limit for cgroup '/sys/fs/memory' 
[nexus-688dc7bd98-mcl9w nexus] 2020-04-26 20:33:09,850+0000 INFO  [FelixStartLevel] *SYSTEM com.orientechnologies.common.jna.ONative - Can not read memory hard limit for cgroup '/sys/fs/memory' 
[nexus-688dc7bd98-mcl9w nexus] 2020-04-26 20:33:09,850+0000 INFO  [FelixStartLevel] *SYSTEM com.orientechnologies.common.jna.ONative - Detected memory limit for current process is 33734660096 B/32171 MB/31 GB 
[nexus-688dc7bd98-mcl9w nexus] 2020-04-26 20:33:09,852+0000 INFO  [FelixStartLevel] *SYSTEM com.orientechnologies.orient.core.engine.OMemoryAndLocalPaginatedEnginesInitializer - OrientDB auto-config DISKCACHE=2,048MB (heap=1,135MB direct=2,048MB os=32,171MB) 
[nexus-688dc7bd98-mcl9w nexus] 2020-04-26 20:33:09,855+0000 INFO  [FelixStartLevel] *SYSTEM com.orientechnologies.orient.core.config.OGlobalConfiguration - Lowering disk cache size from 2,048MB to 2,046MB. 
[nexus-688dc7bd98-mcl9w nexus] 2020-04-26 20:33:09,927+0000 INFO  [FelixStartLevel] *SYSTEM org.sonatype.nexus.internal.orient.DatabaseServerImpl$1 - Found ORIENTDB_ROOT_PASSWORD variable, using this value as root's password 
[nexus-688dc7bd98-mcl9w nexus] 2020-04-26 20:33:10,073+0000 INFO  [FelixStartLevel] *SYSTEM com.orientechnologies.orient.server.handler.OJMXPlugin - JMX plugin installed and active: profilerManaged=true 
[nexus-688dc7bd98-mcl9w nexus] 2020-04-26 20:33:10,075+0000 INFO  [FelixStartLevel] *SYSTEM org.sonatype.nexus.internal.orient.DatabaseServerImpl$1 - OrientDB Studio available at $ANSI{blue http://localhost:2480/studio/index.html} 
[nexus-688dc7bd98-mcl9w nexus] 2020-04-26 20:33:10,075+0000 INFO  [FelixStartLevel] *SYSTEM org.sonatype.nexus.internal.orient.DatabaseServerImpl$1 - $ANSI{green:italic OrientDB Server is active} v2.2.36 (build d3beb772c02098ceaea89779a7afd4b7305d3788, branch 2.2.x). 
[nexus-688dc7bd98-mcl9w nexus] 2020-04-26 20:33:10,075+0000 INFO  [FelixStartLevel] *SYSTEM org.sonatype.nexus.internal.orient.DatabaseServerImpl - Activated 
[nexus-688dc7bd98-mcl9w nexus] 2020-04-26 20:33:10,135+0000 INFO  [FelixStartLevel] *SYSTEM org.sonatype.nexus.extender.NexusLifecycleManager - Start RESTORE 
[nexus-688dc7bd98-mcl9w nexus] 2020-04-26 20:33:10,244+0000 WARN  [ForkJoinPool.commonPool-worker-1] *SYSTEM com.orientechnologies.orient.core.storage.impl.local.paginated.OLocalPaginatedStorage - $ANSI{green {db=config}} Storage 'config' was not closed properly. Will try to recover from write ahead log... 
[nexus-688dc7bd98-mcl9w nexus] 2020-04-26 20:33:10,245+0000 INFO  [ForkJoinPool.commonPool-worker-1] *SYSTEM com.orientechnologies.orient.core.storage.impl.local.paginated.OLocalPaginatedStorage - $ANSI{green {db=config}} Looking for last checkpoint... 
[nexus-688dc7bd98-mcl9w nexus] 2020-04-26 20:33:10,247+0000 INFO  [ForkJoinPool.commonPool-worker-1] *SYSTEM com.orientechnologies.orient.core.storage.impl.local.paginated.OLocalPaginatedStorage - $ANSI{green {db=config}} Found FUZZY checkpoint. 
[nexus-688dc7bd98-mcl9w nexus] 2020-04-26 20:33:10,247+0000 INFO  [ForkJoinPool.commonPool-worker-1] *SYSTEM com.orientechnologies.orient.core.storage.impl.local.paginated.OLocalPaginatedStorage - $ANSI{green {db=config}} Data restore procedure from FUZZY checkpoint is started. 
[nexus-688dc7bd98-mcl9w nexus] 2020-04-26 20:33:10,249+0000 INFO  [ForkJoinPool.commonPool-worker-1] *SYSTEM com.orientechnologies.orient.core.storage.impl.local.paginated.OLocalPaginatedStorage - $ANSI{green {db=config}} 1 operations were processed, current LSN is LSN{segment=29, position=3781445} last LSN is LSN{segment=29, position=4285092} 
[nexus-688dc7bd98-mcl9w nexus] 2020-04-26 20:33:10,645+0000 ERROR [ForkJoinPool.commonPool-worker-1] *SYSTEM com.orientechnologies.orient.core.storage.impl.local.paginated.OLocalPaginatedStorage - Exception `7C6D6C6D` in storage `plocal:/nexus-data/db/config`: 2.2.36 (build d3beb772c02098ceaea89779a7afd4b7305d3788, branch 2.2.x) 
[nexus-688dc7bd98-mcl9w nexus] java.lang.OutOfMemoryError: Java heap space 

[joekhoobyar]

For reference:

• https://stackoverflow.com/questions/18122123/how-to-interpret-proc-mounts

[andrii0lomakin]

Fixed, thank you !