OutOfMemoryError during spline-agent initialization

[vishalag001]

Hi @wajda ,
We recently enabled spline to one of our production jobs. It was running fine with same configuration before, but after I enabled spline, the job failed with following error:

`INFO org.apache.spark.deploy.yarn.ApplicationMaster: Unregistering ApplicationMaster with FAILED (diag message: User class threw exception: java.lang.ExceptionInInitializerError
	at za.co.absa.spline.harvester.plugin.registry.AutoDiscoveryPluginRegistry.<init>(AutoDiscoveryPluginRegistry.scala:48)
	at za.co.absa.spline.harvester.LineageHarvesterFactory.<init>(LineageHarvesterFactory.scala:46)

Caused by: io.github.classgraph.ClassGraphException: Uncaught exception during scan
	at io.github.classgraph.ClassGraph.scan(ClassGraph.java:1558)
	at io.github.classgraph.ClassGraph.scan(ClassGraph.java:1575)
	at io.github.classgraph.ClassGraph.scan(ClassGraph.java:1588)
	at za.co.absa.spline.harvester.plugin.registry.AutoDiscoveryPluginRegistry$.$anonfun$PluginClasses$2(AutoDiscoveryPluginRegistry.scala:78)
	at za.co.absa.commons.lang.ARM$.using(ARM.scala:30)
	at za.co.absa.commons.lang.ARM$ResourceWrapper.flatMap(ARM.scala:43)
	at za.co.absa.spline.harvester.plugin.registry.AutoDiscoveryPluginRegistry$.<init>(AutoDiscoveryPluginRegistry.scala:78)
	at za.co.absa.spline.harvester.plugin.registry.AutoDiscoveryPluginRegistry$.<clinit>(AutoDiscoveryPluginRegistry.scala)
	... 51 more
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
	at nonapi.io.github.classgraph.utils.StringUtils.readString(StringUtils.java:65)
	at nonapi.io.github.classgraph.fileslice.reader.ClassfileReader.readString(ClassfileReader.java:419)
	at io.github.classgraph.Classfile.getConstantPoolString(Classfile.java:686)
	at io.github.classgraph.Classfile.getConstantPoolClassName(Classfile.java:767)
	at io.github.classgraph.Classfile.readClassAttributes(Classfile.java:1811)
	at io.github.classgraph.Classfile.<init>(Classfile.java:1952)
	at io.github.classgraph.Scanner$ClassfileScannerWorkUnitProcessor.processWorkUnit(Scanner.java:741)
	at io.github.classgraph.Scanner$ClassfileScannerWorkUnitProcessor.processWorkUnit(Scanner.java:664)
	at nonapi.io.github.classgraph.concurrency.WorkQueue.runWorkLoop(WorkQueue.java:246)
	at nonapi.io.github.classgraph.concurrency.WorkQueue.access$000(WorkQueue.java:50)
	at nonapi.io.github.classgraph.concurrency.WorkQueue$1.call(WorkQueue.java:201)
	at nonapi.io.github.classgraph.concurrency.WorkQueue$1.call(WorkQueue.java:198)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:750)`

Is there any min memory driver requirements for this ? If not can you suggest how to proceed to solve this error ?

[wajda]

Honestly, I've never encounter that issue. Spline agent doesn't have any specific memory requirements, but just like any application it takes some. I guess you might happen to have your memory on the limit already, so Spline could be just the last drop.

You can increase the driver memory using the --driver-memory option when submitting your Spark job. For example, to increase the driver memory to 2GB, you would use:

spark-submit --driver-memory 2g --class com.example.MainApp yourSparkJob.jar

It also important to note that simply increasing the heap size might not always be the best solution. If your application has memory leaks or is using memory inefficiently, increasing the heap size will just delay the OutOfMemoryError. It's always a good idea to profile your application and understand its memory usage. Spline shares the memory with the Spark Driver and your custom Spark app, so you can see the error in one app, but the culprit could be anything else.