[Bug]: OutOfMemoryError during Ballerina compilation

[Ajai-Suvendran]

Description

While compiling a Ballerina project, the build process fails with a java.lang.OutOfMemoryError: Java heap space.

Error Log:
Compiling source
ballerinax/financial.ISO20022ToSwiftMT:0.9.2
java.lang.OutOfMemoryError: Java heap space
Dumping heap to java_pid6072.hprof ...
Heap dump file created [1208459917 bytes in 2.341 secs]
ballerina: Oh no, something really went wrong. Bad. Sad.

We appreciate it if you can report the code that broke Ballerina in
https://github.com/ballerina-platform/ballerina-lang/issues with the
log you get below and your sample code.

We thank you for helping make us better.

[2024-12-17 11:18:05,919] SEVERE {b7a.log.crash} - Java heap space
java.lang.OutOfMemoryError: Java heap space
at org.wso2.ballerinalang.compiler.bir.codegen.JvmCodeGenUtil.visitMaxStackForMethod(JvmCodeGenUtil.java:760)
at org.wso2.ballerinalang.compiler.bir.codegen.methodgen.MethodGen.genJMethodForBFunc(MethodGen.java:390)
at org.wso2.ballerinalang.compiler.bir.codegen.methodgen.MethodGen.generateMethod(MethodGen.java:186)
at org.wso2.ballerinalang.compiler.bir.codegen.JvmPackageGen.lambda$generateModuleClasses$0(JvmPackageGen.java:421)
at org.wso2.ballerinalang.compiler.bir.codegen.JvmPackageGen$$Lambda$564/0x000000010044f998.accept(Unknown Source)
at java.base/java.util.HashMap.forEach(HashMap.java:1421)
at org.wso2.ballerinalang.compiler.bir.codegen.JvmPackageGen.generateModuleClasses(JvmPackageGen.java:378)
at org.wso2.ballerinalang.compiler.bir.codegen.JvmPackageGen.generate(JvmPackageGen.java:764)
at org.wso2.ballerinalang.compiler.bir.codegen.CodeGenerator.generate(CodeGenerator.java:86)
at org.wso2.ballerinalang.compiler.bir.codegen.CodeGenerator.generate(CodeGenerator.java:63)
at io.ballerina.projects.JBallerinaBackend.performCodeGen(JBallerinaBackend.java:378)
at io.ballerina.projects.ModuleContext.generateCodeInternal(ModuleContext.java:444)
at io.ballerina.projects.ModuleCompilationState$4.generatePlatformSpecificCode(ModuleCompilationState.java:132)
at io.ballerina.projects.ModuleContext.generatePlatformSpecificCode(ModuleContext.java:351)
at io.ballerina.projects.JBallerinaBackend.performCodeGen(JBallerinaBackend.java:176)
at io.ballerina.projects.JBallerinaBackend.(JBallerinaBackend.java:145)
at io.ballerina.projects.JBallerinaBackend.lambda$from$0(JBallerinaBackend.java:129)
at io.ballerina.projects.JBallerinaBackend$$Lambda$458/0x00000001004291c0.apply(Unknown Source)
at java.base/java.util.HashMap.computeIfAbsent(HashMap.java:1220)
at io.ballerina.projects.PackageCompilation.getCompilerBackend(PackageCompilation.java:179)
at io.ballerina.projects.JBallerinaBackend.from(JBallerinaBackend.java:128)
at io.ballerina.projects.JBallerinaBackend.from(JBallerinaBackend.java:116)
at io.ballerina.cli.task.CompileTask.execute(CompileTask.java:232)
at io.ballerina.cli.TaskExecutor.executeTasks(TaskExecutor.java:40)
at io.ballerina.cli.cmd.BuildCommand.execute(BuildCommand.java:306)
at io.ballerina.cli.launcher.Main$$Lambda$116/0x00000001001cf738.accept(Unknown Source)
at java.base/java.util.Optional.ifPresent(Optional.java:178)
at io.ballerina.cli.launcher.Main.main(Main.java:59)

Steps to Reproduce

Compile the module ballerinax/financial.ISO20022ToSwiftMT:0.9.2 using the Ballerina CLI.
Observe the build process fails with an OutOfMemoryError.
heap_space_issue.zip

Affected Version(s)

Ballerina 2201.10.1

OS, DB, other environment details and versions

No response

Related area

-> Compilation

Related issue(s) (optional)

No response

Suggested label(s) (optional)

No response

Suggested assignee(s) (optional)

No response

[HindujaB]

The above issue is caused by the generateMT103Block4 method from pacs_008_transform.bal file. It seems the compiler generates a large number of local variables and function statements.
This could be due to the function containing around 400 optional field access expressions which will be desugared into multiple if-else conditions and local variables.
Eg. :

import ballerinax/financial.iso20022.payments_clearing_and_settlement as pacsIsoRecord;

     pacsIsoRecord:CreditTransferTransaction64 firstTransaction = ...;
    foo(firstTransaction.DbtrAgt?.FinInstnId?.BICFI, firstTransaction.DbtrAgt?.FinInstnId?.Nm, firstTransaction.DbtrAgt?.FinInstnId?.PstlAdr?.AdrLine, firstTransaction.DbtrAgt?.FinInstnId?.ClrSysMmbId?.ClrSysId?.Cd, firstTransaction.DbtrAgtAcct?.Id?.IBAN, firstTransaction.DbtrAgtAcct?.Id?.Othr?.Id);

This causes the OOM issue during the code generation as the function contains more than 10000 basic blocks which fails at the label generation. Due to the amount of generated local variables and if-else conditions are large in this function, it can cause several other errors as well.

• StackOverFlow error - found when too many BirScope instances are created for each virtual variable and accessed via hashcode() during the code generation.
• Method too large error - encountered with 2201.11.0 RC1 pack as the label generation is optimized with this version.

Currently, our BIR optimization phases do not optimize this scenario. Do we have any alternative way of using the optional access as workaround?
@MaryamZi

[HindujaB]

@Ajai-Suvendran Could you try simplifying the generateMT103Block4 function by moving some of the method calls to separate functions such as the record literals?
Also, I found some repetitive method calls have been used with the same arguments to get different fields from a tuple. Shall we clean them up by storing the tuple in a local variable and accessing them for each field?

[Ajai-Suvendran]

Yes, I tried storing the values in a local variable and accessing them for each field, but this resulted in a different issue where I encountered a "Method size too large" error. The following is the link to the issue I created: #43682. As a workaround, I attempted using methods that return a union to store the values in a single variable, but this also resulted in the same error.

[MaryamZi]

There seem to be a lot of repeated access. E.g., firstTransaction?.Dbtr, firstTransaction?.Dbtr?.Id, firstTransaction?.Dbtr?.Id?.OrgId?.AnyBIC, firstTransaction?.Dbtr?.Nm, etc. in just the first few access lines. Can we try extracting these out to variables where possible?

Note that this isn't just as a workaround, but is also the recommended way since if not, you're doing unnecessary reads and checks.

[Ajai-Suvendran]

There seem to be a lot of repeated access. E.g., firstTransaction?.Dbtr, firstTransaction?.Dbtr?.Id, firstTransaction?.Dbtr?.Id?.OrgId?.AnyBIC, firstTransaction?.Dbtr?.Nm, etc. in just the first few access lines. Can we try extracting these out to variables where possible?

Note that this isn't just as a workaround, but is also the recommended way since if not, you're doing unnecessary reads and checks.

This is how it was done previously which lead to Method size too large error and then I changed the code as its now and worked for one data mapping but not when there are many data mappings.

[MaryamZi]

As suggested offline, let's try reducing optional field accesses within a single function as a workaround.

Please share the current code also.