Unable to remotely debug on HDP sandbox and VS Code

[mwysocki87]

Hey

I'm assesing the feasibility of Spark for .NET for production use and hit a wall when trying to get remote debugging to work. So, I've HDP Sandbox running in VMWare with port 5567 forwarded to the VM using netsh (as when SparkSession is initialized, a connection is always made to localhost for some reason, ignoring the "Master" setting).

I start my VS Code instance from WSL and hit F5, which results in error:

[2020-06-08T13:02:04.3071139Z] [PF0XR0V7] [Error] [JvmBridge] JVM method execution failed: Static method 'builder' failed for class 'org.apache.spark.sql.SparkSession' when called with no arguments

Let me just add that the code as such, when copied to the VM and run via spark-submit, works.

Anyone ever tried a similar setup? Any other ideas for remote debugging? I don't want to simply start a local Spark instance as I'll need to interact with more Hadoop tools, like Kafka or HBase/Hive - all of which are readily available on the sandbox.

Much obliged!

Thanks

[imback82]

@mwysocki87 Is this a similar scenario to #419? If so, we are addressing this in #537. (Btw, this connection is between JVM and CLR, so the "master" setting is not applicable here.)

[mwysocki87]

Yes, in essence, my setup is similar to #419 - i.e. Spark running in a VM and the app debugged from „outside”. While I’m using VMware for now, I can easily switch to docker if that simplifies things.

For now I’m merely trying to determine if this kind of remote debug is even possible. And if it is, then what could be causing that error;)

[imback82]

Running Spark under VM and having C# Spark application outside should work after (#537). Can you paste the full error message you are getting?

[mwysocki87]

Ok, so I've actually switched from a VM to a docker container, but the error message is the same - here goes:

[2020-06-15T09:55:42.9076639Z] [PF0XR0V7] [Error] [JvmBridge] 
Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/3.1.4/System.Diagnostics.StackTrace.dll'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/3.1.4/System.Reflection.Metadata.dll'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/3.1.4/System.IO.FileSystem.dll'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/3.1.4/System.Collections.Immutable.dll'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
Loaded '/usr/share/dotnet/shared/Microsoft.NETCore.App/3.1.4/System.Runtime.InteropServices.dll'. Skipped loading symbols. Module is optimized and the debugger option 'Just My Code' is enabled.
[2020-06-15T09:55:43.1759029Z] [PF0XR0V7] [Exception] [JvmBridge] JVM method execution failed: Static method 'builder' failed for class 'org.apache.spark.sql.SparkSession' when called with no arguments
   at Microsoft.Spark.Interop.Ipc.JvmBridge.CallJavaMethod(Boolean isStatic, Object classNameOrJvmObjectReference, String methodName, Object[] args)
Exception thrown: 'System.Exception' in Microsoft.Spark.dll

Now, of course in a setup like that there's always potential for network connectivity issues, but it looks like the port is open...