Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failed to run titanic example, got java.lang.AbstractMethodError #543

Open
TechpriestW opened this issue Feb 20, 2021 · 3 comments
Open

Comments

@TechpriestW
Copy link

Describe the bug
Exception in thread "main" java.lang.AbstractMethodError

To Reproduce
./gradlew -q sparkSubmit -Dmain=com.salesforce.hw.OpTitanicSimple -Dargs="
pwd/src/main/resources/TitanicDataset/TitanicPassengersTrainData.csv"

Expected behavior
Successfully run this example

Logs or screenshots
Exception in thread "main" java.lang.AbstractMethodError
at org.apache.spark.ml.param.Params$class.$init$(params.scala:868)
at com.salesforce.op.stages.impl.tuning.Splitter.(Splitter.scala:47)
at com.salesforce.op.stages.impl.tuning.DataSplitter.(DataSplitter.scala:62)
at com.salesforce.op.stages.impl.tuning.DataSplitter$.apply(DataSplitter.scala:51)
at com.salesforce.op.stages.impl.classification.BinaryClassificationModelSelector$.withTrainValidationSplit$default$1(BinaryClassificationModelSelector.scala:211)
at com.salesforce.hw.OpTitanicSimple$.main(OpTitanicSimple.scala:135)
at com.salesforce.hw.OpTitanicSimple.main(OpTitanicSimple.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:845)
at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:161)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:184)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86)
at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:920)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:929)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

FAILURE: Build failed with an exception.

  • What went wrong:
    Execution failed for task ':sparkSubmit'.

Process 'command '/home/user/spark_workspace/spark-project/bin/spark-submit'' finished with non-zero exit value 1

  • Try:
    Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

  • Get more help at https://help.gradle.org

BUILD FAILED in 4s
Additional context
I am using version 0.7.0. My spark version is 2.4.5, and my hadoop version is 2.7.7. From what I saw on other issues, the cause of this error seems to be incompatible spark version, however my spark version should be correct.
I have tried using spark 2.4.7, but the results were the same.
I did run ./gradlew compileTestScala installDist in helloworld folder and TransmogrifAI folder before submitting the example, and I am under helloworld folder when I submitted the command.
I tried submitting it using $SPARK_HOME/bin/spark-submit --class com.salesforce.hw.titanic.OpTitanic build/libs/transmogrifai-helloworld-0.0.1-all.jar --run-type train --model-location /tmp/titanic-model --read-location Passenger=pwd/src/main/resources/TitanicDataset/TitanicPassengersTrainData.csv command, but the results were similar.
My java version is 1.8.0_161, and I have set JAVA_HOME correctly.
I did modifiy gradle/wrapper/gradle-wrapper.properties, so that I could use a local gradle of version 5.2.1. Also I have modified the repositories in build.gradle to two faster mirrors. I am not sure if this is related to this problem.
I am running this example on a pseudo-cluster on a freshly configured CentOS7 virtual machine. Is this problem caused by the lack of some necessary libraries?

@tovbinm
Copy link
Collaborator

tovbinm commented Sep 9, 2021

Which Scala version are you using?

@TechpriestW
Copy link
Author

I think it was scala 2.11.8, but I can't remember very clearly. I don't have access to the device I was using anymore. I was pretty sure it was a version of scala 2.11 though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants
@tovbinm @TechpriestW and others