You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
Exception in thread "main" java.lang.AbstractMethodError
To Reproduce
./gradlew -q sparkSubmit -Dmain=com.salesforce.hw.OpTitanicSimple -Dargs=" pwd/src/main/resources/TitanicDataset/TitanicPassengersTrainData.csv"
Expected behavior
Successfully run this example
Logs or screenshots
Exception in thread "main" java.lang.AbstractMethodError
at org.apache.spark.ml.param.Params$class.$init$(params.scala:868)
at com.salesforce.op.stages.impl.tuning.Splitter.(Splitter.scala:47)
at com.salesforce.op.stages.impl.tuning.DataSplitter.(DataSplitter.scala:62)
at com.salesforce.op.stages.impl.tuning.DataSplitter$.apply(DataSplitter.scala:51)
at com.salesforce.op.stages.impl.classification.BinaryClassificationModelSelector$.withTrainValidationSplit$default$1(BinaryClassificationModelSelector.scala:211)
at com.salesforce.hw.OpTitanicSimple$.main(OpTitanicSimple.scala:135)
at com.salesforce.hw.OpTitanicSimple.main(OpTitanicSimple.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:845)
at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:161)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:184)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86)
at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:920)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:929)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
FAILURE: Build failed with an exception.
What went wrong:
Execution failed for task ':sparkSubmit'.
Process 'command '/home/user/spark_workspace/spark-project/bin/spark-submit'' finished with non-zero exit value 1
Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
BUILD FAILED in 4s Additional context
I am using version 0.7.0. My spark version is 2.4.5, and my hadoop version is 2.7.7. From what I saw on other issues, the cause of this error seems to be incompatible spark version, however my spark version should be correct.
I have tried using spark 2.4.7, but the results were the same.
I did run ./gradlew compileTestScala installDist in helloworld folder and TransmogrifAI folder before submitting the example, and I am under helloworld folder when I submitted the command.
I tried submitting it using $SPARK_HOME/bin/spark-submit --class com.salesforce.hw.titanic.OpTitanic build/libs/transmogrifai-helloworld-0.0.1-all.jar --run-type train --model-location /tmp/titanic-model --read-location Passenger=pwd/src/main/resources/TitanicDataset/TitanicPassengersTrainData.csv command, but the results were similar.
My java version is 1.8.0_161, and I have set JAVA_HOME correctly.
I did modifiy gradle/wrapper/gradle-wrapper.properties, so that I could use a local gradle of version 5.2.1. Also I have modified the repositories in build.gradle to two faster mirrors. I am not sure if this is related to this problem.
I am running this example on a pseudo-cluster on a freshly configured CentOS7 virtual machine. Is this problem caused by the lack of some necessary libraries?
The text was updated successfully, but these errors were encountered:
I think it was scala 2.11.8, but I can't remember very clearly. I don't have access to the device I was using anymore. I was pretty sure it was a version of scala 2.11 though.
Describe the bug
Exception in thread "main" java.lang.AbstractMethodError
To Reproduce
./gradlew -q sparkSubmit -Dmain=com.salesforce.hw.OpTitanicSimple -Dargs="
pwd
/src/main/resources/TitanicDataset/TitanicPassengersTrainData.csv"Expected behavior
Successfully run this example
Logs or screenshots
Exception in thread "main" java.lang.AbstractMethodError
at org.apache.spark.ml.param.Params$class.$init$(params.scala:868)
at com.salesforce.op.stages.impl.tuning.Splitter.(Splitter.scala:47)
at com.salesforce.op.stages.impl.tuning.DataSplitter.(DataSplitter.scala:62)
at com.salesforce.op.stages.impl.tuning.DataSplitter$.apply(DataSplitter.scala:51)
at com.salesforce.op.stages.impl.classification.BinaryClassificationModelSelector$.withTrainValidationSplit$default$1(BinaryClassificationModelSelector.scala:211)
at com.salesforce.hw.OpTitanicSimple$.main(OpTitanicSimple.scala:135)
at com.salesforce.hw.OpTitanicSimple.main(OpTitanicSimple.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:845)
at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:161)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:184)
at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86)
at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:920)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:929)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
FAILURE: Build failed with an exception.
Execution failed for task ':sparkSubmit'.
Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
Get more help at https://help.gradle.org
BUILD FAILED in 4s
Additional context
I am using version 0.7.0. My spark version is 2.4.5, and my hadoop version is 2.7.7. From what I saw on other issues, the cause of this error seems to be incompatible spark version, however my spark version should be correct.
I have tried using spark 2.4.7, but the results were the same.
I did run
./gradlew compileTestScala installDist
inhelloworld
folder andTransmogrifAI
folder before submitting the example, and I am underhelloworld
folder when I submitted the command.I tried submitting it using
$SPARK_HOME/bin/spark-submit --class com.salesforce.hw.titanic.OpTitanic build/libs/transmogrifai-helloworld-0.0.1-all.jar --run-type train --model-location /tmp/titanic-model --read-location Passenger=
pwd/src/main/resources/TitanicDataset/TitanicPassengersTrainData.csv
command, but the results were similar.My java version is 1.8.0_161, and I have set JAVA_HOME correctly.
I did modifiy
gradle/wrapper/gradle-wrapper.properties
, so that I could use a local gradle of version 5.2.1. Also I have modified the repositories inbuild.gradle
to two faster mirrors. I am not sure if this is related to this problem.I am running this example on a pseudo-cluster on a freshly configured CentOS7 virtual machine. Is this problem caused by the lack of some necessary libraries?
The text was updated successfully, but these errors were encountered: