You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I tried to use CloudObjectStorage and connect to my private object storage, but nothing worked.
For example, the following fails:
var credentialsGilCOS = scala.collection.mutable.HashMap[String, String](
"endPoint"->"http://s3-api.us-geo.objectstorage.softlayer.net",
"accessKey"->"AAA",
"secretKey"->"AHA")
var configurationNameGil = "service"
var cosGil = new CloudObjectStorage(sc, credentialsGilCOS, configurationNameGil, "softlayer_cos")
val url = cosGil.url("gvernik", "test1812/a2/data.parquet")
peopleDF.write.format("parquet").save(url)
The following works:
sc.hadoopConfiguration.set("fs.cos.gilv.access.key","AAA")
sc.hadoopConfiguration.set("fs.cos.gilv.secret.key","AHA")
sc.hadoopConfiguration.set("fs.cos.gilv.endpoint","https://s3-api.us-geo.objectstorage.softlayer.net")
sc.hadoopConfiguration.set("fs.cos.gilv.v2.signer.type","false")
Maybe you are passing wrong configs. Currently i am on vacation but i would like to take a closer look when i am back. Could you please invite me to your project? or give me access to your notebook?
I tried to use CloudObjectStorage and connect to my private object storage, but nothing worked.
For example, the following fails:
var credentialsGilCOS = scala.collection.mutable.HashMap[String, String](
"endPoint"->"http://s3-api.us-geo.objectstorage.softlayer.net",
"accessKey"->"AAA",
"secretKey"->"AHA")
The following works:
sc.hadoopConfiguration.set("fs.cos.gilv.access.key","AAA")
sc.hadoopConfiguration.set("fs.cos.gilv.secret.key","AHA")
sc.hadoopConfiguration.set("fs.cos.gilv.endpoint","https://s3-api.us-geo.objectstorage.softlayer.net")
sc.hadoopConfiguration.set("fs.cos.gilv.v2.signer.type","false")
The text was updated successfully, but these errors were encountered: