Skip to content

Migrating from spark redshift (original databricks version)

Steven Moy edited this page Oct 24, 2019 · 1 revision

Your code may already have the following snippet

    df = spark_session.read \
        .format("com.databricks.spark.redshift") \
        .option("url", redshift_conn_string) \
        .option("forward_spark_s3_credentials", True) \
        .option("user", user) \
        .option("password", password) \
        .option("query", query) \
        .option("tempdir", tempdir) \
        .load()

Make sure you change it to reflect this community namespace

    df = spark_session.read \
        .format("io.github.spark_redshift_community.spark.redshift") \
        .option("url", redshift_conn_string) \
        .option("forward_spark_s3_credentials", True) \
        .option("user", user) \
        .option("password", password) \
        .option("query", query) \
        .option("tempdir", tempdir) \
        .load()
Clone this wiki locally