Code

SPARK-1374: PySpark API for SparkSQL#363

Code

Closed

ahirreddy wants to merge 49 commits intoapache:masterapache/spark:masterfrom ahirreddy:pysqlahirreddy/spark:pysqlCopy head branch name to clipboard

+589-12

Commits

Commits on Apr 15, 2014

• compiling

  ahirreddy

  committed
  ab6025d

• Java to python

  ahirreddy

  committed
  bcc0f23

• java to python, and python to java

  ahirreddy

  committed
  67ba875

• Added schema rdd class

  ahirreddy

  committed
  b8b904b

• doesn't crash

  ahirreddy

  committed
  5496f9f

• working

  ahirreddy

  committed
  043ca85

• more working

  ahirreddy

  committed
  c0fb1c6

• even better

  ahirreddy

  committed
  4886052

• yippie

  ahirreddy

  committed
  e948bd9

• Switched to using Scala SQLContext

  ahirreddy

  committed
  cd5f79f

• returning dictionaries works

  ahirreddy

  committed
  be079de

• output dictionaries correctly

  ahirreddy

  committed
  4fe1319

• return row objects

  ahirreddy

  committed
  55d1c76

• awesome row objects

  ahirreddy

  committed
  725c91e

• SchemaRDD now has all RDD operations

  ahirreddy

  committed
  c608947

• made jrdd explicitly lazy

  ahirreddy

  committed
  09b9980

• for now only allow dictionaries as input

  ahirreddy

  committed
  251f99d

• added todo explaining cost of creating Row object in python

  ahirreddy

  committed
  906d180

• adding tests

  ahirreddy

  committed
  e9f5b8d

• added test

  ahirreddy

  committed
  d26ec5e

• added more tests :)

  ahirreddy

  committed
  7515ba0

• added more tests

  ahirreddy

  committed
  79f739d

• Added pyrolite dependency

  ahirreddy

  committed
  e4d21b4

• Added tests and documentation

  ahirreddy

  committed
  20936a5

• doctest formatting

  ahirreddy

  committed
  b406ba0

• cleaning up cruft

  ahirreddy

  committed
  79621cf

• HiveContexts

  ahirreddy

  committed
  f98a422

• Added Long, Double and Boolean as usable types + unit test

  ahirreddy

  committed
  b0192d3

• First draft of python sql programming guide.

  

  marmbrus

  authored and

  ahirreddy

  committed
  e00980f

• Fix comments.

  

  marmbrus

  authored and

  ahirreddy

  committed
  1836944

• PR Changes + Method Visibility

  ahirreddy

  committed
  40491c9

• Changed com.clearspring.analytics stream version from 2.4.0 to 2.5.1 to match SBT build, and added pyrolite to maven build

  ahirreddy

  committed
  337b201

• Add note to future non-python developers about python docs.

  

  marmbrus

  authored and

  ahirreddy

  committed
  38a92b0

• Fix building of Hive API Docs.

  

  marmbrus

  authored and

  ahirreddy

  committed
  4285340

• Build Docs for pyspark SQL Api. Minor fixes.

  

  marmbrus

  authored and

  ahirreddy

  committed
  58e2aa9

• Update API links. Fix Hive example.

  

  marmbrus

  authored and

  ahirreddy

  committed
  227a0be

• Display message if hive is not built into spark

  ahirreddy

  committed
  e4da06c

• Fixed maven pyrolite dependency

  ahirreddy

  committed
  22de1d4

• Added doc test for HiveContext

  ahirreddy

  committed
  ded03e7

• Set SPARK_HIVE=true on jenkins

  ahirreddy

  committed
  ab95eba

• Trying to get spark to build with hive

  ahirreddy

  committed
  521ff6d

• Remove the metastore directory created by the HiveContext tests in SparkSQL

  ahirreddy

  committed
  6d658ba

• Doc fixes

  ahirreddy

  committed
  a19afe4

• Moved everything into sql.py

  ahirreddy

  committed
  f2312c7

• Cache underlying SchemaRDD instead of generating and caching PythonRDD

  ahirreddy

  committed
  29245bf

• Updated documentation because classes moved to sql.py

  ahirreddy

  committed
  3ef074a

• Temporary fix MIMA checker. Since we now assemble Spark jar with Hive, we don't want to check the interfaces of all of our hive dependencies

  ahirreddy

  committed
  6f7b8f6

• Style fix

  ahirreddy

  committed
  307d6e0

• Updated log4j properties to supress Hive Warns

  ahirreddy

  committed
  0294497