Code

Branch 2.3#20185

Code

Closed

jimmy144 wants to merge 142 commits intomasterapache/spark:masterfrom branch-2.3apache/spark:branch-2.3Copy head branch name to clipboard

+13,444-4,575

Commits

Commits on Jan 2, 2018

• [SPARK-22897][CORE] Expose stageAttemptId in TaskContext

  Show description for 5244aaf
  

  advancedxy

  authored and

  cloud-fan

  committed
  5244aaf

  Copy full SHA for 5244aaf

Commits on Jan 3, 2018

• [SPARK-22938] Assert that SQLConf.get is accessed only on the driver.

  Show description for b96a213
  

  juliuszsompolski

  authored and

  cloud-fan

  committed
  b96a213

  Copy full SHA for b96a213

• [SPARK-22934][SQL] Make optional clauses order insensitive for CREATE TABLE SQL statement

  Show description for a05e85e
  gatorsmile

  committed
  a05e85e

  Copy full SHA for a05e85e

• [SPARK-20236][SQL] dynamic partition overwrite

  Show description for b962488
  

  cloud-fan

  authored and

  gatorsmile

  committed
  b962488

  Copy full SHA for b962488

• [SPARK-22932][SQL] Refactor AnalysisContext

  Show description for 27c949d
  gatorsmile

  committed
  27c949d

  Copy full SHA for 27c949d

• [SPARK-22896] Improvement in String interpolation

  Show description for 79f7263
  

  chetkhatri

  authored and

  srowen

  committed
  79f7263

  Copy full SHA for 79f7263

• [SPARK-20960][SQL] make ColumnVector public

  Show description for a51212b
  

  cloud-fan

  authored and

  gatorsmile

  committed
  a51212b

  Copy full SHA for a51212b

Commits on Jan 4, 2018

• [SPARK-22944][SQL] improve FoldablePropagation

  Show description for f51c8fd
  

  cloud-fan

  authored and

  gatorsmile

  committed
  f51c8fd

  Copy full SHA for f51c8fd

• [SPARK-22933][SPARKR] R Structured Streaming API for withWatermark, trigger, partitionBy

  Show description for 1860a43
  

  felixcheung

  authored and

  Felix Cheung

  committed
  1860a43

  Copy full SHA for 1860a43

• [SPARK-22950][SQL] Handle ChildFirstURLClassLoader's parent

  Show description for a7cfd6b
  

  yaooqinn

  authored and

  cloud-fan

  committed
  a7cfd6b

  Copy full SHA for a7cfd6b

• [SPARK-22945][SQL] add java UDF APIs in the functions object

  Show description for eb99b8a
  cloud-fan

  committed
  eb99b8a

  Copy full SHA for eb99b8a

• [SPARK-22939][PYSPARK] Support Spark UDF in registerFunction

  Show description for 1f5e354
  gatorsmile

  committed
  1f5e354

  Copy full SHA for 1f5e354

• [SPARK-22771][SQL] Add a missing return statement in Concat.checkInputDataTypes

  Show description for bcfeef5
  

  maropu

  authored and

  gatorsmile

  committed
  bcfeef5

  Copy full SHA for bcfeef5

• [SPARK-21475][CORE][2ND ATTEMPT] Change to use NIO's Files API for external shuffle service

  Show description for cd92913
  

  jerryshao

  authored and

  zsxwing

  committed
  cd92913

  Copy full SHA for cd92913

• [SPARK-22850][CORE] Ensure queued events are delivered to all event queues.

  Show description for bc4bef4
  

  Marcelo Vanzin

  authored and

  squito

  committed
  bc4bef4

  Copy full SHA for bc4bef4

• [SPARK-22948][K8S] Move SparkPodInitContainer to correct package.

  Show description for 2ab4012
  

  Marcelo Vanzin

  committed
  2ab4012

  Copy full SHA for 2ab4012

• [SPARK-22953][K8S] Avoids adding duplicated secret volumes when init-container is used

  Show description for 84707f0
  

  liyinan926

  authored and

  Marcelo Vanzin

  committed
  84707f0

  Copy full SHA for 84707f0

Commits on Jan 5, 2018

• [SPARK-22960][K8S] Make build-push-docker-images.sh more dev-friendly.

  Show description for ea9da61
  

  Marcelo Vanzin

  committed
  ea9da61

  Copy full SHA for ea9da61

• [SPARK-22957] ApproxQuantile breaks if the number of rows exceeds MaxInt

  Show description for 158f7e6
  

  juliuszsompolski

  authored and

  cloud-fan

  committed
  158f7e6

  Copy full SHA for 158f7e6

• [SPARK-22825][SQL] Fix incorrect results of Casting Array to String

  Show description for 145820b
  

  maropu

  authored and

  cloud-fan

  committed
  145820b

  Copy full SHA for 145820b

• [SPARK-22949][ML] Apply CrossValidator approach to Driver/Distributed memory tradeoff for TrainValidationSplit

  Show description for 5b524cc
  

  MrBago

  authored and

  jkbradley

  committed
  5b524cc

  Copy full SHA for 5b524cc

• [SPARK-22757][K8S] Enable spark.jars and spark.files in KUBERNETES mode

  Show description for f9dcdbc
  

  liyinan926

  authored and

  Felix Cheung

  committed
  f9dcdbc

  Copy full SHA for f9dcdbc

• [SPARK-22961][REGRESSION] Constant columns should generate QueryPlanConstraints

  Show description for fd4e304
  

  adrian-ionescu

  authored and

  gatorsmile

  committed
  fd4e304

  Copy full SHA for fd4e304

• [SPARK-22940][SQL] HiveExternalCatalogVersionsSuite should succeed on platforms that don't have wget

  Show description for 0a30e93
  

  bersprockets

  authored and

  Marcelo Vanzin

  committed
  0a30e93

  Copy full SHA for 0a30e93

• [SPARK-13030][ML] Follow-up cleanups for OneHotEncoderEstimator

  Show description for d1f422c
  jkbradley

  committed
  d1f422c

  Copy full SHA for d1f422c

Commits on Jan 6, 2018

• [SPARK-22914][DEPLOY] Register history.ui.port

  Show description for 55afac4
  

  gerashegalov

  authored and

  Marcelo Vanzin

  committed
  55afac4

  Copy full SHA for 55afac4

• [SPARK-22937][SQL] SQL elt output binary for binary inputs

  Show description for bf85301
  

  maropu

  authored and

  gatorsmile

  committed
  bf85301

  Copy full SHA for bf85301

• [SPARK-22960][K8S] Revert use of ARG base_image in images

  Show description for 3e3e938
  

  liyinan926

  authored and

  Marcelo Vanzin

  committed
  3e3e938

  Copy full SHA for 3e3e938

• [SPARK-22930][PYTHON][SQL] Improve the description of Vectorized UDFs for non-deterministic cases

  Show description for 7236914
  

  icexelloss

  authored and

  gatorsmile

  committed
  7236914

  Copy full SHA for 7236914

• [SPARK-22793][SQL] Memory leak in Spark Thrift Server

  Show description for e6449e8
  

  zuotingbing

  authored and

  gatorsmile

  committed
  e6449e8

  Copy full SHA for e6449e8

• [SPARK-21786][SQL] When acquiring 'compressionCodecClassName' in 'ParquetOptions', parquet.compression needs to be considered.

  Show description for 0377755
  

  fjh100456

  authored and

  gatorsmile

  committed
  0377755

  Copy full SHA for 0377755

• [SPARK-22901][PYTHON][FOLLOWUP] Adds the doc for asNondeterministic for wrapped UDF function

  Show description for b66700a
  

  HyukjinKwon

  authored and

  gatorsmile

  committed
  b66700a

  Copy full SHA for b66700a

• [HOTFIX] Fix style checking failure

  Show description for f9e7b0c
  gatorsmile

  committed
  f9e7b0c

  Copy full SHA for f9e7b0c

Commits on Jan 7, 2018

• [SPARK-22973][SQL] Fix incorrect results of Casting Map to String

  Show description for 285d342
  

  maropu

  authored and

  cloud-fan

  committed
  285d342

  Copy full SHA for 285d342

Commits on Jan 8, 2018

• [SPARK-22985] Fix argument escaping bug in from_utc_timestamp / to_utc_timestamp codegen

  Show description for 7673e9c
  

  JoshRosen

  authored and

  gatorsmile

  committed
  7673e9c

  Copy full SHA for 7673e9c

• [SPARK-22566][PYTHON] Better error message for _merge_type in Pandas to Spark DF conversion

  Show description for a1d3352
  

  gberger-palantir

  authored and

  ueshin

  committed
  a1d3352

  Copy full SHA for a1d3352

• [SPARK-22979][PYTHON][SQL] Avoid per-record type dispatch in Python data conversion (EvaluatePython.fromJava)

  Show description for 8bf24e9
  

  HyukjinKwon

  authored and

  cloud-fan

  committed
  8bf24e9

  Copy full SHA for 8bf24e9

• [SPARK-22983] Don't push filters beneath aggregates with empty grouping expressions

  Show description for 6964dfe
  

  JoshRosen

  authored and

  gatorsmile

  committed
  6964dfe

  Copy full SHA for 6964dfe

• [SPARK-21865][SQL] simplify the distribution semantic of Spark SQL

  Show description for 4a45f0a
  cloud-fan

  committed
  4a45f0a

  Copy full SHA for 4a45f0a

• [SPARK-22952][CORE] Deprecate stageAttemptId in favour of stageAttemptNumber

  Show description for 06fd842
  

  advancedxy

  authored and

  cloud-fan

  committed
  06fd842

  Copy full SHA for 06fd842

• [SPARK-22992][K8S] Remove assumption of the DNS domain

  Show description for eecd83c
  

  foxish

  authored and

  Marcelo Vanzin

  committed
  eecd83c

  Copy full SHA for eecd83c

Commits on Jan 9, 2018

• [SPARK-22972] Couldn't find corresponding Hive SerDe for data source provider org.apache.spark.sql.hive.orc

  Show description for 8032cf8
  

  xubo245

  authored and

  gatorsmile

  committed
  8032cf8

  Copy full SHA for 8032cf8

• [SPARK-22990][CORE] Fix method isFairScheduler in JobsTab and StagesTab

  Show description for 850b9f3
  

  gengliangwang

  authored and

  cloud-fan

  committed
  850b9f3

  Copy full SHA for 850b9f3

• [SPARK-22984] Fix incorrect bitmap copying and offset adjustment in GenerateUnsafeRowJoiner

  Show description for fd46a27
  

  JoshRosen

  authored and

  cloud-fan

  committed
  fd46a27

  Copy full SHA for fd46a27

• [SPARK-21292][DOCS] refreshtable example

  Show description for 911a4db
  

  felixcheung

  authored and

  Felix Cheung

  committed
  911a4db

  Copy full SHA for 911a4db

• [SPARK-21293][SPARKR][DOCS] structured streaming doc update

  Show description for a23c07e
  

  felixcheung

  authored and

  Felix Cheung

  committed
  a23c07e

  Copy full SHA for a23c07e

• [SPARK-23000] Fix Flaky test suite DataSourceWithHiveMetastoreCatalogSuite in Spark 2.3

  Show description for e79480e
  

  gatorsmile

  authored and

  cloud-fan

  committed
  e79480e

  Copy full SHA for e79480e

• [SPARK-22998][K8S] Set missing value for SPARK_MOUNTED_CLASSPATH in the executors

  Show description for 47f975b
  

  liyinan926

  authored and

  Felix Cheung

  committed
  47f975b

  Copy full SHA for 47f975b

• [SPARK-16060][SQL] Support Vectorized ORC Reader

  Show description for 60f6b99
  

  dongjoon-hyun

  authored and

  cloud-fan

  committed
  60f6b99

  Copy full SHA for 60f6b99

• [SPARK-22981][SQL] Fix incorrect results of Casting Struct to String

  Show description for be59919
  

  maropu

  authored and

  cloud-fan

  committed
  be59919

  Copy full SHA for be59919

• [SPARK-22912] v2 data source support in MicroBatchExecution

  Show description for 44763d9
  

  jose-torres

  authored and

  tdas

  committed
  44763d9

  Copy full SHA for 44763d9

Commits on Jan 10, 2018

• [SPARK-23005][CORE] Improve RDD.take on small number of partitions

  Show description for df047bd
  

  gengliangwang

  authored and

  cloud-fan

  committed
  df047bd

  Copy full SHA for df047bd

• [MINOR] fix a typo in BroadcastJoinSuite

  Show description for 45f5c3c
  cloud-fan

  committed
  45f5c3c

  Copy full SHA for 45f5c3c

• [SPARK-23018][PYTHON] Fix createDataFrame from Pandas timestamp series assignment

  Show description for 20a8c88
  

  BryanCutler

  authored and

  ueshin

  committed
  20a8c88

  Copy full SHA for 20a8c88

• [SPARK-22982] Remove unsafe asynchronous close() call from FileDownloadChannel

  Show description for 162c5be
  

  JoshRosen

  authored and

  cloud-fan

  committed
  162c5be

  Copy full SHA for 162c5be

• [SPARK-16060][SQL][FOLLOW-UP] add a wrapper solution for vectorized orc reader

  Show description for ecc24ec
  cloud-fan

  committed
  ecc24ec

  Copy full SHA for ecc24ec

• [SPARK-22993][ML] Clarify HasCheckpointInterval param doc

  Show description for 2db5239
  

  sethah

  authored and

  Felix Cheung

  committed
  2db5239

  Copy full SHA for 2db5239

• [SPARK-22997] Add additional defenses against use of freed MemoryBlocks

  Show description for 60d4d79
  JoshRosen

  committed
  60d4d79

  Copy full SHA for 60d4d79

• [SPARK-23019][CORE] Wait until SparkContext.stop() finished in SparkLauncherSuite

  Show description for 5b5851c
  

  gengliangwang

  authored and

  Marcelo Vanzin

  committed
  5b5851c

  Copy full SHA for 5b5851c

• [SPARK-22951][SQL] fix aggregation after dropDuplicates on empty data frames

  Show description for eb4fa55
  

  Feng Liu

  authored and

  liancheng

  committed
  eb4fa55

  Copy full SHA for eb4fa55

Commits on Jan 11, 2018

• [SPARK-23009][PYTHON] Fix for non-str col names to createDataFrame from Pandas

  Show description for 551ccfb
  

  BryanCutler

  authored and

  HyukjinKwon

  committed
  551ccfb

  Copy full SHA for 551ccfb

• [SPARK-22587] Spark job fails if fs.defaultFS and application jar are different url

  Show description for 317b0aa
  

  Mingjie Tang

  authored and

  jerryshao

  committed
  317b0aa

  Copy full SHA for 317b0aa

• [SPARK-23001][SQL] Fix NullPointerException when DESC a database with NULL description

  Show description for d9a973d
  

  gatorsmile

  authored and

  cloud-fan

  committed
  d9a973d

  Copy full SHA for d9a973d

• [SPARK-20657][CORE] Speed up rendering of the stages page.

  Show description for b781301
  

  Marcelo Vanzin

  authored and

  cloud-fan

  committed
  b781301

  Copy full SHA for b781301

• [SPARK-22967][TESTS] Fix VersionSuite's unit tests by change Windows path into URI path

  Show description for 7995989
  

  Ngone51

  authored and

  HyukjinKwon

  committed
  7995989

  Copy full SHA for 7995989

• [SPARK-23000][TEST-HADOOP2.6] Fix Flaky test suite DataSourceWithHiveMetastoreCatalogSuite

  Show description for 9ca0f6e
  gatorsmile

  committed
  9ca0f6e

  Copy full SHA for 9ca0f6e

• [SPARK-19732][FOLLOW-UP] Document behavior changes made in na.fill and fillna

  Show description for f624850
  

  gatorsmile

  authored and

  HyukjinKwon

  committed
  f624850

  Copy full SHA for f624850

• [SPARK-22994][K8S] Use a single image for all Spark containers.

  Show description for b94debd
  

  Marcelo Vanzin

  committed
  b94debd

  Copy full SHA for b94debd

• [SPARK-22908] Add kafka source and sink for continuous processing.

  Show description for f891ee3
  

  jose-torres

  authored and

  tdas

  committed
  f891ee3

  Copy full SHA for f891ee3

• [SPARK-23046][ML][SPARKR] Have RFormula include VectorSizeHint in pipeline

  Show description for 2ec3026
  

  MrBago

  authored and

  jkbradley

  committed
  2ec3026

  Copy full SHA for 2ec3026

• Preparing Spark release v2.3.0-rc1

  sameeragarwal

  committed
  964cc2e

  Copy full SHA for 964cc2e

• Preparing development version 2.3.1-SNAPSHOT

  sameeragarwal

  committed
  6bb2296

  Copy full SHA for 6bb2296

Commits on Jan 12, 2018

• [SPARK-23008][ML] OnehotEncoderEstimator python API

  Show description for 55695c7
  

  WeichenXu123

  authored and

  jkbradley

  committed
  55695c7

  Copy full SHA for 55695c7

• [SPARK-22986][CORE] Use a cache to avoid instantiating multiple instances of broadcast variable values

  Show description for 3ae3e1b
  

  ho3rexqj

  authored and

  cloud-fan

  committed
  3ae3e1b

  Copy full SHA for 3ae3e1b

• [SPARK-23008][ML][FOLLOW-UP] mark OneHotEncoder python API deprecated

  Show description for d512d87
  

  WeichenXu123

  authored and

  Nick Pentreath

  committed
  d512d87

  Copy full SHA for d512d87

• [SPARK-23025][SQL] Support Null type in scala reflection

  Show description for 6152da3
  

  mgaido91

  authored and

  gatorsmile

  committed
  6152da3

  Copy full SHA for 6152da3

• [MINOR][BUILD] Fix Java linter errors

  Show description for db27a93
  

  dongjoon-hyun

  authored and

  sameeragarwal

  committed
  db27a93

  Copy full SHA for db27a93

• [SPARK-22975][SS] MetricsReporter should not throw exception when there was no progress reported

  Show description for 02176f4
  

  mgaido91

  authored and

  zsxwing

  committed
  02176f4

  Copy full SHA for 02176f4

• Revert "[SPARK-22908] Add kafka source and sink for continuous processing."

  Show description for 60bcb46
  sameeragarwal

  committed
  60bcb46

  Copy full SHA for 60bcb46

Commits on Jan 13, 2018

• [SPARK-22980][PYTHON][SQL] Clarify the length of each series is of each batch within scalar Pandas UDF

  Show description for ca27d9c
  HyukjinKwon

  committed
  ca27d9c

  Copy full SHA for ca27d9c

• [SPARK-22870][CORE] Dynamic allocation should allow 0 idle time

  Show description for 801ffd7
  

  wangyum

  authored and

  srowen

  committed
  801ffd7

  Copy full SHA for 801ffd7

• [SPARK-23036][SQL][TEST] Add withGlobalTempView for testing

  Show description for 8d32ed5
  

  xubo245

  authored and

  gatorsmile

  committed
  8d32ed5

  Copy full SHA for 8d32ed5

• [SPARK-22790][SQL] add a configurable factor to describe HadoopFsRelation's size

  Show description for 0fc5533
  

  CodingCat

  authored and

  gatorsmile

  committed
  0fc5533

  Copy full SHA for 0fc5533

• [SPARK-21213][SQL][FOLLOWUP] Use compatible types for comparisons in compareAndGetNewStats

  Show description for bcd87ae
  

  maropu

  authored and

  gatorsmile

  committed
  bcd87ae

  Copy full SHA for bcd87ae

Commits on Jan 14, 2018

• [SPARK-23063][K8S] K8s changes for publishing scripts (and a couple of other misses)

  Show description for 1f4a08b
  

  foxish

  authored and

  Felix Cheung

  committed
  1f4a08b

  Copy full SHA for 1f4a08b

• [SPARK-23038][TEST] Update docker/spark-test (JDK/OS)

  Show description for a335a49
  

  dongjoon-hyun

  authored and

  Felix Cheung

  committed
  a335a49

  Copy full SHA for a335a49

• [SPARK-23069][DOCS][SPARKR] fix R doc for describe missing text

  Show description for 0d425c3
  

  felixcheung

  authored and

  HyukjinKwon

  committed
  0d425c3

  Copy full SHA for 0d425c3

• [SPARK-23021][SQL] AnalysisBarrier should override innerChildren to print correct explain output

  Show description for 5fbbd94
  

  maropu

  authored and

  gatorsmile

  committed
  5fbbd94

  Copy full SHA for 5fbbd94

• [SPARK-23051][CORE] Fix for broken job description in Spark UI

  Show description for 9051e1a
  

  smurakozi

  authored and

  srowen

  committed
  9051e1a

  Copy full SHA for 9051e1a

• [SPARK-22999][SQL] show databases like command' can remove the like keyword

  Show description for 2879236
  

  guoxiaolong

  authored and

  gatorsmile

  committed
  2879236

  Copy full SHA for 2879236

Commits on Jan 15, 2018

• [SPARK-23054][SQL] Fix incorrect results of casting UserDefinedType to String

  Show description for 30574fd
  

  maropu

  authored and

  cloud-fan

  committed
  30574fd

  Copy full SHA for 30574fd

• [SPARK-23049][SQL] spark.sql.files.ignoreCorruptFiles should work for ORC files

  Show description for 81b9899
  

  dongjoon-hyun

  authored and

  cloud-fan

  committed
  81b9899

  Copy full SHA for 81b9899

• [SPARK-23023][SQL] Cast field data to strings in showString

  Show description for 188999a
  

  maropu

  authored and

  cloud-fan

  committed
  188999a

  Copy full SHA for 188999a

• [SPARK-19550][BUILD][FOLLOW-UP] Remove MaxPermSize for sql module

  Show description for 3491ca4
  

  wangyum

  authored and

  srowen

  committed
  3491ca4

  Copy full SHA for 3491ca4

• [SPARK-23070] Bump previousSparkVersion in MimaBuild.scala to be 2.2.0

  Show description for c6a3b92
  gatorsmile

  committed
  c6a3b92

  Copy full SHA for c6a3b92

• [SPARK-23035][SQL] Fix improper information of TempTableAlreadyExistsException

  Show description for 706a308
  

  xubo245

  authored and

  gatorsmile

  committed
  706a308

  Copy full SHA for 706a308

Commits on Jan 16, 2018

• [SPARK-23080][SQL] Improve error message for built-in functions

  Show description for bb8e5ad
  

  mgaido91

  authored and

  HyukjinKwon

  committed
  bb8e5ad

  Copy full SHA for bb8e5ad

• [SPARK-23000] Use fully qualified table names in HiveMetastoreCatalogSuite

  Show description for e2ffb97
  

  sameeragarwal

  authored and

  gatorsmile

  committed
  e2ffb97

  Copy full SHA for e2ffb97

• [SPARK-22956][SS] Bug fix for 2 streams union failover scenario

  Show description for e58c4a9
  

  xuanyuanking

  authored and

  zsxwing

  committed
  e58c4a9

  Copy full SHA for e58c4a9

• [SPARK-23020][CORE] Fix races in launcher code, test.

  Show description for 20c6981
  

  Marcelo Vanzin

  authored and

  sameeragarwal

  committed
  20c6981

  Copy full SHA for 20c6981

• [SPARK-22978][PYSPARK] Register Vectorized UDFs for SQL Statement

  Show description for 5c06ee2
  

  gatorsmile

  authored and

  HyukjinKwon

  committed
  5c06ee2

  Copy full SHA for 5c06ee2

• [SPARK-22392][SQL] data source v2 columnar batch reader

  Show description for 863ffdc
  cloud-fan

  committed
  863ffdc

  Copy full SHA for 863ffdc

• [SPARK-23045][ML][SPARKR] Update RFormula to use OneHotEncoderEstimator.

  Show description for 833a584
  

  MrBago

  authored and

  jkbradley

  committed
  833a584

  Copy full SHA for 833a584

Commits on Jan 17, 2018

• [SPARK-23095][SQL] Decorrelation of scalar subquery fails with java.util.NoSuchElementException

  Show description for 41d1a32
  

  dilipbiswal

  authored and

  gatorsmile

  committed
  41d1a32

  Copy full SHA for 41d1a32

• [SPARK-22361][SQL][TEST] Add unit test for Window Frames

  Show description for 08252bb
  

  gaborgsomogyi

  authored and

  gatorsmile

  committed
  08252bb

  Copy full SHA for 08252bb

• [SPARK-22908][SS] Roll forward continuous processing Kafka support with fix to continuous Kafka data reader

  Show description for 0a441d2
  

  jose-torres

  authored and

  tdas

  committed
  0a441d2

  Copy full SHA for 0a441d2

• Revert "[SPARK-23020][CORE] Fix races in launcher code, test."

  Show description for b9339ee
  sameeragarwal

  committed
  b9339ee

  Copy full SHA for b9339ee

• Fix merge between 07ae39d and 1667057

  Show description for 00c744e
  

  jose-torres

  authored and

  zsxwing

  committed
  00c744e

  Copy full SHA for 00c744e

• [SPARK-23072][SQL][TEST] Add a Unicode schema test for file-based data sources

  Show description for 8ef323c
  

  dongjoon-hyun

  authored and

  cloud-fan

  committed
  8ef323c

  Copy full SHA for 8ef323c

• [SPARK-23062][SQL] Improve EXCEPT documentation

  Show description for bfbc2d4
  

  Henry Robinson

  authored and

  gatorsmile

  committed
  bfbc2d4

  Copy full SHA for bfbc2d4

• [SPARK-21783][SQL] Turn on ORC filter push-down by default

  Show description for cbb6bda
  

  dongjoon-hyun

  authored and

  cloud-fan

  committed
  cbb6bda

  Copy full SHA for cbb6bda

• [SPARK-23079][SQL] Fix query constraints propagation with aliases

  Show description for aae73a2
  

  gengliangwang

  authored and

  cloud-fan

  committed
  aae73a2

  Copy full SHA for aae73a2

• [SPARK-23020] Ignore Flaky Test: SparkLauncherSuite.testInProcessLauncher

  Show description for 1a6dfaf
  sameeragarwal

  committed
  1a6dfaf

  Copy full SHA for 1a6dfaf

• [SPARK-23033][SS] Don't use task level retry for continuous processing

  Show description for dbd2a55
  

  jose-torres

  authored and

  tdas

  committed
  dbd2a55

  Copy full SHA for dbd2a55

• [SPARK-23093][SS] Don't change run id when reconfiguring a continuous processing query.

  Show description for 79ccd0c
  

  jose-torres

  authored and

  zsxwing

  committed
  79ccd0c

  Copy full SHA for 79ccd0c

• [SPARK-23047][PYTHON][SQL] Change MapVector to NullableMapVector in ArrowColumnVector

  Show description for 6e509fd
  

  icexelloss

  authored and

  HyukjinKwon

  committed
  6e509fd

  Copy full SHA for 6e509fd

• [SPARK-23132][PYTHON][ML] Run doctests in ml.image when testing

  Show description for b84c2a3
  HyukjinKwon

  committed
  b84c2a3

  Copy full SHA for b84c2a3

Commits on Jan 18, 2018

• [SPARK-23119][SS] Minor fixes to V2 streaming APIs

  Show description for 9783aea
  

  tdas

  authored and

  zsxwing

  committed
  9783aea

  Copy full SHA for 9783aea

• [SPARK-23064][DOCS][SS] Added documentation for stream-stream joins

  Show description for 050c1e2
  

  tdas

  authored and

  zsxwing

  committed
  050c1e2

  Copy full SHA for 050c1e2

• [SPARK-21996][SQL] read files with space in name for streaming

  Show description for f2688ef
  

  xysun

  authored and

  zsxwing

  committed
  f2688ef

  Copy full SHA for f2688ef

• [SPARK-23122][PYTHON][SQL] Deprecate register* for UDFs in SQLContext and Catalog in PySpark

  Show description for 3a80cc5
  

  HyukjinKwon

  authored and

  ueshin

  committed
  3a80cc5

  Copy full SHA for 3a80cc5

• [SPARK-23052][SS] Migrate ConsoleSink to data source V2 api.

  Show description for 2a87c3a
  

  jose-torres

  authored and

  tdas

  committed
  2a87c3a

  Copy full SHA for 2a87c3a

• [SPARK-23140][SQL] Add DataSourceV2Strategy to Hive Session state's planner

  Show description for f801ac4
  

  jerryshao

  authored and

  cloud-fan

  committed
  f801ac4

  Copy full SHA for f801ac4

• [SPARK-22036][SQL] Decimal multiplication with high precision/scale often returns NULL

  Show description for 8a98274
  

  mgaido91

  authored and

  cloud-fan

  committed
  8a98274

  Copy full SHA for 8a98274

• [SPARK-23141][SQL][PYSPARK] Support data type string as a returnType for registerJavaFunction.

  Show description for e0421c6
  

  ueshin

  authored and

  HyukjinKwon

  committed
  e0421c6

  Copy full SHA for e0421c6

• [SPARK-23147][UI] Fix task page table IndexOutOfBound Exception

  Show description for bd0a162
  

  jerryshao

  authored and

  Marcelo Vanzin

  committed
  bd0a162

  Copy full SHA for bd0a162

• [SPARK-23029][DOCS] Specifying default units of configuration entries

  Show description for bfdbdd3
  

  ferdonline

  authored and

  srowen

  committed
  bfdbdd3

  Copy full SHA for bfdbdd3

• [SPARK-23143][SS][PYTHON] Added python API for setting continuous trigger

  Show description for e6e8bbe
  tdas

  committed
  e6e8bbe

  Copy full SHA for e6e8bbe

• [SPARK-23144][SS] Added console sink for continuous processing

  Show description for 1f88fcd
  tdas

  committed
  1f88fcd

  Copy full SHA for 1f88fcd

• [SPARK-23133][K8S] Fix passing java options to Executor

  Show description for b8c6d93
  

  andrusha

  authored and

  Marcelo Vanzin

  committed
  b8c6d93

  Copy full SHA for b8c6d93

• [SPARK-23094] Fix invalid character handling in JsonDataSource

  Show description for a295034
  

  brkyvz

  authored and

  HyukjinKwon

  committed
  a295034

  Copy full SHA for a295034

• [SPARK-22962][K8S] Fail fast if submission client local files are used

  Show description for 7057e31
  

  liyinan926

  authored and

  Marcelo Vanzin

  committed
  7057e31

  Copy full SHA for 7057e31

Commits on Jan 19, 2018

• [SPARK-23142][SS][DOCS] Added docs for continuous processing

  Show description for acf3b70
  tdas

  committed
  acf3b70

  Copy full SHA for acf3b70

• [DOCS] change to dataset for java code in structured-streaming-kafka-integration document

  Show description for 225b1af
  

  brandonJY

  authored and

  srowen

  committed
  225b1af

  Copy full SHA for 225b1af

• [SPARK-23054][SQL][PYSPARK][FOLLOWUP] Use sqlType casting when casting PythonUserDefinedType to String.

  Show description for 541dbc0
  

  ueshin

  authored and

  cloud-fan

  committed
  541dbc0

  Copy full SHA for 541dbc0

• [BUILD][MINOR] Fix java style check issues

  Show description for 54c1fae
  sameeragarwal

  committed
  54c1fae

  Copy full SHA for 54c1fae

• [SPARK-23127][DOC] Update FeatureHasher guide for categoricalCols parameter

  Show description for e582231
  

  Nick Pentreath

  committed
  e582231

  Copy full SHA for e582231

• [SPARK-23048][ML] Add OneHotEncoderEstimator document and examples

  Show description for ef7989d
  

  viirya

  authored and

  Nick Pentreath

  committed
  ef7989d

  Copy full SHA for ef7989d

• [SPARK-23089][STS] Recreate session log directory if it doesn't exist

  Show description for b7a8199
  

  mgaido91

  authored and

  cloud-fan

  committed
  b7a8199

  Copy full SHA for b7a8199

• [SPARK-23000][TEST] Keep Derby DB Location Unchanged After Session Cloning

  Show description for 8d6845c
  

  gatorsmile

  authored and

  cloud-fan

  committed
  8d6845c

  Copy full SHA for 8d6845c

• [SPARK-23149][SQL] polish ColumnarBatch

  Show description for 55efeff
  

  cloud-fan

  authored and

  gatorsmile

  committed
  55efeff

  Copy full SHA for 55efeff

• [SPARK-23104][K8S][DOCS] Changes to Kubernetes scheduler documentation

  Show description for ffe4591
  

  foxish

  authored and

  Marcelo Vanzin

  committed
  ffe4591

  Copy full SHA for ffe4591