Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#2 #1

Merged
merged 469 commits into from
Jul 15, 2018
Merged

#2 #1

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
469 commits
Select commit Hold shift + click to select a range
d440699
[SPARK-24381][TESTING] Add unit tests for NOT IN subquery around null…
mgyucht May 27, 2018
672209f
[SPARK-24334] Fix race condition in ArrowPythonRunner causes unclean …
icexelloss May 28, 2018
de01a8d
[SPARK-24373][SQL] Add AnalysisBarrier to RelationalGroupedDataset's …
mgaido91 May 28, 2018
fa2ae9d
[SPARK-24392][PYTHON] Label pandas_udf as Experimental
BryanCutler May 28, 2018
b31b587
[SPARK-19613][SS][TEST] Random.nextString is not safe for directory n…
dongjoon-hyun May 29, 2018
2ced619
[SPARK-24377][SPARK SUBMIT] make --py-files work in non pyspark appli…
jerryshao May 29, 2018
23db600
[SPARK-24250][SQL][FOLLOW-UP] support accessing SQLConf inside tasks
gatorsmile May 29, 2018
aca65c6
[SPARK-23991][DSTREAMS] Fix data loss when WAL write fails in allocat…
gaborgsomogyi May 29, 2018
900bc1f
[SPARK-24371][SQL] Added isInCollection in DataFrame API for Scala an…
dbtsai May 29, 2018
f489388
[SPARK-24365][SQL] Add Data Source write benchmark
gengliangwang May 30, 2018
a4be981
[SPARK-24331][SPARKR][SQL] Adding arrays_overlap, array_repeat, map_e…
mn-mikke May 30, 2018
0ebb0c0
[SPARK-23754][PYTHON] Re-raising StopIteration in client code
e-dorigatti May 30, 2018
9e7bad0
[SPARK-24419][BUILD] Upgrade SBT to 0.13.17 with Scala 2.10.7 for JDK9+
dbtsai May 30, 2018
1e46f92
[SPARK-24369][SQL] Correct handling for multiple distinct aggregation…
maropu May 30, 2018
b142157
[SPARK-24384][PYTHON][SPARK SUBMIT] Add .py files correctly into Pyth…
HyukjinKwon May 30, 2018
ec6f971
[SPARK-23161][PYSPARK][ML] Add missing APIs to Python GBTClassifier
huaxingao May 30, 2018
1b36f14
[SPARK-23901][SQL] Add masking functions
mgaido91 May 30, 2018
24ef7fb
[SPARK-24276][SQL] Order of literals in IN should not affect semantic…
mgaido91 May 30, 2018
0053e15
[SPARK-24337][CORE] Improve error messages for Spark conf values
PenguinToast May 31, 2018
90ae98d
[SPARK-24146][PYSPARK][ML] spark.ml parity for sequential pattern min…
WeichenXu123 May 31, 2018
698b9a0
[WEBUI] Avoid possibility of script in query param keys
srowen May 31, 2018
7a82e93
[SPARK-24414][UI] Calculate the correct number of tasks for a stage.
May 31, 2018
223df5d
[SPARK-24397][PYSPARK] Added TaskContext.getLocalProperty(key) in Python
tdas May 31, 2018
cc976f6
[SPARK-23900][SQL] format_number support user specifed format as argu…
wangyum May 31, 2018
21e1fc7
[SPARK-24232][K8S] Add support for secret env vars
May 31, 2018
2c9c862
[MINOR][YARN] Add YARN-specific credential providers in debug logging…
HyukjinKwon Jun 1, 2018
cbaa729
[SPARK-24330][SQL] Refactor ExecuteWriteTask and Use `while` in writi…
gengliangwang Jun 1, 2018
b2d0226
[SPARK-24444][DOCS][PYTHON] Improve Pandas UDF docs to explain column…
BryanCutler Jun 1, 2018
22df953
[SPARK-24326][MESOS] add support for local:// scheme for the app jar
Jun 1, 2018
98909c3
[SPARK-23920][SQL] add array_remove to remove all elements that equal…
huaxingao Jun 1, 2018
6039b13
[SPARK-24351][SS] offsetLog/commitLog purge thresholdBatchId should b…
ivoson Jun 1, 2018
d2c3de7
Revert "[SPARK-24369][SQL] Correct handling for multiple distinct agg…
gatorsmile Jun 1, 2018
09e78c1
[INFRA] Close stale PRs.
Jun 1, 2018
8ef167a
[SPARK-24340][CORE] Clean up non-shuffle disk block manager files fol…
jiangxb1987 Jun 1, 2018
a36c1a6
[SPARK-23668][K8S] Added missing config property in running-on-kubern…
liyinan926 Jun 2, 2018
de4feae
[SPARK-24356][CORE] Duplicate strings in File.path managed by FileSeg…
misha-cloudera Jun 3, 2018
a2166ec
[SPARK-24455][CORE] fix typo in TaskSchedulerImpl comment
Jun 4, 2018
416cd1f
[SPARK-24369][SQL] Correct handling for multiple distinct aggregation…
cloud-fan Jun 4, 2018
1d9338b
[SPARK-23786][SQL] Checking column names of csv headers
MaxGekk Jun 4, 2018
0be5aa2
[SPARK-23903][SQL] Add support for date extract
wangyum Jun 4, 2018
7297ae0
[SPARK-21896][SQL] Fix StackOverflow caused by window functions insid…
Jun 4, 2018
b24d3db
[SPARK-24290][ML] add support for Array input for instrumentation.log…
lu-wang-dl Jun 4, 2018
ff0501b
[SPARK-24300][ML] change the way to set seed in ml.cluster.LDASuite.g…
lu-wang-dl Jun 4, 2018
dbb4d83
[SPARK-24215][PYSPARK] Implement _repr_html_ for dataframes in PySpark
xuanyuanking Jun 5, 2018
b3417b7
[SPARK-16451][REPL] Fail shell if SparkSession fails to start.
Jun 5, 2018
e8c1a0c
[SPARK-15784] Add Power Iteration Clustering to spark.ml
WeichenXu123 Jun 5, 2018
2c2a86b
[SPARK-24453][SS] Fix error recovering from the failure in a no-data …
tdas Jun 5, 2018
93df3cd
[SPARK-22384][SQL] Refine partition pruning when attribute is wrapped…
Jun 5, 2018
e9efb62
[SPARK-24187][R][SQL] Add array_join function to SparkR
huaxingao Jun 6, 2018
e76b012
[SPARK-23803][SQL] Support bucket pruning
Jun 6, 2018
1462bba
[SPARK-24119][SQL] Add interpreted execution to SortPrefix expression
bersprockets Jun 8, 2018
2c10020
[SPARK-24224][ML-EXAMPLES] Java example code for Power Iteration Clus…
shahidki31 Jun 8, 2018
a5d775a
[SPARK-24191][ML] Scala Example code for Power Iteration Clustering
shahidki31 Jun 8, 2018
173fe45
[SPARK-24477][SPARK-24454][ML][PYTHON] Imports submodule in ml/__init…
HyukjinKwon Jun 8, 2018
1a644af
[SPARK-23984][K8S] Initial Python Bindings for PySpark on K8s
ifilonenko Jun 8, 2018
b070ded
[SPARK-17756][PYTHON][STREAMING] Workaround to avoid return type mism…
HyukjinKwon Jun 8, 2018
f433ef7
[SPARK-23010][K8S] Initial checkin of k8s integration tests.
ssuchter Jun 8, 2018
36a3409
[SPARK-24412][SQL] Adding docs about automagical type casting in `isi…
raptond Jun 9, 2018
f07c506
[SPARK-24468][SQL] Handle negative scale when adjusting precision for…
mgaido91 Jun 9, 2018
3e5b4ae
[SPARK-23754][PYTHON][FOLLOWUP] Move UDF stop iteration wrapping from…
e-dorigatti Jun 11, 2018
a99d284
[SPARK-19826][ML][PYTHON] add spark.ml Python API for PIC
huaxingao Jun 11, 2018
9b6f242
[MINOR][CORE] Log committer class used by HadoopMapRedCommitProtocol
ejono Jun 11, 2018
2dc047a
[SPARK-24520] Double braces in documentations
Jun 11, 2018
f5af86e
[SPARK-24134][DOCS] A missing full-stop in doc "Tuning Spark".
XD-DENG Jun 11, 2018
0481977
[SPARK-22144][SQL] ExchangeCoordinator combine the partitions of an 0…
liutang123 Jun 12, 2018
dc22465
[SPARK-23732][DOCS] Fix source links in generated scaladoc.
Jun 12, 2018
01452ea
[SPARK-24502][SQL] flaky test: UnsafeRowSerializerSuite
cloud-fan Jun 12, 2018
1d7db65
docs: fix typo
tomsaleeba Jun 12, 2018
5d6a53d
[SPARK-15064][ML] Locale support in StopWordsRemover
dongjinleekr Jun 12, 2018
2824f14
[SPARK-24531][TESTS] Remove version 2.2.0 from testing versions in Hi…
mgaido91 Jun 12, 2018
3af1d3e
[SPARK-24416] Fix configuration specification for killBlacklisted exe…
Jun 12, 2018
f0ef1b3
[SPARK-23931][SQL] Adds arrays_zip function to sparksql
DylanGuedes Jun 12, 2018
cc88d7f
[SPARK-24216][SQL] Spark TypedAggregateExpression uses getSimpleName …
Jun 12, 2018
ada28f2
[SPARK-23933][SQL] Add map_from_arrays function
kiszk Jun 12, 2018
0d3714d
[SPARK-23010][BUILD][FOLLOWUP] Fix java checkstyle failure of kuberne…
jiangxb1987 Jun 12, 2018
f53818d
[SPARK-24506][UI] Add UI filters to tabs added after binding
mgaido91 Jun 12, 2018
9786ce6
[SPARK-22239][SQL][PYTHON] Enable grouped aggregate pandas UDFs as wi…
icexelloss Jun 13, 2018
3352d6f
[SPARK-24466][SS] Fix TextSocketMicroBatchReader to be compatible wit…
HeartSaVioR Jun 13, 2018
4c388bc
[SPARK-24485][SS] Measure and log elapsed time for filesystem operati…
HeartSaVioR Jun 13, 2018
7703b46
[SPARK-24479][SS] Added config for registering streamingQueryListeners
arunmahadevan Jun 13, 2018
299d297
[SPARK-24500][SQL] Make sure streams are materialized during Tree tra…
hvanhovell Jun 13, 2018
1b46f41
[SPARK-24235][SS] Implement continuous shuffle writer for single read…
jose-torres Jun 13, 2018
3bf7691
[SPARK-24531][TESTS] Replace 2.3.0 version with 2.3.1
mgaido91 Jun 13, 2018
534065e
[MINOR][CORE][TEST] Remove unnecessary sort in UnsafeInMemorySorterSuite
jiangxb1987 Jun 14, 2018
fdadc4b
[SPARK-24495][SQL] EnsureRequirement returns wrong plan when reorderi…
mgaido91 Jun 14, 2018
d3eed8f
[SPARK-24563][PYTHON] Catch TypeError when testing existence of HiveC…
icexelloss Jun 14, 2018
b8f27ae
[SPARK-24543][SQL] Support any type as DDL string for from_json's schema
MaxGekk Jun 14, 2018
18cb0c0
[SPARK-24319][SPARK SUBMIT] Fix spark-submit execution where no main …
gaborgsomogyi Jun 14, 2018
270a9a3
[SPARK-24248][K8S] Use level triggering and state reconciliation in s…
mccheah Jun 14, 2018
22daeba
[SPARK-24478][SQL] Move projection and filter push down to physical c…
rdblue Jun 15, 2018
6567fc4
[PYTHON] Fix typo in serializer exception
rberenguel Jun 15, 2018
495d8cf
[SPARK-24490][WEBUI] Use WebUI.addStaticHandler in web UIs
jaceklaskowski Jun 15, 2018
b5ccf0d
[SPARK-24396][SS][PYSPARK] Add Structured Streaming ForeachWriter for…
tdas Jun 15, 2018
90da7dc
[SPARK-24452][SQL][CORE] Avoid possible overflow in int add or multiple
kiszk Jun 15, 2018
e4fee39
[SPARK-24525][SS] Provide an option to limit number of rows in a Memo…
mukulmurthy Jun 15, 2018
c7c0b08
add one supported type missing from the javadoc
Jun 16, 2018
b0a9352
[SPARK-24573][INFRA] Runs SBT checkstyle after the build to work arou…
HyukjinKwon Jun 18, 2018
e219e69
[SPARK-23772][SQL] Provide an option to ignore column of all null val…
maropu Jun 18, 2018
bce1775
[SPARK-24526][BUILD][TEST-MAVEN] Spaces in the build dir causes failu…
Jun 18, 2018
8f225e0
[SPARK-24548][SQL] Fix incorrect schema of Dataset with tuple encoders
viirya Jun 18, 2018
1737d45
[SPARK-24478][SQL][FOLLOWUP] Move projection and filter push down to …
cloud-fan Jun 19, 2018
9a75c18
[SPARK-24542][SQL] UDF series UDFXPathXXXX allow users to pass carefu…
gatorsmile Jun 19, 2018
a78a904
[SPARK-24521][SQL][TEST] Fix ineffective test in CachedTableSuite
icexelloss Jun 19, 2018
9dbe53e
[SPARK-24556][SQL] Always rewrite output partitioning in ReusedExchan…
yucai Jun 19, 2018
13092d7
[SPARK-24534][K8S] Bypass non spark-on-k8s commands
rimolive Jun 19, 2018
2cb9763
[SPARK-24565][SS] Add API for in Structured Streaming for exposing ou…
tdas Jun 19, 2018
bc0498d
[SPARK-24583][SQL] Wrong schema type in InsertIntoDataSourceCommand
maryannxue Jun 19, 2018
bc11146
[SPARK-23778][CORE] Avoid unneeded shuffle when union gets an empty RDD
mgaido91 Jun 20, 2018
c8ef923
[MINOR][SQL] Remove invalid comment from SparkStrategies
HeartSaVioR Jun 20, 2018
c5a0d11
[SPARK-24575][SQL] Prohibit window expressions inside WHERE and HAVIN…
Jun 20, 2018
3f4bda7
[SPARK-24578][CORE] Cap sub-region's size of returned nio buffer
WenboZhao Jun 20, 2018
15747cf
[SPARK-24547][K8S] Allow for building spark on k8s docker images with…
Jun 21, 2018
9de11d3
[SPARK-23912][SQL] add array_distinct
huaxingao Jun 21, 2018
54fcaaf
[SPARK-24571][SQL] Support Char literals
MaxGekk Jun 21, 2018
7236e75
[SPARK-24574][SQL] array_contains, array_position, array_remove and e…
chongguang Jun 21, 2018
c0cad59
[SPARK-24614][PYSPARK] Fix for SyntaxWarning on tests.py
rekhajoshm Jun 21, 2018
b56e9c6
[SPARK-16630][YARN] Blacklist a node if executors won't launch on it
attilapiros Jun 21, 2018
c8e909c
[SPARK-24589][CORE] Correctly identify tasks in output commit coordin…
Jun 21, 2018
b9a6f74
[SPARK-24613][SQL] Cache with UDF could not be matched with subsequen…
maryannxue Jun 21, 2018
dc8a6be
[SPARK-24588][SS] streaming join should require HashClusteredPartitio…
cloud-fan Jun 21, 2018
92c2f00
[SPARK-23934][SQL] Adding map_from_entries function
mn-mikke Jun 22, 2018
39dfaf2
[SPARK-24519] Make the threshold for highly compressed map status con…
Jun 22, 2018
33e77fa
[SPARK-24518][CORE] Using Hadoop credential provider API to store pas…
jerryshao Jun 22, 2018
4e7d867
[SPARK-24372][BUILD] Add scripts to help with preparing releases.
Jun 22, 2018
c7e2742
[SPARK-24190][SQL] Allow saving of JSON files in UTF-16 and UTF-32
MaxGekk Jun 24, 2018
98f363b
[SPARK-24206][SQL] Improve FilterPushdownBenchmark benchmark code
maropu Jun 24, 2018
a5849ad
[SPARK-24324][PYTHON] Pandas Grouped Map UDF should assign result col…
BryanCutler Jun 24, 2018
f596ebe
[SPARK-24327][SQL] Verify and normalize a partition column name based…
maropu Jun 25, 2018
6e0596e
[SPARK-23931][SQL][FOLLOW-UP] Make `arrays_zip` in function.scala `@s…
ueshin Jun 25, 2018
8ab8ef7
Fix minor typo in docs/cloud-integration.md
Jun 25, 2018
bac50aa
[SPARK-24596][SQL] Non-cascading Cache Invalidation
maryannxue Jun 25, 2018
594ac4f
[SPARK-24633][SQL] Fix codegen when split is required for arrays_zip
mgaido91 Jun 25, 2018
5264164
[SPARK-24648][SQL] SqlMetrics should be threadsafe
dbkerkela Jun 25, 2018
baa01c8
[INFRA] Close stale PR.
Jun 25, 2018
6d16b98
[SPARK-24552][CORE][SQL] Use task ID instead of attempt number for wr…
Jun 25, 2018
d48803b
[SPARK-24324][PYTHON][FOLLOWUP] Grouped Map positional conf should ha…
BryanCutler Jun 26, 2018
4c059eb
[SPARK-23776][DOC] Update instructions for running PySpark after buil…
bersprockets Jun 26, 2018
c7967c6
[SPARK-24418][BUILD] Upgrade Scala to 2.11.12 and 2.12.6
dbtsai Jun 26, 2018
e07aee2
[SPARK-24636][SQL] Type coercion of arrays for array_join function
mn-mikke Jun 26, 2018
dcaa49f
[SPARK-24658][SQL] Remove workaround for ANTLR bug
wangyum Jun 26, 2018
02f8781
[SPARK-24423][SQL] Add a new option for JDBC sources
dilipbiswal Jun 26, 2018
16f2c3e
[SPARK-6237][NETWORK] Network-layer changes to allow stream upload.
squito Jun 26, 2018
1b9368f
[SPARK-24659][SQL] GenericArrayData.equals should respect element typ…
rednaxelafx Jun 27, 2018
d08f53d
[SPARK-24605][SQL] size(null) returns null instead of -1
MaxGekk Jun 27, 2018
2669b4d
[SPARK-23927][SQL] Add "sequence" expression
wajda Jun 27, 2018
9a76f23
[SPARK-23927][SQL][FOLLOW-UP] Fix a build failure.
ueshin Jun 27, 2018
a1a64e3
[SPARK-21335][DOC] doc changes for disallowed un-aliased subquery use…
cnZach Jun 27, 2018
6a0b77a
[SPARK-24215][PYSPARK][FOLLOW UP] Implement eager evaluation for Data…
xuanyuanking Jun 27, 2018
78ecb6d
[SPARK-24446][YARN] Properly quote library path for YARN.
Jun 27, 2018
c04cb2d
[SPARK-21687][SQL] Spark SQL should set createTime for Hive partition
debugger87 Jun 27, 2018
776befb
[SPARK-24660][SHS] Show correct error pages when downloading logs
mgaido91 Jun 27, 2018
221d03a
[SPARK-24533] Typesafe rebranded to lightbend. Changing the build dow…
Jun 27, 2018
893ea22
[SPARK-24204][SQL] Verify a schema in Json/Orc/ParquetFileFormat
maropu Jun 27, 2018
c5aa54d
[SPARK-24553][WEB-UI] http 302 fixes for href redirect
SJKallman Jun 27, 2018
bd32b50
[SPARK-24645][SQL] Skip parsing when csvColumnPruning enabled and par…
maropu Jun 28, 2018
1c9acc2
[SPARK-24206][SQL][FOLLOW-UP] Update DataSourceReadBenchmark benchmar…
maropu Jun 28, 2018
6a97e8e
[SPARK-24603][SQL] Fix findTightestCommonType reference in comments
Jun 28, 2018
5b05966
[SPARK-24564][TEST] Add test suite for RecordBinaryComparator
jiangxb1987 Jun 28, 2018
524827f
[SPARK-14712][ML] LogisticRegressionModel.toString should summarize m…
jiayue-zhang Jun 28, 2018
a95a4af
[SPARK-23120][PYSPARK][ML] Add basic PMML export support to PySpark
holdenk Jun 28, 2018
e1d3f80
[SPARK-24408][SQL][DOC] Move abs function to math_funcs group
jaceklaskowski Jun 28, 2018
2224861
[SPARK-24439][ML][PYTHON] Add distanceMeasure to BisectingKMeans in P…
huaxingao Jun 28, 2018
f6e6899
[SPARK-24386][SS] coalesce(1) aggregates in continuous processing
jose-torres Jun 28, 2018
f71e8da
[SPARK-24566][CORE] Fix spark.storage.blockManagerSlaveTimeoutMs defa…
xueyumusic Jun 29, 2018
03545ce
[SPARK-24638][SQL] StringStartsWith support push down
wangyum Jun 30, 2018
797971e
[SPARK-24696][SQL] ColumnPruning rule fails to remove extra Project
maryannxue Jun 30, 2018
d54d8b8
simplify rand in dsl/package.scala
gatorsmile Jun 30, 2018
f825847
[SPARK-24654][BUILD] Update, fix LICENSE and NOTICE, and specialize f…
srowen Jul 1, 2018
8f91c69
[SPARK-24665][PYSPARK] Use SQLConf in PySpark to manage all sql configs
xuanyuanking Jul 2, 2018
8008f9c
[SPARK-24715][BUILD] Override jline version as 2.14.3 in SBT
viirya Jul 2, 2018
f599cde
[SPARK-24507][DOCUMENTATION] Update streaming guide
rekhajoshm Jul 2, 2018
4281554
[SPARK-24683][K8S] Fix k8s no resource
mccheah Jul 2, 2018
85fe129
[SPARK-24428][K8S] Fix unused code
Jul 2, 2018
a7c8f0c
[SPARK-24385][SQL] Resolve self-join condition ambiguity for EqualNul…
mgaido91 Jul 3, 2018
5585c57
[SPARK-24420][BUILD] Upgrade ASM to 6.1 to support JDK9+
dbtsai Jul 3, 2018
776f299
[SPARK-24709][SQL] schema_of_json() - schema inference from an example
MaxGekk Jul 4, 2018
b42fda8
[SPARK-23698] Remove raw_input() from Python 2
Jul 4, 2018
5bf95f2
[BUILD] Close stale PRs
srowen Jul 4, 2018
7c08eb6
[SPARK-24732][SQL] Type coercion between MapTypes.
ueshin Jul 4, 2018
772060d
[SPARK-24704][WEBUI] Fix the order of stages in the DAG graph
stanzhai Jul 4, 2018
b2deef6
[SPARK-24727][SQL] Add a static config to control cache size for gene…
maropu Jul 4, 2018
021145f
[SPARK-24716][SQL] Refactor ParquetFilters
wangyum Jul 4, 2018
1a2655a
[SPARK-24635][SQL] Remove Blocks class from JavaCode class hierarchy
viirya Jul 4, 2018
ca8243f
[MINOR][ML] Minor correction in the powerIterationSuite
shahidki31 Jul 4, 2018
bf764a3
[SPARK-22384][SQL][FOLLOWUP] Refine partition pruning when attribute …
cloud-fan Jul 5, 2018
489a529
[SPARK-17213][SPARK-17213][FOLLOW-UP] Improve the test of
gatorsmile Jul 5, 2018
f997be0
[SPARK-24698][PYTHON] Fixed typo in pyspark.ml's Identifiable class.
mcteo Jul 5, 2018
4be9f0c
[SPARK-24673][SQL] scala sql function from_utc_timestamp second argum…
agilelab-tmnd1991 Jul 5, 2018
32cfd3e
[SPARK-24361][SQL] Polish code block manipulation API
viirya Jul 5, 2018
e58dadb
[SPARK-23820][CORE] Enable use of long form of callsite in logs
michaelmior Jul 5, 2018
7bd6d54
[SPARK-24711][K8S] Fix tags for integration tests
Jul 5, 2018
ac78bcc
[SPARK-24743][EXAMPLES] Update the JavaDirectKafkaWordCount example t…
cluo512 Jul 5, 2018
33952cf
[SPARK-24675][SQL] Rename table: validate existence of new location
gengliangwang Jul 5, 2018
e71e93a
[SPARK-24694][K8S] Pass all app args to integration tests
Jul 5, 2018
01fcba2
[SPARK-24737][SQL] Type coercion between StructTypes.
ueshin Jul 6, 2018
bf67f70
[SPARK-24692][TESTS] Improvement FilterPushdownBenchmark
wangyum Jul 6, 2018
141953f
[SPARK-24535][SPARKR] fix tests on java check error
felixcheung Jul 6, 2018
a381bce
[SPARK-24673][SQL][PYTHON][FOLLOWUP] Support Column arguments in time…
maropu Jul 6, 2018
4de0425
[SPARK-24569][SQL] Aggregator with output type Option should produce …
viirya Jul 7, 2018
fc43690
[SPARK-24749][SQL] Use sameType to compare Array's element type in Ar…
viirya Jul 7, 2018
74f6a92
[SPARK-24739][PYTHON] Make PySpark compatible with Python 3.7
HyukjinKwon Jul 7, 2018
044b33b
[SPARK-24740][PYTHON][ML] Make PySpark's tests compatible with NumPy …
HyukjinKwon Jul 7, 2018
79c6689
[SPARK-24757][SQL] Improving the error message for broadcast timeouts
MaxGekk Jul 7, 2018
e2c7e09
[SPARK-24646][CORE] Minor change to spark.yarn.dist.forceDownloadSche…
jerryshao Jul 9, 2018
034913b
[SPARK-23936][SQL] Implement map_concat
bersprockets Jul 9, 2018
1bd3d61
[SPARK-24268][SQL] Use datatype.simpleString in error messages
mgaido91 Jul 9, 2018
aec966b
Revert "[SPARK-24268][SQL] Use datatype.simpleString in error messages"
gatorsmile Jul 9, 2018
eb6e988
[SPARK-24759][SQL] No reordering keys for broadcast hash join
gatorsmile Jul 9, 2018
4984f1a
[MINOR] Add Sphinx into dev/requirements.txt
HyukjinKwon Jul 10, 2018
a289009
[SPARK-24706][SQL] ByteType and ShortType support pushdown to parquet
wangyum Jul 10, 2018
6fe3286
[SPARK-24678][SPARK-STREAMING] Give priority in use of 'PROCESS_LOCAL…
Jul 10, 2018
e0559f2
[SPARK-21743][SQL][FOLLOWUP] free aggregate map when task ends
cloud-fan Jul 10, 2018
32cb508
[SPARK-24662][SQL][SS] Support limit in structured streaming
mukulmurthy Jul 10, 2018
6078b89
[SPARK-24730][SS] Add policy to choose max as global watermark when s…
tdas Jul 11, 2018
1f94bf4
[SPARK-24530][PYTHON] Add a control to force Python version in Sphinx…
HyukjinKwon Jul 11, 2018
74a8d63
[SPARK-24165][SQL] Fixing conditional expressions to handle nullabili…
mn-mikke Jul 11, 2018
5ff1b9b
[SPARK-23529][K8S] Support mounting volumes
Jul 11, 2018
006e798
[SPARK-23461][R] vignettes should include model predictions for some …
huaxingao Jul 11, 2018
592cc84
[SPARK-24562][TESTS] Support different configs for same test in SQLQu…
mgaido91 Jul 11, 2018
ebf4bfb
[SPARK-24208][SQL] Fix attribute deduplication for FlatMapGroupsInPandas
mgaido91 Jul 11, 2018
290c30a
[SPARK-24470][CORE] RestSubmissionClient to be robust against 404 & n…
rekhajoshm Jul 11, 2018
59c3c23
[SPARK-23254][ML] Add user guide entry and example for DataFrame mult…
WeichenXu123 Jul 11, 2018
ff7f6ef
[SPARK-24697][SS] Fix the reported start offsets in streaming query p…
tdas Jul 11, 2018
e008ad1
[SPARK-24782][SQL] Simplify conf retrieval in SQL expressions
mgaido91 Jul 12, 2018
3ab48f9
[SPARK-24761][SQL] Adding of isModifiable() to RuntimeConfig
MaxGekk Jul 12, 2018
5ad4735
[SPARK-24529][BUILD][TEST-MAVEN] Add spotbugs into maven build process
kiszk Jul 12, 2018
301bff7
[SPARK-23914][SQL] Add array_union function
kiszk Jul 12, 2018
e6c6f90
[SPARK-24691][SQL] Dispatch the type support check in FileFormat impl…
gengliangwang Jul 12, 2018
9fa4a1e
[SPARK-20168][STREAMING KINESIS] Setting the timestamp directly would…
yashs360 Jul 12, 2018
1055c94
[SPARK-24610] fix reading small files via wholeTextFiles
dhruve Jul 12, 2018
395860a
[SPARK-24768][SQL] Have a built-in AVRO data source implementation
gengliangwang Jul 12, 2018
07704c9
[SPARK-23007][SQL][TEST] Add read schema suite for file-based data so…
dongjoon-hyun Jul 12, 2018
1138489
[SPARK-24208][SQL][FOLLOWUP] Move test cases to proper locations
mgaido91 Jul 12, 2018
7572505
[SPARK-24790][SQL] Allow complex aggregate expressions in Pivot
maryannxue Jul 12, 2018
e0f4f20
[SPARK-24537][R] Add array_remove / array_zip / map_from_arrays / arr…
huaxingao Jul 13, 2018
0ce11d0
[SPARK-23486] cache the function name from the external catalog for l…
kevinyu98 Jul 13, 2018
0f24c6f
[SPARK-24713] AppMatser of spark streaming kafka OOM if there are hund…
Jul 13, 2018
dfd7ac9
[SPARK-24781][SQL] Using a reference from Dataset in Filter/Sort migh…
viirya Jul 13, 2018
c1b62e4
[SPARK-24776][SQL] Avro unit test: use SQLTestUtils and replace depre…
gengliangwang Jul 13, 2018
3bcb1b4
Revert "[SPARK-24776][SQL] Avro unit test: use SQLTestUtils and repla…
gatorsmile Jul 13, 2018
3b6005b
[SPARK-23528][ML] Add numIter to ClusteringSummary
mgaido91 Jul 13, 2018
a75571b
[SPARK-23831][SQL] Add org.apache.derby to IsolatedClientLoader
wangyum Jul 13, 2018
f1a99ad
[SPARK-23984][K8S][TEST] Added Integration Tests for PySpark on Kuber…
ifilonenko Jul 14, 2018
e1de341
[SPARK-17091][SQL] Add rule to convert IN predicate to equivalent Par…
wangyum Jul 14, 2018
8aceb96
[SPARK-24754][ML] Minhash integer overflow
srowen Jul 14, 2018
43e4e85
[SPARK-24718][SQL] Timestamp support pushdown to parquet data source
wangyum Jul 15, 2018
3e7dc82
[SPARK-24776][SQL] Avro unit test: deduplicate code and replace depre…
gengliangwang Jul 15, 2018
6999321
[SPARK-24807][CORE] Adding files/jars twice: output a warning and add…
MaxGekk Jul 15, 2018
9603087
[SPARK-24800][SQL] Refactor Avro Serializer and Deserializer
gengliangwang Jul 15, 2018
5d62a98
Doc fix: The Imputer is an Estimator
zoltanctoth Jul 15, 2018
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
157 changes: 58 additions & 99 deletions LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -201,102 +201,61 @@
limitations under the License.


=======================================================================
Apache Spark Subcomponents:

The Apache Spark project contains subcomponents with separate copyright
notices and license terms. Your use of the source code for the these
subcomponents is subject to the terms and conditions of the following
licenses.


========================================================================
For heapq (pyspark/heapq3.py):
========================================================================

See license/LICENSE-heapq.txt

========================================================================
For SnapTree:
========================================================================

See license/LICENSE-SnapTree.txt

========================================================================
For jbcrypt:
========================================================================

See license/LICENSE-jbcrypt.txt

========================================================================
BSD-style licenses
========================================================================

The following components are provided under a BSD-style license. See project link for details.
The text of each license is also included at licenses/LICENSE-[project].txt.

(BSD 3 Clause) netlib core (com.github.fommil.netlib:core:1.1.2 - https://github.com/fommil/netlib-java/core)
(BSD 3 Clause) JPMML-Model (org.jpmml:pmml-model:1.2.7 - https://github.com/jpmml/jpmml-model)
(BSD License) AntLR Parser Generator (antlr:antlr:2.7.7 - http://www.antlr.org/)
(BSD License) ANTLR 4.5.2-1 (org.antlr:antlr4:4.5.2-1 - http://wwww.antlr.org/)
(BSD licence) ANTLR ST4 4.0.4 (org.antlr:ST4:4.0.4 - http://www.stringtemplate.org)
(BSD licence) ANTLR StringTemplate (org.antlr:stringtemplate:3.2.1 - http://www.stringtemplate.org)
(BSD License) Javolution (javolution:javolution:5.5.1 - http://javolution.org)
(BSD) JLine (jline:jline:0.9.94 - http://jline.sourceforge.net)
(BSD) ParaNamer Core (com.thoughtworks.paranamer:paranamer:2.3 - http://paranamer.codehaus.org/paranamer)
(BSD) ParaNamer Core (com.thoughtworks.paranamer:paranamer:2.6 - http://paranamer.codehaus.org/paranamer)
(BSD 3 Clause) Scala (http://www.scala-lang.org/download/#License)
(Interpreter classes (all .scala files in repl/src/main/scala
except for Main.Scala, SparkHelper.scala and ExecutorClassLoader.scala),
and for SerializableMapWrapper in JavaUtils.scala)
(BSD-like) Scala Actors library (org.scala-lang:scala-actors:2.11.8 - http://www.scala-lang.org/)
(BSD-like) Scala Compiler (org.scala-lang:scala-compiler:2.11.8 - http://www.scala-lang.org/)
(BSD-like) Scala Compiler (org.scala-lang:scala-reflect:2.11.8 - http://www.scala-lang.org/)
(BSD-like) Scala Library (org.scala-lang:scala-library:2.11.8 - http://www.scala-lang.org/)
(BSD-like) Scalap (org.scala-lang:scalap:2.11.8 - http://www.scala-lang.org/)
(BSD-style) scalacheck (org.scalacheck:scalacheck_2.11:1.10.0 - http://www.scalacheck.org)
(BSD-style) spire (org.spire-math:spire_2.11:0.7.1 - http://spire-math.org)
(BSD-style) spire-macros (org.spire-math:spire-macros_2.11:0.7.1 - http://spire-math.org)
(New BSD License) Kryo (com.esotericsoftware:kryo:3.0.3 - https://github.com/EsotericSoftware/kryo)
(New BSD License) MinLog (com.esotericsoftware:minlog:1.3.0 - https://github.com/EsotericSoftware/minlog)
(New BSD license) Protocol Buffer Java API (com.google.protobuf:protobuf-java:2.5.0 - http://code.google.com/p/protobuf)
(New BSD license) Protocol Buffer Java API (org.spark-project.protobuf:protobuf-java:2.4.1-shaded - http://code.google.com/p/protobuf)
(The BSD License) Fortran to Java ARPACK (net.sourceforge.f2j:arpack_combined_all:0.1 - http://f2j.sourceforge.net)
(The BSD License) xmlenc Library (xmlenc:xmlenc:0.52 - http://xmlenc.sourceforge.net)
(The New BSD License) Py4J (net.sf.py4j:py4j:0.10.6 - http://py4j.sourceforge.net/)
(Two-clause BSD-style license) JUnit-Interface (com.novocode:junit-interface:0.10 - http://github.com/szeiger/junit-interface/)
(BSD licence) sbt and sbt-launch-lib.bash
(BSD 3 Clause) d3.min.js (https://github.com/mbostock/d3/blob/master/LICENSE)
(BSD 3 Clause) DPark (https://github.com/douban/dpark/blob/master/LICENSE)
(BSD 3 Clause) CloudPickle (https://github.com/cloudpipe/cloudpickle/blob/master/LICENSE)
(BSD 2 Clause) Zstd-jni (https://github.com/luben/zstd-jni/blob/master/LICENSE)
(BSD license) Zstd (https://github.com/facebook/zstd/blob/v1.3.1/LICENSE)

========================================================================
MIT licenses
========================================================================

The following components are provided under the MIT License. See project link for details.
The text of each license is also included at licenses/LICENSE-[project].txt.

(MIT License) JCL 1.1.1 implemented over SLF4J (org.slf4j:jcl-over-slf4j:1.7.5 - http://www.slf4j.org)
(MIT License) JUL to SLF4J bridge (org.slf4j:jul-to-slf4j:1.7.5 - http://www.slf4j.org)
(MIT License) SLF4J API Module (org.slf4j:slf4j-api:1.7.5 - http://www.slf4j.org)
(MIT License) SLF4J LOG4J-12 Binding (org.slf4j:slf4j-log4j12:1.7.5 - http://www.slf4j.org)
(MIT License) pyrolite (org.spark-project:pyrolite:2.0.1 - http://pythonhosted.org/Pyro4/)
(MIT License) scopt (com.github.scopt:scopt_2.11:3.2.0 - https://github.com/scopt/scopt)
(The MIT License) Mockito (org.mockito:mockito-core:1.9.5 - http://www.mockito.org)
(MIT License) jquery (https://jquery.org/license/)
(MIT License) AnchorJS (https://github.com/bryanbraun/anchorjs)
(MIT License) graphlib-dot (https://github.com/cpettitt/graphlib-dot)
(MIT License) dagre-d3 (https://github.com/cpettitt/dagre-d3)
(MIT License) sorttable (https://github.com/stuartlangridge/sorttable)
(MIT License) boto (https://github.com/boto/boto/blob/develop/LICENSE)
(MIT License) datatables (http://datatables.net/license)
(MIT License) mustache (https://github.com/mustache/mustache/blob/master/LICENSE)
(MIT License) cookies (http://code.google.com/p/cookies/wiki/License)
(MIT License) blockUI (http://jquery.malsup.com/block/)
(MIT License) RowsGroup (http://datatables.net/license/mit)
(MIT License) jsonFormatter (http://www.jqueryscript.net/other/jQuery-Plugin-For-Pretty-JSON-Formatting-jsonFormatter.html)
(MIT License) modernizr (https://github.com/Modernizr/Modernizr/blob/master/LICENSE)
(MIT License) machinist (https://github.com/typelevel/machinist)
------------------------------------------------------------------------------------
This product bundles various third-party components under other open source licenses.
This section summarizes those components and their licenses. See licenses/
for text of these licenses.


Apache Software Foundation License 2.0
--------------------------------------

common/network-common/src/main/java/org/apache/spark/network/util/LimitedInputStream.java
core/src/main/java/org/apache/spark/util/collection/TimSort.java
core/src/main/resources/org/apache/spark/ui/static/bootstrap*
core/src/main/resources/org/apache/spark/ui/static/jsonFormatter*
core/src/main/resources/org/apache/spark/ui/static/vis*
docs/js/vendor/bootstrap.js


Python Software Foundation License
----------------------------------

pyspark/heapq3.py


BSD 3-Clause
------------

python/lib/py4j-*-src.zip
python/pyspark/cloudpickle.py
python/pyspark/join.py
core/src/main/resources/org/apache/spark/ui/static/d3.min.js

The CSS style for the navigation sidebar of the documentation was originally
submitted by Óscar Nájera for the scikit-learn project. The scikit-learn project
is distributed under the 3-Clause BSD license.


MIT License
-----------

core/src/main/resources/org/apache/spark/ui/static/dagre-d3.min.js
core/src/main/resources/org/apache/spark/ui/static/*dataTables*
core/src/main/resources/org/apache/spark/ui/static/graphlib-dot.min.js
ore/src/main/resources/org/apache/spark/ui/static/jquery*
core/src/main/resources/org/apache/spark/ui/static/sorttable.js
docs/js/vendor/anchor.min.js
docs/js/vendor/jquery*
docs/js/vendor/modernizer*


Creative Commons CC0 1.0 Universal Public Domain Dedication
-----------------------------------------------------------
(see LICENSE-CC0.txt)

data/mllib/images/kittens/29.5.a_b_EGDP022204.jpg
data/mllib/images/kittens/54893.jpg
data/mllib/images/kittens/DP153539.jpg
data/mllib/images/kittens/DP802813.jpg
data/mllib/images/multi-channel/chr30.4.184.jpg
Loading