Skip to content

Commit

Permalink
[MINOR][DOCS][PYTHON] Adding missing boolean type for replacement val…
Browse files Browse the repository at this point in the history
…ue in fillna

## What changes were proposed in this pull request?

Currently pyspark Dataframe.fillna API supports boolean type when we pass dict, but it is missing in documentation.

## How was this patch tested?
>>> spark.createDataFrame([Row(a=True),Row(a=None)]).fillna({"a" : True}).show()
+----+
|   a|
+----+
|true|
|true|
+----+

Please review http://spark.apache.org/contributing.html before opening a pull request.

Author: Srinivasa Reddy Vundela <[email protected]>

Closes #17688 from vundela/fillna_doc_fix.
  • Loading branch information
Srinivasa Reddy Vundela authored and Felix Cheung committed May 1, 2017
1 parent ae3df4e commit 6613046
Show file tree
Hide file tree
Showing 2 changed files with 5 additions and 1 deletion.
2 changes: 1 addition & 1 deletion python/pyspark/sql/dataframe.py
Original file line number Diff line number Diff line change
Expand Up @@ -1247,7 +1247,7 @@ def fillna(self, value, subset=None):
Value to replace null values with.
If the value is a dict, then `subset` is ignored and `value` must be a mapping
from column name (string) to replacement value. The replacement value must be
an int, long, float, or string.
an int, long, float, boolean, or string.
:param subset: optional list of column names to consider.
Columns specified in subset that do not have matching data type are ignored.
For example, if `value` is a string, and subset contains a non-string column,
Expand Down
4 changes: 4 additions & 0 deletions python/pyspark/sql/tests.py
Original file line number Diff line number Diff line change
Expand Up @@ -1711,6 +1711,10 @@ def test_fillna(self):
self.assertEqual(row.age, None)
self.assertEqual(row.height, None)

# fillna with dictionary for boolean types
row = self.spark.createDataFrame([Row(a=None), Row(a=True)]).fillna({"a": True}).first()
self.assertEqual(row.a, True)

def test_bitwise_operations(self):
from pyspark.sql import functions
row = Row(a=170, b=75)
Expand Down

0 comments on commit 6613046

Please sign in to comment.