udf schema inference should not set column nullable=false

[vkrot-exos]

When udf is applied to dataframe and no return schema hint is provided - schema is infered based on data sample and makes wrong assumption about columns nullability. Looks like if all values in some column are not null - resulting schema nullable=false.
But this is a wrong assumption, cause only a sample of data was examined.
Sample code leading to runtime error:

import databricks.koalas as ks
import numpy as np

data = list()
for i in range(1, 10000):
  data.append((str(i % 100), np.nan if i % 9999 == 0 else float(i),))

sdf = spark.createDataFrame(data, 'a string, b float').repartition(10)
kdf = sdf.to_koalas()

def f(df):
  return df

df = kdf\
  .groupby('a')\
  .apply(f)

df.to_spark().printSchema()
df[df['b'] == -1]

printSchema output:

root
 |-- a: string (nullable = false)
 |-- b: float (nullable = false)

Running this code with koalas version 1.3 throws error:

Job aborted due to stage failure: Task 0 in stage 67.0 failed 4 times, most recent failure: Lost task 0.3 in stage 67.0 (TID 435, 10.147.224.152, executor 0): java.lang.IllegalStateException: Value at index is null

[HyukjinKwon]

As a workaround, you can specify the schema in the type hints (https://koalas.readthedocs.io/en/latest/user_guide/typehints.html#type-hinting-with-names) which should be more performant.