INT96 is convert to datetime

[shiyuhang0]

write with csv

func main() {
	var err error
	md := []string{
		"name=id, type=INT96",
		"name=name, type=BYTE_ARRAY",
	}

	fw, err := local.NewLocalFileWriter("csv.parquet")
	if err != nil {
		log.Println("Can't open file", err)
		return
	}
	pw, err := writer.NewCSVWriter(md, fw, 4)
	if err != nil {
		log.Println("Can't create csv writer", err)
		return
	}

	data := []string{"18446744073709551615", "b"}
	rec := make([]*string, len(data))
	for j := 0; j < len(data); j++ {
		rec[j] = &data[j]
	}
	if err = pw.WriteString(rec); err != nil {
		log.Println("WriteString error", err)
	}

	if err = pw.WriteStop(); err != nil {
		log.Println("WriteStop error", err)
	}
	for _, s := range pw.Footer.Schema {
		println(fmt.Sprintf("%v", *s))
	}
	log.Println("Write Finished")
	fw.Close()
}

read with duckdb

id,name
"4714-11-24 (BC) 00:00:00",b

id become something like datetime, is it reasonable?

[hangxie]

First INT96 was deprecated more than 6 years ago https://github.com/apache/parquet-format/blob/master/CHANGES.md#version-250, you may consider move to something else.

I feel like some tools/libraries don't interpret INT96 properly, I run your code and used online parquet viewers got different results:

1. https://www.parquet-viewer.com/ gives 1717-12-28 19:20:10.805067775
2. https://dataconverter.io/view/parquet/ gives 4713-01-01T11:59:59.999Z
3. https://parquetreader.com/result gives 1970-01-01

Various CLI tools also returned different result, if you expect 1717-12-28 19:20:10.805067775 you may want to try the tool i built:

$ parquet-tools cat csv.parquet
[{"Id":"1717-12-28T19:20:10.805067776Z","Name":"b"}]