fix: Remove opening file object when validating S3 parquet source (fe…

…ast-dev#3217) * Remove opening the file object Let pyarrow handle opening the path using the filesystem. Signed-off-by: Max Z <[email protected]> * fix: linting error Signed-off-by: Max Z <[email protected]> Signed-off-by: Max Z <[email protected]>
franciscojavierarceo · Sep 16, 2022 · a906018 · a906018
1 parent ae37b20
commit a906018
Showing 1 changed file with 1 addition and 3 deletions.
diff --git a/sdk/python/feast/infra/offline_stores/file_source.py b/sdk/python/feast/infra/offline_stores/file_source.py
@@ -160,9 +160,7 @@ def get_table_column_names_and_types(
         if filesystem is None:
             schema = ParquetDataset(path).schema.to_arrow_schema()
         else:
-            schema = ParquetDataset(
-                filesystem.open_input_file(path), filesystem=filesystem
-            ).schema
+            schema = ParquetDataset(path, filesystem=filesystem).schema
 
         return zip(schema.names, map(str, schema.types))