Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Pseudo-column for fileSize and fileModifiedTime #1428

Merged
merged 3 commits into from
Sep 13, 2019

Conversation

Praveen2112
Copy link
Member

Fixes #1423. Added two hidden columns for Hive and Iceberg connector.

Copy link
Member

@kokosing kokosing left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

% comments

I think we could also capture split size, but I am not sure how useful this could be.

@@ -45,6 +45,11 @@
public static final HiveType BUCKET_HIVE_TYPE = HIVE_INT;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WRT to commit message:

Add Hive Connector $file_size hidden column

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bump, for this commit and others

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolved it

@Praveen2112
Copy link
Member Author

@kokosing Thanks for your insights. Have applied your comments.

@Praveen2112 Praveen2112 force-pushed the hive_hidden_columns branch 3 times, most recently from d585b36 to 71aaf3c Compare September 9, 2019 11:07
@Praveen2112 Praveen2112 force-pushed the hive_hidden_columns branch 3 times, most recently from d7b6c9a to 604c512 Compare September 10, 2019 14:16
{
long testStartTime = Instant.now().toEpochMilli();

Thread.sleep(2_000);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead of sleep here you could use testStartTime - 2000

@Praveen2112 Praveen2112 force-pushed the hive_hidden_columns branch 2 times, most recently from 4600615 to c40007c Compare September 11, 2019 14:14
@kokosing kokosing merged commit f5608a3 into trinodb:master Sep 13, 2019
@kokosing
Copy link
Member

Merged, thanks!

@kokosing kokosing mentioned this pull request Sep 13, 2019
6 tasks
@Praveen2112 Praveen2112 deleted the hive_hidden_columns branch September 15, 2019 16:26
@martint martint added this to the 319 milestone Sep 20, 2019

TableMetadata tableMetadata = getTableMetadata(catalog, TPCH_SCHEMA, "test_file_modified_time");

List<String> columnNames = ImmutableList.of("col0", "col1", PATH_COLUMN_NAME, FILE_SIZE_COLUMN_NAME, FILE_MODIFIED_TIME_COLUMN_NAME);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider changing test method name to cover more than just "FileModifiedTimeHiddenColumn"

agrawalreetika added a commit to agrawalreetika/prestodb that referenced this pull request Aug 1, 2021
pettyjamesm pushed a commit to prestodb/presto that referenced this pull request Aug 17, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

Pseudocolumn for "last modified" date
4 participants