Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix Druid metadata refresh. #192

Merged
merged 3 commits into from
Mar 26, 2016
Merged

Fix Druid metadata refresh. #192

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
7a7eb33
Disable Datasource sql polymorphism & Fix refresh Druid metadata.
kim-pham-airbnb Mar 25, 2016
23b6bca
Fix style warning
kim-pham-airbnb Mar 25, 2016
2378fdf
Disable polymorphism in DruidMetric as well
kim-pham-airbnb Mar 25, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 12 additions & 4 deletions dashed/models.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -840,8 +840,12 @@ def latest_metadata(self):
return
max_time = results[0]['result']['maxTime']
max_time = parse(max_time)
intervals = (max_time - timedelta(seconds=1)).isoformat() + '/'
intervals += (max_time + timedelta(seconds=1)).isoformat()
# Query segmentMetadata for 7 days back. However, due to a bug,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mistercrunch I have to query [-7 day, -1 day] range because of a bug in 0.8.0. (see comment).

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why we need to query for 7 days? even 10 minutes is good enough

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Max has a point that some segment might be outside of the range. -7 is a heuristic to get at least one segment.

# we need to set this interval to more than 1 day ago to exclude
# realtime segments, which trigged a bug (fixed in druid 0.8.2).
# https://groups.google.com/forum/#!topic/druid-user/gVCqqspHqOQ
intervals = (max_time - timedelta(days=7)).isoformat() + '/'
intervals += (max_time - timedelta(days=1)).isoformat()
segment_metadata = client.segment_metadata(
datasource=self.datasource_name,
intervals=intervals)
Expand Down Expand Up @@ -1092,7 +1096,9 @@ class DruidMetric(Model):
datasource_name = Column(
String(250),
ForeignKey('datasources.datasource_name'))
datasource = relationship('DruidDatasource', backref='metrics')
# Setting enable_typechecks=False disables polymorphic inheritance.
datasource = relationship('DruidDatasource', backref='metrics',
enable_typechecks=False)
json = Column(Text)
description = Column(Text)

Expand All @@ -1114,7 +1120,9 @@ class DruidColumn(Model):
datasource_name = Column(
String(250),
ForeignKey('datasources.datasource_name'))
datasource = relationship('DruidDatasource', backref='columns')
# Setting enable_typechecks=False disables polymorphic inheritance.
datasource = relationship('DruidDatasource', backref='columns',
enable_typechecks=False)
column_name = Column(String(256))
is_active = Column(Boolean, default=True)
type = Column(String(32))
Expand Down