Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial run for media_stats does not work on databricks #51

Closed
drphrozen opened this issue Aug 31, 2023 · 3 comments
Closed

Initial run for media_stats does not work on databricks #51

drphrozen opened this issue Aug 31, 2023 · 3 comments

Comments

@drphrozen
Copy link

The following line dosen't yield any results when the table is empty in databricks.

and p.start_tstamp > ( select max(last_base_tstamp) from {{ this }} )

@github-actions github-actions bot added the status:needs_triage Needs maintainer triage. label Aug 31, 2023
@rlh1994
Copy link
Contributor

rlh1994 commented Aug 31, 2023

Hi @drphrozen , that line is within an is_incremental block so it won't be that line running on the initial run anyway. Are you saying that when an initial run results in the table being empty that then subsequent runs do not populate because of this, or is your issue around the initial run not having any data in that table?

@rlh1994 rlh1994 removed the status:needs_triage Needs maintainer triage. label Aug 31, 2023
@drphrozen
Copy link
Author

drphrozen commented Aug 31, 2023

The table was never populated, either by the initial or subsequent. Solved our issue by adding a dummy row with a old timestamp and some random media id.

I did not deep dive into the issue.. just ran the compiled section for this SQL and found that this part of the where statement was the issue.

@rlh1994
Copy link
Contributor

rlh1994 commented Aug 31, 2023

Okay that's helpful thanks, I wasn't sure if a date is always greater than null or if it returns null but it looks like it returns null. We're currently in the middle of a big update to the package so we'll make sure it's caught in there. cc @matus-tomlein

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants