-
Notifications
You must be signed in to change notification settings - Fork 3
/
CHANGELOG
284 lines (213 loc) · 14.3 KB
/
CHANGELOG
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
snowplow-media-player 0.9.2 (2024-12-10)
---------------------------------------
## Summary
This feature release introduces dynamic aggregation for passthrough fields in ad_views, allowing greater flexibility in managing session-level analytics.
## Features
**- Dynamic Aggregation for Passthrough Fields:**
Introduced the ability to apply optional aggregation logic (e.g., MAX, MIN) to passthrough fields in the media_player_media_ad_views model. This feature prevents cardinality violations while maintaining compatibility for non-aggregation fields.
**Example configuration:**
```yaml
snowplow__ad_views_passthroughs: [
"v_collector",
{"sql": "v_tracker || app_id", "alias": "tracker_app_id", "agg": "max"},
{"sql": "v_collector", "alias": "v_collector_alias", "agg": "min"}
]
```
This is an optional feature and can be implemented if required by your use case.
## Upgrading
Update the snowplow-media-player version in your packages.yml file.
snowplow-media-player 0.9.1 (2024-10-23)
---------------------------------------
## Summary
This patch release adds app_id as a new field in some of the derived tables and includes a bug fix related to user identification handling in ad_views.
## Features
- Introduced app_id field across some of the derived tables to improve application-level analytics
## Fix
- Optimized user_id aggregation, changed from GROUP BY to MAX(user_id) in queries in ad_view, because it could cause duplicates in an edge case.
## Upgrading
To upgrade, update the snowplow-media-player version in your `packages.yml` file. Please note:
- The new app_id field will be populated for new data points automatically
- Although this is a non-breaking change, historical data will require a [full refresh](https://docs.snowplow.io/docs/modeling-your-data/modeling-your-data-with-dbt/dbt-operation/full-or-partial-refreshes/) to include app_id values, if needed
snowplow-media-player 0.9.0 (2024-10-15)
---------------------------------------
## Summary
This release adds support for Apache Spark with the Iceberg file format and updates integration tests to ensure proper functionality.
## Features
- Add support for Apache Spark with Iceberg file format
## Under the hood
- Modify integration tests to support Spark
## Upgrading
Update the snowplow-media-player version in your `packages.yml` file.
snowplow-media-player 0.8.0 (2024-06-20)
---------------------------------------
## Summary
This release adds support for video playback across multiple pages/screens (e.g., while in picture-in-picture mode).
It also fixes the calculation of impressions in the stats table to use distinct plays instead of page views. This allows for multiple videos on the same page.
## Features
- Add support for picture-in-picture playback of videos across multiple pages/screens (#70)
## Fix
- Fix counting impressions based using distinct play_id instead of page_view_id (#72)
## Under the hood
- Run tests on Redshift in CI (#74)
## 🚨 Breaking Changes 🚨
This version requires a full refresh run if you have been using any previous versions. Otherwise the calculation of the `impressions` and `play_rate` columns in the media stats table will be inconsistent with historical data. Check out the [migration guide](https://docs.snowplow.io/docs/modeling-your-data/modeling-your-data-with-dbt/migration-guides/media-player/) for more information when you upgrade.
snowplow-media-player 0.7.2 (2024-03-18)
---------------------------------------
## Summary
This release makes a few internal tweaks, including adding support for [schema grants](https://docs.snowplow.io/docs/modeling-your-data/modeling-your-data-with-dbt/package-features/table-grants/#granting-usage-on-schemas)
## Features
- Add support for schema grants
## Fix
- Add missing argument in delete from manifest call
## Under the hood
- Enforce full refresh flag to refresh manifest tables
## Upgrading
To upgrade simply bump the snowplow-media-player version in your `packages.yml` file. Note the minimum version of snowplow-utils required is now 0.16.2
snowplow-media-player 0.7.1 (2024-01-26)
---------------------------------------
## Summary
This version bumps the package dependency to add support for the latest snowplow utils package.
## Under the hood
- Bump support for latest utils
## Upgrading
To upgrade simply bump the snowplow-media-player version in your `packages.yml` file.
snowplow-media-player 0.7.0 (2023-12-07)
---------------------------------------
## Summary
This version adds new features powered by a complete refactor of the core processing of the package by moving it out to the new `base` macro functionality provided in `snowplow_utils`. This enables users to now specify custom fields for sessionization and user identification, to add custom entities/SDEs fields to the base events table for redshift/postgres, and to add passthrough fields to the derived tables so you can now more easily add your own fields to our tables.
The default session identifier has been updated from using the domain_sessionid, to now be the media session id (or the page/screen view id if the media session entity is not set). Previously media events from a play that overlapped to a new domain_sessionid were discarded, this update ensures the complete media play is modeled. It is still possible to perform the original session level analysis using the new `domain_sessionid_array` field.
In addition this release adds a more robust unique media identifier. This fixes an issue where duplicate `media_id` values could occur in the media stats table as a result of incorrect tracking implementation (e.g. sharing the same media label across different media types). This release also fixes the incremental materialization of the media_ad_views table by adding a unique primary key.
## Features
- Migrate base models to the new `base` macros for flexibility and consistency
- Updated the default session identifier be the media session id (or page/screen view id if the media session entity is not set)
- Add ability to pass fields through to derived media base and ad views tables
- Add new field `domain_sessionid_array` to derived tables (where applicable)
## Fixes
- Add unique media identifier (close #59)
- Add missing primary key to media_ad_views
- Fix field names in custom session stats model yaml (close #63)
- Fix playback_quality_field macro (close #60)
## 🚨 Breaking Changes 🚨
This version requires a full refresh run if you have been using any previous versions. You will not be able to upgrade and have the package work without doing a full refresh. Check out the [migration guide](https://docs.snowplow.io/docs/modeling-your-data/modeling-your-data-with-dbt/migration-guides/media-player/) for more information when you upgrade.
snowplow-media-player 0.6.1 (2023-10-04)
---------------------------------------
## Summary
This release fixes an issue for users using a utils version 0.15.1 or greater, and fixes an issue with disabling the new contexts on databricks and redshift.
## Fixes
- Fix error with new utils version on end hook
- Fix issue with disabling new contexts on redshift and databricks
## Upgrading
To upgrade simply bump the snowplow-media-player version in your `packages.yml` file.
snowplow-media-player 0.6.0 (2023-09-20)
---------------------------------------
## Summary
This release brings support for new media event and context entity schemas tracked using the JavaScript as well as mobile (iOS and Android) trackers. It also adds new features on top of the new media schemas such as ad tracking models and more accurate playback metrics.
## Features
Add support for new Snowplow media event and entity schemas on Web and mobile (close #49)
Replace snowplow_web with a base that can be compatible with mobile events (close #45)
## Under the hood
Update license to SPAL
Bump support for utils versions
## 🚨 Breaking Changes 🚨
This version requires a full refresh run if you have been using any previous versions. You will not be able to upgrade and have the package work without doing a full refresh. Check out the [migration guide](https://docs.snowplow.io/docs/modeling-your-data/modeling-your-data-with-dbt/migration-guides/media-player/) for more information when you upgrade.
snowplow-media-player 0.5.3 (2023-09-04)
---------------------------------------
## Summary
This release fixes a problem reported with the media_stats model failing during incremental runs when the table is empty.
## Fixes
Fix errors during media_stats incremental runs in case the table is empty (close #51)
## Upgrading
To upgrade simply bump the snowplow-web and snowplow-media-player version in your `packages.yml` file.
snowplow-media-player 0.5.2 (2023-08-16)
---------------------------------------
## Summary
This release fixes an issue on Redshift due to default function names.
## Fixes
Fix current_time issue (Close #46)
## Upgrading
To upgrade simply bump the snowplow-web and snowplow-media-player version in your `packages.yml` file.
snowplow-media-player 0.5.1 (2023-06-27)
---------------------------------------
## Summary
A release to support [dbt-snowplow-web v0.15.0](https://github.com/snowplow/dbt-snowplow-web/releases/tag/0.15.0).
## Features
Bump package versions
## Upgrading
To upgrade simply bump the snowplow-web and snowplow-media-player version in your `packages.yml` file.
snowplow-media-player 0.5.0 (2023-03-29)
---------------------------------------
## Summary
This version contains two major changes, the first is to migrate our models away from the `snowplow_incremental_materialization` and instead move to using the built-in `incremental` with an optimization applied on top. The second is to change the de-duplication logic applied to redshift/postgres to bring it in line with the other warehouses (keeping 1 of the duplicate records, instead of discarding them all). We also upgrade some macros and update some of our docs.
## 🚨 Breaking Changes 🚨
### Changes to materialization
To take advantage of the optimization we apply to the `incremental` materialization, users will need to add the following to their `dbt_project.yml` :
```yaml
# dbt_project.yml
...
dispatch:
- macro_namespace: dbt
search_order: ['snowplow_utils', 'dbt']
```
### Redshift/Postgres custom contexts
The change in de-duplication logic means that now the `events_this_run` table from the web model and downstream tables will contain events that may have duplicates within your self-describing-events or context tables. Previously these events were discarded do there was no risk of duplication when joining a sde/context in a custom model, you must now make sure to de-dupe your sde/context before joining in any custom models. See the docs [here](https://docs.snowplow.io/docs/modeling-your-data/modeling-your-data-with-dbt/dbt-custom-models/#guidelines--best-practice).
For custom models please refer to the [snowplow utils migration guide](https://docs.snowplow.io/docs/modeling-your-data/modeling-your-data-with-dbt/migration-guides/utils/#upgrading-to-0140) and the latest docs on [creating custom incremental models](https://docs.snowplow.io/docs/modeling-your-data/modeling-your-data-with-dbt/dbt-custom-models/#incremental-models).
## Features
- Migrate from `get_cluster_by` and `get_partition_by` to `get_value_by_target_type`
- Migrate all models to use new materialization
- Change de-duplication logic on redshift/postgres
## Docs
- Update readme
## Upgrading
Bump the snowplow-media-player version in your `packages.yml` file, and ensuring you have followed the above steps. You can read more in our [upgrade guide](https://docs.snowplow.io/docs/modeling-your-data/modeling-your-data-with-dbt/migration-guides/media-player/#upgrading-to-0140)
snowplow-media-player 0.4.2 (2023-03-20)
---------------------------------------
## Summary
A release to support dbt-snowplow-web v0.13.0 which introduced a handful of [breaking changes](https://github.com/snowplow/dbt-snowplow-web/releases/tag/0.13.0).
## Features
Add load_tstamp to integration test
snowplow-media-player 0.4.1 (2023-01-26)
---------------------------------------
## Summary
This release fixes a bug that causes data models to fail when a media object is never played. We also add an automatic action to generate our docsite with every new release.
## Features
Add action for generating docs for pages
Fix bug which causes data model to fail when a media object is never played
## Upgrading
To upgrade simply bump the snowplow-web version in your `packages.yml` file.
snowplow-media-player 0.4.0 (2022-12-08)
---------------------------------------
## Summary
**This release drops support for dbt versions below 1.3.** In order to support the new dbt-utils major release, we need to drop support for dbt versions below 1.3. As a result, we've stopped using macros from dbt-utils which removes the deprecation warnings users were seeing.
## Features
Drop support for dbt versions below 1.3 (Close #19)
Add standard actions and templates
## Upgrading
To upgrade simply bump the snowplow-web version in your `packages.yml` file. **You will need dbt v1.3 at least to be able to use this version of the package.**
snowplow-media-player 0.3.4 (2022-12-08)
---------------------------------------
Support dbt-snowplow-web v0.11.0 to take advantage of Databricks performance improvements (Close #30)
snowplow-media-player 0.3.3 (2022-11-30)
---------------------------------------
Remove deduplication for Databricks for get_string_agg macro (Close #23)
Move macros to snowplow_utils (Close #24)
Change incremental logic for media_stats (Close #26)
Handle exception when duration equals zero (Close #27)
snowplow-media-player 0.3.2 (2022-10-26)
---------------------------------------
Migrate core docs to https://docs.snowplow.io/docs (Close #21)
snowplow-media-player 0.3.1 (2022-09-06)
---------------------------------------
Optimize performance in Databricks for incremental models (Close #14)
Improve the GitHub workflow (Close #15)
snowplow-media-player 0.3.0 (2022-08-12)
---------------------------------------
Add support for Databricks (Close #9)
Add automated testing for the media player model (Close #10)
Rename session_stats for custom yml (Close #12)
snowplow-media-player 0.2.0 (2022-06-28)
---------------------------------------
Add support for Snowflake and BigQuery (Close #5)
Make Media Player choice optional (Close #6)
snowplow-media-player 0.1.0 (2022-05-13)
---------------------------------------
Add Media Player model with Redshift and Postgres support (Close #3)