Releases: snowplow/snowplow
Releases · snowplow/snowplow
Release 112 Baalbek (2019-02-19)
This release brings reliability improvements to the batch pipeline
EmrEtlRunner
- Bump to 0.34.0 (#3935)
- Add support for running steps on a persistent EMR cluster (#3930)
- Recover from request timeout (#3943)
- Set timeout to 120 seconds (#3949)
- Leverage compaction steps when copying to S3 (#3940)
- Recover from S3 internal errors (#3950)
- Rename EMR steps (#3925)
- Scrub credentials from stderr (#2815)
- Bump elasticity to 6.0.14 (#3948)
- Extend copyright notice to 2019 (#3978)
Clojure Collector
- Bump to 2.1.3 (#3947)
- Add ebextension to increase the number of file descriptors to 65536 (#3876)
- Extend copyright notice to 2019 (#3979)
Storage
Release 111 Selinunte (2018-10-02)
Release 110 Valle dei Templi (2018-09-07)
This release brings a new enrichment process running on Google Cloud Dataflow: Beam Enrich
Beam Enrich
- Implement a barebone port of Stream Enrich (#3735)
- Support enrichments relying on local files (#3736)
- Add support for the PII enrichment (#3888)
- Add metrics (#3737)
- Build Docker image (#3815)
- Add README (#3773)
- Add CI/CD (#3757)
Stream Enrich
Clojure Collector
Common
Release 109 Lambaesis (2018-08-21)
This release is a maintenance real-time release although it also upgrades Spark Enrich as well as brings a bugfix to EmrEtlRunner.
Scala Stream Collector
- Bump to 0.14.0 (#3862)
- Respect a do not track cookie (#3825)
- Add a way to customize the response from the root path (#3670)
- Support HEAD requests (#3827)
- Allow for multiple domains in crossdomain.xml (#3740)
- Allow overriding of the kinesis endpoint url in the configuration (#3846)
- Turn BufferConfig's byteLimit and recordLimit into longs (#3807)
Scala Common Enrich
- Bump to 0.35.0 (#3861)
- Externalize ua-parser rule file (#3793)
- Bump ua-parser to 1.4.0 (#3811)
- Make the list of files to cache available from the EnrichmentRegistry (#3789)
- Change Iglu adapter to consider arrays as multiple events (#3858)
- Update CloudfrontAccessLogAdapter to support newer 26 field format (#3816)
- Leverage the x-forwarded-for field in CloudfrontLoader (#2859)
- Handle comma-separated list of ips (#3771)
- Bump SBT Bintray to 0.5.4 (#3840)
- Bump SBT to 1.1.6 (#3839)
Stream Enrich
- Bump to 0.19.0 (#3864)
- Bump scala-common-enrich to 0.35.0 (#3863)
- Allow overriding of the kinesis endpoint url in the configuration (#3775)
- Decorrelate the need for a pii stream and the pii enrichment (#3828)
- Bump SBT to 1.1.6 (#3841)
- Fix configuration example (#3820)
Spark Enrich
- Bump to 1.1.6.0 (#3874)
- Bump scala-common-enrich to 0.35.0 (closes #3869)
- Bump SBT Assembly to 0.14.7 (#3844)
- Bump SBT to 1.1.6 (#3843)
EmrEtlRunner
- Bump to 0.33.1 (#3867)
- Replace recursive functions by their iterative versions (#3866)
- Retrieve correct latest run id when using s3a (#3871)
- Update spark_enrich version in config samples to 1.16.0 (#3859)
Common
- Add pull request template (#3818)
- Add issue template (#3819)
- Update CONTRIBUTING.md (#3530)
- Update to new logo in README (#3855)
- Remove Vagrant setup (#3851)
- Add Gitter badge (#3838)
- Bump Travis Scala version to 2.11.12 (#3837)
Miscellaneous
Release 108 Val Camonica (2018-07-24)
This release brings end-to-end encryption capabilities to the batch pipeline as well as update the Clojure Collector.
EmrEtlRunner
- Bump to 0.33.0 (#3800)
- Add ability to specify an EMR security configuration (#3798)
- Handle SSE-S3 encrypted S3 buckets (#3456)
- Replace Sluice by aws-sdk-s3 (#3524)
- Add --ignore-lock-on-start option (#3537)
- Check the processing folder for emptiness when resuming from enrich (#3803)
- Make port in Snowplow monitoring configurable (#3236)
- Make protocol in Snowplow monitoring configurable (#3791)
Clojure Collector
- Bump to 2.1.0 (#3801)
- Make cookie path configurable (#2739)
- Do not allow dependencies requiring an HTTP repository (#3559)
- Bump lein-ring to 0.12.4 (#3783)
- Bump commons-codec to 1.11 (#3782)
- Bump metrics-clojure to 2.10.0 (#3781)
- Bump compojure to 1.6.1 (#3780)
- Bump clojure to 1.9.0 (#3779)
- Bump ring to 1.6.3 (#3778)
- Remove lein-beanstalk (#3784)
Release 107 Trypillia (2018-07-17)
This release brings a new enrichment to detect bots and spiders as well as two new webhooks integration (Vero and Marketo).
Scala Common Enrich
Add adapter to pre-process Marketo webhooks (#2616)
Add adapter to pre-process Vero webhooks (#2757)
Propagate the currency code to all the contexts which need it in the GA adapter (#3733)
Add IAB Spiders & Robots Enrichment (#937)
Bump to 0.34.0 (#3758)
Stream Enrich
Bump scala-common-enrich to 0.34.0 (#3730)
Add support for the IAB enrichment (#3797)
Force jackson-databind to 2.9.3 for all projects (#3767)
Rename force-ip-lookups-download to force-cached-files-download (#3809)
Bump to 0.18.0 (#3727)
Spark Enrich
Bump scala-common-enrich to 0.34.0 (#3729)
Add support for the IAB Enrichment (#3772)
Bump to 1.15.0 (#3728)
Misc
Fix travis deployment test (#3805)
Update example config for PII to version 2-0-0 (#3812)
Release 106 Acropolis (2018-06-14)
This release brings a new version of the PII enrichment for both the batch and real-time pipelines.
Scala Common Enrich
- Add formats as ScalazJson4sUtils.extract as implicit parameter (#3668)
- Add salt to PII Enrichment (#3648)
- Bump to 0.33.0 (#3763)
- Bump user-agent-utils to 1.21 (#3656)
- Extend PII Enrichment to include identification events in EnrichedEvent (#3580)
- Fix platform specific error checking in IpLookupsEnrichmentSpec (#3762)
- Fix unnecessarily-created JSON object as a result of the PII Enrichment (#3636)
Spark Enrich
- Apply automated code formatting (#3655)
- Bump scala-common-enrich to 0.33.0 (#3764)
- Bump to 1.14.0 (#3765)
- Ignore PII identification events from Scala Common Enrich (#3582)
- Use automated code formatting (#3654)
Stream Enrich
- Add context for parent event when generating PII event (#3724)
- Add end-to-end test using mock streaming (#3639)
- Apply automated code formatting (#3651)
- Bump scala-common-enrich to 0.33.0 (#3607)
- Bump to 0.17.0 (#3608)
- Extend PII Enrichment to output a stream of PII identification events (#3581)
- Update config.hocon.sample to include a PII output stream (#3579)
- Use automated code formatting (#3644)
EmrEtlRunner
Release 105 Pompeii (2018-05-07)
Release 104 Stoplesteinan (2018-04-30)
Fixes issues with the Stream Enrich mode introduced in EmrEtlRunner in R102.
EmrEtlRunner
- Bump to 0.32.0 (#3723)
- Fail fast when trying to skip staging or enrich in stream enrich mode (#3726)
- Factor out steps-generating function (#3718)
- Uncompress enriched files when copying to HDFS (#3719)
- Fix srcPattern for copying stream enriched data to HDFS (#3722)
- Check if whole enriched.good is non-empty in stream enrich mode (#3717)
Common
- Remove trailing hyphen from CHANGELOG entry for R103 (#3731)
Release 103 Paestum (2018-04-17)
Upgrading the IP Lookups Enrichment
Scala Common Enrich
- Bump to 0.32.0 (#3673)
- Bump scala-maxmind-iplookups to 0.4.0 (#3675)
- Update IP Lookups Enrichment to support non-legacy database (#3672)
- Support extraction of IP addresses in the Forwarded header (#3475)
- Support IPv6 addresses in the IpAddressExtractor (#3474)
- Bump mandrill event versions to 1-0-1 (#3372)
Stream Enrich
- Bump to 0.16.0 (#3698)
- Bump scala-common-enrich to 0.32.0 (#3676)
- Force jackson-dataformat-cbor to 2.9.3 (#3701)
Spark Enrich
Clojure Collector
- Bump to 2.0.0 (#3708)
- Make Flash access domains and secure configurable (#2914)
- Retrieve configuration only through JVM properties (#3709)
- Allow HTTP repositories (#3707)
- Add CI/CD (#3712)
Config
- Update database value in config/enrichments/ip_lookups.json (#3671)
EmrEtlRunner
Update spark_enrich version in config.yml.sample to 1.13.0 (#3710)