Adloox (adloox.com) RTD provider, analytic and adserver modules #6174

jimdigriz · 2021-01-10T19:55:11Z

Type of change

New Analytics adapter (documentation)
- New integration example
New RTD provider (documentation)
New Ad Server for Video (documentation)

N.B. not yet to be merged(!), submitting a PR to start the requests for comments and review process

Description of change

Introduction of analytics, RTD provider and Ad Server for Video modules that support Adloox's publisher services.

contact email of the adapter’s maintainer: [email protected] and [email protected]
this is an official submission

Disclosures:

Analytics: on BID_WON adds the Adloox post-buy verification script (~25kB gzipped from j.adlooxtracking.com) and locates it immediately after the ad slot in the DOM
RTD: loads the generic segmentation script (~3kB gzipped from p.adlooxtracking.com) and provides no user orientated segments (only page, slot quality and IVT/fraud related); used to provide a consistent experience with our GPT clients as it also includes observability/tracing for our monitoring purposes which would not be appropriate to be included here.

These modules automate the complex tasks an AdOps teams must perform when integrating with Adloox, including the script loading listed in the disclosures above. If the any of the scripts fail to load, it impacts only Adloox's measurement and does not affect the operation of Prebid.js or inventory fill for the publisher.

I do want to say, though early days for us with Prebid.js, it has been a positive experience from both the developer and AdOps perspectives. Our thanks go out to everyone involved in contributing and maintaining an exciting project that makes solving hard problems easy.

Other information

analytics module provides a command processing queue to the other Adloox modules to provide configuration sharing and macro/URL expansion facilities
- exists to reduce the burden on AdOps teams by preventing typos between each modules configuration that should be keep identical (eg. {client,tag,platform}id, id[0-9] parameters, ...)
- created as events.js does not support custom events and is a shared event bus
- works by exporting the same internal command array between all the modules; does not use the global namespace
  - these intra-module imports do trip the the eslint's 'prebid/validate-imports' check which I have made a warning for now to highlight them during the review process; the plan is to later to disable those checks pre-merge
Ad Server for Video module is required to bring MRC accredited viewability metrics to video inventory
- converts the VAST URL into an Adloox wrapped VAST/VPAID tag (j.adlooxtracking.com/ads/vast/tag.php)
- optionally (enabled by default) walks the VAST chain and rewrites the fetched VAST tags in the chain to use Blob URLs to avoid duplicate requests; solves the no-fill correlator related problems you get with GAM tags
RTD module
- Looking for feedback on what to do about getGlobal().adUnits in the RTD module; I looked to use auctionManager.getAdUnits() but it does not seem to be initialised early on enough to have any adunits in there and though supposedly a no-no I see the other RTD modules are using getGlobal() so I just copied them on this one. Is there something better I could be doing here?
8a2811f (slot hunting) and 2ad2d5d (legacy) are planned to be squashed/fixed up into the main commit (69110f7) after some further review with clients
'KLUDGE' commit (73217b7) will be removed once I figure out (advice welcomed!) a better integration example and merge the changes in tag-dev.php into tag.php in production on Adloox's infrastructure

jimdigriz · 2021-01-11T08:53:01Z

I am unable to see what CircleCI is complaining about (it wants full read/write access to all my public/private repositories which I am not prepared to do) but tests work as expected for me and coverage for our modules is ~95%:

alex@aineko:~/src/adloox/Prebid.js$ gulp test
[08:51:15] Using gulpfile ~/src/adloox/Prebid.js/gulpfile.js
[08:51:15] Starting 'test'...
[08:51:15] Starting 'clean'...
[08:51:15] Finished 'clean' after 49 ms
[08:51:15] Starting 'lint'...
[08:51:37] 
/home/alex/src/adloox/Prebid.js/modules/adlooxAdServerVideo.js
  11:1  warning  import "./adlooxAnalyticsAdapter.js" cannot require module entry point  prebid/validate-imports

/home/alex/src/adloox/Prebid.js/modules/adlooxRtdProvider.js
  14:1  warning  import "./adlooxAnalyticsAdapter.js" cannot require module entry point  prebid/validate-imports

✖ 2 problems (0 errors, 2 warnings)

[08:51:37] Finished 'lint' after 21 s
[08:51:37] Starting 'test'...

START:
⚠ ｢wdm｣: 
ℹ ｢wdm｣: Compiled with warnings.
ℹ ｢wdm｣: Compiling...
⚠ ｢wdm｣: 
ℹ ｢wdm｣: Compiled with warnings.
11 01 2021 08:52:11.249:INFO [karma-server]: Karma v4.4.1 server started at http://0.0.0.0:9876/
11 01 2021 08:52:11.250:INFO [launcher]: Launching browsers ChromeHeadless with concurrency 5
11 01 2021 08:52:11.252:INFO [launcher]: Starting browser ChromeHeadless
11 01 2021 08:52:12.062:INFO [HeadlessChrome 87.0.4280 (Linux 0.0.0)]: Connected on socket dYLL2RqbRh8wnXOsAAAA with id 7427373

Finished in 8.081 secs / 5.058 secs @ 08:52:21 GMT+0000 (Greenwich Mean Time)

SUMMARY:
✔ 7305 tests completed
ℹ 5 tests skipped
[08:52:21] Finished 'test' after 44 s
[08:52:21] Finished 'test' after 1.08 min

aleksatr · 2021-01-12T11:01:07Z

I am unable to see what CircleCI is complaining about (it wants full read/write access to all my public/private repositories which I am not prepared to do) but tests work as expected for me and coverage for our modules is ~95%:

alex@aineko:~/src/adloox/Prebid.js$ gulp test
[08:51:15] Using gulpfile ~/src/adloox/Prebid.js/gulpfile.js
[08:51:15] Starting 'test'...
[08:51:15] Starting 'clean'...
[08:51:15] Finished 'clean' after 49 ms
[08:51:15] Starting 'lint'...
[08:51:37] 
/home/alex/src/adloox/Prebid.js/modules/adlooxAdServerVideo.js
  11:1  warning  import "./adlooxAnalyticsAdapter.js" cannot require module entry point  prebid/validate-imports

/home/alex/src/adloox/Prebid.js/modules/adlooxRtdProvider.js
  14:1  warning  import "./adlooxAnalyticsAdapter.js" cannot require module entry point  prebid/validate-imports

✖ 2 problems (0 errors, 2 warnings)

[08:51:37] Finished 'lint' after 21 s
[08:51:37] Starting 'test'...

START:
⚠ ｢wdm｣: 
ℹ ｢wdm｣: Compiled with warnings.
ℹ ｢wdm｣: Compiling...
⚠ ｢wdm｣: 
ℹ ｢wdm｣: Compiled with warnings.
11 01 2021 08:52:11.249:INFO [karma-server]: Karma v4.4.1 server started at http://0.0.0.0:9876/
11 01 2021 08:52:11.250:INFO [launcher]: Launching browsers ChromeHeadless with concurrency 5
11 01 2021 08:52:11.252:INFO [launcher]: Starting browser ChromeHeadless
11 01 2021 08:52:12.062:INFO [HeadlessChrome 87.0.4280 (Linux 0.0.0)]: Connected on socket dYLL2RqbRh8wnXOsAAAA with id 7427373

Finished in 8.081 secs / 5.058 secs @ 08:52:21 GMT+0000 (Greenwich Mean Time)

SUMMARY:
✔ 7305 tests completed
ℹ 5 tests skipped
[08:52:21] Finished 'test' after 44 s
[08:52:21] Finished 'test' after 1.08 min

Hi @jimdigriz , from what I see that's a warning reported by linter, saying:

/home/aleksatr/workspace/rtk.io/Prebid.js/modules/adlooxAdServerVideo.js
  11:1  warning  import "./adlooxAnalyticsAdapter.js" cannot require module entry point  prebid/validate-imports

/home/aleksatr/workspace/rtk.io/Prebid.js/modules/adlooxRtdProvider.js
  14:1  warning  import "./adlooxAnalyticsAdapter.js" cannot require module entry point  prebid/validate-imports

Tracing it to the source validateImports.js:

// don't allow import of any files directly within modules folder or index.js files within modules' sub-folders
    if (
      path.dirname(absImportPath) === absModulePath || (
        absImportPath.startsWith(absModulePath) &&
        path.basename(absImportPath) === 'index.js'
      )
    ) {
      context.report(node, `import "${importPath}" cannot require module entry point`);
    }

jimdigriz · 2021-01-12T11:04:53Z

Hi @jimdigriz , from what I see that's a warning reported by linter, saying:

/home/aleksatr/workspace/rtk.io/Prebid.js/modules/adlooxAdServerVideo.js
  11:1  warning  import "./adlooxAnalyticsAdapter.js" cannot require module entry point  prebid/validate-imports

/home/aleksatr/workspace/rtk.io/Prebid.js/modules/adlooxRtdProvider.js
  14:1  warning  import "./adlooxAnalyticsAdapter.js" cannot require module entry point  prebid/validate-imports

Tracing it to the source validateImports.js:
[snipped]

I mentioned this in the 'Other information' above: "these intra-module imports do trip the the eslint's 'prebid/validate-imports' check which I have made a warning for now to highlight them during the review process; the plan is to later to disable those checks pre-merge"

If this is the only reason CircleCI is upset, excellent means that everything is good at least on that front! My current plan is to have eslint ignore those errors completely unless an alternative approach is suggested.

Thanks

bretg · 2021-01-14T17:45:05Z

modules/adlooxAdServerVideo.md

+
+Ad Server Video for adloox.com. Contact [email protected] for information.
+
+Prebid.js does support sending the [`bidWon` event for video](https://github.com/prebid/prebid.github.io/issues/1320) so for this situation this module can be used to synthesise this event for the [Adloox Analytics Adapter](./adlooxAnalyticsAdapter.md).


Assume you mean does not support.

This statement is awkward and the purpose of this module is buried well into the documentation. Here's what I think you're trying to say:

"This module provides video events to the Adloox analytics adapter.

Prebid.js doesn't support bidsWon events for in-stream video, and the Instream Video Ads Tracking feature it does offer has limitations.

This module has two modes of operation configurable with the wrap parameter below:

..."

I rejiggled it in fc2892e, let me know if it makes things clearer. Thanks

modules/adlooxRtdProvider.md

bretg · 2021-01-14T18:33:58Z

modules/adlooxRtdProvider.md

+
+RTD provider for adloox.com. Contact [email protected] for information.
+
+In addition to populating the ad server key-value targeting, fetched segments (prefixed with `adl_...`) will also populate [First Party Data](https://docs.prebid.org/features/firstPartyData.html), some examples of the segments as described by the Adloox 'Google Publisher Tag Targeting Guidelines' and where they are placed are:


Please describe the key-value targeting. Is this an integral part of the service or can it be broken out?

Sorry, I do not understand your question. Do not all RTD modules provide key-value targeting? What are you proposing or hinting could be broken out here?

Our clients make use of the key-value targeting in GAM as that is where (for example) they decide to swap in a house ad when an IVT segment lights up.

I guess I could add a flag whether to make getTargetingData() a NOOP if this is what you are referring to?

bretg · 2021-01-14T18:35:30Z

modules/adlooxRtdProvider.md

+      ...
+
+      realTimeData: {
+        auctionDelay: 700,             // recommended to be at least 700ms


I do not like this recommendation at all. Nor the requirement that waitForIt to be true. Will discuss with the community.

The intention for RTD modules is that they are fast. 50ms. 100ms maybe. 700 is so far out of range it is likely to destroy a publishers ability to get successful bids.

The 700ms is not for us, as described in the documentation below that, but as I have not seen publishers differentiate between office leased lines and contended mobile dongles we had to pick something; those that do use smarts would ignore this advice anyway. I settled on this number as there are only really two RTD modules with recommendations to glean from; browsiRtdProvider.js (1000ms) and jwplayerRtdProvider.js (100ms).

My understanding is that auctionDelay is only a deadline, if the bidders (and RTD sources) all return before that time limit then that phase is over and we move on, right?

We definitely do not intend to delay anything by 700ms and we would be just as comfortable with a ~100ms which is enough to cross either the US or the EU...my packet-pushing-hat does raise an eyebrow that a 3G/4G phone could reliably meet that deadline whilst simultaneously pulling in publisher content. Does it? Anyway, more than happy to lower the threshold warning or better still replace it with a warning that we missed the auction instead?

On the waitForIt note, can you describe the concern as the publisher controls this and not us. I do not understand how an RTD could function if the publisher is not willing to wait at all for it to call out to an API and pull in the segments to use during the auction process? I am missing something here, can you explain what I should be doing instead?

Maybe I am confused and should be using one of and not both waitForIt or auctionDelay and this is what I am doing wrong? I understood auctionDelay to be a NOOP unless you have waitForIt?

The behaviour I am looking for is that we want a maximum of Xms (auctionDelay?) and if we miss that our RTD does not get to participate in setting FPD for the auction.

bretg · 2021-01-20T21:07:26Z

On the waitForIt note, can you describe the concern as the publisher controls this and not us.

The idea behind waitForIt is that publishers can decide which modules are worth waiting for and which better hustle. Imagine a bus stop. The bus driver will wait up to N ms for important passengers A, J, and X. Once these 3 passengers are on the bus, it will leave immediately. Other potential passengers need to get on before those 3 or they will be left behind.

This isn't well documented -- I'll come up with something.

What we object to in this PR is that your module is specifically requiring at the code level to be one of the "important" passengers. Not allowed. You can ask the publisher in the docs to consider you important. You can show the customer data that the flag makes a difference.

I understood auctionDelay to be a NOOP unless you have waitForIt

Nope. Clearly the docs are lacking here. AuctionDelay is the required thing. waitForIt is a list of VIPs. If there aren't any VIPs, the bus will wait for the full AuctionDelay before leaving.

browsiRtdProvider.js (1000ms)

Thanks for pointing this out. @Asafsham - please lower this value in the Browsi docs. 100ms?

Maybe I am confused and should be using one of and not both waitForIt or auctionDelay

The sub-modules themselves shouldn't look at either value. They are control levers. Your sub-module should let the levers work and not complain about their values. Again, you're welcome to request that publishers set certain values in your documentation, but not enforce that in code.

The behaviour I am looking for is that we want a maximum of Xms (auctionDelay?) and if we miss that our RTD does not get to participate in setting FPD for the auction

This is what happens automatically. You get a chance to be a passenger on the bus. Be to the station on time or miss is. Your code should just be as fast as it can be.

a 3G/4G phone [might not] reliably meet that deadline

It's possible that mobile web pages might need higher timeouts. Those publishers can decide what values work best for them.
javascript is single threaded. on busy web pages, we've seen requests take longer than the timeout and still succeed because of the order things are placed on the javascript queue.

gglas · 2021-01-20T21:36:35Z

In addition to Bret's feedback, which thanks for uncovering the gap in our documentation, we spoke with the community and we'd like to request that these are broken out into 3 separate PR's each for its own module. It's fine to denote in documentation that they work in tandem and communicate with one another, but the code itself should be implementable standalone without affecting any other prebid functionality. We'll then provide relevant feedback on each PR for each item in order to keep things clean -- thank you so much!

bretg · 2021-01-20T21:54:00Z

Hopefully this will close the documentation gap -- prebid/prebid.github.io#2642

jimdigriz · 2021-01-20T22:27:35Z

In addition to Bret's feedback, which thanks for uncovering the gap in our documentation, we spoke with the community and we'd like to request that these are broken out into 3 separate PR's each for its own module.

Not a problem. Quick question, once I have submitted those PRs and added references to them here, do you want me to close this PR (#6174 )?

This workaround is needed for the demo to work DO NOT USE IN PRODUCTION!

jimdigriz · 2021-02-13T19:09:54Z

Moved conversation into #6308, #6309 and #6310

jimdigriz force-pushed the feature/adloox branch 3 times, most recently from 408ef08 to 881ca29 Compare January 10, 2021 22:55

jimdigriz force-pushed the feature/adloox branch from 881ca29 to 6d987bc Compare January 11, 2021 09:05

smenzer assigned aleksatr Jan 11, 2021

smenzer added feature minor needs review labels Jan 11, 2021

jimdigriz force-pushed the feature/adloox branch 4 times, most recently from 60700ba to 3c1f966 Compare January 12, 2021 10:10

jimdigriz force-pushed the feature/adloox branch 5 times, most recently from d5ac550 to 73217b7 Compare January 12, 2021 17:42

bretg reviewed Jan 14, 2021

View reviewed changes

jimdigriz added a commit to corememltd/Prebid.js that referenced this pull request Jan 14, 2021

ws (prebid#6174 (comment))

c11a140

jimdigriz added a commit to corememltd/Prebid.js that referenced this pull request Jan 14, 2021

ws (prebid#6174 (comment))

fc2892e

jimdigriz added 3 commits February 13, 2021 16:24

Adloox analytic module

8df5fe6

Adloox (video) adserver module

3d73c47

KLUDGE: get video tracking working

4ddcffa

This workaround is needed for the demo to work DO NOT USE IN PRODUCTION!

Adloox real time data module

eddc700

jimdigriz force-pushed the feature/adloox branch from fb7b601 to eddc700 Compare February 13, 2021 18:18

This was referenced Feb 13, 2021

Adloox Analytics Adapter: add new analytics adapter #6308

Merged

Adloox Video Adserver Module: add new adserver module #6309

Merged

Adloox RTD Module: add new RTD module #6310

Merged

jimdigriz closed this Feb 13, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adloox (adloox.com) RTD provider, analytic and adserver modules #6174

Adloox (adloox.com) RTD provider, analytic and adserver modules #6174

jimdigriz commented Jan 10, 2021 •

edited

Loading

jimdigriz commented Jan 11, 2021 •

edited

Loading

aleksatr commented Jan 12, 2021

jimdigriz commented Jan 12, 2021 •

edited

Loading

bretg Jan 14, 2021

jimdigriz Jan 14, 2021 •

edited

Loading

bretg Jan 14, 2021

jimdigriz Jan 14, 2021 •

edited

Loading

bretg Jan 14, 2021

jimdigriz Jan 14, 2021 •

edited

Loading

bretg commented Jan 20, 2021

gglas commented Jan 20, 2021

bretg commented Jan 20, 2021

jimdigriz commented Jan 20, 2021

jimdigriz commented Feb 13, 2021


		Ad Server Video for adloox.com. Contact [email protected] for information.

		Prebid.js does support sending the [`bidWon` event for video](https://github.com/prebid/prebid.github.io/issues/1320) so for this situation this module can be used to synthesise this event for the [Adloox Analytics Adapter](./adlooxAnalyticsAdapter.md).


		RTD provider for adloox.com. Contact [email protected] for information.

		In addition to populating the ad server key-value targeting, fetched segments (prefixed with `adl_...`) will also populate [First Party Data](https://docs.prebid.org/features/firstPartyData.html), some examples of the segments as described by the Adloox 'Google Publisher Tag Targeting Guidelines' and where they are placed are:

Adloox (adloox.com) RTD provider, analytic and adserver modules #6174

Adloox (adloox.com) RTD provider, analytic and adserver modules #6174

Conversation

jimdigriz commented Jan 10, 2021 • edited Loading

Type of change

Description of change

Other information

jimdigriz commented Jan 11, 2021 • edited Loading

aleksatr commented Jan 12, 2021

jimdigriz commented Jan 12, 2021 • edited Loading

bretg Jan 14, 2021

Choose a reason for hiding this comment

jimdigriz Jan 14, 2021 • edited Loading

Choose a reason for hiding this comment

bretg Jan 14, 2021

Choose a reason for hiding this comment

jimdigriz Jan 14, 2021 • edited Loading

Choose a reason for hiding this comment

bretg Jan 14, 2021

Choose a reason for hiding this comment

jimdigriz Jan 14, 2021 • edited Loading

Choose a reason for hiding this comment

bretg commented Jan 20, 2021

gglas commented Jan 20, 2021

bretg commented Jan 20, 2021

jimdigriz commented Jan 20, 2021

jimdigriz commented Feb 13, 2021

jimdigriz commented Jan 10, 2021 •

edited

Loading

jimdigriz commented Jan 11, 2021 •

edited

Loading

jimdigriz commented Jan 12, 2021 •

edited

Loading

jimdigriz Jan 14, 2021 •

edited

Loading

jimdigriz Jan 14, 2021 •

edited

Loading

jimdigriz Jan 14, 2021 •

edited

Loading