[Native] Slot targeting not sent to DFP with nativ position and banner position

[haroldpussat]

Type of issue

question

Description

Still testing native implementation, I'm facing an unsolved issue with combination of native position and legacy banner position on a same page. What I can see in a case I mix nativ position and legacy banner position is that slot targeting to DFP is not sent to DFP and in that case, there is no ad served to the page.
On my test page, I have several position

• pub_banniere_haute, pub_banniere_basse, pub_pave1, pub_pave2, pub_pave3, pub_banniere_medium
  All positions are running fine with Prebid.

Case 1 : One position setup with prebid : pub_banniere_medium is declared as native with mediaType and a size of [1,1] --> Results : Ok, targeting is done correctly and ad is delivered

Case 2 : 2 positions setup with prebid : pub_banniere_medium is declared as native with mediaType and a size of [1,1], pub_pave1 is declared as banner with mediaType and also size
--> Results : OK, targeting is done correctly and ads are delivered

Case 3 (PROD target): 6 positions setup with Prebid : pub_banniere_medium is declared as native with mediaType and a size of [1,1], all other positions are declared as banner with mediaType and also size
--> Results : KO Often, there is no ad delivered at all, even if prebid answered with valid bids.
When I received bid for native, it seems some of them are running fine and some other are not running correctly
For example : If I go on https://www.ouestfrance-auto.com/ and pick up a random car to get retargeted. I often get a bid from OuestFrance auto and the issue is raised
For example : If I go on https://www.homly-you.com/ to get retargeted, I often get a bid from Homly You and the native ad is diplayed correctly)

I have some clue about the behaviour, from my understanding the slot targeting is sent to DFP via params "prev_scp". For case 1 and case 2, this params seems correct :

For case 3, this params is not visible :

Even if I received prebid bid

Even if the googletag.slot seems to have a correct targeting. This is an output of the object slot for pub_banniere_medium (the native), after the pbjs.setTargetingForGPTAsync and before the googletag.refresh

I'm not able to get the rootcause of this issue (no error from DFP, or DFP console, or browser console), and this is blocking for us to go live. I don't know if it's an issue from Prebid or DFP or from my setup

I did test with several prebid version
1.16
1.18-pre
1.18-pre custom (with encodeURIComponent for native key-value only, to fix #2854 issue

Steps to reproduce

Prebid version 1.*
Chrome / Firefox / Opera : last version
Declared an ad position with mediaType : native : {} and a size of [1,1]
EnableSendAllBids : true
I also try to play with SuppressEmptyKeys (true | none)
Have a simple native mediaType for at least one position and other positions are legacy banner
The issue is raised more often with https://www.ouestfrance-auto.com/ native ad (I can see with them if needed, how they did setup their native ad)

Test page

All test pages are with Prebid 1.18-pre custom (including a encodeURIComponent for native value)
Case 1 (only native) :
http://www.precom-multimedia.com/interne/hpu/test_nativ5/index_only_native.html?dfptest=40&pbjs_debug=true

Case 2 (one native + 1 legacy banner) :
http://www.precom-multimedia.com/interne/hpu/test_nativ5/index_only_native_2_positions.html?dfptest=40&pbjs_debug=true

Case 3 (one native + 5 legacy banner) :
http://www.precom-multimedia.com/interne/hpu/test_nativ5/index_custom.html?dfptest=40&pbjs_debug=true

Expected results

Case 1 : adex ad for other positions. Native prebid ad for pub_banniere_medium
Case 2 : adex ad for other positions. Legacy banner prebid / adex ad for pub_pave1, Native prebid ad for pub_banniere_medium
Case 3 : adex ad for other positions. Legacy banner prebid / adex ad for pub_pave1, Native prebid ad for pub_banniere_medium

Actual results

Case 3 : After being retargeted, No ad delivered at all and slot targeting to DFP seems not sent (via prev_scp)

Platform details

Prebid 1.*

Other information

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[mkendall07]

@haroldpussat
I've not been able to reproduce the issue in scenario 3. A few thoughts:

1. Try turning on DFP debug by appending ?googfc to the end of the URL
2. There is a lot of output in console - can you clean that up?
3. The screenshot with the output of the google slot seems to suggest the issue is with DFP - since it's got all the keys set but it is not sent in the URL. Maybe you are hitting a key/value limit on the slot? Can you disable "send all bids" to see if that remedies the issue?
4. Are you saying that you are not seeing an add at all for scenario 3?

[haroldpussat]

Thanks for the feedback, I was OOO and I'm back now.
For 4) : Yes, I'm not seeing any ad at all with scenario 3.
I'll check for the other point and will try with sendAllBids -> false.

As mentionnent by Christian from Appnexus, maybe there is a limit reached with DFP (8192 characters per request on asynchronous render) : https://support.google.com/admanager/answer/1628457?hl=en

I'll let you know when it's done.

[haroldpussat]

1. The googfc doesn't raise any error on all the pages (except some warning due to missing listener, it does interact with ad delivery)
2. I clean log message
3. I create a test page based on case 3 (one native + 5 legacy banner) with sendAllBids --> false, you can check : http://www.precom-multimedia.com/interne/hpu/test_nativ5/index_notallsendbids.html?dfptest=40&pbjs_debug=true

On this enableSendAllBid->false, I can see ad (not all ad I want as I didn't create several lineItem ready for SendAllBid -> false). The requested URL to DFP has a length of 6603 characters

[mkendall07]

@haroldpussat
Thanks for the update. I think we are hitting DFP limits and that's causes the no ad to render condition. I will try and confirm this theory when I get a chance. I think the best recommendation for now is to not send all bids. I will review with the team if we can do anything to mitigate this issue as well in the future.

[mkendall07]

Just a quick update here. I was able to confirm that DFP will silently drop keywords when it hits the max URL length defined. Because max URL length for DFP is influenced by a number of factors (number of adUnits, Sizes etc) I cannot give an absolute max for keywords but in my test with a single adUnit it started to be a problem when my total character count (Key + value) was over 6,325.

Unfortunately we most likely cannot figure out at run time the max to use, so Prebid.js will have to figure out some sensible rules here to prevent this from happening.

[wqi1972]

We faced exactly same problem. Not like other bids, native bid has too many data to pass, based on my test, when there are more than 3 native bids returns inside one call, then the problem happens.
Can't really figure out a complete solution for it, since the limitation is inside dfp key/value, and technically the same problem happens for normal bids as well, say: inside one call, returns 1000 available bids, it will end up the same problem.

Workaround we use to mitigate the issue:
if we look into the key/value data passed to dfp, we can find the key value for each slots includes: data for all available bids, and data for winning bids, technically only data for winning bids are needed, and data for available bids are only for reporting purpose. i.e. from the example provided, it has:
hb_adid hb_adid_criteo hb_adid_appnexusAs hb_native_body hb_native_body_appne
we can see hb_adid == hb_adid_appnexusAs, and hb_native_body == hb_native_body_appne, and hb_adid_criteo is not really needed unless you want create dfp report to analysis it. From key/value size point of view, the part hb_native_body == hb_native_body_appne is the problem, since native fields data normally quite big, and it does not make sense to pass the same value twice in the key/value and I don't think anyone want to create dfp report to analysis beaten native bid's native details.
So I created following function:

  const NativeKeys = ['hb_native_cta_', 'hb_native_linkurl_', 'hb_native_icon_', 'hb_native_image_', 'hb_native_brand_', 'hb_native_body_', 'hb_native_title_'];
  function removeUnnecessaryNativeKeys(slots) {
    slots.forEach(function (slot) {
      if (slot) {
        var keys = slot.getTargetingKeys();
        keys.forEach( function(key) {
        	NativeKeys.forEach( function(nativeKey) {
        		if (key.indexOf(nativeKey) === 0) {
        			slot.clearTargeting(key);
        		} 
        	});
        });
      }
    });
  }

and call this function right before window.googletag.pubads().refresh()

With this change, it reduces size of dfp request with native bid about 40%, and I notice one ad call now can support up to about 5 native bids. I think it works for most scenarios, at least it covers most of scenarios for us.
Hope it helps.

[haroldpussat]

If I'm not mistaken the solution provided with removeUnnecessaryNativeKeys is nearly like SendAllBids : false ? With SendAllBids : false, only winning bid will be sent to DFP, even if you have more than one native actor, there is only on native key / value sent to DFP.
We already go on with SendAllBids : false on PROD and I'm afraid it is not sufficient to avoir DFP limit to be reached. FYI, with 2 natives positions + 5 IAB positions on a same page we reach the limit (and so no ad were delivered). That's why we operate a rollback today, by desactivate one of the native placement. I guess it depends on how long are the URL / Image / Icon URL length from partner and it is not only linked to mount of native answer.

[wqi1972]

Not 100% understand the issue you refer to. Inside http://www.precom-multimedia.com/interne/hpu/test_nativ5/index_notallsendbids.html?dfptest=40&pbjs_debug=true,
I see prev_scp: pos=banniere_haute&amznbid=2&amznp=2&id=eda5b12f-cb19-11e8-ae98-98f2b3e84d4c|pos=banniere_basse&amznbid=2&amznp=2&id=eda5b130-cb19-11e8-ae98-98f2b3e84d4c&hb_format=banner&hb_source=client&hb_size=970x90&hb_pb=0.35&hb_adid=1070b8db9a0324e&hb_bidder=rubicon|pos=pave1&amznbid=2&amznp=2&id=eda5b131-cb19-11e8-ae98-98f2b3e84d4c&hb_format=banner&hb_source=client&hb_size=300x600&hb_pb=0.37&hb_adid=1114b1292883a2a&hb_bidder=rubicon|pos=pave2&amznbid=2&amznp=2&id=eda5b132-cb19-11e8-ae98-98f2b3e84d4c&hb_format=banner&hb_source=client&hb_size=300x600&hb_pb=0.37&hb_adid=12398fa6d4f91ae&hb_bidder=rubicon|pos=pave3&amznbid=2&amznp=2&id=eda5b133-cb19-11e8-ae98-98f2b3e84d4c&hb_format=banner&hb_source=client&hb_size=300x250&hb_pb=0.20&hb_adid=13a4be3f09eef44&hb_bidder=rubicon|pos=banniere_medium&amznbid=2&amznp=2&id=eda5b134-cb19-11e8-ae98-98f2b3e84d4c, and

So it looks to me, the IAB bids beat native ones, and because you setup SendAllBids : false, so no native bid key/value passed. They look good to me, do I misunderstood something?

[mkendall07]

currently working on a solution to this issue by not sending the click_url data to DFP. More soon.

[robertrmartinez]

currently working on a solution to this issue by not sending the click_url data to DFP. More soon.

@naoto0822 For your reference

[naoto0822]

@robertrmartinez Thx!

[naoto0822]

@mkendall07 @robertrmartinez
Hi.
Was this issue solved by #3573 ?

[mkendall07]

yes #3573 solved this.