Spec percent-scripts-timeout (WICG#1341)

* Reserved.once specification. * Some metrics stuff, but it's a bit off for re-run, so need to think some more. Actually implement the get. * Fix handwaveyness of fetch metrics. * Fix the re-run thing, and a bit of prep for later. * Fix formatting. * Fix bug + misc found in feedback. * Typo fix. * Wire timeout from most script helpers. * Start on actual computation. * k-anon only. The impl gets it wrong. * generateBid computation. * report* * And add the base value itself. * Fix participating IG count computation. * Back to always. * Fix the reportAdditionalWin case. * Improve report-win enum at suggestion. * Fix edit mishap * Apply feedback * Apply feedback * + PA contributions type * More feedback * More phrasing tweaks. * More add -> increment * Some feedback * Fix the error case Qingxin pointed out. * Explain 110% * Fix the parse error case Qingxin pointed out, lack of timeout on top-level run is orthogonal. * Fix the lack of timeout on ScriptExecution. * More feedback --------- Co-authored-by: Maks Orlovich <[email protected]>
qingxinwu · Nov 26, 2024 · 4a5cbfb · 4a5cbfb
1 parent 35bc3e7
commit 4a5cbfb
Showing 1 changed file with 89 additions and 29 deletions.
diff --git a/spec.bs b/spec.bs
@@ -2372,10 +2372,14 @@ or "component-auction", a [=currency tag=] |componentAuctionExpectedCurrency|, a
 1. [=Add a sample to an averager=] given |metrics|'s [=per participant metrics/code fetch time
   averager=] and |decisionLogicFetcher|'s [=script fetcher/fetch duration=].
 1. Let « |scoreAdResult|, |debugWinReportUrl|, |debugLossReportUrl|, |realTimeContributions|,
-  |paContributions| » be the result of [=evaluating a scoring script=] with |decisionLogicScript|,
-  |adMetadata|, |bidValue|'s [=bid with currency/value=], |auctionConfig|, |reportingContext|,
-  |sameOriginTrustedScoringSignals|, |crossOriginTrustedScoringSignals|, |browserSignals|,
-  |directFromSellerSignalsForSeller|, and |auctionConfig|'s [=auction config/seller timeout=].
+  |paContributions|, |executionMetrics| » be the result of [=evaluating a scoring script=] with
+  |decisionLogicScript|, |adMetadata|, |bidValue|'s [=bid with currency/value=], |auctionConfig|,
+  |reportingContext|, |sameOriginTrustedScoringSignals|, |crossOriginTrustedScoringSignals|,
+  |browserSignals|, |directFromSellerSignalsForSeller|, and |auctionConfig|'s [=auction config/
+  seller timeout=].
+1. Increment |metrics|'s [=per participant metrics/script executions attempted=] by 1.
+1. If |executionMetrics|'s [=execution metrics/script timed out=] is true, increment |metrics|'s
+  [=per participant metrics/script timeouts occurred=] by 1.
 1. If |generatedBid|'s [=generated bid/for k-anon auction=] is true:
 
   Note: Non-k-anonymous bids do not participate in reporting (except for platform real-time
@@ -2918,13 +2922,17 @@ and a [=global object=] |global|:
      |sellerReportingScriptFetcher|.
   1. [=Add a sample to an averager=] given |metrics|'s [=per participant metrics/code fetch time
     averager=] and |sellerReportingScriptFetcher|'s [=script fetcher/fetch duration=].
-  1. Let « |sellerSignals|, |reportUrl|, |reportingBeaconMap|, ignored, |paContributions| » be the
-    result of [=evaluating a reporting script=] with |sellerReportingScript|, "`reportResult`",
-    |reportingContext|, |config|'s [=auction config/seller=], |config|'s
+  1. Let « |sellerSignals|, |reportUrl|, |reportingBeaconMap|, ignored, |paContributions|,
+    |executionMetrics| » be the result of [=evaluating a reporting script=] with
+    |sellerReportingScript|, "`reportResult`",  |reportingContext|,
+    |config|'s [=auction config/seller=], |config|'s
     [=auction config/seller Private Aggregation coordinator=],  |config|'s [=auction config/
     config idl=]'s {{AuctionAdConfig/reportingTimeout}}, and
     « |config|'s [=auction config/config idl=], |browserSignals|, |directFromSellerSignals| ».
-   1. Let |reportingResult| be a [=reporting result=] with the following [=struct/items=]:
+  1. Set |metrics|'s [=per participant metrics/script executions attempted=] to 1.
+  1. If |executionMetrics|'s [=execution metrics/script timed out=] is true, set |metrics|'s
+    [=per participant metrics/script timeouts occurred=] to 1.
+  1. Let |reportingResult| be a [=reporting result=] with the following [=struct/items=]:
      : [=reporting result/report url=]
      :: |reportUrl|
      : [=reporting result/reporting beacon map=]
@@ -3021,14 +3029,18 @@ a {{ReportingBrowserSignals}} |browserSignals|, a [=direct from seller signals=]
   1. Let |reportFunctionName| be "`reportWin`".
   1. If |winner|'s [=generated bid/provided as additional bid=] is true:
     1. Set |reportFunctionName| be "`reportAdditionalBidWin`".
-  1. Let « ignored, |resultUrl|, |reportingBeaconMap|, |reportingMacroMap|, |paContributions| » be
-    the result of [=evaluating a reporting script=] with |buyerReportingScript|,
-    |reportFunctionName|, |reportingContext|, |ig|'s [=interest group/owner=], |ig|'s [=interest
-    group/Private Aggregation coordinator=], |leadingBidInfo|'s [=leading bid info/auction config=]'s
+  1. Let « ignored, |resultUrl|, |reportingBeaconMap|, |reportingMacroMap|, |paContributions|,
+    |executionMetrics| » be the result of [=evaluating a reporting script=] with
+    |buyerReportingScript|, |reportFunctionName|, |reportingContext|, |ig|'s
+    [=interest group/owner=], |ig|'s [=interest group/
+    Private Aggregation coordinator=], |leadingBidInfo|'s [=leading bid info/auction config=]'s
     [=auction config/config idl=]'s {{AuctionAdConfig/reportingTimeout}}, and
     « |leadingBidInfo|'s [=leading bid info/auction config=]'s [=auction config/config idl=]'s
       {{AuctionAdConfig/auctionSignals}}, |perBuyerSignalsForBuyer|, |sellerSignals|,
       |reportWinBrowserSignals|, |directFromSellerSignals| ».
+  1. Set |metrics|'s [=per participant metrics/script executions attempted=] to 1.
+  1. If |executionMetrics|'s [=execution metrics/script timed out=] is true, set |metrics|'s
+    [=per participant metrics/script timeouts occurred=] to 1.
   1. [=Commit private aggregation contributions=] given |paContributions|, |winner|'s [=generated
     bid/reporting id=] and |reportingContext|.
   1. Set |leadingBidInfo|'s [=leading bid info/buyer reporting result=] to a [=reporting result=]
@@ -3755,6 +3767,10 @@ A <dfn>reporting context map</dfn> is a [=map=] from [=auction config=] to [=rep
     |ig|'s [=interest group/owner=], [=worklet function/generate-bid=].
   1. [=Merge samples to an averager=] given |metrics|'s [=per participant metrics/code fetch time
     averager=] and |executionMetrics|'s [=execution metrics/code fetch time averager=].
+  1. Set |metrics|'s [=per participant metrics/script executions attempted=] to
+    |metrics|'s [=per participant metrics/participating interest group count=].
+  1. If |executionMetrics|'s [=execution metrics/script timed out=] is true, increment |metrics|'s
+    [=per participant metrics/script timeouts occurred=] by 1.
   1. [=set/Insert=] |id| into |reportingContext|'s [=reporting context/bidder participants=].
   1. [=Commit private aggregation contributions=] given |paContributions|, |id| and
     |reportingContext|.
@@ -4189,6 +4205,9 @@ A signal base value is one of the following:
   auction, after considering prioritization and capabilities. Interest groups included in this might
   not actually get to bid if the cumulative timeout expires, or the script fails to load, etc; or
   might decide not to bid, but they would have gotten a chance if nothing went wrong.
+: "<dfn><code>percent-scripts-timeout</code></dfn>"
+:: The numeric value is percentage of executions of this script that hit their individual timeout,
+  out of all executions that were expected to happen.
 
 </dl>
 
@@ -4277,6 +4296,9 @@ metrics collected from a single execution of a worklet function:
 <dl dfn-for="execution metrics">
   : <dfn>code fetch time averager</dfn>
   :: An [=averager=].
+  : <dfn>script timed out</dfn>
+  :: A [=boolean=], initially false. This refers to the script hitting its own time out, and does
+    not include the cumulative timeout affecting buyers.
 </dl>
 
 A <dfn>per participant metrics</dfn> is a [=struct=] with the following [=struct/items=],
@@ -4286,6 +4308,10 @@ representing metrics aggregated over a particular participant (e.g. bidder or se
   :: A {{long}}, initially 0.
   : <dfn>code fetch time averager</dfn>
   :: An [=averager=].
+  : <dfn>script timeouts occurred</dfn>
+  :: A {{long}}, initially 0.
+  : <dfn>script executions attempted</dfn>
+  :: A {{long}}, initially 0.
 </dl>
 
 <h4 dfn-type=dfn>Private Aggregation contributions</h4>
@@ -4647,6 +4673,26 @@ They return a {{double}}.
     participant metrics/code fetch time averager=].
 1. If |signalBaseValue| is "<code>[=signal base value/participating-ig-count=]</code>":
   1. Return |bidAndScoreMetrics|'s [=per participant metrics/participating interest group count=].
+1. If |signalBaseValue| is "<code>[=signal base value/percent-scripts-timeout=]</code>":
+  1. Return the result of [=computing a percentage metric=] given |metrics|'s [=per participant
+    metrics/script timeouts occurred=] and |metrics|'s [=per participant metrics/script executions
+    attempted=].
+
+</div>
+
+<div algorithm>
+To <dfn>compute a percentage metric</dfn> given {{long}}s |numerator| and |denominator|:
+1. If |denominator| is 0, return 0.
+1. Let |result| be 100.0 * |numerator| / |denominator|, performing the computation with
+  {{double}}s.
+1. If |result| &gt; 110.0, set |result| to 110.0
+
+  Note: Since [=perform storage maintenance|Interest Group Storage Maintenance=] happens
+  periodically, metrics measuring storage utilization <span class="allow-2119">may</span> report
+  values that exceed 100%. The 110% cap exists to make it easier to allocate histogram bucket space
+  for those metrics, by providing a predictable upper bound.
+
+1. Return |result|.
 
 </div>
 
@@ -5596,7 +5642,8 @@ of the following global objects:
     1. Let |crossOriginTrustedBiddingSignalsJS| be |crossOriginTrustedBiddingSignals|
        [=converted to ECMAScript values=].
     1. Let |startTime| be |settings|'s [=environment settings object/current monotonic time=].
-    1. Let |result| be the result of [=evaluating a script=] with |realm|, |script|, "`generateBid`",
+    1. Let (|result|, |executionMetrics|'s [=execution metrics/script timed out=]) be the
+      result of [=evaluating a script=] with |realm|, |script|, "`generateBid`",
       « |igJS|, |auctionSignalsJS|, |perBuyerSignalsJS|, |sameOriginTrustedBiddingSignalsJS|,
         |browserSignalsJS|, |directFromSellerSignalsJS|, |crossOriginTrustedBiddingSignalsJS| »,
         and |timeout|.
@@ -5679,7 +5726,9 @@ of the following global objects:
     1. Let |directFromSellerSignalsJs| be |directFromSellerSignalsForSeller|
       [=converted to ECMAScript values=].
     1. Let |startTime| be |settings|'s [=environment settings object/current monotonic time=].
-    1. Let |scoreAdResult| be the result of [=evaluating a script=] with |realm|, |script|, "`scoreAd`",
+    1. Let |executionMetrics| be a new [=execution metrics=].
+    1. Let (|scoreAdResult|, |executionMetrics|'s [=execution metrics/script timed out=]) be
+      the result of [=evaluating a script=] with |realm|, |script|, "`scoreAd`",
       «|adMetadata|, |bidValue|, |auctionConfigJS|, |sameOriginTrustedScoringSignalsJS|,
       |browserSignalsJS|, |directFromSellerSignalsJs|, |crossOriginTrustedScoringSignalsJS|»,
       and |timeout|.
@@ -5701,7 +5750,7 @@ of the following global objects:
     1. Let |paContributions| be the result of [=extracting private aggregation contributions=] given
       |global|.
     1. Return « |scoreAdResult|, |debugWinReportUrl|, |debugLossReportUrl|, |realTimeContributions|,
-        |paContributions| ».
+        |paContributions|, |executionMetrics| ».
 </div>
 
 <div algorithm>
@@ -5719,9 +5768,12 @@ of the following global objects:
       and |privateAggregationCoordinator|.
     1. Let |argumentsJS| be the result of [=converting a Web IDL arguments list to an ECMAScript
       arguments list|converting=] |arguments| to an ECMAScript arguments list. If this
-      [=exception/throws=] an exception, return « "null", null, null, null ».
-    1. Let |result| be the result of [=evaluating a script=] with |realm|, |script|,
-      |functionName|, |argumentsJS|, and |timeout|.
+      [=exception/throws=] an exception, return « "null", null, null, null, a new [=Private
+      Aggregation Contributions=], |executionMetrics| ».
+    1. Let |executionMetrics| be a new [=execution metrics=].
+    1. Let (|result|, |executionMetrics|'s [=execution metrics/script timed out=]) be the
+      result of [=evaluating a script=] with |realm|, |script|, |functionName|, |argumentsJS|,
+      and |timeout|.
     1. If |result| is an [=ECMAScript/abrupt completion=], return « "null", null, null, null ».
     1. Let |resultJSON| be "null".
     1. If |functionName| is "`reportResult`", then set |resultJSON| to the result of
@@ -5738,20 +5790,23 @@ of the following global objects:
       |global|.
     1. Return « |resultJSON|, |reportURL|,
       |global|'s [=InterestGroupReportingScriptRunnerGlobalScope/reporting beacon map=], |macroMap|,
-      |paContributions| ».
+      |paContributions|, |executionMetrics| ».
 </div>
 
 <div algorithm>
   To <dfn>evaluate a script</dfn> with a [=ECMAScript/realm=] |realm|, [=string=] |script|, [=string=]
   |functionName|, a [=list=] |arguments|, and an integer millisecond [=duration=] |timeout|, run these steps.
-  They return a [=ECMAScript/Completion Record=], which is either an [=ECMAScript/abrupt completion=] (in
-  the case of a parse failure or execution error), or a [=ECMAScript/normal completion=] populated with the
-  [=ECMAScript/ECMAScript language value=] result of invoking |functionName|.
+  They return a tuple of a [=ECMAScript/Completion Record=], which is either an [=ECMAScript/abrupt
+  completion=] (in the case of a parse failure or execution error), or a [=ECMAScript/normal
+  completion=] populated with the [=ECMAScript/ECMAScript language value=] result of invoking
+  |functionName|, and a [=boolean=] stating whether the script was interrupted due to reaching
+  |timeout|.
 
     1. [=Assert=] that these steps are running [=in parallel=].
 
-    1. If |timeout| &le; 0, [=immediately=] interrupt the execution and set |finalCompletion| to a
-      new [=ECMAScript/throw completion=] given null.
+    1. If |timeout| &le; 0, return (new [=ECMAScript/throw completion=] given null, true).
+
+    1. Let |timedOut| be false.
 
     1. Let |global| be |realm|'s [=realm/global object=], and run these steps in |realm|'s [=realm/agent=]:
 
@@ -5761,7 +5816,7 @@ of the following global objects:
        unlike traditional [=scripts=] on the web platform.
 
     1. If |result| is a list of errors, return
-       Completion { \[[Type]]: `throw`, \[[Value]]: |result|, \[[Target]]: `empty` }.
+       (Completion { \[[Type]]: `throw`, \[[Value]]: |result|, \[[Target]]: `empty` }, false).
 
     1. [=Assert=]: |result| is a [=ECMAScript/Script Record=].
 
@@ -5771,6 +5826,11 @@ of the following global objects:
 
     1. Let |evaluationStatus| be the result of [$ScriptEvaluation$](result).
 
+      In |timeout| milliseconds, if the invocation of [$ScriptEvaluation$] has not completed,
+      [=immediately=] interrupt the execution, set |finalCompletion| to a new
+      [=ECMAScript/throw completion=] given null, set |timedOut| to true, and jump to the step
+      labeled <i><a href="#evaluate-script-return">return</a></i>.
+
     1. If |evaluationStatus| is an [=ECMAScript/abrupt completion=], jump to the step labeled <i>
        <a href="#evaluate-script-return">return</a></i>.
 
@@ -5781,9 +5841,9 @@ of the following global objects:
     1. Set |finalCompletion| be [=ECMAScript/Completion Record|Completion=]([$Call$](F, `undefined`,
        |arguments|)).
 
-       In |timeout| milliseconds, if the invocation of [$Call$] has not completed,
-       [=immediately=] interrupt the execution and set |finalCompletion| to a new
-       [=ECMAScript/throw completion=] given null.
+       In |timeout| milliseconds minus the execution time of [$ScriptEvaluation$], if the invocation
+       of [$Call$] has not completed, [=immediately=] interrupt the execution, set |finalCompletion|
+       to a new [=ECMAScript/throw completion=] given null, and set |timedOut| to true.
 
     1. <i id=evaluate-script-return>Return</i>: at this point |finalCompletion| will be set to a
        [=ECMAScript/Completion Record=].
@@ -5792,7 +5852,7 @@ of the following global objects:
          execution context|running JavaScript execution context=], and remove it from the
          [=ECMAScript/execution context stack|JavaScript execution context stack=].
 
-      1. Return |finalCompletion|.
+      1. Return (|finalCompletion|, timedOut).
 </div>
 
 ## Global scopes ## {#global-scopes}