pagebuilder prefers an overfull frame with worse badness over an underfull one

[zavislak]

SILE v0.12.5 (Lua 5.2)

\begin{document}
\script[src=packages/frametricks]
\script[src=packages/lorem]
\font[size=12pt]
\set[parameter=document.baselineskip,value=36pt]

\showframe
\lorem[words=500]

\end{document}

Running sile:

$ sile bug2.sil
SILE v0.12.5 (Lua 5.2)
<bug2.sil>

! Overfull frame: 16.288224285pt shrinkability required to fit but only 0pt available at in "lorem ipsum dolor …" near bug2.sil:8:1: in \loremtable: 0x55e1d17840e0
[1] [2]

Output:

The reason why this exceeds the content frame is because none of the other possible breaks meet the minimum leastC that would trigger setting bestBreak. As a result, the pagebuilder waits until it goes over the target before setting bestBreak to the worst break.

Note that a naive attempt at fixing it (keep updating bestBreak if it's badness is <= the best break so far) seems to break the current pagebuilder in another way.

[Omikhleia]

Over the course of my current project, I had noticed too, without such a detailed analysis, that overfull frames were apparently "preferred" to underfull ones, but I had not reported it (yet). Of course a small bit of overflow (vs. huge underfull pages) could be acceptable, so whatever the fix would be, perhaps it would also need some setting for fine-tuning (e.g. I have tables with pictures in one cell, overflowing vertically by a few pt only... I'd be ok with some of them as long as they do not overwrite the folio space / page bottom content ;)

[alerque]

I think there are at least two bugs here, but I'm having trouble putting my finger on either.

In the mean time I'll note that the main way of combating this problem is giving the typesetter some vertical flexibility. This can be leading that has some stretch/shrink (even 1pt across each line on a page can give it enough to fit or not fit one more line cleanly) or of course parskip or other places.

[applesud]

Hi! Thanks for making SILE :)

Sorry if I'm being silly—I'm not very familiar with the SILE codebase—but it looks like this is a problem in the rateBadness function?

Because there's no stretch, it decides that all of the breaking opportunities are infinitely bad: in other words, each opportunity is so bad that there's no point deciding between them, so it's okay to just choose the last one. I think the idea behind infinite badness is that in practise, there should almost never be a case where a break worse than inf_bad is actually the best one; however, that's not what's happening here: the best break is worse than inf_bad, and it isn't even a rare occurence—it happens frequently when working without shrinkable glue.

If we change rateBadness, so that it rates the badness as (I think) we would expect, it seems to behave sensibly?

utilities.rateBadness = function(inf_bad, shortfall, spring)
  local bad = math.floor(100 * math.abs(shortfall / (spring + 50) ^ 3)
  return math.min(inf_bad, bad)
end

(This function seemed more theoretically optimal, and worked for this mwe; it hasn't been properly investigated.)

Again, sorry if I'm missing something/alot, and/or stating the obvious :)