rustdoc-search: use set ops for ranking and filtering

[notriddle]

This commit adds ranking and quick filtering to type-based search, improving performance and having it order results based on their type signatures.

Preview

Profiler output: https://notriddle.com/rustdoc-html-demo-6/profile-8/index.html

Preview: https://notriddle.com/rustdoc-html-demo-6/ranking-and-filtering-v2/std/index.html

Motivation

If I write a query like str -> String, a lot of functions come up. That's to be expected, but String::from should come up on top, and it doesn't right now. This is because the sorting algorithm is based on the functions name, and doesn't consider the type signature at all. slice::join even comes up above it!

To fix this, the sorting should take into account the function's signature, and the closer match should come up on top.

Guide-level description

When searching by type signature, types with a "closer" match will show up above types that match less precisely.

Reference-level explanation

Functions signature search works in three major phases:

• A compact "fingerprint," based on the bloom filter technique, is used to check for matches and to estimate the distance. It sometimes has false positive matches, but it also operates on 128 bit contiguous memory and requires no backtracking, so it performs a lot better than real unification.

  The fingerprint represents the set of items in the type signature, but it does not represent nesting, and it ignores when the same item appears more than once.

  The result is rejected if any query bits are absent in the function, or if the distance is higher than the current maximum and 200 results have already been found.

• The second step performs unification. This is where nesting and true bag semantics are taken into account, and it has no false positives. It uses a recursive, backtracking algorithm.

  The result is rejected if any query elements are absent in the function.

Drawbacks

This makes the code bigger.

More than that, this design is a subtle trade-off. It makes the cases I've tested against measurably faster, but it's not clear how well this extends to other crates with potentially more functions and fewer types.

The more complex things get, the more important it is to gather a good set of data to test with (this is arguably more important than the actual benchmarking ifrastructure right now).

Rationale and alternatives

Throwing a bloom filter in front makes it faster.

More than that, it tries to take a tactic where the system can not only check for potential matches, but also gets an accurate distance function without needing to do unification. That way it can skip unification even on items that have the needed elems, as long as they have more items than the currently found maximum.

If I didn't want to be able to cheaply do set operations on the fingerprint, a cuckoo filter is supposed to have better performance. But the nice bit-banging set intersection doesn't work AFAIK.

I also looked into minhashing, but since it's actually an unbiased estimate of the similarity coefficient, I'm not sure how it could be used to skip unification (I wouldn't know if the estimate was too low or too high).

This function actually uses the number of distinct items as its "distance function." This should give the same results that it would have gotten from a Jaccard Distance $1-\frac{|F\cap{}Q|}{|F\cup{}Q|}$, while being cheaper to compute. This is because:

• The function $F$ must be a superset of the query $Q$, so their union is just $F$ and the intersection is $Q$ and it can be reduced to $1-\frac{|Q|}{|F|}.

• There are no magic thresholds. These values are only being used to compare against each other while sorting (and, if 200 results are found, to compare with the maximum match). This means we only care if one value is bigger than the other, not what it's actual value is, and since $Q$ is the same for everything, it can be safely left out, reducing the formula to $1-\frac{1}{|F|} = \frac{|F|}{|F|}-\frac{1}{|F|} = |F|-1$. And, since the values are only being compared with each other, $|F|$ is fine.

Prior art

This is significantly different from how Hoogle does it.
It doesn't account for order, and it has no special account for nesting, though Box<t> is still two items, while t is only one.

This should give the same results that it would have gotten from a Jaccard Distance $1-\frac{|A\cap{}B|}{|A\cup{}B|}$, while being cheaper to compute.

Unresolved questions

[] and (), the slice/array and tuple/union operators, are ignored while building the signature for the query. This is because they match more than one thing, making them ambiguous. Unfortunately, this also makes them a performance cliff. Is this likely to be a problem?

Right now, the system just stashes the type distance into the same field that levenshtein distance normally goes in. This means exact query matches show up on top (for example, if you have a function like fn nothing(a: Nothing, b: i32), then searching for nothing will show it on top even if there's another function with fn bar(x: Nothing) that's technically a closer match in type signature.

Future possibilities

It should be possible to adopt more sorting criteria to act as a tie breaker, which could be determined during unification.

[rustbot]

r? @fmease

(rustbot has picked a reviewer for you, use r? to override)

[rustbot]

Some changes occurred in HTML/CSS/JS.

cc @GuillaumeGomez, @jsha

[notriddle]

r? @GuillaumeGomez

[GuillaumeGomez]

Overall, I have no issue with this change. But for future optimizations which require to increase code size, I'd really like us to have a wide enough testsuite to ensure that it doesn't impact negatively other cases. Considering this, I'm wondering if we shouldn't start by writing these performance checks (even if not run in rustc's CI) before landing any new performance improvement PR. What do you think?

[notriddle]

That does seem like a good idea, yeah.

Any suggestions about where to start?

[notriddle]

I've added a couple more benchmarks:

• stm32f4
• ripgrep

I've also added more queries to the arti benchmark, and improved the bloom filter hashing (it gets a lot fewer false positives now, and the many nearly identical queries now all run in similar time).

[notriddle]

@bors r=GuillaumeGomez

[bors]

📌 Commit 34b7265 has been approved by GuillaumeGomez

It is now in the queue for this repository.

[bors]

☔ The latest upstream changes (presumably #118900) made this pull request unmergeable. Please resolve the merge conflicts.

[notriddle]

@bors r=GuillaumeGomez

Rebased with the variable changes.

[bors]

📌 Commit bec6672 has been approved by GuillaumeGomez

It is now in the queue for this repository.

[bors]

⌛ Testing commit bec6672 with merge eeff92a...

[bors]

☀️ Test successful - checks-actions
Approved by: GuillaumeGomez
Pushing eeff92a to master...

[rust-timer]

Finished benchmarking commit (eeff92a): comparison URL.

Overall result: no relevant changes - no action needed

@rustbot label: -perf-regression

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	2.4%	[0.7%, 4.4%]	4
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	2.4%	[0.7%, 4.4%]	4

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean	range	count
Regressions ❌ (primary)	0.8%	[0.8%, 0.8%]	1
Regressions ❌ (secondary)	3.3%	[2.2%, 4.1%]	3
Improvements ✅ (primary)	-0.5%	[-0.5%, -0.5%]	1
Improvements ✅ (secondary)	-2.0%	[-2.0%, -2.0%]	1
All ❌✅ (primary)	0.2%	[-0.5%, 0.8%]	2

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 671.752s -> 671.988s (0.04%)
Artifact size: 312.39 MiB -> 312.40 MiB (0.00%)