faster hnsw CPU index training

[alexanderguzhva]

This change decreases the training time for 1M x 768 dataset down to 10 minutes from 13 minutes in our experiments.

Please verify and benchmark.

[facebook-github-bot]

@kuarora has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[mengdilin]

I microbenchmarked IndexHNSWFlat::add on google benchmark on this change and compared it against the master commit. The results are looking good (we are seeing 7-11% CPU time improvement for the microbenchmarks) with a reasonably small p-value from the U tests.

With parameters:
d=64, n=1000, M=32, threads=1, we get a 7% CPU time improvement on average (P1567300587)
d=128, n=1000, M=16, threads=1, we get a 7% CPU time improvement on average (P1567316788)
d=128, n=1000, M=32, threads=5, we get a 11% CPU time improvement on average(P1567363053)
d=64, n=2000, M=16, threads=5, we get a 10% CPU time improvement on average (P1567491190)
d=128, n=1000, M=16, threads=5, we get a 5% CPU time improvement on average (P1567371095) but this result is not statistically significant enough to be counted

I think we can verify actual impact on production with @mnorris11's work on observability for the internal customers

@kuarora and @mdouze: are there other parameters we would want to try out here?

[mdouze]

@alexanderguzhva you are usually leaving the "simple" version of the code in comments, which is better than nothing.

Would you mind instead using a local boolean variable reference_code (or something) which is set to false and do

bool reference_code = false; 

if(reference_code) {
  ... The old short code 
}  else {
   ... Your optimized long code 
}

In this way the compiler will compile the old code but optimize it away.
The reason is that (1) it should be easy to switch back to the ref code and (2) if the interface changes somehow, we don't want the two branches to diverge

[alexanderguzhva]

@mdouze done

[facebook-github-bot]

@kuarora has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[facebook-github-bot]

@kuarora merged this pull request in e261725.