speed up step_boundaries

[stevengj]

step_boundaries copies all of the chunk's boundary points into a buffer, sends them via MPI (if the destination is a different process), and then copies the received buffer to the destination boundary. This code was all written many years ago under the assumption that the computational cost was not significant (since it is proportional to the surface area rather than the volume). However, in practice it appears to be a substantial portion of the time in some cases.

Fortunately, the fact that it was never seriously optimized also means that there are probably many ways to speed it up. Here are three ideas:

1. The copy to/from buffer implementation uses an array of pointers to the source/destination locations, which means that there is a lot of pointer chasing in the inner loops. One simple optimization would be to (a) sort the pointers by address and (b) replace arithmetic sequences (sequences of n addresses with constant offsets s) by a (pointer,s,n) triplet so that the sequence can be read using a simple loop.

2. Similarly, the phase factors are stored and computed individually per boundary point here, when in fact many of these phases are the same, so they could be compressed with run-length encoding.

3. When the source and destination process are the same (e.g. copying from a chunk to a neighboring PML chunk on the same process), we can presumably skip the buffer and copy directly to the destination.

[stevengj]

Would be good to have some profiling data to know what fraction of the time is typically spent on copying to/from these buffers, to know if it is worth optimizing further.