[pex] Memoize calls to Crawler.crawl() for performance win in find-links based resolution.

[kwlzn]

While investigating a user-reported performance issue in the creation of pex files (via pants), profiling revealed around ~60% of the total time being spent in pex's Crawler.crawl() function with over 47 calls. Note that calls to Crawler.crawl() involve use of the re module to match html tags from a given index page (in our case, with roughly ~5000+ files) - with excessive application of the re module being a known culprit for slowness.

I was able to approximately repro the same scenario directly with pex:

pex --disable-cache --no-pypi -f http://$URL -f http://$URL/dist \
    pex psutil requests ansible jsonschema -o /tmp/throwaway

..and further inspection here revealed upwards of 24 non-cached Crawler.crawl() calls just for these 5 dependencies.

This PR memoizes calls to Crawler.crawl() to save overhead on subsequent calls for a ~2x/200% speedup in the above test case:

before

real    0m19.943s
user    0m5.437s
sys     0m2.427s

after

real    0m8.288s
user    0m3.420s
sys     0m1.986s

[kwlzn]

merged @ fcdee8a