Selection parsing overhaul. New syntax features.

[mnmelo]

Following @orbeckst's suggestions in #345 that our parsing of selection strings be rewritten using pyparsing I did a bit of reading and some (very basic) testing. I think pyparsing can be implemented relatively easily.

This will allow an explosion of parsing capabilities, perhaps best suited for milestone 1.0. Setting this target would also allow this change to come at the same time that AtomGroups are planned to become structured arrays (#363), which again will open many more possibilities for easy selections.

Deploying earlier than that is also possible, although at a risk that much code will end up having to be rewritten once the new AtomGroups are in.

If we go down the pyparsing overhaul road I'd put other efforts to clean or extend the current implementation on hold.

I also think it's important we start chipping in what new aspects of the syntax we'd like to have.

I recently posted issues for two such enhancements: a global keyword, for better scope control when selecting from an AtomGroup(#268); and the implicit VMD-style OR'ing of search terms (#345).

Other features I'd like to have in MDAnalysis' selections (some, admittedly, from my happy experience with VMD):

1. arithmetic in comparisons; also, drop the requirement for 'prop': selectAtoms("x + y < 100");
2. full-fledged regular expressions in named searches: selectAtoms("name '.*H(1|2|3)' ");
3. periodic and non-periodic keywords to override whether selections are done with PBC in mind (right now, many selections simply ignore even MDAnalysis.core.flags['use_periodic_selections']).

Please chip in and add your own requests!
(Perhaps it's a good idea to ask in the lists what our devs and users think?)

Related issues #268, #345, #347, and #362 would be put on hold until this is done.

[richardjgowers]

I think once 0.11 is out the door, I'm ready to start working on the AtomGroup array, so we could do these two in parallel (and never mix "old" and "new" systems). I think before either can happen we need to have the desired functionality set in stone (in the form of tests).

[mnmelo]

We can start working in tandem then.

I agree that first setting out a set of tests that we know will fail (because the functionality isn't there yet) is the best way to guide our efforts.