persist should be lattice typed, e.g. to enforce uniqueness of sets

[jhellerstein]

Currently to persist and remove dups you'd have to write:

source... -> unique() -> persist()

That pays for storing the state twice.

What we'd really like is

source...->persist<SetUnion<T>>()

or maybe to support deletion we'd like:

source...->persist<MapUnionWithTombstones<HashMap<K, SetUnionHashSet<V>>, HashSet<K>>>

[MingweiSamuel]

Use lattice_fold/lattice_reduce with flatten after (bit clunky) or, in the future, state

[MingweiSamuel]

state is the new persist #1058