Add edit tree lemmatizer

[adrianeboyd]

Description

Add edit tree lemmatizer, converted from spacy_experimental.edit_tree_lemmatizer

Types of change

Enhancement

Checklist

• I confirm that I have the right to submit this contribution under the project's MIT license.
• I ran the tests, and all new and existing tests passed.
• My changes don't require a change to the documentation, or if they do, I've added all required information.

[adrianeboyd]

@danieldk: In particular, can you double-check all the types?

[adrianeboyd]

I'm not sure that spacy.pipelines was the best place for edit_trees, but there wasn't an immediately obvious better place to put it.

I'm also not sure that edit_tree_lemmatizer is the name we should go with in the end, since it's kind of opaque/jargon-y for users (I can't think of the actual name for this ... where the name is from the developers' perspective rather than the users').

My original intention was to have this kind of component as a mode of lemmatizer, which has tried to follow the TrainablePipe API enough to make it possible in the future (mainly having a model parameter), but now I think it will be an overall API headache to mix trainable and not-trainable modes.

[danieldk]

I'm also not sure that edit_tree_lemmatizer is the name we should go with in the end, since it's kind of opaque/jargon-y for users (I can't think of the actual name for this ... where the name is from the developers' perspective rather than the users').

Maybe TrainableLemmatizer/trainable_lemmatizer?

[adrianeboyd]

So what I mean is the default component name / factory name and not the class name, where I think that EditTreeLemmatizer is fine.

Nothing else is trainable_x and "trainable" is expressed in other ways. But maybe there aren't any better alternatives...

[svlandeg]

Really excited to have this as part of the main library! During review, I mainly had some small comments about typing.

[svlandeg]

I'm not sure that spacy.pipelines was the best place for edit_trees, but there wasn't an immediately obvious better place to put it.

I was thinking that too. But having spacy.pipeline._edittree_internals like we have for the parser is probably overkill.

For the entity_linker, we did end up deciding having kb.pyx and kb.pxd in the root spacy module though.

[adrianeboyd]

Maybe _internals and we move the parser stuff there at some point, too?

[svlandeg]

Nothing else is trainable_x and "trainable" is expressed in other ways. But maybe there aren't any better alternatives...

trainable_x does make the distinction more clear to users, I think, but I don't like it for the same reason you mention. Alternatively: neural_lemmatizer. Though it has the same issues, at least it's shorter :|

[svlandeg]

Maybe _internals and we move the parser stuff there at some point, too?

I don't think that'll work, because the internal parser classes are so genericly named that then it might become more difficult to see what all the parser code is...

[danieldk]

Tested German with the validation/serialization changes, there doesn't seem to be a regression.

[svlandeg]

Looks like this is ready to merge?

We can revise the cast thing later, it'd require an update of Thinc and I don't want to hold up this PR over something like that.

[adrianeboyd]

No, not yet. I would wait a bit until the potential integrations with Lemmatizer have been explored a bit more.

[svlandeg]

Oh, ok. I thought we didn't want to go there initially, but I'll leave that up to you Adriane. I'll put this in draft to signal that we shouldn't merge yet!

[adrianeboyd]

We decided that an integration with Lemmatizer or the lemmatizer factory is not a particular useful choice. It's already supported to do something like this:

nlp.add_pipe("trainable_lemmatizer", name="lemmatizer")

[adrianeboyd]

Now this needs docs...

[danieldk]

Now this needs docs...

Added, hope I didn't miss anything.