Inter-annotator agreement for span layer independently of features

[bgittel]

I'm looking for a way to calculate agreement for the spans of a span layer independently of the features attached to it. In the drop-down menu to select the item for agreement calculation, there are only features of certain layers (I'm using v. 34.2).

So it would be really nice to have the possibility to calculate agreement for the spans alone to see how good annotators are at identifying a certain phenomenon before classifying it further. Ideally, the agreement calculation would give you an option to count spans with large overlap (in contrast to identical spans) as agreement.

[reckart]

@bgittel looks like this can be done with the Krippendorff Alpha Unitizing measure. See screenshots here: #5192