Search API: Support search without an API Token

[michbarsinai]

Search should be as if used by :guest.

[michbarsinai]

related: #1299

[michbarsinai]

Also related: #1838

[michbarsinai]

Implemented, ready for QA.

[pdurbin]

Implemented, ready for QA.

@michbarsinai what about code review?

[pdurbin]

To me this goes all the way back to #1809 (comment) and reverses a policy decision made by @mercecrosas for the out-of-the-box behavior of Dataverse.

Also, it seems like all the logic surrounding :SearchApiNonPublicAllowed has been circumvented. In #1299 is was decided that we're only supporting public search.

In short, there are a lot of policy changes in pull request #3908, a lot of implications to unpack. I am very supportive of work toward "Consider options for opening APIs without tokens" in #1838 but I feel that we need to make sure whatever changes happen are what we want as a project.

[pdurbin]

In standup I suggested that @djbrooke could ping @mercecrosas about the policy considerations. Also, I forgot to mention something I said to @michbarsinai which is that if we open up a lot of APIs (#1838) we should consider rate limiting the API (#1339) like GitHub and others. Otherwise, you're down at a low level blocking individual abusive IP addresses with iptables or whatever.

[djbrooke]

Talked with @mercecrosas and @scolapasta about this yesterday. We should go ahead with this change. If so many people are using our Search API that we have load issues or need to IP ban a few bad apples, we'll revisit or determine some other strategy to handle this.

[pdurbin]

@djbrooke that's fantastic news! In IQSS/dataverse-android#1 I've been talking about the need to add authentication to my Dataverse Android app but once the Search API is open, the Android app should "just work" again!

[pdurbin]

I'm planning on doing some code review of pull request #3908.

[pdurbin]

I closed pull request #3908 in favor of pull request #3940 that I just made. Mine preserves the documented behavior of the Search API that only public data is exposed, which we don't intend to work on until #1299.

[pdurbin]

I gave @sekmiller a brain dump of what's changing and what's staying the same.

Staying the same: the Search API still only returns published data. #1299 is about also returning unpublished data.

Changing: Dataverse 4.6.2 and lower will bark at you if you try to use the Search API without a token saying "Please provide a key query parameter (?key=XXX) or via the HTTP header X-Dataverse-key". As of pull request #3940 the new out of the box behavior is that you don't have to supply a token. If you want the old 4.6.2 behavior, you can set :SearchApiRequiresToken to true.

[pdurbin]

Changing: Dataverse 4.6.2 and lower will bark at you if you try to use the Search API without a token

@kcondon as we discussed, since my pull request missed the 4.7 train, I updated the docs in 0308476 to explain that Dataverse 4.7 and lower will bark at you about not supplying a token.

[pdurbin]

@kcondon found that :SearchApiRequiresToken=true wasn't restoring the 4.7 behavior. Fixed in 6fcd1ce.