Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

API: View data in dataverse only works with authenticated user key even on public dv. Needs doc or change. #1809

Closed
kcondon opened this issue Mar 30, 2015 · 17 comments

Comments

@kcondon
Copy link
Contributor

kcondon commented Mar 30, 2015

The view data in dataverse works but only if you are an authenticated user and supply the key.
If the dataverse is unpublished, you need perms.

This is not mentioned in the doc but does indicate a key is needed on failure.

  1. Is this how we want it to work for published dvs?
  2. Probably want to add a note and update the URI in the doc:
    View data about the dataverse identified by $id. $id can be the id number of the dataverse, its alias, or the special value :root.

GET http://$SERVER/api/dataverses/$id

@michbarsinai
Copy link
Member

We had discuss this under the "should guest user have an API key" subject. The decision was that all APIs require API key an thus a registered user of some sort.

Technically, it would be very easy to just return guest user if no API key is provided, and take it from there.

@pdurbin
Copy link
Member

pdurbin commented Apr 1, 2015

Without an API key you can use the Search API, the Data Access API, the "meta" API (used by TwoRavens), and probably others, as I mentioned in a recent comment on a Google Doc.

For the Search API I'm using GuestUser until I get around to finishing #1299

@scolapasta
Copy link
Contributor

Discussed with @mcrosas and @kcondon and we think it makes sense that all public APIs keys should require API tokens, so we know who's calling them. Will discuss with @pdurbin and @landreev to see if there's any issues with this.

@pdurbin
Copy link
Member

pdurbin commented Apr 1, 2015

I'm sure requiring API keys/tokens will prevent potential collaborators from even trying out our APIs. They'll see "API token required. Please sign up." If they bother to try to sign up they'll see "give us your email address." Some will do this. Many will stop being interested in our APIs.

I prefer the GitHub model where they let you use their APIs without sign up but they rate-limit you. It's much friendlier to potential collaborators.

@scolapasta
Copy link
Contributor

For 4.0 we will block.

Opened issue #1838 for future consideration of opening.

@scolapasta
Copy link
Contributor

Passing to @pdurbin to add token logic to search.

@scolapasta scolapasta assigned pdurbin and unassigned scolapasta Apr 1, 2015
@pdurbin
Copy link
Member

pdurbin commented Apr 2, 2015

As @scolapasta and I discussed, since we're not supporting public search until #1299 we'll require an API token for the Search API but then execute the search as the guest user.

@pdurbin
Copy link
Member

pdurbin commented Apr 2, 2015

I just had another thought about the implications of requiring an API token for Saved Search. It will mean that my little proof of concept Android app at https://github.com/IQSS/dataverse-android will stop working because it (currently) doesn't do any authentication. My plan would be to have it make use of the new "retrieve your API token" endpoint developed in #1818.

But who cares about my little Android app. There's more.

From chatting with @landreev last night it came to my attention that we are considering also requiring an API token for downloading files. This adds friction for people who would like to download unrestricted published data. I'm reminded of the "Direct access to data via URL?" thread on our mailing list that began with this:

Hi, I'm about to publish a paper and I want to distribute a script to
reproduce the analysis along with the paper. In order to do this, I
need to provide the users with direct access to the data files.
However, as far as I can tell, all of my data are behind a 'terms of
use' wall that prevents direct download. Is there any way around this
restriction?

In the scenario where downloads required an API token, what would the script described above look like? Would it prompt you for your Dataverse username and password and then retrieve your API token, and then proceed to download the files? If the person trying to download the files with the script doesn't have a Dataverse account would it prompt you to create a local/builtin account? (Shibboleth accounts can only be created via a browser and likewise Shibboleth users can not retrieve their API token via the API.) Is this what we'll recommend to researchers who want to write scripts like the one described above? Even if their desire is to publish their files with the fewest restrictions possible?

/cc @mcrosas @scolapasta @landreev @michbarsinai

@mercecrosas
Copy link
Member

This issue about terms of use will not happen moving forward for datasets that have a CC0 and the data files are not restricted.

Since there are several use cases to consider, let's make a final decision next week on the tokens for public APIs with @scolapasta @landreev @michbarsinai @pdurbin @kcondon

(I do care about the Android app, but we also want to plan this well and safely! )

pdurbin added a commit that referenced this issue Apr 2, 2015
@pdurbin
Copy link
Member

pdurbin commented Apr 2, 2015

As of 8be10bb the Search API requires an API token. It checks for a valid token but executes the search as the guest user (non-guest searches are coming in #1299).

Passing to @landreev for any changes he may need to make to other API endpoints.

@pdurbin
Copy link
Member

pdurbin commented Apr 3, 2015

(I do care about the Android app, but we also want to plan this well and safely! )

Of course, as predicted above, it no longer works. I opened a ticket to add the ability to retrieve one's API token: IQSS/dataverse-android#1

@pdurbin
Copy link
Member

pdurbin commented Apr 3, 2015

@scolapasta and @mcrosas I believe your intention is that even unrestricted published files should not be able to be downloaded without using an API token. Is that correct?

I bring this up because I just pushed the latest code (e945eeb) to https://apitest.dataverse.org and I can still download files without any authentication:

pdurbin@tabby:~$ wget https://apitest.dataverse.org/api/access/datafile/12
--2015-04-03 10:38:53--  https://apitest.dataverse.org/api/access/datafile/12
Resolving apitest.dataverse.org (apitest.dataverse.org)... 140.247.115.118
Connecting to apitest.dataverse.org (apitest.dataverse.org)|140.247.115.118|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [image/png]
Saving to: ‘12’

    [ <=>                                   ] 8,361       --.-K/s   in 0.01s   

2015-04-03 10:38:54 (638 KB/s) - ‘12’ saved [8361]

pdurbin@tabby:~$ file 12
12: PNG image data, 141 x 78, 8-bit/color RGBA, non-interlaced
pdurbin@tabby:~$ 

Please note that you can also download preview images with no authentication at https://apitest.dataverse.org/api/access/fileCardImage/12 for example.

@mcrosas and @scolapasta is this what we want? I'm changing the status of this ticket to Critical.

@mercecrosas
Copy link
Member

To be discussed with @scolapasta @pdurbin @landreev on Monday (4/6)

pdurbin added a commit to IQSS/dataverse-apitester that referenced this issue Apr 4, 2015
@mercecrosas mercecrosas modified the milestones: Dataverse 4.0: Release Patch, In Review - 4.0 Apr 12, 2015
@mercecrosas
Copy link
Member

Moved to review for release patch

@mercecrosas mercecrosas modified the milestones: 4.0.1, Dataverse 4.0: Release Patch Apr 17, 2015
@mercecrosas
Copy link
Member

Left to solve: Download API needs to deal with Terms of Use (in cases that files are unrestricted) @landreev moved to 4.0.1

@mercecrosas
Copy link
Member

For 4.0.1 we'll block the API for these cases (wither for everybody, or everybody except data owners/curators ...)

@mercecrosas mercecrosas modified the milestones: In Design, Candidates for 4.0.1 May 8, 2015
@mercecrosas mercecrosas assigned scolapasta and unassigned landreev Jun 25, 2015
@mercecrosas mercecrosas modified the milestones: In Design, In Review Nov 30, 2015
@scolapasta scolapasta removed their assignment Jan 27, 2016
@scolapasta scolapasta removed this from the Not Assigned to a Release milestone Jan 28, 2016
@pdurbin pdurbin removed the zTriaged label Jun 30, 2017
@pdurbin pdurbin added User Role: API User Makes use of APIs and removed zPriority: High labels Jul 12, 2017
@mheppler
Copy link
Contributor

mheppler commented Mar 5, 2018

Reviewed this issue with @scolapasta who agrees that we can close it, and continue to track further development related to API's and token in the new issue he opened, Consider options for opening APIs without tokens #1838.

@mheppler mheppler closed this as completed Mar 5, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

7 participants