Cannot use Portus REST API to create or update

[wolfch]

I am working an a Python script to be able to clone a V2 registry with as little manual intervention as possible. If I look at the Rails Routes via /rails/info/routes, it appears that this should be possible. I have found all the GET APIs work, but upon trying a POST API, I get a ActionController::InvalidAuthenticityToken exception with this stack trace:

actionpack (4.2.2) lib/action_controller/metal/request_forgery_protection.rb:181:in `handle_unverified_request'
actionpack (4.2.2) lib/action_controller/metal/request_forgery_protection.rb:209:in `handle_unverified_request'
devise (3.5.1) lib/devise/controllers/helpers.rb:251:in `handle_unverified_request'
actionpack (4.2.2) lib/action_controller/metal/request_forgery_protection.rb:204:in `verify_authenticity_token'
activesupport (4.2.2) lib/active_support/callbacks.rb:432:in `block in make_lambda'
activesupport (4.2.2) lib/active_support/callbacks.rb:164:in `call'

Then a did a "view source" of the "create team" page and, sure enough, I see the HTML tags:

    <meta name="csrf-param" content="authenticity_token"/>
    <meta name="csrf-token" content="2vvKRkLFqq13cp2SIx7oTyakKt6tGpdoHKNXlvcAO1+145O/Qmk5J2OU5J/yr+zMsQnF7k4nW1f/D44to5WS0g=="/>

...which my Python API code will have no knowledge of. In other words, the create/update APIs only seem to support Ajax from HTML Forms, rather then general RESTful API access, unless I'm missing something such as session cookies, etc.

Upon searching around, I find there should be a way to conditionally turn CSRF on/off based on whether the API is called from a form or as a service:
http://stackoverflow.com/questions/16258911/rails-4-authenticity-token

In fact, the RoR docs make this suggestion:
http://api.rubyonrails.org/classes/ActionController/RequestForgeryProtection.html

So I modified Portus/app/controllers/application_controller.rb to conform to the above suggestion, but I still got InvalidAuthenticityToken... :(

Here is the fragment of code in my Python REST client that tries to create a team in Portus:

    def create_team(self, name, hidden, description):
        teams = [{"name": name, "hidden": hidden, "description": description}]
        response = requests.post(self.urlbase + '/teams.json', json=teams,
            headers={'Content-Type':'application/json', 'Accept': 'application/json'},
            auth=(self.username, self.password), verify=False)
        if response.status_code is not 200:
            raise APIException(response.text, response.status_code)
        data = response.json()
        self.log.debug(data)

Notice that my resource path is appended with .json, which I thought would inhibit the CSRF mechanism with my code change from the RoR docs:
http://api.rubyonrails.org/classes/ActionController/RequestForgeryProtection.html

Also I noticed that there is no user CRUD API.

In summary - how can we use the REST API from a script to create/modify teams, namespaces, etc.?

[wolfch]

I changed my REST client code to first load the form page, e.g. /teams, then parse out the csrf-token from the page and send it via the X-CSRF-Token header upon posting to /teams.json and I still get InvalidAuthenticityToken...

[wolfch]

Will there ever be support for RESTful changes to the registry? i.e. add/delete/modify?

[mssola]

Hi @wolfch. When designing/implementing Portus we never had in mind opening a REST API to control it. Honestly, it didn't even occur to me 😅. Since we have already settled on the features for the next release and this would be quite a big change, don't expect us to work on this anytime soon. That being said, I'm generally in favor of this change.

@flavio what do you think ?

[flavio]

@flavio what do you think ?

I'm not entirely convinced about that. Why don't you issue commands against distribution's API?

[wolfch]

Hi @flavio - sorry, I was not clear - what I want to accomplish is cloning/replicating a registry via API calls. Yes, image-related activity would be done via the distribution API, but to replicate the Portus metadata, i.e. users, namespaces, teams, roles, registriy registration, etc. are purely Portus objects that distribution/registry knows nothing about.

[BastienM]

Hello there,

Not the same goal as wolfch but I hope it will match the topic.

I am trying to create a team/namespace by using the exposed API (basically copying the request made by the browser) but always end up getting a 422 Unprocessable Entity.

The goals behind that is to be able to provision our Portus instance using LDAP (creating teams/users/namespaces) as features in #734 are still not implemented.

Here's my current script :

#!/bin/bash                                                                                                           

BASE_URL="https://portus.domain.local/teams"
TOKEN=$(curl -ski "${BASE_URL}/teams" -u "user:password" | grep "csrf-token" | sed 's/<meta name="csrf-token" content="//' | sed 's/" \/>.*//')

declare -A HEADERS
HEADERS=(
    [Host]="https://portus.domain.local"
    [Accept]="*/*;q=0.5, text/javascript, application/javascript, application/ecmascript, application/x-ecmascript"
    [X-CSRF-Token]=$TOKEN
    [Content-Type]="application/x-www-form-urlencoded; charset=UTF-8"
    [X-Requested-With]="XMLHttpRequest"
    [Referer]="https://portus.domain.local/teams"
)

function set_header {
    echo "-H $1: ${HEADERS[$1]}"
}

curl -vki "${BASE_URL}.json" \
"$(set_header Host)" \
"$(set_header Accept)" \
--compressed \
"$(set_header X-CSRF-Token)" \
"$(set_header Content-Type)" \
"$(set_header X-Requested-With)" \
"$(set_header Referer)" \
--data 'utf8=%E2%9C%93&team%5Bname%5D=bash&team%5Bdescription%5D=bash&commit=Add'h
-u "user:password"

Which give :

*   Trying 172.16.18.41...
* Connected to portus.domain.local (172.16.18.41) port 443 (#0)
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: none
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: C=FR; O=LINAGORA; OU=SERVER; CN=docker-ci.linagora.com
*  start date: Mar  7 21:38:16 2016 GMT
*  expire date: Mar  7 21:38:16 2018 GMT
*  issuer: C=FR; O=LINAGORA; OU=0002 431473669; CN=LINAGORA Service CA
*  SSL certificate verify result: self signed certificate in certificate chain (19), continuing anyway.
> POST /teams.json HTTP/1.1
> Host: portus.domain.local
> User-Agent: curl/7.48.0
> Accept: */*
> Accept-Encoding: deflate, gzip
>  Host: https://portus.domain.local
>  Accept: */*;q=0.5, text/javascript, application/javascript, application/ecmascript, application/x-ecmascript
>  X-CSRF-Token: 6b0qSIlvToY3xGbNFPX3mijHX92/cywfkJHqpr5Zi3hCG2OetR7Nb/mBynadE5BelsVIDO9p+CrdSc+1Z5IuXg==
>  Content-Type: application/x-www-form-urlencoded; charset=UTF-8
>  X-Requested-With: XMLHttpRequest
>  Referer: https://portus.domain.local/teams
> Content-Length: 72
> Content-Type: application/x-www-form-urlencoded
> 
* upload completely sent off: 72 out of 72 bytes
< HTTP/1.1 422 Unprocessable Entity
HTTP/1.1 422 Unprocessable Entity
< Date: Mon, 23 May 2016 09:22:11 GMT
Date: Mon, 23 May 2016 09:22:11 GMT
< Server: Apache/2.4.18 (Unix) OpenSSL/1.0.1k
Server: Apache/2.4.18 (Unix) OpenSSL/1.0.1k
< Content-Type: text/html; charset=utf-8
Content-Type: text/html; charset=utf-8
< X-Request-Id: 54aa76de-9393-465b-856c-3926f3e0ab13
X-Request-Id: 54aa76de-9393-465b-856c-3926f3e0ab13
< X-Runtime: 0.004051
X-Runtime: 0.004051
< Content-Length: 0
Content-Length: 0

< 
* Connection #0 to host portus.domain.local left intact

It is by design or am I missing something ?

[mssola]

We are starting to develop a REST API. For now we have only included users and application tokens, but more will come in the future. Closing this in favor of #1412.