API: add endpoint POST /v1/containers

VERSION

@@ -1 +1 @@
-1.1.32
+1.1.33

docs/_docs/client.md

@@ -125,21 +125,14 @@ We use `-U` for unsigned.
 
 ### Push Logic
 
+ - If you push an image to a non existing collection, the collection will be created first (version `1.1.32`)
+ - If you push an image to a non existing container, the container will be created first (version `1.1.33`) (\*)
+ - If you push a new image, it will be added.
+ - If you push a new tag, it will be added.
  - If you push an existing tag, if the container is unfrozen, it will be replaced
  - If you push an existing tag and the container is frozen (akin to protected) you'll get permission denied.
- - If you push a new tag, it will be added.
- - If you push a new image, it will also be added.
- - If you push an image to a non existing collection, the collection will be created first, then the image will be added (version `1.1.32`).
-
-Unlike the Sylabs API, when the GET endpoint is made to `v1/containers` and the image doesn't exist,
-we return a response for the collection (and not 404). In other words, [this response](https://github.com/sylabs/scs-library-client/blob/acb520c8fe6456e4223af6fbece956449d790c79/client/push.go#L140) is always returned. We do this because
-the Sylabs library client has a strange logic where it doesn't tag images until after the fact,
-and doesn't send the user's requested tag to any of the get or creation endpoints. This means
-that we are forced on the registry to create a dummy holder tag (that is guaranteed to be unique)
-and then to find the container at the end to [set tags](https://github.com/sylabs/scs-library-client/blob/acb520c8fe6456e4223af6fbece956449d790c79/client/push.go#L187) based on the id of the image
-that is created with the [upload request](https://github.com/sylabs/scs-library-client/blob/acb520c8fe6456e4223af6fbece956449d790c79/client/push.go#L174). I didn't see a logical way to create the container using the POST endpoint to
-"v1/containers" given that we do not know the tag or version, and would need to know the exact container id
-to return later when the container push is requested.
+
+(\*) The Singularity Registry Server data model is different from this of Sylabs library. In the former, a container is created from an image. In the latter, containers are just placeholders for images with the same name, and can be empty. Hence Singularity Registry Server just [mimics the container creation without actually doing it](#mimic-empty-container-creation). The container is created afterwards at the upload step.
 
 ### Push Size
 
@@ -328,7 +321,7 @@ As of version `1.1.32` it is possible to create a new collection via the API. It
 First retrieve the numeric `id` associated with your username with a GET request to the endpoint `/v1/entities/<username>`.
 
 ```bash
-$ curl -s -H 'Authorization: Bearer <token>' /v1/entities/<username>
+$ curl -s -H 'Authorization: Bearer <token>' http://127.0.0.1/v1/entities/<username>
 ```
 Here is a response made pretty by piping into json_pp:
 ```
@@ -396,3 +389,94 @@ You can then see the response that the collection was created, and it will appea
 
 The `private` key is optional. If not provided, it defaults to the servers's configured default for collection creation.
 In case of a `singularity push` to a non existing collection, the client triggers the collection creation first, using this endpoint, then pushes the image.
+
+## Mimic empty container creation
+
+As of version `1.1.33` it is possible to mimic the Sylabs library endpoint for new container creation. It requires authentication.
+
+First retrieve the numeric `id` associated with the holding collection with a GET request to the endpoint `/v1/collections/<entity>/<collection_name>`, where `<entity>` is your username.
+
+```bash
+$ curl -s -H 'Authorization: Bearer <token>' http://127.0.0.1/v1/collections/<entity>/<collection_name>
+```
+
+This is the value associated with the `id` key of the answer. For example:
+```
+{
+  "data": {
+    "deletedAt": "0001-01-01T00:00:00Z",
+    "entityName": "pini",
+    "deleted": false,
+    "name": "pini-private",
+    "owner": "1",
+    "size": 4,
+    "description": "My private collection",
+    "customData": "",
+    "entity": "1",
+    "updatedBy": "1",
+    "id": "6",
+    "createdBy": "1",
+    "private": true,
+    "createdAt": "2021-02-20T20:21:13.692215Z",
+    "containers": [
+      "5",
+      "6",
+      "7",
+      "8"
+    ],
+    "updatedAt": "2021-02-20T20:21:13.777940Z"
+  }
+}
+```
+
+Then we can issue a POST request to the endpoint `/v1/containers` with the payload:
+```
+{
+  "collection": "<collection_numeric_id>"
+  "name": "<new_container_name>"
+}
+```
+```bash
+$ curl -X POST -H 'Authorization: Bearer <token>' -H "Content-Type: application/json" --data '{"collection": 6, "name": "fish"}' http://127.0.0.1/v1/containers 
+```
+
+There is no container creation, because it is not needed with the Singularity Registry Server data model. But the response is as if the container was created so that any workflow using this endpoint could work the same as with Sylabs library:
+
+```bash
+{
+  "data": {
+    "collection": "6",
+    "description": "My private collection",
+    "collectionName": "pini-private",
+    "owner": "1",
+    "readOnly": false,
+    "id": "6",
+    "size": 4,
+    "imageTags": {},
+    "createdBy": "1",
+    "private": true,
+    "downloadCount": null,
+    "updatedAt": "2021-02-20T20:21:13.777940Z",
+    "fullDescription": "Test-private Collection",
+    "deletedAt": "0001-01-01T00:00:00Z",
+    "entityName": "pini",
+    "deleted": false,
+    "name": "test-private",
+    "stars": 0,
+    "customData": "",
+    "images": [],
+    "entity": "1",
+    "updatedBy": "1",
+    "archTags": {
+      "amd64": {}
+    },
+    "createdAt": "2021-02-20T20:21:13.692215Z",
+    "containers": [
+      "5",
+      "6",
+      "7",
+      "8"
+    ]
+  }
+}
+```

shub/apps/library/views/images.py

@@ -13,7 +13,7 @@
 from sregistry.utils import parse_image_name
 
 from shub.apps.logs.utils import generate_log
-from shub.apps.main.utils import format_collection_name
+from shub.apps.main.utils import format_collection_name, format_container_name
 from shub.apps.main.models import Collection, Container
 from shub.settings import (
     MINIO_BUCKET,
@@ -708,18 +708,19 @@ def post(self, request, collection_id):
 
 
 class ContainersView(RatelimitMixin, APIView):
-    """Return a simple list of containers
-    GET /v1/containers
-    """
 
     renderer_classes = (JSONRenderer,)
     ratelimit_key = "ip"
     ratelimit_rate = settings.VIEW_RATE_LIMIT
     ratelimit_block = settings.VIEW_RATE_LIMIT_BLOCK
-    ratelimit_method = "GET"
+    ratelimit_method = ("GET", "POST")
     renderer_classes = (JSONRenderer,)
 
     def get(self, request):
+        """Return a simple list of containers.
+
+        GET /v1/containers
+        """
 
         print("GET ContainersView")
         print(request.data)
@@ -732,6 +733,85 @@ def get(self, request):
         # collections = generate_collections_list(token.user)
         return Response(data={}, status=200)
 
+    def post(self, request):
+        """Mimic the creation a new container.
+
+        POST /v1/containers
+
+        Body parameters:
+        * collection: collection numeric id as a string
+        * name: new container name
+
+        Sylabs library has an optional 'private' parameter, which we
+        ignore here because containers' privacy is inherited from the
+        collection they belong to.
+
+        Return new container's data (but it is not created actually).
+
+        This endpoint only mimics the Sylabs library's one, without actually
+        creating the container object. Because in the sregistry data model a
+        container cannot exist with no images.
+
+        It is provided to improve compatibility with other singularity
+        clients.
+        """
+
+        print("POST ContainersView")
+        if not validate_token(request):
+            message = {"error": {"code": 403, "message": "Token not valid"}}
+            return Response(message, status=403)
+
+        # body should have {'collection': collection_id, 'name': new_container_name}
+        # 'private' is optional and unused
+        # {"collection": "22", "name": "my_container"}
+        body = json.loads(request.body.decode("utf-8"))
+        if not ("collection" in body and "name" in body):
+            message = {"error": {"code": 400, "message": "Invalid payload."}}
+            return Response(message, status=400)
+
+        try:
+            collection_id = int(body["collection"])
+        except ValueError:
+            message = {"error": {"code": 400, "message": "Invalid payload."}}
+            return Response(message, status=400)
+
+        # check permissions
+        # return 403 when collection does not exist or user is not an owner
+        token = get_token(request)
+        try:
+            collection = Collection.objects.get(id=collection_id)
+        except Collection.DoesNotExist:
+            collection = None
+        if (not collection) or (token.user not in collection.owners.all()):
+            message = {
+                "error": {
+                    "code": 403,
+                    "message": "Permission denied {0} {1}".format(
+                        token.user.id, body["entity"]
+                    ),
+                }
+            }
+            return Response(message, status=403)
+
+        # does a container with the same name exist already?
+        name = format_container_name(body["name"])
+        containers = collection.containers.filter(name=name)
+
+        if containers:
+            message = {
+                "error": {
+                    "code": 403,
+                    "message": "A container named '{0}' exists already for collection id '{1}'!".format(
+                        name, collection_id
+                    ),
+                }
+            }
+            return Response(message, status=403)
+
+        # We don't need to create the specific container here
+        details = generate_collection_details(collection, [], token.user)
+        return Response(data={"data": details}, status=200)
+
 
 class GetNamedCollectionView(RatelimitMixin, APIView):
     """Given a collection, return the associated metadata.
@@ -806,10 +886,15 @@ def get(self, request, username, name, container):
             return Response(status=403)
 
         # We don't need to create the specific container here
-        containers = collection.containers.filter(name=container) or []
-
-        # Even if the container doesn't exist, we return response that it does,
-        # And it's created in the next view.
-
-        data = generate_collection_details(collection, containers, token.user)
-        return Response(data={"data": data}, status=200)
+        containers = collection.containers.filter(name=container)
+        if containers:
+            data = generate_collection_details(collection, containers, token.user)
+            return Response(data={"data": data}, status=200)
+        else:
+            message = {
+                "error": {
+                    "code": 404,
+                    "message": "Error retrieving container: not found",
+                }
+            }
+            return Response(message, status=404)