Implement GSP-49 Add CreateDir Operation

[Xuanwo]

RFC: https://github.com/aos-dev/specs/blob/master/rfcs/49-add-create-dir-operation.md

Please leave a comment telling us what part you would like to do.

Roadmap

• Add CreateDir operation in specs
• Update go-storage storage: Implement AOS-49 Add CreateDir Operation #561
• Add integration tests for CreateDir storage: Implement Direr tests go-integration-test#20
• Implement CreateDir in services
  • azblob Add CreateDir go-service-azblob#26 Implement GSP-90, GSP-97, GSP-109 and GSP-117 go-service-azblob#27
  • cos storage: Add CreateDir go-service-cos#28 Implement GSP-97, GSP-109 and GSP-117 go-service-cos#32
  • dropbox storage: Add CreateDir go-service-dropbox#20 Implement GSP-97 and GSP-117, add CreateDir go-service-dropbox#22
  • fs Create dir go-service-fs#45
  • gcs Implement GSP-93, GSP-109 and GSP-117 go-service-gcs#35
  • kodo Implement GSP-93, GSP-109 and GSP-117 go-service-kodo#29
  • oss Implement GSP-93, GSP-97, GSP-109 and GSP-117 go-service-oss#31
  • qingstor Implement GSP-93, GSP-97, GSP-109 and GSP-117 go-service-qingstor#58
  • s3 storage: Add CreateDir go-service-s3#114 Implement GSP-97, GSP-109 and GSP-117 go-service-s3#126
  • uss storage: Add CreateDir go-service-uss#12

[JinnyYi]

I have some questions:

• Is it necessary to check the path parameter?
  For object storage, may be we need to add / at the end of path. But paths may not end with a slash or whitespace in dropbox.
• Is it allowed to create root directory?
• How to deal with file and directory with the same name?
  • For file system, a file and a subdirectory in the same directory with the same name are not allowed.
  • For object storage, test and test/ exist simultaneously is allowed. Should we check whether the file or directory exist if we implement CreateDir through store.Write("abc/", nil, 0, ps.WithContentType("application/x-directory")) in object storage?
• How to deal with creating an existing dir?
  • For file system, create an existing dir will get File exists error.
  • For dropbox, calling CreateFolderV2 to create an existing dir will get path/conflict/folder/. error.
  • For object storage, calling PutObject to create an existing object will overwrite it.
• Impact on other interfaces:
  • CreateAppend/Create(with MultipartId)/write: the path parameter shoud not be abc/ liked (abc/xxx allowed).
  • Delete: Only allowed to delete a file or an empty directory (If allowed to delete recursively, os.RemoveAll() should be called in fs, List need to call first for object storage, config.Folder should be set true for uss)?
  • ...
• How to implement CreateDir in kodo?
  An alternative way is to Put an empty file first, then modify the mimetype to application/x-directory through ChangeMime. But we will get the MD5 of the simulated folder from ListFiles which is puzzling.

[Xuanwo]

Nice question list, I think we should add them in both GSP and CreateDir comments.

---

Is it necessary to check the path parameter?

We can add a limit for CreateDir: path should not be ended with /.

Object storage should add / at the end of path to simulate a directory.

Is it allowed to create root directory?

No, and it's not needed: WorkDir should be created by services while init.

How to deal with file and directory with the same name?

It's storage service behavior, let's just follow them. I think it's OK. Just make it clear in CreateDir's comment.

• For fs, we return an error.
• For object storage, we allow test and test/ to exist at the same time.

How to deal with creating an existing dir?

It's storage service behavior, let's just follow them. I think it's OK. Just make it clear in CreateDir's comment.

• For fs, we return an error.
• For object storage, just overwrite them.

Notice: CreateDir for object storage service should be a virtual function, users should enable them and we should add comments for all these limitations.

CreateAppend/Create(with MultipartId)/write

For all API, we don't allow user input path ends with /.

In Create, we create a dir object like: Create("test", ps.WithObjectMode(ObjectModeDir))

Delete

We only support deleting a single object which means a normal object or an empty dir object.

And we should never call API like os.RemoveAll() in our services.

How to implement CreateDir in kodo?

Let's discuss this in kodo's PR.

[Xuanwo]

Now we have GSP-93, the behavior should be changed.

[JinnyYi]

Pause for beyondstorage/specs#107.