Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[curvefs] recycle blacklist or writelist #2230

Open
cw123 opened this issue Feb 6, 2023 · 17 comments
Open

[curvefs] recycle blacklist or writelist #2230

cw123 opened this issue Feb 6, 2023 · 17 comments
Labels
enhancement improve feature good first issue Good for newcomers pending repairing bug repairing

Comments

@cw123
Copy link
Contributor

cw123 commented Feb 6, 2023

Description:

当前curvefs已经有一个实现了基础功能的回收站,用来对误删除操作兜底。当文件从回收站删除之后,可以去回收站找回误删的文件。
如果所有删除的文件都进入回收站,可能会把存储系统的存储空间撑爆炸。
希望回收站能够增加一个白名单或者黑名单功能。
白名单:符合白名单条件的文件才会进入回收站,其他文件删除的时候直接删除。
黑名单:符合黑名单条件的文件不会进入回收站,删除的时候直接删除,其余文件删除时进入回收站。

Currently, curvefs already has a recycle bin that implements basic functions, which is used to cover accidental deletion operations. When files are deleted from the recycle bin, you can go to the recycle bin to retrieve the accidentally deleted files.
If all deleted files go into the recycle bin, it may blow up the storage space of the storage system.
I hope that the recycle bin can add a whitelist or blacklist function.
Whitelist: Only files that meet the whitelist conditions will enter the recycle bin, and other files will be deleted directly when they are deleted.
Blacklist: Files that meet the conditions of the blacklist will not enter the recycle bin, and will be deleted directly when deleted, and other files will enter the recycle bin when deleted.

Proposed solution:

白名单或者黑名单可通过配置文件配置,client挂载的时候,读取配置文件,获取回收站黑名单或者白名单。在client的fuse_client.cppCURVEFS_ERROR FuseClient::RemoveNode(fuse_req_t req, fuse_ino_t parent, const char *name, FsFileType type) 时候,判断文件是否需要进入回收站。

The whitelist or blacklist can be configured through the configuration file. When the client mounts, it reads the configuration file and obtains the blacklist or whitelist of the recycle bin. When CURVEFS_ERROR FuseClient::RemoveNode(fuse_req_t req, fuse_ino_t parent, const char *name, FsFileType type) of fuse_client.cpp of the client determines whether the file needs to enter the recycle bin.

@cw123 cw123 added the enhancement improve feature label Feb 6, 2023
@cw123 cw123 changed the title curvefs recycle blacklist or writelist [curvefs] recycle blacklist or writelist Feb 6, 2023
@wuhongsong wuhongsong added the good first issue Good for newcomers label Feb 15, 2023
@Alipebt
Copy link

Alipebt commented Mar 4, 2023

I will try to do this .Please assign me.

@ilixiaocui
Copy link
Contributor

Since the developer @Alipebt has more important matters and cannot continue this issue, interested students can continue to try.

@CodeFarmerPK
Copy link
Contributor

I want to try to deal with this issue.

@ilixiaocui
Copy link
Contributor

I want to try to deal with this issue.

welcome!

@Cyber-SiKu
Copy link
Contributor

@CodeFarmerPK Have you encountered any difficulties?

@CodeFarmerPK
Copy link
Contributor

@CodeFarmerPK Have you encountered any difficulties?
I'm still deal with it, and i have a little trouble when build test environment.
The recycle bin has been merged in branch release2.6 recently, and the earlier image can't biuld the test environment. When I made image with newest branch master, I can't create filesystem when mount. Now I'm trying to use branch release2.6 to create a new image to build test environment.

@caoxianfei1
Copy link
Contributor

caoxianfei1 commented Jun 30, 2023

@CodeFarmerPK Have you encountered any difficulties?
I'm still deal with it, and i have a little trouble when build test environment.
The recycle bin has been merged in branch release2.6 recently, and the earlier image can't biuld the test environment. When I made image with newest branch master, I can't create filesystem when mount. Now I'm trying to use branch release2.6 to create a new image to build test environment.

Is there any specific error message when using the image of master branch? And feel free to continue.

@CodeFarmerPK
Copy link
Contributor

CodeFarmerPK commented Jul 7, 2023

@CodeFarmerPK Have you encountered any difficulties?
I'm still deal with it, and i have a little trouble when build test environment.
The recycle bin has been merged in branch release2.6 recently, and the earlier image can't biuld the test environment. When I made image with newest branch master, I can't create filesystem when mount. Now I'm trying to use branch release2.6 to create a new image to build test environment.

Is there any specific error message when using the image of master branch? And feel free to continue.

No problem when I build image with branch master or branch release2.6. The earlier image given in document like "opencurvedocker/curve-base:build-debian9" does not support recycle bin.

And now I am not in work, I was sick for almost two weeks, ops.

@CodeFarmerPK
Copy link
Contributor

I found the way to filter files when delete, but recycle bin has some other functions to do, and I am trying to design how to support other functions.
For I do not have full time to deal with it, it will cost a lot of time.

@CodeFarmerPK
Copy link
Contributor

The first version, I want to set witelist or black list when create fs.
And next version I want to create a new tool for recycle bin configuration. Include witelist、blacklist、change recycletimehour and other functions.
For that I don't know how to create tool now and I need to learn how to realize it.
Any problem or suggestion for this programme?

@CodeFarmerPK
Copy link
Contributor

I am under ill for almost one month, so this porblen has been delayed for so long, maybe it will still wait a long time to reslove for that I will not recover soon.
But I still trying to learn the codes and deal with it.

@caoxianfei1
Copy link
Contributor

Ok, we temporarily unassigne to you, so that other people can choose this issue to deal with, but you can still continue. Also wish you a speedy recovery.

@CodeFarmerPK
Copy link
Contributor

Ok, we temporarily unassigne to you, so that other people can choose this issue to deal with, but you can still continue. Also wish you a speedy recovery.

OK, but it is almost done now, and I am testing it.

@caoxianfei1
Copy link
Contributor

Ok, we temporarily unassigne to you, so that other people can choose this issue to deal with, but you can still continue. Also wish you a speedy recovery.

OK, but it is almost done now, and I am testing it.

Got it.

@CodeFarmerPK
Copy link
Contributor

CodeFarmerPK commented Aug 1, 2023

Finished function test and it works.
And i am performing boundary and exception test to improve the codes, so I assign the issue to myself.

@wuhongsong
Copy link
Contributor

@CodeFarmerPK doing now

@caoxianfei1
Copy link
Contributor

@CodeFarmerPK For a long time there was no corresponding work and we unassign to you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement improve feature good first issue Good for newcomers pending repairing bug repairing
Projects
None yet
Development

No branches or pull requests

7 participants