Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add filter directory #197

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

add filter directory #197

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
6476d25
add filter directory
yangzong18 Oct 12, 2022
491a228
Merge branch 'master' into dev-2209-exclude
yangzong18 Nov 11, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
533 changes: 498 additions & 35 deletions lib/command_test.go
View file
Open in desktop

Large diffs are not rendered by default.

58 changes: 47 additions & 11 deletions lib/cp.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@ type copyOptionType struct {
type filterOptionType struct {
name string
pattern string
isDir bool
}

type fileInfoType struct {
Expand Down Expand Up @@ -212,7 +213,6 @@ var specChineseCopy = SpecText{
?：匹配单个字符
[sequence]：匹配sequence的任意字符
[!sequence]：匹配不在sequence的任意字符
注意：规则不支持带目录的格式，e.g.，--include "/usr/*/test/*.jpg"。

--include和--exclude可以出现多次。当多个规则出现时，这些规则按从左往右的顺序应用。例如：
当前目录下包含3个文件：
Expand All @@ -231,6 +231,23 @@ var specChineseCopy = SpecText{
$ ossutil cp . oss://my-bucket/path --exclude '*.jpg' --include 'testfile*.jpg' --exclude 'testfile?.jpg'
上传testfile2.txt到oss://my-bucket/path/testfile2.txt
上传testfile33.jpg到oss://my-bucket/path/testfile33.jpg

当前目录下包含3个文件一个目录：
testfile1.jpg
testfiel2.txt
testfile33.jpg
testdir1
-testfile1.txt
-testfile2.txt
-testfile3.txt
$ ossutil cp . oss://my-bucket/path --include 'testdir1/*'
上传testdir1目录下的文件到oss://my-bucket/path/testdir1/

$ ossutil cp . oss://my-bucket/path --exclude 'testdir1/*'
上传testfile1.jpg到oss://my-bucket/path/testfile1.jpg
上传testfile2.txt到oss://my-bucket/path/testfile2.txt
上传testfile33.jpg到oss://my-bucket/path/testfile33.jpg


--meta选项

Expand Down Expand Up @@ -745,7 +762,6 @@ var specEnglishCopy = SpecText{
?: Matches any single character
[sequence]: Matches any character in sequence
[!sequence]: Matches any character not in sequence
Note: does not support patterns containing directory info. e.g., --include "/usr/*/test/*.jpg"

Any number of these parameters can be passed to a command. You can do this by providing an --exclude
or --include argument multiple times, e.g.,
Expand Down Expand Up @@ -776,6 +792,22 @@ var specEnglishCopy = SpecText{
upload testfile2.txt to oss://my-bucket/path/testfile2.txt
upload testfile33.jpg to oss://my-bucket/path/testfile33.jpg

e.g., 3 files in current dir and 1 sub directory
testfile1.jpg
testfiel2.txt
testfile33.jpg
testdir1
-testfile1.txt
-testfile2.txt
-testfile3.txt
$ ossutil cp . oss://my-bucket/path --include 'testdir1/*'
upload files in testdir1 directory to oss://my-bucket/path/testdir1/

$ ossutil cp . oss://my-bucket/path --exclude 'testdir1/*'
upload testfile1.jpg to oss://my-bucket/path/testfile1.jpg
upload testfile2.txt to oss://my-bucket/path/testfile2.txt
upload testfile33.jpg to oss://my-bucket/path/testfile33.jpg

--meta option

This option will set the specified objects' meta data. If --recursive option is specified,
Expand Down Expand Up @@ -1797,7 +1829,10 @@ func (cc *CopyCommand) getFileListStatistic(dpath string) error {

if f.IsDir() {
if fpath != dpath {
cc.monitor.updateScanNum(1)
if doesSingleFileMatchPatterns(fpath, cc.cpOption.filters) {
cc.monitor.updateScanNum(1)
}

}
return nil
}
Expand Down Expand Up @@ -1831,7 +1866,7 @@ func (cc *CopyCommand) getFileListStatistic(dpath string) error {
return nil
}
}
if doesSingleFileMatchPatterns(f.Name(), cc.cpOption.filters) {
if doesSingleFileMatchPatterns(fpath, cc.cpOption.filters) {
cc.monitor.updateScanSizeNum(realFileSize, 1)
}
return nil
Expand Down Expand Up @@ -1930,11 +1965,14 @@ func (cc *CopyCommand) getFileList(dpath string, chFiles chan<- fileInfoType) er

if f.IsDir() {
if fpath != dpath {
if strings.HasSuffix(fileName, "\\") || strings.HasSuffix(fileName, "/") {
chFiles <- fileInfoType{fileName, name}
} else {
chFiles <- fileInfoType{fileName + string(os.PathSeparator), name}
if doesSingleFileMatchPatterns(fpath, cc.cpOption.filters) {
if strings.HasSuffix(fileName, "\\") || strings.HasSuffix(fileName, "/") {
chFiles <- fileInfoType{fileName, name}
} else {
chFiles <- fileInfoType{fileName + string(os.PathSeparator), name}
}
}

}
return nil
}
Expand All @@ -1961,8 +1999,7 @@ func (cc *CopyCommand) getFileList(dpath string, chFiles chan<- fileInfoType) er
return nil
}
}

if doesSingleFileMatchPatterns(fileName, cc.cpOption.filters) {
if doesSingleFileMatchPatterns(fpath, cc.cpOption.filters) {
chFiles <- fileInfoType{fileName, name}
}
return nil
Expand Down Expand Up @@ -2404,7 +2441,6 @@ func (cc *CopyCommand) downloadFiles(srcURL CloudURL, destURL FileURL) error {
prefix = srcURL.object[:index+1]
relativeKey = srcURL.object[index+1:]
}

go cc.objectStatistic(bucket, srcURL)
err := cc.downloadSingleFileWithReport(bucket, objectInfoType{prefix, relativeKey, -1, time.Now()}, filePath)
return cc.formatResultPrompt(err)
Expand Down
Loading