-
Notifications
You must be signed in to change notification settings - Fork 62
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: multi task import #117
Conversation
server/README.md
Outdated
| HandleImportAction | /api-nebula/task/import/action | POST | | ||
| QueryImportStats | /api-nebula/task/import/stats | POST | | ||
| DownloadConfig | /api-nebula/task/import/config | POST | | ||
| DownloadLog | /api-nebula/task/import/log | GET | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why not use restful style api?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, I will modify it
server/README.md
Outdated
address: host | ||
postStart: null | ||
preStop: null | ||
logPath: E:\NebulaProject\新建文件夹 (4)\nebula-studio\server\data\tasks\1/import.log |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do not use Chinese.
server/README.md
Outdated
|
||
The request : | ||
|
||
http://localhost:9000/api-nebula/task/import/log/?pathName=E:\NebulaProject\nebula-studio\server\data\tasks\1\err\err 1.log |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here is what to download logs? Why not use the task id nor path?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Because there is need to download two types of files with same id.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The client should not know the path of the service.
You can use parameters and paths to differentiate.
server/README.md
Outdated
"bbbbbbbbbbb", | ||
"ccccccccccc" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What is this?
@@ -266,9 +394,51 @@ Response: | |||
"code": 0, | |||
"message": "", | |||
"data": { | |||
"taskDir": "E:\\NebulaProject\\test\\nebula-studio\\server\\data\\tasks", | |||
"uploadDir": "E:\\NebulaProject\\test\\nebula-studio\\server\\data\\upload" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why expose this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To get all files from this dir
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if err != nil { | ||
return err | ||
} | ||
if err != nil { | ||
return err | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
dup?
err = ioutil.WriteFile(path, []byte(utf8String), 0666) | ||
if err != nil { | ||
return err | ||
} | ||
return nil |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can just return ioutil.WriteFile(path, []byte(utf8String), 0666)
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok
ctx.SendFile(configPath, "config.yaml") | ||
return base.Response{ | ||
Code: base.Success, | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there any abnormality call ctx.JSON(&result)
after ctx.SendFile
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Forget?
if index == -1 { | ||
index = strings.LastIndex(path, "\\") | ||
} | ||
filename := path[index:] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How about index == -1
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The path may be under linux or windows env.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I means that the value of index
maybe -1
here, so it will panic.
zap.L().Warn("open file error", zap.Error(err)) | ||
return nil, err | ||
} | ||
scanner := bufio.NewScanner(file) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Wouldn't that be more concise?
for lineIndex := int64(0); scanner.Scan() && lineIndex < offset+limit; lineIndex++ {
if lineIndex >= offset {
res = append(res, scanner.Text())
}
}
defer func() { | ||
*config.NebulaClientSettings.Connection.Address = address | ||
*config.NebulaClientSettings.Connection.User = user | ||
*config.NebulaClientSettings.Connection.Password = password | ||
}() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's need defer
? It's already at the end of func.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If err happen,it also need to do
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
No description provided.