-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Mpro coleta #5
base: main
Are you sure you want to change the base?
Mpro coleta #5
Conversation
thyagopereira
commented
Apr 27, 2021
- Criação do file só acontece com sudo, trabalhando nisso.
- Avanços estruturais, e de coleta. Me atrasou um pouco a necessidade de lidar com o id de sessão.
crawler.go
Outdated
page, err := ioutil.ReadAll(resp.Body) | ||
htmlCode := string(page) | ||
|
||
id := strings.Split(htmlCode, "Constants.viewingSessionId = \"") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Poderia, por favor, fazer um teste de unidade para essa função? para tornar o teste mais fácil, pode utilizar a bilioteca httptest
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Estamos quase chegando lá! 😄
|
||
// Retorna as url para download de cada planilha em questão | ||
func requestURL(year, month int) (urlRequests, error) { | ||
remuIDURL := fmt.Sprint("https://servicos-portal.mpro.mp.br/plcVis/frameset?__report=..%2FROOT%2Frel%2Fcontracheque%2Fmembros%2FremuneracaoMembrosAtivos.rptdesign&anomes=", year, fmt.Sprintf("%02d", month), "&nome=&cargo=&lotacao=") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Aplicar o mesmo estilo da sugestão abaixo aqui.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Bacana! Favor endereçar os comentários abaixo antes de fazer merge
} | ||
defer resp.Body.Close() | ||
|
||
_, err = os.Stat(outputPath) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
colocar esse assinalamento no if abaixo
var fileName = fmt.Sprintf("%d_%02d_remu.xls", year, month) | ||
var filePath = fmt.Sprint(outputPath, "/", fileName) | ||
|
||
err = download(request.remunerationURL, filePath, outputPath) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Colocar esse assinalamento no if abaixo
var fileName = fmt.Sprintf("%d_%02d_vi.xls", year, month) | ||
var filePath = fmt.Sprint(outputPath, "/", fileName) | ||
|
||
err = download(request.benefitsURL, filePath, outputPath) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
mover assinalamento para dentro do if abaixo
@@ -14,7 +14,7 @@ import ( | |||
type Environment struct { | |||
Month int `envconfig:"MONTH" required:"true"` | |||
Year int `envconfig:"YEAR" required:"true"` | |||
OutputFolder string `envconfig:"OUTPUT_FOLDER" default:"/output"` | |||
OutputFolder string `envconfig:"OUTPUT_FOLDER" default:"./output"` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
remover "."
fileNames := Crawl(month, year, outputPath) | ||
fileNames, err := Crawl(month, year, outputPath) | ||
if err != nil { | ||
os.Exit(1) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
usar pacote status
|
||
paths = append(paths, filePath) | ||
case viURLType: | ||
var fileName = fmt.Sprintf("%d_%02d_vi.xls", year, month) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
var fileName = fmt.Sprintf("%d_%02d_vi.xls", year, month) | |
fileName := fmt.Sprintf("%d_%02d_vi.xls", year, month) |
paths = append(paths, filePath) | ||
case viURLType: | ||
var fileName = fmt.Sprintf("%d_%02d_vi.xls", year, month) | ||
var filePath = fmt.Sprint(outputPath, "/", fileName) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
var filePath = fmt.Sprint(outputPath, "/", fileName) | |
filePath := fmt.Sprint(outputPath, "/", fileName) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
usar filepath.Join