-
Notifications
You must be signed in to change notification settings - Fork 269
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Nessus: skipped report due to encoding issue #208
Comments
Hi @svalo, |
I am also getting the same issue.
Affected module Expected behavior System in which VulnWhisperer runs (please complete the following information): OS: CentOS Vulnwhisperer has indexed a large portion of documents but a particular scan has an area which hangs. Once the error is triggered, the entire process halts and no additional scans are processed. Full log:
|
Hi @qmontal, thank for your answer, I'll try to provide the debug log. The issue anyway was that python wasn't able to properly decode the csv as utf-8 because of a character at some position in the file. As the file was coming from a nessus used in central Europe I tried first using ascii encoding and later on using latin1. It turned out the latter was the good one. I'm sure my patch doesn't cover all the cases, it was meant as a possible approach to the issue :) I think It's a bit hazardous to assume the exported CSV would always be encoded as utf-8, on the other hand handling alle the possible encodings would be a nightmare. I checked whether it's possible to provide a list of supported encoding as HTTP headers to requests but it seems that it's an indication of preference and it's not sure Nessus would honor it. I'm not even sure the problem would still be present once the project moves to python3 where there are many differences in regards to encoding. |
@alta01 based on the behaviour you reported it seems that the commit I mentioned in my ticket isn't present as it handles the failures skipping the problematic docs, what does |
Describe the bug
While trying to run VW against a Nessus instance I receive an error regarding a document which gets downloaded but not analyzed and saved due to encoding problem.
The issue appears to happen here whre VW assumes the document to be UTF-8 and doesn't manage to parse the text.
I managed to read the CSV patching the code and trying to parse the report as 'latin1'. As I'm not too confident with the codebase I'm not sure that's the only point where it could be needed.
Would you consider adding something like that to be able to parse CSV which are not encoded in UTF-8?
My patch is the following
Affected module
Nessus
Expected behavior
Documents which are not UTF8-encoded gets downloaded and passed to logstash
System in which VulnWhisperer runs (please complete the following information):
Additional context
Before commit e1ca9fa the software would stop processing documents due to the encoding error, now it just skips the document.
The text was updated successfully, but these errors were encountered: