Skip to content

Commit

Permalink
Feature: CSV export supports detailed analysis
Browse files Browse the repository at this point in the history
  • Loading branch information
rfc-st committed Dec 28, 2024
1 parent 5a42376 commit 8327a17
Show file tree
Hide file tree
Showing 8 changed files with 1,300 additions and 1,255 deletions.
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
<a target="_blank" href="https://www.python.org/downloads/" title="Minimum Python version required to run this tool"><img src="https://img.shields.io/badge/Python-%3E%3D3.8-blue?labelColor=343b41"></a>
<a target="_blank" href="LICENSE" title="License of this tool"><img src="https://img.shields.io/badge/License-MIT-blue.svg?labelColor=343b41"></a>
<a target="_blank" href="https://github.com/rfc-st/humble/releases" title="Latest release of this tool"><img src="https://img.shields.io/github/v/release/rfc-st/humble?display_name=release&label=Latest%20Release&labelColor=343b41"></a>
<a target="_blank" href="https://github.com/rfc-st/humble/commits/master" title="Latest commit of this tool"><img src="https://img.shields.io/badge/Latest_Commit-2024--12--24-blue.svg?labelColor=343b41"></a>
<a target="_blank" href="https://github.com/rfc-st/humble/commits/master" title="Latest commit of this tool"><img src="https://img.shields.io/badge/Latest_Commit-2024--12--28-blue.svg?labelColor=343b41"></a>
<a target="_blank" href="https://github.com/rfc-st/humble/actions?query=workflow%3ACodeQL" title="Results of the last analysis of this tool with CodeQL"><img src="https://github.com/rfc-st/humble/workflows/CodeQL/badge.svg"></a>
<a target="_blank" href="https://pkg.kali.org/pkg/humble" title="Official tool in Kali Linux"><img src="https://img.shields.io/badge/Kali%20Linux-Tool-blue?labelColor=343b41"></a>
<br />
Expand Down Expand Up @@ -132,7 +132,7 @@ Options used: -f -g -p -U -s --hints
<img src="https://github.com/rfc-st/humble/blob/master/screenshots/humble_fng.jpg" alt="(Linux) - List of HTTP fingerprint headers based on a specific term" width=70% height=70%>
</p>
<br />
.: (Linux) - Brief analysis saved as CSV. <a href="https://github.com/rfc-st/humble/raw/master/samples/humble_https_facebook_com_20241121_174721_en.csv">Example.</a><br />
.: (Linux) - Detailed analysis saved as CSV. <a href="https://github.com/rfc-st/humble/raw/master/samples/humble_https_facebook_com_20241228_202046_en.csv">Example.</a><br />
<p></p>
<p align="center">
<img src="https://github.com/rfc-st/humble/blob/master/screenshots/humble_csv_s.PNG" alt="(Linux) - Brief analysis saved as CSV" width=70% height=70%>
Expand Down Expand Up @@ -264,7 +264,7 @@ $ docker rmi humble:1.42
usage: humble.py [-h] [-a] [-b] [-c] [-df] [-e [TESTSSL_PATH]] [-f [FINGERPRINT_TERM]] [-g] [-grd] [-if INPUT_FILE] [-l {es}] [-lic] [-o {csv,html,json,pdf,txt,xml}]
[-of OUTPUT_FILE] [-op OUTPUT_PATH] [-r] [-s [SKIP_HEADERS ...]] [-u URL] [-ua USER_AGENT] [-v]

'humble' (HTTP Headers Analyzer) | https://github.com/rfc-st/humble | v.2024-12-20
'humble' (HTTP Headers Analyzer) | https://github.com/rfc-st/humble | v.2024-12-28

options:
-h, --help show this help message and exit
Expand All @@ -276,10 +276,10 @@ options:
-f [FINGERPRINT_TERM] Shows fingerprint statistics; if 'FINGERPRINT_TERM' (e.g., 'Google') is omitted the top 20 results will be shown
-g Shows guidelines for enabling security HTTP response headers on popular frameworks, servers and services
-grd Shows the checks to grade an analysis, along with advice for improvement
-if INPUT_FILE Analyzes 'INPUT_FILE': must contain HTTP response headers and values separated by ': '; E.g. 'server: nginx'.
-if INPUT_FILE Analyzes 'INPUT_FILE': must contain HTTP response headers and values separated by ': '; E.g. 'server: nginx'
-l {es} Defines the language for displaying analysis, errors and messages; if omitted, will be shown in English
-lic Shows the license for 'humble', along with permissions, limitations and conditions.
-o {csv,html,json,pdf,txt,xml} Exports analysis to 'humble_scheme_URL_port_yyyymmdd_hhmmss_language.ext' file; csv/json/xml will have a brief analysis
-o {csv,html,json,pdf,txt,xml} Exports analysis to 'humble_scheme_URL_port_yyyymmdd_hhmmss_language.ext' file; json/xml will have a brief analysis
-of OUTPUT_FILE Exports analysis to 'OUTPUT_FILE'; if omitted the default filename of the parameter '-o' will be used
-op OUTPUT_PATH Exports analysis to 'OUTPUT_PATH'; must be absolute. If omitted the PATH of 'humble.py' will be used
-r Shows HTTP response headers and a detailed analysis; '-b' parameter will take priority
Expand Down
2,372 changes: 1,186 additions & 1,186 deletions additional/fingerprint.txt

Large diffs are not rendered by default.

14 changes: 7 additions & 7 deletions humble.py
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,7 @@
URL_STRING = ('rfc-st', ' URL : ', 'caniuse')

current_time = datetime.now().strftime("%Y/%m/%d - %H:%M:%S")
local_version = datetime.strptime('2024-12-24', '%Y-%m-%d').date()
local_version = datetime.strptime('2024-12-28', '%Y-%m-%d').date()


class SSLContextAdapter(requests.adapters.HTTPAdapter):
Expand Down Expand Up @@ -766,7 +766,7 @@ def print_basic_info(export_filename):
def print_extended_info(args, reliable, status_code):
if args.skip_headers:
print_skipped_headers(args)
if args.output in ('csv', 'json', 'xml'):
if args.output in ('json', 'xml'):
print(get_detail('[limited_analysis_note]', replace=True))
if (status_code is not None and 400 <= status_code <= 451) or reliable or \
args.redirects or args.skip_headers:
Expand Down Expand Up @@ -857,7 +857,7 @@ def get_fingerprint_headers():
encoding='utf8') as fng_source:
l_fng_ex = [line.strip() for line in
islice(fng_source, SLICE_INT[0], None) if line.strip()]
l_fng = [line.split(' [')[0].strip() for line in l_fng_ex]
l_fng = [line.split(' (')[0].strip() for line in l_fng_ex]
titled_fng = [item.title() for item in l_fng]
return l_fng_ex, l_fng, titled_fng

Expand Down Expand Up @@ -1358,7 +1358,7 @@ def format_html_fingerprint(args, ln, sub_d, l_fng):
for i in l_fng:
if (ln and i in ln and not args.brief):
try:
idx = ln.index(' [')
idx = ln.index(' (')
except ValueError:
return ln
if 'class="ko"' not in ln:
Expand Down Expand Up @@ -1700,7 +1700,7 @@ def custom_help_formatter(prog):
the checks to grade an analysis, along with advice for improvement")
parser.add_argument("-if", dest='input_file', type=str, help="Analyzes \
'INPUT_FILE': must contain HTTP response headers and values separated by ': ';\
E.g. 'server: nginx'.")
E.g. 'server: nginx'")
parser.add_argument("-l", dest='lang', choices=['es'], help="Defines the \
language for displaying analysis, errors and messages; if omitted, will be \
shown in English")
Expand All @@ -1709,7 +1709,7 @@ def custom_help_formatter(prog):
parser.add_argument("-o", dest='output', choices=['csv', 'html', 'json', 'pdf',
'txt', 'xml'], help="Exports\
analysis to 'humble_scheme_URL_port_yyyymmdd_hhmmss_language.ext' file; \
csv/json/xml will have a brief analysis")
json/xml will have a brief analysis")
parser.add_argument("-of", dest='output_file', type=str, help="Exports \
analysis to 'OUTPUT_FILE'; if omitted the default filename of the parameter \
'-o' will be used")
Expand Down Expand Up @@ -1795,7 +1795,7 @@ def custom_help_formatter(prog):
args.URL_A is None):
print_error_detail('[args_several]')

if args.output in ['csv', 'json', 'xml'] and not args.brief:
if args.output in ['json', 'xml'] and not args.brief:
print_error_detail('[args_brief_filetype]')

skip_list, unsupported_headers = [], []
Expand Down
4 changes: 2 additions & 2 deletions l10n/details.txt
Original file line number Diff line number Diff line change
Expand Up @@ -872,7 +872,7 @@ HTTP Response Headers
Note : The analysis may not be reliable because of the time it took for the URL to respond.

[limited_analysis_note]
Note : Exporting to CSV/JSON/XML is currently limited to a brief analysis
Note : Exporting to JSON/XML is currently limited to a brief analysis

[analysis_redirects_note]
Note : The exact URL will be analyzed, without following redirects.
Expand Down Expand Up @@ -1558,7 +1558,7 @@ Error: The parameters '-b', '-c', '-df', '-'o', '-r' and '-s' require the parame
Error: The parameter '-of' requires the parameters '-u' and '-o'.

[args_brief_filetype]
Error: The parameters '-o csv', '-o json' and '-o xml' require the parameter '-b'.
Error: The parameters '-o json' and '-o xml' require the parameter '-b'.

[notestssl_file]
Error: 'testssl.sh' is not found in that PATH.
Expand Down
4 changes: 2 additions & 2 deletions l10n/details_es.txt
Original file line number Diff line number Diff line change
Expand Up @@ -874,7 +874,7 @@ Cabeceras de respuesta HTTP
Nota : El análisis puede no ser fiable por el tiempo que tardó en responder la URL.

[limited_analysis_note]
Nota: Exportar a CSV/JSON/XML limita actualmente a un análisis breve
Nota: Exportar a JSON/XML limita actualmente a un análisis breve

[analysis_redirects_note]
Nota : Se analizará la URL exacta, sin seguir redireccionamientos.
Expand Down Expand Up @@ -1563,7 +1563,7 @@ Error: Los parámetros '-b', 'c', '-df', -'o', '-r' y '-s' requieren el parámet
Error: El parámetro -of' requiere los parámetros '-u' y '-o'.

[args_brief_filetype]
Error: Los parámetros '-o csv', '-o json' y '-o xml' requieren el parámetro '-b'.
Error: Los parámetros '-o json' y '-o xml' requieren el parámetro '-b'.

[notestssl_file]
Error: 'testssl.sh' no existe en la ruta indicada.
Expand Down
53 changes: 0 additions & 53 deletions samples/humble_https_facebook_com_20241121_174721_en.csv

This file was deleted.

Loading

0 comments on commit 8327a17

Please sign in to comment.