web_spider sslv3 handshake failure

[andersonDadario]

The web_spider plugin got an error while requesting "https://dadario.com.br/blog/apple-entenda-o-vazamento-de-udids-parte-1/". Reason: "[Errno 1] _ssl.c:510: error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure"

As servers mitigate POODLE attack (sslv3), the web_spider can't connect anymore. Need add support for TLSv1.2 TLSv1.1 TLSv1

[andresriancho]

https://bugs.launchpad.net/ubuntu/+source/openssl/+bug/965371

[andresriancho]

https://gist.github.com/flandr/74be22d1c3d7c1dfefdd

[andresriancho]

Conditions of satisfaction:

• We have a unittest with a server that only supports TLSv1
• We have a unittest with a server that only supports SSLv2
• We have a unittest with a server that only supports SSLv3
• We perform detection of the SSL protocol supported by the remote end one time, and then use that protocol for all the scan

[andresriancho]

http://bugs.python.org/msg128686

[andresriancho]

https://github.com/sqlmapproject/sqlmap/blob/master/lib/request/httpshandler.py

[andresriancho]

Test script:

# webspider

plugins
output console,text_file
output config text_file
set output_file output-w3af.txt
set verbose True
back
output config console
set verbose True
back

crawl web_spider
crawl config web_spider
set only_forward True
back
back

target
set target https://dadario.com.br/
back

start

exit

Recommend enabling DEBUG in keepalive.py

Strange...

SSL connection error occurred with protocol 1: '[Errno 1] _ssl.c:510: error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure'
SSL connection error occurred with protocol 3: '[Errno 1] _ssl.c:510: error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure'
SSL connection error occurred with protocol 2: '[Errno 1] _ssl.c:510: error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure'

Why does it say routines:SSL3_READ_BYTES:sslv3 when I'm using protocol 3 (which is TLS)?

[andresriancho]

@andersonDadario any idea on the last question?

[andersonDadario]

Some ideia I guess. The TLS 1.0 is similiar, but different from SSLv3, so there may be some confusion.

I'm not a crypto specialist it's just a little research at google, as you can see at [1] and [2] saying the same thing.

Note: My blog (dadario.com.br) is hosted at AWS (static website served from S3) so I will not be able to print the server config here if you need to :(

[1] http://serverfault.com/questions/178561/what-are-the-exact-protocol-level-differences-between-ssl-and-tls
[2] http://stackoverflow.com/questions/385634/is-there-a-difference-between-sslv3-and-tls1-0

[andersonDadario]

But you can test against a nginx web server that mitigates Poodle [1] in case you need to know the web server config.

"How to test for SSL POODLE vulnerability?
$ openssl s_client -connect google.com:443 -ssl3
If there is a handshake failure then the server is not supporting SSLv3 and it is secure from this vulnerability. Otherwise it is required to disable SSLv3 support.

How to disable the SSLv3 support on Nginx?
In nginx configuration, just after the "ssl on;" line, add the following to allow only TLS protocols:
ssl_protocols TLSv1.2 TLSv1.1 TLSv1;"

[1] http://blog.adityapatawari.com/2014/10/how-to-check-for-ssl-poodle-sslv3-bug.html

[andresriancho]

@andersonDadario maybe it's an ubuntu bug? Which OS do you use? Can you checkout 4d3da21 and run the script above to see if you get the same results?

The messages will appear if you set DEBUG to True in keepalive.py

[andersonDadario]

As I told you it's a managed service From AWS to serve static sites from S3 (actually CloudFront is on the front), so I don't have access to it, but I will test against my servers that disabled sslv3 and let you know (maybe tonight). Right now there is no way, unfortunately, but I'll do it.

Enviado via iPhone

Anderson Dadario
https://dadario.com.br

On 28/10/2014, at 14:41, Andres Riancho [email protected] wrote:

@andersonDadario maybe it's an ubuntu bug? Which OS do you use? Can you checkout 4d3da21 and run the script above to see if you get the same results?

The messages will appear if you set DEBUG to True in keepalive.py

—
Reply to this email directly or view it on GitHub.

[andresriancho]

I was asking about your desktop OS, not the server where the domain is at.

[andersonDadario]

Sorry I misunderstood, it's Ubuntu.

The result after changing keepalive.py and running the script above:

./w3af_console -s test.w3af
w3af>>> plugins
w3af/plugins>>> output console,text_file
w3af/plugins>>> output config text_file
w3af/plugins/output/config:text_file>>> set output_file output-w3af.txt
w3af/plugins/output/config:text_file>>> set verbose True
w3af/plugins/output/config:text_file>>> back
The configuration has been saved.
w3af/plugins>>> output config console
w3af/plugins/output/config:console>>> set verbose True
w3af/plugins/output/config:console>>> back
The configuration has been saved.
w3af/plugins>>> crawl web_spider
w3af/plugins>>> crawl config web_spider
w3af/plugins/crawl/config:web_spider>>> set only_forward True
w3af/plugins/crawl/config:web_spider>>> back
The configuration has been saved.
w3af/plugins>>> back
w3af>>> target
w3af/config:target>>> set target https://dadario.com.br/
w3af/config:target>>> back
The configuration has been saved.
w3af>>> start
Called w3afCore.start()
Enabling _dns_cache()
Called build_openers
DNS response from DNS server for domain: dadario.com.br
Incrementing global error count. GEC: 0
Failed to HTTP "GET" "https://dadario.com.br/". Reason: "Unable to create a SSL connection using protocol: 2", going to retry.
Traceback for this error: Traceback (most recent call last):
File "/usr/local/share/w3af/w3af/core/data/url/extended_urllib.py", line 485, in _send
res = self._opener.open(req)
File "/usr/lib/python2.7/urllib2.py", line 404, in open
response = self._open(req, data)
File "/usr/lib/python2.7/urllib2.py", line 422, in _open
'_open', req)
File "/usr/lib/python2.7/urllib2.py", line 382, in _call_chain
result = func(*args)
File "/usr/local/share/w3af/w3af/core/data/url/handlers/keepalive.py", line 804, in https_open
return self.do_open(req)
File "/usr/local/share/w3af/w3af/core/data/url/handlers/keepalive.py", line 624, in do_open
self._start_transaction(conn, req)
File "/usr/local/share/w3af/w3af/core/data/url/handlers/keepalive.py", line 766, in _start_transaction
conn.endheaders()
File "/usr/lib/python2.7/httplib.py", line 969, in endheaders
self._send_output(message_body)
File "/usr/lib/python2.7/httplib.py", line 829, in _send_output
self.send(msg)
File "/usr/lib/python2.7/httplib.py", line 791, in send
self.connect()
File "/usr/local/share/w3af/w3af/core/data/url/handlers/keepalive.py", line 909, in connect
self.connect_multi_ssl()
File "/usr/local/share/w3af/w3af/core/data/url/handlers/keepalive.py", line 949, in connect_multi_ssl
raise HTTPRequestException(msg % protocol)
HTTPRequestException: Unable to create a SSL connection using protocol: 2

Re-sending request "<HTTPRequest "https://dadario.com.br/" (cookies:True, cache:False)>" after initial exception: "Unable to create a SSL connection using protocol: 2"
Incrementing global error count. GEC: 1
Failed to HTTP "GET" "https://dadario.com.br/". Reason: "Unable to create a SSL connection using protocol: 2", going to retry.
Traceback for this error: Traceback (most recent call last):
File "/usr/local/share/w3af/w3af/core/data/url/extended_urllib.py", line 485, in _send
res = self._opener.open(req)
File "/usr/lib/python2.7/urllib2.py", line 404, in open
response = self._open(req, data)
File "/usr/lib/python2.7/urllib2.py", line 422, in _open
'_open', req)
File "/usr/lib/python2.7/urllib2.py", line 382, in _call_chain
result = func(*args)
File "/usr/local/share/w3af/w3af/core/data/url/handlers/keepalive.py", line 804, in https_open
return self.do_open(req)
File "/usr/local/share/w3af/w3af/core/data/url/handlers/keepalive.py", line 624, in do_open
self._start_transaction(conn, req)
File "/usr/local/share/w3af/w3af/core/data/url/handlers/keepalive.py", line 766, in _start_transaction
conn.endheaders()
File "/usr/lib/python2.7/httplib.py", line 969, in endheaders
self._send_output(message_body)
File "/usr/lib/python2.7/httplib.py", line 829, in _send_output
self.send(msg)
File "/usr/lib/python2.7/httplib.py", line 791, in send
self.connect()
File "/usr/local/share/w3af/w3af/core/data/url/handlers/keepalive.py", line 909, in connect
self.connect_multi_ssl()
File "/usr/local/share/w3af/w3af/core/data/url/handlers/keepalive.py", line 949, in connect_multi_ssl
raise HTTPRequestException(msg % protocol)
HTTPRequestException: Unable to create a SSL connection using protocol: 2

strategy.start() is raising exception "The remote web server is not answering our HTTP requests, multiple errors have been found while trying to GET a response from the server.

In most cases this means that the configured target is incorrect, the port is closed, there is a firewall blocking our packets or there is no HTTP daemon listening on that port.

Please verify your target configuration and try again."

IMPORTANT The following error was detected by w3af and couldn't be resolved:
The remote web server is not answering our HTTP requests, multiple errors have been found while trying to GET a response from the server.

In most cases this means that the configured target is incorrect, the port is closed, there is a firewall blocking our packets or there is no HTTP daemon listening on that port.

Please verify your target configuration and try again.

Scan finished in 0 seconds.
Stopping the core...
w3af>>> exit
w3af>>> The user stopped the core, finishing threads...
0 seconds. were needed to stop the core.

Liked it? Sponsor the project!

[andresriancho]

Trying to fix this once again could you give me the output of running https://gist.github.com/andresriancho/77d1d163c9987966ce77 in your system please? Also log the OS and version, thanks!

[andresriancho]

My output is:

[pablo:/home/pablo/pch/w3af] feature/poodle(+1/-1) ± python conn.py 
Traceback (most recent call last):
  File "conn.py", line 5, in <module>
    sock = ssl.wrap_socket(sock, None, None, ssl_version=ssl.PROTOCOL_TLSv1)
  File "/usr/lib/python2.7/ssl.py", line 487, in wrap_socket
    ciphers=ciphers)
  File "/usr/lib/python2.7/ssl.py", line 243, in __init__
    self.do_handshake()
  File "/usr/lib/python2.7/ssl.py", line 405, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [Errno 1] _ssl.c:510: error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure
[pablo:/home/pablo/pch/w3af] feature/poodle(+1/-1) 1 ± 

[andersonDadario]

Mine is:

$ python

import socket
import ssl

... sock = socket.create_connection(('dadario.com.br', 443))
sock = ssl.wrap_socket(sock, None, None, ssl_version=ssl.PROTOCOL_TLSv1)
Traceback (most recent call last):
File "", line 1, in
File "/usr/lib/python2.7/ssl.py", line 487, in wrap_socket
ciphers=ciphers)
File "/usr/lib/python2.7/ssl.py", line 243, in init
self.do_handshake()
File "/usr/lib/python2.7/ssl.py", line 405, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [Errno 1] _ssl.c:510: error:14094410:SSL
routines:SSL3_READ_BYTES:sslv3 alert handshake failure

2015-01-30 22:39 GMT-02:00 Andres Riancho [email protected]:

My output is:

[pablo:/home/pablo/pch/w3af] feature/poodle(+1/-1) ± python conn.py
Traceback (most recent call last):
File "conn.py", line 5, in
sock = ssl.wrap_socket(sock, None, None, ssl_version=ssl.PROTOCOL_TLSv1)
File "/usr/lib/python2.7/ssl.py", line 487, in wrap_socket
ciphers=ciphers)
File "/usr/lib/python2.7/ssl.py", line 243, in init
self.do_handshake()
File "/usr/lib/python2.7/ssl.py", line 405, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [Errno 1] _ssl.c:510: error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure
[pablo:/home/pablo/pch/w3af] feature/poodle(+1/-1) 1 ±

—
Reply to this email directly or view it on GitHub
#5802 (comment).

Anderson Dadario https://www.linkedin.com/in/andersondadario@andersonmvd
https://twitter.com/andersonmvdhttp://dadario.com.br

[andresriancho]

@andersonDadario which OS and version on your workstation? (I know I asked this before, but not specifically for this test)

Anderson sent data over email, same OS and version as mine.

[andresriancho]

http://runnable.com/VM6q-SvfvRRn_GLj/output

[andresriancho]

Read somewhere that this could be because of SNI (requests module had this issue)

• https://bugs.launchpad.net/ubuntu/+source/python-requests/+bug/1299075
• https://pypi.python.org/pypi/ndg-httpsclient
• http://stackoverflow.com/a/19477363/1347554
• http://bugs.python.org/issue5639

These guys say it works?

• http://good-for-nothing.org/2013/10/an-empirical-study-of-sni-support-in-different-http-clients/

[andresriancho]

http://my.gauntlet.io/ has SSLv3 disabled and the test script works well against it

[andresriancho]

After some more tests:

• It seems to be safe to say that: w3af (in feature/poodle branch at least) can scan a site that has sslv3 disabled
• just checked and w3af (develop branch) can also scan gauntlet.io
• the fix I added in feature/poodle doesn't seem to help (or break) anything related with ssl

This guides us in the direction of a SNI error, and not the initial assessment from Anderson about "As servers mitigate POODLE attack (sslv3), the web_spider can't connect anymore. Need add support for TLSv1.2 TLSv1.1 TLSv1"

[andresriancho]

• Python's ssl module implements SNI only in late releases that most linux distributions don't have yet.
• pyOpenSSL does have SNI support
• There is a hack which I could use to fix my issue...

[andresriancho]

Tested with a fix for SNI

[andersonDadario]

You're amazing. Congratulations for the fix :P

2015-02-01 22:18 GMT-02:00 Andres Riancho [email protected]:

Tested with a fix for SNI
https://gist.github.com/andresriancho/77d1d163c9987966ce77#comment-1384823

—
Reply to this email directly or view it on GitHub
#5802 (comment).

Anderson Dadario https://www.linkedin.com/in/andersondadario@andersonmvd
https://twitter.com/andersonmvdhttp://dadario.com.br

[andresriancho]

Latest commit has the following features:

• Timeout works for HTTP
• I can scan dadario.com.br site (SNI enabled)

Missing tests:

• Can we still scan a site that doesn't have a valid certificate? (unittest)
• Timeout http working as expected
• Timeout https working. This one won't be working until Setting socket timeouts breaks handshake pyca/pyopenssl#168 is fixed. Created reminder at SSL connection timeout doesn't work #7989
• Proxy SSL is working as expected (this build says no but I tested with mitmproxy from the command line and the keepalive.py changes are working, see Proxy daemon SSL bug Proxy daemon SSL bug #183)

[andresriancho]

Just need to make all tests pass at circleci and I'll merge this into develop

[andresriancho]

Fuc**** never ending story! Now httpretty doesn't support openssl so all my mocked tests which use https are failing! A list of all the feature/poodle builds can be found here

This reproduces the OpenSSL un-mocked issue:

nosetests -s w3af/plugins/tests/infrastructure/test_afd.py:TestFoundHttpsAFD

Sent some tweets to the httpretty maintainer asking if there was openssl support for his project, waiting answer.

[andresriancho]

The fix for HTTPretty:

• httplib with OpenSSL mock support gabrielfalcao/HTTPretty#215
• https://travis-ci.org/gabrielfalcao/HTTPretty

[andresriancho]

Much better but still not PASS:

• https://circleci.com/gh/andresriancho/w3af/1346

[andresriancho]

Looks much better, only one test is failing 👍
https://circleci.com/gh/andresriancho/w3af/1352

[andresriancho]

All tests but the test_rfi.py are PASSing, I want to fix that one because my OCD won't let me sleep.

[andresriancho]

Fixed in 03eaf0b

[andresriancho]

OMGWTFBBQ

[andersonmvd]

Nice job :)

Anderson Dadario https://www.linkedin.com/in/andersondadario@andersonmvd
https://twitter.com/andersonmvd
http://dadario.com.br

2015-02-04 19:08 GMT-02:00 Andres Riancho [email protected]:

OMGWTFBBQ

—
Reply to this email directly or view it on GitHub
#5802 (comment).