[ANCHOR-403] Prevent SSRF through SEP-1 TOML redirect #1043

reecexlm · 2023-08-05T17:40:44Z

This PR addresses SDP-001. If an error is encountered while fetching stellar.toml via http, the exception is obfuscated with an error message to avoid ability to use the method to leak aws instance metadata via ssrf.

PR Structure

This PR has reasonably narrow scope (if not, break it down into smaller PRs).
This PR avoids mixing refactoring changes with feature changes (split into two PRs
otherwise).

Thoroughness

To test this I used the Anchor Platform e2e test. I updated the profile/config.env to use stellar toml type url and value to be localhost:9000

sep1.toml.type=url
sep1.toml.value=http://localhost:9000/.well-known/stellar.toml

I ran the local flask code which performed redirect to itself and simulated a metadata endpoint.

First to reproduce the error I received the exception in the to local flask app. can see the redirect and then returns a 200 for the fake metadata.

❯ python3 app1.py
 * Serving Flask app 'app1'
 * Debug mode: off
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on all addresses (0.0.0.0)
 * Running on http://127.0.0.1:9000
 * Running on http://192.168.1.106:9000
Press CTRL+C to quit
127.0.0.1 - - [09/Aug/2023 15:26:30] "GET /.well-known/stellar.toml HTTP/1.1" 302 -
127.0.0.1 - - [09/Aug/2023 15:26:30] "GET /hostname HTTP/1.1" 200 -

the fake metadata sent looks like this

{"amiId":"ami-12345678","instanceId":"i-1234567890abcdef0","instanceType":"t2.micro","region":"us-west-2"}

the sep service had exception in toml parse

AnchorPlatformEnd2EndTest > initializationError FAILED
    java.lang.IllegalStateException: Key is not followed by an equals sign on line 1: {"amiId":"ami-12345678","instanceId":"i-1234567890abcdef0","instanceType":"t2.micro","region":"us-west-2"}
        at com.moandjiezana.toml.Toml.read(Toml.java:140)
        at org.stellar.anchor.util.Sep1Helper$TomlContent.<init>(Sep1Helper.java:25)
        at org.stellar.anchor.util.Sep1Helper.parse(Sep1Helper.java:13)
        at org.stellar.anchor.platform.AbstractIntegrationTest.setupTests(AbstractIntegrationTest.kt:44)

Throwing an exception with a customer error message obfuscates the exception message, preventing the leak

AnchorPlatformEnd2EndTest > initializationError FAILED
    java.lang.RuntimeException: Failed to parse TOML content
        at org.stellar.anchor.util.Sep1Helper.parse(Sep1Helper.java:17)
        at org.stellar.anchor.platform.AbstractIntegrationTest.setupTests(AbstractIntegrationTest.kt:44)
        at org.stellar.anchor.platform.AbstractIntegrationTest.setUp(AbstractIntegrationTest.kt:34)
        at org.stellar.anchor.platform.AnchorPlatformEnd2EndTest$Companion.construct(AnchorPlatformEnd2EndTest.kt:14)
        at org.stellar.anchor.platform.AnchorPlatformEnd2EndTest.construct(AnchorPlatformEnd2EndTest.kt)

stellar-jenkins · 2023-08-05T17:43:21Z

Reference Server Preview is available here:
https://anchor-ref-pr1043.previews.kube001.services.stellar-ops.com/
SEP Server Preview is available here:
https://anchor-sep-pr1043.previews.kube001.services.stellar-ops.com/

stellar-jenkins · 2023-08-06T01:22:48Z

Reference Server Preview is available here:
https://anchor-ref-pr1043.previews.kube001.services.stellar-ops.com/
SEP Server Preview is available here:
https://anchor-sep-pr1043.previews.kube001.services.stellar-ops.com/

stellar-jenkins · 2023-08-06T12:21:52Z

Reference Server Preview is available here:
https://anchor-ref-pr1043.previews.kube001.services.stellar-ops.com/
SEP Server Preview is available here:
https://anchor-sep-pr1043.previews.kube001.services.stellar-ops.com/

stellar-jenkins · 2023-08-08T21:46:43Z

Reference Server Preview is available here:
https://anchor-ref-pr1043.previews.kube001.services.stellar-ops.com/
SEP Server Preview is available here:
https://anchor-sep-pr1043.previews.kube001.services.stellar-ops.com/

…ation step before the assertions

stellar-jenkins · 2023-08-09T02:37:23Z

Reference Server Preview is available here:
https://anchor-ref-pr1043.previews.kube001.services.stellar-ops.com/
SEP Server Preview is available here:
https://anchor-sep-pr1043.previews.kube001.services.stellar-ops.com/

stellar-jenkins · 2023-08-09T14:57:05Z

Reference Server Preview is available here:
https://anchor-ref-pr1043.previews.kube001.services.stellar-ops.com/
SEP Server Preview is available here:
https://anchor-sep-pr1043.previews.kube001.services.stellar-ops.com/

platform/src/main/resources/config/anchor-config-default-values.yaml

core/src/main/java/org/stellar/anchor/util/NetUtil.java

stellar-jenkins · 2023-08-09T15:43:31Z

Reference Server Preview is available here:
https://anchor-ref-pr1043.previews.kube001.services.stellar-ops.com/
SEP Server Preview is available here:
https://anchor-sep-pr1043.previews.kube001.services.stellar-ops.com/

stellar-jenkins · 2023-08-09T15:57:30Z

Reference Server Preview is available here:
https://anchor-ref-pr1043.previews.kube001.services.stellar-ops.com/
SEP Server Preview is available here:
https://anchor-sep-pr1043.previews.kube001.services.stellar-ops.com/

stellar-jenkins · 2023-08-09T23:25:28Z

Reference Server Preview is available here:
https://anchor-ref-pr1043.previews.kube001.services.stellar-ops.com/
SEP Server Preview is available here:
https://anchor-sep-pr1043.previews.kube001.services.stellar-ops.com/

stellar-jenkins · 2023-08-09T23:31:25Z

Something went wrong with PR preview build please check

stellar-jenkins · 2023-08-09T23:45:41Z

Something went wrong with PR preview build please check

stellar-jenkins · 2023-08-09T23:51:37Z

Reference Server Preview is available here:
https://anchor-ref-pr1043.previews.kube001.services.stellar-ops.com/
SEP Server Preview is available here:
https://anchor-sep-pr1043.previews.kube001.services.stellar-ops.com/

stellar-jenkins · 2023-08-10T00:25:00Z

Reference Server Preview is available here:
https://anchor-ref-pr1043.previews.kube001.services.stellar-ops.com/
SEP Server Preview is available here:
https://anchor-sep-pr1043.previews.kube001.services.stellar-ops.com/

stellar-jenkins · 2023-08-10T06:14:18Z

Reference Server Preview is available here:
https://anchor-ref-pr1043.previews.kube001.services.stellar-ops.com/
SEP Server Preview is available here:
https://anchor-sep-pr1043.previews.kube001.services.stellar-ops.com/

philipliu

LGTM

Comments resolved

JakeUrban · 2023-08-10T17:24:54Z

core/src/main/java/org/stellar/anchor/util/NetUtil.java

+    if (!response.isSuccessful()) {
+      throw new IOException(String.format("Unable to fetch data from %s", url));
+    }


Are we sure we want to throw an exception in this case? We aren't interested in the response body of a 400 transaction?

thanks! I will add the response code. will that satisfy ?

making additional changes per discussion in slack.

Ifropc · 2023-08-10T17:51:38Z

core/src/main/java/org/stellar/anchor/util/Sep1Helper.java

+    try {
+      return new TomlContent(tomlString);
+    } catch (Exception e) {
+      // obfuscate exception message to prevent metadata leaks


Should we still log it on ERR level? Otherwise it's hard to debug what went wrong

* ANCHOR-403-reece-SSRF-metadata * use assertThrows * additional cleanup * remove custom exception * make sure that the log event is actually captured by adding a verification step before the assertions * temporary setting toml to url to test * just testing. ignore * try other file * patch toml parser as well * replace comment * revert * lint * debugF * revert config changes * revert defaults

* [ANCHOR-403] Prevent SSRF through SEP-1 TOML redirect (#1043) * ANCHOR-403-reece-SSRF-metadata

ANCHOR-403-reece-SSRF-metadata

d4f087b

reecexlm changed the title ~~ANCHOR-403-reece-SSRF-metadata~~ ANCHOR-403-HOTFIX-WIP-DO-NOT-MERGE-reece-SSRF-metadata Aug 5, 2023

reecexlm added WIP DO NOT MERGE HOTFIX labels Aug 5, 2023

use assertThrows

2d74b54

reecexlm changed the base branch from develop to main August 6, 2023 12:09

additional cleanup

c797ff5

lijamie98 assigned reecexlm Aug 7, 2023

remove custom exception

30c7943

make sure that the log event is actually captured by adding a verific…

93acffc

…ation step before the assertions

reecexlm marked this pull request as ready for review August 9, 2023 02:57

temporary setting toml to url to test

05c167d

lijamie98 previously requested changes Aug 9, 2023

View reviewed changes

platform/src/main/resources/config/anchor-config-default-values.yaml Outdated Show resolved Hide resolved

core/src/main/java/org/stellar/anchor/util/NetUtil.java Show resolved Hide resolved

just testing. ignore

eb31590

try other file

4e9c934

patch toml parser as well

1c2ed0d

reecexlm changed the title ~~ANCHOR-403-HOTFIX-WIP-DO-NOT-MERGE-reece-SSRF-metadata~~ ANCHOR-403-HOTFIX-Prevent-SSRF-metadata-leak Aug 9, 2023

reecexlm removed WIP DO NOT MERGE labels Aug 9, 2023

replace comment

e73c208

revert

8da260c

lint

19c84e5

debugF

3c6f9b0

revert config changes

807ff46

revert defaults

33369dd

philipliu approved these changes Aug 10, 2023

View reviewed changes

philipliu changed the title ~~ANCHOR-403-HOTFIX-Prevent-SSRF-metadata-leak~~ [ANCHOR-403] Prevent SSRF through SEP-1 TOML redirect Aug 10, 2023

philipliu requested a review from lijamie98 August 10, 2023 17:03

reecexlm removed the request for review from lijamie98 August 10, 2023 17:06

reecexlm merged commit 574e8c7 into main Aug 10, 2023

reecexlm deleted the ANCHOR-403-reece-SDP001-aws-metadata branch August 10, 2023 17:08

JakeUrban reviewed Aug 10, 2023

View reviewed changes

Ifropc reviewed Aug 10, 2023

View reviewed changes

reecexlm mentioned this pull request Aug 14, 2023

backmerge main to develop #1054

Closed

reecexlm added a commit that referenced this pull request Aug 14, 2023

Anchor 403 backmerge to develop (#1055)

e6b2c01

* [ANCHOR-403] Prevent SSRF through SEP-1 TOML redirect (#1043) * ANCHOR-403-reece-SSRF-metadata

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ANCHOR-403] Prevent SSRF through SEP-1 TOML redirect #1043

[ANCHOR-403] Prevent SSRF through SEP-1 TOML redirect #1043

reecexlm commented Aug 5, 2023 •

edited

Loading

stellar-jenkins commented Aug 5, 2023

stellar-jenkins commented Aug 6, 2023

stellar-jenkins commented Aug 6, 2023

stellar-jenkins commented Aug 8, 2023

stellar-jenkins commented Aug 9, 2023

stellar-jenkins commented Aug 9, 2023

stellar-jenkins commented Aug 9, 2023

stellar-jenkins commented Aug 9, 2023

stellar-jenkins commented Aug 9, 2023

stellar-jenkins commented Aug 9, 2023

stellar-jenkins commented Aug 9, 2023

stellar-jenkins commented Aug 9, 2023

stellar-jenkins commented Aug 10, 2023

stellar-jenkins commented Aug 10, 2023

philipliu left a comment

JakeUrban Aug 10, 2023

reecexlm Aug 10, 2023 •

edited

Loading

reecexlm Aug 10, 2023

Ifropc Aug 10, 2023

[ANCHOR-403] Prevent SSRF through SEP-1 TOML redirect #1043

[ANCHOR-403] Prevent SSRF through SEP-1 TOML redirect #1043

Conversation

reecexlm commented Aug 5, 2023 • edited Loading

PR Structure

Thoroughness

stellar-jenkins commented Aug 5, 2023

stellar-jenkins commented Aug 6, 2023

stellar-jenkins commented Aug 6, 2023

stellar-jenkins commented Aug 8, 2023

stellar-jenkins commented Aug 9, 2023

stellar-jenkins commented Aug 9, 2023

stellar-jenkins commented Aug 9, 2023

stellar-jenkins commented Aug 9, 2023

stellar-jenkins commented Aug 9, 2023

stellar-jenkins commented Aug 9, 2023

stellar-jenkins commented Aug 9, 2023

stellar-jenkins commented Aug 9, 2023

stellar-jenkins commented Aug 10, 2023

stellar-jenkins commented Aug 10, 2023

philipliu left a comment

Choose a reason for hiding this comment

JakeUrban Aug 10, 2023

Choose a reason for hiding this comment

reecexlm Aug 10, 2023 • edited Loading

Choose a reason for hiding this comment

reecexlm Aug 10, 2023

Choose a reason for hiding this comment

Ifropc Aug 10, 2023

Choose a reason for hiding this comment

reecexlm commented Aug 5, 2023 •

edited

Loading

reecexlm Aug 10, 2023 •

edited

Loading