-
Notifications
You must be signed in to change notification settings - Fork 364
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update the logging properties to opt-out of the prefix events #844 #845
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So far, no objections, but I need to think about the tests a bit more. In the meantime, @mickeyz07, perhaps you can respond to my comments.
configuration/esapi/ESAPI.properties
Outdated
@@ -407,6 +407,10 @@ Logger.UserInfo=true | |||
# Determines whether ESAPI should log the session id and client IP. | |||
Logger.ClientInfo=true | |||
|
|||
# Determines whether ESAPI should log the prefix of [EVENT_TYPE - APPLICATION NAME]. | |||
# If all above Logger entries are set to false and LogIgnorePrefix is true, then the output would be the same like if no ESAPI was used | |||
Logger.LogIgnorePrefix=true |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree with @jeremiahjstacey about the non-intuitive property name. But also we can't have this new property be set in a manner that is going to change the default log output as it has previously existed since ESAPI 2.0. As project co-lead, I'm flat out not going to allow that. So whatever the default is set to, the logs have to be formatted as they were before. The only times we break backward compatibility is to:
- If doing so is the only way to fix a security issue.
- Something behavior of a dependency forces us to do so.
- Without first deprecating something (generally for a 2 year period) before changing the behavior.
- When changing major # in semantic versioning (e.g., going from ESAPI 1.x to 2.x or 2.x to 3.x).
We've already learned that people do not read the release notes. Instead, they post GitHub issues, send the team private emails, post on Stack Overflow, etc. and we are left trying to provide answers. So, no, the default logging behavior is not going to change on my watch. If people like yourself want it changed, you will have to tweak the value in your application's private copy.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, VERY important note: We can't assume that people will update their ESAPI.properties file. They in fact often don't until something goes wrong. So be sure to test it to make sure that it behaves as it does in 2.5.4.0 and earlier even if the completely omit that new property file. We can't be changing behavior and more importantly, we don't want to throw a ConfigurationException if that property is missing.
@@ -111,6 +111,7 @@ public final class PropNames { | |||
public static final String LOG_ENCODING_REQUIRED = "Logger.LogEncodingRequired"; | |||
public static final String LOG_APPLICATION_NAME = "Logger.LogApplicationName"; | |||
public static final String LOG_SERVER_IP = "Logger.LogServerIP"; | |||
public static final String LOG_IGNORE_PREFIX = "Logger.LogIgnorePrefix"; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If the property name changes as per @jeremiahjstacey's suggestion, then please rename this variable in a corresponding fashion as well.
@@ -30,18 +30,27 @@ public class EventTypeLogSupplier // implements Supplier<String> | |||
{ | |||
/** EventType reference to supply log representation of. */ | |||
private final EventType eventType; | |||
/** Whether to log or not the event type */ | |||
private boolean ignoreLogEventType = false; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If you take @jeremiahjstacey's suggestion, maybe instead it makes more sense to call this keepLogEventType
(or even just logEventType
it that's not used and you don't think it would be confusing) and set it to true. Of course, you'd have to invert all your logic, but long term, it probably would result in more readability, especially if the property is renamed as @jeremiahjstacey recommends.
src/main/java/org/owasp/esapi/logging/appender/EventTypeLogSupplier.java
Show resolved
Hide resolved
src/main/java/org/owasp/esapi/logging/appender/LogPrefixAppender.java
Outdated
Show resolved
Hide resolved
configuration/esapi/ESAPI.properties
Outdated
@@ -407,6 +407,10 @@ Logger.UserInfo=true | |||
# Determines whether ESAPI should log the session id and client IP. | |||
Logger.ClientInfo=true | |||
|
|||
# Determines whether ESAPI should log the prefix of [EVENT_TYPE - APPLICATION NAME]. | |||
# If all above Logger entries are set to false and LogIgnorePrefix is true, then the output would be the same like if no ESAPI was used |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess I'm missing something here. Are you saying that for this new property to have the effect of omitting the event type, that ALL of these other Logger.*
property names from lines 400 through 408 inclusive MUST be false? That seems overly complicated to me.
src/main/java/org/owasp/esapi/logging/appender/ServerInfoSupplier.java
Outdated
Show resolved
Hide resolved
public EventTypeLogSupplierIgnoreEventTypeTest(Logger.EventType eventType, String result) { | ||
this.eventType = eventType; | ||
this.expectedResult = result; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Suggestion: Add blank line here for improved readability.
@@ -439,6 +439,10 @@ Logger.UserInfo=true | |||
# Determines whether ESAPI should log the session id and client IP. | |||
Logger.ClientInfo=true | |||
|
|||
# Determines whether ESAPI should log the prefix of [EVENT_TYPE - APPLICATION NAME]. | |||
# If all above Logger entries are set to false and LogIgnorePrefix is true, then the output would be the same like if no ESAPI was used | |||
Logger.LogIgnorePrefix=true |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Change this property name in accordance with @jeremiahjstacey's suggestion in configuration/esapi/ESAPI.properties.
src/main/java/org/owasp/esapi/logging/appender/EventTypeLogSupplier.java
Outdated
Show resolved
Hide resolved
src/main/java/org/owasp/esapi/logging/appender/LogPrefixAppender.java
Outdated
Show resolved
Hide resolved
src/main/java/org/owasp/esapi/logging/appender/LogPrefixAppender.java
Outdated
Show resolved
Hide resolved
src/main/java/org/owasp/esapi/logging/appender/ServerInfoSupplier.java
Outdated
Show resolved
Hide resolved
src/test/java/org/owasp/esapi/logging/appender/LogPrefixAppenderTest.java
Outdated
Show resolved
Hide resolved
@kwwall - This all looks good to me. if your concerns have been addressed please merge or leave a comment and I will follow up and get it merged this week |
@jeremiahjstacey - I will take care of merging this, but probably won't get to it by today though, as I had hoped. But I want to run a few dynamic tests first to make sure the defaults are all set so as to be backward compatible. I think they are, but it's a lot easier to try it than try to judge based on the code. I hope to have it merged by this weekend unless I find something that needs fixed |
@mickeyz07 - Sorry I didn't notice this before. Jeremiah and I were focused on the code review itself. However, I just noticed that you have not signed your commits with a GPG / PGP key. For commits that will be merged to the 'develop' or 'main' branches, we require those commits to be signed. From our CONTRIBUTING-TO-ESAPI.txt file:
There are detailed instructions on how to set this up in our ESAPI Release Steps document. Unfortunately, I seem to recall this before. GitHub doesn't warn us about this until after we have 2 people from the team approve the PR. I think it would then probably then force you to go back and set up signing and you'd have to made at least one commit to all that you've modified and then we'd have to get 2 of us to approve it again since there would be a commit after the approval. So, if you could, could you take care of that. I'm not sure if it would fix it though, because I think ti may require ALL commits to be signed, not just the last one in a PR. (@xeno6696 or @jeremiahjstacey - do either of you know?) I'm guessing it may not have flagged this because you forked it and created it on your 'issue-844' branch (which, was as instructed). I think there is an option where I can set up our repo to require signed commits on ALL branches, but I didn't do that because I figured it my break things like Dependabot and Snyk generated PRs. Anyhow, can you see if you can somehow fix this? Thanks. |
There's no reason we couldn't have a "signature" commit, just a simple merge with a text change detailing it as a signature. Just to get it into history and I think that ought to be sufficient. Going back and resigning old commits is more trouble than its worth.
…________________________________
From: Kevin W. Wall ***@***.***>
Sent: Saturday, July 6, 2024 7:38 AM
To: ESAPI/esapi-java-legacy ***@***.***>
Cc: Matt Seil ***@***.***>; Mention ***@***.***>
Subject: Re: [ESAPI/esapi-java-legacy] Update the logging properties to opt-out of the prefix events #844 (PR #845)
@mickeyz07<https://github.com/mickeyz07> - Sorry I didn't notice this before. Jeremiah and I were focused on the code review itself. However, I just noticed that you have not signed your commits with a GPG / PGP key. For commits that will be merged to the 'develop' or 'main' branches, we require those commits to be signed.
From our CONTRIBUTING-TO-ESAPI.txt file:
A Special Note Regarding Making Commits for PRs
Shortly after the 2.5.1.0 ESAPI release in late November 2022, the ESAPI
team decided to lock down the 'develop' amd 'main' branches. Merges from
PRs are done to the 'develop' branch. That means that if you intend to
contribute to ESAPI, you must be signing your commits. Please see the
GitHub instructions at
https://docs.github.com/en/authentication/managing-commit-signature-verification/signing-commits
for details.
There are detailed instructions on how to set this up in our ESAPI Release Steps document.<https://github.com/ESAPI/esapi-java-legacy/blob/develop/documentation/ESAPI-release-steps.pdf> Unfortunately, I seem to recall this before. GitHub doesn't warn us about this until after we have 2 people from the team approve the PR. I think it would then probably then force you to go back and set up signing and you'd have to made at least one commit to all that you've modified and then we'd have to get 2 of us to approve it again since there would be a commit after the approval.
So, if you could, could you take care of that. I'm not sure if it would fix it though, because I think ti may require ALL commits to be signed, not just the last one in a PR. ***@***.***<https://github.com/xeno6696> or @jeremiahjstacey<https://github.com/jeremiahjstacey> - do either of you know?)
I'm guessing it may not have flagged this because you forked it and created it on your 'issue-844' branch (which, was as instructed). I think there is an option where I can set up our repo to require signed commits on ALL branches, but I didn't do that because I figured it my break things like Dependabot and Snyk generated PRs.
Anyhow, can you see if you can somehow fix this? Thanks.
—
Reply to this email directly, view it on GitHub<#845 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ACIQAQL6ZLKRZTPRV43QNTTZK76P5AVCNFSM6AAAAABJ36H722VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMJRG44DKMRVGU>.
You are receiving this because you were mentioned.Message ID: ***@***.***>
|
@xeno6696 - That's sort of what I was thinking of having him do, but I wasn't sure if that would pass the muster of the GitHub signature verification. I thought it required every single commit to be signed if it was to end up on the 'develop' or 'main' branch. That said, maybe it would work if we did a squash-merge? I don't know. |
So, @mickeyz07 found a severe must fix issue in your code. I made this simple change to simulate users NOT updating their ESAPI.properties file (which I can guarantee you that many will do): $ git diff src/test/resources/esapi/ESAPI.properties
diff --git a/src/test/resources/esapi/ESAPI.properties b/src/test/resources/esapi/ESAPI.properties
index 8ffc61f6..a84328ab 100644
--- a/src/test/resources/esapi/ESAPI.properties
+++ b/src/test/resources/esapi/ESAPI.properties
@@ -441,7 +441,7 @@ Logger.ClientInfo=true
# Determines whether ESAPI should log the prefix of [EVENT_TYPE - APPLICATION NAME].
# If all above Logger entries are set to false, as well as LogPrefix, then the output would be the same as if no ESAPI was used
-Logger.LogPrefix=true
+# Logger.LogPrefix=true
#===========================================================================
# ESAPI Intrusion Detection And when I re-ran 'mvn test' I got this:
A similar error happens if one sets the value of Logger.LogPrefix=off or even something absurd as: Logger.LogPrefix=scoopy_do you have to treat is as though they had: Logger.LogPrefix=true It's okay if you want to output an error to stdout using Make sense? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changes discussed in #845 (comment) are required before this will be approved.
… fifth iteration
@kwwall @jeremiahjstacey I have made changes and signed the commit with GPG key. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not going to approve this unless Matt or Jeremiah find this kludge acceptable.
String property = properties.getProperty( propertyName ); | ||
if ( property == null ) { | ||
if (propertyName.startsWith("Logger.")) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@mickeyz07 -- IMO, this is a major implementation kludge to me. Generally, I was expecting in whatever static initializer that checks the value for the Logger.LogPrefix
property, that any error / exception that would be thrown that you would just treat it as 'true' and then log something via logSpecial
. If for some reason that approach turns out to be impossible, then the next best alternative would be to create a special dedicated getLogPrefix method. Another preferred alternative (something I've been meaning to do for a while now) is to create a new Boolean getBooleanProp(String propName, boolean default)
and use that, and just pass in 'true' for the default, so if the specified property name is not a boolean, it defaults to the specified default value (which should be true for Logger.LogPrefix.
I am also very confused why this is even written the way that it it; it appears that you could omit the entire:
if (propertyName.startsWith("Logger.")) {
and just went with the specific property name check that follows. (And why this specific check? I would have expected Logger.LogPrefix
here, since that's the new property. So why Logger.LogEncodingRequired
? I'm confused.)
@jeremiahjstacey and @xeno6696 - I'd like your opinion on this, but I don't like this at all. Can either of you think of any alternatives?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think I agree @kwwall - Updates in this file will lead to additional scope that I think we'll be better off not trying to tackle in this effort.
@mickeyz07 - My understanding of what the desire here is to try to solve the problem at the lowest level possible. I think I agree with the intent, but I think you can agree that it's not the cleanest or most maintainable due to current restrictions in the library's interfaces.
To facilitate a faster integration of this feedback, I would like to propose the following:
- Revert all changes to DefaultSecurityConfiguration.java
- Update JavaLogFactory and SLF4JLogFactory implementations to default to TRUE, attempt to read from properties, and useLogSpecial on failure
boolean logPrefix = true;
try {
logPrefix = ESAPI.securityConfiguration().getBooleanProp(LOG_PREFIX);
} catch (ConfigurationException ex) {
ESAPI.securityConfiguration().logSpecial("Failed to read Log Prefix configuration. Defaulting to enabled. ", ex);
}
It's not the best possible solution, but I think that's the best we can do in the current library implementation.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Okay, my thoughts, exactly, and what you described was my preferred approach at resolving this.
@mickeyz07 - Please make the change that @jeremiahjstacey detailed above. Thanks.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changes requested. See previous comments by Jeremiah and myself. It should be pretty straightforward since he pretty much laid out the code changes.
@kwwall @jeremiahjstacey I have added the new method:
|
@mickeyz07 We are getting close, so please don't give up. But there are 2 things that I see missing right out of the gate:
|
… seventh iteration
@kwwall I have added the required tests for all 3 configuration test files you noted. Please check. |
@@ -33,6 +33,13 @@ public interface EsapiPropertyLoader { | |||
*/ | |||
public Boolean getBooleanProp(String propertyName) throws ConfigurationException; | |||
|
|||
/** | |||
* Get any Boolean type property from security configuration. | |||
* If property does not exist in configuration or has incorrect type, defaultValue is returned |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
From the Javadoc: for this interface: "or has incorrect type".
Well, that is the $64k question, isn't it? What is the INTUITIVE behavior if one calls something like:
Boolean propValue = getBooleanProp("SomePropName", true)
but somewhere, say in ESAPI.properties or elsewhere, we have defined
SomePropName=This is not a boolean property
Then what is the expected / intuitive behavior for that call? I think that I'd expect a ConfigurationException
to be thrown rather than having propValue
silently being set to true
. Or maybe I shouldn't say that is the intuitive behavior, but rather the safe behavior. I think because copy/paste errors abound and it would be easy to make a typo with the property name, that's the only way that we're going to help people catch these bizarre accidental edge cases. Otherwise, we're going to cause them to curse at us because they'll have to figure up a debugger to figure out what the hell is going on in cases like this.
@xeno6696 and @jeremiahjstacey - I'd like your opinion here. I know this requires yet another code change, but if we do a release and document it this way this (likely bug, IMO) will become a feature because changing it later on would break backward compatibility. If we were requiring JDK 9 or later we could just not export this method and keep it hidden, but since Java 8 is the minimal version, once we add this, it becomes available to the world.
I personally am leaning towards only using the default value if the property is not set at all; that is if
System.getProperty("SomePropValue");
returns null
, but I'd like to know what the 2 of you think. If you both think it's more intuitive have it return the default boolean value if the property name is found to have a non-Boolean value I will reluctantly go along with that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall, I think this is more scope than we should put time into to modify the opt-out request from the original issue. I appreciate the initiative; however, we're adding this case to expand for something that could have been solved in two copies of an 6 line block.
This is overengineering in this scope. We're blocking value, and a new possible release for this API change.
My suggestion is to take it out. Use the duplicate snippet from my previous comment in the 2 factories and revert the changes in the DefaultSecurityConfiguration. Create another task for the default API that we can address (reuse this code if you want), and complete it in a non-blocking scope. We don't need this to fix the original problem, and as you can see the discussion of edge cases and caveats will continue to grow. Let's get the opt-out behavior in in a first form, then improve upon it in another task to expand the SecurityConfiguration API to allow defaults to be specified.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I concur. If in the future, you want to do a new PR to get a new interface for
public Boolean getBooleanProp(String propertyName, Boolean defaultValue) throws ConfigurationException
I am fine with that, but as my review of this as surfaced, this is a little more nuanced than intuition alone might have us believe. If we are going to add that a future time, I think we need to start out with a GitHub issue and discuss the specifications there first, before waiting for an implementation to discuss them. So, to proceed with this PR, please rip that part of the code out and follow Jeremiah's suggestion.
@Override | ||
public Boolean getBooleanProp(String propertyName, Boolean defaultValue) { | ||
for (AbstractPrioritizedPropertyLoader loader : loaders) { | ||
return loader.getBooleanProp(propertyName, defaultValue); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the way this is currently implemented, I don't think it will ever reach the final return on line 87 here. Maybe I'm wrong about that. Do we have a test for that case? Of course, if we change it as per my previous comment, we may still need it. Or may need to add a try / catch here and just re-throw any ConfigurationException
.
if (property.equalsIgnoreCase("false") || property.equalsIgnoreCase("no")) { | ||
return false; | ||
} | ||
return defaultValue; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Depending on if we decide to change the behavior here in accordance to my first comment, we may wish to change line 88 to something along the following lines:
if ( property != null ) {
throw new ConfigurationException("Property name '" + propertyName + "' has non-Boolean value of '" + property + "'.");
}
if (property.equalsIgnoreCase("false") || property.equalsIgnoreCase("no")) { | ||
return false; | ||
} | ||
return defaultValue; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ditto here. Similar to comment on StandardEsapiPropertyLoader.java.
if ( property.equalsIgnoreCase("false") || property.equalsIgnoreCase("no") ) { | ||
return false; | ||
} | ||
return defaultValue; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ditto comment in EsapiPropertyManager.java about throwing ConfigurationException
here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@mickeyz07 - I'm holding off on requesting changes until I get @xeno6696 and @jeremiahjstacey feedback about how getBooleanProp(String, Boolean) should treat a property that it encounters that has a non-Boolean value. I think we need to resolve that first. If we decide to do that, I will give you the option to finish it up, but if you don't, I'll probably approve this and then do those final changes myself in a separate PR. But getting this behavior of this new method is important and we'll likely only get one chance at getting it right.
@xeno6696 and @jeremiahjstacey -- waiting to hear back from you on this.
// then | ||
assertEquals(expectedPropertyValue, propertyValue); | ||
} | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we decide to revise this behavior to throw a ConfigurationException
if getBooleanProperty(String, Boolean)
is called and a non-Boolean property if found, we will need a new test here.
Ditto in StandardEsapiPropertyLoaderTest.java and XmlEsapiProertperLoaderTest.java.
… eigth iteration
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Two lines where I'd like to see some very minor tweaks, otherwise LGTM.
try { | ||
logPrefix = ESAPI.securityConfiguration().getBooleanProp(LOG_PREFIX); | ||
} catch (ConfigurationException ex) { | ||
System.out.println("ESAPI: Failed to read Log Prefix configuration. Defaulting to enabled" + |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please change the output to something that explicitly mentions the property name on the output sent to stderr. E.g., something like this:
System.out.println("ESAPI: Failed to read Log Prefix configuration, " + LOG_PREFIX + ". Defaulting to enabled" +
That will make it easier for those getting this failure to know exactly what property to look for if they wish to change it in their ESAPI.properties file.
try { | ||
logPrefix = ESAPI.securityConfiguration().getBooleanProp(LOG_PREFIX); | ||
} catch (ConfigurationException ex) { | ||
System.out.println("ESAPI: Failed to read Log Prefix configuration. Defaulting to enabled" + |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same as previous comment.
… ninth iteration
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me as well
I'm going to go ahead and bypass the 'branch protection' and do the 'squash and merge' because @mickeyz07 signed most of his commits and we reviewed all the code, so someone usurped his GitHub username and snuck something malicious past us that's one us. |
Determines whether ESAPI should log the prefix of [EVENT_TYPE - APPLICATION NAME].
If all Logger.* settings in the ESAPI.properties file are false (including Logger.LogPrefix), then the log output would be the same as if no ESAPI was used.
That can be important, especially for clients that are using cloud providers. The charges for extra logging can quickly add up.
So, the user can sanitise untrusted data used to construct log entries by using a safe logging mechanism in the OWASP ESAPI Logger without any extra logging information.
However, only by changing a couple of settings in the ESAPI.properties file, a user can have a very detailed log if required.