add MDC for sqlFile used in a rollback (DAT-13420) (#3843)

* add MDC interfaces and base classes (DAT-12601) (#3567) Create the basic APIs for MDC in support of structured logging. * API changes to support structured logging (DAT-12652) (#3600) Primarily, this PR allows the JavaLogService to supply its own custom formatter class that is used on the handler for the logging library. * add configuration option for log format (DAT-12727) (#3644) Add a log format configuration option. * Add mdc to update command (DAT-12602) (#3647) * add MDC interfaces and base classes (DAT-12601) (#3567) Create the basic APIs for MDC in support of structured logging. * set log format in log service * add deprecated annotation * Revert "set log format in log service" This reverts commit 096b067. * use log formatter for JSON logging * add MDC to structured log output * fix test * Include "path" in databasechangelog's description column for all change types with "path" attributes (#3244) Extends the DESCRIPTION column of the DATABASECHANGELOG for sqlFile by the path specification, for example sqlFile path=IUNT200.sql Co-authored-by: michaelmatthiaskern <[email protected]> Co-authored-by: Nathan Voxland <[email protected]> * #1466: Add ignore:true changeset attribute to Formatted SQL changeLogs (#3377) Co-authored-by: skrivenko <[email protected]> Co-authored-by: filipe lautert <[email protected]> * [DAT-12602] Initial update mdc implementation. * add configuration option for log format * [DAT-12602] Adds remaining mdc values. Reworks to use static strings over enum. * [DAT-12602] Remove unused keys, fixed formatting. * [DAT-12602] Adds system information to mdc. * [DAT-12602] Fixes formatting for mdckey. * [DAT-12602] Simplifies getting mdc for command name. * updated logFormat description * warn about invalid values for log format * [DAT-12602] Add changeset outcome to info log, fix deployment outcome to actually show in info. * Revert "Merge remote-tracking branch 'origin/DAT-11693' into DAT-12602" This reverts commit 6e9163d, reversing changes made to 1ceec55. * Revert "Merge remote-tracking branch 'origin/DAT-12727' into DAT-12602" This reverts commit 8554e29, reversing changes made to 10d540b. * Revert "Revert "Merge remote-tracking branch 'origin/DAT-11693' into DAT-12602"" This reverts commit 10d540b. * Revert "Revert "Merge remote-tracking branch 'origin/DAT-12727' into DAT-12602"" This reverts commit 12f6539. * [DAT-12602] Fix liquibase system information mdc. * [DAT-12602] Fix logged liquibase version. * Revert change to build.properties * safely cast logformat - fixes using the default value * [DAT-12602] Adds mdc for failed changeset outcome. Co-authored-by: Steven Massaro <[email protected]> Co-authored-by: MichaelKern-IVV <[email protected]> Co-authored-by: michaelmatthiaskern <[email protected]> Co-authored-by: Nathan Voxland <[email protected]> Co-authored-by: skrivenko <[email protected]> Co-authored-by: skrivenko <[email protected]> Co-authored-by: filipe lautert <[email protected]> * Updates MdcKeys (DAT-12915) (#3679) * add MDC interfaces and base classes (DAT-12601) (#3567) Create the basic APIs for MDC in support of structured logging. * set log format in log service * add deprecated annotation * Revert "set log format in log service" This reverts commit 096b067. * use log formatter for JSON logging * add MDC to structured log output * fix test * Include "path" in databasechangelog's description column for all change types with "path" attributes (#3244) Extends the DESCRIPTION column of the DATABASECHANGELOG for sqlFile by the path specification, for example sqlFile path=IUNT200.sql Co-authored-by: michaelmatthiaskern <[email protected]> Co-authored-by: Nathan Voxland <[email protected]> * #1466: Add ignore:true changeset attribute to Formatted SQL changeLogs (#3377) Co-authored-by: skrivenko <[email protected]> Co-authored-by: filipe lautert <[email protected]> * [DAT-12602] Initial update mdc implementation. * add configuration option for log format * [DAT-12602] Adds remaining mdc values. Reworks to use static strings over enum. * [DAT-12602] Remove unused keys, fixed formatting. * [DAT-12602] Adds system information to mdc. * [DAT-12602] Fixes formatting for mdckey. * [DAT-12602] Simplifies getting mdc for command name. * updated logFormat description * warn about invalid values for log format * [DAT-12602] Add changeset outcome to info log, fix deployment outcome to actually show in info. * Revert "Merge remote-tracking branch 'origin/DAT-11693' into DAT-12602" This reverts commit 6e9163d, reversing changes made to 1ceec55. * Revert "Merge remote-tracking branch 'origin/DAT-12727' into DAT-12602" This reverts commit 8554e29, reversing changes made to 10d540b. * Revert "Revert "Merge remote-tracking branch 'origin/DAT-11693' into DAT-12602"" This reverts commit 10d540b. * Revert "Revert "Merge remote-tracking branch 'origin/DAT-12727' into DAT-12602"" This reverts commit 12f6539. * [DAT-12602] Fix liquibase system information mdc. * [DAT-12602] Fix logged liquibase version. * [DAT-12915] Replaces liquibaseTargetType/liquibaseTargetUrl with liquibaseTargetUrl and unused liquibaseRefUrl. * Revert change to build.properties * safely cast logformat - fixes using the default value * [DAT-12602] Adds mdc for failed changeset outcome. Co-authored-by: Steven Massaro <[email protected]> Co-authored-by: MichaelKern-IVV <[email protected]> Co-authored-by: michaelmatthiaskern <[email protected]> Co-authored-by: Nathan Voxland <[email protected]> Co-authored-by: skrivenko <[email protected]> Co-authored-by: skrivenko <[email protected]> Co-authored-by: filipe lautert <[email protected]> * add pretty-printed-json format for log output (DAT-13000) (#3714) Add an option to print structured logs in JSON format, without minifying them. * add MDC for rollback command (DAT-12698) (#3732) * add changelog properties to update MDC (DAT-12972) (#3757) * Adds deploymentOutcomeCount to MDC (DAT-12979) (#3766) * [DAT-12979] Adds deploymentOutcomeCount mdc. * [DAT-12979] Handles when hub is updating. * [DAT-12979] Reworks usage of hubChangeExecListener combined with defaultChangeExecutor. * [DAT-12979] Revert changes to HubChangeExecListener * reload the log service after registering value providers (DAT-12854) (#3771) * initial fix * remove unnecessary public static variable * Change exception message level for change set to severe DAT-12984 (#3739) * Include deploymentId and liquibaseTargetUrl mdc values during exception log (DAT-13012) (#3747) * move existing MDC logging to separate method * refactor adding outcomes to MDC and immediate removal * add MDC for rollback * [DAT-13012] Persist deploymentId and liquibaseTargetUrl mdc values during exception log. This change will also include any other mdc values that have not been manually removed from scope if an exception is encountered. --------- Co-authored-by: Steven Massaro <[email protected]> * MDC for rollback-count command, support custom objects in MDC output (DAT-13023) (#3774) Add MDC for rollback-count command. Allow custom objects in MDC output. * add MDC for rollback-one-changeset (DAT-13024) (#3782) * add MDC for rollback-one-update (DAT-13025) (#3792) * Add changesetComment, changesetLabel mdc values (DAT-12919) (#3773) * [DAT-12919] Adds changeset labels, changeset comments to mdc. * [DAT-12919] Handles null comments/labels. * Adds deploymentOutcome mdc when change fails validation (DAT-12977) (#3744) * [DAT-12977] Adds deploymentOutcome: fail to mdc when change fails validation due to checksum error (or other error). * Trigger build * [DAT-12977] Fixes multiple instances of deploymentOutcome when change fails validation. * Adds contexts, labels, contextFilters and labelFilters to mdc (DAT-12958) (#3789) * [DAT-12958] Adds changesetLabelFilter, changesetContextFilter mdcs keys. * [DAT-12958] Adds label filter/context filter to update to tag. * [DAT-12958] Adds mdc for updateCount, updateToTag. * [DAT-12958] Adds method for setting up labelfilter/contextfilter. * [DAT-12958] Renames filter keys, refactors to DRY for deploymentOutcome and deploymentOutcomeCount. * [DAT-12958] Refactors deploymentId generation. --------- Co-authored-by: Steven Massaro <[email protected]> * MDC code to add labels and contexts DAT-13146 (#3795) MDC code to add labels and contexts DAT-13146 * Remove log message that caused duplicate structured log messages DAT-12961 (#3781) * Standardize commandLabelFilter/commandContextFilter mdc value format (DAT-12958) (#3801) * [DAT-12958] Formats all empty filters from "()" to "" in mdc. * [DAT-12958] Reworks to rely on originalString not just checking if it matches empty parenths. * add MDC for rollback-to-date (DAT-13027) (#3796) * add MDC for rollback-to-date * code cleanup and deduplication * DAT-13145: set log level to severe if using a log file with no log level set (#3805) * hide log format parameter (DAT-13501) (#3822) hide log format parameter * Change show summary default DAT-13497 (#3818) * Add labels/context information to rollback operations DAT-13146 * Set default for show summary to OFF DAT-13147 * Adds databaseChangelogSQL mdc key (DAT-13092) (#3810) * [DAT-13092] Adds databaseChangelogSQL mdc key, populates via StandardChangelogHistoryService and StandardLockService. Moves changesetSql mdc population into Changeset class. * [DAT-13092] Fixes lock test. * [DAT-13092] Handles rollback and update statements. * [DAT-13092] Fixes logic to only generate rollback statements when actually applicable. * [DAT-13092] Only add sql to mdc if it is supported. * [DAT-13092] Adds javadocs to new util method. * [DAT-13092] Adds javadocs * Adds databaseChangelogTableOutcome mdc (DAT-12978) (#3815) * [DAT-12978] Log databaseChangelogTableOutcome mdc. * [DAT-12978] Fix test. * add MDC for sqlFile used in a rollback * add javadoc to RollbackSqlFile * fix NPE * fix test failure --------- Co-authored-by: Alex <[email protected]> Co-authored-by: MichaelKern-IVV <[email protected]> Co-authored-by: michaelmatthiaskern <[email protected]> Co-authored-by: Nathan Voxland <[email protected]> Co-authored-by: skrivenko <[email protected]> Co-authored-by: skrivenko <[email protected]> Co-authored-by: filipe lautert <[email protected]> Co-authored-by: Wesley Willard <[email protected]> Co-authored-by: Wesley willard <[email protected]>
liquibase · Feb 23, 2023 · 393fd7a · 393fd7a
1 parent 06673b6
commit 393fd7a
Show file tree

Hide file tree

Showing 4 changed files with 100 additions and 0 deletions.
diff --git a/liquibase-core/src/main/java/liquibase/Liquibase.java b/liquibase-core/src/main/java/liquibase/Liquibase.java
@@ -1255,6 +1255,7 @@ public void run() throws Exception {
                             new DbmsChangeSetFilter(database),
                             new IgnoreChangeSetFilter(),
                             new CountChangeSetFilter(changesToRollback));
+
                     doRollback(bufferLog, rollbackScript, logIterator, contexts, labelExpression, hubUpdater, rollbackOperation);
                 }
                 catch (Throwable t) {
@@ -1539,6 +1540,7 @@ public void run() throws Exception {
                             new LabelChangeSetFilter(labelExpression),
                             new IgnoreChangeSetFilter(),
                             new DbmsChangeSetFilter(database));
+
                     doRollback(bufferLog, rollbackScript, logIterator, contexts, labelExpression, hubUpdater, rollbackOperation);
                 }
                 catch (Throwable t) {
@@ -1688,6 +1690,7 @@ public void run() throws Exception {
                             new LabelChangeSetFilter(labelExpression),
                             new IgnoreChangeSetFilter(),
                             new DbmsChangeSetFilter(database));
+
                     doRollback(bufferLog, rollbackScript, logIterator, contexts, labelExpression, hubUpdater, rollbackOperation);
                 }
                 catch (Throwable t) {

diff --git a/liquibase-core/src/main/java/liquibase/changelog/ChangeSet.java b/liquibase-core/src/main/java/liquibase/changelog/ChangeSet.java
@@ -6,6 +6,7 @@
 import liquibase.change.*;
 import liquibase.change.core.EmptyChange;
 import liquibase.change.core.RawSQLChange;
+import liquibase.change.core.SQLFileChange;
 import liquibase.changelog.visitor.ChangeExecListener;
 import liquibase.configuration.LiquibaseConfiguration;
 import liquibase.database.Database;
@@ -17,6 +18,7 @@
 import liquibase.executor.LoggingExecutor;
 import liquibase.logging.Logger;
 import liquibase.logging.mdc.MdcKey;
+import liquibase.logging.mdc.customobjects.RollbackSqlFile;
 import liquibase.parser.ChangeLogParserConfiguration;
 import liquibase.parser.core.ParsedNode;
 import liquibase.parser.core.ParsedNodeException;
@@ -843,6 +845,9 @@ public void rollback(Database database, ChangeExecListener listener) throws Roll
                     //
                     SqlStatement[] changeStatements = change.generateStatements(database);
                     addSqlMdc(change, database, false);
+                    if (change instanceof SQLFileChange) {
+                        addSqlFileMdc((SQLFileChange) change);
+                    }
                     if (changeStatements != null) {
                         statements.addAll(Arrays.asList(changeStatements));
                     }
@@ -896,6 +901,11 @@ public void rollback(Database database, ChangeExecListener listener) throws Roll
 
     }
 
+    private void addSqlFileMdc(SQLFileChange change) {
+        RollbackSqlFile rollbackSqlFile = new RollbackSqlFile(change);
+        Scope.getCurrentScope().addMdcValue(MdcKey.ROLLBACK_SQL_FILE, rollbackSqlFile);
+    }
+
     private boolean ignoreSpecificChangeTypes(Change change, Database database) {
         return ((change instanceof DbmsTargetedChange) && !DatabaseList.definitionMatches(((DbmsTargetedChange) change).getDbms(), database, true))
              || ((change instanceof RawSQLChange) && "empty".equalsIgnoreCase(((RawSQLChange)change).getSql()));

diff --git a/liquibase-core/src/main/java/liquibase/logging/mdc/MdcKey.java b/liquibase-core/src/main/java/liquibase/logging/mdc/MdcKey.java
@@ -34,4 +34,5 @@ public class MdcKey {
     public static final String ROLLBACK_TO_DATE = "rollbackToDate";
     public static final String DATABASE_CHANGELOG_SQL = "databaseChangelogSQL";
     public static final String DATABASE_CHANGELOG_OUTCOME = "databaseChangelogTableOutcome";
+    public static final String ROLLBACK_SQL_FILE = "rollbackSqlFile";
 }
diff --git a/liquibase-core/src/main/java/liquibase/logging/mdc/customobjects/RollbackSqlFile.java b/liquibase-core/src/main/java/liquibase/logging/mdc/customobjects/RollbackSqlFile.java
@@ -0,0 +1,86 @@
+package liquibase.logging.mdc.customobjects;
+
+import liquibase.change.core.SQLFileChange;
+import liquibase.logging.mdc.CustomMdcObject;
+
+/**
+ * This class is used to represent MDC data related to a sqlFile type change inside a rollback.
+ */
+public class RollbackSqlFile implements CustomMdcObject {
+    private String dbms;
+    private String encoding;
+    private String endDelimiter;
+    private String path;
+    private Boolean relativeToChangelogFile;
+    private Boolean splitStatements;
+    private Boolean stripComments;
+
+    public RollbackSqlFile() {
+    }
+
+    public RollbackSqlFile(SQLFileChange change) {
+        this.dbms = change.getDbms();
+        this.endDelimiter = change.getEndDelimiter();
+        this.splitStatements = change.isSplitStatements();
+        this.stripComments = change.isStripComments();
+        this.encoding = change.getEncoding();
+        this.path = change.getPath();
+        this.relativeToChangelogFile = change.isRelativeToChangelogFile();
+    }
+
+    public String getDbms() {
+        return dbms;
+    }
+
+    public void setDbms(String dbms) {
+        this.dbms = dbms;
+    }
+
+    public String getEncoding() {
+        return encoding;
+    }
+
+    public void setEncoding(String encoding) {
+        this.encoding = encoding;
+    }
+
+    public String getEndDelimiter() {
+        return endDelimiter;
+    }
+
+    public void setEndDelimiter(String endDelimiter) {
+        this.endDelimiter = endDelimiter;
+    }
+
+    public String getPath() {
+        return path;
+    }
+
+    public void setPath(String path) {
+        this.path = path;
+    }
+
+    public Boolean getRelativeToChangelogFile() {
+        return relativeToChangelogFile;
+    }
+
+    public void setRelativeToChangelogFile(Boolean relativeToChangelogFile) {
+        this.relativeToChangelogFile = relativeToChangelogFile;
+    }
+
+    public Boolean getSplitStatements() {
+        return splitStatements;
+    }
+
+    public void setSplitStatements(Boolean splitStatements) {
+        this.splitStatements = splitStatements;
+    }
+
+    public Boolean getStripComments() {
+        return stripComments;
+    }
+
+    public void setStripComments(Boolean stripComments) {
+        this.stripComments = stripComments;
+    }
+}