Toggle formatting inline with spotless:off and spotless:on

CHANGES.md

@@ -10,6 +10,8 @@ This document is intended for Spotless developers.
 We adhere to the [keepachangelog](https://keepachangelog.com/en/1.0.0/) format (starting after version `1.27.0`).
 
 ## [Unreleased]
+### Added
+* `PipeStepPair` which allows extracting blocks of text in one step, then injecting those blocks back in later. Currently only used for `spotless:off` `spotless:on`, but could also be used to [apply different steps in different places](https://github.com/diffplug/spotless/issues/412) ([#691](https://github.com/diffplug/spotless/pull/691)).
 ### Changed
 * When applying license headers for the first time, we are now more lenient about parsing existing years from the header ([#690](https://github.com/diffplug/spotless/pull/690)).
 

README.md

@@ -39,6 +39,7 @@ output = [
 '| --------------------------------------------- | ------------- | ------------ | ------------ | --------|',
 '| Automatic [idempotency safeguard](PADDEDCELL.md) | {{yes}}    | {{yes}}      | {{yes}}      | {{no}}  |',
 '| Misconfigured [encoding safeguard](https://github.com/diffplug/spotless/blob/08340a11566cdf56ecf50dbd4d557ed84a70a502/testlib/src/test/java/com/diffplug/spotless/EncodingErrorMsgTest.java#L34-L38) | {{yes}}    | {{yes}}      | {{yes}}       | {{no}}  |',
+'| Toggle with [`spotless:off` and `spotless:on`](plugin-gradle/#spotlessoff-and-spotlesson) | {{yes}}    | {{yes}}      | {{no}}       | {{no}}  |',
 '| [Ratchet from](https://github.com/diffplug/spotless/tree/main/plugin-gradle#ratchet) `origin/main` or other git ref | {{yes}}    | {{yes}}      | {{no}}       | {{no}}  |',
 '| Define [line endings using git](https://github.com/diffplug/spotless/tree/main/plugin-gradle#line-endings-and-encodings-invisible-stuff) | {{yes}}    | {{yes}}      | {{yes}}       | {{no}}  |',
 '| Fast incremental format and up-to-date check   | {{yes}}      | {{no}}       | {{no}}       | {{no}}  |',
@@ -73,6 +74,7 @@ extra('wtp.EclipseWtpFormatterStep')             +'{{yes}}       | {{yes}}
 | --------------------------------------------- | ------------- | ------------ | ------------ | --------|
 | Automatic [idempotency safeguard](PADDEDCELL.md) | :+1:    | :+1:      | :+1:      | :white_large_square:  |
 | Misconfigured [encoding safeguard](https://github.com/diffplug/spotless/blob/08340a11566cdf56ecf50dbd4d557ed84a70a502/testlib/src/test/java/com/diffplug/spotless/EncodingErrorMsgTest.java#L34-L38) | :+1:    | :+1:      | :+1:       | :white_large_square:  |
+| Toggle with [`spotless:off` and `spotless:on`](plugin-gradle/#spotlessoff-and-spotlesson) | :+1:    | :+1:      | :white_large_square:       | :white_large_square:  |
 | [Ratchet from](https://github.com/diffplug/spotless/tree/main/plugin-gradle#ratchet) `origin/main` or other git ref | :+1:    | :+1:      | :white_large_square:       | :white_large_square:  |
 | Define [line endings using git](https://github.com/diffplug/spotless/tree/main/plugin-gradle#line-endings-and-encodings-invisible-stuff) | :+1:    | :+1:      | :+1:       | :white_large_square:  |
 | Fast incremental format and up-to-date check   | :+1:      | :white_large_square:       | :white_large_square:       | :white_large_square:  |

lib/src/main/java/com/diffplug/spotless/generic/PipeStepPair.java

@@ -0,0 +1,157 @@
+/*
+ * Copyright 2020 DiffPlug
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.diffplug.spotless.generic;
+
+import java.io.Serializable;
+import java.util.ArrayList;
+import java.util.Objects;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+import com.diffplug.spotless.FormatterStep;
+
+import edu.umd.cs.findbugs.annotations.SuppressFBWarnings;
+
+public class PipeStepPair {
+	/** The two steps will be named `<name>In` and `<name>Out`. */
+	public static Builder named(String name) {
+		return new Builder(name);
+	}
+
+	public static String defaultToggleName() {
+		return "toggle";
+	}
+
+	public static String defaultToggleOff() {
+		return "spotless:off";
+	}
+
+	public static String defaultToggleOn() {
+		return "spotless:on";
+	}
+
+	public static class Builder {
+		String name;
+		Pattern regex;
+
+		private Builder(String name) {
+			this.name = Objects.requireNonNull(name);
+		}
+
+		/** Defines the opening and closing markers. */
+		public Builder openClose(String open, String close) {
+			return regex(Pattern.quote(open) + "([\\s\\S]*?)" + Pattern.quote(close));
+		}
+
+		/** Defines the pipe via regex. Must have *exactly one* capturing group. */
+		public Builder regex(String regex) {
+			return regex(Pattern.compile(regex));
+		}
+
+		/** Defines the pipe via regex. Must have *exactly one* capturing group. */
+		public Builder regex(Pattern regex) {
+			this.regex = regex;
+			return this;
+		}
+
+		public PipeStepPair buildPair() {
+			return new PipeStepPair(name, regex);
+		}
+	}
+
+	final FormatterStep in, out;
+
+	private PipeStepPair(String name, Pattern pattern) {
+		StateIn stateIn = new StateIn(pattern);
+		StateOut stateOut = new StateOut(stateIn);
+		in = FormatterStep.create(name + "In", stateIn, state -> state::format);
+		out = FormatterStep.create(name + "Out", stateOut, state -> state::format);
+	}
+
+	public FormatterStep in() {
+		return in;
+	}
+
+	public FormatterStep out() {
+		return out;
+	}
+
+	@SuppressFBWarnings("SE_TRANSIENT_FIELD_NOT_RESTORED")
+	static class StateIn implements Serializable {
+		private static final long serialVersionUID = -844178006407733370L;
+
+		final Pattern regex;
+
+		public StateIn(Pattern regex) {
+			this.regex = regex;
+		}
+
+		final transient ArrayList<String> groups = new ArrayList<>();
+
+		private String format(String unix) {
+			groups.clear();
+			Matcher matcher = regex.matcher(unix);
+			while (matcher.find()) {
+				groups.add(matcher.group(1));
+			}
+			return unix;
+		}
+	}
+
+	@SuppressFBWarnings("SE_TRANSIENT_FIELD_NOT_RESTORED")
+	static class StateOut implements Serializable {
+		private static final long serialVersionUID = -1195263184715054229L;
+
+		final StateIn in;
+
+		StateOut(StateIn in) {
+			this.in = in;
+		}
+
+		final transient StringBuilder builder = new StringBuilder();
+
+		private String format(String unix) {
+			if (in.groups.isEmpty()) {
+				return unix;
+			}
+			builder.setLength(0);
+			Matcher matcher = in.regex.matcher(unix);
+			int lastEnd = 0;
+			int groupIdx = 0;
+			while (matcher.find()) {
+				builder.append(unix, lastEnd, matcher.start(1));
+				builder.append(in.groups.get(groupIdx));
+				lastEnd = matcher.end(1);
+				++groupIdx;
+			}
+			if (groupIdx == in.groups.size()) {
+				builder.append(unix, lastEnd, unix.length());
+				return builder.toString();
+			} else {
+				// throw an error with either the full regex, or the nicer open/close pair
+				Matcher openClose = Pattern.compile("\\\\Q([\\s\\S]*?)\\\\E" + "\\Q([\\s\\S]*?)\\E" + "\\\\Q([\\s\\S]*?)\\\\E")
+						.matcher(in.regex.pattern());
+				String pattern;
+				if (openClose.matches()) {
+					pattern = openClose.group(1) + " " + openClose.group(2);
+				} else {
+					pattern = in.regex.pattern();
+				}
+				throw new Error("An intermediate step removed a match of " + pattern);
+			}
+		}
+	}
+}

plugin-gradle/CHANGES.md

@@ -3,6 +3,8 @@
 We adhere to the [keepachangelog](https://keepachangelog.com/en/1.0.0/) format (starting after version `3.27.0`).
 
 ## [Unreleased]
+### Added
+* New option [`toggleOffOn()`](README.md#spotlessoff-and-spotlesson) which allows the tags `spotless:off` and `spotless:on` to protect sections of code from the rest of the formatters ([#691](https://github.com/diffplug/spotless/pull/691)).
 ### Changed
 * When applying license headers for the first time, we are now more lenient about parsing existing years from the header ([#690](https://github.com/diffplug/spotless/pull/690)).
 

plugin-gradle/README.md

@@ -33,7 +33,7 @@ output = prefixDelimiterReplace(input, 'https://javadoc.io/static/com.diffplug.s
 
 Spotless is a general-purpose formatting plugin used by [4,000 projects on GitHub (August 2020)](https://github.com/search?l=gradle&q=spotless&type=Code).  It is completely à la carte, but also includes powerful "batteries-included" if you opt-in.
 
-To people who use your build, it looks like this ([IDE support also available]()):
+To people who use your build, it looks like this ([IDE support also available](IDE_HOOK.md)):
 
 ```console
 user@machine repo % ./gradlew build
@@ -78,6 +78,7 @@ Spotless supports all of Gradle's built-in performance features (incremental bui
 - **Language independent**
   - [License header](#license-header) ([slurp year from git](#retroactively-slurp-years-from-git-history))
   - [How can I enforce formatting gradually? (aka "ratchet")](#ratchet)
+  - [`spotless:off` and `spotless:on`](#spotlessoff-and-spotlesson)
   - [Line endings and encodings (invisible stuff)](#line-endings-and-encodings-invisible-stuff)
   - [Custom steps](#custom-steps)
   - [Multiple (or custom) language-specific blocks](#multiple-or-custom-language-specific-blocks)
@@ -694,11 +695,23 @@ However, we strongly recommend that you use a non-local branch, such as a tag or
 
 This is especially helpful for injecting accurate copyright dates using the [license step](#license-header).
 
+## `spotless:off` and `spotless:on`
+
+Sometimes there is a chunk of code  which you have carefully handcrafted, and you would like to exclude just this one little part from getting clobbered by the autoformat. Some formatters have a way to do this, many don't, but who cares.  If you setup your spotless like this:
+
+```gradle
+spotless {
+  java { // or kotlin, or c, or python, or whatever
+    toggleOffOn()
+```
+
+Then whenever Spotless encounters a pair of `spotless:off` / `spotless:on`, it will exclude the code between them from formatting, regardless of all other rules. If you want, you can change the tags to be whatever you want, e.g. `toggleOffOn('fmt:off', 'fmt:on')`. If you decide to change the default, be sure to [read this](https://github.com/diffplug/spotless/pull/691) for some gotchas.
+
 <a name="invisible"></a>
 
 ## Line endings and encodings (invisible stuff)
 
-Spotless uses UTF-8 by default, but you can use [any encoding which Java supports](https://docs.oracle.com/javase/8/docs/technotes/guides/intl/encoding.doc.html).  You can set it globally, and you can also set it per-format.
+Spotless uses UTF-8 by default, but you can use [any encoding which the JVM supports](https://docs.oracle.com/javase/8/docs/technotes/guides/intl/encoding.doc.html).  You can set it globally, and you can also set it per-format.
 
 ```gradle
 spotless {

plugin-gradle/src/main/java/com/diffplug/gradle/spotless/FormatExtension.java

@@ -53,6 +53,7 @@
 import com.diffplug.spotless.generic.IndentStep;
 import com.diffplug.spotless.generic.LicenseHeaderStep;
 import com.diffplug.spotless.generic.LicenseHeaderStep.YearMode;
+import com.diffplug.spotless.generic.PipeStepPair;
 import com.diffplug.spotless.generic.ReplaceRegexStep;
 import com.diffplug.spotless.generic.ReplaceStep;
 import com.diffplug.spotless.generic.TrimTrailingWhitespaceStep;
@@ -623,12 +624,46 @@ public EclipseWtpConfig eclipseWtp(EclipseWtpFormatterStep type, String version)
 		return new EclipseWtpConfig(type, version);
 	}
 
+	/**
+	 * Given a regex with *exactly one capturing group*, disables formatting
+	 * inside that captured group.
+	 */
+	public void toggleOffOnRegex(String regex) {
+		this.togglePair = PipeStepPair.named(PipeStepPair.defaultToggleName()).regex(regex).buildPair();
+	}
+
+	/** Disables formatting between the given tags. */
+	public void toggleOffOn(String off, String on) {
+		this.togglePair = PipeStepPair.named(PipeStepPair.defaultToggleName()).openClose(off, on).buildPair();
+	}
+
+	/** Disables formatting between `spotless:off` and `spotless:on`. */
+	public void toggleOffOn() {
+		toggleOffOn(PipeStepPair.defaultToggleOff(), PipeStepPair.defaultToggleOn());
+	}
+
+	/** Undoes all previous calls to {@link #toggleOffOn()} and {@link #toggleOffOn(String, String)}. */
+	public void toggleOffOnDisable() {
+		this.togglePair = null;
+	}
+
+	private @Nullable PipeStepPair togglePair;
+
 	/** Sets up a format task according to the values in this extension. */
 	protected void setupTask(SpotlessTask task) {
 		task.setEncoding(getEncoding().name());
 		task.setExceptionPolicy(exceptionPolicy);
 		FileCollection totalTarget = targetExclude == null ? target : target.minus(targetExclude);
 		task.setTarget(totalTarget);
+		List<FormatterStep> steps;
+		if (togglePair != null) {
+			steps = new ArrayList<>(this.steps.size() + 2);
+			steps.add(togglePair.in());
+			steps.addAll(this.steps);
+			steps.add(togglePair.out());
+		} else {
+			steps = this.steps;
+		}
 		task.setSteps(steps);
 		task.setLineEndingsPolicy(getLineEndings().createPolicy(getProject().getProjectDir(), () -> totalTarget));
 		if (spotless.project != spotless.project.getRootProject()) {

plugin-gradle/src/test/java/com/diffplug/gradle/spotless/ToggleOffOnTest.java

@@ -0,0 +1,48 @@
+/*
+ * Copyright 2020 DiffPlug
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package com.diffplug.gradle.spotless;
+
+import java.io.IOException;
+
+import org.junit.Test;
+
+public class ToggleOffOnTest extends GradleIntegrationHarness {
+	@Test
+	public void toggleOffOn() throws IOException {
+		setFile("build.gradle").toLines(
+				"plugins { id 'com.diffplug.spotless' }",
+				"spotless {",
+				"  format 'toLower', {",
+				"    target '**/*.md'",
+				"    custom 'lowercase', { str -> str.toLowerCase() }",
+				"    toggleOffOn()",
+				"  }",
+				"}");
+		setFile("test.md").toLines(
+				"A B C",
+				"spotless:off",
+				"D E F",
+				"spotless:on",
+				"G H I");
+		gradleRunner().withArguments("spotlessApply").build();
+		assertFile("test.md").hasLines(
+				"a b c",
+				"spotless:off",
+				"D E F",
+				"spotless:on",
+				"g h i");
+	}
+}

plugin-maven/CHANGES.md

@@ -3,6 +3,8 @@
 We adhere to the [keepachangelog](https://keepachangelog.com/en/1.0.0/) format (starting after version `1.27.0`).
 
 ## [Unreleased]
+### Added
+* New option [`<toggleOffOn />`](README.md#spotlessoff-and-spotlesson) which allows the tags `spotless:off` and `spotless:on` to protect sections of code from the rest of the formatters ([#691](https://github.com/diffplug/spotless/pull/691)).
 ### Changed
 * When applying license headers for the first time, we are now more lenient about parsing existing years from the header ([#690](https://github.com/diffplug/spotless/pull/690)).
 

plugin-maven/README.md

@@ -59,6 +59,7 @@ user@machine repo % mvn spotless:check
   - [Generic steps](#generic-steps)
   - [License header](#license-header) ([slurp year from git](#retroactively-slurp-years-from-git-history))
   - [How can I enforce formatting gradually? (aka "ratchet")](#ratchet)
+  - [`spotless:off` and `spotless:on`](#spotlessoff-and-spotlesson)
   - [Line endings and encodings (invisible stuff)](#line-endings-and-encodings-invisible-stuff)
   - [Disabling warnings and error messages](#disabling-warnings-and-error-messages)
   - [How do I preview what `mvn spotless:apply` will do?](#how-do-i-preview-what-mvn-spotlessapply-will-do)
@@ -654,6 +655,18 @@ However, we strongly recommend that you use a non-local branch, such as a tag or
 
 This is especially helpful for injecting accurate copyright dates using the [license step](#license-header).
 
+## `spotless:off` and `spotless:on`
+
+Sometimes there is a chunk of code  which you have carefully handcrafted, and you would like to exclude just this one little part from getting clobbered by the autoformat. Some formatters have a way to do this, many don't, but who cares. If you setup your spotless like this:
+
+```xml
+<configuration>
+  <java> <!-- or scala, or c, or whatever -->
+    <toggleOffOn />
+    ...
+```
+
+Then whenever Spotless encounters a pair of `spotless:off` / `spotless:on`, it will exclude that subsection of code from formatting. If you want, you can change the tags to be whatever you want, e.g. `<toggleOffOn><off>fmt:off</off><on>fmt:on</on></toggleOffOn>')`. If you change the default, [read this](https://github.com/diffplug/spotless/pull/691) for some gotchas.
 
 ## Line endings and encodings (invisible stuff)