Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cover block upload completion processor #11541

Merged
merged 32 commits into from
Apr 22, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
3f6e8e0
Update gutenberg-mobile reference
mkevins Mar 27, 2020
cccadc8
Update gutenberg-mobile reference
mkevins Mar 27, 2020
89cce9d
Add some patterns for media upload completion processing
mkevins Apr 6, 2020
e56de3e
fixup
mkevins Apr 6, 2020
8acc71d
Add some tests for cover block upload completion processing
mkevins Apr 6, 2020
15e381b
Add cover block upload completion processor
mkevins Apr 6, 2020
8972433
fixup
mkevins Apr 6, 2020
17d76d4
fixup
mkevins Apr 6, 2020
5a22a08
Handle nested blocks in processPost
mkevins Apr 6, 2020
3949fce
Update gutenberg-mobile reference
mkevins Apr 6, 2020
a476918
Merge branch 'develop' into feature/cover-block-uploads
mkevins Apr 6, 2020
57bc35d
Move json attributes processor to abstract block processor
mkevins Apr 6, 2020
812ee7f
Remove some cruft from upload completion processor patterns
mkevins Apr 6, 2020
5e1c92d
Use json parsing in other media-block processors
mkevins Apr 6, 2020
c38ba61
Use static map in media block type enum
mkevins Apr 7, 2020
d2c8a2f
Use static pattern in media block type enum
mkevins Apr 7, 2020
80ffb78
Add some comments and cleanup lint
mkevins Apr 7, 2020
c50b104
Update gutenberg-mobile reference
mkevins Apr 10, 2020
0a02cf8
Merge branch 'develop' into feature/cover-block-uploads
mkevins Apr 10, 2020
e6c78af
Update gutenberg-mobile reference
mkevins Apr 10, 2020
5acd7dc
Add unit test for malformed Cover block
mkevins Apr 14, 2020
57dcd82
Fix base case in processContent method
mkevins Apr 14, 2020
aba1654
Update gutenberg-mobile reference
mkevins Apr 14, 2020
d4da054
Merge branch 'develop' into feature/cover-block-uploads
mkevins Apr 14, 2020
29d19b4
Update gutenberg-mobile reference
mkevins Apr 15, 2020
f1e2f6e
Add upload processor test with different inline style order
mkevins Apr 21, 2020
4ba5103
Add upload processor test with space in inline styles
mkevins Apr 21, 2020
4716f03
Allow for spaces in inline styles for cover block upload processor
mkevins Apr 21, 2020
fcfbe8f
Update gutenberg-mobile reference
mkevins Apr 21, 2020
4e11d7b
Merge branch 'develop' into feature/cover-block-uploads
mkevins Apr 21, 2020
f1e302d
Fix lint
mkevins Apr 21, 2020
f053f12
Fix ktlint
mkevins Apr 21, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -426,7 +426,7 @@ public static String replaceMediaFileWithUrlInGutenbergPost(@NonNull String post
.notNullStr(Utils.escapeQuotes(mediaFile.getFileURL()));
MediaUploadCompletionProcessor processor = new MediaUploadCompletionProcessor(localMediaId, mediaFile,
siteUrl);
postContent = processor.processPost(postContent);
postContent = processor.processContent(postContent);
}
return postContent;
}
Expand Down
Original file line number Diff line number Diff line change
@@ -1,13 +1,17 @@
package org.wordpress.android.ui.posts.mediauploadcompletionprocessors;

import com.google.gson.JsonObject;
import com.google.gson.JsonParser;

import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Document.OutputSettings;
import org.wordpress.android.editor.Utils;
import org.wordpress.android.util.helpers.MediaFile;

import java.util.regex.Matcher;
import java.util.regex.Pattern;

import static org.wordpress.android.ui.posts.mediauploadcompletionprocessors.MediaUploadCompletionProcessorPatterns.PATTERN_BLOCK_CAPTURES;

/**
* Abstract class to be extended for each enumerated {@link MediaBlockType}.
Expand All @@ -27,11 +31,12 @@ abstract class BlockProcessor {
String mRemoteId;
String mRemoteUrl;

private Pattern mBlockPattern;
private String mHeaderComment;
private String mBlockContent;
private String mBlockName;
private JsonObject mJsonAttributes;
private Document mBlockContentDocument;
private String mClosingComment;


/**
* @param localId The local media id that needs replacement
* @param mediaFile The mediaFile containing the remote id and remote url
Expand All @@ -40,51 +45,38 @@ abstract class BlockProcessor {
mLocalId = localId;
mRemoteId = mediaFile.getMediaId();
mRemoteUrl = org.wordpress.android.util.StringUtils.notNullStr(Utils.escapeQuotes(mediaFile.getFileURL()));
mBlockPattern = Pattern.compile(String.format(getBlockPatternTemplate(), localId), Pattern.DOTALL);
}

// TODO: consider processing block header JSON in a more robust way (current processing uses RexEx)
/**
* @param block The raw block contents of the block to be matched
* @return A {@link Matcher} to extract block contents and splice the header with a remote id. The matcher has the
* following capture groups:
*
* <ol>
* <li>Block header before id</li>
* <li>The localId (to be replaced)</li>
* <li>Block header after id</li>
* <li>Block contents</li>
* <li>Block closing comment and any following characters</li>
* </ol>
*/
Matcher getMatcherForBlock(String block) {
return mBlockPattern.matcher(block);
private JsonObject parseJson(String blockJson) {
JsonParser parser = new JsonParser();
return parser.parse(blockJson).getAsJsonObject();
}

private Document parseHTML(String blockContent) {
// create document from block content
Document document = Jsoup.parse(blockContent);
document.outputSettings(OUTPUT_SETTINGS);
return document;
}

boolean matchAndSpliceBlockHeader(String block) {
Matcher matcher = getMatcherForBlock(block);
private boolean splitBlock(String block) {
Matcher captures = PATTERN_BLOCK_CAPTURES.matcher(block);

boolean matchFound = matcher.find();
boolean capturesFound = captures.find();

if (matchFound) {
mHeaderComment = new StringBuilder()
.append(matcher.group(1))
.append(mRemoteId) // here we substitute remote id in place of the local id
.append(matcher.group(3))
.toString();
mBlockContent = matcher.group(4);
mClosingComment = matcher.group(5);
if (capturesFound) {
mBlockName = captures.group(1);
mJsonAttributes = parseJson(captures.group(2));
mBlockContentDocument = parseHTML(captures.group(3));
mClosingComment = captures.group(4);
return true;
} else {
mHeaderComment = null;
mBlockContent = null;
mBlockName = null;
mJsonAttributes = null;
mBlockContentDocument = null;
mClosingComment = null;
return false;
}

return matchFound;
}

String getHeaderComment() {
return mHeaderComment;
}

/**
Expand All @@ -94,46 +86,28 @@ String getHeaderComment() {
* @param block The raw block contents
* @return A string containing content with ids and urls replaced
*/
String processBlock(String block) {
if (matchAndSpliceBlockHeader(block)) {
// create document from block content
Document document = Jsoup.parse(mBlockContent);
document.outputSettings(OUTPUT_SETTINGS);

if (processBlockContentDocument(document)) {
// return injected block
return new StringBuilder()
.append(getHeaderComment())
.append(document.body().html()) // parser output
.append(mClosingComment)
.toString();
}
}

// leave block unchanged
return block;
}

/**
* All concrete implementations must implement this method to return a regex pattern template for the particular
* block type.<br>
* <br>
* The pattern template should contain a format specifier for the local id that needs to be matched and
* replaced in the block header, and the format specifier should be within its own capture group, e.g. `(%1$s)`.<br>
* <br>
* The pattern template should result in a matcher with the following capture groups:
*
* <ol>
* <li>Block header before id</li>
* <li>The format specifier for the local id (to be replaced by the local id when generating the pattern)</li>
* <li>Block header after id</li>
* <li>Block contents</li>
* <li>Block closing comment and any following characters</li>
* </ol>
*
* @return String with the regex pattern template
*/
abstract String getBlockPatternTemplate();
String processBlock(String block) {
if (splitBlock(block)) {
if (processBlockJsonAttributes(mJsonAttributes)) {
if (processBlockContentDocument(mBlockContentDocument)) {
// return injected block
return new StringBuilder()
.append("<!-- wp:")
.append(mBlockName)
.append(" ")
.append(mJsonAttributes) // json parser output
.append(" -->\n")
.append(mBlockContentDocument.body().html()) // HTML parser output
.append(mClosingComment)
.toString();
}
} else {
return processInnerBlock(block); // delegate to inner blocks if needed
}
}
// leave block unchanged
return block;
}

/**
* All concrete implementations must implement this method for the particular block type. The document represents
Expand All @@ -146,4 +120,33 @@ String processBlock(String block) {
* @return A boolean value indicating whether or not the block contents should be replaced
*/
abstract boolean processBlockContentDocument(Document document);

/**
* All concrete implementations must implement this method for the particular block type. The jsonAttributes object
* is a {@link JsonObject} parsed from the block header attributes. This object can be used to check for a match,
* and can be directly mutated if necessary.<br>
* <br>
* This method should return true to indicate success. Returning false will result in the block contents being
* unmodified.
*
* @param jsonAttributes the attributes object used to check for a match with the local id, and mutated if necessary
* @return
*/
abstract boolean processBlockJsonAttributes(JsonObject jsonAttributes);

/**
* This method can be optionally overriden by concrete implementations to delegate further processing via recursion
* when {@link BlockProcessor#processBlockJsonAttributes(JsonObject)} returns false (i.e. the block did not match
* the local id being replaced). This is useful for implementing mutual recursion with
* {@link MediaUploadCompletionProcessor#processContent(String)} for block types that have media-containing blocks
* within their inner content.<br>
* <br>
* The default implementation provided is a NOOP that leaves the content of the block unchanged.
*
* @param block The raw block contents
* @return A string containing content with ids and urls replaced
*/
String processInnerBlock(String block) {
return block;
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason to not make the default processor process inner blocks? It will come surprisingly often that the blocks we care about are nested, for example, when people build with the full-site-editing layout blocks, or the Group block, or the Quote block, or the Gallery block…

Copy link
Contributor Author

@mkevins mkevins Apr 13, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason to not make the default processor process inner blocks?

For the current implementation of the processors, they are not full-blown parsers, but part of their behavior resembles a parser. Instead, the processors are only concerned with a subset of block types (image, video, etc.). To hopefully clarify: when processContent encounters a Group block, it does not match on that block, but if that block's inner contents contains a media-containing block, that will match and be processed further (via processBlock), while all content preceeding that match is treated as a raw string. This is a way to produce the desired output without the need to implement any serialization logic for all potential block types.

}
}
Original file line number Diff line number Diff line change
Expand Up @@ -5,19 +5,22 @@
import java.util.HashMap;
import java.util.Map;

import static org.wordpress.android.ui.posts.mediauploadcompletionprocessors.MediaBlockType.COVER;
import static org.wordpress.android.ui.posts.mediauploadcompletionprocessors.MediaBlockType.GALLERY;
import static org.wordpress.android.ui.posts.mediauploadcompletionprocessors.MediaBlockType.IMAGE;
import static org.wordpress.android.ui.posts.mediauploadcompletionprocessors.MediaBlockType.MEDIA_TEXT;
import static org.wordpress.android.ui.posts.mediauploadcompletionprocessors.MediaBlockType.VIDEO;

class BlockProcessorFactory {
private final MediaUploadCompletionProcessor mMediaUploadCompletionProcessor;
private final Map<MediaBlockType, BlockProcessor> mMediaBlockTypeBlockProcessorMap;

/**
* This factory initializes block processors for all media block types and provides a method to retrieve a block
* processor instance for a given block type.
*/
BlockProcessorFactory() {
BlockProcessorFactory(MediaUploadCompletionProcessor mediaUploadCompletionProcessor) {
mMediaUploadCompletionProcessor = mediaUploadCompletionProcessor;
mMediaBlockTypeBlockProcessorMap = new HashMap<>();
}

Expand All @@ -32,6 +35,8 @@ BlockProcessorFactory init(String localId, MediaFile mediaFile, String siteUrl)
mMediaBlockTypeBlockProcessorMap.put(VIDEO, new VideoBlockProcessor(localId, mediaFile));
mMediaBlockTypeBlockProcessorMap.put(MEDIA_TEXT, new MediaTextBlockProcessor(localId, mediaFile));
mMediaBlockTypeBlockProcessorMap.put(GALLERY, new GalleryBlockProcessor(localId, mediaFile, siteUrl));
mMediaBlockTypeBlockProcessorMap.put(COVER, new CoverBlockProcessor(localId, mediaFile,
mMediaUploadCompletionProcessor));

return this;
}
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
package org.wordpress.android.ui.posts.mediauploadcompletionprocessors;

import com.google.gson.JsonObject;

import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.wordpress.android.util.helpers.MediaFile;

import java.util.regex.Matcher;
import java.util.regex.Pattern;

public class CoverBlockProcessor extends BlockProcessor {
/**
* Template pattern used to match and splice cover inner blocks
*/
private static final Pattern PATTERN_COVER_INNER = Pattern.compile(new StringBuilder()
.append("(^.*?<div class=\"wp-block-cover__inner-container\">\\s*)")
.append("(.*)") // inner block contents
.append("(\\s*</div>\\s*</div>\\s*<!-- /wp:cover -->.*)").toString(), Pattern.DOTALL);
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oops. I had left a comment here that was wrong. you can take some inspiration from hpq which @aduth wrote. something like jsoup (random internet search) could automate the inference of the block attributes as defined in the block implementation, rather than falling back to attempts at parsing hTML with RegExp

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @dmsnell 👋 😃 , thanks for taking a look at this one. I'm using jsoup already, which is really helpful. IIRC, I encountered some minor differences in output in certain scenarios, possible because it parses to an AST and not a parse tree, however, it is probably worth revisiting, as the variations might still be acceptable to pass validation. 👍

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I encountered some minor differences in output in certain scenarios, possible because it parses to an AST and not a parse tree

Do you remember what those differences were? Any difference would have to be a bug since we're dealing with HTML/CSS selectors…

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also it looks like this entire pattern shouldn't be necessary if we process the inner blocks using a spec-compliant parser

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you remember what those differences were? Any difference would have to be a bug

Off-hand, I recall that jsoup was adding closing tags, which was problematic during at least one of the iterations toward the current implementation. It's likely this is resolved now, as the implementation no longer processes units of non-self-contained HTML, so probably worth revisiting. To be sure, this was not a bug in jsoup, but rather a limitation in a particular iteration of this implementation, in that it required serialization of the parse tree, while jsoup is designed to serialize abstract trees.

also it looks like this entire pattern shouldn't be necessary if we process the inner blocks using a spec-compliant parser

Indeed, using a spec-compliant parser would obviate the need for this, as we'd be able to handle inner blocks an a much more general way, albeit for all block types.


/**
* Pattern to match background-image url in cover block html content
*/
private static final Pattern PATTERN_BACKGROUND_IMAGE_URL = Pattern.compile(
"background-image:\\s*url\\([^\\)]+\\)");

private final MediaUploadCompletionProcessor mMediaUploadCompletionProcessor;

public CoverBlockProcessor(String localId, MediaFile mediaFile,
MediaUploadCompletionProcessor mediaUploadCompletionProcessor) {
super(localId, mediaFile);
mMediaUploadCompletionProcessor = mediaUploadCompletionProcessor;
}

@Override String processInnerBlock(String block) {
Matcher innerMatcher = PATTERN_COVER_INNER.matcher(block);
boolean innerCapturesFound = innerMatcher.find();

// process inner contents recursively
if (innerCapturesFound) {
String innerProcessed = mMediaUploadCompletionProcessor.processContent(innerMatcher.group(2)); //
return new StringBuilder()
.append(innerMatcher.group(1))
.append(innerProcessed)
.append(innerMatcher.group(3))
.toString();
}

return block;
}

@Override boolean processBlockJsonAttributes(JsonObject jsonAttributes) {
if (jsonAttributes.get("id").getAsInt() == Integer.parseInt(mLocalId, 10)) {
jsonAttributes.addProperty("id", Integer.parseInt(mRemoteId, 10));
jsonAttributes.addProperty("url", mRemoteUrl);
return true;
}

return false;
}

@Override boolean processBlockContentDocument(Document document) {
// select cover block div
Element targetDiv = document.select(".wp-block-cover").first();

// if a match is found, proceed with replacement
if (targetDiv != null) {
// replace background-image url in style attribute
String style = PATTERN_BACKGROUND_IMAGE_URL.matcher(targetDiv.attr("style"))
.replaceFirst(String.format("background-image:url(%1$s)", mRemoteUrl));
targetDiv.attr("style", style);
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hm…I'm seeing the cover block actually store the background imagine inside the JSON attributes. this will deifnitely update the style, but won't the newly-uploaded image get wiped out after someone loads the post in the editor again and there's a mismatch between what the attributes indicate and what HTML has been generated?

we probably need to update both places.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we probably need to update both places

The Json attributes are updated in an earlier step via the abstract method processBlockJsonAttributes. For Cover block, the id is used to check for a match first, and the url is mutated a match with the local id is found. The replacement within the style attribute only occurs subsequently, if that condition is met.


// return injected block
return true;
}

return false;
}
}
Loading