WordPress 4.7.4: File integrity issues?

[strarsis]

After upgrading from WordPress 4.7.3 to 4.7.4 I get core file integrity issues reported by WordFence.
Hence I downloaded the official WordPress 4.7.4 release zip and composer-required the johnpbloch/wordpress package and then compared both directories.

And indeed (besides extra misc files like composer.json) there are differences
in minified JavaScript files and inlined code parts in some PHP files
between official WordPress and composer package releases.
About 100 files are affected.

Example: Deminified core.min.js files from both WordPress 4.7.4 sources:

• core.min.js from official WordPress 4.7.4 zip
• core.min.js from composer WordPress 4.7.4 package

[retlehs]

also seeing this

vagrant@example:/srv/www/example.com/current$ wp core verify-checksums
Warning: File doesn't verify against checksum: wp-admin/js/user-profile.min.js
Warning: File doesn't verify against checksum: wp-admin/js/word-count.min.js
Warning: File doesn't verify against checksum: wp-admin/js/updates.min.js
Warning: File doesn't verify against checksum: wp-admin/js/postbox.min.js
Warning: File doesn't verify against checksum: wp-admin/js/nav-menu.min.js
Warning: File doesn't verify against checksum: wp-admin/js/theme.min.js
Warning: File doesn't verify against checksum: wp-admin/js/press-this.min.js
Warning: File doesn't verify against checksum: wp-admin/js/link.min.js
Warning: File doesn't verify against checksum: wp-admin/js/post.min.js
Warning: File doesn't verify against checksum: wp-admin/js/inline-edit-post.min.js
Warning: File doesn't verify against checksum: wp-admin/js/dashboard.min.js
Warning: File doesn't verify against checksum: wp-admin/js/tags.min.js
Warning: File doesn't verify against checksum: wp-admin/js/comment.min.js
Warning: File doesn't verify against checksum: wp-admin/js/common.min.js
Warning: File doesn't verify against checksum: wp-admin/js/revisions.min.js
Warning: File doesn't verify against checksum: wp-admin/js/image-edit.min.js
Warning: File doesn't verify against checksum: wp-admin/js/widgets.min.js
Warning: File doesn't verify against checksum: wp-admin/js/customize-nav-menus.min.js
Warning: File doesn't verify against checksum: wp-admin/js/custom-background.min.js
Warning: File doesn't verify against checksum: wp-admin/js/svg-painter.min.js
Warning: File doesn't verify against checksum: wp-admin/js/customize-widgets.min.js
Warning: File doesn't verify against checksum: wp-admin/js/editor-expand.min.js
Warning: File doesn't verify against checksum: wp-admin/js/color-picker.min.js
Warning: File doesn't verify against checksum: wp-admin/js/gallery.min.js
Warning: File doesn't verify against checksum: wp-admin/js/tags-box.min.js
Warning: File doesn't verify against checksum: wp-admin/js/password-strength-meter.min.js
Warning: File doesn't verify against checksum: wp-admin/js/editor.min.js
Warning: File doesn't verify against checksum: wp-admin/js/user-suggest.min.js
Warning: File doesn't verify against checksum: wp-admin/js/edit-comments.min.js
Warning: File doesn't verify against checksum: wp-admin/js/customize-controls.min.js
Warning: File doesn't verify against checksum: wp-admin/js/tags-suggest.min.js
Warning: File doesn't verify against checksum: wp-admin/css/themes.min.css
Warning: File doesn't verify against checksum: wp-admin/css/themes-rtl.min.css
Warning: File doesn't verify against checksum: wp-includes/js/media-audiovideo.min.js
Warning: File doesn't verify against checksum: wp-includes/js/customize-preview.min.js
Warning: File doesn't verify against checksum: wp-includes/js/media-views.min.js
Warning: File doesn't verify against checksum: wp-includes/js/autosave.min.js
Warning: File doesn't verify against checksum: wp-includes/js/twemoji.min.js
Warning: File doesn't verify against checksum: wp-includes/js/media-editor.min.js
Warning: File doesn't verify against checksum: wp-includes/js/plupload/handlers.min.js
Warning: File doesn't verify against checksum: wp-includes/js/plupload/wp-plupload.min.js
Warning: File doesn't verify against checksum: wp-includes/js/wp-emoji.min.js
Warning: File doesn't verify against checksum: wp-includes/js/customize-base.min.js
Warning: File doesn't verify against checksum: wp-includes/js/wp-custom-header.min.js
Warning: File doesn't verify against checksum: wp-includes/js/wplink.min.js
Warning: File doesn't verify against checksum: wp-includes/js/utils.min.js
Warning: File doesn't verify against checksum: wp-includes/js/customize-selective-refresh.min.js
Warning: File doesn't verify against checksum: wp-includes/js/json2.min.js
Warning: File doesn't verify against checksum: wp-includes/js/heartbeat.min.js
Warning: File doesn't verify against checksum: wp-includes/js/media-models.min.js
Warning: File doesn't verify against checksum: wp-includes/js/colorpicker.min.js
Warning: File doesn't verify against checksum: wp-includes/js/admin-bar.min.js
Warning: File doesn't verify against checksum: wp-includes/js/jquery/ui/datepicker.min.js
Warning: File doesn't verify against checksum: wp-includes/js/jquery/ui/dialog.min.js
Warning: File doesn't verify against checksum: wp-includes/js/jquery/ui/progressbar.min.js
Warning: File doesn't verify against checksum: wp-includes/js/jquery/ui/mouse.min.js
Warning: File doesn't verify against checksum: wp-includes/js/jquery/ui/slider.min.js
Warning: File doesn't verify against checksum: wp-includes/js/jquery/ui/droppable.min.js
Warning: File doesn't verify against checksum: wp-includes/js/jquery/ui/position.min.js
Warning: File doesn't verify against checksum: wp-includes/js/jquery/ui/draggable.min.js
Warning: File doesn't verify against checksum: wp-includes/js/jquery/ui/effect.min.js
Warning: File doesn't verify against checksum: wp-includes/js/jquery/ui/spinner.min.js
Warning: File doesn't verify against checksum: wp-includes/js/jquery/ui/accordion.min.js
Warning: File doesn't verify against checksum: wp-includes/js/jquery/ui/resizable.min.js
Warning: File doesn't verify against checksum: wp-includes/js/jquery/ui/widget.min.js
Warning: File doesn't verify against checksum: wp-includes/js/jquery/ui/autocomplete.min.js
Warning: File doesn't verify against checksum: wp-includes/js/jquery/ui/core.min.js
Warning: File doesn't verify against checksum: wp-includes/js/jquery/ui/menu.min.js
Warning: File doesn't verify against checksum: wp-includes/js/jquery/ui/tabs.min.js
Warning: File doesn't verify against checksum: wp-includes/js/jquery/ui/sortable.min.js
Warning: File doesn't verify against checksum: wp-includes/js/jquery/ui/tooltip.min.js
Warning: File doesn't verify against checksum: wp-includes/js/jquery/ui/button.min.js
Warning: File doesn't verify against checksum: wp-includes/js/wp-pointer.min.js
Warning: File doesn't verify against checksum: wp-includes/js/wp-embed-template.min.js
Warning: File doesn't verify against checksum: wp-includes/js/shortcode.min.js
Warning: File doesn't verify against checksum: wp-includes/js/quicktags.min.js
Warning: File doesn't verify against checksum: wp-includes/js/imagesloaded.min.js
Warning: File doesn't verify against checksum: wp-includes/js/mediaelement/wp-playlist.min.js
Warning: File doesn't verify against checksum: wp-includes/js/wp-emoji-release.min.js
Warning: File doesn't verify against checksum: wp-includes/js/mce-view.min.js
Warning: File doesn't verify against checksum: wp-includes/js/customize-models.min.js
Warning: File doesn't verify against checksum: wp-includes/js/wp-emoji-loader.min.js
Warning: File doesn't verify against checksum: wp-includes/js/customize-loader.min.js
Warning: File doesn't verify against checksum: wp-includes/js/hoverIntent.min.js
Warning: File doesn't verify against checksum: wp-includes/js/tinymce/plugins/wordpress/plugin.min.js
Warning: File doesn't verify against checksum: wp-includes/js/tinymce/plugins/wpdialogs/plugin.min.js
Warning: File doesn't verify against checksum: wp-includes/js/tinymce/plugins/wplink/plugin.min.js
Warning: File doesn't verify against checksum: wp-includes/js/tinymce/plugins/wptextpattern/plugin.min.js
Warning: File doesn't verify against checksum: wp-includes/js/tinymce/plugins/wpeditimage/plugin.min.js
Warning: File doesn't verify against checksum: wp-includes/js/tinymce/plugins/wpview/plugin.min.js
Warning: File doesn't verify against checksum: wp-includes/js/tinymce/wp-tinymce.js.gz
Warning: File doesn't verify against checksum: wp-includes/js/wp-api.min.js
Warning: File doesn't verify against checksum: wp-includes/js/customize-preview-nav-menus.min.js
Warning: File doesn't verify against checksum: wp-includes/js/customize-preview-widgets.min.js
Warning: File doesn't verify against checksum: wp-includes/js/wp-embed.min.js
Warning: File doesn't verify against checksum: wp-includes/js/tw-sack.min.js
Warning: File doesn't verify against checksum: wp-includes/js/media-grid.min.js
Warning: File doesn't verify against checksum: wp-includes/formatting.php
Warning: File doesn't verify against checksum: wp-includes/embed.php
Error: WordPress install doesn't verify against checksums.

[johnpbloch]

Since I build the core package from source, the compiled and minified scripts and stylesheets have slightly different contents. My package is not the same as the .org package.

If you're interested in reviewing how I'm generating the core package, check out the build script at https://github.com/johnpbloch/build-wp

[strarsis]

@johnpbloch: Would it be possible to make the minified files match the checksums?
There are several reasons why this would be much better,
including integrity checks, WAF like WordFence scans.

[johnpbloch]

I think it's certainly possible for future tagged versions. It's honestly not a high priority for me. I'd certainly be happy to accept a contribution to the build script if you or another interested person sent in a pull request.

[strarsis]

@johnpbloch: I just rebuilt 4.7.4 (using the receipt in the build-wp repository you linked to above) and
now the text files are all identical - so it would work correctly with the current 4.7.4 svn tag.

Interestingly, there are no differences for 4.7.3 between official zip and composer package.

With the last 4.7.4 composer package, only the minified JavaScript files and inlined code parts differed.
The deobfuscated code also differed - maybe the WordPress devs changed something last-minute and re-tagged? It is unlikely the uglify minifier being not deterministic using same input source.

[strarsis]

@johnpbloch: I think an additional release (with an extra segment after patch level,
deviating from semver but following the composer version specs/examples (see https://getcomposer.org/doc/articles/versions.md)) will use the now
correct 4.7.4 svn tagged commit and result in a correct composer package and
an update-successor the current 4.7.4 composer package.

[johnpbloch]

@strarsis I took some time to adjust the build script today. Tags no longer build from develop.svn but rather simply use the zip file distributed on wordpress.org for building tagged releases. What that means is that going forward, this shouldn't be a problem again, and releases should get to packagist faster to boot.
I'm going to look into getting a new set of releases tagged in the repo, probably doing a X.X.X.1 pattern to avoid needing to delete tags.

[ollietreend]

Great work @johnpbloch – thanks for this!

I came here wondering why my app has just updated to 4.7.4.1 despite that not being an official release – I was a little confused, but this makes sense. Using the official ZIP releases makes sense rather than rebuilding from source.

[wunc]

I just want to note a side-effect: since the twentyeleven through twentyfourteen themes are not included in the zip (but are in the source), they are no longer installed since @johnpbloch updated the build script to build from the zip instead of the source. (At least I think that's the reason.)

This threw me for a bit of a loop because I had activated them on a network site, and a couple of the sub-sites that were using them broke when I updated today. I had to manually add them back from wpackagist.