Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Version 1.0 problems in manifest.json when importing from composer vendor folder #907

Closed
minimit opened this issue Feb 1, 2021 · 11 comments · Fixed by shellscape/webpack-manifest-plugin#249 or #921
Labels
Bug Bug Fix

Comments

@minimit
Copy link
Contributor

minimit commented Feb 1, 2021

Hello I'm trying to update sylius webpack encore to 1.0.3 but the manifest.json is all wrong, maybe it's because sylius entry point is a javascript that imports files from vendor?

This is the entry point. sylius is inside vendor.

import 'sylius/bundle/AdminBundle/Resources/private/entry';

and the content of manifest.json is this

{
  "build/shop/shop-entry.css": "/build/shop/shop-entry.css",
  "build/shop/shop-entry.js": "/build/shop/shop-entry.js",
  "build/shop/shop-entry.eot": "/build/shop/fonts/outline-icons.752905fa.eot",
  "build/shop/shop-entry.svg": "/build/shop/images/outline-icons.9c4845b4.svg",
  "build/shop/shop-entry.ttf": "/build/shop/fonts/outline-icons.53671035.ttf",
  "build/shop/shop-entry.woff": "/build/shop/fonts/outline-icons.ddae9b1b.woff",
  "build/shop/shop-entry.woff2": "/build/shop/fonts/outline-icons.687a4990.woff2",
  "build/shop/shop-entry.png": "/build/shop/images/logo.72a9465e.png",
  "build/shop/shop-entry.gif": "/build/shop/images/loading.f657825a.gif",
  "build/shop/shop-entry.jpg": "/build/shop/images/homepage-banner.8d8f072a.jpg",
  "build/shop/node_modules/semantic-ui-css/themes/default/assets/fonts/brand-icons.svg": "/build/shop/images/brand-icons.6729d297.svg",
  "build/shop/node_modules/semantic-ui-css/themes/default/assets/fonts/icons.svg": "/build/shop/images/icons.62d9dae4.svg",
  "build/shop/vendor/sylius/sylius/src/Sylius/Bundle/ShopBundle/Resources/private/img/homepage-banner.jpg": "/build/shop/images/homepage-banner.8d8f072a.jpg",
  "build/shop/node_modules/semantic-ui-css/themes/default/assets/fonts/outline-icons.svg": "/build/shop/images/outline-icons.9c4845b4.svg",
  "build/shop/node_modules/semantic-ui-css/themes/default/assets/fonts/icons.eot": "/build/shop/fonts/icons.a01e3f2d.eot",
  "build/shop/node_modules/semantic-ui-css/themes/default/assets/fonts/icons.ttf": "/build/shop/fonts/icons.c656b8ca.ttf",
  "build/shop/node_modules/semantic-ui-css/themes/default/assets/fonts/brand-icons.eot": "/build/shop/fonts/brand-icons.d68fa3e6.eot",
  "build/shop/node_modules/semantic-ui-css/themes/default/assets/fonts/brand-icons.ttf": "/build/shop/fonts/brand-icons.65a2fb6d.ttf",
  "build/shop/node_modules/semantic-ui-css/themes/default/assets/fonts/brand-icons.woff": "/build/shop/fonts/brand-icons.cac87dc0.woff",
  "build/shop/node_modules/semantic-ui-css/themes/default/assets/fonts/brand-icons.woff2": "/build/shop/fonts/brand-icons.278156e4.woff2",
  "build/shop/node_modules/semantic-ui-css/themes/default/assets/fonts/icons.woff": "/build/shop/fonts/icons.425399f8.woff",
  "build/shop/vendor/sylius/sylius/src/Sylius/Bundle/ShopBundle/Resources/private/img/sylius-plus-banner.png": "/build/shop/images/sylius-plus-banner.fff4fe72.png",
  "build/shop/node_modules/semantic-ui-css/themes/default/assets/fonts/icons.woff2": "/build/shop/fonts/icons.38c6d8ba.woff2",
  "build/shop/node_modules/semantic-ui-css/themes/default/assets/fonts/outline-icons.eot": "/build/shop/fonts/outline-icons.752905fa.eot",
  "build/shop/node_modules/semantic-ui-css/themes/default/assets/fonts/outline-icons.ttf": "/build/shop/fonts/outline-icons.53671035.ttf",
  "build/shop/node_modules/semantic-ui-css/themes/default/assets/images/flags.png": "/build/shop/images/flags.99f63ae7.png",
  "build/shop/vendor/sylius/sylius/src/Sylius/Bundle/UiBundle/Resources/private/img/logo.png": "/build/shop/images/logo.72a9465e.png",
  "build/shop/node_modules/semantic-ui-css/themes/default/assets/fonts/outline-icons.woff": "/build/shop/fonts/outline-icons.ddae9b1b.woff",
  "build/shop/node_modules/semantic-ui-css/themes/default/assets/fonts/outline-icons.woff2": "/build/shop/fonts/outline-icons.687a4990.woff2",
  "build/shop/node_modules/lightbox2/dist/images/loading.gif": "/build/shop/images/loading.f657825a.gif",
  "build/shop/vendor/sylius/sylius/src/Sylius/Bundle/UiBundle/Resources/private/img/avatar.png": "/build/shop/images/avatar.fa261429.png",
  "build/shop/node_modules/lightbox2/dist/images/prev.png": "/build/shop/images/prev.0edc57cc.png",
  "build/shop/node_modules/lightbox2/dist/images/next.png": "/build/shop/images/next.62074ac7.png",
  "build/shop/node_modules/lightbox2/dist/images/close.png": "/build/shop/images/close.0cfd6489.png"
}

Should be this

  "build/shop/shop-entry.css": "/build/shop/shop-entry.css",
  "build/shop/shop-entry.js": "/build/shop/shop-entry.js",
  "build/shop/fonts/brand-icons.woff2": "/build/shop/fonts/brand-icons.278156e4.woff2",
  "build/shop/fonts/brand-icons.ttf": "/build/shop/fonts/brand-icons.65a2fb6d.ttf",
  "build/shop/fonts/brand-icons.woff": "/build/shop/fonts/brand-icons.cac87dc0.woff",
  "build/shop/fonts/brand-icons.eot": "/build/shop/fonts/brand-icons.d68fa3e6.eot",
  "build/shop/fonts/icons.woff2": "/build/shop/fonts/icons.38c6d8ba.woff2",
  "build/shop/fonts/icons.woff": "/build/shop/fonts/icons.425399f8.woff",
  "build/shop/fonts/icons.eot": "/build/shop/fonts/icons.a01e3f2d.eot",
  "build/shop/fonts/icons.ttf": "/build/shop/fonts/icons.c656b8ca.ttf",
  "build/shop/fonts/outline-icons.ttf": "/build/shop/fonts/outline-icons.53671035.ttf",
  "build/shop/fonts/outline-icons.woff2": "/build/shop/fonts/outline-icons.687a4990.woff2",
  "build/shop/fonts/outline-icons.eot": "/build/shop/fonts/outline-icons.752905fa.eot",
  "build/shop/fonts/outline-icons.woff": "/build/shop/fonts/outline-icons.ddae9b1b.woff",
  "build/shop/images/avatar.png": "/build/shop/images/avatar.fa261429.png",
  "build/shop/images/brand-icons.svg": "/build/shop/images/brand-icons.6729d297.svg",
  "build/shop/images/close.png": "/build/shop/images/close.0cfd6489.png",
  "build/shop/images/flags.png": "/build/shop/images/flags.99f63ae7.png",
  "build/shop/images/homepage-banner.jpg": "/build/shop/images/homepage-banner.8d8f072a.jpg",
  "build/shop/images/icons.svg": "/build/shop/images/icons.62d9dae4.svg",
  "build/shop/images/loading.gif": "/build/shop/images/loading.f657825a.gif",
  "build/shop/images/logo.png": "/build/shop/images/logo.72a9465e.png",
  "build/shop/images/next.png": "/build/shop/images/next.62074ac7.png",
  "build/shop/images/outline-icons.svg": "/build/shop/images/outline-icons.9c4845b4.svg",
  "build/shop/images/prev.png": "/build/shop/images/prev.0edc57cc.png",
  "build/shop/images/sylius-plus-banner.png": "/build/shop/images/sylius-plus-banner.fff4fe72.png"
}
@ureimers
Copy link

ureimers commented Feb 2, 2021

This all seems to be related with the webpack-manifest-plugin in connection with Webpack 5.0. The currently open tickets that are related to this are:
Manifest keys contain source path: shellscape/webpack-manifest-plugin#238
Manifest populates main entry point with unrelated files with webpack 5: shellscape/webpack-manifest-plugin#237

There are other issues too that might be related (see current list of issues in https://github.com/shellscape/webpack-manifest-plugin/issues).

I fear that the webpack-manifest-plugin currently isn't working with Webpack 5.0 and that we have to move back to webpack-encore prior to v1.0 which is a bummer since I spent half the day migrating and fixing our webpack.config.js but our manifest.json is in such a bad shape and even assets that are correctly mapped in it are then prefixed with the build path twice when using Symfonys assets function (bundle/build/icons/spritemap.svg becomes /bundle/build/bundle/build/spritemap.svg in the rendered HTML). We'll try to figure it out piece by piece...

@weaverryan
Copy link
Member

Hi everyone!

Thanks for the issue and the detailed update @ureimers - that's very helpful!

I need to dig deeper, but since we need to get this fixed, I see 2 options:

  1. Find the "bug" in webpack-manifest-plugin, embed that plugin temporarily into Encore, then fix it. We have done this in the past - it's not ideal, but it gets the job done. Then, ideally, we could get that patch merged upstream and drop our copy of the plugin.

  2. Try using assets-webpack-plugin instead. We already have this package in Encore - it generates the entrypoints.json file. I do not know if it suffers from the same bug or not. It actually has way less downloads than webpack-manifest-plugin but it seems to be more active.

If anyone wants to play with assets-webpack-plugin to see if it can be used for the manifest.json file (with the same format as we have now), that would be awesome and help us get this fixed faster.

Cheers!

@ureimers
Copy link

ureimers commented Feb 3, 2021

Hi @weaverryan,

glad I could help kicking this off.

  1. I'm no JS expert, but I could still try to find out what's causing the problem with webpack-manifest-plugin. It's more a matter of having enough free time to do so. The package itself was taken over by shellscape end of october last year (Manifest keys contain hashes in production build when using file-loader shellscape/webpack-manifest-plugin#209 (comment)). He's seems to be quite busy at the moment and it's hard for me to see how active all the other contributors are. Still, embedding the plugin into Encore and then fixing it seems to be the best option here, because:

  2. The problem, I see with the alterantive assets-webpack-plugin, is, that it doesn't produce manifest.json files like Webpack Encore needs them (or is used to have). Its documentation describes the format like this (https://github.com/ztoben/assets-webpack-plugin#example-output):

The output is a JSON object in the form:

{
   "bundle_name": {
       "asset_kind": "/public/path/to/asset"
   }
}

Where:

"bundle_name" is the name of the bundle (the key of the entry object in your webpack config, or "main" if your entry is an array).
"asset_kind" is the camel-cased file extension of the asset

I'm not sure if I get the naming right here, but every single mapped asset would need to be a bundle name, and the result would have a lot of superfluous "asset_kind" keys. For example:

// Before
{
  "build/form.js": "/relaunch2020/build/form.cd8233e8.js",
  "build/spritemap.svg": "/build/spritemap.2970a35c.svg",
  "build/fonts/28EB38_0_0.woff": "/build/fonts/28EB38_0_0.3c310b9b.woff"
}
// After
{
  "build/form.js": {
    "js": "/relaunch2020/build/form.cd8233e8.js"
  },
  "build/spritemap.svg": {
    "svg": "/build/spritemap.2970a35c.svg"
  },
  "build/fonts/28EB38_0_0.woff": {
    "woff": "/build/fonts/28EB38_0_0.3c310b9b.woff"
  }
}

It's possible, but it doesn't quite seem to be the right format. What do you think?

Regards,

Ulf

@weaverryan
Copy link
Member

If you're having this problem, could you please test #921 and tell me if it works for you or doesn't work for you?

Thanks!

@minimit
Copy link
Contributor Author

minimit commented Feb 8, 2021

If you're having this problem, could you please test #921 and tell me if it works for you or doesn't work for you?

Thanks!

It works great the output is the same as version 0.33.0 with the right manifest!

@ureimers
Copy link

ureimers commented Feb 8, 2021

If you're having this problem, could you please test #921 and tell me if it works for you or doesn't work for you?

Thanks!

@weaverryan Looks a lot better now, thank you!

There are three issues left:

1. Spritemaps (fixed)
We're also using svg-spritemap-webpack-plugin like this:

Encore.addPlugin(new SVGSpriteMapPlugin('./public/bundles/my-bundle/icons/**/*.svg', {
        output: {
            filename: Encore.isProduction()
                ? 'spritemap.[contenthash].svg'
                : 'spritemap.svg',
        // ...
}

This worked in Webpack Encore 0.24 (we hadn't updated in a while) but with the new Webpack Encore 1.0 / webpack-manifest-plugin combination the manifest's source filename would include the [contenthash] too. Like:

"build/spritemap.9a7fc4ab.js":"build/spritemap.9a7fc4ab.js"

I fixed this by using and modifying the new removeHashKey method from webpack-manifest-plugin like I described here: #808 (comment)

2. Stylus assets (no fix needed?)
Also all of our inlined font files (using Encore.enableStylusLoader() and .styl files) are now in the map, which they weren't before. But I would say that this new behaviour is better since the fonts might be updated in the future and thus, having them use/support cash busting is helpful.

3. Numerically named files (conditional imports?)
There are now three numerically named files in the manifest.json:

  "build/217.js": "build/217.9ec7e8c9.js",
  "build/181.js": "build/181.9120a97c.js",
  "build/54.css": "build/54.4acca938.css",

They weren't there before (Encore 0.24) and are only created in a production build. As they are minified it's not that easy to find their source files but I was able to locate the .css file: It's conditionally imported inside a .vue file which itself is imported in a .js file which is added as an entry to Encore (Encore.addEntry() -> .js file -> import -> .vue file -> if() import -> .css file).

I'll try to investigate this further.

@stof
Copy link
Member

stof commented Feb 9, 2021

@ureimers if fonts are referenced from the CSS, they don't need to be in the map (as the map is meant for usages in Symfony's asset() function in Twig). Cache busting of fonts is unrelated to them being in the map or no.

The fact that they are in the map is related to a change in webpack-manifest-plugin that is being reverted (see the associated PR).
The fact that chunks for conditional imports are there too might be related to the same issue in the plugin.

@ureimers
Copy link

ureimers commented Feb 10, 2021

@stof You're absolutely right: As the hashed font names are already written into all built assets they aren't needed in the manifest.

Which associated PR did you mean? shellscape/webpack-manifest-plugin#249 doesn't seem to revert a change to webpack-manifest-plugin but that's the only "hot" PR I can see (and am watching) right now.

Btw. @weaverryan Great work so far Sherlock! Following your track of comments and findings through the different packages (webpack-manifest-plugin, as well as webpack itself) is like reading an investigative story :-) Sorry to hear, that npm on Windows almost killed you ;-)

@weaverryan
Copy link
Member

weaverryan commented Feb 10, 2021

Hey @ureimers!

Following your track of comments and findings through the different packages (webpack-manifest-plugin, as well as webpack itself) is like reading an investigative story :-) Sorry to hear, that npm on Windows almost killed you ;-)

Ha! I'm super happy to know that someone else followed that rabbit hole of insanity! Thanks for the nice words :)

I fixed this by using and modifying the new removeHashKey method from webpack-manifest-plugin like I described here: #808 (comment)

I think this might, unfortunately, be the "correct" solution. That removeHashKey was added to fix a bug with CopyWebpackPlugin where the same things was happening. And now that I know the code super well, it doesn't surprise me: probably webpack-manifest-plugin is simply only aware of the final filename. It may be possible to fix that in webpack-manifest-plugin, I'm not sure. There is an issue (you probably saw) on Webpack itself to make this "manifest" thing simpler - it's pretty crazy complex at the moment.

There are now three numerically named files in the manifest.json:

If you build in "dev" mode, these should have super long names - like node_modules_something_else_foo.js. You mentioned that the CSS is "conditionally" imported - I assume you mean via the import() function? Yes, that's the type of thing that causes Webpack to split into multiple files (but not the only reason). Anyways, Webpack does a lot of file splitting, and in prod mode, it gives those files numbers like this :).

Which associated PR did you mean? shellscape/webpack-manifest-plugin#249 doesn't seem to revert a change to webpack-manifest-plugin but that's the only "hot" PR I can see (and am watching) right now.

My PR has rapidly changed in the past 36 hours :)

Cheers!

@weaverryan
Copy link
Member

Hi!

Please try version 1.0.6: https://github.com/symfony/webpack-encore/releases/tag/v1.0.6

If you continue to have any manifest.json problems, please open a new issue. We rely on a 3rd party library for this feature, and we've been working with them to improve their Webpack 5 support (it's actually a surprisingly complex problem to solve).

Cheers!

@minimit
Copy link
Contributor Author

minimit commented Feb 12, 2021

Great you fixed this pretty fast with much work! 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Bug Fix
Projects
None yet
4 participants