Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move to manifest v3 #59

Closed
Nimpruda opened this issue Jan 5, 2022 · 48 comments
Closed

Move to manifest v3 #59

Nimpruda opened this issue Jan 5, 2022 · 48 comments

Comments

@Nimpruda
Copy link

Nimpruda commented Jan 5, 2022

With google deciding to make manifest v2 extensions (see https://developer.chrome.com/blog/mv2-transition/) wouldn't it be wise to move the repo to the new format ?
I can't find the article but I read somewhere that firefox will do the same

@catch6
Copy link

catch6 commented Jan 12, 2022

here is the Manifest V2 support timeline
image

@tmkx
Copy link
Collaborator

tmkx commented Jan 12, 2022

I've made a simple migration on this branch, but it can work on build only.

HMR is also been unavailable, see:
https://bugs.chromium.org/p/chromium/issues/detail?id=1247690

If anyone is familiar with mv3, PR is welcome. ❤️

@Serendipity96
Copy link

I've made a simple migration on this branch, but it can work on build only.

HMR is also been unavailable, see: https://bugs.chromium.org/p/chromium/issues/detail?id=1247690

If anyone is familiar with mv3, PR is welcome. ❤️

hi, @tmkx , thanks for your template.

When I coding, I replaced "dev": "npm run clear && cross-env NODE_ENV=development run-p dev:*" with "dev": "vite build --watch" without starting localhost.

The chrome extension compiled successfully and supported HMR.

I don't know if it is a good solution, if so, we can simplify the template dev-related script together.

@aliuq
Copy link

aliuq commented Apr 14, 2022

I think because of this reason, in manifest v3, we can no longer load script from remote server

@tmkx tmkx pinned this issue Apr 17, 2022
@coure2011
Copy link

@tmkx thx for providing a workaround, at least we can build the extension.
Any way we can achieve the HMR using the dev mode?

@tmkx
Copy link
Collaborator

tmkx commented Apr 19, 2022

@tmkx thx for providing a workaround, at least we can build the extension. Any way we can achieve the HMR using the dev mode?

Reference: https://groups.google.com/a/chromium.org/g/chromium-extensions/c/pYtdXH0f46E

@coure2011
Copy link

btw, Chrome store is no longer accepting v2 extensions and this repo is somewhat not use able out of box for new extensions.
image

@coure2011
Copy link

I've made a simple migration on this branch, but it can work on build only.

HMR is also been unavailable, see: https://bugs.chromium.org/p/chromium/issues/detail?id=1247690

If anyone is familiar with mv3, PR is welcome. ❤️

Using ur branch it builds the extension, after now if an extension remain idle for some ~5mins, it throws error whenever it sendsMessage
index.global.js:1 Uncaught (in promise) Error: Attempting to use a disconnected port object

@coure2011
Copy link

Found that the disconnected port object error is fixed in v. 5.0.4 of webext-bridge
serversideup/webext-bridge#18

@lautr
Copy link

lautr commented May 11, 2022

For people also waiting for the HMR issue to be resolved, a working nodemon command for the prod build:

pnpm add nodemon

pnpm nodemon --ignore "*.d.ts" --watch "src" -e ts,js,mjs,json,vue --exec "pnpm build"

@xlzy520
Copy link

xlzy520 commented May 15, 2022

@lautr
image

image

@lautr
Copy link

lautr commented May 16, 2022

@xlzy520
You wrote it in a way that triggers recursion, since your script is called nodemon which is the name of the command its calling, so it ends up calling itself, naming it something else should solve this issue.

@xlzy520
Copy link

xlzy520 commented May 17, 2022

@lautr sorry.

@tmkx
Copy link
Collaborator

tmkx commented May 21, 2022

I've made a simple migration on this branch, but it can work on build only.

Hi all, branch refactor/mv3 supports HMR now

changes: main...refactor/mv3


For anyone who is interested in how did that:

In MV3, we can't use dynamic codes (eval, server file, CDN, .etc) anymore, so we need to put files to the file system, just like devServer.devMiddleware.writeToDisk in Webpack. Vite doesn't provide this ability directly, thankfully we can do the same thing with the help of configureServer hook.

After putting files to FS, we are facing another problem: file extension. The HTTP server can use Content-Type to tell the browser the correct file type, even if the file extension is not match. According to that, we need to append .js by hand in the correct time to make the browser can recognize .ts/virtual files correctly.

@MccRay-s
Copy link

I've made a simple migration on this branch, but it can work on build only.

Hi all, branch refactor/mv3 supports HMR now

changes: main...refactor/mv3

For anyone who is interested in how did that:

In MV3, we can't use dynamic codes (eval, server file, CDN, .etc) anymore, so we need to put files to the file system, just like devServer.devMiddleware.writeToDisk in Webpack. Vite doesn't provide this ability directly, thankfully we can do the same thing with the help of configureServer hook.

After putting files to FS, we are facing another problem: file extension. The HTTP server can use Content-Type to tell the browser the correct file type, even if the file extension is not match. According to that, we need to append .js by hand in the correct time to make the browser can recognize .ts/virtual files correctly.

@tmkx
尝试运行 pnpm run dev

Waiting for the debugger to disconnect...
E:\code\my\webext\vitesse-webext\node_modules.pnpm\[email protected]\node_modules\fs-extra\lib\mkdirs\utils.js:16
const error = new Error(Path contains invalid characters: ${pth})
^
Error: Path contains invalid characters: E:\code\my\webext\vitesse-webext\extension@fs\E:\code\my\webext\vitesse-webext\node_modules.vite\deps
at checkPath (E:\code\my\webext\vitesse-webext\node_modules.pnpm\[email protected]\node_modules\fs-extra\lib\mkdirs\utils.js:16:21)
at Object.module.exports.makeDir (E:\code\my\webext\vitesse-webext\node_modules.pnpm\[email protected]\node_modules\fs-extra\lib\mkdirs\make-dir.js:12:3)
at Object.defineProperty.value (E:\code\my\webext\vitesse-webext\node_modules.pnpm\[email protected]\node_modules\universalify\index.js:21:45)
at E:\code\my\webext\vitesse-webext\vite.config.ts:145:45
at Generator.next ()
at fulfilled (E:\code\my\webext\vitesse-webext\vite.config.ts:42:24) {
code: 'EINVAL'
}

应该是 vite-mv3-hmr.ts 内 await fs.ensureDir(dirname(targetFile))

@tmkx
Copy link
Collaborator

tmkx commented May 23, 2022

应该是 vite-mv3-hmr.ts 内 await fs.ensureDir(dirname(targetFile))

Sorry 我还没在 Windows 上验证过,你试试 WSL 能不能运行,我回头找个 Win 电脑看下,现在居家 🥲

Sorry, I haven't verified it on Windows, please try WSL, I will find a Win computer to check, WFH now..

@eugenesvk
Copy link

@MccRay-s you could try running with this PR, should resolve your error with pnpm run dev Error(Path contains invalid characters: ${pth})

@xlzy520
Copy link

xlzy520 commented Jun 8, 2022

"scripts": {
    "dev": "npm run clear && cross-env NODE_ENV=development run-p dev:*",
    "dev:prepare": "esno scripts/prepare.ts",
    "dev:web": "vite",
    "dev:js": "npm run build:js -- --mode development",
    "dev:bg": "npm run build:bg -- --mode development",
    "build": "cross-env NODE_ENV=production run-s clear build:web build:prepare build:js build:bg",
    "build:prepare": "esno scripts/prepare.ts",
    "build:web": "vite build",
    "build:js": "vite build --config vite.config.content.ts",
    "build:bg": "vite build --config vite.config.bg.ts",
    "watch:bg": "nodemon --ignore \"*.d.ts\" --watch \"src/background\" -e ts,js,mjs,json,vue --exec \"pnpm dev:bg\""
  }

use watch:bg

@xlzy520
Copy link

xlzy520 commented Jun 8, 2022

build

我只调试background的时候,这样可以加快速度

@lautr
Copy link

lautr commented Jun 10, 2022

@MccRay-s you could try running with this PR, should resolve your error with pnpm run dev Error(Path contains invalid characters: ${pth})

just tried the new version and now getting this on windows, any ideas?

build started...

  vite v2.9.10 dev server running at:

  > Local: http://localhost:3303/
  > Network: use `--host` to expose

  ready in 5901ms.

transforming (1) contentScripts\index.ts PRE  write manifest.json
transforming (6) ..\..\..\node_modules\.pnpm\[email protected]\node_modules\vue\dist\vue.runtime.esm-bund PRE 
 stub options
 PRE  stub popup
transforming (13) contentScripts\views\App.vuenode:internal/process/promises:279
            triggerUncaughtException(err, true /* fromPromise */);
            ^

[Error: UNKNOWN: unknown error, open 'C:\Users\xxx\Documents\GitHub\app-v4\packages\webext\extension\@fs\Users\xxx\Documents\GitHub\app-v4\packages\webext\node_modules\.vite\deps\vue.js'] {
  errno: -4094,
  code: 'UNKNOWN',
  syscall: 'open',
  path: 'C:\\Users\\xxx\\Documents\\GitHub\\app-v4\\packages\\webext\\extension\\@fs\\Users\\xxx\\Documents\\GitHub\\app-v4\\packages\\webext\\node_modules\\.vite\\deps\\vue.js'
}
 ELIFECYCLE  Command failed with exit code 1.
transforming (17) ..\~icons\pixelarticons\powerERROR: "dev:web" exited with 1.
 ELIFECYCLE  Command failed with exit code 1.

@eugenesvk
Copy link

how would I know, it's an UNKNOWN error :)
this branch works for me as is (with yarn and node_modules) until it crashes later due to another Windows fswatcher bug

You could try this branch with yarn PNP, maybe it would work?

@ManUtopiK
Copy link
Contributor

Hi!
Just an idea, I just found this file from PlasmoHQ that auto-generate manifest V3. Maybe it's worth taking a look...

@zmwangx
Copy link

zmwangx commented Jun 16, 2022

@tmkx Thanks for the work on manifest v3! I tried the refactor/mv3 branch (61b47be at the time of writing) and found two problems:

  1. src/background/contentScriptHMR.ts got removed and the functionality was never restored anywhere, so it seems content scripts are never injected into the page in dev mode. A quick patch of mine:

    diff --git a/src/background/contentScriptHMR.ts b/src/background/contentScriptHMR.ts
    new file mode 100644
    index 0000000..00687cb
    --- /dev/null
    +++ b/src/background/contentScriptHMR.ts
    @@ -0,0 +1,21 @@
    +import browser from 'webextension-polyfill'
    +import { isFirefox, isForbiddenUrl } from '~/env'
    +
    +// Firefox fetch files from cache instead of reloading changes from disk,
    +// hmr will not work as Chromium based browser
    +browser.webNavigation.onCommitted.addListener(({ tabId, frameId, url }) => {
    +  // Filter out non main window events.
    +  if (frameId !== 0)
    +    return
    +
    +  if (isForbiddenUrl(url))
    +    return
    +
    +  // inject the latest scripts
    +  browser.scripting
    +    .executeScript({
    +      target: { tabId },
    +      files: [`${isFirefox ? '' : '.'}/dist/contentScripts/index.global.js`],
    +    })
    +    .catch(error => console.error(error))
    +})
    diff --git a/src/background/index.ts b/src/background/index.ts
    index 3b28a58..d2e14e1 100644
    --- a/src/background/index.ts
    +++ b/src/background/index.ts
    @@ -2,6 +2,9 @@ import type { Tabs } from 'webextension-polyfill'
     import browser from 'webextension-polyfill'
     import { onMessage, sendMessage } from 'webext-bridge'
     
    +if (__DEV__)
    +  import('./contentScriptHMR')
    +
     browser.runtime.onInstalled.addListener((): void => {
       // eslint-disable-next-line no-console
       console.log('Extension installed')
    diff --git a/src/manifest.ts b/src/manifest.ts
    index 52b947a..c52ab00 100644
    --- a/src/manifest.ts
    +++ b/src/manifest.ts
    @@ -60,7 +60,7 @@ export async function getManifest() {
         // we use a background script to always inject the latest version
         // see src/background/contentScriptHMR.ts
         delete manifest.content_scripts
    -    manifest.permissions?.push('webNavigation')
    +    manifest.permissions?.push('scripting', 'webNavigation')
       }
     
       return manifest

    Note: this will inject the content script into any page under host_permissions; you may need to further limit injection based on url.

  2. I tried to use this template with Tailwind v3 + PostCSS (the standard setup: https://tailwindcss.com/docs/guides/vite) instead of Windi.css because I'm more familiar with Tailwind and v3 has more features I like. Unfortunately it failed spectacularly with the MV3Hmr plugin. Presumably due to some weird interaction between Tailwind's JIT and the MV3Hmr plugin's way to detect changes, writeToDisk is called in a seemingly infinite loop, and the node process quickly runs out of memory and crashes. I didn't further investigate this.

@gwokhov
Copy link

gwokhov commented Jun 27, 2022

I found that filename like Options.vue?vue&type=style&index=0&lang.css.js(style from SFC) which has query string cannot be loaded expectably.

@amirhoseinsalimi
Copy link

I'm having the exact same problems as you @zmwangx. Did you find a solution for these 2 problems?

@zmwangx
Copy link

zmwangx commented Jun 28, 2022

I'm having the exact same problems as you @zmwangx. Did you find a solution for these 2 problems?

I already posted a patch for the first problem.

I didn't look into the second one, I just stuck to Windi even though it has some shortcomings.

@amirhoseinsalimi
Copy link

Thank you, man.

@lautr
Copy link

lautr commented Aug 2, 2022

For everyone like me that still has failing dev builds to to file system issues on windows, this helped me

install concurrently

pnpm add -d concurrently 

entry in package.json

"dev": "pnpm run clear && cross-env NODE_ENV=development && concurrently --restart-tries -1 --restart-delay 3000 pnpm:dev:*",

@zmwangx
Copy link

zmwangx commented Aug 23, 2022

Noticed another problem with refactor/mv3: global defines aren't available to option, popup, etc., since compared to the normal @vite/client, mv3client.mjs does not import vite/src/client/env.ts where the __DEFINES__ are. Not sure what's the best solution, but a quick and dirty way to fix this is to manually bind each define to window in scripts/client.ts, e.g.

declare global {
  interface Window {
    __APP_VERSION__: string;
  }
}

window.__APP_VERSION__ = __APP_VERSION__;

@iMuFeng
Copy link

iMuFeng commented Aug 26, 2022

@zmwangx here is my solution for problem 2:

const cacheMap = new Map<string, string>()

async function writeToDisk(url: string) {
  const result = await server.transformRequest(url.replace(/^\/@id\//, ''))

  if (result?.etag && result?.etag === cacheMap.get(url))
    return

  // Add to cache
  cacheMap.set(url, result?.etag || '')

  // ....
}

@HelloAlexPan
Copy link

HelloAlexPan commented Oct 4, 2022

Hey guys — currently developing a webextension using this template and 85% through stumbled across this thread 🤦‍♂️

Is there anything we can help with re porting this template to MV3? Is the MV3 branch ok for production?

@ChxGuillaume
Copy link

Is there anything we can help with re porting this template to MV3? Is the MV3 branch ok for production?

I would say it is really close to, I am redoing an extension with it, so far doing great when not taking in consideration those bullshit MV3 changes 😄

@farazshuja
Copy link

My extension (mv3) is already on chrome store. Everything other than HMR is working fine!

@tjx666
Copy link

tjx666 commented Nov 20, 2022

vite had published v3, current scripts/client.ts should be updated.
What's your main change in that file? @tmkx

@tmkx
Copy link
Collaborator

tmkx commented Nov 20, 2022

vite had published v3, current scripts/client.ts should be updated. What's your main change in that file? @tmkx

https://github.com/antfu/vitesse-webext/tree/refactor/mv3 has upgraded vite to v3 now.

@tjx666
Copy link

tjx666 commented Nov 20, 2022

@tmkx latest commit has two problems:

  1. hmr doesn't work, when I modify options/main.ts, will not regenerate latest options/main.ts.js in the extension folder
  2. not suport sourcemap

I have some other ideas:

  1. maybe vite should builtin support write to disk feature @antfu, or publish vite-mv3-hmr.ts to a npm vite plugin
  2. every time the vite upgrade, we need to check the scripts/client, I think this is not a good solution to deal with hmr.
  3. I check the scripts/client.ts, but not vary understand why wee need change the default client.ts

@id3vz
Copy link

id3vz commented Nov 25, 2022

anyone managed to get a working example with mv3?

@GarinZ
Copy link

GarinZ commented Dec 3, 2022

  1. hmr doesn't work, when I modify options/main.ts, will not regenerate latest options/main.ts.js in the extension folder

I have exactly the same problem @tmkx

@seahindeniz
Copy link

Hey @antfu, any plans on supporting this thread? It's nearly 2023 and the steps taken may require your attention

@seahindeniz
Copy link

seahindeniz commented Dec 9, 2022

Seems like manifest v3 isn't supported by Firefox 💀

Error: Unsupported manifest version: 3

terminal output
➜  test-extension-2 git:(test-release) ✗ npm run start:firefox

> [email protected] start:firefox
> web-ext run --source-dir ./extension --target=firefox-desktop

Applying config file: ./package.json
Running web extension from /Users/sahindeniz/projects/test-extension-2/extension
Use --verbose or --devtools to see logging

WebExtError: installTemporaryAddon: Error: Error: Could not install add-on at '/Users/sahindeniz/projects/test-extension-2/extension': Error: Unsupported manifest version: 3
    at RemoteFirefox.installTemporaryAddon (file:///Users/sahindeniz/projects/test-extension-2/node_modules/web-ext/lib/firefox/remote.js:90:13)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async FirefoxDesktopExtensionRunner.startFirefoxInstance (file:///Users/sahindeniz/projects/test-extension-2/node_modules/web-ext/lib/extension-runners/firefox-desktop.js:218:27)
    at async FirefoxDesktopExtensionRunner.run (file:///Users/sahindeniz/projects/test-extension-2/node_modules/web-ext/lib/extension-runners/firefox-desktop.js:51:5)
    at async Promise.all (index 0)
    at async MultiExtensionRunner.run (file:///Users/sahindeniz/projects/test-extension-2/node_modules/web-ext/lib/extension-runners/index.js:68:5)
    at async run (file:///Users/sahindeniz/projects/test-extension-2/node_modules/web-ext/lib/cmd/run.js:178:3)
    at async Program.execute (file:///Users/sahindeniz/projects/test-extension-2/node_modules/web-ext/lib/program.js:263:7)
    at async file:///Users/sahindeniz/projects/test-extension-2/node_modules/web-ext/bin/web-ext.js:13:1

Upstream library web-ext has a solution for this problem, mentioned in this comment
However, we still need to maintain two manifest files for each browser. service_worker isn't yet supported https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/manifest.json/background#browser_compatibility
Here's the workaround mozilla/web-ext#2379 (comment)

@iMuFeng
Copy link

iMuFeng commented Dec 9, 2022

@seahindeniz

This comment was marked as off-topic.

@Elinia
Copy link

Elinia commented Dec 20, 2022

https://bugs.chromium.org/p/chromium/issues/detail?id=1247690#c35

the CSP has been updated to allow localhost entries for unpacked extensions.

It seems the issue that MV3 CSP doesn't allow localhost scripts for unpacked extensions has been solved in chromium.

@seahindeniz
Copy link

MV3 sunset is under review. I guess we can expect some breaking changes
https://developer.chrome.com/docs/extensions/mv3/mv2-sunset/
https://groups.google.com/u/1/a/chromium.org/g/chromium-extensions/c/zQ77HkGmK9E
image

@tjx666
Copy link

tjx666 commented Feb 14, 2023

How come webpack implementation make use of HMR with MV3 while Vite can't?

Because Vite doesn't support write requested file to disk

@ameinhardt
Copy link

ameinhardt commented Feb 14, 2023

Thanks for this good inititive!
I also had problems with scss and transforming cascaded mixins that were 'inherited' by imports from some parent files. Maybe
https://github.com/antfu/vitesse-webext/blob/cfb69ba1d3a695cb5207311d286c63bee5406143/vite-mv3-hmr.ts#L51
does not have to dive to full depth, e.g.

const hasImport = !!code && (code.includes(mod.url) || mod.url.charCodeAt(0) === 0)
if (!hasImport)
  continue

Also, to me it seems the response of configureServer() is too early, as dependencies are not analysed yet. Should it rather be?:

async configureServer(_server) {
  server = _server
},
async buildStart() {
...
  await Promise.all(Object.keys(server.config.build.rollupOptions.input!).map(entry => writeToDisk(`/${entry}/main.ts`)))
}

As a minor suggestion, it might be possible to generalize the escaping

// .replace(/__x00__plugin-vue:export-helper/g, '~~x00__plugin-vue:export-helper.js')
.replace(/__x00__([-:_\w]+)/g, '~~x00__$1.js')

At the end, I found crxjs to be working well

@tmkx
Copy link
Collaborator

tmkx commented May 8, 2023

hello all, Chromium has allowed localhost sources in extension CSP, and I tested in the latest Edge version(113), HMR also works, so there's no need for the trick anymore. The main branch has been upgraded #124.

https://chromium.googlesource.com/chromium/src/+/b81d6dcb2f2a89075b2bd8619b59bbf00994a01d

@tmkx tmkx closed this as completed May 8, 2023
@tmkx tmkx unpinned this issue May 8, 2023
@myfunc
Copy link

myfunc commented Mar 20, 2024

@MccRay-s you could try running with this PR, should resolve your error with pnpm run dev Error(Path contains invalid characters: ${pth})

just tried the new version and now getting this on windows, any ideas?

build started...

  vite v2.9.10 dev server running at:

  > Local: http://localhost:3303/
  > Network: use `--host` to expose

  ready in 5901ms.

transforming (1) contentScripts\index.ts PRE  write manifest.json
transforming (6) ..\..\..\node_modules\.pnpm\[email protected]\node_modules\vue\dist\vue.runtime.esm-bund PRE 
 stub options
 PRE  stub popup
transforming (13) contentScripts\views\App.vuenode:internal/process/promises:279
            triggerUncaughtException(err, true /* fromPromise */);
            ^

[Error: UNKNOWN: unknown error, open 'C:\Users\xxx\Documents\GitHub\app-v4\packages\webext\extension\@fs\Users\xxx\Documents\GitHub\app-v4\packages\webext\node_modules\.vite\deps\vue.js'] {
  errno: -4094,
  code: 'UNKNOWN',
  syscall: 'open',
  path: 'C:\\Users\\xxx\\Documents\\GitHub\\app-v4\\packages\\webext\\extension\\@fs\\Users\\xxx\\Documents\\GitHub\\app-v4\\packages\\webext\\node_modules\\.vite\\deps\\vue.js'
}
 ELIFECYCLE  Command failed with exit code 1.
transforming (17) ..\~icons\pixelarticons\powerERROR: "dev:web" exited with 1.
 ELIFECYCLE  Command failed with exit code 1.

It happens cause @crxj/vite-plugin uses require("path").posix.isAbsolute that doesn't work correctly on Windows.
For windows require("path").isAbsolute should be used.
2 versions of isAbsolute works in different ways. posix version doesn't detect absolute win32 path as absolute.

https://github.com/crxjs/chrome-extension-tools/blob/main/packages/vite-plugin/src/node/plugin-hmr.ts#L49
In that part path.posix.isAbsolute is used instead of path.isAbsolute
crxjs/chrome-extension-tools#473

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests