Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(deps)!: migrate fast-glob to tinyglobby #18243

Merged
merged 21 commits into from
Oct 17, 2024
Merged
Show file tree
Hide file tree
Changes from 20 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
c08bd72
chore(deps): add `tinyglobby` in `packages/vite`
ziebam Sep 10, 2024
3f4a699
chore(deps): replace `fast-glob` with `tinyglobby`
ziebam Sep 10, 2024
2fb69ab
chore(deps): use `tinyglobby` in `be-integration`
ziebam Sep 10, 2024
90cc406
chore(deps): use `tinyglobby` in `playground/css`
ziebam Sep 10, 2024
70ce3a5
Merge 'upstream/main' into 'fast-glob-to-tinyglobby'
ziebam Sep 30, 2024
7c04f36
chore(deps): bump `tinyglobby` to 0.2.7
ziebam Sep 30, 2024
28a4b52
chore(deps): update the first glob argument
ziebam Sep 30, 2024
70ffc04
chore(deps): migrate remaining `fast-glob` usages
ziebam Sep 30, 2024
21e759e
chore(deps): update `fast-glob` references in docs
ziebam Sep 30, 2024
24a808a
chore(deps): fix failing tests
ziebam Oct 1, 2024
4799f70
chore(deps): fix another test failure
ziebam Oct 1, 2024
e6e0ab0
chore(deps): bump `tinyglobby`
ziebam Oct 1, 2024
04413fe
Merge branch 'main' of https://github.com/vitejs/vite into fast-glob-…
ziebam Oct 1, 2024
a19b70a
chore(deps): update the comment that mentions fast-glob
ziebam Oct 2, 2024
dbd3b0a
chore(deps): improve globbing in css.ts
ziebam Oct 2, 2024
ab2962f
Merge branch 'main' of https://github.com/vitejs/vite into fast-glob-…
ziebam Oct 2, 2024
905f3ee
chore(deps): simplify ignored pattern
ziebam Oct 3, 2024
d592bda
Merge branch 'main' of https://github.com/vitejs/vite into fast-glob-…
ziebam Oct 3, 2024
8a72fee
chore(deps): revert the change
ziebam Oct 3, 2024
866a27c
chore(deps): simplify globbing (again)
ziebam Oct 4, 2024
c5be962
Merge 'upstream/main' into 'fast-glob-to-tinyglobby'
ziebam Oct 17, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/config/dep-optimization-options.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

By default, Vite will crawl all your `.html` files to detect dependencies that need to be pre-bundled (ignoring `node_modules`, `build.outDir`, `__tests__` and `coverage`). If `build.rollupOptions.input` is specified, Vite will crawl those entry points instead.

If neither of these fit your needs, you can specify custom entries using this option - the value should be a [fast-glob pattern](https://github.com/mrmlnc/fast-glob#basic-syntax) or array of patterns that are relative from Vite project root. This will overwrite default entries inference. Only `node_modules` and `build.outDir` folders will be ignored by default when `optimizeDeps.entries` is explicitly defined. If other folders need to be ignored, you can use an ignore pattern as part of the entries list, marked with an initial `!`. If you don't want to ignore `node_modules` and `build.outDir`, you can specify using literal string paths (without fast-glob patterns) instead.
If neither of these fit your needs, you can specify custom entries using this option - the value should be a [`tinyglobby` pattern](https://github.com/SuperchupuDev/tinyglobby) or array of patterns that are relative from Vite project root. This will overwrite default entries inference. Only `node_modules` and `build.outDir` folders will be ignored by default when `optimizeDeps.entries` is explicitly defined. If other folders need to be ignored, you can use an ignore pattern as part of the entries list, marked with an initial `!`. If you don't want to ignore `node_modules` and `build.outDir`, you can specify using literal string paths (without `tinyglobby` patterns) instead.

## optimizeDeps.exclude

Expand Down
2 changes: 1 addition & 1 deletion docs/config/server-options.md
Original file line number Diff line number Diff line change
Expand Up @@ -184,7 +184,7 @@ The error that appears in the Browser when the fallback happens can be ignored.

Warm up files to transform and cache the results in advance. This improves the initial page load during server starts and prevents transform waterfalls.

`clientFiles` are files that are used in the client only, while `ssrFiles` are files that are used in SSR only. They accept an array of file paths or [`fast-glob`](https://github.com/mrmlnc/fast-glob) patterns relative to the `root`.
`clientFiles` are files that are used in the client only, while `ssrFiles` are files that are used in SSR only. They accept an array of file paths or [`tinyglobby`](https://github.com/SuperchupuDev/tinyglobby) patterns relative to the `root`.

Make sure to only add files that are frequently used to not overload the Vite dev server on startup.

Expand Down
2 changes: 1 addition & 1 deletion docs/guide/features.md
Original file line number Diff line number Diff line change
Expand Up @@ -542,7 +542,7 @@ Note that:

- This is a Vite-only feature and is not a web or ES standard.
- The glob patterns are treated like import specifiers: they must be either relative (start with `./`) or absolute (start with `/`, resolved relative to project root) or an alias path (see [`resolve.alias` option](/config/shared-options.md#resolve-alias)).
- The glob matching is done via [`fast-glob`](https://github.com/mrmlnc/fast-glob) - check out its documentation for [supported glob patterns](https://github.com/mrmlnc/fast-glob#pattern-syntax).
- The glob matching is done via [`tinyglobby`](https://github.com/SuperchupuDev/tinyglobby).
- You should also be aware that all the arguments in the `import.meta.glob` must be **passed as literals**. You can NOT use variables or expressions in them.

## Dynamic Import
Expand Down
29 changes: 29 additions & 0 deletions packages/vite/LICENSE.md
Original file line number Diff line number Diff line change
Expand Up @@ -2894,6 +2894,35 @@ Repository: git+https://github.com/antfu/strip-literal.git

---------------------------------------

## tinyglobby
License: MIT
By: Superchupu
Repository: git+https://github.com/SuperchupuDev/tinyglobby.git

> MIT License
>
> Copyright (c) 2024 Madeline Gurriarán
>
> Permission is hereby granted, free of charge, to any person obtaining a copy
> of this software and associated documentation files (the "Software"), to deal
> in the Software without restriction, including without limitation the rights
> to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
> copies of the Software, and to permit persons to whom the Software is
> furnished to do so, subject to the following conditions:
>
> The above copyright notice and this permission notice shall be included in all
> copies or substantial portions of the Software.
>
> THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
> IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
> FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
> AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
> LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
> OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
> SOFTWARE.

---------------------------------------

## to-regex-range
License: MIT
By: Jon Schlinkert, Rouven Weßling
Expand Down
2 changes: 1 addition & 1 deletion packages/vite/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -120,7 +120,6 @@
"escape-html": "^1.0.3",
"estree-walker": "^3.0.3",
"etag": "^1.8.1",
"fast-glob": "^3.3.2",
"http-proxy": "^1.18.1",
"launch-editor-middleware": "^2.9.1",
"lightningcss": "^1.27.0",
Expand Down Expand Up @@ -148,6 +147,7 @@
"source-map-support": "^0.5.21",
"strip-ansi": "^7.1.0",
"strip-literal": "^2.1.0",
"tinyglobby": "^0.2.9",
"tsconfck": "^3.1.3",
"tslib": "^2.7.0",
"types": "link:./types",
Expand Down
8 changes: 4 additions & 4 deletions packages/vite/src/node/optimizer/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ import colors from 'picocolors'
import type { BuildContext, BuildOptions as EsbuildBuildOptions } from 'esbuild'
import esbuild, { build } from 'esbuild'
import { init, parse } from 'es-module-lexer'
import glob from 'fast-glob'
import { isDynamicPattern } from 'tinyglobby'
import type { ResolvedConfig } from '../config'
import {
createDebugger,
Expand Down Expand Up @@ -151,8 +151,8 @@ export type DepOptimizationOptions = DepOptimizationConfig & {
* will crawl those entry points instead.
*
* If neither of these fit your needs, you can specify custom entries using
* this option - the value should be a fast-glob pattern or array of patterns
* (https://github.com/mrmlnc/fast-glob#basic-syntax) that are relative from
* this option - the value should be a tinyglobby pattern or array of patterns
* (https://github.com/SuperchupuDev/tinyglobby) that are relative from
* vite project root. This will overwrite default entries inference.
*/
entries?: string | string[]
Expand Down Expand Up @@ -826,7 +826,7 @@ export async function addManuallyIncludedOptimizeDeps(
const includes = [...optimizeDepsInclude]
for (let i = 0; i < includes.length; i++) {
const id = includes[i]
if (glob.isDynamicPattern(id)) {
if (isDynamicPattern(id)) {
const globIds = expandGlobIds(id, environment.getTopLevelConfig())
includes.splice(i, 1, ...globIds)
i += globIds.length - 1
Expand Down
33 changes: 16 additions & 17 deletions packages/vite/src/node/optimizer/resolve.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import path from 'node:path'
import glob from 'fast-glob'
import micromatch from 'micromatch'
import { globSync } from 'tinyglobby'
import type { ResolvedConfig } from '../config'
import { escapeRegex, getNpmPackageName } from '../utils'
import { resolvePackageData } from '../packages'
Expand Down Expand Up @@ -81,27 +81,24 @@ export function expandGlobIds(id: string, config: ResolvedConfig): string[] {

// "./dist/glob/*-browser/*.js" => "./dist/glob/**/*-browser/**/*.js"
// NOTE: in some cases, this could expand to consecutive /**/*/**/* etc
// but it's fine since fast-glob handles it the same.
// but it's fine since `tinyglobby` handles it the same.
const exportValuePattern = exportsValue.replace(/\*/g, '**/*')
// "./dist/glob/*-browser/*.js" => /dist\/glob\/(.*)-browser\/(.*)\.js/
const exportsValueGlobRe = new RegExp(
exportsValue.split('*').map(escapeRegex).join('(.*)'),
)

possibleExportPaths.push(
...glob
.sync(exportValuePattern, {
cwd: pkgData.dir,
ignore: ['node_modules'],
})
...globSync(exportValuePattern, {
cwd: pkgData.dir,
expandDirectories: false,
ziebam marked this conversation as resolved.
Show resolved Hide resolved
ignore: ['node_modules'],
})
.map((filePath) => {
// ensure "./" prefix for inconsistent fast-glob result
// glob.sync("./some-dir/**/*") -> "./some-dir/some-file"
// glob.sync("./**/*") -> "some-dir/some-file"
if (
exportsValue.startsWith('./') &&
!filePath.startsWith('./')
) {
// `tinyglobby` returns paths as they are formatted by the underlying `fdir`.
// Both `globSync("./some-dir/**/*")` and `globSync("./**/*")` result in
// `"some-dir/somefile"` being returned, so we ensure the correct prefix manually.
if (exportsValue.startsWith('./')) {
filePath = './' + filePath
}

Expand Down Expand Up @@ -146,9 +143,11 @@ export function expandGlobIds(id: string, config: ResolvedConfig): string[] {
return matched
} else {
// for packages without exports, we can do a simple glob
const matched = glob
.sync(pattern, { cwd: pkgData.dir, ignore: ['node_modules'] })
.map((match) => path.posix.join(pkgName, slash(match)))
const matched = globSync(pattern, {
cwd: pkgData.dir,
expandDirectories: false,
ignore: ['node_modules'],
}).map((match) => path.posix.join(pkgName, slash(match)))
matched.unshift(pkgName)
return matched
}
Expand Down
7 changes: 3 additions & 4 deletions packages/vite/src/node/optimizer/scan.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@ import fs from 'node:fs'
import fsp from 'node:fs/promises'
import path from 'node:path'
import { performance } from 'node:perf_hooks'
import glob from 'fast-glob'
import type {
BuildContext,
Loader,
Expand All @@ -13,6 +12,7 @@ import type {
import esbuild, { formatMessages, transform } from 'esbuild'
import type { PartialResolvedId } from 'rollup'
import colors from 'picocolors'
import { glob, isDynamicPattern } from 'tinyglobby'
import {
CSS_LANGS_RE,
JS_TYPES_RE,
Expand Down Expand Up @@ -332,12 +332,13 @@ function orderedDependencies(deps: Record<string, string>) {

function globEntries(pattern: string | string[], environment: ScanEnvironment) {
const resolvedPatterns = arraify(pattern)
if (resolvedPatterns.every((str) => !glob.isDynamicPattern(str))) {
if (resolvedPatterns.every((str) => !isDynamicPattern(str))) {
return resolvedPatterns.map((p) =>
normalizePath(path.resolve(environment.config.root, p)),
)
}
return glob(pattern, {
absolute: true,
cwd: environment.config.root,
ignore: [
'**/node_modules/**',
Expand All @@ -347,8 +348,6 @@ function globEntries(pattern: string | string[], environment: ScanEnvironment) {
? []
: [`**/__tests__/**`, `**/coverage/**`]),
],
absolute: true,
suppressErrors: true, // suppress EACCES errors
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Such errors are suppressed by default in tinyglobby.

})
}

Expand Down
11 changes: 5 additions & 6 deletions packages/vite/src/node/plugins/css.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@ import fsp from 'node:fs/promises'
import path from 'node:path'
import { createRequire } from 'node:module'
import { fileURLToPath, pathToFileURL } from 'node:url'
import glob from 'fast-glob'
import postcssrc from 'postcss-load-config'
import type {
ExistingRawSourceMap,
Expand All @@ -27,6 +26,7 @@ import type { TransformOptions } from 'esbuild'
import { formatMessages, transform } from 'esbuild'
import type { RawSourceMap } from '@ampproject/remapping'
import { WorkerWithFallback } from 'artichokie'
import { globSync } from 'tinyglobby'
import { getCodeWithSourcemap, injectSourcesContent } from '../server/sourcemap'
import type { EnvironmentModuleNode } from '../server/moduleGraph'
import {
Expand Down Expand Up @@ -1405,11 +1405,10 @@ async function compileCSS(
} else if (message.type === 'dir-dependency') {
// https://github.com/postcss/postcss/blob/main/docs/guidelines/plugin.md#3-dependencies
const { dir, glob: globPattern = '**' } = message
const pattern =
glob.escapePath(normalizePath(path.resolve(path.dirname(id), dir))) +
`/` +
globPattern
const files = glob.sync(pattern, {
const files = globSync(globPattern, {
absolute: true,
ziebam marked this conversation as resolved.
Show resolved Hide resolved
cwd: path.resolve(path.dirname(id), dir),
expandDirectories: false,
ignore: ['**/node_modules/**'],
})
for (let i = 0; i < files.length; i++) {
Expand Down
16 changes: 7 additions & 9 deletions packages/vite/src/node/plugins/importMetaGlob.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,10 @@ import type {
} from 'estree'
import type { CustomPluginOptions, RollupAstNode, RollupError } from 'rollup'
import MagicString from 'magic-string'
import fg from 'fast-glob'
import { stringifyQuery } from 'ufo'
import type { GeneralImportGlobOptions } from 'types/importGlob'
import { parseAstAsync } from 'rollup/parseAst'
import { escapePath, glob } from 'tinyglobby'
import type { Plugin } from '../plugin'
import type { EnvironmentModuleNode } from '../server/moduleGraph'
import type { ResolvedConfig } from '../config'
Expand Down Expand Up @@ -354,7 +354,7 @@ function findCorrespondingCloseParenthesisPosition(

const importPrefix = '__vite_glob_'

const { basename, dirname, relative, join } = posix
const { basename, dirname, relative } = posix

export interface TransformGlobImportResult {
s: MagicString
Expand Down Expand Up @@ -396,13 +396,12 @@ export async function transformGlobImport(
async ({ globsResolved, isRelative, options, index, start, end }) => {
const cwd = getCommonBase(globsResolved) ?? root
const files = (
await fg(globsResolved, {
cwd,
await glob(globsResolved, {
absolute: true,
cwd,
dot: !!options.exhaustive,
ignore: options.exhaustive
? []
: [join(cwd, '**/node_modules/**')],
expandDirectories: false,
ignore: options.exhaustive ? [] : ['**/node_modules/**'],
})
)
.filter((file) => file !== id)
Expand Down Expand Up @@ -515,8 +514,7 @@ type IdResolver = (

function globSafePath(path: string) {
// slash path to ensure \ is converted to / as \ could lead to a double escape scenario
// see https://github.com/mrmlnc/fast-glob#advanced-syntax
return fg.escapePath(normalizePath(path))
return escapePath(normalizePath(path))
ziebam marked this conversation as resolved.
Show resolved Hide resolved
}

function lastNthChar(str: string, n: number) {
Expand Down
5 changes: 3 additions & 2 deletions packages/vite/src/node/server/warmup.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import fs from 'node:fs/promises'
import path from 'node:path'
import glob from 'fast-glob'
import colors from 'picocolors'
import { glob } from 'tinyglobby'
import { FS_PREFIX } from '../constants'
import { normalizePath } from '../utils'
import type { ViteDevServer } from '../index'
Expand Down Expand Up @@ -71,7 +71,8 @@ function fileToUrl(file: string, root: string) {

function mapFiles(files: string[], root: string) {
return glob(files, {
cwd: root,
absolute: true,
cwd: root,
expandDirectories: false,
})
}
6 changes: 3 additions & 3 deletions packages/vite/src/node/watch.ts
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
import { EventEmitter } from 'node:events'
import path from 'node:path'
import glob from 'fast-glob'
import type { FSWatcher, WatchOptions } from 'dep-types/chokidar'
import type { OutputOptions } from 'rollup'
import colors from 'picocolors'
import { escapePath } from 'tinyglobby'
import { withTrailingSlash } from '../shared/utils'
import { arraify, normalizePath } from './utils'
import type { Logger } from './logger'
Expand Down Expand Up @@ -59,12 +59,12 @@ export function resolveChokidarOptions(
'**/.git/**',
'**/node_modules/**',
'**/test-results/**', // Playwright
glob.escapePath(cacheDir) + '/**',
escapePath(cacheDir) + '/**',
...arraify(ignoredList || []),
]
if (emptyOutDir) {
ignored.push(
...[...resolvedOutDirs].map((outDir) => glob.escapePath(outDir) + '/**'),
...[...resolvedOutDirs].map((outDir) => escapePath(outDir) + '/**'),
)
}

Expand Down
2 changes: 1 addition & 1 deletion playground/backend-integration/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,6 @@
"devDependencies": {
"sass": "^1.79.4",
"tailwindcss": "^3.4.13",
"fast-glob": "^3.3.2"
"tinyglobby": "^0.2.9"
}
}
10 changes: 6 additions & 4 deletions playground/backend-integration/vite.config.js
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import path from 'node:path'
import glob from 'fast-glob'
import { globSync } from 'tinyglobby'
import { defineConfig, normalizePath } from 'vite'

/**
Expand All @@ -14,9 +14,11 @@ function BackendIntegrationExample() {
const root = path.join(sourceCodeDir, 'entrypoints')
const outDir = path.relative(root, path.join(projectRoot, 'dist/dev'))

const entrypoints = glob
.sync(`${normalizePath(root)}/**/*`, { onlyFiles: true })
.map((filename) => [path.relative(root, filename), filename])
const entrypoints = globSync(`${normalizePath(root)}/**/*`, {
absolute: true,
expandDirectories: false,
onlyFiles: true,
}).map((filename) => [path.relative(root, filename), filename])

entrypoints.push(['tailwindcss-colors', 'tailwindcss/colors.js'])
entrypoints.push(['bar.css', path.resolve(__dirname, './dir/foo.css')])
Expand Down
4 changes: 2 additions & 2 deletions playground/css/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,12 @@
"@vitejs/test-css-js-dep": "file:./css-js-dep",
"@vitejs/test-css-proxy-dep": "file:./css-proxy-dep",
"@vitejs/test-scss-proxy-dep": "file:./scss-proxy-dep",
"fast-glob": "^3.3.2",
"less": "^4.2.0",
"postcss-nested": "^6.2.0",
"sass": "^1.79.4",
"stylus": "^0.63.0",
"sugarss": "^4.0.1"
"sugarss": "^4.0.1",
"tinyglobby": "^0.2.9"
},
"imports": {
"#imports": "./imports-field.css"
Expand Down
4 changes: 2 additions & 2 deletions playground/css/postcss.config.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import fs from 'node:fs'
import path from 'node:path'
import glob from 'fast-glob'
import { globSync } from 'tinyglobby'
import { normalizePath } from 'vite'
import postcssNested from 'postcss-nested'

Expand All @@ -19,7 +19,7 @@ function testDirDep() {
const pattern = normalizePath(
path.resolve(path.dirname(result.opts.from), './glob-dep/**/*.css'),
)
const files = glob.sync(pattern)
const files = globSync(pattern, { expandDirectories: false })
const text = files.map((f) => fs.readFileSync(f, 'utf-8')).join('\n')
atRule.parent.insertAfter(atRule, text)
atRule.remove()
Expand Down
Loading