Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Install state file #998

Merged
merged 47 commits into from
Feb 27, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
f8029cf
Add virtual state in options
crubier Feb 24, 2020
d85c40b
Change some stuff
crubier Feb 24, 2020
d0771b3
Update doc
crubier Feb 24, 2020
45311ac
Tests
crubier Feb 24, 2020
bb92de8
again
crubier Feb 24, 2020
8c53164
ok
crubier Feb 24, 2020
dbfd9dc
Come on
crubier Feb 24, 2020
dca63ab
Sneaky Typescript
crubier Feb 24, 2020
2f1d184
Made generateVirtualState look like generateLockFile
crubier Feb 24, 2020
a66e6e8
Add tim
crubier Feb 24, 2020
ca7eb66
time
crubier Feb 24, 2020
e6d309d
find timing
crubier Feb 24, 2020
88d9bd9
Fix
crubier Feb 24, 2020
ca0c227
Generate the right virtual state file
crubier Feb 24, 2020
e639b46
Fix
crubier Feb 24, 2020
821647a
Compare old vs new option
crubier Feb 24, 2020
f969b66
No battery left on laptop, think Im almost done
crubier Feb 24, 2020
d656142
Trying to fix
crubier Feb 25, 2020
1c1e104
Fixes the serialization / hydratation
arcanis Feb 26, 2020
6a4187e
Merge pull request #2 from yarnpkg/mael/crubier-work
crubier Feb 26, 2020
dd7861e
Add timing
crubier Feb 26, 2020
f5d1a41
Merge branch 'hydrate-virtual-packages' into install-state-file
crubier Feb 26, 2020
be4a095
Persist all like crazy
crubier Feb 26, 2020
ae518e4
Call restore on load
crubier Feb 26, 2020
8f2af6a
Restore should start working...
crubier Feb 26, 2020
85ba5e8
RestoreState does not work...
crubier Feb 26, 2020
679272e
Merge branch 'master' into install-state-file
crubier Feb 26, 2020
9960569
JSON for install state
crubier Feb 26, 2020
dc8a6da
Works! :pray:
crubier Feb 26, 2020
f2d6e82
update test
crubier Feb 26, 2020
7f308f1
Cleaning
crubier Feb 26, 2020
c48c6a8
Optimized serialization
crubier Feb 27, 2020
63c8fc3
Remove install state
crubier Feb 27, 2020
0e33dba
doc
crubier Feb 27, 2020
f5c0d97
Clean
crubier Feb 27, 2020
16c12aa
Enable restoreInstallState on all commands
crubier Feb 27, 2020
50249c0
Cleanup
crubier Feb 27, 2020
15f4d99
More cleanup
crubier Feb 27, 2020
68efc66
Control checksums to ensure cache is up to date
crubier Feb 27, 2020
fff0881
Small additional optimization
crubier Feb 27, 2020
1f04daa
clean dirty stuff
crubier Feb 27, 2020
1f1c42b
Slight changes
arcanis Feb 27, 2020
61ca9e4
Adds versions
arcanis Feb 27, 2020
2948411
Remove test "echo" script from package.json
crubier Feb 27, 2020
7ff03ed
Fixes dlx
arcanis Feb 27, 2020
3fcbea3
Fixes constraints
arcanis Feb 27, 2020
169ead1
Merge branch 'install-state-file' of github.com:crubier/berry into pr…
arcanis Feb 27, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,9 @@ package.tgz
# This is the Yarn build state; it's local to each clone
/.yarn/build-state.yml

# This is the Yarn install state cache, it can be rebuilt anytime
/.yarn/install-state.gz

# Those files are meant to be local-only
/packages/*/bundles

Expand Down
1 change: 1 addition & 0 deletions .pnp.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

29 changes: 29 additions & 0 deletions .yarn/versions/1e50b08a.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
releases:
"@yarnpkg/cli": prerelease
"@yarnpkg/core": prerelease
"@yarnpkg/plugin-dlx": prerelease
"@yarnpkg/plugin-essentials": prerelease
"@yarnpkg/plugin-npm-cli": prerelease
"@yarnpkg/plugin-pack": prerelease
"@yarnpkg/plugin-patch": prerelease
"@yarnpkg/plugin-version": prerelease

declined:
- "@yarnpkg/plugin-compat"
- "@yarnpkg/plugin-constraints"
- "@yarnpkg/plugin-exec"
- "@yarnpkg/plugin-file"
- "@yarnpkg/plugin-git"
- "@yarnpkg/plugin-github"
- "@yarnpkg/plugin-http"
- "@yarnpkg/plugin-init"
- "@yarnpkg/plugin-interactive-tools"
- "@yarnpkg/plugin-link"
- "@yarnpkg/plugin-node-modules"
- "@yarnpkg/plugin-npm"
- "@yarnpkg/plugin-pnp"
- "@yarnpkg/plugin-stage"
- "@yarnpkg/plugin-typescript"
- "@yarnpkg/plugin-workspace-tools"
- "@yarnpkg/builder"
- "@yarnpkg/doctor"
12 changes: 7 additions & 5 deletions packages/gatsby/content/advanced/questions-and-answers.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,16 +49,18 @@ Lockfiles should **always** be kept within the repository. Continuous integratio
If you're using Zero-Installs:

```gitignore
.yarn/unplugged
.yarn/build-state.yml
.yarn/*
!.yarn/cache
!.yarn/releases
!.yarn/plugins
```

If you're not using Zero-Installs:

```gitignore
.yarn/cache
.yarn/unplugged
.yarn/build-state.yml
.yarn/*
!.yarn/releases
!.yarn/plugins
.pnp.*
```

Expand Down
7 changes: 2 additions & 5 deletions packages/plugin-dlx/sources/commands/dlx.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import {BaseCommand, WorkspaceRequiredError} from '@yarnpkg/cli';
import {Configuration, Project, ThrowReport} from '@yarnpkg/core';
import {Configuration, Project} from '@yarnpkg/core';
import {scriptUtils, structUtils} from '@yarnpkg/core';
import {Filename, PortablePath, npath, ppath, toFilename, xfs} from '@yarnpkg/fslib';
import {Command, Usage} from 'clipanion';
Expand Down Expand Up @@ -64,10 +64,7 @@ export default class DlxCommand extends BaseCommand {
if (workspace === null)
throw new WorkspaceRequiredError(project.cwd, tmpDir);

await project.resolveEverything({
lockfileOnly: true,
report: new ThrowReport(),
});
await project.restoreInstallState();

return await scriptUtils.executeWorkspaceAccessibleBinary(workspace, command, this.args, {
cwd: this.context.cwd,
Expand Down
13 changes: 5 additions & 8 deletions packages/plugin-essentials/sources/commands/bin.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import {BaseCommand} from '@yarnpkg/cli';
import {Configuration, Project, ThrowReport, StreamReport} from '@yarnpkg/core';
import {scriptUtils, structUtils} from '@yarnpkg/core';
import {Command, Usage, UsageError} from 'clipanion';
import {BaseCommand} from '@yarnpkg/cli';
import {Configuration, Project, StreamReport} from '@yarnpkg/core';
import {scriptUtils, structUtils} from '@yarnpkg/core';
import {Command, Usage, UsageError} from 'clipanion';

// eslint-disable-next-line arca/no-default-export
export default class BinCommand extends BaseCommand {
Expand Down Expand Up @@ -37,10 +37,7 @@ export default class BinCommand extends BaseCommand {
const configuration = await Configuration.find(this.context.cwd, this.context.plugins);
const {project, locator} = await Project.find(configuration, this.context.cwd);

await project.resolveEverything({
lockfileOnly: true,
report: new ThrowReport(),
});
await project.restoreInstallState();

if (this.name) {
const binaries = await scriptUtils.getPackageAccessibleBinaries(locator, {project});
Expand Down
15 changes: 6 additions & 9 deletions packages/plugin-essentials/sources/commands/run.ts
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
import {BaseCommand} from '@yarnpkg/cli';
import {Configuration, Project, Workspace, ThrowReport} from '@yarnpkg/core';
import {scriptUtils, structUtils} from '@yarnpkg/core';
import {Command, Usage, UsageError} from 'clipanion';
import {BaseCommand} from '@yarnpkg/cli';
import {Configuration, Project, Workspace} from '@yarnpkg/core';
import {scriptUtils, structUtils} from '@yarnpkg/core';
import {Command, Usage, UsageError} from 'clipanion';

import {pluginCommands} from '../pluginCommands';
import {pluginCommands} from '../pluginCommands';

// eslint-disable-next-line arca/no-default-export
export default class RunCommand extends BaseCommand {
Expand Down Expand Up @@ -67,10 +67,7 @@ export default class RunCommand extends BaseCommand {
const configuration = await Configuration.find(this.context.cwd, this.context.plugins);
const {project, workspace, locator} = await Project.find(configuration, this.context.cwd);

await project.resolveEverything({
lockfileOnly: true,
report: new ThrowReport(),
});
await project.restoreInstallState();

const effectiveLocator = this.topLevel
? project.topLevelWorkspace.anchoredLocator
Expand Down
19 changes: 8 additions & 11 deletions packages/plugin-essentials/sources/commands/why.ts
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
import {BaseCommand, WorkspaceRequiredError} from '@yarnpkg/cli';
import {Configuration, LocatorHash, Package, ThrowReport} from '@yarnpkg/core';
import {IdentHash, Project} from '@yarnpkg/core';
import {miscUtils, structUtils} from '@yarnpkg/core';
import {Command, Usage} from 'clipanion';
import {Writable} from 'stream';
import {asTree} from 'treeify';
import {BaseCommand, WorkspaceRequiredError} from '@yarnpkg/cli';
import {Configuration, LocatorHash, Package} from '@yarnpkg/core';
import {IdentHash, Project} from '@yarnpkg/core';
import {miscUtils, structUtils} from '@yarnpkg/core';
import {Command, Usage} from 'clipanion';
import {Writable} from 'stream';
import {asTree} from 'treeify';

type TreeNode = {[key: string]: TreeNode};

Expand Down Expand Up @@ -42,10 +42,7 @@ export default class WhyCommand extends BaseCommand {
if (!workspace)
throw new WorkspaceRequiredError(project.cwd, this.context.cwd);

await project.resolveEverything({
lockfileOnly: true,
report: new ThrowReport(),
});
await project.restoreInstallState();

const identHash = structUtils.parseIdent(this.package).identHash;

Expand Down
21 changes: 9 additions & 12 deletions packages/plugin-npm-cli/sources/commands/npm/publish.ts
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
import {BaseCommand, WorkspaceRequiredError} from '@yarnpkg/cli';
import {Configuration, MessageName, Project, ReportError, StreamReport, Workspace, ThrowReport} from '@yarnpkg/core';
import {miscUtils, structUtils} from '@yarnpkg/core';
import {npmConfigUtils, npmHttpUtils} from '@yarnpkg/plugin-npm';
import {packUtils} from '@yarnpkg/plugin-pack';
import {Command, Usage, UsageError} from 'clipanion';
import {createHash} from 'crypto';
import ssri from 'ssri';
import {BaseCommand, WorkspaceRequiredError} from '@yarnpkg/cli';
import {Configuration, MessageName, Project, ReportError, StreamReport, Workspace} from '@yarnpkg/core';
import {miscUtils, structUtils} from '@yarnpkg/core';
import {npmConfigUtils, npmHttpUtils} from '@yarnpkg/plugin-npm';
import {packUtils} from '@yarnpkg/plugin-pack';
import {Command, Usage, UsageError} from 'clipanion';
import {createHash} from 'crypto';
import ssri from 'ssri';

// eslint-disable-next-line arca/no-default-export
export default class NpmPublishCommand extends BaseCommand {
Expand Down Expand Up @@ -47,10 +47,7 @@ export default class NpmPublishCommand extends BaseCommand {
if (workspace.manifest.name === null || workspace.manifest.version === null)
throw new UsageError(`Workspaces must have valid names and versions to be published on an external registry`);

await project.resolveEverything({
lockfileOnly: true,
report: new ThrowReport(),
});
await project.restoreInstallState();

// We store it so that TS knows that it's non-null
const ident = workspace.manifest.name;
Expand Down
5 changes: 1 addition & 4 deletions packages/plugin-pack/sources/commands/pack.ts
Original file line number Diff line number Diff line change
Expand Up @@ -60,10 +60,7 @@ export default class PackCommand extends BaseCommand {
report: new ThrowReport(),
});
} else {
await project.resolveEverything({
lockfileOnly: true,
report: new ThrowReport(),
});
await project.restoreInstallState();
}
}

Expand Down
15 changes: 6 additions & 9 deletions packages/plugin-patch/sources/commands/patch.ts
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
import {BaseCommand, WorkspaceRequiredError} from '@yarnpkg/cli';
import {Cache, Configuration, Project, ThrowReport, structUtils, StreamReport, MessageName} from '@yarnpkg/core';
import {npath} from '@yarnpkg/fslib';
import {Command, Usage, UsageError} from 'clipanion';
import {BaseCommand, WorkspaceRequiredError} from '@yarnpkg/cli';
import {Cache, Configuration, Project, structUtils, StreamReport, MessageName} from '@yarnpkg/core';
import {npath} from '@yarnpkg/fslib';
import {Command, Usage, UsageError} from 'clipanion';

import * as patchUtils from '../patchUtils';
import * as patchUtils from '../patchUtils';

// eslint-disable-next-line arca/no-default-export
export default class PatchCommand extends BaseCommand {
Expand All @@ -25,10 +25,7 @@ export default class PatchCommand extends BaseCommand {
if (!workspace)
throw new WorkspaceRequiredError(project.cwd, this.context.cwd);

await project.resolveEverything({
lockfileOnly: true,
report: new ThrowReport(),
});
await project.restoreInstallState();

let locator = structUtils.parseLocator(this.package);

Expand Down
15 changes: 6 additions & 9 deletions packages/plugin-patch/sources/commands/patchCommit.ts
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
import {BaseCommand, WorkspaceRequiredError} from '@yarnpkg/cli';
import {Cache, Configuration, Project, ThrowReport, structUtils, execUtils, miscUtils} from '@yarnpkg/core';
import {npath, xfs, Filename, ppath} from '@yarnpkg/fslib';
import {Command, Usage, UsageError} from 'clipanion';
import {BaseCommand, WorkspaceRequiredError} from '@yarnpkg/cli';
import {Cache, Configuration, Project, structUtils, execUtils, miscUtils} from '@yarnpkg/core';
import {npath, xfs, Filename, ppath} from '@yarnpkg/fslib';
import {Command, Usage, UsageError} from 'clipanion';

import * as patchUtils from '../patchUtils';
import * as patchUtils from '../patchUtils';

// eslint-disable-next-line arca/no-default-export
export default class PatchCommitCommand extends BaseCommand {
Expand All @@ -27,10 +27,7 @@ export default class PatchCommitCommand extends BaseCommand {
if (!workspace)
throw new WorkspaceRequiredError(project.cwd, this.context.cwd);

await project.resolveEverything({
lockfileOnly: true,
report: new ThrowReport(),
});
await project.restoreInstallState();

const folderPath = ppath.resolve(this.context.cwd, npath.toPortablePath(this.patchFolder));
const metaPath = ppath.join(folderPath, `.yarn-patch.json` as Filename);
Expand Down
36 changes: 15 additions & 21 deletions packages/plugin-version/sources/commands/version/check.tsx
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
import {WorkspaceRequiredError} from '@yarnpkg/cli';
import {CommandContext, Configuration, MessageName, Project, StreamReport, Workspace, structUtils, ThrowReport} from '@yarnpkg/core';
import {ppath} from '@yarnpkg/fslib';
import {ScrollableItems} from '@yarnpkg/libui/sources/components/ScrollableItems';
import {FocusRequest} from '@yarnpkg/libui/sources/hooks/useFocusRequest';
import {useListInput} from '@yarnpkg/libui/sources/hooks/useListInput';
import {renderForm} from '@yarnpkg/libui/sources/misc/renderForm';
import {Command, Usage, UsageError} from 'clipanion';
import {Box, Color} from 'ink';
import React, {useCallback, useState} from 'react';
import semver from 'semver';

import * as versionUtils from '../../versionUtils';
import {WorkspaceRequiredError} from '@yarnpkg/cli';
import {CommandContext, Configuration, MessageName, Project, StreamReport, Workspace, structUtils} from '@yarnpkg/core';
import {ppath} from '@yarnpkg/fslib';
import {ScrollableItems} from '@yarnpkg/libui/sources/components/ScrollableItems';
import {FocusRequest} from '@yarnpkg/libui/sources/hooks/useFocusRequest';
import {useListInput} from '@yarnpkg/libui/sources/hooks/useListInput';
import {renderForm} from '@yarnpkg/libui/sources/misc/renderForm';
import {Command, Usage, UsageError} from 'clipanion';
import {Box, Color} from 'ink';
import React, {useCallback, useState} from 'react';
import semver from 'semver';

import * as versionUtils from '../../versionUtils';

type Releases = Map<Workspace, Exclude<versionUtils.Decision, versionUtils.Decision.UNDECIDED>>;

Expand Down Expand Up @@ -53,10 +53,7 @@ export default class VersionApplyCommand extends Command<CommandContext> {
if (!workspace)
throw new WorkspaceRequiredError(project.cwd, this.context.cwd);

await project.resolveEverything({
lockfileOnly: true,
report: new ThrowReport(),
});
await project.restoreInstallState();

const versionFile = await versionUtils.openVersionFile(project);
if (versionFile === null || versionFile.releaseRoots.size === 0)
Expand Down Expand Up @@ -273,10 +270,7 @@ export default class VersionApplyCommand extends Command<CommandContext> {
if (!workspace)
throw new WorkspaceRequiredError(project.cwd, this.context.cwd);

await project.resolveEverything({
lockfileOnly: true,
report: new ThrowReport(),
});
await project.restoreInstallState();

const report = await StreamReport.start({
configuration,
Expand Down
1 change: 1 addition & 0 deletions packages/yarnpkg-core/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@
"@types/got": "^8.3.4",
"@types/is-ci": "^2.0.0",
"@types/micromatch": "^3.1.0",
"@types/node": "^13.7.0",
"@types/semver": "^7.1.0",
"@types/tar": "^4.0.0",
"@types/tmp": "^0.0.33",
Expand Down
5 changes: 5 additions & 0 deletions packages/yarnpkg-core/sources/Configuration.ts
Original file line number Diff line number Diff line change
Expand Up @@ -195,6 +195,11 @@ export const coreDefinitions: {[coreSettingName: string]: SettingsDefinition} =
type: SettingsType.STRING,
default: DEFAULT_LOCK_FILENAME,
},
installStatePath: {
description: `Path of the file where the install state will be persisted`,
type: SettingsType.ABSOLUTE_PATH,
default: `./.yarn/install-state.gz`,
},
rcFilename: {
description: `Name of the files where the configuration can be found`,
type: SettingsType.STRING,
Expand Down
Loading