Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

yarn 3: invalid result when Yarn up modifies package.json files #20281

Closed
mtlewis opened this issue Feb 7, 2023 · 15 comments · Fixed by #21309
Closed

yarn 3: invalid result when Yarn up modifies package.json files #20281

mtlewis opened this issue Feb 7, 2023 · 15 comments · Fixed by #21309
Labels
manager:npm package.json files (npm/yarn/pnpm) status:requirements Full requirements are not yet known, so implementation should not be started type:bug Bug fix of existing functionality

Comments

@mtlewis
Copy link

mtlewis commented Feb 7, 2023

How are you running Renovate?

Mend Renovate hosted app on github.com

If you're self-hosting Renovate, tell us what version of Renovate you run.

No response

If you're self-hosting Renovate, select which platform you are using.

None

If you're self-hosting Renovate, tell us what version of the platform you run.

No response

Was this something which used to work for you, and then stopped?

I never saw this working

Describe the bug

It seems like Renovate is failing to correctly apply update-lockfile updates when a single package is depended on via different version ranges across packages in a monorepo.

Here's a minimal repro: https://github.com/mtlewis/renovate-yarn-repro

You'll notice that package-1 depends on @types/express@*, while package-2 depends on @types/express@^4.17.16. In main, the package version is locked to v4.17.16.

While renovate correctly opened mtlewis/renovate-yarn-repro#1 to update the package to v4.17.17, the yarn.lock file:

  • Incorrectly changes the version range for the package-2 dependency to on @types/express to *.
  • Does not update the locked version of @types/express at all - the diff does not reflect the PR description, in that even after re-running yarn locally, the version of @types/express in use in the repo stays at 4.17.16.

Relevant debug logs

Logs
DEBUG: No dangling containers to remove
INFO: Repository started
{
  "renovateVersion": "34.124.2"
}
DEBUG: Using localDir: /mnt/renovate/gh/mtlewis/renovate-yarn-repro
DEBUG: PackageFiles.clear() - Package files deleted
DEBUG: initRepo("mtlewis/renovate-yarn-repro")
DEBUG: Using queue: host=api.github.com, concurrency=10
DEBUG: mtlewis/renovate-yarn-repro default branch = main
DEBUG: Using app token for git init
DEBUG: Repository cache is restored from revision 13
DEBUG: Resetting npmrc
DEBUG: checkOnboarding()
DEBUG: isOnboarded()
DEBUG: Checking cached config file name
DEBUG: Existing config file confirmed
DEBUG: Repository config
{
  "fileName": ".github/renovate.json5",
  "config": {
    "$schema": "https://docs.renovatebot.com/renovate-schema.json",
    "extends": [
      "config:base"
    ],
    "rangeStrategy": "update-lockfile"
  }
}
DEBUG: Repo is onboarded
DEBUG: migrateAndValidate()
DEBUG: No config migration necessary
DEBUG: massaged config
{
  "config": {
    "$schema": "https://docs.renovatebot.com/renovate-schema.json",
    "extends": [
      "github>whitesource/merge-confidence:beta",
      "config:base"
    ],
    "rangeStrategy": "update-lockfile"
  }
}
DEBUG: migrated config
{
  "config": {
    "$schema": "https://docs.renovatebot.com/renovate-schema.json",
    "extends": [
      "github>whitesource/merge-confidence:beta",
      "config:base"
    ],
    "rangeStrategy": "update-lockfile"
  }
}
DEBUG: Setting hostRules from config
DEBUG: Found repo ignorePaths
{
  "ignorePaths": [
    "**/node_modules/**",
    "**/bower_components/**",
    "**/vendor/**",
    "**/examples/**",
    "**/__tests__/**",
    "**/test/**",
    "**/tests/**",
    "**/__fixtures__/**"
  ]
}
DEBUG: Using queue: host=api.github.com, concurrency=10
DEBUG: No vulnerability alerts found
DEBUG: No vulnerability alerts found
DEBUG: findIssue(Dependency Dashboard)
DEBUG: Retrieving issueList
DEBUG: Retrieved 1 issues
DEBUG: Found issue 2
DEBUG: No baseBranches
DEBUG: extract()
DEBUG: Cached extract result cannot be used due to base branch SHA change (old=8c4c29987b96954c3f83a280e0df9cb9b77fb82d, new=e5d0c02e1b06d86544c57c2759f802ae9be7c60c)
DEBUG: Setting current branch to main
DEBUG: Initializing git repository into /mnt/renovate/gh/mtlewis/renovate-yarn-repro
DEBUG: Performing blobless clone
DEBUG: git clone completed
{
  "durationMs": 1051
}
DEBUG: latest repository commit
{
  "latestCommit": {
    "hash": "e5d0c02e1b06d86544c57c2759f802ae9be7c60c",
    "date": "2023-02-07T14:54:02+00:00",
    "message": "restore range strategy",
    "refs": "HEAD -> main, origin/main, origin/HEAD",
    "body": "",
    "author_name": "MT Lewis",
    "author_email": "[email protected]"
  }
}
DEBUG: latest commit
{
  "branchName": "main",
  "latestCommitDate": "2023-02-07T14:54:02+00:00"
}
DEBUG: Using file match: (^|/)tasks/[^/]+\.ya?ml$ for manager ansible
DEBUG: Using file match: (^|/)requirements\.ya?ml$ for manager ansible-galaxy
DEBUG: Using file match: (^|/)galaxy\.ya?ml$ for manager ansible-galaxy
DEBUG: Using file match: (^|/)\.tool-versions$ for manager asdf
DEBUG: Using file match: azure.*pipelines?.*\.ya?ml$ for manager azure-pipelines
DEBUG: Using file match: (^|/)batect(-bundle)?\.yml$ for manager batect
DEBUG: Using file match: (^|/)batect$ for manager batect-wrapper
DEBUG: Using file match: (^|/)WORKSPACE(|\.bazel)$ for manager bazel
DEBUG: Using file match: \.bzl$ for manager bazel
DEBUG: Using file match: (^|\/)\.bazelversion$ for manager bazelisk
DEBUG: Using file match: (^|/)\.?bitbucket-pipelines\.ya?ml$ for manager bitbucket-pipelines
DEBUG: Using file match: buildkite\.ya?ml for manager buildkite
DEBUG: Using file match: \.buildkite/.+\.ya?ml$ for manager buildkite
DEBUG: Using file match: (^|/)Gemfile$ for manager bundler
DEBUG: Using file match: \.cake$ for manager cake
DEBUG: Using file match: (^|/)Cargo\.toml$ for manager cargo
DEBUG: Using file match: (^|/)\.circleci/config\.yml$ for manager circleci
DEBUG: Using file match: (^|/)cloudbuild\.ya?ml for manager cloudbuild
DEBUG: Using file match: (^|/)Podfile$ for manager cocoapods
DEBUG: Using file match: (^|/)([\w-]*)composer\.json$ for manager composer
DEBUG: Using file match: (^|/)conanfile\.(txt|py)$ for manager conan
DEBUG: Using file match: (^|/)(?:deps|bb)\.edn$ for manager deps-edn
DEBUG: Using file match: (^|/)(?:docker-)?compose[^/]*\.ya?ml$ for manager docker-compose
DEBUG: Using file match: (^|/|\.)Dockerfile$ for manager dockerfile
DEBUG: Using file match: (^|/)Dockerfile[^/]*$ for manager dockerfile
DEBUG: Using file match: (^|/)\.drone\.yml$ for manager droneci
DEBUG: Using file match: (^|/)fleet\.ya?ml for manager fleet
DEBUG: Using file match: (^|\/)flux-system\/(?:.+\/)?gotk-components\.yaml$ for manager flux
DEBUG: Using file match: (^|\/)\.fvm\/fvm_config\.json$ for manager fvm
DEBUG: Using file match: (^|/)\.gitmodules$ for manager git-submodules
DEBUG: Using file match: ^(workflow-templates|\.github\/workflows)\/[^/]+\.ya?ml$ for manager github-actions
DEBUG: Using file match: (^|\/)action\.ya?ml$ for manager github-actions
DEBUG: Using file match: \.gitlab-ci\.yml$ for manager gitlabci
DEBUG: Using file match: \.gitlab-ci\.yml$ for manager gitlabci-include
DEBUG: Using file match: (^|/)go\.mod$ for manager gomod
DEBUG: Using file match: \.gradle(\.kts)?$ for manager gradle
DEBUG: Using file match: (^|\/)gradle\.properties$ for manager gradle
DEBUG: Using file match: (^|\/)gradle\/.+\.toml$ for manager gradle
DEBUG: Using file match: \.versions\.toml$ for manager gradle
DEBUG: Using file match: (^|\/)versions.props$ for manager gradle
DEBUG: Using file match: (^|\/)versions.lock$ for manager gradle
DEBUG: Using file match: (^|/)gradle/wrapper/gradle-wrapper\.properties$ for manager gradle-wrapper
DEBUG: Using file match: (^|/)requirements\.yaml$ for manager helm-requirements
DEBUG: Using file match: (^|/)values\.yaml$ for manager helm-values
DEBUG: Using file match: (^|/)helmfile\.yaml$ for manager helmfile
DEBUG: Using file match: (^|/)Chart\.yaml$ for manager helmv3
DEBUG: Using file match: (^|/)bin/hermit$ for manager hermit
DEBUG: Using file match: ^Formula/[^/]+[.]rb$ for manager homebrew
DEBUG: Using file match: \.html?$ for manager html
DEBUG: Using file match: (^|/)plugins\.(txt|ya?ml)$ for manager jenkins
DEBUG: Using file match: (^|/)jsonnetfile\.json$ for manager jsonnet-bundler
DEBUG: Using file match: ^.+\.main\.kts$ for manager kotlin-script
DEBUG: Using file match: (^|/)kustomization\.ya?ml$ for manager kustomize
DEBUG: Using file match: (^|/)project\.clj$ for manager leiningen
DEBUG: Using file match: (^|/|\.)pom\.xml$ for manager maven
DEBUG: Using file match: ^(((\.mvn)|(\.m2))/)?settings\.xml$ for manager maven
DEBUG: Using file match: (^|\/).mvn/wrapper/maven-wrapper.properties$ for manager maven-wrapper
DEBUG: Using file match: (^|/)package\.js$ for manager meteor
DEBUG: Using file match: (^|\/)Mintfile$ for manager mint
DEBUG: Using file match: (^|/)mix\.exs$ for manager mix
DEBUG: Using file match: (^|\/)flake\.nix$ for manager nix
DEBUG: Using file match: (^|/)\.node-version$ for manager nodenv
DEBUG: Using file match: (^|/)package\.json$ for manager npm
DEBUG: Using file match: \.(?:cs|fs|vb)proj$ for manager nuget
DEBUG: Using file match: \.(?:props|targets)$ for manager nuget
DEBUG: Using file match: (^|\/)dotnet-tools\.json$ for manager nuget
DEBUG: Using file match: (^|\/)global\.json$ for manager nuget
DEBUG: Using file match: (^|/)\.nvmrc$ for manager nvm
DEBUG: Using file match: (^|/)src/main/features/.+\.json$ for manager osgi
DEBUG: Using file match: (^|/)([\w-]*)requirements\.(txt|pip)$ for manager pip_requirements
DEBUG: Using file match: (^|/)setup\.py$ for manager pip_setup
DEBUG: Using file match: (^|/)Pipfile$ for manager pipenv
DEBUG: Using file match: (^|/)pyproject\.toml$ for manager poetry
DEBUG: Using file match: (^|/)\.pre-commit-config\.yaml$ for manager pre-commit
DEBUG: Using file match: (^|/)pubspec\.ya?ml$ for manager pub
DEBUG: Using file match: (^|\/)Puppetfile$ for manager puppet
DEBUG: Using file match: (^|/)\.python-version$ for manager pyenv
DEBUG: Using file match: (^|/)\.ruby-version$ for manager ruby-version
DEBUG: Using file match: \.sbt$ for manager sbt
DEBUG: Using file match: project/[^/]*.scala$ for manager sbt
DEBUG: Using file match: (^|/)setup\.cfg$ for manager setup-cfg
DEBUG: Using file match: (^|/)Package\.swift for manager swift
DEBUG: Using file match: \.tf$ for manager terraform
DEBUG: Using file match: (^|/)\.terraform-version$ for manager terraform-version
DEBUG: Using file match: (^|/)terragrunt\.hcl$ for manager terragrunt
DEBUG: Using file match: (^|/)\.terragrunt-version$ for manager terragrunt-version
DEBUG: Using file match: \.tflint\.hcl$ for manager tflint-plugin
DEBUG: Using file match: ^\.travis\.yml$ for manager travis
DEBUG: Using file match: (^|/)\.vela\.ya?ml$ for manager velaci
DEBUG: Using file match: ^\.woodpecker(?:\/[^/]+)?\.ya?ml$ for manager woodpecker
DEBUG: Matched 4 file(s) for manager npm: package.json, packages/package-1/package.json, packages/package-2/package.json, packages/package-3/package.json
DEBUG: npm file package.json has name "renovate-yarn-repro"
DEBUG: npm file packages/package-1/package.json has name "@renovate-yarn-repro/package-1"
DEBUG: npm file packages/package-2/package.json has name "@renovate-yarn-repro/package-2"
DEBUG: npm file packages/package-3/package.json has name "@renovate-yarn-repro/package-3"
DEBUG: Detecting pnpm Workspaces
DEBUG: Detecting Lerna and Yarn Workspaces
DEBUG: Finding locked versions
DEBUG: Found npm package files
DEBUG: Found 4 package file(s)
INFO: Dependency extraction complete
{
  "baseBranch": "main",
  "stats": {
    "managers": {
      "npm": {
        "fileCount": 4,
        "depCount": 5
      }
    },
    "total": {
      "fileCount": 4,
      "depCount": 5
    }
  }
}
DEBUG: Using queue: host=registry.npmjs.org, concurrency=10
DEBUG: PackageFiles.add() - Package file saved for base branch
{
  "baseBranch": "main"
}
DEBUG: Package releases lookups complete
{
  "baseBranch": "main"
}
DEBUG: branchifyUpgrades
DEBUG: detectSemanticCommits()
DEBUG: getCommitMessages
DEBUG: semanticCommits: detected "unknown"
DEBUG: semanticCommits: disabled
DEBUG: 2 flattened updates found: @types/express, @types/express
DEBUG: Returning 1 branch(es)
DEBUG: config.repoIsOnboarded=true
DEBUG: packageFiles with updates
{
  "baseBranch": "main",
  "config": {
    "npm": [
      {
        "packageFile": "package.json",
        "deps": [
          {
            "depType": "packageManager",
            "depName": "yarn",
            "currentValue": "3.4.1",
            "datasource": "npm",
            "commitMessageTopic": "Yarn",
            "packageName": "@yarnpkg/cli",
            "prettyDepType": "packageManager",
            "depIndex": 0,
            "updates": [],
            "warnings": [],
            "versioning": "npm",
            "sourceUrl": "https://github.com/yarnpkg/berry",
            "registryUrl": "https://registry.npmjs.org",
            "sourceDirectory": "packages/yarnpkg-cli",
            "currentVersion": "3.4.1",
            "fixedVersion": "3.4.1"
          }
        ],
        "packageJsonName": "renovate-yarn-repro",
        "packageFileVersion": "0.0.0-development",
        "yarnLock": "yarn.lock",
        "managerData": {
          "yarnZeroInstall": false,
          "hasPackageManager": true
        },
        "skipInstalls": true,
        "yarnWorkspacesPackages": [
          "packages/*"
        ],
        "constraints": {
          "yarn": "3.4.1"
        },
        "lockFiles": [
          "yarn.lock"
        ]
      },
      {
        "packageFile": "packages/package-1/package.json",
        "deps": [
          {
            "depType": "devDependencies",
            "depName": "@types/express",
            "currentValue": "*",
            "datasource": "npm",
            "prettyDepType": "devDependency",
            "lockedVersion": "4.17.16",
            "depIndex": 0,
            "updates": [
              {
                "bucket": "non-major",
                "newVersion": "4.17.17",
                "newValue": "*",
                "releaseTimestamp": "2023-02-03T21:32:52.154Z",
                "newMajor": 4,
                "newMinor": 17,
                "updateType": "patch",
                "isRange": true,
                "isLockfileUpdate": true,
                "branchName": "renovate/express-4.x-lockfile"
              }
            ],
            "warnings": [],
            "versioning": "npm",
            "sourceUrl": "https://github.com/DefinitelyTyped/DefinitelyTyped",
            "registryUrl": "https://registry.npmjs.org",
            "sourceDirectory": "types/express",
            "homepage": "https://github.com/DefinitelyTyped/DefinitelyTyped/tree/master/types/express",
            "currentVersion": "4.17.16",
            "isSingleVersion": true,
            "fixedVersion": "4.17.16"
          }
        ],
        "packageJsonName": "@renovate-yarn-repro/package-1",
        "packageFileVersion": "0.0.0-development",
        "yarnLock": "yarn.lock",
        "managerData": {
          "yarnZeroInstall": false,
          "hasPackageManager": true
        },
        "skipInstalls": true,
        "constraints": {
          "yarn": "3.4.1"
        },
        "hasYarnWorkspaces": true,
        "lockFiles": [
          "yarn.lock"
        ]
      },
      {
        "packageFile": "packages/package-2/package.json",
        "deps": [
          {
            "depType": "devDependencies",
            "depName": "@types/express",
            "currentValue": "^4.17.16",
            "datasource": "npm",
            "prettyDepType": "devDependency",
            "lockedVersion": "4.17.16",
            "depIndex": 0,
            "updates": [
              {
                "bucket": "non-major",
                "newVersion": "4.17.17",
                "newValue": "^4.17.16",
                "releaseTimestamp": "2023-02-03T21:32:52.154Z",
                "newMajor": 4,
                "newMinor": 17,
                "updateType": "patch",
                "isRange": true,
                "isLockfileUpdate": true,
                "branchName": "renovate/express-4.x-lockfile"
              }
            ],
            "warnings": [],
            "versioning": "npm",
            "sourceUrl": "https://github.com/DefinitelyTyped/DefinitelyTyped",
            "registryUrl": "https://registry.npmjs.org",
            "sourceDirectory": "types/express",
            "homepage": "https://github.com/DefinitelyTyped/DefinitelyTyped/tree/master/types/express",
            "currentVersion": "4.17.16",
            "isSingleVersion": true,
            "fixedVersion": "4.17.16"
          }
        ],
        "packageJsonName": "@renovate-yarn-repro/package-2",
        "packageFileVersion": "0.0.0-development",
        "yarnLock": "yarn.lock",
        "managerData": {
          "yarnZeroInstall": false,
          "hasPackageManager": true
        },
        "skipInstalls": true,
        "constraints": {
          "yarn": "3.4.1"
        },
        "hasYarnWorkspaces": true,
        "lockFiles": [
          "yarn.lock"
        ]
      },
      {
        "packageFile": "packages/package-3/package.json",
        "deps": [
          {
            "depType": "dependencies",
            "depName": "@renovate-yarn-repro/package-1",
            "currentValue": "workspace:^",
            "skipReason": "internal-package",
            "prettyDepType": "dependency",
            "isInternal": true,
            "depIndex": 0,
            "updates": []
          },
          {
            "depType": "dependencies",
            "depName": "@renovate-yarn-repro/package-2",
            "currentValue": "workspace:^",
            "skipReason": "internal-package",
            "prettyDepType": "dependency",
            "isInternal": true,
            "depIndex": 1,
            "updates": []
          }
        ],
        "packageJsonName": "@renovate-yarn-repro/package-3",
        "packageFileVersion": "0.0.0-development",
        "yarnLock": "yarn.lock",
        "managerData": {
          "yarnZeroInstall": false,
          "hasPackageManager": true
        },
        "skipInstalls": true,
        "constraints": {
          "yarn": "3.4.1"
        },
        "hasYarnWorkspaces": true,
        "lockFiles": [
          "yarn.lock"
        ]
      }
    ]
  }
}
DEBUG: detectSemanticCommits()
DEBUG: semanticCommits: returning "disabled" from cache
DEBUG: processRepo()
DEBUG: Processing 1 branch: renovate/express-4.x-lockfile
DEBUG: Calculating hourly PRs remaining
DEBUG: getPrList success
{
  "pullsTotal": 1,
  "requestsTotal": 1,
  "apiQuotaAffected": true
}
DEBUG: currentHourStart=2023-02-07T14:00:00.000+00:00
DEBUG: PR hourly limit remaining: 1
DEBUG: Calculating prConcurrentLimit (10)
DEBUG: getBranchPr(renovate/express-4.x-lockfile)
DEBUG: findPr(renovate/express-4.x-lockfile, undefined, open)
DEBUG: findPr(renovate/express-4.x-lockfile, undefined, closed)
DEBUG: Found PR #1
DEBUG: Found autoclosed PR for branch
{
  "autoclosedPr": {
    "bodyStruct": {
      "debugData": {
        "createdInVer": "34.124.2",
        "updatedInVer": "34.124.2"
      },
      "hash": "689c29a2ed63cad07e5f99aefbd2fcc315f7f5764cf6cfb79db2aef548262073",
      "rebaseRequested": true
    },
    "closedAt": "2023-02-07T14:51:23Z",
    "createdAt": "2023-02-07T14:23:57Z",
    "displayNumber": "Pull Request #1",
    "hasReviewers": true,
    "labels": [],
    "node_id": "PR_kwDOI65vr85Jb--d",
    "number": 1,
    "sha": "633623ed5a1331d8ce6563cf3aa474b221d864f6",
    "sourceBranch": "renovate/express-4.x-lockfile",
    "sourceRepo": "mtlewis/renovate-yarn-repro",
    "state": "closed",
    "title": "Update dependency @types/express to v4.17.17 - autoclosed",
    "updated_at": "2023-02-07T14:51:25Z"
  }
}
DEBUG: Recreated autoclosed branch renovate/express-4.x-lockfile with sha 633623ed5a1331d8ce6563cf3aa474b221d864f6
INFO: Successfully reopened autoclosed PR
{
  "branchName": "renovate/express-4.x-lockfile",
  "title": "Update dependency @types/express to v4.17.17",
  "number": 1
}
DEBUG: 1 PRs are currently open
DEBUG: PR concurrent limit remaining: 9
DEBUG: Calculated maximum PRs remaining this run: 1
DEBUG: PullRequests limit = 1
DEBUG: Calculating hourly PRs remaining
DEBUG: currentHourStart=2023-02-07T14:00:00.000+00:00
DEBUG: PR hourly limit remaining: 1
DEBUG: Calculating branchConcurrentLimit (10)
DEBUG: 0 already existing branches found:
DEBUG: Branch concurrent limit remaining: 10
DEBUG: Calculated maximum branches remaining this run: 1
DEBUG: Branches limit = 1
DEBUG: syncBranchState()(branch="renovate/express-4.x-lockfile")
DEBUG: syncBranchState(): Branch cache not found, creating minimal branchState(branch="renovate/express-4.x-lockfile")
DEBUG: getBranchPr(renovate/express-4.x-lockfile)(branch="renovate/express-4.x-lockfile")
DEBUG: findPr(renovate/express-4.x-lockfile, undefined, open)(branch="renovate/express-4.x-lockfile")
DEBUG: Found PR #1(branch="renovate/express-4.x-lockfile")
DEBUG: branchExists=false(branch="renovate/express-4.x-lockfile")
DEBUG: dependencyDashboardCheck=undefined(branch="renovate/express-4.x-lockfile")
DEBUG: PR rebase requested=true(branch="renovate/express-4.x-lockfile")
DEBUG: Checking schedule(at any time, null)(branch="renovate/express-4.x-lockfile")
DEBUG: No schedule defined(branch="renovate/express-4.x-lockfile")
DEBUG: Manual rebase requested via Dependency Dashboard(branch="renovate/express-4.x-lockfile")
DEBUG: Using reuseExistingBranch: false(branch="renovate/express-4.x-lockfile")
DEBUG: Setting current branch to main(branch="renovate/express-4.x-lockfile")
DEBUG: latest commit(branch="renovate/express-4.x-lockfile")
{
  "branchName": "main",
  "latestCommitDate": "2023-02-07T14:54:02+00:00"
}
DEBUG: manager.getUpdatedPackageFiles() reuseExistingBranch=false(branch="renovate/express-4.x-lockfile")
DEBUG: npm.updateLockedDependency: @types/[email protected] -> 4.17.17 [yarn.lock](branch="renovate/express-4.x-lockfile")
DEBUG: Yarn 2+ unsupported(branch="renovate/express-4.x-lockfile")
DEBUG: npm.updateLockedDependency: @types/[email protected] -> 4.17.17 [yarn.lock](branch="renovate/express-4.x-lockfile")
DEBUG: Yarn 2+ unsupported(branch="renovate/express-4.x-lockfile")
DEBUG: No package files need updating(branch="renovate/express-4.x-lockfile")
DEBUG: Getting updated lock files(branch="renovate/express-4.x-lockfile")
DEBUG: Writing package.json files(branch="renovate/express-4.x-lockfile")
{
  "packageFiles": [
    "package.json",
    "packages/package-1/package.json",
    "packages/package-2/package.json",
    "packages/package-3/package.json"
  ]
}
DEBUG: Writing any updated package files(branch="renovate/express-4.x-lockfile")
DEBUG: npmrc file found in repository(branch="renovate/express-4.x-lockfile")
DEBUG: Writing updated .npmrc file to .npmrc(branch="renovate/express-4.x-lockfile")
DEBUG: Generating yarn.lock for .(branch="renovate/express-4.x-lockfile")
DEBUG: Spawning yarn install to create yarn.lock(branch="renovate/express-4.x-lockfile")
DEBUG: No node constraint found - using latest(branch="renovate/express-4.x-lockfile")
DEBUG: Enabling global cache as zero-install is not detected(branch="renovate/express-4.x-lockfile")
DEBUG: Performing lockfileUpdate (yarn)(branch="renovate/express-4.x-lockfile")
DEBUG: Setting CONTAINERBASE_CACHE_DIR to /tmp/containerbase(branch="renovate/express-4.x-lockfile")
DEBUG: Using docker to execute(branch="renovate/express-4.x-lockfile")
{
  "image": "sidecar"
}
DEBUG: Resolved stable matching version(branch="renovate/express-4.x-lockfile")
{
  "toolName": "node",
  "constraint": null,
  "resolvedVersion": "v18.14.0"
}
DEBUG: Resolved stable matching version(branch="renovate/express-4.x-lockfile")
{
  "toolName": "corepack",
  "resolvedVersion": "0.15.3"
}
DEBUG: containerbaseDir is separate from cacheDir(branch="renovate/express-4.x-lockfile")
DEBUG: Resolved tag constraint(branch="renovate/express-4.x-lockfile")
{
  "image": "docker.io/renovate/sidecar"
}
DEBUG: Fetching Docker image: docker.io/renovate/sidecar(branch="renovate/express-4.x-lockfile")
DEBUG: Finished fetching Docker image docker.io/renovate/sidecar@sha256:38451e75e26e586419661c2c780e6a6edf26ad51edada285d82e892183e18436(branch="renovate/express-4.x-lockfile")
DEBUG: Executing command(branch="renovate/express-4.x-lockfile")
{
  "command": "docker run --rm --name=renovate_sidecar --label=renovate_child -v \"/mnt/renovate/gh/mtlewis/renovate-yarn-repro\":\"/mnt/renovate/gh/mtlewis/renovate-yarn-repro\" -v \"/tmp/renovate-cache\":\"/tmp/renovate-cache\" -v \"/tmp/containerbase\":\"/tmp/containerbase\" -e NPM_CONFIG_CACHE -e npm_config_store -e CI -e YARN_ENABLE_IMMUTABLE_INSTALLS -e YARN_HTTP_TIMEOUT -e YARN_GLOBAL_FOLDER -e YARN_ENABLE_GLOBAL_CACHE -e BUILDPACK_CACHE_DIR -e CONTAINERBASE_CACHE_DIR -w \"/mnt/renovate/gh/mtlewis/renovate-yarn-repro\" docker.io/renovate/sidecar bash -l -c \"install-tool node v18.14.0 && install-tool corepack 0.15.3 && yarn install --mode=update-lockfile && yarn up @types/express@^4.17.16 @types/express@* --mode=update-lockfile\""
}
DEBUG: exec completed(branch="renovate/express-4.x-lockfile")
{
  "durationMs": 17713,
  "stdout": "installing v2 tool node v18.14.0\nlinking tool node v18.14.0\nnode: v18.14.0 /usr/local/bin/node\nnpm: 9.3.1  /usr/local/bin/npm\nInstalled v2 /usr/local/buildpack/tools/v2/node.sh in 7 seconds\nskip cleanup, not a docker build: 456f23f92bd7\ninstalling v2 tool corepack v0.15.3\nlinking tool corepack v0.15.3\n0.15.3\nInstalled v2 /usr/local/buildpack/tools/v2/corepack.sh in 3 seconds\nskip cleanup, not a docker build: 456f23f92bd7\n➤ YN0000: ┌ Resolution step\n➤ YN0000: └ Completed\n➤ YN0000: ┌ Fetch step\n➤ YN0000: └ Completed\n➤ YN0000: ┌ Link step\n➤ YN0073: │ Skipped due to mode=update-lockfile\n➤ YN0000: └ Completed\n➤ YN0000: Done with warnings in 0s 137ms\n➤ YN0000: ┌ Resolution step\n➤ YN0000: └ Completed\n➤ YN0000: ┌ Fetch step\n➤ YN0000: └ Completed\n➤ YN0000: ┌ Link step\n➤ YN0073: │ Skipped due to mode=update-lockfile\n➤ YN0000: └ Completed\n➤ YN0000: Done with warnings in 0s 119ms\n",
  "stderr": ""
}
DEBUG: yarn.lock needs updating(branch="renovate/express-4.x-lockfile")
DEBUG: updateYarnOffline resolvedPaths(branch="renovate/express-4.x-lockfile")
{
  "resolvedPaths": [
    ".yarn/cache",
    ".pnp.cjs",
    ".pnp.js",
    ".pnp.loader.mjs"
  ]
}
DEBUG: Updated 1 lock files(branch="renovate/express-4.x-lockfile")
{
  "updatedArtifacts": [
    "yarn.lock"
  ]
}
DEBUG: Getting comments for #1(branch="renovate/express-4.x-lockfile")
DEBUG: Found 0 comments(branch="renovate/express-4.x-lockfile")
DEBUG: 1 file(s) to commit(branch="renovate/express-4.x-lockfile")
DEBUG: Preparing files for committing to branch renovate/express-4.x-lockfile(branch="renovate/express-4.x-lockfile")
DEBUG: Setting git author name: Renovate Bot(branch="renovate/express-4.x-lockfile")
DEBUG: Setting git author email: [email protected](branch="renovate/express-4.x-lockfile")
DEBUG: git commit(branch="renovate/express-4.x-lockfile")
{
  "deletedFiles": [],
  "ignoredFiles": [],
  "result": {
    "author": null,
    "branch": "renovate/express-4.x-lockfile",
    "commit": "8ecf87e4ef536e8809f8ba2f4d4043ad5cb2ca73",
    "root": false,
    "summary": {
      "changes": 1,
      "insertions": 2,
      "deletions": 2
    }
  }
}
DEBUG: POST https://api.github.com/repos/mtlewis/renovate-yarn-repro/git/refs = (code=ERR_NON_2XX_3XX_RESPONSE, statusCode=422 retryCount=0, duration=336)(branch="renovate/express-4.x-lockfile")
DEBUG: 422 Error thrown from GitHub(branch="renovate/express-4.x-lockfile")
{
  "err": {
    "name": "HTTPError",
    "code": "ERR_NON_2XX_3XX_RESPONSE",
    "timings": {
      "start": 1675781679712,
      "socket": 1675781679712,
      "lookup": 1675781679725,
      "connect": 1675781679725,
      "secureConnect": 1675781679736,
      "upload": 1675781679737,
      "response": 1675781680048,
      "end": 1675781680048,
      "phases": {
        "wait": 0,
        "dns": 13,
        "tcp": 0,
        "tls": 11,
        "request": 1,
        "firstByte": 311,
        "download": 0,
        "total": 336
      }
    },
    "message": "Response code 422 (Unprocessable Entity)",
    "stack": "HTTPError: Response code 422 (Unprocessable Entity)\n    at Request.<anonymous> (/home/ubuntu/renovateapp/node_modules/got/dist/source/as-promise/index.js:118:42)\n    at runMicrotasks (<anonymous>)\n    at processTicksAndRejections (node:internal/process/task_queues:96:5)",
    "options": {
      "headers": {
        "user-agent": "Renovate Bot (GitHub App 2740)",
        "accept": "application/json, application/vnd.github.machine-man-preview+json",
        "authorization": "***********",
        "content-type": "application/json",
        "content-length": "99",
        "accept-encoding": "gzip, deflate, br"
      },
      "url": "https://api.github.com/repos/mtlewis/renovate-yarn-repro/git/refs",
      "hostType": "github",
      "username": "",
      "password": "",
      "method": "POST",
      "http2": false
    },
    "response": {
      "statusCode": 422,
      "statusMessage": "Unprocessable Entity",
      "body": {
        "message": "Reference already exists",
        "documentation_url": "https://docs.github.com/rest/reference/git#create-a-reference"
      },
      "headers": {
        "server": "GitHub.com",
        "date": "Tue, 07 Feb 2023 14:54:39 GMT",
        "content-type": "application/json; charset=utf-8",
        "content-length": "122",
        "x-github-media-type": "github.v3; param=machine-man-preview",
        "x-github-api-version-selected": "2022-11-28",
        "x-ratelimit-limit": "5000",
        "x-ratelimit-remaining": "4961",
        "x-ratelimit-reset": "1675783409",
        "x-ratelimit-used": "39",
        "x-ratelimit-resource": "core",
        "access-control-expose-headers": "ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset",
        "access-control-allow-origin": "*",
        "strict-transport-security": "max-age=31536000; includeSubdomains; preload",
        "x-frame-options": "deny",
        "x-content-type-options": "nosniff",
        "x-xss-protection": "0",
        "referrer-policy": "origin-when-cross-origin, strict-origin-when-cross-origin",
        "content-security-policy": "default-src 'none'",
        "vary": "Accept-Encoding, Accept, X-Requested-With",
        "x-github-request-id": "C83C:28CD:1468467:152A9E0:63E2662F",
        "connection": "close"
      },
      "httpVersion": "1.1",
      "retryCount": 0
    }
  }
}
DEBUG: Platform-native commit: unknown error(branch="renovate/express-4.x-lockfile")
{
  "branchName": "renovate/express-4.x-lockfile",
  "err": {
    "hostType": "github",
    "err": {
      "name": "HTTPError",
      "code": "ERR_NON_2XX_3XX_RESPONSE",
      "timings": {
        "start": 1675781679712,
        "socket": 1675781679712,
        "lookup": 1675781679725,
        "connect": 1675781679725,
        "secureConnect": 1675781679736,
        "upload": 1675781679737,
        "response": 1675781680048,
        "end": 1675781680048,
        "phases": {
          "wait": 0,
          "dns": 13,
          "tcp": 0,
          "tls": 11,
          "request": 1,
          "firstByte": 311,
          "download": 0,
          "total": 336
        }
      },
      "message": "Response code 422 (Unprocessable Entity)",
      "stack": "HTTPError: Response code 422 (Unprocessable Entity)\n    at Request.<anonymous> (/home/ubuntu/renovateapp/node_modules/got/dist/source/as-promise/index.js:118:42)\n    at runMicrotasks (<anonymous>)\n    at processTicksAndRejections (node:internal/process/task_queues:96:5)",
      "options": {
        "headers": {
          "user-agent": "Renovate Bot (GitHub App 2740)",
          "accept": "application/json, application/vnd.github.machine-man-preview+json",
          "authorization": "***********",
          "content-type": "application/json",
          "content-length": "99",
          "accept-encoding": "gzip, deflate, br"
        },
        "url": "https://api.github.com/repos/mtlewis/renovate-yarn-repro/git/refs",
        "hostType": "github",
        "username": "",
        "password": "",
        "method": "POST",
        "http2": false
      },
      "response": {
        "statusCode": 422,
        "statusMessage": "Unprocessable Entity",
        "body": {
          "message": "Reference already exists",
          "documentation_url": "https://docs.github.com/rest/reference/git#create-a-reference"
        },
        "headers": {
          "server": "GitHub.com",
          "date": "Tue, 07 Feb 2023 14:54:39 GMT",
          "content-type": "application/json; charset=utf-8",
          "content-length": "122",
          "x-github-media-type": "github.v3; param=machine-man-preview",
          "x-github-api-version-selected": "2022-11-28",
          "x-ratelimit-limit": "5000",
          "x-ratelimit-remaining": "4961",
          "x-ratelimit-reset": "1675783409",
          "x-ratelimit-used": "39",
          "x-ratelimit-resource": "core",
          "access-control-expose-headers": "ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset",
          "access-control-allow-origin": "*",
          "strict-transport-security": "max-age=31536000; includeSubdomains; preload",
          "x-frame-options": "deny",
          "x-content-type-options": "nosniff",
          "x-xss-protection": "0",
          "referrer-policy": "origin-when-cross-origin, strict-origin-when-cross-origin",
          "content-security-policy": "default-src 'none'",
          "vary": "Accept-Encoding, Accept, X-Requested-With",
          "x-github-request-id": "C83C:28CD:1468467:152A9E0:63E2662F",
          "connection": "close"
        },
        "httpVersion": "1.1",
        "retryCount": 0
      }
    },
    "message": "external-host-error",
    "stack": "Error: external-host-error\n    at handleGotError (/home/ubuntu/renovateapp/node_modules/renovate/dist/util/http/github.js:93:16)\n    at GithubHttp.request (/home/ubuntu/renovateapp/node_modules/renovate/dist/util/http/github.js:230:19)\n    at runMicrotasks (<anonymous>)\n    at processTicksAndRejections (node:internal/process/task_queues:96:5)\n    at async GithubHttp.requestJson (/home/ubuntu/renovateapp/node_modules/renovate/dist/util/http/index.js:194:21)\n    at async pushFiles (/home/ubuntu/renovateapp/node_modules/renovate/dist/modules/platform/github/index.js:1374:13)\n    at async Proxy.commitFiles (/home/ubuntu/renovateapp/node_modules/renovate/dist/modules/platform/github/index.js:1393:24)\n    at async processBranch (/home/ubuntu/renovateapp/node_modules/renovate/dist/workers/repository/update/branch/index.js:384:25)\n    at async writeUpdates (/home/ubuntu/renovateapp/node_modules/renovate/dist/workers/repository/process/write.js:120:21)\n    at async update (/home/ubuntu/renovateapp/node_modules/renovate/dist/workers/repository/process/extract-update.js:144:15)\n    at async Object.renovateRepository (/home/ubuntu/renovateapp/node_modules/renovate/dist/workers/repository/index.js:60:25)\n    at async renovateRepository (/home/ubuntu/renovateapp/app/worker/index.js:260:26)\n    at async /home/ubuntu/renovateapp/app/worker/index.js:454:5"
  }
}
DEBUG: Ensuring Dependency Dashboard
DEBUG: ensureIssue(Dependency Dashboard)
DEBUG: Patching issue
DEBUG: Issue updated
DEBUG: Removing any stale branches
DEBUG: config.repoIsOnboarded=true
DEBUG: No renovate branches found
DEBUG: Cleaning up Renovate refs: refs/renovate/*
DEBUG: PackageFiles.clear() - Package files deleted
DEBUG: Branch summary
{
  "cacheModified": true,
  "baseBranches": [
    {
      "branchName": "main",
      "sha": "e5d0c02e1b06d86544c57c2759f802ae9be7c60c"
    }
  ],
  "branches": [],
  "inactiveBranches": [
    "renovate/express-4.x-lockfile"
  ]
}
DEBUG: Renovate repository PR statistics
{
  "stats": {
    "total": 1,
    "open": 1,
    "closed": 0,
    "merged": 0
  }
}
DEBUG: Repository result: done, status: onboarded, enabled: true, onboarded: true
DEBUG: Repository timing splits (milliseconds)
{
  "splits": {
    "init": 3039,
    "extract": 3608,
    "lookup": 789,
    "onboarding": 0,
    "update": 24740
  },
  "total": 34568
}
DEBUG: Package cache statistics
{
  "get": {
    "count": 4,
    "avgMs": 20,
    "medianMs": 19,
    "maxMs": 29
  },
  "set": {
    "count": 0
  }
}
DEBUG: http statistics
{
  "urls": {
    "https://api.github.com/graphql (POST,200)": 2,
    "https://api.github.com/repos/mtlewis/renovate-yarn-repro/contents/.github/renovate.json5 (GET,200)": 1,
    "https://api.github.com/repos/mtlewis/renovate-yarn-repro/git/commits (POST,201)": 1,
    "https://api.github.com/repos/mtlewis/renovate-yarn-repro/git/refs (POST,201)": 1,
    "https://api.github.com/repos/mtlewis/renovate-yarn-repro/git/refs (POST,422)": 1,
    "https://api.github.com/repos/mtlewis/renovate-yarn-repro/git/trees (POST,201)": 1,
    "https://api.github.com/repos/mtlewis/renovate-yarn-repro/issues/1/comments (GET,200)": 1,
    "https://api.github.com/repos/mtlewis/renovate-yarn-repro/issues/2 (GET,200)": 2,
    "https://api.github.com/repos/mtlewis/renovate-yarn-repro/issues/2 (PATCH,200)": 1,
    "https://api.github.com/repos/mtlewis/renovate-yarn-repro/pulls (GET,200)": 1,
    "https://api.github.com/repos/mtlewis/renovate-yarn-repro/pulls/1 (PATCH,200)": 1,
    "https://api.github.com/repos/whitesource/merge-confidence/contents/beta.json (GET,200)": 1,
    "https://registry.npmjs.org/@yarnpkg%2Fcli (GET,200)": 1
  },
  "hostStats": {
    "api.github.com": {
      "requestCount": 14,
      "requestAvgMs": 359,
      "queueAvgMs": 1
    },
    "registry.npmjs.org": {
      "requestCount": 1,
      "requestAvgMs": 167,
      "queueAvgMs": 0
    }
  },
  "totalRequests": 15
}
DEBUG: dns cache
{
  "hosts": [
    "api.github.com",
    "registry.npmjs.org"
  ]
}
INFO: Repository finished
{
  "cloned": true,
  "durationMs": 34568
}

Have you created a minimal reproduction repository?

I have linked to a minimal reproduction repository in the bug description

@mtlewis mtlewis added priority-5-triage status:requirements Full requirements are not yet known, so implementation should not be started type:bug Bug fix of existing functionality labels Feb 7, 2023
@rarkins
Copy link
Collaborator

rarkins commented Feb 7, 2023

This is the yarn command run: yarn up @types/express@^4.17.16 @types/express@* --mode=update-lockfile

@viceice
Copy link
Member

viceice commented Feb 7, 2023

this is an edge case. the yarn up command updates all references in all workspace projects to the same passed version of a package. so we can't support update-lockfile range strategy when multiple different versions of a package are used. this seems to be a yarn limitation.

@viceice viceice added the manager:npm package.json files (npm/yarn/pnpm) label Feb 7, 2023
@mtlewis
Copy link
Author

mtlewis commented Feb 7, 2023

Perhaps it makes sense to open a yarn issue to cover this? Based on my (admittedly naive!) interpretation of the situation, it seems like there's a bug (rather than a limitation) with the way yarn handles yarn up @types/express@^4.17.16 @types/express@* --mode=update-lockfile - it doesn't upgrade the version of @types/express in yarn.lock at all, it makes changes in package.json, and it leaves yarn.lock in a state where running yarn produces further changes.

@viceice
Copy link
Member

viceice commented Feb 8, 2023

no, it also changes the lockfile, but renovate isn't expecting a package.json change, so that file is omitted from git commit. that's why you see them missing on PR.

also yarn seems to use the last dep from cli args if the same package is passed multiple times.

@rarkins maybe we need to check for changed package.json files and commit them in case yarn changed them?

@mtlewis
Copy link
Author

mtlewis commented Feb 8, 2023

no, it also changes the lockfile, but renovate isn't expecting a package.json change, so that file is omitted from git commit. that's why you see them missing on PR.

Surely if mode is update-lockfile, it shouldn't change package.json files.

also yarn seems to use the last dep from cli args if the same package is passed multiple times.

Want to highlight that even if changes to package.json were included, mtlewis/renovate-yarn-repro#1 would still be incorrect. The version of @types/express in the lockfile in the PR is unchanged.

@rarkins
Copy link
Collaborator

rarkins commented Apr 1, 2023

Reproduction forked to https://github.com/renovate-reproductions/20281

@rarkins
Copy link
Collaborator

rarkins commented Apr 1, 2023

I updated the commands we run to target workspaces, e.g. yarn workspace @renovate-yarn-repro/package-2 up '@types/express@^4.17.16'.

yarn install --mode=update-lockfile predictably changes nothing in this case.

yarn workspace @renovate-yarn-repro/package-2 up '@types/express@^4.17.16' results in this diff:

diff --git a/packages/package-1/package.json b/packages/package-1/package.json
index 9057fe3..9759a33 100644
--- a/packages/package-1/package.json
+++ b/packages/package-1/package.json
@@ -4,6 +4,6 @@
   "license": "UNLICENSED",
   "private": true,
   "devDependencies": {
-    "@types/express": "*"
+    "@types/express": "^4.17.16"
   }
 }
diff --git a/yarn.lock b/yarn.lock
index 73e629a..19272df 100644
--- a/yarn.lock
+++ b/yarn.lock
@@ -9,7 +9,7 @@ __metadata:
   version: 0.0.0-use.local
   resolution: "@renovate-yarn-repro/package-1@workspace:packages/package-1"
   dependencies:
-    "@types/express": "*"
+    "@types/express": ^4.17.16
   languageName: unknown
   linkType: soft
 
@@ -49,26 +49,26 @@ __metadata:
   languageName: node
   linkType: hard
 
-"@types/express-serve-static-core@npm:^4.17.31":
-  version: 4.17.31
-  resolution: "@types/express-serve-static-core@npm:4.17.31"
+"@types/express-serve-static-core@npm:^4.17.33":
+  version: 4.17.33
+  resolution: "@types/express-serve-static-core@npm:4.17.33"
   dependencies:
     "@types/node": "*"
     "@types/qs": "*"
     "@types/range-parser": "*"
-  checksum: 009bfbe1070837454a1056aa710d0390ee5fb8c05dfe5a1691cc3e2ca88dc256f80e1ca27cb51a978681631d2f6431bfc9ec352ea46dd0c6eb183d0170bde5df
+  checksum: dce580d16b85f207445af9d4053d66942b27d0c72e86153089fa00feee3e96ae336b7bedb31ed4eea9e553c99d6dd356ed6e0928f135375d9f862a1a8015adf2
   languageName: node
   linkType: hard
 
-"@types/express@npm:*, @types/express@npm:^4.17.16":
-  version: 4.17.16
-  resolution: "@types/express@npm:4.17.16"
+"@types/express@npm:^4.17.16":
+  version: 4.17.17
+  resolution: "@types/express@npm:4.17.17"
   dependencies:
     "@types/body-parser": "*"
-    "@types/express-serve-static-core": ^4.17.31
+    "@types/express-serve-static-core": ^4.17.33
     "@types/qs": "*"
     "@types/serve-static": "*"
-  checksum: 43f3ed2cea6e5e83c7c1098c5152f644e975fd764443717ff9c812a1518416a9e7e9f824ffe852c118888cbfb994ed023cad08331f49b19ced469bb185cdd5cd
+  checksum: 0196dacc275ac3ce89d7364885cb08e7fb61f53ca101f65886dbf1daf9b7eb05c0943e2e4bbd01b0cc5e50f37e0eea7e4cbe97d0304094411ac73e1b7998f4da
   languageName: node
   linkType: hard

i.e. it's change the package.json in package-1.

So assuming you still think that's wrong, unfortunately it's yarn doing it. Unless/until we have some yarn commands which "do the right thing" for this case, we can't change anything on the Renovate end.

@rarkins
Copy link
Collaborator

rarkins commented Apr 1, 2023

Created an issue in the Yarn repo: yarnpkg/berry#5365

@rarkins rarkins changed the title yarn 3: multiple dependencies on different versions of the same package in workspaces aren't handled correctly yarn 3: invalid result when Yarn up modifies package.json files Apr 2, 2023
@rarkins
Copy link
Collaborator

rarkins commented Apr 2, 2023

I've modified the title of this to try to capture the broader problem here. It's like this:

  1. yarn up and yarn workspace x up can both modify an unrelated workspace's package.json. Maybe this is a bug of Yarn and once it's fixed then we don't even have any work to do in Renovate
  2. Renovate today is not even looking for changes to package.json after yarn up is run, so we commit only the lock file changes, which means the PR fails yarn install --immutable

If we get an answer back from the Yarn project soon (with much appreciation to @arcanis if possible) that it's a bug which will be soon addressed, then we'll wait for that and probably close this issue once the updated Yarn is available.

If it's not a bug, or it isn't likely to be fixed soon, we'll at least need to adjust Renovate's behavior to look for package.json changes and commit them too.

@arcanis
Copy link

arcanis commented Apr 2, 2023

yarn up and yarn workspace x up can both modify an unrelated workspace's package.json. Maybe this is a bug of Yarn and once it's fixed then we don't even have any work to do in Renovate

That's expected. The purpose of the yarn up command is update the dependency across the entire project at once (unlike yarn add, which only update it in the active workspace).

@rarkins
Copy link
Collaborator

rarkins commented Apr 2, 2023

@arcanis thanks. So if you run yarn workspace A up X then is the workspace A part redundant/not used? It looks like it updates X across all workspaces, like what you state above. Would "workspace-constrained yarn up commands" be a valid feature request for Yarn, or is there a reason not to?

In the reproduction scenario:

  • workspace a relies on "@types/express": "*"
  • workspace b relies on "@types/express": "^4.17.16"

Locked version (common) for them is 4.17.16 currently.

We can't figure out a way to upgrade the locked version to 4.17.7 without one of the workspaces having their package.json constraint changed.

@arcanis
Copy link

arcanis commented Apr 2, 2023

So if you run yarn workspace A up X then is the workspace A part redundant/not used?

Yep - yarn workspace WORKSPACE CMD just runs CMD from the cwd of the specified workspace directory, and in the default case of up it doesn't have any special effect as far as I can remember.

That said, what you describe (upgrading without bumping the version in the package.json files) can be achieved by adding the -R flag to yarn up. Quoting the documentation:

If -R,--recursive is set the command will change behavior and no other switch will be allowed. When operating under this mode yarn up will force all ranges matching the selected packages to be resolved again (often to the highest available versions) before being stored in the lockfile. It however won't touch your manifests anymore, so depending on your needs you might want to run both yarn up and yarn up -R to cover all bases.

Keep in mind that:

  • As the description says, it won't touch the package.json file, so the resolution won't go past what the initial range would allow (if you have a ~1.0.0 range, it won't be re-resolved to 1.1.0).

  • In Yarn, a single range is always resolved to the exact same version across the project, so two workspaces that list the exact same "lodash": "^1.0.0" dependency cannot depend on different versions (in the case you described it's not the case since you have two different ranges, so I only mention it for completeness).

@rarkins
Copy link
Collaborator

rarkins commented Apr 3, 2023

Thanks @arcanis, I have raised #21309 in order to fix the problem.

I think that our users would prefer that Renovate could specify the exact version to update a locked dependency to, but that doesn't appear possible right now. We might for example tell them we're updating a range to 1.0.8 but actually update to 1.0.9, but that's hopefully a rare edge case.

@renovate-release
Copy link
Collaborator

🎉 This issue has been resolved in version 35.32.1 🎉

The release is available on:

Your semantic-release bot 📦🚀

@mtlewis
Copy link
Author

mtlewis commented Apr 8, 2023

Hey folks, really appreciate the fix! Just re-ran renovate in the original repro PR above, and it now seems to me to be doing the correct thing. Thanks a lot!

@github-actions github-actions bot locked as resolved and limited conversation to collaborators May 9, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
manager:npm package.json files (npm/yarn/pnpm) status:requirements Full requirements are not yet known, so implementation should not be started type:bug Bug fix of existing functionality
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants