Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tools: Make "check-composer-deps" into "check-intra-monorepo-deps" #20297

Merged
merged 2 commits into from
Jul 7, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/files/renovate-helper.sh
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,7 @@ fi

# Update deps and lock files.
echo "::group::Updating dependencies on changed packages"
tools/check-composer-deps.sh -uv
tools/check-intra-monorepo-deps.sh -uv
echo "::endgroup::"

# Create and push the commit.
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/linting.yml
Original file line number Diff line number Diff line change
Expand Up @@ -783,7 +783,7 @@ jobs:
- run: .github/files/check-lock-files.sh

### Check that monorepo packages are correctly referenced.
# Local equivalent: tools/check-composer-deps.sh -v && .github/files/check-monorepo-package-repos.sh
# Local equivalent: tools/check-intra-monorepo-deps.sh -v && .github/files/check-monorepo-package-repos.sh
monorepo_package_refs:
name: Monorepo package version refs
runs-on: ubuntu-latest
Expand All @@ -796,7 +796,7 @@ jobs:
run: |
which jq
jq --version
- run: tools/check-composer-deps.sh -v
- run: tools/check-intra-monorepo-deps.sh -v
- run: .github/files/check-monorepo-package-repos.sh

### Checks against project structure, e.g. that composer.json exists.
Expand Down
6 changes: 3 additions & 3 deletions pnpm-lock.yaml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
Significance: patch
Type: changed

Updated package dependencies.
2 changes: 1 addition & 1 deletion projects/packages/connection-ui/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
"dependencies": {
"@automattic/calypso-build": "6.5.0",
"@automattic/jetpack-components": "workspace:^0.2.0-alpha",
"@automattic/jetpack-connection": "workspace:^0.4.0-alpha",
"@automattic/jetpack-connection": "workspace:^0.4.0",
"@babel/core": "7.12.10",
"@babel/helper-module-imports": "7.12.5",
"@babel/preset-env": "7.12.11",
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
Significance: patch
Type: changed

Updated package dependencies.
2 changes: 1 addition & 1 deletion projects/plugins/backup/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@
"extends @wordpress/browserslist-config"
],
"dependencies": {
"@automattic/jetpack-connection": "workspace:^0.4.0-alpha",
"@automattic/jetpack-connection": "workspace:^0.4.0",
"@wordpress/data": "4.27.3",
"@wordpress/i18n": "3.20.0",
"react": "16.14.0",
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
Significance: patch
Type: other

Updated package dependencies.
2 changes: 1 addition & 1 deletion projects/plugins/jetpack/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@
"@automattic/components": "1.0.0-alpha.3",
"@automattic/format-currency": "1.0.0-alpha.0",
"@automattic/jetpack-components": "workspace:^0.2.0-alpha",
"@automattic/jetpack-connection": "workspace:^0.4.0-alpha",
"@automattic/jetpack-connection": "workspace:^0.4.0",
"@automattic/popup-monitor": "1.0.0",
"@automattic/request-external-access": "1.0.0",
"@automattic/social-previews": "1.1.1",
Expand Down
74 changes: 55 additions & 19 deletions tools/check-composer-deps.sh → tools/check-intra-monorepo-deps.sh
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ function usage {
cat <<-EOH
usage: $0 [-u] [-v]

Check that all composer dependencies between monorepo projects are up to date.
Check that all composer and pnpm dependencies between monorepo projects are up to date.

If \`-u\` is passed, update any that aren't and add changelogger change files
for the updates.
Expand Down Expand Up @@ -60,12 +60,14 @@ fi
function get_packages {
PACKAGES1=$(jq -nc 'reduce inputs as $in ({}; .[$in.name] |= if $in.extra["branch-alias"]["dev-master"] then [ $in.extra["branch-alias"]["dev-master"], ( $in.extra["branch-alias"]["dev-master"] | sub( "^(?<v>\\d+\\.\\d+)\\.x-dev$"; "^\(.v)" ) ) ] else [ "@dev" ] end )' "$BASE"/projects/packages/*/composer.json)
PACKAGES2=$(jq -c '( .[][0] | select( . != "@dev" ) ) |= empty' <<<"$PACKAGES1")
JSPACKAGES=$(jq -nc 'reduce inputs as $in ({}; if $in.name then .[$in.name] |= [ "workspace:^\($in.version)", "workspace:\($in.version)" ] else . end )' "$BASE"/projects/js-packages/*/package.json)
}

get_packages

# Use a temp variable so pipefail works
TMP="$(tools/get-build-order.php 2>/dev/null)"
TMP=monorepo$'\n'"$TMP"
SLUGS=()
mapfile -t SLUGS <<<"$TMP"

Expand Down Expand Up @@ -111,50 +113,84 @@ for SLUG in "${SLUGS[@]}"; do
else
PACKAGES="$PACKAGES1"
fi
FILE="projects/$SLUG/composer.json"
if [[ "$SLUG" == monorepo ]]; then
DOCL=false
DIR=.
PHPFILE=composer.json
JSFILE=package.json
else
DOCL=true
DIR="projects/$SLUG"
PHPFILE="projects/$SLUG/composer.json"
JSFILE="projects/$SLUG/package.json"
fi
if $UPDATE; then
JSON=$(jq --argjson packages "$PACKAGES" -r 'def ver(e): if $packages[e.key] then if e.value[0:1] == "^" then $packages[e.key][1] else null end // $packages[e.key][0] else e.value end; if .require then .require |= with_entries( .value = ver(.) ) else . end | if .["require-dev"] then .["require-dev"] |= with_entries( .value = ver(.) ) else . end' "$FILE" | tools/prettier --parser=json-stringify)
DIDCL=false
if [[ "$JSON" != "$(<"$FILE")" ]]; then
info "Dependencies of $SLUG changed!"
echo "$JSON" > "$FILE"

info "Creating changelog entry for $SLUG"
changelogger "$SLUG" 'Updated package dependencies.'
DIDCL=true
JSON=$(jq --argjson packages "$PACKAGES" -r 'def ver(e): if $packages[e.key] then if e.value[0:1] == "^" then $packages[e.key][1] else null end // $packages[e.key][0] else e.value end; if .require then .require |= with_entries( .value = ver(.) ) else . end | if .["require-dev"] then .["require-dev"] |= with_entries( .value = ver(.) ) else . end' "$PHPFILE" | tools/prettier --parser=json-stringify)
if [[ "$JSON" != "$(<"$PHPFILE")" ]]; then
info "PHP dependencies of $SLUG changed!"
echo "$JSON" > "$PHPFILE"

if $DOCL; then
info "Creating changelog entry for $SLUG"
changelogger "$SLUG" 'Updated package dependencies.'
DOCL=false
fi
fi
if [[ -e "$JSFILE" ]]; then
JSON=$(jq --argjson packages "$JSPACKAGES" -r 'def ver(e): if $packages[e.key] then if e.value[0:1] == "^" then $packages[e.key][1] else null end // $packages[e.key][0] else e.value end; def proc(k): if .[k] then .[k] |= with_entries( .value = ver(.) ) else . end; proc("dependencies") | proc("devDependencies") | proc("peerDependencies") | proc("optionalDependencies")' "$JSFILE" | tools/prettier --parser=json-stringify)
if [[ "$JSON" != "$(<"$JSFILE")" ]]; then
info "JS dependencies of $SLUG changed!"
echo "$JSON" > "$JSFILE"

if $DOCL; then
info "Creating changelog entry for $SLUG"
changelogger "$SLUG" 'Updated package dependencies.'
DOCL=false
fi
fi
fi
if [[ -n "$(git -c core.quotepath=off ls-files "projects/$SLUG/composer.lock")" ]]; then
PROJECTFOLDER="$BASE/projects/$SLUG"
if [[ -n "$(git -c core.quotepath=off ls-files "$DIR/composer.lock")" ]]; then
PROJECTFOLDER="$BASE/$DIR"
cd "$PROJECTFOLDER"
debug "Updating $SLUG composer.lock"
OLD="$(<composer.lock)"
"$BASE/tools/composer-update-monorepo.sh" --quiet "$PROJECTFOLDER"
if [[ "$OLD" != "$(<composer.lock)" ]] && ! $DIDCL; then
if [[ "$OLD" != "$(<composer.lock)" ]] && $DOCL; then
info "Creating changelog entry for $SLUG composer.lock update"
changelogger "$SLUG" '' 'Updated composer.lock.'
DIDCL=true
DOCL=false
fi
cd "$BASE"
fi
else
while IFS=" " read -r PKG VER; do
while IFS=$'\t' read -r FILE PKG VER; do
EXIT=1
LINE=$(grep --line-number --fixed-strings --max-count=1 "$PKG" "$FILE")
if [[ -n "$CI" ]]; then
M="::error file=$FILE"
[[ -n "$LINE" ]] && M="$M,line=${LINE%%:*}"
echo "$M::Must depend on monorepo package $PKG version $VER%0AYou might use \`tools/check-composer-deps.sh -u\` to fix this."
echo "$M::Must depend on monorepo package $PKG version $VER%0AYou might use \`tools/check-intra-monorepo-deps.sh -u\` to fix this."
else
M="$FILE"
[[ -n "$LINE" ]] && M="$M:${LINE%%:*}"
error "$M: Must depend on monorepo package $PKG version $VER"
fi
done < <( jq --argjson packages "$PACKAGES" -r '.require // {}, .["require-dev"] // {} | to_entries[] | select( $packages[.key] as $vals | $vals and ( [ .value ] | inside( $vals ) | not ) ) | .key + " " + ( $packages[.key] | join( " or " ) )' "$FILE" )
done < <(
jq --argjson packages "$PACKAGES" -r '.require // {}, .["require-dev"] // {} | to_entries[] | select( $packages[.key] as $vals | $vals and ( [ .value ] | inside( $vals ) | not ) ) | [ input_filename, .key, ( $packages[.key] | join( " or " ) ) ] | @tsv' "$PHPFILE"
if [[ -e "$JSFILE" ]]; then
jq --argjson packages "$JSPACKAGES" -r '.dependencies // {}, .devDependencies // {}, .peerDependencies // {}, .optionalDependencies // {} | to_entries[] | select( $packages[.key] as $vals | $vals and ( [ .value ] | inside( $vals ) | not ) ) | [ input_filename, .key, ( $packages[.key] | join( " or " ) ) ] | @tsv' "$JSFILE"
fi
)
fi
done

if $UPDATE; then
debug "Updating pnpm-lock.yaml"
pnpm install --silent
fi

if ! $UPDATE && [[ "$EXIT" != "0" ]]; then
jetpackGreen 'You might use `tools/check-composer-deps.sh -u` to fix these errors.'
jetpackGreen 'You might use `tools/check-intra-monorepo-deps.sh -u` to fix these errors.'
fi

exit $EXIT