Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restructure extension to better support multiple task versions #1326

Merged

Conversation

rhyskoedijk
Copy link
Contributor

@rhyskoedijk rhyskoedijk commented Sep 3, 2024

What are you trying to achieve?

Restructure the extension so that when #1318 is reviewed, there are less changes to review.

Changes

  • Moved extension\task => extension\tasks\dependabot\dependabotV1 to match the folder structure for multiple build task versions within one extension.
  • Moved *.test.ts to the same directory as the file it is testing; This is going to help with clarity later where there are different tests for V1 and V2.
  • Refactored build:prod script to a post dependency lifecycle script. This runs whenever npm install runs, simplifying the build steps for local development.
  • Added npm start script for easier dev testing.
  • Added npm run package script for easier dev testing.
  • Updated extension development guide documentation.
  • Reset the minor/patch version numbers in all config files, for clarity. They are automatically set in CI workflow.

Folder structure

  • tasks
    • dependabot
      • dependabotV1
        • utils
          • parseConfigFile.ts
          • parseConfigFile.test.ts
          • etc...
        • task.json
        • icon.png
        • index.ts
  • tests
    • config
      • dependabot.yaml
      • sample-registries.yaml

I have published the extension to DevOps and confirmed that it runs without error after these changes.

@rhyskoedijk rhyskoedijk changed the title Restructure extension task to better support multiple version Restructure extension to better support multiple task versions Sep 3, 2024
@rhyskoedijk rhyskoedijk marked this pull request as ready for review September 4, 2024 00:53
@rhyskoedijk
Copy link
Contributor Author

rhyskoedijk commented Sep 4, 2024

@mburumaxwell it appears that each task version folder needs to be self-contained and cannot reference shared scripts from parent directories when published to a .vsix package. Because of this, some of the util scripts will need to be duplicated when V2 is added.

It seems some extension work around this issue by using build scripts to copy the shared scripts into the task directory prior to publishing. I don't have much experience with TypeScript and am unsure how to make this work whilst also correcting the module import paths, so I have not done this.

It might be something that needs to be revisited later if code duplication is a big concern.

Happy to make further changes if any of this doesn't sit well with you.

@mburumaxwell
Copy link
Contributor

@mburumaxwell it appears that each task version folder needs to be self-contained and cannot reference shared scripts from parent directories when published to a .vsix package. Because of this, some of the util scripts will need to be duplicated when V2 is added.

It seems some extension work around this issue by using build scripts to copy the shared scripts into the task directory prior to publishing. I don't have much experience with TypeScript and am unsure how to make this work whilst also correcting the module import paths, so I have not done this.

It might be something that needs to be revisited later if code duplication is a big concern.

Happy to make further changes if any of this doesn't sit well with you.

Duplication is not a concern at the moment. However, we may sort it by binding using tsup sometime later.

@mburumaxwell mburumaxwell merged commit 03e65cc into tinglesoftware:main Sep 4, 2024
3 checks passed
@rhyskoedijk rhyskoedijk deleted the feature/task_v1_restructure branch September 4, 2024 05:53
kzhuklinets added a commit to kirillcoso/dependabot-azure-devops that referenced this pull request Oct 4, 2024
* Update the server version/component to support directories and groups (tinglesoftware#1294)

* Update .gitignore

* Formatting fixes

* Update dependabot.yml

* Add prettier formatting (tinglesoftware#1304)

This will mostly format files in the extension but can format some other non-js/ts files that it supports like json, yaml, etc.

At the root `package.json` and `package-lock.json` files are added but a workspace is not used because we do not need it and it is preferred for the extension to be independent to ensure it is self-contained during deployment. This may change if we can use `tsup` to package it.

* Update packages to be included in the microsoft group

* Bump @types/node in /extension in the js-ts-types group (tinglesoftware#1296)

* Bump ts-jest from 29.2.4 to 29.2.5 in /extension in the jest group (tinglesoftware#1295)

* Bump axios from 1.7.4 to 1.7.5 in /extension (tinglesoftware#1298)

* Bump vcr from 6.2.0 to 6.3.1 in /updater (tinglesoftware#1302)

* Bump Azure.Monitor.Query from 1.4.0 to 1.5.0 in the azure group (tinglesoftware#1299)

* Add missing EFCore migration for directories and groups support (tinglesoftware#1305)

I added support for directories and groups in tinglesoftware#1294 but forgot to add the database migration. Possibly because I do not have unit tests against the actual database.

* Bump Microsoft.NET.Test.Sdk from 17.10.0 to 17.11.0 (tinglesoftware#1306)

* Bump dotnet-ef from 8.0.7 to 8.0.8 (tinglesoftware#1303)

Bumps [dotnet-ef](https://github.com/dotnet/efcore) from 8.0.7 to 8.0.8.
- [Release notes](https://github.com/dotnet/efcore/releases)
- [Commits](dotnet/efcore@v8.0.7...v8.0.8)

---
updated-dependencies:
- dependency-name: dotnet-ef
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump azure-pipelines-task-lib from 4.16.0 to 4.17.0 in /extension (tinglesoftware#1308)

Bumps [azure-pipelines-task-lib](https://github.com/Microsoft/azure-pipelines-task-lib) from 4.16.0 to 4.17.0.
- [Commits](https://github.com/Microsoft/azure-pipelines-task-lib/commits)

---
updated-dependencies:
- dependency-name: azure-pipelines-task-lib
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Test that token replacement works when padded with spaces

* Update docs and README.md

* Add swift to `ECOSYSTEM_LOOKUP` for vulnerabilities (tinglesoftware#1309)

* Improvement to credentials handling (tinglesoftware#1310)

- Handle setting `index-url` instead of `url` for `python_index` registries in the server version/component.
- Handle setting both `host` and `url` for `composer_repository` registries.
- Better logic around building extra credentials.

> This is likely the last modification before we try to move to the proxy used by `dependabot-cli`.

* Merge `mv:prod` script into `build:prod`

* Formatting fixes

* Exclude .idea folders from prettier

* Regenerate Gemfile.lock which should vulnerability in Rexml

* Register span processor for sentry to fix OTel export (tinglesoftware#1311)

This should fix the OpenTelemetry export issue.
It may be reverted in the future when `dependabot-core/updater` makes other changes.

See https://docs.sentry.io/platforms/ruby/tracing/instrumentation/opentelemetry/

* Update updater.yml

* Update extension task node version from v16 to v20.1 (tinglesoftware#1313)

* Update task node version from 16 to 20.1

* Bump minimum agent version

* Fix vNext AZURE_VIRTUAL_DIRECTORY is not working bug (tinglesoftware#1316)

* Bump @types/node (tinglesoftware#1322)

* Bump YamlDotNet from 16.0.0 to 16.1.0 (tinglesoftware#1325)

* Bump Microsoft.Data.SqlClient from 5.2.1 to 5.2.2 (tinglesoftware#1324)

* Bump axios from 1.7.5 to 1.7.7 in /extension (tinglesoftware#1323)

* Bump opentelemetry-exporter-otlp in /updater in the opentelemetry group (tinglesoftware#1319)

* Restructure extension to better support multiple task versions (tinglesoftware#1326)

* Restructure extension task to better support multiple version

* Fix typos

* Fix build

* Fix build

* Bump @types/node (tinglesoftware#1327)

Bumps the js-ts-types group with 1 update in the /extension directory: [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node).


Updates `@types/node` from 22.5.2 to 22.5.3
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node)

---
updated-dependencies:
- dependency-name: "@types/node"
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: js-ts-types
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump Microsoft.NET.Test.Sdk from 17.11.0 to 17.11.1 (tinglesoftware#1329)

Bumps [Microsoft.NET.Test.Sdk](https://github.com/microsoft/vstest) from 17.11.0 to 17.11.1.
- [Release notes](https://github.com/microsoft/vstest/releases)
- [Changelog](https://github.com/microsoft/vstest/blob/main/docs/releases.md)
- [Commits](microsoft/vstest@v17.11.0...v17.11.1)

---
updated-dependencies:
- dependency-name: Microsoft.NET.Test.Sdk
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump @types/node (tinglesoftware#1332)

Bumps the js-ts-types group with 1 update in the /extension directory: [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node).


Updates `@types/node` from 22.5.3 to 22.5.4
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node)

---
updated-dependencies:
- dependency-name: "@types/node"
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: js-ts-types
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump dependabot-omnibus from 0.268.0 to 0.274.0 in /updater (tinglesoftware#1331)

Bumps [dependabot-omnibus](https://github.com/dependabot/dependabot-core) from 0.268.0 to 0.274.0.
- [Release notes](https://github.com/dependabot/dependabot-core/releases)
- [Changelog](https://github.com/dependabot/dependabot-core/blob/main/CHANGELOG_ARCHIVE_2019_TO_SWITCH_TO_GITHUB_RELEASES.md)
- [Commits](dependabot/dependabot-core@v0.268.0...v0.274.0)

---
updated-dependencies:
- dependency-name: dependabot-omnibus
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Format fix

* Install ice and jq inside the @azure/cli task

* Bump typescript from 5.5.4 to 5.6.2 in /extension (tinglesoftware#1337)

* Bump azure-pipelines-task-lib from 4.17.0 to 4.17.2 in /extension (tinglesoftware#1338)

* Bump dependabot-omnibus from 0.274.0 to 0.275.0 in /updater (tinglesoftware#1339)

Bumps [dependabot-omnibus](https://github.com/dependabot/dependabot-core) from 0.274.0 to 0.275.0.
- [Release notes](https://github.com/dependabot/dependabot-core/releases)
- [Changelog](https://github.com/dependabot/dependabot-core/blob/main/CHANGELOG_ARCHIVE_2019_TO_SWITCH_TO_GITHUB_RELEASES.md)
- [Commits](dependabot/dependabot-core@v0.274.0...v0.275.0)

---
updated-dependencies:
- dependency-name: dependabot-omnibus
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump YamlDotNet from 16.1.0 to 16.1.2 (tinglesoftware#1340)

Bumps [YamlDotNet](https://github.com/aaubry/YamlDotNet) from 16.1.0 to 16.1.2.
- [Release notes](https://github.com/aaubry/YamlDotNet/releases)
- [Commits](aaubry/YamlDotNet@v16.1.0...v16.1.2)

---
updated-dependencies:
- dependency-name: YamlDotNet
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump @types/jest in /extension in the jest group across 1 directory (tinglesoftware#1341)

Bumps the jest group with 1 update in the /extension directory: [@types/jest](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/jest).


Updates `@types/jest` from 29.5.12 to 29.5.13
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/jest)

---
updated-dependencies:
- dependency-name: "@types/jest"
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: jest
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump @types/node (tinglesoftware#1342)

Bumps the js-ts-types group with 1 update in the /extension directory: [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node).


Updates `@types/node` from 22.5.4 to 22.5.5
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node)

---
updated-dependencies:
- dependency-name: "@types/node"
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: js-ts-types
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Sync updater files to version 0.275.0 (tinglesoftware#1343)

* Bump Azure.ResourceManager.AppContainers in the azure group (tinglesoftware#1336)

Bumps the azure group with 1 update: [Azure.ResourceManager.AppContainers](https://github.com/Azure/azure-sdk-for-net).


Updates `Azure.ResourceManager.AppContainers` from 1.2.0 to 1.3.0
- [Release notes](https://github.com/Azure/azure-sdk-for-net/releases)
- [Commits](Azure/azure-sdk-for-net@Azure.ResourceManager.AppContainers_1.2.0...Azure.ResourceManager.AppContainers_1.3.0)

---
updated-dependencies:
- dependency-name: Azure.ResourceManager.AppContainers
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: azure
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Remove workaround for reading ContainerApp Job execution (tinglesoftware#1344)

The library has since been updated; we no longer need a workaround.

See also, Azure/azure-sdk-for-net#38385

* Add Task V2; Use Dependabot-CLI to perform updates (tinglesoftware#1318)

* Split extension task in to V1 and V2

* Basic support for running update using dependabot-cli

* Add missing update job configs

* Move update output processing to dedicated class; Add DevOps API client

* Codespell skip package-lock.json

* Clean-up

* Implement create pull request

* Implement groups, auto-complete, auto-approve, pull request properties, and updating existing pull requests

* Clean up

* Set task V1 as deprecated, task V2 as preview

* Restructure extension task to better support multiple version

* Fix typos

* Fix build

* Fix build

* Fix merge issues

* Remove unused task inputs

* Fix for 'convertPlaceholder' not accepted built-in DevOps variable names containing '.'

* Implement closing pull requests

* Implement updating pull requests

* Use default branch name if target branch not configured

* Implement approving pull requests

* Add task inputs for pr commit author email and name

* Implement open pull request limit config

* Cleanup temporary files after task completion

* Add configuration placeholders for dependabot component images

* Implement more config options

* Implement dependency list snapshots, which are stored in the DevOps project properties

* Add task input option for storing dependency list

* Implement experiments

* Implement requirements-update-strategy and lockfile-only configs

* Fix typo

* Implement PR reviewers, work item references, and labels

* Add start commands for each task version, use V2 by default

* Update V1 task.json version numbers when publishing the extension

* Update documentation

* Update documentation

* Update documentation

* Update documentation

* Typo

* Update documentation

* Fix reference to undefined `this.cachedUserIds`

* Use case insensitive comparision when parsing "System.Debug" variable

* Fix dependabot tool path detection in agents where `$PATH` does not contain `$GOPATH/bin`

* Add more logging

* Fix for task reporting success when pull request creation failed

* Add more logging; Fix formatting

* Fix 'labels' config parsing

* Implement "targetUpdateIds" task input option

* Fix error when using multiple update blocks in dependabot.yml with the same package manager

* Only install dependabot once; cache the tool path once known

* Add migration warning to complete V1 pull requests before migrating to V2

* Process updates synchronously when using multiple update blocks in dependabot.yml

* Fix typos

* Report the total number of failed update jobs in the task result

* Include stack trace when errors are logged, to help with diagnosing issues

* Fix inverted logic for "abandonUnwantedPullRequests"

* Fix error handling

* Check formatting (prettier) in the extension workflow (tinglesoftware#1345)

* Fix version for `azure-devops-node-api`

* Set `$schema` on vss-extension.json

* Reduce nesting of tasks by one level (tinglesoftware#1346)

This is likely to help with discovery which only checks upto 1 level

https://learn.microsoft.com/en-us/azure/devops/extend/develop/integrate-build-task?toc=%2Fazure%2Fdevops%2Fmarketplace-extensibility%2Ftoc.json&view=azure-devops#multiple-versions-layout

* Change identifier for v2 task

* Revert "Change identifier for v2 task"

This reverts commit c5d84ce.

* Change name for v2 task

* Bump prettier-plugin-organize-imports from 4.0.0 to 4.1.0 (tinglesoftware#1352)

* Bump xunit from 2.9.0 to 2.9.1 in the xunit group (tinglesoftware#1351)

* Bump the tingle group with 3 updates (tinglesoftware#1350)

* Add contributing guide; update developer guides (tinglesoftware#1347)

* Add contributing guide

* Waffling

* Update toc

* Fix broken link

* Bump dependabot-omnibus from 0.275.0 to 0.276.0 in /updater (tinglesoftware#1349)

Bumps [dependabot-omnibus](https://github.com/dependabot/dependabot-core) from 0.275.0 to 0.276.0.
- [Release notes](https://github.com/dependabot/dependabot-core/releases)
- [Changelog](https://github.com/dependabot/dependabot-core/blob/main/CHANGELOG_ARCHIVE_2019_TO_SWITCH_TO_GITHUB_RELEASES.md)
- [Commits](dependabot/dependabot-core@v0.275.0...v0.276.0)

---
updated-dependencies:
- dependency-name: dependabot-omnibus
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Revert "Change name for v2 task"

This reverts commit 1e24c54.

* Set `argumentFormat`

* Do not override major version numbers in task.json files

* Fix undefined iteration

* Fix "cannot read property of undefined" error; Fix GitHub image pull rate-limiting (tinglesoftware#1353)

* Fix reading undefined reference when there is no dependency list stored for the project yet (i.e. a first-time run)

* Forward GitHub access token to dependabot-cli to avoid rate-limiting when pulling Docker images; Set Dependabot job id to aid with debugging

* Fix formatting

* Fix another "cannot read property of undefined" error (tinglesoftware#1354)

* Fix reading undefined reference when there are no existing pull requests to update yet

* Improved logging to ensure error stack traces are always dumped; helps with debugging

* Regenerate updater/Gemfile.lock, should fix some security concerns

* Sync updater files to version 0.276.0

* Attempt to fix brain fart from https://github.com/rhyskoedijk

See tinglesoftware#1354 (comment)

* Set `IncludeSourceRevisionInInformationalVersion=false` to allow GitVersion to work

See https://learn.microsoft.com/en-us/dotnet/core/compatibility/sdk/8.0/source-link

* Update known experiments list (tinglesoftware#1361)

* Fix "cannot read properties of null" (tinglesoftware#1359)

* Fix "undefined is not iterable" (tinglesoftware#1358)

* Fix GitHub rate limiting when generating pull request descriptions (tinglesoftware#1362)

* Set github registry credentials to avoid rate-limiting when generating pull request descriptions

* Set github registry credentials to avoid rate-limiting when generating pull request descriptions

* Run `go install dependabot` async so logs are streamed to task output instead of blocking until process has completed (tinglesoftware#1357)

* Log the number of pull requests that would have been updated when `skipPullRequests` is set (tinglesoftware#1360)

* Fix "cannot read properties of undefined" (reading 'filter') (tinglesoftware#1364)

* Fix "pull request cannot be activated because the source and/or the target branch no longer exists" error (tinglesoftware#1366)

* Strip reference prefix from the default branch name, the caller doesn't need to know this

* Use regex, match start of string, ignore case

* Upload flame graph report to pipeline timeline when `System.Debug` is `true` (tinglesoftware#1363)

* Attach flamegraph report to pipeline timeline when `System.Debug` is `true`

* Add documentation

* Fix formatting

* Bump xunit from 2.9.1 to 2.9.2 in the xunit group (tinglesoftware#1369)

* Bump Azure.Identity from 1.12.0 to 1.12.1 in the azure group (tinglesoftware#1368)

* Bump the sentry group in /updater with 2 updates (tinglesoftware#1373)

* Bump @types/node (tinglesoftware#1371)

* Bump YamlDotNet from 16.1.2 to 16.1.3 (tinglesoftware#1370)

* Bump azure-devops-node-api from 14.0.2 to 14.1.0 in /extension (tinglesoftware#1372)

* Bump dependabot-omnibus from 0.276.0 to 0.278.0 in /updater (tinglesoftware#1374)

Bumps [dependabot-omnibus](https://github.com/dependabot/dependabot-core) from 0.276.0 to 0.278.0.
- [Release notes](https://github.com/dependabot/dependabot-core/releases)
- [Changelog](https://github.com/dependabot/dependabot-core/blob/main/CHANGELOG_ARCHIVE_2019_TO_SWITCH_TO_GITHUB_RELEASES.md)
- [Commits](dependabot/dependabot-core@v0.276.0...v0.278.0)

---
updated-dependencies:
- dependency-name: dependabot-omnibus
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Fix merge strategy parsing and default value (tinglesoftware#1376)

* Fix "unknown package manager" error (tinglesoftware#1377)

* Add support for the "pull-request-branch-name.separator" config option (tinglesoftware#1378)

* Add support for "pull-request-branch-name.separator" config

* Update migration documentation

* Apply suggestions from code review

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: Maxwell Weru <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Rhys Koedijk <[email protected]>
Co-authored-by: Yurii Yakymenko <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants