tools/data-api-repository: consolidating the reading/writing of data into data-api-repository / updated support for Common Types
#4217
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Pedantic, but enables us to support `serve-watch` in the future/have these defined once for the tool
… take the SourceDataType Each `Repository` instance is tied to a SourceDataType, therefore we'll need one for Resource Manager and one for Microsoft Graph within the Data API - however the Importers can each use a single instance.
…I Version. Whilst this information is stored alongside these as the key to the associated value; it's useful to have this information stored alongside, too.
…ry function changing
This conflicts with `TenantProperties`
…dels with no Operations In the event there's only Constants and Models these aren't usable, so we may as well skip these?
This both consolidates the logic from `./tools/data-api/internal/repository` (and `./tools/data-api/internal/endpoints/transforms`) as well as refactors more of the logic into the `internal` package, with the intention to only expose the public surface area and not the types used for direct access on-disk.
…for loading data
…dding `Name` and `APIVersion` to Service and APIVersion respectively
…values (since this is currently mixed) Matches the others
…common-types` and ensuring it's not loaded as a Service
… are stored, so that each Stage is more specific to a Service
…sdk`: changing Common Types to be tied to an API Version Chatting with @manicminer we want this tied to an API Version (for example `V1` and `Beta` for MSGraph), so this commit changes the endpoints and loads that data from disk in a directory named `common-types`
tombuildsstuff
added
tool/data-api
…e to corporate tooling intefering with io..
…ses in the tests
… on errors rather than raising them
This was referenced Jun 20, 2024
Merged
manicminer
approved these changes
Jun 20, 2024
There was a problem hiding this comment.
Thanks @tombuildsstuff! Just a couple minor comments but this otherwise looks good 👍
Co-authored-by: Tom Bamford <[email protected]>
Co-authored-by: Tom Bamford <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR consolidates the Reading and Writing of data into
tools/data-api-repository- meaning that this becomes the single source-of-truth for where data is loaded/persisted from disk.This PR also:
tools/data-apiintotools/data-api-repositorytools/data-apito use this new repositorytools/importer-rest-api-specsto use the newPurgeExistingDatamethod (which I’ve incorporated fromtools/importer-msgraph-metadata: refactor to generate JSON definitions usingdata-api-sdk#4167)tools/importer-rest-api-specsandtools/data-api-repositorywhere APIResources containing Constants and Models with no Operations (and thus weren’t usable) were being output - but not included in the list of APIResources for an APIVersion. As a result of this change, in the next generation, a handful of unused types will be removed.sourceInformationwithin./api-definitions/handwritten-resource-managerso this parses correctlyinternalpackage, so that these remain scoped to this package.AADB2C/2021-04-01-preview/Tenants/Model-UpdateTenantProperties.json) where theUpdateTenantPropertiesmodel was unintentionally namedTenantProperties, meaning that this could get overwritten when loading the data (and the fields removed).Notably this PR does not:
data-api-repositorypackage to align with Meta: refactoring #3754. Whilst that will be done, this is intentionally being done in the next stage, since it’s an internal refactor and not changing the public surface area.Ultimately this PR unblocks #4167