Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix bug For Github Issues #24226

Merged
merged 4 commits into from
Jun 12, 2023
Merged

Fix bug For Github Issues #24226

merged 4 commits into from
Jun 12, 2023

Conversation

Jingshu923
Copy link
Contributor

@Jingshu923 Jingshu923 commented May 31, 2023

ARM API Information (Control Plane)

Changelog

Add a changelog entry for this PR by answering the following questions:

  1. What's the purpose of the update?
    • new service onboarding
    • new API version
    • update existing version for new feature
    • update existing version to fix swagger quality issue in s360
    • Other, please clarify
  2. When are you targeting to deploy the new service/feature to public regions? Please provide the date or, if the date is not yet available, the month.
  3. When do you expect to publish the swagger? Please provide date or, the the date is not yet available, the month.
  4. By default, Azure SDKs of all languages (.NET/Python/Java/JavaScript for both management-plane SDK and data-plane SDK, Go for management-plane SDK only ) MUST be refreshed with/after swagger of new version is published. If you prefer NOT to refresh any specific SDK language upon swagger updates in the current PR, please leave details with justification here.

Contribution checklist (MS Employees Only):

If any further question about AME onboarding or validation tools, please view the FAQ.

ARM API Review Checklist

Applicability: ⚠️

If your changes encompass only the following scenarios, you should SKIP this section, as these scenarios do not require ARM review.

  • Change to data plane APIs
  • Adding new properties
  • All removals

Otherwise your PR may be subject to ARM review requirements. Complete the following:

  • Check this box if any of the following apply to the PR so that the label "ARMReview" and "WaitForARMFeedback" will be added by bot to kick off ARM API Review. Missing to check this box in the following scenario may result in delays to the ARM manifest review and deployment.

    • Adding a new service
    • Adding new API(s)
    • Adding a new API version
      -[ ] To review changes efficiently, ensure you copy the existing version into the new directory structure for first commit and then push new changes, including version updates, in separate commits. You can use OpenAPIHub to initialize the PR for adding a new version. For more details refer to the wiki. Note that this doesn't apply if you are trying to merge a PR that was previously in the private repository.
  • Ensure you've reviewed following guidelines including ARM resource provider contract and REST guidelines. Estimated time (4 hours). This is required before you can request review from ARM API Review board.

  • If you are blocked on ARM review and want to get the PR merged with urgency, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.

Breaking Change Review Checklist

If you have any breaking changes as defined in the Breaking Change Policy, request approval from the Breaking Change Review Board.

Action: to initiate an evaluation of the breaking change, create a new intake using the template for breaking changes. Additional details on the process and office hours are on the Breaking Change Wiki.

NOTE: To update API(s) in public preview for over 1 year (refer to Retirement of Previews)

Please follow the link to find more details on PR review process.

@openapi-pipeline-app
Copy link

openapi-pipeline-app bot commented May 31, 2023

Swagger Validation Report

️❌BreakingChange: 11 Errors, 0 Warnings failed [Detail]
compared swaggers (via Oad v0.10.4)] new version base version
LinkedService.json 2018-06-01(be64441) 2018-06-01(main)
Pipeline.json 2018-06-01(be64441) 2018-06-01(main)
LinkedService.json 2020-12-01(be64441) 2020-12-01(main)
Pipeline.json 2020-12-01(be64441) 2020-12-01(main)
Rule Message
1026 - TypeChanged The new version has a different type 'object' than the previous one 'string'.
New: stable/2018-06-01/entityTypes/Pipeline.json#L2273:9
Old: stable/2018-06-01/entityTypes/Pipeline.json#L2273:9
1026 - TypeChanged The new version has a different type 'object' than the previous one 'string'.
New: stable/2018-06-01/entityTypes/Pipeline.json#L2294:9
Old: stable/2018-06-01/entityTypes/Pipeline.json#L2301:9
1026 - TypeChanged The new version has a different type 'object' than the previous one 'string'.
New: stable/2020-12-01/entityTypes/Pipeline.json#L1902:9
Old: stable/2020-12-01/entityTypes/Pipeline.json#L1902:9
1026 - TypeChanged The new version has a different type 'object' than the previous one 'string'.
New: stable/2020-12-01/entityTypes/Pipeline.json#L1921:9
Old: stable/2020-12-01/entityTypes/Pipeline.json#L1929:9
1045 - AddedOptionalProperty The new version has a new optional property 'authentication' that was not found in the old version.
New: stable/2018-06-01/entityTypes/LinkedService.json#L1884:7
Old: stable/2018-06-01/entityTypes/LinkedService.json#L1884:7
1045 - AddedOptionalProperty The new version has a new optional property 'authentication' that was not found in the old version.
New: stable/2020-12-01/entityTypes/LinkedService.json#L1591:7
Old: stable/2020-12-01/entityTypes/LinkedService.json#L1591:7
1045 - AddedOptionalProperty The new version has a new optional property 'authentication' that was not found in the old version.
New: stable/2020-12-01/entityTypes/LinkedService.json#L1653:7
Old: stable/2020-12-01/entityTypes/LinkedService.json#L1649:7
1049 - RemovedXmsEnum The new version is missing a 'x-ms-enum' found in the old version.
New: stable/2018-06-01/entityTypes/Pipeline.json#L2273:9
Old: stable/2018-06-01/entityTypes/Pipeline.json#L2273:9
1049 - RemovedXmsEnum The new version is missing a 'x-ms-enum' found in the old version.
New: stable/2018-06-01/entityTypes/Pipeline.json#L2294:9
Old: stable/2018-06-01/entityTypes/Pipeline.json#L2301:9
1049 - RemovedXmsEnum The new version is missing a 'x-ms-enum' found in the old version.
New: stable/2020-12-01/entityTypes/Pipeline.json#L1902:9
Old: stable/2020-12-01/entityTypes/Pipeline.json#L1902:9
1049 - RemovedXmsEnum The new version is missing a 'x-ms-enum' found in the old version.
New: stable/2020-12-01/entityTypes/Pipeline.json#L1921:9
Old: stable/2020-12-01/entityTypes/Pipeline.json#L1929:9
️️✔️Breaking Change(Cross-Version) succeeded [Detail] [Expand]
There are no breaking changes.
️️✔️CredScan succeeded [Detail] [Expand]
There is no credential detected.
️🔄LintDiff inProgress [Detail]
️️✔️Avocado succeeded [Detail] [Expand]
Validation passes for Avocado.
️️✔️ApiReadinessCheck succeeded [Detail] [Expand]
️️✔️~[Staging] ServiceAPIReadinessTest succeeded [Detail] [Expand]
Validation passes for ServiceAPIReadinessTest.
️❌SwaggerAPIView: 0 Errors, 0 Warnings failed [Detail]
️️✔️CadlAPIView succeeded [Detail] [Expand]
️️✔️TypeSpecAPIView succeeded [Detail] [Expand]
️️✔️ModelValidation succeeded [Detail] [Expand]
Validation passes for ModelValidation.
️️✔️SemanticValidation succeeded [Detail] [Expand]
Validation passes for SemanticValidation.
️️✔️PoliCheck succeeded [Detail] [Expand]
Validation passed for PoliCheck.
️️✔️PrettierCheck succeeded [Detail] [Expand]
Validation passes for PrettierCheck.
️️✔️SpellCheck succeeded [Detail] [Expand]
Validation passes for SpellCheck.
️️✔️Lint(RPaaS) succeeded [Detail] [Expand]
Validation passes for Lint(RPaaS).
️️✔️CadlValidation succeeded [Detail] [Expand]
Validation passes for CadlValidation.
️️✔️TypeSpec Validation succeeded [Detail] [Expand]
Validation passes for TypeSpec Validation.
️️✔️PR Summary succeeded [Detail] [Expand]
Validation passes for Summary.
Posted by Swagger Pipeline | How to fix these errors?

@openapi-workflow-bot
Copy link

Hi @Jingshu923, this service has enabled Service API Toolset. All spec updates MUST be initiated from service ADO project repo, so that to guarantee it to be the source of truth., please review this pull request if it's intentional or reject it if it's not expected. Normally, all the specification or example changes should start with PR created in ADO.

  • You can refer to Service API Toolset Introduction for details.
  • Please contact service focal contacts or Service API Toolset Support Channel if need further help.
  • @openapi-pipeline-app
    Copy link

    openapi-pipeline-app bot commented May 31, 2023

    Swagger Generation Artifacts

    ️️✔️ApiDocPreview succeeded [Detail] [Expand]

    Only 0 items are rendered, please refer to log for more details.

    ️❌SDK Breaking Change Tracking failed [Detail]

    Only 0 items are rendered, please refer to log for more details.

    ️❌ azure-sdk-for-net-track2 failed [Detail]

    Only 0 items are rendered, please refer to log for more details.

    ️⚠️ azure-sdk-for-python-track2 warning [Detail]

    Only 0 items are rendered, please refer to log for more details.

    ️⚠️ azure-sdk-for-java warning [Detail]

    Only 0 items are rendered, please refer to log for more details.

    ️️✔️ azure-sdk-for-go succeeded [Detail] [Expand]

    Only 0 items are rendered, please refer to log for more details.

    ️️✔️ azure-sdk-for-js succeeded [Detail] [Expand]

    Only 0 items are rendered, please refer to log for more details.

    ️⚠️ azure-resource-manager-schemas warning [Detail]

    Only 0 items are rendered, please refer to log for more details.

    ️❌ azure-powershell failed [Detail]

    Only 0 items are rendered, please refer to log for more details.

    Posted by Swagger Pipeline | How to fix these errors?

    @openapi-pipeline-app
    Copy link

    openapi-pipeline-app bot commented May 31, 2023

    Generated ApiView

    Language Package Name ApiView Link
    Go sdk/resourcemanager/datafactory/armdatafactory https://apiview.dev/Assemblies/Review/962dd4f0c6b046c0b756a3715beeaf3d
    Java azure-resourcemanager-datafactory https://apiview.dev/Assemblies/Review/0da745e55841458e908110ceffc7b4cf
    JavaScript @azure/arm-datafactory https://apiview.dev/Assemblies/Review/6b01ee774234444280e0de560cc9be82

    @ghost ghost added the Synapses label May 31, 2023
    @openapi-workflow-bot openapi-workflow-bot bot added the DoNotMerge <valid label in PR review process> use to hold merge after approval label May 31, 2023
    @openapi-workflow-bot
    Copy link

    Hi, @Jingshu923 Thanks for your PR. I am workflow bot for review process. Here are some small tips.

  • Please ensure to do self-check against checklists in first PR comment.
  • PR assignee is the person auto-assigned and responsible for your current PR reviewing and merging.
  • For specs comparison cross API versions, Use API Specs Comparison Report Generator
  • If there is CI failure(s), to fix CI error(s) is mandatory for PR merging; or you need to provide justification in PR comment for explanation. How to fix?

  • Any feedback about review process or workflow bot, pls contact swagger and tools team. [email protected]

    @openapi-workflow-bot
    Copy link

    Hi @Jingshu923, one or multiple breaking change(s) is detected in your PR. Please check out the breaking change(s), and provide business justification in the PR comment and @ PR assignee why you must have these change(s), and how external customer impact can be mitigated. Please ensure to follow breaking change policy to request breaking change review and approval before proceeding swagger PR review.
    Action: To initiate an evaluation of the breaking change, create a new intake using the template for breaking changes. Addition details on the process and office hours are on the Breaking change Wiki.
    If you want to know the production traffic statistic, please see ARM Traffic statistic.
    If you think it is false positive breaking change, please provide the reasons in the PR comment, report to Swagger Tooling Team via https://aka.ms/swaggerfeedback.
    Note: To avoid breaking change, you can refer to Shift Left Solution for detecting breaking change in early phase at your service code repository.

    @JeffreyRichter
    Copy link
    Member

    Why are you removing x-ms-enum and turning the properties into simple strings? Especially since the docs state what the valid string values are.

    @Jingshu923
    Copy link
    Contributor Author

    @JeffreyRichter In our UX, it could support expression, so there are customers ask we support expression in swagger and sdk
    image

    @JeffreyRichter
    Copy link
    Member

    With x-ms-enums, if you set model-as-string to true, then any string can actually be passed/returned and this would work for your dynamic content. I think your enums were already marked this way. The reason why your proposal is breaking is because all SDKs that now have enums due to the way the swagger was, will take the enum types away and this will break any customer code that referred to the enum types. So, I'm encouraging you to leave the swagger the way it was to avoid breaking your customers.

    @JeffreyRichter
    Copy link
    Member

    With x-ms-enums, if you set model-as-string to true, then any string can actually be passed/returned and this would work for your dynamic content. I think your enums were already marked this way. The reason why your proposal is breaking is because all SDKs that now have enums due to the way the swagger was will take the enum types away and this will break any customer code that referred to the enum types.

    @Jingshu923
    Copy link
    Contributor Author

    @JeffreyRichter I know this will break existing customers using it for get value. but we could not avoid this kind of change.
    Previous customers can only set it as string
    "readBehavior": "Query".

    But now customers can set it as string or expression
    "readBehavior": "Query"
    or
    "readBehavior":{"type":"Expression","value":"anything"}

    sdk link: https://github.com/Azure/azure-sdk-for-net/blob/3c4795de8e87f31065ef0b3a790d573cd678ba6f/sdk/datafactory/Microsoft.Azure.Management.DataFactory/src/Generated/Models/SalesforceSource.cs#LL83C26-L83C26

    @JeffreyRichter
    Copy link
    Member

    This doesn't look right to me. In the SDK link you sent the Query property is of type 'object' - what are customers supposed to do with this? When data comes in over the wire, what would it get deserialized to. If Query were set to some object (any object) that it gets serialized to be sent to the service - how will the service handle this? This design looks very questionable and unusable to me.

    Maybe we need a design session to design this better?

    @Frey-Wang
    Copy link
    Member

    Hi @JeffreyRichter, this is the expression feature in ADF that has been widely used across many different properties. Due to the limitation of swagger, we can't use oneOf, and thus we have to set it to object. From SDK perspective, we are working with SDK team on the new Track2 SDK and Josh has proposed a design on this to help customer construct the properties that support expression, please check here.

    @JeffreyRichter
    Copy link
    Member

    We have to think about customers who do not use an SDK at all and we also have to think about customers using SDK languages other than C# (Josh's prototype). I still think this property should be modeled as an expression STRING in the REST API and then customers/SDKs can parse this string however they want (or not at all). Does the service store this as an arbitrary JSON object - whatever the client passes to it? What if the JSON object is 5 megabytes in size? How does the service validate this JSN object? Plus, you're introducing a breaking change to customers and we always try to avoid that.

    @JeffreyRichter JeffreyRichter added the Approved-BreakingChange DO NOT USE! OBSOLETE label. See https://github.com/Azure/azure-sdk-tools/issues/6374 label Jun 8, 2023
    @lmazuel
    Copy link
    Member

    lmazuel commented Jun 12, 2023

    Merging, as @JeffreyRichter approved the breaking change

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    Approved-BreakingChange DO NOT USE! OBSOLETE label. See https://github.com/Azure/azure-sdk-tools/issues/6374 Approved-SdkBreakingChange-Python BreakingChangeReviewRequired <valid label in PR review process>add this label when breaking change review is required CI-BreakingChange-Go CI-BreakingChange-JavaScript Data Factory data-plane DoNotMerge <valid label in PR review process> use to hold merge after approval resource-manager ShiftLeftViolation Synapses
    Projects
    None yet
    Development

    Successfully merging this pull request may close these issues.

    9 participants