Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove scope from marketplace registration definitions operations #12397

Merged
merged 8 commits into from
Feb 3, 2021

Conversation

skayani
Copy link
Contributor

@skayani skayani commented Jan 12, 2021

MSFT employees can try out our new experience at OpenAPI Hub - one location for using our validation tools and finding your workflow.

Changelog

Please ensure to add changelog with this PR by answering the following questions.

  1. What's the purpose of the update?
    • new service onboarding
    • new API version
    • update existing version for new feature
    • update existing version to fix swagger quality issue in s360
    • Other, please clarify
  2. When you are targeting to deploy new service/feature to public regions? Please provide date, or month to public if date is not available yet.
  3. When you expect to publish swagger? Please provide date, or month to public if date is not available yet.
  4. If it's an update to existing version, please select SDKs of specific language and CLIs that require refresh after swagger is published.
    • SDK of .NET (need service team to ensure code readiness)
    • SDK of Python
    • SDK of Java
    • SDK of Js
    • SDK of Go
    • PowerShell
    • CLI
    • Terraform
    • No, no need to refresh for updates in this PR

Contribution checklist:

If any further question about AME onboarding or validation tools, please view the FAQ.

ARM API Review Checklist

  • Ensure to check this box if one of the following scenarios meet updates in the PR, so that label “WaitForARMFeedback” will be added automatically to involve ARM API Review. Failure to comply may result in delays for manifest application. Note this does not apply to data plane APIs, all “removals” and “adding a new property” no more require ARM API review.

    • Adding new API(s)
    • Adding a new API version
    • Adding a new service
  • Please ensure you've reviewed following guidelines including ARM resource provider contract and REST guidelines. Estimated time (4 hours). This is required before you can request review from ARM API Review board.

  • If you are blocked on ARM review and want to get the PR merged with urgency, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.

Breaking Change Review Checklist

If there are following updates in the PR, ensure to request an approval from API Review Board as defined in the Breaking Change Policy.

  • Removing API(s) in stable version
  • Removing properties in stable version
  • Removing API version(s) in stable version
  • Updating API in stable version with Breaking Change Validation errors
  • Updating API(s) in preview over 1 year

Action: to initiate an evaluation of the breaking change, create a new intake using the template for breaking changes. Addition details on the process and office hours are on the Breaking change Wiki.

Please follow the link to find more details on PR review process.

@skayani skayani requested a review from dramuy January 12, 2021 21:24
@openapi-workflow-bot
Copy link

Hi, @skayani Thanks for your PR. I am workflow bot for review process. Here are some small tips.

  • Please ensure to do self-check against checklists in first PR comment.
  • PR assignee is the person auto-assigned and responsible for your current PR reviewing and merging.
  • For specs comparison cross API versions, Use API Specs Comparison Report Generator
  • If there is CI failure(s), to fix CI error(s) is mandatory for PR merging; or you need to provide justification in PR comment for explanation. How to fix?

  • Any feedback about review process or workflow bot, pls contact swagger and tools team. [email protected]

    @openapi-pipeline-app
    Copy link

    openapi-pipeline-app bot commented Jan 12, 2021

    Swagger Validation Report

    ️❌BreakingChange: 4 Errors, 0 Warnings failed [Detail]

    Rule Message
    1038 - AddedPath The new version is adding a path that was not found in the old version.
    New: Microsoft.ManagedServices/preview/2020-02-01-preview/managedservices.json#L425:5
    1038 - AddedPath The new version is adding a path that was not found in the old version.
    New: Microsoft.ManagedServices/preview/2020-02-01-preview/managedservices.json#L503:5
    1038 - AddedPath The new version is adding a path that was not found in the old version.
    New: Microsoft.ManagedServices/stable/2019-09-01/managedservices.json#L425:5
    1038 - AddedPath The new version is adding a path that was not found in the old version.
    New: Microsoft.ManagedServices/stable/2019-09-01/managedservices.json#L503:5
    ️️✔️LintDiff succeeded [Detail] [Expand]
    Validation passes for LintDiff.

    ️️✔️Avocado succeeded [Detail] [Expand]
    Validation passes for Avocado.
    ️️✔️ModelValidation succeeded [Detail] [Expand]
    Validation passes for ModelValidation.
    ️️✔️SemanticValidation succeeded [Detail] [Expand]
    Validation passes for SemanticValidation.
    ️️✔️[Staging] Cross Version BreakingChange (Base on preview version) succeeded [Detail] [Expand]
    There are no breaking changes.
    ️️✔️[Staging] Cross Version BreakingChange (Base on stable version) succeeded [Detail] [Expand]
    There are no breaking changes.
    ️️✔️CredScan succeeded [Detail] [Expand]
    There is no credential detected.
    Posted by Swagger Pipeline | How to fix these errors?

    @openapi-pipeline-app
    Copy link

    openapi-pipeline-app bot commented Jan 12, 2021

    Swagger Generation Artifacts

    ️️✔️ azure-resource-manager-schemas succeeded [Detail] [Expand]
    • ️✔️Succeeded [Logs]Release - Generate from d606d4b. Schema Automation 14.0.0
      warn	Skip initScript due to not configured
      command	npx n 12
      command	autorest --use=@autorest/[email protected] --version=3.0.6369 --azureresourceschema --multiapi --title=none --pass-thru:subset-reducer --azureresourceschema-folder=/home/vsts/work/1/s/azure-resource-manager-schemas/azure-resource-manager-schemas ../../azure-rest-api-specs/specification/managedservices/resource-manager/readme.md
    • ️✔️managedservices [View full logs]  [Release Schema Changes]
      cmderr	[resource-manager] node-pre-gyp
      cmderr	[resource-manager] WARN Using request for node-pre-gyp https download
      cmderr	[resource-manager]  WARN [email protected] No repository field.
      cmderr	[resource-manager] npm WARN optional SKIPPING OPTIONAL DEPENDENCY: [email protected] (node_modules/fsevents):
      cmderr	[resource-manager] npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for [email protected]: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
      cmderr	[resource-manager]  ERR! code ELIFECYCLE
      cmderr	[resource-manager] npm ERR! errno 11
      cmderr	[resource-manager]  [email protected] test: `mocha -r ts-node/register tests.ts`
      cmderr	[resource-manager] npm ERR! Exit status 11
      cmderr	[resource-manager] npm ERR!
      cmderr	[resource-manager] npm ERR! Failed at the [email protected] test script.
      cmderr	[resource-manager] npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
      cmderr	[resource-manager]  A complete log of this run can be found in:
      cmderr	[resource-manager] npm ERR!     /home/vsts/.npm/_logs/2021-02-03T05_00_12_974Z-debug.log
      cmderr	[resource-manager] Error: Command failed: npm run test --prefix tools
      cmderr	[resource-manager]     at checkExecSyncError (child_process.js:635:11)
      cmderr	[resource-manager]     at Object.execSync (child_process.js:671:15)
      cmderr	[resource-manager]     at Object.<anonymous> (/home/vsts/work/1/s/azure-resource-manager-schemas/azure-resource-manager-schemas/sdkauto_afterscript.js:13:28)
      cmderr	[resource-manager]     at Module._compile (internal/modules/cjs/loader.js:999:30)
      cmderr	[resource-manager]     at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10)
      cmderr	[resource-manager]     at Module.load (internal/modules/cjs/loader.js:863:32)
      cmderr	[resource-manager]     at Function.Module._load (internal/modules/cjs/loader.js:708:14)
      cmderr	[resource-manager]     at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:60:12)
      cmderr	[resource-manager]     at internal/main/run_main_module.js:17:47 {
      cmderr	[resource-manager]   status: 11,
      cmderr	[resource-manager]   signal: null,
      cmderr	[resource-manager]   output: [ null, null, null ],
      cmderr	[resource-manager]   pid: 2961,
      cmderr	[resource-manager]   stdout: null,
      cmderr	[resource-manager]   stderr: null
      cmderr	[resource-manager] }
    ️⚠️ azure-sdk-for-js warning [Detail]
    • ⚠️Warning [Logs]Release - Generate from d606d4b. SDK Automation 14.0.0
      warn	Skip initScript due to not configured
      command	autorest --version=V2 --typescript --license-header=MICROSOFT_MIT_NO_VERSION [email protected]/[email protected] --typescript-sdks-folder=/home/vsts/work/1/s/azure-sdk-for-js/azure-sdk-for-js ../../azure-rest-api-specs/specification/managedservices/resource-manager/readme.md
      warn	No file changes detected after generation
      warn	No package detected after generation
    ️️✔️ azure-sdk-for-net succeeded [Detail] [Expand]
    • ️✔️Succeeded [Logs]Release - Generate from d606d4b. SDK Automation 14.0.0
      warn	Skip initScript due to not configured
      command	sudo apt-get install -y dotnet-sdk-5.0
      command	autorest --version=V2 --csharp --reflect-api-versions --license-header=MICROSOFT_MIT_NO_VERSION [email protected]/[email protected] --csharp-sdks-folder=/home/vsts/work/1/s/azure-sdk-for-net/sdk ../azure-rest-api-specs/specification/managedservices/resource-manager/readme.md
      cmderr	[Autorest] realpath(): Permission denied
      cmderr	[Autorest] realpath(): Permission denied
      cmderr	[Autorest] realpath(): Permission denied
      cmderr	[Autorest] realpath(): Permission denied
      cmderr	[Autorest] realpath(): Permission denied
      cmderr	[Autorest] realpath(): Permission denied
    • ️✔️Microsoft.Azure.Management.ManagedServices [View full logs]  [Release SDK Changes]
    ️⚠️ azure-sdk-for-python warning [Detail]
    • ⚠️Warning [Logs]Release - Generate from d606d4b. SDK Automation 14.0.0
      command	sh scripts/automation_init.sh ../azure-sdk-for-python_tmp/initInput.json ../azure-sdk-for-python_tmp/initOutput.json
      cmderr	[automation_init.sh] Ignoring mock: markers 'python_version <= "2.7"' don't match your environment
      cmderr	[automation_init.sh]   Failed building wheel for ConfigArgParse
      cmderr	[automation_init.sh]   Failed building wheel for yarl
      cmderr	[automation_init.sh]   Failed building wheel for wrapt
      cmderr	[automation_init.sh]   Failed building wheel for multidict
      cmderr	[automation_init.sh] Cannot uninstall requirement azure-nspkg, not installed
      cmderr	[automation_init.sh] Command '['/home/vsts/work/1/s/azure-sdk-for-python_tmp/venv-sdk/bin/python', '-m', 'pip', 'uninstall', '-y', 'azure-nspkg']' returned non-zero exit status 1.
      command	sh scripts/automation_generate.sh ../azure-sdk-for-python_tmp/generateInput.json ../azure-sdk-for-python_tmp/generateOutput.json
    • ️✔️azure-mgmt-managedservices [View full logs]  [Release SDK Changes]
      info	[Changelog] **Features**
      info	[Changelog]
      info	[Changelog]   - Model RegistrationDefinitionProperties has a new parameter eligible_authorizations
      info	[Changelog]   - Model RegistrationAssignmentPropertiesRegistrationDefinitionProperties has a new parameter eligible_authorizations
      info	[Changelog]   - Model Authorization has a new parameter principal_id_display_name
      info	[Changelog]   - Model Authorization has a new parameter delegated_role_definition_ids
      info	[Changelog]   - Added operation group MarketplaceRegistrationDefinitionsOperations
      info	[Changelog]   - Added operation group MarketplaceRegistrationDefinitionsWithoutScopeOperations
    ️⚠️ azure-sdk-for-java warning [Detail]
    • ⚠️Warning [Logs]Release - Generate from d606d4b. SDK Automation 14.0.0
      command	./eng/mgmt/automation/init.sh ../azure-sdk-for-java_tmp/initInput.json ../azure-sdk-for-java_tmp/initOutput.json
      command	./eng/mgmt/automation/generate.py ../azure-sdk-for-java_tmp/generateInput.json ../azure-sdk-for-java_tmp/generateOutput.json
      cmderr	[generate.py] 2021-02-03 05:01:40 INFO [VERSION][Not Found] cannot find version for "com.azure.resourcemanager:azure-resourcemanager-managedservices"
      cmderr	[generate.py] 2021-02-03 05:01:40 INFO [VERSION][Not Found] cannot find stable version, current version "1.0.0-beta.1"
      cmderr	[generate.py] 2021-02-03 05:01:40 INFO autorest --version=3.0.6350 --use=@autorest/[email protected] --java.azure-libraries-for-java-folder=/home/vsts/work/1/s/azure-sdk-for-java --java.output-folder=/home/vsts/work/1/s/azure-sdk-for-java/sdk/managedservices/azure-resourcemanager-managedservices --java.namespace=com.azure.resourcemanager.managedservices   --java --pipeline.modelerfour.additional-checks=false --pipeline.modelerfour.lenient-model-deduplication=true --azure-arm --verbose --sdk-integration --fluent=lite --java.fluent=lite --java.license-header=MICROSOFT_MIT_SMALL ../azure-rest-api-specs/specification/managedservices/resource-manager/readme.md
      cmderr	[generate.py] 2021-02-03 05:02:22 INFO [CI][Success] Write to ci.yml
      cmderr	[generate.py] 2021-02-03 05:02:22 INFO [POM][Process] dealing with pom.xml
      cmderr	[generate.py] 2021-02-03 05:02:22 INFO [POM][Skip] pom already has module azure-resourcemanager-managedservices
      cmderr	[generate.py] 2021-02-03 05:02:22 INFO [POM][Success] Write to pom.xml
      cmderr	[generate.py] 2021-02-03 05:02:22 INFO [POM][Process] dealing with root pom
      cmderr	[generate.py] 2021-02-03 05:02:22 INFO [POM][Success] Write to root pom
      cmderr	[generate.py] WARNING: An illegal reflective access operation has occurred
      cmderr	[generate.py] WARNING: Illegal reflective access by org.codehaus.groovy.reflection.ReflectionUtils (file:/home/vsts/.m2/repository/org/codehaus/groovy/groovy/3.0.0-beta-2/groovy-3.0.0-beta-2-indy.jar) to method java.util.Collections$UnmodifiableCollection.toString()
      cmderr	[generate.py] WARNING: Please consider reporting this to the maintainers of org.codehaus.groovy.reflection.ReflectionUtils
      cmderr	[generate.py] WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
      cmderr	[generate.py] WARNING: All illegal access operations will be denied in a future release
    • ️✔️azure-resourcemanager-managedservices [View full logs]  [Release SDK Changes]
      cmderr	[Inst] 2021-02-03 05:05:46 DEBUG Got artifact_id: azure-resourcemanager-managedservices
      cmderr	[Inst] 2021-02-03 05:05:46 DEBUG Got artifact: pom.xml
      cmderr	[Inst] 2021-02-03 05:05:46 DEBUG Got artifact: azure-resourcemanager-managedservices-1.0.0-beta.1.jar
      cmderr	[Inst] 2021-02-03 05:05:46 DEBUG Match jar package: azure-resourcemanager-managedservices-1.0.0-beta.1.jar
      cmderr	[Inst] 2021-02-03 05:05:46 DEBUG output: {"full": "```sh\ncurl -L \"https://portal.azure-devex-tools.com/api/sdk-dl-pub?p=Azure/12397/azure-sdk-for-java/azure-resourcemanager-managedservices/azure-resourcemanager-managedservices-1.0.0-beta.1.jar\" -o azure-resourcemanager-managedservices-1.0.0-beta.1.jar\nmvn install:install-file -DgroupId=com.azure.resourcemanager -DartifactId=azure-resourcemanager-managedservices -Dversion=1.0.0-beta.0 -Dfile=azure-resourcemanager-managedservices-1.0.0-beta.1.jar -Dpackaging=jar -DgeneratePom=true
      ```"}
    ️️✔️ azure-sdk-for-go succeeded [Detail] [Expand]
    • ️✔️Succeeded [Logs]Release - Generate from d606d4b. SDK Automation 14.0.0
      command	sh ./initScript.sh ../../../../../azure-sdk-for-go_tmp/initInput.json ../../../../../azure-sdk-for-go_tmp/initOutput.json
      command	go run ./tools/generator/main.go ../../../../../azure-sdk-for-go_tmp/generateInput.json ../../../../../azure-sdk-for-go_tmp/generateOutput.json
    • ️✔️managedservices/mgmt/2019-06-01/managedservices [View full logs]  [Release SDK Changes]
      info	[Changelog] No exported changes
    • ️✔️preview/managedservices/mgmt/2019-04-01/managedservices [View full logs]  [Release SDK Changes]
      info	[Changelog] No exported changes
    • ️✔️preview/managedservices/mgmt/2018-06-01/managedservices [View full logs]  [Release SDK Changes]
      info	[Changelog] No exported changes
    ️️✔️[Staging] ApiDocPreview succeeded [Detail] [Expand]
     Please click here to preview with your @microsoft account. 
    Posted by Swagger Pipeline | How to fix these errors?

    @openapi-workflow-bot
    Copy link

    Hi @skayani, one or multiple breaking change(s) is detected in your PR. Please check out the breaking change(s), and provide business justification in the PR comment and @ PR assignee why you must have these change(s), and how external customer impact can be mitigated. Please ensure to follow breaking change policy to request breaking change review and approval before proceeding swagger PR review.
    Action: To initiate an evaluation of the breaking change, create a new intake using the template for breaking changes. Addition details on the process and office hours are on the Breaking change Wiki.
    If you want to know the production traffic statistic, please see ARM Traffic statistic.
    If you think it is false positive breaking change, please provide the reasons in the PR comment, report to Swagger Tooling Team via https://aka.ms/swaggerfeedback.

    @openapi-workflow-bot
    Copy link

    NewApiVersionRequired reason:

    A service’s API is a contract with customers and is represented by using the api-version query parameter. Changes such as adding an optional property to a request/response or introducing a new operation is a change to the service’s contract and therefore requires a new api-version value. This is critically important for documentation, client libraries, and customer support.

    EXAMPLE: if a customer calls a service in the public cloud using api-version=2020-07-27, the new property or operation may exist but if they call the service in a government cloud, air-gapped cloud, or Azure Stack Hub cloud using the same api-version, the property or operation may not exist. Because there is no clear relationship between the service api-version and the new property/operation, customers can’t trust the documentation and Azure customer have difficulty helping customers diagnose issues. In addition, each client library version documents the service version it supports. When an optional property or new operation is added to a service and its Swagger, new client libraries must be produced to expose this functionality to customers. Without updating the api-version, it is unclear to customers which version of a client library supports these new features.

    @skayani
    Copy link
    Contributor Author

    skayani commented Jan 13, 2021

    The 'scope' part of the URI path that was removed is optional for marketplace registration definition operations. Initially I tried replacing the 'scope' parameter with a new 'optionalScope' parameter that was required=false but this lead to semantic validation errors so I believe this is the right approach. Please correct me if I'm wrong.

    @@ -380,17 +380,14 @@
    }
    }
    },
    "/{scope}/providers/Microsoft.ManagedServices/marketplaceRegistrationDefinitions": {
    Copy link
    Contributor

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    suggest to have {scopeOptional} as this API supports tenant and subscription level as well.

    Copy link
    Contributor

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    agree with removing; as I see for operations API also we have removed it.

    @ArcturusZhang
    Copy link
    Member

    Hi @skayani we need to go through the breaking change review because this PR is introducing breaking changes. You could find the guidance in a comment by @openapi-workflow-bot

    @skayani
    Copy link
    Contributor Author

    skayani commented Jan 14, 2021

    Hi @skayani we need to go through the breaking change review because this PR is introducing breaking changes. You could find the guidance in a comment by @openapi-workflow-bot

    I've gone ahead and done that, thanks

    @skayani
    Copy link
    Contributor Author

    skayani commented Jan 21, 2021

    Hi @ArcturusZhang , I wanted to ask this question directly to you as in case I don't get a response from the Swagger tools team alias. Here it is:

    I tried looking for this in the docs but couldn't find a similar scenario. For our service we have these Azure resource types called Registration Definitions (RDs). We allow our users to retrieve RDs for a particular scope or for an entire tenant. This means our service allows for the following GET RDs URIs:

    • /{scope}/providers/Microsoft.ManagedServices/registrationDefinitions
    • /providers/Microsoft.ManagedServices/registrationDefinitions

    Since according to the Swagger docs, we can't have an {optionalScope} parameter (i.e. we cannot have a parameter in the path that is required=false), what is the next best course? Do we specify the above two as two separate Swagger paths (even though they return the same type of result)? Is there a better option?

    @JeffreyRichter JeffreyRichter added the Approved-BreakingChange DO NOT USE! OBSOLETE label. See https://github.com/Azure/azure-sdk-tools/issues/6374 label Jan 25, 2021
    @skayani skayani added the DoNotMerge <valid label in PR review process> use to hold merge after approval label Jan 25, 2021
    @ArcturusZhang
    Copy link
    Member

    Hi @ArcturusZhang , I wanted to ask this question directly to you as in case I don't get a response from the Swagger tools team alias. Here it is:

    I tried looking for this in the docs but couldn't find a similar scenario. For our service we have these Azure resource types called Registration Definitions (RDs). We allow our users to retrieve RDs for a particular scope or for an entire tenant. This means our service allows for the following GET RDs URIs:

    • /{scope}/providers/Microsoft.ManagedServices/registrationDefinitions
    • /providers/Microsoft.ManagedServices/registrationDefinitions

    Since according to the Swagger docs, we can't have an {optionalScope} parameter (i.e. we cannot have a parameter in the path that is required=false), what is the next best course? Do we specify the above two as two separate Swagger paths (even though they return the same type of result)? Is there a better option?

    taking the existing examples, we need to have both of these paths in swagger. We have some similar cases in the policy swaggers. It is not exactly the same but quite similar

    @openapi-workflow-bot
    Copy link

    Hi @skayani, Your PR has some issues. Please fix the CI sequentially by following the order of Avocado, semantic validation, model validation, breaking change, lintDiff.

    TaskHow to fixPrioritySupport (Microsoft alias)
    AvocadoFix-AvocadoHighruowan
    Semantic validationFix-SemanticValidation-ErrorHighraychen, jianyxi
    Model validationFix-ModelValidation-ErrorHighraychen,jianyxi
    LintDiffFix-LintDiffhighjianyxi, ruoxuan
    If you need further help, please feedback via swagger feedback."

    @skayani skayani removed the DoNotMerge <valid label in PR review process> use to hold merge after approval label Jan 28, 2021
    @skayani
    Copy link
    Contributor Author

    skayani commented Jan 29, 2021

    @ArcturusZhang Hey, I've gone ahead and added two paths for our marketplaceRegistrationDefinitions endpoints. I initially received breaking changes approval for removing scope. Will the breaking changes approval still apply to this PR?

    @ArcturusZhang ArcturusZhang merged commit d606d4b into Azure:master Feb 3, 2021
    mkarmark pushed a commit to mkarmark/azure-rest-api-specs that referenced this pull request Jul 21, 2021
    …ure#12397)
    
    * Remove scope from marketplace registration definitions operations
    
    * Swagger fixes
    
    * Changed operation ids
    
    * Validation fixes
    
    * Unique x-ms-example names
    
    * Name fix for examples
    
    * Removed sub from example responses
    
    * Added empty eligibleAuths to 2020-02-01-preview MRD examples
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    Approved-BreakingChange DO NOT USE! OBSOLETE label. See https://github.com/Azure/azure-sdk-tools/issues/6374
    Projects
    None yet
    Development

    Successfully merging this pull request may close these issues.

    4 participants