SDK Automation Tooling's Enhancements

[raych1]

Overview
This issue is created to track all the work items related to the Enhancements-to-SDK-Automation specification.

1. Support Scenarios

• TypeSpec green field services.
  This is currently supported by all four languages (.NET, Java, Python, JS).
  Add the support for Go language.
• TypeSpec brown field services.
  Generation strategy to handle the existing customized SDK code:
  • option 1: keep the existing customized SDK code and generate the SDK code based on TypeSpec specs.
    If the customized SDK code is not compatible with the new specs, the generation should fail in the build/compile step providing the comprehensive error message, such as "The SDK has customized SDK code and might not be compatible with the new specs, the service team may need to update the customized code or the new specs."
  • option 2: remove the existing customized code then generate the SDK code from scratch based on the TypeSpec specs
• Service group with a mix of service using TypeSpec and service using the Swagger.
  The prerequisite is each service must have individual SDK package.

Tasks

Preview Give feedback

1. [FEATURE REQ] Support SDK generation in CI pipeline for the services migrated from Swagger to TypeSpec azure-sdk-for-net#41746
   DPG EngSys +2
   
2. [Automation Script Bug] Use 'service.csproj' to build generated SDK projects rather than using '*.sln' file azure-sdk-for-net#42159
   Client EngSys +2
   
3. [SDK Automation - Java] Support SDK generation in CI pipeline for the services migrated from Swagger to TypeSpec #8055
   Automation Tool +1
   
4. [SDK Automation - Python] Support SDK generation in CI pipeline for the services migrated from Swagger to TypeSpec #8056
   Automation Tool +1
   
5. [SDK Automation - JS] Support SDK generation in CI pipeline for the services migrated from Swagger to TypeSpec #8057
   Automation Tool +1
   
6. [SDK Automation - Go] Support the release from TypeSpec #8308
   Automation Tool +1
   
7. [SDK Automation - Go] Support the SDK generation for services using TypeSpec in the spec PR pipeline #8309
   Automation Tool +1
   
8. [SDK-Automation] Service with swagger spec is skipped from SDK generation in spec PR #8712
   Automation Tool +1
   

2. Define the Required Steps
There are a bunch of steps to be executed during the SDK generation process. Take .NET for example, it includes the following steps:

• Environment setup
• SDK generation
• SDK code building, which include test/sample code building
• SDK code packing
• API export
• Breaking change validation
• API View generating

Any failure of the above steps would cause the SDK generation CI check fail. We will need to identify the required steps to run in the spec pull request phase for each SDK language. It's important to note that the required steps may vary across languages, as the requirement could differ from one language to another.

Tasks

Preview Give feedback

1. [SDK Automation - Java] Adjust the steps to run in the automation script #8040
   Automation Tool +1
   
2. [SDK Automation - Python] Adjust the steps to run in the automation script Automation Tool #8041
   Automation Tool +1
   
3. [SDK Automation - JS] Adjust the steps to run in the automation script Automation Tool #8042
   Automation Tool +1
   
4. [SDK Automation - .NET] Adjust the steps to run in the automation script Automation Tool #8043
   Automation Tool +1
   
5. [SDK Automation - Go] Adjust the steps to run in the automation script Automation Tool #8044
   Automation Tool +1
   
6. [SDK Automation - .NET] Test code build failure should not impact the API view generation #8059
   Automation Tool +1
   
7. [SDK Automation - .NET] Support the release from TypeSpec in MPG #8417
   Automation Tool +1
   
8. [SDK Automation - Python] Support the release from TypeSpec in MPG #8416
   Automation Tool +1
   
9. [SDK Automation - Java] Support the release from TypeSpec for MPG #8415
   Automation Tool +1
   

3. Breaking Change Detection
The automation should be able to detect the breaking changes against SDK API level.

Tasks

Preview Give feedback

1. [SDK Automation - Python] Enable breaking change detection for DPG & MPG #8045
   Automation Tool +1
   
2. [SDK Automation - JS] Support and Enable Breaking Change Detection for Modular Client #8046
   5 of 5
   Automation Tool P0 +2
   
3. [SDK Automation - Java] Support breaking change detection for DPG & MPG #8477
   Automation Tool +1
   

4. tsp-client adoption
Adopt tsp-client to replace the common powershell script to interact with tsp-config.yaml, fetch the TypeSpec specs, and run TypeSpec language emitters.
This could improve the logging experience for the steps tsp-client tool covers.

Tasks

Preview Give feedback

1. [SDK Automation - Python] Adopt tsp-client in automation script #8047
   Automation Tool +1
   
2. [SDK Automation - JS] Adopt tsp-client in automation script #8048
   SDK Automation +1
   
3. [SDK Automation - .NET] Adopt tsp-client in automation script #8049
   1 of 1
   Automation Tool +1
   
4. [SDK Automation - Java] Adopt tsp-client in automation script #8050
   Automation Tool +1
   

5. Logging Improvement
Refine the error logs to enhance clarity and conciseness, provide guidance on corrective actions, and include relevant document links as necessary. The logging should include three parts information if possible.

• error code or error message
• next steps or suggested actions
• associated document links if have

Tasks

Preview Give feedback

1. [SDK Automation - JS] Build error reported without clear output #8058
   Automation Tool +1
   
2. [SDK Automation] Log the output of the 'tsp compile' command to the pipeline #7958
   P0 SDK Automation +2
   
3. [SDK Automation - Java] Add validation for both 'package-dir' and 'namespace' configurations in 'tspconfig.yaml' #8095
   Automation Tool +1
   
4. [SDK Automation - Java] Improve the logging to be concise and actionable #8096
   Automation Tool +1
   
5. [SDK Automation - Python] Improve the logging to be concise and actionable #8097
   Automation Tool +1
   
6. [SDK Automation - JS] Improve the logging to be concise and actionable #8098
   Automation Tool P0 +2
   
7. [SDK Automation - .NET] Improve the logging to be concise and actionable #8099
   Automation Tool +1
   
8. [SDK Automation - Go] Improve the logging to be concise and actionable #8100
   Automation Tool +1
   
9. [SDK Automation] Logging improvement for CI framework part #8122
   Automation Tool P0 +2
   
10. [SDK Automation - Go] Log the release tags array null reference error gracefully #8250
    Automation Tool +1
    
11. [SDK Automation - .NET] provide how-to-fix suggestion when fail #8495
    Automation Tool Spec PR Tools +2
    
12. [SDK-Automation] The filtered logs are printed twice in the CI check result #8688
    Automation Tool +1
    
13. [Go-SDK-Automation] Error should be reported when the readme.go.md has incorrect format #8658
    Automation Tool +1
14. [SDK-Automation] The warnings entry is rendered multiple times in the PR CI check result #8649
    Automation Tool +1
    
15. [DotNet-Automation] Exception should be handled gracefully when the artifact doesn't exist in debug folder #8563
    Automation Tool +1
    
16. [Go SDK Automation] Remove the duplicated output from the logs #8722
    Automation Tool +1
    

6. Bugs identified

Tasks

Preview Give feedback

1. Pipeline result fails to be published #7631
   Automation Tool SDK Automation +2
   
2. Add/modify title and description of autogenerated language PRs #7760
   Central-EngSys P1 SDK Automation Spec PR Tools +4
   
3. Provide suppression mechanism to skip analyzer errors #8013
   Automation Tool +1
   
4. [SDK azure-sdk-for-js] Step PublishTestResults fails with OOM #7458
   P0 Spec PR Tools +2
   
5. JS automation tool should generate code in generated folder if having customization #7950
   Automation Tool +1
   
6. [SDK Automation] Python script fails with 'ModuleNotFoundError' when generate the SDK #7957
   Automation Tool P0 +2
   
7. [SDK Automation] JS fails to build the generated SDK project #7968
   Automation Tool +1
   

   
8. [SDK Automation] JS automation script doesn't need to install 'node' repeatedly #7967
   Automation Tool +1
   
9. [SDK Automation] Remove ANSI code from displaying in the SDK automation result #8087
   Automation Tool +1
   
10. [SDK Automation] Java automation script doesn't need to install node 18 repeatedly #7966
    Automation Tool +1
    
11. [SDK Automation - Python] Automation script doesn't need to install node 18 repeatedly #8101
    Automation Tool +1
    
12. [SDK Automation] Beta version is flagged incorrectly for JS SDK #8218
    Automation Tool SDK Automation +2
    
13. [SDK Automation] The check SDK $(sdkName) is stuck in progress when AutoRest config (README.md) does not define any language SDKs #7235
    Central-EngSys SDK Automation Spec PR Tools bug +4
    
14. TypeError is thrown in SDK automation code when the SDK generation config is not in place #8360
    SDK Automation +1
    
15. [SDK Automation] Invalid branch error is thrown after SDK generation #8380
    SDK Automation +1
    
16. [SDK Automation] Breaking change should be re-evaluated on each run #8381
    SDK Automation +1
    
17. Python SDK generation is not triggered and status is not updated #8603
    Automation Tool SDK Automation +2
    
18. Compile failure caused by duplicated definitions created under same namespace autorest.csharp#4946
    Mgmt v3 +2
    

7. Documentation
Provide documentation of error definitions and suggestions for the fixes for each language and the common SDK automation framework.

8. Additional Tasks

Tasks

Preview Give feedback

1. [SDK Automation] Deprecate the call to openapi portal #8111
   Automation Tool P1 +2
   
2. [Spring Grove] Use workload identity to access Azure blob in SDK automation #8353
   SDK Automation +1
   
3. [SDK-Automation] Pipeline job result in pipeline view should align with the result reported in the spec PR #8701
   Automation Tool +1
   
4. [SDK-Automation] All the SDK artifacts and APIViews should be available even if one package fails to generate in the spec PR #8710
   Automation Tool +1
   
5. [SDK Automation] Support generated SDK artifacts downloading in spec PR pipeline #8418
   Automation Tool +1
   

[raych1]

All of the linked tasks have been completed except for the one to add how-to-fix suggestions for .NET. I will close this EPIC and track that in API to SDK automation board separately.