Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update npmauth to handle retries cleanly #15914

Merged
merged 7 commits into from
Feb 18, 2022
Merged

Conversation

mastrzyz
Copy link
Contributor

@mastrzyz mastrzyz commented Feb 11, 2022

Task name: NPMAuthenticate

Description:

The NPM Authenticate task leverages live services and can be very flaky for us , so we used the retryOnTaskFailure setting in the main task to try and mitigate this.

We encounter the following error when we have a retry :

Adding authentication to the .npmrc file at D:\a_work\3\s*********.npmrc
##[debug]SAVE_NPMRC_PATH=D:\a_work\3\npmAuthenticate\FyuhLH
##[debug]SAVE_NPMRC_PATH=D:\a_work\3\npmAuthenticate\FyuhLH
##[debug]NPM_AUTHENTICATE_TEMP_DIRECTORY=D:\a_work\3\npmAuthenticate
##[debug]NPM_AUTHENTICATE_TEMP_DIRECTORY=D:\a_work\3\npmAuthenticate
##[debug]rm -rf D:\a_work\3\npmAuthenticate
##[debug]task result: Failed
##[error]Error: ENOENT: no such file or directory, open 'D:\a_work\3\npmAuthenticate\FyuhLH\index.json'
##[debug]Processed: ##vso[task.issue type=error;]Error: ENOENT: no such file or directory, open 'D:\a_work\3\npmAuthenticate\FyuhLH\index.json'
##[debug]Processed: ##vso[task.complete result=Failed;]Error: ENOENT: no such file or directory, open 'D:\a_work\3\npmAuthenticate\FyuhLH\index.json'
##[warning]RetryHelper encountered task failure, will retry (attempt #: 2 out of 4) after 4000 ms
This looks to be due to the fact that after the first attempt to authenticate we timed out and the following catch code is run :

main().catch(error => {
if(tl.getVariable("NPM_AUTHENTICATE_TEMP_DIRECTORY")) {
tl.rmRF(tl.getVariable("NPM_AUTHENTICATE_TEMP_DIRECTORY"));
}
tl.setResult(tl.TaskResult.Failed, error);
});
This wipes the directory NPM_AUTHENTICATE_TEMP_DIRECTORY but not the variables.

thus on retry the following code gets executed :

if (tl.getVariable("SAVE_NPMRC_PATH")) {
     saveNpmrcPath = tl.getVariable("SAVE_NPMRC_PATH");
}
else {
    let tempPath = tl.getVariable('Agent.BuildDirectory') || tl.getVariable('Agent.TempDirectory');
    tempPath = path.join(tempPath, 'npmAuthenticate');
    tl.mkdirP(tempPath);
    saveNpmrcPath = fs.mkdtempSync(tempPath + path.sep); 
    tl.setVariable("SAVE_NPMRC_PATH", saveNpmrcPath, false);
    tl.setVariable("NPM_AUTHENTICATE_TEMP_DIRECTORY", tempPath, false);
}

And since SAVE_NPMRC_PATH was previously set, the code think the directory should be there, but it is not.

Simple solution would be to wipe this variable alongside wiping the root directory.

main().catch(error => {
if(tl.getVariable("NPM_AUTHENTICATE_TEMP_DIRECTORY")) {
tl.rmRF(tl.getVariable("NPM_AUTHENTICATE_TEMP_DIRECTORY"));
// Clear the variales after we rm-rf the main root directory
tl.setVariable("SAVE_NPMRC_PATH", "", false);
tl.setVariable("NPM_AUTHENTICATE_TEMP_DIRECTORY", "", false);
}
tl.setResult(tl.TaskResult.Failed, error);
});

Does that seem good? am I missing something important here.

Documentation changes required: (Y/N) N

Added unit tests: (Y/N) -> Current task has no UT's // TODO - add real tests

Attached related issue: (Y/N) #15913

Checklist:

  • Task version was bumped - please check instruction how to do it
  • Checked that applied changes work as expected -> Would appreciate some guidance here, how would I try this out in our repository?

I tried the following code to test out my changes :

Commented out my 'fix' and called main immediately after the failure to fake a retry attempt

main().catch(error => {
    if (tl.getVariable("NPM_AUTHENTICATE_TEMP_DIRECTORY")) {
        tl.rmRF(tl.getVariable("NPM_AUTHENTICATE_TEMP_DIRECTORY"));
        // Clear the variales after we rm-rf the main root directory
      //  tl.setVariable("SAVE_NPMRC_PATH", "", false);
      //  tl.setVariable("NPM_AUTHENTICATE_TEMP_DIRECTORY", "", false);
    }
    main()
    tl.setResult(tl.TaskResult.Failed, error);
});

Result ->
image

Uncommented my 'fix'

main().catch(error => {
if (tl.getVariable("NPM_AUTHENTICATE_TEMP_DIRECTORY")) {
tl.rmRF(tl.getVariable("NPM_AUTHENTICATE_TEMP_DIRECTORY"));
// Clear the variales after we rm-rf the main root directory
tl.setVariable("SAVE_NPMRC_PATH", "", false);
tl.setVariable("NPM_AUTHENTICATE_TEMP_DIRECTORY", "", false);
}
main()
tl.setResult(tl.TaskResult.Failed, error);
});

No more error!

@mastrzyz mastrzyz marked this pull request as ready for review February 11, 2022 19:41
@mastrzyz mastrzyz changed the title Update npmauth.ts Update npmauth to handle retries cleanly Feb 11, 2022
@mastrzyz
Copy link
Contributor Author

@aasim provided an update with how I tested this, let me know if it seems good.

@zhenghao104 zhenghao104 self-requested a review February 17, 2022 21:25
@zhenghao104
Copy link
Contributor

/azp run

@azure-pipelines
Copy link

Azure Pipelines successfully started running 4 pipeline(s).

@zhenghao104 zhenghao104 merged commit 2983272 into microsoft:master Feb 18, 2022
zhenghao104 pushed a commit that referenced this pull request Feb 19, 2022
* Update npmauth.ts

* Update task.json

* Update task.loc.json

* Update npmauth.ts

* RED

Co-authored-by: Your Name <[email protected]>
zhenghao104 added a commit that referenced this pull request Feb 22, 2022
* Update npmauth.ts

* Update task.json

* Update task.loc.json

* Update npmauth.ts

* RED

Co-authored-by: Your Name <[email protected]>

Co-authored-by: Marcin Strzyz <[email protected]>
Co-authored-by: Your Name <[email protected]>
aasim pushed a commit that referenced this pull request Mar 8, 2022
* Update npmauth.ts

* Update task.json

* Update task.loc.json

* Update npmauth.ts

* RED

Co-authored-by: Your Name <[email protected]>
aasim added a commit that referenced this pull request Mar 8, 2022
* Fixed casing for packaging api data

* Updated task version

* [Maven tasks] Bump the codecoverage-tools package at the dependencies (#15928)

* Update cc-tools package for MavenV2 task

* Update cc-tools package for MaveV3 task

* Disabled percent encoding (#15931)

* Update npmauth to handle retries cleanly (#15914)

* Update npmauth.ts

* Update task.json

* Update task.loc.json

* Update npmauth.ts

* RED

Co-authored-by: Your Name <[email protected]>

* Updated Helm download Url (#15821)

* updated helm download url

* bumped task vesrions required for this change

* bumped task versions

* [FtpUploadV2] Resolve dependabot alert about shelljs (#15881)

* fix dependencies

* fix build

build error are caused by tasklib 3.x return types change from "<type>" to "<type> | undefined"
functionally, these method calls should be identical

* bump task version

* Add simple unit tests to check getting input parameters and start of task

* Delete debug

* Fix EOL

* Updated task version to 201

* Added test for successful execution

Co-authored-by: Tatyana Kostromskaya (Akvelon) <[email protected]>
Co-authored-by: Tatyana Kostromskaya <[email protected]>
Co-authored-by: Denis Tikhomirov <[email protected]>
Co-authored-by: Denis Tikhomirov <[email protected]>
Co-authored-by: Anatoly Bolshakov <[email protected]>

* Fix README.md section on installing Invoke-SqlCommand on an agent (#15693)

* Add another way of installing Invoke-SqlCommand.
* Fix dead link.

Co-authored-by: Kim Jämiä <[email protected]>
Co-authored-by: Philipson Joseph V <[email protected]>

* updating  the azure-pipelines-tasks-azure-arm-rest-v2 to 2.198.2  (#15935)

* version update of common module

* version update

Co-authored-by: rajnish-byte <[email protected]>

* Fix shelljs vulnerability for tasks (#15972)

* bump shelljs for ArchiveFilesV2

* bump shelljs for BashV3

* bump shelljs for CMakeV1

* bump shelljs for CocoaPodsV0

* bump affected tasks versions

* 15646 - Machine Names required", but should not be required, and machine names being prepended to destination folder #15646 (#15870)

* Making Machines,AdminLogin,Password as Mandatory

* Making Machines,Admin Login and Password required fields

* fix no wait result bug (#15957)

* fix no wait result bug

* bump version

Co-authored-by: Anatoly Bolshakov <[email protected]>
Co-authored-by: Nikita Ezzhev <[email protected]>

* [UsePythonV0] Download python from registry (#15820)

* download python from registry if absent in cache

* add unit test

* fixes & debug messages

* fix manifest branch name

* fix debug message for python archive file name

* remove unneeded function

* fix unit tests failing on unix

* only check os version once

* add unit test for ubuntu version lookup

* adjust error messages

* add input to allow unstable versions

* fix ubuntu test

* add test for unstable version download

* add comments

* adjust comments

* select arch based on input

* extract interfaces into separate file

* make allowUnstable optional

* add a separate debug message for downloading

* explain what the registry is more clearly

* bump task version due to a new sprint

* fix allowUnstable default value

* format osutil to unify code style

* Updated version

Co-authored-by: Aasim Malladi <[email protected]>
Co-authored-by: Konstantin Tyukalov <[email protected]>
Co-authored-by: Anatoly Bolshakov <[email protected]>
Co-authored-by: Marcin Strzyz <[email protected]>
Co-authored-by: Your Name <[email protected]>
Co-authored-by: rajnish-byte <[email protected]>
Co-authored-by: Daniil Shmelev <[email protected]>
Co-authored-by: Tatyana Kostromskaya (Akvelon) <[email protected]>
Co-authored-by: Tatyana Kostromskaya <[email protected]>
Co-authored-by: Denis Tikhomirov <[email protected]>
Co-authored-by: Denis Tikhomirov <[email protected]>
Co-authored-by: Kim Jämiä <[email protected]>
Co-authored-by: Kim Jämiä <[email protected]>
Co-authored-by: Philipson Joseph V <[email protected]>
Co-authored-by: v-hinti <[email protected]>
Co-authored-by: v-nagarajku <[email protected]>
Co-authored-by: Ruoyu Wang <[email protected]>
Co-authored-by: Nikita Ezzhev <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants