Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

Use cuDNN for conv bias and bias grad #20771

Merged
merged 5 commits into from
Dec 17, 2021
Merged

Use cuDNN for conv bias and bias grad #20771

merged 5 commits into from
Dec 17, 2021

Conversation

mk-61
Copy link
Contributor

@mk-61 mk-61 commented Dec 7, 2021 •
edited
Loading

Description

This change uses cuDNN implementation of bias and bias gradient, if available. It mitigates performance regression in convolutional networks in 2.0 vs 1.x, as tested with ResNet training script.

Checklist

Essentials

  • Changes are complete (i.e. I finished coding on this PR)
  • All changes have test coverage
  • Code is well-documented

Other changes

  • Fix AMP for ndarray.numpy

@mxnet-bot
Copy link

Hey @mk-61 , Thanks for submitting the PR
All tests are already queued to run once. If tests fail, you can trigger one or more tests again with the following commands:

  • To trigger all jobs: @mxnet-bot run ci [all]
  • To trigger specific jobs: @mxnet-bot run ci [job1, job2]

CI supported jobs: [clang, centos-cpu, unix-cpu, windows-gpu, windows-cpu, miscellaneous, unix-gpu, website, sanity, centos-gpu, edge]

Note:
Only following 3 categories can trigger CI :PR Author, MXNet Committer, Jenkins Admin.
All CI tests must pass before the PR can be merged.

@mseth10 mseth10 added the pr-work-in-progress PR is still work in progress label Dec 7, 2021
@mk-61 mk-61 force-pushed the pr-fix-perf-vs-1x branch 2 times, most recently from 464a07e to ff7e5a1 Compare December 8, 2021 22:53
@mk-61 mk-61 force-pushed the pr-fix-perf-vs-1x branch 2 times, most recently from de9b39a to 0ac4a0b Compare December 9, 2021 21:58
@mk-61 mk-61 requested a review from szha as a code owner December 10, 2021 00:20
@mk-61 mk-61 changed the title [WIP] Use cuDNN for conv bias and bias grad Use cuDNN for conv bias and bias grad Dec 10, 2021
@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-awaiting-review PR is waiting for code review pr-work-in-progress PR is still work in progress and removed pr-work-in-progress PR is still work in progress pr-awaiting-review PR is waiting for code review pr-awaiting-testing PR is reviewed and waiting CI build and test labels Dec 10, 2021
@mk-61
Copy link
Contributor Author

mk-61 commented Dec 10, 2021

@mxnet-bot run ci [unix-cpu, windows-gpu]

@mxnet-bot
Copy link

Jenkins CI successfully triggered : [unix-cpu, windows-gpu]

@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress and removed pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test labels Dec 10, 2021
@mk-61
Copy link
Contributor Author

mk-61 commented Dec 10, 2021

@mxnet-bot run ci [windows-gpu]

@mxnet-bot
Copy link

Jenkins CI successfully triggered : [windows-gpu]

@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress and removed pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test labels Dec 10, 2021
@mk-61
Copy link
Contributor Author

mk-61 commented Dec 10, 2021

@mxnet-bot run ci [windows-gpu]

@mxnet-bot
Copy link

Jenkins CI successfully triggered : [windows-gpu]

@mseth10 mseth10 added pr-work-in-progress PR is still work in progress and removed pr-awaiting-testing PR is reviewed and waiting CI build and test labels Dec 10, 2021
@mk-61
Copy link
Contributor Author

mk-61 commented Dec 10, 2021

@mxnet-bot run ci [unix-gpu]

@mxnet-bot
Copy link

Jenkins CI successfully triggered : [unix-gpu]

@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress and removed pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test labels Dec 10, 2021
@mk-61
Copy link
Contributor Author

mk-61 commented Dec 11, 2021

@mxnet-bot run ci [unix-cpu]

@mxnet-bot
Copy link

Jenkins CI successfully triggered : [unix-cpu]

@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress and removed pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test labels Dec 11, 2021
@mk-61
Copy link
Contributor Author

mk-61 commented Dec 11, 2021

@mxnet-bot run ci [unix-cpu]

@mxnet-bot
Copy link

Jenkins CI successfully triggered : [unix-cpu]

@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-work-in-progress PR is still work in progress and removed pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test labels Dec 11, 2021
@mk-61
Copy link
Contributor Author

mk-61 commented Dec 11, 2021

@mxnet-bot run ci [unix-cpu]

@mxnet-bot
Copy link

Jenkins CI successfully triggered : [unix-cpu]

@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test pr-awaiting-review PR is waiting for code review and removed pr-work-in-progress PR is still work in progress pr-awaiting-testing PR is reviewed and waiting CI build and test labels Dec 11, 2021
@TristonC
Copy link
Contributor

@szha please help review.

@TristonC
Copy link
Contributor

@josephevans Could you help to check these CI failure? It seems not from the MXNet.

ptrendx
ptrendx approved these changes Dec 17, 2021
View reviewed changes
Copy link
Member

@ptrendx ptrendx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks!

@ptrendx ptrendx merged commit f0ef9d8 into apache:master Dec 17, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@ptrendx ptrendx ptrendx approved these changes

@szha szha Awaiting requested review from szha szha is a code owner

Assignees
No one assigned
Labels
pr-awaiting-review PR is waiting for code review
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@mk-61 @mxnet-bot @TristonC @ptrendx @mseth10