Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[GraphRuntime] Layerwise timing in debug graph runtime #3232

Merged
merged 1 commit into from
May 23, 2019

Conversation

hlu1
Copy link
Contributor

@hlu1 hlu1 commented May 23, 2019

The debug_run in the debug graph runtime runs each layer individually and only once, which is not the best way to get the layerwise timing. The PR replaces debug_run with the more accurate run_individual (implemented in #2569) in the python debug graph runtime.

Before:

Node Name  Ops    Time(us)  Time(%)  Start Time       End Time         Shape  Inputs  Outputs  
---------  ---    --------  -------  ----------       --------         -----  ------  -------  
add        myadd  4.54      83.5     21:16:40.659529  21:16:40.659548  (4,)   1       1        
Node Name  Ops    Time(us)  Time(%)  Start Time       End Time         Shape  Inputs  Outputs  
---------  ---    --------  -------  ----------       --------         -----  ------  -------  
add        myadd  1.15      78.68    21:16:40.794619  21:16:40.794770  (4,)   1       1        

After:

Node Name  Ops    Time(us)  Time(%)  Shape  Inputs  Outputs  
---------  ---    --------  -------  -----  ------  -------  
add        myadd  0.071     100.0    (4,)   1       1        
Node Name  Ops    Time(us)  Time(%)  Shape  Inputs  Outputs  
---------  ---    --------  -------  -----  ------  -------  
add        myadd  0.06      100.0    (4,)   1       1        

Note myadd actually only takes about 0.06 - 0.07 us to run, if you warm the cache up first. In the case of a full model, the right way to get layerwise timing is to let the model run continuously with minimal interruptions for a fixed amount of time, and then take the average to get the runtime of individual layers.

@icemelon9 and @antinucleon please review

@hlu1 hlu1 force-pushed the debug_runtime branch from 0a126bd to 94ce8e1 Compare May 23, 2019 04:40
@tqchen tqchen merged commit e1e91f1 into apache:master May 23, 2019
@tqchen
Copy link
Member

tqchen commented May 23, 2019

Thanks, @hlu1 ! this is now merged

hlu1 added a commit to hlu1/tvm that referenced this pull request May 24, 2019
@hlu1 hlu1 deleted the debug_runtime branch May 31, 2019 21:19
wweic pushed a commit to wweic/tvm that referenced this pull request Jun 26, 2019
wweic pushed a commit to neo-ai/tvm that referenced this pull request Jun 27, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants