-
Notifications
You must be signed in to change notification settings - Fork 95
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[DOC] multi-echo reports #457
Merged
Merged
Changes from all commits
Commits
Show all changes
61 commits
Select commit
Hold shift + click to select a range
58a2c32
Added html template for report
b61fb1e
doc: initial commit of reporting template
emdupre 34c9133
doc: save out html for testing
emdupre fcaf13b
fix: add missing bokeh js script
emdupre 942a469
fix: unescape HTML ugly way
emdupre b7262ab
fix: add missing cdn's
emdupre fbb2659
enh: switch to generate_report structure
emdupre 31bf94f
sty: switch back to pure CSS from bootstrap
emdupre f0db0f5
fix: differently update template for content
emdupre 1901196
sty: add jupyter notebooks to git ignore
emdupre 2bffccf
enh:initial commit of dynamic plots
javiergcas 1627b10
fix: patch unpaired bracket
emdupre 89fd655
Merge remote-tracking branch 'emdupre/doc/me-reports' into dynamic_re…
javiergcas 8bf5310
enh:modularized dynamic kappa/rho plots
javiergcas 0b44fa0
enh:docstrings added to KappaRho_DynPlot.py
javiergcas 70bbe10
Force figures default, fold viz into reporting module
emdupre d834b30
enh:TS and Spectrum are now dynamic plots
javiergcas 27f7ac8
Merge pull request #20 from javiergcas/dynamic_reports
emdupre ebea3c9
sty: initial refactor of dynamic figures
emdupre 5ff0d3d
fix: re-introduce func to load comp_table
emdupre ed44834
sty: finalize rough refactor
emdupre 7302e1e
fix: patch linting errors
emdupre d82e284
Merge remote-tracking branch 'upstream/master' into doc/me-reports
emdupre ef318b6
doc: continue refactor
emdupre 632900b
Merge branch 'master' of https://github.com/me-ica/tedana into doc/me…
eurunuela 96a896f
Advanced on dynamic_figures
eurunuela d1a417b
Made the refactored code work
eurunuela b9b9117
fix: new year, new approach
emdupre 636d9ed
tmp: non-functioning reporting code
emdupre 8986fab
Merge remote-tracking branch 'refs/remotes/origin/doc/me-reports' int…
emdupre bc668d0
Merge remote-tracking branch 'upstream/master' into doc/me-reports
emdupre 748c9a9
Merge remote-tracking branch 'upstream/master' into doc/me-reports
emdupre 8192fb2
Fixes bug
eurunuela d7c3745
Fixes bug
eurunuela ed3500a
Added new function to generate html report and removed tempita calls
eurunuela 3ce5d4d
Added FFT and time series plots
eurunuela b66d097
Made template subsitute work and updated necessary bokeh version
eurunuela 8a8fe7e
Changes html title
eurunuela b48aa7f
remove time series, fft plots
emdupre 55e5900
fix: remove unescapes
emdupre 2fd4683
fix: patch incorrect imports
emdupre 46c3eb4
Merge remote-tracking branch 'upstream/master' into doc/me-reports
emdupre 53b2248
fix: add missing bokeh import
emdupre de61e24
fix: spelling mistake 🤦
emdupre 4f8be84
fix: reflect reorg in get_coeffs call
emdupre 797f735
fix: correct expected test outputs
emdupre 133b7e2
fix: don't designate file path when saving report
emdupre 464bef0
fix: combine generate, html report functions
emdupre b66d578
fix: ignore missing columns in df.drop, to better handle multiple params
emdupre a8f9900
Dynamic figures are now two rows, static figure is on the right. Remo…
eurunuela 131dc82
Added new line after References in report.txt
eurunuela 1a4a174
Removes BeautifulSoup dependency in favor of an object tag and change…
eurunuela 3f162a5
Merge pull request #23 from eurunuela/enh/dynamic_reports
emdupre 72cd396
enh: switch to grid layout, style About section, add links to navbar
emdupre 1363c01
doc: update no-png to no-reports
emdupre 2b5a94b
modifiation of setup.py to properly install all reporting directories
javiergcas ba68bfd
Merge pull request #24 from javiergcas/doc/me-reports
emdupre 05e91a4
Use relative path for static figures on report
eurunuela 663f5eb
Merge pull request #25 from eurunuela/doc/me-reports
emdupre 1b517b6
Add version check for bokeh figures
emdupre 4817b8a
Merge branch 'doc/me-reports' of https://github.com/emdupre/tedana in…
emdupre File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -104,4 +104,8 @@ ENV/ | |
.mypy_cache/ | ||
|
||
# vscode | ||
.vscode | ||
.vscode | ||
|
||
# jupyter notebooks | ||
.ipynb_checkpoints/ | ||
*.ipynb |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -7,3 +7,4 @@ pandas | |
scikit-learn>=0.22 | ||
scipy>=1.3.3 | ||
threadpoolctl | ||
bokeh | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,8 @@ | ||
""" | ||
Reporting code for tedana | ||
""" | ||
|
||
from .html_report import generate_report | ||
from .static_figures import comp_figures | ||
|
||
__all__ = ['generate_report', 'comp_figures'] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
This directory contains data required for tedana reporting. | ||
|
||
html/ : HTML templates |
Empty file.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,25 @@ | ||
<style type="text/css"> | ||
.content { | ||
padding: 35px 20px 10px 20px; | ||
margin-left: 50px; | ||
margin-top: 50px; | ||
height: 900px; | ||
} | ||
.about { | ||
position: relative; | ||
top: -50px; | ||
margin-left: 5%; | ||
margin-right: 5%; | ||
white-space: pre-line; | ||
font-family: Arial, Helvetica, sans-serif; | ||
font-size: larger; | ||
} | ||
</style> | ||
<div class="content"> | ||
$content | ||
</div> | ||
<div class="about"> | ||
$about | ||
</div> | ||
|
||
$javascript |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,38 @@ | ||
<!DOCTYPE html> | ||
<html lang="en"> | ||
|
||
<head> | ||
<title>tedana report</title> | ||
<meta name="viewport" content="width = device-width, initial-scale = 1" charset="UTF-8"> | ||
<script src="https://cdn.bokeh.org/bokeh/release/bokeh-2.0.1.min.js" | ||
crossorigin="anonymous"></script> | ||
<script src="https://cdn.bokeh.org/bokeh/release/bokeh-widgets-2.0.1.min.js" | ||
crossorigin="anonymous"></script> | ||
<script src="https://cdn.bokeh.org/bokeh/release/bokeh-tables-2.0.1.min.js" | ||
crossorigin="anonymous"></script> | ||
<link rel="stylesheet" href="https://unpkg.com/[email protected]/build/pure-min.css" | ||
integrity="sha384-oAOxQR6DkCoMliIh8yFnu25d7Eq/PHS21PClpwjOTeU2jRSq11vu66rf90/cZr47" crossorigin="anonymous"> | ||
<!-- CSS for the report --> | ||
<style type="text/css"> | ||
body { | ||
padding: 10px 20px 10px 20px; | ||
} | ||
</style> | ||
|
||
</head> | ||
|
||
<nav> | ||
<div class="pure-menu pure-menu-horizontal"> | ||
<a href="https://github.com/ME-ICA/tedana" class="pure-menu-heading pure-menu-link">tedana $version</a> | ||
<ul class="pure-menu-list"> | ||
<li class="pure-menu-item"> | ||
<a href="https://tedana.readthedocs.io" class="pure-menu-link">Reporting documentation</a> | ||
</li> | ||
</ul> | ||
</div> | ||
</nav> | ||
|
||
<body> | ||
$body | ||
</body> | ||
</html> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,284 @@ | ||
import numpy as np | ||
from math import pi | ||
import pandas as pd | ||
from sklearn.preprocessing import MinMaxScaler | ||
from bokeh import (events, models, plotting, transform) | ||
|
||
color_mapping = {'accepted': '#2ecc71', | ||
'rejected': '#e74c3c', | ||
'ignored': '#3498db'} | ||
|
||
tap_callback_jscode = """ | ||
// Accessing the selected component ID | ||
var data = source_comp_table.data; | ||
var selected_idx = source_comp_table.selected.indices; | ||
if(selected_idx > 0) { | ||
// A component has been selected | ||
// ----------------------------- | ||
var components = data['component'] | ||
var selected = components[selected_idx] | ||
var selected_padded = '' + selected; | ||
while (selected_padded.length < 2) { | ||
selected_padded = '0' + selected_padded; | ||
} | ||
var selected_padded_forIMG = '0' + selected_padded | ||
var selected_padded_C = 'ica_' + selected_padded | ||
|
||
// Find color for selected component | ||
var colors = data['color'] | ||
var this_component_color = colors[selected_idx] | ||
|
||
// Image Below Plots | ||
div.text = "" | ||
var line = "<span><img src='./figures/comp_"+selected_padded_forIMG+".png'" + | ||
" alt='Component Map'><span>\\n"; | ||
console.log('Linea: ' + line) | ||
var text = div.text.concat(line); | ||
var lines = text.split("\\n") | ||
if (lines.length > 35) | ||
lines.shift(); | ||
div.text = lines.join("\\n"); | ||
|
||
} else { | ||
// No component has been selected | ||
// ------------------------------ | ||
// Set Component color to Black | ||
var this_component_color = '#000000' | ||
|
||
// Image Below Plots | ||
div.text = "" | ||
var line = "<p>Please select an individual component to view it in more detail</p>\\n" | ||
var text = div.text.concat(line); | ||
|
||
} | ||
""" | ||
|
||
|
||
def _create_data_struct(comptable_path, color_mapping=color_mapping): | ||
""" | ||
Create Bokeh ColumnDataSource with all info dynamic plots need | ||
|
||
Parameters | ||
---------- | ||
comptable: str | ||
file path to component table, JSON format | ||
|
||
Returns | ||
------- | ||
cds: bokeh.models.ColumnDataSource | ||
Data structure with all the fields to plot or hover over | ||
""" | ||
unused_cols = ['normalized variance explained', | ||
'countsigFR2', 'countsigFS0', | ||
'dice_FS0', 'countnoise', 'dice_FR2', | ||
'signal-noise_t', 'signal-noise_p', | ||
'd_table_score', 'kappa ratio', | ||
'rationale', 'd_table_score_scrub'] | ||
|
||
df = pd.read_json(comptable_path) | ||
df.drop('Description', axis=0, inplace=True) | ||
df.drop('Method', axis=1, inplace=True) | ||
df = df.T | ||
n_comps = df.shape[0] | ||
|
||
# remove space from column name | ||
df.rename(columns={'variance explained': 'var_exp'}, inplace=True) | ||
|
||
# For providing sizes based on Var Explained that are visible | ||
mm_scaler = MinMaxScaler(feature_range=(4, 20)) | ||
df['var_exp_size'] = mm_scaler.fit_transform( | ||
df[['var_exp', 'normalized variance explained']])[:, 0] | ||
|
||
# Calculate Kappa and Rho ranks | ||
df['rho_rank'] = df['rho'].rank(ascending=False).values | ||
df['kappa_rank'] = df['kappa'].rank(ascending=False).values | ||
df['var_exp_rank'] = df['var_exp'].rank(ascending=False).values | ||
|
||
# Remove unused columns to decrease size of final HTML | ||
# set errors to 'ignore' in case some columns do not exist in | ||
# a given data frame | ||
df.drop(unused_cols, axis=1, inplace=True, errors='ignore') | ||
|
||
# Create additional Column with colors based on final classification | ||
df['color'] = [color_mapping[i] for i in df['classification']] | ||
|
||
# Create additional column with component ID | ||
df['component'] = np.arange(n_comps) | ||
|
||
# Compute angle and re-sort data for Pie plots | ||
df['angle'] = df['var_exp'] / df['var_exp'].sum() * 2 * pi | ||
df.sort_values(by=['classification', 'var_exp'], inplace=True) | ||
|
||
cds = models.ColumnDataSource(data=dict( | ||
kappa=df['kappa'], | ||
rho=df['rho'], | ||
varexp=df['var_exp'], | ||
kappa_rank=df['kappa_rank'], | ||
rho_rank=df['rho_rank'], | ||
varexp_rank=df['var_exp_rank'], | ||
component=[str(i) for i in df['component']], | ||
color=df['color'], | ||
size=df['var_exp_size'], | ||
classif=df['classification'], | ||
angle=df['angle'])) | ||
|
||
return cds | ||
|
||
|
||
def _create_kr_plt(comptable_cds): | ||
""" | ||
Create Dymamic Kappa/Rho Scatter Plot | ||
|
||
Parameters | ||
---------- | ||
comptable_cds: bokeh.models.ColumnDataSource | ||
Data structure containing a limited set of columns from the comp_table | ||
|
||
Returns | ||
------- | ||
fig: bokeh.plotting.figure.Figure | ||
Bokeh scatter plot of kappa vs. rho | ||
""" | ||
# Create Panel for the Kappa - Rho Scatter | ||
kr_hovertool = models.HoverTool(tooltips=[('Component ID', '@component'), | ||
('Kappa', '@kappa{0.00}'), | ||
('Rho', '@rho{0.00}'), | ||
('Var. Expl.', '@varexp{0.00}%')]) | ||
fig = plotting.figure(plot_width=400, plot_height=400, | ||
tools=["tap,wheel_zoom,reset,pan,crosshair,save", kr_hovertool], | ||
title="Kappa / Rho Plot") | ||
fig.circle('kappa', 'rho', size='size', color='color', alpha=0.5, source=comptable_cds, | ||
legend_group='classif') | ||
fig.xaxis.axis_label = 'Kappa' | ||
fig.yaxis.axis_label = 'Rho' | ||
fig.toolbar.logo = None | ||
fig.legend.background_fill_alpha = 0.5 | ||
fig.legend.orientation = 'horizontal' | ||
fig.legend.location = 'bottom_right' | ||
return fig | ||
|
||
|
||
def _create_sorted_plt(comptable_cds, n_comps, x_var, y_var, title=None, | ||
x_label=None, y_label=None): | ||
""" | ||
Create dynamic sorted plots | ||
|
||
Parameters | ||
---------- | ||
comptable_ds: bokeh.models.ColumnDataSource | ||
Data structure containing a limited set of columns from the comp_table | ||
|
||
x_var: str | ||
Name of variable for the x-axis | ||
|
||
y_var: str | ||
Name of variable for the y-axis | ||
|
||
title: str | ||
Plot title | ||
|
||
x_label: str | ||
X-axis label | ||
|
||
y_label: str | ||
Y-axis label | ||
|
||
Returns | ||
------- | ||
fig: bokeh.plotting.figure.Figure | ||
Bokeh plot of components ranked by a given feature | ||
""" | ||
hovertool = models.HoverTool(tooltips=[('Component ID', '@component'), | ||
('Kappa', '@kappa{0.00}'), | ||
('Rho', '@rho{0.00}'), | ||
('Var. Expl.', '@varexp{0.00}%')]) | ||
fig = plotting.figure(plot_width=400, plot_height=400, | ||
tools=["tap,wheel_zoom,reset,pan,crosshair,save", hovertool], | ||
title=title) | ||
fig.line(x=np.arange(1, n_comps + 1), | ||
y=comptable_cds.data[y_var].sort_values(ascending=False).values, | ||
color='black') | ||
fig.circle(x_var, y_var, source=comptable_cds, | ||
size=5, color='color', alpha=0.7) | ||
fig.xaxis.axis_label = x_label | ||
fig.yaxis.axis_label = y_label | ||
fig.x_range = models.Range1d(-1, n_comps + 1) | ||
fig.toolbar.logo = None | ||
|
||
return fig | ||
|
||
|
||
def _create_varexp_pie_plt(comptable_cds, n_comps): | ||
fig = plotting.figure(plot_width=400, plot_height=400, title='Variance Explained View', | ||
tools=['hover,tap,save'], | ||
tooltips=[('Component ID', ' @component'), | ||
('Kappa', '@kappa{0.00}'), | ||
('Rho', '@rho{0.00}'), | ||
('Var. Exp.', '@varexp{0.00}%')]) | ||
fig.wedge(x=0, y=1, radius=.9, | ||
start_angle=transform.cumsum('angle', include_zero=True), | ||
end_angle=transform.cumsum('angle'), | ||
line_color="white", | ||
fill_color='color', source=comptable_cds, fill_alpha=0.7) | ||
fig.axis.visible = False | ||
fig.grid.visible = False | ||
fig.toolbar.logo = None | ||
|
||
circle = models.Circle(x=0, y=1, size=150, fill_color='white', line_color='white') | ||
fig.add_glyph(circle) | ||
|
||
return fig | ||
|
||
|
||
def _tap_callback(comptable_cds, div_content, out_dir): | ||
""" | ||
Javacript function to animate tap events and show component info on the right | ||
|
||
Parameters | ||
---------- | ||
CDS: bokeh.models.ColumnDataSource | ||
Data structure containing a limited set of columns from the comp_table | ||
div: bokeh.models.Div | ||
Target Div element where component images will be loaded | ||
|
||
Returns | ||
------- | ||
CustomJS: bokeh.models.CustomJS | ||
Javascript function that adds the tapping functionality | ||
""" | ||
return models.CustomJS(args=dict(source_comp_table=comptable_cds, | ||
div=div_content, | ||
outdir=out_dir), code=tap_callback_jscode) | ||
|
||
|
||
def _link_figures(fig, comptable_ds, div_content, out_dir): | ||
""" | ||
Links figures and adds interaction on mouse-click. | ||
|
||
Parameters | ||
---------- | ||
fig : bokeh.plotting.figure | ||
Figure containing a given plot | ||
|
||
comptable_ds : bokeh.models.ColumnDataSource | ||
Data structure with a limited version of the comptable | ||
suitable for dynamic plot purposes | ||
|
||
div_content : bokeh.models.Div | ||
Div element for additional HTML content. | ||
|
||
out_dir : str | ||
Output directory of tedana results. | ||
|
||
Returns | ||
------- | ||
fig : bokeh.plotting.figure | ||
Same as input figure, but with a linked method to | ||
its Tap event. | ||
|
||
""" | ||
fig.js_on_event(events.Tap, | ||
_tap_callback(comptable_ds, | ||
div_content, | ||
out_dir)) | ||
return fig |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I hit an error on my first run, as my version of bokeh was apparently outdated, should this be set to specific version num? Updating from 1.3.4 (?!) to 2.0.1 fixed it.
For completion sake, error:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think version 1.4 is the minimum required but I'm not completely sure right now. Let me check.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Strange, on OS X I got a working version. Would be good to set a minimum version in the
.txt
file though.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was using an anaconda install, from not so long ago (~months). As you can see, the error message is super useless - so I just had to assume it was a version issue.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Interestingly, if we require Bokeh 2.0.0 then we have to drop python 3.5 support (since it requires python 3.6+). How would you feel about that @tsalo ?