Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DOC] Reporting documentation #465

Merged
merged 84 commits into from
Oct 19, 2020
Merged
Show file tree
Hide file tree
Changes from 83 commits
Commits
Show all changes
84 commits
Select commit Hold shift + click to select a range
58a2c32
Added html template for report
Nov 11, 2019
b61fb1e
doc: initial commit of reporting template
emdupre Nov 11, 2019
34c9133
doc: save out html for testing
emdupre Nov 11, 2019
fcaf13b
fix: add missing bokeh js script
emdupre Nov 11, 2019
942a469
fix: unescape HTML ugly way
emdupre Nov 11, 2019
b7262ab
fix: add missing cdn's
emdupre Nov 11, 2019
fbb2659
enh: switch to generate_report structure
emdupre Nov 11, 2019
31bf94f
sty: switch back to pure CSS from bootstrap
emdupre Nov 11, 2019
f0db0f5
fix: differently update template for content
emdupre Nov 11, 2019
1901196
sty: add jupyter notebooks to git ignore
emdupre Nov 12, 2019
2bffccf
enh:initial commit of dynamic plots
javiergcas Nov 12, 2019
1627b10
fix: patch unpaired bracket
emdupre Nov 12, 2019
89fd655
Merge remote-tracking branch 'emdupre/doc/me-reports' into dynamic_re…
javiergcas Nov 12, 2019
8bf5310
enh:modularized dynamic kappa/rho plots
javiergcas Nov 12, 2019
0b44fa0
enh:docstrings added to KappaRho_DynPlot.py
javiergcas Nov 12, 2019
70bbe10
Force figures default, fold viz into reporting module
emdupre Nov 12, 2019
d834b30
enh:TS and Spectrum are now dynamic plots
javiergcas Nov 12, 2019
27f7ac8
Merge pull request #20 from javiergcas/dynamic_reports
emdupre Nov 12, 2019
ebea3c9
sty: initial refactor of dynamic figures
emdupre Nov 13, 2019
5ff0d3d
fix: re-introduce func to load comp_table
emdupre Nov 13, 2019
ed44834
sty: finalize rough refactor
emdupre Nov 13, 2019
7302e1e
fix: patch linting errors
emdupre Nov 13, 2019
d82e284
Merge remote-tracking branch 'upstream/master' into doc/me-reports
emdupre Nov 13, 2019
c9f1396
Initial commit of documentation page for reports
javiergcas Nov 14, 2019
84cdee1
subsections added
javiergcas Nov 14, 2019
b496dee
doc:plot descriptions completed. new figures.
javiergcas Nov 14, 2019
ef318b6
doc: continue refactor
emdupre Nov 14, 2019
41f6d0a
doc:additional info and pics of icons
javiergcas Nov 14, 2019
879b297
doc: finalized first draft of report docs
javiergcas Nov 15, 2019
632900b
Merge branch 'master' of https://github.com/me-ica/tedana into doc/me…
eurunuela Nov 15, 2019
96a896f
Advanced on dynamic_figures
eurunuela Nov 15, 2019
d1a417b
Made the refactored code work
eurunuela Nov 15, 2019
b9b9117
fix: new year, new approach
emdupre Jan 3, 2020
636d9ed
tmp: non-functioning reporting code
emdupre Jan 17, 2020
8986fab
Merge remote-tracking branch 'refs/remotes/origin/doc/me-reports' int…
emdupre Jan 17, 2020
bc668d0
Merge remote-tracking branch 'upstream/master' into doc/me-reports
emdupre Jan 21, 2020
748c9a9
Merge remote-tracking branch 'upstream/master' into doc/me-reports
emdupre Mar 20, 2020
8192fb2
Fixes bug
eurunuela Mar 21, 2020
d7c3745
Fixes bug
eurunuela Mar 21, 2020
ed3500a
Added new function to generate html report and removed tempita calls
eurunuela Apr 8, 2020
3ce5d4d
Added FFT and time series plots
eurunuela Apr 13, 2020
b66d097
Made template subsitute work and updated necessary bokeh version
eurunuela Apr 13, 2020
8a8fe7e
Changes html title
eurunuela Apr 13, 2020
b48aa7f
remove time series, fft plots
emdupre Apr 14, 2020
55e5900
fix: remove unescapes
emdupre Apr 14, 2020
2fd4683
fix: patch incorrect imports
emdupre Apr 14, 2020
46c3eb4
Merge remote-tracking branch 'upstream/master' into doc/me-reports
emdupre Apr 14, 2020
53b2248
fix: add missing bokeh import
emdupre Apr 14, 2020
de61e24
fix: spelling mistake 🤦
emdupre Apr 14, 2020
4f8be84
fix: reflect reorg in get_coeffs call
emdupre Apr 14, 2020
797f735
fix: correct expected test outputs
emdupre Apr 14, 2020
133b7e2
fix: don't designate file path when saving report
emdupre Apr 14, 2020
464bef0
fix: combine generate, html report functions
emdupre Apr 14, 2020
b66d578
fix: ignore missing columns in df.drop, to better handle multiple params
emdupre Apr 14, 2020
a8f9900
Dynamic figures are now two rows, static figure is on the right. Remo…
eurunuela Apr 17, 2020
131dc82
Added new line after References in report.txt
eurunuela Apr 17, 2020
1a4a174
Removes BeautifulSoup dependency in favor of an object tag and change…
eurunuela Apr 18, 2020
3f162a5
Merge pull request #23 from eurunuela/enh/dynamic_reports
emdupre Apr 18, 2020
72cd396
enh: switch to grid layout, style About section, add links to navbar
emdupre Apr 19, 2020
1363c01
doc: update no-png to no-reports
emdupre Apr 21, 2020
7d5ec6b
Addresses PR comments
eurunuela Apr 25, 2020
88ae8ab
Merge pull request #1 from eurunuela/doc/reports
javiergcas May 4, 2020
2b5a94b
modifiation of setup.py to properly install all reporting directories
javiergcas May 9, 2020
ba68bfd
Merge pull request #24 from javiergcas/doc/me-reports
emdupre May 10, 2020
05e91a4
Use relative path for static figures on report
eurunuela Jun 4, 2020
663f5eb
Merge pull request #25 from eurunuela/doc/me-reports
emdupre Jun 4, 2020
1b517b6
Add version check for bokeh figures
emdupre Jun 11, 2020
4817b8a
Merge branch 'doc/me-reports' of https://github.com/emdupre/tedana in…
emdupre Jun 11, 2020
b4cf000
Fix typo
eurunuela Oct 15, 2020
cfbfb1c
Merge branch 'reports_doc' of https://github.com/javiergcas/tedana in…
eurunuela Oct 15, 2020
e61acaa
Removed visual reports and added new reports page to sidebar
eurunuela Oct 15, 2020
91a9f61
Update docs/reporting.rst
eurunuela Oct 16, 2020
47d2c75
Update docs/reporting.rst
eurunuela Oct 16, 2020
e99cea0
Merge with upstream master
eurunuela Oct 16, 2020
605535d
Update docs/reporting.rst
eurunuela Oct 16, 2020
78963cf
Update docs/reporting.rst
eurunuela Oct 16, 2020
470799f
Update docs/reporting.rst
eurunuela Oct 16, 2020
e9cbc54
Update docs/reporting.rst
eurunuela Oct 18, 2020
1b59407
Update reporting page figures and text
eurunuela Oct 18, 2020
635405e
Update beta map text
eurunuela Oct 18, 2020
f3965af
Update docs/reporting.rst
eurunuela Oct 19, 2020
d72a5b8
Update docs/reporting.rst
eurunuela Oct 19, 2020
27a0652
Update docs/reporting.rst
eurunuela Oct 19, 2020
252f7c3
Move note higher up the page
eurunuela Oct 19, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added docs/_static/rep01_betaMap.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/_static/rep01_fftPlot.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/_static/rep01_kappaScree.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/_static/rep01_kapparhoScatter.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/_static/rep01_overallview.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/_static/rep01_rhoScree.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/_static/rep01_tool_areazoom.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/_static/rep01_tool_crosshair.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/_static/rep01_tool_hover.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/_static/rep01_tool_pan.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/_static/rep01_tool_reset.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/_static/rep01_tool_save.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/_static/rep01_tool_select.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/_static/rep01_tool_wheelzoom.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/_static/rep01_tools.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/_static/rep01_tsPlot.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/_static/rep01_varexpPie.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -157,6 +157,7 @@ tedana is licensed under GNU Lesser General Public License version 2.1.
usage
approach
outputs
reporting
faq
support
contributing
Expand Down
78 changes: 0 additions & 78 deletions docs/outputs.rst
Original file line number Diff line number Diff line change
Expand Up @@ -198,81 +198,3 @@ An example report
Sørensen, T. J. (1948). A method of establishing groups of equal amplitude in plant sociology based on similarity of species content and its application to analyses of the vegetation on Danish commons. I kommission hos E. Munksgaard.

Van Der Walt, S., Colbert, S. C., & Varoquaux, G. (2011). The NumPy array: a structure for efficient numerical computation. Computing in Science & Engineering, 13(2), 22.

Visual reports
--------------
Static visual reports can be generated by using the ``--png`` flag when calling
tedana from the command line.
Images are created and placed within the output directory, in a folder labeled
``figures``.

These reports consist of three main types of images.

Component Images
````````````````
.. image:: /_static/example_good_component.png
:align: center

For each component identified by tedana, a single image will be created.
Above is an example of an accepted component.
These are designed for an up-close inspection of both the spatial and temporal
aspects of the component, as well as ancillary information.

The title of the plot provides information about variance, kappa and rho values
as well as the reasons for rejection, if any (see above for codes).

Below this is the component timeseries, color coded on the basis of its
classification.
Green for accepted, Red for rejected, Black for ignored or unclassified.

Slices are then selected from sagittal, axial and coronal planes, to highlight
the component pattern.
By default these images used the red-blue colormap and are scaled to 10% of the
max beta value.

.. note::
You can select your own colormap to use by specifying its name when calling
tedana with ``--png-cmap``.
For example, to use the bone colormap, you would add ``--png-cmap bone``.

Finally, the bottom of the image shows the Fast Fourier Transform of the
component timeseries.

.. tip::
Look for your fundamental task frequencies here!

.. image:: /_static/example_bad_component.png
:align: center

Above, you can review a component that was rejected.
In this case, the subject moved each time the task was performed - which
affected single slices of the fMRI volume.
This scan used multiband imaging (collecting multiple slices at once), so
the motion artifact occurs in more than once slice.


Kappa vs Rho Scatter Plot
`````````````````````````
.. image:: /_static/example_Kappa_vs_Rho_Scatter.png
:align: center

This diagnostic plot shows the relationship between kappa and rho values for
each component.

This can be useful for getting a big picture view of your data or for comparing
denoising performance with various fMRI sequences.

Double Pie Chart
````````````````
.. image:: /_static/example_Component_Overview.png
:align: center

This diagnostic plot shows the relative variance explained by each
classification type in the outer ring, with individual components on the inner
ring.
If a low amount of variance is explained, this will be shown as a gap in the
ring.

.. tip::
Sometimes large variance is due to singular components, which can be
easily seen here.
168 changes: 168 additions & 0 deletions docs/reporting.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,168 @@
#####################
ICA Components Report
#####################

The reporting page for the tedana decomposition presents a series
of interactive plots designed to help you evaluate the quality of your
analyses. This page describes the plots forming the reports and well as
information on how to take advantage of the interactive functionalities.
You can also play around with `our demo`_.

.. _our demo: https://me-ica.github.io/tedana-ohbm-2020/

Report Structure
================

The image below shows a representative report, which has two sections: a) the summary view,
and b) the individual component view.

.. image:: /_static/rep01_overallview.png
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this reflects the state of the reports any more, unfortunately, since we removed the individual component view. The figure should be updated

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1b59407 should take care of this.

I included the individual component view section in the end, as we do show a static view of individual components.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That looks right, thanks !! Should we note that the individual component view will be empty by default, since no component is pre-selected ?

:align: center
Summary View
------------
This view provides an overview of the decomposition and component
selection results. It includes four different plots.

* **Kappa/Rho Scatter:** This is a scatter plot of `Kappa` vs. `Rho` features for all components.
In the plot, each dot represents a different component. The x-axis represents the kappa feature, and the
y-axis represents the rho feature. These are two of the most
informative features describing the likelihood of the component
being BOLD or non-BOLD. Additional information is provided via color
and size. In particular, color informs about its classification
status (e.g., accepted, rejected); while size relates to
the amount of variance explained by the component (larger dot,
larger variance).
.. image:: /_static/rep01_kapparhoScatter.png
:align: center
:height: 400px

* **Kappa Scree Plot:** This scree plot provides a view of the components ranked by `Kappa`.
As in the previous plot, each dot represents a component. The color of the dot informs us
about classification status. In this plot, size is not related to variance explained.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we explain what the axes are ?

.. image:: /_static/rep01_kappaScree.png
:align: center
:height: 400px

* **Rho Scree Plot:** This scree plot provides a view of the components ranked by `Rho`.
As in the previous plot, each dot represents a component. The color of the dot informs us
about classification status. In this plot, size is not related to variance explained.
.. image:: /_static/rep01_rhoScree.png
:align: center
:height: 400px

* **Variance Explained Plot:** This pie plot provides a summary of how much variance is explained
by each individual component, as well as the total variance explained by each of the three
classification categories (i.e., accepted, rejected, ignored). In this plot, each component is
represented as a wedge, whose size is directly related to the amount of variance explained. The
color of the wedge inform us about the classification status of the component. For this view,
components are sorted by classification first, and inside each classification group by variance
explained.
.. image:: /_static/rep01_varexpPie.png
:align: center
:height: 400px

Individual Component View
-------------------------
This view provides detailed information about an individual
component (selected in the summary view, see below). It includes three different plots.

* **Time series:** This plot shows the time series associated with a given component
(selected in the summary view). The x-axis represents time (in units of TR), and the
y-axis represents signal levels (in arbitrary units). Finally, the color of the trace
informs us about the component classification status.

.. image:: /_static/rep01_tsPlot.png
:align: center
:height: 150px

* **Component beta map:** This plot shows the map of the beta coefficients associated with
a given component (selected in the summary view). The colorbar represents the amplitude
of the beta coefficients.

.. image:: /_static/rep01_betaMap.png
:align: center
:height: 400px

* **Spectrum:** This plot shows the spectrogram associated with a given component
(selected in the summary view). The x-axis represents frequency (in Hz), and the
y-axis represents spectral amplitude.

.. image:: /_static/rep01_fftPlot.png
:align: center
:height: 150px

.. note::
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is what I was thinking of 😸 Maybe let's move it up around L19 ?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes! That's a great idea. Thank you!

When a report is initially loaded, as no component is selected on the
summary view, the individual component view appears empty.

Reports User Interactions
=========================

As previously mentioned, all summary plots in the report allow user interactions. While the
Kappa/Rho Scatter plot allows full user interaction (see the toolbar that accompanies the plot
and the example below), the other three plots allow the user to select components and update the
figures.

.. image:: /_static/rep01_tools.png
:align: center
:height: 25px

The table below includes information about all available interactions

.. |Reset| image:: /_static/rep01_tool_reset.png
:height: 25px

.. |WZoom| image:: /_static/rep01_tool_wheelzoom.png
:height: 25px

.. |BZoom| image:: /_static/rep01_tool_areazoom.png
:height: 25px

.. |CHair| image:: /_static/rep01_tool_crosshair.png
:height: 25px

.. |Pan| image:: /_static/rep01_tool_pan.png
:height: 25px

.. |Hover| image:: /_static/rep01_tool_hover.png
:height: 25px

.. |Sel| image:: /_static/rep01_tool_select.png
:height: 25px

.. |Save| image:: /_static/rep01_tool_save.png
:height: 25px

============ ======= =======================================================
Interaction Icon Description
============ ======= =======================================================
Reset |Reset| Resets the data bounds of the plot to their values when
the plot was initially created.

Wheel Zoom |WZoom| Zoom the plot in and out, centered on the current
mouse location.

Box Zoom |BZoom| Define a rectangular region of a plot to zoom to by
dragging the mouse over the plot region.

Crosshair |CHair| Draws a crosshair annotation over the plot, centered on
the current mouse position

Pan |Pan| Allows the user to pan a plot by left-dragging a mouse
across the plot region.

Hover |Hover| If active, the plot displays informational tooltips
whenever the cursor is directly over a plot element.

Selection |Sel| Allows user to select components by tapping on the dot
or wedge that represents them. Once a component is
selected, the plots forming the individual component
view update to show component specific information.

Save |Save| Saves an image reproduction of the plot in PNG format.
============ ======= =======================================================

.. note::
Specific user interactions can be switched on/off by clicking on their associated icon within
the toolbar of a given plot. Active interactions show an horizontal blue line underneath their
icon, while inactive ones lack the line.