Multivariate Cross Classification

Overview

The former MVPA technique has the ability to detect subtle differences in brain activation patterns. Thus, this powerful capacity could be used to study how these patterns are consistent across different cognitive contexts. In general, the consistency of the information across different sets of data can be analyzed with these techniques. To this end, classification models are trained with one set of data and the consistency is assessed by testing these models with another data sets, belonging to a different experimental condition. This technique is called Multivariate Cross-Classification (MVCC) and is growing in popularity in recent years.

It is important to stress that different results can be obtained depending on which set is used for training and which one for testing ( Train:A → Test:B or Train:B → Test:A). This is called classification direction. The observation of classification direction asymmetries in MVCC can be explained by several and very different phenomena, including complex neurocognitive mechanisms or a simple signal-to-noise ratio difference across datasets. For this reason, reporting results in both directions is highly recommended. By default, MVPAlab computes and reports both directions separately.

Figure 1. Time-resolved MVCC analysis results. (a) Group-level decoding performance (F1-score) for both cross-classification directions. Single subject plots are represented in dashed and dotted lines. Significant clusters are highlighted using horizontal colored bars. Shaded areas represent the standard error of the mean.

Demo scripts:

To compute the MVCC analysis, the function mvpalab_mvcc() should be called after the feature extraction stage:

mvcc_demo.m

%% MVPAlab TOOLBOX - (mvcc_demo.m)
% -------------------------------------------------------------------------
% Brain, Mind and Behavioral Research Center - University of Granada.
% Contact: [email protected]
% -------------------------------------------------------------------------

% Initialize project and run configuration file:
cfg = mvpalab_init();
run cfg_file; % cfg_file_advanced for advanced configuration.

% Load data, generate conditions and feature extraction:
[cfg,data,fv] = mvpalab_import(cfg);

% Compute MVCC analysis:
[result,cfg] = mvpalab_mvcc(cfg,fv);

% Compute permutation maps and run statistical analysis
[permaps,cfg] = mvpalab_cpermaps(cfg,fv);
stats = mvpalab_permtest(cfg,result,permaps);

% Save cfg file:
mvpalab_savecfg(cfg);

% Plot the results:
run mvcc_plot;

Similar to previous analysis, this function mvpalab_mvcc() returns an updated version of the configuration structure cfg and the result variable. In this case, time resolved accuracy values are stored for both classification directions in:

result.acc.ab
result.acc.ba

Here you can find a configuration template for the former analysis:

cfg_file_advanced.m

%% Advanced configuration file for MVCC analysis - Folder and data files:

cfg.analysis = 'MVCC';
cfg.location = pwd;

cfg.study.conditionIdentifier{1,1} = 'condition_1';
cfg.study.conditionIdentifier{1,2} = 'condition_2';
cfg.study.conditionIdentifier{2,1} = 'condition_3';
cfg.study.conditionIdentifier{2,2} = 'condition_4';

cfg.study.dataPaths{1,1} = 'C:\Users\Cimcyc\Desktop\data\condition_1\';
cfg.study.dataPaths{1,2} = 'C:\Users\Cimcyc\Desktop\data\condition_2\';
cfg.study.dataPaths{2,1} = 'C:\Users\Cimcyc\Desktop\data\condition_3\';
cfg.study.dataPaths{2,2} = 'C:\Users\Cimcyc\Desktop\data\condition_4\';

cfg.study.dataFiles{1,1} = {'1.mat','2.mat','3.mat'};
cfg.study.dataFiles{1,2} = {'1.mat','2.mat','3.mat'};
cfg.study.dataFiles{2,1} = {'1.mat','2.mat','3.mat'};
cfg.study.dataFiles{2,2} = {'1.mat','2.mat','3.mat'};

%% FEATURE EXTRACTION:

cfg.feature = 'voltage';

% cfg.feature = 'voltage'  - Raw voltage as feature.
% cfg.feature = 'envelope' - Power evelope as feature.

cfg.powenv.method = 'analytic';
cfg.powenv.uplow  = 'upper';
cfg.powenv.length = 5;

% cfg.powenv.method = 'analytic' - Envelope using the analytic signal.
% cfg.powenv.method = 'peak'     - Peak envelopes.

% cfg.powenv.uplow = 'upper' - Select upper envelope.
% cfg.powenv.uplow = 'lower' - Select lower envelope.

%% TRIAL AVERAGE:

cfg.trialaver.flag     = true;
cfg.trialaver.ntrials  = 5;
cfg.trialaver.order    = 'rand';

% cfg.trialaver.order = 'rand' - Random order.
% cfg.trialaver.order = 'seq'  - Secuential order.

%% BALANCED DATASETS:

cfg.classsize.match = true;
cfg.classsize.matchkfold = true;

%% DIMENSION REDUCTION:

% cfg.dimred.method = 'none' - Diemnsion reduction disabled.
% cfg.dimred.method = 'pca'  - Principal Component Analysis.

cfg.dimred.method = 'none';
cfg.dimred.ncomp  = 0;

%% DATA NORMALIZATION:

% cfg.normdata = 0 - raw data
% cfg.normdata = 1 - z-score (across features)
% cfg.normdata = 2 - z-score (across time)
% cfg.normdata = 3 - z-score (across trials)
% cfg.normdata = 4 - std_nor (across trials)

cfg.normdata = 4; 

%% DATA SMOOTHING:

% cfg.smoothdata.method = 'none'   - Data smooth disabled.
% cfg.smoothdata.method = 'moving' - Moving average method.

cfg.smoothdata.method   = 'moving';
cfg.smoothdata.window   = 5;

%% ANALYSIS TIMING:

cfg.tm.tpstart   = -200;
cfg.tm.tpend     = 1500;
cfg.tm.tpsteps   = 3;

%% CLASSIFICATION ALGORITHM:

% cfg.classmodel.method = 'svm' - Support Vector Machine.
% cfg.classmodel.method = 'da'  - Linear Discriminant Analysis.

% cfg.classmodel.kernel = 'linear'     - Support Vector Machine.
% cfg.classmodel.kernel = 'gaussian'   - Support Vector Machine.
% cfg.classmodel.kernel = 'rbf'        - Support Vector Machine.
% cfg.classmodel.kernel = 'polynomial' - Support Vector Machine.

% cfg.classmodel.kernel = 'linear' - Discriminant Analysis.
% cfg.classmodel.kernel = 'quadratic' - Discriminant Analysis.

cfg.classmodel.method = 'svm';
cfg.classmodel.kernel = 'linear';

%% PERFORMANCE METRICS:

cfg.classmodel.roc       = false;
cfg.classmodel.auc       = false;
cfg.classmodel.confmat   = false;
cfg.classmodel.precision = false;
cfg.classmodel.recall    = false;
cfg.classmodel.f1score   = false;
cfg.classmodel.wvector   = false;

%% EXTRA CONFIGURATION:

cfg.classmodel.tempgen = false;
cfg.classmodel.extdiag = false;
cfg.classmodel.permlab = false;

% Enable parallel computation if the Distrib_Computing_Toolbox is installed:

if license('test','Distrib_Computing_Toolbox')
    cfg.classmodel.parcomp = true;
else
    cfg.classmodel.parcomp = false;
end

%% CROSS-VALIDATIONN PROCEDURE:

% cfg.cv.method = 'kfold' - K-Fold cross-validation.
% cfg.cv.method = 'loo'   - Leave-one-out cross-validation.

cfg.cv.method  = 'kfold';
cfg.cv.nfolds  = 5;

%% PERMUTATION TEST

cfg.stats.flag   = 0;
cfg.stats.nper   = 100;
cfg.stats.nperg  = 1e5;
cfg.stats.pgroup = 99.9;
cfg.stats.pclust = 99.9;
cfg.stats.shownulldis = 0;

And finally, the following scripts generate a graphic representation of the result:

% Initialize and configure plots:
graph = mvpalab_plotinit();

% Load results if needed: 
load results/time_resolved/acc/result.mat

% Axis limits:
graph.xlim = [-200 1500];
graph.ylim = [.3 .95];

% Axes labels and titles:
graph.xlabel = 'Time (ms)';
graph.ylabel = 'Classifier performance';
graph.title = 'Demo plot (no statistical significance)';

% Smooth results:
graph.smoothdata = 1; % (1 => no smoothing)

% Mean accuracy plot (no statistical significance)
figure;
hold on

% Colors A - B:
graph.shadecolor = graph.colors.mvpalab{10};
mvpalab_plotdecoding(graph,cfg,result.ab);

% Colors B - A:
graph.shadecolor = graph.colors.mvpalab{1};
mvpalab_plotdecoding(graph,cfg,result.ba);

% Initialize and configure plots:
graph = mvpalab_plotinit();

% Load results if needed: 
load results/time_resolved/acc/result.mat
load results/time_resolved/acc/stats.mat

% Axis limits:
graph.xlim = [-200 1500];
graph.ylim = [.3 .95];

% Axes labels and titles:
graph.xlabel = 'Time (ms)';
graph.ylabel = 'Classifier performance';
graph.title = 'Demo plot (statistical significance)';

% Plot significant clusters (above and below chance):
graph.stats.above = true;
graph.stats.below = true;

% Significant indicator:
graph.sigh = .4;

% Smooth results:
graph.smoothdata = 1; % (1 => no smoothing)

% Mean accuracy plot (no statistical significance)
figure;
hold on

% Colors A - B:
graph.shadecolor = graph.colors.mvpalab{10};
graph.sigc = graph.colors.mvpalab{10};
mvpalab_plotdecoding(graph,cfg,result.ab,stats.ab);

% Colors B - A:
graph.sigh = .45;
graph.shadecolor = graph.colors.mvpalab{1};
graph.sigc = graph.colors.mvpalab{1};
mvpalab_plotdecoding(graph,cfg,result.ba,stats.ba);