#### searchlight correlation in label ####
-------------------------------------------
funct: "SearchLight CrossCorr at Label Vtxs (left-click-X)"
  set label <infile>     [constraint region from label file]
  corr_over_label <neitype> <crit> <cmptype>
-------------------------------------------
funct: "Searchlight CrossCorr at Annot RGB (mid-click-X)"
  [constraint region from r,g,b of annotation]
  corr_over_annotcol <neitype> <crit> <cmptype> <r> <g> <b>
-------------------------------------------
funct: "Calculate RawData Covariance (shift-left-click-X)"
  rawdatasubbrik_to_paintable 0
  paint_surface 0
  find_uniqsamp_vertices
  calc_rawdata_covar
-------------------------------------------
funct: "Write RawData Covariance (shift-mid-click-X)"
  write_rawdata_covar
-------------------------------------------
funct: "Read RawData Covariance (shift-mid-click-X)"
  read_rawdata_covar
-------------------------------------------
funct: "Display Correlation for CurrVectex (shift-right-click-X)"
  [select vertex]
  copy_rowcovar_val
-------------------------------------------


The "X" button on the "label:" entry line
performs searchlight cross correlations within a
label or annotation region, and can also
calculate, write, and display covariance of raw
data time series.


Detailed Description of Label "X" Button Actions

-------------------------------------------
funct: "SearchLight CrossCorr at Label Vtxs (left-click-X)"
  set label <infile>     [constraint region from label file]
  corr_over_label <neitype> <crit> <cmptype>
-------------------------------------------
funct: "Searchlight CrossCorr at Annot RGB (mid-click-X)"
  [constraint region from r,g,b of annotation]
  corr_over_annotcol <neitype> <crit> <cmptype> <r> <g> <b>
-------------------------------------------

The first two functions are identical searchlight
cross correlations performed within a region
defined by either a tksurfer label or a single
colored region in an MGH annot file.  This is
done to constrain this potentially time-consuming
operation.  To cover the entire cortex, simply
FILL everything.

The tksurfer.tcl wrapper function used to run
corr_over_label and corr_over_annotcol is:
corrctrls (which calls do_corr).

Calculate Pearson Product-Moment Correlation:

                 Sum[(x - xav)*(y - yav)]
corr =   -----------------------------------
          sqrt[Sum(x - xav)^2 * Sum(y - yav)^2]

between two vertexwise data sets for a set of
searchlights that each include nearest neighbors
extending to a radius defined by simple vertex
count, surface area (mm^2), or radius (mm).  The
correlated values x and y at each vertex can each
be either real numbers or complex amplitudes (see
below).

If there are two complex-valued inputs and
comparison type ($cmptype) 4 is chosen, a
circular correlation is performed instead:

                 Sum[sin(x - xav)*sin(y - yav)]
corr =  ----------------------------------------
        sqrt[Sum[sin^2(x - xav)] * Sum[sin^2(y - yav)]]

where x and y are now angles at each vertex, and
xav and yav are now angular averages (computed by
normalizing complex input vectors before vector
averaging).

The set of searchlight centers is defined by a
label (left-click the "X") or by a single MGH
annotation region r,g,b color (middle-click the
"X" after picking a region with a left-click).

Loading Input Data Sets

In addition to a label specifying the searchlight
centers, this function requires that two data
sets (each can be either real or complex) to
correlate have been loaded into the visible
.val(.val2) data field(s) and the invisible
.valbak(.val2bak) data field(s).

This can be done by reading wfiles (*.w or
*.curv/area or 1D *.mgh, *.vtk, *.gii), or label
files (*.label).  It is OK to mix, but if you
read a label, don't forget to reload the label
specifying searchlight centers afterward.

To read the values in a wfile (or 1D .mgh file)
into the val field(s), use the "R" button on
"val:" line, which uses the tcl funct
read_binary_values.  This will auto-detect the
following formats:

  *.w -- freesurfer wfile, usu. vertex subset
  *.w/*.curv -- new curv/area, req's data all vtxs
  *.mgh -- 1D float real data (req's *.mgh suff)
  *.vtk -- 1D ASCII real (req's *.vtk suff)
  *.vtk-like -- 1D ASCII beginning w/POINT_DATA
  *.gii -- single-frame vertex-data GIFTI file

To read values in a *label* into the val field(s),
use use the "R" button on the "label:" line (which
uses the tcl funct read_label_to_val).

How to Load the Second Data Set

There is a stack of possible values at each
vertex.  Only the top (or top pair) is visible on
the displayed surface (the .val field for
real-valued data if complexvalflag is 0, or the
.val/.val2 fields for complex-valued data if
complexvalflag is 1).  The stack looks like this:

    val         (always visible)
    val2       (visible if complexvalflag is 1)

    stats       (invisible)
    pval       (invisible)

    valbak    (invisible)
    val2bak  (invisible)

To swap values between real and imaginary fields
on the top (visible) part of the stack, use a
middle-click on "S/V":

  swap_val_val2

To swap *pairs* of values between top (visible)
and bottom (invisible) parts of the stack, use
a shift-middle-click "S/V":

  swap_valval2_valbakval2bak

N.B.: also use this to swap just the top and
bottom real values (val and valbak).

Output Correlation Coefficient

The correlation is calculated and saved in the
.stat field.   If the operation is done by clicking
the "X" button in the interface, the result will
displayed when done.   In a tcl script, when the
functions corr_over_label or corr_over_annotcol
finish, do swap_stat_val and reset the colscale
(see example below).

To display a .stat, which is normally invisible
(only visible effect is optional stat masking),
the .stat has to be swapped to the visible .val
field.  A counterintuitive effect of this is when
clicking on a vertex, the swapped .stat value
will now be reported as "val" (the number
reported as "stat" will be whatever was
previously in the "val" field).

Output Correlation Coefficient

The vertexwise output correlation coefficient
will automatically be saved as a wfile with a
prefix taken from the "xcorr_outstem" entry on
the pop-up, and an "_s" infix ("_r" is in use for
real values).  The file will be saved in the
parent directory of the current "val:" file (asks
overwrite).

Alternate Output Z-Score

To save a z-score instead of the correlation
coefficient (z = atanh(r)), tick "r2zscoreflag".
In this case, the output file will have a "_z"
infix.

The correlation coefficient (or z-score) can also
be saved afterward to an ASCII label using the
"W" on the label line.  To save only the vertices
over which the correlation was performed, first
click "C" (recut current label) on the "label:"
line before saving (else the label will contain
all vertices).

Alternate Output -- Covariance

If the "xcorrcovarflag" in the Searchlight
Correlation popup is ticked, the numerator of the
Pearson (or circular) correlation equation is
instead divided by the number of vertices, and
the result is saved in .stat instead of the
correlation (or circular correlation).  The
p-value of the correlation is calculated as usual
(see next), but the then correlation (r) itself
is discarded.

The output wfile will have the same "_s" infix as
the correlation coefficient output.

N.B.: "xcorrcovarflag" is incompatible with
"r2zscoreflag".  You will have to untick one or
the other to procede.

Output p-Value for Circ Correlation Coefficient

This is calculated as follows:

  avgssx = 1/n * Sum[sin^2(x - xav)
  avgssy = 1/n * Sum[sin^2(y - yav)
  avgssxy = 1/n * Sum[sin^2(y - yav) * sin^2(y - yav)]
  t = sqrt((n*avgssx*avgssy) / avgssxy) * corr
  p = 2.0 * (1.0 - normcdf(fabs(t)))

N.B.: the equations for circular correlation as
well as the p-value calculations in Berens (2009)
(http://www.jstatsoft.org/v31/i10) contain
multiple errors, but the associated MATLAB code
is correct.  For the correct equations, go, for
example, here:

  http://ncss.wpengine.netdna-cdn.com/wp-content/themes/ncss/pdf/Procedures/NCSS/Circular_Data_Analysis.pdf

For convenience in using tksurfer color scales,
the output p-value is saved as the positive power
of ten (e.g., p<0.01 is saved as 2):

  vertex[i].pval = -log10(p)

The maximum positive p-value exponent is clamped
to 37.

To view (or save) the p-value, first swap it to
the visible position using the tcl function
swap_pval_val.  This can be done interactively
with a shift-left-click on the S/V button.

Searchlight Operation and Parameters

For a left-click on "X", the searchlight centers
are defined by every vertex in the file ($label)
that is currently in the "label:" field.  The
label doesn't have to be displayed.  the 2nd
through 5th (or 6th) entries on each label line
are ignored, so a 2-line label header (#!ascii,
vertexcount) followed by a single column of
vertex numbers is a sufficient label here.

For a middle-click on "X", the searchlight
centers are defined by every vertex in an
individual MGH annotation region with a certain
r,g,b color.  To select a region, load an
annotation (with "D") and click a vertex in one
of the regions to capture this region's color in
the r,g,b fields (to the right of the MESH
button) before middle-clicking the "X" button.

The vertices are first sorted by distance from
each center vertex by Euclidean distance.  To use
approximate geodesic distances (shortest
distances along the surface), use the inflated
surface.  Using a folded surface will instead
find the nearest vertices within a sphere around
each center vertex; this approximates a standard
searchlight in 3D, but one that is helpfully
restricted to the gray matter.

The number of vertices in the searchlight can be
specified directly (by number), or by however
many vertices are required to reach a specified
surface area (in mm^2), or a specified radius
(entered in mm).

To see the size of given searchlight, left-click
somewhere on the surface, and use the bottom
right "N", "A", and "R" buttons ("UNDO" to
restore surface).

When "X" is clicked, a popup first appears with 3
adjustable parameters that control the
searchlight, with the following definitions:

  $fillneartype (int: 0-2)

    0: corr over nearest n neighbors
    1: corr over nearest up to area
    2: corr over nearest up to radius

  $num_sqmm_mm (criterion, int or float)

    fillneartype=0: number of vertices
    fillneartype=1: area in mm^2
    fillneartype=2: radius in mm

  $cmptype (int: 0-4)

    0: real/real
    1: complex-amp/real
    2: real/complex-amp
    3: complex-amp/complex-amp
    4: complex-phase/complex-phase (circular corr)

The last parameter controls how the data loaded
into the val (val2) and valbak (val2bak) fields
at each vertex is treated.  There will be an
error if any of the required members for a
$cmptype have not been loaded.

    0: val <==> valbak
    1: amp(val,val2) <==> valbak
    2: val <==> amp(valbak,val2bak)
    3: hypot(val,val2) <==> hypot(valbak,val2bak)
    4: atan2(val,val2) <==> atan2(valbak,val2bak)

To illustrate the progess of the searchlight
without slowing it down, the center of one out of
every 200 searchlights computed are marked on the
surface.

Note that once the value fields have been loaded,
there is no check as to whether the fields are
stale relative to the current selection of
$cmptype.  For example, one could load two
complex-valued data sets, then perform a spurious
real-to-complex comparison.

To clear all fields before starting, do a
middle-click on the CLR button on the "val:" line
(*not* the CLR button on the "label:" line).
This clears:

  vertex[i].val
  vertex[i].val2
  vertex[i].valbak
  vertex[i].val2bak
  vertex[i].stat
  vertex[i].pval
  vertex[i].uniqvoxnum
  vertex[i].uniqvoxcnt

Use Representative Vertices (upsampling correction)

If "searchlightuniqflag" in the Searchlight
Correlation popup is ticked, each Pearson or
circular correlation searchlight is calculated
across 'representative' unique vertices -- that
is, using only one vertex for each uniquely
sampled 3D stat voxel.

This overcomes surface-based upsampling, the
result of a surface mesh that is typically finer
than standard 3D functional data sets (i.e., each
functional voxel is typically nearest-neighbor
sampled to more than one 2D surface vertex).

Unique vertices for each 3D functional voxel
(vertex in set sampling a single voxel nearest
the average location of all vertices in the set)
must first be calculated.  To do this, click
"val:" to get "val3d:" and do:

 1) select 3D stat BRIK from "val3d:" dropdown
 2) read 3D data with "val3d:" "R" button
 3) sample 3D data to surface with "PAINT"
 4) click UQ

The searchlight will still be calculated for
every vertex, but will only use 'representative'
vertices for calculating correlations.  This
resulting picture is less smooth and will require
a larger searchlight (still measured by
number/radius/area using all vertices) to obtain
the same number of data points to correlate.

To force the same number of vertices to be used
for each searchlight when "searchlightuniqflag"
is ticked, use fillneartype=0 (num of vertices)
and also tick "numafteruniqflag".  Note that this
expands the overall area of each searchlight in
order to make up for the skipped vertices.

---------------------------------
Example -- real-cplxamp by interface clicks
---------------------------------

## load 2nd data set (e.g., real)
select valfile from "val:" dropdown
read valfile ("R" on "val:" line)
           or
select labelfile from "label:" dropdown
read labelfile to val ("R" on "label:" line)

## swap pair (empty imaginary OK) to bottom of stack
shift-middle-click "S/V"

## load 1st data set (e.g., complex-valued)
select real (or imaginary) valfile from "val:" dropdown
read valfile ("R" on "val:" line)
=> auto-reads both real, imaginary
           or
select complex-valued labelfile from "label:" dropdown
read label file ("R" on "label:" line)

## select label for searchlight centers
select labelfile from "label:" dropdown

## set searchlight parms (7mm radius) and run corr's
click "X", and enter paramters in popup
  fillneartype -> 2          ;# use radius
  num_sqmm_mm -> 7.0  ;# 7 mm
  cmptype -> 1             ;# compare complex-amp to real
click "READY"
confirm choices and click "Searchlight Correlation"
=> takes a few minutes

## view result
when done, .stat field result swapped, colscale reset

---------------------------------
Same example -- by tcl script
---------------------------------

## load 1st data set (e.g., real), move to .valbak
setfile */scan1/scan+orig-lh_a.w
read_binary_values
swap_valval2_valbakval2bak
## load 2nd data set (e.g., complex-valued)
setfile */scan1/scan+orig-lh_i.w
read_binary_values
swap_val_val2
setfile */scan1/scan+orig-lh_r.w
read_binary_values
## set label of vtxs for searchlight centers
setfile */scan1/lh-searchregion.label
## searchlight
set fillneartype 2
set num_sqmm_mm 7.0
set cmptype 1
corr_over_label $fillneartype $num_sqmm_mm $cmptype
## view
swap_stat_val
set complexvalflag 0
set revphaseflag 0
set colscale 6
set fthresh 0.001
set fmid 0.6
set fslope 2.0
redraw

-------------------------------------------
funct: "Calculate RawData Covariance (shift-left-click-X)"
  rawdatasubbrik_to_paintable 0
  paint_surface 0
  find_uniqsamp_vertices
  calc_rawdata_covar
-------------------------------------------

This function calculates a covariance matrix for
a raw time-series data (AFNI BRIK containing
shorts) after sampling it to the surface, using
current parameters for distance along the local
surface normals.  To constrain the size of the
covariance matrix and avoid redundantly sampling
the data, 'representative vertices' are used.

A 'representative' (unique) vertex is the one
closest to average location of all the vertices
that have sampled one voxel.  The first three
functions generate the 'representative' vertex
subset.  For one hemisphere, this will be about
13K vertices (out of about 140K) for a 3.2 mm
width functional scan.

Each covariance matrix entry (for a pair of
vertices) is computed in the usual way as follows
(sum is across t):

           Sum[(v1 - v1_avg) * (v2 - v2_avg)]
corr =   -----------------------------------------
                               n - 1

which results in a symmetric matrix with
variances along the diagonal.  Before running
this function the policy for sampling the 3D
along the local surface normal should be set
(left-click the "label:" label to get a popup).
First tick or untick:

  normdsampsearchflag     # turn norm search on/off
  normdfracflag                # choose fraction of ctx thk vs. mm

then, depending on the state of the second flag,
set one of the following two pairs to the same
value (i.e., sample at a point):

  normdmax       # max sample dist in mm
  normdsamp     # min sample dist in mm

or:

  normdfracmax      # max sample dist in ctx thk fraction
  normdfracsamp    # min sample dist in ctx thk fraction

Then load a raw data time series (and
register.dat) as follows:

 1) click "val:" to toggle it to "val3d:" 
 2) click "RD" to get read rawdata popup 
 3) select rawdata BRIK from popup "val3d:" dropdown 
 4) click "READ RAWDATA and REGISTRATION"

Finally, R-click the "X" button on the "label:"
line to get a buttonbar popup and click
"Calculate RawData Covariance" (or
shift-left-click-X).  This takes about a minute.

The result can be written out, or a row of the
matrix can be displayed using the two functions
below.

N.B.: to see the 'representative' vertices, click
"UQ" on the "val3d:" line after "Calculate
RawData Covariance" has run.

-------------------------------------------
funct: "Write RawData Covariance (shift-mid-click-X)"
  write_rawdata_covar
-------------------------------------------

The covariance matrix can be written out as a
headerless binary file of floats, which will be
given a standard name in the current $scandir:

  $hemi-CovarianceMatrix.float

The indices of the rows and columns of this
covariance matrix are the sequentially numbered
'representative' vertex subset.  So in addition,
an ASCII file of the original full-surface vertex
numbers of these 'representative' vertices will
be written to:

  $hemi-CovarianceMatrixVnums.txt

The headerless binary file of floats is written
in standard freesurfer byte order (MSB).  Use
AFNI 4swap to convert to Intel (LSB) byte order
before reading the binary data into an external
program expecting LSB byte order.  For example:

  4swap rh-CovarianceMatrix.float

-------------------------------------------
funct: "Read RawData Covariance (shift-mid-click-X)"
  read_rawdata_covar
-------------------------------------------

This function reads the two files written above
(if they exist), so correlations can immediately
be displayed without re-calculating the
covariance matrix.

N.B.: this won't work right if the *.float file
has been 4swap'd to LSB :-}

-------------------------------------------
funct: "Display Correlation for CurrVectex (shift-right-click-X)"
  [select vertex]
  copy_rowcovar_val
-------------------------------------------

To display a row of the covariance matrix
(correlations for particular vertex), select a
vertex and click "Display Correlation for
CurrVertex".

First, the unique 'representative' vertex nearest
the selected vertex is found.

This index is then used to copy a row of the
covariance matrix to each other representative
vertex.  Before display, this data is first
sparse-smoothed (i.e., representative vertices
are fixed and only surrounding undefined vertices
are interpolated), for 20 steps. This uses the
C/tcl function, smooth_val_sparse, which can be
accessed directly from the SMOOTH button with the
"sp" option.

Finally, the first time a correlation is
displayed, the following sane colscale parameters
are set.  The tcl command used are:

  copy_rowcovar_val
  smooth_val_sparse 20
  set overlayflag 1
  set complexvalflag 0
  set statmaskampflag 0
  set truncphaseflag 0
  set colscale 11
  set fslope 0.1
  set fthresh 5.0
  set fmid 10.0

