Validation of Automatic Spike-Sorting Methods

A workshop in Ski, Norway, 19.-20. May 2011.

Organized by: Gaute T. Einevoll, Espen Hagen, Felix Franke

Background

Sorting of spikes recorded by tetrodes or other multielectrodes is difficult. At the moment this spike sorting is typically done with a large manual component, making the procedure very labor intensive and unreliable. Further, different labs often use their own method making it difficult to compare their results. The presently used spike-sorting procedures have also not been validated in a quantitative way, at least not comprehensively. This makes the accuracy difficult to assess and the methods difficult to trust.

An international collaborative effort seems called for to improve the situation.

In May 2010 Gyorgy Buzsaki and Dima Rinberg organized a nice workshop on the topic at Janelia Farm. For the workshop at Ski, organized exactly a year later, the aim was to build on this initiative and to stimulate the development of a coordinated international effort to address the issue.

Several automatic spike-sorting methods have been developed, some of them by workshop participants, but an obstacle for this development has been the limited availability of reliable test data, i.e., multielectrode recordings where one knows the "true" underlying firing sequences of the relevant neurons. Also, good virtual meeting places where people with spike-sorting algorithms can get access to test data has been lacking.

We believe that significant progress can be made if a motivated group of (i) spike-sorting algorithm developers, (ii) users of such algorithms, (iii) people who can provide test data (either experimental or obtained by modeling), and (iv) people who can provide collaborative web-tools meet and join forces.

Participants

Clemens Boucsein Isabel Delgado-Ruz Gaute T. Einevoll
Felix Franke Marianne Fyhn Samuel Garcia
Torkel Hafting Espen Hagen Ken Harris
Dan Hill Anders Johansson Shabnam Kadir
Emilio Kropff Henrik Linden Philipp Meier
Luis A. Camunas Mesa Christophe Pouzat Raphael Ritz
Simon Schultz Shy Shoham Erik Schomburg
Andrey Sobolev Eran Stark Ariel Tankus
Palmi Thor Thorbergsson

Session 1: Algorithms

Algorithms for spike sorting data from large, dense electrode arrays

Kenneth Harris, Imperial College, London, GB

Spike sorting of non-stationary data (slides)

Eran Stark, Rutgers Univ.

Off-line and on-line spike sorting with a heuristic neural gas algorithm

Emilio Kropff, NTNU

Robust automatic classification of single units, multi-units and optogenetic validation (slides)

Shy Shoham and Ariel Tankus, Technion

Template matching with linear filters (slides)

Felix Franke, TU-Berlin

Session 2: Applications

Using multi-electrode arrays to localise and classify neurons in vivo

Simon Schultz, Imperial

Needs and hopes for spike sorting algorithms: analyzing data from the rodent visual system

Clemens Boucsein, U Freiburg

Session 3: Data for testing and validation of algorithms

Simulation of extracellular recordings

Luis Camunas, U Leicester

Modeling extracellular potentials in the rat hippocampus (slides)

Erik Schomburg, Caltech

Biophysically detailed test data from LFPy (slides)

Espen Hagen, Henrik Linden, UMB

Session 4: Electrodes

Electrodes and data acquisition for neural recordings

Anders Johanson, U Lund

Session 5: Metrics for comparison of spike trains

Estimation of false positive and negative error rates for sorted single-unit spike trains (slides)

Dan Hill, TU Munich

Spike sorting and sorting quality evaluation through generative model estimation, data simulation, flexible coarse classification and refined template matching (slides, high-res)

Christophe Pouzat, CNRS & U Paris

Session 6: Facilitation of collaboration

What we hope to achieve today 

Gaute T. Einevoll, UMB

OpenElectrophy's spikesorting framework: a bridge between good methods and experimentalists (slides)

Samuel Garcia, CNRS, Lyon

Tools for collaborative data sharing at the German INCF Node (slides, odp)

Andrey Sobolev, CNRS, Lyon

How can INCF assist? (slides)

Raphael Ritz, INCF Secr., Stockholm

Demonstration of evaluation page

Felix Franke and Espen Hagen