Administrative Preamble

It has been my experience over some thirty years of teaching that the first day of class is mostly shopping and impedance matching. Students are asking ``Is this a good class for me to take?'' for some interpretation of ``good''. And so I treat it as such and prepare my lectures accordingly. I'll start with an administrative overview and try to answer your questions about grades, projects and participation. Once we have that out of the way, I'll talk about the course content and provide an introduction to the problems you'll be learning about and developing solutions to in this class. This introduction will be relatively high level for reasons I'll soon explain.

You can take the course for a letter grade or pass-fail. The grade breakdown hasn't changed from years past. The final project involves the most work, but I expect you to take the proposal seriously as it is a chance to work with me to define a project that is challenging but won't require more work than is reasonable. Your attendance in class is required except under the usual extenuating circumstances involving illness, death in the family, etc. Anticipating the inevitable unanticipated compelling exceptions, each of you has two passes for exigencies that fall outside the university rules.

• Class participation including presentation (30%)

• Project proposal due around midterm (20%)

• Project report due around finals week (50%)

Participation is important because each class will feature one of the scientists and engineers who run the experiments and analyze the resulting data or that build the scientific instruments and computer-systems infrastructure for making the necessary measurements and processing the data. You'll have one or two papers to read in preparation for each class, generally written by the scientist who'll be joining us for discussion. I don't expect you to completely comprehend each paper, and you're encouraged to ask any questions you have that might help you in understanding the papers. Nowadays systems neuroscience is fundamentally multi-disciplinary and it is a very rare individual whose knowledge spans all of the relevant fields.

Introductory Comments

The instructions for delivering a Nobel Prize lecture are simple: (a) talk about a subject connected with the work for which the prize was awarded, and (b) stick to the prescribed 45-minute duration. If there is more than one Laureate, they usually work out in advance who will say what with regards to the prize-winning work, so as to avoid repetition.

What you shouldn't do is use your brief time on stage to discredit a rival, especially if that person happens to be there to receive the Nobel Prize with you. And yet, that's exactly what the Italian anatomist Camillo Golgi did in 1906, when he launched into a malicious attack on his Medicine co-Laureate, the Spanish anatomist Santiago Ramón y Cajal.

Both scientists had developed staining methods that revealed the complex anatomy of the nervous system in exquisite detail. Working by candlelight in a hospital kitchen that he had converted into a laboratory, Golgi discovered a technique in the 1870s for impregnating brain and other tissue with a silver solution in such a way that made it possible to stain nerve cells black and view under the microscope:

Golgi viewed the nervous system as being a seamless, continuous network of interconnected cells, with nerve signals firing along in all directions. Cajal, on the other hand, proposed that the brain is composed of billions of individual cells, or neurons (a term coined by the German anatomist Heinrich Wilhelm Gottfried von Waldeyer-Hart), receiving information at one end and transmitting it in one direction along to the next cell.

Golgi used his 45 minutes of Nobel spotlight to attack Cajal's theory. His unrelenting efforts to find evidence discrediting the neuron theory, almost comical political antics and the lengths he went to avoid talking Cajal are legend. At the Nobel ceremony, Cajal refused to take the bait and opened his lecture with:

In accordance with the tradition followed by the illustrious orators honoured before me with the Nobel Prize, I am going to talk to you about the principal results of my scientific work in the realm of the histology and physiology of the nervous system.

Golgi might have thought that he won the battle on that day in Stockholm, but Cajal won the war. The neuron doctrine remains a fundamental principle for understanding the central nervous system. (SOURCE)

Here is a modern retrospective of Cajal's work [9] including samples of his detailed renderings of individual neurons and neural circuits that inspired generations of neuroscientists and continue to do so to this day: The histological slides and drawings of Cajal, by Pablo Garcia-Lopez, Virginia Garcia-Marin and Miguel Freire.

@article{Garcia-LopezetalFiN-10,
    author = {Pablo Garcia-Lopez and Virginia Garcia-Marin and Miguel Freire},
     title = {The histological slides and drawings of Cajal},
   Journal = {Frontiers in Neuroanatomy},
    volume = {4},
     issue = {9},
      year = {2010},
     pages = {1-16},
  abstract = {Ram\`{o}n} y Cajal's studies in the field of neuroscience provoked a radical change in the course of its history. For this reason he is considered as the father of modern neuroscience. Some of his original preparations are housed at the Cajal Museum (Cajal Institute, CSIC, Madrid, Spain). In this article, we catalogue and analyse more than 4,500 of Cajal's histological preparations, the same preparations he used during his scientific career. Furthermore, we catalogued Cajal's original correspondence, both manuscripts and personal letters, drawings and plates. This is the first time anyone has compiled an account of Cajal's enormous scientific production, offering some curious insights into his work and his legacy.},
}

In the decades following, scientists fleshed out the neuron theory, created mathematical models that accurately predicted the behavior of individual neurons [13], and mapped the behaviour of circuits in primary visual cortex back to the stimuli that gave rise to them [15, 14]. With the discovery neurotransmitters that propagate signals from one neuron to another by chemical means, it was believed we had ``cracked the code'', but there were many surprises to come including the discovery of signalling by purely electrical means [8]: A network of fast-spiking cells in the neocortex connected by electrical synapses, by Mario Galarreta and Shaul Hestrin.

@article{GalarretaandHestrinNATURE-99,
       author = {Mario Galarreta and Shaul Hestrin},
        title = {A network of fast-spiking cells in the neocortex connected by electrical synapses},
      journal = {Nature},
       volume = 402,
         year = 1999,
        pages = {72-75},
     abstract = {Encoding of information in the cortex is thought to depend on synchronous firing of cortical neurons. Inhibitory neurons are known to be critical in the coordination of cortical activity, but how interaction among inhibitory cells promotes synchrony is not well understood. To address this issue directly, we have recorded simultaneously from pairs of fast-spiking (FS) cells, a type of \gamma{}-aminobutyric acid (GABA)-containing neocortical interneuron. Here we report a high occurrence of electrical coupling among FS cells. Electrical synapses were not found among pyramidal neurons or between FS cells and other cortical cells. Some FS cells were interconnected by both electrical and GABAergic synapses. We show that communication through electrical synapses allows excitatory signalling among inhibitory cells and promotes their synchronous spiking. These results indicate that electrical synapses establish a network of fast-spiking cells in the neocortex which may play a key role in coordinating cortical activity},
}

The use of electron microscopy to see in more detail than allowed by conventional light microscopy had been an essential tool in all branches of biology including the field of neuroscience. In the last few decades, the technologies for both tissue preparation (generally referred to as staining) and higher-resolution, wider-field microscopes has advanced considerably [6], to the point where it became feasible to image relatively large blocks of neural tissue [25, 24]: Kevin Briggman's Slides for CS379C in Spring 2013:

@article{HelmstaedteretalNATURE-13,
    author = {Helmstaedter, Moritz and Briggman, Kevin L. and Turaga, Srinivas C. and Jain, Viren and Seung, H. Sebastian and Denk, Winfried},
     title = {Connectomic reconstruction of the inner plexiform layer in the mouse retina},
   journal = {Nature},
 publisher = {Nature Publishing Group},
    volume = 500,
     issue = 7461,
      year = 2013,
     pages = {168-174},
  abstract = {Comprehensive high-resolution structural maps are central to functional exploration and understanding in biology. For the nervous system, in which high resolution and large spatial extent are both needed, such maps are scarce as they challenge data acquisition and analysis capabilities. Here we present for the mouse inner plexiform layer\emdash{}the main computational neuropil region in the mammalian retina\emdash{}the dense reconstruction of 950 neurons and their mutual contacts. This was achieved by applying a combination of crowd-sourced manual annotation and machine-learning-based volume segmentation to serial block-face electron microscopy data. We characterize a new type of retinal bipolar interneuron and show that we can subdivide a known type based on connectivity. Circuit motifs that emerge from our data indicate a functional mechanism for a known cellular response in a ganglion cell that detects localized motion, and predict that another ganglion cell is motion sensitive.}
}

The gold standard for recording electrical activity is the so-called patch-clamp method which involves the insertion of a miniature pipette through the cell membrane in order to measure the voltage drop across membrane. This method is practical for only small circuits consisting of two or three neurons, but can be used to construct very accurate models: Reconstruction of an average cortical column in silico, by Mauritz Helmstaedter and C.P.J. de Kock and D. Feldmeyer and R.M. Bruno and B. Sakmann:

@article{HelmstaedteretalBRR-07,
        title = {Reconstruction of an average cortical column in silico},
       author = {M. Helmstaedter and C.P.J. de Kock and D. Feldmeyer and R.M. Bruno and B. Sakmann},
      journal = {Brain Research Reviews},
       volume = 55,
       number = 2,
         year = 2007,
        pages = {193-203},
     abstract = [The characterization of individual neurons by Golgi and Cajal has been the basis of neuroanatomy for a century. A new challenge is to anatomically describe, at cellular resolution, complete local circuits that can drive behavior. In this essay, we review the possibilities to obtain a model cortical column by using in vitro and in vivo pair recordings, followed by anatomical reconstructions of the projecting and target cells. These pairs establish connection modules that eventually may be useful to synthesize an average cortical column in silico. Together with data on sensory evoked neuronal activity measured in vivo, this will allow to model the anatomical and functional cellular basis of behavior based on more realistic assumptions than previously attempted.}
}

Probes consisting of a single electrode or an array of electrodes make it possible to record from larger numbers of electrons, but all of these methods are invasive and can damage cells producing atypical behavior. The next big breakthrough came about due to the confluences of two new technologies: two-photon microscopy [12, 5] and genetically encoded calcium imaging [26, 11]: Network anatomy and in vivo physiology of visual cortical neurons, by Bock, Davi D. and Lee, Wei-Chung Allen and Kerlin, Aaron M. and Andermann, by Mark L. and Hood, Greg and Wetzel, Arthur W. and Yurgenson, Sergey and Soucy, Edward R. and Kim, Hyon Suk and Reid, R. Clay:

@article{BocketalNATURE-11,
        title = {Network anatomy and in vivo physiology of visual cortical neurons},
       author = {Bock, Davi D. and Lee, Wei-Chung Allen and Kerlin, Aaron M. and Andermann, Mark L. and Hood, Greg and Wetzel, Arthur W. and Yurgenson, Sergey and Soucy, Edward R. and Kim, Hyon Suk and Reid, R. Clay},
      journal = {Nature},
    publisher = {Nature Publishing Group},
       volume = 471,
       number = 7337,
         year = 2011,
        pages = {177-182},
     abstract = {In the cerebral cortex, local circuits consist of tens of thousands of neurons, each of which makes thousands of synaptic connections. Perhaps the biggest impediment to understanding these networks is that we have no wiring diagrams of their interconnections. Even if we had a partial or complete wiring diagram, however, understanding the network would also require information about each neuron's function. Here we show that the relationship between structure and function can be studied in the cortex with a combination of in vivo physiology and network anatomy. We used two-photon calcium imaging to characterize a functional property--the preferred stimulus orientation--of a group of neurons in the mouse primary visual cortex. Large-scale electron microscopy of serial thin sections was then used to trace a portion of these neurons' local network. Consistent with a prediction from recent physiological experiments, inhibitory interneurons received convergent anatomical input from nearby excitatory neurons with a broad range of preferred orientations, although weak biases could not be rejected.}
}

Now all the pieces are in place to scale the process of data collection to record from large ensembles of neurons in the quest to understand complicated neural circuits. This quest is at the heart of the United States BRAIN Initiative, the European Union Human Brain Project [7] and the Allen Institute for Brain Science's MindScope Project [17]. It is also the quest of this class and the project I lead at Google.

Supplementary Notes

During the quarter, I maintain a notebook that provides corrections, explanations, references and technical details relevant to the lectures and classroom discussion. Here are some examples taken from my lab notebooks that are representative:

March 9, 2015

Here is a note that I sent to a number of my Stanford colleagues asking them to recommend the class to their students:

I'll be teaching cs379c Computational Models of the Neocortex again this spring, but with a different focus than in previous years. This generation of computer scientists will be the first in history to have access to brain data in sufficient quantity and quality for large-scale structural and functional connectomics, and this year I'm trying to attract computer science students and computer-savvy engineers and neuroscientists interested in tackling some of the machine-learning and signal-processing challenges in analyzing such data.

In collaboration with the Allen Institute for Brain Science (AIBS), HHMI Janelia Farm, Max Planck Institute for Medical Research and MIT, we are compiling EM (Electron Microscopy) datasets that will enable computer scientists to reconstruct the neural circuits for several model organisms, and co-registered activity recordings using calcium imaging (CI) from which we hope to glean algorithmic insights by fitting various artificial neural network models to account for observed input / output behavior.

We'll be working with two teams of scientists and engineers who are building the tools to acquire this data. We have several relatively-small (10TB) EM datasets (including ground truth) that students interested in circuit tracing (structural connectomics) can use in projects. Scientists at AIBS have volunteered to help students in understanding the data and technologies used to collect it. In addition, engineers from my team at Google will supply examples of algorithms that have worked well for us.

Inferring function from CI data is more challenging since until recently there haven't been good datasets to work with. We now have several such datasets provided by our collaborators that can be used in student projects. In addition, we'll be generating synthetic datasets for cortical circuits of 5-50K neurons using Hodgkin-Huxley models¹ developed at AIBS and EPFL. These models and their associated simulators provide a controlled environment in which to experiment with and evaluate machine-learning technologies for functional connectomics.

The prerequisites are basic high-school biology, good math skills, and familiarity with machine learning. Some background in computer vision and signal processing will be important for projects in structural connectomics. Familiarity with modern artificial neural network technologies is a plus for projects in functional connectomics. Please encourage your qualified students to consider taking the course. As an added incentive, I have a group of extraordinary scientists and engineers lined up to help make it a great course.

March 7, 2015

I wrote up some notes including a few papers and technical reports intended to help students better understand the challenges involved in structural and functional connectomics and the strategies we are deploying for addressing them at Google. The challenges are divided into three categories: (1) connectomics (circuits), (2) recordings (activity), and (3) analyses (function), where the last is the least well defined in terms of agreed-upon outcomes and priorities for pursuing them:

1. CIRCUITS: Here's a pretty reasonable extrapolation of existing and emerging technologies leading to economical whole-brain connectomics which I've excerpted from [19]. Check out the full document. I think the authors have done a good job including the front-runners as well as some of the most promising alternatives. The time frame for whole-brain connectomes run from the two to ten years, depending on the organism and technology. It's obviously much easier predicting how the technologies of incumbents like Zeiss will fare than the more exotic ideas coming out of the academic labs:

   Due to advances in parallel-beam instrumentation, whole mouse brain electron microscopic image acquisition could cost less than $100 million, with total costs presently limited by image analysis to trace axons through large image stacks. Optical microscopy at 50 to 100 nm isotropic resolution could potentially read combinatorially multiplexed molecular information from individual synapses, which could indicate the identities of the pre-synaptic and post-synaptic cells without relying on axon tracing. An optical approach to whole mouse brain connectomics may be achievable for less than $10 million and could be enabled by emerging technologies to sequence nucleic acids in-situ in fixed tissue via fluorescent microscopy. Novel strategies relying on bulk DNA sequencing, which would extract the connectome without direct imaging of the tissue, could produce a whole mouse brain connectome for $100k to $1 million or a mouse cortical connectome for $10k to $100k. Anticipated further reductions in the cost of DNA sequencing could lead to a $1000 mouse cortical connectome.

   We're putting most of our money on reconstruction from EM using current and soon-to-be-current technologies like the new Zeiss line of multi-beam microscopes that Winfried Denk is now working with while at the same time developing his extra-wide, perfect-crystal, whole-brain, serial-sectioning diamond-knife [6], but we are also placing side bets on Boyden's expansion-microscopy technology [3] which we believe is very promising and keeping close tabs on some of the work coming out of the Church [20] and Zador [18] labs.

2. ACTIVITY: This is an area full of opportunity with lots of new ideas and talent from complementary disciplines. We wrote a technical report on neural recording technologies that is still pretty current [4]. One of my colleagues Adam Marblestone — Ph.D. with George Church and currently a postdoc with Ed Boyden — corralled a group biologists, chemists, physiologists, physicists, electrical engineers, etc, to put together a somewhat more speculative — understandably so given the additional complexity in working with an awake, behaving organism — extrapolation that is definitely worth your time reading [21].

   For the time being, we are banking on calcium imaging as being the recording technology that is likely to scale to satisfy our requirements. The current GECIs have much improved response kinetics and signal amplitudes compared with earlier generations [16], the necessary GECI-expressing transgenic mouse lines already exist and the Allen Institute has world-class neuroscientists with expertise in working with them. There has been some work on miniature fluorescence microscopes suitable for mounting on the head of a mouse, thereby allowing the animal limited mobility [10], but so far the incumbent GECI and fixed-camera technologies seems way out in front in terms of scale and reliability.

3. FUNCTION: This is by far the least well explored of the three technical categories. The reason is pretty obvious: we have never had data on the scale that we anticipate from the Allen Institute MindScope Project. There has been speculation about the structure and function of cortical columns, but no compelling evidence to support any of the current hypotheses. We've been working with Costas Anastassiou and his team at AIBS in developing simulations of small portions of cortex consisting of 5,000-50,000 neurons, but this doesn't even account for a single cortical column. We could simulate much larger models at Google, but at this point in time it doesn't much matter, since, if we wanted to create a model of a small patch of cortex spanning multiple cortical columns using state-of-the-art neural modeling tools, we would be hard pressed to do so given our limited knowledge of cortical cell types, connectivity and dynamics.

   We're designing a series of progressively more difficult modeling challenges. The first couple of challenges involve learning the input-output functions of a set of artificial neural network (ANN) models. We don't pretend these models are necessarily good models of biological networks; however, if we can't learn a reasonably well-behaved network we've engineered, there isn't much sense in trying to learn a real neural network given all the unknowns associated with biological systems. The next set of experiments will make use of the models that Costas' team is developing. These models consist of networks of reconstructed, multi-compartmental, virtually-instrumented and spiking pyramidal neurons and basket cells, plus ion- and voltage-dependent currents and local field potentials so we can generate the same sort of rasters we expect to collect during calcium imaging. Once again we have a highly-controlled sandbox in which to evaluate machine-learning technologies.

   We hope to start getting recorded activity data from AIBS by end of summer if not sooner. We may get access to data from experiments carried out by Clay Reid while still at Harvard that we can play with while waiting for MindScope data. Given real data, the first order of business is to see if we can replicate the training data and generalize to the test data. Interpreting success will be challenging; the best anyone can do may be to capture summary statistics of the output or identify emergent, dynamical-system behaviour. In order to have any chance of reproducing spiking behavior, we may have to restrict our attention to smaller circuits assuming we can identify their boundaries. We'll also want to exploit any connectomic, proteomic or transcriptomic information we glean from the fixed and registered tissue after the activity-recording stage. Once we have mouse recordings, we are in terra incognita with much to learn.

March 5, 2015

It is easy to fall into the trap of thinking that just because it is possible to trace individual processes over substantial distances in dense neural tissue, tracing all of the neurons in an entire mammalian brain is just a matter of scale. The mouse brain has ~10⁸ neurons and ~10¹¹ synapses in a volume of ~500³ mm. Kilometers of neuronal wiring passes through any cubic millimeter of tissue and the relevant anatomical features are on the scale of 100 nm [19].

Accurate tracing of individual axonal processes is certainly possible using a number of laboratory techniques. Both anterograde (soma to synapse) and retrograde (synapse to soma) tracing that work by exploiting different methods of labeling and axonal transport are reasonably well developed but still require special care to administer. Some progress has been made by using retroviruses to propagate labels from one neuron to another [22, 2] and now there transsynaptic anterograde tracers, which can cross the synaptic cleft, labeling multiple neurons along an extended path [23, 1].

These methods suffer from the problem that, while an individual process is relatively easy to trace, once you label all or most of the processes in a given volume, you end up with many of the same problems that surface in tracing processes stained with conventional preparations using the sort of protocols and algorithms we've discussed elsewhere. Alternative methods that rely on propagating either unique or one of many distinguishable labels may provide a solution².

The Zador et al [27] method for attaching unique molecular molecular barcodes to each neuronal connection works by converting the problem of tracing connectivity into a form that can be solved by high-throughput DNA sequencing. A related method leveraging fluorescent in situ nucleic acid sequencing [18] offers similar functionality with more detailed annotation — additional markers for diverse molecules — and a theoretically simpler method for reading off the information encoded in the neural tissue [20].

For more on the technical details as well as other alternative technologies, take a look at the Dean et al report [4] produced by the Spring 2013 class of CS379C or the Marblestone et al report [21] describing the physical principles relevant to scaling neural recording.

Questions:

Prime directive in statistical modeling and machine learning is ``know your data''. What do you think this means and how might it apply in the case of the problems addressed in this class? Suppose you were charged with developing a classifier to sort freshly caught fish into different species, e.g., salmon, halibut, swordfish, etc., in preparation for separate processing and packing in a food-processing factory. How would you apply the prime directive to this problem?

Data collection and processing involves many different steps which are combined in a pipeline. Pipelines are serial and hence processing time and the probability of introducing errors and program crashes is often dominated by the weakest link in the chain. The same principles apply during technology development where the key factors involve coming up with a suitable solution for each step and writing the code. What do you think are choke points in the connectomics problem?

References