|Home | About | Journals | Submit | Contact Us | Français|
A fundamental question in neuroscience is how entire neural circuits generate behavior and adapt it to changes in sensory feedback. Here we use two-photon calcium imaging to record activity of large populations of neurons at the cellular level throughout the brain of larval zebrafish expressing a genetically-encoded calcium sensor, while the paralyzed animals interact fictively with a virtual environment and rapidly adapt their motor output to changes in visual feedback. We decompose the network dynamics involved in adaptive locomotion into four types of neural response properties, and provide anatomical maps of the corresponding sites. A subset of these signals occurred during behavioral adjustments and are candidates for the functional elements that drive motor learning. Lesions to the inferior olive indicate a specific functional role for olivocerebellar circuitry in adaptive locomotion. This study enables the analysis of brain-wide dynamics at single-cell resolution during behavior.
The generation of motor output and the influence of sensory input on future motor programs engage neural activity in many neurons across multiple brain regions. However, past measurements of neural activity during behavior have been hampered by the inability to exhaustively monitor all neurons in the brain of a behaving animal. Although it is possible to record activity from behaving animals1,2,3,4,5,6, the large size and opacity of the vertebrate brain constrains experimenters to focus on small fractions of the total number of neurons. Here, we develop a preparation in which neuronal activity can be monitored anywhere in the brain via two-photon calcium imaging in paralyzed larval zebrafish that interact with a virtual environment and adjust their behavior to changes in visual feedback.
When visual feedback following a motor command does not meet expectation, animals can learn to adapt the strength of subsequent motor commands. In the past this has been studied in controlled laboratory settings by perturbing visual feedback in the context of insect flight7,8,9, the vestibulo-ocular reflex10,11 and reaching movements12,13. Here we study adaptive control of locomotion in larval zebrafish14. This animal swims in discrete swim bouts during which the visual environment moves relative to its retina. One hypothesis is that this optic flow is used as a measure of displacement, and serves to tune the strength of future motor commands to the desired travel distance7,8,9. Such sensorimotor recalibration is especially important during the optomotor response (OMR)15,16, in which animals move in the direction of motion of the visual surround, thereby stabilizing their location in the presence of e.g. water flow, and which occurs in many animal species. If motor output is not correctly calibrated to visual feedback, the fish may systematically overshoot or undershoot the desired travel distance, instead of stabilizing its location. Sensorimotor recalibration is necessary for accurate locomotion because the rate of optic flow following a motor command is affected by temperature-dependent changes in muscle strength, viscosity of the water and distance of objects from the retina.
To examine neural dynamics across brain areas that drive sensorimotor recalibration, we developed a system to study neural activity at cellular resolution17,18 by two-photon microscopy19 anywhere in the brain20 during closed-loop optomotor behavior in larval zebrafish. These animals have a small and transparent brain which is readily accessible for optogenetic recording and stimulation21,22, electrophysiology23 and single-cell ablation24. To remove motion artifacts25,26 we developed a swim simulator for completely paralyzed larvae (Fig. 1a). Motor commands, or ‘fictive swims’, are recorded at the motor neuron level27,28,8 (Fig. 1c,d) and translated, in real time, into visual feedback that resembles the optic flow of freely swimming fish (Methods 1). This constitutes a fictively-driven virtual-reality setup. Simultaneously, a two-photon microscope scanning over a transgenic fish expressing GCaMP229 in almost all neurons30,20 allows activity to be monitored throughout the brain at single-neuron resolution. Since the experimenter is in complete control over the visual feedback, this allowed us to study neural dynamics during visually-guided motor adaptation throughout the brain at the cellular level.
To study motor adaptation, we used a closed-loop paradigm and simulated a one-dimensional environment in which the fish is swept backward by a virtual water flow, a motion the fish could compensate for by swimming forward, as in the OMR. In the fictive virtual-reality setup, this corresponds to a whole field visual stimulus that is moving forward, but can be momentarily accelerated backward by a fictive swim of the fish (Fig. 1b; Methods 1) so that the fish can stabilize its virtual location over time. Remarkably, paralyzed larval zebrafish behaved readily in this closed-loop paradigm, exhibiting similar behavior as freely swimming fish do when following whole-field motion24, and were not noticeably compromised by the absence of vestibular, proprioceptive and somatosensory feedback that accompanies unrestrained swimming.
An important free parameter in this closed-loop paradigm is the feedback gain31,12,10,11,14 — the factor that translates the strength of the fictive swim signal to the change in stimulus velocity (Methods 1). The higher the feedback gain, the larger the velocity change following a motor command (e.g. dashed red line in Fig. 1b), so that high feedback gain corresponds to a ‘strong virtual fish’ and low feedback gain to a ‘weak virtual fish’. Accurate motor control would require the motor output to adapt to the feedback gain. Periodically switching the feedback gain between low and high values in the virtual environment resulted in compensatory changes in motor output: a change to a lower gain resulted in the gradual increase of the amplitude and duration of the fictive swim signals (Fig. 1c; a weak fish sends more impulses to the muscles), whereas a switch to a higher gain setting led to an incremental decrease (a strong fish sends fewer impulses to the muscles). The duration of fictive swim bouts was modulated in an analogous manner. This behavior was tested in more detail via the scheme illustrated in Figure 1e which was repeated up to 50 times per fish. We analyzed the power of motor nerve bursts, equivalent to the number of fictive tail oscillations (Fig. 1d), in each of the first 12 swim bouts that occurred during 30 seconds of motor adaptation to either high or low feedback gain (Fig. 1e, phase I). The first swim bouts following a switch from low to high gain, or from high to low gain, were indistinguishable (Fig. 1f, t-test, p > 0.5). This implies that fish do not adjust their motor output once a motor command has been issued despite the presence of immediate visual feedback. Starting at the second swim bout, the number of bursts diverges in the high and low gain conditions (p < 10−4). Behavioral adjustment plateaus after about ten bouts, which corresponds to about 7-10 seconds (Fig. 1g; Suppl. Fig. S4).
To determine whether the larvae are learning a new sensorimotor transformation or merely responding to different patterns of visual stimulation during the high and low gain periods, fish were exposed to a ten second ‘rest’ period (Fig. 1e, II) during which constant velocity backward gratings were shown in open-loop, a stimulus that tends to inhibit swimming, followed by a closed-loop ‘test’ period of medium feedback gain (Fig. 1e, III). We found that the strength of the first swim bouts in the ‘test’ period was determined by the gain setting during the preceding adaptation session (I), which demonstrates that the retention of the increased or decreased locomotor drive outlasts ten seconds of fixed optic flow (Fig. 1h, dark vs. light histograms; Suppl. Fig. S5). Thus, motor adaptation in larval zebrafish is not merely a response to different patterns of visual stimulation, but instead involves a short-term learned change in the sensorimotor transformation.
After verifying that neural activity in the reticulospinal system is modulated by locomotor drive (Suppl. Fig. S6a-f), as suggested by previous studies24, we next looked for signals relating to adaptive motor control throughout the entire brain (Suppl. Video 2). We generated a transgenic fish expressing the genetically encoded calcium indicator GCaMP229 driven by the panneuronal elavl3 (previously known as HuC) promoter30,20 (Fig. 2a). We used a paradigm in which 30 seconds of high gain alternated with 30s low gain, without open-loop intermissions. Behavioral variables such as swim frequency, number of bursts, and power changed in an analogous manner (Suppl. Fig. S4c). A single Z plane was imaged for six repetitions of gain switches. The brain volume that can be covered in a single fish depends on the duration of the paradigm and the size of the imaging plane. With relatively short assays (about 2 minutes) the entire brain of single fish can be imaged in one experiment. We chose instead to use a longer assay — 10 minutes — to cope with our relatively complicated behavioral paradigm and the low signal-to-noise ratio of GCaMP2. Thus, we sampled on average 20% of each fish’s brain and created a composite brain for final analysis.
Data analysis was automated and carried out along the following lines. Every experiment generated a number of fluorescence movies with associated fictive swim recordings and information about the stimulus. A custom-written signal identification and localization algorithm extracted fluorescence time-series from single-neurons or 4×4 μm regions of neuropil (Fig. 2c-e; Methods 3). These fluorescence time series (Fig. 2e) were then related to the stimulus and behavioral traces (Fig. 2f) via methods described below. Finally, to identify the ROIs of multiple fish with anatomical loci in a reference brain, all imaged planes were mapped via an image-registration algorithm to a high resolution reference brain of a 6 dpf larva. Although small variations existed between brains, the multitude of landmarks made reasonable localization between different fish possible (within about 25μm, see methods 3).
To search the brain for neural activity related to motor output, we first needed to solve the problem that in closed-loop, one cannot easily distinguish motor- from visual-related activity, as both are directly linked in this setting. A period of open-loop stimulus presentation was added to the paradigm (yellow area in Fig. 2e,f) during which the stimulus experienced by the animal during a preceding closed-loop period (black bar in Fig. 2e) was repeated. Activity of visually-driven neurons during this ‘replay’ period will resemble the activity of the preceding period — formalized by using ccFF, the correlation coefficient of fluorescence during and before replay, as a measure of the degree of visually-driven activity — whereas activity of motor-related neurons will instead correlate to motor output — formalized by ccFM, the correlation coefficient between motor output and fluorescence during replay, as a measure of motor-related activity (Methods 5). High ccFM indicates activity related to locomotion (e.g. Fig. 2e), and high ccFF indicates visually-driven activity.
As in Fig. 2f, most fish’s swimming behavior became erratic during open-loop stimulus replay. This observation illustrates that the behavioral state of an animal depends strongly on the presence of appropriate sensory feedback following a motor command.
Figure 2g displays the density of neurons in the reference brain (N=32 fish) whose activity was strongly correlated to fictive motor output during replay (high ccFM, Methods 5). Clusters of such neurons can be seen in the caudal hindbrain (1) including the inferior olive, in the cerebellum (2), near the nucMLF and pretectum (3), and in the forebrain (4). Asymmetries in the anatomical maps may arise from the limited sensitivity of GCaMP2 and the limited sampling of the brain. Figure 2h displays the density of neurons whose activity correlates with visual input (high ccFF; Methods 5). Here we used a more liberal criterion because only a small fraction of all possible visual input could be sampled, so that this map is by necessity less complete. Nevertheless, densities were found in the area of the pretectum (4), the tectum (3), the cerebellum (2) and the hindbrain including the inferior olive (1). Thus, regions throughout the brain involved in locomotion could be identified via correlational analysis of neural activity during behavior.
As a first step toward understanding the dynamics that occur during motor adaptation, activity of all identified sites across all fish was visualized by embedding it in a three dimensional phase space via principal components analysis (PCA)32,33 (traces averaged over six low-high gain repetitions, see Methods 5).
As expected, the trajectory loops back to the starting point (Fig. 3a,c), reflecting the periodicity of the neural activity induced by the paradigm, which consisted of repeating periods of high and low feedback gain. The velocity through the principal component space is initially high after a change in feedback gain, then slows down (Fig. 3d) to reach one of two approximate-steady-states (β and δ in Fig. 3c). Notably, the periods of fast change in network space (α and γ) coincide with the period of behavioral change (figure 1g). Indeed, the first two temporal principal components shown in Fig. 3b reflect steady-state activity (TCP1) and transient activity (TCP2) after a decrease in feedback gain. In summary, network activity evolves quickly following a change in feedback gain, and then settles into one of two steady states depending on the setting of the feedback gain. Network changes coincide with changes in behavior, and steady network states correspond to periods of stable behavior. To determine what neural activity induces the two transient and the two steady phases, we next looked for neurons that showed correlated activity with these four phases, i.e. during the four phases α - δ in Fig. 3d.
Neurons exhibiting raised activity during the low-gain, high locomotor drive phase were termed ‘motor’ related neurons (average fluorescence Fβ > Fδ, paired t-test on six repetitions, p < 0.005). The neurons of Figs. 2e and and4a4a are two examples. Activity of these two neurons was more related to locomotion than to visual input, as determined by ccFM > ccFF during replay (Fig. 2e: ccFM = 0.58 > ccFF = 0.21, Fig. 4a: ccFM = 0.154 > ccFF = 0.052). This was the case for almost all members of the ‘motor’ population (Fig. 4f). The population average of ‘motor’ neurons activity is shown in figure 4e.
‘Motor’ units were found in areas shown in Fig. 5a: in the posterior hindbrain34,35 (white arrow), with an especially dense concentration just caudal to the cerebellum (yellow arrow), and in the inferior olive (black arrow). (2) In the reticulospinal system (Suppl. Fig. S6) as suggested by previous studies24,25. (3) Throughout the cerebellum (red), especially in the corpus, including in the areas of the Purkinje (larger, more dorsal cell bodies) and granule cell (smaller, deeper cell bodies) layers36,37,38 and in the deep cerebellum (more lateral). (4) In the midbrain, ventral to the optic tectum, near the nucMLF and in the pretectum (orange)39,38. (5) In the habenula (green). (6) In the pallium (blue).
Neuronal populations active after a decrease in gain may be responsible for driving the animal into a state of high locomotor drive. Signals during this period are also consistent with the detection of discrepancies between expected and received visual feedback, which are thought to drive many forms of adaptive behavior40,41,31,12,10,11. An example from the ‘gain-down’ population (Fα > Fβ , Fα > Fδ; two t-tests on six gain repetitions, p < 0.005), at the ventrolateral border of the cerebellum, is shown in figure 4b: this neuron exhibits transient activity, after a gain decrease, that returns to baseline while the increase in locomotor drive is maintained. The stimulus replay period (yellow region) shows that the neuron is not visually driven, but is instead functionally related to motor output (ccFM = 0.32 > ccFF = 0.15), as are most other neurons of this population (figure 4f).
‘Gain-down’ units were found in the cerebellum (red) and in the inferior olive (black) (Fig. 5b). In the cerebellum they appeared in both the areas of the Purkinje and granule cell layers, and in the deep cerebellum. In the dorsal areas they appeared medial (red), and in the deep cerebellum they appeared lateral (red with white outline). The anatomical localization of gain-down units to the cerebellum is consistent with findings in mammals, where the cerebellum is a locus of motor learning10,11. Some units were also found in the hindbrain just caudal to the cerebellum.
On the other hand, figure 4c shows calcium signals of a ‘gain-up’ neuron in the optic tectum which exhibits transient signals after an increase in gain (Fγ > Fβ, Fγ > Fδ). This neuron is mainly driven by visual input since the calcium trace during the replay period resembles the trace during the matched preceding period (ccFM = −0.21 < ccFF = 0.59, representative of the population, see Fig. 4f). Not many neurons with the ‘gain-up’ property were found (N=39), but a concentration existed in the inferior olive (black arrow), and most were visually-driven (Fig. 4f).
A fourth class of neurons exhibited raised activity during periods of weak locomotion or absence of locomotion, termed the ‘motor-off’ class (Fδ > Fβ). Figure 4d shows an example of such a neuron, in the dorsal hindbrain, whose activity is elevated during periods of high gain and low locomotor drive. Remarkably, during the stimulus replay period, the calcium signal still peaks during periods of no swimming, suggesting that this is motor-related neuron instead of a visually driven neuron (ccFM = −0.40 > ccFF = 0.18). It might be involved in inhibiting motor output, or in suppressing behaviors that should not be executed during vigorous swimming. Not all motor-off neurons had this property; some were more visually-driven (Fig. 4f).
‘Motor-off’ units were concentrated in the dorsal hindbrain (Fig. 5d, pink), in the cerebellum (red), in the inferior olive (black), in the ventral midbrain near the nucMLF and the pretectum (orange), and in the habenula and pallium (green, blue). For completeness, correlation-based maps of sites strongly correlating or anticorrelating with locomotion during ‘replay’, as measured via ccFM, are shown in Fig. 5e,f, are generally consistent with panels a and d, and reveal additional structure such as two arcs in the dorsal hindbrain (panel f, pink).
Population averages summarizing the above four types of neural dynamics are shown in figure 4e (individual traces are shown in Supplementary Figures S13-S16). The detected units were not false-positives resulting from noisy measurements of neural activity, as is confirmed by shuffling the fluorescence time-series, which causes an 8-fold drop in detected units (Fig. 4g, Methods 5). Fig. 4h-k shows that neurons generally only belong to one functional class, but there is some overlap between motor and gain-down (46% of gain-down units have significant sustained activity during low gain), and no overlap between gain-down and gain-up units, consistent with previously observed asymmetries between gain adaptation in opposite directions42. Supplementary Video 4 contains anatomical stacks with superimposed clusters of the functionally identified neural classes (same data as Fig. 5). Since GCaMP2 has relatively weak signal-to-noise ratio, and the brain was sampled a limited number of times, the anatomical maps are not exhaustive, so we quantified the uncertainty in detecting or missing functional units (Suppl. Fig. S10).
Thus, the four types of neural dynamics identified via a dimensionally-reduced representation of network activity could be mapped to distinct brain areas. This represents to the best of our knowledge the first brain-wide imaging at the cellular level of activity related to adaptive motor control in a vertebrate brain.
The ability to monitor neural activity at single cell resolution throughout the whole brain of a behaving animal creates new opportunities for studying circuit function during behavior. The demonstration that paralyzed larval zebrafish interact readily with a virtual environment, and the remarkable finding that these animals still exhibited short-term forms of motor learning in the fictive virtual-reality setup, provided an exciting opportunity to study the circuit dynamics occurring during this behavior.
Here we identified neural populations activated during specific phases of adaptive locomotion that span multiple areas of the larval zebrafish brain. Both the inferior olive and the cerebellum contained many neurons correlating with adaptive motor control. In mammals, cerebellar circuits play an important role in motor control11 and in fish the cerebellum has been shown to be involved in the selection of motor programs43,44,45. Furthermore, the structure of olivo-cerebellar circuitry in zebrafish is remarkably similar to that of mammals36,37,46 (Suppl. Fig. S23a) and the transient gain-down activity observed in the inferior olive and cerebellum may represent error signals driving motor learning mechanisms40,41,11,47.
To test whether the inferior olive is necessary for motor adaptation, we next lesioned it with an infrared laser24. Post-lesion, the power of swim bouts in the high- and low-gain settings became statistically indistinguishable (figure 6). Although damage to passing axons cannot be ruled out, similar lesions in the dorsal anterior hindbrain did not affect motor adaptation (p<0.001 pre-lesion; p=0.01 post-lesion; Fig. 6c and Suppl. Fig. S28). The optomotor response was still intact (Suppl. Fig. S24). These results indicate that the inferior olive is necessary for successful adaptation of motor programs to external feedback gain. One hypothesis is that an error signal is computed in the inferior olive via subtractive interaction of an efference copy of motor output (e.g. via inhibitory connections from the deep cerebellum) and visual feedback from a swim bout (e.g. via pretectal projections; see also supplementary figure S27), which then activates appropriate circuits in the cerebellum via climbing fibers. Cerebellar activity may drive changes in motor programs via the deep cerebellum, which in mammals projects to premotor circuits.
While synaptic plasticity underlies much of motor learning11,12, it is only one of multiple candidate mechanisms for the behavior observed here. An alternative hypothesis is that sustained firing rates of neuronal populations, perhaps subsets of the motor and motor-off populations, implement and maintain over prolonged periods the different levels of locomotor drive via e.g. attractor states48.
The function of the observed adaptive sensorimotor behavior may be multifold. On long timescales, development of muscle and body shape will require continuous adjustment of sensorimotor control. On medium timescales, fluctuating body mass due to eating, and fluctuating viscosity of the water require some adaptation of swimming behavior. In addition, temperature fluctuations cause changes in muscle efficacy49 that must be counterbalanced by adjustments of locomotor drive. On short timescales, the relationship between the speed of optic flow following a swim bout depends on the distance to the optical surround (objects farther away induce smaller optic flow), requiring recalibration of the sensorimotor loop on the timescale of seconds. In our experiments, we observed adaptation occurring on such a timescale. Human motor control faces many similar challenges and is subject to continuous recalibration to cope with changing conditions (e.g. leg injury; walking on a slippery floor; carrying a heavy bag). Thus, the current study of brain-wide activity during adaptive locomotion represents an important step for understanding entire circuits in the precise context for which they evolved: the flexible control of behavior in changing environments.
Wildtype WIK larvae were used for behavioral experiments; for reticulospinal imaging, nacre fish on a WIK background were used; for two-photon imaging, nacre fish expressing GCaMP2 under control of the elavl3 30,20 promoter were used (previously known as HuC), again on a WIK background. All experiments were approved by Harvard University’s Standing Committee on the Use of Animals in Research and Training. Zebrafish larvae ages 6 to 7 dpf were anesthetized with MS222 and paralyzed by injection with a 1mg/ml bungarotoxin solution (Sigma-Aldrich), then suspended from structural pipettes (Suppl. Fig. S3), or embedded in agarose after which the agarose around the tail was removed. Motor nerve recordings were made with a Multiclamp 700B amplifier, simultaneously with two-photon imaging. Experiments were done at room temperature in filtered facility fish water. Visual scenes were projected onto a diffusive screen underneath the petri dish containing the fish via a mini projector, whose light source was replaced by a red Luxeon Rebel LED that was pulsed in synchrony with the fast scan mirror, so that illumination only occurred at the edges of the image where the scan mirror changed direction (typically at 800 Hz) to avoid any corruption of the two-photon images. Visual scenes consisted of square gratings with spatial period 12mm moving at 1cm/s from tail to head in the absence of motor nerve signals (−1 cm/s). When the processed swim signal was above an automatically set threshold (see Suppl. Meth. 1.3 and Suppl. Fig. S2), the locomotor drive was defined as the area underneath the curve of the processed swim signal during the current and previous video frame. The processed swim signal was defined to be the standard deviation of the raw swim signal in a sliding window of 15ms (see Fig. 1d). In the presence of such motor nerve signals, the instantaneous virtual fish velocity was set, 60 times per second (at the rate of the 60Hz projector), to −1cm/s + [gain] × [instantaneous locomotor drive], where the gain was set experimentally, after which the velocity decayed back to −1cm/s at a rate of 15cm/s2, approximately matched to freely swimming fish dynamics (Supplementary figure S1). The high gain was chosen to be two to five times higher than the low gain and these values bracketed the ‘natural’ gain setting that described the transformation of motor activity into optic flow in a freely swimming fish. The high- and low-gain settings were manually adjusted for each fish, as different fish showed different ranges of adaptability. Some fish exhibited a transient increase in fictive motor output followed by a decrease after a gain-down change; these fish were discarded from the gain-down dataset because transient neural activity could not be distinguished from motor-related activity (rejection criterion: p < 0.03, paired t-test on fictive signal averages over seconds 0-15 versus averages over seconds 15-30 after gain-down change).
A custom built laser scanning two-photon microscope, utilizing a Spectra Physics Mai Tai pulsed infrared laser, was used to monitor fluorescence in the brain. A PMT with a green bandpass filter and a laser blocking filter monitored green fluorescence. The fish was illuminated from below by a near-infrared LED light source whose light path was combined with the light path of the projector via a dichroic mirror (a ‘hot mirror’, Edmund Optics; not shown in Fig. 1). The image of the fish was registered above the objective via another dichroic mirror by a camera (AVT Stingray). Areas of the brain were scanned at 1.5 to 3 Hz, typically with 4-8 μm between imaging planes. Fish typically showed robust behavior for a period of 3-5 hours, so that not the entire brain of every fish could be scanned (although near 100% coverage was possible in some fish). 32 fish were used to non-homogeneously cover 6 times the volume of the brain (see Supplementary Figure S9). The electrophysiology, stimulus presentation and the two-photon microscope were controlled by a single piece of software custom written in C# (Microsoft). Two data acquisition cards (National Instruments) were used to acquire imaging and electrophysiology data, and were synchronized by periodic digital pulses.
Image analysis software was written in Matlab (Mathworks). We developed a novel method for automatically extracting regions of interest from two-photon movies that contain many neurons. A square ROI, half the size of a neuron, is swept over all locations of the imaging plane. At each location, a fluorescence time-series, averaged over the ROI, is extracted and converted to a statistic for the ‘peaky-ness’ of the fluorescence signal at that point. If fx,y(t) is the fluorescence of the pixel at x, y at time t, and fx,y is the average fluorescence at pixel x, y, and f is the average of fx,y, then the statistic is defined to be , where denotes the spatial average over the square ROI and means average over time. This measure was chosen because it bears resemblance to the usual ΔF/F but contains an offset to counteract the undesired amplification of noise in areas of low fluorescence; the third power was chosen because it nonlinearly converts peaks in the fluorescence signal to larger values of the statistic. This measure — one of several tested — yielded spatial signal maps mx,y which tended to be at least as sensitive as sets of ROIs selected manually from observation of the raw and ΔF/F movies. The maps also tended to be more complete than ROIs obtained via PCA-ICA methods50, which detected large events in which many neurons participated, but often missed small, sparse activation of single neurons. Thus, a spatial map of activity is obtained as in the top panel of Fig. 2c. This map is smoothed by convolution with a Gaussian the size of a neuron, and the maxima designated as points of interest. Points less than a neuron distance from one another, including ‘chains’ of such points, as sometimes occurred in regions of activated neuropil, were pruned to a single point at the center via an automatic procedure. These points of interest formed the center of new ROIs and were found to be positioned at the center of neurons in the majority of cases (e.g. red circle in Fig. 2c). The fluorescence time-series of these ROIs formed the basis of the functional classification. The time-series could be correlated with the time series of all pixels in a region around the point of interest, leading to a spatial map of correlation coefficients (Fig. 2d, top panel). This method extracted shapes of single neurons because pixel values within a single neuron tend to correlate highly over time. In many cases, the correlational maps also revealed other neurons whose activity was highly correlated with the neuron at the point of interest (Suppl. Fig. S7).
An entire 6 dpf larval brain (Tg(elavl3:GCaMP2), fish treated with 1-phenyl 2-thiourea [PTU] to inhibit eye pigmentation) brain was scanned and used as a reference brain. Individual imaged planes of other fish were registered onto the reference brain as follows (software was written in Matlab). The reference stack was intensity-normalized per Z-plane, and thresholded to dampen the effect of brightly labeled neurons on the registration. The peak of the cross-correlation between the mean-subtracted Z-planes reference stack and a mean-subtracted image was calculated for each Z-plane of the reference stack, giving the most likely X and Y positions for that Z plane (Fig. 2b, Suppl. Fig. S8, and Supplementary Movie 3). The Z location was chosen to be the peak of these correlation values, which fixed the X, Y and Z positions in the entire stack. Manual inspection showed that 82% of images were correctly registered; the remaining images were localized manually. To estimate the accuracy of localization, the reference brain (A) was used as well as a second brain (B). Five hundred times, a plane (250 by 270 μm) was randomly selected from brain A, and mapped to brain B by the algorithm. Next, the corresponding plane in brain B was mapped back to A. Thus the original location could be compared to the location after two mappings by the algorithm; thus, 500 discrepancies were obtained. The uncertainty in the mapping algorithm is then approximated by the standard deviation of these 500 discrepancies, divided by (because two errors accumulate by the double mapping). This was 19.5μm, so we assumed the slightly higher value of 25μm for the precision of the algorithm. In 14% of cases, a gross error occurred >60μm. These were easily spotted by eye (cf. the manual mapping of our data after gross discrepancies above). The unidirectional gross error rate was therefore estimated to be 7%.
Since each fish contributed a partial brain to the dataset, a total volume of 6 times the volume of the brain was obtained (assuming a 4μm z-resolution), with certain areas covered more densely than others (see Suppl. Figs. S9,S10). The density maps of figure 5 were normalized by the sampling density in order not to bias the maps toward the most densely sampled areas. Roughly 1% of all neurons were classified as being active by our activity detection algorithm. Of these, under our statistical criteria, about 20% could be related to the behavior by classification into the four groups described in the text. All major clusters in figure 5 appeared in more than one fish; units with no unit within the same class from at least one different fish within a 50μm sphere were removed.
After extracting fluorescence time-series from the ROIs (Methods 3), the traces were analyzed in more detail. To distinguish neural activity driven by visual input from activity relating to locomotion, we first smoothed and combined the left and right fictive channel recordings to generate a single quantitative descriptor of motor output, M. The motor output and the fluorescence trace were cross-correlated during the ‘replay’ period (see text) to yield a fluorescence-motor correlation coefficient ccFM. The correlation coefficient of the fluorescence trace during and before replay was termed ccFF. High values of ccFM indicate neural activity relating to locomotion, and high values of ccFF indicate visually-driven activity (since the stimulus during ‘replay’ is a repetition of the preceding stimulus; visually driven neurons should thus respond in a similar way). In the ccFF versus ccFM histograms (Fig. 4f), there is a region in the center that cannot be distinguished from noise, which was derived from a fish that did not receive visual input and that did not swim (Suppl. Figs. S17 and S18), and which is therefore left blank in the figures.
For dimensionality reduction, principal components of the matrix containing the fluorescence traces of all detected sites (size Nsites × Ntime points) were found using the princomp function in Matlab, and consisted of vectors of length Nsites. Activity traces from all detected sites, averaged over the six low-high gain repetitions, were projected (by the dot product) onto the first three of these principal components, to obtain three time-series (length Ntime points). These time-series served as a ‘summary’ of activity of all sites across brain regions and fish, since they can be used to approximately reconstruct (via the principal components) the activity of the detected sites. The three time-series were then visualized by plotting them on three axes, rendering a three-dimensional curve as shown in figure 3.
The ‘motor’, ‘gain decrease’, ‘gain increase’ and ‘motor-off’ activity patterns were detected as described in the text. Since a large number of units (N=9814 over 32 fish) contributed to the dataset, a concern is that the detected units are a result of false positives, arising from randomly fluctuating signals being classified purely by chance in one of the four functional categories. To address this, we applied a shuffle test, in which fluorescence traces were cut at 16 random time points and randomly rearranged, after which analysis proceeded as normal. The fact that the number of units in the four categories fell by a factor of 8.1 (Fig. 4g) indicates that the detected units did not arise by chance.
The ‘motor’-… ‘motor-off’-based description of neural activity (Fig. 5a-d), and the ccFM- and ccFF-based description (Fig. 5e-f), are complementary, with the first being derived from the gain-adaptation assay, and the second from the ‘replay’ period, and both being useful functional descriptors of neural activity.
IO lesions were performed by preselecting about 60 sites in an averaged two-photon image, then shining an infrared laser on them at 850nm, at 900mW power outside the cage. The pulses lasted 200ms per site. During the exposure, the laser beam spiraled over a circle of 1μm24. Brief but large increases in emitted light intensity indicated a successful lesion; in the absence of such signals, the site was exposed once more, and abandoned if the second attempt failed.
We are grateful to David Schoppik for teaching M.B.A. fictive swimming techniques, to Kuo-Hua Huang for doing spinal calcium green injections, and to Miguel Concha, Robert Baker and Leung-Hang Ma for advice on anatomy. We thank Markus Meister, Bence Ölveczky, Daniel Wolpert, Eran Mukamel, Michael Yartsev, David Schoppik, David Hildebrand, Eva Naumann, Adam Kampff, Peter Latham, Timothy Dunn, and members of the Engert lab for useful discussions and comments on the manuscript. We thank Pablo Oteiza and Renate Hellmiss for help with anatomy and figures, to Alain Viel for use of lab space, and to reviewers for helpful suggestions. M.B.A. thanks Daniel Wolpert and Etenilza Santos for limitless support. This work was supported by a Sir Henry Wellcome Fellowship from the Wellcome Trust (M.B.A.), a K99 grant no. 5K99NS62780-2 (M.B.O.), and NIH grants 5R01EY014429 and RC2NS069407 (F.E.).
Author contributions. M.B.A. developed the fictive virtual-reality paradigm, did the experiments, analyzed the data, and built the setup/software. M.B.A., F.E. and R.P. conceived of the experiments. M.B.O., D.N.R., J.M.L., and A.F.S. generated the Tg(elavl3:GCaMP2) fishline. M.B.O. generated the Tg(alpha tubulin:C3PA-GFP) fishline. All authors discussed the data and the manuscript. M.B.A. wrote the manuscript with assistance of R.P., M.B.O., and F.E.