Jack Gallant

Professor

University of California, Berkeley
Country flagUnited States
ApplyKite TwinLearns as you apply

Make your next move.

Prepare your application

Review drafts before you send them.

Explore related searches

Articles (11)

An essential aspect of human cognition is the ability to explicitly think about semantic relations between concepts. Neuroimaging studies have found that individual concepts are encoded by distributed patterns of cortical activity, but relatively little is known about how semantic relations between concepts are encoded in the brain. Some theoretical models suggest that relation representations are embedded within concept representations, while others suggest that relation representations are independent of any specific concept pair. We designed a study to compare how semantic relations and concepts are encoded across the cerebral cortex. To characterize how relations are encoded across cortex, fMRI was used to record brain activity while six participants each answered over one thousand questions about different semantic relations. We find that relations are encoded independently of the specific concepts that are connected in any particular instance of the relation. Our results further suggest that relations and concepts are represented in the same set of cortical regions, and that, within these regions, each location is preferentially selective for specific relations. Overall, these results suggest that in the human cerebral cortex, relations and concepts may have the same type of functional representation.

Year:

2026

Encoding models using word embeddings or artificial neural network (ANN) features reliably predict brain responses to naturalistic stimuli, yet interpreting these models remains challenging. A central limitation is superposition: distinct semantic features become entangled along correlated directions in dense embeddings when latent features outnumber embedding dimensions. This entanglement renders regression weights non-identifiable—different combinations of semantic directions can produce identical predictions, precluding principled interpretation of voxel selectivity. To address this, we introduce the Sparse Concept Encoding Model, which transforms dense embeddings into a higher-dimensional, sparse, non-negative space of learned concept atoms. This transformation yields an axis-aligned semantic basis where each dimension corresponds to an interpretable concept, enabling direct readout of conceptual selectivity from voxel weights. When applied to fMRI data collected during story listening, our model matches the prediction performance of conventional dense models while substantially enhancing interpretability. It enables novel neuroscientific analyses such as disentangling overlapping cortical representations of time, space, and number, and revealing structured similarity among distributed conceptual maps. This framework offers a scalable and interpretable bridge between ANN-derived features and human conceptual representations in the brain.

Year:

2025

One of the major goals of cognitive neuroscience is understanding how the brain represents information about its own internal states and about the external world. This goal can be addressed by creating encoding models that reveal the information represented explicitly in measured brain activity. Here we describe the Voxelwise Encoding Model (VEM) framework for creating encoding models with functional magnetic resonance imaging (fMRI) data. The VEM framework provides several key advantages over traditional neuroimaging approaches. First, the VEM framework is applicable to most experimental designs, from classic factorial designs to complex naturalistic experiments such as movie watching or video games. This flexibility enables researchers to study brain function across multiple domains with the same analytical approach. Second, hypotheses about functional representations are defined and tested quantitatively by extracting feature spaces from experimental stimuli or tasks. These feature spaces quantify specific types of information potentially represented in brain activity and can range from simple human-derived labels to complex features generated by deep neural networks. If a feature space can be used to linearly predict brain activity, the brain is considered to explicitly represent features within that feature space. This prediction approach provides a systematic way to test hypotheses about functional representations. Third, the VEM framework implements robust data science methods to improve model estimation and minimize false positive results. Encoding models are estimated on a training set and validated on an independent test set. Testing in an independent dataset provides direct evidence that experimental findings generalize beyond the dataset used for model estimation. Finally, voxelwise encoding models can be created in each participant's native brain space without unnecessary information loss due to spatial averaging or template resampling required by conventional group analyses. This enables the VEM approach to reveal fine-grained functional organization in individual participants that might otherwise be ignored. In this article, we provide a comprehensive guide to the Voxelwise Encoding Model framework through all phases of research, from experimental design and data collection to the estimation and interpretation of voxelwise encoding models.

Year:

2025

One of the major goals of cognitive neuroscience is understanding how the brain represents information about its own internal states and about the external world. This goal can be addressed by creating encoding models that reveal the information represented explicitly in measured brain activity. Here we describe the Voxelwise Encoding Model (VEM) framework for creating encoding models with functional magnetic resonance imaging (fMRI) data. The VEM framework provides several key advantages over traditional neuroimaging approaches. First, the VEM framework is applicable to most experimental designs, from classic factorial designs to complex naturalistic experiments such as movie watching or video games. This flexibility enables researchers to study brain function across multiple domains with the same analytical approach. Second, hypotheses about functional representations are defined and tested quantitatively by extracting feature spaces from experimental stimuli or tasks. These feature spaces quantify specific types of information potentially represented in brain activity and can range from simple human-derived labels to complex features generated by deep neural networks. If a feature space can be used to linearly predict brain activity, the brain is considered to explicitly represent features within that feature space. This prediction approach provides a systematic way to test hypotheses about functional representations. Third, the VEM framework implements robust data science methods to improve model estimation and minimize false positive results. Encoding models are estimated on a training set and validated on an independent test set. Testing in an independent dataset provides direct evidence that experimental findings generalize beyond the dataset used for model estimation. Finally, voxelwise encoding models can be created in each participant's native brain space without unnecessary information loss due to spatial averaging or template resampling required by conventional group analyses. This enables the VEM approach to reveal fine-grained functional organization in individual participants that might otherwise be ignored. In this article, we provide a comprehensive guide to the Voxelwise Encoding Model framework through all phases of research, from experimental design and data collection to the estimation and interpretation of voxelwise encoding models.

Year:

2025

Each person experiences the world through a unique conceptual lens, shaped by personal experiences, natural variations, or disease. These individual differences have remained largely inaccessible to cognitive neuroscience and clinical neurology, limiting the development of precision medicine approaches to cognitive disorders. To overcome this limitation, here we develop a new statistical framework to measure and interpret individual differences in functional brain representations. We apply this framework to characterize how different individuals represent the same concepts. Twenty-four participants listened to narrative stories while their brain activity was measured with functional MRI (fMRI). Encoding models were used to recover how hundreds of concepts were represented in each person’s brain. Despite listening to identical stories, participants showed systematic individual differences in conceptual representations. These differences reveal person-specific biases in how concepts are represented in the brain. Individual variability was highest in regions that represent social information. Because these regions are thought to integrate sensory information with personal beliefs and experiences, the observed individual differences may reflect cognitive traits unique to each person. Our work reveals that individual differences are a systematic, measurable principle of conceptual representations in the human brain. By enabling researchers to measure and interpret differences in person-specific functional brain representations, our work establishes a new paradigm for precision neuroscience. This paradigm provides a rigorous foundation for developing fMRI applications in precision medicine to diagnose and monitor cognitive disorders.

Year:

2025

The Voxelwise Encoding Model framework (VEM) is a powerful approach for functional brain mapping. In the VEM framework, features are extracted from the stimulus (or task) and used in an encoding model to predict brain activity. If the encoding model is able to predict brain activity in some part of the brain, then one may conclude that some information represented in the features is also encoded in the brain. In VEM, a separate encoding model is fitted on each spatial sample (i.e., each voxel). VEM has many benefits compared to other methods for analyzing and modeling neuroimaging data. Most importantly, VEM can use large numbers of features simultaneously, which enables the analysis of complex naturalistic stimuli and tasks. Therefore, VEM can produce high-dimensional functional maps that reflect the selectivity of each voxel to large numbers of features. Moreover, because model performance is estimated on a separate test dataset not used during fitting, VEM minimizes overfitting and inflated Type I error confounds that plague other approaches, and the results of VEM generalize to new subjects and new stimuli. Despite these benefits, VEM is still not widely used in neuroimaging, partly because no tutorials on this method are available currently. To demystify the VEM framework and ease its dissemination, this paper presents a series of hands-on tutorials accessible to novice practitioners. The VEM tutorials are based on free open-source tools and public datasets, and reproduce the analysis presented in previously published work.

Year:

2025

The Voxelwise Encoding Model framework (VEM) is a powerful approach for functional brain mapping. In the VEM framework, features are extracted from the stimulus (or task) and used in an encoding model to predict brain activity. If the encoding model is able to predict brain activity in some part of the brain, then one may conclude that some information represented in the features is also encoded in the brain. In VEM, a separate encoding model is fitted on each spatial sample (i.e., each voxel). VEM has many benefits compared to other methods for analyzing and modeling neuroimaging data. Most importantly, VEM can use large numbers of features simultaneously, which enables the analysis of complex naturalistic stimuli and tasks. Therefore, VEM can produce high-dimensional functional maps that reflect the selectivity of each voxel to large numbers of features. Moreover, because model performance is estimated on a separate test dataset not used during fitting, VEM minimizes overfitting and inflated Type I error confounds that plague other approaches, and the results of VEM generalize to new subjects and new stimuli. Despite these benefits, VEM is still not widely used in neuroimaging, partly because no tutorials on this method are available currently. To demystify the VEM framework and ease its dissemination, this paper presents a series of hands-on tutorials accessible to novice practitioners. The VEM tutorials are based on free open-source tools and public datasets, and reproduce the analysis presented in previously published work.

Year:

2025

The Voxelwise Encoding Model framework (VEM) is a powerful approach for functional brain mapping. In the VEM framework, features are extracted from the stimulus (or task) and used in an encoding model to predict brain activity. If the encoding model is able to predict brain activity in some part of the brain, then one may conclude that some information represented in the features is also encoded in the brain. In VEM, a separate encoding model is fitted on each spatial sample (i.e., each voxel). VEM has many benefits compared to other methods for analyzing and modeling neuroimaging data. Most importantly, VEM can use large numbers of features simultaneously, which enables the analysis of complex naturalistic stimuli and tasks. Therefore, VEM can produce high-dimensional functional maps that reflect the selectivity of each voxel to large numbers of features. Moreover, because model performance is estimated on a separate test dataset not used during fitting, VEM minimizes overfitting and inflated Type I error confounds that plague other approaches, and the results of VEM generalize to new subjects and new stimuli. Despite these benefits, VEM is still not widely used in neuroimaging, partly because no tutorials on this method are available currently. To demystify the VEM framework and ease its dissemination, this paper presents a series of hands-on tutorials accessible to novice practitioners. The VEM tutorials are based on free open-source tools and public datasets, and reproduce the analysis presented in previously published work.

Year:

2025

V4 is a key area within the visual processing hierarchy, and it represents features of intermediate complexity. However, no current computational model explains V4 responses under natural conditions. To address this, we developed a new hierarchical convolutional energy (HCE) model reflecting computations thought to occur in areas V1, V2, and V4, but which consists entirely of simple- and complex-like units like those found in V1. In contrast to prior models, the HCE model is trained end-to-end on neurophysiology data, without relying on pre-trained network features. We recorded 313 V4 neurons during full-color nature video stimulation and fit the HCE model to each neuron. The model’s predicted optimal patterns (POPs) revealed complex spatiotemporal pattern selectivity in V4, supporting its role in representing space, time, and color. These findings indicate that area V4 is crucial for image segmentation and grouping operations that are essential for complex vision. Thus, responses of V4 neurons under naturalistic conditions can be explained by a hierarchical three-stage model where each stage consists entirely of units like those found in area V1.

Year:

2024

Deep brain stimulation (DBS) has garnered widespread use as an effective treatment for advanced Parkinson’s Disease (PD). Conventional DBS (cDBS) provides electrical stimulation to the basal ganglia at fixed amplitude and frequency, yet patients’ therapeutic needs are often dynamic with residual symptom fluctuations or side-effects. Adaptive DBS (aDBS) is an emerging technology that modulates stimulation with respect to real-time clinical, physiological, or behavioral states, enabling therapy to dynamically align with patient-specific symptoms. Here, we report a novel aDBS algorithm intended to mitigate movement slowness by delivering targeted stimulation increases during movement using decoded motor signals from the brain. Compared to cDBS and a control algorithm that decreases stimulation during movement, our approach demonstrates enhanced clinical efficacy, with improvements in movement speed, reduced dyskinesia, and better patient-reported therapeutic quality. Furthermore, we demonstrate proof-of-principle of a machine learning pipeline capable of remotely optimizing aDBS parameters in a home setting. This work illustrates the potential of movement-responsive aDBS as a new therapeutic approach and highlights how machine learning assisted programming can simplify optimization to facilitate translational scalability.

Year:

2024

Year:

2024

Collaborators (3)

Jeffrey Herron

Associate Professor

University of Washington

UNITED STATES
View details

Simon Little

Assistant Professor Neurology

University of California, San Francisco

UNITED STATES
View details

Fatma Deniz

Professor

Technische Universität Berlin

GERMANY
View details