Sanjay Shakkottai

Professor

University of Texas at Austin
Country flagUnited States

Research Interests

ApplyKite TwinLearns as you apply

Make your next move.

Prepare your application

Review drafts before you send them.

Explore related searches

About

Professor Sanjay Shakkottai is a faculty member at the University of Texas at Austin. His research areas include stochastic optimization, multi-agent systems, and bandit algorithms, as reflected in his recent publications on topics such as adaptive algorithms for Markovian settings and robust multi-agent bandit frameworks. His work aims to advance the understanding and application of learning algorithms in various stochastic environments.

Articles (12)

Year:

2026

We study a variant of the contextual bandit problem where an agent can intervene through a set of stochastic expert policies. Given a fixed context, each expert samples actions from a fixed conditional distribution. The agent seeks to remain competitive with the “best” among the given set of experts. We propose the Divergence-based Upper Confidence Bound (D-UCB) algorithm that uses importance sampling to share information across experts and provide horizon-independent constant regret bounds that only scale linearly in the number of experts. We also provide the Empirical D-UCB (ED-UCB) algorithm that can function with only approximate knowledge of expert distributions. Further, we investigate the episodic setting where the agent interacts with an environment that changes over episodes. Each episode can have different context and reward distributions resulting in the best expert changing across episodes. We show that by bootstrapping from \(\mathcal {O}(N\log (NT^2\sqrt {E}))\) samples, ED-UCB guarantees a regret that scales as \(\mathcal {O}(E(N+1) + \frac{N\sqrt {E}}{T^2})\) for N experts over E episodes, each of length T . We finally empirically validate our findings through simulations.

Year:

2024

The stochastic approximation (SA) method stands as the foundational mathematical tool for modern large-scale optimization and machine learning. Therefore, gaining a theoretical understanding of SA algorithms is of fundamental interest. In their paper titled “A Lyapunov Theory for Finite-Sample Guarantees of Markovian Stochastic Approximation,” Chen et al. present a unified Lyapunov framework for the finite-sample analysis of a Markovian SA algorithm under a contractive operator with respect to an arbitrary norm. The key novelty lies in the construction of a smooth Lyapunov function called the generalized Moreau envelope. The authors demonstrate the effectiveness of their SA results in the context of reinforcement learning (RL), specifically through popular algorithms such as variants of temporal difference (TD) learning and Q-learning. As byproducts, the results provide theoretical insights into the efficiency of bootstrapping in TD learning with eligibility traces and the bias-variance tradeoff in off-policy learning.

Year:

2024

Year:

2024

Year:

2023

Year:

2023

Year:

2023

Year:

2022

Year:

2022

Year:

2021

Year:

2021

Year:

2020

Collaborators (1)

Siva Theja Maguluri

Associate Professor

Georgia Institute of Technology

UNITED STATES
View details