Bum Jun Kim

Researcher

University of Tokyo
Country flagJapan

Research Interests

ApplyKite TwinLearns as you apply

Make your next move.

Prepare your application

Review drafts before you send them.

Explore related searches

Articles (18)

Year:

2026

Year:

2026

Year:

2026

Year:

2026

Year:

2026

Year:

2025

Year:

2025

Year:

2025

L 2 regularization for weights in neural networks is widely used as a standard training trick. In addition to weights, the use of batch normalization involves an additional trainable parameter γ, which acts as a scaling factor. However, L 2 regularization for γ remains an undiscussed mystery and is applied in different ways depending on the library and practitioner. In this article, we study whether L 2 regularization for γ is valid. To explore this issue, we consider two approaches: (1) variance control to make the residual network behave like an identity mapping and (2) stable optimization through the improvement of effective learning rate. Through two analyses, we specify the desirable and undesirable γ to apply L 2 regularization and propose four guidelines for managing them. In several experiments, we observed that applying L 2 regularization to applicable γ increased 1% to 4% classification accuracy, whereas applying L 2 regularization to inapplicable γ decreased 1% to 3% classification accuracy, which is consistent with our four guidelines. Our proposed guidelines were further validated through various tasks and architectures, including variants of residual networks and transformers.

Year:

2024

Year:

2024

Year:

2024

Year:

2023

Year:

2023

Year:

2023

Year:

2023

Year:

2022

Year:

2022

Year:

2022

Collaborators (2)

Anastasis Kratsios

Assistant Professor

McMaster University

CANADA
View details

Sang Woo Kim

Professor

Pohang University of Science and Technology

SOUTH KOREA
View details