I will be appointed as a Presidential Young Professor at NUS from Fall 2021. I am looking for PhD students, postdocs and visiting scholars. Scholarships are available. Please contact me if you are interested.
Selected Publications
Conference Papers
(* indicates equal contribution)
Kenji Kawaguchi. On the Theory of Implicit Deep Learning: Global Convergence with Implicit Layers.
In International Conference on Learning Representations (ICLR), 2021.
[pdf] [BibTeX] Selected for ICLR Spotlight (5% accept rate)
Linjun Zhang*, Zhun Deng*, Kenji Kawaguchi*, Amirata Ghorbani and James Zou. How Does Mixup Help With Robustness and Generalization?
In International Conference on Learning Representations (ICLR), 2021.
[pdf] [BibTeX] Selected for ICLR Spotlight (5% accept rate)
[pdf] [BibTeX] Ferran Alet*, Dylan Doblar*, Allan Zhou, Joshua B. Tenenbaum, Kenji Kawaguchi and Chelsea Finn. Noether Networks: meta-learning useful conserved quantities. Advances in Neural Information Processing Systems (NeurIPS), 2021.
[pdf] [BibTeX] Zhun Deng, Linjun Zhang, Kailas Vodrahalli, Kenji Kawaguchi and James Zou. Adversarial Training Helps Transfer Learning via Better Representations. Advances in Neural Information Processing Systems (NeurIPS), 2021.
[pdf] [BibTeX] Clement Gehring, Kenji Kawaguchi, Jiaoyang Huang and Leslie Pack Kaelbling. Understanding End-to-End Model-Based Reinforcement Learning Methods as Implicit Parameterization. Advances in Neural Information Processing Systems (NeurIPS), 2021.
[pdf] [BibTeX] Ferran Alet, Maria Bauza Villalonga, Kenji Kawaguchi, Nurullah Giray Kuru, Tomás Lozano-Pérez and Leslie Pack Kaelbling. Tailoring: encoding inductive biases by optimizing unsupervised objectives at prediction time. Advances in Neural Information Processing Systems (NeurIPS), 2021.
[pdf] [BibTeX] Juncheng Liu, Kenji Kawaguchi, Bryan Hooi, Yiwei Wang and Xiaokui Xiao. EIGNN: Efficient Infinite-Depth Graph Neural Networks. Advances in Neural Information Processing Systems (NeurIPS), 2021.
[pdf] [BibTeX]
Keyulu Xu*, Mozhi Zhang, Stefanie Jegelka and Kenji Kawaguchi*. Optimization of Graph Neural Networks: Implicit Acceleration by Skip Connections and More Depth.
International Conference on Machine Learning (ICML), 2021.
[pdf] [BibTeX]
Vikas Verma, Minh-Thang Luong, Kenji Kawaguchi, Hieu Pham and Quoc V Le. Towards Domain-Agnostic Contrastive Learning.
International Conference on Machine Learning (ICML), 2021.
[pdf] [BibTeX]
Vikas Verma, Meng Qu, Kenji Kawaguchi, Alex Lamb, Yoshua Bengio, Juho Kannala and Jian Tang. GraphMix: Improved Training of GNNs for Semi-Supervised Learning.
In Proceedings of the 35th AAAI Conference on Artificial Intelligence (AAAI), 2021.
[pdf] [BibTeX]
Kenji Kawaguchi* and Qingyun Sun*. A Recipe for Global Convergence Guarantee in Deep Neural Networks.
In Proceedings of the 35th AAAI Conference on Artificial Intelligence (AAAI), 2021.
[pdf] [BibTex]
Kenji Kawaguchi* and Haihao Lu*. Ordered SGD: A New Stochastic Optimization Framework for Empirical Risk Minimization.
In Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics (AISTATS), 2020.
[pdf] [BibTeX] [Code]
Kenji Kawaguchi and Leslie Pack Kaelbling. Elimination of All Bad Local Minima in Deep Learning.
In Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics (AISTATS), 2020.
[pdf] [BibTeX]
Kenji Kawaguchi and Jiaoyang Huang. Gradient Descent Finds Global Minima for Generalizable Deep Neural Networks of Practical Sizes.
In Proceedings of the 57th Allerton Conference on Communication, Control, and Computing (Allerton), IEEE, 2019.
[pdf] [BibTex] [Video]
Kenji Kawaguchi*, Bo Xie*, Vikas Verma, and Le Song. Deep Semi-Random Features for Nonlinear Function Approximation.
In Proceedings of the 32nd AAAI Conference on Artificial Intelligence (AAAI), 2018.
[pdf] [BibTex]
Kenji Kawaguchi. Deep Learning without Poor Local Minima. Advances in Neural Information Processing Systems (NeurIPS), 2016.
[pdf] [BibTex] [Spotlight Video] [Talk] Selected for NeurIPS oral presentation (2% accept rate)
Kenji Kawaguchi. Bounded Optimal Exploration in MDP.
In Proceedings of the 30th AAAI Conference on Artificial Intelligence (AAAI), 2016.
[pdf] [BibTex]
Kenji Kawaguchi, Leslie Pack Kaelbling and Tomás Lozano-Pérez. Bayesian Optimization with Exponential Convergence. Advances in Neural Information Processing Systems (NeurIPS), 2015.
[pdf] [BibTex]
[Code]
Journal Articles
(* indicates equal contribution)
Ameya D. Jagtap, Kenji Kawaguchi and George E. Karniadakis. Adaptive Activation Functions Accelerate Convergence in Deep and Physics-informed Neural Networks. Journal of Computational Physics, 404, 109136, 2020.
[pdf] [BibTex]
Ameya D. Jagtap*, Kenji Kawaguchi* and George E. Karniadakis. Locally adaptive activation functions with slope recovery for deep and physics-informed neural networks. Proceedings of the Royal Society A, 476, 20200334, 2020.
[pdf] [BibTex]
Kenji Kawaguchi and Yoshua Bengio. Depth with Nonlinearity Creates No Bad Local Minima in ResNets.
Neural Networks, 118, 167-174, 2019.
[pdf] [BibTex] [Video]
Kenji Kawaguchi, Jiaoyang Huang and Leslie Pack Kaelbling. Effect of Depth and Width on Local Minima in Deep Learning. Neural Computation, 31(7), 1462-1498, MIT press, 2019.
[pdf] [BibTex]
Kenji Kawaguchi, Yu Maruyama and Xiaoyu Zheng. Global Continuous Optimization with Error Bound and Fast Convergence.
Journal of Artificial Intelligence Research (JAIR), 56, 153-195, 2016.
[pdf] [BibTex]
Xiaoyu Zheng, Hiroto Itoh, Kenji Kawaguchi, Hitoshi Tamaki and Yu Maruyama. Application of Bayesian nonparametric models to the uncertainty and sensitivity analysis of source term in a BWR severe accident.
Reliability Engineering and System Safety (RESS), 138, 253-262, 2015.
[pdf] [BibTeX]
Jun Ishikawa, Kenji Kawaguchi and Yu Maruyama. Analysis for iodine release from unit 3 of Fukushima Dai-ichi nuclear power plant with consideration of water phase iodine chemistry.
Journal of Nuclear Science and Technology (JNST), 52(3), 308-315, 2015.
[pdf] [BibTeX]
Book Chapter
Kenji Kawaguchi, Leslie Pack Kaelbling and Yoshua Bengio. Generalization in deep learning. In Mathematical Aspects of Deep Learning. Cambridge University Press, 2022.
[pdf] [BibTex] [Code]
Technical Reports
Tomaso Poggio, Kenji Kawaguchi, Qianli Liao, Brando Miranda, Lorenzo Rosasco, Xavier Boix, Jack Hidary and Hrushikesh Mhaskar. Theory of Deep Learning III: explaining the
non-overfitting puzzle.
Massachusetts Institute of Technology CBMM Memo No. 73, 2018.
[pdf] [BibTex]
Qianli Liao, Kenji Kawaguchi and Tomaso Poggio. Streaming Normalization: Towards Simpler and More Biologically-plausible Normalizations for Online and Recurrent Learning.
Massachusetts Institute of Technology CBMM Memo No. 57, 2016.
[pdf] [BibTeX]
Selected Service and Activity
Invited Conference reviewer:
• Conference on Neural Information Processing Systems: NeurIPS 2022, NeurIPS 2021, NeurIPS 2020, and NeurIPS 2019
• International Conference on Machine Learning: ICML 2022, ICML 2021, and ICML 2020
• International Conference on Learning Representations: ICLR 2022, ICLR 2021
Program Committee Member:
• Conference on Learning Theory: COLT 2022, COLT 2021
• AAAI Conference on Artificial Intelligence: AAAI 2020 and AAAI 2019
• Conference on Uncertainty in Artificial Intelligence: UAI 2019
Invited Journal reviewer:
• Journal of Machine Learning Research (JMLR)
• Annals of Statistics (Ann. Stat.)
• Neural Computation (MIT press)
• Neural Networks (Elsevier)
• IEEE Transactions on Neural Networks and Learning Systems (IEEE TNNLS)
Invited research visits:
• Microsoft Research (MSR), Redmond, Summer 2018.
• TTIC, Chicago, Fall 2019.
• University of Cambridge: Invited to be in the participant list of the program on "Mathematics of Deep Learning" by the organization team of Prof. Peter Bartlett, Prof. Arnulf Jentzen, Prof. Anders Hansen, Prof. Gitta Kutyniok, Prof. Stephane Mallat, and Prof. Carola Schönlieb.
Invited talks
Seminar, university, and research lab:
• Harvard University / Special talk on Deep Learning, invited by Professor Jun Liu, 2021.
• Brown University / CRUNCH seminar, invited by Professor George Em Karniadakis, 2021.
• Max Planck Institute + UCLA / Math Machine Learning seminar, 2020.
• University of Michigan, Ann Arbor / Seminar, 2020.
• Brown University / Seminar, 2020.
• National University of Singapore / Seminar, 2020.
• University of British Columbia / Seminar, 2020.
• Stanford University / CS theory lunch, 2019.
• Harvard University / Professor Horng-Tzer Yau lab, 2019.
• Carnegie Mellon University (CMU) / AI Seminar Series, 2019.
• Carnegie Mellon University (CMU) / Professor Eric P. Xing lab, 2019.
• Toyota Technological Institute at Chicago (TTIC) / Young Researcher Seminar Series, 2019.
• Purdue University / Seminar at School of Industrial Engineering, 2019.
• PhILMs center / invited by Professor George Em Karniadakis at Brown University, 2019.
• Google Research (at Cambridge) / invited by Dr. Dilip Krishnan (Research Scientist at Google), 2017.
• MIT / Professor David Sontag lab, 2017.
• MIT / Professor Tomaso Poggio lab, 2016.
• MIT / Machine Learning Tea, 2016.
Invited talk at International Conference:
• Minisymposium on Theoretical Foundations of Deep Learning, ICIAM 2019, Spain.
Education / Positions
Harvard University 2020-Present
Postdoctoral fellow
Mentor: Horng-Tzer Yau, Department of Mathematics
Massachusetts Institute of Technology 2020
Ph.D., Computer Science
Advisor: Leslie Pack Kaelbling
Thesis committee: Yoshua Bengio and Suvrit Sra
Massachusetts Institute of Technology 2016
S.M., Electrical Engineering and Computer Science
Advisors: Leslie Pack Kaelbling and Tomás Lozano-Pérez.