Mathew Monfort

I am currently an Applied Scientist at Amazon AWS working on problems related to machine learning fairness and explainability. Previously I was a research scientist at the Computer Science and Artificial Intelligence Laboratory (CSAIL) at the Massachusetts Institute of Technology (MIT). Before that I was a postdoctoral associate at MIT CSAIL working with Aude Oliva, Antonio Torralba, and Wojciech Matusik.

At CSAIL I worked on deep learning approaches to video understanding and visual navigation with the Toyota-CSAIL Research Center and the IBM-MIT Watson Laboratory.

While earning my PhD I researched scalable inverse optimal control as part of the Purposeful Prediction Lab at the University of Illinois at Chicago (UIC) under Brian Ziebart.



Spoken Moments: Learning Joint Audio-Visual Representations from Video Descriptions
Mathew Monfort*, SouYoung Jin*, Alexander Liu, David Harwath, Rogerio Feris, Aude Oliva
Computer Vision and Pattern Recognition (CVPR), 2021. [PDF] [Webpage]

Multi-Moments in Time: Learning and Interpreting Models for Multi-Action Video Understanding
Mathew Monfort, Bowen Pan, Kandan Ramakrishnan, Alex Andonian,
Barry A McNamara, Alex Lascelles, Quanfu Fan, Dan Gutfreund, Rogerio Feris, Aude Oliva
IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 2021. [PDF] [Webpage] [BIB]

We Have So Much In Common: Modeling Semantic Relational Set Abstractions in Videos
Alex Andonian*, Camilo Fosco*, Mathew Monfort, Allen Lee, Rogerio Feris, Carl Vondrick, Aude Oliva
European Conference on Computer Vision (ECCV), 2020. [PDF] [Webpage]

Reasoning about Human-Object Interactions through Dual Attention Networks
Tete Xiao, Quanfu Fan, Danny Gutfreund, Mathew Monfort, Aude Oliva, Bolei Zhou
International Conference on Computer Vision (ICCV), 2019. [PDF] [Webpage]

Convolutional Spatial Fusion for Multi-Agent Trajectory Prediction
Tianyang Zhao, Yifei Xu, Mathew Monfort, Wongun Choi, Chris Baker, Yibiao Zhao, Yizhou Wang, Ying Nian Wu
Computer Vision and Pattern Recognition (CVPR), 2019. [PDF]

Moments in Time Dataset: one million videos for event understanding
Mathew Monfort, Bolei Zhou, Sarah Adel Bargal, Alex Andonian, Tom Yan,
Kandan Ramakrishnan, Lisa Brown, Quanfu Fan, Dan Gutfruend, Carl Vondrick, Aude Oliva
IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 2019. [PDF] [Webpage]

A Deep Learning Approach to Identifying Shock Locations in Turbulent Combustion Tensor Fields
Mathew Monfort, Timothy Luciani, Jon Komperda, Brian D. Ziebart, Farzad Mashayek, G. Elisabeta Marai
Modeling, Analysis, and Visualization of Anisotropy, Springer, 2017. [PDF]

Asynchronous Data Aggregation for Training End to End Visual Control Networks
Mathew Monfort, Matthew Johnson, Aude Oliva, Katja Hofmann
International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 2017. [PDF]

Goal-Predictive Robotic Teleoperation from Noisy Sensors
Christopher Schultz, Sanket Gaurav, Mathew Monfort, Lingfei Zhang, Brian Ziebart
IEEE International Conference on Robotics and Automation (ICRA), 2017. [PDF]

Adversarial Inverse Optimal Control for General Imitation Learning Losses and Embodiment Transfer
Xiangli Chen, Mathew Monfort, Brian D. Ziebart, Peter Carr
Uncertainty in Artificial Intelligence (UAI), 2016. [PDF]

End to End Learning for Self-Driving Cars
Mariusz Bojarski, Davide Del Testa, Daniel Dworakowski, Bernhard Firner, Beat Flepp, Prasoon Goyal,
Lawrence D. Jackel, Mathew Monfort, Urs Muller, Jiakai Zhang, Xin Zhang, Jake Zhao, Karol Zieba
arXiv:1604.07316, 2016. [PDF]

Robust Covariate Shift Regression
Xiangli Chen, Mathew Monfort, Anqi Liu, Brian Ziebart
Artificial Intelligence and Statistics (AISTATS), 2016. [PDF]

Softstar: Heuristic-Guided Probabilistic Inference
Mathew Monfort, Brenden M Lake, Brian Ziebart, Patrick Lucey, Josh Tenenbaum
Advances in Neural Information Processing Systems (NIPS), 2015. [PDF]

Graph-based inverse optimal control for robot manipulation
Arunkumar Byravan, Mathew Monfort, Brian Ziebart, Byron Boots, Dieter Fox
International Joint Conference on Artificial Intelligence (IJCAI), 2015. [PDF]

Intent Prediction and Trajectory Forecasting via Predictive Inverse Linear-Quadratic Regulation
Mathew Monfort, Anqi Liu, Brian D Ziebart
Advances in Artificial Intelligence (AAAI), 2015. [PDF]

“Quality vs Quantity”: Improved Shot Prediction in Soccer using Strategic Features from Spatiotemporal Data
Patrick Lucey, Alina Bialkowski, Mathew Monfort, Peter Carr, Iain Matthews
MIT Sloan Sports Analytics Conference, 2014. [PDF]

Layered Hybrid Inverse Optimal Control for Learning Robot Manipulation from Demonstration
Arunkumar Byravan, Mathew Monfort, Brian Ziebart, Byron Boots, Dieter Fox
NIPS Workshop on Autonomously Learning Robots, 2014. [PDF]

Trajectory Forecasting and Intent Recognition via Predictive Inverse Linear-Quadratic Regulation
Mathew Monfort, Anqi Liu, Brian D Ziebart
IROS Workshop on Assistance and Service Robotics in a Human Environment, 2014.

Predictive Inverse Optimal Control in Large Decision Processes via Heuristic-based Search
Mathew Monfort, Brenden M Lake, Brian Ziebart, Josh Tenenbaum
ICML Workshop on Robot Learning, 2014. [PDF]


Methods in Large Scale Inverse Optimal Control
Mathew Monfort
PhD Thesis. Department of Computer Science, University of Illinois at Chicago, 2016. [PDF]

An Ensemble of Convolutional Neural Networks for the Recognition of Handwritten Digits
Mathew Monfort
Master's Thesis. Department of Computer Science, Florida State University, 2011. [PDF]