Heng-Jui Chang

Heng-Jui Chang (pronunciation)

I'm a Ph.D. candidate at MIT CSAIL in the Spoken Language Systems Group, advised by Dr. James Glass. I obtained my S.M. in EECS from MIT in 2024. I graduated from National Taiwan University in 2021 with my B.S. in Electrical Engineering, supervised by Prof. Lin-shan Lee and Prof. Hung-yi Lee. I am working on data-efficient speech representation learning.
Email: hengjui [at] mit.edu


2022–Present	2023 & 2024 Summer	2017–2021

News

(Apr 2024) I received the IEEE Ganesh N. Ramaswamy Memorial Student Grant for my ICASSP paper.
(Mar 2024) My paper (R-Spin) was accepted to NAACL 2024.
(Feb 2024) I have obtained my Master of Science degree in EECS from MIT.
(Dec 2023) My paper done during an internship at Meta was accepted to ICASSP 2024.

Publications (* equal contribution)

2024

DC-Spin: A Speaker-invariant Speech Tokenizer for Spoken Language Models
Heng-Jui Chang, Hongyu Gong, Changhan Wang, James Glass, Yu-An Chung
Preprint 2024
arxiv

R-Spin: Efficient Speaker and Noise-invariant Representation Learning with Acoustic Pieces
Heng-Jui Chang, James Glass
Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL) 2024
arxiv / acl anthology / code

CoLLD: Contrastive Layer-to-layer Distillation for Compressing Multilingual Pre-trained Speech Encoders
Heng-Jui Chang, Ning Dong, Ruslan Mavlyutov, Sravya Popuri, Yu-An Chung
IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2024
IEEE Ganesh N. Ramaswamy Memorial Student Grant
arxiv / ieee xplore / blog

A Large-Scale Evaluation of Speech Foundation Models
Shu-wen Yang, Heng-Jui Chang*, Zili Huang*, Andy T. Liu*, Cheng-I Lai*, Haibin Wu*, Jiatong Shi, Xuankai Chang, Hsiang-Sheng Tsai, Wen-Chin Huang, Tzu-hsun Feng, Po-Han Chi, Yist Y. Lin, Yung-Sung Chuang, Tzu-Hsien Huang, Wei-Cheng Tseng, Kushal Lakhotia, Abdelrahman Mohamed, Shang-Wen Li, Shinji Watanabe, Hung-yi Lee
IEEE/ACM Transactions on Audio, Speech, and Language Processing (TASLP) 2024
arxiv / ieee xplore

SpeechCLIP+: Self-supervised Multi-task Representation Learning for Speech via CLIP and Speech-image Data
Hsuan-Fu Wang, Yi-Jen Shih, Heng-Jui Chang, Layne Berry, Puyuan Peng, Hung-yi Lee, Hsin-Min Wang, David Harwath
ICASSP Self-supervision in Audio, Speech, and Beyond (SASB) Workshop 2024
arxiv

2023

DinoSR: Self-Distillation and Online Clustering for Self-supervised Speech Representation Learning
Alexander H. Liu, Heng-Jui Chang, Michael Auli, Wei-Ning Hsu, James Glass
Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS) 2023
arxiv / openreview / neurips proceedings / code

Self-supervised Fine-tuning for Improved Content Representations by Speaker-invariant Clustering
Heng-Jui Chang, Alexander H. Liu, James Glass
Interspeech 2023
arxiv / isca / code

M-SpeechCLIP: Leveraging Large-Scale, Pre-Trained Models for Multilingual Speech to Image Retrieval
Layne Berry, Yi-Jen Shih, Hsuan-Fu Wang, Heng-Jui Chang, Hung-yi Lee, David Harwath
IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2023
arxiv / ieee xplore

2022

SpeechCLIP: Integrating Speech with Pre-Trained Vision and Language Model
Yi-Jen Shih, Hsuan-Fu Wang, Heng-Jui Chang, Layne Berry, Hung-yi Lee, David Harwath
IEEE Spoken Language Technology Workshop (SLT) 2022
arxiv / ieee xplore / code

SUPERB-SG: Enhanced Speech processing Universal PERformance Benchmark for Semantic and Generative Capabilities
Hsiang-Sheng Tsai*, Heng-Jui Chang*, Weng-Chin Huang*, Zili Huang*, Kushal Lakhotia*, Shu-wen Yang, Shuyan Dong, Andy T. Liu, Cheng-I Lai, Jiatong Shi, Xuankai Chang, Phil Hall, Hsuan-Jui Chen, Shang-Wen Li, Shinji Watanabe, Abdelrahman Mohamed, Hung-yi Lee
Annual Meeting of the Association for Computational Linguistics (ACL) 2022
arxiv / acl anthology / code / website

DistilHuBERT: Speech Representation Learning by Layer-wise Distillation of Hidden-unit BERT
Heng-Jui Chang, Shu-wen Yang, Hung-yi Lee
IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2022
arxiv / ieee xplore / code / poster / huggingface / video

Mandarin-English Code-switching Speech Recognition with Self-supervised Speech Representation Model
Liang-Hsuan Tseng*, Yu-Kuan Fu*, Heng-Jui Chang, Hung-yi Lee
AAAI Workshop on Self-supervised Learning for Audio and Speech Processing 2022
arxiv

2021

Non-autoregressive Mandarin-English Code-switching Speech Recognition
Shun-Po Chuang*, Heng-Jui Chang*, Sung-Feng Huang, Hung-yi Lee
IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) 2021
arxiv / ieee xplore / video

Towards Lifelong Learning of End-to-end ASR
Heng-Jui Chang, Hung-yi Lee, Lin-shan Lee
Interspeech 2021
ISCA Student Travel Grant
arxiv / isca / poster / short video / long video

End-to-end Whispered Speech Recognition with Frequency-weighted Approaches and Pseudo Whisper Pre-training
Heng-Jui Chang, Alexander H. Liu, Hung-yi Lee, Lin-shan Lee
IEEE Spoken Language Technology Workshop (SLT) 2021
arxiv / ieee xplore / video