Sang-gil Lee

Email  /  CV  /  LinkedIn  /  Google Scholar  /  X (Twitter)  /  GitHub

I am a research scientist at NVIDIA. I work on deep generative models for sequences, with a particular focus on speech and audio.

I received my Ph.D. from the Data Science & AI Lab (DSAIL) at Seoul National University. During my Ph.D., I served as a research intern at NVIDIA, under the advisement of Wei Ping and Boris Ginsburg. Prior to that, I completed internships at Microsoft Research Asia, where I was advised by Xu Tan, Tao Qin (speech), and Bin Shao (bioinformatics). I received my B.S. in Electrical and Computer Engineering from Seoul National University.

My research interest spans a wide range of deep generative models (AR, flow, GAN, diffusion, etc.) applied to sequential data. Specifically, I am working on building multi-modal large language models with a focus on audio.

During my Ph.D., I focused on time-domain waveform data (speech and audio) to advance generative modeling for audio.

I am also broadly interested in speech and audio applications, including text-to-speech, voice conversion, music generation, neural audio codecs, and audio language models.

Representative papers are highlighted.

ETTA is the first text-to-audio model with emergent abilities, capable of synthesizing entirely novel, imaginative sounds beyond the real world by leveraging large-scale synthetic audio captions (AF-Synthetic).

BigVGAN is a universal audio synthesizer that achieves unprecedented zero-shot performance on various unseen environments using anti-aliased periodic nonlinearity and large-scale training.

A neural audio codec that leverages finite scalar quantization and adversarial training with large speech language models to achieve high-quality audio compression with a 1.89 kbps bitrate and 21.5 frames per second.

AF-AudioSet is a large-scale audio dataset featuring synthetic captions generated by Audio Flamingo, enabling significant improvements in text-to-audio models.

VoiceTailor is a one-shot speaker-adaptive text-to-speech model, which proposes combining low-rank adapters to perform speaker adaptation in a parameter-efficient manner.

Edit-A-Video is a diffusion-based one-shot video editing model that solves a background inconsistency problem using a new sparse-causal mask blending method.

PriorGrad presents an efficient method for constructing a data-dependent non-standard Gaussian prior for training and sampling from diffusion models applied to speech synthesis.

NanoFlow uses a single neural network for multiple transformation stages in normalizing flows, which provides an efficient compression for flow-based generative models.

FloWaveNet is one of the first flow-based generative models for fast and parallel synthesis of audio waveforms, enabling a likelihood-based neural vocoder without any auxiliary loss.

Template-based one-shot text-to-SQL generative model based on a Candidate Search Network & Pointer Network.

This work investigates an efficient musical word representation from polyphonic MIDI data for SeqGAN, simultaneously capturing chords and melodies with dynamic timings.

Investigates various initialization and backward control schemes of the membrane potential for training deep spiking networks.

During my time at DSAIL, I collaborated with Seoul National University Hospital on a computer-aided diagnosis project for liver cancer. The project yielded a high-performance medical object detection model to help reduce human errors from radiologists for the early detection of liver disease.

GSSD++ provides robustness to unregistered multi-phase CT images for detecting liver lesions using attention-guided multi-phase alignment with deformable convolutions.

GSSD pioneers a focal liver lesion detection model from multi-phase CT images, which reflects a real-world clinical practice of radiologists.

Last update: Jan 2025. Template borrowed from here.