An Introduction to Fisher Information

Posted on October 3, 2021

A common question among statisticians and data analysts is how accurately we can estimate the parameters of a distribution given some samples from it. Fisher information can help answer this question by quantifying the amount of information that the samples contain about the unknown parameters of the distribution.

Read More

The History of Speech Recognition to the Year 2030

Posted on August 3, 2021

Given the remarkable changes in the state of speech recognition over the previous decade, what can we expect over the coming decade? I attempt to forecast the state of speech recognition research and applications by the year 2030.

Read More

The Label Bias Problem

Posted on November 8, 2019

Many sequence classification models suffer from the label bias problem. Understanding the label bias problem and when a certain model suffers from it is subtle but is essential to understand the design of models like conditional random fields and graph transformer networks.

Read More

Training Sequence Models with Attention

Posted on December 17, 2017

Here are a few practical tips for training sequence-to-sequence models with attention. If you have experience training other types of deep neural networks, pretty much all of it applies here. This article focuses on a few tips you might not know about, even with experience training other models.

Read More

Speech Recognition Is Not Solved

Posted on October 11, 2017

Ever since Deep Learning hit the scene in speech recognition, word error rates have fallen dramatically. But despite articles you may have read, we still don’t have human-level speech recognition. Speech recognizers have many failure modes. Acknowledging these and taking steps towards solving them is critical to progress. It’s the only way to go from ASR which works for some people, most of the time to ASR which works for all people, all of the time.

Read More

PyTorch or TensorFlow?

Posted on August 17, 2017

This is a guide to the main differences I’ve found between PyTorch and TensorFlow. This post is intended to be useful for anyone considering starting a new project or making the switch from one deep learning framework to another. The focus is on programmability and flexibility when setting up the components of the training and deployment deep learning stack. I won’t go into performance (speed / memory usage) trade-offs.

Read More