U.S. Department of Energy

Pacific Northwest National Laboratory

Tensor Deep Stacking Networks

Wednesday, November 11, 2015
Dr. Brian Hutchinson
The Tensor Deep Stacking Network (T-DSN) is a deep learning architecture that has demonstrated strong results on speech and vision tasks. The model consists of a set of stacked blocks, where the output predictions of each block are fed (along with the original data) into the following block. Each individual block is a neural network with a special structure that incorporates higher order features via two parallel hidden layers and a hidden-to-output weight tensor. Unlike deep neural networks, model performance steadily improves when trained with larger batch sizes. Fortunately, full batch training is feasible even for large datasets using a parallel training procedure. In this talk we will explore the model, its relationship to other deep architectures, the parallel training procedure, and experimental results demonstrating the T-DSN's effectiveness.
Speaker Bio

Brian Hutchinson received his Ph.D. in electrical engineering from the University of Washington in 2013, where he also earned a Master's in the same field. He also holds degrees in computer science (B.S., M.S.) and linguistics (B.A.) from Western Washington University, where he has been an Assistant Professor of Computer Science since 2013. His research interests include machine learning, speech and language processing, and optimization. In particular, he is interested in novel deep learning architectures and their application across domains.

| Pacific Northwest National Laboratory