Why are GPUs so hard to program - or are they?

Wen-mei Hwu
University of Illinois, Urbana-Champaign and MulticoreWare Inc.

When we designed Blue Waters, the most powerful petascale supercomputer for the NSF community, the single most pressing concern about its success was that “GPUs are too hard to program.” The rise of GPU computing has significantly boosted the pace of progress in numeric methods, algorithm design and programming techniques for scalability at the processor architecture level. There is now a wealth of publications, teaching material as well as publicly available software libraries. For example, the CUDA counter part of the Math Kernel Library (MKL) now contains hundreds of functions covering nearly every domain of computational science. We also have the experience of teaching more than 10,000 students to write scalable parallel programs using CUDA in a Coursera MOOC. However, there has been a lack of practical languages and compilers that relieve programmers from heavy lifting. I will review the community’s progress in developing scalable, portable, and numerically stable software, with some deeper discussions of contributions from the IMPACT group. I will also present some recent advances and current efforts in languages and compilers for developing scalable numerical applications.

Slides:

Download the slides for the presentation in PDF format.

About the speaker:

Wen-mei W. Hwu is a Professor and holds the Sanders-AMD Endowed Chair in the Department of Electrical and Computer Engineering, University of Illinois at Urbana-Champaign. He is also CTO of MulticoreWare Inc., Chief Scientist of UIUC Parallel Computing Institute and director of the IMPACT research group He directs (impact.crhc.illinois.edu/default.aspx), the UIUC CUDA Center of Excellence, and serves as one of the principal investigators of the $208M NSF Blue Waters Petascale computer project. For his contributions, he received the ACM SigArch Maurice Wilkes Award, the ACM Grace Murray Hopper Award, the ISCA Influential Paper Award, and the Distinguished Alumni Award in Computer Science of the University of California, Berkeley. He is a fellow of IEEE and ACM. Dr. Hwu received his Ph.D. degree in Computer Science from the University of California, Berkeley.

Contact information:

Wen-mei Hwu
University of Illinois, Urbana-Champaign