Lecture: May 14, 2013

www.stanford.edu/class/ee392n


Energy Insights from Big Data

Drew Hylbert and Jeff Kolesky, OPower

Bio

Drew Hylbert is Opower’s Engineering Director, overseeing the architecture team and overall software and systems design from the company’s San Francisco office. He was the evangelist behind the adoption of the Hadoop-based data storage infrastructure for time-series and event data, allowing Opower to scale data storage and processing infrastructure quickly and cost-effectively. Prior to Opower, Drew was Vice President of Engineering at hi5.com, a social gaming and virtual good transaction platform where he was responsible for architecture and infrastructure teams. His work there led to the creation of a “friend relevancy system,” which processed ~1B events/day and returned relevant ordered lists in less than 100ms. Prior to hi5, Drew was a member of Yahoo’s map/reduce and data warehousing infrastructure team. His work there included the implementation of distributed processes across Yahoo’s network of content, which consumed about 3TB of data for a particular day’s worth of traffic. Drew earned his B.S. in Computer Science from the University of Illinois Urbana-Champaign.

Jeff Kolesky is Opower's Chief Software Architect, responsible for the overarching design of Opower's energy information platform. As the founding engineer, Jeff laid the groundwork for Opower's innovative data analytics and report generation system which directly led to a terawatt hour of energy savings in a short four years of operation. Jeff has spent his entire career working for and with start-ups, from specializing in business-to-business web applications during the dot-com boom to architecting a massive educational testing platform. These startups include Fanlib.com, acquired by Disney, Stylehive, Edusoft, and Scient. Jeff holds a double B.S.E. in electrical engineering and computer science from the Pratt School of Engineering at Duke University.

Abstract

In 2007, Opower set out to conduct a behavioral science experiment with the goal of achieving material consumer energy savings with minimal cost to, and effort from consumers. The idea was to partner with utilities to combine consumer energy consumption data with climate, parcel, and occupancy data, and then provide households easy-to-comprehend energy usage reports encouraging consumers to use less energy via normative comparison and providing actionable information. These reports, known as Home Energy Reports (HER), were successful resulting in an average of 2% reduction in consumer energy consumption per household. The Opower HER product was then deemed to be not only a valid energy efficiency program for regulated energy utilities but also the most cost effective program on the market. In addition to the HERs, Opower has added other useful consumer tools like Unusual Usage Alerts and Bill Forecasts. In the last two years, Opower has also partnered with Facebook and Honeywell to leverage social networking and Wi-Fi thermostat technology in an effort to advance home energy savings.

Today, with 86 utility partners in six countries, Opower processes energy consumption data for 70 million households and has helped consumers save more than $280 million on their energy bills. Each household can require processing multiple gigabytes of data to produce just one analytics insight for a consumer. With new products offering lower latency communication channels, Opower has adapted from monthly to daily processing resulting in significant data storage and processing challenges. As a result, Opower has made large investments to build a consumption data storage and processing platform using Hadoop. By utilizing open source tools Opower can operate a linearly scalable big data infrastructure that does the hard work of understanding energy consumption patterns at large scale so consumers can easily make decisions that will save money and the environment. These insights have shown to be effective, as Opower has saved more than two terawatt hours of electricity, which is enough to take every home in the city of Sacramento off the grid for an entire year.

Lecture Notes

Presentation on AMI Big Data by Drew Hylbert and Jeff Kolesky (pdf)