University of Cambridge > Talks.cam > Microsoft Research Summer School > Tools and services for data intensive research

Tools and services for data intensive research

Add to your list(s) Download to your calendar using vCal

If you have a question about this talk, please contact Dr Fabien Petitcolas.

Abstract: For many important research investigations, especially in science, efficiently analyzing large data sets is a major challenge. Microsoft’s Dryad is a high-performance, general-purpose distributed computing engine that handles some of the most difficult aspects of cluster-based distributed computing. It’s powerful: Microsoft routinely uses Dryad applications to analyze petabytes of data on clusters of thousands of computers. Microsoft Research has also developed DryadLINQ, which allows developers to use an extended version of the LINQ programming model and API to implement Dryad applications in managed code. DryadLINQ code is similar to what you’ll see in a conventional LINQ -to-objects application, and the application core is often only a few lines of code. Behind the scenes though, a DryadLINQ provider automatically converts the LINQ query into a Dryad job and executes the query as a distributed application on a cluster. Using Dryad through DryadLINQ, even a novice at parallel processing or cluster-based computing can implement a high-performance distributed application to efficiently analyze terabytes of data.

In this talk we present an introduction to Dryad and DryadLINQ for data intensive research and we compare and contrast it to other related technologies. We describe our ongoing efforts to collaborate with external researchers to explore the application Dryad and DryadLINQ to big data research problems in science. We also highlight our efforts to offer software and services to researchers across the world, through the academic release of Dryad and DryadLINQ with associated programming user documentation.

Biography: Roger Barga is Principal Architect at Microsoft Research, where he leads the Advanced Research Services and Tools (ARTS) team. The ARTS team is responsible for developing innovative tools and services using Microsoft products and technology to revolutionize and accelerate research, and it provides strategic and tactical hands-on technological leadership to many research projects. Roger joined Microsoft in 1997 as a Researcher in the Database Group of Microsoft Research, where he directed both systems research and product development efforts in database, workflow and stream processing systems. He has developed ideas from basic research, through proof of concept prototypes to incubation efforts in product groups. Roger has published over 50 peer reviewed papers, filed over 30 patent applications, and served more than 70 times on program committees for more than 30 different international conferences and workshops.

This talk is part of the Microsoft Research Summer School series.

Tell a friend about this talk:

This talk is included in these lists:

Note that ex-directory lists are not shown.

 

© 2006-2020 Talks.cam, University of Cambridge. Contact Us | Help and Documentation | Privacy and Publicity