Category Archives: Quentin Clark

Using Windows Azure to Speed up Genome Research and Save Millions of Dollars

A year ago at Virginia Tech researchers needed 2 weeks to analyze just 1 genome. Today, they can analyze 100 genomes each day. Why is this important? Scientists can learn more about our DNA and uncover more effective strategies for detecting, diagnosing, and treating diseases such as cancer. What’s helping to make this possible? An innovative solution developed by Virginia Polytechnic Institute and State University (Virginia Tech) that’s based on Windows Azure and the Windows Azure HDInsight Service.

There are currently an estimated 2,000 DNA sequencers generating around 15 petabytes of data every year.  Additionally, data volumes are doubling every 8 months, significantly ahead of Moore’s law of compute capability’s which is doubling only every 24 months.  Most institutions can’t afford to scale data centers fast enough to store and analyze all of the new information. To overcome this challenge, Virginia Tech developed a high-performance computing (HPC) solution with Windows Azure. It gives global researchers a highly scalable, on-demand IT infrastructure in the cloud that they can use to store and analyze Big Data, accelerate genome research, and increase collaboration.    

To make it easy for researchers to use the solution, Virginia Tech developed two cloud applications. One streamlines the creation of Genome Analysis Toolkit (GATK) pipelines (for DNA sequencing) using Windows Azure HDInsight. The other program simplifies the use of Hadoop MapReduce pipelines to automate data transfers and analyze information that resides on local and cloud-based systems in a hybrid scenario.

The new solution is saving Virginia Tech—and other organizations—millions of dollars because scientists pay only for the resources that they use. This includes Windows Azure Blob storage for temporary or long-term data storage and HDInsight clusters for on-demand HPC nodes. Provisioning a new resource takes just seconds.

Global scientists can also collaborate with less effort because they can now easily share insights and data sets virtually anytime, anywhere—and with any device. As a result, in the future scientists or doctors may be able to use the solution to develop custom treatments for individual patients faster, by engaging in genome analysis directly at hospitals.  

You can learn more about Virginia Tech’s solution by watching the video below or reading the detailed case study here.   

Making Big Data Work for Everyone

This has been a really exciting week for the Microsoft data platform. On Monday, we announced the general availability of Power BI for Office 365. This morning, we announced that support for Hadoop 2.2 clusters is now in preview for Windows Azure HDInsight—our cloud-based Hadoop service. And, I just presented at Strata, one of our industry’s premier big data events, where I talked about making data work for everyone.

A few weeks ago, I shared some thoughts about the changing nature of data in the modern business enterprise, leading to the conclusion that there is a “new value” of data. At Microsoft, we have a bold vision of bringing big data to a billion people. That’s how we think of big data: a billion people are more informed and have a better perspective of the world around them. 

When I say, “bringing big data to a billion people,” I’m talking about changing the way people live and work through data. And, I am talking about everyone, not just data scientists and experts. Big data is valuable for people in all walks of life. It’s certainly valuable for the enterprise; people in businesses and organizations of all types clearly need the power of big data to drive faster decisions and insights. But it’s also valuable for consumers, citizens and people of all generations, whether they are seeking answers and insights for some other aspect of their lives or just satisfying their own curiosity.

Let me share a personal example of “satisfying curiosity.” A week ago, my boys and I were talking about the World Cup and how it’s being hosted in Brazil and how the Winter Olympics are being hosted in Russia. Somehow, this dialog led to us wondering about the populations of countries by continent. I was able to produce the view below in just a couple of minutes using some of our business intelligence tools, Power BI for Office 365 and Excel. I opened Excel, did a Power Query search on country population, found the data I wanted and then added that data to a worksheet. I then searched on countries by continent, found the data and added that to another worksheet. I inserted both worksheets into Power View, linked them together, dragged the right data columns into the Power View canvas, made the continents a slicer (a type of filter), and voila!, I had the information my boys were curious about. 

This little story illuminates the art of what’s possible when disparate data sources are quickly and easily brought together for analysis and visualization. Our family had a better, more informed dialog over lunch because of the power of structured information that is accessible in a friction-free way. Now just imagine what the professionals in finance and engineering I work with every day are doing – Power BI has changed how I run my business and organization day-to-day.

At Microsoft, we are striving to put the power of data insights into the hands of the people who are closest to real-world challenges. We want subject matter experts to analyze and visualize data and explore solutions as easily as they can create a spreadsheet. Think of all the different kinds of people that could benefit from this: How would farmers change their planting and animal-husbandry practices? How might law enforcement practices change? How will city planners create better experiences for all citizens of their communities?

Our view is that it takes the combined effect of three elements to bring big data to a billion people: robust tools that everyday people can use, easy access to all kinds of data sets, and a complete data platform.

Today, with the latest additions and updates to our data platform, we are another step closer to our vision. As I mentioned at the outset, on Monday, we announced the general availability of Power BI for Office 365, which brings together Microsoft’s strengths in cloud computing, productivity software, and business intelligence to enable people to easily find, analyze and visualize data in Excel, discover valuable insights, and share and collaborate on those insights from anywhere with Office 365. More than that, it does so in a turnkey way, empowering users while giving IT the ability to help manage and maximize the work. In addition, we are making available a preview of Windows Azure HDInsight with support for Hadoop 2.2. Windows Azure HDInsight combines the best of open source and flexibility of cloud deployment with the security and manageability required to accelerate the adoption of Hadoop in the enterprise. With this new version, we’ve updated HDInsight to take advantage of some key capabilities in the latest version of Hadoop.

It’s a really exciting time in the data industry. We are at the tipping point of bringing bring big data to a billion people. It’s happening now: doctors, farmers, lawyers, teachers and marketing teams are incorporating big data into their daily decision-making and fundamentally reinventing the way they do business.   

Quentin Clark
Corporate Vice President
Data Platform Group

Microsoft to Join the Big Data Conversation as an Elite Sponsor at Strata, Santa Clara, CA

Join the big data conversations at Strata Santa ClaraOn February 11-13, Microsoft will join 2,700 of the world’s leading big data scientists, CIOs and executives at the Strata conference in Santa Clara, CA. For some, Microsoft does not come immediately to mind when thinking about leadership in data; however, along with the power of HDInsight, Microsoft is working to democratize big data – connecting companies to previously untouched data sources and enabling everyone to gain insight through familiar and powerful tools in Microsoft Office Excel.

Quentin Clark, Corporate Vice President of the Data Platform Group will share some of that vision in his keynote address followed by an interview with O’Reilly’s Mac Slocum – streamed live from the Strata conference website. Additionally, Herain Oberoi, Director Product Marketing, Microsoft will deliver a joint session with Bryan Hurd, Director of Advanced Analytics, Microsoft Cybercrime Center, highlighting how they use Microsoft’s portfolio of Big Data solutions, including simple, familiar end user tools and cutting-edge cloud technologies, to fight organized crime and BotNets that commit hundreds of millions of dollars of identity fraud ever year (view the Cybercrime Center video on YouTube).  Visit the Strata Website for full Event Agenda.

Microsoft is also hosting a hack-a-thon on February 8-9.  Not limited to technology, this big data hack-a-thon is aimed at spending 1½ days hacking for a good cause. Extra credit will go to those who develop using HDInsight and Power BI, but you are free to use whichever technology you like.  Sign up here.

Join the conversation – attend one of our sessions in person, visit our exhibit booth (#401) and interact with our data experts or follow us on and Twitter @SQLServer. You will how Microsoft’s complete data platform delivers Big Data: enabling you to enrich your data with external data sources; connect any type of data – structured or unstructured; and extend the accessibility of data analysis beyond scientists into the hands of every employee.

Windows Azure HDInsight Released for GA

Today, Microsoft announced the general availability of Windows Azure HDInsight, its cloud-based distribution of Hadoop. Further expanding on Microsoft’s big data strategy, Microsoft CVP Quentin Clark will deliver a keynote at O’Reilly Strata + Hadoop World tomorrow, where he will discuss Microsoft’s vision to democratize big data by providing easy to use tools and a unified data platform. Clark further detailed this approach in a post on the Official Microsoft Blog.

News Highlights from  Strata + Hadoop World Conference

  • Windows Azure HDInsight, available today, is an open and flexible platform that provides 100% Apache Hadoop as a service in the cloud. HDInsight builds on top of the Hortonworks Data Platform (HDP) to ensure full compatibility with Apache Hadoop. HDInsight offers customers:
    • Insights with familiar tools: Through integration with Microsoft’s business intelligence tools, HDInsight enables customers to easily analyze data in Excel.
    • Deployment Agility: With HDInsight customers can deploy and provision a Hadoop cluster, no matter the size, in minutes instead of hours or days, without losing any data.
    • Enterprise-ready Hadoop: HDInsight offers enterprise-class security and manageability with a secure node in every cluster and extensive support for PowerShell scripting.
    • Rich developer experience: Developers have a choice of languages including .NET, Java and others. .NET developers can exploit the full power of language-integrated query with LINQ to Hive, and database developers can use existing skills to query and transform data through Hive.
  • Customers such as Christian Hansen, City of Barcelona, and Virginia Tech are using HDInsight to give their data the power of Hadoop and the scalability of Windows Azure.
  • Interested customers can begin using Windows Azure HDInsight now.

For more information on today’s news, check out Clark’s post on the Official Microsoft Blog. To follow the news from O’Reilly Strata + Hadoop World, follow the conversation on Twitter at @SQLServer and #strataconf.


What’s Your Favorite Feature of SQL Server 2012?

PASS Summit in November was a perfect opportunity to catch up with SQL Server community members to ask them about their favorite features of SQL Server 2012. We caught up with many of them at a local restaurant and captured their responses in this video to kick off Quentin Clark’s keynote.

Perhaps not surprisingly, the favorite features named were exceedingly diverse, but there were some commonalities in the outcomes people were looking for.  These benefits included:

  • Reductions in application downtime
  • Improvements in database and application performance
  • Improvements in productivity
  • Costs savings
  • Empowering end-users with BI tools to improve decision making

So if any of these outcomes are critical to your next project, watch the full video above and see what features of SQL Server 2012 can help you achieve these aims.  And for those that are interested in the Business Intelligence benefits for your next project, you may want to hear more by attending the PASS Business Analytics Conference on April 10-12 in Chicago.  That would be a great opportunity to catch up and hear more about your favorite feature of SQL Server 2012!

Many of the customers featured in the video have already worked on published SQL Server 2012 customer stories.  You can find a complete list of these case studies at

David Hobbs-Mallyon, Senior Product Marketing Manager


PASS Summit 2012 Recap & the Milestones of SQL Server 2012

Microsoft_VP_Ted_KummertLast week marked the completion of a great week at PASS Summit 2012, the world’s largest technical training conference for SQL Server professionals and BI experts alike. During this year’s 3-day conference, nearly 4,000 attendees heard firsthand about the great advances being made toward managing big data. Over the course of two keynote speeches by Microsoft Corporate Vice Presidents Ted Kummert (Data Platform Group) and Quentin Clark (SQL Program Management), Microsoft announced the following:

  • Project codename “Hekaton,” a new in-memory technology that will be built directly into the data platform, will ship in the next major version of SQL Server.  Currently in private technology preview with a small set of customers, Hekaton completes the company’s portfolio of in-memory technologies across analytics, transactions, streaming and caching workloads, enabling business acceleration by shrinking the time from raw data to insights.
  • SQL Server 2012 Parallel Data Warehouse (PDW), the next version of Microsoft’s enterprise-class appliance, will be available during the first half of 2013.  SQL Server 2012 PDW includes PolyBase, a fundamental breakthrough in data processing that will enable queries across relational data and non-relational Hadoop data.
  • SQL Server 2012 SP1, which supports Office 2013 by offering business users enhanced, new capabilities for self-service business intelligence using familiar tools such as Excel and Sharepoint, is now available for download here

What’s more, on the final day of PASS Summit 2012, attendees were treated to the presentation, “Big Data Meets SQL Server 2012” by Microsoft Technical Fellow David DeWitt. 

PASS_Summit_2012All the while, conference participants attended a wide variety of technical sessions presented by industry experts in addition to a host of other programs. From on-site certification testing, to hands-on-labs, attendees were able to boost their technical skills using these resources, as well as work through technical issues with top Microsoft Customer Service and Support (CSS) engineers and get architectural guidance from the SQL Server, Business Intelligence and Azure Customer Advisory Teams (CAT). Of course, the learning didn’t stop there; attendees were invited to new, “I Made That!” Developer Chalk Talks, which featured 30 minute casual talks with the Microsoft developers who worked on specific features and functionalities of SQL Server 2012. The topics appealed to many, ranging from AlwaysOn to Hekaton. You can see more great photos from PASS Summit 2012 on the SQL Server Facebook page or access the video interviews with Dave Campbell, Quentin Clark, and David DeWitt available at the SQL Server virtual press room.

And so, as we close on another year of PASS Summit, it’s the perfect time to look back and see how far we’ve come since the launch of SQL Server 2012.  Join us below, as we take a celebratory look at the milestones we’ve hit along the way, and let’s look together toward the bright future ahead!