Microsoft is making a bid for the world of big data with a number of new products and announcements.

The details were given at Microsoft’s PASS Summit for SQL Server developers, where Microsoft Corporate Vice President Ted Kummert talked about Microsoft’s roadmap for helping customers manage and analyze any data, of any size, anywhere - on premises, and in the private or public cloud.

In practical terms, this vision was backed up by details of new BI facilities for SQL Server 2012, a new data analysis component, and an Apache Hadoop-based distribution for Windows Server and Windows Azure. The finalising of the name SQL Server 2012 and the fact it’ll be launched in the first half of 2012 were almost non-events compared to the raft of other announcements.

It’s important to Microsoft that SQL Server and Azure don’t become sidelined in the ‘big data’ market, as according to some estimates that now makes up 80 percent of available data. The fact that Microsoft is including Hadoop shows just how much importance it is giving to the idea of working with data as a whole rather than being limited to simply relational data sets. Hadoop is the cloud database framework distributed under Apache as open source.

Hadoop’s advantage is that is that the framework allows for the distributed processing of large data sets across clusters of computers using a simple programming model. Its data warehousing system, Hive, comes with its own SQL-like language called HiveQL that is used to create queries to analyse the data. Microsoft is working on a Hive ODBC driver that will let you create queries against Hive from within Microsoft applications such as Excel and PowerPivot. Microsoft expects to have a CTP version of the Hadoop-based service for Windows Azure ready by the end of 2011.

Microsoft also announced that the final versions of the Hadoop Connectors for SQL Server and Parallel Data Warehouse are now available. One of the surprising things about the announcement is that Microsoft has until now been working on its own competitors to Hadoop. The Dryad family (Dryad, DSC, and LINQtoHPC) are a set of technologies aimed at .NET developers that support data-intensive computing applications that run on a Windows High Performance Computing cluster. Doug Leland, General Manager of Product Management in Microsoft's Business Platform Marketing Group, said that Microsoft is continuing to work on various alternatives to Java-based Hadoop and MapReduce and is "still committed" to these efforts. Kummert showed how SQL Server 2010 will include better self-service BI capabilities with the inclusion of Power View (formerly Project "Crescent,") and SQL Server Data Tools (formerly "Juneau").

PowerView is a browser-based Silverlight application launched from SharePoint Server that can be used to provide ad-hoc reporting for business users who can create and interact with views of data from tabular models based on PowerPivot workbooks published in a PowerPivot Gallery, or tabular models deployed to SQL Server 2012. The fact PowerView is browser-based supports Microsoft’s plans to make it, alongside SQL Server Reporting Services and PerformancePoint, run in browsers on Apple’s iOS and Android. Kummert demoed PowerView on Windows Phone Mango, an iPad 2 and an Android tablet at PASS. The final element demoed was Data Explorer, a new component for Microsoft’s Azure MarketPlace that will give a way to work with public data. Data Explorer is intended to give people a way to discover data no matter where it is located, either inside or outside their company, including in the Data Marketplace. Once discovered, the data can be brought in, integrated with existing data, transformed, and merged with other data. A CTP of Data Explorer is expected to be made available later this year in the SQL Azure Labs at http://www.SQLAzureLabs.com.

Kummert summed up the overall plan as

"Imagine if everyone, regardless of what type of data frameworks or platforms they use, could achieve deep business insights by amassing and analyzing enormous amounts of data not just from their own organization, but from all over the world using a global data marketplace.”

No-one can say Microsoft isn’t good at putting out a grand vision.

To be informed about new articles on I Programmer, subscribe to the RSS feed, follow us on Twitter or Facebook or sign up for our weekly newsletter.