Microsoft has dropped its own big data project to concentrate on Hadoop and is working with Hortonworks to develop a distribution of Hadoop for Azure and for Windows Server.

Microsoft had been working on its own project, LINQ to HPC, codenamed Dryad, but in a blog post on the Windows HPC Team Blog about a new preview release of Dryad, team member Don Pattee posted that “this will be the final preview and we do not plan to move forward with a production release. In line with our announcement in October at the PASS conference we will focus our effort on bringing Apache Hadoop to both Windows Server and Windows Azure.”

Microsoft said at the SQL Server PASS conference that it plans to integrate Hadoop with SQL Server. As an open source project, Hadoop currently works on Linux, so there’s work to be done before it appears on Windows Server.

Dryad was well received by beta testers who were testing its ability to analyse big data sets across clusters of Windows servers. The move to Hadoop has left developers who have been working towards using it in applications processing unstructured data wondering just what the future holds. The response to a question about future plans for Hadoop and .NET on the Windows HPC Developer’s forum doesn’t sound like there will be any quick answers. When a developer pointed out that, as a Java platform, Hadoop apps are best written in native Java, and asked whether Microsoft intended to provide support for its .NET stack, the answer was that while Microsoft plans to integrate Apache Hadoop with the rest of its technology components including developer tools and Frameworks, there’s no announced timeframe:

“At this point however, Microsoft is not making any specific commitments to when Microsoft’s Hadoop distribution will support .NET."

Microsoft is, hoowever, already working with third party company Hortonworks to develop a Windows Server distribution of Hadoop, along with one for Azure.

While the Community Technology Preview of the Azure version is expected before the new year, the only date for a Windows Server pre-release version of Hadoop is “some time in 2012”, and that doesn’t even begin to answer the point about .NET development.

More Information:

Bringing Apache Hadoop to Windows

Related News:

Announcements from PASS 2011

New Hadoop connectors

To be informed about new articles on I Programmer, subscribe to the RSS feed, follow us on Google+, Twitter or Facebook or sign up for our weekly newsletter.