There’s a new version of Microsoft’s Cloud Numerics lab that you can use to deploy and run data analysis C# applications on large data sets on Windows Azure.
The Cloud Numerics lab is one of the SQL Azure Labs where you can try out Microsoft’s prototype technologies for developers ahead of any eventual public release.
The Cloud Numerics lab consists of a numerical library of 400 functions covering mathematics, statistics and linear algebra, with a customized Visual Studio project that you can use to work with the functions on distributed data structures.
Both make use of a distributed array object that is capable of holding large data sets by partitioning memory across several computers. The idea is that you develop and debug the apps on your desktop then deploy to Windows Azure.
The refresh of the lab has added support for sparse data structures and algorithms so you can now model big data sets that have many missing values in the tables. This is a fairly normal pattern for large data sets, so the support is an important improvement to the lab.
More functions have been added to support for descriptive statistics, Fourier transforms and linear algebra. Other improvements are a framework for embarrassingly parallel workloads and parametric sweeps, and improved IO enabling parallel reads and writes from a variety of formats and data sources including Azure Blob storage.
This video video shows the lab in action:
You'll need a Windows Azure subscription to deploy your Cloud Numerics application to Azure and can sign up for a 90-day free trial here.
It's the busiest time of the year for online shopping and, as the biggest online retailer, Amazon is handling millions of orders. In order to meet the demand it has increased its workforce - both huma [ ... ]
One of the biggest problems programmers face today is making a single code base work across a range of systems. How a giant company like Google solves the problem is obviously going to be interesting. [ ... ]