Azure Data Lake Tools for Visual Studio is now generally available. The toolkit gives you a code editor for developing big data queries, including U-SQL scripts, from within Visual Studio. 

Azure Data Lake is a hyper-scale data store for big data analytic workloads. It combines analysis options with an exabyte-scale big data store as a fully managed service. Azure Data Lake is part of the Cortana Analytics Suite, and is made up of the Data Lake Store, a single data repository that can be used for data of any size, with the data being accessible for processing and analytics from HDFS applications and tools.

The editor in the toolkit is mainly of use for writing U-SQL queries.U-SQL is based on T-SQL while it uses C# types as default. It started life in Microsoft's internal Big Data language called SCOPE, and along with the SQL-like declarative language and the use of C# types and the C# expression language, it includes big data processing concepts such as “schema on reads”, custom processors and reducers. You can use U-SQL to query and combine data from sources including Azure Data Lake Storage, Azure Blob Storage, and Azure SQL DB, Azure SQL Data Warehouse, and SQL Server instances running in Azure VMs. The video below shows the editor in use:

The U-SQL support in the editor includes features such as IntelliSense, Syntax Highlight, Error Marker, Go to Definition, and Auto Format. In addition to writing U-SQL scripts, you can add assembly references and perform U-SQL extensibility through code behind. The Code Behind feature lets you create a C# file for custom code development. The tool auto generates code behind and performs the code behind registering and unregistering dynamically during the compile process.

Another feature of the toolkit is the ability to integrate with ADLA (Azure Data Lake Analytics). ADLA is a service built on Apache YARN that dynamically scales. The advantage of U-SQL on ADLA is that it can be used as a Job as a Service, so customers don't have to manage deployment of clusters, just submit their jobs to ADLA. The support in the toolkit means that when you sign in to Azure, you can see your ADLA Metadata entities through a list of customized VSCode command items, and can navigate through ADLA Metadata based on its hierarchy.

The final option in the toolkit is ADLS integration. Azure Data Lake Store is a repository for big data analytic workloads, and the integration with the toolkit means you can navigate, preview your ADLS resources, and upload your file into ADLS..

More Information

Azure Data Lake Tools for Visual Studio

Related Articles

SQL Server 2016 And Azure Data Lake

Azure Big Data Announcements

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info