Major Update to Google BigQuery
Written by Kay Ewbank   
Monday, 23 September 2013

Google has announced a major update to BigQuery, adding real-time support to its service for analyzing large amounts of data.

BigQuery has already been updated this year to provide support for big scale JOINs and GROUP BYs and unlimited result sizes. The new improvements add the option of querying subsets of the latest data, more functions and browser tool improvements alongside the ability to run queries in real time. The browser has been improved with a buttons for common tasks in the query history panel, and more information made visible about the queries.

 

(click in screenshot for larger version)

 

The real-time support has been added through a simple API call, tabledata().insertAll(), that lets you store data as it comes in and query it instantly. Talking about the improvements on the Google Developer’s Blog, Felipe Hoffa, Developer Programs Engineer at Google said this feature is ideal for time sensitive use cases like log analysis and alerts generation, and that you can use it simply by calling the new endpoint with your data in a JSON object (with a single row or multiple rows of data).

Streaming data into BigQuery is free until January 1st, 2014, after when it will be billed at a flat rate of 1 cent per 10,000 rows inserted. The traditional jobs().insert() method will continue to be free.

The second improvement is the ability to define queries that only scan a range or spot in the previous 24 hours. Traditionally BigQuery has always done a "full column scan" when querying data, while the new syntax will allow you to focus only on a specific subset of the latest data, so lowering the costs of queries. You can query only the last hour of inserted data, or what was inserted before that hour, or get a snapshot of the table at a specific time.

Google has also added new window functions, namely SUM(), COUNT(), AVG(), MIN(), MAX(), FIRST_VALUE, and LAST_VALUE(), along with the statistical functions COVAR_POP(), COVAR_SAMP(), STDDEV_POP(), STDDEV_SAMP(), VAR_POP() and VAR_SAMP(). Window functions let you carry out calculations on a specific partition, or "window", of a result set.

 

 

More Information

Google BigQuery goes real-time

Google BigQuery

 

Related Articles

BigQuery Updated and Repriced

Google BigQuery Updated

BigQuery Now Open to All

Google BigQuery Service

 

To be informed about new articles on I Programmer, install the I Programmer Toolbar, subscribe to the RSS feed, follow us on, Twitter, Facebook, Google+ or Linkedin,  or sign up for our weekly newsletter.

 

blog comments powered by Disqus

 

Banner


Amazon Robot Picking Challenge 2015
09/10/2014

Amazon has lots and lots of warehouses, sorry fulfilment centers, all over the world and they employ lots of humans to find the stuff you buy. The Amazon Picking Challenge is about getting robots to d [ ... ]



ShellShock - Yet Another Code Injection Vulnerability
25/09/2014

A vulnerability in the Bash shell has the Internet in full alarmed mode and like all good security holes it even has a catchy name - Shell Shock. How does it work?


More News

 

Last Updated ( Monday, 23 September 2013 )
 
 

   
RSS feed of news items only
I Programmer News
Copyright © 2014 i-programmer.info. All Rights Reserved.
Joomla! is Free Software released under the GNU/GPL License.