Apache Bigtop Adds OpenJDK 8 Support
Apache Bigtop Adds OpenJDK 8 Support
Written by Kay Ewbank   
Monday, 11 December 2017

Apache has released Bigtop 1.2.1 with support for OpenJDK 8, and a new sandbox feature that lets you run big data pseudo clusters on Docker. Bigtop is an Apache Foundation project that you can use for packaging, testing, and configuration of the big name open source big data components that make up the Hadoop infrastructure.

bigtop

Bigtop supports a wide range of components and projects, including Hadoop, HBase and Spark. The primary goal of Bigtop is to build a community around the packaging, deployment and interoperability testing of Hadoop-related projects. This includes testing at various levels, including packaging, platform, runtime, and upgrade, and focussing on the system as a whole, rather than individual projects.

While Hadoop is generally used to refer to the central collection of tools, Bigtop looks at the wider selection that makes up the Hadoop-related projects, including Hbase, Pig, Hive MapReduce, Zookeeper and Avro among others.  Bigtop packages Hadoop RPMs and DEBs, so that you can manage and maintain your Hadoop cluster, and it provides an integrated smoke testing framework, alongside a suite of 50 test files. It also helps with virtualization testing, with vagrant recipes, raw images, and (work-in-progress) docker recipes for deploying Hadoop from zero, and you can use Bigtop Provisioner to spin-up a virtual cluster with a single command.

The new release of Bigtop adds a Sandbox feature that lets you use Docker to run pseudo clusters. Creating it is a single line command, and for HDFS the process takes around 30 seconds. You then get a local WebUI to play around it. You can run HDFS and Spark standalong, or HDFS, Yarn, Hive and Pig - there are simple instructions on the Bigtop site.

This presentation from DataWorks Summit 2017 on using the Sandbox comes from Apache Bigtop Project Committer and PMC member, Evans Ye:

 
The new release also includes a faster Docker Provisioner which has been rewritten to fully embrace the Docker ecosystem. The OpenJDK support in the new release means that all the components are now built on JDK8.
The major components have been updated to recent versions, including Hadoop 2.7.3, Spark 2.1.1, HBase 1.1.9, and Zeppelin 0.72, and most of the ecosystem projects have also been updated, including Apex, Crunch, Flume, Ignite, Mahout, Oozie, and Phoenix, among others.

 bigtop

More Information

Apache Bigtop Site 

Related Articles

Hadoop Adds Resource Estimator 

Hadoop Adds In-Memory Caching

Hadoop SQL Query Engine Launched

Hadoop 2 Introduces YARN

Spark Gets NLP Library

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on, Twitter, FacebookGoogle+ or Linkedin.

 

Banner


As We Are - A 14-Foot Selfie
27/12/2017

How far we have come from "As We May Think" to "As We Are". The selfie has has transformed from a modern wonder to an irritation and occasionally a danger to life and limb. But the selfie can be art.  [ ... ]



Fear And Loathing In The App Store 19 - Apple Rejects Net Neutrality App
20/01/2018

UPDATE  Apple bowed to concerted pressure and the Wehe App has been approved by Apple and is now available in the App Store.

Initially the app, which tests to see if your ISP is applying throttl [ ... ]


More News

 

 
 

 

blog comments powered by Disqus

Last Updated ( Monday, 11 December 2017 )
 
 

   
Banner
RSS feed of news items only
I Programmer News
Copyright © 2018 i-programmer.info. All Rights Reserved.
Joomla! is Free Software released under the GNU/GPL License.