| IBM Introduces Hybrid Cloud AI/ML Framework |
| Thursday, 15 July 2021 |
|
IBM is introducing CodeFlare, a serverless framework that aims to reduce the time and effort developers spend training and preparing AI and machine learning models for deployment in hybrid cloud environments. CodeFlare is a framework to simplify the integration, scaling and acceleration of complex multi-step analytics and machine learning pipelines on the cloud.
CodeFlare can be used for pipeline execution and scaling with tools to define and execute parallel pipelines. The developers say it unifies pipeline workflows across multiple frameworks while providing nearly optimal scale-out parallelism on pipelined computations. It supports a serverless user experience by integrating Red Hat OpenShift and IBM Cloud Code Engine and providing adapters and connectors for loading data and connecting to data services. CodeFlare is built on top of Ray, an open-source distributed computing framework for machine learning applications. CodeFlare adds elements to make it easier to scale workflows, so extending the capabilities of Ray. CodeFlare also has a Python-based interface for creating pipelines that simplifies the task of training and optimizing the model through data cleaning, feature extraction, and model optimization. The researchers say the goal of the new framework is to unify pipeline workflows across multiple platforms without requiring data scientists to learn a new workflow language. The team says CodeFlare makes the process of analyzing and optimizing pipelines significantly faster, giving the example of one user using the framework for approximately 100,000 pipelines for training machine learning models, who found CodeFlare cut the time it took to execute each pipeline from 4 hours to 15 minutes. CodeFlare has been made available open source on GitHub, and the developers have written a series of technical blog posts on how it works and what you need to get started using it. They say they've started applying this tech to things they’re building at IBM in AI research. Future plans include support for more complex pipelines, enhanced fault-tolerance and consistency, along with improved integration and data management for external sources, and adding support for pipeline visualization.
More InformationRelated ArticlesIBM Releases CodeNet Dataset For AI Coding Google Comics Factory Makes ML Easy Nearly A Third Of Devs Using AI And ML To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin. 
Comments
or email your comment to: comments@i-programmer.info |
| Last Updated ( Thursday, 15 July 2021 ) |
