| IBM Open Sources CodeFlare |
| Written by Kay Ewbank |
| Friday, 14 January 2022 |
|
IBM has announced improvements to CodeFlare, its serverless framework that aims to reduce the time and effort developers spend training and preparing AI and machine learning models for deployment in hybrid cloud environments. CodeFlare has also now been made open source. CodeFlare is a framework that simplifies the integration, scaling and acceleration of complex multi-step analytics and machine learning pipelines on the cloud.
CodeFlare can be used for pipeline execution and scaling with tools to define and execute parallel pipelines. The improved version of CodeFlare has been refined so that it can take on foundation models, and is now available as open-source software. The CodeFlare team says this effectively takes CodeFlare from an exploratory tool for data science researchers to a tool that can automate AI and ML workflows on IBM’s hybrid cloud. The main improvement to the actual framework is faster, automated foundation model training. Foundation models underpin much of machine analysis, but it takes a long time, running into weeks, to gather and train an AI model on the right body of data, combined with multiple upstream and downstream tasks. The CodeFlare team's goal is to make the generation of downstream models ‘one-click’ easy for data scientists. Foundation model pipelines consist of a sequence of multiple, often heterogeneous steps that can be difficult to create. CodeFlare provides a Python-based interface for the foundation model pipelines, making it possible to fully automate the tasks of preprocessing, validating, and adapting foundation models. The developers provide an example of sentiment analysis in which CodeFlare starts off by cleaning up the input data, including de-duplication, and removing unsafe or biased content. Following this CodeFlare tunes a foundation model for all of the specific tasks needed for the organization’s sentiment analysis. The developers say that: "with just a few lines of code, a data scientist can operationalize hundreds of such pipelines and automate these tasks whenever they need to make any changes. Our goal is to make generating downstream models as easy for data scientists as a single mouse click." CodeFlare is available on GitHub now.
More InformationRelated ArticlesIBM Introduces Hybrid Cloud AI/ML Framework IBM Releases CodeNet Dataset For AI Coding Google Comics Factory Makes ML Easy Nearly A Third Of Devs Using AI And ML
To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin. 
Comments
or email your comment to: comments@i-programmer.info |
