IBM is launching a new set of AI software engineering agents designed to autonomously resolve GitHub issues. The agents are being made available in an open-source licensing model.

In the announcement made at IBM's TechXchange Conference in Florida, IBM demonstrated the collection of AI agents that are powered by open LLMs.

Avi Sil, Manager & Principal Research Scientist at IBM, Language & Agents, said the agent can take a developer's Github code-base and given any new issues/ bugs can automatically resolve those bugs. Sil said that unlike most SWE Agents that use Frontier/ closed-source models, IBM SWE-Agent is built only on top of open-source models, meaning that developers can run it on their code base without worrying of data leakage to LLM APIs. Sil also said that SWE-Agent achieves competitive performance compared to agents on the SWE-Bench leaderboard.

In a blog post about the release, IBM said that on average, the SWE agents can localize and fix problems within five minutes, and in testing, they managed a 23.7% success rate on the SWE-bench tests, which measure how efficiently AI agents can solve real-world problems found on GitHub. The score places the IBM SWE agent in the upper reaches of the SWE-bench leaderboard, above many other agents relying on massive frontier models, like GPT-4o and Claude 3.

SWE-Agent uses a "Mixture of Agents" paradigm that includes several open source models such as Meta LLaMa 3.1, Mistral AI Large and IBM's own IBM Granite LLMs.

IBM's goal is to offer cost efficient SWE agents to run wherever developer's code resides, including behind firewalls, and still achieve good performance, according to Ruchir Puri, chief scientist at IBM Research.

The tool will join other AI agents IBM Research has developed for developers, including one for editing lines of code based on developer requests, which relies on IBM's Granite LLM on watsonx, and another that can be used for developing and executing tests to ensure that code will run as intended.

In use, a developer can open a bug report they’ve received on GitHub, tag it with “ibm-swe-agent-1.0” and the agent will work in the background to find the troublesome code. Once it’s found the location, it will suggest a fix that the developer could implement to resolve the issue. The developer can then review the proposed fix, and either accept and implement the fix or potentially find out what other agents suggest.

IBM says it will be releasing a research paper on the new agents "soon".

More Information

SWE-bench tests

Related Articles

IBM Launches The Granite Code LLM Series

IBM Releases Watsonx Granite Models

IBM Announces WatsonX AI Platform

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info