GitHub Copilot Provides Productivity Boost
Written by Sue Gee   
Wednesday, 07 September 2022

GitHub set out to discover the impact its new AI-based tool, Copilot is having on developers' productivity. The results seem pretty conclusive - 88% of developers surveyed reported they were more productive and an empirical study revealed developers were 55% faster coding with Copilot that without it. 

copilotbanner

Launched in technical preview in June 2021, GitHub Copilot is described as "Your AI pair programmer" which uses the OpenAI Codex to suggest code and entire functions in real-time from your editor.

As Mike James explained in our report GitHub Copilot - Your Programming Pal:

GitHub Copilot is an AI-based system that will take notice of what you are coding and make suggestions for what it is you are about to type. You can think of it as IntelliSense on steroids, but users are claiming that it is a big step up. The idea is all based on the sort of AI that lead to the GP-3 language system. Take a very big network and train it on a huge corpus of natural language data and it will extract the regularities.

More details were provided by Kay Ewbank who reported on Copilot's release in June 2022

The system has been trained to work out how to complete lines and sections of code by taking completed code, deleting parts of it, and training the AI to correctly fill in the missing part. The AI means that the examples used are remembered, along with the underlying principles so that the system is capable of extrapolation to things that it has never seen.

By the time of its release it worked with with Neovim, JetBrains IDEs, Visual Studio, and Visual Studio Code.

From the outset GitHub was confident that Copilot would improve developer productivity and now Eirini Kalliamvakou has revealed the research findings to prove it, both qualitative and quantitative, on the GitHub blog

The qualitative results - which rely on user's perceptions - are from a survey of more than 2,000 developers in which questions were based on the SPACE productivity framework. SPACE is an acronym for Satisfaction, Performance, Activity, Communication, and Efficiency and the framework, which was devised by Microsoft and GitHub in collaboration with University of Victoria had previously been used by GitHub for its Good Day Project, which I reported on in What Ruins A Developer's Day.  

With regards to Productivity, an overwhelming 88% of respondees agreed with the statement

As to Satisfaction, 59% felt less frustrated when coding;  60% reported being more fulfilled with their job; and 74% agreed that using Copilot let them focus on more satisfying work.

Questions on Efficiency elicited the findings that Copilot helped them stay in the flow (73%) and preserve mental effort during repetitive tasks (87%). The researchers comment:

That’s developer happiness right there, since we know from previous research that context switches and interruptions can ruin a developer’s day, and that certain types of work are draining. 

Using Copilot also led to specific productivity gains with 88% reporting faster completion in general and 96% being faster with repetitive tasks.

To provided quantitative evidence the researchers conducted an empirical test in which 95 developers were split into two groups to determine their speed in completing the task of writing a web server in JavaScript with and without Copilot.

As shown in the infographic below those who used Copilot had a higher rate of completing the task (78%, compared to 70% for non-users) and even more impressive Copilot users were able to complete tasks 55% faster than those who didn’t use it.

copilotresults

At the end of the day the advantages of using AI Pair Programming seem to be pretty clear are are neatly summed up by a quote from a Senior Software Engineer who participated in the research:

“[With Copilot] I have to think less, and when I have to think it’s the fun stuff. It sets off a little spark that makes coding more fun *and* more efficient.”
 

copilotsq

More Information

Copilot Website

Related Articles

GitHub Copilot Released

GitHub Copilot - Your Programming Pal

What Ruins A Developer's Day   

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Banner


Alexa Teacher Models Outperform GPT-3
31/08/2022

Researchers at Amazon Alexa AI are making breakthroughs in conversational AI and natural language processing using models that learn new concepts and transfer knowledge from one languag [ ... ]



Jakarta EE 10 - A New Era For Java On The Cloud
23/09/2022

After a lot of iterations and false starts in terms of dates, Jakarta EE 10 is finally here and primarily focused on the Cloud and microservices.


More News

pythondata

 



 

Comments




or email your comment to: comments@i-programmer.info

Last Updated ( Wednesday, 07 September 2022 )