GitHub Copilot reviews: What devs are saying about it so far

GitHub Copilot is an AI that has been making a serious splash in the software development community. After the technical preview in 2021, GitHub made the tool generally available to developers in June of last year. And at the recent GitHub Universe in November, GitHub announced businesses would be able to purchase Copilot seat licenses.

For those unfamiliar with it, GitHub Copilot leverages the artificial intelligence model of OpenAICodex to suggest code and functions in real time from your editor, going beyond code completion.

GitHub certainly has high hopes for its new tool. In the announcement, it was described pretty dramatically: “Every once in a while, a new technology comes along that changes everything,” and GitHub suggests Copilot can increase developer productivity by as much as 55%.

At Zenhub, we’re pretty invested in discovering new ways developers can become more efficient. So, we asked a few developers to review GitHub Copilot for us and asked them how it impacted their day-to-day workflows and overall productivity. The results were mixed but insightful.

Overall GitHub Copilot review: improved productivity, decent accuracy

By and large, the consensus of the GitHub Copilot reviews was that it largely succeeds at its mission, improving the developer experience through enhanced efficiency and code quality. It saves time and eliminates busy work, making developers more productive and letting them focus on more exciting work.

Omer Ishag, a software engineer at Laimuna, has been using GitHub Copilot since it was in beta and is currently a paid subscriber. “I am using it mostly for my personal one-man startup, and I am very happy with it,” Ishag said. “I see a great deal of enthusiasm and excitement around it from my friends and colleagues as well. It is extremely useful for tasks that don’t require deep thinking and are repetitive, which is a lot of tasks in software development.” He adds that he considers it great value for his money and plans to use it as long as it’s available.

For software developer Edwin Miller, Copilot “is a helpful tool that can improve repositories.” Speaking with other developers, his general impression is that “developers seem to find GitHub Copilot helpful. Its suggestions are generally accurate, and it has become an integral part of some workflows.” He noted a couple of key aspects of this:

• It helps identify best practices and generally makes accurate recommendations. One developer noted it suggested removing an unused file, which improved repository performance.
• It helps identify potential problems and recommend changes to improve performance or address issues.

Developer Manual Thomas wasn’t sure what to make of GitHub Copilot at first. “But as I started to use it to generate code more and more, I began to realize just how valuable it is. The tool makes many suggestions based on my activity on GitHub, and its recommendations are usually spot-on. This has made it an essential part of my workflow, and I now turn to Copilot whenever I need help evaluating pull requests or choosing a good next step for my project.” He adds that the AI assistance saves time when digging through code and makes it easier to stay on top of his repository.

Critical GitHub CoPilot reviews: where there’s “room for improvement”

Unsurprisingly, though, developers had some notes. For some, it was a case of bad first impressions, such as for Aleksei Kankov. “My first impression was that it was a terrible thing. Most of the autosuggestions were not useful at all. So, after playing with it for a couple of days, I stopped using it.”

Kankov said he tried it again two months later and found it much improved. Fast forward to the official launch, though, and he decided not to spend the $10 a month for it. “But after spending a couple of weeks without Copilot, I realized that I had missed it. I missed the suggestions and the ability to write code quickly without thinking about it. And here we are. I’m using Copilot daily and am happy to pay for it.”

Katelynn Smith, a software developer and product manager at Airgram, has been making good use of Copilot but believes it needs more finetuning. “Some of the bizarre predictions can put you off completely. Also, the fact that I can’t place 100% trust in the suggestions provided by Copilot means that I have to spend more time reviewing the code. I certainly could do without that,” she said. “Even then, I still wouldn’t dare work without the tool. It ticks all the boxes most of the time, and it will certainly continue improving with time.”

Developer Edwin Miller noted the criticism he’s heard most: Copilot was too cautious. “Some developers have criticized it for being too conservative in its suggestions or for not identifying all potential problems.”

To Dolya Tolmachev, head of software engineering at TechStack Ltd., Copilot feels a bit like a solution in search of a problem. In Tolmachev’s experience, Copilot “shows itself well on some simple scripts, where everything goes straight, but as soon as the size grows or understanding of the context is necessary, it immediately fails.” While Copilot “copes well with writing some simple things,” he said, “these things are often well handled by the IDE autocompletion or encapsulated within libraries or at the level of the language you write in, so it is not entirely clear why in your application you need to reinvent the wheel.” That said, he certainly hasn’t written it off. “Definitely, we will follow and look at it further as the technology develops.”

Is GitHub Copilot changing how developers work?

Copilot reviewers also noted that beyond actual functionality, Copilot could change how they work and what they spend time on.

Developer Marco Rossini was skeptical of Copilot when it first came out, and while he’s integrated it into his workflow, he said that it has changed his approach to coding. “If you need to do something, just write a comment on what you need to do in the next lines of code, and you will get a suggestion from Copilot auto-complete.”

But to Ivan Chiklichki, a senior software engineer and resource manager, this change in approach could be risky. “Though Copilot aids in perfecting lines of code, programmers risk atrophying their creative muscles. Relying heavily on such AI pair programmers could lead to codependency, especially for novice developers.”

And this change in approach is at least partly why Copilot didn’t find a home with Tolmachov’s team. “Everyone was under the impression that it was not ready for serious work on large projects with many internal dependencies,” he said. “It lacks an understanding of the subject area and the environment or abstractions that are ubiquitous in programming that interferes with efficient work.”

Copilot isn’t the only way devs are enhancing productivity

Of course, Copilot is not the only AI coding assistance tool out there – we’ve named a few in our developer productivity tools blog if you want to fill in some of GitHub Copilot’s gaps. 

If you’re not convinced AI coding tools can help improve your team’s efficiency or want additional help with your productivity, don’t fret! There are many other ways to become a more efficient developer, from adopting new working techniques and processes to automated tools.  Check out how Zenhub has been enhancing other areas of a developer’s workflow with AI by joining our waitlist here.