My initial experiences with Tabnine AI code assistant (hopeful)

So I signed up for the 90-day free trial to Tabnine to explore it's AI code assistance capabilities in a real-life scenario that I was working on.

As stated on their website, they support a number of IDEs and I happen to be using VS Code. Instructions are provided on how to install the Tabnine module (i.e., plugin) on VS Code.

Feature: Code Completion

Code completion is pretty much available with Tabnine on the fly as you're typing. This is no different than what we see in Google and Microsoft products such as Outlook these days. It can be a blessing or an annoyance based on how you look at it.

Do you know how Google seems to be able to read your mind when you're typing up a query in the search bar? I had the same feeling with Tabnine's auto-code-completion.

It appears that Tabnine does not operate as your standard IDE auto-complete where it just shows you the options available for the function you're typing in, but rather it has already analyzed the entire code and provides more relevant recommendations.

Verdict: GOOD

Feature: Troubleshooting Errors

As you can see in line 81, I have some type of error in that line of code, denoted by the light bulb icon. And during execution, I received the error SyntaxError: 'await' outside async function. Normally I'd leverage Google or ChatGPT to figure out what's going on, but I clicked on the light bulb and used the troubleshooting feature.

Upon using the Fix with Tabnine feature the first time, you're prompted to select a model for Tabnine Chat. Do you want Codestral? Claude? ChatGPT?

Eventually it provided a recommendation as shown in the code snippet in the black box below. Though the error did in fact disappear, it did not solve my coding problem because by removing the await keyword, it converted my async call to a sync one (which would not have worked anyway with the openai.ChatCompletion.acreate API that I was using).

ChatGPT didn't give me the exact answer either, but the solution was hidden in plain sight in my chat history. I seemed to have forgotten to add the async keyword before the function name.

Verdict: AVERAGE

Feature: Fixing Errors

I typed the wrong variable names in the errored line below (v_title = title1 + title2). This was a simple error but I don't know why I didn't figure it out in a few seconds (old age maybe?). Clicking on the bulb, I selected "Fix with Tabnine" and it actually fixed the error, replacing the incorrect variables names with the correct ones.

Then... I tried repeating the same action to capture screenshots for this blog. Every subsequent fix by Tabnine resulted in what you see in the screenshot below.

First, it kept the old invalid line. Then it stripped of half the line in the last line of code shown. Finally, it injected the string ">>>>>>> Tabnine >>>>>>>" in my code just for kicks sake.

I'm not sure what caused this mixup after it worked perfectly the first time.

Verdict: NOT GOOD

Feature: Code Generation

I had a function called read_word_document() and I wanted to create a similar read_excel_document() one. If you notice on the screenshot, you'll see an "Ask" button. I clicked it and asked Tabnine to "Create a function similar to this but call it read_excel_document() and have it read a Microsoft Excel document instead of a Microsoft Word document".

It did the job and gave me a 2 line function which I copied and pasted into my code.

It did the job, but... when I asked ChatGPT (gpt-4o) to do the same thing, the function was a little more elaborate, adding additional logic to convert the Excel dataframe to a string, row by row.

In the next screenshot, you can see my original Excel content, and how it was created using the ChatGPT generated function and the Tabnine generated function. The outputs and formatting were slightly different.

The ChatGPT function worked perfectly, but for some odd reason it removed the first line. The Tabnine function also did the job, but it looked ugly.

Verdict: AVERAGE

Is Tabnine increasing my coding efficiency? Yes. Up to 40%? That's questionable based on my brief experience with it.

Security considerations aside, as a developer, would I personally pay $12/month to get Tabnine Pro? Probably. The reason it's not a firm 'yes' is because I'd still have to be paying another $20/month to ChatGPT to augment my development efforts. But these code assist tools are indeed helpful.

There is a Tabnine Enterprise deployment model which allows for self-hosting, among a slew of other features, that may be better suited for the privacy needs of a corporation. The cost jumps to $39/month/developer.