AI Code Generation

Comparing Cline and GitHub Copilot for Autonomous AI Code Generation

Ahmed Aboulnaga

• 6 min read
Comparing Cline and GitHub Copilot for Autonomous AI Code Generation

This will be a quick blog post as I share my raw feedback. I haven't captured screenshots and this won't be an exhaustive comparison, but it will describe my initial reactions to doing the same exercise in both Cline and GitHub Copilot.

I previously conducted basic tests with Cline in a previous blog post and it was generally positive. One disadvantage that hampered adoption of Cline within our organization was it does not support any IDE other than VS Code.

But for those who know me, I'm interested in not only AI code generation, but autonomous AI code generation. I want to explore true autonomous coding, not just an AI companion or AI assistant.

Let's take a look at an example I took.

My Use Case - Developing a C# REST API

For the record, I am not a .NET developer let alone a C# developer. I don't know anything about the .NET framework. But here's what I wanted to develop:

(1) Create a REST web service in .NET C#
(2) This web service will have two operations, a GET operation which will read from a CSV file, and a POST operation which will insert a new entry into the CSV file
(3) For the POST operation, this web service will accept the following input in JSON format: first name (text), last name (text), date of hire (date), salary (numeric), and it will save this information in a comma-delimited CSV file in C:\temp with the filename cline_data.csv, and it will return an HTTP 200 if successful
(4) For the GET operation, this web service will accept the following input: first name (text), and it will open the CSV file and locate all records that match the first name, and return the results in JSON format

I used this as the basis of my autonomous coding exercise in Cline and Copilot.

My Results - Cline vs. GitHub Copilot

I used the Cline and Copilot plugins in VS Code. Cline requires an API key to some provider, in which I used Anthropic, so there is a cost per call (my entire development exercise here cost about $1). Copilot is free for individual use and requires a GitHub account (you can choose an underlying model such as GPT-4o or Claude 3.5 Sonnet).

Cline is an extension with the prompts appearing on the left as shown in the screenshot below, while Copilot, although also an extension, appears in the sidebar on the right.

Cline

  • Autonomous Coding [EXCELLENT]. Zero lines of code were written by me.
  • Code Fixing [EXCELLENT]. The code first generated was not working. Repeated prompts eventually fixed it. When an issue is encountered, Cline asks for permission to update the files and performed the fix.
  • Explanation [EXCELLENT]. Full, detailed explanations along the way.
  • ReadMe [EXCELLENT]. See below. Cline went above and beyond and created batch scripts for Windows, Linux, and Mac
# Employee Service

A REST web service built with ASP.NET Core that provides operations to manage employee data in a CSV file.

## Features

- **GET Operation**: Retrieve employees by first name
- **POST Operation**: Add a new employee to the CSV file

## API Endpoints

### GET /api/employees

Retrieves employees by first name.

**Query Parameters:**
- `firstName` (required): The first name to search for

**Example Request:**
```
GET /api/employees?firstName=John
```

**Example Response:**
```json
[
  {
    "firstName": "John",
    "lastName": "Doe",
    "dateOfHire": "2023-01-15T00:00:00",
    "salary": 75000.00
  },
  {
    "firstName": "John",
    "lastName": "Smith",
    "dateOfHire": "2022-05-10T00:00:00",
    "salary": 82000.00
  }
]
```

### POST /api/employees

Adds a new employee to the CSV file.

**Request Body:**
```json
{
  "firstName": "Jane",
  "lastName": "Doe",
  "dateOfHire": "2023-03-20T00:00:00",
  "salary": 78000.00
}
```

**Response:**
- HTTP 200 OK if successful
- HTTP 400 Bad Request if the request is invalid
- HTTP 500 Internal Server Error if there's a server error

## CSV File

The service stores employee data in a CSV file located at `C:\temp\cline_data.csv`. The file has the following format:

```
FirstName,LastName,DateOfHire,Salary
John,Doe,2023-01-15,75000.00
Jane,Doe,2023-03-20,78000.00
```

## Running the Service

### Using the Run Scripts

#### Windows
1. Ensure you have .NET SDK installed
2. Double-click the `RunService.bat` file or run it from the command line

#### Linux/macOS
1. Ensure you have .NET SDK installed
2. Make the script executable: `chmod +x run-service.sh`
3. Run the script: `./run-service.sh`

### Manual Run
1. Ensure you have .NET SDK installed
2. Navigate to the project directory
3. Run the following command:
   ```
   dotnet run
   ```

The service will be available at:
- HTTP: http://localhost:5000
- HTTPS: https://localhost:5001

## Web Interface

A simple web interface is available to test the service:
- Open a web browser and navigate to `http://localhost:5000/index.html`
- The interface allows you to:
  - Add new employees
  - Search for employees by first name

## Swagger Documentation

Swagger documentation is available at `/swagger` when running the service in development mode.

## PowerShell Test Script

A PowerShell script (`TestService.ps1`) is included to test the service from the command line:
1. Start the service using one of the methods above
2. Open a PowerShell window
3. Navigate to the project directory
4. Run the script: `.\TestService.ps1`
  • Above and beyond [EXCELLENT]. Cline developed a simple, clean HTML form to assist in the testing.
  • Code Quality [EXCELLENT]. Cline developed the project on the .NET Framework 8.x and the code ran the first time (though it had some bugs).

Copilot

  • Autonomous Coding [EXCELLENT]. Zero lines of code were written by me.
  • Code Fixing [AVERAGE]. The code first generated was not working. Repeated prompts eventually fixed it, but there were much more errors during the build, startup, and runtime in Copilot compared to Cline (about 3x more), taking more time to resolve. Secondly, Copilot gives you a few recommendations in which you have to step through one by one, and you must manually open the correct file before asking Copilot to make the fix.
  • Explanation [AVERAGE]. Inferior compared to Cline.
  • ReadMe [VERY GOOD]. See below.
# Csv Web Service

This project is a RESTful web service built using .NET that provides operations to read from and write to a CSV file. It includes a GET operation to retrieve entries based on a first name and a POST operation to add new entries to the CSV file.

## Project Structure

```
CsvWebService
├── Controllers
│   └── CsvController.cs
├── Models
│   └── CsvEntry.cs
├── Services
│   └── CsvService.cs
├── Data
│   └── sample.csv
├── CsvWebService.csproj
├── Program.cs
├── Startup.cs
└── README.md
```

## Setup Instructions

1. **Clone the repository**:
   ```
   git clone <repository-url>
   cd CsvWebService
   ```

2. **Restore dependencies**:
   ```
   dotnet restore
   ```

3. **Run the application**:
   ```
   dotnet run
   ```

4. **Access the API**:
   - The API will be available at `http://localhost:5000`.

## API Endpoints

### GET /api/csv?firstName={firstName}

- **Description**: Retrieves entries from the CSV file filtered by the provided first name.
- **Parameters**:
  - `firstName`: The first name to filter the entries.
- **Response**: Returns a JSON array of matching entries.

### POST /api/csv

- **Description**: Adds a new entry to the CSV file.
- **Request Body**: A JSON object representing a `CsvEntry` with the following properties:
  - `FirstName`: string
  - `LastName`: string
  - `DateOfHire`: DateTime
  - `Salary`: decimal
- **Response**: Returns HTTP 200 status if successful.

## Example Usage

### GET Example

```
GET http://localhost:5000/api/csv?firstName=John
```

### POST Example

```
POST http://localhost:5000/api/csv
Content-Type: application/json

{
  "FirstName": "Jane",
  "LastName": "Doe",
  "DateOfHire": "2023-01-01T00:00:00",
  "Salary": 60000.00
}
```

## Notes

- Ensure that the CSV file path is correctly set in the application.
- The sample CSV file can be found in the `Data` directory for testing purposes.
  • Code Quality [GOOD]. Copilot developed the project on the older .NET Frameworks 6.x and 8.x and one of the files generated was missing closing brackets.

My Final Thoughts

Cline was far superior from the get-go. It provides a "Plan" phase detailing out all the requirements and provides you with a plan on what it intends to do. You have the opportunity to update or correct it at that point. Then the "Act" phase actually creates the code.

Cline went above and beyond in many areas, from testing to debugging to documentation. The amount of bug fixes, troubleshooting, and getting the code to work was much easier and faster with Cline as compared to Copilot. I'm not sure had I chosen gpt-4o as the model for Cline if it would have performed similarly to Copilot or not. Keep in mind that Cline supports Claude 3.7 Sonnet while Copilot only support Claude 3.5 Sonnet at the time of writing.

At this point, I would find it hard pressed to recommend Copilot over Cline. Cline's biggest limitation is that the fact that the plugin is only available on VS Code.