Avante - A Cursor like experience for Neovim

This is a continuation of my previous blog entry about setting up Neovim with Ollama but this time we are using new plugin called Avante which provides a Cursor like experience.

If you are not familiar with Cursor it's an IDE leveraging AI to increase developer productivity by providing features like code suggestions and an integrated AI chat which is able to reference your code.

Installing the plugin

Let's start by installing and configuring it via the Neovim plugin manager of choice. In my case I'm using lazy.nvim, but you can find other examples in the README of the Avante GitHub repository.

require('lazy').setup({
  {
    "yetone/avante.nvim",
    event = "VeryLazy",
    lazy = false,
    version = false, -- Set this to "*" to always pull the latest release version, or set it to false to update to the latest code changes.
    opts = {
      -- add any opts here
    },
    -- if you want to build from source then do `make BUILD_FROM_SOURCE=true`
    build = "make",
    -- build = "powershell -ExecutionPolicy Bypass -File Build.ps1 -BuildFromSource false" -- for windows
    dependencies = {
      "stevearc/dressing.nvim",
      "nvim-lua/plenary.nvim",
      "MunifTanjim/nui.nvim",
      --- The below dependencies are optional,
      "echasnovski/mini.pick", -- for file_selector provider mini.pick
      "nvim-telescope/telescope.nvim", -- for file_selector provider telescope
      "hrsh7th/nvim-cmp", -- autocompletion for avante commands and mentions
      "ibhagwan/fzf-lua", -- for file_selector provider fzf
      "nvim-tree/nvim-web-devicons", -- or echasnovski/mini.icons
      -- "zbirenbaum/copilot.lua", -- for providers='copilot'
      {
        -- support for image pasting
        "HakonHarnes/img-clip.nvim",
        event = "VeryLazy",
        opts = {
          -- recommended settings
          default = {
            embed_image_as_base64 = false,
            prompt_for_file_name = false,
            drag_and_drop = {
              insert_mode = true,
            },
            -- required for Windows users
            use_absolute_path = true,
          },
        },
      },
      {
        -- Make sure to set this up properly if you have lazy=true
        'MeanderingProgrammer/render-markdown.nvim',
        opts = {
          file_types = { "markdown", "Avante" },
        },
        ft = { "markdown", "Avante" },
      },
    },
  }
})

Choosing a Model

When choosing a model it is essential to account for your available hardware capabilities. As a general rule of thumb, the size of your VRAM should be at least 1.2 times greater than the size of the model you are trying to run.

For example given the table below which contains the currently available options of the DeepSeek-R1 reasoning model we would need 10.8 GB of VRAM or more to run the 14b parameter model.

We can further test this with a couple of commands. First we are going to check how Ollama is actually going to allocate our resources for running the model.

ollama run deepseek-r1:14b
>>> /bye
ollama ps
NAME               ID              SIZE     PROCESSOR    UNTIL
deepseek-r1:14b    ea35dfe18182    11 GB    100% GPU     4 minutes from now

As you can see we are able to fit the full 14b parameter model into the VRAM of our GPU. Next we are going to look at the general performance of the model when we query it. Anything above 10 tokens/s should provide a somewhat acceptable experience for our use case.

ollama run deepseek-r1:14b --verbose
>>> What is the largest animal in the world?

...

total duration:       13.010056912s
load duration:        12.018553ms
prompt eval count:    12 token(s)
prompt eval duration: 5ms
prompt eval rate:     2400.00 tokens/s
eval count:           534 token(s)
eval duration:        12.992s
eval rate:            41.10 tokens/s

Configuring the Model

Since we already went over how to install Ollama last time I'm not going to reiterate that here, so please refer to the previous entry or the official documentation. At the time of writing using a local LLM via Ollama is not officially supported by the plugin, but the community has provided a workaround in one of the issues of the project.

-- Ollama API Documentation https://github.com/ollama/ollama/blob/main/docs/api.md#generate-a-completion
local role_map = {
  user = "user",
  assistant = "assistant",
  system = "system",
  tool = "tool",
}

---@param opts AvantePromptOptions
local parse_messages = function(self, opts)
  local messages = {}
  local has_images = opts.image_paths and #opts.image_paths > 0
  -- Ensure opts.messages is always a table
  local msg_list = opts.messages or {}
  -- Convert Avante messages to Ollama format
  for _, msg in ipairs(msg_list) do
    local role = role_map[msg.role] or "assistant"
    local content = msg.content or "" -- Default content to empty string
    -- Handle multimodal content if images are present
    -- *Experimental* not tested
    if has_images and role == "user" then
      local message_content = {
        role = role,
        content = content,
        images = {},
      }

      for _, image_path in ipairs(opts.image_paths) do
        local base64_content = vim.fn.system(string.format("base64 -w 0 %s", image_path)):gsub("\n", "")
        table.insert(message_content.images, "data:image/png;base64," .. base64_content)
      end

      table.insert(messages, message_content)
    else
      table.insert(messages, {
        role = role,
        content = content,
      })
    end
  end

  return messages
end

local parse_curl_args = function(self, code_opts)
  -- Create the messages array starting with the system message
  local messages = {
    {
      role = "system",
      content = code_opts.system_prompt,
    },
  }

  -- Extend messages with parsed conversation messages
  vim.list_extend(messages, self:parse_messages(code_opts))
  -- Construct options separately for claritty
  local options = {
    num_ctx = (self.options and self.options.num_ctx) or 4096,
    temperature = code_opts.temperature or (self.options and self.options.temperature) or 0,
  }

  -- Check if tools table is empty
  local tools = (code_opts.tools and next(code_opts.tools)) and code_opts.tools or nil
  -- Return the final request table
  return {
    url = self.endpoint .. "/api/chat",
    headers = {
      Accept = "application/json",
      ["Content-Type"] = "application/json",
    },
    body = {
      model = self.model,
      messages = messages,
      options = options,
      -- tools = tools, -- Optional tool support
      stream = true, -- Keep streaming enabled
    }
  }
end

local parse_stream_data = function (data, handler_opts)
  local json_data = vim.fn.json_decode(data)
  if json_data then
    if json_data.done then
      handler_opts.on_stop({ reason = json_data.done_reason or "stop" })
      return
    end

    if json_data.message then
      local content = json_data.message.content
      if content and content ~= "" then
        handler_opts.on_chunk(content)
      end
    end

    -- Handle tool calls if present
    if json_data.tool_calls then
      for _, tool in ipairs(json_data.tool_calls) do
        handler_opts.on_tool(tool)
      end
    end
  end
end

---@type AvanteProvider
local ollama = {
  api_key_name = "",
  endpoint = "http://127.0.0.1:11434",
  model = "deepseek-r1:14b",
  parse_messages = parse_messages,
  parse_curl_args = parse_curl_args,
  parse_stream_data = parse_stream_data,
}

-- Configure provider via plugin options
require('lazy').setup({
  {
    "yetone/avante.nvim",
    ...
    opts = {
      auto_suggestions_provider = "ollama",
      debug = true,
      provider = "ollama",
      vendors = {
        ollama = ollama,
      },
    },
    ...
  }
  ...
})

Usage

After we are done with the configuration we can now interact with Avante either by pressing Leader a a (the default Neovim leader key is the backslash key) to open the Avante Chat window and entering our question there or alternatively via the :AvanteAsk [question] command. Avante will then generate code suggestions in reference to the current file using the configured model which we can then apply one by one.

Another option is to select some code in visual mode and then press the key binding Leader a e and specify what we would like to be modified about it to trigger another code suggestion.

Sources

• ollama/ollama - Official GitHub Repository
• avante.nvim - Plugin by yetone.
• Avante + Ollama #1 - Issue.
• Avante + Ollama #2 - Issue.