Running NVIDIA's Free LLMs Inside VSCode

🎯 The short version

You can use current LLMs for free with nothing but a phone number. No credit card. And you can drive those models from a coding agent inside VSCode. The setup is four fields.

API Provider : OpenAI Compatible
Base URL     : https://integrate.api.nvidia.com/v1
API Key      : nvapi-...
Model ID     : qwen/qwen2.5-coder-32b-instruct

That’s the whole answer. The rest of this post is why it works and where I tripped.

😵 Why I went looking

If you use Claude Pro seriously, you hit the rate limit. It always lands mid-edit, right when you had momentum. Adding another subscription didn’t feel like the answer either.

So I decided to keep a backup model on standby. Something I ignore most days and switch to only when the limit hits. Free would be nice.

📰 What the article taught me

The starting point was this post: LLMOps#1: Using the NVIDIA NIM API

NVIDIA NIM lends you models that NVIDIA has tuned for its own GPUs, served over an API. It began as a paid enterprise product, but they opened a free tier so developers can evaluate it.

Here’s the shape of it.

Item	Detail
Sign-up	Phone verification at build.nvidia.com
Cost	Free. No card on file, so nothing can be charged
Key lifetime	Six months
Models	100+, covering text, embeddings, images, and speech

I wondered why a free service would skip credit cards. Turns out the phone number is the anti-abuse check. That’s all they needed.

The OpenAI-compatible URL is the real key

Technically, one line matters more than the rest.

https://integrate.api.nvidia.com/v1

It means NVIDIA accepts requests shaped exactly like OpenAI’s. Think of wall sockets. Plug shapes differ by country, and NVIDIA decided to just use the common one.

That matters because most developer tools are already built to speak OpenAI’s shape. So you keep the tool, swap the address, and a different company’s model runs in its place. No code changes.

🔧 So I wired it into VSCode

The original article stops at calling the API from Python. I wanted to go one step further. What if a coding agent used this model instead of me calling it from a script?

VSCode has an extension called Cline. It isn’t just a chat window. It reads your project files, edits them, and runs terminal commands. Search the marketplace for Cline and install it.

Click the Cline icon in the left sidebar, then the gear icon at the top right. Four fields, and you’re done.

Field	Value
API Provider	`OpenAI Compatible`
Base URL	`https://integrate.api.nvidia.com/v1`
API Key	your `nvapi-...` key
Model ID	`qwen/qwen2.5-coder-32b-instruct`

To switch models, you change the Model ID field and nothing else. These are the three I keep around.

for coding      qwen/qwen2.5-coder-32b-instruct
heavyweight     nvidia/nemotron-3-super-120b-a12b
general         meta/llama-3.3-70b-instruct

Save, and it works immediately. The whole thing takes under two minutes.

❌ Codex refused

This might be the most useful section here.

I tried the same trick on the Codex CLI. Set some environment variables, point it at NVIDIA, done. That was the theory.

$env:OPENAI_API_BASE="https://integrate.api.nvidia.com/v1"
$env:OPENAI_API_KEY="nvapi-..."
$env:OPENAI_MODEL="qwen/qwen2.5-coder-32b-instruct"

codex

It doesn’t work. Codex ignores those variables and launches on its own default model. Recent versions read an internal config file and the official endpoint, and nothing else.

The workaround is to stand up a translator in the middle. LiteLLM does that, and it needs two terminals.

# Terminal 1 - run the proxy
pip install "litellm[proxy]" websockets
litellm --model openai/qwen/qwen2.5-coder-32b-instruct `
        --api_base https://integrate.api.nvidia.com/v1 `
        --port 1000

# Terminal 2 - point Codex at the proxy
$env:OPENAI_API_BASE="http://localhost:1000/v1"
$env:OPENAI_API_KEY="dummy"
codex

This does work. The Codex banner still shows its original model name, but the request travels through the proxy to NVIDIA. You can watch the proxy terminal log each call to confirm.

But I decided not to keep it. Two terminals running at all times, plus a proxy restart every time I change models, is a lot of ceremony for an emergency backup. Cline does it in two clicks.

So the split is this. If you want to switch models with a mouse, use Cline in VSCode. If you live in the CLI, use the LiteLLM proxy. I picked the first one.

💸 That 14.6k number is not money

After my first message, 14.6k appeared in the top right corner. My immediate reaction was that I’d burned fourteen thousand of something and it was going to cost me.

It isn’t money. It’s how much conversation the model is currently holding in memory. The actual cost sits next to it, and it reads $0.0000.

There’s a reason the number starts high. An agent like Cline sends your project structure and its own operating rules along with every message, even a one-word one. So you begin somewhere around 10k to 15k. That’s normal.

When you fill the 128k window, the model starts forgetting the early parts. Hit the + button for a new task and it resets to zero.

⚠️ Go easy on Auto-Approve!

Cline has an auto-approve setting that skips the confirmation dialog for reading files, editing them, and running terminal commands.

I strongly recommend enabling Read files only. If you auto-approve edits and command execution, the agent can touch the wrong file and you won’t notice. Read-only auto-approval already removes most of the friction while keeping things safe!

💡 Quick tips for getting more out of Cline

Cline isn’t just a basic chat AI. It’s a full-fledged Agent that directly reads your project directory, creates/modifies files, and executes terminal commands.

Here are two core features worth knowing:

Mentioning specific files/folders (using @) Type @ in the chat window to quickly pick a file or directory from your project.

Example: @src/main.py Refactor the exception handling logic in this file
Auto-suggesting terminal commands (Terminal Execution) When package installation (pip install, npm install) or test runs (pytest) are needed after code modifications, Cline automatically suggests terminal commands. Just hit the Run Command button at the bottom to execute it right away!

💭 Closing thoughts

The best part is that I got a backup without adding a single subscription. Hitting the limit doesn’t rattle me now. I just open Cline and keep going!

And failing on Codex turned out to be the more valuable half. I learned that “just set the environment variables” was a guess, not a fact, and that tools differ in which configuration they’ll even look at. Success would have taught me none of that.

One thing is still open. I want to run the same prompt through all three models and compare speed and output quality, but I haven’t measured properly yet. I’ll write that up separately once I have real numbers! 😃

Let’s keep figuring it out together.