Using Local AI Models for Salesforce Development

The email came from InfoSec on a Tuesday. We'd been trialling an AI coding assistant on a financial services build, and the security lead — Anneliese, who'd read every line of the vendor's data-processing addendum — had one question that ended the trial. Where does the source code go when the model autocompletes a method? The honest answer was "to a server we don't control." That was the end of cloud AI on that project.

This is a common wall at APRA-regulated orgs and government agencies. The source code is the intellectual property, the schema leaks business logic, and legal will not sign off on shipping either to a third-party inference endpoint, no matter how clean the SOC 2 report looks. For a while that meant no AI assistance at all. Then I started running models locally, and the wall turned into a workflow.

What Running a Model Locally Actually Means

Local means the model weights sit on a disk you own and inference happens on hardware you control. Nothing leaves the machine. No API key, no network call, no data-processing agreement to argue over. The tool I reach for is Ollama — it wraps the awkward parts of running open-weight models behind a couple of commands.

# Pull a coding model and run it — the weights live on your disk, nothing phones home
ollama pull qwen2.5-coder:32b
ollama run qwen2.5-coder:32b

# Or drive it straight from the CLI with a prompt
ollama run qwen2.5-coder:32b "Write an Apex test for AccountTriggerHandler.afterUpdate, bulk case, 200 records"

That last command runs entirely on the box. You can pull the network cable out of the wall and it keeps working, which is the demonstration that actually convinces a security team. I've literally done that in a meeting room to end an argument. Once someone watches a model generate a test class with the ethernet unplugged, the compliance conversation is over.

The Hardware Is the Whole Game

Here's the part the enthusiastic posts skip. The quality of a local model tracks its size, and size costs memory. A 7B model runs on almost anything. A 32B model at 4-bit quantisation wants around 19 to 20GB of VRAM, which in practice means a single 24GB card like an RTX 4090, or nothing. A 70B model wants two of those, or a very patient CPU and an afternoon to spare.

On that financial services build we settled on Qwen2.5-Coder at 32B, 4-bit, on a shared workstation with a single 24GB card. It scored around 28 tokens a second — fast enough to feel usable, slow enough that you'd notice the wait on a long test class. I wanted the 70B for the extra accuracy. It wouldn't fit, and the budget for a second card never materialised before the quarter closed. So we ran the 32B and lived with it. Six developers, one shared box, a booking spreadsheet like it was 2009. Over the quarter one of them, Ravi, generated something like 40 test classes against it. Good enough. Not great. The performance tuning is still on the list I never get back to.

A 7B model is a toy for Apex, and I'll say that plainly. It autocompletes a single line fine. Ask it for a bulkified handler method and it produces something that compiles and quietly assumes it will only ever see one record. The jump from 7B to 32B is the difference between a demo and a tool, and that jump is entirely a hardware conversation before it's a software one.

The Models Worth Your Disk Space

By late 2024 the open-weight coding models are genuinely usable, which was not true even a year earlier. Qwen2.5-Coder is my default — the 32B handles Apex idioms, understands trigger context, and writes reasonable SOQL. Codestral at 22B is a strong second and lighter on memory if you're stuck on a 16GB card. DeepSeek-Coder-V2 is worth a look if your work leans heavily into integration code and you want the broader language coverage.

None of them know your org. This is where local and cloud models fail identically — they'll write SOQL that's syntactically perfect and operationally dangerous, because they cannot see whether External_Id__c is actually indexed or whether Industry has five picklist values or fifty. The SOQL patterns I check by hand are the exact ones I verify in local output, every single time.

The failure that stuck with me was subtler than a syntax error. The 7B model handed back this, with a comment above it claiming the query was secure:

// Securely fetch accounts for the given industry
List<Account> accts = [
    SELECT Id, Name, AnnualRevenue
    FROM Account
    WHERE Industry = :industry
];

It isn't secure. No field-level security check, no sharing enforcement, no LIMIT. The model had pattern-matched the word "secure" from its training data onto code that would fail a review on sight. If you're generating Apex that touches regulated data, you own the CRUD and FLS enforcement yourself. The model will happily lie about it in a comment, and a junior who trusts the comment ships the gap.

Wiring Ollama Into the Editor

Running the model in a terminal is fine for one-off questions. For day-to-day work I wire Ollama into the editor with the Continue extension, which speaks to the local endpoint the same way it would to a cloud provider. The config just points at localhost.

{
  "models": [
    {
      "title": "Qwen2.5 Coder 32B (local)",
      "provider": "ollama",
      "model": "qwen2.5-coder:32b",
      "apiBase": "http://localhost:11434"
    }
  ],
  "tabAutocompleteModel": {
    "title": "Qwen2.5 Coder 7B (autocomplete)",
    "provider": "ollama",
    "model": "qwen2.5-coder:7b"
  }
}

I run the big model for chat and generation and a small 7B for inline autocomplete, because autocomplete needs to be fast far more than it needs to be smart. The whole thing works offline. On a plane, on a locked-down VDI, in a government environment where the network itself is the threat model, the workflow doesn't change at all. That portability is the quiet second benefit that has nothing to do with compliance.

Where Cloud Models Still Win

I'm not going to pretend this is a free lunch. A frontier cloud model is better than anything I can run on a single card, and it isn't close for the hard stuff. When the constraint isn't compliance, the cloud workflow I run with Claude Code is faster and produces better first drafts, especially on multi-file refactors where the model needs to hold a lot of context at once. Local models have smaller context windows and less headroom, and you feel it the moment a task spans more than a couple of files.

The trade is capability for control. If your data is genuinely fine to send to a cloud endpoint, send it — you'll move faster and the drafts will be cleaner. Local models are what you reach for when "send it" was never on the table. Treating one as a replacement for the other in every situation is how you end up disappointed with both.

What I'd Do Differently

I'd size the hardware before choosing the model. We picked the model we wanted and then discovered the card couldn't hold it, which is exactly backwards. Decide the tokens-per-second you can tolerate and the model tier you actually need, then buy the VRAM that clears it. A second 24GB card up front would have paid for itself in the hours we spent nursing the 32B through work the 70B would have handled cleanly.

I'd also set expectations with the team earlier. Developers coming off polished cloud tools feel the drop in quality immediately, and if nobody frames it as "this is the AI you're allowed to have, and it's still a real gain over none," they quietly stop using it within a fortnight. Frame the constraint honestly and it sticks. Leave it unsaid and the workstation goes idle. If you're standing up a local stack for a regulated org and want a second pair of eyes on the model choice, the hardware, and the review discipline that has to sit around it, that's the kind of review I do.

Cloud AI gives you the best model and asks you to trust someone else with your source code. Local AI gives you a worse model and asks you to trust no one. For a lot of the organisations I work with, that second trade is the only one they're allowed to make.