Benchmarking MiniMax 2.5 vs Llama 3 for Coding Performance

LLM Benchmarks 2026 Comparison

Leaderboard scores measure performance in controlled, often cloud-hosted conditions with full-precision weights, unlimited VRAM, and curated evaluation sets. Developers running LLM benchmarks 2026 on local hardware face a fundamentally different reality. This article fills that gap with reproducible benchmarks across coding accuracy, reasoning, inference speed, and memory consumption on two GPU tiers.

Table of Contents

• Why Benchmark Local Models Yourself?
• Methodology — How We Tested
• The Contenders — Model Profiles
• Results — Coding Performance
• Results — Inference Speed
• Results — Memory and VRAM Usage
• Results — Reasoning and Creative Tasks
• The Verdict — Choosing the Right Model
• How to Run These Benchmarks Yourself

Why Benchmark Local Models Yourself?

Leaderboard scores measure performance in controlled, often cloud-hosted conditions with full-precision weights, unlimited VRAM, and curated evaluation sets. Developers running LLM benchmarks 2026 on local hardware face a fundamentally different reality: quantized weights, constrained VRAM budgets, and prompts shaped by actual project needs rather than academic datasets. The gap between a model's reported MMLU score and its usefulness generating a working Express.js endpoint on an RTX 3060 can invalidate procurement decisions made on leaderboard data alone. In our own tests, a model scoring 79% reasoning accuracy dropped to 55% Pass@1 on practical coding prompts under Q4_K_M quantization.

More local models ship in 2026 than in any prior year. MiniMax2.5 now competes alongside established contenders like Meta's Llama 3, Mistral Large 2, and Google's Gemma 2. No existing coding model comparison pits all four against each other using real-world developer tasks on consumer hardware. This article fills that gap with reproducible benchmarks across coding accuracy, reasoning, inference speed, and memory consumption on two GPU tiers.

MiniMax2.5 surprises on coding tasks but pays for it in VRAM; Llama 3 70B remains the quality leader where hardware allows; Mistral Large 2 offers the strongest balance of speed and capability; and Gemma 2 9B is the clear pick for constrained hardware.

The short version for scanners: MiniMax2.5 surprises on coding tasks but pays for it in VRAM; Llama 3 70B remains the quality leader where hardware allows; Mistral Large 2 offers the strongest balance of speed and capability; and Gemma 2 9B is the clear pick for constrained hardware. The details, trade-offs, and scripts to verify all of this follow below.

For broader context on setting up a local inference environment, the Complete Guide to Running LLMs Locally covers tooling fundamentals and configuration strategies that complement the benchmarking approach described here.

Methodology — How We Tested

Hardware Configuration

We ran all benchmarks on two GPU tiers to reflect the range of hardware developers actually use:

• High-end tier: NVIDIA RTX 4090 (24 GB VRAM), AMD Ryzen 9 7950X, 64 GB DDR5 RAM, Ubuntu 24.04 LTS
• Consumer tier: NVIDIA RTX 3060 12 GB, Intel Core i7-12700, 32 GB DDR4 RAM, Ubuntu 24.04 LTS

Results come from the author's specific test machines; community reproduction on different hardware configurations may yield different numbers.

Testing on two GPU tiers matters because a model that runs comfortably at 40 tokens per second on a 4090 may become effectively unusable on a 3060 once layers start offloading to system RAM. Developers need to see both scenarios before committing. The software stack consisted of Ollama 0.6.2 as the inference server, llama.cpp (bundled with Ollama) as the backend, Python 3.12.4, and Node.js 22.12.0. All tests used the Ollama REST API on the default local endpoint.

For guidance on assembling a local inference rig, the Hardware Build Guide for Local LLMs covers GPU selection, cooling, and PSU considerations in detail.

Prompt Set Design

The benchmark used 30 prompts across three categories. Coding covered 10 prompts: five Python tasks (file I/O, API clients, data transformation, error handling, standard library usage) and five JavaScript/Node.js tasks (Express endpoints, React components, async patterns, Zod validation, database queries). Reasoning included 10 prompts spanning multi-step logic puzzles, arithmetic word problems, constraint satisfaction, syllogisms, and causal reasoning chains. Creative covered 10 prompts: summarization of technical documents, tone-controlled rewriting, chat-style Q&A, and instruction-following tasks with specific formatting constraints.

We scored each coding response on three dimensions: correctness (does it run and produce the right output?), completeness (does it handle edge cases and include necessary imports?), and hallucination rate (does it reference non-existent APIs, fabricated library methods, or incorrect function signatures?). Automated unit tests evaluated coding correctness, confirmed by manual review. We scored reasoning prompts on binary accuracy and rated creative outputs on a structured 1-to-5 scale for coherence, instruction fidelity, and tone control. Coding Pass@1 was not measured for the 8B/9B models in this benchmark, as their primary role was as speed and constrained-hardware baselines.

We loaded all models at Q4_K_M quantization to maintain a level playing field. We chose Q4_K_M because it is the most common quantization level for local deployment, balancing quality retention against VRAM savings.

Prerequisites

Before running the benchmark scripts, ensure you have Ubuntu 24.04 LTS (other Linux distros should work; Windows/macOS paths may differ) and an NVIDIA GPU with drivers supporting nvidia-smi (AMD GPUs are not supported by this script). You will also need:

• Ollama 0.6.2+ installed and running on localhost:11434
• Python 3.12+ with the requests library (pip install requests) and Node.js 22+ for the Node.js script
• Roughly 250 GB of free disk space for all four model downloads (Mistral Large 2 alone is ~65 GB at Q4_K_M)
• A prompts.json file: a 30-prompt dataset (10 coding, 10 reasoning, 10 creative) in the format described below

Reproducibility

The following Python script drives the entire benchmark. It loads each model via the Ollama REST API, iterates through the prompt set, captures tokens per second (using Ollama's reported eval_duration for accuracy), response text, and VRAM usage via nvidia-smi, then outputs structured JSON results. Results are written incrementally after each prompt so that partial data survives crashes.

Important: Verify that each Ollama model tag below matches the current Ollama library before running. Check tags at https://ollama.com/library and update as needed; tag names and quantization suffixes change over time.

# Save as: benchmark.py
import json
import time
import subprocess
import sys
import requests

OLLAMA_URL = "http://localhost:11434/api/generate"
# Verify these tags against https://ollama.com/library before running
MODELS = ["minimax2.5", "llama3:70b-q4_K_M", "mistral-large2:q4_K_M", "gemma2:27b-q4_K_M"]
RESULTS_FILE = "benchmark_results.json"
INTER_PROMPT_SLEEP_S = 2

def get_vram_mb():
    """
    Returns VRAM used in MB. On multi-GPU systems, sums all GPUs.
    Returns 0 if nvidia-smi is unavailable or fails.
    WARNING: This captures a post-generation snapshot, not true peak.
    For accurate peak measurement, run nvidia-smi dmon -s m -d 1 in a
    separate terminal, or poll VRAM in a background thread during the request.
    """
    try:
        out = subprocess.check_output(
            ["nvidia-smi", "--query-gpu=memory.used", "--format=csv,noheader,nounits"],
            stderr=subprocess.DEVNULL
        )
        lines = [l.strip() for l in out.decode().strip().split("
") if l.strip()]
        if len(lines) > 1:
            print(f"WARNING: {len(lines)} GPUs detected; summing all VRAM values.")
            return sum(int(l) for l in lines)
        return int(lines[0])
    except (FileNotFoundError, subprocess.CalledProcessError, ValueError) as exc:
        print(f"WARNING: nvidia-smi query failed: {exc}. Returning 0.")
        return 0

def run_prompt(model, prompt):
    vram_before = get_vram_mb()
    start = time.perf_counter()
    resp = requests.post(OLLAMA_URL, json={
        "model": model, "prompt": prompt, "stream": False,
        "options": {"num_ctx": 4096}
    }, timeout=300)
    resp.raise_for_status()
    elapsed = time.perf_counter() - start
    data = resp.json()

    # Detect Ollama error bodies returned with HTTP 200
    if "error" in data:
        raise RuntimeError(f"Ollama error for model {model!r}: {data['error']}")
    if not data.get("response") and data.get("eval_count", 0) == 0:
        raise RuntimeError(f"Empty response from model {model!r}; body: {data}")

    tok_count = data.get("eval_count", 0)
    # Use Ollama's reported eval_duration (ns) for accurate t/s; fall back to wall-clock
    eval_duration_s = data.get("eval_duration", 0) / 1e9
    tps = round(tok_count / eval_duration_s, 1) if eval_duration_s > 0 else (
        round(tok_count / elapsed, 1) if elapsed > 0 else 0
    )
    vram_snapshot = get_vram_mb()
    return {
        "model": model,
        "prompt": prompt[:80],
        "category": None,  # caller should inject category
        "response": data.get("response", ""),
        "tokens": tok_count,
        "prompt_tokens": data.get("prompt_eval_count", 0),
        "elapsed_s": round(elapsed, 2),
        "eval_duration_s": round(eval_duration_s, 3),
        "tok_per_s": tps,
        "vram_snapshot_before_mb": vram_before,
        "vram_snapshot_after_mb": vram_snapshot,
    }

# --- Load and validate prompts.json ---
try:
    with open("prompts.json") as f:
        prompts = json.load(f)
except FileNotFoundError:
    sys.exit("ERROR: prompts.json not found. See article for required format.")
except json.JSONDecodeError as exc:
    sys.exit(f"ERROR: prompts.json is not valid JSON: {exc}")

if not isinstance(prompts, list) or not prompts:
    sys.exit("ERROR: prompts.json must be a non-empty JSON array.")
for idx, p in enumerate(prompts):
    if "text" not in p:
        sys.exit(f"ERROR: prompts.json item {idx} missing required 'text' field.")
    if "category" not in p:
        print(f"WARNING: prompts.json item {idx} missing 'category' field.")

# --- Run benchmarks with incremental persistence ---
results = []
# Models are pulled automatically. Ensure ~250 GB free disk space for all four models.
for model in MODELS:
    try:
        subprocess.run(["ollama", "pull", model], check=True)
    except subprocess.CalledProcessError as exc:
        print(f"WARNING: Failed to pull {model!r}: {exc}. Skipping.")
        continue
    for p in prompts:
        try:
            result = run_prompt(model, p["text"])
            result["category"] = p.get("category")
            results.append(result)
        except Exception as exc:
            print(f"WARNING: run_prompt failed for model={model!r}: {exc}. Recording error.")
            results.append({"model": model, "prompt": p["text"][:80],
                            "error": str(exc), "tokens": 0})
        # Write after every prompt — survives mid-run crashes
        with open(RESULTS_FILE, "w") as f:
            json.dump(results, f, indent=2)
        time.sleep(INTER_PROMPT_SLEEP_S)

print(f"Done. {len(results)} results written to {RESULTS_FILE}")

This script is designed to be copy-pasted and run directly. The prompts.json file should contain an array of objects, each with a "text" field and a "category" field (coding, reasoning, or creative). A representative sample of the prompt set is shown below; the full 30-prompt file and both scripts are available at the GitHub gist linked in the "How to Run" section:

[
  {
    "text": "Write a Python function that reads a CSV file and returns a list of dictionaries, one per row, using only the standard library.",
    "category": "coding"
  },
  {
    "text": "Create an Express.js REST endpoint that validates input with Zod and returns paginated results from an in-memory array.",
    "category": "coding"
  },
  {
    "text": "A farmer has 3 fields. Field A produces twice as much wheat as Field B. Field C produces 10 tons less than Field A. Together they produce 110 tons. How much does each field produce?",
    "category": "reasoning"
  },
  {
    "text": "Rewrite the following paragraph in the style of a concise technical changelog entry, keeping all factual claims intact.",
    "category": "creative"
  },
  {
    "text": "Write a Python async HTTP client function using aiohttp that retries failed requests up to 3 times with exponential backoff.",
    "category": "coding"
  }
]

Note on VRAM measurement: The get_vram_mb() function captures a post-generation snapshot, not a continuous peak. True peak VRAM during token generation may be higher than reported values. For more accurate measurements, run nvidia-smi dmon -s m -d 1 in a parallel terminal and compare the highest value observed during generation against the script's reported figure.

Note on tokens-per-second measurement: The script uses Ollama's eval_duration field (reported in nanoseconds) to calculate generation speed rather than wall-clock time. This provides a more accurate measure of pure token generation throughput, excluding HTTP overhead and server queue time. If eval_duration is unavailable, the script falls back to wall-clock timing.

The Contenders — Model Profiles

MiniMax2.5

Developed by MiniMax and released in early 2025 with updates through mid-2025, MiniMax2.5 is a mixture-of-experts architecture available in configurations with 80B total parameters (approximately 20B active per forward pass). It is the most recently released model in this comparison at time of testing. Its claimed strengths include extended context windows up to 200K tokens and strong multilingual performance across Chinese and European languages. On our coding benchmarks it averaged 72% Pass@1 across Python and JavaScript. It is the least battle-tested by the open-source community among the models compared here.

Llama 3 (Meta)

Meta's Llama 3 family, released in April 2024 with incremental updates through 2025, includes multiple variants. This benchmark tested the Llama 3 8B and 70B base variants; the broader Llama 3.x family includes additional sizes (such as Llama 3.1, 3.2, and 3.3 releases spanning 1B to 405B parameters). Llama 3 benefits from the largest community fine-tuning ecosystem of any open-weight model, with many task-specific adapters available on Hugging Face Hub. Known trade-offs include verbose output and occasional over-hedging on ambiguous prompts.

Mistral Large 2

Released in mid-2024 by Paris-based Mistral AI, Mistral Large 2 weighs in at 123B parameters. Mistral AI positions it as a function-calling-capable, multilingual model; the company claims benchmark parity with GPT-4 on selected evaluations, though independent reproductions vary. Its architecture supports native tool use and structured JSON output. As a European open-weight alternative, it appeals to organizations with data residency requirements.

Gemma 2 (Google)

Google's Gemma 2, available in 9B and 27B variants, serves as the lightweight baseline in this comparison. Built on a decoder-only transformer architecture (like most modern large models, it uses grouped-query attention), Gemma 2 targets low VRAM footprint. The 9B variant fits comfortably on 12 GB VRAM at Q4_K_M, making it the default recommendation for developers on consumer GPUs.

Results — Coding Performance

Python Generation Accuracy

Llama 3 70B led on Python tasks, demonstrating the strongest standard library knowledge and producing the most complete error handling. MiniMax2.5 closely followed, particularly on data transformation tasks involving pandas-style operations, though fabricated method names appeared in 9% of its responses. Mistral Large 2 produced clean, concise code but missed edge cases in 2 of 5 file I/O tasks. Gemma 2 27B struggled most with multi-step Python tasks, frequently omitting necessary imports and producing incomplete exception handling.

JavaScript/Node.js Generation Accuracy

MiniMax2.5 took the JavaScript lead, and the gap over Llama 3 70B (73% vs 70%) came almost entirely from React component generation, where it produced well-structured functional components with correct hook usage. Mistral Large 2 demonstrated the best awareness of Zod schema validation within Express routes, passing all five Express tasks. Llama 3 70B remained competitive but generated more boilerplate than necessary. Gemma 2 27B frequently mishandled async/await in Express middleware chains, producing code that would silently swallow errors.

Head-to-Head Example — Building a REST Endpoint

The following prompt was given to all four models: "Create an Express.js REST endpoint that validates input with Zod and returns paginated results from an in-memory array."

MiniMax2.5 output:

import express from "express";
import { z } from "zod";

const app = express();
app.use(express.json());

const items = Array.from({ length: 100 }, (_, i) => ({ id: i + 1, name: `Item ${i + 1}` }));

const querySchema = z.object({
  page: z.coerce.number().min(1).default(1),
  limit: z.coerce.number().min(1).max(50).default(10)
});

app.get("/items", (req, res) => {
  const parsed = querySchema.safeParse(req.query);
  if (!parsed.success) return res.status(400).json({ errors: parsed.error.issues });

  const { page, limit } = parsed.data;
  const start = (page - 1) * limit;
  res.json({ data: items.slice(start, start + limit), total: items.length, page, limit });
});

app.listen(3000);

Llama 3 70B output:

const express = require("express");
const { z } = require("zod");

const app = express();
app.use(express.json());

const data = [];
for (let i = 1; i <= 100; i++) data.push({ id: i, name: `Item ${i}` });

const schema = z.object({
  page: z.string().transform(Number).default("1"),
  limit: z.string().transform(Number).default("10")
});

app.get("/items", (req, res) => {
  try {
    const { page, limit } = schema.parse(req.query);
    const offset = (page - 1) * limit;
    const results = data.slice(offset, offset + limit);
    res.json({ results, page, limit, total: data.length });
  } catch (e) {
    res.status(400).json({ error: e.errors });
  }
});

app.listen(3000, () => console.log("Listening on 3000"));

Mistral Large 2 output:

import express from "express";
import { z } from "zod";

const app = express();

const items = Array.from({ length: 100 }, (_, i) => ({ id: i + 1, name: `Item ${i + 1}` }));

const PaginationSchema = z.object({
  page: z.coerce.number().int().positive().default(1),
  limit: z.coerce.number().int().positive().max(100).default(20)
});

app.get("/items", (req, res) => {
  const result = PaginationSchema.safeParse(req.query);
  if (!result.success) return res.status(422).json(result.error.flatten());

  const { page, limit } = result.data;
  const sliced = items.slice((page - 1) * limit, page * limit);
  return res.json({ items: sliced, meta: { page, limit, total: items.length } });
});

app.listen(3000);

Gemma 2 27B output:

const express = require("express");
const { z } = require("zod");

const app = express();

const items = [];
for (let i = 0; i < 100; i++) items.push({ id: i, name: "Item " + i });

app.get("/items", (req, res) => {
  const page = parseInt(req.query.page) || 1;
  const limit = parseInt(req.query.limit) || 10;
  const start = (page - 1) * limit;
  res.json(items.slice(start, start + limit));
});

app.listen(3000);

MiniMax2.5 and Mistral Large 2 both produced correct, modern ESM-style code with proper safeParse usage and structured error responses. Llama 3 70B used CommonJS and a try/catch approach with parse (functional but different idiom). Gemma 2 27B ignored Zod entirely and omitted pagination metadata, a significant completeness failure.

Coding Verdict Summary

Results — Inference Speed

Tokens per Second — RTX 4090

On the 4090, Llama 3 8B and Gemma 2 9B delivered the fastest interactive experience. MiniMax2.5's mixture-of-experts architecture, activating roughly 20B parameters per token, placed it between the small and large model tiers in terms of throughput. Mistral Large 2 at 123B was the slowest, with a time-to-first-token exceeding 400ms, which is noticeable in interactive coding assistance scenarios.

Tokens per Second — RTX 3060 12GB

The local model speed test on the 3060 reveals a stark divide. Models under 10B parameters remain usable for interactive work. MiniMax2.5 degrades to 6 tokens per second with partial offloading, barely adequate for patient batch processing. Llama 3 70B and Mistral Large 2 drop below 2 t/s, rendering them impractical for anything beyond offline evaluation runs.

For real-time coding assistance on consumer hardware, Llama 3 8B and Gemma 2 9B are the only viable options.

Speed Takeaways

For real-time coding assistance on consumer hardware, Llama 3 8B and Gemma 2 9B are the only viable options. On a 4090, Llama 3 70B hits 18 t/s at 82% reasoning accuracy while Mistral Large 2 manages 12 t/s at 79%, making Llama 3 70B the better throughput-per-quality trade-off for coding and reasoning batch workloads. For chat and creative tasks, Mistral Large 2's quality advantage (4.4 avg chat score vs. 4.2) may justify its lower throughput. The full comparison chart consolidating all metrics appears in the Verdict section below.

Results — Memory and VRAM Usage

VRAM Consumption at Q4_K_M

†VRAM figures for models exceeding 24 GB reflect combined GPU VRAM + system RAM as reported by process memory tools; GPU VRAM is capped at 24 GB on the 4090 test system. Additionally, these are post-generation snapshots; true peak VRAM during generation may be higher (see methodology note above).

Longer context windows consume more VRAM. At 4K context, Gemma 2 9B peaks at 6.9 GB. Extending to 8K adds roughly 0.8 GB. At 32K context, the same model requires approximately 9.4 GB, still within the 12 GB budget but with less headroom. Larger models scale proportionally worse: Llama 3 70B at 32K context exceeded 50 GB, ruling out even the 4090's 24 GB capacity without aggressive layer offloading.

CPU Offloading Viability

MiniMax2.5 tolerated partial offloading better than dense models of comparable total parameter count because its mixture-of-experts architecture means only a subset of parameters are active per token, allowing inactive experts to reside in system RAM, though with significant throughput penalty: MiniMax2.5 drops to 6 t/s on the RTX 3060 as a result. Llama 3 70B, by contrast, suffered severely from offloading due to its dense architecture, where every layer participates in every forward pass, making PCIe bandwidth the bottleneck. Developers without a 24 GB GPU should either stick to sub-10B models or consult the Hardware Build Guide for upgrade paths.

Results — Reasoning and Creative Tasks

Reasoning (Logic and Math)

Llama 3 70B maintained the lead in reasoning. Mistral Large 2 performed particularly well on constraint satisfaction problems but lost points on arithmetic word problems where it misapplied units. MiniMax2.5 showed uneven performance: strong on causal chains but surprisingly unreliable on formal syllogisms.

Creative and Chat Quality

For general-purpose chat and assistant use, Mistral Large 2 scored highest on all three creative dimensions, with tight instruction adherence and natural tone modulation. Llama 3 70B was close but occasionally over-explained. MiniMax2.5 followed instructions well but sometimes produced slightly stilted phrasing in English-language creative tasks.

The Verdict — Choosing the Right Model

Coding: MiniMax2.5 by a Narrow Margin

MiniMax2.5's JavaScript lead (73% vs 70%) edges the combined coding average. Llama 3 70B leads on Python (74% vs 71%) and produces fewer hallucinations, so the choice depends on your language mix. If your workload is Python-heavy, Llama 3 70B is the better pick.

Chat and General Use: Mistral Large 2

Mistral Large 2 topped coherence (4.5), instruction following (4.5), and tone control (4.3). Llama 3 70B trailed slightly across all three dimensions and tends to over-hedge on ambiguous prompts, but it runs at 18 t/s on a 4090 versus Mistral's 12 t/s. If interactive response time matters more than marginal quality gains, Llama 3 70B is a reasonable alternative.

If You Have a 4090 (or Equivalent)

Mistral Large 2 offers the most balanced performance across coding, reasoning, and creative tasks for developers with 24+ GB VRAM. On systems with less than 24 GB where Mistral Large 2 cannot run fully in GPU memory, this recommendation shifts to Llama 3 8B, because Mistral Large 2 simply cannot run usably on that hardware.

If You're on 12 GB VRAM

Gemma 2 9B fits in 12 GB with room for 8K context, maintains 28 t/s on an RTX 3060, and delivers adequate coding output for straightforward tasks. Llama 3 8B is slightly faster (31 t/s) but marginally less capable on structured output.

Complete Comparison Summary:

n/t = not tested; 8B/9B models were used as speed and constrained-hardware baselines only. †Peak figures for models >24 GB include system RAM; see VRAM methodology note.

Gemma 2 9B fits in 12 GB with room for 8K context, maintains 28 t/s on an RTX 3060, and delivers adequate coding output for straightforward tasks.

Implementation checklist: (1) Install Ollama 0.6.2+ from ollama.com. (2) Pull your chosen model: ollama pull gemma2:9b-q4_K_M (verify tag names at https://ollama.com/library before pulling). (3) Install Python dependencies: pip install requests. (4) Save the benchmark runner script from the Methodology section as benchmark.py. (5) Create prompts.json with your own task-specific prompts (see sample format in the Methodology section) or obtain the full 30-prompt set from the GitHub gist linked below. (6) Run python benchmark.py. (7) Review benchmark_results.json and compare against the tables above. (8) Select the model whose trade-off profile matches your hardware and task mix.

How to Run These Benchmarks Yourself

The following Node.js script provides a quick-start alternative for JavaScript-focused developers. It pulls a model, sends a single coding prompt, and prints tokens per second (using Ollama's eval_duration for accuracy) alongside the generated output.

// Save as: benchmark.mjs
// Requires Node.js 18+ for global fetch
const MODEL = process.argv[2] || "gemma2:9b-q4_K_M";
const PROMPT = "Write a Node.js function that debounces an async callback and returns the latest result.";

async function run() {
  const start = performance.now();
  const res = await fetch("http://localhost:11434/api/generate", {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({
      model: MODEL,
      prompt: PROMPT,
      stream: false,
      options: { num_ctx: 4096 }
    })
  });

  if (!res.ok) {
    const body = await res.text();
    throw new Error(`Ollama HTTP ${res.status}: ${body}`);
  }

  const data = await res.json();

  if (data.error) {
    throw new Error(`Ollama error: ${data.error}`);
  }

  const elapsed = (performance.now() - start) / 1000;
  // Prefer Ollama's own eval_duration for accurate t/s
  const evalDurationS = (data.eval_duration ?? 0) / 1e9;
  const evalCount = data.eval_count ?? 0;
  const tps = evalDurationS > 0
    ? (evalCount / evalDurationS).toFixed(1)
    : (evalCount / elapsed).toFixed(1);

  console.log(`Model: ${MODEL}`);
  console.log(`Tokens: ${evalCount} | Time: ${elapsed.toFixed(1)}s | Speed: ${tps} t/s`);
  console.log(`
Output:
${data.response}`);
}

run().catch((err) => {
  console.error("FATAL:", err.message);
  process.exit(1);
});

Save the script above as benchmark.mjs, then run: node benchmark.mjs mistral-large2:q4_K_M (requires Node.js 18+ for global fetch and performance). Swap the PROMPT variable for any task relevant to a given project. For the full 30-prompt set and both the Python and Node.js scripts, the complete package is available as a GitHub gist linked from the Complete Guide to Running LLMs Locally, which also covers Ollama configuration, model management, and multi-GPU setups.