Underdog’s Saluki 27B Shrinks Qwen3.8-27B to 7.89 GB: Tool Calling Up on Its Own Test, Competition Maths Down

Underdog, the company behind a local AI assistant app, has published Saluki 27B 1.0, a heavily compressed version of Qwen3.8-27B. The trade-off is the story. Underdog says it passed 88 of 120 tasks on its own tool-calling test against 84 for the full model, but scored roughly 15 to 18% lower on two competition-maths benchmarks. Every performance figure here is Underdog’s own.

Spotted via Md Ismail Šojal (@0x0SojalSec) on X; original release post by Underdog AI (@UnderdogAI) on X.

What was released

Confirmed. The model card on Hugging Face, in the ConwayResearch organisation, offers a 7.89 GB file under the Apache 2.0 licence. It is a “standard GGUF for stock llama.cpp” [1]. Underdog compares that with 54 GB for the full model, which is its own rounded figure [1].

It is not a Qwen release. The card lists two base models: Qwen’s Qwen3.8-27B and ISTA-DASLab’s Qwen3.8-27B-GSQ-RCO-GGUF, an existing 2-bit version. A NOTICE file says Saluki is “built on this release” and “Modified by Underdog” [1][2]. Underdog calls it a “preview checkpoint” and says it will publish its full training “soon” [3].

What quantisation means

A model is billions of stored numbers, called weights. Normally each takes 16 bits of storage. Quantisation stores each number with fewer bits, which is a bit like rounding prices to the nearest pound instead of the nearest penny. The file shrinks and fits in less memory. The cost is that rounding throws away detail, so some answers can get worse. The card calls Saluki “2-bit”, meaning very coarse rounding. TSN’s general explainer on quantisation formats is here.

The trade-off, in Underdog’s numbers

Developer-reported: where it is ahead. On “Underdog Bench”, 120 tasks that Underdog drew from the Berkeley Function Calling Leaderboard (BFCL v4) and says it froze before testing, Saluki passed 88 against 84 for the full model. On 100 parallel tool-call tasks, Underdog reports 42 against 35. Underdog itself says: “We don’t claim they’re smarter than the original. 120 tasks is a modest test, and part of that lead is likely run-to-run variation” [3].

Developer-reported: where it is behind. On AIME 2025 (average of four runs) Saluki scored 79.2 against 96.7 for the full model. On AIME 2026 it scored 80.0 against 94.6. That is about 18% and 15% lower (TSN’s arithmetic); the card says Saluki “keeps about 82 to 85%” of the full model’s competition-maths score. Those full-model scores are public figures “from a different harness” than Underdog’s own runs, so the comparison is not like for like [1]. It also trails on SWE-bench Verified (30 against 33 of 50 issues) and MBPP+ (78.0 against 83.9) [1].

The 96% headline. The card gives “96% average retention across 9 benchmarks” [1]. TSN’s check of the card’s nine ratios gives about 96%, so the arithmetic holds. But it blends Underdog’s own runs with public scores, and one ratio is 120% (parallel calls), which lifts the average. It is not 96% on every task.

What this does not prove

• That Saluki is better than the full model. The tool-calling lead is on Underdog’s own test, by four tasks of 120, and Underdog says some of it may be variation [3].
• That the 96% figure holds for your work. It is an average of nine mixed benchmarks, and maths fell furthest [1].
• How it behaves on particular hardware. The card names none; it gives only the file size and the llama.cpp format [1]. Underdog’s own page says “Runs on 16 GB laptop” [3], which is the company’s statement.
• That the results are independent. TSN found no outside reproduction of Underdog Bench or the 96% figure. Some X posts go further than the card (see Sources).

The Bottom Line

Saluki is a real, openly licensed 7.89 GB file built on ISTA-DASLab’s 2-bit version of Qwen3.8-27B. Underdog says it holds up well on tool calling and gives up the most on competition maths. Until someone else runs the tests, treat the numbers as the developer’s claims. This is not investment advice.

Related on TSN: Quantization Deep Dive: GGUF, AWQ, GPTQ, EXL2 Compared (2026 Guide)

Sources

1. Underdog, “Underdog Saluki 27B 1.0”, Hugging Face model card (ConwayResearch/Underdog-Saluki-27B-1.0), repository created 8 October 2026; Apache 2.0 (developer claims). https://huggingface.co/ConwayResearch/Underdog-Saluki-27B-1.0
2. Underdog, NOTICE file in the same repository (names Qwen3.8-27B and ISTA-DASLab’s Qwen3.8-27B-GSQ-RCO-GGUF as the works it modifies). https://huggingface.co/ConwayResearch/Underdog-Saluki-27B-1.0/blob/main/NOTICE
3. Underdog, “Meet Underdog Saluki 27B”, dated 7 October 2026 (developer’s own page; its promotional claims about the base model are not used). https://underdog.ai/saluki
4. ISTA-DASLab, Qwen3.8-27B-GSQ-RCO-GGUF, the 2-bit release Saluki builds on. https://huggingface.co/ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF
5. Underdog AI (@UnderdogAI), X post, 8 October 2026, 03:27 BST (read via the X API; verified). Goes beyond the model card with “beats Qwen at tool calling” and “the best model <8GB that runs in Underdog”, which are the company’s claims about its own app, not a general ranking. https://x.com/UnderdogAI/status/2108021482983133395
6. Md Ismail Šojal (@0x0SojalSec), X post, 9 October 2026, 22:47 BST (read via the X API; verified; an aggregator account, credited as the finder). Goes beyond the model card with “beats the full-size version at tool calling”, “keeps 96% of Qwen 3.8 27B performance” and “runs entirely on your Mac locally”; the card supports none of these as worded, and TSN does not repeat them. https://x.com/0x0SojalSec/status/2108675821343027212