# Introduction
Qwythos-9B-Claude-Mythos-5-1M is a 9B reasoning and coding mannequin primarily based on Qwen3.5, designed for native coding workflows, agentic improvement, and long-context duties. What makes it attention-grabbing is that it’s sufficiently small to run on shopper {hardware}, whereas nonetheless being succesful sufficient to assist with sensible coding duties.
On this information, I’ll present you learn how to run the Mythos-enhanced Qwythos mannequin domestically utilizing llama.cpp, then join it to Pi so you need to use it as an area coding agent. I can be utilizing an RTX 4070 Ti Tremendous with 16GB of VRAM, which permits me to run the Q6_K MTP quantization comfortably. When you have an 8GB GPU, I like to recommend beginning with the Q4_K_M variant as a result of it presents a greater steadiness between high quality, pace, and reminiscence utilization.
# Putting in llama.cpp
First, set up the official llama.cpp command-line interface (CLI). This will provide you with entry to the llama command, together with llama serve, which we are going to use to run the mannequin domestically.
curl -LsSf https://llama.app/set up.sh | sh
Subsequent, add the set up listing to your shell path so your terminal can discover the llama command:
echo 'export PATH="$HOME/.native/bin:$PATH"' >> ~/.bashrc
supply ~/.bashrc
To verify that the set up labored, run:
If every part is put in appropriately, you must see the obtainable llama.cpp instructions and choices.
# Setting a Hugging Face Cache Listing
Set the Hugging Face cache someplace with sufficient free storage. That is particularly helpful on cloud machines the place the default residence listing has restricted disk area.
export HF_HOME="/workspace/huggingface"
mkdir -p "$HF_HOME"
To persist it throughout new terminal periods, add it to your shell configuration:
echo 'export HF_HOME="/workspace/huggingface"' >> ~/.bashrc
supply ~/.bashrc
# Beginning the Qwythos MTP Mannequin
Run the Q6_K MTP GGUF mannequin with all obtainable GPU layers:
llama serve
-hf "empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF:MTP-Q6_K"
--alias "qwythos-9b-mtp"
--host 0.0.0.0
--port 8910
--n-gpu-layers all
--ctx-size 100000
--parallel 1
--batch-size 1024
--ubatch-size 512
--flash-attn on
--cache-type-k q8_0
--cache-type-v q8_0
--spec-type draft-mtp
--spec-draft-n-max 6
--threads 12
--threads-batch 24
--temp 0.6
--top-p 0.95
--top-k 20
--repeat-penalty 1.05
--jinja
--perf
The primary time you run this command, llama.cpp will obtain the mannequin information from Hugging Face.
After that, it’ll load the mannequin into GPU reminiscence and begin an area server.
As soon as the mannequin is operating, open the native interface in your browser:
This additionally provides you an OpenAI-compatible native API endpoint at:
The Q6_K variant provides higher output high quality, however it additionally makes use of extra reminiscence. If you happen to run into VRAM or RAM points, scale back --ctx-size first. If that’s nonetheless not sufficient, strive the Q4_K_M variant as an alternative.
Listed below are crucial flags within the command:
| Flag | Function |
|---|---|
--n-gpu-layers all |
Offloads all supported layers to the GPU. |
--ctx-size 100000 |
Allocates a 100K-token context window. Scale back this when you run out of VRAM or RAM. |
--flash-attn on |
Permits Flash Consideration the place supported. |
--cache-type-k q8_0 and --cache-type-v q8_0 |
Reduces KV-cache reminiscence utilization whereas holding good high quality. |
--spec-type draft-mtp |
Permits MTP speculative decoding. |
--spec-draft-n-max 6 |
Permits as much as six speculative tokens per step. |
--jinja |
Makes use of the mannequin’s embedded chat template. |
--perf |
Prints efficiency stats equivalent to token throughput. |
On my RTX 4070 Ti Tremendous, I used to be getting round 81.74 tokens per second. In my take a look at, the mannequin was additionally in a position to purpose by means of the duty, name the required instruments, and return the newest gold value when used contained in the native agent workflow.
# Putting in Pi and the llama.cpp Integration
Pi can hook up with the native llama.cpp server and use the mannequin as a coding agent. The pi-llama plugin is designed to attach Pi with a operating native llama.cpp server, while not having an exterior API key.
Set up Pi:
curl -fsSL https://pi.dev/set up.sh | sh
Set up the llama.cpp plugin for Pi:
pi set up git:github.com/huggingface/pi-llama
Create a small take a look at venture:
mkdir new-project
cd new-project
By default, the plugin appears for llama.cpp on port 8080. On this information, our native server is operating on port 8910, so we have to set the right native API deal with earlier than beginning Pi:
export LLAMA_BASE_URL="http://127.0.0.1:8910/v1"
Now launch Pi:
Inside Pi, sort:
Choose the native mannequin alias:
It is best to see the mannequin listed inside Pi, and as soon as chosen, you can begin utilizing it as an area coding agent.
# Testing the Mythos-Enhanced Coding Mannequin with Pi
Now it’s time to take a look at the native mannequin on actual coding duties inside Pi. I used two easy however sensible duties: a browser recreation and a small Python CLI instrument.
// Constructing a Easy Browser Sport
Begin with a immediate that asks Pi to create a small browser recreation known as “Beat the AI.”
Create a easy browser recreation known as “Beat the AI”.
The participant has 30 seconds to reply brief pattern-recognition questions. Every appropriate reply will increase the rating by one. Present a countdown timer, rating show, progress bar, and a remaining outcomes display screen.
Necessities:
– Use HTML, CSS, and vanilla JavaScript solely.
– Generate not less than three sorts of questions, equivalent to quantity patterns, fast math, and phrase logic.
– Add a restart button after the sport ends.
– Make the interface really feel playful and polished.
– Hold all code simple to know and keep away from pointless information.Take a look at the sport within the browser earlier than ending.
In my take a look at, Pi created the total recreation in a single HTML file with embedded CSS and JavaScript, which retains the venture easy and straightforward to examine.
Pi then summarized what it constructed, together with the 30-second countdown, rating monitoring, progress bar, query varieties, and restart button.
After that, I opened the sport within the browser to confirm that it labored appropriately.
The outcome was a refined little recreation with:
- A countdown timer
- A rating show
- A progress bar
- A number of query varieties
- A restart movement after the sport ends
It is a good instance of how the mannequin can deal with an entire front-end job domestically while not having an exterior API.
// Constructing a CSV-to-Excel Python CLI
For the second take a look at, ask Pi to construct a small Python CLI that converts a CSV file into an Excel .xlsx file.
Construct a easy Python CLI that converts a CSV file into an Excel .xlsx file, accepts enter and output file paths as arguments, preserves column headers, validates lacking information, and prints clear success or error messages. Earlier than ending, create a small dummy CSV file with pattern knowledge, use it to check the CLI, confirm that the Excel file is created appropriately, after which summarize the take a look at outcome.
Pi created a Python script named csv2excel.py, generated a pattern CSV file, ran the take a look at command, after which summarized the outcomes.
python csv2excel.py sample_data.csv output.xlsx
In my run, the abstract confirmed that the script:
- Accepted enter and output file paths.
- Preserved the column headers.
- Appropriately transformed the pattern knowledge.
- Dealt with lacking information with clear error messages.
- Dealt with lacking output paths correctly.
I additionally opened the generated Excel file to verify that the output was appropriate.
The pattern output preserved the rows and headers appropriately, which confirmed that the CLI labored as anticipated.
# Last Ideas
I actually like this small native coding mannequin. It’s quick, correct sufficient for on a regular basis coding duties, and doesn’t want an enormous quantity of VRAM to be helpful. You need to use it with Pi, Claude Code, OpenCode, or any coding setup that helps native OpenAI-compatible or Anthropic-compatible endpoints.
For primary front-end apps, Python scripts, CLI instruments, and fast prototypes, it really works surprisingly properly. You possibly can construct helpful tasks domestically with out relying on exterior APIs or paying for each request.
To get even higher outcomes, I extremely suggest including internet search expertise, Context7, and different helpful Pi integrations. You too can take a look at my full information on optimizing your Pi coding agent setup right here: Tips on how to Set Up Kimi K2.7 Code with Pi: The Final AI Coding Atmosphere.
Abid Ali Awan (@1abidaliawan) is a licensed knowledge scientist skilled who loves constructing machine studying fashions. At the moment, he’s specializing in content material creation and writing technical blogs on machine studying and knowledge science applied sciences. Abid holds a Grasp’s diploma in expertise administration and a bachelor’s diploma in telecommunication engineering. His imaginative and prescient is to construct an AI product utilizing a graph neural community for college students battling psychological sickness.
# Introduction
Qwythos-9B-Claude-Mythos-5-1M is a 9B reasoning and coding mannequin primarily based on Qwen3.5, designed for native coding workflows, agentic improvement, and long-context duties. What makes it attention-grabbing is that it’s sufficiently small to run on shopper {hardware}, whereas nonetheless being succesful sufficient to assist with sensible coding duties.
On this information, I’ll present you learn how to run the Mythos-enhanced Qwythos mannequin domestically utilizing llama.cpp, then join it to Pi so you need to use it as an area coding agent. I can be utilizing an RTX 4070 Ti Tremendous with 16GB of VRAM, which permits me to run the Q6_K MTP quantization comfortably. When you have an 8GB GPU, I like to recommend beginning with the Q4_K_M variant as a result of it presents a greater steadiness between high quality, pace, and reminiscence utilization.
# Putting in llama.cpp
First, set up the official llama.cpp command-line interface (CLI). This will provide you with entry to the llama command, together with llama serve, which we are going to use to run the mannequin domestically.
curl -LsSf https://llama.app/set up.sh | sh
Subsequent, add the set up listing to your shell path so your terminal can discover the llama command:
echo 'export PATH="$HOME/.native/bin:$PATH"' >> ~/.bashrc
supply ~/.bashrc
To verify that the set up labored, run:
If every part is put in appropriately, you must see the obtainable llama.cpp instructions and choices.
# Setting a Hugging Face Cache Listing
Set the Hugging Face cache someplace with sufficient free storage. That is particularly helpful on cloud machines the place the default residence listing has restricted disk area.
export HF_HOME="/workspace/huggingface"
mkdir -p "$HF_HOME"
To persist it throughout new terminal periods, add it to your shell configuration:
echo 'export HF_HOME="/workspace/huggingface"' >> ~/.bashrc
supply ~/.bashrc
# Beginning the Qwythos MTP Mannequin
Run the Q6_K MTP GGUF mannequin with all obtainable GPU layers:
llama serve
-hf "empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF:MTP-Q6_K"
--alias "qwythos-9b-mtp"
--host 0.0.0.0
--port 8910
--n-gpu-layers all
--ctx-size 100000
--parallel 1
--batch-size 1024
--ubatch-size 512
--flash-attn on
--cache-type-k q8_0
--cache-type-v q8_0
--spec-type draft-mtp
--spec-draft-n-max 6
--threads 12
--threads-batch 24
--temp 0.6
--top-p 0.95
--top-k 20
--repeat-penalty 1.05
--jinja
--perf
The primary time you run this command, llama.cpp will obtain the mannequin information from Hugging Face.
After that, it’ll load the mannequin into GPU reminiscence and begin an area server.
As soon as the mannequin is operating, open the native interface in your browser:
This additionally provides you an OpenAI-compatible native API endpoint at:
The Q6_K variant provides higher output high quality, however it additionally makes use of extra reminiscence. If you happen to run into VRAM or RAM points, scale back --ctx-size first. If that’s nonetheless not sufficient, strive the Q4_K_M variant as an alternative.
Listed below are crucial flags within the command:
| Flag | Function |
|---|---|
--n-gpu-layers all |
Offloads all supported layers to the GPU. |
--ctx-size 100000 |
Allocates a 100K-token context window. Scale back this when you run out of VRAM or RAM. |
--flash-attn on |
Permits Flash Consideration the place supported. |
--cache-type-k q8_0 and --cache-type-v q8_0 |
Reduces KV-cache reminiscence utilization whereas holding good high quality. |
--spec-type draft-mtp |
Permits MTP speculative decoding. |
--spec-draft-n-max 6 |
Permits as much as six speculative tokens per step. |
--jinja |
Makes use of the mannequin’s embedded chat template. |
--perf |
Prints efficiency stats equivalent to token throughput. |
On my RTX 4070 Ti Tremendous, I used to be getting round 81.74 tokens per second. In my take a look at, the mannequin was additionally in a position to purpose by means of the duty, name the required instruments, and return the newest gold value when used contained in the native agent workflow.
# Putting in Pi and the llama.cpp Integration
Pi can hook up with the native llama.cpp server and use the mannequin as a coding agent. The pi-llama plugin is designed to attach Pi with a operating native llama.cpp server, while not having an exterior API key.
Set up Pi:
curl -fsSL https://pi.dev/set up.sh | sh
Set up the llama.cpp plugin for Pi:
pi set up git:github.com/huggingface/pi-llama
Create a small take a look at venture:
mkdir new-project
cd new-project
By default, the plugin appears for llama.cpp on port 8080. On this information, our native server is operating on port 8910, so we have to set the right native API deal with earlier than beginning Pi:
export LLAMA_BASE_URL="http://127.0.0.1:8910/v1"
Now launch Pi:
Inside Pi, sort:
Choose the native mannequin alias:
It is best to see the mannequin listed inside Pi, and as soon as chosen, you can begin utilizing it as an area coding agent.
# Testing the Mythos-Enhanced Coding Mannequin with Pi
Now it’s time to take a look at the native mannequin on actual coding duties inside Pi. I used two easy however sensible duties: a browser recreation and a small Python CLI instrument.
// Constructing a Easy Browser Sport
Begin with a immediate that asks Pi to create a small browser recreation known as “Beat the AI.”
Create a easy browser recreation known as “Beat the AI”.
The participant has 30 seconds to reply brief pattern-recognition questions. Every appropriate reply will increase the rating by one. Present a countdown timer, rating show, progress bar, and a remaining outcomes display screen.
Necessities:
– Use HTML, CSS, and vanilla JavaScript solely.
– Generate not less than three sorts of questions, equivalent to quantity patterns, fast math, and phrase logic.
– Add a restart button after the sport ends.
– Make the interface really feel playful and polished.
– Hold all code simple to know and keep away from pointless information.Take a look at the sport within the browser earlier than ending.
In my take a look at, Pi created the total recreation in a single HTML file with embedded CSS and JavaScript, which retains the venture easy and straightforward to examine.
Pi then summarized what it constructed, together with the 30-second countdown, rating monitoring, progress bar, query varieties, and restart button.
After that, I opened the sport within the browser to confirm that it labored appropriately.
The outcome was a refined little recreation with:
- A countdown timer
- A rating show
- A progress bar
- A number of query varieties
- A restart movement after the sport ends
It is a good instance of how the mannequin can deal with an entire front-end job domestically while not having an exterior API.
// Constructing a CSV-to-Excel Python CLI
For the second take a look at, ask Pi to construct a small Python CLI that converts a CSV file into an Excel .xlsx file.
Construct a easy Python CLI that converts a CSV file into an Excel .xlsx file, accepts enter and output file paths as arguments, preserves column headers, validates lacking information, and prints clear success or error messages. Earlier than ending, create a small dummy CSV file with pattern knowledge, use it to check the CLI, confirm that the Excel file is created appropriately, after which summarize the take a look at outcome.
Pi created a Python script named csv2excel.py, generated a pattern CSV file, ran the take a look at command, after which summarized the outcomes.
python csv2excel.py sample_data.csv output.xlsx
In my run, the abstract confirmed that the script:
- Accepted enter and output file paths.
- Preserved the column headers.
- Appropriately transformed the pattern knowledge.
- Dealt with lacking information with clear error messages.
- Dealt with lacking output paths correctly.
I additionally opened the generated Excel file to verify that the output was appropriate.
The pattern output preserved the rows and headers appropriately, which confirmed that the CLI labored as anticipated.
# Last Ideas
I actually like this small native coding mannequin. It’s quick, correct sufficient for on a regular basis coding duties, and doesn’t want an enormous quantity of VRAM to be helpful. You need to use it with Pi, Claude Code, OpenCode, or any coding setup that helps native OpenAI-compatible or Anthropic-compatible endpoints.
For primary front-end apps, Python scripts, CLI instruments, and fast prototypes, it really works surprisingly properly. You possibly can construct helpful tasks domestically with out relying on exterior APIs or paying for each request.
To get even higher outcomes, I extremely suggest including internet search expertise, Context7, and different helpful Pi integrations. You too can take a look at my full information on optimizing your Pi coding agent setup right here: Tips on how to Set Up Kimi K2.7 Code with Pi: The Final AI Coding Atmosphere.
Abid Ali Awan (@1abidaliawan) is a licensed knowledge scientist skilled who loves constructing machine studying fashions. At the moment, he’s specializing in content material creation and writing technical blogs on machine studying and knowledge science applied sciences. Abid holds a Grasp’s diploma in expertise administration and a bachelor’s diploma in telecommunication engineering. His imaginative and prescient is to construct an AI product utilizing a graph neural community for college students battling psychological sickness.
