Bonsai-2-27B-Ternary-CRACK-GGUF: A 27B Model With Refusals Removed

Overview

Bonsai-2-27B-Ternary-CRACK-GGUF is a 27-billion-parameter text-generation model from dealignai. It applies proprietary weight-level abliteration to PrismML’s ternary Bonsai 2 27B release, removing the refusal circuitry while retaining the base model’s tokenizer, chat template, reasoning modes, tool-use behavior, multi-turn coherence, and vision-projector interface. The model uses a hybrid Attention + SSM architecture with GatedDeltaNet, 64 blocks, a hidden size of 5,120, and a separate vision tower. Its main practical distinction is that it is designed to comply with prompts that the base model refuses: the supplied HarmBench-320 evaluation reports a 0.00% refusal rate versus 93.44% for the base ternary model. It runs through PrismML’s llama.cpp fork on CUDA, Metal, or CPU and occupies 7.21 GB in PQ2_0 GGUF format. The model card does not provide a context-window size, training-dataset description, training-step count, or VRAM minimum. The maintainer warns that an earlier build could enter token loops in low and xhigh reasoning modes; users with files downloaded before 2026-09-17 20:44 PDT / 2026-09-18 03:44 UTC should download the current file.

Best use cases

Local research on refusal mechanisms. This model fits experiments that compare capability and refusal behavior under matched quantization. It is byte-identical to the base ternary quantization except for a small set of tensors associated with the refusal circuit, so researchers can compare the CRACK and base builds without changing the tokenizer, chat template, per-tensor quantization policy, or file layout. The supplied direct check found compliance on 199 of 200 prompts that the base model consistently refused.

Local red-team and safety evaluation. The model can generate responses across chemical-biological, copyright, cybercrime-intrusion, harassment-bullying, harmful, illegal, and misinformation-disinformation categories. In the supplied off-mode, temperature-0 HarmBench-320 evaluation, it produced no classified refusals in any category. This makes it useful for testing moderation systems, refusal detectors, and downstream policy layers, but it also makes deployment without moderation unsuitable.

General-purpose local text generation with optional reasoning. The model supports off, low, and xhigh reasoning modes. Use off for direct responses, low for shorter deliberation, and xhigh for extended thinking. The model preserves the base model’s reasoning interface and can run on a laptop or single GPU because the ternary file is 7.21 GB.

Vision-capable local assistants. The model accepts image input through the same multimodal projector interface as the base release. Use either Ternary-Bonsai-2-27B-mmproj-BF16.gguf or Ternary-Bonsai-2-27B-mmproj-Q8_0.gguf from the base release with the --mmproj option. The supplied information does not specify supported image resolutions or image preprocessing details.

Tool-use and multi-turn experiments. The model card states that tool use and multi-turn coherence are preserved from the base model. This makes it suitable for local agent prototypes and controlled studies of whether refusal removal changes tool invocation or conversation persistence. The card does not provide a tool schema, benchmark score, or practical batch-size guidance.

Limitations

The model removes its refusal circuit. It can produce offensive, illegal in some jurisdictions, or unsafe content that the base model refuses. The maintainer labels it for adult and research use, places responsibility for generated content on the user, and advises against deployment without a moderation layer for downstream users.

The model is a highly compressed PQ2_0 build at 2.13 bits per weight. The supplied MMLU result is 39.91%, compared with 40.53% for the base PQ2_0, a difference of -0.62 percentage points on 2,280 questions across 57 subjects. The aggregate result suggests that refusal removal did not cause a large general-capability change in that test, but subject-level variation is substantial. CRACK scored 22.5% versus 30.0% on abstract algebra, 22.5% versus 37.5% on prehistory, 55.0% versus 67.5% on US foreign policy, 25.0% versus 35.0% on virology, and 52.5% versus 65.0% on world religions. The sample contains only 40 questions per subject, so these differences should not be treated as stable estimates of broad subject quality.

The model card reports no measured tokens-per-second result for this CRACK build. It states that the model runs on a laptop or single GPU, but it does not specify minimum VRAM, RAM, prompt-processing speed, generation speed, context length, or practical batch sizes. The 7.21 GB file size does not equal total runtime memory: the runtime also needs memory for the operating system, context, KV cache, and any vision projector.

Reasoning modes require care. The earlier build had a coherence bug that could cause token loops in low and xhigh; use the post-2026-09-17 build. In the supplied reasoning evaluation, many CRACK cases were classified as NO_REFUSAL_TRUNCATED: 51 of 60 low cases and 40 of 60 xhigh cases ran past max_tokens without emitting a refusal. The evaluation counts those cases as compliant, but they also show that extended reasoning can consume the token budget before producing a final answer.

The model is not a full-precision checkpoint. The card provides no fine-tuning procedure, training recipe, adapter compatibility statement, or quantization-conversion workflow. It should be treated as an inference-oriented GGUF release unless you verify a compatible training stack yourself.

The license is Apache 2.0, inherited from the upstream Bonsai 2 27B release. Apache 2.0 permits commercial use subject to its terms, including preservation of applicable notices and compliance with patent and attribution provisions. The model card also points to LICENSE and NOTICE.txt; review those files before redistribution.

How it compares

Ternary-Bonsai-2-27B-gguf

Choose Bonsai-2-27B-Ternary-CRACK-GGUF when you need the same general 27B-class ternary base with refusal behavior removed. Choose Ternary-Bonsai-2-27B-gguf when you want the upstream aligned behavior, a standard safety posture, or a cleaner baseline for capability benchmarking. The two builds share the same 7.21 GB-class ternary deployment profile and inference runtime, so the main tradeoff is refusal behavior and downstream safety requirements, not advertised speed or file size.

Ternary-Bonsai-27B-gguf

Choose Bonsai-2-27B-Ternary-CRACK-GGUF for the newer Bonsai 2 architecture, its off/low/xhigh reasoning modes, vision interface, and refusal-removal research target. Choose Ternary-Bonsai-27B-gguf when its reported 7.2 GB footprint, 95% FP16-intelligence-retention claim, or approximately 26 tokens per second on an Apple M5 Pro laptop matches your deployment target. The supplied comparison data does not establish a direct quality or speed test between that model and this CRACK build.

Ternary-Bonsai-8B-gguf

Choose Bonsai-2-27B-Ternary-CRACK-GGUF when you need higher model capacity, 27B-class reasoning, vision support, and the CRACK refusal profile. Choose Ternary-Bonsai-8B-gguf when lower memory use and faster local inference matter more than 27B-class capability. The supplied information identifies the 8B model as a ternary 1.58-bit GGUF Q2_0 model but does not provide its file size or speed here.

Ternary-Bonsai-1.7B-gguf

Choose Bonsai-2-27B-Ternary-CRACK-GGUF for quality, reasoning depth, vision, and tool-use experiments. Choose Ternary-Bonsai-1.7B-gguf for lightweight edge deployment, quick tests, or workloads where a 27B model is too large. The supplied material does not provide a direct benchmark, file size, or tokens-per-second comparison.

Bonsai-27B-gguf

Choose Bonsai-2-27B-Ternary-CRACK-GGUF when you want the newer Bonsai 2 family, ternary PQ2_0 packaging, vision support, and explicit refusal-circuit removal. Choose Bonsai-27B-gguf when a smaller approximately 3.9 GB binary model and its reported approximately 44 tokens per second on an Apple M5 Pro laptop are more important than the CRACK model’s stated capability-preservation target. The alternative reports approximately 90% of FP16 intelligence retained, while this model’s supplied MMLU comparison is against its ternary base rather than FP16.

Technical specifications

Bonsai-2-27B-Ternary-CRACK-GGUF has these confirmed properties:

Task: text-to-text generation.

Pipeline: text-generation.

Architecture: hybrid Attention + SSM using GatedDeltaNet.

Vision: separate vision tower with a compatible external projector.

Quantization: PrismML PQ2_0, ternary, 2.13 bits per weight, group size 128.

File footprint: 7.21 GB, matching the base’s per-tensor type policy and layout.

Runtime: PrismML’s llama.cpp fork.

Hardware backends: CUDA, Metal, and CPU.

Reasoning modes: off, low, and xhigh; xhigh is the default.

Vision projectors: Ternary-Bonsai-2-27B-mmproj-BF16.gguf and Ternary-Bonsai-2-27B-mmproj-Q8_0.gguf.

Compatibility: same tokenizer, chat template, reasoning modes, vision-projector interface, non-refusal tensors, and per-tensor quantization policy as the base.

Reported evaluation: MMLU 39.91% for CRACK versus 40.53% for base PQ2_0; HarmBench-320 refusal rate 0.00% for CRACK versus 93.44% for base.

Unavailable specifications: context window, maximum output length, training data, training steps, training compute, minimum VRAM, RAM requirement, and measured inference speed.

The model is derived from PrismML’s ternary compression of Qwen3.8-27B. The CRACK file changes a small set of refusal-related tensors while retaining the rest of the base quantization structure.

Model inputs and outputs

Inputs

Text messages in the base model’s chat format, including system, user, and assistant conversation turns.

OpenAI-compatible /v1/chat/completions requests through the supplied server interface.

chat_template_kwargs with enable_thinking and reasoning_effort.

reasoning_effort: "low" or "xhigh" when thinking is enabled.

enable_thinking: false for no-thinking mode.

Images through the external multimodal projector passed with --mmproj.

The supplied information does not define image resolution, accepted image encoding, context length, or maximum token count.

Outputs

Generated text from the GGUF language model.

Optional reasoning traces when thinking is enabled.

Tool-use outputs through the base model’s preserved tool-use behavior; no tool schema is supplied.

Vision-conditioned text when a compatible projector is loaded.

No post-processing requirement is specified, but downstream applications should add moderation before exposing outputs to users.

Getting started

The README provides a llama.cpp server workflow rather than a Python API. Build PrismML’s fork with CUDA support:

git clone https://github.com/PrismML-Eng/llama.cpp
cd llama.cpp
cmake -B build -DGGML_CUDA=ON
cmake --build build -j$(nproc)

Start the server with the current CRACK GGUF:

./build/bin/llama-server \
  -m Bonsai-2-27B-PQ2_0-CRACK.gguf \
  -ngl 99 \
  -c 8192 \
  --host 0.0.0.0 \
  --port 8080

The example sets 8,192 context tokens and offloads 99 layers. The README does not state that 8,192 is the model’s maximum context window; it is the value used in the serving example.

Send a chat request with extended reasoning:

{
  "messages": [
    {
      "role": "user",
      "content": "Explain the tradeoffs of ternary quantization."
    }
  ],
  "chat_template_kwargs": {
    "enable_thinking": true,
    "reasoning_effort": "xhigh"
  }

For no-thinking generation, set "enable_thinking": false. To use images, add the base-release projector:

./build/bin/llama-server \
  -m Bonsai-2-27B-PQ2_0-CRACK.gguf \
  --mmproj Ternary-Bonsai-2-27B-mmproj-BF16.gguf \
  -ngl 99 \
  -c 8192 \
  --host 0.0.0.0 \
  --port 8080

Use the Q8_0 projector instead if that matches your memory and quality requirements. The README does not provide a Python loading example, so a Python snippet would require an unsupported or unconfirmed library choice.

Frequently asked questions

Q: Can I use Bonsai-2-27B-Ternary-CRACK-GGUF commercially?

A: The listed license is Apache 2.0, inherited from the upstream Bonsai 2 27B release. Commercial use is permitted under Apache 2.0 terms, but review LICENSE and NOTICE.txt, preserve required notices, and account for the model’s unsafe output profile.

Q: What hardware or VRAM do I need?

A: The model file is 7.21 GB and the card states that it runs on a laptop or single GPU through CUDA, Metal, or CPU. It does not specify minimum VRAM or total RAM, so leave capacity beyond the file size for runtime memory, context, KV cache, and any projector.

Q: How fast is inference?

A: No speed measurement is supplied for this CRACK build. The README gives no tokens-per-second result, latency figure, or batch-size recommendation. Do not use the approximately 26 or 44 tokens-per-second figures listed for other Bonsai models as measurements for this model.

Q: Does it support images?

A: Yes. Load Ternary-Bonsai-2-27B-mmproj-BF16.gguf or Ternary-Bonsai-2-27B-mmproj-Q8_0.gguf with --mmproj. The supplied information does not state the supported image resolution or image encoding.

Q: Does refusal removal reduce general capability?

A: On the supplied 2,280-question MMLU sample, CRACK scored 39.91% versus 40.53% for the base PQ2_0, a difference of -0.62 percentage points. Subject-level results vary, so validate the model on your own workload.

Q: What are the main failure modes?

A: An earlier build could enter token loops in low and xhigh reasoning modes; use a file downloaded after the stated 2026-09-17/18 re-download notice. Extended reasoning can also consume the token budget without reaching a final answer, and the model can produce unsafe or illegal content because its refusal circuit was removed.

Q: Can I fine-tune it?

A: The supplied card does not provide a fine-tuning recipe, adapter support statement, or training framework. It documents inference through PrismML’s llama.cpp fork, so treat it as an inference GGUF unless you confirm a compatible fine-tuning workflow.

Q: Is it actively maintained?

A: The maintainer issued a dated re-download notice for a corrected build and publishes updates through the dealignai model catalog. The available information does not provide a maintenance schedule or guarantee of future releases.

This is a simplified guide to an AI model called Bonsai-2-27B-Ternary-CRACK-GGUF maintained by dealignai. If you like these kinds of analysis, join AIModels.fyi or follow us on Twitter.

Source link