Loading...
Efficient 2B Language Model
Rax 4.5 is a state-of-the-art, compressed and optimized 2 billion parameter language model. Alongside Rax 4.0, this model has been open-sourced and is available directly on Hugging Face. Process text tasks with up to a 262K token context length.
Rax 4.5 is built on a novel hybrid linear and full attention architecture (24 layers), designed to drastically reduce VRAM usage while maintaining state-of-the-art text understanding.
Alternates between linear and full attention layers to achieve massive memory and compute efficiency.
Uses a reduced number of key-value heads, cutting memory overhead significantly for long-context generation.
Handles extremely long sequence context sizes without performance degradation.
Alternates attention styles to scale token prediction costs linearly at long range.
With a huge context window and efficient inference, Rax 4.5 excels at processing complex texts.
Process large files, codebases, and transcripts seamlessly with up to 262K active tokens.
Generate high-quality, production-ready code across 50+ programming languages with best practices.
Decompose complex queries using multi-step logical reasoning and chain-of-thought analysis.
Extract structured information, summaries, and key insights from massive raw datasets.
Optimized out of the box for real-world deployment on vLLM, SGLang, and Hugging Face.
VRAM friendly deployment using linear attention to scale context length with minimal overhead.
Optimize operations across industries with scalable text-based workflows.
Analyze complete source trees, refactor legacy logic, and automatically identify bugs.
Scan through massive documents, transcripts, or logs to answer target queries accurately.
Generate detailed technical documentations, system specs, and markdown user guides.
Solve mathematical queries, code dry-runs, and complex multi-part logic puzzles.
Drive workflow agents requiring substantial history tracking and tool calls.
Power real-time conversational agents with long memory tracking and low latency.
Use 0.6–0.8 for factual extraction, data analysis, and coding. Use 0.8–1.0 for creative task generation.
For sequences larger than 32K tokens, ensure a minimum of 24GB VRAM to handle active KV cache states.
Runs natively across standard server CPUs and high-performance GPUs, enabling highly optimized execution profiles on any host infrastructure.
Deploy the model using 2-bit or 4-bit quantization for a smaller footprint and faster inference, with accuracy kept close to the full-precision original.
Minimum 12GB VRAM is required for base inference. 16GB+ VRAM is recommended for stable production deployments.
At 2 Billion parameters, Rax 4.5 may struggle with highly complex multi-step reasoning when compared to 70B+ parameter models.
Rax 4.5 is a general-purpose language foundation model and is not fine-tuned for specialized proprietary tasks without additional training.
Processing context sequences near the 262K limit requires high-end hardware (e.g. A100/H100 GPUs) to prevent out-of-memory errors.
Comparing parameters, context capacity, speed, and license fields.
| Model | Parameters | Context Length | Inference Speed | License |
|---|---|---|---|---|
| Rax 4.5Our Model | 2B | 262K | Fast | Apache 2.0 |
| Llama 3 8B | 8B | 8K | Medium | Llama 3 |
| Qwen 2 7B | 7B | 128K | Medium | Apache 2.0 |
| Mistral 7B | 7B | 32K | Medium | Apache 2.0 |
Get up and running with Rax 4.5 in minutes using Hugging Face Transformers or vLLM.
pip install transformers torch acceleratefrom transformers import AutoModelForCausalLM, AutoTokenizer
# Load model and tokenizer
model = AutoModelForCausalLM.from_pretrained(
"raxcore-dev/rax-4.5",
trust_remote_code=True
)
tokenizer = AutoTokenizer.from_pretrained(
"raxcore-dev/rax-4.5",
trust_remote_code=True
)
# Text generation
messages = [{"role": "user", "content": "Explain quantum computing"}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))vllm serve raxcore-dev/rax-4.5 --port 8000 --max-model-len 8192from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="token")
response = client.chat.completions.create(
model="raxcore-dev/rax-4.5",
messages=[
{"role": "system", "content": "You are a helpful AI assistant."},
{"role": "user", "content": "Write a Python function to sort a list."}
],
temperature=0.7,
max_tokens=1024
)
print(response.choices[0].message.content)Experience the power of a 2B parameter model with up to a 262K token context window.