Qwengram-0.8B: Transferring Qwen3.8 Flash-Next n-gram Memory to 0.8B Model

Core summary

One sentence to understand this update

Research successfully transferred Qwen3.8 Flash-Next's n-gram memory to a smaller Qwen3.5-0.8B model, resulting in a 5.05% lower validation perplexity.

Impact & opportunity

What this could mean

This technique offers new avenues for improving the performance of smaller models in resource-constrained environments, prompting developers to explore new knowledge distillation and model optimization methods.