Qwengram-0.8B Improves Qwen3.5-0.8B with Qwen3.8 Flash-Next's N-gram Memory

Core summary

One sentence to understand this update

A developer successfully transferred Qwen3.8 Flash-Next’s pretrained n-gram memory to Qwen3.5-0.8B, resulting in a 5.05% lower validation perplexity for the smaller model.

Impact & opportunity

What this could mean

This demonstrates a promising technique for improving smaller LLMs by transferring knowledge from larger, more advanced models, even with limited resources.