CODEBLOCK: Supervising Code LLMs with Optimal Granularity

Core summary

One sentence to understand this update

CODEBLOCK is a new approach that addresses the challenge of supervised fine-tuning for code LLMs by learning to apply supervision at the right granularity, moving beyond uniform cross-entropy loss.

Impact & opportunity

What this could mean

Developers fine-tuning code LLMs can explore CODEBLOCK to potentially achieve more effective and targeted model training by optimizing the supervision granularity.