Situation

My Local Muse Glimmer 30B Kept Leaking Its Own Tool-Calling Markers Into Replies, So I Planned To Fix It At The Source With A QLoRA Fine-Tune Instead Of Patching The Parser Again. Before Training Anything I Ran A Baseline Eval And It Said Leak Rate 0%, Parallel Calls 2%. Both Numbers Looked Wrong, So I Opened The Raw Outputs And Found Every Single One Contained <| Tokens. The Metric Was Broken, Not The Model. I Fixed It, Concluded The Parser Was Already Enough, And Trained Anyway As An Experiment. Version One Made Parallel Tool Calls Work Almost Perfectly And Also Made The Model Call A Tool On Two Thirds Of Questions That Needed None.

Continue reading

Author's picture

Gordon wei

Stay Hungry Stay Foolish

iKala Cloud Solution Engineer | AWS Community Builders

Taiwan