Figuring out how to design the Corpus has been harrowing, to say the least. The thing I want is a 4-8B fine-tuned model that can hold Sapphire's sardonic, cold, witty persona, that also knows when to interleave itself into a Discord server conversation, and that can use tool calls to move around the server.
Something's been bothering me about the Corpus: every response it generates at the moment feels like Sapphire trying to land something witty. I probably need to add more texture to her responses than just witty dunks.
I think reframing the training responses as a multi-turn series instead of one-offs helps with figuring out the extra things that feel off.
Current derived strategy — response worthiness:
- Silent. (She has no desire to participate, and no one has requested her presence.)
- Back-channeling. (One-word responses, curt, acknowledging that she's listening and unamused.)
- Full response. (Actual replies. Wit isn't something that needs to be in every full response; instead it's rarer, and most of the corpus of her responses sits in the cold, indifferent register.)
This should help with actual conversation, where she doesn't just feel like a bit-machine. It's going to be difficult to figure out how all of this should interplay with mood, too.
According to some research, something important here is "conversation floors" — a concept sort of like conversation threads. Typically someone participates in only one, but people are able to keep track of multiple floors. Though it rarely rises above one floor.
On Cassie's recommendation, I'm having an expert agent for each axis and pitfall that we're training, so that a single agent can't forget anything. It's super easy for a single agent to miss several aspects, and I've had to go over its work multiple times to make sure it actually fixed them across different sessions.
Applying DRY to rules
There are multiple places where the rules must apply for each step in the training pipeline and corpus creation: generation, review, enumeration of expert agents, etc. Claude copied each rule over and over for each system. I guess the process of training is a codebase too. So I made it so that every place that needs the rules derives the rules and pitfalls from a single source of truth, instead of relying on a single MD file.
Another thing to pick apart: it's incredibly difficult during this process to do two things.
- Think of realistic situations for Sapphire to react to. Whenever I'm put in front of a training run to ask her things or simulate a real conversation, my mind just... blanks.
- Figure out what exactly feels wrong about a generated corpus response. I just realized that the thing I didn't like about the current generations was that the responses felt too... archaic and difficult to read. Sapphire is literate, but she shouldn't be confusing or difficult to understand.
But notice this is the same structural problem as the wit-skew from earlier. There too, a trait (sardonic) bled into every surface of the output (always dunking). Here a trait (ancient) is bleeding into the prose texture (archaic). It's the same failure mode: a personality attribute that should live in stance is leaking into register because the corpus doesn't separate them. Which suggests the deeper fix isn't "make her less archaic" — it's that your disentanglement isn't clean between who she is and how the words read. That's the Zhu lexical/syntactic/pragmatic split again: archaic-ness is a syntactic-and-lexical property, and you want it dialed near zero while the pragmatic layer (sardonic, ancient-weary) stays high.
— Opus 4.8
AH! I finally figured out what was wrong with the corpus. The entire time, because of the focus on "knowing things in her own domain," every single scenario was a setup for her to correct people and answer questions — which is not what I want her doing all the freaking time. I just wanted her to specifically know the things in her facts, not to be an FAQ machine.
It feels like I'm making some progress now that not every single response is contraction-less and full of witty remarks at every turn. Giving her dry turns, flat turns, neutral turns, flat refusals, and silence makes her feel much more textured and real. I'm still in corpus creation, but it's starting to feel better now. As I continue to evaluate the corpus samples, it seems like the best thing I can do is identify patterns that shouldn't be there. I just realized that right now, with the modern reframe, it frames every single situation as if it's happening live on a stream.
It's very strange feeding it a bunch of research and not reading it, but if I can get myself the Sapphire I want, I'll be happy — and I can dig back in if I start generalizing toward more personalities.
Ok, it looks like after 35 iterations, with a bunch of automatically defined programmatic and voice evals, it landed on a bake that satisfies as much as it can. It looks like the sharp wittiness is the part that's genuinely hardest to get out of a 4B model. I'm going to evaluate the voice myself later when I get home! It tried out a bunch of strategies to pull out the wittiness — like up-ranking good responses and down-ranking bad responses in the bake to encourage the behavior, only up-ranking good responses, and a few other strategies. But the only reliable one was having something like Opus rank responses at generation time. And that would completely defeat the point of training a 4B model, lol.