Chapter 88: Seven Billion and Holding
[Gardner Analytics research floor 6:07 PM Tuesday, September 29, 2015]
The dashboard had not moved in nine days.
7,143,220,084 tokens. The same number at 6 PM Tuesday that it was at 6 PM Monday, and the Monday before that, and the Monday before that. The progress bar sat at thirty five percent and had settled there like something geological.
Twelve researchers occupied the long table in the glass walled conference room on two. Priya's team. Half of them were looking at the dashboard. The other half were looking at Priya, drinking espresso from a ceramic mug she had once described, in a company wide Slack thread, as a war crime against Arabica beans. It said I survived peer review in serif font. The espresso was dark enough to use as a resist.
Ethan stood at the whiteboard end, six minutes in now. Nobody had asked him to sit down.
"The licensing split is the hold," Priya said not to him. To the room, with him in it, which was a different register entirely. "Six of you want to include the Common Crawl pull with attribution stripped. Five of you want to exclude it pending a licensing review that Lauren Kim's team cannot complete before November. One of you has not voted."
The one who had not voted was looking at his laptop screen. His name was Derek. He was not the Derek from sales. This Derek was a computational linguist with a 7.1 rating and a persistent issue with authority that read as shyness in meetings and as brilliance in code reviews.
"That's interesting," Priya said.
It was the compositional ratio on the whiteboard she meant, the one Ethan had written there twenty minutes ago. It specified a 40/35/25 split between curated web text, published technical corpora, and synthetic dialogue. The ratio was correct. It was a fact that preceded evidence, the same way he'd known the freight elevator's bearing was failing before anyone called a repairman.
Everyone in the room who had worked with Priya for more than a month went still for a half second. That's interesting, in her register, was not curiosity. It was the sound of a claim being weighed on a balance that did not credit intuition.
"Where does the ratio come from?" she said.
"Scaling problems," Ethan said. "The crossover point at this parameter count favors a narrower, cleaner corpus. A larger dirty corpus buys you volume, but the noise overhead at the attention layers eats the marginal gain past about thirty five percent synthetic."
"That's the mechanism. I'm asking about the provenance."
He had expected the question just not for it to arrive this precisely. The last time she had asked, six weeks ago, about a different compositional call, she had said "where does that number come from" and accepted the deflection after one follow up. This was the second follow up, and it was sharper.
"The crossover point is sensitive to tokenizer quality and context window distribution," he said. "At 175 billion parameters with a 2048 token context window, the dirty corpus underperforms past a certain ratio. I can't show you the ablation because we haven't run it. I'm asking you to run it."
"You're asking me to redirect six researchers to validate a compositional preference you can't source."
"I'm giving you a budget and a deadline. Nine days to 9.4 billion tokens. If the ablation fails, we fall back to the larger corpus and lose two weeks. If it holds, we gain five."
Priya set the mug down. The ceramic hit the table with a sound like a period at the end of a sentence. Off came her glasses the tell every person in the room recognized, the one Ethan had catalogued months ago: a model of the world was being revised.
"Maya," she said.
Maya looked up from her laptop. She was twenty four, rated 6.5, and had the useful habit of asking for decisions in front of people who did not want to make them.
"Can you implement the 40/35/25 split by Friday?"
"If licensing clears the synthetic source by Thursday, yes."
"Then that's the question." Priya turned to Ethan. "Licensing. Not composition. If you can get Lauren Kim to sign off on the synthetic corpus by Thursday, the split runs Friday. I'm not overruling a licensing concern on an unsourced preference. I am implementing a testable hypothesis with a deadline."
She put her glasses back on.
"That's interesting" had not been followed by "but." It had been followed by a budget and a timeline, which was worse. She had accepted the deflection without believing it and built a process around it that would either validate or invalidate the call independent of its source. Handling his unsourced certainty the way you handle a radioactive sample: with tongs, behind glass, at a measured distance.
He did not dislike the sentence when it formed; he disliked that it had formed at all. Somewhere along the way, he had begun sorting the people in his building by how much of his unexplained certainty they would absorb before the cost exceeded the value. Priya was the most expensive person in the building to lie to. He had just done it anyway, because the corpus ratio was correct, and the January training window would not wait for him to explain how he knew.
The meeting broke at 6:40. Maya stayed to configure the pipeline. Two researchers followed Priya out, already arguing about the synthetic licensing language. Derek closed his laptop last.
Ethan stayed at the whiteboard. The ratio was still up there in blue marker, specific to a decimal place that implied prior observation. He had written it in front of the one person in the building who had said, out loud, in a meeting, that specificity implies prior observation. She had not said it tonight. She had not needed to.
His phone buzzed a calendar notification. Expecting the Thursday licensing call, he opened it.
It was a cancellation. Kai's mentor, the senior researcher, 7.8, the one who had taught two of the people in this room how to build tokenizers, had canceled his one on one. Not rescheduled. Canceled. The calendar note said only personal.
This was the second cancellation two reschedules before it, then a cancel. Ethan scrolled back. The first reschedule had been the week Hooli's call list went live. Diana had started logging the list quietly, in a spreadsheet she had not shared with him yet, though he knew it existed anyway. Diana built spreadsheets the way other people breathed. Two of Priya's twelve were on it, and he had known which two for a week.
At the top of the stairs, the research floor was half dark. The dashboards were live. Maya's pipeline was already pulling the synthetic corpus, three hours ahead of the Thursday deadline Priya had set. Maya was the kind of person who treated a Friday deadline as a Thursday night, and a Thursday night as a Wednesday afternoon.
The cancellation notification remained on his phone. Personal. He read it again, and left the app open instead of closing it. What he opened next was a note, not SHOW_NOTES, which was gone, just the phone's native notes app, the gray default icon. He typed the date, the name, and the word personal in quotes. Then he stared at the three lines and considered whether he was keeping a list or building an inventory.
The phone went into his pocket. The freight elevator ground open on the ground floor. The pastrami smell was faint at this hour, an overnight trace, the building settling into the version of itself that existed after everyone went home.
Priya was at her desk, visible through the glass wall. Not looking at a screen writing in a notebook, not the company laptop, not the shared doc. A physical notebook with a black cover, one he had seen once before, on her desk, closed. She was writing the date. The rest he could not see. The glass was frosted on her side, and he did not go closer.
He took the stairs down. The Honda was four blocks away. It started on the first try, which was unusual and which he did not notice.
✦ ━━━━━━━━━━━━━━━━━━━━━━━━ ✦
Read more chapters for free at unwrittenrealm.com
✦ ━━━━━━━━━━━━━━━━━━━━━━━━ ✦
TL;DR — Patreon has the chapters ahead.
Silver / 10 ahead
Gold / 15 ahead
Platinum / 20 ahead, no weekly wait
That's it. patreon.com/fanficwriter1
If reviews aren't your thing, no pressure — they do help though.