[Gardner Analytics Office, Third Floor (Mezzanine) — July 2015, 6:00 AM]
The mezzanine had become Ethan's refuge. The main floors — first and second — were populated from eight to seven with seventy-five people whose keyboards and conversations and coffee-machine rituals generated a constant acoustic baseline that made deep technical work impossible. The mezzanine, originally storage, now held four meeting rooms and a narrow strip of desk space that nobody used because the ceiling was low and the lighting was industrial. Ethan used it because the ceiling didn't bother him and the solitude was worth the ambiance.
The pressure had been building for two weeks. Not the headache of forced resolution — that belonged to Phase 1, the early stage where architectures were fog and clarity required effort. This was different. Denser. The architectural awareness expanding beyond GPT-2's now-familiar terrain into territory that was vast in a way the previous generations hadn't been.
GPT-2 sat in his mind at Phase 2, approaching Phase 3. Three months of production deployment — the documentation product running on the optimized model, six enterprise clients generating revenue, the compressed version running on consumer hardware through the Pied Piper integration — had deepened his understanding to the point where the 1.5-billion-parameter architecture was as familiar as the original Transformer. He could mentally trace attention patterns through all forty-eight layers, predict how the model would respond to specific prompt structures, identify the scaling behaviors that Priya's evaluation battery had confirmed as emergent.
And beyond that familiarity, at the edge of the architectural landscape, something enormous was taking shape.
Not four options this time. One. A single structure, so large that its dimensions occupied the entirety of his expanded awareness like a mountain filling a valley. The blueprint was forming with the slow resolution of a photograph developing in chemical solution — details emerging from noise, contours sharpening from blur, the shape of something that was unmistakably GPT-3.
175 billion parameters.
The number arrived in his awareness the way the previous generation specifications had arrived — not as a calculation but as a spatial fact, a dimension of the blueprint that existed independently of his analysis. 175 billion. Not 100 billion. Not 200 billion. The specific parameter count that, in his previous life, had produced the model that changed everything — the model that could write code, compose poetry, answer questions across every domain, engage in multi-turn conversation with contextual awareness that felt, to its users, like understanding.
The training cost calculation was automatic. ChronoCloud's V100-equivalents at $50 per hour. A training cluster of sixty-four instances — four times the GPT-2 configuration. Estimated training time: three thousand GPU-hours. Estimated cost: $4.6 million.
Four-point-six million dollars for a single training run. The number sat in his awareness beside the company's bank balance — approximately $9.5 million after the past months of operational costs, salaries, and the GPT-2 training — and the arithmetic was immediate: one GPT-3 training run would consume nearly half their remaining capital. Multiple runs for hyperparameter optimization could consume all of it.
He sat on the mezzanine in the pre-dawn dark, the low ceiling pressing overhead, the server rack's hum drifting up from the floor below. The blue marker was in his hand. A portable whiteboard — the kind that rolled on wheels, purchased for the meeting rooms — stood in front of him. He drew.
The GPT-3 architecture. The same decoder stack as GPT-2, scaled by a factor that transformed the diagram from a building into a skyscraper. 96 layers. 96 attention heads. Embedding dimension: 12288. Vocabulary: 50,257 tokens (the BPE tokenization that the blueprint specified with characteristic precision). Context window: 2048 tokens.
The dimensions were specific. Exact. The kind of specificity that Priya had challenged in December — "you drew these without a single question mark" — applied now to a structure so large that the compute requirements alone would determine whether the company survived to build it.
Priya arrived at seven-thirty. She'd developed the habit of checking the mezzanine before going to her desk, because Ethan's predawn whiteboard sessions had become a leading indicator of strategic shifts that would eventually reshape her research agenda. She stopped at the top of the narrow staircase, saw the diagram, and stood still for ten seconds.
"That's not GPT-2," she said.
"No."
"The layer count is 96. The embedding dimension is 12288. The estimated parameter count is—" She ran the calculation in her head, the mental arithmetic of a nine-rated theoretical mind processing dimensional specifications the way a pianist processes sheet music. "175 billion. Approximately."
"Approximately."
Priya walked to the whiteboard. Her red marker appeared. She didn't annotate — she just stood and read, tracing the architecture with her eyes the way she traced loss curves on monitoring screens. The assessment was comprehensive: layer dimensions, attention head count, feed-forward network sizes, positional encoding limits.
"The training compute for this model," she said, "at our current hardware configuration, would require approximately three thousand GPU-hours. At sixteen V100-equivalents, that's approximately eight days of continuous training. At our ChronoCloud rate, the cost is—"
"Four-point-six million."
"Which is roughly half our bank balance." She capped her marker. "Ethan. This model would be a hundred times larger than GPT-2. The scaling hypothesis — that bigger models produce qualitatively better output — has been confirmed at 1.5 billion parameters. But a hundred-X jump isn't incremental scaling. It's a phase transition."
"I know."
"The emergent capabilities at this scale would be... I don't have a theoretical framework for predicting what a 175-billion-parameter language model would do. Nobody does. The scaling laws suggest capability improvement, but the type of capability that emerges at this scale is genuinely unknown."
She was wrong, and he couldn't tell her. In his previous life, GPT-3 had emerged as the model that bridged the gap between "AI that writes well" and "AI that reasons." Few-shot learning. In-context adaptation. The ability to perform tasks it had never been explicitly trained for, simply by receiving examples in the prompt. The model that had made the world take artificial intelligence seriously — not as a research curiosity but as a technology that would reshape every industry.
"Series B," Ethan said. "We need to raise before we can train at this scale. The current capital covers operations and product development. It doesn't cover a four-million-dollar training run."
"Series B timing?"
"Six months. Eighteen months after the Series A — standard for a company with our growth trajectory. We'll have a year of revenue data, the partnership metrics, GPT-2's deployed capabilities. The pitch is strong."
"And the training happens after?"
"The training happens after." He set the blue marker on the whiteboard's ledge. The GPT-3 diagram blazed against the white surface — 96 layers of decoder blocks, each one a column of attention and transformation, the whole structure a tower that reached toward a ceiling that Ethan's architectural awareness couldn't yet see beyond. "But we prepare now. Data curation. Training infrastructure optimization. Hyperparameter search at reduced scale. By the time the Series B closes, we're ready to launch the training run on day one."
Priya picked up her coffee — the I survived peer review mug, refilled from the commercial machine that she still described as "a war crime against beans" without any reduction in consumption. She drank, studied the whiteboard, and made a decision visible in the set of her jaw: she would work on this without asking how Ethan knew the specifications. The tolerance she'd described as finite had, apparently, expanded its limits in proportion to the quality of the problems she was asked to solve.
"I'll start the data curation pipeline," she said. "The GPT-2 corpus was two billion tokens. For this scale, we'll need at least ten times that. Possibly more."
"Twenty billion tokens. Minimum."
The number was from the blueprint. Priya didn't question it. She turned and descended the mezzanine stairs, already typing on her phone — probably texting her research team, mobilizing the data infrastructure that would need to be built from scratch because no existing pipeline could handle twenty billion tokens of curated training data.
Ethan stayed on the mezzanine. The whiteboard glowed in the morning light that filtered through the narrow windows near the ceiling. The GPT-3 architecture — the model that would change the world a second time, built on temporal hardware by a team that didn't know the future but was creating it — waited on the surface like a blueprint pinned to a construction-site wall.
The first step was money. The second step was data. The third step was compute. The fourth step was whatever came after — the deployment, the product, the impact that a model of this scale would have on the industry, the economy, and the fundamental relationship between humans and machines.
But the first step was money. Always money. The fuel that everything else ran on, the resource that determined whether the architecture in his head became a machine in the world or remained a drawing on a whiteboard in a mezzanine above a sandwich shop.
Series B. Six months. The climb continued.
Author's Note / Promotion: Your Reviews and Power Stones are the best way to show support. They help me know what you're enjoying and bring in new readers! You don't have to. Get instant access to more content by supporting me on Patreon. I have three options so you can pick how far ahead you want to be: 🪙 Silver Tier ($6): Read 10 chapters ahead of the public site. 👑 Gold Tier ($9): Get 15-20 chapters ahead of the public site. 💎 Platinum Tier ($15): The ultimate experience. Get new chapters the second I finish them . No waiting for weekly drops, just pure, instant access. Your support helps me write more . 👉 Find it all at patreon.com/fanficwriter1