Chapter 117: The Door He Closed
[Gardner Analytics mezzanine Tuesday December 22, 2015 10:40 PM]
The mezzanine smelled like blue marker and dust.
Ethan sat on the floor because the four desks Sarah had carved out of banner storage didn't have chairs yet. The floor was honest about what this room was: a landing between two staircases that nobody used, because the freight elevator worked most of the time.
`PHASE 3: GPT 3 175B parameters. Series B. The climb continues.`
Blue marker, Sarah's hand. The sentence had been up since September. Below it, an unfinished line from two weeks ago `PHASE 4:` and then nothing, because he hadn't known what to write and still didn't.
The laptop on his knees had the Series B term sheet open. Paragraph 7(c): technical milestone covenant. Rebecca Zhou's addition, dated October, the one she'd added after he'd failed to answer a question about the GPT 3 blueprint's attention scaling. The language was boilerplate. It required the company to demonstrate a converged frontier scale model within eighteen months of close. Boilerplate, because every Series B AI deal in 2015 had some version of it. Boilerplate, because nobody on the other side of the table thought the founder couldn't do it.
He read the paragraph again. Then he closed the laptop and set it on the floor beside him and reached.
This was the word he used inside his own head reached because it had a shape. Every generation boundary since 2014 had felt the same: a door, and on the other side of it, a corridor. Sometimes wide, sometimes narrow. Gen 1 had been a hallway. Gen 2 had been a fork. Gen 3 had been a single door. Gen 4 the one that gave him 175B as a spatial fact had been a wall with one opening in it. The opening was exactly his size.
He reached now the same way he'd reached then. The same motion. The same internal posture not thinking, not looking, something between the two like reaching into the dark for a doorknob, certain it was there.
Nothing.
Not nothing like empty. More like a sealed wall, no trace of the door that used to be there. The surface was smooth. No seam, no handle, no give. He pressed against it the same way he'd pressed against every generation boundary. It did not move. The not moving had a texture he recognized he'd felt it once before, at Gen 2, when he'd tried to reach a path he'd already closed.
But he hadn't closed this one. He hadn't chosen a door at Gen 5. There was no door to have chosen. The corridor hadn't opened.
He sat with this for a long time. The freight elevator's bearing ground through the wall a sound Marcus had filed in the operations risk register as "probably fine until it isn't". The pastrami trace rose through the floorboards, faint, the overnight ghost of the daytime smell. He'd liked that smell on Saturday. He liked it less now without knowing why.
The diagnosis arrived the way diagnoses did for him: not as a feeling, but as a sentence he couldn't unhear.
He had not written a line of GPT 3's core.
The architecture, he'd approved. The parameter count, he'd specified. He'd reviewed the tokenizer stack. He'd signed off on the data curation protocol. He'd directed the training configuration every piece of it. Everything a CEO does to a model, he had done. None of what an architect does to a model, he had not. The distinction had never mattered before, because the distinction had never existed. For four years he had been both the man who wrote the code and the man who approved it. Those had been the same man. Now they weren't.
Arch HR4. Personal Work. The requirement was specific in the way only power rules were specific: personal, meaning his hands. Core components, meaning the load bearing parts. Deep understanding, meaning he could reconstruct it from a blank screen. No delegation, meaning no delegation.
He tested it. He had to test it because the rule was the rule and he'd been wrong about rules before.
The first test: the tokenizer. He'd written the original GPT 2 tokenizer in March, every line, at 5x, in a single sitting. He could feel it: the stack, the merge rules, the byte pair encoding. He remembered exactly where the unicode handling broke, on the seventeenth pass. He reached for it. It was there. Familiar, warm, his. The door was open. The tokenizer was something he had personally written and deeply understood. HR4 satisfied for the tokenizer.
The second test: the attention mechanism. He'd designed the original architecture in 2014, published it, and personally implemented every variant through Gen 3. GPT 3's attention was a scaling of that, and he could feel the scaling as a fact but he could not feel the implementation, because there wasn't one. His team had written it. Marcus had configured the distributed training. Priya's team had curated the data. He had approved it, and approval was not personal work.
The door was closed. He felt the same wall.
The third test: the training loop itself. The optimizer, the gradient clipping, the learning rate schedule. He had specified all of it. He had not written any of it. He could describe the schedule to a whiteboard, but the requirement wasn't describe. It was written. It was deep. It was personal.
Three tests. Three walls. Each one in a different place, each one the same texture, each one confirming the same sentence he could not say aloud: the ladder would not move on approval.
He sat on the mezzanine floor and the sentence sat with him and neither of them moved.
A hundred and fifty days. That was the number. Mid July to December twenty second. A hundred and fifty days of being good at the wrong job: calendar management, hiring decisions, patent strategy, investor calls. A thousand small acts of competence that had kept the company alive through Prometheus. Each one reasonable. Each one a meeting. Each one an hour not spent at a keyboard writing a core component he could have written in his sleep. He hadn't, because the company had needed him to be the CEO instead of the architect. And he had been a good CEO.
Operation Prometheus's structural theory keep him in meetings had worked. And nobody at Hooli knew there was anything to work on.
That was the part he sat with longest. Not the wall. Not the closed door. The fact that there was nobody to be angry at. Gavin hadn't done this. Vincent hadn't timed this. The patent hadn't caused this. He had caused this by being competent at the job he'd been hired to do. That job was not the job the power required. The power did not care about the distinction.
He opened the laptop again. Not the term sheet. His calendar.
The training run was nine days out. Compute booked, data curated, team staffed. The round's close was contingent on the technical milestone covenant. Hooli's second poaching wave was inside three weeks. He could feel it on the research floor the slope rising, the variance widening, the mean holding but the edges fraying. Monica had a board dinner on the fourteenth. There was an investor call on the sixteenth. The poaching wave would land on the seventeenth or the eighteenth.
He blocked out seventy hours in three chunks. Tuesday the twenty ninth through Thursday the thirty first. Sunday the third through Tuesday the fifth. Thursday the eighth through Friday the ninth. The title of the blocks was a single word: `Review`.
The board dinner conflicted with the first block. He moved the dinner. The investor call conflicted with the second. He moved the call. The second poaching wave would conflict with the third. He would not move the wave. The wave would be Marcus's problem, Sarah's problem, and Monica's problem before it was his. Every one of those three conflicts would be somebody else's problem before it was his.
He saved the calendar. He closed the laptop. He did not tell anyone.
The Blue Bottle drip was on the ledge. Sarah. He hadn't heard her come in, hadn't heard her leave. The cup was still warm. She had left it and asked one question and accepted one answer and gone, because that was what she did.
"What are you doing up here?"
"Thinking about deployment constraints."
"Okay."
She had not believed him. She had not disbelieved him. She had done what she had been doing since 2014: provided context, accepted the deflection, and left a coffee.
He picked up the drip. It was good. He drank it, looking at the unfinished `PHASE 4:` on the wall. The blue marker. The dust. The freight elevator's bearing, grinding through the wall. He was entirely alone with the loneliest fact of the year: there was nobody to tell.