3DVR · AI Systems Lab← Portal
Field Guide 001 · August 2026

From scaling to agents.
Building the open computing layer around intelligence.

A working field guide inspired by Jeff Dean’s February 10, 2026 Princeton Distinguished Colloquium and a February 12 companion interview. It connects the history of modern AI to a practical 3DVR roadmap: inference-time reasoning, model routing, agent orchestration, verification, OpenClaw, Codex, RUNE, local AI, and eventually open hardware.

Start the guide ↓Jump to the build roadmap
Distinction matters: Dean’s statements are summarized as research context; the 3DVR architecture and roadmap are our interpretation.
01 · Core thesis

AI progress is a systems story.

The useful unit is not “the model.” It is the whole stack: algorithms, architectures, data, training, inference, hardware, distributed systems, tools, evaluators, and human control.

Better algorithms × more scale × better systems = nonlinear capability.
🧠

Architectures

Representation learning, sequence models, attention, Transformers, multimodality.

⚙️

Systems

Distributed training, serving, memory movement, batching, latency, reliability.

🧱

Hardware

Specialized accelerators appear when important workloads expose expensive bottlenecks.

🧭

Inference

Hard tasks can receive more computation, tools, attempts, retrieval, and critique than easy ones.

🤖

Agents

Models become workers when they can plan, act through tools, coordinate, and persist over time.

✅

Verification

“I did it” is not evidence. Reliable systems check outputs against tests, artifacts, constraints, and human approval.

02 · How we got here

The ideas were early. The world had to catch up.

Neural networks were not suddenly invented in the 2010s. What changed was our ability to train large systems, learn useful internal representations, and build hardware that made the workloads economically possible.

Neural networks + parallelismEarly ideas about learned representations and spreading training across processors existed long before today’s scale.
Unsupervised representation learningLarge systems began learning useful internal concepts directly from raw data instead of relying only on hand-coded rules.
Word vectors + sequence-to-sequenceMeaning and transformations could be represented geometrically and learned from examples.
Specialized acceleratorsNeural workloads pushed Google toward TPUs: build hardware around the expensive computation rather than forcing every workload onto a generic machine.
Transformers + self-supervisionAttention architectures and vast self-supervised datasets turned scaling into a remarkably general capability engine.
Multimodal foundation modelsText, image, audio, video, code, and other modalities increasingly live inside unified systems.
Inference-time reasoning + agentsThe system now spends computation after the prompt: plan, search, use tools, compare candidates, critique, retry, and coordinate workers.
The TPU lesson for 3DVR: do not start by designing a RISC-V computer because custom silicon sounds exciting. First discover the computation the future system performs constantly. Measure the bottleneck. Then design the machine that makes that workload cheap.
03 · Inference-time compute

Training builds capability. Inference decides how much of it to spend.

The old mental model was prompt → model → answer. The emerging model gives difficult problems more time, attempts, tools, context, retrieval, and evaluation.

Why this matters

A trivial question and a hard proof no longer need the same computational treatment. The system can adapt effort to difficulty.

Economic implication

Smaller fast models can handle routine work while expensive frontier models are reserved for the moments that justify them.

04 · Gemini + model routing

Stop building intelligence as disconnected islands.

Dean described Gemini as a push toward a unified multimodal effort instead of fragmenting researchers and compute across separate language and modality programs. The systems lesson is broader than any one model family.

Fast model

Low latency, high-volume actions, lightweight coding assistance, classification, routing.

Frontier model

Deep reasoning, difficult planning, ambiguous evaluation, complex synthesis.

Local model

Privacy, offline continuity, cheap repeated tasks, user-owned memory and control.

05 · Models → agents

The interface shifts from operating tools to managing capabilities.

An agent is not merely a chatbot with a longer prompt. It can decompose goals, call tools and other models, maintain state, inspect results, and continue until the mission is complete or blocked.

The future UI may look less like “open the right app” and more like “tell the system what outcome you want.”

Dean has discussed a future in which one person might coordinate many virtual workers, organized into manageable teams rather than directly micromanaging every agent. The practical lesson for us is not “spawn 50 agents tomorrow.” It is to design clean delegation boundaries now.

06 · Verification

“Done” is a claim. Evidence makes it real.

Agent reliability is increasingly a verification problem. Coding is especially powerful because many outputs can be compiled, tested, benchmarked, rendered, and inspected.

Machine-verifiable

Tests, types, HTTP responses, benchmarks, screenshots, file diffs, deterministic constraints.

Model-verifiable

Critics can compare candidates, check rubrics, identify omissions, and challenge assumptions.

Human-verifiable

Approvals remain essential for consequential, subjective, irreversible, or trust-sensitive actions.

Gauntlet principle: change made ≠ task completed. A blocked evidence path should produce BLOCKED, not a confident success message. That is a feature of a trustworthy agent system.
07 · Practical runtime

OpenClaw should coordinate. Codex should specialize.

Do not rebuild a coding agent inside the orchestrator. Let the control plane understand missions, delegate bounded jobs, collect evidence, enforce approvals, and swap workers as the ecosystem changes.

Mission

What outcome are we trying to create?

Worker

Which model, agent, person, or tool should own this bounded job?

Constraints

What must not change? What requires permission?

Evidence

What artifacts prove the work actually happened?

Evaluator

How do we decide pass, fail, retry, or blocked?

Approval

Where must a human explicitly take responsibility?

08 · Specification + RUNE

The new programming skill is describing intent precisely.

Agentic systems make specifications operational. A good mission says what success means, what evidence is required, and where the system must stop for a person.

RUNE · mission sketch
mission homepage_repair {
    goal:
        homepage renders correctly

    worker:
        codex

    constraints:
        preserve existing design
        no production deployment
        do not modify unrelated files

    verify:
        build passes
        tests pass
        homepage returns 200
        browser console has no errors
        screenshot exists

    on_failure:
        retry 2

    approval:
        human before merge
}

RUNE does not need to begin as another giant general-purpose language. Its first useful form could be a human-readable mission specification language for intelligent systems. That is a smaller, testable path toward “computers directed by intention.”

09 · Hardware after workload

Eventually, the workload tells us what machine to build.

Dean emphasizes how expensive data movement can be relative to arithmetic. That makes memory hierarchy, model placement, batching, and accelerator design central to efficient AI systems.

Software → measurement → architecture → silicon. Not silicon first.

Possible future local stack

RISC-V host CPU + open inference accelerator + large shared memory + small persistent local model + user-owned memory.

Cloud escalation

Local systems handle private and routine work; frontier services are called only for problems that justify the cost or capability.

10 · 3DVR build roadmap

Build the layer above the models first.

We do not need to compete with frontier labs at foundation-model training. We can build the open layer that gives intelligence memory, tools, verification, routing, permissions, and a human-centered interface.

Now · 30 days
Tiny agent runtime

Goal → plan → worker → evidence → evaluator → approval. Treat receipts and BLOCKED states as first-class product features.

1–3 months
Bounded specialist workers

Coder, researcher, browser/operator, designer, sysadmin, and sales research—delegated by one orchestrator, not six giant personalities.

3–6 months
Interchangeable model router

Choose local, cheap, coding, vision, or frontier reasoning models by task, privacy, latency, and cost.

6–12 months
RUNE mission language

Turn constraints, success criteria, retries, evidence, permissions, and approvals into a portable open specification.

1–2 years
Hardware exploration from real telemetry

Only after workloads become obvious: evaluate open accelerators, RISC-V hosts, shared memory, local model persistence, and low-power agent runtimes.

North star

Give ordinary people computers they can direct by intention.

The old computer waits for commands. The new computer understands a goal, makes a plan, recruits resources, acts, checks itself, asks permission when necessary, learns from the result, and reports back.

Underneath that human interface can remain everything 3DVR cares about: open source, Linux, RISC-V, local AI, distributed systems, custom languages, user-owned data, and community infrastructure.

Sources + context

Primary anchors for this field guide.

This is a synthesis and product roadmap, not a transcript. Where the guide moves into 3DVR design, it is explicitly our interpretation.

Princeton CS · Important Trends in AIOfficial event description for Jeff Dean’s February 10, 2026 Distinguished Colloquium.
Latent Space · Jeff Dean interviewFebruary 12, 2026 companion discussion covering Gemini, agents, verification, latency, model orchestration, and hardware themes.
OpenAI CodexCurrent example of a coding specialist that can inspect repositories, edit code, run commands, and validate work.