Production traces and the future of data

A decade ago, helping train AI meant drawing a box around a pedestrian. Today, it involves writing a legal brief or building a financial model. The progression of human data vendors and their shapes of choice reveals a clear strategy for improving models: find people who can do harder things.

How far can this approach take us? As models rapidly outperform the world’s experts, the pool of humans who can reliably stump the model frontier shrinks while the industry competes harder for the same experts. Contractor-generated training data starts to look like a race to the bottom.

Yet even a system with superhuman abilities will encounter unfamiliar businesses, changing requirements, and problems it has never seen. We believe the products through which AI does real work will accumulate evidence about those problems and what it takes to solve them. That manifests in the form of production traces.

In this way, the economy itself can become the largest source of training data for advanced AI. As models improve, people entrust them with harder work, thereby creating new problems to learn from. A data company built around that experience can keep supplying useful training material long after models surpass human experts. This is the company we are building at San Francisco Data.

Why production traces matter

We define a production trace as the chronology of AI working on a real task. Depending on the application, it includes the user’s request, the context available to the model, its tool calls, and subsequent feedback. Traces preserve enough information to understand both the intended outcome and the path taken toward it. That makes them the optimal starting point for studying the gap between a model’s capabilities and its intended real-world output.

Having only a finished artifact often leaves this gap impossible to reconstruct. While a static repository shows the assumptions someone ultimately settled on, the surrounding interaction may explain why an apparently reasonable solution failed or why a requirement that looked minor was essential. When that context is recorded, it gives us a way to build training tasks around decisions that would otherwise be impossible to reverse engineer.

The supply of this information can grow with the use of AI itself. An agentic product generates new sessions as users work on new problems. Each deployment creates opportunities to observe where the system fails.

Stronger models increase the complexity of available tasks. As a stronger agent handles a familiar class of problems reliably, users delegate even larger changes, exposing a different set of difficulties. Our belief is that production data will keep moving toward the boundary of what models can accomplish, because people will keep asking useful systems to do more.

Turning traces into training data

Turning traces into high quality training data that verifiably improves models is a challenging and open research problem.

The first problem is identifying sessions that contain something worth learning. Many production traces will be repetitive, incomplete, or too basic to contain content valuable enough for frontier model development. A failed attempt may reveal a broken environment rather than a model limitation.

The next problem is turning those sessions into reproducible tasks. This requires recovering the user’s explicit and implicit objectives, including requirements introduced through later corrections, and enough of the original environment for another model to attempt the work. A missing input or unavailable service can make an otherwise valuable trace unusable without further augmentation. Preserving the initial prompt alone may produce a task that rewards precisely the behavior the user rejected.

The final problem is evaluation. Existing tests may overlook an important constraint, and naïve user approval is not always evidence of correctness. A verifier must capture the behavior the user actually needed and resist attempts to exploit the evaluation. Otherwise, additional training can make a model better at passing the grader without improving its ability to do the work.

The value of this work ultimately depends on the capabilities it produces. Internal post-training experiments help establish which data improves performance on new tasks and which merely add volume.

The volume of these traces will continue to grow at an incredible rate, and the training value will increase accordingly. Properly diffusing this value requires a market.

Building a market for production traces

We do not believe the world will end with everyone labeling RL environments. Companies have sources of competitive advantage beyond their data, including but not limited to network effects, social capital, distribution moats, and regulatory advantages. A model trained on a company's activity can learn the work, but it cannot inherit the market position, relationships, and customer trust that win the company its business.

There is a commercial opportunity alongside the technical one. We expect useful traces to emerge across many products, each serving different users and observing different kinds of work. The companies operating those products should remain laser focused on making them useful. Developing a research pipeline for training data, understanding what individual lab teams need, and managing data licensing requires expertise far outside their core business.

For many companies, monetizing data will strengthen the business they already have. A frontier lab can often extract substantially more value from the training value of traces than the company producing it can capture on its own. The lab can use that data to improve models deployed across many applications, while the source company can share in the proceeds without having to build a frontier model itself. It can then reinvest that capital in better products, stronger distribution, and the relationships that sustain its business.

The relationship has to begin with a clear understanding of what data the partner can provide and how it can be leveraged. Production sessions often contain customer information and proprietary material so permissions and appropriate filtering inform which tasks we can construct. A valuable stream is one we can repeatedly turn into useful training data under an agreement that supports that work.

These principles are core to how we operate at SF Data. We do not work at arm’s length. Instead, we structure our partnerships to collaborate closely with each partner, helping them understand what their traces can become. Sometimes that means helping a company see value in data it didn’t know it had. Other times it means designing a new pricing model for a kind of data no one has sold before.

We often describe our ambition as a Glencore for production traces. Glencore did not need to own every mine or oil field. Its advantage came from access to fragmented supply, expertise in processing and structuring it, and relationships with concentrated buyers. Raw traces are not training assets any more than crude oil is jet fuel.

The future of data

The majority of economically valuable traces generated today come from coding agents, and that’s where we are beginning. As AI comes to perform a greater share of economic activity, its interactions with the world will become an enormous source of evidence about how that activity works. Model progress creates more capable systems, those systems encounter harder tasks, and the resulting data will support further model progress.

We want San Francisco Data to become the company through which that data reaches the systems learning to do the work. Our long-term vision is to build a continuous connection between AI deployed across the economy and the training processes that determine what AI can do next.

To build this future, we’re actively hiring incredible human engineers and operators. If you are interested in collaborating with us, you can reach us at founders@sfdata.com.