Nostr notes by Travis Frisinger (Unofficial)

Software Was First. Expertise Is Next. Every organization has ...

2026-10-16T04:44:47Z

Software Was First. Expertise Is Next.

Every organization has someone who just knows how things work.
Maybe it's Sally in accounting. She's been there for fifteen years, understands how operations affect the books, knows which exceptions actually matter, and can spot a problem three departments away before anyone else realizes something is wrong.
She's not necessarily the most senior person in the company. But when something unusual happens, everyone knows who to call.
Now imagine asking Sally what she does all day. You'd probably hear about reports, reconciliations, approvals, and the occasional fire she has to put out.
If you're looking for AI opportunities, those workflows seem like obvious candidates. Automate the reports, reconcile the numbers, reduce interruptions, and give Sally back a few hours every week.
That's a perfectly reasonable place to start. But we're missing something much more valuable.
The Value Isn't in the Workflow
What makes Sally exceptional isn't her ability to reconcile accounts. It's the knowledge she's accumulated about how the business operates and, more importantly, how different parts of the business affect one another.
She understands accounting and operations. She knows where they intersect, which rules can bend, which ones cannot, and what usually happens when someone makes an innocent-looking change upstream.
Some of that knowledge is documented. Most of it isn't.
We've spent decades building software to support people like Sally. Dashboards, reporting tools, workflow engines, and increasingly sophisticated automation. Yet the thing that makes them valuable often remains outside those systems entirely.
It lives in their judgment.
The bigger opportunity is capturing that judgment, not simply automating the tasks around it.
What Are We Actually Trying to Improve?
Suppose a business believes it could increase revenue by 5% if it could eliminate a particularly painful operational process.
The obvious response is to automate it. Study the steps, map the workflow, identify integrations, and build something that moves information through the business more efficiently.
But what if that process exists because three departments interpret the same information differently?
What if one experienced employee has spent years translating between them?
What if the real constraint isn't the process at all, but the organization's inability to consistently apply knowledge that only a handful of people possess?
Now we're solving a different problem.
We could automate the repetitive work so Sally can spend more time improving operations. We could capture the reasoning she applies to exceptions so less experienced employees can handle more complicated situations. Or we could discover that her understanding of accounting and operations could support an entirely new service the business couldn't previously offer.
Those are very different opportunities, and they require different systems.
The common thread is finding where human knowledge creates value and making more of that value available.
Software Is Becoming Cheap Enough to Ask Bigger Questions
We're moving beyond using AI to help engineers write code. Increasingly, we're building systems capable of performing substantial portions of software development autonomously, from implementation and testing to debugging, review, and architectural exploration.
I've spent much of this year experimenting with that transition. Building software factories, pushing their manufacturing capacity, discovering their constraints, and learning how to relieve bottlenecks as the systems take on more responsibility.
And somewhere along the way, my perspective shifted.
I started out asking how much software we could manufacture. I'm increasingly interested in what becomes worth manufacturing when software itself becomes dramatically cheaper.
Think about that specialized system a small organization could never justify building. The internal capability that previously required an entire engineering team. The unusual combination of business rules, expertise, and judgment that doesn't fit into any existing SaaS product.
What happens when we can afford to build those things?
Of course, inexpensive implementation doesn't make the entire problem inexpensive. Understanding the business, capturing knowledge, establishing correctness, and integrating into real operations remain difficult.
But those challenges are exactly where the opportunity is moving.
And we've tried something like this before.
Expert systems in the 1980s promised to capture specialist knowledge and make it available through software. They demonstrated real value, but three persistent problems limited their reach. Encoding expertise was expensive. Maintaining that knowledge as circumstances changed was difficult. And discovering where the system's reasoning failed often required substantial expert effort.
What's changing now is the economics of all three.
Software factories can dramatically reduce implementation costs and make continuous adaptation more practical. Modern AI systems can help extract knowledge from examples, documents, conversations, and observed decisions. And rapid development and evaluation cycles make it possible to put working systems in front of experts far earlier.
An expert may struggle to explain every rule they apply, but showing them a concrete result often reveals missing context, exceptions, and assumptions.
Their corrections become part of the knowledge capture process.
This doesn't eliminate the need for careful validation. It changes how quickly we can discover what we don't yet understand.
The Software Isn't the Valuable Part
Return to Sally.
Imagine spending time understanding how she actually makes decisions. Examining real cases, capturing exceptions, identifying the information she relies on, and establishing how she recognizes a good outcome.
Then imagine manufacturing a system capable of applying some of that expertise consistently.
It might contain traditional software, deterministic rules, language models, simulations, or autonomous agents. Probably some combination of them.
The achievement is taking knowledge that was previously difficult to distribute and turning it into something the organization can reliably use.
That capability could handle routine decisions, help junior employees tackle more difficult problems, or let Sally apply her expertise across ten times as many situations without personally intervening in every one.
It might even reveal an entirely new business opportunity.
And Sally has a stake in how this happens. Done well, her expertise can reach further than she ever could alone, while giving her more opportunity to shape how the business operates. But capturing knowledge also changes who controls it and how it gets used.
The strongest systems will be developed with the expert, not simply built around them.
This isn't automating what Sally does. It's making what Sally knows executable.
That distinction changes the question we should be asking.
Instead of starting with "What workflows should we automate?", start with:
What does your organization know how to do exceptionally well that it cannot easily replicate?
Sometimes the answer will be a workflow. Sometimes an expert's judgment, an unusual intersection of disciplines, or decades of accumulated institutional experience.
Sometimes the answer might be something nobody has thought to turn into a system before.
Making Human Knowledge Executable
I've been thinking about this through the lens of software manufacturing, but software engineering is simply the first domain where I could prove something much larger.
We're entering a period where enormous portions of human knowledge can increasingly be expressed as goals, constraints, decisions, evaluations, and executable systems.
Not everything can be captured cleanly. Some expertise is contextual, some knowledge is tacit, and some decisions involve human relationships and values that resist formalization.
And making something executable certainly doesn't make it correct.
But look at the direction we're heading.
We can increasingly capture expertise, encode how it gets applied, manufacture systems around that knowledge, and evaluate whether those systems produce useful outcomes.
The implications extend far beyond software engineering. Manufacturing, finance, medicine, construction, logistics, scientific discovery, and almost every other knowledge-intensive field have capabilities that remain constrained by the people who possess them.
The economics of software changed first. The economics of expertise are next.
I've started calling this making human knowledge executable.
Over the next three to five years, I believe some of the most interesting opportunities will come from discovering things organizations have known how to do for decades but never had the means to scale.
How much extraordinary value is hiding in plain sight inside our organizations, waiting to become executable?

https://aibuddy.software/software-was-first-expertise-is-next/

In a Room Full of Builders, Where Are the Architects? Harness ...

2026-10-16T03:52:47Z

In a Room Full of Builders, Where Are the Architects?

Harness engineering is transforming how software gets built. But are we preparing engineers to work differently?
For the past year, I've been engineering software without reading the code.
I don't review pull requests. I don't inspect implementations. I don't spend my days supervising agents as they work through individual changes. I describe what I want, evaluate what comes back, and correct what doesn't meet my expectations. I'm still responsible for the architecture, the quality, and the outcome. I've simply stopped treating direct involvement in implementation as a prerequisite for engineering.
And that brings me to one of the most interesting developments in AI engineering right now: harness engineering.
It's becoming another buzzword in an industry that produces them at an astonishing rate. But this one represents something genuinely transformative. We're moving beyond simply asking AI to write code. We're designing the environments, constraints, tools, feedback loops, and verification mechanisms that allow agents to perform increasingly complex engineering work with greater autonomy.
Yet much of the conversation remains focused on the machinery. How do we build better harnesses? How do we orchestrate agents? Where do we put humans in the loop? How do we review everything the agents produce?
Those are useful questions, but I think we're missing a much bigger one. We're building systems that no longer require engineers to participate in every implementation decision. And yet, we're still organizing much of our engineering practice around exactly that involvement.
I don't need a software factory to work differently. I can use ordinary AI coding tools and operate at the same level of abstraction. The factory makes that approach repeatable, autonomous, and scalable. But the technology isn't what makes the operating model possible. It's the engineering discipline behind how we direct, constrain, and evaluate the work.
As organizations begin delivering against this model, the gap is becoming difficult to ignore. We have tools capable of executing work at extraordinary speed. What we don't have is enough engineers prepared to direct that work without reverting to the development practices they've spent their careers learning. That's not a tooling problem. It's a capability problem.
Six weeks, one operator
On September 4, HydraFlow v2, 8th Light's software factory, was a specification. No code. A design, a plan, and a collection of opinions about how software should be produced when agents do the building.
Six weeks later, it has merged roughly 1,100 pull requests. It plans its own work, implements changes, runs verification, reviews the results, and merges what passes. A supervisor agent monitors the system, recovers stalled processes, and communicates with me through Slack.
The pull request count is a measure of throughput, not proof of quality. But the interesting part isn't how much code the factory produced. It's what I actually did during those six weeks, because it wasn't building software.
What the architect did
On day two, staging went red. Three pull requests had passed their individual checks but broke the build when merged together. I didn't debug the code. I changed the rule: nothing merges unless CI is green against the combined result. One architectural decision changed how the factory would handle every subsequent merge, and I didn't need to inspect the implementation to make it.
A few weeks later, I encountered the opposite problem. The factory was waiting on approvals and controls I'd established earlier. Those gates had seemed reasonable when I introduced them, but the evidence increasingly showed that I was becoming the bottleneck. Some controls were removed, others relaxed or delegated to automated verification. The important question wasn't whether we could eliminate human involvement. It was whether each control still justified its cost.
Knowing when to remove a control is just as much an architectural decision as knowing when to introduce one. In traditional development, an unnecessary approval might delay a developer. In an autonomous factory, the same approval can constrain an entire production system.
Another example came when the factory proposed a cohesion gate. It flagged 44% of production modules and 48% of test files. At first glance, those numbers suggested a significant quality problem. But the similarity between two different populations raised a more interesting question: was the measure identifying poor design, or merely describing characteristics common to both populations? I challenged the measure rather than automatically turning it into another gate.
No code inspection was required. What mattered was understanding what the metric actually told us and whether it justified an architectural constraint. These are the decisions I spend my time making now, not how to implement a feature, but what conditions must hold for the system to implement features reliably.
One operator, a hundred engineers' velocity, a thousand engineers' chaos
There's a phrase I've started using to describe what operating a software factory feels like: one person moving at the velocity of a hundred engineers, creating the chaos of a thousand.
It's not a literal headcount comparison. It's what happens when the rate of change is no longer limited by the number of people producing it.
Recently, in another factory experiment, I moved more of the workload onto dedicated hardware. On-premise CI reduced the cost of verification, additional CPU capacity allowed more builds to run concurrently, and centralized infrastructure gave the system room to expand. But removing those constraints exposed an entirely different class of problems.
During one particularly difficult evening, a change to shared branch history created 52 file conflicts across 23 dependent branches. Workflow state drifted from reality after restarts. Changes to the factory's own machinery interfered with work already moving through the system. None of these were fundamentally problems with generating code. They were problems of coordination, shared state, and operational control.
A large engineering organization has established ways of managing these problems. Teams coordinate migrations, freeze changes, manage ownership, and introduce controls around integration. Much of that discipline is embedded in the organization and its people. A factory operating at comparable throughput doesn't inherit those safeguards automatically, and one operator cannot personally coordinate every decision at that velocity.
So the architecture has to do it.
The system needs to reconcile its own state after failures, coordinate changes to shared infrastructure, limit conflicting operations, and test changes to its own machinery before applying them broadly. Perhaps most importantly, it needs to learn collectively. A failure discovered by one agent or workflow should become a test, constraint, or operational rule that protects every subsequent execution. Knowledge cannot remain trapped in individual sessions or depend on the operator remembering what happened.
The goal isn't to make one person capable of supervising a thousand changes. It's to architect a system that doesn't require them to.
That's where harness engineering becomes much more interesting than another collection of tools and prompts. We're designing the conditions under which software can be produced reliably at a scale that human-centered development practices were never designed to manage. And it demands a different kind of engineering judgment.
The engineering shift: from writing and reviewing code to defining intent, establishing constraints, and designing the systems that produce and verify software autonomously.
The industry needs to pivot
Here's where this becomes more than a story about software factories.
We have an industry full of extraordinarily capable engineers. People who understand distributed systems, security, architecture, infrastructure, and the complexities of production software. Yet give those same engineers increasingly capable AI tools, and we often see familiar development processes reappear. Tasks are assigned, implementations inspected, pull requests reviewed, and humans remain responsible for moving work through the system. We've changed the tools without fundamentally changing how we practice engineering.
This isn't a criticism of those engineers. It's what the industry has spent decades teaching us to do. Our processes, organizational structures, career ladders, and definitions of engineering excellence developed around a world where producing software required substantial human involvement in implementation. That world is changing, and the constraint is shifting from our capacity to produce software to our capacity to direct, constrain, and validate its production.
My experience scaling factory throughput has reinforced that shift. Every time we remove one constraint, another becomes visible. Implementation gives way to verification. Verification gives way to compute capacity. Compute capacity gives way to coordination, integration, and system coherence. The bottleneck keeps moving, and the problems become increasingly architectural.
The interesting thing is that we already possess much of the knowledge needed to address these problems. Reconciliation loops, trunk-based development, canary deployments, backpressure, and failure recovery aren't new inventions. What's changing is where that knowledge needs to be applied. Rather than relying on people to perform these activities around the development process, we need to encode more of that discipline into the systems doing the work.
That requires us to reconsider what engineering responsibility looks like. The technical depth we've accumulated remains enormously valuable, but we need to apply it differently. Architecture, verification, and operational control can no longer be practices that surround implementation. They increasingly need to define the environment in which implementation happens. Human judgment determines the boundaries, acceptable risks, and evidence required for the system to proceed, while the machinery handles the execution.
There's also a shift in how we think about knowledge. Traditional engineering teams rely heavily on what experienced people carry in their heads: why a decision was made, which components are fragile, what caused the last incident, and which changes shouldn't happen together. At machine speed, that knowledge needs to become durable and executable. Architectural decisions become constraints, lessons from incidents become automated checks, and operational experience becomes recovery behavior. The architect's job increasingly becomes turning engineering judgment into something the system can consistently apply.
This is the capability gap we need to address, and another workshop on coding agents won't close it. Experienced engineers need opportunities to practice directing execution, defining constraints, establishing evidence, and correcting systems rather than supervising every implementation. Our delivery practices, expectations, and measures of engineering effectiveness need to support that change. We cannot promise a fundamentally different approach to software delivery while continuing to organize the work as though nothing has changed.
And none of this requires building a software factory. The transition can begin with the tools we already have. The factory simply makes the limitations of our existing habits impossible to ignore.
In a room full of builders
The more I work this way, the more I find myself asking the same question. In a room full of builders, where are the architects?
Not the architects who produce diagrams and hand specifications to development teams. I'm talking about engineers who understand the shape of a system, establish the boundaries within which autonomous work can happen, evaluate the evidence it produces, and remain accountable for the result. People who can direct execution without needing to personally supervise every step.
For decades, we built our engineering practices around the human effort required to produce software. That assumption is beginning to fall away, but the profession hasn't fully adjusted to what comes next. This isn't a question for the next generation of developers. It's a question for all of us, particularly those who have spent our careers defining what good engineering looks like.
We've spent enormous effort teaching AI how to build software. Now we need to reconsider what it means to engineer when AI does the building.
The tools are changing how software is produced. It's time we changed how we practice engineering.

https://aibuddy.software/in-a-room-full-of-builders-where-are-the-architects/