Intent Router

(This essay is an early draft of a four part essay series that explains the motivation behind tiles.run (opens in a new tab). See part 1, "The New Intelligence" (opens in a new tab), part 2, "Squishy Software" (opens in a new tab), and part 3, "Hybrid AI" (opens in a new tab).)

Illustration: Maggie Appleton

I.

The software industry is witnessing a paradigm shift that will fundamentally change the nature of software development and user interaction. With the advent of large language models (LLMs) and AI agents, the cost of creating software is rapidly approaching zero, just as the internet drove the cost of creating and distributing content to zero. This shift is particularly evident in language understanding, which was once an extremely challenging task but has now become a commodity. However, the real challenge lies not in understanding user intent, but in effectively fulfilling it. As a result, we will see the rise of ephemeral, intent-driven software that focuses on fulfilling user intent efficiently rather than capturing and monetizing user attention.

In this new world, users will express their high-level goals or intents, and lightweight, disposable software will dynamically assemble itself to achieve those goals. This approach aligns closely with the concept of home-cooked software – small-scale, personalized applications created by individuals for their own use or for their immediate community. The rise of AI-powered development tools will democratize the creation of home-cooked software, making it accessible to a wider range of people, including "barefoot developers"^[1] – a term coined by Maggie Appleton to describe individuals with some technical savvy but not necessarily professional programming skills.

II.

Illustration: Maggie Appleton

The development of local-first^[2] AI technologies will further accelerate this shift by enabling software to adapt to users' needs and contexts in real-time, without the need for constant server communication. Interestingly, the concept of breaking down app silos has historical precedent. The Apple Newton, released nearly 30 years ago, featured a design that allowed entire apps to be nested within one another, with the nested app able to access properties of the surrounding app. This approach facilitated app compositionality and data sharing. Looking forward, the future architecture of intent-driven, home-cooked software will need to address the siloed nature of data across different apps by enabling cross-app data access, potentially facilitated through a new kind of operating environment that uses local AI to responsibly manage and utilize data from multiple applications while respecting user privacy and control.

For consumers, this shift will lead to a significant reduction in the cost of digital experiences and increased personalization. Users will have access to a fluid, AI-driven interface that can pull together the exact functionality they need at any given moment, saving money, reducing cognitive load, and increasing productivity. More importantly, it empowers users to create and customize their own software solutions, tailored precisely to their needs and preferences.

Businesses can leverage the AI's ability to access and interpret data across multiple apps (with user permission) to offer hyper-personalized services that truly understand and anticipate user needs. This could lead to more loyal customers and new revenue streams based on the value provided rather than attention captured. However, businesses will need to adapt to a world where users have more control and where generic, one-size-fits-all solutions may no longer suffice.

The democratization of software creation through AI aligns with the long-standing goal of end-user programming^[3], empowering all computer users to modify and create their own software tools. In this new paradigm, users won't just be consumers of pre-packaged software; they'll be active participants in shaping their digital tools. The barrier between using and creating software will blur, allowing users to naturally extend and customize their digital environments.

III.

Illustration: Jim Fan

The core idea, as articulated by industry experts like Ben Thompson of Stratechery in an interview^[4] with Daniel Gross, is that "The most important agent in AI is going to be the local agent that decides where to dispatch jobs. It doesn't need to be big, it doesn't need to be complex, but it is the linchpin and will control all the value." This local agent — which I can call an intent router — would serve as the user's primary interface to AI capabilities, understanding their needs and routing requests to the most appropriate AI models and tools. A small "reasoning core" should reside on the device to maximize user intent extraction and provide responses by separating reasoning from knowledge, reducing the reliance on large models that memorize facts for benchmarks. Instead, it leverages tools like browsers and code verifiers, emphasizing post-training inference as the key to long-term performance, as demonstrated by OpenAI's o1 (aka Strawberry) release^[5].

Steven Sinofsky, former President of the Windows Division at Microsoft, has emphasized that AI may indeed require a new OS^[6]. This new paradigm is not just about creating smarter AI; it's about making that intelligence accessible, intuitive, and seamlessly woven into existing workflows. The true challenge lies in bridging the gap between raw AI capabilities and practical, user-friendly applications. This insight underscores the importance of rethinking our approach to operating systems in the age of AI, aligning closely with the concept of an intent router as a core component of future OS design.

A critical aspect of this new paradigm is the relationship between the user's device and cloud services. In a user-centric AI ecosystem, the cloud should feel like an extension of the user's computer, not vice versa. Apple's Private Cloud Compute (PCC)^[7] exemplifies this approach by implementing a revolutionary security architecture for cloud AI processing. In the PCC model, the user's device encrypts requests directly to cryptographically certified PCC nodes, ensuring end-to-end encryption and data privacy. The intent router, as the user's personal agent, remains the authoritative source of user intent and permissions. Cloud services should respond to and extend the capabilities of the local intent router, rather than the other way around. This principle is embodied in PCC's design, which maintains strict privacy guarantees through technologies such as stateless computation, non-targetability, and verifiable transparency. By adhering to this model, even when leveraging powerful cloud-based AI models, user data remains protected and under the user's control, with cloud services enhancing rather than superseding local capabilities.

Apple's recent unveiling of their AI strategy offers an early glimpse of this paradigm, with a local LLM kernel on the device that listens to requests and decides how to handle them. This approach allows Apple to provide a unified AI experience while maintaining flexibility on the backend. As local AI models improve, more tasks can be handled on-device. Complex requests can be routed to cloud services. And third-party AI providers can be integrated when needed.

IV.

Illustration: Sequoia AI Ascent

Anthropic's introduction of Artifacts^[8] represents a significant step towards making AI a true collaborative partner in the workplace. This feature allows users to manipulate, refine, publish, share, and remix AI-generated content^[9] in real-time, fostering a collaborative environment where users can iterate on AI-created content. This concept aligns with James Addison's vision of future AI interfaces^[10] that allow users to create and consume media in whatever way feels most natural, including seamless semantic transformations^[11] between different media types and complex manipulations. By providing a middle ground between fully automated content creation and manual work, Artifacts addresses the lack of creative control in AI-generated content, enabling users to think about media at a higher level, fill in gaps, and simplify tasks without compromising their ability to express themselves.

The concept of intent-based architectures in systems highlights potential risks and challenges in adopting intents, while emphasizing their shift towards a declarative paradigm. Intents allow users to specify "what" they want to achieve rather than "how" to achieve it, offering improved flexibility and efficiency. This declarative approach enables users to outsource complex execution details to sophisticated third parties, potentially improving user experience and reducing inefficiencies. However, it's important to be aware of risks such as centralization, trust issues, and opacity in intent-based systems.

A key insight is that if an intent router serves as the backend, then goal workspaces provide an ideal frontend interface. Rather than manually organizing information in tools like Notion, users could interact with dynamic, AI-powered workspaces that automatically adapt to support their current goals and tasks. These goal workspaces would provide a more natural and fluid way of working compared to static, manually-maintained digital spaces. Similar to how ChatGPT's Code Interpreter^[12] allows users to dynamically execute code and perform data analysis within a conversational interface, these goal workspaces could integrate computational capabilities, enabling users to seamlessly process information, run simulations, or visualize data as part of their workflow, all while maintaining context and adapting to the user's evolving objectives.

V.

Illustration: Kasey Klimes

The age of software designed to capture user attention is drawing to a close. In its place, we are entering an era where software is summoned on demand to serve user intent – an era where home-cooked software, crafted by and for individuals and communities, will flourish. This shift, powered by AI and embracing principles of end-user programming and local-first design, will put more power and agency back into the hands of users.

As the traditional app concept fades, a new app store-like ecosystem centered around intelligent agents will emerge. In this ecosystem, agents define the data, while an intelligent operating environment renders interfaces and understands user intent, seamlessly routing it to the appropriate agents. We are already witnessing the gradual rise of this transition through features like Chat Participants in VSCode^[13] and GPT Store^[14]. These are just the beginning – bridges that will lead us from the world of static apps to a future where agents and environments dynamically interact to serve user goals.

As we move into this new era, we have the opportunity to build a digital world that is more responsive to individual needs, more respectful of user privacy and agency, and more aligned with the values of local communities. The end of software as we know it marks the beginning of a more empowering, personalized, and human-centered computing experience.

References

I always welcome feedback: @feynon, @feynon.bsky.social, @feynon@mastodon.social, or hey@ankeshbharti.com.

Calendar | Github | Discord | Are.na | Instagram | Goodreads | Letterboxd | IMDb | Last.fm | PayPal | Resources | Subscribe