Talking to Agents: The Missing Context

2 Sents

To get true value out of AI agents, we must stop barking one-off commands and start having stateful, contextual conversations, just like we would with a human colleague. The leverage comes from building subcontext and shared understanding over time, not from polishing a single prompt in isolation.

I've been thinking a lot recently about how we communicate with AI, and specifically how it differs from how we communicate with the people who do the jobs we are trying to have AI do.

Consider how you work with a Product Manager. You don't just bark orders. You give them data, you show them screenshots, you walk through designs, and most importantly, you have conversations. There is a back-and-forth flow of information that builds a shared understanding.

Contrast that with how we currently try to work with an AI agent. We dictate. We tell them what to do in a text box and expect a perfect result. It's a one-way street, and frankly, it's inefficient for complex tasks.

I believe we are going to need to build new tools and entirely new ways to interact with AI: specifically, Agentic AI.

To leverage AI in ways that we currently use people, we need to move beyond simple prompts. We need to work on:

Conversations: Real, stateful dialogue where the AI can ask clarifying questions and we can provide iterative feedback.
Context: Providing the full picture, not just the immediate request.
Subcontext: Understanding the nuances within an image, video, audio file, or document. The goal is to grasp intent behind a design, not merely ingest the file as text.
Images & Multi-modality: seamlessly weaving visual information into the workflow.
Timing: Knowing when to act and when to wait.
External Context: Understanding the world outside the immediate chat window: project goals, company value, or user history.

These aren't just "nice-to-have" features; they are the improvements necessary to leverage AI in ways that we're currently using people. If we want AI to act as a true partner, we need to give it the same richness of input we give our human colleagues.

This is a conversation we need to have as a society moving forward. We need to figure out how to design the interfaces of the future to support this level of collaboration.

2 Sents

I've been thinking a lot recently about how we communicate with AI, and specifically how it differs from how we communicate with the people who do the jobs we are trying to have AI do.

I believe we are going to need to build new tools and entirely new ways to interact with AI: specifically, Agentic AI.

To leverage AI in ways that we currently use people, we need to move beyond simple prompts. We need to work on:

Conversations: Real, stateful dialogue where the AI can ask clarifying questions and we can provide iterative feedback.
Context: Providing the full picture, not just the immediate request.
Subcontext: Understanding the nuances within an image, video, audio file, or document. The goal is to grasp intent behind a design, not merely ingest the file as text.
Images & Multi-modality: seamlessly weaving visual information into the workflow.
Timing: Knowing when to act and when to wait.
External Context: Understanding the world outside the immediate chat window: project goals, company value, or user history.

This is a conversation we need to have as a society moving forward. We need to figure out how to design the interfaces of the future to support this level of collaboration.