How to decide what an AI agent can handle by itself and what still needs a person: a guide for small teams
The short version
A practical checklist for small teams on which tasks an AI agent can run alone and which need a person, drawn from ShipStation Global and AWS examples.
In October 2026, AWS published three blog posts that together show a small team how much work it can hand to an AI agent. In one, ShipStation Global's Slack help-desk agent answers most questions by itself, but a person still confirms every new ticket. The other two cover AWS's "Trust in Depth" approach and how an agent's answers can go wrong while every dashboard looks fine. Combined, they give you a checklist to run through before you let any agent work without someone watching.
The short answer
Let an agent act alone when a mistake would be cheap, easy to spot and easy to undo, and when you can write the rule down exactly. Keep a person on anything that needs judgment, touches money or access, or can't be reversed. Then give the agent more to do only as the evidence builds.
AWS puts it this way: let the agent act on its own when its actions can be verified, and save human judgment for the decisions that need it. Having a person approve every step doesn't scale either. AWS notes that people asked to check everything stop catching the things that matter.
What a working split looks like
ShipStation Global's engineering team built Deckhand, an AI help-desk agent that works inside Slack and runs on Amazon Bedrock AgentCore, AWS's platform for hosting agents. Deckhand answers routine IT and company questions from a curated knowledge base (the set of documents it's allowed to search). It also reads screenshots of error messages and creates and updates Jira Service Management tickets.
Here are the results, per ShipStation Global and AWS. Deckhand launched in April 2026. In its first three months it handled 1,935 messages and replied to 97.5% of them. It had 1,221 conversations: 812 were handled in chat and 409 became tickets. A third of the questions came in after hours, and Deckhand answered 94% of those. Positive ratings rose from 82% in April to 96% in June, after the team fixed what people complained about.
The more useful lesson is where the team kept people in charge:
- Tickets need a click. Deckhand offers a Create Ticket button, and nothing gets filed until a person confirms.
- It knows when to stay quiet. It skips announcements, social chat and conversations between people. It also goes silent once a human helper has replied.
- Some requests are off-limits. Its instructions tell it never to ask for passwords or multi-factor authentication codes.
- Its source material is checked. When the team mined old Slack threads for answers, a thread only counted as resolved if the full, working fix was visible in the thread.
- Every answer can be rated. Each answer has thumbs-up and thumbs-down buttons. A thumbs-down asks what went wrong, and the team uses those replies to fix the instructions and the knowledge base.
The team also points out that rules written into the agent's instructions shape how it behaves, but they don't replace real access controls.
A four-question checklist
AWS's Trust in Depth approach asks four questions at different points in an agent's life. Where a rule can be stated exactly, it adds a fifth step: proving that the rule holds.
- Scope, before it runs. What is the agent for, which tools and data does it need, and what must it never do? Give it only the access the job requires. Watch for permissions that come bundled with a tool or role. AWS warns that an admin role granted for a single task can give the agent permissions across your whole environment.
- Know, when you deploy it. Who authorized the agent, and can you trace every action back to it?
- Bound, while it runs. Are its limits enforced from outside the agent? AWS's advice is to let the agent choose how to do the task but never rely on it to enforce its own limits.
- Observe, after each run. Can you reconstruct what it did and why?
AWS also separates rules into two kinds. "Only a manager can approve access" can be checked mechanically. "Keep a friendly tone" cannot. Software can enforce the first kind. For the second kind, AWS says to limit what the agent can do, check how it behaves and send anything genuinely unclear to a person.
AWS's own example is an agent that offboards a departing employee. It gets exactly six tools and no tool for deleting the account. The system won't let it share the person's mailbox until their sign-in is blocked, and it won't let it change their license until it has checked for a legal hold.
Watch for answers that look right but aren't
An agent can be wrong without anything looking broken. Before launch, AWS's Sales, Marketing, and Global Services Operations team tested one of its internal agents. The agent reported $410.08 of revenue as "$410.08 Million". It offered a revenue estimate that appeared nowhere in the data, and it presented its own calculations as sourced facts. Their monitoring stayed green throughout.
The team says this risk grows with every change to data, instructions, code or models. It grows even without changes, because the data underneath refreshes every week. In one pre-production test, a data change, a tool change and the matching instruction update landed at different times. That gap alone dropped the pass rate from 78% to 38%.
The team's fix is a "golden set": a curated list of questions with known correct answers, rerun every time something changes. Over three months, tracking failures by type moved the pass rate from 93% to a stable 96%, per AWS.
What it means for your business
You don't need AWS's tooling to use these ideas. Take a hypothetical 10-person online shop that wants an agent to handle customer emails. Here's how the checklist applies:
- Write the job in one line. For example: "Answer order-status and return-policy questions from our help docs." Then list the tools the agent gets, and leave out anything it doesn't need. If it can't issue refunds, it can't issue a wrong one.
- Sort your rules. "Refunds only within 30 days of delivery" is exact, so software can enforce it. "Calm down an angry customer" is a judgment call, so the agent hands that thread to a person.
- Put a button in front of anything you can't undo. Like Deckhand's Create Ticket button, the agent drafts the refund or the reply, and a person clicks send.
- Decide when it should stay quiet. Once a staff member has replied in a thread, the agent should step back.
- Read the thumbs-downs every week. They tell you what to fix in your docs and instructions.
- Keep a small golden set. Write down 20 or so real questions with the correct answers. Rerun them whenever you change the instructions, update the help docs or switch AI models, and compare the results.
Then give the agent more to do slowly. AWS's own advice: "Start with one agent, one workflow, and one boundary you can prove." Once a month of results shows the agent stays within its limits, you can hand it the next task.
Sources
- How ShipStation Global built an AI IT help desk on Amazon Bedrock AgentCore · Amazonhttps://aws.amazon.com/blogs/machine-learning/how-shipstation-global-built-an-ai-it-help-desk-on-amazon-bedrock-agentcore/
- Trust in Depth: Giving agents more autonomy without losing control · Amazonhttps://aws.amazon.com/blogs/machine-learning/trust-in-depth-giving-agents-more-autonomy-without-losing-control/
- Building trust in AI agents: A production-ready evaluation framework with Amazon Bedrock AgentCore · Amazonhttps://aws.amazon.com/blogs/machine-learning/building-trust-in-ai-agents-a-production-ready-evaluation-framework-with-amazon-bedrock-agentcore/
Drafted by Elle Evate, our Head of AI Content (an AI system). Reviewed and approved by a Persistent Momentum editor. How we work: /editorial-policy. Spot an error? /corrections.