Custom AI Agents for Business: The 7-Point Scoping Test

Custom AI agents for business are easy to demo and hard to commission. A demo needs one clean input, one cooperative API, and ten minutes without an edge case. The system you pay for has to survive incomplete records, permission changes, model updates, rate limits, angry customers, and a Monday morning volume spike.

That gap is where budgets disappear. Gartner predicts that more than 40% of agentic AI projects will be canceled by the end of 2027, citing escalating costs, unclear business value, and inadequate risk controls. The uncomfortable part is that most of those problems are visible before development starts. They are simply missing from the proposal.

I use a seven-point test before I would commission a custom agent: workflow owner, data access, approval gates, evaluation set, failure handling, run cost, and exit plan. Score each point from zero to two. A proposal needs at least 12 out of 14, with no zero on data access, approval gates, evaluation, or exit.

Use this as a scope acceptance test. It forces any AI agent developer to show what will be built, how it will be judged, who will operate it, and how you get out if the system never earns its place.

Custom AI agents for business need a scope

The market has made agent development look like a model selection exercise. Pick a model, connect a few tools, add a chat interface, and call the result custom. That is configuration. The custom part is the operating contract around the model.

An agent becomes custom when it reflects the details that are unique to your business:

• Which event starts the work
• Which records the agent may read
• Which fields it may change
• Which decisions stay with a person
• Which exceptions stop the run
• Which result counts as success
• Which team owns the system after launch

Production teams are already doing this. A 2026 study of deployed systems found that 68% of production agents execute no more than ten steps before human intervention, 70% use prompting with existing models rather than model training, and 74% depend primarily on human evaluation. The architecture that survives is usually narrower than the one in the sales deck.

The first decision is whether you need custom agent development at all.

Situation	Best starting point	Why
Fixed rules, stable inputs, no judgment	Conventional automation	Deterministic code is cheaper to test and easier to debug
Common job with a mature product category	Packaged agent	You benefit from edge cases the product has already solved
Unique workflow, proprietary data, or unusual system mix	Custom agent	The business logic and integrations are the advantage
High consequence actions across several systems	Custom agent with approval gates	Flexibility is useful, but autonomy must stay bounded
Unclear workflow with no baseline	Manual process first	You cannot automate a process nobody can explain or measure

If a standard product fits, use it. Our field guide to the best AI agents for business maps packaged agents by job and risk. Custom work should begin only when your workflow, data, or required controls create a real gap.

The 7-point custom AI agent development test

The test is deliberately small. I do not want a 70-row procurement spreadsheet that rewards the vendor with the most polished security boilerplate. I want seven answers that reveal whether the proposal describes an operable system.

Point	The proposal must answer	Evidence worth two points	Automatic concern
1. Workflow owner	Who owns the result and exceptions?	Named owner, baseline, target, exception queue	”The AI team will own it”
2. Data access	What can the agent read, write, and retain?	Field-level access map and retention rule	Shared admin credentials
3. Approval gates	Which actions require a person?	Risk tiers enforced in code	Approval mentioned only in the prompt
4. Evaluation set	What cases prove it works?	Versioned real cases and release thresholds	A few handpicked demos
5. Failure handling	How does every known failure end?	Retry limits, safe stop, escalation, rollback	”The model will ask for help”
6. Run cost	What does one successful outcome cost?	Volume model with tools, review, and support	Token price presented as total cost
7. Exit plan	What happens when either side leaves?	Export, shutdown, migration, and IP schedule	No answer until contract stage

1. Name the workflow owner

Every proposal needs one person who owns the business result. Not a committee. Not the vendor’s project manager. The owner is the person who can say whether the agent completed the work correctly and what should happen when it cannot.

For a lead routing agent, that might be the RevOps manager. For a lifecycle email agent, it might be the marketing operations lead. For an invoice exception agent, it might be the controller. The builder owns the software. The workflow owner owns the definition of correct.

I want five things stated in one paragraph:

1. The trigger that starts the workflow
2. The input the agent receives
3. The decision or action it may produce
4. The person or queue that receives exceptions
5. The metric and current baseline

“Build an AI sales agent” fails this test. “When a new demo request enters HubSpot, enrich the account, score it against the approved ICP, draft a routing recommendation, and send uncertain cases to RevOps” is a scope.

This is also where you stop a vendor from selling a strategic project you did not ask for. Your company brings the workflow, ICP, and message that already work. The agent runs them. If the engagement quietly expands into positioning, sales process design, or a strategy retainer, the scope has changed.

2. Map data access before architecture

The model is rarely the hardest integration. Identity and permissions are.

A credible proposal includes a data access matrix before it includes an architecture diagram. For each system, it should name the objects, fields, operations, credential type, retention period, and environment. “Connect to Salesforce” is not a data plan. “Read account domain and industry, write only the routing status field, use a dedicated service account, retain traces for 30 days, and use synthetic records in staging” is one.

Data question	Weak answer	Commissionable answer
What can it read?	”CRM data”	Named objects and fields, with a business reason for each
What can it change?	”Records as needed”	An allowlist of actions and fields
Which identity does it use?	A team member’s login	Dedicated service account with least privilege
What enters the model?	”Only relevant context”	A documented data flow, including subprocessors and regions
What is retained?	”Logs for debugging”	Named log fields, masking rules, retention, and deletion process
How is staging isolated?	A test tag in production	Sandbox systems or synthetic data that cannot reach customers

Respondents to NIST’s 2026 request for information agreed that AI agents introduce novel security threats that require established cybersecurity practices to be adapted. An agent can choose tools and create side effects. Access is part of the product, not an infrastructure detail for later.

I would reject any proposal that asks for a shared admin account during discovery. Least privilege is easiest before the integration exists and hardest after every tool assumes broad access.

3. Turn approval gates into code

“Human in the loop” is often a reassuring label attached to an undefined process. Who is the human? What do they see? Which action waits? How long can it wait? What happens if they do nothing?

Put actions into three risk tiers:

Tier	Example	Default control
Reversible and internal	Add a tag, draft a note, create a review task	Agent may act, with an audit trail
External or financially meaningful	Send an email, change a deal stage, approve a credit	Named person approves before execution
Irreversible or regulated	Delete data, sign terms, move money, make a legal determination	Keep outside the agent’s authority

The gate belongs in the tool layer. A prompt that says “always ask before sending” is guidance. A send function that cannot run without an approval token is a control.

Nubank offers a useful production example. Its customer assistant handles up to five automated interactions before escalating, while its broader system manages more than 2 million monthly chats and resolves 55% of tier one inquiries. I care about the bounded interaction count and explicit handoff more than the model name.

The proposal should include the approval screen, timeout behavior, escalation destination, and audit record. If the AI agent developer cannot sketch that flow during discovery, autonomy is being sold before control.

4. Build the evaluation set before the agent

An evaluation set is the buyer’s acceptance test. Without one, “it works” means the latest demo looked convincing.

Start with 50 to 100 historical cases for a narrow workflow. Label the expected result with the workflow owner. Include the boring majority and the ugly minority: missing fields, conflicting policies, mixed languages, duplicate records, tool timeouts, prompt injection, unclear intent, and requests outside scope.

Score more than the final answer:

• Task result: Did the agent complete the right job?
• Tool choice: Did it use the right system and function?
• Parameters: Did it act on the correct record with valid values?
• Policy compliance: Did it stay inside permissions and approval rules?
• Recovery: Did it stop or escalate correctly when blocked?
• Economics: Did the run finish inside the cost and latency budget?

LangChain’s 2026 survey of more than 1,300 professionals found that 57.3% had agents in production, but only 52.4% reported offline evaluations on test sets and 37.3% reported online evaluations. Nearly 89% had some observability. Teams are getting better at seeing runs than proving a release is better.

Your contract should name the release threshold before the first model call. For a low risk internal workflow, that might be 90% task success, zero unauthorized actions, and 95% correct escalation on the exception set. Those are starting points, not universal benchmarks. The workflow owner should make them stricter as consequence rises.

The same versioned evaluation set must run after any meaningful change to the prompt, model, tool, retrieval source, or policy. That is how a custom AI agent development project becomes software instead of a permanent demo session.

5. Design the failure path

The happy path is the cheapest part of the build. Ask the vendor to show a failure matrix.

Failure	Safe behavior	Evidence to request
Required field is missing	Ask once or route to review	Test case and queue screenshot
CRM returns a rate limit	Back off, retry within a cap, then stop	Retry policy and trace
Action succeeds but response times out	Check idempotency key before retry	Integration test
Retrieved sources conflict	Refuse the action and cite the conflict	Evaluation case
User attempts prompt injection	Ignore untrusted instruction and restrict tools	Adversarial test result
Model or tool cost spikes	Stop at task and daily budgets	Budget alert and kill switch
Vendor service is unavailable	Preserve work and route to manual process	Continuity runbook

Failure handling must answer three questions: can the action safely be retried, who owns the exception, and how does the normal business process continue? “Escalate to a human” is incomplete until the human has a queue, context, service level, and authority.

Quality is still the largest production barrier, cited by 32% of respondents in LangChain’s survey. A good vendor will not promise that the model never fails. It will show how failures become bounded, visible, and recoverable.

This is distinct from launch readiness. Our 25-step AI agent deployment checklist covers the controls needed after a scope is accepted and before real customers depend on it. At commissioning, I only need enough evidence to know those controls are part of the build, not an expensive phase two surprise.

6. Price the operating loop

A build quote covers the first cost. The operating loop decides whether the economics hold after launch.

DataRobot’s 2026 survey of 413 AI practitioners and leaders found that 72% had exceeded expected operating budgets, while 71% said running agents cost more than building them. Vendor research has an incentive behind it, but the procurement question is valid: what will one successful business outcome cost after launch?

Require a volume model with these lines:

Cost layer	What belongs in it	Useful unit
Models	Input, output, reasoning, embeddings	Cost per attempted run
Tools and data	Search, enrichment, voice, email, databases	Cost per attempted run
Infrastructure	Hosting, queues, storage, tracing	Monthly fixed plus variable
Human review	Approval and exception handling time	Minutes per successful outcome
Maintenance	Regression testing, prompt and integration changes	Monthly hours or managed fee
Failure waste	Retries, abandoned runs, rework	Cost per failed run
Business result	Completed task that meets acceptance criteria	Total cost per successful outcome

Architecture changes the bill. Anthropic reports that its multi-agent research system outperformed a single agent by 90.2% on work requiring parallel exploration, while warning that multi-agent systems consume tokens rapidly and add operational complexity. That is a trade, not an automatic upgrade.

Ask the vendor to price three volumes, expected, double expected, and a failure week with elevated retries. Add task and daily cost ceilings. Compare vendors on cost per accepted outcome, not model price and not day rate.

The financial baseline should also survive comparison with a person. The AI agent versus hiring framework separates execution volume from human judgment so you do not justify an agent by pretending every automated minute removes payroll.

7. Put the exit plan in the proposal

Exit is an architecture requirement. It decides where state lives, how credentials are separated, which artifacts remain portable, and whether the buyer can keep operating when the relationship ends.

The proposal should contain an exit schedule covering:

• Customer data and output exports, including format and delivery time
• Service accounts, API keys, and credential rotation
• Prompts, policies, evaluation cases, and runbooks
• Trace and audit log retention after termination
• Source code, reusable platform IP, and custom integration ownership
• Model and vendor dependencies that cannot be replaced directly
• Shutdown steps and confirmation that scheduled runs have stopped
• Transition support, hours, rates, and response time

An exit plan does not require every provider to transfer its platform. A managed agent company may retain code and infrastructure, while a bespoke development firm may assign the custom repository. Both can be valid. The proposal has to state the boundary before the buyer funds integration work.

I would give zero points if ownership is deferred to “legal” after technical discovery. Technical choices already determine portability. If the evaluation set exists only inside the vendor’s account or production credentials are mixed with its platform keys, the contract cannot easily fix the architecture later.

How to score an AI agent developer’s proposal

Score the written proposal, not the sales call. The same evidence rule sits behind our broader guide on how to evaluate a growth partner, but an agent proposal needs technical proof that a conventional engagement never touches.

Score	Meaning	What you should see
0	Missing, unsafe, or contradicted	Vague promise, broad access, no owner, or no deliverable
1	Mentioned but not testable	A reasonable approach without thresholds, artifacts, or named responsibility
2	Specific and verifiable	Deliverable, owner, threshold, evidence, and acceptance method

The maximum is 14. My decision rule is:

• 12 to 14: Commission a narrow pilot, subject to commercial and security review.
• 9 to 11: Pay for discovery only. Do not approve the production build.
• 8 or lower: The proposal is a concept, not an operating plan.
• Automatic stop: Any zero on data access, approval gates, evaluation set, or exit plan.

The automatic stops matter more than the total. A vendor can score well on workflow and cost while still asking for unsafe access. Another can show excellent security paperwork while keeping every evaluation asset in a proprietary account. A high average does not repair a critical hole.

Ask for one artifact per point. Seven artifacts are enough: workflow map, data access matrix, approval flow, evaluation plan, failure matrix, cost model, and exit schedule. If the vendor cannot produce them during a paid discovery sprint, it will not magically produce them after the build invoice.

Three custom agent builds that reveal the scope

The headline metrics tempt buyers. I look at the boundary that produced them.

avatarin and Yamada Denki: Ground the agent in owned expertise

avatarin and Yamada Holdings built a 24-hour multilingual shopping agent around product data and experienced sales associates’ knowledge. In a two-week public campaign, about 30,000 shoppers used it and 92% of survey responses were positive.

The scoping lesson is specific. The agent did not receive a generic goal to sell appliances. It used retrieval over product information, conversation flows shaped by category knowledge, and guardrails that kept the exchange focused. The implementation team also optimized API costs for an always-on voice service. Workflow, data, boundaries, and run cost were part of the same design.

Nubank: Bound the conversation before full automation

Nubank began with custom enterprise search used by more than 5,000 employees each month, then expanded into customer support. Its customer assistant now handles more than 2 million monthly chats, resolves 55% of tier one inquiries, and reduced response time by 70%.

The maximum of five automated interactions before escalation is a concrete approval and failure boundary. You can test it, monitor it, and change it. “The agent escalates when needed” gives the operator nothing to work with.

Cars24: Start with a constrained revenue workflow

Cars24 did not begin by automating an entire automotive marketplace. It started with constrained middle and bottom funnel conversations, where the agent could collect needs, recommend inventory, book visits, and reengage dropped leads. Its system now handles more than 1 million conversation minutes each month and recovers 12% of seller leads that were previously lost.

The sequence matters. One valuable workflow created the data and operational confidence to expand. That is a better commissioning model than buying a broad “AI employee” and searching for work after launch.

A 10-day commissioning sprint

Do not compare five glossy proposals for a month. Run a short evidence sprint with the two strongest AI agent vendors.

Days	Buyer work	Vendor deliverable	Decision
1 to 2	Supply process map, sample cases, baseline, and constraints	Restated workflow with open questions	Does the vendor understand the work?
3 to 4	Approve systems and action tiers	Data access matrix and approval flow	Can the design stay inside safe boundaries?
5 to 6	Label representative cases	Draft evaluation set and thresholds	Can both sides define correct?
7 to 8	Review exception history and expected volume	Failure matrix and three-volume cost model	Is the system operable and affordable?
9	State commercial model and internal capability	Exit schedule and ownership map	Can you change direction later?
10	Score all seven points	Revised proposal with seven artifacts	Pilot, discovery only, or stop

Pay for the sprint. Good discovery is real work, and paying lets you demand usable artifacts. The losing vendor should still leave you with a clearer workflow and risk map, subject to the agreed terms.

The pilot that follows should prove one workflow on real but bounded data. Set the acceptance thresholds before it begins. Compare the result against the existing manual process and include human review time. Our AI agent ROI measurement framework shows how to connect that pilot to revenue impact, cost displacement, capacity reinvestment, and risk reduction.

Proposal red flags that end the conversation

I would stop the process when I see any of these:

1. The proposal leads with a multi-agent diagram. Complexity appears before the workflow demands it.
2. Accuracy is a single percentage with no test set. A number without cases and labels is decoration.
3. The vendor needs production admin access for the pilot. The access model is already broken.
4. Human approval exists only in prompt text. The control disappears when the model ignores it.
5. The quote includes build price but no per-run model. Operating cost will arrive as a surprise.
6. Every failure becomes a human escalation. The vendor has moved exception design into your payroll.
7. Ownership and exports wait until handover. By then, the lock-in is already built.
8. Success means tasks completed. The metric ignores correctness, rework, and business value.

One more red flag is subtler: the vendor agrees with every request. A credible builder will tell you when a fixed workflow is better than an agent, when a requested action should remain human, and when the data is too weak to support the proposed automation. The first evidence of judgment is a useful no.

Commission the smallest system that can earn expansion

Custom AI agents for business should not begin as digital employees with broad job descriptions. They should begin as narrow systems with a workflow owner, limited access, explicit approval, a hard evaluation gate, designed failure paths, known unit economics, and a clean exit.

That scope may feel less ambitious than the demo. It is more ambitious operationally because it can survive real work.

At Momentum Nexus, we run AI growth agents for pipeline, CRM and RevOps, content, lifecycle email, conversion, product analytics, and retention as a managed service. We set them up, host them, watch every run, and tune them weekly. We do not hand clients a fragile workflow and ask them to operate it alone.

If you are evaluating a growth agent, use the seven artifacts in this test against our proposal too. Review the managed agent plans at Momentum Nexus AI Agents, then make us show the scope, controls, economics, and operating responsibility before you commission anything.