— Curated insights from the Marketo field —

Monday, May 4, 2026 Marketo Ops Radar Curated insights from the Marketo field
AI & Automation

Onboarding AI Agents: A 'Human-in-the-Loop' Strategy

April 30, 2026  · Juan Pablo Garcia  · 7 min read
Onboarding AI Agents: A 'Human-in-the-Loop' Strategy

Original source: Adobe Marketo Engage User Groups
This article is an editorial summary and interpretation of that content. The ideas belong to the original authors; the selection and writing are by Marketo Ops Radar.

This video from Adobe Marketo Engage User Groups covered a lot of ground. 8 segments stood out as worth your time. Everything below links directly to the timestamp in the original video.

Are you considering integrating AI agents into your Marketo operations? Learn how to strategically onboard them to maximize trust and effectiveness.

Onboarding AI Agents: A 'Human-in-the-Loop' Strategy

When integrating AI agents, treat them like new team members, beginning with human oversight at every step. This 'human-in-the-loop' approach builds confidence in the AI's capabilities before gradually transitioning to automated guardrails, ensuring reliability and accuracy. One practical method involves using a 'shadow mode' to run processes in a simulated environment, verifying outputs before live implementation. This iterative testing helps refine AI operations and minimizes potential errors in production.

"I would set up human in the loop step at every possible step until you build confidence. Run a whole campaign, but it doesn't send to anyone. It just sends to me."

▶ Watch this segment — 23:48

Architecting AI Agents for Scalability and Control

AI agents operate in distinct modes: 'human-in-the-loop' for intricate program builds requiring oversight, and 'human-on-the-loop' for scalable, automated tasks like callable agents. The choice depends on the specific use case and desired level of autonomy, balancing precision with efficiency. Marketing operations teams must architect instances with robust evaluation steps and clear escalation paths. This ensures that unexpected results from autonomous agents are flagged, allowing for administrator intervention and continuous optimization of AI-driven processes.

"If I go and call this agent, I'm expecting one of these 12 values to come back. If the next flow step is if the value is not this, then call out and say, 'hey, something's gone wrong here and flag that to your admin.'"

▶ Watch this segment — 24:51

Empowering Non-Marketo Users with Co-pilot for Data Access

The Marketo Co-pilot (MCP) extends Marketo's utility beyond core users, enabling non-Marketo practitioners like event managers or sales teams to access crucial data via conversational interfaces. This significantly reduces the need for manual reporting requests, democratizing data access. By leveraging both public and private APIs, MCP facilitates complex queries for historical lead data, account-level activity, and even assists in building smart lists and flow steps. This transformative capability empowers a broader audience to self-serve information, enhancing operational efficiency across the organization.

"An event manager wanting to know their attendance status should just be something you could kind of ask in the other one. Find out what is the history of this person or even maybe something that's been quite a challenge in the past of looking at an account level."

▶ Watch this segment — 36:28

Strengthening Documentation for Effective AI Agent Performance

The advent of AI agents elevates the importance of a robust center of excellence and meticulous documentation within Marketo operations. Providing detailed description fields and clear instructions offers AI agents the necessary context to perform tasks accurately and align with organizational workflows. AI agents understand Marketo's functionalities but require explicit guidance on specific team processes. Well-documented procedures, including detailed naming conventions and instructions, are crucial for achieving optimal results and preventing errors, much like onboarding a new human team member.

"The AI can the agents can read that and then they you're giving them more context to work from. The agents know how to work with Marketo, they don't know how you work with Marketo."

▶ Watch this segment — 44:47

Preparing for AI Automation: Identify and Document Key Tasks

To effectively integrate AI into Marketo operations, practitioners should start by identifying routine, time-consuming tasks that AI could automate. Common examples include list imports, data normalization, and frequently requested reporting queries, freeing up valuable human resources. Preparation involves thoroughly documenting prescriptive, step-by-step instructions for these tasks. This detailed guidance provides the necessary framework for AI agents to perform accurately, transforming complex manual processes into streamlined, automated workflows and improving overall operational efficiency.

"Thinking through what you do day-to-day. Could that be something that's replaced? Okay, we already know list imports is off my table now. If it's not prescriptive to like a step-by-step detail, that would be something that's probably not wasted effort to start preparing."

▶ Watch this segment — 58:28

Revolutionizing List Imports and QA with AI Agents

AI agents are set to transform tedious Marketo tasks like list imports by automating data structuring and normalization, resolving common pain points associated with manual data handling. This includes ensuring data integrity and correct formatting, such as country ISO values, significantly reducing effort and errors. The introduction of an AI-powered QA agent will also enhance confidence in email sends, verifying links and content automatically. These advancements promise substantial time savings and a reduction in post-send damage control, addressing long-standing operational challenges for marketing teams.

"The two that I'm super excited to see come to life are definitely one for list imports. That's always a painful job. The other one is the QA one there, just as another level of confidence that will come and support."

▶ Watch this segment — 13:22

Expanding Marketo Access with AI-Powered Co-pilot Features

AI features, such as the Marketo Co-pilot (MCP), provide alternative access points for non-Marketo users, enabling broader engagement with marketing data. Field marketers, for example, can quickly check event attendance via a chat interface, bypassing the need for direct Marketo logins or manual reporting requests. For key Marketo users, AI streamlines complex tasks like list imports, introducing safety nets and accelerating processes. This dual benefit—democratizing data access for casual users and enhancing efficiency for power users—underscores AI's potential to broaden Marketo's utility across an organization.

"There's also the user population that are less in Marketo or maybe not even in at all. Something like that with the NCP and accessing it through a different interface that they use daily for other things."

▶ Watch this segment — 19:53

Building Trust and Control in AI-Driven Operations

Achieving trust in AI results requires a balance of fine-tuning and robust control mechanisms. A practitioner emphasized that full autonomy for AI without human intervention is not advisable, stressing the critical need for embedded evaluation steps and clear escalation paths. Ongoing optimization of AI systems, including continuous refinement of guardrails and inputs, is anticipated to become a core responsibility for marketing operations teams. This proactive approach ensures that AI outputs remain aligned with expectations and mitigate potential issues before they impact downstream processes.

"I don't think I'd ever fully trust it. I would always want to put in that kind of the evaluation step so that has that traceability or at least some escalation path saying if a value comes back that is not expected."

▶ Watch this segment — 27:20

Also mentioned in this video

Summarised from Adobe Marketo Engage User Groups · 1:02:09. All credit belongs to the original creators. Marketo Ops Radar summarises publicly available video content.

AI & Automation AI Implementation Human-in-the-Loop AI Testing Shadow Mode Operational Confidence AI Architecture Human-on-the-Loop Evaluation Steps Escalation Paths Scalable AI Josh Arrington Adobe Marketo Engage User Groups
Share

More from