Function Calling, Tool Use, and Memory: How an AI Agent Gets Work Done

I’m Maya Chen. I’m thirty-nine, a software engineering manager in Seattle, and I’ve spent years turning flashy demos into dependable systems. Deadlines land like bricks. Edge cases arrive brittle as glass. Frontline staff wear the fatigue of a thousand small fails. And somewhere behind the scenes, a decision is made that software is a product, not just a model.

This piece follows a single bounded task from request to action, showing three mechanisms in action: function calling, tool orchestration, and memory. It’s not a tutorial. It’s a briefing about how the pieces fit, where they break, and why the breaks matter for teams that ship.

A real task, concrete example Imagine a team wants to check a customer’s eligibility for a loan. The demo shows a smart assistant that can read policy rules, fetch customer data, run a risk model, and return a decision. Great. But the demo doesn’t stop there. A real system has to handle uncertain inputs, data access constraints, and the occasional system fault without tripping the entire process.

I’m not going to pretend there’s one magic switch. There isn’t. The three mechanisms, defined tools and schemas, disciplined orchestration, and memory that persists across turns, work together, and they create both capability and risk. This is how a developer makes a powerful but auditable automation.

Function definitions and argument validation In any system built for production, you start with a contract. A function, the thing the AI can ask to do, has a name, a description, and a clear schema of arguments. The schema is not decorative. It’s the guardrail that keeps the system from spinning off into nonsensical requests.

For our loan check, you’d define:

• fetch_customer_data(customer_id: string) -> CustomerRecord
• fetch_policy_rules(region: string) -> PolicySet
• run_risk_model(input: RiskInput) -> RiskScore
• decide_loan(risk_score: float, income: float, debt: float) -> Decision

Each input must be validated before the function runs. The model should not blindly trust a string or a numeric guess. It should be explicit about types and required fields. If arguments are missing or invalid, the system should fail fast with a clear error message, not later in a tangled thread of logs. This keeps the human in control and makes failures legible.

Orchestration: planning, then doing The human goal isn’t to have a clever chat. It’s to move through a sequence of reliable steps, each with a defined input and output. The agent begins with a plan that it can adjust on the fly as assumptions prove false. The plan might look like:

• Validate input payload
• Retrieve customer data
• Retrieve policy rules
• Run risk model
• Make a decision
• Log the outcome for audit

The important thing is that the model doesn’t just pick one function and pretend it’s done. It coordinates several functions, passing results from one to the next and catching failures early. If data retrieval fails, it may retry a bounded number of times or escalate. If the risk model returns unusual values, it flags them and halts the decision path until an operator reviews.

Memory: short-term context vs long-term memory Memory is not a single thing. It’s a layered approach to remember what’s happened in the current conversation (short-term context) and what’s needed for ongoing workflows (long-term state).

• Short-term context: what the model has learned in the present session. It helps prevent repeating questions, preserves the current decision point, and keeps a running summary of inputs, outputs, and decisions. It’s essential for coherence in a multi-step task.
• Long-term state: a store that persists across sessions and tasks. It records policy versions, user preferences, and task histories. Long-term memory enables reuse of previously computed results, faster onboarding of new team members, and easier audits.

The balance matters. Too much memory without governance invites stale or biased results. Too little memory makes automation brittle, forcing the system to re-derive facts it could have reused. A well-structured memory strategy keeps operations efficient and auditable.

Permissions and access control Tool use is a superpower with a price. If the agent can fetch customer data, it must also respect privacy and authorization boundaries. You implement permissions at the tool level and the data level.

• Tool permissions: only allow the agent to call a subset of functions it needs for the current task. The model should be aware of what it can and cannot do. If the action is outside its remit, it should fail in a controlled way.
• Data permissions: the plan should enforce data access policies. If a customer record is restricted, the agent should surface a compliant alternative or escalate to a human.
• Least privilege: deploy the smallest set of capabilities required for the task. The principle reduces blast radius when things go wrong.

Error handling: when the world breaks No system is perfect. The moment you allow external tools to execute actions, you invite failures. Timeouts, partial results, malformed responses, and unexpected data formats. The key is to handle these gracefully and transparently.

• Timeouts and retries: implement bounded retries with backoff. If a data fetch times out, you should retry a fixed number of times, then raise a clear, auditable error.
• Validation failures: if a tool returns data that fails business validation, stop the current path and prompt for human review or alternative flows.
• Partial results: when a multi-step task returns partial data, the system should either complete with the partial data if safe or halt and surface the gap to a human.
• Safe defaults: where appropriate, fall back to safe defaults that do not violate policy, and clearly annotate why a default was used.

Audit log: traceability that survives outages An auditable automation needs a robust trail. Every decision, every tool call, and every memory write should be logged with a time stamp and a user-visible identifier.

• Tool calls: record which function was invoked, with the exact arguments (sanitized as needed), and the results.
• Decision points: capture why a particular path was chosen, not just what was chosen. This helps future inquiries into product and management decisions.
• Data access: log who accessed what data, when, and under what permission. This is critical for security reviews.
• Memory writes: log what persisted to long-term memory and when. This makes the system auditable and reproducible.

What constitutes a scalable pattern The patterns aren’t about being clever. They’re about being predictable and auditable.

• Clear schemas: every tool exposes a strict input/output contract. The code paths that implement the tools enforce these contracts.
• Deterministic orchestration: the plan is explicit, and deviations are logged. The same inputs yield the same decisions unless policy or data changes.
• Memory discipline: memory is a deliberate, governed part of the pipeline, not a stray variable that floats around in a chat.
• Observability: metrics, traces, and logs are readily accessible to engineers. If a failure happens, you can see where it started, what data flowed through, and why it ended the way it did.

Three mechanisms in action: a bounded loan check Let me walk through the bounded task in a concrete way, not as a recipe but as a demonstration of how the pieces interact.

• The model starts with a request: “Check if this applicant qualifies for a loan under policy X in region Y.” It first validates the input and checks permissions. If the input is incomplete, it returns a clean error that the frontend can surface to the user.
• It then identifies the needed tools: fetch_customer_data, fetch_policy_rules, run_risk_model, and decide_loan. Each tool has a defined schema. The model emits a structured plan: call fetch_customer_data with customer_id, then fetch_policy_rules(region), then run_risk_model with the combined data, then decide_loan.
• As results come back, the model composes them into a coherent decision. If the risk score is outside expected ranges, it flags it and halts the auto-decision path. The audit log records every call and result, including the exact arguments passed and the outputs returned.
• The memory layer stores the session’s state: the customer record, the policy set used, the risk score, and the final decision. This memory is accessible to future steps in the same workflow and to subsequent sessions that need to reference policy versions or prior decisions.
• If something goes wrong, one tool times out, or the policy version is out of date, the system either retries with a backoff or escalates. The escalation path is explicit in the plan and appears in the audit log.

Patterns you’ll recognize, and why they matter

• Function or tool schema: the contract that makes automation legible and testable. It is the cornerstone of safe tool use.
• Argument validation: prevents cascading failures. It’s cheaper to fail fast on input than to chase a chain of bad assumptions.
• Orchestration: the pragmatic choreography of multiple tools. It’s where edge cases live and where decisions must still be auditable.
• Short- and long-term state: memory that survives a single request and memory that informs future decisions. Both are necessary for reliability and speed.
• Permissions: guardrails that keep automation honest and aligned with policy.
• Error handling: a disciplined playbook for when the world disagrees with the plan.
• Audit log: the record that proves what happened, why, and when.

What I’m not pretending This isn’t an API walkthrough. It isn’t a glossary, and it isn’t a story about one heroic model. It’s about the scaffolding that makes a demo survive a real day. The actual code, tests, and deployments aren’t the point of this piece; the point is why the scaffolding exists, how it behaves under pressure, and what risks it introduces.

Research anchors From industry documentation and analysis on agent tools and memory, a few consistent themes emerge: tools are defined with explicit schemas; memory must be purposefully partitioned and governed; and robust auditing is non-negotiable for compliance and reliability. The core idea is not to replace human judgment but to structure machine actions so they can be inspected, repeated, and improved over time. This framing helps teams move from “watch this demo” to “this is how we operate at scale”.

A pragmatic take on failures Public AI failures rarely live in the model alone. They’re product decisions, data access choices, and management tradeoffs all tangled together. If a tool returns borderline data, the system should surface a decision path that a human can review, not pretend the model is a universal oracle. The memory and audit trails don’t just log what happened; they reveal why it happened, which informs the next change to policy, data access, or the tool contract.

Closing thought: when fluent becomes action The moment a fluent answer becomes an action in someone else’s system is the moment the loop becomes real. The user sees a decision; a human sees the path to that decision; the system logs the path for future scrutiny. It’s not magic. It’s a disciplined chain of contracts, orchestrations, and memories, all designed to be observable and revisable.

After the Demo In the hours after a demo, the room changes. Edge cases move from the blackboard to real data. The audit logs grow teeth. People patch tiny leaks before they become floods. The memory store gets updated with what worked and what didn’t, so the next demo can be leaner, faster, and closer to safe production. And the team learns which decisions should stay in automation and which should stay in human hands. After the demo, the system is not perfect. It’s trackable, improvable, and, for a moment, trustworthy.

After the Demo