Start With What The Assistant Is Allowed To Know
When connecting ERP data to an AI assistant, the first decision should not be which model to use. The first decision is what role the assistant plays in the business process. Looking up order status, summarizing receivables, flagging inventory issues, and preparing a sales briefing all require different levels of access. If the project begins by treating the entire ERP database as a searchable knowledge base, the team will usually run into broad permissions, answers that cannot be traced, and unclear responsibility when something leaks or is misread.
A practical starting point is data classification that engineering can actually enforce. Separate common reference data from transactional data. Identify fields that contain personal information, commercial terms, pricing logic, internal notes, or financial exposure. Also identify data that should not be read by an AI layer at all. This classification does not have to be perfect on day one, but it must map to real controls: API scopes, table filters, field masks, query policies, and audit records. A policy document alone will not protect the system.
Choose The Data Path: Sync, Live Query, Or Hybrid
There are three common ways to connect ERP information into an AI assistant. The first is to sync selected data into a search index or vector database. This works well for product descriptions, specifications, customer notes, manuals, process documents, and other material that is read often and changes at a manageable pace. The benefit is fast retrieval and better semantic search. The cost is that the team must handle freshness, deletion, re-indexing, and permission synchronization.
The second approach is live querying through backend APIs when the assistant needs current operational state. This is usually a better fit for order status, stock availability, receivables, payables, approval status, and shipment progress. The advantage is that the ERP remains the source of truth. The downside is that the ERP and integration layer must tolerate AI-driven query patterns, including repeated lookups and broad questions from users. The API should constrain what can be queried, how many rows can be returned, and which fields can be exposed.
The third approach is a hybrid. Use an index for semantic retrieval and explanatory context, then use live tools for transactional status and permission-sensitive data. This is often the most robust enterprise pattern, but it requires explicit routing. The model should not be free to invent its own data access plan. The backend tool layer should decide which tools exist, which arguments are allowed, and what data shape can be returned to the model.
- Freshness:Highly current data usually belongs behind a live API, while slower-moving context can be indexed.
- Permission complexity:If different roles can see different columns or rows, authorization must sit inside the query path.
- Traceability:Answers tied to business decisions should point back to source records, timestamps, and query conditions.
- Load control:An assistant may turn one natural language request into several queries, so rate limits, row limits, and caching matter.
Permissions Are Not A Prompting Problem
A common integration mistake is assuming that a system prompt can enforce security. Prompts can guide behavior, but they are not a security boundary. Real authorization must happen in identity, backend services, API gateways, database filters, and response shaping. The assistant should never receive data that the user is not allowed to access. It is not enough to retrieve sensitive data and ask the model not to mention it.
Tool calls should inherit the user identity and role wherever possible. If a service account is required, the backend should re-apply user-level permissions before any ERP data reaches the model. This includes row-level rules, field-level masking, and function-level access. Responses should also be minimized. A query for open customer orders may not need contact details, payment terms, margin information, or internal collection notes. Giving the model less data is often the simplest way to reduce risk.
Prompt injection also needs attention. ERP comments, customer messages, supplier documents, and imported files can contain text that looks like instructions. Once those records enter retrieval or tool output, the model may confuse data with commands. The safer design is to label external content clearly as data, require the assistant to treat it as untrusted context, and prevent retrieved text from enabling actions such as exports, approvals, email sending, or record updates.
Build Observability And Human Review Into The Workflow
An ERP-connected AI assistant needs operational visibility from the beginning. The system should record who asked the question, which tools were called, which records were queried, which sources were cited, and what answer was returned. These logs are not only for compliance. They are essential for debugging. When a user says the assistant gave the wrong answer or exposed something unexpected, the team needs to know whether the issue came from permissions, stale sync, retrieval ranking, model reasoning, or UI presentation.
High-impact operations should remain human-confirmed. Creating a purchase order, changing credit limits, releasing shipments, sending quotes, updating customer master data, or approving exceptions should usually stop at a draft or recommendation. The assistant can gather evidence, prefill fields, explain anomalies, and prepare the next step, but the final write should pass through the existing approval flow with a clear human owner.
The goal is not to make the assistant capable of touching everything. The goal is to make it reliable inside a clearly defined boundary. Start with read-only workflows, narrow data scopes, and measurable auditability. Add write tools later, after the access model and failure modes are understood. For companies that do not already have strong integration practice, working with a team that understands ERP, cloud architecture, data governance, and AI tool orchestration can prevent expensive design mistakes.