AI inside an application
Before touching any model, you need to decide where the code that talks to it will live, so that everything that comes next stays tidy and separate from the business logic.
Reading time: 3 min
The AI layer
The AI layer is the part of the application that holds all the code that talks to language models. Instead of scattering API calls across controllers or data models, they are grouped in a place of their own, with their own classes, configuration and tests.
The reason is that an LLM is a boundary with an external system that has three awkward properties: it is not deterministic (the same question can get different answers), it is paid (every call costs money) and it is slow (seconds, not milliseconds). Anything with these properties is worth isolating, just as you would isolate a payment gateway.
Having it isolated is what later lets you version prompts, test without real calls, measure costs or switch models without touching the rest of the application.
Domain service
A domain service exposes an operation with a business name, such as summariseTicket($ticket) or suggestReply($ticket), and hides the details: which model is used, with which prompt, with which parameters and how the response is interpreted.
Whoever calls it doesn't need to know there is an LLM behind it. This lets you change the prompt, add a cache or replace the model with a simple rule without modifying any other part of the code. It also makes testing easier, because you can swap the service for a simulated response.
Synchronous calls and queues
A synchronous call makes the user wait until the model responds; a queue (a background job) accepts the request, responds to the user straight away and processes it later. With models that take between 2 and 30 seconds, this decision directly affects the user experience and robustness.
Rule of thumb: if the user needs the result to carry on and the call is short, it can be synchronous. If it is long, bulk or may fail and need retrying (processing 500 tickets, indexing documents), it goes to the queue. Queues also give you automatic retries when the provider fails or rate-limits your requests.
Provider abstraction
Provider abstraction is a layer, your own or an SDK's, that unifies the way you talk to OpenAI, Anthropic, Google or an open model, so that switching providers is a configuration change rather than a rewrite.
Models change in price, quality and availability every few months, and it often makes sense to use different models for different tasks. Without this layer, every change means touching code spread across the whole application. With it, you can even make a request fall back to another provider when one goes down.
Human review by design
Designing with human review means that whatever the AI generates comes in as a proposal (a draft, a suggestion) that a person validates before it has any real effect, such as sending a reply to a customer or issuing a refund.
It is not a temporary measure until the model "is good enough": it is what makes it viable to put AI into real processes from day one, because the model's mistakes never reach production directly. As you gather data on how many proposals are accepted unchanged, you can decide where it makes sense to give it more autonomy.
Test yourself on this module
Copy this prompt and paste it into your AI (ChatGPT, Claude, Gemini…). It will give you a 20-question multiple-choice test on the module's concepts and then suggest a hands-on exercise.
Act as the examiner for Dani Pérez's "AI Engineering Guide". Examine me on the module "AI inside an application" (https://daniperez.pro/en/resources/ai-engineering-guide/ai-layer). Concepts covered by the exam: - The AI layer - Domain service - Synchronous calls and queues - Provider abstraction - Human review by design Exam: 1. 20 multiple-choice questions, each with 4 options (a, b, c, d) and a single correct answer. 2. Ask about understanding and judgement (what each thing is for and when NOT to use it), not about memorising definitions. 3. Spread the position of the correct answer evenly across a, b, c and d. 4. Ask me the questions in 4 rounds of 5. Don't give any example answer (like "1a 2b 3c 4d 5a"): I already know to answer with the letters. Don't tell me whether I got them right until I have answered all 20. 5. At the end, mark them all: for each question, my answer, the correct one and a short explanation. Give me my score out of 20 and tell me which concepts I should review. Hands-on exercise (after marking): 6. Ask me what application I have or want to build, and which language and framework I work with. If I don't have one, use this: a customer support application for an online shop, with tickets, customers, orders and a knowledge base (FAQ and return policies). In that case, focus the exercise on: designing where the AI layer lives and making a first model call wrapped behind a domain method. 7. Suggest an exercise that applies this module's concepts to that application: goal, requirements, criteria to consider it done and common mistakes to avoid. 8. Don't solve it for me. When I bring you my solution, review it against those criteria.