To add AI to an existing business application, start with one workflow where people spend time reading, writing, classifying or searching, then add a small server-side service that calls a model with only the data that workflow needs, validates what comes back, and lets a person stay in control of anything important. You rarely need to rebuild the application. The hard parts are not the API call; they are choosing the right use case, handling data and permissions properly, and planning for the times the model is wrong.
Key takeaways
- Start from a costly, repetitive task. Not from “we should use AI somewhere”.
- Keep AI calls on the server. Behind your existing authentication, permissions and logging.
- Validate every output. Treat model responses as untrusted input to your system.
- Budget for running costs. Usage-based pricing grows with adoption.
Choose a use case that is worth building
Good first AI features share three traits: the task is frequent, it involves language or unstructured content, and a mistake is cheap to catch. Examples that usually fit:
- Drafting replies, summaries or reports that a person reviews before sending.
- Classifying and routing incoming emails, tickets or documents.
- Extracting fields from invoices, forms or contracts into your existing data model.
- Answering questions from your own documentation or policies (see our guide to RAG assistants).
- Searching records by meaning rather than exact keywords.
Before building, write down how the task is done today, how long it takes, and what “good enough” looks like. That becomes your test set and your business case.
When AI is the wrong tool
| Situation | Better option |
|---|---|
| Rules are fixed and known | Ordinary code or a rules engine |
| Exact numbers or calculations | Deterministic code, perhaps with AI to explain the result |
| Every answer must be correct, with no review | Redesign the workflow so a person approves |
| Searching a small, well-structured dataset | Database queries and filters |
| Moving data between systems | A normal API integration |
A lot of “AI projects” turn out to be integration or workflow projects. If that is your situation, our guide to replacing manual data entry with API integrations may be the better starting point.
How the integration usually works
The pattern that fits most existing applications looks like this:
- The user triggers an action in your existing interface, such as “summarise this ticket”.
- Your backend checks the user’s permissions exactly as it would for any other request.
- It gathers only the data needed for this task, from your own database or APIs.
- It calls the model provider’s API from the server, with your key kept in server-side configuration.
- It validates the response, ideally structured output checked against a schema.
- It stores or displays the result, often as a suggestion the user can edit or approve.
For tasks that take more than a few seconds, run them as background jobs and notify the user when they finish, rather than holding a web request open while the model responds.
Choosing a model
You do not need to pick the most capable model for every task. Start with a strong general model to prove the feature works, build your test set of real examples, then try smaller and cheaper models against the same tests. Classification and extraction often run well on smaller models; long reasoning or writing tasks may need a larger one. Providers publish model comparisons covering capability, context size and limits, and these change often, so revisit the choice every few months.
Keep the model call behind one small module in your code. That makes switching models or providers a contained change rather than a rewrite, and it gives you one place to add logging, limits and fallbacks.
Design the interface around review
Most AI features work best as suggestions, not decisions. Show the output where the user already works, make it easy to edit, accept or reject, and show what the suggestion was based on where you can. Record whether users accept or change the output; that feedback is the most honest measure of quality you will get, and it shows you which examples to add to your test set.
Data access and permissions
The AI feature must never see more than the current user is allowed to see. The easiest way to guarantee this is to fetch data through the same permission-checked code paths your application already uses, rather than giving the AI service broad database access. This matters even more for agents that take actions: give them the narrowest permissions possible, and require confirmation for anything that changes or deletes data.
Privacy and contracts
- Read the provider’s terms on data retention and whether your data is used for training.
- Send the minimum: strip or mask personal data the task does not need.
- Update your privacy notice and customer contracts to cover the processing. In the UK, the ICO’s guidance on the right to be informed explains what people must be told.
- For regulated sectors, check whether data must stay in a particular region.
Security risks specific to AI features
The OWASP Top 10 for LLM and Generative AI Applications lists prompt injection first: instructions hidden in content the model reads, such as an email or uploaded document, can try to change its behaviour. Defences include separating instructions from user content, limiting what the model can do, never letting model output execute directly as code or queries, and treating every output as untrusted input. Our AI security checklist includes these checks.
Error handling and fallbacks
AI APIs time out, hit rate limits and occasionally return malformed output. Model providers apply rate limits that depend on your usage tier. Design for this from the start:
- Set timeouts and retry with backoff for transient failures.
- Validate structured output and retry or fall back when it does not match.
- Keep the non-AI path working, so the workflow continues if the AI feature is unavailable.
- Show users clearly when content is AI-generated and may need checking.
Cost control and monitoring
Track cost per feature and per customer from day one. Set monthly budgets and per-user limits, cache results that do not change, and send the smallest context that does the job. Then measure quality: keep a set of real examples with known good answers and rerun them whenever you change the prompt or model, so improvements are evidence-based rather than anecdotal.
A realistic first project
| Stage | What happens |
|---|---|
| Define | Pick one use case, collect 30 to 100 real examples, agree what good output looks like |
| Prototype | Test prompts and models against the examples, outside the product |
| Integrate | Build the server-side call, permissions, validation, logging and the review interface |
| Harden | Timeouts, fallbacks, limits, prompt injection checks and cost tracking |
| Release | Internal users first, then a small customer group, measuring acceptance and cost |
Keeping the first project to a single, well-defined use case is what makes it shippable in weeks rather than months. A second use case is far easier once the plumbing, monitoring and review patterns exist.
Roll it out gradually
Release to internal users first, then a small group of customers, collecting feedback on each output. Expand when quality and cost look right. If your use case is answering questions from documents, our detailed guide on building an AI knowledge assistant from your documents covers retrieval, citations and evaluation.