Firms point a retrieval system at their own documents so a question returns a passage with its source attached, instead of a colleague’s memory. It works when three things are solved first: permissions, whose information it is, and provenance.
Key takeaways
Every professional firm has the same quiet problem. The answer to the question a junior is about to spend four hours on already exists — in a memo written in 2023, in a precedent on a matter nobody remembers, in a deliverable for a client in another office. The knowledge is not missing. It is unfindable, and the only reliable retrieval mechanism is asking the one person who was there.
That mechanism does not scale, does not survive departures, and produces a firm where the same work is done repeatedly at full cost. Retrieval over the firm’s own material is the direct fix, and it is the most defensible internal use of AI in a professional practice, because the source of every answer is a document the firm already owns.
Documents are indexed. A question is matched against that index. The relevant passages are retrieved, and a model composes an answer from those passages, with a link to each one. The load-bearing word is “from”: a system that answers from the model’s general knowledge rather than from your documents is a different product with a different risk profile, and the difference is visible in one test — ask it something your firm has never written about and see whether it says so.
A firm’s document store is not uniformly readable, and the reasons are professional rather than administrative. Conflicts arrangements, confidential searches, matters where a team has been walled off, personnel files, partnership material. An index built by pointing a crawler at the shared drive flattens all of that, and the search tool becomes the fastest route around a wall the firm built deliberately.
In an Ontario law firm the constraints are explicit: the Rules of Professional Conduct set the duty of confidentiality at rule 3.3 and the conflicts rules at 3.4, and a lawyer must assume complete professional responsibility for the practice and directly supervise assigned work under rule 6.1-1. Accountants, engineers and consultants have their own equivalents, and a firm elsewhere in Canada should read its own regulator’s code rather than this one. The design rule is the same everywhere: the index mirrors existing permissions, or it is not deployed.
Practically, that means access is enforced at retrieval, not by prompt instruction. “Do not show documents from walled matters” is not a control. A permissions model that never retrieves them is.
Most of what a professional firm holds belongs to a client, and some of it is personal information about identifiable people. PIPEDA governs the second category, and the definition of personal information is wider than most firms assume.
The Office of the Privacy Commissioner’s principles for generative AI are the operating spec. Know and document your legal authority for the use. Limit collection, use and disclosure to what the identified purpose requires, and avoid function creep — material gathered to deliver one engagement is not automatically available to answer questions on another. Be open about what is collected and why. Keep accountability internal, with defined roles and an assessment of the impact before deployment rather than after.
Contractually, the confidentiality obligation you owe each client is the boundary. It is worth being clear on how a confidentiality clause differs from a standalone NDA and on what a breach exposes you to before deciding whether client work product may be indexed for firm-wide retrieval. For many firms the honest answer is that some engagements can be and some cannot, and the index has to know the difference.
An answer without a source cannot be checked, and in a professional firm an unchecked answer is the thing you are being paid to prevent. Require a citation on every answer — document name, date, matter reference, and the passage. That single requirement converts the tool from an oracle into a search engine that writes well, which is what you actually want.
Staleness is the failure that catches firms out. A precedent superseded by a rule change still sits in the index, still matches the question, and still answers confidently. Three defences: date every document and surface the date with the answer; mark superseded material explicitly rather than deleting it, so the system can say so; and give a named person responsibility for reviewing the top queries each quarter. Retention is the same conversation from the other direction — material you were obliged to destroy should not be answering questions.
A short specification, agreed before anything is built. Scope: which repositories, and which are excluded and why. Permissions: mirrored from the source systems and enforced at retrieval. Citation: every answer names its sources or declines. Freshness: dates surfaced, superseded material flagged. Owner: a named partner or manager, not a committee. Review: a quarterly look at the most-asked questions and the worst answers.
Notice that none of those six is a model choice. The model is the least consequential decision in the project.
A 40-person consultancy with offices in Toronto and Calgary indexes three things: completed client deliverables older than 12 months, internal methodology notes, and its proposal library. It deliberately excludes live engagement folders, anything covered by a client-specific restriction, and all personnel material.
Permissions mirror the existing folder structure, so a consultant sees only what they could already open. Every answer returns with the document name, the date and the engagement code, and answers drawn from documents older than three years carry a visible age warning.
The first month’s query log is the most useful artefact the project produces. The three most common questions are all about the same internal process, which turns out not to be documented anywhere — the system found the gap by failing to answer. The firm writes the missing note, and the tool starts answering it correctly.
One partner asks for live engagement folders to be added. The answer is no, on confidentiality grounds, until the engagement terms for those clients have been read. Two clients turn out to permit it; the rest do not. That is the project working.
Index one repository you are certain you may index, require citations, and log every query for a month. The log tells you what your firm actually does not know, which is a better roadmap than any vendor demo. Pair it with a sign-off workflow if the retrieved material starts feeding client-facing drafts, and treat the adoption question as seriously as the technical one — a knowledge system nobody trusts is a knowledge system nobody uses.
Can we index client work product?
Sometimes, and it is an engagement-terms question rather than a technology one. Read what you agreed on confidentiality and permitted use, remember that the privacy guidance warns against function creep, and be prepared for the answer to differ client by client.
Does this replace a knowledge manager?
No, it exposes the need for one. Retrieval makes gaps visible — the questions the system cannot answer are the documents that were never written. Somebody has to write them.
What stops it answering from the model instead of our documents?
Configuration, and a test you run yourself: ask about something your firm has never worked on. A correctly configured retrieval system says it has nothing. One that produces a confident general answer is not doing knowledge management.
Is a firm-wide search tool a conflicts risk?
It is, if the index ignores your walls. In Ontario the confidentiality and conflicts rules sit at 3.3 and 3.4 of the Rules of Professional Conduct; every profession has an equivalent. Mirror the permissions at retrieval and the risk goes back to where it was.
A 30-minute call is enough to tell you whether AI pays for itself here.
