On this page
Start with one document groupHow the system worksBuild the first versionTest search before testing the answerPrepare for a teamBuild the answer from evidenceWhat if my PDF is a scan?SourcesA useful document assistant needs clean files, good search, clear sources, correct permissions, and a real test set.
Build the first document collection
A private document assistant has two important jobs. It must find the correct passage and then explain that passage without making up missing facts.
- 1
Create a clean folder
Make a new folder named approved-documents. Add only current files that users are allowed to search.
- 2
Check every file
Open each PDF. Make sure text can be selected. Use OCR first when a page is only a scanned image.
- 3
Choose a local document app
Install a trusted local RAG tool that supports your engine. Confirm in its privacy settings that files and embeddings stay on your device.
- 4
Add five documents
Do not begin with hundreds. Import five clean documents and let the app create its search index.
- 5
Test known questions
Ask ten questions with known pages. Confirm the correct passage appears before judging the final AI answer.
- Answers show the file name and source passage.
- The assistant says when the answer is missing.
- Removing a user's file permission also removes it from that user's search results.
If something goes wrong
- If the wrong passage appears, fix document extraction or chunking before changing the writing model.
- If scanned pages are empty, run OCR and import the corrected file again.
Here is what matters, why it matters and how to check it on your own setup.
Start with one document group
Choose one useful group, such as staff policies, product manuals, or approved procedures. Decide who will use it and what questions it should answer.
Remove old copies, duplicate files, unreadable scans, and documents with unclear ownership.
How the system works
The system reads each document and breaks it into smaller pieces. It turns those pieces into numbers that help a search tool compare meaning.
When a user asks a question, the system finds the best pieces and sends them to the AI model. The model should answer from those pieces and show the source.
Build the first version
- Store approved files locally.
- Extract the text and check tables and scanned pages.
- Break text into useful pieces and keep each source name.
- Create a local search index.
- Send only the best passages to the model.
- Show the source and allow the answer “not found.”
Test search before testing the answer
Create questions with known source passages. First check whether the search tool finds the correct passage.
If search does not find the evidence, a bigger AI model may not fix the problem. Improve the files, text pieces, or search settings first.
Prepare for a team
- Add user accounts and role-based access.
- Protect and back up files and search indexes.
- Keep only the logs you truly need.
- Place the model and search service behind the app.
- Create rules for updating and deleting documents.
- Test again after every important change.
Build the answer from evidence
A document assistant has several jobs: read the file, split it into pieces, find useful pieces and write an answer from them. If the wrong piece reaches the writing model, a larger model may still answer incorrectly. Fix the search before changing the model.
Start with three clean documents. Ask a question with a known page. Look at the text found by the search step. Then check the final answer and citation. Add a question with no answer in the files. The assistant should explain that the information is missing.
A model download alone does not create a document assistant. You also need file reading, search, access rules and a user interface.
What if my PDF is a scan?
Try to select a sentence. If the page is only an image, you need text recognition. Check that names, dates and table values were read correctly before indexing it.
Words you may see in this guide
- RAM
- The working memory used by your computer and apps.
- VRAM
- The memory on a dedicated graphics card. Shared graphics may use system RAM.
- Context
- The text and other input the model can use for its current answer.
- Quantization
- A way to make model weights smaller. It can change memory use and answer quality.
- Engine
- The software that loads the model and runs it.
- RAG
- A workflow that finds useful document passages before asking a model to answer.
- Benchmark
- A repeatable test. A result is useful only when you know the setup and task.
Official facts and real user evidence
Official documentation supports product and model facts. Community discussions show real setups, failures, and questions. A community result is supporting evidence, not a promise that another computer will perform the same way.
Watch a real test. Check what fits your computer.
See practical local AI tests from DeviceTerra, then use MamiLens to build a hardware-aware shortlist for your own setup.
Need help building your private AI system?
Request a quote for installation, a private document assistant or a small business deployment. We first check whether your hardware and task are suitable, then agree the scope and price before any paid work begins.
The MamiLens calculator, guides and launch configuration review remain free. Sending an enquiry does not commit you to buying.
Request a project quoteShare your goal and hardware details first. Do not send confidential documents in your initial email.
This guide is educational. Model software, licenses, and hardware support can change. Check official sources before an important deployment.
