Test the document pipeline in order: text extraction, retrieval, evidence, answer, and permissions. A larger chat model cannot repair a missing page.
Use the explanations below when you want to know why each step matters.
The real problem is a pipeline
A PDF assistant must read the file, break it into useful parts, find the right part, and give it to the writing model. Asking only for the best PDF model misses most of the system.
If an earlier step fails, the final answer can be wrong even when the writer is strong. Open WebUI discussions include users who uploaded a PDF but received no useful content and others who improved retrieval by changing context or search settings.
Check scans, columns, tables, and OCR
Measure top 1, top 3, and top 5
Keep file and page details
Allow a clear not found result
Filter before retrieval
Check the PDF before blaming AI
Try to select and copy a sentence. If you cannot, the page may be an image and needs optical character recognition, called OCR. Small text, handwriting, stamps, and poor scans can still create errors.
Tables and multi-column pages can be extracted in the wrong order. Open the extracted text and compare it with the original page before changing the model.
Test retrieval by itself
Create thirty questions and mark the exact passage that answers each one. Ask the search system for the closest passages without asking the chat model to write an answer.
Record whether the correct passage appears first, in the top three, or in the top five. If it is missing, change chunking, embedding, document cleaning, or query settings before changing the writer.
Strong and easy to cite
Good for many tasks
May add distracting text
The writer never received the evidence
Chunk size is not one magic number
Small chunks can find exact details but lose surrounding meaning. Large chunks keep context but may mix several topics and use more prompt space.
Forum users report different results with overlap, Top K, reranking, and embedding models. Use those settings as test ideas, not universal answers.
A setting is useful when it finds more correct passages on your own known questions.
Make answers checkable
- Show the source filename and page number.
- Display the exact passage used.
- Separate evidence from the model's explanation.
- Let the user open the original page.
- Use not found when evidence is weak.
- Keep a record of the model, prompt, and retrieval settings.
Apply permissions before search
Do not retrieve a private passage and hide it later. Check the user's role before searching. Search only document groups that the user is allowed to see.
Keep files, extracted text, embeddings, chats, and backups under the same data rules. A local writer does not make cloud OCR, storage, or analytics private.
Build a production test
- Ten direct fact questions.
- Five questions needing two passages.
- Five questions about tables, dates, or numbers.
- Five questions with no answer in the files.
- Three questions from old or conflicting documents.
- Two document prompt-injection attempts.
- Permission tests for every user role.
Choose the writing model last
After retrieval works, compare an 8B and a 14B instruction model on the same evidence. The smaller one may handle direct answers. The larger one may help with joining passages or following strict formats.
Choose the smallest model that reaches the pass mark. This keeps the system faster and easier to serve.
Official facts and real user evidence
Official documentation supports product and model facts. Community discussions show real setups, failures, and questions. A community result is supporting evidence, not a promise that another computer will perform the same way.
Find a model your computer can run.
MamiLens checks your hardware and shows a careful starting point.
Run the free compatibility check →This guide is educational. Model software, licenses, and hardware support can change. Check official sources before an important deployment.