When an AI assistant searches a company’s documents, two design decisions shape what it finds: how the documents are divided, and what information travels with each part. Chunking handles the first question. Metadata handles the second.

Chunking: divide the source into useful pieces

A long policy or manual is often too large to retrieve as one useful search result. Chunking divides it into smaller passages. A simple method uses a fixed number of words or tokens. A structure-aware method follows headings, paragraphs, or other meaningful boundaries. The right size depends on the task: a chunk should contain enough context to answer a question without burying the answer in unrelated material.

Metadata: describe each piece

Metadata is information about a document or chunk. For a knowledge base, it may include a document title, source URL, section heading, owner, approval status, publication date, language, or access level. Those fields help the system filter results, show citations, and avoid returning material a user should not see.

How they work together

Imagine a 40-page benefits handbook. The system could split it by section, then attach the handbook title, year, section name, and permission level to every chunk. A question about leave eligibility could retrieve the relevant section and cite its source. If the handbook is replaced, the year and status metadata help remove stale passages.

  • Bad chunking can cut an answer away from its heading or exception.
  • Missing metadata can make a good passage hard to identify, filter, or cite.
  • Neither fixes an incorrect or outdated source document. Maintain the source first.

What to test

Use real questions from users. Check whether the right passage is retrieved, whether it includes the exception or qualifier, whether the citation points to the current source, and whether access rules are respected. Adjust chunk boundaries and metadata fields based on those failures rather than optimizing a single technical metric.

In an AI knowledge base, chunking decides what each search result contains; metadata tells the system how to filter, identify, and cite it. They work best when designed and tested together.