Knowledge Sources

Index PDFs, pasted notes, and GitHub Markdown/MDX files so agents can retrieve cited document context during chat and execution.

Knowledge sources are document-RAG data sources for unstructured material. They let agents search indexed documents, retrieve the most relevant chunks, and answer with source-backed context instead of relying only on prompt text or model memory.

Knowledge sources live under Data Sources in Studio with a knowledge label. Use them when the agent needs reusable reference material such as product docs, onboarding notes, policies, runbooks, research PDFs, or GitHub-hosted Markdown.

Loading screenshot

What You Can Import

Source type	Use it for	Notes
GitHub repository	Documentation, runbooks, specs, and Markdown content stored in a repo	Imports .md, .mdx, and .markdown files. Private repositories require a GitHub credential.
PDF upload	Decks, policies, reports, contracts, and exported documents	The browser uploads directly to private object storage with a short-lived signed URL, then PromptRails extracts and indexes the text.
Paste	Quick notes, copied docs, policy fragments, and manually curated context	Paste Markdown or plain text directly into the source.

GitHub Repository Import

GitHub import is designed for documentation-style repositories. Enter owner/repo or the full GitHub URL, then optionally narrow the import with a branch and path.

For private repositories, add or select a GitHub credential. The token must have repository access and Contents: Read-only permission. GitHub describes this permission as read access to code and metadata.

Technical details/GitHub import behavior

PromptRails:

1. Lists Markdown-compatible files from the selected repository, branch, and folder.
2. Creates a document record for each .md, .mdx, or .markdown file.
3. Fetches file content with the selected GitHub credential when the repository is private.
4. Splits the content into markdown-aware chunks with heading-path citations.
5. Embeds each chunk for semantic retrieval.

Use the advanced import options when you only want a specific branch or folder, such as docs, content/docs, or handbook.

Website Shadow Markdown

Some website pages are implemented as TSX marketing pages instead of Markdown or MDX. If those pages should be available to a repository-based knowledge source, add a markdown shadow file under content/knowledge-docs.

Each shadow file should include knowledge_source_only: true in frontmatter and a source_url pointing to the canonical page. That flag marks the file as documentation ingestion material only; it should not be added to the public docs navigation or treated as a routed docs page.

PDF Uploads

PDF uploads use a two-step flow:

1. PromptRails creates a short-lived signed upload URL.
2. The browser uploads the file directly to private object storage.
3. PromptRails confirms the document and starts background ingestion.

During ingestion, PromptRails extracts text, chunks the content, embeds it, and marks the document as ready. Failed documents stay visible with an error so you can retry or delete them.

Pasted Markdown and Text

Paste mode is useful when the source material is small, changing quickly, or not stored anywhere yet. Add an optional title, paste Markdown or plain text, and PromptRails indexes it like any other document.

Retrieval Preview

Use the retrieval preview before attaching a source to an agent. Ask a question and PromptRails searches ready chunks from that knowledge source. This helps confirm that the documents are indexed, the chunks are meaningful, and the expected passages are retrievable.

Attach Knowledge to an Agent

After documents are ready, attach the knowledge source from the agent's Knowledge tab. During execution, PromptRails exposes a built-in knowledge_search tool for the agent. The tool searches attached sources, returns matching passages, and records retrieval in the trace.

This keeps the source reusable: one knowledge source can be attached to multiple agents, and one agent can use multiple knowledge sources.

Technical details/Runtime retrieval flow

At runtime:

1. The agent receives the user's message or execution input.
2. The model can call the built-in knowledge_search tool.
3. PromptRails performs semantic search over active, attached knowledge sources.
4. Matching chunks are returned with document titles and source paths.
5. The model uses those passages to answer.
6. The tool call and retrieved context appear in the trace for inspection.

Document Health and Resync

The document list shows ingestion status, chunk count, and errors. Use it to:

• Search or filter long document lists
• Retry failed documents
• Delete documents that should no longer be used
• Resync GitHub-imported sources after repository content changes

Keep source names clear enough for agent owners to understand what they are attaching. For example, use Product docs, Support handbook, or GitHub station docs instead of a raw repository ID.

Related Topics

• Data Sources -- Structured SQL and static-file sources
• Credentials -- GitHub and storage-related credential setup
• Agents -- Attaching knowledge sources to agents
• Tracing -- Inspecting retrieval calls and returned passages