> For the complete documentation index, see [llms.txt](https://docs.uptiq.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.uptiq.ai/console/knowledge/knowledge-datasets.md).
# Knowledge Datasets
Group source files from uploads, cloud drives, or a website crawl.
A **dataset** is a named grouping of source files — the page describes itself as managing *logical document groupings*. It holds the files, tracks their ingestion, and accepts more later. It does not interpret what the files say; that happens when you build a knowledge base from them.
Open it from **Knowledge → All Datasets**.
## At a glance
| Task | How | Result |
| ---------------------- | ----------------------------------------- | --------------------------------------- |
| Create a dataset | **Create Dataset** | A named grouping, with or without files |
| Add files | **Upload**, a cloud drive, or **Website** | Files attached and queued for ingestion |
| Crawl a site | **Website** → **Start crawl** | A dataset built from the crawled pages |
| Reach retiring options | **More Datasets ▾** | Legacy Dataset and RAG Data Store |
When no dataset exists yet, the page reads **No datasets found — Start by creating your first dataset**.
## Create a dataset
Select **Create Dataset** and fill in the dialog.
**Name** — required. Identifies the dataset in the list.
**Description** — optional. Worth adding when several datasets hold similar material. Note the asymmetry: a description is optional here but **required** when you create a knowledge base.
**Source Assets** — optional at creation. The dialog says so directly: *"You can add source assets later on also."* Five sources are offered:
| Source | Best for | Authorization |
| ---------------- | ---------------------------------------- | --------------- |
| **Upload** | Files on your computer | None |
| **Google Drive** | Files in Google Drive | Google OAuth |
| **One Drive** | Personal or work OneDrive files | Microsoft OAuth |
| **SharePoint** | Files in a SharePoint site you can reach | Microsoft OAuth |
| **Website** | Public pages you want crawled | None |
Cloud sources ask you to authorize Qore the first time. If your administrator manages OAuth grants, ask them before you start.
{% hint style="info" %}
Datasets accept `.pdf`, `.png`, `.jpg`, `.jpeg`, `.webp`, `.tiff`, `.bmp`, and other image formats. Each file can be up to `15 MB`, and each upload can include up to `25` files. Files over either limit are refused. Uploading from the agent canvas instead has a lower limit of 10 MB per file.
{% endhint %}
## Add a website
**Website** crawls a site and builds a dataset from what it finds, rather than you downloading pages by hand.
**Website URL** — required, for example `https://docs.example.com`.
**Advanced** holds two settings that decide how much of the site is taken. Both matter more than they look:
**Maximum pages** — how many pages to ingest at most. Default `25`, allowed `1`–`200`. The dialog is explicit about the consequence: *"Reaching this limit means only part of the site is ingested."* A crawl that hits the cap stops there, so a large site silently produces partial knowledge.
**Link depth** — how many link levels below the URL to follow. Default `2`, allowed `1`–`10`. At `1` the crawl takes only the pages linked directly from your URL.
Select **Start crawl**, or **Discard** to leave. Crawling **runs in the background and creates its own dataset** — you do not wait on the dialog, and the result arrives as a separate dataset rather than joining the one you were editing.
{% hint style="warning" %}
**Raise the page limit before you crawl, not after.** A crawl that stops at the cap gives you a knowledge base that looks complete and answers confidently from a fraction of the site. If the site is larger than the limit, set the limit to match it first.
{% endhint %}
Knowledge built from a crawl is named for its source — `Web — en.wikipedia.org — 9c1ad5f5`.
## Review files and track ingestion
Open a dataset to see its files. Each shows its name, type, size, and ingestion status. Search by file name when the list is long.
**Completed** means processing finished and the file is ready to contribute to a knowledge base. Files can also be `in-progress`, `queued`, or `failed`.
Wait for **Completed** before building. A knowledge base built from unfinished files is missing their content, and nothing warns you — it simply answers from less than you think it has.
If a file stays unprocessed longer than expected, wait a few minutes and refresh. Persistent problems are usually platform-wide rather than specific to your file; raise it with your administrator.
## Search
Dataset and file search match names and descriptions. Neither searches inside the documents. To search content, build a knowledge base and query it with **Try Knowledge**.
## Migrate older datasets
**More Datasets ▾** holds `Legacy Dataset` and `RAG Data Store`, both being retired:
> *"The Legacy and RAG Dataset options will be deprecated in the upcoming release. Migrate any files in them to the Knowledge Store."*
The notice still says *Knowledge Store*, which is the former name for Knowledge. Migrate to a dataset here.
{% hint style="danger" %}
**Support for the legacy options ends on 30 September 2026**, as stated in the product. Migrate before that date. What happens to content left in a legacy option after it is unavailable has not been confirmed — if you have files you cannot move in time, raise it with your platform administrator now rather than after the date.
{% endhint %}
{% hint style="warning" %}
**Migration is manual.** Create a new dataset, add the same source files, then stop using the older option. Nothing moves them for you.
{% endhint %}
## Notes and limitations
* **You cannot delete a dataset or an individual file.** Ask your administrator.
* **A dataset does not read your files.** Grouping them changes nothing about their content — interpretation happens when a knowledge base is built.
### Related
Title
Description
Target
Knowledge
Build queryable knowledge from these files, then attach it.
---
# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.
## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.
Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:
```
GET https://docs.uptiq.ai/console/knowledge/knowledge-datasets.md?ask=&goal=
```
`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.