Datasets
Group source files from uploads, cloud drives, or a website crawl.
A dataset groups source files for reuse. It tracks ingestion and accepts new files. Create a knowledge base to process and query the content.
At a glance
Create a dataset
Create Dataset
A reusable source-file group
Add files
Attach Files
Files ready to sync
Crawl a site
Website → Start crawl
Website content added as one file
Replace a file
Upload new version
The next version of that file
Inside a knowledge base, Datasets shows source files, versions, and sync status. See Sync & versions.
Create a dataset
Select Create Dataset, then provide the following details:
Name is required. It identifies the dataset.
Description is optional. Use it to distinguish similar datasets.
Source Assets are optional. Add them now or later.
Add files from your computer. Each file can be up to 15 MB. Add up to 25 files at once.
Add files from Google Drive, OneDrive, or SharePoint. Google Drive uses Google OAuth. OneDrive and SharePoint use Microsoft OAuth.
Crawl a public website and add its content to the dataset.
Add files
To add files later, select Upload files to dataset for the dataset. Use Attach Files to upload, select a cloud provider, or add a website.
New files show Ready to sync. They remain outside knowledge-base answers until you sync them. With automatic sync enabled, syncing begins automatically. See Sync & versions.
Add a website
Use Website to crawl public pages without downloading them first.
Website URL
—
Valid URL
The crawl starting point
Maximum pages
25
1–200
Maximum pages ingested
Link depth
2
1–10
Link levels followed; 1 includes directly linked pages only
Select Start crawl. Crawling runs in the background. The crawled site is added as one file, so it syncs, versions, and detaches as one unit.
Set the page limit before starting. A crawl that reaches its limit contains only part of the site.
Review files and track ingestion
Each dataset lists a file's name, type, size, and ingestion status. Within a knowledge base, files also show their version and sync status. See File status.
File actions
View document
Opens the file
Upload new version
Adds the next file version under the same name, from your computer, a cloud drive, or a website. See File versions.
Version history
Lists earlier versions
View extracted content
Shows processed content. See Document processing.
Download original
Downloads the uploaded file
Detach from knowledge
Removes the file from this knowledge base only
Delete from dataset
Removes the file from the dataset
Completed
Processing finished. The file can contribute after syncing.
in-progress
Processing is underway.
queued
Processing is waiting to start.
failed
Processing did not finish.
Wait for Completed before syncing. If processing does not complete, wait a few minutes and refresh. Contact your administrator if the issue persists.
Search and discovery
Dataset and file search match names and descriptions.
Search does not inspect document content.
Use Try Knowledge to query processed content.
Files are also indexed for semantic discovery. Preparing for discovery means indexing is underway. Discoverable means indexing is complete. Discovery and sync are independent, so a discoverable file can still be Ready to sync.
Discovery searches your data sources by meaning, not by name. Use it to find which dataset holds what you need, for example which one covers this year's rate card:
On a dataset's page, use Search this dataset.
In the Knowledge Builder chat or the Agent Builder chat, ask for it in your own words.
Only Discoverable files are found. A file still Preparing for discovery doesn't appear in results yet.
Related
Maintained by Abhishek Paul
AI-assisted, human-approved
Last updated
Was this helpful?