Open the Knowledge Base Indexer.
Last updated on
Jul 27, 2026
Define the starting points, scope, and processing limits for a website crawl.
The Knowledge Base Indexer begins with one or more target URLs and follows links according to the settings you define. Configure the crawl first, then use Discovery Mode to test it without uploading content or Upload Mode to add indexed content to a Knowledge Base in Acrobat collection.
Before you begin
- Install the Knowledge Base Indexer.
- Make sure the computer has an internet connection.
- Identify one or more HTTPS URLs to crawl.
Configure a website crawl
In Target URLs, enter one or more starting URLs.
Enter each URL on a separate line. URLs must begin with https://.
Set Crawl Depth.
Crawl depth determines how many levels of links the Indexer follows from the starting URLs.
- A value of 1 processes pages linked directly from the starting URLs.
- Higher values follow additional levels of links.
Set Page Limit.
Page limit restricts the maximum number of pages processed during the crawl.
- Default: 5
- Maximum: 5,000
Set Number of Workers.
Workers control how many crawlers process pages in parallel.
- Default: 4
- Recommended for most systems: 4–8, depending on the number of processor cores.
Enter an Authentication URL when the Indexer must authenticate before crawling protected content.
Public websites do not require this value.
How the crawl settings work
Target URLs |
Defines where the crawl begins. |
Enter one HTTPS URL per line. |
Crawl Depth |
Determines how many levels of links the Indexer follows. |
A value of 1 follows links directly from the starting URLs. |
Page Limit |
Restricts the maximum number of pages processed. |
Default: 5; maximum: 5,000. |
Number of Workers |
Controls how many crawlers process pages in parallel. |
Default: 4; recommended: 4–8 for most systems. |
Authentication URL |
Provides a URL the Indexer can use to authenticate before crawling protected content. |
Optional for public websites. |
Best practices
- Begin with a shallow crawl depth and a small page limit.
- Target a specific website section instead of an entire domain.
- Increase the number of workers only when the computer has sufficient resources.
- Test the configured crawl in Discovery Mode before uploading content.
Things to know
- Only HTTPS URLs are supported.
- Local network addresses are blocked as a security measure.
- Loopback addresses and certain restricted IP addresses are also rejected.
- Invalid URLs generate errors in the logs when the crawl runs.
- Higher crawl depths can include more pages than intended.
- Large page limits can increase processing time.
- Higher worker counts can improve speed but use more system resources.
- Configuring the crawl does not start processing or upload content.
Next steps
- To test the configured crawl without uploading content, see [Discover website content].
- To run the crawl and upload indexed content, see [Upload website content to a collection].
