Web Content (Sitemap) Connector Limitations

Learn about the functional limitations of the Web Content (Sitemap) connector that affect certain behaviors and outcomes.

Some Sitemap Types Aren't Supported

Yoast sitemaps aren’t supported, see example
Some non-XML sitemaps are not supported.

Unsupported File Types Aren't Ingested

HTML and PDF pages are supported.
Supported image formats: .jpg, .jpeg, .png.
Video and audio files (.mp3, .mp4, etc.) aren’t supported.

URL Query Strings Aren't Supported

When parsing links, the connector automatically removes the entire query string (all text following the ?) from the URL. This limitation applies to all pages except the root page. As the connector removes everything that follows the query string ?, any pages included in the URL following the query string are not ingested.

Client-Side Rendering Isn't Supported

The Web Content (Sitemap) connector doesn’t support content that is dynamically rendered via JavaScript at runtime. Any content rendered in this manner isn’t ingested.

Rate Limit

The connector’s rate limit is 5 records (pages) per second. If your website doesn’t support this rate limit, the connector may fail.

Apache HttpClient Requests

The ingested site needs to allow Apache HttpClient Requests.

General Data 360 Limitations

Learn about Data 360’s general limitations which may create issues for this specific connector.