Sajari does not fully crawl third party sites until our pingback code is installed. Please don't add "amazon.com", "google.com", or other domains that you do not own as your domains to be crawled - we do not fully crawl third party domains without authentication via our snippet.
You can also check if our crawler is having issues with your site by using our website debug tool. If our crawler encounters errors or if you have redirects and canonicals set that are causing redirect loops we may abandon crawling.
Commons errors we see:
- Canonical tags that are all set to your homepage - in this case we may only ever end up indexing your homepage. If you are unable to set your canonical tags to the correct page then they should be left blank.
- Path-Relative URLs - we do not follow path-relative URLs (i.e. if there is no base tag on the page <a href="example-page">). Path-relative URLs can cause redirect loops. Please make sure you use Root-Relative (e.g. <a href="/example-page">) or Absolute URLs (e.g. <a href="https://www.acme.com/example-page">) on your site. For further reading on URL types please see here.
If you have checked the above and are still having issues, please contact us for assistance.