Why are there no records in my website collection?

Our crawler may have encountered issues with your site during initial crawl. If our crawler encounters errors or if you have redirects or canonicals which cause redirect loops, we will abandon crawling.

  1. The first thing you should do is to add your website’s homepage URL to page debug tool.
  2. The page debug tool will indicate whether the website crawler was able to crawl your homepage. Read the fetch log to see if there were any redirects before the page was parsed(crawled) or any errors
  3. There are common issues we see:
  4. Canonical tags that are all set to your homepage - in this case we may only ever end up indexing your homepage. To resolve, update the canonical tags to the correct page or they should be left blank.
  5. Website homepage is in a canonical loop (i.e. homepage redirects to a different page, which redirects back to the homepage, and ends in a loop). To resolve, either remove the canonical tag from the website’s homepage or add the correct canonical tag.
  6. Using path-relative URLs instead of root-relative URLs - Path-relative URLs can cause redirect loops. The crawler does not follow path-relative URLs (i.e. if there is no base path on the page <a href="example-page">). Please make sure you use root-relative (e.g. <a href="/example-page">) or Absolute URLs (e.g. <a href="https://www.acme.com/example-page">) on your site. For further reading on URL types </a></a></a>please see here.

Notes:

  • Sajari does not fully crawl third party sites until our pingback code is installed.
  • Please don't add "amazon.com", "google.com", or other domains that you do not own as your domains to be crawled - we do not fully crawl third party domains without authentication.

How to hide a field in a search interface?

Background

When you generate an interface via console for a Site Search collection, we return title, description, URL, and image(optional) in the search results. In some instances, you might want to hide title, description, or URL.

Limitation

Our default interface uses URL field for click-tracking, and it must be returned in response, otherwise, the click-tracking won’t function. Hence, if you try to remove URL field, it will return an error:


tracking field 'url' missing from result.

Instructions

To hide ‘title’ or the ‘description’ field from the search interface:

  1. Generate an interface from the Integrate section in the console.
  2. After choosing the relevant options, and generating an interface, click on “View code“
  3. Add the “fields” parameter in the values object. See example below which will only return and render ‘title’ and ‘URL’:

values: {"q.override": true, "resultsPerPage": "10","q": getUrlParam("q"), "fields":'title,url'}