What customizations can be applied to the crawler?

You can setup a variety of rules for crawling your site.

From your Console, you can choose what domains are stored in your collection, and if crawling is active for each domain.

You can also create exclude rules based on URL structure, domain, and a variety of metadata.

Exclude rules will remove all matching records from your collection, and Sajari will not re-crawl any records that match in the future.

How to hide a field in a search interface?

Background

When you generate an interface via console for a Site Search collection, we return title, description, URL, and image(optional) in the search results. In some instances, you might want to hide title, description, or URL.

Limitation

Our default interface uses URL field for click-tracking, and it must be returned in response, otherwise, the click-tracking won’t function. Hence, if you try to remove URL field, it will return an error:


tracking field 'url' missing from result.

Instructions

To hide ‘title’ or the ‘description’ field from the search interface:

  1. Generate an interface from the Integrate section in the console.
  2. After choosing the relevant options, and generating an interface, click on “View code“
  3. Add the “fields” parameter in the values object. See example below which will only return and render ‘title’ and ‘URL’:

values: {"q.override": true, "resultsPerPage": "10","q": getUrlParam("q"), "fields":'title,url'}