How does the crawler handle 301 or 302 redirect?

Sajari Site Search handles HTTP redirects automatically when a page is reindexed. The redirected pages are removed from the Collection(index) when they have a 301 or 302 HTTP status code.

The destination page of the redirect gets added to the Collection and will be shown in the search results.

Note: Any page that doesn’t have any meta changes detected will be re-crawled after 3-6 days in any case. The redirected pages might still appear in the search results until the next re-crawl takes place.

How to remove redirected pages immediately from a Collection:

You can also manually trigger the crawler to remove the page if you don't want to wait by following these steps:

  1. Log in to your Sajari account
  2. Select the relevant Collection
  3. Navigate to 'Domains' and click on 'Diagnose'
  4. Enter the URL that you have removed/redirected and press "Diagnose"
  5. The result and details of the record would be returned. Press "Add to Index"

The page will be removed from the index in a few minutes, and the "State" would change to "Redirect" the next time it is diagnosed. See a screenshot below:

How to hide a field in a search interface?

Background

When you generate an interface via console for a Site Search collection, we return title, description, URL, and image(optional) in the search results. In some instances, you might want to hide title, description, or URL.

Limitation

Our default interface uses URL field for click-tracking, and it must be returned in response, otherwise, the click-tracking won’t function. Hence, if you try to remove URL field, it will return an error:


tracking field 'url' missing from result.

Instructions

To hide ‘title’ or the ‘description’ field from the search interface:

  1. Generate an interface from the Integrate section in the console.
  2. After choosing the relevant options, and generating an interface, click on “View code“
  3. Add the “fields” parameter in the values object. See example below which will only return and render ‘title’ and ‘URL’:

values: {"q.override": true, "resultsPerPage": "10","q": getUrlParam("q"), "fields":'title,url'}