How does crawling and indexing work?

Without our ping-back code installed, indexing is done by following links starting with the root of the domain.

For instance, for the crawler starts with and follows links from there. Here's some more information on how Sajari works with domains.

With the ping-back code installed, new content is indexed instantly and made searchable, rather than waiting for a periodic crawl.