This documentation is currently only available in English.
"The first rule of web crawling is you do not harm the website. The second rule of web crawling is you do NOT harm the website. We’re supporters of the democratization of web data, but not at the expense of the website’s owners. "
A polite crawler respects robots.txt A polite crawler never degrades a website’s performance A polite crawler identifies its creator with contact information A polite crawler is not a pain in the buttocks of system administrators
What data do we scrape?
We just index metadata and link to the original source whenever possible. Our tools are aimed to get your content more visitors and readers. You can see it in action when you use the chatbot or the Browser Extension. The listed reviews are displayed as linked headings only with your organization name included.