Crawls external links as well.

Recommend this page to a friend!

      Email Scraper  >  All threads  >  Crawls external links as well.  >  (Un) Subscribe thread alerts  
Subject:Crawls external links as well.
Summary:Crawls links to external domains - why?
Messages:1
Author:Jason
Date:2015-08-21 17:12:51
 

  1. Crawls external links as well.   Reply   Report abuse  
Picture of Jason Jason - 2015-08-21 17:12:51
Thank you for making your code available. I've tried two websites so far (one is mine). The first website I tried has links to books on Amazon. Those links are higher up in the page, so they get crawled first, and the script starts going on a marathon crawling every page on Amazon.com.

I haven't looked too deep in the class yet, but wondered if you had any suggestion.

Thank you!

 

For more information send a message to info at phpclasses dot org.