Harvest is the crawler we have developped in order to feed our visual search engine with data : RSS feeds, XML files or HTML sites.
Its state-of-the-art distributed architecture makes it both fast and scalable. Importing 1M items with Harvest takes less than an hour !
Thanks to a great auto-scaling feature it is able to optimize spare resources such as men and money. No kidding !
Last but not least it comes with a fancy web based User Interface which makes it very easy to administrate.
