Webcrawler

Envato Customers Customer Hangout
#1

Hello everyone,

I need a software/plugin to select data (company, description, company-link, logo) from different websites and add them automatically to a database.

Somethign similiar to: Scraper - Content Crawler Plugin for Wordpress

Any other similiar plugins/softwares you know?

Best regards
cowe

#2

@cowe
You can use Scraper - Content Crawler Plugin.
Then use export/import plugin (there are more very good options out there) in order to build XML or CSV/Excel. Though you did not specify what kind of database you actually need.
Let’s say this is the low cost solution…

#3

@kriszta thanks for your help.

With database I mean just simple (WP Database) or another CMS

#4

@cowe
You are welcome :yum:
And then what"s wrong with that Scraper - Content Crawler Plugin for Wordpress?
Is there anything you can not achieve with it?

#5

scrap not all the selected content but just the one which includes a keyword I look for

E.g. Website structure is:

  1. Company A - description bla bla bla
  2. Company B - description bla bla bla
  3. Company C - description bla bla bla KEYWORD

The plugin would scrapp all the content 1 - 3 cause it selects the HTML tags.

What I need is also to consider the content and select just Company C with the Keyword

#6

It strongly depends on your workflow. Working with xPath could solve your needs, it enables to choose sampling.

But it is simply not possible to tell you the exact solution without knowing your real life example.
You might even need other methods to work with…

Though I do not expect you to tell us your business idea, indeed :slight_smile:

I strongly suggest to contact the the author of the item they will guide you and dig a little deeper for you.