Multi Site scraper
wpBots Support – The best crawlers for WordPress › Forums › SCRAPER (pre-sales) › General enquiry › Multi Site scraper
Tagged: star rating, multisite
- This topic has 5 replies, 2 voices, and was last updated 4 years, 6 months ago by Suman M..
-
AuthorPosts
-
May 1, 2020 at 6:16 am #1975lex van DommelenParticipant
Hi,
I’m really excited about this product and can’t wait to start. Two questions though.
1. I collect reviews to create a Metascore. But I just collect specific reviews of a product. Is there a way in scraper to collect bulk reviews from multiple sites?
For example:
I have a new game I want to collect reviews of, it’s called ‘Halo Combat Evolved’. This game is reviewed on multiple sites. I google the URL’s, past them in the bulk scraper and get the review content from these websites.
https://www.windowscentral.com/halo-combat-evolved-anniversary-pc-review-quality-port-notable-problems
https://www.pcgamer.com/halo-combat-evolved-anniversary-review/
https://www.ign.com/articles/2011/11/14/halo-combat-evolved-anniversary-review
https://www.pcgamesn.com/halo-the-master-chief-collection/halo-combat-evolved-review
https://www.eurogamer.net/articles/2011-11-14-halo-combat-evolved-anniversary-review-review2. Some websites use star ratings, how would you collect them. See example:
https://www.gamesradar.com/halo-combat-evolved-anniversary-review/
May 1, 2020 at 6:43 am #1991lex van DommelenParticipantThank you for approving this post!
A way for 2. is getting the <script type=”application/ld+json”> (xpath: //script[@type=”application/ld+json”]) and get the value of ‘reviewRating.ratingValue’ and use ‘bestRating’ so it can match my site’s rating. Only question is how to scrape the right application/ld+json (some sites have multiple application/ld+json) to get the ‘ratingValue’ key from the ‘reviewRating’ json object.
May 1, 2020 at 7:50 am #1992Suman M.KeymasterHi,
1) You can scrape the URLs in bulk but these should have same HTML structure (basically it’s for scraping from different urls of same website) – https://support.wpbots.net/documentation/scraping-urls-in-bulk/
In your case as it’s different websites, the better option would be to create separate task for each. You may also use shortcode feature – https://support.wpbots.net/documentation/creating-dynamic-wordpress-shortcode/2) Unfortunately, you cannot scrape information from images like these. Also, I checked the site’s page source and there is no text representation of the rating.
Regards!
May 1, 2020 at 8:07 am #1995lex van DommelenParticipant1) Ok, would be a nice feature to auto detect the url’s inputed and scrape based in templates.
2) Yup I saw this too. That’s ashame. Is there a way to scrape the <script type=”application/ld+json”>? I tested the demo but didn’t work
May 1, 2020 at 8:18 am #1996lex van DommelenParticipantBy the way, 1 can be solved if scraping for <script type=”application/ld+json”> can be done and using that Json object to fill out the fields.
May 1, 2020 at 11:30 am #1998Suman M.KeymasterYou can scrape the site/content loaded via Ajax/Script using Scraper Pro option. You can check it in the demo – https://scraper.site/visual-editor/?purchase_code=demo-account
( https://www.screencast.com/t/eeLvVHIw5I ) -
AuthorPosts
- You must be logged in to reply to this topic.