June 6, 2020 at 10:07 pm #2248
I bought this Plugin to scrape content from Steam, where i want to get Items from the steam workshop on my site
It doesn’t work accordingly, it fails almost everytime completely but sometimes it scrapes only 1 item. It is essential for me to get this working, otherwise I really don’t need this plugin. I suspect, that there is some kind of blocking serverside by steam to prevent such content fetching.
I already tried to increase the delay per processed URL and I increased the PHP memory and time limits but it didn’t help at all.
Thank you in advance!June 7, 2020 at 6:24 am #2249
Hi, thanks for contacting us. Please try using the following XPath for Item’s Path ( https://www.screencast.com/t/tl1RP4rtaYr )
I used it and could scrape from the given site. Do let us know.June 7, 2020 at 3:15 pm #2251
Hey thanks for your quick answer,
unfortunately there must be some mistake in the syntax you gave me, i checken a few times and copied and pasted it correctly but there isn’t any item it could find with the xpath provided
it says Items count 0 and i cant advance to the Post content.June 7, 2020 at 4:33 pm #2252
Oops! Sorry for that. The forum’s text editor has changed the double quotes. Below is the correct XPath:
//div[contains(@class,"workshopItem")]//div[contains(@class,"workshopItemTitle")]/ancestor::aJune 30, 2020 at 6:22 am #2324
Sorry to say but after a few more attempts it still makes problems. It looks like the target has some anti-scraping mechanics activated. So I tried some other pages with different problems. this plugin might be usefull, but I can say that I have absoluetly no use for it in my case because all the things i want to achieve with it dont work.
Sorry but I want my pruchase to be rolled back/refundedJune 30, 2020 at 3:51 pm #2329
I tried to scrape from above source site and could fetch the content. Can you please let us know your website’s wp-admin login details so that we can check the issue there?
If it doesn’t work we’ll be happy to make the refund. Thanks for your understanding.July 1, 2020 at 8:02 pm #2335
Unfortunately I cannot give you administrative access because we have some sensitive customer data of thousands of users and this would not be allowed due to GDPR without a proper data execution contract. But can you tell be the Xpaths with wich you manged to grab the players from this leaderboard to present them on our page?
Additionally how did you manage to get the things from the steam workshop wothout being soft banned from scraping their contents? :/July 2, 2020 at 11:14 am #2344
Alright, we understand your privacy. I could scrape from https://steamcommunity.com. I have given the XPath in my previous post – https://support.wpbots.net/support/topic/scraping-for-content-is-not-working/page/3/#post-2252
Please let us know your scraper task ID so that we can check – https://www.screencast.com/t/vG5tD4XGW
July 2, 2020 at 11:54 am #2346
- This reply was modified 1 week, 3 days ago by Suman M..
As said, the steam Workshop scraping worked at first but it failed after a number of items, I will have an extensive look on both cases in a few hours when i am at home at let you know the details and problems in both cases.July 2, 2020 at 10:06 pm #2351
The Task ID is 141377c3718331d2ca882a31866a13c3July 3, 2020 at 5:52 am #2353This reply has been marked as private.July 6, 2020 at 2:17 pm #2364
Yep as said in #2324 😉 Thats the problem. So this seems not to work accordingly, therefor the only use-case that might be useful left would be to scrape these leaderboards: https://cnc.community/command-and-conquer-remastered/leaderboard/tiberian-dawn but I am not sure how to start here :/July 6, 2020 at 4:25 pm #2371This reply has been marked as private.July 7, 2020 at 5:35 am #2375
Error 404 🙁
Task ID is: 7a8822db99cb26a45e81ee0be51997ef
scraping for the complete HTML table is no option, due to very different formatting. it looks really bad :/July 7, 2020 at 1:15 pm #2376
We tried to scrape from this site and it’s not returning any data to the bots. Thus this site cannot be scraped.
You must be logged in and have valid license to reply to this topic.