wpBots Support

Helpdesk for Best Scraper Plugin

Search in:
Suggested Search: installation, demo data, icons

SLgreen site images/gallery/content

wpBots Support – The best crawlers for WordPress › Forums › SCRAPER (after-sales) › Tasks Troubleshooting › SLgreen site images/gallery/content

Tagged: ,

Viewing 10 posts - 1 through 10 (of 10 total)
  • Author
    Posts
  • March 6, 2020 at 5:34 am #1588
    Blake O'RuairiBlake O’Ruairi
    Participant

    Hi all, Thanks for the product = hopefully it’ll work well for what I need.

    This site : https://slgreen.com/properties?list_view=true

    I need to run a serial that pulls ALL data from each individual entry.
    https://slgreen.com/properties/110-east-42nd-street?
    info to be pulled needs to be both the standard data in top content block plus the “Available units” block.

    I’ve tried 8 different scrape setups; yours and one other could even ‘see’ the images thru visual selector, all others couldn’t view images.

    I currently have bot installed at this work address =
    if its going to be simpler for you, here’s access:

    Really truly need this sorted out as soon as possible, folks, and Your hlep will surely get us the project.

    Best
    Blake

    • This topic was modified 4 years, 8 months ago by Suman M.Suman M..
    • This topic was modified 4 years, 8 months ago by Suman M.Suman M..
    March 6, 2020 at 5:40 am #1596
    Suman M.Suman M.
    Keymaster
    This reply has been marked as private.
    March 6, 2020 at 5:54 am #1597
    Suman M.Suman M.
    Keymaster

    I’ve created a task “slgreen.com” in your site. Please check it and let us know.

    March 6, 2020 at 2:22 pm #1598
    Blake O'RuairiBlake O’Ruairi
    Participant

    This looks absolutely perfect..I Thank You, and after all’s run and placed pulls from site I’ll be presenting later today.

    two item’s however.

    1 – scrape only seems to pull main single image, not gallery images; how can I include those gallery images, please? Or in this case, for speed, I’d ask if perhaps you could adjust
    2 – beneath each listing is an area called ‘Available units’ – I need to include that in scraped data also

    The other thing, which im sure is ME somehow lols, is that I cant seem to make a full serial scrape here, have only been able to do 2 then scrape stopped working at all.

    I increased execution time and turned off caching in that work folder area for now, while scrapes are worked. Runs fine now apart from those items I mention above~

    You guys are Truly excellent, and I’ll most assuredly be leaving that review for you. right now. Which is odne~
    Tks
    Blake

    March 6, 2020 at 6:25 pm #1599
    Suman M.Suman M.
    Keymaster

    1 – scrape only seems to pull main single image, not gallery images; how can I include those gallery images, please? Or in this case, for speed, I’d ask if perhaps you could adjust
    >> If you check the scrape preview, you’ll see that gallery images are also being scraped – https://www.screencast.com/t/fL7oCgVS. Aren’t they fetched when the content is imported to your site? Is there any specific gallery field in your site? Also, scraping and importing gallery images might need more memory in the server.

    2 – beneath each listing is an area called ‘Available units’ – I need to include that in scraped data also
    >> I checked it and is difficult to scrape this particular content as there is no unique HTML/CSS-class-name defined for it. So I have tried to fetch it as 2 different variables “availability_header” and “availability_content”. The first one scrapes the header part and second one scrapes the availability data. These are then merged into 3rd variable “available_units”.

    Note: You’ll need to add CSS for these scraped HTML codes, or else you can copy CSS code from the source site.

    March 6, 2020 at 7:04 pm #1600
    Blake O'RuairiBlake O’Ruairi
    Participant

    Hi Suman

    1 – no, gallery images aren’t been shown in the run I’ve had on site. I’m not sure at all why not.
    I also couldn’t get system to run a full scrape – this setup only allowed me to run single scrapes then stopped working.
    There’s no specific gallery field or anything like that on site, purely basic.

    Right now, system has 512mb ram to use, i really don’t see it being a memory allocation issue on server but i can increase over that if needed.
    If you’ve run a scrape that pulled all gallery images then it should be displaying.

    2 – thats perfect, exactly what would be needed.

    I have been away for a few hours and have not looked at, since last i posted here.
    I’ll go review that now, and see how things are at present to get items posted.

    March 6, 2020 at 8:33 pm #1601
    Blake O'RuairiBlake O’Ruairi
    Participant

    Hi there.

    I’ve looked at many items on site; I’ve added a server info section at admin pls review it, ensure its up to requirements for scraper.

    All seems to go ok, preview shows items – gallery and other data, yet when i ‘Run’ things, nothing seems to happen.

    I cloned your original scrape to ‘slgreen b’ and have been using that to test run with.
    It seems as if it just wont run

    March 8, 2020 at 6:37 pm #1602
    Suman M.Suman M.
    Keymaster
    This reply has been marked as private.
    March 9, 2020 at 11:31 am #1604
    Blake O'RuairiBlake O’Ruairi
    Participant

    Hi.

    I have php execution set to 300.
    However when I spoke with hosting they stated that its at a max. execution of 30, serverwide, and cannot be increased at that hosting instance.
    Does your plugin require over 30 execution? If it does I’ll need to change servers, let me know asap on that pls.
    I’d hope it wouldn’t need that as so many hostings are similar to this.

    I added server info to admin, pls review there.

    March 9, 2020 at 3:53 pm #1606
    Suman M.Suman M.
    Keymaster

    Hi, I’m confused! Is the max execution time limit is 30 or 300 in your server? If it’s 30 then it’s less, and 300 shall be enough.
    Also, when I run the scraper task, first 2 items are scraped and imported and then it throws error. Can you please run the task (first delete the previously imported items), and then check your server error log for the detailed error message to know what exactly is creating the timeout error.

  • Author
    Posts
Viewing 10 posts - 1 through 10 (of 10 total)

You must be logged in and have valid license to reply to this topic.

License required for the following item
Scraper - Automatic Content Crawler Plugin for WordPress
Login and Registration Log in · Register