How to scrap IMDB lists?

Tagged: 

Viewing 10 posts - 1 through 10 (of 10 total)
  • Author
    Posts
  • #1799
    Varun MahajanVarun Mahajan
    Participant

    Hi,

    I need to scrap IMDB lists and movies into databases or tables plugin via shortcode or any method available. How to do it?

    #1803
    Suman M.Suman M.
    Keymaster

    Hi, you can scrape from imdb.com. Can you please let us know the exact source url to scrape from? And you can also create dynamic shortcodes using scraper plugin – https://support.wpbots.net/documentation/creating-dynamic-wordpress-shortcode/

    #1805
    Varun MahajanVarun Mahajan
    Participant

    I want to scrap this list: https://www.imdb.com/list/ls053573902/

    The list keeps updating after a few days.

    How can I extract all information available here and present in my own format on my page.

    #1815
    Suman M.Suman M.
    Keymaster

    You can either scrape it as a single post or multiple post. But as you want to show them in your own format it’s recommended to scrape as Multiple Post – https://www.screencast.com/t/LNhIooGkj
    This will import each item as separate post in your website and then you can display them in your site in any format you want.

    #1823
    Varun MahajanVarun Mahajan
    Participant
    This reply has been marked as private.
    #1825
    Suman M.Suman M.
    Keymaster

    I have created a task “imdb – best tv shows” in your website. It’s set to import the items as wordpress post, but you may also import them as any custom post type.
    Please check and let us know!

    #1826
    Varun MahajanVarun Mahajan
    Participant

    Hi, thanks for creating the task. I run the task, the first time it gave 500 error. Now, it is giving this error:

    There is no post to process, the task completed loop. (Please check your “Total Run” limit, if there is any).

    #1829
    Suman M.Suman M.
    Keymaster

    This means you have already run the task and the posts have been imported. If you want to run the task again, you’ll first need to reset the task and then run it – https://www.screencast.com/t/iBFqnM1A

    #1831
    Varun MahajanVarun Mahajan
    Participant

    Okay, I reset the task. Then I ran again – see same gateway error.

    https://gyazo.com/baf6add87784360ffea09a2afb51d38c

    #1838
    Suman M.Suman M.
    Keymaster

    I could no more login to your site using above credentials. The gateway timeout error occurs if your site/server terminates the process. Please make sure that your site/server’s PHP memory limit is at least 512M and max execution time limit is minimum 300 seconds. Let us know!

Viewing 10 posts - 1 through 10 (of 10 total)

You must be logged in and have valid license to reply to this topic.

License required for the following item
Login and Registration Log in · Register