I want to rip the contents of a pay website, but I have to log in to their web site on a web page to get access

Does anyone have any good tools for Windows for that?

I’m guessing that any such tools must have a built in browser, or be a browser plugin for it to work.

  • zabadoh@lemmy.mlOP
    link
    fedilink
    English
    arrow-up
    4
    ·
    1 year ago

    I have an account, so that’s not a problem. The problem is how to automate going into every little content page and downloading the content, including the hi-res files.

      • zabadoh@lemmy.mlOP
        link
        fedilink
        arrow-up
        2
        ·
        edit-2
        1 year ago

        Webcopy looks promising if I can get the crawler part of it to work with this site’s authentication…

        edit: I couldn’t get Webcopy’s spider to authenticate correctly.

        Webcopy uses the deprecated version of Internet Explorer in Windows 10 as a module, and I can log into the website using the Capture Forms browser dialog, but the cookies or whatever else don’t translate over to the spider.