i interested in automating repetitive data entry in forms website frequent. far tools i've looked provide support in headless fashion selenium webdriver , mechanize.
my question is, there fundamental technical difference in using once versus other? selenium used testing. i've noticed folks use doing i'm looking for, , that's automating data entry. testing becomes second benefit in case.
is there reasons not use selenium want on mechanize? not matter , both of these tools work?
i'm not asking better, i'm asking right tool job. perhaps i'm not understanding premise behind purpose of each tool.
these different tools "cross" in web-scraping, web automation, automated data extraction scope.
mechanize mature , widely-used tool programmatic web-browsing lot of built-in features, cookie handing, browser history, form submissions. key thing understand here mechanize.browser not real browser, cannot execute , understand javascript, cannot send asynchronous requests needed form web page.
this selenium comes play - browser automation tool used in web-scraping. selenium becomes "fall-back" tool - when cannot web-scrape site mechanize or robobrowser or mechanicalsoup (note - alternatives) because of, instance, it's javascript "heaviness", choice selenium. selenium can go headless, automating phantomjs browser, or having virtual display. commonly mentioned drawback, performance mentioned - selenium working target site real user in web browser, loading additional files needed form page, making xhr requests, rendering etc.
and not mean should use selenium everywhere - choose tool wisely, choose because fits problem better, not because more familiar instrument.
also note should, first, consider using api (if provided target website) instead of going down web-scraping. and, if comes it, web-scraping citizen:
Comments
Post a Comment