How To Scrape Websites Using Python + Selenium

There is endless amounts of data on the internet, so let’s go ahead and pull some data from any given website using Python! In this example we’ll use Python 3 & a package called Selenium! If you need python installed use this article I wrote to install it! Otherwise, let’s get started!

First off, let’s go ahead and install Selenium, go into your terminal / command prompt and type in “pip install selenium” (I already have mine installed):

At this point, let’s make our way over to our Python script, let’s start off by importing Selenium as well as the web driver, just like this:

In this case, we’re going to go ahead and use the Chrome web browser as our base driver, this means you will have to have Google Chrome installed on your PC (you most probably already have this installed), with Selenium you also have the ability to use Safari and Firefox browsers as well. Selenium is a little bit different than other web scrapers because it acts as an actual web browser, so it will literally boot up a google chrome instance on your computer, this may or may not be beneficial to you but you can easily turn the actual GUI off if needed.