Introduction of web scrapping with Python

In this series of blog posts, we will introduce 2 methods to perform web scrapping with Python:

Method 1: Using requests and BeautifulSoup

This method is applicable for web page which having static or server side rendered content. However, this method is not applicable for web page that rendered by client side Javascript.

Method 2: Using Selenium and Geckodriver / Chromedriver

This method is applicable for all types of web pages like static, server side rendered and client side rendered content.

Implementation for Method 1:

First install python libraries requests and beautiful Soup 4 with command below:

pip install requests BeautifulSoup4

In this implementation, we will list out all blog posts’ title and author at byprogrammers.com

Implementation for Method 2:

For method 2, before you jump into the implementation, ensure you have install Geckodriver or Chromedriver in your environment.

And also please run command below in terminal to install selenium library to your environment:

pip install selenium

In this implementation, similar with method #1, we will list out all blog posts’ title and author at byprogrammers.com

For further details, please feel free to checkout my Github repo for this blog post.

Happy Coding ?!