Master Selenium with Python: From Installation to Advanced Browser Automation

Introduction

Selenium is a powerful Python library for web application testing and automation, enabling you to control browsers like Chrome, Firefox, Safari, Edge, and more.

Preparation

Install Selenium

pip install selenium

Install Browser Driver

You can install the ChromeDriver manually by matching the Chrome version or automatically using webdriver_manager:

from webdriver_manager.chrome import ChromeDriverManager
from selenium import webdriver
browser = webdriver.Chrome(ChromeDriverManager().install())

Basic Usage

Initialize Browser

from selenium import webdriver
# Standard browser
browser = webdriver.Chrome()
# Headless mode
options = webdriver.ChromeOptions()
options.add_argument("headless")
browser = webdriver.Chrome(options=options)

Open a Page

browser.get("https://www.baidu.com")

Set Window Size

# Full screen
browser.maximize_window()
# Custom size
browser.set_window_size(500, 500)

Refresh and Navigation

browser.refresh()
browser.back()
browser.forward()

Page Properties

print(browser.title)
print(browser.current_url)
print(browser.page_source)

Element Location

By ID: browser.find_element_by_id('kw') By name: browser.find_element_by_name('wd') By class name: browser.find_element_by_class_name('s_ipt') By tag name: browser.find_element_by_tag_name('input') By link text: browser.find_element_by_link_text('新闻') By partial link text: browser.find_element_by_partial_link_text('闻') By XPath: browser.find_element_by_xpath("//*[@id='kw']") By CSS selector: browser.find_element_by_css_selector('#kw') General method:

from selenium.webdriver.common.by import By
browser.find_element(By.ID, 'kw')

Element Attributes

Retrieve attributes, text, ID, location, tag name, and size using methods like get_attribute(), .text, .id, .location, .tag_name, and .size.

Interactions

Input text: element.send_keys('python') Click: element.click() Clear text: element.clear() Submit (Enter): element.submit() Radio buttons and checkboxes: locate and click() Dropdowns using Select:

from selenium.webdriver.support.select import Select
select = Select(browser.find_element_by_name('example'))
select.select_by_index(2)
select.select_by_value('value')
select.select_by_visible_text('Option Text')

Multiple Windows and Frames

Switch to frames with browser.switch_to.frame() and back with browser.switch_to.parent_frame(). Manage tabs using browser.window_handles and browser.switch_to.window(handle).

Mouse Actions

Use ActionChains for advanced mouse operations:

Right click: ActionChains(browser).context_click(element).perform() Double click: ActionChains(browser).double_click(element).perform() Drag and drop:

ActionChains(browser).drag_and_drop(source, target).perform()

Hover:

ActionChains(browser).move_to_element(element).perform()

Keyboard Actions

Import Keys and send special keys:

from selenium.webdriver.common.keys import Keys
input.send_keys(Keys.ENTER)
input.send_keys(Keys.BACK_SPACE)
input.send_keys(Keys.CONTROL, 'a')  # select all

Wait Strategies

Explicit wait with WebDriverWait and expected conditions:

Implicit wait: browser.implicitly_wait(10) Hard sleep:

import time; time.sleep(5)

Other Utilities

Execute JavaScript:

Cookie management:

This tutorial provides a complete workflow for using Selenium in Python, covering installation, basic operations, element handling, user interactions, waiting mechanisms, JavaScript execution, and cookie control, enabling reliable web automation and scraping tasks.