How to Scrape QQ Group Member Data with Selenium: Step‑by‑Step Guide

This guide demonstrates how to automate the extraction of QQ group member information using Python and Selenium.

1. Open QQ group website and log in

First, navigate to https://qun.qq.com/ and trigger the login dialog.

url = 'https://qun.qq.com/'
browser = webdriver.Chrome()
browser.get(url)
# Click the login link
browser.find_element_by_css_selector('#headerInfo p a').click()

2. Handle the login iframe

The login form appears inside an iframe. Use an explicit wait, retrieve the iframe src, and load it directly.

WebDriverWait(browser, 1000).until(
    EC.presence_of_all_elements_located((By.CSS_SELECTOR, '#loginWin iframe'))
)
print('Login iframe loaded')
iframe_url = browser.find_element_by_css_selector('#loginWin iframe').get_attribute('src')
browser.get(iframe_url)
# Click the quick‑login avatar
WebDriverWait(browser, 1000).until(
    EC.presence_of_all_elements_located((By.ID, 'qlogin_list'))
)
browser.find_element_by_css_selector('#qlogin_list a').click()
print('Login successful')

3. Switch to group management

After logging in, click the "Group Management" tab and then the "Member Management" link.

# Click Group Management
WebDriverWait(browser, 1000).until(
    EC.presence_of_all_elements_located((By.XPATH, './/ul[@id="headerNav"]/li[4]'))
)
browser.find_element_by_xpath('.//ul[@id="headerNav"]/li[4]').click()
# Click Member Management
WebDriverWait(browser, 1000).until(
    EC.presence_of_all_elements_located((By.CLASS_NAME, 'color-tit'))
)
browser.find_element_by_class_name('color-tit').click()
# Switch to the new window opened by Member Management
browser.switch_to.window(browser.window_handles[1])

4. Locate my groups

Wait for the group list to appear, then collect the li elements representing each group I have joined.

WebDriverWait(browser, 1000).until(
    EC.presence_of_all_elements_located((By.CLASS_NAME, 'my-all-group'))
)
lis = browser.find_elements_by_xpath('.//div[@class="my-all-group"]/ul[2]/li')

5. Iterate over groups and paginate

For each group, click it, determine the total member count, and paginate (21 members per page). The example limits pagination to a few pages for speed.

num = 0
while True:
    try:
        lis[num].click()
        WebDriverWait(browser, 1000).until(
            EC.presence_of_all_elements_located((By.CLASS_NAME, 'list'))
        )
        groupMemberNum = eval(browser.find_element_by_id('groupMemberNum').text)
        count = groupMemberNum // 21 + 1
        if count > 5:
            count = 5
        while count:
            count -= 1
            browser.execute_script('document.documentElement.scrollTop=100000')
            time.sleep(2)
        time.sleep(3)
        # Extract member rows
        trs = browser.find_elements_by_class_name('mb')
        if trs:
            for tr in trs:
                tds = tr.find_elements_by_tag_name('td')[2:]
                if len(tds) == 8 or len(tds) == 9:
                    qq_name = tds[0].text
                    group_name = tds[1].text
                    qq_number = tds[2].text
                    gender = tds[3].text
                    qq_year = tds[4].text
                    join_time = tds[5].text
                    if len(tds) == 9:
                        level = tds[6].text
                        end_time = tds[7].text
                    else:
                        level = None
                        end_time = tds[6].text
                    data_dict = {
                        'qq_name': qq_name,
                        'group_name': group_name,
                        'qq_number': qq_number,
                        'gender': gender,
                        'qq_year': qq_year,
                        'join_time': join_time,
                        'level': level,
                        'end_time': end_time
                    }
                    print(data_dict)
        # Return to group list
        browser.find_element_by_id('changeGroup').click()
        time.sleep(3)
        WebDriverWait(browser, 1000).until(
            EC.presence_of_all_elements_located((By.CLASS_NAME, 'ui-dialog'))
        )
        lis = browser.find_elements_by_xpath('.//div[@class="my-all-group"]/ul[2]/li')
        num += 1
    except Exception:
        lis = browser.find_elements_by_xpath('.//div[@class="my-all-group"]/ul[2]/li')
        num += 1
        continue

6. Save results

After collecting all dictionaries, write them to a JSON file and then to a CSV file.

with open('data_json.json', 'a+', encoding='utf-8') as f:
    json.dump(data_list, f)
print('JSON file written')
with open('data_csv.csv', 'w', encoding='utf-8-sig', newline='') as f:
    writer = csv.DictWriter(f, data_list[0].keys())
    writer.writeheader()
    writer.writerows(data_list)
print('CSV file written')

Illustrative Screenshots

By following these steps, you can programmatically collect detailed member data from QQ groups and store it for further analysis.