Automating Login and Cookie Management with WebDriver to Save Topology Data

The author, a frontend engineer and W3C performance group member, faced a situation where a new feature added a "save" button to a topology page, but nearly 200 existing pages lacked saved node information, making manual processing impractical.

WebDriver is a W3C‑defined standard derived from the Selenium WebDriver framework, designed to simulate user interactions with local or remote browsers for automation and testing purposes.

Before using WebDriver, the following preparations are required:

Download and install the appropriate ChromeDriver (or FirefoxDriver) and place it in the system PATH.

Install the Selenium WebDriver package for the language of choice; the example uses the Node.js package selenium-webdriver .

With the environment ready, the author wrote a simple demo that opens a topology page and clicks the save button. The core code is:

const webdriver = require('selenium-webdriver');
async function () {
  let driver = await new webdriver.Builder().forBrowser('chrome').build();
  await driver.get('http://mydomain/topo/1234'); // replace with actual URL
  driver.findElement(webdriver.By.css('.topo-component .btn-save')).click(); // replace selector as needed
}();

Running this demo opened a browser window and navigated to the URL, but the page immediately redirected to a login screen because each WebDriver instance starts with a fresh browser session that lacks the required authentication cookies.

To solve the "identity" problem, the author first logs in manually, captures the cookies, and stores them locally for later reuse.

The login and cookie‑saving logic is shown below:

// Store cookies to a local file
async function storeCookieToFile(driver) {
  const cks = await driver.manage().getCookies();
  const ckString = JSON.stringify(cks, null, 2);
  // Write the cookie JSON to a file named cookies.txt
  fs.writeFile('cookies.txt', ckString, (err) => {
    err && console.log(err);
  });
}

async function login() {
  let driver = await new webdriver.Builder().forBrowser('chrome').build();
  // Open the login page
  await driver.get('http://mydomain/data_need_authorization');
  // Fill username and password
  await driver.wait(driver.findElement(webdriver.By.css('input[name=userName]')), 1000);
  await driver.findElement(webdriver.By.css('input[name=userName]')).sendKeys('myUserName');
  await driver.findElement(webdriver.By.css('input[name=password]')).sendKeys('myPassword');
  await driver.findElement(webdriver.By.css('input[type=submit]')).click();
  // Optional: wait for manual captcha entry
  try {
    await driver.wait(driver.findElement(webdriver.By.css('input[name=captcha]')), 2000);
    await holdOn(5000); // pause for manual captcha solving
  } finally {
    // Additional business logic can be placed here
  }
  // Verify login succeeded
  await driver.wait(driver.findElement(webdriver.By.css('.user-info')), 5000);
  await storeCookieToFile(driver);
}();

The two most important WebDriver APIs used are sendKeys() for typing into input fields and getCookies() for retrieving the session cookies. Captcha handling is left to manual intervention via a short pause.

After obtaining the cookies, they can be applied to subsequent requests. Because cookies can only be added after the browser has visited a page on the same domain, the correct sequence is:

Read the cookie JSON from the local file.

Navigate to a non‑protected page on the target domain (e.g., http://mydomain ).

Add each cookie to the browser session.

Optionally wait a few seconds.

Navigate to the protected page again; the login state is now retained.

The implementation of this sequence is:

async function () {
  let driver = await new webdriver.Builder().forBrowser('chrome').build();
  await driver.get('http://mydomain');
  const cookies = await retrieveCookieFromFile(); // implement this helper
  JSON.parse(cookies).forEach(async ck => {
    await driver.manage().addCookie(ck);
  });
  await holdOn(2000);
  await driver.get('http://mydomain');
  // Verify cookie was set
  driver.manage().getCookie('XXX').then(cookie => {
    console.log('get cookie', cookie);
  });
}();

With authentication handled, the author proceeds to the business logic: iterating over the 200 topology URLs, waiting for the page to render, and clicking the save button for each. The loop uses the same get() , holdOn() , and click() methods demonstrated earlier.

Although the full loop code is omitted, readers can combine the presented snippets to build a complete automation script that logs in once, reuses cookies, and saves all topology data automatically.