AI & Automation

What is Selenium?

Selenium is a browser automation framework that drives real browsers through the W3C WebDriver protocol. It has official bindings for Java, Python, C#, Ruby and JavaScript, and Selenium Grid spreads runs across many machines. Long-standing use makes it the default in enterprise QA, though newer libraries handle waiting and proxy credentials with far less code.

Your script does not touch the browser directly. It sends HTTP requests with JSON bodies to a driver binary - chromedriver, geckodriver, msedgedriver - and that driver speaks the browser's own control protocol. Selenium 4 retired the legacy JSON Wire Protocol in favour of the W3C WebDriver standard, which is why browser vendors ship and maintain those drivers themselves. Selenium Manager, added in the 4.x line, downloads a driver matching the installed browser, which removed most of the driver-version mismatches that once filled Selenium bug reports.

Proxies go in through the browser options. You can build a Proxy object with httpProxy, sslProxy, socksProxy, a socksVersion and a noProxy exclusion list, or pass --proxy-server directly as a Chromium argument. Credentials are the sore point, because WebDriver defines no standard way to send a proxy username and password, and the browser's native authentication dialog then blocks the session. The common answers are IP authentication, which sidesteps the prompt, and a small packed Chrome extension that answers the auth request programmatically.

Waiting is the skill Selenium demands. An implicit wait applies one polling timeout to every element lookup and interacts badly with explicit waits, so mixing them produces timeouts nobody can explain. Explicit WebDriverWait with a condition - clickable, visible, present - is the approach that survives contact with a real site. StaleElementReferenceException is the other rite of passage: the element you found vanished when the page re-rendered, and the fix is to re-find it rather than to add another sleep.

Selenium still wins on breadth. It runs against Safari on macOS, against real devices through Appium, and against every commercial cloud grid, all of which speak WebDriver as their wire format. A team writing tests in Java or C# has no equally mature alternative. What it lacks is native network interception and request mocking, which on Chromium require dropping to a DevTools Protocol session, and the automatic waiting that keeps modern suites from being flaky by construction.

Where you meet it

Selenium normally arrives with the job rather than by choice, in the shape of an existing regression suite and a Grid already wired into CI. The recurring tickets are familiar. A driver version drifts out of step, a test fails only in headless, or a proxy that works fine in curl hangs the browser on an authentication prompt. Knowing where credentials can enter the stack saves most of a day.

Common questions

How do I use an authenticated proxy in Selenium?

WebDriver has no field for proxy credentials, so the browser shows its own login prompt. The reliable route is IP authentication with the proxy provider, which removes the prompt. Otherwise, load a small Chrome extension that responds to the authentication request, or drive the browser's DevTools session directly on Chromium.

Is Selenium still worth using in 2026?

For cross-browser QA in Java or C#, and for anything already running on Grid or a commercial device lab, yes. For a new scraping project, Playwright generally needs less code and produces fewer flaky runs, mainly because of auto-waiting and built-in request interception.

Why does my Selenium script get blocked when manual browsing works?

Driver-controlled browsers expose automation signals such as navigator.webdriver, and they start from a default profile with no history or cookies. Scripts also click faster and more precisely than people do. Add a datacenter IP and the risk score tips over. Slowing interactions and exiting through residential or mobile addresses both cut the flag rate.

Related terms

Playwright
Playwright is Microsoft's browser automation library. A single API drives Chromium, Firefox and WebKit, with official bindings for JavaScript, Python, .NET and Java. It has auto-waiting, isolated browser contexts, per-context proxy settings and a trace viewer that replays a failed run step by step, which makes it the usual pick for new automation projects.
Puppeteer
Puppeteer is a Node.js library that drives Chrome and Chromium through the Chrome DevTools Protocol. From JavaScript it launches a browser, opens pages, clicks and types, intercepts network traffic and renders screenshots or PDFs. Teams run it to scrape pages that only assemble their content after JavaScript executes, and to automate logged-in workflows.
AI Agent
An AI agent is a program that lets a language model choose its own next action toward a goal, calling tools such as a browser, an HTTP client or a shell between reasoning steps and feeding each result back into its context. A fixed script sets the order in advance; an agent derives it from what the last step returned.
MCP
MCP, the Model Context Protocol, is an open standard for connecting AI assistants to external tools and data. A server declares the tools, resources and prompts it provides; a client such as Claude Desktop, Claude Code or an IDE discovers them at runtime over JSON-RPC and calls them on the model's behalf, so one integration works across every compatible assistant.
Browser Automation
Browser automation is the scripted control of a real browser, navigating, clicking, typing and reading the rendered DOM in place of a person doing it by hand. Playwright, Puppeteer and Selenium drive Chromium, Firefox and WebKit over a debugging or WebDriver protocol, headless or with a visible window, for testing, scraping and agent work.
Scrapy
Scrapy is an open-source Python framework for large-scale web crawling. Its asynchronous engine keeps many requests in flight at once, while spiders, item pipelines and downloader middlewares keep parsing separate from transport. Proxies plug into the middleware chain, so a crawl can change exit IP addresses without any edit to spider code.
API
An API (application programming interface) is a defined contract that lets one program call another. On the web that usually means HTTP endpoints returning JSON, with an authentication header, documented parameters and stable error codes. Proxy providers publish APIs so a script can list endpoints, trigger an IP rotation or read bandwidth usage without anyone opening a dashboard.

Real 4G/5G mobile and residential IPs

PROXIES.SX runs carrier IPs in 100+ countries with HTTP and SOCKS5 on every endpoint. $4/GB down to $2.40/GB at volume, free endpoints and rotation, and your GB never expire.