What is SERP?
A SERP is a search engine results page, meaning everything an engine returns for one query. Alongside organic links it can carry ads, a local map pack, People Also Ask, images, video, shopping units and an AI-generated summary. What appears changes with the query, the searcher's location, language and device, so two SERPs for the same words are rarely identical.
The organic list is one part of the page, and that complicates any claim about rank. A tool reporting "position 1" may mean the first organic link or the first block of any kind. On a query carrying three ads, a map pack and an answer box above it, the top organic result can sit most of a screen below the fold while its organic position stays unchanged. Rank trackers that record the organic rank alongside the pixel or block position give a far more honest picture of visibility.
Parsing a SERP is continuing maintenance rather than a one-time build. Class names in the markup are generated and rotate without warning. Mobile and desktop layouts differ structurally, and some blocks arrive through JavaScript instead of the initial HTML. Parsers anchored on stable structural cues, such as link containers and heading hierarchy, break less often than ones matching CSS classes. Teams extracting SERPs at scale keep a small set of fixture pages and re-run the parser against them whenever results start looking implausible.
Answer-style results have added a second thing to track. Google's AI Overviews, Perplexity and ChatGPT's search mode return a written answer with citations rather than a ranked list, so the question shifts from what position a site holds to whether it is cited at all, and in which sentence. Each of those products has a different page shape and its own citation format, and each tolerates automated querying to a different degree. A pipeline built for ten blue links needs separate extraction and separate storage for citation data.
Collecting SERPs at volume runs into the same defences as any other automated fetching. Engines watch request rate and pattern per address, and they answer suspicion with an interstitial rather than an error code, which means a naive scraper records "no results" instead of failing loudly. Log the HTTP status and a hash of the response body so you can tell a challenge page from an empty result. Spread requests across countries so each address stays within normal usage for its region.
Where you meet it
You hit this the first morning your SERP parser returns zero rows for every keyword and nothing in the logs looks broken. The fetches were 200 OK, and the bodies were challenge pages or a re-laid-out results block. It also comes up in reporting disputes, when two tools disagree about position because one counts ads and answer boxes and the other does not. Define what a position means before you argue about it.
Common questions
What is the difference between a SERP and organic results?
The SERP is the whole page an engine returns. Organic results are the unpaid links ranked by the engine's algorithm, which are one component of it. Ads, local packs, image carousels, People Also Ask and AI summaries occupy the same page and often push organic listings well down the screen without altering their organic rank.
Why do two people searching the same keyword see different SERPs?
An engine assembles results per request from the searcher's approximate location, language setting, device type, logged-in state and prior activity, along with experiments that expose different layouts to different users. Location has the largest effect on anything with local intent, which is why rank tracking specifies a city and a device rather than one global position.
Is scraping SERPs allowed?
The results are publicly visible, but the major engines' terms prohibit automated querying, and they enforce with rate limits and challenges rather than lawsuits in most cases. Legal exposure varies by jurisdiction and by what you do with the data, so read the terms that apply to you and take advice if the dataset is commercial.
Related terms
Real 4G/5G mobile and residential IPs
PROXIES.SX runs carrier IPs in 100+ countries with HTTP and SOCKS5 on every endpoint. $4/GB down to $2.40/GB at volume, free endpoints and rotation, and your GB never expire.