Our position is easy to state: we are Geonode and we sell proxies, which has nothing to do with class selection. The one relevant observation is that class-based selectors are the most fragile kind, and fragility looks exactly like a network problem from the outside — the request succeeds, the page arrives, and your extraction returns nothing or the wrong thing. If you are debugging a scraper that stopped working, check whether the class names changed before you check anything about how the page was fetched. On a site using a modern build pipeline, they may change on every deploy.
The Problem With the Class Attribute
class holds a space-separated list of tokens, and XPath has no concept of that list. It sees one string.
<div class="card featured large">...</div>
To XPath, @class is the string "card featured large". There is no built-in way to ask "is card one of the tokens", which is exactly the question a CSS selector answers natively with .card.
That gap produces the two failure modes.
Exact matching is too strict:
//div[@class='card']
Matches only class="card" with nothing else and no extra whitespace. It misses class="card featured", class="featured card", and class=" card ". On a real page it will miss most of what you wanted.
Substring matching is too loose:
//div[contains(@class, 'card')]
Matches class="card" correctly, and also class="card-large", class="postcard", class="discard" and class="card-footer". On a page with any related naming, it returns a superset of what you asked for — and your code takes the first result, silently.
The second failure is the more dangerous one, because it produces results. Something is returned, it looks reasonable, and it is the wrong element.
The Correct Idiom
The standard solution pads both sides so that only whole tokens can match:
//div[contains(concat(' ', normalize-space(@class), ' '), ' card ')]
Read it step by step:
normalize-space(@class) trims leading and trailing whitespace and collapses internal runs of whitespace to single spaces. " card featured " becomes "card featured".
concat(' ', ..., ' ') wraps the result in spaces, giving " card featured ". Now every token is delimited by a space on both sides.
contains(..., ' card ') looks for the target surrounded by spaces.
Check it against the cases that broke the naive versions:
| Class attribute | Padded string | Contains ' card '? |
|---|---|---|
card | " card " | Yes |
card featured | " card featured " | Yes |
featured card | " featured card " | Yes |
card-large | " card-large " | No |
discard | " discard " | No |
postcard footer | " postcard footer " | No |
card | " card " | Yes |
Correct in every case. The normalize-space() step is not decorative — without it, class="card featured" with a double space would produce " card featured ", which still contains " card " and happens to work, but class="card\nfeatured" with a newline would not.
It is verbose. Wrap it:
def has_class(name):
return f"contains(concat(' ', normalize-space(@class), ' '), ' {name} ')"
tree.xpath(f"//div[{has_class('card')}]")
const hasClass = name =>
`contains(concat(' ', normalize-space(@class), ' '), ' ${name} ')`;
Every codebase doing serious extraction with XPath ends up with some version of this helper. Writing it once is better than getting the padding wrong in one expression out of twenty.
Multiple Classes
Combine with and:
//div[contains(concat(' ', normalize-space(@class), ' '), ' card ')
and contains(concat(' ', normalize-space(@class), ' '), ' featured ')]
Which is where the verbosity becomes genuinely painful — that is 140 characters to express what CSS says in div.card.featured.
For "either of two classes", use or:
//div[contains(concat(' ', normalize-space(@class), ' '), ' card ')
or contains(concat(' ', normalize-space(@class), ' '), ' tile ')]
For "has this class but not that one":
//div[contains(concat(' ', normalize-space(@class), ' '), ' card ')
and not(contains(concat(' ', normalize-space(@class), ' '), ' hidden '))]
At two or more class conditions, seriously consider whether a CSS selector would do. div.card.featured:not(.hidden) is the same logic in a fifth of the characters, and every mainstream parsing library supports CSS selectors alongside XPath. There is no rule requiring you to pick one language for the whole file.
Generated and Hashed Class Names
The modern complication, and it changes the advice.
Many front-end build tools generate scoped class names to avoid collisions — CSS Modules, styled-components, various CSS-in-JS libraries. The result looks like this:
<div class="ProductCard_container__3xK9p">...</div>
<div class="css-1x9dj2k">...</div>
The hash portion changes whenever the component's styles change, which in practice means on many deploys. A selector anchored to the full name breaks without warning.
Three ways to handle it, in order of preference.
Anchor on the stable prefix, where the tooling produces one:
//div[starts-with(@class, 'ProductCard_container__')]
CSS Modules conventionally produce ComponentName_elementName__hash, so the part before the final double underscore is stable across builds. This works well when it applies.
Find a data-* attribute instead. Many applications include data-testid, data-test or similar attributes precisely so that automated tools have a stable hook:
//div[@data-testid='product-card']
If one exists, use it. It is more stable than any class name because it was chosen deliberately rather than generated.
Anchor on something else entirely. Structure relative to a heading, text content, or an element type. //h2[normalize-space()='Featured']/following-sibling::div[1] does not care what the classes are called.
And for the fully opaque case — css-1x9dj2k with no stable component — class-based selection is simply not viable, and pretending otherwise produces a scraper that breaks weekly. Look for structured data on the page, or for the API the page itself calls.
Class Order and Whitespace
Two things that trip people up and are worth being precise about.
Class order in the attribute is meaningless. class="card featured" and class="featured card" are equivalent to the browser and to CSS. The padded-concat idiom handles both correctly; exact matching handles neither reliably. If you find yourself writing an expression that depends on the order, that is a signal something is wrong.
Whitespace can be anything. Tabs and newlines are valid separators in a class attribute, and they appear in hand-formatted HTML:
<div class="card
featured">
normalize-space() collapses all of it, which is why the idiom includes it. An expression using concat(' ', @class, ' ') without normalising will fail on that markup, and the failure is invisible in a rendered page.
Case matters. Class names are case-sensitive in HTML documents parsed as XHTML and treated case-insensitively in standards-mode HTML for CSS matching — but XPath string comparison is always case-sensitive. If a page mixes Card and card, the padded idiom will treat them as different. XPath 1.0 has no lower-case() function, so the workaround is translate() with explicit alphabets, at which point the expression becomes genuinely unreadable and a CSS selector is clearly the better choice.
Getting the Ancestor or Descendant of a Class
Class selection is usually a means rather than an end. Two patterns cover most of it.
From a class to something inside it:
//div[contains(concat(' ',normalize-space(@class),' '),' card ')]//span[@class='price']
Or, in code that already has the card element, use a relative expression — and the leading dot is essential:
for card in tree.xpath(f"//div[{has_class('card')}]"):
price = card.xpath(".//span[@class='price']/text()")
.//span searches within the card. //span searches the whole document from the root, which returns the first price on the page for every card. This produces uniform, plausible, wrong output and it is one of the most common bugs in extraction code.
From an element up to its container:
//span[@class='price']/ancestor::div[contains(concat(' ',normalize-space(@class),' '),' card ')][1]
The [1] matters: ancestor:: is a reverse axis, so position 1 is the nearest matching ancestor rather than the outermost. Without it you get every matching ancestor, and with nested containers that is rarely what you want.
Library-Specific Notes
The idiom is the same everywhere; the surrounding API is not, and a few library differences cause avoidable confusion.
lxml (Python). Supports both languages, and cssselect is the pragmatic choice for class work:
from lxml import html
tree = html.fromstring(source)
tree.cssselect('div.card.featured') # clear
tree.xpath(f"//div[{has_class('card')}]") # when inside a larger expression
cssselect translates CSS to XPath internally, so the two are equivalent in capability for the selectors it supports. Note that it is a separate package from lxml itself and needs installing.
BeautifulSoup (Python). Has no XPath support at all — a fact that surprises people arriving from other ecosystems. It offers select() for CSS selectors and its own find_all(class_='card') API, which does correct token matching natively. If you need XPath specifically, you need lxml.
Selenium. Accepts both By.XPATH and By.CSS_SELECTOR, and uses the browser's engine for each. That means XPath 1.0 only, and it means CSS support matches whatever the browser supports — including :has().
Playwright. Auto-detects the locator type from the string, so page.locator('div.card') and page.locator('//div[@id="x"]') both work without a prefix. It also offers text-based locators — page.getByText(), page.getByRole() — which cover a large share of what people previously needed XPath's text matching for, and are more readable than either language.
Scrapy. Provides .css() and .xpath() on selectors, and they chain, which is genuinely useful:
for card in response.css('div.card'):
price = card.xpath(".//dt[normalize-space()='Price']/following-sibling::dd[1]/text()").get()
CSS for the structural pass, XPath for the label lookup, in the same expression chain. Note the leading . on the XPath — Scrapy's chained selectors have the same root-versus-relative trap as everything else.
Browser console. $x('//div[@class="card"]') evaluates XPath in Chrome and Firefox dev tools, and document.querySelectorAll('div.card') handles CSS. Testing an expression here before putting it in code is worth the ten seconds, with the caveat that the browser's DOM is post-JavaScript and your parser's is not — an expression that works in the console may find nothing in raw HTML.
When to Use CSS Instead
Stated plainly, because for this specific job the answer is usually yes.
Use CSS when your criteria are classes. div.card, div.card.featured, div.card:not(.hidden) — all clearer, shorter and correct by default. CSS understands the class attribute as a token list, which is exactly the thing XPath lacks.
Use XPath when the class is incidental and the real criterion is something CSS cannot express — text content above all. //div[contains(@class,'card')][.//span[contains(., 'Sold out')]] needs XPath for the text condition, and the class part comes along for the ride.
Mix them. Every mainstream parsing library supports both. lxml has cssselect alongside xpath; Selenium accepts both locator strategies; Playwright accepts both. Using CSS for the structural selection and XPath for the text conditions is not a compromise, it is the arrangement that produces the shortest readable code.
The one case for XPath class selection is where you need it inside a larger XPath expression and switching languages mid-expression is not possible. That is a real constraint and it is why the idiom exists — but it is narrower than the amount of XPath class matching you will find in the wild.
People Also Ask
How do I select an element by class in XPath?
Use //div[contains(concat(' ', normalize-space(@class), ' '), ' card ')]. The padding ensures only whole class tokens match. Plain @class='card' misses elements with additional classes, and contains(@class,'card') also matches card-large and discard.
Why does contains(@class, 'name') match the wrong elements?
Because it is a plain substring test with no notion of word boundaries. The class attribute is a space-separated list, but XPath sees a single string, so contains(@class,'card') matches any class name containing those four characters anywhere.
How do I select an element with two classes in XPath?
Combine two padded-concat conditions with and. It works and it is around 140 characters. At two or more class conditions, a CSS selector — div.card.featured — is dramatically clearer, and most parsing libraries let you use one.
Does class order matter in XPath?
Not with the padded-concat idiom, which matches a token wherever it appears in the attribute. It does matter with exact comparison, which is one of several reasons @class='card featured' is a poor choice. Class order is meaningless in HTML, so a selector that depends on it is a bug waiting to happen.
How do I handle randomly generated class names?
Anchor on the stable prefix with starts-with() where the tooling produces one, use a data-testid attribute if the application provides one, or anchor on something else entirely such as text or structure. For fully opaque hashed names with no stable part, class-based selection is not viable.
Is XPath or CSS better for selecting by class?
CSS, clearly. It treats the class attribute as a token list, which is what it is, so div.card is both shorter and correct. XPath needs a seventy-character idiom to express the same thing. Use XPath when you also need something CSS cannot do, such as matching text.
How do I find a parent element by class in XPath?
//span[@class='price']/ancestor::div[contains(concat(' ',normalize-space(@class),' '),' card ')][1]. The [1] selects the nearest matching ancestor, since ancestor:: is a reverse axis where position 1 means closest rather than outermost.
Why does my relative XPath return the same value for every element?
Because you used // instead of .// inside the loop. A leading // searches from the document root regardless of context, so every iteration finds the first match on the whole page. .// searches within the current element, which is what you meant.
Wrapping Up
Class selection is where XPath shows its age. The class attribute is a token list and XPath has no token operations, so expressing "has this class" requires padding the string with spaces and searching for a padded token — a seventy-character idiom for what CSS says in nine.
Learn the idiom, because you will need it inside larger XPath expressions, and wrap it in a helper so you write it once rather than twenty times. The normalize-space() step is not optional: real markup contains newlines and tabs in class attributes, and an unnormalised expression fails on them invisibly.
But the more useful conclusion is the one that avoids the idiom. If your selection criteria are classes and nothing else, write a CSS selector. Every mainstream library supports both, mixing them in one file is normal, and choosing the right tool per expression produces shorter code that a colleague can read.
And treat class names as the least stable anchor available. Generated and hashed names change on deploy; even hand-written ones change whenever someone restyles a component. A data-testid, a heading's text, or an element's relationship to something identifiable will all outlast the class name — and the failure mode when a class disappears is not an error, it is quiet, well-formed, empty output.