A quick note on why we are writing this. We are Geonode; we sell proxies to people who extract data, so selector questions come to us constantly. The relevant disclaimer is short: selector bugs and proxy problems produce completely different symptoms, and confusing them wastes hours. A blocked request returns a challenge page or an error status. A broken contains() returns a perfectly good page and an empty result. If the HTML is there and your expression finds nothing, this article is the right place and the network is fine.
The Function Itself
The XPath 1.0 specification is terse:
The contains function returns true if the first argument string contains the second argument string, and otherwise returns false.
Both arguments are strings. Two arguments, a boolean result, case-sensitive, no wildcards, no regular expressions.
//a[contains(@href, 'download')]
//div[contains(@class, 'product')]
//p[contains(text(), 'Price')]
The interesting behaviour is entirely in what happens when the argument you supply is not a string, which is most of the time.
The Node-Set Trap: contains() Only Sees the First Node
This is the bug that produces "my XPath works on some pages and not others", and the specification explains it precisely:
A node-set is converted to a string by returning the string-value of the node in the node-set that is first in document order. If the node-set is empty, an empty string is returned.
So when you write something that produces a node-set and pass it to contains(), XPath silently discards everything except the first node.
<div>
<p>Introduction</p>
<p>Price: £42</p>
<p>Availability</p>
</div>
contains(//p, 'Price') → false
False, because //p is a node-set of three, string conversion takes the first one — "Introduction" — and that does not contain "Price". The other two paragraphs were never considered.
The fix is to make the predicate apply per node rather than converting a set:
//p[contains(., 'Price')] → the second paragraph
Here contains() is evaluated once for each p, with . being that individual node. This is the difference between asking "does the set contain this?" and "which members of the set contain this?", and only the second is usually what anyone means.
The same trap appears with text(), which is also a node-set:
//div[contains(text(), 'Price')]
text() returns all direct text-node children, and string conversion takes the first. If the element has text broken across several nodes — which happens whenever there is nested markup — you are testing only the first fragment.
. Versus text(): The Other Half of the Same Problem
The specification defines the string-value of an element as:
the concatenation of the string-values of all text node descendants of the element node in document order
That is the crucial difference between the two forms.
<p>Total: <strong>£42.00</strong> including VAT</p>
contains(., '£42.00') → true (all descendant text, concatenated)
contains(text(), '£42.00') → false (first direct text node only: "Total: ")
. reaches into nested elements. text() sees only direct children, and only the first of them.
Use . by default. It matches what a reader would consider "the text of this element", and it survives markup changes such as someone wrapping a value in a <span>.
Use text() deliberately when you specifically want to exclude nested content — for instance matching a label without matching text inside a child badge or tooltip.
For "any of the text nodes contains this", the correct form applies the predicate to the text nodes themselves:
//div[text()[contains(., 'Price')]]
Verbose, and correct.
Whitespace: normalize-space() Is Not Optional
Real HTML is pretty-printed, and the whitespace goes into the string value.
<td>
In stock
</td>
The string-value of that cell is "\n In stock\n", so an exact comparison against 'In stock' fails.
The specification defines the fix:
The normalize-space function returns the argument string with whitespace normalized by stripping leading and trailing whitespace and replacing sequences of whitespace characters by a single space.
//td[normalize-space() = 'In stock']
//td[contains(normalize-space(), 'In stock')]
Note that normalize-space() with no argument operates on the context node, which is what you want inside a predicate.
For contains() specifically, whitespace matters less at the ends and a great deal in the middle. A search for 'In stock' fails against "In stock" unless you normalise first. If you write only one habit into your selectors, make it wrapping text comparisons in normalize-space().
Matching Classes Properly
The most common misuse of contains(), and the one most likely to be silently wrong.
//div[contains(@class, 'btn')]
That matches class="btn". It also matches class="btn-primary", class="unbtn", and class="sidebar-btn-group". Because @class is a single space-separated string and contains() is a plain substring test, it knows nothing about word boundaries.
The correct idiom pads both the attribute and the target with spaces so that only whole tokens match:
//div[contains(concat(' ', normalize-space(@class), ' '), ' btn ')]
Read it as: take the class attribute, normalise its whitespace, wrap it in spaces so every token is space-delimited on both sides, and look for the target surrounded by spaces. class="btn-primary" becomes " btn-primary ", which does not contain " btn ". class="icon btn large" becomes " icon btn large ", which does.
It is ugly. It is also correct, and it is what every mature scraping codebase ends up with. Wrap it in a helper:
def has_class(name):
return (f"contains(concat(' ', normalize-space(@class), ' '), ' {name} ')")
When you need two classes:
//div[contains(concat(' ', normalize-space(@class), ' '), ' btn ')
and contains(concat(' ', normalize-space(@class), ' '), ' primary ')]
At which point a CSS selector — div.btn.primary — is dramatically more readable and does exactly the right thing. If you are matching on classes and nothing else, use CSS. XPath earns its place when you need text matching or backwards navigation, not for the things CSS already does well. We compared the two in XPath preceding-sibling.
Case Sensitivity and the translate() Workaround
contains() is case-sensitive, and XPath 1.0 has no lower-case() function. Browsers and Selenium implement XPath 1.0, so this limitation is live wherever it matters most.
The workaround uses translate(), defined as returning "the first argument string with occurrences of characters in the second argument string replaced by the character at the corresponding position in the third argument string":
//p[contains(translate(., 'ABCDEFGHIJKLMNOPQRSTUVWXYZ',
'abcdefghijklmnopqrstuvwxyz'), 'price')]
Character-by-character transliteration, ASCII only. It will not lowercase accented characters unless you extend both strings to cover them, which gets unwieldy fast.
Two better options where available:
Match a case-stable substring. If the page says "Price" or "PRICE" but never "price", matching 'rice' is ugly and works. Frequently the pragmatic answer.
Use a library with a richer XPath. Server-side parsers such as lxml support EXSLT extensions including re:test() for regular expressions, which handles case and much else. Browsers do not, so a snippet you found for lxml may fail in Selenium for exactly this reason.
If you find yourself writing a long translate(), that is a signal you have outgrown XPath 1.0 for this task.
The Related String Functions
contains() is one of a small family, and the others are often more precise.
starts-with() — "returns true if the first argument string starts with the second argument string". More specific than contains() and correspondingly less likely to over-match:
//a[starts-with(@href, 'https://')]
There is no ends-with() in XPath 1.0. The workaround uses substring() and string-length(), and it is unpleasant enough that a different approach is usually better.
substring-before() and substring-after() — the first "returns the substring of the first argument string that precedes the first occurrence of the second argument string... or the empty string if the first argument string does not contain the second". Useful for splitting a value inside the expression:
substring-after(//span[@class='price'], '£')
normalize-space() — covered above, and the one you should use most.
translate() — case folding, and also stripping characters by mapping them to nothing:
translate(., ',', '')
string-length() — filtering out empty or truncated values:
//td[string-length(normalize-space()) > 0]
Combining them is where the value is:
//tr[contains(normalize-space(td[1]), 'Weight')]/td[2]
The second cell of any row whose first cell mentions "Weight", whitespace-insensitive.
Patterns That Come Up Constantly
The expressions below cover most real extraction work and are worth keeping to hand.
Find the value next to a label. The single most common requirement in scraping structured pages:
//dt[contains(normalize-space(), 'Price')]/following-sibling::dd[1]
//th[contains(normalize-space(), 'Weight')]/following-sibling::td[1]
Note the [1] — without it, following-sibling::td returns every subsequent cell in the row, and your code silently takes the first while you assume it was the only one.
Find a link by its visible text rather than its href:
//a[contains(normalize-space(), 'Download')]
More robust than matching the URL when the URL is a hashed identifier, and more fragile when the site is translated. Pick according to which changes more often.
Find a container by something inside it:
//div[contains(concat(' ', normalize-space(@class), ' '), ' card ')][.//span[contains(., 'Sold out')]]
Two predicates in sequence: a card, containing a span mentioning "Sold out". This composes well and reads better than trying to express it in one condition.
Exclude rather than include. Often the clearer formulation:
//tr[not(contains(@class, 'header'))]
//li[not(contains(normalize-space(), 'Advertisement'))]
Match a button whether it is a button or an a:
//*[self::button or self::a][contains(normalize-space(), 'Continue')]
Find the row containing a value and take a different column:
//tr[td[contains(normalize-space(), 'SKU-1234')]]/td[3]
Read outwards: rows that have a cell mentioning the SKU, then the third cell of that row. This is the table-lookup pattern, and it survives column reordering far better than an absolute index into the whole table.
Guard against empty matches. A predicate that filters out blanks costs nothing and prevents a class of downstream confusion:
//td[string-length(normalize-space()) > 0][contains(., 'Ltd')]
Where contains() Is the Wrong Tool
When you mean equality. contains(., 'Price') also matches "Historic Price" and "Price excluding VAT". If you want exactly that label, use normalize-space() = 'Price'. Over-matching is silent — your code takes the first result and never knows there were three.
When you are matching classes and nothing else. CSS does it properly and readably. See above.
When you need a regular expression. XPath 1.0 has none. Extract with contains() if you must, then apply a regex in your host language, where you can also see what matched.
When there is a usable identifier. An id, a data attribute, or embedded JSON-LD is more stable than any text match. Text is content, and content changes — a redesign, a translation, or a copy edit breaks a text-matched selector and nothing warns you.
When the string comes from user input. Interpolating untrusted text into an XPath expression is XPath injection. Use your library's variable binding where it exists, and escape properly where it does not — quotes in particular, since XPath 1.0 has no escape sequence for a quote inside a string literal and you have to use concat() to build one.
People Also Ask
What does contains() do in XPath?
It returns true when the first string argument contains the second as a substring. Both arguments are strings, the comparison is case-sensitive, and there are no wildcards or regular expressions. Its usual role in extraction is matching an element by part of its text or attribute value.
Why does my XPath contains() not find anything?
Most often because you passed it a node-set. XPath converts a node-set to a string by taking the first node in document order and ignoring the rest, so contains(//p, 'x') only ever examines the first paragraph. Apply the predicate per node instead: //p[contains(., 'x')].
What is the difference between contains(.) and contains(text())?
. uses the element's string-value, which the spec defines as the concatenation of all descendant text nodes — so it reaches into nested markup. text() returns direct text-node children, and string conversion takes only the first. Use . unless you specifically want to exclude nested content.
How do I match a class with XPath?
Use contains(concat(' ', normalize-space(@class), ' '), ' name '), which pads the attribute so only whole tokens match. A plain contains(@class, 'btn') also matches btn-primary and unbtn. If you are matching only on classes, a CSS selector is clearer and correct by default.
Is XPath contains() case-sensitive?
Yes, and XPath 1.0 has no lower-case() function. The standard workaround is translate() with explicit uppercase and lowercase alphabets, which handles ASCII only. Server-side libraries such as lxml support EXSLT regular expressions; browsers and Selenium do not.
How do I use contains() with multiple conditions?
Combine predicates with and and or: //div[contains(@class, 'card') and contains(., 'In stock')]. Each contains() is a separate boolean test evaluated against the same context node.
Does XPath have a starts-with or ends-with function?
starts-with() exists and is worth preferring over contains() where it fits, since it over-matches less. There is no ends-with() in XPath 1.0 — the workaround uses substring() with string-length() and is unpleasant enough that a different approach is usually better.
Why does contains() match more elements than expected?
Because it is a substring test with no notion of word boundaries. contains(., 'Price') matches "Historic Price" and "Price excluding VAT" too. Use normalize-space() = 'Price' for equality, or the padded-concat idiom for class tokens.
Wrapping Up
contains() is simple in specification and full of traps in use, and nearly all of them come from one place: XPath converts a node-set to a string by taking the first node and discarding the rest. That single rule explains why contains(//p, 'x') gives a confident wrong answer, why contains(text(), 'x') misses text split across nodes, and why the same expression works on one page and fails on the next.
The habits that avoid it are few. Apply contains() inside a predicate so it evaluates per node. Use . rather than text() unless you have a reason. Wrap text comparisons in normalize-space(), because real HTML is pretty-printed. And for class matching, either use the padded-concat idiom or — better — use a CSS selector, which was designed for exactly that job.
Reserve XPath for what it uniquely does: matching on text, and navigating backwards. Those are real capabilities with no CSS equivalent, and they are worth the syntax. Using it to select div.card is paying the cost without collecting the benefit.