Selector-based scraping extracts values from HTML elements using a rule such as a CSS selector or XPath expression. It works best when pages share a stable layout and the values already appear in the fetched HTML.
For example, h1 selects heading elements, while .product-price selects elements with a product-price class. A regular expression is a separate text-pattern method; it does not navigate the HTML tree like a selector.
CSS selector examples for scraping
These examples describe a hypothetical product page. Inspect the actual site's HTML before using them; they are not universal product selectors.
| Selector | Matches | Useful extraction |
|---|---|---|
h1 | Heading elements | Text for a page title |
.product-price | Elements with the product-price class | Displayed price text |
#product-details | An element with that ID | Description text or HTML |
.product-card a | Links inside product-card elements | The href attribute for product URLs |
meta[name="description"] | A matching meta element | The content attribute for the meta description |
Use the MDN selector reference to check syntax. The selector engine in your scraper may support a different subset than a browser; test the expression on representative fetched pages.
Selector scraping in Datablist
Use Bulk Scraper for CSS selectors or regular expressions. This enrichment takes existing rows with a page URL; it is different from a source that generates one row per product.
- Import your URLs and map the URL column to Url to parse.
- Add a rule under Css Selectors, enter the CSS Selector, and give the output a useful Name.
- Choose CSS Selector Content: inner text, raw HTML, or an attribute. For an attribute, set Selector Attribute to the name, such as
hreforcontent. - Map the result and its status to separate properties, then run a small sample.
For a page with “EUR 45” in .product-price, inner-text extraction returns that text. An attribute extraction from a link can return a relative path such as /products/sensor-200; resolve it against the source URL before treating it as a full URL.
If several elements match, Bulk Scraper joins their nonempty values with semicolons in the output. It does not automatically turn each matching card into a separate row. Independently extracted lists can also become misaligned when one card lacks a price or link. For repeated records, use a workflow that keeps each record's fields together, such as Website AI Scraper.
Troubleshooting empty or incorrect results
Check both URL Scraping Status and the status for the individual selector. A page fetch can succeed while a selector finds no matching element or only empty text.
Inspect the fetched layout when a rule fails. A changed class, an overly broad selector, or a missing attribute can change the result. When the content requires JavaScript, Render HTML is available with Use Proxy when needed enabled. Rendering can change costs and does not guarantee access or make an incorrect selector work.
Test several page layouts before increasing volume. Preserve the source URL beside every result and sample the output after site changes. For variable layouts or semantic extraction, consider AI web scraping; for separate results pages, plan pagination explicitly.