Transformers Overview
Transform, clean, parse, and re-format DOM data with composable transformers.
What are Transformers?
By default, CSS selectors extract the innerText of matching HTML elements as a string.
In real-world web data extraction, raw text often needs formatting: trimming whitespace, casting to numbers, parsing embedded JSON, extracting attribute values, or resolving relative URLs. Transformers provide a declarative, chained pipeline to perform these manipulations.
selector: .price
transform:
- trim
- 'replace("$", "")'
- numberArgument Syntax & Quoting Rules
Transformers support parameter syntax with specific quoting rules:
-
Bare (unquoted) arguments: Simple identifiers matching
[\w-]+(letters, digits, underscores, hyphens) can be written without quotes:attr(href)trimsplit(-)
-
Quoted arguments (
"..."or'...'): Arguments containing spaces, commas, colons, or regex metacharacters must be enclosed in single or double quotes:replace("\s+", " ")split(", ")join(" - ")urlQueryParam("utm_source")
-
Escaping inside quotes: A backslash escapes the enclosing quote character (
\"inside"..."or\'inside'...') or a literal backslash (\\). Regex escape sequences like\s,\d, and\ware passed untouched to regex engines. -
YAML quoting tip: In YAML configurations, enclose the transform expression in quotes to avoid syntax collisions with YAML parsers:
transform: 'replace("\s+", " ")'
Available Transformers
| Transformer | Description |
|---|---|
attr | Extract element attributes |
boolean | Cast values to boolean using truthy semantics |
capitalize | Uppercase the first character, lowercase the rest |
exists | Check whether a selector matched any element |
html | Extract innerHTML of the element |
join | Join array elements into a string with a delimiter |
json | Parse JSON string into an object or array |
jsonpath | Query JSON structures using JSONPath syntax |
length | Count array items, object keys, or string length |
lower | Convert string to lowercase |
number | Parse string into a numeric value |
removeLastPathSection | Remove the trailing segment from a URL path |
removeUrlQueryParam | Remove query parameters from a URL |
replace | Replace substrings or regex patterns |
resolve | Resolve relative URLs against the document base URL |
split | Split string into an array by delimiter or whitespace |
trim | Remove leading and trailing whitespace |
upper | Convert string to uppercase |
urlQueryParam | Extract query parameters from a URL string |