Signature
Parameters
ColumnOrName
required
Boolean expression string to evaluate
Variable name and value pairs (var_name1, var_value1, var_name2, var_value2, …)
Returns
Column
Boolean result of the expression evaluation
Other
barrier (Python)
Prevent filter pushdown and control predicate evaluation order in complex spatial joins.
Prevent filter pushdown and control predicate evaluation order in complex spatial joins.
This function creates an optimization barrier by evaluating boolean expressions at runtime.
Example:
df.where(barrier(‘rating > 4.0 AND stars >= 4’,
‘rating’, col(‘r.rating’),
‘stars’, col(‘h.stars’)))