Logical operators in Python are lazy. With the following definition:
def func(s):
print(s)
return True
calling the or
operator
>>> func('s') or func('t')
's'
True
only evaluates the first function call, because or
recognizes that the expression evaluates to
True
, regardless of the return value of the second function call. and
behaves analogously.
However, when using any()
(analogously: all()
) in the following way:
>>> any([func('s'), func('t')])
's'
't'
True
all function calls are evaluated, because the inner list is constructed first, before any
starts to iterate over the boolean values of its items.
That way we lose the power of any
being short-circuit, which means that it breaks as soon as the first element of the iterable is truish. If the function calls are expensive, evaluating all the functions up front is a big loss and is a waste of this ability of any
. In some sense, one could call this a Python gotcha, because it might be unexpected for users trying to leverage this feature of any
, and because any
is often thought as being just another syntactic way of chaining a sequence of or
statements. But any
is just short-circuit, not lazy, and that is a difference here.
any
accepts an iterable. So, there should be a way of creating an iterator which does not evaluate its elements up front but pass them unevaluated to any
and lets them evaluate inside of any
only, in order to achieve a fully lazy evaluation.
So, the question is: How can we use any
with truly lazy function evaluation? That means: How can we make an iterator of function calls which any
can consume, without evaluating all the function calls in advance?
We can use a generator expression, passing the functions and their arguments separately and evaluating only in the generator like so:
>>> any(func(arg) for arg in ('s', 't'))
's'
True
For different functions with different signatures, this could look like the following:
any(
f(*args)
for f, args in [(func1, ('s',)), (func2, (1, 't'))]
)
That way, any
will stop iterating over the generator as soon as one function call evaluates to True
, and that means that the function evaluation is fully lazy.
Another neat way to postpone the function evaluation is to use lambda expressions, like so:
>>> any(
... f()
... for f in [lambda: func('s'), lambda: func('t')]
... )
's'
True
It is sad that any()
and all()
don’t have the logical functionality and are limited by this somewhat artificial constraint. Instead, a once-thru loop construction can be handy, particularly if there are intermediate results that need to be generated and used. This is related also to a function with early returns.
for _ in range(1):
val = func1()
if not val:
break
val1 = intermediate_func1(val)
if not func3(val1):
break
val2 = intermediate_func2(val1)
if not func4(val2):
break
result = the_really_expensive_function(val, val2)
if result:
return True, result
return False, None
similar construct using a function and early returns.
def example():
val = func1()
if not val:
return False, None
val1 = intermediate_func1(val)
if not func3(val1):
return False, None
val2 = intermediate_func2(val1)
if not func4(val2):
return False, None
result = the_really_expensive_function(val, val2)
if result:
return True, result
return False, None
What I wanted to use (but can’t; and this is only feasible with :=
operator):
if all(
val := func1(),
func3(val1 := intermediate_func1(val)),
func4(val2 := intermediate_func2(val1)),
result := the_really_expensive_function(val, val2),
):
return True, result
return False, None
Maybe in the future this will be feasible.
There are all manner of iterator producing things in Python. For example
map()
:any(map(func, ('s', 't')))
. Most people seem to have the opposite problem in Python3 — they want list outputs and python gives them generators and maps!This has nothing to do with
any
. The evaluations happen while constructing the list.