Introduce temporary named expressions for match subjects

[sterliakov]

Fixes #18440. Fixes #17230. Fixes #16650. Improves behavior in #14731 (but still thinks that match is non-exhaustive, "missing return" false positive remains).

#16503 did this specifically for CallExpr, but that isn't the only kind of such statements. I propose to expand this for more general expressions and believe that a blacklist is more reasonable here: we do not want to introduce a temporary name only for certain expressions that either are already named or can be used to infer contained variables (inline tuple/list/dict/set literals).

Writing logic to generate a name for every other kind of expression would be quite cumbersome - I circumvent this by using a simple counter to generate unique names on demand.

[sterliakov]

Whoa, I expected this to produce at least a couple of unused ignore diagnostics, but apparently pattern matching isn't very popular in wild...

[sterliakov]

This interferes with recently added narrowing by item/attribute (aka discriminated union, testMatchNarrowingUnionClassViaAttribute). So a simple solution like this won't fly. This information for index/attr should be stored in some other way to support narrowing of attr/item and the object itself at the same time. This may still work for other, less informative nodes (await?)

[sterliakov]

To support narrowing the subject and its "dependencies" simultaneously, I propose to just push the narrowed type of subject to typemap as the type of its original form, allowing for any inference on it.

[ilevkivskyi]

LG, but I have one suggestion

[ilevkivskyi]

IIUC match statement handling re-uses binder in some code paths. Then I I think instead of this list we should do the same that binder does:

from mypy.literals import literal  # name is confusing, but this is what you want
if not literal(expr):
    <make dummy name>

[sterliakov]

Nice idea, but are you sure?

mypy/mypy/literals.py

Lines 117 to 124 in fc991a0

	def literal(e: Expression) -> int:
	"""Return the literal kind for an expression."""
	if isinstance(e, ComparisonExpr):
	return min(literal(o) for o in e.operands)
	elif isinstance(e, OpExpr):
	return min(literal(e.left), literal(e.right))

This will break testMatchOperations case added here, for example, as 1 < 2 will return LITERAL_YES, and we do not store types of comparison exprs in binder AFAIC.

[ilevkivskyi]

Right, binder also does isinstance(expr, (IndexExpr, MemberExpr, NameExpr)) before checking for literal(). But then this list will not be simplified much. Even then probably it would make sense to somehow re-use binder logic here.

Btw I am not super-confident about AssignmentExpr, what about something like x := y[z] (where z is not a literal)? This will have troubles with binder.

[ilevkivskyi]

Sorry, I meant the opposite x[y] := z with non-literal y.

[sterliakov]

Sorry, I meant the opposite x[y] := z with non-literal y

Did I miss a new PEP? If I didn't, this should be reported elsewhere as a SyntaxError...

[sterliakov]

Maybe we can add a common ancestor to {Str,Bytes,Int,Float,Complex}Expr? Semantically they should have a "primitive literal" superclass, then this list will look much saner - just "name or primitive".

[sterliakov]

Hm. I only decided to allow primitive literals as an optimization, but nothing will break if we store their types. Matching of primitive literals sounds like something no one should be doing (often). Let's remove them altogether!

[ilevkivskyi]

What I wanted is something like this:

        if isinstance(subject, (IndexExpr, MemberExpr, NameExpr) and literal(subject):
            <use existing expression>
        else:
            <create fake name>

plus an explicit comment that this should be kept in sync with corresponding logic in binder. This way it will be easier to spot a bug if something changes in binder.

[sterliakov]

Hm, okay, you finally made me realize that the problem with walrus is not situated here - thanks! I think the new version should be closer.

[github-actions]

According to mypy_primer, this change doesn't affect type check results on a corpus of open source code. ✅