[VPlan] Skip disconnected exit blocks in hasEarlyExit.

[fhahn]

Currently hasEarlyExit returns true, if there are multiple exit blocks. ExitBlocks contains the wrapped original IR exit blocks. Without checking the predecessors we incorrectly return true for loops with multiple countable exits, that have been vectorized by requiring a scalar epilogue. In that case, the exit blocks will get disconnected.

Fix this by filtering out disconnected exit blocks.

Currently this should only impact the 'early-exit vectorized' statistic.

[llvmbot]

@llvm/pr-subscribers-vectorizers

@llvm/pr-subscribers-llvm-transforms

Author: Florian Hahn (fhahn)

Changes

Currently hasEarlyExit returns true, if there are multiple exit blocks. ExitBlocks contains the wrapped original IR exit blocks. Without checking the predecessors we incorrectly return true for loops with multiple countable exits, that have been vectorized by requiring a scalar epilogue. In that case, the exit blocks will get disconnected.

Fix this by filtering out disconnected exit blocks.

Currently this should only impact the 'early-exit vectorized' statistic.

---

Full diff: https://github.com/llvm/llvm-project/pull/151718.diff

2 Files Affected:

• (modified) llvm/lib/Transforms/Vectorize/VPlan.h (+4-1)
• (modified) llvm/test/Transforms/LoopVectorize/vect.stats.ll (+1-1)

diff --git a/llvm/lib/Transforms/Vectorize/VPlan.h b/llvm/lib/Transforms/Vectorize/VPlan.h
index a5de5933d5ff1..758aea48eb930 100644
--- a/llvm/lib/Transforms/Vectorize/VPlan.h
+++ b/llvm/lib/Transforms/Vectorize/VPlan.h
@@ -4229,7 +4229,10 @@ class VPlan {
   /// block with multiple predecessors (one for the exit via the latch and one
   /// via the other early exit).
   bool hasEarlyExit() const {
-    return ExitBlocks.size() > 1 ||
+    return count_if(ExitBlocks,
+                    [](VPIRBasicBlock *EB) {
+                      return EB->getNumPredecessors() != 0;
+                    }) > 1 ||
            (ExitBlocks.size() == 1 && ExitBlocks[0]->getNumPredecessors() > 1);
   }
 
diff --git a/llvm/test/Transforms/LoopVectorize/vect.stats.ll b/llvm/test/Transforms/LoopVectorize/vect.stats.ll
index 033907e079a7d..e3240c8181519 100644
--- a/llvm/test/Transforms/LoopVectorize/vect.stats.ll
+++ b/llvm/test/Transforms/LoopVectorize/vect.stats.ll
@@ -5,7 +5,7 @@
 ; vectorized) and the third one is not.
 
 ; CHECK: 4 loop-vectorize               - Number of loops analyzed for vectorization
-; CHECK: 2 loop-vectorize               - Number of early exit loops vectorized
+; CHECK: 1 loop-vectorize               - Number of early exit loops vectorized
 ; CHECK: 3 loop-vectorize               - Number of loops vectorized
 
 target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64-S128"

[annamthomas]

good catch! Thank you.

I was using this statistic to check how many loops we early-exit vectorize (downstream).