[DWARF] Speedup .gdb_index dumping

[itrofimow]

This patch drastically speed ups dumping .gdb_index for large indexes

[llvmbot]

@llvm/pr-subscribers-debuginfo

Author: None (itrofimow)

Changes

This patch drastically speed ups dumping .gdb_index for large indexes

---

Full diff: https://github.com/llvm/llvm-project/pull/151806.diff

1 Files Affected:

• (modified) llvm/lib/DebugInfo/DWARF/DWARFGdbIndex.cpp (+20-5)

diff --git a/llvm/lib/DebugInfo/DWARF/DWARFGdbIndex.cpp b/llvm/lib/DebugInfo/DWARF/DWARFGdbIndex.cpp
index 987e63963a068..c0ad2a38df373 100644
--- a/llvm/lib/DebugInfo/DWARF/DWARFGdbIndex.cpp
+++ b/llvm/lib/DebugInfo/DWARF/DWARFGdbIndex.cpp
@@ -17,6 +17,7 @@
 #include <cinttypes>
 #include <cstdint>
 #include <set>
+#include <unordered_map>
 #include <utility>
 
 using namespace llvm;
@@ -60,6 +61,24 @@ void DWARFGdbIndex::dumpSymbolTable(raw_ostream &OS) const {
                ", filled slots:",
                SymbolTableOffset, (uint64_t)SymbolTable.size())
      << '\n';
+
+  std::unordered_map<uint32_t, decltype(ConstantPoolVectors)::const_iterator>
+      CuVectorMap{};
+  CuVectorMap.reserve(ConstantPoolVectors.size());
+  const auto FindCuVector =
+      [&CuVectorMap, notFound = ConstantPoolVectors.end()](uint32_t vecOffset) {
+        const auto it = CuVectorMap.find(vecOffset);
+        if (it != CuVectorMap.end()) {
+          return it->second;
+        }
+
+        return notFound;
+      };
+  for (auto it = ConstantPoolVectors.begin(); it != ConstantPoolVectors.end();
+       ++it) {
+    CuVectorMap.emplace(it->first, it);
+  }
+
   uint32_t I = -1;
   for (const SymTableEntry &E : SymbolTable) {
     ++I;
@@ -72,11 +91,7 @@ void DWARFGdbIndex::dumpSymbolTable(raw_ostream &OS) const {
     StringRef Name = ConstantPoolStrings.substr(
         ConstantPoolOffset - StringPoolOffset + E.NameOffset);
 
-    auto CuVector = llvm::find_if(
-        ConstantPoolVectors,
-        [&](const std::pair<uint32_t, SmallVector<uint32_t, 0>> &V) {
-          return V.first == E.VecOffset;
-        });
+    auto CuVector = FindCuVector(E.VecOffset);
     assert(CuVector != ConstantPoolVectors.end() && "Invalid symbol table");
     uint32_t CuVectorId = CuVector - ConstantPoolVectors.begin();
     OS << format("      String name: %s, CU vector index: %d\n", Name.data(),

[itrofimow]

I have a binary with gdb-index of size ~250Mb, and for that binary llvm-dwarfdump --gdb-index takes basically forever (10+ minutes) to complete.
With the patch applied it takes ~5s

[dwblaikie]

Thanks for the improvement!

Hmm, actually at a high level: I guess this ConstantPoolVectors isn't sorted, is it? So we can't do a binary search... could we sort it? I guess not - since we do want to dump it in a way that matches the input too (in case the on-disk ordering is important to debugging the data at some point)?

(oh, and high level question, if you're interested/able: What's your interest in gdb_index? Myself, I've worked on various indexing solutions at Google due to the large size of single binaries we have, for a while but we rarely see traction/interest in these tools outside of Google - so it's always interesting to make friends with folks who are facing similar problems)

[dwblaikie]

Probably drop these {} as they're redundant.

[dwblaikie]

Probably use a DenseMap? ( https://llvm.org/docs/ProgrammersManual.html#picking-the-right-data-structure-for-a-task )

[dwblaikie]

LLVM's coding style skips {} on single-line blocks, and reduces scope where possible, so probably something like:

if (auto it = CuVectorMap.find(vecOffset); it != CuVectorMap.end())
  return it->second;

[dwblaikie]

I probably wouldn't bother separately capturing things like this in a locally scoped non-erased functor. Probably just use [&] and return ConstantPoolVectors.end(); in the fail-case.

[dwblaikie]

Could drop the {} here.

Could use a range-based for loop, and instead of putting iterators as values in the map, use pointers (then you can get a pointer to the value in the range based for loop where there aren't any visible/name-able iterators)