kernel/git/stable/linux.git/tools/perf/util/maps.h, branch master

perf maps: Add maps__mutate_mapping

2026-06-15T12:01:27+00:00

During kernel ELF symbol parsing (dso__process_kernel_symbol), proc kallsyms image loading (dso__load_kernel_sym, dso__load_guest_kernel_sym), and dynamic kernel memory map alignment updates (machine__update_kernel_mmap), the loader directly modifies live virtual address boundary keys fields on map objects. If these boundaries are mutated while the map pointer actively resides inside the parent maps cache array list (kmaps) outside of any lock closure, an unsafe concurrent window is exposed where parallel worker lookup threads (e.g., inside perf top) can mistakenly assume the cache remains sorted based on stale parameters, executing binary search queries (bsearch) across an unsorted range and triggering lookup failures. Fix this by introducing maps__mutate_mapping() that explicitly acquires the parent maps write semaphore lock, executes an incoming mutation callback block to perform the field updates under lock protection, and invalidates the sorted tracking flags prior to releasing the write lock. This guarantees synchronization invariants, closing the concurrent lookup race window. The adjacent module alignment pass inside machine__create_kernel_maps() is safely preserved as a high-performance lockless pass, as its invocation lifecycle bounds remain strictly single-threaded by contract during session initialization construction. To safely support this unconditional down_write write lock mutator without recursive read-to-write self-deadlock upgrades during lazy symbol loading, we introduce a public maps__load_maps() API. It copies map pointers under a brief read lock and force-loads all modules locklessly outside the lock. Callers (such as perf inject) must pre-load all kernel symbol maps up front at startup using maps__load_maps(), completely bypassing dynamic runtime mutations. Fixes: 39b12f781271 ("perf tools: Make it possible to read object code from vmlinux") Assisted-by: Antigravity:gemini-3.1-pro Signed-off-by: Ian Rogers Tested-by: James Clark Cc: Adrian Hunter Cc: Gabriel Marin Cc: Ingo Molnar Cc: Jiri Olsa Cc: Namhyung Kim Cc: Peter Zijlstra Signed-off-by: Arnaldo Carvalho de Melo

perf unwind-libunwind: Remove libunwind-local

2026-05-15T19:19:50+00:00

Local unwinding only works on the machine libunwind is built for, rather than cross platform, the APIs for remote and local unwinding are similar but types like unw_word_t depend on the included header. Place the architecture specific code into the appropriate libunwind-.c file. Put generic code in unwind-libunwind.c and use libunwind-arch to choose the correct implementation based on the thread's e_machine. Structuring the code this way avoids including the unwind-libunwind-local.c for each architecture of remote unwinding. Data is moved into the struct unwind_info to simplify the architecture and generic code, trying to keep as much code as possible generic. Signed-off-by: Ian Rogers Cc: Adrian Hunter Cc: Albert Ou Cc: Alexander Shishkin Cc: Alexandre Ghiti Cc: Andrew Jones Cc: Athira Rajeev Cc: Dapeng Mi Cc: Dmitrii Dolgov <9erthalion6@gmail.com> Cc: Florian Fainelli Cc: Howard Chu Cc: Ingo Molnar Cc: James Clark Cc: Jiri Olsa Cc: John Garry Cc: Leo Yan Cc: Li Guan Cc: Namhyung Kim Cc: Palmer Dabbelt Cc: Paul Walmsley Cc: Peter Zijlstra Cc: Shimin Guo Cc: Thomas Richter Cc: Tomas Glozar Cc: Will Deacon Signed-off-by: Arnaldo Carvalho de Melo

perf unwind-libunwind: Move flush/finish access out of local

2026-05-15T19:19:50+00:00

Flush and finish access are relatively simple calls into libunwind, move them out struct unwind_libunwind_ops. So that the correct version can be called, add an e_machine variable to maps. This size regression will go away when the unwind_libunwind_ops no longer need stashing in the maps. To set the e_machine up pass it into unwind__prepare_access, which no longer needs to determine the unwind operations based on a map dso because of this. This also means the maps copying code can call unwind__prepare_access once for the e_machine rather than once per map. Signed-off-by: Ian Rogers Cc: Adrian Hunter Cc: Albert Ou Cc: Alexander Shishkin Cc: Alexandre Ghiti Cc: Andrew Jones Cc: Athira Rajeev Cc: Dapeng Mi Cc: Dmitrii Dolgov <9erthalion6@gmail.com> Cc: Florian Fainelli Cc: Howard Chu Cc: Ingo Molnar Cc: James Clark Cc: Jiri Olsa Cc: John Garry Cc: Leo Yan Cc: Li Guan Cc: Namhyung Kim Cc: Palmer Dabbelt Cc: Paul Walmsley Cc: Peter Zijlstra Cc: Shimin Guo Cc: Thomas Richter Cc: Tomas Glozar Cc: Will Deacon Signed-off-by: Arnaldo Carvalho de Melo

perf unwind-libdw: Don't discard loaded ELF/DWARF after every unwind

2026-01-20T18:43:56+00:00

The unwind-libdw dwfl has ELF binaries associated with mmap addresses. Experimenting with using the per dso dwfl it is required to alter the address to be 0 based variant. Unfortunately libdwfl doesn't allow a single unwind and then an update to the return address to be 0 based as there are assertions that registers aren't updated once an unwind has started, etc. As removing the dwfl didn't prove possible, an alternative is to just not discard the dwfl when the unwind ends. The dwfl is valid for a process unless a dso is loaded at the same address as a previous one. So keep the dwfl with the maps, invalidate it if a map is removed (in case a new map replaces it) and recycle the dwfl in the unwinding code. A wrinkly in the implementation of this is that the attached thread argument is remembered by the dwfl and so it needs to be a pointer to memory that also persists with the dwfl (struct dwfl_ui_thread_info in the code). Recording 10 seconds of system wide data with --call-graph=dwarf and then processing with perf report shows a total runtime improvement from 41.583s to 2.279s (an 18x speedup). Signed-off-by: Ian Rogers Cc: Aditya Bodkhe Cc: Adrian Hunter Cc: Albert Ou Cc: Alexandre Ghiti Cc: Andi Kleen Cc: Athira Rajeev Cc: Chun-Tse Shao Cc: Dmitriy Vyukov Cc: Dr. David Alan Gilbert Cc: Guo Ren Cc: Haibo Xu Cc: Howard Chu Cc: Ingo Molnar Cc: James Clark Cc: Jiri Olsa Cc: John Garry Cc: Krzysztof Łopatowski Cc: Leo Yan Cc: Mark Wielaard Cc: Namhyung Kim Cc: Palmer Dabbelt Cc: Paul Walmsley Cc: Peter Zijlstra Cc: Sergei Trofimovich Cc: Shimin Guo Cc: Stephen Brennan Cc: Thomas Falcon Cc: Will Deacon Signed-off-by: Arnaldo Carvalho de Melo

perf maps: Locking tidy up of nr_maps

2024-02-12T20:35:41+00:00

After this change maps__nr_maps is only used by tests, existing users are migrated to maps__empty. Compute maps__empty under the read lock. Signed-off-by: Ian Rogers Acked-by: Namhyung Kim Cc: K Prateek Nayak Cc: James Clark Cc: Vincent Whitchurch Cc: Alexey Dobriyan Cc: Colin Ian King Cc: Changbin Du Cc: Masami Hiramatsu Cc: Song Liu Cc: Leo Yan Cc: Athira Rajeev Cc: Liam Howlett Cc: Artem Savkov Cc: bpf@vger.kernel.org Signed-off-by: Namhyung Kim Link: https://lore.kernel.org/r/20240210031746.4057262-7-irogers@google.com

perf maps: Hide maps internals

2024-02-12T20:35:41+00:00

Move the struct into the C file. Add maps__equal to work around exposing the struct for reference count checking. Add accessors for the unwind_libunwind_ops. Move maps_list_node to its only use in symbol.c. Signed-off-by: Ian Rogers Acked-by: Namhyung Kim Cc: K Prateek Nayak Cc: James Clark Cc: Vincent Whitchurch Cc: Alexey Dobriyan Cc: Colin Ian King Cc: Changbin Du Cc: Masami Hiramatsu Cc: Song Liu Cc: Leo Yan Cc: Athira Rajeev Cc: Liam Howlett Cc: Artem Savkov Cc: bpf@vger.kernel.org Signed-off-by: Namhyung Kim Link: https://lore.kernel.org/r/20240210031746.4057262-6-irogers@google.com

perf maps: Switch from rbtree to lazily sorted array for addresses

2024-02-12T20:35:14+00:00

Maps is a collection of maps primarily sorted by the starting address of the map. Prior to this change the maps were held in an rbtree requiring 4 pointers per node. Prior to reference count checking, the rbnode was embedded in the map so 3 pointers per node were necessary. This change switches the rbtree to an array lazily sorted by address, much as the array sorting nodes by name. 1 pointer is needed per node, but to avoid excessive resizing the backing array may be twice the number of used elements. Meaning the memory overhead is roughly half that of the rbtree. For a perf record with "--no-bpf-event -g -a" of true, the memory overhead of perf inject is reduce fom 3.3MB to 3MB, so 10% or 300KB is saved. Map inserts always happen at the end of the array. The code tracks whether the insertion violates the sorting property. O(log n) rb-tree complexity is switched to O(1). Remove slides the array, so O(log n) rb-tree complexity is degraded to O(n). A find may need to sort the array using qsort which is O(n*log n), but in general the maps should be sorted and so average performance should be O(log n) as with the rbtree. An rbtree node consumes a cache line, but with the array 4 nodes fit on a cache line. Iteration is simplified to scanning an array rather than pointer chasing. Overall it is expected the performance after the change should be comparable to before, but with half of the memory consumed. To avoid a list and repeated logic around splitting maps, maps__merge_in is rewritten in terms of maps__fixup_overlap_and_insert. maps_merge_in splits the given mapping inserting remaining gaps. maps__fixup_overlap_and_insert splits the existing mappings, then adds the incoming mapping. By adding the new mapping first, then re-inserting the existing mappings the splitting behavior matches. Signed-off-by: Ian Rogers Acked-by: Namhyung Kim Cc: K Prateek Nayak Cc: James Clark Cc: Vincent Whitchurch Cc: Alexey Dobriyan Cc: Colin Ian King Cc: Changbin Du Cc: Masami Hiramatsu Cc: Song Liu Cc: Leo Yan Cc: Athira Rajeev Cc: Liam Howlett Cc: Artem Savkov Cc: bpf@vger.kernel.org Signed-off-by: Namhyung Kim Link: https://lore.kernel.org/r/20240210031746.4057262-2-irogers@google.com

perf maps: Reduce scope of map_rb_node and maps internals

2023-12-20T18:02:17+00:00

Avoid exposing the implementation of maps so that the internals can be refactored. Signed-off-by: Ian Rogers Cc: Adrian Hunter Cc: Alexander Shishkin Cc: Andi Kleen Cc: Athira Rajeev Cc: Changbin Du Cc: Colin Ian King Cc: Dmitrii Dolgov <9erthalion6@gmail.com> Cc: German Gomez Cc: Guilherme Amadio Cc: Huacai Chen Cc: Ingo Molnar Cc: James Clark Cc: Jiri Olsa Cc: K Prateek Nayak Cc: Kajol Jain Cc: Kan Liang Cc: Leo Yan Cc: Li Dong Cc: Liam Howlett Cc: Mark Rutland Cc: Masami Hiramatsu Cc: Miguel Ojeda Cc: Ming Wang Cc: Namhyung Kim Cc: Nick Terrell Cc: Paolo Bonzini Cc: Peter Zijlstra Cc: Ravi Bangoria Cc: Sandipan Das Cc: Sean Christopherson Cc: Steinar H. Gunderson Cc: Vincent Whitchurch Cc: Wenyu Liu Cc: Yang Jihong Link: https://lore.kernel.org/r/20231207011722.1220634-22-irogers@google.com Signed-off-by: Arnaldo Carvalho de Melo

perf maps: Add find next entry to give entry after the given map

2023-12-20T18:02:02+00:00

Use to remove map_rb_node use from machine.c. Signed-off-by: Ian Rogers Cc: Adrian Hunter Cc: Alexander Shishkin Cc: Andi Kleen Cc: Athira Rajeev Cc: Changbin Du Cc: Colin Ian King Cc: Dmitrii Dolgov <9erthalion6@gmail.com> Cc: German Gomez Cc: Guilherme Amadio Cc: Huacai Chen Cc: Ingo Molnar Cc: James Clark Cc: Jiri Olsa Cc: K Prateek Nayak Cc: Kajol Jain Cc: Kan Liang Cc: Leo Yan Cc: Li Dong Cc: Liam Howlett Cc: Mark Rutland Cc: Masami Hiramatsu Cc: Miguel Ojeda Cc: Ming Wang Cc: Namhyung Kim Cc: Nick Terrell Cc: Paolo Bonzini Cc: Peter Zijlstra Cc: Ravi Bangoria Cc: Sandipan Das Cc: Sean Christopherson Cc: Steinar H. Gunderson Cc: Vincent Whitchurch Cc: Wenyu Liu Cc: Yang Jihong Link: https://lore.kernel.org/r/20231207011722.1220634-21-irogers@google.com Signed-off-by: Arnaldo Carvalho de Melo

perf maps: Add maps__load_first()

2023-12-20T17:56:10+00:00

Avoid bpf_lock_contention_read touching the internal maps data structure by adding a helper function. As access is done directly on the map in maps, hold the read lock to stop it being removed. Signed-off-by: Ian Rogers Cc: Adrian Hunter Cc: Alexander Shishkin Cc: Andi Kleen Cc: Athira Rajeev Cc: Changbin Du Cc: Colin Ian King Cc: Dmitrii Dolgov <9erthalion6@gmail.com> Cc: German Gomez Cc: Guilherme Amadio Cc: Huacai Chen Cc: Ingo Molnar Cc: James Clark Cc: Jiri Olsa Cc: K Prateek Nayak Cc: Kajol Jain Cc: Kan Liang Cc: Leo Yan Cc: Li Dong Cc: Liam Howlett Cc: Mark Rutland Cc: Masami Hiramatsu Cc: Miguel Ojeda Cc: Ming Wang Cc: Namhyung Kim Cc: Nick Terrell Cc: Paolo Bonzini Cc: Peter Zijlstra Cc: Ravi Bangoria Cc: Sandipan Das Cc: Sean Christopherson Cc: Steinar H. Gunderson Cc: Vincent Whitchurch Cc: Wenyu Liu Cc: Yang Jihong Link: https://lore.kernel.org/r/20231207011722.1220634-20-irogers@google.com Signed-off-by: Arnaldo Carvalho de Melo