kernel/git/stable/linux.git/Documentation/vm/page_owner.rst, branch master

docs: rename Documentation/vm to Documentation/mm

2022-06-27T19:52:53+00:00

so it will be consistent with code mm directory and with Documentation/admin-guide/mm and won't be confused with virtual machines. Signed-off-by: Mike Rapoport Suggested-by: Matthew Wilcox Tested-by: Ira Weiny Acked-by: Jonathan Corbet Acked-by: Wu XiangCheng

docs: vm/page_owner: tweak literal block in STANDARD FORMAT SPECIFIERS

2022-05-13T14:20:18+00:00

A semantic conflict between commit 5603f9bdea68 ("docs: vm/page_owner: use literal blocks for param description") and a change queued for v5.19 authored by Jiajian Ye ("tools/vm/page_owner_sort.c: support sorting blocks by multiple keys") results in a warning from "make htmldocs" saying: [...]/vm/page_owner.rst:176: WARNING: Literal block expected; none found. This is because a literal block in ReST ends at a line which has the same indent as the paragraph preceding it. In this case the one with no indent. Indent the two "For --xxxx option:" lines by two columns and make the whole section a literal block. While at it, fix indents by white spaces of "ator" keys. Link: https://lkml.kernel.org/r/fdfecc82-d41e-6d8a-738d-4beb6faa27fb@gmail.com Signed-of-by: Akira Yokosawa Reported-by: Shenghong Han Cc: Jiajian Ye Cc: Chongxi Zhao Cc: Yinan Zhang Cc: Yixuan Cao Cc: Yongqiang Liu Cc: Yuhong Feng Cc: Haowen Bai Cc: Jonathan Corbet Signed-off-by: Andrew Morton

tools/vm/page_owner_sort.c: provide allocator labelling and update --cull and --sort options

2022-04-29T06:15:57+00:00

An application is suspected of having memory leak when its memory consumption is high and keeps increasing. There are several commonly used memory allocators: slab, cma, vmalloc, etc. The memory leak identification can be sped up if the page information allocated by an allocator can be analyzed separately. This patch provides supports for memory allocator labelling for slab, vmalloc, and cma. The pages allocated by slab and cma can be confirmed from the "PFN" line according to the kernel codes, and the label of the vmalloc allocator can be obtained by analyzing the stack trace. Thanks for Vlastimil Babka's constructive suggestions. Based on Yinan Zhang's study, the call chain of vmalloc() is vmalloc() -> ... -> __vmalloc_node_range() -> __vmalloc_area_node(). __vmalloc_area_node() requests memory through the interface of buddy allocation system. In the current version, __vmalloc_area_node() uses four interfaces: alloc_pages_bulk_array_mempolicy(), alloc_pages_bulk_array_node(), alloc_pages() and alloc_pages_node(). By disassembling the code, we find that __vmalloc_area_node() is expanded in __vmalloc_node_range(). So __vmalloc_area_node is not in the stack trace. On the test machine, the stack trace of pages allocated by vmalloc has the following four forms: __alloc_pages_bulk+0x230/0x6a0 __vmalloc_node_range+0x19c/0x598 alloc_pages_bulk_array_mempolicy+0xbc/0x278 __vmalloc_node_range+0x1e8/0x598 __alloc_pages+0x160/0x2b0 __vmalloc_node_range+0x234/0x598 alloc_pages+0xac/0x150 __vmalloc_node_range+0x44c/0x598 Therefore, in two consecutive lines of stacktrace, if the first line contains the word "alloc_pages" and the second line contains the word "__vmalloc_node_range", it can be determined that the page is allocated by vmalloc. And the function offset and size are not the same on different machines, so there is no need to match them. At the same time, this patch updates the --cull and --sort options to support allocator-based merge statistics and sorting. The added functions are fully compatible with the original work. When using, you can use "allocator", or abbreviated as "ator". Relevant updates have also been made in the documentation(Documentation/vm/page_owner.rst). Example: ./page_owner_sort --cull=st,pid,name,allocator ./page_owner_sort --sort=ator,pid,name This work is coauthored by Jiajian Ye, Yinan Zhang, Shenghong Han, Chongxi Zhao, Yuhong Feng and Yongqiang Liu. Link: https://lkml.kernel.org/r/20220410132932.9402-1-caoyixuan2019@email.szu.edu.cn Signed-off-by: Yixuan Cao Cc: Chongxi Zhao Cc: Haowen Bai Cc: Jiajian Ye Cc: Sean Anderson Cc: Shenghong Han Cc: Yinan Zhang Cc: Yongqiang Liu Cc: Yuhong Feng Signed-off-by: Andrew Morton

tools/vm/page_owner_sort.c: support sorting blocks by multiple keys

2022-04-29T06:15:57+00:00

When viewing page owner information, we may want to sort blocks of information by multiple keys, since one single key does not uniquely identify a block. Therefore, following adjustments are made: 1. Add a new --sort option to support sorting blocks of information by multiple keys. ./page_owner_sort --sort= ./page_owner_sort --sort is a single argument in the form of a comma-separated list, which offers a way to specify sorting order. Sorting syntax is [+|-]key[,[+|-]key[,...]]. The ascending or descending order can be specified by adding the + (ascending, default) or - (descend -ing) prefix to the key: ./page_owner_sort [option] --sort -key1,+key2,key3... For example, to sort the blocks first by task command name in lexicographic order and then by pid in ascending numerical order, use the following: ./page_owner_sort --sort=name,+pid To sort the blocks first by pid in ascending order and then by timestamp of the page when it is allocated in descending order, use the following: ./page_owner_sort --sort=pid,-alloc_ts 2. Add explanations of a newly added --sort option in the function usage() and the document(Documentation/vm/page_owner.rst). This work is coauthored by Yixuan Cao Shenghong Han Yinan Zhang Chongxi Zhao Yuhong Feng Yongqiang Liu Link: https://lkml.kernel.org/r/20220401024856.767-3-yejiajian2018@email.szu.edu.cn Signed-off-by: Jiajian Ye Cc: Chongxi Zhao Cc: Shenghong Han Cc: Yinan Zhang Cc: Yixuan Cao Cc: Yongqiang Liu Cc: Yuhong Feng Cc: Haowen Bai Cc: Sean Anderson Signed-off-by: Andrew Morton

tools/vm/page_owner_sort.c: support for multi-value selection in single argument

2022-04-29T06:15:56+00:00

When viewing page owner information, we may want to select blocks whose PID/TGID/TASK_COMM_NAME appears in a user-specified list for data analysis and aggregation. But currently page_owner_sort only supports selecting blocks associated with only one specified PID/TGID/TASK_COMM_NAME. Therefore, following adjustments are made to fix the problem: 1. Enhance selecting function to support the selection of multiple PIDs/TGIDs/TASK_COMM_NAMEs. The enhanced usages are as follows: --pid Select by pid. This selects the blocks whose PID numbers appear in . --tgid Select by tgid. This selects the blocks whose TGID numbers appear in . --name Select by task command name. This selects the blocks whose task command name appear in . Where , , are single arguments in the form of a comma-separated list,which offers a way to specify individual selecting rules. For example, if you want to select blocks whose tgids are 1, 2 or 3, you have to use 4 commands as follows: ./page_owner_sort --tgid=1 ./page_owner_sort --tgid=2 ./page_owner_sort --tgid=3 cat > With this patch, you can use only 1 command to obtain the same result as above: ./page_owner_sort --tgid=1,2,3 2. Update explanations of --pid, --tgid and --name in the function usage() and the document(Documents/vm/page_owner.rst). This work is coauthored by Yixuan Cao Shenghong Han Yinan Zhang Chongxi Zhao Yuhong Feng Yongqiang Liu Link: https://lkml.kernel.org/r/20220401024856.767-2-yejiajian2018@email.szu.edu.cn Signed-off-by: Jiajian Ye Cc: Chongxi Zhao Cc: Shenghong Han Cc: Yinan Zhang Cc: Yixuan Cao Cc: Yongqiang Liu Cc: Yuhong Feng Cc: Haowen Bai Cc: Sean Anderson Signed-off-by: Andrew Morton

docs: vm/page_owner: use literal blocks for param description

2022-04-27T20:28:48+00:00

Sphinx generates hard-to-read lists of parameters at the bottom of the page. Fix them by putting literal-block markers of "::" in front of them. Link: https://lkml.kernel.org/r/cfd3bcc0-b51d-0c68-c065-ca1c4c202447@gmail.com Signed-off-by: Akira Yokosawa Fixes: 57f2b54a9379 ("Documentation/vm/page_owner.rst: update the documentation") Cc: Shenghong Han Cc: Haowen Bai Cc: Jonathan Corbet Cc: Alex Shi Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds

doc/vm/page_owner.rst: remove content related to -c option

2022-04-01T18:46:09+00:00

-c option has been removed from page_owner_sort.c. Remove the usage of -c option from Documentation. This work is coauthored by Shenghong Han Yixuan Cao Chongxi Zhao Jiajian Ye Yuhong Feng Yongqiang Liu Link: https://lkml.kernel.org/r/20220326085920.1470081-2-zhangyinan2019@email.szu.edu.cn Signed-off-by: Yinan Zhang Cc: Jonathan Corbet Cc: Stephen Rothwell Cc: Sean Anderson Cc: Tang Bin Cc: Zhenliang Wei Cc: Georgi Djakov Cc: Chongxi Zhao Cc: Jiajian Ye Cc: Yixuan Cao Cc: Yuhong Feng Cc: Yongqiang Liu Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds

tools/vm/page_owner_sort.c: support for user-defined culling rules

2022-03-25T02:06:45+00:00

When viewing page owner information, we may want to cull blocks of information with our own rules. So it is important to enhance culling function to provide the support for customizing culling rules. Therefore, following adjustments are made: 1. Add --cull option to support the culling of blocks of information with user-defined culling rules. ./page_owner_sort --cull= ./page_owner_sort --cull is a single argument in the form of a comma-separated list to specify individual culling rules, by the sequence of keys k1,k2, .... Mixed use of abbreviated and complete-form of keys is allowed. For reference, please see the document(Documentation/vm/page_owner.rst). Now, assuming two blocks in the input file are as follows: Page allocated via order 0, mask xxxx, pid 1, tgid 1 (task_name_demo) PFN xxxx prep_new_page+0xd0/0xf8 get_page_from_freelist+0x4a0/0x1290 __alloc_pages+0x168/0x340 alloc_pages+0xb0/0x158 Page allocated via order 0, mask xxxx, pid 32, tgid 32 (task_name_demo) PFN xxxx prep_new_page+0xd0/0xf8 get_page_from_freelist+0x4a0/0x1290 __alloc_pages+0x168/0x340 alloc_pages+0xb0/0x158 If we want to cull the blocks by stacktrace and task command name, we can use this command: ./page_owner_sort --cull=stacktrace,name The output would be like: 2 times, 2 pages, task_comm_name: task_name_demo prep_new_page+0xd0/0xf8 get_page_from_freelist+0x4a0/0x1290 __alloc_pages+0x168/0x340 alloc_pages+0xb0/0x158 As we can see, these two blocks are culled successfully, for they share the same pid and task command name. However, if we want to cull the blocks by pid, stacktrace and task command name, we can this command: ./page_owner_sort --cull=stacktrace,name,pid The output would be like: 1 times, 1 pages, PID 1, task_comm_name: task_name_demo prep_new_page+0xd0/0xf8 get_page_from_freelist+0x4a0/0x1290 __alloc_pages+0x168/0x340 alloc_pages+0xb0/0x158 1 times, 1 pages, PID 32, task_comm_name: task_name_demo prep_new_page+0xd0/0xf8 get_page_from_freelist+0x4a0/0x1290 __alloc_pages+0x168/0x340 alloc_pages+0xb0/0x158 As we can see, these two blocks are failed to cull, for their PIDs are different. 2. Add explanations of --cull options to the document. This work is coauthored by Yixuan Cao Shenghong Han Yinan Zhang Chongxi Zhao Yuhong Feng Link: https://lkml.kernel.org/r/20220312145834.624-1-yejiajian2018@email.szu.edu.cn Signed-off-by: Jiajian Ye Cc: Yixuan Cao Cc: Shenghong Han Cc: Yinan Zhang Cc: Chongxi Zhao Cc: Yuhong Feng Cc: Stephen Rothwell Cc: Sean Anderson Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds

tools/vm/page_owner_sort.c: support for selecting by PID, TGID or task command name

2022-03-25T02:06:45+00:00

When viewing page owner information, we may also need to select the blocks by PID, TGID or task command name, which helps to get more accurate page allocation information as needed. Therefore, following adjustments are made: 1. Add three new options, including --pid, --tgid and --name, to support the selection of information blocks by a specific pid, tgid and task command name. In addtion, multiple options are allowed to be used at the same time. ./page_owner_sort [input] [output] --pid ./page_owner_sort [input] [output] --tgid ./page_owner_sort [input] [output] --name Assuming a scenario when a multi-threaded program, ./demo (PID = 5280), is running, and ./demo creates a child process (PID = 5281). $ps PID TTY TIME CMD 5215 pts/0 00:00:00 bash 5280 pts/0 00:00:00 ./demo 5281 pts/0 00:00:00 ./demo 5282 pts/0 00:00:00 ps It would be better to filter out the records with tgid=5280 and the task name "demo" when debugging the parent process, and the specific usage is ./page_owner_sort [input] [output] --tgid 5280 --name demo 2. Add explanations of three new options, including --pid, --tgid and --name, to the document. This work is coauthored by Shenghong Han , Yixuan Cao , Yinan Zhang , Chongxi Zhao , Yuhong Feng . Link: https://lkml.kernel.org/r/1646835223-7584-1-git-send-email-yejiajian2018@email.szu.edu.cn Signed-off-by: Jiajian Ye Cc: Sean Anderson Cc: Stephen Rothwell Cc: Zhenliang Wei Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds

tools/vm/page_owner_sort: support for sorting by task command name

2022-03-25T02:06:45+00:00

When viewing page owner information, we may also need to the block to be sorted by task command name. Therefore, the following adjustments are made: 1. Add a member variable to record task command name of block. 2. Add a new -n option to sort the information of blocks by task command name. 3. Add -n option explanation in the document. Link: https://lkml.kernel.org/r/20220306030640.43054-2-yejiajian2018@email.szu.edu.cn Signed-off-by: Jiajian Ye Cc: Stephen Rothwell Cc: Sean Anderson Cc: Yixuan Cao Cc: Zhenliang Wei Cc: Cc: Cc: Signed-off-by: Andrew Morton Signed-off-by: Linus Torvalds