diff options
34 files changed, 1272 insertions, 202 deletions
diff --git a/Documentation/admin-guide/bootconfig.rst b/Documentation/admin-guide/bootconfig.rst index f712758472d5..3d6412458c8b 100644 --- a/Documentation/admin-guide/bootconfig.rst +++ b/Documentation/admin-guide/bootconfig.rst @@ -234,6 +234,87 @@ Kconfig option selected. Note that even if you set this option, you can override the embedded bootconfig by another bootconfig which attached to the initrd. +Rendering Embedded kernel.* Keys at Build Time +---------------------------------------------- + +By default, the embedded bootconfig (``CONFIG_BOOT_CONFIG_EMBED=y``) is +parsed at runtime, after ``parse_early_param()`` has already run. Early +parameter handlers (``mem=``, ``earlycon=``, ``loglevel=``, ...) therefore +cannot see values supplied via the embedded ``kernel`` subtree. + +``CONFIG_CMDLINE_FROM_BOOTCONFIG`` resolves this by rendering the +``kernel`` subtree of ``CONFIG_BOOT_CONFIG_EMBED_FILE`` into a flat cmdline +string at kernel build time (via ``tools/bootconfig -C``) and prepending +it to ``boot_command_line`` during early architecture setup, so the keys +are visible to ``parse_early_param()``. + +The option requires ``CONFIG_BOOT_CONFIG_EMBED=y``, a non-empty +``CONFIG_BOOT_CONFIG_EMBED_FILE``, ``CONFIG_CMDLINE`` to be empty, and +an architecture that selects ``CONFIG_ARCH_SUPPORTS_CMDLINE_FROM_BOOTCONFIG``. +Currently only x86 selects it; on other architectures the embedded +bootconfig still works, but only through the late runtime parser. + +The same ``bootconfig`` opt-in applies as elsewhere: the rendered keys +are prepended only when ``bootconfig`` (in any form) appears on the +kernel command line, or when ``CONFIG_BOOT_CONFIG_FORCE`` is set, which +defaults to ``y`` when ``CONFIG_BOOT_CONFIG_EMBED`` is set. + +For example, given:: + + kernel { + loglevel = 7 + mem = 4G + } + +the kernel boots as if ``loglevel=7 mem=4G`` had been prepended to the +bootloader command line, with the values visible to early-parsed +handlers. Comma-separated values are still expanded into multiple +cmdline entries per the bootconfig array convention -- the embedded +``kernel.earlycon = "uart8250,io,0x3f8"`` must be quoted to land as a +single ``earlycon=`` entry, exactly as for the runtime parser. + +If the rendered string would not fit in ``COMMAND_LINE_SIZE`` together +with the existing command line, the prepend is skipped and an error is +logged, so an oversized embedded bootconfig cannot brick a boot. + +Interaction with other command line and bootconfig sources +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +With ``CONFIG_CMDLINE_FROM_BOOTCONFIG=y`` the rendered ``kernel`` +subtree behaves like a build-time command line (similar to +``CONFIG_CMDLINE``), not like a bootconfig source. It is prepended to +``boot_command_line`` in ``setup_arch()``, before ``parse_early_param()`` +and long before the runtime parser looks at an initrd. Options can reach +the kernel from up to four places: + +- Bootloader command line: the arguments the boot loader passes. The + embedded cmdline is prepended in front of them, so for last-one-wins + parameters a bootloader option still overrides the embedded value. + Visible in /proc/cmdline. +- Embedded cmdline (this option): the rendered ``kernel`` subtree, + prepended early so it is seen by ``parse_early_param()``. Visible in + /proc/cmdline. +- Initrd bootconfig: parsed late in ``setup_boot_config()``; its + ``kernel`` keys are placed ahead of ``boot_command_line``, i.e. before + the embedded cmdline, so last-wins favors the embedded values. As a + bootconfig source, an initrd bootconfig still replaces the embedded + bootconfig. Visible in /proc/cmdline and /proc/bootconfig. +- Embedded bootconfig (runtime): parsed late, only when no initrd + bootconfig is present. Visible in /proc/cmdline and /proc/bootconfig. + +So with this option the embedded ``kernel.*`` values take precedence +over an initrd bootconfig's ``kernel.*`` values: for early parameters +the initrd is not parsed yet, and for ordinary parameters the embedded +keys land later in the command line. If you need an initrd bootconfig to +override the embedded ``kernel.*`` keys, leave this option off and rely +on the runtime parser. + +The rendered string is part of the command line, so it appears in +/proc/cmdline. It is deliberately not shown in /proc/bootconfig: that +file keeps reporting the parsed bootconfig tree -- the initrd bootconfig +if present, otherwise the embedded bootconfig -- independent of whether +build-time cmdline rendering is enabled. + Kernel parameters via Boot Config ================================= diff --git a/Documentation/trace/eprobetrace.rst b/Documentation/trace/eprobetrace.rst index fe3602540569..680e0af43d5d 100644 --- a/Documentation/trace/eprobetrace.rst +++ b/Documentation/trace/eprobetrace.rst @@ -49,7 +49,12 @@ Synopsis of eprobe_events (STRUCT)FIELD->MEMBER[->MEMBER] : If BTF is supported, typecast FIELD to a pointer to STRUCT and then derference the pointer defined by ->MEMBER. Note that when this is used, the FIELD name does not - need to be prefixed with a '$'. + need to be prefixed with a '$'. ASGN can be specified optionally. + If ASGN is specified, FIELD will be cast to the same offset + position as the ASGN member, rather than to the beginning of + the STRUCT. + (STRUCT)(FETCHARG)->MEMBER[->MEMBER] : typecast can nest, so the above can + also be used with another FETCHARG instead of FIELD. Types ----- diff --git a/Documentation/trace/fprobetrace.rst b/Documentation/trace/fprobetrace.rst index b4c2ca3d02c1..3439bc9bd351 100644 --- a/Documentation/trace/fprobetrace.rst +++ b/Documentation/trace/fprobetrace.rst @@ -50,13 +50,23 @@ Synopsis of fprobe-events $argN : Fetch the Nth function argument. (N >= 1) (\*2) $retval : Fetch return value.(\*3) $comm : Fetch current task comm. + $current : Fetch the address of the current task_struct. +|-[u]OFFS(FETCHARG) : Fetch memory at FETCHARG +|- OFFS address.(\*4)(\*5) + this_cpu_read(FETCHARG) : Read the value of the per-CPU variable FETCHARG on the current CPU. + this_cpu_ptr(FETCHARG) : Get the address of the per-CPU variable FETCHARG on the current CPU. \IMM : Store an immediate value to the argument. NAME=FETCHARG : Set NAME as the argument name of FETCHARG. FETCHARG:TYPE : Set TYPE as the type of FETCHARG. Currently, basic types (u8/u16/u32/u64/s8/s16/s32/s64), hexadecimal types (x8/x16/x32/x64), "char", "string", "ustring", "symbol", "symstr" and bitfield are supported. + (STRUCT[,ASGN])FIELD->MEMBER[->MEMBER] : If BTF is supported, typecast FIELD to + a pointer to STRUCT and then derference the pointer defined by + ->MEMBER. ASGN can be specified optionally. If ASGN is specified, + FIELD will be cast to the same offset position as the ASGN member, + rather than to the beginning of the STRUCT. + (STRUCT[,ASGN])(FETCHARG)->MEMBER[->MEMBER] : typecast can nest, so the above can + also be used with another FETCHARG instead of FIELD. (\*1) This is available only when BTF is enabled. (\*2) only for the probe on function entry (offs == 0). Note, this argument access diff --git a/Documentation/trace/kprobetrace.rst b/Documentation/trace/kprobetrace.rst index 3b6791c17e9b..9ae330eb0a52 100644 --- a/Documentation/trace/kprobetrace.rst +++ b/Documentation/trace/kprobetrace.rst @@ -53,7 +53,10 @@ Synopsis of kprobe_events $argN : Fetch the Nth function argument. (N >= 1) (\*1) $retval : Fetch return value.(\*2) $comm : Fetch current task comm. + $current : Fetch the address of the current task_struct. +|-[u]OFFS(FETCHARG) : Fetch memory at FETCHARG +|- OFFS address.(\*3)(\*4) + this_cpu_read(FETCHARG) : Read the value of the per-CPU variable FETCHARG on the current CPU. + this_cpu_ptr(FETCHARG) : Get the address of the per-CPU variable FETCHARG on the current CPU. \IMM : Store an immediate value to the argument. NAME=FETCHARG : Set NAME as the argument name of FETCHARG. FETCHARG:TYPE : Set TYPE as the type of FETCHARG. Currently, basic types @@ -61,6 +64,14 @@ Synopsis of kprobe_events (x8/x16/x32/x64), VFS layer common type(%pd/%pD), "char", "string", "ustring", "symbol", "symstr" and bitfield are supported. + (STRUCT[,ASGN])FIELD->MEMBER[->MEMBER] : If BTF is supported, typecast FIELD to + a pointer to STRUCT and then derference the pointer defined by + ->MEMBER. Note that this is available only when the probe is + on function entry. ASGN can be specified optionally. If ASGN + is specified, FIELD will be cast to the same offset position + as the ASGN member, rather than to the beginning of the STRUCT. + (STRUCT[,ASGN])(FETCHARG)->MEMBER[->MEMBER] : typecast can nest, so the above can + also be used with another FETCHARG instead of FIELD. (\*1) only for the probe on function entry (offs == 0). Note, this argument access is best effort, because depending on the argument type, it may be passed on diff --git a/MAINTAINERS b/MAINTAINERS index 44ceebcdd764..85b9fa378dd4 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -9845,6 +9845,7 @@ F: fs/proc/bootconfig.c F: include/linux/bootconfig.h F: lib/bootconfig-data.S F: lib/bootconfig.c +F: lib/embedded-cmdline.S F: tools/bootconfig/* F: tools/bootconfig/scripts/* @@ -1558,6 +1558,22 @@ prepare: tools/bpf/resolve_btfids endif endif +# tools/bootconfig renders the embedded bootconfig into a cmdline at build time. +ifdef CONFIG_CMDLINE_FROM_BOOTCONFIG +prepare: tools/bootconfig +endif + +# tools/bootconfig is run on the build host during prepare, so force a host +# binary here; its own Makefile keeps $(CC) for standalone and cross builds. +# CROSS_COMPILE= is cleared so tools/scripts/Makefile.include does not inject +# the target's --target=/--sysroot= flags into the host clang invocation under +# LLVM=1 cross builds (which would produce a target binary that fails to exec). +tools/bootconfig: export CC := $(HOSTCC) +tools/bootconfig: FORCE + $(Q)mkdir -p $(objtree)/tools + $(Q)$(MAKE) O=$(abspath $(objtree)) subdir=tools -C $(srctree)/tools/ \ + bootconfig CROSS_COMPILE= + # The tools build system is not a part of Kbuild and tends to introduce # its own unique issues. If you need to integrate a new tool into Kbuild, # please consider locating that tool outside the tools/ tree and using the @@ -1584,6 +1600,15 @@ ifneq ($(wildcard $(objtool_O)),) $(Q)$(MAKE) -sC $(abs_srctree)/tools/objtool O=$(objtool_O) srctree=$(abs_srctree) $(patsubst objtool_%,%,$@) endif +PHONY += bootconfig_clean + +bootconfig_O = $(abspath $(objtree))/tools/bootconfig + +bootconfig_clean: +ifneq ($(wildcard $(bootconfig_O)),) + $(Q)$(MAKE) -sC $(srctree)/tools/bootconfig O=$(bootconfig_O) clean +endif + tools/: FORCE $(Q)mkdir -p $(objtree)/tools $(Q)$(MAKE) O=$(abspath $(objtree)) subdir=tools -C $(srctree)/tools/ @@ -1759,7 +1784,7 @@ vmlinuxclean: $(Q)$(CONFIG_SHELL) $(srctree)/scripts/link-vmlinux.sh clean $(Q)$(if $(ARCH_POSTLINK), $(MAKE) -f $(ARCH_POSTLINK) clean) -clean: archclean vmlinuxclean resolve_btfids_clean objtool_clean +clean: archclean vmlinuxclean resolve_btfids_clean objtool_clean bootconfig_clean # mrproper - Delete all generated files, including .config # diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 50e00d8417aa..cda1a2c42411 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -127,6 +127,7 @@ config X86 select ARCH_SUPPORTS_NUMA_BALANCING if X86_64 select ARCH_SUPPORTS_KMAP_LOCAL_FORCE_MAP if NR_CPUS <= 4096 select ARCH_SUPPORTS_CFI if X86_64 + select ARCH_SUPPORTS_CMDLINE_FROM_BOOTCONFIG select ARCH_USES_CFI_TRAPS if X86_64 && CFI select ARCH_SUPPORTS_LTO_CLANG select ARCH_SUPPORTS_LTO_CLANG_THIN diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c index 46882ce79c3a..88b055a46591 100644 --- a/arch/x86/kernel/setup.c +++ b/arch/x86/kernel/setup.c @@ -6,6 +6,7 @@ * parts of early kernel initialization. */ #include <linux/acpi.h> +#include <linux/bootconfig.h> #include <linux/console.h> #include <linux/cpu.h> #include <linux/crash_dump.h> @@ -880,7 +881,6 @@ static void __init x86_report_nx(void) * * Note: On x86_64, fixmaps are ready for use even before this is called. */ - void __init setup_arch(char **cmdline_p) { #ifdef CONFIG_X86_32 @@ -924,6 +924,18 @@ void __init setup_arch(char **cmdline_p) builtin_cmdline_added = true; #endif +#ifdef CONFIG_CMDLINE_FROM_BOOTCONFIG + /* + * Prepend the build-time-rendered embedded "kernel" keys here so + * parse_early_param() below sees them, using the same opt-in as the + * runtime parser, plus the build-time CONFIG_BOOT_CONFIG_FORCE. + */ + if (bootconfig_cmdline_requested(boot_command_line, NULL) || + IS_ENABLED(CONFIG_BOOT_CONFIG_FORCE)) + xbc_prepend_embedded_cmdline(boot_command_line, + COMMAND_LINE_SIZE); +#endif + strscpy(command_line, boot_command_line, COMMAND_LINE_SIZE); *cmdline_p = command_line; diff --git a/include/asm-generic/kprobes.h b/include/asm-generic/kprobes.h index 5290a2b2e15a..16f16963d503 100644 --- a/include/asm-generic/kprobes.h +++ b/include/asm-generic/kprobes.h @@ -2,7 +2,7 @@ #ifndef _ASM_GENERIC_KPROBES_H #define _ASM_GENERIC_KPROBES_H -#if defined(__KERNEL__) && !defined(__ASSEMBLY__) +#if defined(__KERNEL__) && !defined(__ASSEMBLER__) #ifdef CONFIG_KPROBES /* * Blacklist ganerating macro. Specify functions which is not probed @@ -21,6 +21,6 @@ static unsigned long __used \ # define __kprobes # define nokprobe_inline inline #endif -#endif /* defined(__KERNEL__) && !defined(__ASSEMBLY__) */ +#endif /* defined(__KERNEL__) && !defined(__ASSEMBLER__) */ #endif /* _ASM_GENERIC_KPROBES_H */ diff --git a/include/linux/bootconfig.h b/include/linux/bootconfig.h index 1c7f3b74ffcf..deda507500da 100644 --- a/include/linux/bootconfig.h +++ b/include/linux/bootconfig.h @@ -308,4 +308,18 @@ static inline const char *xbc_get_embedded_bootconfig(size_t *size) } #endif +/* Bootconfig opt-in detection, shared by setup_arch() and setup_boot_config() */ +#ifdef CONFIG_BOOT_CONFIG +bool __init bootconfig_cmdline_requested(const char *boot_cmdline, int *end_offset); +#endif + +/* Build-time-rendered bootconfig cmdline prepended in setup_arch() */ +#ifdef CONFIG_CMDLINE_FROM_BOOTCONFIG +void __init xbc_prepend_embedded_cmdline(char *dst, size_t size); +bool __init xbc_embedded_cmdline_applied(void); +#else +static inline void xbc_prepend_embedded_cmdline(char *dst, size_t size) { } +static inline bool xbc_embedded_cmdline_applied(void) { return false; } +#endif + #endif diff --git a/init/Kconfig b/init/Kconfig index 10f2013b5321..fa42fb264c9c 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -1569,6 +1569,42 @@ config BOOT_CONFIG_EMBED_FILE This bootconfig will be used if there is no initrd or no other bootconfig in the initrd. +config ARCH_SUPPORTS_CMDLINE_FROM_BOOTCONFIG + bool + help + Silent symbol; no C code reads it directly. Architectures + select it once their setup_arch() calls + xbc_prepend_embedded_cmdline() before parse_early_param(). + Its only role is to gate the user-visible + CMDLINE_FROM_BOOTCONFIG option per-arch, the same + ARCH_SUPPORTS_* idiom used by ARCH_SUPPORTS_CFI, etc. + +config CMDLINE_FROM_BOOTCONFIG + bool "Render embedded bootconfig as kernel cmdline at build time" + depends on BOOT_CONFIG_EMBED_FILE != "" + depends on ARCH_SUPPORTS_CMDLINE_FROM_BOOTCONFIG + depends on CMDLINE = "" + default n + help + Render the "kernel" subtree of the embedded bootconfig file into a + flat cmdline string at kernel build time and prepend it to + boot_command_line during early architecture setup. This makes + early_param() handlers (e.g. mem=, earlycon=, loglevel=) see the + values supplied via the embedded bootconfig. + + The runtime bootconfig parser is unaffected, so tree-structured + consumers such as ftrace boot-time tracing keep working. + + Note: when an initrd also carries a bootconfig, its "kernel" + subtree is still parsed at runtime, but the embedded "kernel" + keys remain in boot_command_line for parse_early_param() and + end up later than the initrd keys in saved_command_line, so + parse_args() last-wins favors the embedded values. If you need + initrd to override embedded kernel.* keys, leave this option + off. + + If unsure, say N. + config CMDLINE_LOG_WRAP_IDEAL_LEN int "Length to try to wrap the cmdline when logged at boot" default 1021 diff --git a/init/main.c b/init/main.c index 92d34e496a33..2613d3f9b3ce 100644 --- a/init/main.c +++ b/init/main.c @@ -357,45 +357,33 @@ static char * __init xbc_make_cmdline(const char *key) return new_cmdline; } -static int __init bootconfig_params(char *param, char *val, - const char *unused, void *arg) -{ - if (strcmp(param, "bootconfig") == 0) { - bootconfig_found = true; - } - return 0; -} - static int __init warn_bootconfig(char *str) { - /* The 'bootconfig' has been handled by bootconfig_params(). */ + /* The 'bootconfig' option is handled by setup_boot_config(). */ return 0; } static void __init setup_boot_config(void) { - static char tmp_cmdline[COMMAND_LINE_SIZE] __initdata; const char *msg, *data; - int pos, ret; + int pos, ret, offs; size_t size; - char *err; + bool from_embedded = false; /* Cut out the bootconfig data even if we have no bootconfig option */ data = get_boot_config_from_initrd(&size); /* If there is no bootconfig in initrd, try embedded one. */ - if (!data) + if (!data) { data = xbc_get_embedded_bootconfig(&size); + from_embedded = true; + } - strscpy(tmp_cmdline, boot_command_line, COMMAND_LINE_SIZE); - err = parse_args("bootconfig", tmp_cmdline, NULL, 0, 0, 0, NULL, - bootconfig_params); - - if (IS_ERR(err) || !(bootconfig_found || IS_ENABLED(CONFIG_BOOT_CONFIG_FORCE))) + bootconfig_found = bootconfig_cmdline_requested(boot_command_line, &offs); + if (!(bootconfig_found || IS_ENABLED(CONFIG_BOOT_CONFIG_FORCE))) return; - /* parse_args() stops at the next param of '--' and returns an address */ - if (err) - initargs_offs = err - tmp_cmdline; + /* Offset of the init arguments after a "--", located by the helper. */ + initargs_offs = offs; if (!data) { /* If user intended to use bootconfig, show an error level message */ @@ -422,8 +410,24 @@ static void __init setup_boot_config(void) } else { xbc_get_info(&ret, NULL); pr_info("Load bootconfig: %ld bytes %d nodes\n", (long)size, ret); - /* keys starting with "kernel." are passed via cmdline */ - extra_command_line = xbc_make_cmdline("kernel"); + /* + * keys starting with "kernel." are passed via cmdline. When + * this bootconfig came from the embedded source and + * setup_arch() already prepended the rendered "kernel" subtree + * to boot_command_line, rendering again here would duplicate + * the keys in saved_command_line and make accumulating handlers + * (console=, earlycon=, ...) re-register the same value. Skip + * only when the prepend really happened. + * + * On arches that do not select ARCH_SUPPORTS_CMDLINE_FROM_BOOTCONFIG, + * CONFIG_CMDLINE_FROM_BOOTCONFIG is unselectable and + * xbc_embedded_cmdline_applied() collapses to a stub returning + * false, so this path still runs and the embedded "kernel" + * keys reach the cmdline via the runtime parser exactly as + * before this series. + */ + if (!from_embedded || !xbc_embedded_cmdline_applied()) + extra_command_line = xbc_make_cmdline("kernel"); /* Also, "init." keys are init arguments */ extra_init_args = xbc_make_cmdline("init"); } diff --git a/kernel/trace/Kconfig b/kernel/trace/Kconfig index 084f34dc6c9f..0ab5916575a9 100644 --- a/kernel/trace/Kconfig +++ b/kernel/trace/Kconfig @@ -779,6 +779,18 @@ config PROBE_EVENTS_BTF_ARGS kernel function entry or a tracepoint. This is available only if BTF (BPF Type Format) support is enabled. +config PROBE_EVENTS_DUMP_FETCHARG + bool "Dump of dynamic probe event fetch-arguments" + depends on PROBE_EVENTS + default n + help + This shows the dump of fetch-arguments of dynamic probe events + alongside their event definitions in the dynamic_events file + as comment lines. This is useful to debug the probe events. + Since this exposes the raw values in the dynamic_events file, + it might be a security risk. Only enable it if you need to debug + probe events themselves. + config KPROBE_EVENTS depends on KPROBES depends on HAVE_REGS_AND_STACK_ACCESS_API diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c index 1146b83b711a..c9e182d40059 100644 --- a/kernel/trace/trace.c +++ b/kernel/trace/trace.c @@ -4320,14 +4320,16 @@ static const char readme_msg[] = "\t args: <name>=fetcharg[:type]\n" "\t fetcharg: (%<register>|$<efield>), @<address>, @<symbol>[+|-<offset>],\n" #ifdef CONFIG_HAVE_FUNCTION_ARG_ACCESS_API - "\t $stack<index>, $stack, $retval, $comm, $arg<N>,\n" + "\t $stack<index>, $stack, $retval, $comm, $arg<N>, $current\n" #ifdef CONFIG_PROBE_EVENTS_BTF_ARGS - "\t <argname>[->field[->field|.field...]],\n" + "\t [(structname[,field])]<argname>[->field[->field|.field...]],\n" + "\t [(structname[,field])](fetcharg)->field[->field|.field...],\n" #endif #else - "\t $stack<index>, $stack, $retval, $comm,\n" + "\t $stack<index>, $stack, $retval, $comm, $current\n" #endif "\t +|-[u]<offset>(<fetcharg>), \\imm-value, \\\"imm-string\"\n" + "\t this_cpu_read(<fetcharg>), this_cpu_ptr(<fetcharg>)\n" "\t kernel return probes support: $retval, $arg<N>, $comm\n" "\t type: s8/16/32/64, u8/16/32/64, x8/16/32/64, char, string, symbol,\n" "\t b<bit-width>@<bit-offset>/<container-size>, ustring,\n" diff --git a/kernel/trace/trace_eprobe.c b/kernel/trace/trace_eprobe.c index 50518b071414..462c31145733 100644 --- a/kernel/trace/trace_eprobe.c +++ b/kernel/trace/trace_eprobe.c @@ -87,6 +87,8 @@ static int eprobe_dyn_event_show(struct seq_file *m, struct dyn_event *ev) seq_printf(m, " %s=%s", ep->tp.args[i].name, ep->tp.args[i].comm); seq_putc(m, '\n'); + trace_probe_dump_args(m, &ep->tp); + return 0; } diff --git a/kernel/trace/trace_fprobe.c b/kernel/trace/trace_fprobe.c index 9f5f08c0e7c2..536781cd4c47 100644 --- a/kernel/trace/trace_fprobe.c +++ b/kernel/trace/trace_fprobe.c @@ -764,7 +764,7 @@ static int unregister_fprobe_event(struct trace_fprobe *tf) return trace_probe_unregister_event_call(&tf->tp); } -static int __regsiter_tracepoint_fprobe(struct trace_fprobe *tf) +static int __register_tracepoint_fprobe(struct trace_fprobe *tf) { struct tracepoint_user *tuser __free(tuser_put) = NULL; struct module *mod __free(module_put) = NULL; @@ -836,7 +836,7 @@ static int __register_trace_fprobe(struct trace_fprobe *tf) tf->fp.flags &= ~FPROBE_FL_DISABLED; if (trace_fprobe_is_tracepoint(tf)) - return __regsiter_tracepoint_fprobe(tf); + return __register_tracepoint_fprobe(tf); /* TODO: handle filter, nofilter or symbol list */ return register_fprobe(&tf->fp, tf->symbol, NULL); @@ -1449,6 +1449,8 @@ static int trace_fprobe_show(struct seq_file *m, struct dyn_event *ev) seq_printf(m, " %s=%s", tf->tp.args[i].name, tf->tp.args[i].comm); seq_putc(m, '\n'); + trace_probe_dump_args(m, &tf->tp); + return 0; } diff --git a/kernel/trace/trace_kprobe.c b/kernel/trace/trace_kprobe.c index a8420e6abb56..cfa807d8e760 100644 --- a/kernel/trace/trace_kprobe.c +++ b/kernel/trace/trace_kprobe.c @@ -1320,6 +1320,8 @@ static int trace_kprobe_show(struct seq_file *m, struct dyn_event *ev) seq_printf(m, " %s=%s", tk->tp.args[i].name, tk->tp.args[i].comm); seq_putc(m, '\n'); + trace_probe_dump_args(m, &tk->tp); + return 0; } diff --git a/kernel/trace/trace_probe.c b/kernel/trace/trace_probe.c index d17cfee77d9c..18c212122344 100644 --- a/kernel/trace/trace_probe.c +++ b/kernel/trace/trace_probe.c @@ -345,6 +345,109 @@ static int parse_trace_event(char *arg, struct fetch_insn *code, return -EINVAL; } +/* this_cpu_* parser */ +#define THIS_CPU_PTR_PREFIX "this_cpu_ptr(" +#define THIS_CPU_READ_PREFIX "this_cpu_read(" +#define THIS_CPU_PTR_LEN (sizeof(THIS_CPU_PTR_PREFIX) - 1) +#define THIS_CPU_READ_LEN (sizeof(THIS_CPU_READ_PREFIX) - 1) + +static int +parse_probe_arg(char *arg, const struct fetch_type *type, + struct fetch_insn **pcode, struct fetch_insn *end, + struct traceprobe_parse_context *ctx); + +/* handle dereference nested call */ +static inline int handle_dereference(char *arg, struct fetch_insn **pcode, + struct fetch_insn *end, struct traceprobe_parse_context *ctx, + int deref, long offset) +{ + const struct fetch_type *type = find_fetch_type(NULL, ctx->flags); + struct fetch_insn *code = *pcode; + int cur_offs = ctx->offset; + char *tmp; + int ret; + + tmp = strrchr(arg, ')'); + if (!tmp) { + trace_probe_log_err(ctx->offset + strlen(arg), + DEREF_OPEN_BRACE); + return -EINVAL; + } + + *tmp = '\0'; + ret = parse_probe_arg(arg, type, &code, end, ctx); + if (ret) + return ret; + ctx->offset = cur_offs; + if (code->op == FETCH_OP_COMM || code->op == FETCH_OP_IMMSTR) { + trace_probe_log_err(ctx->offset, COMM_CANT_DEREF); + return -EINVAL; + } + + /* + * this_cpu_ptr(@SYM) does not use SYM value, but use SYM address. + * So we overwrite the last FETCH_OP_DEREF with FETCH_OP_CPU_PTR. + */ + if (!(deref == FETCH_OP_CPU_PTR && *arg == '@')) { + code++; + if (code == end) { + trace_probe_log_err(ctx->offset, TOO_MANY_OPS); + return -EINVAL; + } + } + *pcode = code; + + code->op = deref; + code->offset = offset; + /* Reset the last type if used */ + ctx->last_type = NULL; + return 0; +} + +static int parse_this_cpu(char *arg, struct fetch_insn **pcode, + struct fetch_insn *end, + struct traceprobe_parse_context *ctx) +{ + struct fetch_insn *code; + bool is_ptr = false; + int ret; + + /* + * This is only for kernel probes, excluding eprobe, because per-cpu + * pointer should not be recorded by events. + */ + if (!(ctx->flags & TPARG_FL_KERNEL) || + (ctx->flags & TPARG_FL_TEVENT)) { + trace_probe_log_err(ctx->offset, NOSUP_PERCPU); + return -EINVAL; + } + if (str_has_prefix(arg, THIS_CPU_PTR_PREFIX)) { + arg += THIS_CPU_PTR_LEN; + ctx->offset += THIS_CPU_PTR_LEN; + is_ptr = true; + } else if (str_has_prefix(arg, THIS_CPU_READ_PREFIX)) { + arg += THIS_CPU_READ_LEN; + ctx->offset += THIS_CPU_READ_LEN; + } else + return -EINVAL; + + ret = handle_dereference(arg, pcode, end, ctx, FETCH_OP_CPU_PTR, 0); + if (ret || is_ptr) + return ret; + + /* this_cpu_read(VAR) -> +0(this_cpu_ptr(VAR)) */ + code = *pcode; + code++; + if (code == end) { + trace_probe_log_err(ctx->offset, TOO_MANY_OPS); + return -EINVAL; + } + code->op = FETCH_OP_DEREF; + code->offset = 0; + *pcode = code; + return 0; +} + #ifdef CONFIG_PROBE_EVENTS_BTF_ARGS static u32 btf_type_int(const struct btf_type *t) @@ -356,9 +459,8 @@ static bool btf_type_is_char_ptr(struct btf *btf, const struct btf_type *type) { const struct btf_type *real_type; u32 intdata; - s32 tid; - real_type = btf_type_skip_modifiers(btf, type->type, &tid); + real_type = btf_type_skip_modifiers(btf, type->type, NULL); if (!real_type) return false; @@ -375,14 +477,13 @@ static bool btf_type_is_char_array(struct btf *btf, const struct btf_type *type) const struct btf_type *real_type; const struct btf_array *array; u32 intdata; - s32 tid; if (BTF_INFO_KIND(type->info) != BTF_KIND_ARRAY) return false; array = (const struct btf_array *)(type + 1); - real_type = btf_type_skip_modifiers(btf, array->type, &tid); + real_type = btf_type_skip_modifiers(btf, array->type, NULL); intdata = btf_type_int(real_type); return !(BTF_INT_ENCODING(intdata) & BTF_INT_SIGNED) @@ -570,6 +671,64 @@ static int split_next_field(char *varname, char **next_field, return ret; } +/* Inner loop for solving dot operator ('.'). Return bit-offset of the given field */ +static int get_bitoffset_of_field(char **pfieldname, const struct btf_type **ptype, + struct traceprobe_parse_context *ctx) +{ + const struct btf_type *type = *ptype; + const struct btf_member *field; + struct btf *btf = ctx_btf(ctx); + char *fieldname = *pfieldname; + int bitoffs = 0; + u32 anon_offs; + char *next; + int is_ptr; + + do { + next = NULL; + is_ptr = split_next_field(fieldname, &next, ctx); + if (is_ptr < 0) + return is_ptr; + + anon_offs = 0; + field = btf_find_struct_member(btf, type, fieldname, + &anon_offs); + if (IS_ERR(field)) { + trace_probe_log_err(ctx->offset, BAD_BTF_TID); + return PTR_ERR(field); + } + if (!field) { + trace_probe_log_err(ctx->offset, NO_BTF_FIELD); + return -ENOENT; + } + /* Add anonymous structure/union offset */ + bitoffs += anon_offs; + + /* Accumulate the bit-offsets of the dot-connected fields */ + if (btf_type_kflag(type)) { + bitoffs += BTF_MEMBER_BIT_OFFSET(field->offset); + ctx->last_bitsize = BTF_MEMBER_BITFIELD_SIZE(field->offset); + } else { + bitoffs += field->offset; + ctx->last_bitsize = 0; + } + + type = btf_type_skip_modifiers(btf, field->type, NULL); + if (!type) { + trace_probe_log_err(ctx->offset, BAD_BTF_TID); + return -EINVAL; + } + + if (next) + ctx->offset += next - fieldname; + fieldname = next; + } while (!is_ptr && fieldname); + + *pfieldname = fieldname; + *ptype = type; + + return bitoffs; +} /* * Parse the field of data structure. The @type must be a pointer type * pointing the target data structure type. @@ -579,16 +738,13 @@ static int parse_btf_field(char *fieldname, const struct btf_type *type, struct traceprobe_parse_context *ctx) { struct fetch_insn *code = *pcode; - const struct btf_member *field; - u32 bitoffs, anon_offs; - bool is_struct = ctx->struct_btf != NULL; struct btf *btf = ctx_btf(ctx); - char *next; - int is_ptr; - s32 tid; + bool is_first_field = true; + int bitoffs; do { - if (!is_struct) { + /* For the first field of typecast, @type will be the target structure type. */ + if (!(is_first_field && ctx->struct_btf)) { /* Outer loop for solving arrow operator ('->') */ if (BTF_INFO_KIND(type->info) != BTF_KIND_PTR) { trace_probe_log_err(ctx->offset, NO_PTR_STRCT); @@ -596,66 +752,31 @@ static int parse_btf_field(char *fieldname, const struct btf_type *type, } /* Convert a struct pointer type to a struct type */ - type = btf_type_skip_modifiers(btf, type->type, &tid); + type = btf_type_skip_modifiers(btf, type->type, NULL); if (!type) { trace_probe_log_err(ctx->offset, BAD_BTF_TID); return -EINVAL; } } - /* Only the first type can skip being a pointer */ - is_struct = false; - - bitoffs = 0; - do { - /* Inner loop for solving dot operator ('.') */ - next = NULL; - is_ptr = split_next_field(fieldname, &next, ctx); - if (is_ptr < 0) - return is_ptr; - - anon_offs = 0; - field = btf_find_struct_member(btf, type, fieldname, - &anon_offs); - if (IS_ERR(field)) { - trace_probe_log_err(ctx->offset, BAD_BTF_TID); - return PTR_ERR(field); - } - if (!field) { - trace_probe_log_err(ctx->offset, NO_BTF_FIELD); - return -ENOENT; - } - /* Add anonymous structure/union offset */ - bitoffs += anon_offs; - - /* Accumulate the bit-offsets of the dot-connected fields */ - if (btf_type_kflag(type)) { - bitoffs += BTF_MEMBER_BIT_OFFSET(field->offset); - ctx->last_bitsize = BTF_MEMBER_BITFIELD_SIZE(field->offset); - } else { - bitoffs += field->offset; - ctx->last_bitsize = 0; - } - - type = btf_type_skip_modifiers(btf, field->type, &tid); - if (!type) { - trace_probe_log_err(ctx->offset, BAD_BTF_TID); - return -EINVAL; - } - - ctx->offset += next - fieldname; - fieldname = next; - } while (!is_ptr && fieldname); + bitoffs = get_bitoffset_of_field(&fieldname, &type, ctx); + if (bitoffs < 0) + return bitoffs; if (++code == end) { trace_probe_log_err(ctx->offset, TOO_MANY_OPS); return -EINVAL; } code->op = FETCH_OP_DEREF; /* TODO: user deref support */ code->offset = bitoffs / 8; + if (is_first_field && ctx->struct_btf) { + /* The first field can be typecasted with field option. */ + code->offset -= ctx->prefix_byteoffs; + } *pcode = code; ctx->last_bitoffs = bitoffs % 8; ctx->last_type = type; + is_first_field = false; } while (fieldname); return 0; @@ -674,7 +795,9 @@ static int parse_btf_arg(char *varname, int i, is_ptr, ret; u32 tid; - if (!ctx->funcname && !(ctx->flags & TPARG_FL_TEVENT)) + /* Note: field is not separated at this point, so check prefix. */ + if (!str_has_prefix(varname, "$current") && + !ctx->funcname && !(ctx->flags & TPARG_FL_TEVENT)) return -EINVAL; is_ptr = split_next_field(varname, &field, ctx); @@ -687,22 +810,23 @@ static int parse_btf_arg(char *varname, return -EOPNOTSUPP; } - if (ctx->flags & TPARG_FL_TEVENT) { - ret = parse_trace_event(varname, code, ctx); + if (!strcmp(varname, "$current")) { + code->op = FETCH_OP_CURRENT; + /* If no typecast is specified for $current, use task_struct by default */ + ret = bpf_find_btf_id("task_struct", BTF_KIND_STRUCT, &ctx->struct_btf); if (ret < 0) { - trace_probe_log_err(ctx->offset, BAD_ATTACH_ARG); - return ret; + trace_probe_log_err(ctx->offset, NO_BTF_ENTRY); + return -ENOENT; } - /* TEVENT is only here via a typecast */ - if (WARN_ON_ONCE(ctx->struct_btf == NULL)) - return -EINVAL; - type = ctx->last_struct; + tid = (u32)ret; + type = ctx->last_struct = + btf_type_skip_modifiers(ctx->struct_btf, tid, NULL); goto found_type; } if (ctx->flags & TPARG_FL_RETURN && !strcmp(varname, "$retval")) { code->op = FETCH_OP_RETVAL; - /* Check whether the function return type is not void */ + /* Check whether the function return type is not void, even with typecast. */ if (query_btf_context(ctx) == 0) { if (ctx->proto->type == 0) { trace_probe_log_err(ctx->offset, NO_RETVAL); @@ -755,7 +879,7 @@ static int parse_btf_arg(char *varname, return -ENOENT; found: - type = btf_type_skip_modifiers(ctx->btf, tid, &tid); + type = btf_type_skip_modifiers(ctx->btf, tid, NULL); found_type: if (!type) { trace_probe_log_err(ctx->offset, BAD_BTF_TID); @@ -825,17 +949,89 @@ static int query_btf_struct(const char *sname, struct traceprobe_parse_context * return 0; } +static int parse_btf_casttype(char *casttype, struct traceprobe_parse_context *ctx) +{ + char *field; + int ret; + + /* Field option - evaluated later. */ + field = strchr(casttype, ','); + if (field) + *field++ = '\0'; + + ret = query_btf_struct(casttype, ctx); + if (ret < 0) { + trace_probe_log_err(ctx->offset, NO_PTR_STRCT); + return -EINVAL; + } + + if (field) { + struct btf_type *type = (struct btf_type *)ctx->last_struct; + + ctx->offset += field - casttype; + ret = get_bitoffset_of_field(&field, &ctx->last_struct, ctx); + if (ret < 0) + return ret; + if (ret % 8) { + trace_probe_log_err(ctx->offset, TYPECAST_NOT_ALIGNED); + return -EINVAL; + } + if (field != NULL) { + /* this means @field skips an arrow operator ("->"). */ + trace_probe_log_err(ctx->offset - 2, TYPECAST_BAD_ARROW); + return -EINVAL; + } + ctx->prefix_byteoffs = ret / 8; + /* Restore the original struct type (overwritten by get_bitoffset_of_field) */ + ctx->last_struct = type; + } + + return ret; +} + +/* Find the matching closing parenthesis for a given opening parenthesis. */ +static char *find_matched_close_paren(char *s) +{ + char *p = s; + int count = 0; + + while (*p) { + if (*p == '(') + count++; + else if (*p == ')') { + if (--count == 0) + return p; + } + p++; + } + return NULL; +} + static int handle_typecast(char *arg, struct fetch_insn **pcode, struct fetch_insn *end, struct traceprobe_parse_context *ctx) { + int orig_offset = ctx->offset; + char *close; char *tmp; int ret; - /* Currently this only works for eprobes */ - if (!(ctx->flags & TPARG_FL_TEVENT)) { - trace_probe_log_err(ctx->offset, TYPECAST_NOT_EVENT); - return -EINVAL; + if (!(tparg_is_event_probe(ctx->flags) || + tparg_is_function_entry(ctx->flags) || + tparg_is_function_return(ctx->flags))) { + trace_probe_log_err(ctx->offset, NOSUP_BTFARG); + return -EOPNOTSUPP; + } + + /* + * Always consider the token after typecast as a nested call + * For example: (STRUCT)VAR->FIELD and (STRUCT)(VAR)->FIELD are same. + * VAR is solved in the nested call. + */ + ctx->nested_level++; + if (ctx->nested_level > TRACEPROBE_MAX_NESTED_LEVEL) { + trace_probe_log_err(ctx->offset, TOO_MANY_NESTED); + return -E2BIG; } tmp = strchr(arg, ')'); @@ -844,19 +1040,92 @@ static int handle_typecast(char *arg, struct fetch_insn **pcode, DEREF_OPEN_BRACE); return -EINVAL; } - *tmp = '\0'; - ret = query_btf_struct(arg + 1, ctx); - *tmp = ')'; + *tmp++ = '\0'; - if (ret < 0) { - trace_probe_log_err(ctx->offset + 1, NO_PTR_STRCT); - return -EINVAL; + ctx->offset += tmp - arg; + if (*tmp == '(') { + close = find_matched_close_paren(tmp); + + if (!close) { + trace_probe_log_err(ctx->offset, DEREF_OPEN_BRACE); + return -EINVAL; + } + /* We expect a field access for typecast */ + if (close[1] != '-' || close[2] != '>') { + trace_probe_log_err(ctx->offset + close - tmp + 1, + TYPECAST_REQ_FIELD); + return -EINVAL; + } + /* Skip '(' */ + ctx->offset += 1; + tmp++; + } else if (*tmp == '+' || *tmp == '-' || + str_has_prefix(tmp, THIS_CPU_PTR_PREFIX) || + str_has_prefix(tmp, THIS_CPU_READ_PREFIX)) { + /* Dereference can have another field access inside it. */ + char *open = strchr(tmp + 1, '('); + + if (!open) { + trace_probe_log_err(ctx->offset, + DEREF_NEED_BRACE); + return -EINVAL; + } + close = find_matched_close_paren(open); + if (!close) { + trace_probe_log_err(ctx->offset + strlen(tmp), + DEREF_OPEN_BRACE); + return -EINVAL; + } + close++; + /* We expect a field access for typecast */ + if (close[0] != '-' || close[1] != '>') { + trace_probe_log_err(ctx->offset + close - tmp, + TYPECAST_REQ_FIELD); + return -EINVAL; + } + } else { + if (tmp[0] == '@') { + /* @sym+offset is not allowed without parenthesized */ + close = strpbrk(tmp, "+-"); + if (close && isdigit(close[1])) { + trace_probe_log_err(ctx->offset, + TYPECAST_SYM_OFFSET); + return -EINVAL; + } + } + /* Inner variable name */ + close = strchr(tmp, '-'); + if (!close || close[1] != '>') { + trace_probe_log_err(ctx->offset + strlen(tmp), + TYPECAST_REQ_FIELD); + return -EINVAL; + } } + *close = '\0'; - tmp++; + /* We need to parse the nested one */ + ret = parse_probe_arg(tmp, find_fetch_type(NULL, ctx->flags), + pcode, end, ctx); + if (ret < 0) + return ret; + ctx->nested_level--; + clear_struct_btf(ctx); - ctx->offset += tmp - arg; - ret = parse_btf_arg(tmp, pcode, end, ctx); + /* Let tmp point the field name. */ + if (close[1] == '-') + tmp = close + 3; /* Skip "->" after closing parenthesis */ + else + tmp = close + 2; /* Skip ">" after inner variable name */ + + /* resolve the typecast struct name */ + ctx->offset = orig_offset + 1; /* for the '(' */ + ret = parse_btf_casttype(arg + 1, ctx); + if (ret < 0) + return ret; + + ctx->offset = orig_offset + tmp - arg; + ret = parse_btf_field(tmp, ctx->last_struct, pcode, end, ctx); + ctx->prefix_byteoffs = 0; return ret; } @@ -1118,6 +1387,24 @@ static int parse_probe_vars(char *orig_arg, const struct fetch_type *t, return 0; } + /* $current returns the address of the current task_struct. */ + if (str_has_prefix(arg, "current")) { + /* $current is only supported by kernel probe. */ + if (!(ctx->flags & TPARG_FL_KERNEL)) { + err = TP_ERR_BAD_VAR; + goto inval; + } + arg += strlen("current"); + if (*arg == '-' && IS_ENABLED(CONFIG_PROBE_EVENTS_BTF_ARGS)) + return parse_btf_arg(orig_arg, pcode, end, ctx); + + if (*arg != '\0') + goto inval; + + code->op = FETCH_OP_CURRENT; + return 0; + } + #ifdef CONFIG_HAVE_FUNCTION_ARG_ACCESS_API len = str_has_prefix(arg, "arg"); if (len) { @@ -1230,7 +1517,7 @@ parse_probe_arg(char *arg, const struct fetch_type *type, code->op = FETCH_OP_IMM; code->immediate = param; } else if (arg[1] == '+') { - /* kprobes don't support file offsets */ + /* Kernel probes do not support file offsets */ if (ctx->flags & TPARG_FL_KERNEL) { trace_probe_log_err(ctx->offset, FILE_ON_KPROBE); return -EINVAL; @@ -1294,43 +1581,16 @@ parse_probe_arg(char *arg, const struct fetch_type *type, } ctx->offset += (tmp + 1 - arg) + (arg[0] != '-' ? 1 : 0); arg = tmp + 1; - tmp = strrchr(arg, ')'); - if (!tmp) { - trace_probe_log_err(ctx->offset + strlen(arg), - DEREF_OPEN_BRACE); - return -EINVAL; - } else { - const struct fetch_type *t2 = find_fetch_type(NULL, ctx->flags); - int cur_offs = ctx->offset; - - *tmp = '\0'; - ret = parse_probe_arg(arg, t2, &code, end, ctx); - if (ret) - break; - ctx->offset = cur_offs; - if (code->op == FETCH_OP_COMM || - code->op == FETCH_OP_DATA) { - trace_probe_log_err(ctx->offset, COMM_CANT_DEREF); - return -EINVAL; - } - if (++code == end) { - trace_probe_log_err(ctx->offset, TOO_MANY_OPS); - return -EINVAL; - } - *pcode = code; - - code->op = deref; - code->offset = offset; - /* Reset the last type if used */ - ctx->last_type = NULL; - } + ret = handle_dereference(arg, pcode, end, ctx, deref, offset); + if (ret < 0) + return ret; break; case '\\': /* Immediate value */ if (arg[1] == '"') { /* Immediate string */ ret = __parse_imm_string(arg + 2, &tmp, ctx->offset + 2); if (ret) break; - code->op = FETCH_OP_DATA; + code->op = FETCH_OP_IMMSTR; code->data = tmp; } else { ret = str_to_immediate(arg + 1, &code->immediate); @@ -1344,15 +1604,28 @@ parse_probe_arg(char *arg, const struct fetch_type *type, ret = handle_typecast(arg, pcode, end, ctx); break; default: - if (isalpha(arg[0]) || arg[0] == '_') { /* BTF variable */ + if (str_has_prefix(arg, THIS_CPU_PTR_PREFIX) || + str_has_prefix(arg, THIS_CPU_READ_PREFIX)) { + ret = parse_this_cpu(arg, pcode, end, ctx); + } else if (isalpha(arg[0]) || arg[0] == '_') { + /* BTF variable or event field*/ + if (ctx->flags & TPARG_FL_TEVENT) { + ret = parse_trace_event(arg, *pcode, ctx); + if (ret < 0) { + trace_probe_log_err(ctx->offset, + NO_EVENT_FIELD); + return -EINVAL; + } + break; + } if (!tparg_is_function_entry(ctx->flags) && !tparg_is_function_return(ctx->flags)) { trace_probe_log_err(ctx->offset, NOSUP_BTFARG); return -EINVAL; } ret = parse_btf_arg(arg, pcode, end, ctx); - break; } + break; } if (!ret && code->op == FETCH_OP_NOP) { /* Parsed, but do not find fetch method */ @@ -1485,7 +1758,7 @@ static int finalize_fetch_insn(struct fetch_insn *code, } else { if (code->op != FETCH_OP_DEREF && code->op != FETCH_OP_UDEREF && code->op != FETCH_OP_IMM && code->op != FETCH_OP_COMM && - code->op != FETCH_OP_DATA && code->op != FETCH_OP_TP_ARG) { + code->op != FETCH_OP_IMMSTR && code->op != FETCH_OP_TP_ARG) { trace_probe_log_err(ctx->offset + type_offset, BAD_STRING); return -EINVAL; @@ -1494,7 +1767,7 @@ static int finalize_fetch_insn(struct fetch_insn *code, if (!strcmp(parg->type->name, "symstr") || (code->op == FETCH_OP_IMM || code->op == FETCH_OP_COMM || - code->op == FETCH_OP_DATA) || code->op == FETCH_OP_TP_ARG || + code->op == FETCH_OP_IMMSTR) || code->op == FETCH_OP_TP_ARG || parg->count) { /* * IMM, DATA and COMM is pointing actual address, those @@ -1620,6 +1893,11 @@ static int traceprobe_parse_probe_arg_body(const char *argv, ssize_t *size, ctx); if (ret < 0) goto fail; + /* nested_level must be 0 here, otherwise there is a bug. */ + if (WARN_ON_ONCE(ctx->nested_level)) { + ret = -EINVAL; + goto fail; + } /* Update storing type if BTF is available */ if (IS_ENABLED(CONFIG_PROBE_EVENTS_BTF_ARGS) && @@ -1670,7 +1948,7 @@ fail: if (ret < 0) { for (code = tmp; code < tmp + FETCH_INSN_MAX; code++) if (code->op == FETCH_NOP_SYMBOL || - code->op == FETCH_OP_DATA) + code->op == FETCH_OP_IMMSTR) kfree(code->data); } kfree(tmp); @@ -1769,7 +2047,7 @@ void traceprobe_free_probe_arg(struct probe_arg *arg) while (code && code->op != FETCH_OP_END) { if (code->op == FETCH_NOP_SYMBOL || - code->op == FETCH_OP_DATA) + code->op == FETCH_OP_IMMSTR) kfree(code->data); code++; } @@ -2396,3 +2674,99 @@ int trace_probe_print_args(struct trace_seq *s, struct probe_arg *args, int nr_a } return 0; } + +#ifdef CONFIG_PROBE_EVENTS_DUMP_FETCHARG + +struct fetch_op_decode { + const char *name; + void (*decode)(struct seq_file *m, struct fetch_insn *insn); +}; + +static const struct fetch_op_decode fetch_op_decode[]; + +static void fetcharg_decode_none(struct seq_file *m, struct fetch_insn *insn) +{ + seq_puts(m, fetch_op_decode[insn->op].name); +} + +static void fetcharg_decode_param(struct seq_file *m, struct fetch_insn *insn) +{ + seq_printf(m, "%s(%u)", fetch_op_decode[insn->op].name, insn->param); +} + +static void fetcharg_decode_imm(struct seq_file *m, struct fetch_insn *insn) +{ + seq_printf(m, "%s(0x%lx)", fetch_op_decode[insn->op].name, insn->immediate); +} + +static void fetcharg_decode_string(struct seq_file *m, struct fetch_insn *insn) +{ + seq_printf(m, "%s(%s)", fetch_op_decode[insn->op].name, (char *)insn->data); +} + +static void fetcharg_decode_symbol(struct seq_file *m, struct fetch_insn *insn) +{ + seq_printf(m, "%s(%s)", fetch_op_decode[insn->op].name, (char *)insn->data); +} + +static void fetcharg_decode_offset(struct seq_file *m, struct fetch_insn *insn) +{ + seq_printf(m, "%s(offset=%d)", fetch_op_decode[insn->op].name, insn->offset); +} + +static void fetcharg_decode_store(struct seq_file *m, struct fetch_insn *insn) +{ + if (insn->op == FETCH_OP_ST_RAW) + seq_printf(m, "%s(size=%u)", fetch_op_decode[insn->op].name, insn->size); + else + seq_printf(m, "%s(offset=%d,size=%u)", fetch_op_decode[insn->op].name, + insn->offset, insn->size); +} + +static void fetcharg_decode_bf(struct seq_file *m, struct fetch_insn *insn) +{ + seq_printf(m, "%s(basesize=%u,lshift=%u,rshift=%u)", + fetch_op_decode[insn->op].name, insn->basesize, insn->lshift, insn->rshift); +} + +static void fetcharg_decode_tp_arg(struct seq_file *m, struct fetch_insn *insn) +{ + struct ftrace_event_field *field = insn->data; + + seq_printf(m, "%s(%s)", fetch_op_decode[insn->op].name, field->name); +} + +#define FETCH_OP(opname, decode_fn) \ + [FETCH_OP_##opname] = { .name = #opname, .decode = fetcharg_decode_##decode_fn } + +static const struct fetch_op_decode fetch_op_decode[] = FETCH_OP_LIST; +#undef FETCH_OP + +static void trace_probe_dump_arg(struct seq_file *m, struct probe_arg *parg) +{ + int i; + + seq_printf(m, "# %s: ", parg->name); + for (i = 0; i < FETCH_INSN_MAX; i++) { + struct fetch_insn *insn = parg->code + i; + + if (insn->op >= ARRAY_SIZE(fetch_op_decode) || !fetch_op_decode[insn->op].decode) + seq_printf(m, "unknown(%d)", insn->op); + else + fetch_op_decode[insn->op].decode(m, insn); + + if (insn->op == FETCH_OP_END) + break; + seq_puts(m, " -> "); + } + seq_putc(m, '\n'); +} + +void trace_probe_dump_args(struct seq_file *m, struct trace_probe *tp) +{ + int i; + + for (i = 0; i < tp->nr_args; i++) + trace_probe_dump_arg(m, &tp->args[i]); +} +#endif /* CONFIG_PROBE_EVENTS_DUMP_FETCHARG */ diff --git a/kernel/trace/trace_probe.h b/kernel/trace/trace_probe.h index 0f09f7aaf93f..e6268a8dc378 100644 --- a/kernel/trace/trace_probe.h +++ b/kernel/trace/trace_probe.h @@ -83,38 +83,48 @@ static nokprobe_inline u32 update_data_loc(u32 loc, int consumed) /* Printing function type */ typedef int (*print_type_func_t)(struct trace_seq *, void *, void *); -enum fetch_op { - FETCH_OP_NOP = 0, - // Stage 1 (load) ops - FETCH_OP_REG, /* Register : .param = offset */ - FETCH_OP_STACK, /* Stack : .param = index */ - FETCH_OP_STACKP, /* Stack pointer */ - FETCH_OP_RETVAL, /* Return value */ - FETCH_OP_IMM, /* Immediate : .immediate */ - FETCH_OP_COMM, /* Current comm */ - FETCH_OP_ARG, /* Function argument : .param */ - FETCH_OP_FOFFS, /* File offset: .immediate */ - FETCH_OP_DATA, /* Allocated data: .data */ - FETCH_OP_EDATA, /* Entry data: .offset */ - // Stage 2 (dereference) op - FETCH_OP_DEREF, /* Dereference: .offset */ - FETCH_OP_UDEREF, /* User-space Dereference: .offset */ - // Stage 3 (store) ops - FETCH_OP_ST_RAW, /* Raw: .size */ - FETCH_OP_ST_MEM, /* Mem: .offset, .size */ - FETCH_OP_ST_UMEM, /* Mem: .offset, .size */ - FETCH_OP_ST_STRING, /* String: .offset, .size */ - FETCH_OP_ST_USTRING, /* User String: .offset, .size */ - FETCH_OP_ST_SYMSTR, /* Kernel Symbol String: .offset, .size */ - FETCH_OP_ST_EDATA, /* Store Entry Data: .offset */ - // Stage 4 (modify) op - FETCH_OP_MOD_BF, /* Bitfield: .basesize, .lshift, .rshift */ - // Stage 5 (loop) op - FETCH_OP_LP_ARRAY, /* Array: .param = loop count */ - FETCH_OP_TP_ARG, /* Trace Point argument */ - FETCH_OP_END, - FETCH_NOP_SYMBOL, /* Unresolved Symbol holder */ -}; +#define FETCH_OP_LIST { \ + /* Stage 1 (load) ops */ \ + FETCH_OP(NOP, none), /* NOP */ \ + FETCH_OP(REG, param), /* Register: .param = offset */ \ + FETCH_OP(STACK, param), /* Stack: .param = index */ \ + FETCH_OP(STACKP, none), /* Stack pointer */ \ + FETCH_OP(RETVAL, none), /* Return value */ \ + FETCH_OP(IMM, imm), /* Immediate: .immediate */ \ + FETCH_OP(COMM, none), /* Current comm */ \ + FETCH_OP(CURRENT, none), /* Current task_struct address */\ + FETCH_OP(ARG, param), /* Argument: .param = index */ \ + FETCH_OP(FOFFS, imm), /* File offset: .immediate */ \ + FETCH_OP(IMMSTR, string), /* Allocated string: .data */ \ + FETCH_OP(EDATA, offset), /* Entry data: .offset */ \ + FETCH_OP(TP_ARG, tp_arg), /* Tracepoint argument: .data */\ + /* Stage 2 (dereference) ops */ \ + FETCH_OP(DEREF, offset), /* Dereference: .offset */ \ + FETCH_OP(UDEREF, offset), /* User-space dereference: .offset */\ + FETCH_OP(CPU_PTR, none), /* Per-CPU pointer: .offset */ \ + /* Stage 3 (store) ops */ \ + FETCH_OP(ST_RAW, store), /* Raw value: .size */ \ + FETCH_OP(ST_MEM, store), /* Memory: .offset, .size */ \ + FETCH_OP(ST_UMEM, store), /* User memory: .offset, .size */\ + FETCH_OP(ST_STRING, store), /* String: .offset, .size */ \ + FETCH_OP(ST_USTRING, store), /* User string: .offset, .size */\ + FETCH_OP(ST_SYMSTR, store), /* Symbol name: .offset, .size */\ + FETCH_OP(ST_EDATA, offset), /* Entry data: .offset */ \ + /* Stage 4 (modify) op */ \ + FETCH_OP(MOD_BF, bf), /* Bitfield: .basesize, .lshift, .rshift*/\ + /* Stage 5 (loop) op */ \ + FETCH_OP(LP_ARRAY, param), /* Loop array: .param = count */\ + /* End */ \ + FETCH_OP(END, none), \ + /* Unresolved Symbol holder */ \ + FETCH_OP(NOP_SYMBOL, symbol), /* Non loaded symbol: .data = symbol name */\ +} + +#define FETCH_OP(opname, decode_fn) FETCH_OP_##opname +enum fetch_op FETCH_OP_LIST; +#undef FETCH_OP + +#define FETCH_NOP_SYMBOL FETCH_OP_NOP_SYMBOL struct fetch_insn { enum fetch_op op; @@ -370,6 +380,13 @@ bool trace_probe_match_command_args(struct trace_probe *tp, int trace_probe_create(const char *raw_command, int (*createfn)(int, const char **)); int trace_probe_print_args(struct trace_seq *s, struct probe_arg *args, int nr_args, u8 *data, void *field); +#ifdef CONFIG_PROBE_EVENTS_DUMP_FETCHARG +void trace_probe_dump_args(struct seq_file *m, struct trace_probe *tp); +#else +static inline void trace_probe_dump_args(struct seq_file *m, struct trace_probe *tp) +{ +} +#endif #ifdef CONFIG_HAVE_FUNCTION_ARG_ACCESS_API int traceprobe_get_entry_data_size(struct trace_probe *tp); @@ -414,6 +431,11 @@ static inline bool tparg_is_function_return(unsigned int flags) return (flags & TPARG_FL_LOC_MASK) == (TPARG_FL_KERNEL | TPARG_FL_RETURN); } +static inline bool tparg_is_event_probe(unsigned int flags) +{ + return !!(flags & TPARG_FL_TEVENT); +} + struct traceprobe_parse_context { struct trace_event_call *event; /* BTF related parameters */ @@ -430,8 +452,13 @@ struct traceprobe_parse_context { struct trace_probe *tp; unsigned int flags; int offset; + int nested_level; + int prefix_byteoffs; /* The byte offset of the prefix field of typecast */ }; +/* Each typecast consumes nested level. So the max number of typecast is 3. */ +#define TRACEPROBE_MAX_NESTED_LEVEL 3 + extern int traceprobe_parse_probe_arg(struct trace_probe *tp, int i, const char *argv, struct traceprobe_parse_context *ctx); @@ -516,7 +543,7 @@ extern int traceprobe_define_arg_fields(struct trace_event_call *event_call, C(BAD_MEM_ADDR, "Invalid memory address"), \ C(BAD_IMM, "Invalid immediate value"), \ C(IMMSTR_NO_CLOSE, "String is not closed with '\"'"), \ - C(FILE_ON_KPROBE, "File offset is not available with kprobe"), \ + C(FILE_ON_KPROBE, "File offset is not available for kernel probes"), \ C(BAD_FILE_OFFS, "Invalid file offset value"), \ C(SYM_ON_UPROBE, "Symbol is not available with uprobe"), \ C(TOO_MANY_OPS, "Dereference is too much nested"), \ @@ -559,6 +586,7 @@ extern int traceprobe_define_arg_fields(struct trace_event_call *event_call, C(NO_PTR_STRCT, "This is not a pointer to union/structure."), \ C(NOSUP_DAT_ARG, "Non pointer structure/union argument is not supported."),\ C(BAD_HYPHEN, "Failed to parse single hyphen. Forgot '>'?"), \ + C(NO_EVENT_FIELD, "This event field is not found."), \ C(NO_BTF_FIELD, "This field is not found."), \ C(BAD_BTF_TID, "Failed to get BTF type info."),\ C(BAD_TYPE4STR, "This type does not fit for string."),\ @@ -566,7 +594,13 @@ extern int traceprobe_define_arg_fields(struct trace_event_call *event_call, C(TOO_MANY_ARGS, "Too many arguments are specified"), \ C(TOO_MANY_EARGS, "Too many entry arguments specified"), \ C(EVENT_TOO_BIG, "Event too big (too many fields?)"), \ - C(TYPECAST_NOT_EVENT, "Typecasts are only for eprobe fields"), + C(TYPECAST_NOT_EVENT, "Typecasts are only for eprobe fields"), \ + C(TYPECAST_REQ_FIELD, "Typecast requires a field access"), \ + C(TOO_MANY_NESTED, "Too many nested typecasts/dereferences"), \ + C(TYPECAST_SYM_OFFSET, "@SYM+/-OFFSET with typecast needs parentheses"), \ + C(TYPECAST_NOT_ALIGNED, "Typecast field option is not byte-aligned"), \ + C(TYPECAST_BAD_ARROW, "Typecast field option does not support -> operator"), \ + C(NOSUP_PERCPU, "Per-cpu variable access is only for kernel probes"), #undef C #define C(a, b) TP_ERR_##a diff --git a/kernel/trace/trace_probe_tmpl.h b/kernel/trace/trace_probe_tmpl.h index f39b37fcdb3b..8db12f758fda 100644 --- a/kernel/trace/trace_probe_tmpl.h +++ b/kernel/trace/trace_probe_tmpl.h @@ -109,9 +109,12 @@ process_common_fetch_insn(struct fetch_insn *code, unsigned long *val) case FETCH_OP_COMM: *val = (unsigned long)current->comm; break; - case FETCH_OP_DATA: + case FETCH_OP_IMMSTR: *val = (unsigned long)code->data; break; + case FETCH_OP_CURRENT: + *val = (unsigned long)current; + break; default: return -EILSEQ; } @@ -126,25 +129,35 @@ process_fetch_insn_bottom(struct fetch_insn *code, unsigned long val, struct fetch_insn *s3 = NULL; int total = 0, ret = 0, i = 0; u32 loc = 0; - unsigned long lval = val; + unsigned long lval, llval = val; stage2: /* 2nd stage: dereference memory if needed */ do { - if (code->op == FETCH_OP_DEREF) { - lval = val; + lval = val; + switch (code->op) { + case FETCH_OP_DEREF: ret = probe_mem_read(&val, (void *)val + code->offset, sizeof(val)); - } else if (code->op == FETCH_OP_UDEREF) { - lval = val; + break; + case FETCH_OP_UDEREF: ret = probe_mem_read_user(&val, (void *)val + code->offset, sizeof(val)); - } else break; + case FETCH_OP_CPU_PTR: + val = (unsigned long)this_cpu_ptr((void __percpu *)val); + ret = 0; + break; + default: + lval = llval; + goto out; + } if (ret) return ret; + llval = lval; code++; } while (1); +out: s3 = code; stage3: diff --git a/kernel/trace/trace_uprobe.c b/kernel/trace/trace_uprobe.c index c274346853d1..b2e264a4b96c 100644 --- a/kernel/trace/trace_uprobe.c +++ b/kernel/trace/trace_uprobe.c @@ -765,6 +765,9 @@ static int trace_uprobe_show(struct seq_file *m, struct dyn_event *ev) seq_printf(m, " %s=%s", tu->tp.args[i].name, tu->tp.args[i].comm); seq_putc(m, '\n'); + + trace_probe_dump_args(m, &tu->tp); + return 0; } diff --git a/lib/Makefile b/lib/Makefile index 531a0be88062..132147b3c547 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -272,6 +272,22 @@ filechk_defbconf = cat $(or $(real-prereqs), /dev/null) $(obj)/default.bconf: $(CONFIG_BOOT_CONFIG_EMBED_FILE) FORCE $(call filechk,defbconf) +obj-$(CONFIG_CMDLINE_FROM_BOOTCONFIG) += embedded-cmdline.o +$(obj)/embedded-cmdline.o: $(obj)/embedded_cmdline.bin + +# Render the bootconfig "kernel" subtree to a flat cmdline string using +# the userspace tools/bootconfig parser (-C mode). The runtime prepend +# helper enforces COMMAND_LINE_SIZE at boot, so no build-time size +# check is performed here (COMMAND_LINE_SIZE is an arch header +# constant, not a Kconfig value). +quiet_cmd_render_cmdline = BCONF2C $@ + cmd_render_cmdline = \ + $(objtree)/tools/bootconfig/bootconfig -C $< > $@ + +targets += embedded_cmdline.bin +$(obj)/embedded_cmdline.bin: $(obj)/default.bconf $(objtree)/tools/bootconfig/bootconfig FORCE + $(call if_changed,render_cmdline) + obj-$(CONFIG_RBTREE_TEST) += rbtree_test.o obj-$(CONFIG_INTERVAL_TREE_TEST) += interval_tree_test.o diff --git a/lib/bootconfig.c b/lib/bootconfig.c index 2ed9ee3dc81c..89c88e359179 100644 --- a/lib/bootconfig.c +++ b/lib/bootconfig.c @@ -19,9 +19,13 @@ #include <linux/errno.h> #include <linux/cache.h> #include <linux/compiler.h> +#include <linux/init.h> +#include <linux/moduleparam.h> +#include <linux/printk.h> #include <linux/sprintf.h> #include <linux/memblock.h> #include <linux/string.h> +#include <asm/setup.h> /* COMMAND_LINE_SIZE */ #ifdef CONFIG_BOOT_CONFIG_EMBED /* embedded_bootconfig_data is defined in bootconfig-data.S */ @@ -34,7 +38,129 @@ const char * __init xbc_get_embedded_bootconfig(size_t *size) return (*size) ? embedded_bootconfig_data : NULL; } #endif -#endif + +#ifdef CONFIG_CMDLINE_FROM_BOOTCONFIG +/* embedded_kernel_cmdline is defined in embedded-cmdline.S */ +extern __visible const char embedded_kernel_cmdline[]; +extern __visible const char embedded_kernel_cmdline_end[]; + +/* Set once the embedded cmdline has actually been prepended. */ +static bool xbc_cmdline_applied __initdata; + +/* + * str_prepend() - Prepend @src in front of the string in @dst, in place + * @dst: NUL-terminated destination buffer, currently @dst_len bytes long + * @dst_len: length of the current @dst string (excluding its NUL) + * @src: bytes to prepend (not NUL-terminated) + * @src_len: number of bytes from @src to prepend + * + * The caller must guarantee @dst has room for src_len + dst_len + 1 bytes. + * Moving dst_len + 1 bytes carries @dst's NUL terminator too, so an empty + * @dst needs no special case. + */ +static void __init str_prepend(char *dst, size_t dst_len, + const char *src, size_t src_len) +{ + memmove(dst + src_len, dst, dst_len + 1); + memcpy(dst, src, src_len); +} + +/** + * xbc_prepend_embedded_cmdline() - Prepend embedded bootconfig cmdline + * @dst: cmdline buffer to prepend into (must already contain a NUL byte) + * @size: total capacity of @dst in bytes + * + * Prepend the build-time-rendered "kernel" subtree of the embedded + * bootconfig to @dst. The rendered string already ends with a single + * space (the xbc_snprint_cmdline() invariant), which serves as the + * separator between the embedded keys and any existing content of @dst. + * On overflow, log an error and leave @dst untouched rather than + * silently truncating: booting without the embedded values is better + * than refusing to boot, and the error message tells the user why + * their embedded keys are missing. + * + * Intended to be called from setup_arch() before parse_early_param() so + * that early_param() handlers see the embedded values. + */ +void __init xbc_prepend_embedded_cmdline(char *dst, size_t size) +{ + size_t embed_len = embedded_kernel_cmdline_end - embedded_kernel_cmdline; + size_t dst_len; + + if (!size || embed_len <= 1) /* trailing NUL only */ + return; + embed_len--; /* exclude trailing NUL byte */ + + dst_len = strnlen(dst, size); + if (embed_len + dst_len + 1 > size) { + pr_err("embedded bootconfig cmdline (%zu bytes) does not fit in COMMAND_LINE_SIZE with %zu bytes already used; ignoring embedded values\n", + embed_len, dst_len); + return; + } + + str_prepend(dst, dst_len, embedded_kernel_cmdline, embed_len); + xbc_cmdline_applied = true; +} + +/** + * xbc_embedded_cmdline_applied() - Did the embedded cmdline get prepended? + * + * Return true if xbc_prepend_embedded_cmdline() actually prepended the + * embedded "kernel" subtree. setup_boot_config() uses this to avoid + * rendering the same keys a second time. + */ +bool __init xbc_embedded_cmdline_applied(void) +{ + return xbc_cmdline_applied; +} +#endif /* CONFIG_CMDLINE_FROM_BOOTCONFIG */ + +/* parse_args() callback: flag when the "bootconfig" parameter is present. */ +static int __init bootconfig_optin(char *param, char *val, + const char *unused, void *arg) +{ + if (!strcmp(param, "bootconfig")) + *(bool *)arg = true; + return 0; +} + +/** + * bootconfig_cmdline_requested() - Was "bootconfig" passed on the cmdline? + * @boot_cmdline: kernel command line to inspect (not modified) + * @end_offset: if non-NULL, set to the offset of the init arguments that + * follow a "--" separator, or 0 when there is none + * + * Parse a private copy of @boot_cmdline (parse_args() is destructive) and + * report whether "bootconfig" is present before the "--" separator. + * setup_arch() uses this to gate prepending the build-time embedded cmdline; + * setup_boot_config() uses it for the runtime opt-in and to locate the init + * arguments via @end_offset. Sharing one parser keeps the early and late + * paths agreeing on what counts as opt-in. CONFIG_BOOT_CONFIG_FORCE is not + * folded in here; callers apply it where they need it. + */ +bool __init bootconfig_cmdline_requested(const char *boot_cmdline, int *end_offset) +{ + static char tmp_cmdline[COMMAND_LINE_SIZE] __initdata; + bool found = false; + char *err; + + if (end_offset) + *end_offset = 0; + + strscpy(tmp_cmdline, boot_cmdline, COMMAND_LINE_SIZE); + err = parse_args("bootconfig", tmp_cmdline, NULL, 0, 0, 0, + &found, bootconfig_optin); + if (IS_ERR(err)) + return false; + + /* parse_args() stops at "--" and returns the address of the rest. */ + if (end_offset && err) + *end_offset = err - tmp_cmdline; + + return found; +} + +#endif /* __KERNEL__ */ /* * Extra Boot Config (XBC) is given as tree-structured ascii text of @@ -440,6 +566,17 @@ int __init xbc_snprint_cmdline(char *buf, size_t size, struct xbc_node *root) * itself is well defined and returns the would-be length. */ xbc_node_for_each_key_value(root, knode, val) { + /* + * An empty or value-only @root (e.g. "kernel {}" or + * "kernel = x", possibly alongside "kernel.foo = bar") + * yields @root itself here. Skip it: composing a key for it + * would fail with -EINVAL, yet any real descendant keys must + * still be rendered. An entirely empty subtree then renders + * nothing and returns 0 rather than an error. + */ + if (knode == root) + continue; + ret = xbc_node_compose_key_after(root, knode, xbc_namebuf, XBC_KEYLEN_MAX); if (ret < 0) diff --git a/lib/embedded-cmdline.S b/lib/embedded-cmdline.S new file mode 100644 index 000000000000..bda81b4a42be --- /dev/null +++ b/lib/embedded-cmdline.S @@ -0,0 +1,16 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Embed the build-time-rendered bootconfig "kernel" subtree as a flat + * cmdline string. setup_arch() prepends this to boot_command_line on + * architectures that select ARCH_SUPPORTS_CMDLINE_FROM_BOOTCONFIG. + * + * Copyright (c) 2026 Meta Platforms, Inc. and affiliates + * Copyright (c) 2026 Breno Leitao <leitao@debian.org> + */ + .section .init.rodata.embed_cmdline, "a", %progbits + .global embedded_kernel_cmdline +embedded_kernel_cmdline: + .incbin "lib/embedded_cmdline.bin" + .byte 0 + .global embedded_kernel_cmdline_end +embedded_kernel_cmdline_end: diff --git a/samples/trace_events/trace-events-sample.c b/samples/trace_events/trace-events-sample.c index 0b7a6efdb247..ca5d98c360cb 100644 --- a/samples/trace_events/trace-events-sample.c +++ b/samples/trace_events/trace-events-sample.c @@ -94,6 +94,20 @@ static int simple_thread_fn(void *arg) static DEFINE_MUTEX(thread_mutex); static int simple_thread_cnt; +static struct foo_timer_data *foo_timer_data; + +static void sample_timer_cb(struct timer_list *t) +{ + struct foo_timer_data *data = container_of(t, struct foo_timer_data, timer); + + get_cpu(); + trace_foo_timer_fn(data); + (*this_cpu_ptr(data->counter))++; + put_cpu(); + + mod_timer(t, jiffies + HZ); +} + int foo_bar_reg(void) { mutex_lock(&thread_mutex); @@ -132,9 +146,27 @@ void foo_bar_unreg(void) static int __init trace_event_init(void) { + foo_timer_data = kzalloc_obj(*foo_timer_data, GFP_KERNEL); + if (!foo_timer_data) + return -ENOMEM; + + foo_timer_data->name = "sample_timer_counter"; + foo_timer_data->counter = alloc_percpu(int); + if (!foo_timer_data->counter) { + kfree(foo_timer_data); + return -ENOMEM; + } + + timer_setup(&foo_timer_data->timer, sample_timer_cb, 0); + mod_timer(&foo_timer_data->timer, jiffies + HZ); + simple_tsk = kthread_run(simple_thread, NULL, "event-sample"); - if (IS_ERR(simple_tsk)) - return -1; + if (IS_ERR(simple_tsk)) { + timer_shutdown_sync(&foo_timer_data->timer); + free_percpu(foo_timer_data->counter); + kfree(foo_timer_data); + return PTR_ERR(simple_tsk); + } return 0; } @@ -147,6 +179,10 @@ static void __exit trace_event_exit(void) kthread_stop(simple_tsk_fn); simple_tsk_fn = NULL; mutex_unlock(&thread_mutex); + + timer_shutdown_sync(&foo_timer_data->timer); + free_percpu(foo_timer_data->counter); + kfree(foo_timer_data); } module_init(trace_event_init); diff --git a/samples/trace_events/trace-events-sample.h b/samples/trace_events/trace-events-sample.h index 1a05fc153353..816848a456a2 100644 --- a/samples/trace_events/trace-events-sample.h +++ b/samples/trace_events/trace-events-sample.h @@ -247,12 +247,14 @@ */ /* - * It is OK to have helper functions in the file, but they need to be protected - * from being defined more than once. Remember, this file gets included more - * than once. + * It is OK to have helper functions and data structures in the file, but they + * need to be protected from being defined more than once. Remember, this file + * gets included more than once. */ #ifndef __TRACE_EVENT_SAMPLE_HELPER_FUNCTIONS #define __TRACE_EVENT_SAMPLE_HELPER_FUNCTIONS +#include <linux/timer.h> + static inline int __length_of(const int *list) { int i; @@ -270,6 +272,13 @@ enum { TRACE_SAMPLE_BAR = 4, TRACE_SAMPLE_ZOO = 8, }; + +struct foo_timer_data { + const char *name; + struct timer_list timer; + int __percpu *counter; +}; + #endif /* @@ -595,6 +604,25 @@ TRACE_EVENT(foo_rel_loc, __get_rel_bitmask(bitmask), __get_rel_cpumask(cpumask)) ); + +TRACE_EVENT(foo_timer_fn, + + TP_PROTO(struct foo_timer_data *data), + + TP_ARGS(data), + + TP_STRUCT__entry( + __string( name, data->name ) + __field( int, count ) + ), + + TP_fast_assign( + __assign_str(name); + __entry->count = *this_cpu_ptr(data->counter); + ), + + TP_printk("name=%s count=%d", __get_str(name), __entry->count) +); #endif /***** NOTICE! The #if protection ends here. *****/ diff --git a/tools/bootconfig/Makefile b/tools/bootconfig/Makefile index 90eb47c9d8de..3cb8066d5141 100644 --- a/tools/bootconfig/Makefile +++ b/tools/bootconfig/Makefile @@ -15,7 +15,7 @@ override CFLAGS += -Wall -g -I$(CURDIR)/include ALL_TARGETS := bootconfig ALL_PROGRAMS := $(patsubst %,$(OUTPUT)%,$(ALL_TARGETS)) -all: $(ALL_PROGRAMS) test +all: $(ALL_PROGRAMS) $(OUTPUT)bootconfig: main.c include/linux/bootconfig.h $(LIBSRC) $(CC) $(filter %.c,$^) $(CFLAGS) $(LDFLAGS) -o $@ @@ -27,4 +27,4 @@ install: $(ALL_PROGRAMS) install $(OUTPUT)bootconfig $(DESTDIR)$(bindir) clean: - $(RM) -f $(OUTPUT)*.o $(ALL_PROGRAMS) + rm -f $(OUTPUT)*.o $(ALL_PROGRAMS) diff --git a/tools/bootconfig/scripts/ftrace2bconf.sh b/tools/bootconfig/scripts/ftrace2bconf.sh index 1603801cf126..8eed445c295e 100755 --- a/tools/bootconfig/scripts/ftrace2bconf.sh +++ b/tools/bootconfig/scripts/ftrace2bconf.sh @@ -57,6 +57,8 @@ EOF kprobe_event_options() { cat $TRACEFS/kprobe_events | while read p args; do case $p in + \#*) + continue;; r*) cat 1>&2 << EOF # WARN: A return probe found but it is not supported by bootconfig. Skip it. diff --git a/tools/testing/selftests/ftrace/test.d/dynevent/btf_probe_event.tc b/tools/testing/selftests/ftrace/test.d/dynevent/btf_probe_event.tc new file mode 100644 index 000000000000..bf71368c31a4 --- /dev/null +++ b/tools/testing/selftests/ftrace/test.d/dynevent/btf_probe_event.tc @@ -0,0 +1,51 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0 +# description: BTF event with typecast and percpu access +# requires: dynamic_events "this_cpu_read(<fetcharg>)":README "[(structname[,field])]<argname>[->field[->field|.field...]]":README + +# Check if the sample module is loaded +if ! lsmod | grep -q trace_events_sample; then + modprobe trace-events-sample || exit_unresolved +fi + +echo 0 > events/enable +echo > dynamic_events + +# The sample_timer_cb(struct timer_list *t) is called. +# We want to check (STRUCT,FIELD)VAR typecast and this_cpu_read() access. +# (foo_timer_data,timer)t converts t to struct foo_timer_data * using container_of. +# data->counter is a per-cpu pointer to int. +# this_cpu_read(data->counter) should give the value of the counter. + +echo 'f:mysample/myevent sample_timer_cb name=(foo_timer_data,timer)t->name:string count=this_cpu_read((foo_timer_data,timer)t->counter)' >> dynamic_events + +echo 1 > events/mysample/myevent/enable +echo 1 > events/sample-trace/foo_timer_fn/enable + +sleep 2 + +echo 0 > events/mysample/myevent/enable +echo 0 > events/sample-trace/foo_timer_fn/enable + +# Compare the values. +MATCH=0 +while read line; do + if echo $line | grep -q "foo_timer_fn:"; then + NAME=`echo $line | sed 's/.*name=\([^ ]*\) .*/\1/'` + COUNT=`echo $line | sed 's/.*count=\([^ ]*\).*/\1/'` + if grep -q "myevent:.*name=\"${NAME}\" count=$COUNT" trace; then + MATCH=$((MATCH+1)) + fi + fi +done < trace + +if [ $MATCH -eq 0 ]; then + echo "No matching events found" + exit_fail +fi + +# Clean up +echo 0 > events/mysample/myevent/enable +echo 0 > events/sample-trace/foo_timer_fn/enable +echo > dynamic_events +clear_trace diff --git a/tools/testing/selftests/ftrace/test.d/dynevent/btf_typecast_accepted.tc b/tools/testing/selftests/ftrace/test.d/dynevent/btf_typecast_accepted.tc new file mode 100644 index 000000000000..dd5552727054 --- /dev/null +++ b/tools/testing/selftests/ftrace/test.d/dynevent/btf_typecast_accepted.tc @@ -0,0 +1,103 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0 +# description: BTF typecast and percpu access syntax validation +# requires: dynamic_events "this_cpu_read(<fetcharg>)":README "[(structname[,field])]<argname>[->field[->field|.field...]]":README + +KPROBES= +FPROBES= + +if grep -qF "p[:[<group>/][<event>]] <place> [<args>]" README ; then + KPROBES=yes +fi +if grep -qF "f[:[<group>/][<event>]] <func-name>[%return] [<args>]" README ; then + FPROBES=yes +fi + +if [ -z "$KPROBES" -a -z "$FPROBES" ] ; then + exit_unsupported +fi + +echo 0 > events/enable +echo > dynamic_events + +# Load trace-events-sample module if available to have per-CPU counter structure defined +if ! lsmod | grep -q trace_events_sample; then + modprobe trace-events-sample || exit_unresolved +fi + +if [ "$FPROBES" ] ; then + # 1. Test basic typecast on fprobe + echo 'f:fpevent1 vfs_read name=(file)file->f_path.dentry->d_name.name:string' >> dynamic_events + # 2. Test parenthesized typecast target on fprobe + echo 'f:fpevent2 vfs_read name=(file)(file)->f_path.dentry->d_name.name:string' >> dynamic_events + # 3. Test nested typecasts on fprobe + echo 'f:fpevent3 vfs_read name=(dentry)((file)file->f_path.dentry)->d_name.name:string' >> dynamic_events + # 4. Test container_of-style typecast with field option on fprobe + echo 'f:fpevent4 vfs_read name=(file,f_path)file->f_mode' >> dynamic_events + # 5. Test typecast on return value on fprobe + echo 'f:fpevent5 vfs_read%return name=(file)$retval->f_path.dentry->d_name.name:string' >> dynamic_events + # 6. Test $current variable support on fprobe + echo 'f:fpevent6 vfs_read pid=$current->pid' >> dynamic_events + echo 'f:fpevent7 vfs_read pid=(task_struct)$current->pid' >> dynamic_events + echo 'f:fpevent8 vfs_read pid=(task_struct,group_leader)$current->pid' >> dynamic_events + + # Test this_cpu_read and this_cpu_ptr on fprobe + echo 'f:fpevent9 sample_timer_cb name=(foo_timer_data,timer)t->name:string count=this_cpu_read((foo_timer_data,timer)t->counter)' >> dynamic_events + echo 'f:fpevent10 sample_timer_cb ptr=this_cpu_ptr((foo_timer_data,timer)t->counter)' >> dynamic_events +fi + +if [ "$KPROBES" ] ; then + # 7. Test basic typecast on kprobe + echo 'p:kpevent1 vfs_read name=(file)file->f_path.dentry->d_name.name:string' >> dynamic_events + # 8. Test parenthesized typecast target on kprobe + echo 'p:kpevent2 vfs_read name=(file)(file)->f_path.dentry->d_name.name:string' >> dynamic_events + # 9. Test nested typecasts on kprobe + echo 'p:kpevent3 vfs_read name=(dentry)((file)file->f_path.dentry)->d_name.name:string' >> dynamic_events + # 10. Test container_of-style typecast with field option on kprobe + echo 'p:kpevent4 vfs_read name=(file,f_path)file->f_mode' >> dynamic_events + # 11. Test typecast on return value on kretprobe + echo 'r:kpevent5 vfs_read name=(file)$retval->f_path.dentry->d_name.name:string' >> dynamic_events + # 12. Test $current variable support on kprobe + echo 'p:kpevent6 vfs_read pid=$current->pid' >> dynamic_events + echo 'p:kpevent7 vfs_read pid=(task_struct)$current->pid' >> dynamic_events + echo 'p:kpevent8 vfs_read pid=(task_struct,group_leader)$current->pid' >> dynamic_events + + # Test this_cpu_read and this_cpu_ptr on kprobe + echo 'p:kpevent9 sample_timer_cb name=(foo_timer_data,timer)t->name:string count=this_cpu_read((foo_timer_data,timer)t->counter)' >> dynamic_events + echo 'p:kpevent10 sample_timer_cb ptr=this_cpu_ptr((foo_timer_data,timer)t->counter)' >> dynamic_events +fi + +# Verify the events exist in dynamic_events +if [ "$FPROBES" ] ; then + grep -q "fpevent1 " dynamic_events + grep -q "fpevent2 " dynamic_events + grep -q "fpevent3 " dynamic_events + grep -q "fpevent4 " dynamic_events + grep -q "fpevent5 " dynamic_events + grep -q "fpevent6 " dynamic_events + grep -q "fpevent7 " dynamic_events + grep -q "fpevent8 " dynamic_events + if lsmod | grep -q trace_events_sample; then + grep -q "fpevent9 " dynamic_events + grep -q "fpevent10 " dynamic_events + fi +fi + +if [ "$KPROBES" ] ; then + grep -q "kpevent1 " dynamic_events + grep -q "kpevent2 " dynamic_events + grep -q "kpevent3 " dynamic_events + grep -q "kpevent4 " dynamic_events + grep -q "kpevent5 " dynamic_events + grep -q "kpevent6 " dynamic_events + grep -q "kpevent7 " dynamic_events + grep -q "kpevent8 " dynamic_events + if lsmod | grep -q trace_events_sample; then + grep -q "kpevent9 " dynamic_events + grep -q "kpevent10 " dynamic_events + fi +fi + +# Clean up +echo > dynamic_events +clear_trace diff --git a/tools/testing/selftests/ftrace/test.d/dynevent/eprobes_syntax_errors.tc b/tools/testing/selftests/ftrace/test.d/dynevent/eprobes_syntax_errors.tc index 2a680c086047..c2e3f9d19f13 100644 --- a/tools/testing/selftests/ftrace/test.d/dynevent/eprobes_syntax_errors.tc +++ b/tools/testing/selftests/ftrace/test.d/dynevent/eprobes_syntax_errors.tc @@ -10,7 +10,7 @@ check_error() { # command-with-error-pos-by-^ check_error 'e ^a.' # NO_EVENT_INFO check_error 'e ^.b' # NO_EVENT_INFO check_error 'e ^a.b' # BAD_ATTACH_EVENT -check_error 'e syscalls/sys_enter_openat ^foo' # BAD_ATTACH_ARG +check_error 'e syscalls/sys_enter_openat ^foo' # NO_EVENT_FIELD check_error 'e:^/bar syscalls/sys_enter_openat' # NO_GROUP_NAME check_error 'e:^12345678901234567890123456789012345678901234567890123456789012345/bar syscalls/sys_enter_openat' # GROUP_TOO_LONG @@ -19,11 +19,19 @@ check_error 'e:^ syscalls/sys_enter_openat' # NO_EVENT_NAME check_error 'e:foo/^12345678901234567890123456789012345678901234567890123456789012345 syscalls/sys_enter_openat' # EVENT_TOO_LONG check_error 'e:foo/^bar.1 syscalls/sys_enter_openat' # BAD_EVENT_NAME -check_error 'e:foo/bar syscalls/sys_enter_openat arg=^dfd' # BAD_FETCH_ARG check_error 'e:foo/bar syscalls/sys_enter_openat arg=^$foo' # BAD_ATTACH_ARG +check_error 'e:foo/bar syscalls/sys_enter_openat arg=^COMM' # NO_EVENT_FIELD +if grep -q "\$current.*" README; then + check_error 'e:foo/bar syscalls/sys_enter_openat arg=^current' # NO_EVENT_FIELD +fi + if grep -q '<attached-group>\.<attached-event>.*\[if <filter>\]' README; then check_error 'e:foo/bar syscalls/sys_enter_openat if ^' # NO_EP_FILTER fi +if grep -q 'this_cpu_read(<fetcharg>)' README; then + check_error 'e:foo/bar syscalls/sys_enter_openat arg=^this_cpu_read(file)' # NOSUP_PERCPU +fi + exit 0 diff --git a/tools/testing/selftests/ftrace/test.d/dynevent/fprobe_syntax_errors.tc b/tools/testing/selftests/ftrace/test.d/dynevent/fprobe_syntax_errors.tc index fee479295e2f..e9d7e6919c7f 100644 --- a/tools/testing/selftests/ftrace/test.d/dynevent/fprobe_syntax_errors.tc +++ b/tools/testing/selftests/ftrace/test.d/dynevent/fprobe_syntax_errors.tc @@ -112,6 +112,18 @@ check_error 'f vfs_read%return $retval->^foo' # NO_PTR_STRCT check_error 'f vfs_read file->^foo' # NO_BTF_FIELD check_error 'f vfs_read file^-.foo' # BAD_HYPHEN check_error 'f vfs_read ^file:string' # BAD_TYPE4STR +if grep -qF "[(structname" README ; then +check_error 'f vfs_read arg1=(task_struct)file^' # TYPECAST_REQ_FIELD +check_error 'f vfs_read arg1=(a)((b)((c)(^(d)file->d)->c)->b)->a' # TOO_MANY_NESTED +check_error 'f vfs_read arg1=(task_struct,^in_execve)file->comm' # TYPECAST_NOT_ALIGNED +check_error 'f vfs_read arg1=(task_struct,^foo_bar)file->pid' # NO_BTF_FIELD +check_error 'f vfs_read arg1=(^task_struct1234)file->pid' # NO_PTR_STRCT +check_error 'f vfs_read arg1=(task_struct,se^->group_node)file->comm' # TYPECAST_BAD_ARROW +check_error 'f vfs_read arg1=(task_struct,^->pid)file->comm' # NO_BTF_FIELD +check_error 'f vfs_read arg1=(task_struct,^.pid)file->comm' # NO_BTF_FIELD +check_error 'f vfs_read arg1=(task_struct,^.)file->comm' # NO_BTF_FIELD +check_error 'f vfs_read arg1=(task_struct)^@symbol+10->comm' # TYPECAST_SYM_OFFSET +fi fi else diff --git a/tools/testing/selftests/ftrace/test.d/kprobe/kprobe_syntax_errors.tc b/tools/testing/selftests/ftrace/test.d/kprobe/kprobe_syntax_errors.tc index 8f1c58f0c239..21ce8414459f 100644 --- a/tools/testing/selftests/ftrace/test.d/kprobe/kprobe_syntax_errors.tc +++ b/tools/testing/selftests/ftrace/test.d/kprobe/kprobe_syntax_errors.tc @@ -115,6 +115,18 @@ check_error 'p vfs_read+20 ^$arg*' # NOFENTRY_ARGS check_error 'p vfs_read ^hoge' # NO_BTFARG check_error 'p kfree ^$arg10' # NO_BTFARG (exceed the number of parameters) check_error 'r kfree ^$retval' # NO_RETVAL +if grep -qF "[(structname" README ; then +check_error 'p vfs_read arg1=(task_struct)file^' # TYPECAST_REQ_FIELD +check_error 'p vfs_read arg1=(a)((b)((c)(^(d)file->d)->c)->b)->a' # TOO_MANY_NESTED +check_error 'p vfs_read arg1=(task_struct,^in_execve)file->comm' # TYPECAST_NOT_ALIGNED +check_error 'p vfs_read arg1=(task_struct,^foo_bar)file->pid' # NO_BTF_FIELD +check_error 'p vfs_read arg1=(^task_struct1234)file->pid' # NO_PTR_STRCT +check_error 'p vfs_read arg1=(task_struct,se^->group_node)file->comm' # TYPECAST_BAD_ARROW +check_error 'p vfs_read arg1=(task_struct,^->pid)file->comm' # NO_BTF_FIELD +check_error 'p vfs_read arg1=(task_struct,^.pid)file->comm' # NO_BTF_FIELD +check_error 'p vfs_read arg1=(task_struct,^.)file->comm' # NO_BTF_FIELD +check_error 'p vfs_read arg1=(task_struct)^@symbol+10->comm' # TYPECAST_SYM_OFFSET +fi else check_error 'p vfs_read ^$arg*' # NOSUP_BTFARG fi diff --git a/tools/testing/selftests/ftrace/test.d/kprobe/uprobe_syntax_errors.tc b/tools/testing/selftests/ftrace/test.d/kprobe/uprobe_syntax_errors.tc index c817158b99db..e12dc967ec76 100644 --- a/tools/testing/selftests/ftrace/test.d/kprobe/uprobe_syntax_errors.tc +++ b/tools/testing/selftests/ftrace/test.d/kprobe/uprobe_syntax_errors.tc @@ -28,4 +28,9 @@ if grep -q ".*symstr.*" README; then check_error 'p /bin/sh:10 $stack0:^symstr' # BAD_TYPE fi +# $current is not supported by uprobe +if grep -q "\$current.*" README; then +check_error 'p /bin/sh:10 ^$current:u8' # BAD_VAR +fi + exit 0 |