kernel/git/stable/linux.git/arch/riscv/net, branch linux-7.2.y

bpf: Restrict JIT predictor flush to cBPF

2026-07-01T08:33:38+00:00

Currently predictor flush on memory reuse is done for all BPF JIT allocations, but only cBPF programs can be loaded by an unprivileged user. eBPF is privileged by default, and flushing predictors for all CPUs on every eBPF reuse penalizes the common case for no security benefit. eBPF allocations can be frequent on busy systems, only flush predictors for cBPF programs. Trampoline and dispatcher allocations also skip the flush as they are eBPF-only. Signed-off-by: Pawan Gupta Acked-by: Daniel Borkmann Signed-off-by: Daniel Borkmann

bpf: Add struct bpf_tramp_node object

2026-06-07T17:03:01+00:00

Adding struct bpf_tramp_node to decouple the link out of the trampoline attachment info. At the moment the object for attaching bpf program to the trampoline is 'struct bpf_tramp_link': struct bpf_tramp_link { struct bpf_link link; struct hlist_node tramp_hlist; u64 cookie; } The link holds the bpf_prog pointer and forces one link - one program binding logic. In following changes we want to attach program to multiple trampolines but we want to keep just one bpf_link object. Splitting struct bpf_tramp_link into: struct bpf_tramp_link { struct bpf_link link; struct bpf_tramp_node node; }; struct bpf_tramp_node { struct bpf_link *link; struct hlist_node tramp_hlist; u64 cookie; }; The 'struct bpf_tramp_link' defines standard single trampoline link and 'struct bpf_tramp_node' is the attachment trampoline object with pointer to the bpf_link object. This will allow us to define link for multiple trampolines, like: struct bpf_tracing_multi_link { struct bpf_link link; ... int nodes_cnt; struct bpf_tracing_multi_node nodes[] __counted_by(nodes_cnt); }; Cc: Hengqi Chen Signed-off-by: Jiri Olsa Link: https://lore.kernel.org/r/20260606123955.345967-9-jolsa@kernel.org Signed-off-by: Alexei Starovoitov

bpf, riscv: inline bpf_get_current_task() and bpf_get_current_task_btf()

2026-06-05T22:26:18+00:00

On RISC-V, the current task pointer is stored in the thread pointer register (tp). Emit a single `mv a5, tp` instead of a full helper call for BPF_FUNC_get_current_task and BPF_FUNC_get_current_task_btf. Register bpf_jit_inlines_helper_call() entries for both helpers so the verifier treats them as inlined, and add the expected `mv a5, tp` annotation to the riscv64 selftests. The following show changes before and after this patch. Before patch: auipc t1,0x817a # load upper PC-relative address jalr -2004(t1) # call bpf_get_current_task helper mv a5,a0 # move return value to BPF_REG_0 After patch: mv a5,tp # directly: a5 = current (tp = thread pointer) Benchmark (bpf_prog_test_run wrapping bpf_get_current_task in loop, batch=100, 10s, QEMU RISC-V): | runs/sec | helper-calls/sec | ns/call -------------+-----------+------------------+--------- Before patch | 173,490 | 17,349,090 | 57 After patch | 320,497 | 32,049,780 | 31 -------------+-----------+------------------+--------- Improvement | +84.7% | +84.7% | -45.6% Signed-off-by: Varun R Mallya Acked-by: Björn Töpel Link: https://lore.kernel.org/r/20260602205847.102825-3-varunrmallya@gmail.com Signed-off-by: Alexei Starovoitov

bpf: Pass bpf_verifier_env to JIT

2026-04-16T14:03:40+00:00

Pass bpf_verifier_env to bpf_int_jit_compile(). The follow-up patch will use env->insn_aux_data in the JIT stage to detect indirect jump targets. Since bpf_prog_select_runtime() can be called by cbpf and lib/test_bpf.c code without verifier, introduce helper __bpf_prog_select_runtime() to accept the env parameter. Remove the call to bpf_prog_select_runtime() in bpf_prog_load(), and switch to call __bpf_prog_select_runtime() in the verifier, with env variable passed. The original bpf_prog_select_runtime() is preserved for cbpf and lib/test_bpf.c, where env is NULL. Now all constants blinding calls are moved into the verifier, except the cbpf and lib/test_bpf.c cases. The instructions arrays are adjusted by bpf_patch_insn_data() function for normal cases, so there is no need to call adjust_insn_arrays() in bpf_jit_blind_constants(). Remove it. Reviewed-by: Anton Protopopov # v8 Reviewed-by: Emil Tsalapatis # v12 Acked-by: Hengqi Chen # v14 Signed-off-by: Xu Kuohai Link: https://lore.kernel.org/r/20260416064341.151802-3-xukuohai@huaweicloud.com Signed-off-by: Alexei Starovoitov

bpf: Move constants blinding out of arch-specific JITs

2026-04-16T14:03:40+00:00

During the JIT stage, constants blinding rewrites instructions but only rewrites the private instruction copy of the JITed subprog, leaving the global env->prog->insnsi and env->insn_aux_data untouched. This causes a mismatch between subprog instructions and the global state, making it difficult to use the global data in the JIT. To avoid this mismatch, and given that all arch-specific JITs already support constants blinding, move it to the generic verifier code, and switch to rewrite the global env->prog->insnsi with the global states adjusted, as other rewrites in the verifier do. This removes the constants blinding calls in each JIT, which are largely duplicated code across architectures. Since constants blinding is only required for JIT, and there are two JIT entry functions, jit_subprogs() for BPF programs with multiple subprogs and bpf_prog_select_runtime() for programs with no subprogs, move the constants blinding invocation into these two functions. In the verifier path, bpf_patch_insn_data() is used to keep global verifier auxiliary data in sync with patched instructions. A key question is whether this global auxiliary data should be restored on the failure path. Besides instructions, bpf_patch_insn_data() adjusts: - prog->aux->poke_tab - env->insn_array_maps - env->subprog_info - env->insn_aux_data For prog->aux->poke_tab, it is only used by JIT or only meaningful after JIT succeeds, so it does not need to be restored on the failure path. For env->insn_array_maps, when JIT fails, programs using insn arrays are rejected by bpf_insn_array_ready() due to missing JIT addresses. Hence, env->insn_array_maps is only meaningful for JIT and does not need to be restored. For subprog_info, if jit_subprogs fails and CONFIG_BPF_JIT_ALWAYS_ON is not enabled, kernel falls back to interpreter. In this case, env->subprog_info is used to determine subprogram stack depth. So it must be restored on failure. For env->insn_aux_data, it is freed by clear_insn_aux_data() at the end of bpf_check(). Before freeing, clear_insn_aux_data() loops over env->insn_aux_data to release jump targets recorded in it. The loop uses env->prog->len as the array length, but this length no longer matches the actual size of the adjusted env->insn_aux_data array after constants blinding. To address it, a simple approach is to keep insn_aux_data as adjusted after failure, since it will be freed shortly, and record its actual size for the loop in clear_insn_aux_data(). But since clear_insn_aux_data() uses the same index to loop over both env->prog->insnsi and env->insn_aux_data, this approach results in incorrect index for the insnsi array. So an alternative approach is adopted: clone the original env->insn_aux_data before blinding and restore it after failure, similar to env->prog. For classic BPF programs, constants blinding works as before since it is still invoked from bpf_prog_select_runtime(). Reviewed-by: Anton Protopopov # v8 Reviewed-by: Hari Bathini # powerpc jit Reviewed-by: Pu Lehui # riscv jit Acked-by: Hengqi Chen # loongarch jit Signed-off-by: Xu Kuohai Link: https://lore.kernel.org/r/20260416064341.151802-2-xukuohai@huaweicloud.com Signed-off-by: Alexei Starovoitov

bpf, riscv: Remove redundant bpf_flush_icache() after pack allocator finalize

2026-04-15T19:09:46+00:00

bpf_flush_icache() calls flush_icache_range() to clean the data cache and invalidate the instruction cache for the JITed code region. However, since commit 48a8f78c50bd ("bpf, riscv: use prog pack allocator in the BPF JIT"), this flush is redundant. bpf_jit_binary_pack_finalize() copies the JITed instructions to the ROX region via bpf_arch_text_copy() -> patch_text_nosync(), and patch_text_nosync() already calls flush_icache_range() on the written range. The subsequent bpf_flush_icache() repeats the same cache maintenance on an overlapping range. Remove the redundant bpf_flush_icache() call and its now-unused definition. Fixes: 48a8f78c50bd ("bpf, riscv: use prog pack allocator in the BPF JIT") Acked-by: Song Liu Signed-off-by: Puranjay Mohan Reviewed-by: Pu Lehui Tested-by: Paul Chaignon Link: https://lore.kernel.org/r/20260413191111.3426023-3-puranjay@kernel.org Signed-off-by: Alexei Starovoitov

Convert 'alloc_obj' family to use the new default GFP_KERNEL argument

2026-02-22T01:09:51+00:00

This was done entirely with mindless brute force, using git grep -l '\

treewide: Replace kmalloc with kmalloc_obj for non-scalar types

2026-02-21T09:02:28+00:00

This is the result of running the Coccinelle script from scripts/coccinelle/api/kmalloc_objs.cocci. The script is designed to avoid scalar types (which need careful case-by-case checking), and instead replace kmalloc-family calls that allocate struct or union object instances: Single allocations: kmalloc(sizeof(TYPE), ...) are replaced with: kmalloc_obj(TYPE, ...) Array allocations: kmalloc_array(COUNT, sizeof(TYPE), ...) are replaced with: kmalloc_objs(TYPE, COUNT, ...) Flex array allocations: kmalloc(struct_size(PTR, FAM, COUNT), ...) are replaced with: kmalloc_flex(*PTR, FAM, COUNT, ...) (where TYPE may also be *VAR) The resulting allocations no longer return "void *", instead returning "TYPE *". Signed-off-by: Kees Cook

bpf, riscv: add fsession support for trampolines

2026-02-13T22:14:27+00:00

Implement BPF_TRACE_FSESSION support in the RISC-V trampoline JIT. The logic here is similar to what we did in x86_64. In order to simply the logic, we factor out the function invoke_bpf() for fentry and fexit. Signed-off-by: Menglong Dong Tested-by: Björn Töpel Acked-by: Björn Töpel Link: https://lore.kernel.org/r/20260208053311.698352-3-dongml2@chinatelecom.cn Signed-off-by: Alexei Starovoitov Reviewed-by: Pu Lehui Signed-off-by: Alexei Starovoitov

bpf, riscv: introduce emit_store_stack_imm64() for trampoline

2026-02-13T22:14:27+00:00

Introduce a helper to store 64-bit immediate on the trampoline stack with a help of a register. Signed-off-by: Menglong Dong Tested-by: Björn Töpel Acked-by: Björn Töpel Link: https://lore.kernel.org/r/20260208053311.698352-2-dongml2@chinatelecom.cn Signed-off-by: Alexei Starovoitov Reviewed-by: Pu Lehui Signed-off-by: Alexei Starovoitov