kernel/git/next/linux-next.git/drivers/nvdimm/virtio_pmem.c, branch master

nvdimm: virtio_pmem: drain requests in freeze

2026-08-19T10:38:46+00:00

virtio_pmem_freeze() currently deletes virtqueues and resets the device without waking threads waiting for a virtqueue descriptor or a host completion. Mark the request virtqueue broken before reset. This makes new submissions fail fast and lets -ENOSPC waiters leave the wait list. Reset the device before draining used and unused request tokens, then delete the virtqueues. This wakes waiters with -EIO. It also keeps the detach call on a quiesced device. Clear req_vq after del_vqs(). Make drain tolerate a NULL queue so remove after freeze does not dereference a stale virtqueue pointer. Also make virtio_pmem_flush() stop checking req_vq once the broken state is visible. A waiter woken by freeze/remove can resume after del_vqs() has cleared req_vq. Signed-off-by: Li Chen Signed-off-by: Michael S. Tsirkin Message-ID: <20260630092338.2094628-13-me@linux.beauty>

nvdimm: virtio_pmem: converge broken virtqueue to -EIO

2026-08-19T10:38:46+00:00

dmesg reports virtqueue failure and device reset: virtio_pmem virtio2: failed to send command to virtio pmem device, no free slots in the virtqueue virtio_pmem virtio2: virtio pmem device needs a reset virtio_pmem_flush() can wait for a free virtqueue descriptor (-ENOSPC). It can also wait for host completion. If the request virtqueue breaks, those waiters may never make progress. One example is notify failure from virtqueue_kick(). Track a device-level broken state and converge the failure to -EIO. New requests fail fast, -ENOSPC waiters are unlinked and woken, and the currently submitted request is woken so its host_acked waiter can return without waiting forever for host completion. Completed requests are forced to report an error after the queue is marked broken. Also serialize async parent-bio flush work against the broken state with pmem_lock. That way remove and freeze either drain work queued before virtio_pmem_mark_broken(), or later callers see nvdimm_flush() complete the parent bio synchronously with -EIO instead of queuing work after the drain point. Do not detach unused buffers from an active virtqueue. Runtime broken-queue handling only stops new submissions and wakes local waiters. Removal resets the device first. It then drains request tokens. After that, the device no longer owns the buffers when the virtqueue reference is dropped. Closes: https://lore.kernel.org/r/202512250116.ewtzlD0g-lkp@intel.com/ Signed-off-by: Li Chen Link: https://lore.kernel.org/r/202512250116.ewtzlD0g-lkp@intel.com/ Signed-off-by: Michael S. Tsirkin Message-ID: <20260630092338.2094628-12-me@linux.beauty>

nvdimm: virtio_pmem: stop allocating child flush bio

2026-08-19T10:38:45+00:00

pmem_submit_bio() passes the parent bio to nvdimm_flush() for REQ_FUA. For virtio-pmem this makes async_pmem_flush() allocate and submit a child PREFLUSH bio chained to the parent. That child allocation is in the block submit path. Making it blocking with GFP_NOIO can consume the same global bio mempool that submit_bio() uses, while making it GFP_ATOMIC can fail under pressure. A forced failure of the child allocation produced: virtio_pmem: forcing child bio allocation failure for test Buffer I/O error on dev pmem0, logical block 0, lost sync page write EXT4-fs (pmem0): I/O error while writing superblock EXT4-fs (pmem0): mount failed Avoid the child bio without turning REQ_FUA into a synchronous submit-path wait. Let provider flush callbacks return NVDIMM_FLUSH_ASYNC after taking ownership of parent bio completion. pmem_submit_bio() returns in that case, and virtio-pmem queues an ordered WQ_MEM_RECLAIM work item that runs the existing host flush path and completes the parent bio. This keeps the asynchronous completion model of the child-bio path while removing the child bio allocation from the submit path. Signed-off-by: Li Chen Signed-off-by: Michael S. Tsirkin Message-ID: <20260630092338.2094628-5-me@linux.beauty>

nvdimm: virtio_pmem: serialize flush requests

2026-02-04T19:16:40+00:00

Under heavy concurrent flush traffic, virtio-pmem can overflow its request virtqueue (req_vq): virtqueue_add_sgs() starts returning -ENOSPC and the driver logs "no free slots in the virtqueue". Shortly after that the device enters VIRTIO_CONFIG_S_NEEDS_RESET and flush requests fail with "virtio pmem device needs a reset". Serialize virtio_pmem_flush() with a per-device mutex so only one flush request is in-flight at a time. This prevents req_vq descriptor overflow under high concurrency. Reproducer (guest with virtio-pmem): - mkfs.ext4 -F /dev/pmem0 - mount -t ext4 -o dax,noatime /dev/pmem0 /mnt/bench - fio: ioengine=io_uring rw=randwrite bs=4k iodepth=64 numjobs=64 direct=1 fsync=1 runtime=30s time_based=1 - dmesg: "no free slots in the virtqueue" "virtio pmem device needs a reset" Fixes: 6e84200c0a29 ("virtio-pmem: Add virtio pmem driver") Signed-off-by: Li Chen Acked-by: Pankaj Gupta Acked-by: Michael S. Tsirkin Link: https://patch.msgid.link/20260203021353.121091-1-me@linux.beauty Signed-off-by: Ira Weiny

virtio_pmem: Add freeze/restore callbacks

2024-11-12T23:07:24+00:00

Add basic freeze/restore PM callbacks to support hibernation (S4): - On freeze, delete vq and quiesce the device to prepare for snapshotting. - On restore, re-init vq and mark DRIVER_OK. Signed-off-by: Philip Chen Message-Id: <20240815004617.2325269-1-philipchen@chromium.org> Signed-off-by: Michael S. Tsirkin

nvdimm: virtio_pmem: drop owner assignment

2024-05-22T12:31:18+00:00

virtio core already sets the .owner, so driver does not need to. Acked-by: Dave Jiang Signed-off-by: Krzysztof Kozlowski Message-Id: <20240331-module-owner-virtio-v2-21-98f04bfaf46a@linaro.org> Signed-off-by: Michael S. Tsirkin Reviewed-by: Pankaj Gupta

virtio_pmem: support feature SHMEM_REGION

2024-01-10T18:01:37+00:00

This patch adds the support for feature VIRTIO_PMEM_F_SHMEM_REGION (virtio spec v1.2 section 5.19.5.2 [1]). During feature negotiation, if VIRTIO_PMEM_F_SHMEM_REGION is offered by the device, the driver looks for a shared memory region of id 0. If it is found, this feature is understood. Otherwise, this feature bit is cleared. During probe, if VIRTIO_PMEM_F_SHMEM_REGION has been negotiated, virtio pmem ignores the `start` and `size` fields in device config and uses the physical address range of shared memory region 0. [1] https://docs.oasis-open.org/virtio/virtio/v1.2/csd01/virtio-v1.2-csd01.html#x1-6480002 Signed-off-by: Changyuan Lyu Message-Id: <20231220204906.566922-1-changyuanl@google.com> Signed-off-by: Michael S. Tsirkin Acked-by: Jason Wang

virtio_pmem: populate numa information

2023-02-15T11:46:22+00:00

Compute the numa information for a virtio_pmem device from the memory range of the device. Previously, the target_node was always 0 since the ndr_desc.target_node field was never explicitly set. The code for computing the numa node is taken from cxl_pmem_region_probe in drivers/cxl/pmem.c. Signed-off-by: Michael Sammler Reviewed-by: Pasha Tatashin Reviewed-by: Pankaj Gupta Tested-by: Mina Almasry Message-Id: <20221115214036.1571015-1-sammler@google.com> Signed-off-by: Michael S. Tsirkin

virtio_pmem: set device ready in probe()

2022-08-11T08:06:37+00:00

The NVDIMM region could be available before the virtio_device_ready() that is called by virtio_dev_probe(). This means the driver tries to use device before DRIVER_OK which violates the spec, fixing this by set device ready before the nvdimm_pmem_region_create(). Note that this means the virtio_pmem_host_ack() could be triggered before the creation of the nd region, this is safe since the pmem_lock has been initialized and whether or not any available buffer is added before is validated by virtio_pmem_host_ack(). Fixes 6e84200c0a29 ("virtio-pmem: Add virtio pmem driver") Acked-by: Pankaj Gupta Signed-off-by: Jason Wang Message-Id: <20220628083430.61856-2-jasowang@redhat.com> Signed-off-by: Michael S. Tsirkin

virtio_pmem: initialize provider_data through nd_region_desc

2022-08-11T08:06:37+00:00

We used to initialize the provider_data manually after nvdimm_pemm_region_create(). This seems to be racy if the flush is issued before the initialization of provider_data[1]. Fixing this by initializing the provider_data through nd_region_desc to make sure the provider_data is ready after the pmem is created. [1]: [ 80.152281] nd_pmem namespace0.0: unable to guarantee persistence of writes [ 92.393956] BUG: kernel NULL pointer dereference, address: 0000000000000318 [ 92.394551] #PF: supervisor read access in kernel mode [ 92.394955] #PF: error_code(0x0000) - not-present page [ 92.395365] PGD 0 P4D 0 [ 92.395566] Oops: 0000 [#1] PREEMPT SMP PTI [ 92.395867] CPU: 2 PID: 506 Comm: mkfs.ext4 Not tainted 5.19.0-rc1+ #453 [ 92.396365] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.16.0-0-gd239552ce722-prebuilt.qemu.org 04/01/2014 [ 92.397178] RIP: 0010:virtio_pmem_flush+0x2f/0x1f0 [ 92.397521] Code: 55 41 54 55 53 48 81 ec a0 00 00 00 65 48 8b 04 25 28 00 00 00 48 89 84 24 98 00 00 00 31 c0 48 8b 87 78 03 00 00 48 89 04 24 <48> 8b 98 18 03 00 00 e8 85 bf 6b 00 ba 58 00 00 00 be c0 0c 00 00 [ 92.398982] RSP: 0018:ffff9a7380aefc88 EFLAGS: 00010246 [ 92.399349] RAX: 0000000000000000 RBX: ffff8e77c3f86f00 RCX: 0000000000000000 [ 92.399833] RDX: ffffffffad4ea720 RSI: ffff8e77c41e39c0 RDI: ffff8e77c41c5c00 [ 92.400388] RBP: ffff8e77c41e39c0 R08: ffff8e77c19f0600 R09: 0000000000000000 [ 92.400874] R10: 0000000000000000 R11: 0000000000000000 R12: ffff8e77c0814e28 [ 92.401364] R13: 0000000000000000 R14: 0000000000000000 R15: ffff8e77c41e39c0 [ 92.401849] FS: 00007f3cd75b2780(0000) GS:ffff8e7937d00000(0000) knlGS:0000000000000000 [ 92.402423] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 92.402821] CR2: 0000000000000318 CR3: 0000000103c80002 CR4: 0000000000370ee0 [ 92.403307] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 92.403793] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ 92.404278] Call Trace: [ 92.404481] [ 92.404654] ? mempool_alloc+0x5d/0x160 [ 92.404939] ? terminate_walk+0x5f/0xf0 [ 92.405226] ? bio_alloc_bioset+0xbb/0x3f0 [ 92.405525] async_pmem_flush+0x17/0x80 [ 92.405806] nvdimm_flush+0x11/0x30 [ 92.406067] pmem_submit_bio+0x1e9/0x200 [ 92.406354] __submit_bio+0x80/0x120 [ 92.406621] submit_bio_noacct_nocheck+0xdc/0x2a0 [ 92.406958] submit_bio_wait+0x4e/0x80 [ 92.407234] blkdev_issue_flush+0x31/0x50 [ 92.407526] ? punt_bios_to_rescuer+0x230/0x230 [ 92.407852] blkdev_fsync+0x1e/0x30 [ 92.408112] do_fsync+0x33/0x70 [ 92.408354] __x64_sys_fsync+0xb/0x10 [ 92.408625] do_syscall_64+0x43/0x90 [ 92.408895] entry_SYSCALL_64_after_hwframe+0x46/0xb0 [ 92.409257] RIP: 0033:0x7f3cd76c6c44 Fixes 6e84200c0a29 ("virtio-pmem: Add virtio pmem driver") Acked-by: Pankaj Gupta Reviewed-by: Dan Williams Signed-off-by: Jason Wang Message-Id: <20220628083430.61856-1-jasowang@redhat.com> Signed-off-by: Michael S. Tsirkin