David Vernet 0e7ffff1b8 scx: Fix raciness in scx_ops_bypass() ... scx_ops_bypass() can currently race on the ops enable / disable path as follows: 1. scx_ops_bypass(true) called on enable path, bypass depth is set to 1 2. An op on the init path exits, which schedules scx_ops_disable_workfn() 3. scx_ops_bypass(false) is called on the disable path, and bypass depth is decremented to 0 4. kthread is scheduled to execute scx_ops_disable_workfn() 5. scx_ops_bypass(true) called, bypass depth set to 1 6. scx_ops_bypass() races when iterating over CPUs While it's not safe to take any blocking locks on the bypass path, it is safe to take a raw spinlock which cannot be preempted. This patch therefore updates scx_ops_bypass() to use a raw spinlock to synchronize, and changes scx_ops_bypass_depth to be a regular int. Without this change, we observe the following warnings when running the 'exit' sched_ext selftest (sometimes requires a couple of runs): .[root@virtme-ng sched_ext]# ./runner -t exit ===== START ===== TEST: exit ... [ 14.935078] WARNING: CPU: 2 PID: 360 at kernel/sched/ext.c:4332 scx_ops_bypass+0x1ca/0x280 [ 14.935126] Modules linked in: [ 14.935150] CPU: 2 UID: 0 PID: 360 Comm: sched_ext_ops_h Not tainted 6.11.0-virtme #24 [ 14.935192] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Arch Linux 1.16.3-1-1 04/01/2014 [ 14.935242] Sched_ext: exit (enabling+all) [ 14.935244] RIP: 0010:scx_ops_bypass+0x1ca/0x280 [ 14.935300] Code: ff ff ff e8 48 96 10 00 fb e9 08 ff ff ff c6 05 7b 34 e8 01 01 90 48 c7 c7 89 86 88 87 e8 be 1d f8 ff 90 0f 0b 90 90 eb 95 90 <0f> 0b 90 41 8b 84 24 24 0a 00 00 eb 97 90 0f 0b 90 41 8b 84 24 24 [ 14.935394] RSP: 0018:ffffb706c0957ce0 EFLAGS: 00010002 [ 14.935424] RAX: 0000000000000009 RBX: 0000000000000001 RCX: 00000000e3fb8b2a [ 14.935465] RDX: 0000000000000001 RSI: 0000000000000004 RDI: ffffffff88a4c080 [ 14.935512] RBP: 0000000000009b56 R08: 0000000000000004 R09: 00000003f12e520a [ 14.935555] R10: ffffffff863a9795 R11: 0000000000000000 R12: ffff8fc5fec31300 [ 14.935598] R13: ffff8fc5fec31318 R14: 0000000000000286 R15: 0000000000000018 [ 14.935642] FS: 0000000000000000(0000) GS:ffff8fc5fe680000(0000) knlGS:0000000000000000 [ 14.935684] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 14.935721] CR2: 0000557d92890b88 CR3: 000000002464a000 CR4: 0000000000750ef0 [ 14.935765] PKRU: 55555554 [ 14.935782] Call Trace: [ 14.935802] <TASK> [ 14.935823] ? __warn+0xce/0x220 [ 14.935850] ? scx_ops_bypass+0x1ca/0x280 [ 14.935881] ? report_bug+0xc1/0x160 [ 14.935909] ? handle_bug+0x61/0x90 [ 14.935934] ? exc_invalid_op+0x1a/0x50 [ 14.935959] ? asm_exc_invalid_op+0x1a/0x20 [ 14.935984] ? raw_spin_rq_lock_nested+0x15/0x30 [ 14.936019] ? scx_ops_bypass+0x1ca/0x280 [ 14.936046] ? srso_alias_return_thunk+0x5/0xfbef5 [ 14.936081] ? __pfx_scx_ops_disable_workfn+0x10/0x10 [ 14.936111] scx_ops_disable_workfn+0x146/0xac0 [ 14.936142] ? finish_task_switch+0xa9/0x2c0 [ 14.936172] ? srso_alias_return_thunk+0x5/0xfbef5 [ 14.936211] ? __pfx_scx_ops_disable_workfn+0x10/0x10 [ 14.936244] kthread_worker_fn+0x101/0x2c0 [ 14.936268] ? __pfx_kthread_worker_fn+0x10/0x10 [ 14.936299] kthread+0xec/0x110 [ 14.936327] ? __pfx_kthread+0x10/0x10 [ 14.936351] ret_from_fork+0x37/0x50 [ 14.936374] ? __pfx_kthread+0x10/0x10 [ 14.936400] ret_from_fork_asm+0x1a/0x30 [ 14.936427] </TASK> [ 14.936443] irq event stamp: 21002 [ 14.936467] hardirqs last enabled at (21001): [<ffffffff863aa35f>] resched_cpu+0x9f/0xd0 [ 14.936521] hardirqs last disabled at (21002): [<ffffffff863dd0ba>] scx_ops_bypass+0x11a/0x280 [ 14.936571] softirqs last enabled at (20642): [<ffffffff863683d7>] __irq_exit_rcu+0x67/0xd0 [ 14.936622] softirqs last disabled at (20637): [<ffffffff863683d7>] __irq_exit_rcu+0x67/0xd0 [ 14.936672] ---[ end trace 0000000000000000 ]--- [ 14.953282] sched_ext: BPF scheduler "exit" disabled (unregistered from BPF) [ 14.953352] ------------[ cut here ]------------ [ 14.953383] WARNING: CPU: 2 PID: 360 at kernel/sched/ext.c:4335 scx_ops_bypass+0x1d8/0x280 [ 14.953428] Modules linked in: [ 14.953453] CPU: 2 UID: 0 PID: 360 Comm: sched_ext_ops_h Tainted: G W 6.11.0-virtme #24 [ 14.953505] Tainted: [W]=WARN [ 14.953527] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Arch Linux 1.16.3-1-1 04/01/2014 [ 14.953574] RIP: 0010:scx_ops_bypass+0x1d8/0x280 [ 14.953603] Code: c6 05 7b 34 e8 01 01 90 48 c7 c7 89 86 88 87 e8 be 1d f8 ff 90 0f 0b 90 90 eb 95 90 0f 0b 90 41 8b 84 24 24 0a 00 00 eb 97 90 <0f> 0b 90 41 8b 84 24 24 0a 00 00 eb 92 f3 0f 1e fa 49 8d 84 24 f0 [ 14.953693] RSP: 0018:ffffb706c0957ce0 EFLAGS: 00010046 [ 14.953722] RAX: 0000000000000001 RBX: 0000000000000000 RCX: 0000000000000001 [ 14.953763] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff8fc5fec31318 [ 14.953804] RBP: 0000000000000000 R08: 0000000000000001 R09: 0000000000000000 [ 14.953845] R10: ffffffff863a9795 R11: 0000000000000000 R12: ffff8fc5fec31300 [ 14.953888] R13: ffff8fc5fec31318 R14: 0000000000000286 R15: 0000000000000018 [ 14.953934] FS: 0000000000000000(0000) GS:ffff8fc5fe680000(0000) knlGS:0000000000000000 [ 14.953974] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 14.954009] CR2: 0000557d92890b88 CR3: 000000002464a000 CR4: 0000000000750ef0 [ 14.954052] PKRU: 55555554 [ 14.954068] Call Trace: [ 14.954085] <TASK> [ 14.954102] ? __warn+0xce/0x220 [ 14.954126] ? scx_ops_bypass+0x1d8/0x280 [ 14.954150] ? report_bug+0xc1/0x160 [ 14.954178] ? handle_bug+0x61/0x90 [ 14.954203] ? exc_invalid_op+0x1a/0x50 [ 14.954226] ? asm_exc_invalid_op+0x1a/0x20 [ 14.954250] ? raw_spin_rq_lock_nested+0x15/0x30 [ 14.954285] ? scx_ops_bypass+0x1d8/0x280 [ 14.954311] ? __mutex_unlock_slowpath+0x3a/0x260 [ 14.954343] scx_ops_disable_workfn+0xa3e/0xac0 [ 14.954381] ? __pfx_scx_ops_disable_workfn+0x10/0x10 [ 14.954413] kthread_worker_fn+0x101/0x2c0 [ 14.954442] ? __pfx_kthread_worker_fn+0x10/0x10 [ 14.954479] kthread+0xec/0x110 [ 14.954507] ? __pfx_kthread+0x10/0x10 [ 14.954530] ret_from_fork+0x37/0x50 [ 14.954553] ? __pfx_kthread+0x10/0x10 [ 14.954576] ret_from_fork_asm+0x1a/0x30 [ 14.954603] </TASK> [ 14.954621] irq event stamp: 21002 [ 14.954644] hardirqs last enabled at (21001): [<ffffffff863aa35f>] resched_cpu+0x9f/0xd0 [ 14.954686] hardirqs last disabled at (21002): [<ffffffff863dd0ba>] scx_ops_bypass+0x11a/0x280 [ 14.954735] softirqs last enabled at (20642): [<ffffffff863683d7>] __irq_exit_rcu+0x67/0xd0 [ 14.954782] softirqs last disabled at (20637): [<ffffffff863683d7>] __irq_exit_rcu+0x67/0xd0 [ 14.954829] ---[ end trace 0000000000000000 ]--- [ 15.022283] sched_ext: BPF scheduler "exit" disabled (unregistered from BPF) [ 15.092282] sched_ext: BPF scheduler "exit" disabled (unregistered from BPF) [ 15.149282] sched_ext: BPF scheduler "exit" disabled (unregistered from BPF) ok 1 exit # ===== END ===== And with it, the test passes without issue after 1000s of runs: .[root@virtme-ng sched_ext]# ./runner -t exit ===== START ===== TEST: exit DESCRIPTION: Verify we can cleanly exit a scheduler in multiple places OUTPUT: [ 7.412856] sched_ext: BPF scheduler "exit" enabled [ 7.427924] sched_ext: BPF scheduler "exit" disabled (unregistered from BPF) [ 7.466677] sched_ext: BPF scheduler "exit" enabled [ 7.475923] sched_ext: BPF scheduler "exit" disabled (unregistered from BPF) [ 7.512803] sched_ext: BPF scheduler "exit" enabled [ 7.532924] sched_ext: BPF scheduler "exit" disabled (unregistered from BPF) [ 7.586809] sched_ext: BPF scheduler "exit" enabled [ 7.595926] sched_ext: BPF scheduler "exit" disabled (unregistered from BPF) [ 7.661923] sched_ext: BPF scheduler "exit" disabled (unregistered from BPF) [ 7.723923] sched_ext: BPF scheduler "exit" disabled (unregistered from BPF) ok 1 exit # ===== END ===== ============================= RESULTS: PASSED: 1 SKIPPED: 0 FAILED: 0 Fixes: f0e1a0643a ("sched_ext: Implement BPF extensible scheduler class") Signed-off-by: David Vernet <void@manifault.com> Signed-off-by: Tejun Heo <tj@kernel.org>		2024-10-25 11:10:51 -10:00
..
bpf	struct fd layout change (and conversion to accessor helpers)	2024-09-23 09:35:36 -07:00
cgroup	struct fd layout change (and conversion to accessor helpers)	2024-09-23 09:35:36 -07:00
configs	tinyconfig: remove unnecessary 'is not set' for choice blocks	2024-09-01 20:34:38 +09:00
debug
dma	dma-mapping: report unlimited DMA addressing in IOMMU DMA path	2024-09-23 08:38:56 +02:00
entry
events	struct fd layout change (and conversion to accessor helpers)	2024-09-23 09:35:36 -07:00
futex	fault-inject: improve build for CONFIG_FAULT_INJECTION=n	2024-09-01 20:43:33 -07:00
gcov
irq	pci-v6.12-changes	2024-09-23 12:47:06 -07:00
kcsan
livepatch
locking	Many singleton patches - please see the various changelogs for details.	2024-09-21 08:20:50 -07:00
module	struct fd layout change (and conversion to accessor helpers)	2024-09-23 09:35:36 -07:00
power	PM: hibernate: Remove unused stub for saveable_highmem_page()	2024-09-10 20:11:40 +02:00
printk	drm next for 6.12-rc1	2024-09-19 10:18:15 +02:00
rcu	slab updates for 6.12	2024-09-18 08:53:53 +02:00
sched	scx: Fix raciness in scx_ops_bypass()	2024-10-25 11:10:51 -10:00
time	In the v6.12 scheduler development cycle we had 63 commits from 18 contributors:	2024-09-19 15:55:58 +02:00
trace	ring-buffer: Updates for v6.12:	2024-09-22 09:47:16 -07:00
.gitignore
acct.c
async.c
audit.c	audit: Make use of str_enabled_disabled() helper	2024-09-03 16:35:16 -04:00
audit.h
audit_fsnotify.c
audit_tree.c
audit_watch.c
auditfilter.c	audit: use task_tgid_nr() instead of task_pid_nr()	2024-08-28 16:48:28 -04:00
auditsc.c	audit: use task_tgid_nr() instead of task_pid_nr()	2024-08-28 16:48:28 -04:00
backtracetest.c
bounds.c
capability.c
cfi.c
compat.c
configs.c
context_tracking.c
cpu.c	Updates for timers and timekeeping:	2024-09-17 07:25:37 +02:00
cpu_pm.c
crash_core.c	Document/kexec: generalize crash hotplug description	2024-09-01 20:43:37 -07:00
crash_reserve.c	crash: fix crash memory reserve exceed system memory bug	2024-09-01 20:43:30 -07:00
cred.c
delayacct.c
dma.c
elfcorehdr.c
exec_domain.c
exit.c	ALong with the usual shower of singleton patches, notable patch series in	2024-09-21 07:29:05 -07:00
exit.h
extable.c
fail_function.c
fork.c	sched_ext: Initial pull request for v6.12	2024-09-21 09:44:57 -07:00
freezer.c
gen_kheaders.sh
groups.c
hung_task.c
iomem.c
irq_work.c
jump_label.c
kallsyms.c
kallsyms_internal.h
kallsyms_selftest.c
kallsyms_selftest.h
kcmp.c
Kconfig.freezer
Kconfig.hz
Kconfig.kexec
Kconfig.locks
Kconfig.preempt
kcov.c	Updates for KCOV instrumentation on x86:	2024-09-17 12:40:34 +02:00
kexec.c
kexec_core.c
kexec_elf.c
kexec_file.c	kexec_file: fix elfcorehdr digest exclusion when CONFIG_CRASH_HOTPLUG=y	2024-09-01 17:59:01 -07:00
kexec_internal.h	kexec: use atomic_try_cmpxchg_acquire() in kexec_trylock()	2024-09-01 20:43:23 -07:00
kheaders.c
kprobes.c
ksyms_common.c
ksysfs.c
kthread.c	kthread: Fix task state in kthread worker if being frozen	2024-09-10 09:51:14 +02:00
latencytop.c
Makefile	mm: move kernel/numa.c to mm/	2024-09-03 21:15:26 -07:00
module_signature.c
notifier.c
nsproxy.c
padata.c	This update includes the following changes:	2024-09-16 06:28:28 +02:00
panic.c	drm next for 6.12-rc1	2024-09-19 10:18:15 +02:00
params.c
pid.c
pid_namespace.c
pid_sysctl.h
profile.c
ptrace.c
range.c
reboot.c
regset.c
relay.c
resource.c	ALong with the usual shower of singleton patches, notable patch series in	2024-09-21 07:29:05 -07:00
resource_kunit.c	resource, kunit: add test case for region_intersects()	2024-09-17 01:07:00 -07:00
rseq.c
scftorture.c
scs.c
seccomp.c
signal.c	struct fd layout change (and conversion to accessor helpers)	2024-09-23 09:35:36 -07:00
smp.c
smpboot.c
smpboot.h
softirq.c
stackleak.c
stacktrace.c
static_call.c
static_call_inline.c
stop_machine.c
sys.c	struct fd layout change (and conversion to accessor helpers)	2024-09-23 09:35:36 -07:00
sys_ni.c
sysctl-test.c
sysctl.c
task_work.c
taskstats.c
torture.c
tracepoint.c
tsacct.c
ucount.c
uid16.c
uid16.h
umh.c
up.c
user-return-notifier.c
user.c	uidgid: make sure we fit into one cacheline	2024-09-12 12:16:09 +02:00
user_namespace.c	user_namespace: use kmemdup_array() instead of kmemdup() for multiple allocation	2024-09-09 16:47:42 -07:00
usermode_driver.c
utsname.c
utsname_sysctl.c
vhost_task.c
vmcore_info.c	mm: support only one page_type per page	2024-09-03 21:15:43 -07:00
watch_queue.c
watchdog.c	watchdog: handle the ENODEV failure case of lockup_detector_delay_init() separately	2024-09-01 20:43:32 -07:00
watchdog_buddy.c
watchdog_perf.c
workqueue.c	workqueue: Changes for v6.12	2024-09-18 06:59:44 +02:00
workqueue_internal.h