kthread: disable preemption during complete()
Peter Boonstoppel [Thu, 19 Jul 2012 21:58:10 +0000 (14:58 -0700)]
After a kthread is created it signals the requester using complete()
and enters TASK_UNINTERRUPTIBLE. However, since complete() wakes up
the requesting thread this can cause a preemption. The preemption will
not remove the task from the runqueue (for that schedule() has to be
invoked directly).

This is a problem if directly after kthread creation you try to do a
kthread_bind(), which will block in HZ steps until the thread is off
the runqueue.

This patch disables preemption during complete(), since we call
schedule() directly afterwards, so it will correctly enter
TASK_UNINTERRUPTIBLE. This speeds up kthread creation/binding during
cpu hotplug significantly.

Change-Id: I856ddd4e01ebdb198ba90f343b4a0c5933fd2b23
Signed-off-by: Peter Boonstoppel <pboonstoppel@nvidia.com>

Rebase-Id: Rae8fc889bf6abbe91080417be672311948b6a8dd

kernel/kthread.c

index 760e86d..9cd89ee 100644 (file)
@@ -18,6 +18,7 @@
 #include <linux/freezer.h>
 #include <linux/ptrace.h>
 #include <linux/uaccess.h>
+#include <linux/preempt.h>
 #include <trace/events/sched.h>
 
 static DEFINE_SPINLOCK(kthread_create_lock);
@@ -190,7 +191,17 @@ static int kthread(void *_create)
        /* OK, tell user we're spawned, wait for stop or wakeup */
        __set_current_state(TASK_UNINTERRUPTIBLE);
        create->result = current;
+
+       /*
+        * Disable preemption so we enter TASK_UNINTERRUPTIBLE after
+        * complete() instead of possibly being preempted. This speeds
+        * up clients that do a kthread_bind() directly after
+        * creation.
+        */
+       preempt_disable();
        complete(&create->done);
+       preempt_enable_no_resched();
+
        schedule();
 
        ret = -EINTR;