Low Latency Kernel Explanatory Comments ... Roger Larsson

MusicStation Intro This explanation was given in the context of examining alternatives for a low latency patch for the Linux kernel. March 00. A related document is Benno Senoner's Low-Latency Mini-Howto.

  

Standard kernel:

  To get low latency from process wake up (done by IRQ routine) to
  the actual process running. 

A) The waken process has to have higher priority than the currently
     running, the currently running will be marked with (need_resched).

B) When the IRQ routine returns; the (need_resched) flag will be checked
    it it is true and the current thread is not running in kernel space
    it will reschedule.

C) BUT if it is running in kernel space it won't be rescheduled until
    it exits back to user space (end of executing system call). This can
    take a while... (worst case more than 100 ms)

Low latency kernel:

A) same
B) same

C) at certain _spots_ in the kernel where it is known to stay for a
   while(looping throu buffer info...) a test if the process needs
   to be rescheduled (need_resched) and a schedule call is done if
   needed.

   This code 'conditional_schedule()' gives a problem too. You leave
   the current spot in the kernel, the higher priority process starts.
   What if it makes a system call and ends up in the same kernel
   routine and modifies the same structures as the original process?

   This means you have to be extremely careful where you place those
   conditional_schedule() and the code has to be reentrant.

"Fully" preemptive kernel:

   Nowadays the kernel has to be reentrant, since the other
   processor in a SMP system may enter the same code before the first
   will leave - code segments that can not be reentrant are protected
   by spinlocks/semaphores ...
   [There is more to this - a rescheduled process might be passed in
    code by the higher priority one. This is unlikely to happen to
    SMP processes rarely runs at different speed, then they would
    not be called SMP (S = Symetrical). This situation would more
    resemble running a kernel with CPUs of different speed...
    Nice is it not? [more... this could happen with the same process
    speed too, an interrupt gets served on one of the processors and
    the other passes, ouch...]]

   Why not use this? It would allow rescheduling of any process
   executing in kernel space that is not holding a lock of any kind?

   When releasing the last lock it should also check if rescheduling
   is wanted to be able to switch as soon as possible. This part can
   be skipped - you will soon get another IRQ at least with 100 HZ
   but then it would not give you the low latencies you want...

-  It would add extra code in lock handling, and since it is executed
   very often in critical patch... = not good

A) the same

B) after handling an IRQ do not reschedule if the current process that
   (need_resched) is executing in kernel with a lock. Only then it can
   not reschedule.
   => can reschedule directly after an IRQ if we were executing in
   user space or in kernel space without holding a lock. And locks are
   held for as little time as possible.

C) We will reschedule from most places in the kernel but with a big
   penalty when it is not needed.

Ingo preemptive kernel: 

   It is a mix. Instead of making everything but places where locks are
   held preemptive. It marks portions that ARE preemptive.

	preempt_on();
	memclear_highpage_flush(page, partial, PAGE_CACHE_SIZE-partial);
	preempt_off();

   First at preempt_on() it checks for pending reschedules and anytime
   during the enclosed part it may reschedule immediately when needed.


   Note:
        preempt_on();
        preempt_off();
     Is semantically equal to:
        conditional_schedule();
     But with more overhead, a
        preempt_point();
     can be introduced.

A) the same
B) as with preemptive kernel
C) is a mix of lowlatency kernel and preemptive kernel

   You know where there are problematic code. Now you will not need
   to make a periodic check inside a loop, like in memclear_...

When a patch with all spots from lowlatency kernel is prepared it
would be a kernel with even lower latencies but with slightly less
performance...

(roger.larsson@norran.net)
 http://www.norran.net/nra02596/