Scheduling the CPU

Definitions:

Scheduling is the act of deciding which process or thread gets to use the CPU, and when it gets to do so.
CPU utilization is the fraction of the time that the cpu is currently running a task. CPU_Utilization = (total_wall_time - idle_time) / total_wall_time.
Total wait time is the sum for all processes of all the time ready to run but spent waiting for the CPU.

How does a scheduler work?

This is really a three part question. You should ask "How does a scheduler keep track of who can run?" and "How can a scheduler switch from one process to another?" and "What process should the scheduler pick next?"

How can a scheduler switch from one process to another?

Switching from one process to another is called a context switch. The basic idea revolves around a structure that can store the state of a running process such that it can be restored later. If done correctly, the process has no idea it was ever interrupted. This structure has been called a process block or a task structure among other things.

Save the CPU's registers (including the program counter) to the stack for the process.
Change stack to the OS's stack
Mark the process as stopped (typically by moving from the run queue to some other queue).
Choose a new process to run.
Load the memory map from the new task structure into the MMU.
Mark that process as running (typically by moving from some other queue to the run queue).
Change the stack to the new processes stack (as loaded from the task-structure)
Load all the registers from that stack
Jump to the saved location of the program counter.

Item 1 is saving the state onto the task-structure.
Item 5-9 is preparing the new process for running.

How does a scheduler keep track of who can run?

Typically, there are a set of queues. Each queue is just a linked list of pointers to task structures. Processes currently running on a CPU on on the run queue. Processes waiting for the CPU are on the ready queue. Processes waiting for a disk drive number 3 to finish are on the disk3-wait queue. Processes waiting for network IO to finish are on the network-queue. Typically there is a queue for every reason a processes might be waiting to run. There are lots of queues.

What process should the scheduler pick next?

That's tough. Here are some objectives:

The scheduler should maintain the illusion that interactive tasks are always waiting for user input.
The scheduler should maximize CPU utilization.
The scheduler should be fair (whatever that means).
The scheduler should honor human priorities.
The scheduler should minimize total wait time.
The scheduler should be simple to understand.
For multi-cpu systems, the scheduler should schedule the same process on the same CPU.
For parallel processing tasks, the scheduler should schedule all threads of a process simultainiously.
For real time systems, the scheduler should meet all deadlines.

Types of Schedulers

There are two types of schedulers.

A non-preemptive scheduler is a scheduler that will allow a process to keep a cpu until the process performs an act that releases the CPU. Under a non-preemptive scheduler, if a process wants to keep the CPU forever, it can. Typically, a process indicates it's willingness to loose the CPU by making any I/O call, or calling yield. Failing to do so on a regular basis locks up the computer. Windows95 and the MacOS are examples of this type of scheduler.
A premptive scheduler is a scheduler that will remove control of the CPU from a process whenever it chooses. Linux and Windows 2000 are examples of this.

Non-Preemptive schedulers offer more flexibilty to the application programmer, but force more responsibilty on him. They are rapidly going out of style.

Scheduler Policies

This is the important part. Having the wrong scheduler policy destroys the entire systems usability.

Policy	Preemptive?	Basic Idea	Starvation?	Advantages	Disadvantages	Used?
Optimal	No	Given a set of processes, known running times, a known objective function, and lots of time to do math, it is easy to compute the optimal schedule.	In this scheme there are a finite number of processes. The question makes little sense.	Perfect schedule	Takes O(n*n) time, and the preconditions are rarely true.	Not really
First Come First Serve	No	Put the tasks in order by arival time. Run them in that order. This is the model the cashier uses at McDonalds	Nope. Unless someone has an infinite loop, whcih is unfair.	Easy to understand. It's fair in some sense.	Makes little attempt to optimize the objectives above.	Windows 95 MacOS
Shortest Job Next	No	Learn the time requirements of all the processes. Pick the smallest.	Yep	Optimal wait-times given non-preemption	Got to learn the requirements somehow.	?
Shortest Job Next with Preemption	Duh	Learn the time requirements of all the processes. Pick the smallest. If a new process comes in the middle of an existsing one, recompute the times anyway.	Yep	Optimal wait-times	Optimal wait-times	?
Priority	If a new process arrives, but not otherwise.	Rank all the processes by arbitrary priority. Run the highest priority.	Probably, but it depends on how you assign priority. Aging prevents this.	Does what you want when you want it.	You need some scheme to assign priority	?
Shortest Deadline Next	If a new process arrives, but not otherwise.	Run the process who's deadline to finish is next	Yep, but all real time OS's have starvation	If the deadlines can be met, this is guarenteed to do it.	Someone has to tell the scheduler the deadlines. But if you're doing a real time OS, you need to do that anyway	Most real time OS's
Round Robin	Yes	Do them for a short while. Then do them again. Repeat.	No	Feels fair to most people. Easy to do.	Does not win in most benchmarks of real world tasks.	Sort of where people start, and then they add modifications to make a real scheduler.