Kernel (operating system)

The operating-system kernel is the core of an operating system, controlling access to resources and supplying mechanisms used by operating-system processes. Processor privilege levels let it enforce boundaries that ordinary programs cannot simply overwrite. The operating-system kernel connects process scheduling, virtual memory and file system operations; a system call requests a service across that boundary. A monolithic kernel and a microkernel divide services differently, so the location of every file system service is a design choice.

For the toy file system, opening a resolves its directory entry to inode 42, establishes an open file description, and returns file descriptor 3. Later reads use that file descriptor to reach the existing open file description, without resolving the spelling a again. This explains how unlinking an open file preserves access: the open reference survives the directory entry. The per-operating-system process table interpreting 3 is maintained by the operating-system kernel; 3 itself is the small integer passed by the program, not inode 42. ^[ref]

A request from a program for an operating-system kernel service. In the Unix examples here, opening, reading, linking and unlinking files request controlled changes or access to operating system resources. The operating-system kernel checks the request and performs the permitted operation; an interface specified by POSIX does not by itself dictate every internal implementation detail.

A system call creating a child operating-system process from a parent. The Linux kernel documents copy-on-write for ordinary inherited private memory pages, with separate address spaces but shared open file descriptions for inherited file descriptors. ^[ref]

A concrete operating-system kernel whose documented file system interface distinguishes a name-associated dentry, an object-associated inode, and a file object created for an open instance. An inode can have several name-associated objects. This illustrates the distinction among directory entries, inodes and open file descriptions; it does not imply identical structures in every operating system. ^[ref]

// Target: computer-science.bigb

A kernel design in which major operating-system services execute within one privileged address space and can call one another directly. Avoiding cross-domain communication can improve performance, while increasing the trusted code base and the reach of a faulty service.

// Destination: computer-science.bigb

An operating-system kernel design retaining essential protected mechanisms while moving services into user-space services. Address spaces, software threads and IPC form a useful small foundation: the operating-system kernel protects private state and controls communication, while a file system service interprets requests about files. This separation of mechanism and policy permits different service implementations to use the same protected interface. ^[ref]

The useful question is which authority a service needs. A device driver may require access to a device while having no permission to overwrite an unrelated application's computer memory. A microkernel makes such protection domains possible; their actual boundaries depend on the chosen memory protection and security capabilities.

Consider an original toy file system service in address space B and a client in address space A, on one CPU. The service holds a five-byte cached file containing LIME followed by a newline. The client already has service handle 9, permission to read that file, and a security capability to send requests to IPC endpoint E. Handle 9 is interpreted by the service; it is neither a virtual address nor an operating-system kernel object identifier. All relevant memory pages are resident, and this read needs no device driver.

This protocol is an illustrative design, not an actual seL4 file system interface. Its call/reply structure follows seL4's documented IPC: seL4_Call waits using reply authority, and small payloads can travel in processor registers. ^[ref]

An ordinary pointer in A refers to A's address space. Sending its numeric value does not make B's private computer memory appear there. This example explicitly transfers bytes. Sharing a mapped memory page instead would require intentional memory protection permissions and a protocol for ownership and synchronization; passing a pointer alone establishes neither.

The same IPC mechanism could carry WRITE, but this client could still be refused by the service's read-only handle policy. Conversely, possession of an IPC endpoint security capability permits communication without automatically granting access to every file the service manages. The separation of mechanism and policy appears twice: endpoint authority gates delivery, and the service's handle policy gates the requested operation.

Suppose the user-space service stops after receiving request 7, before replying. Assume that its private memory pages are not writable mappings of the client's or an unrelated service's computer memory, and that it lacks security capabilities to change their software threads. A failed store in the service cannot directly overwrite that private state through ordinary CPU address translation. Those assumptions establish a particular isolation boundary, not a successful read.

The client can remain blocked waiting for request 7. An unrelated runnable service may continue if process scheduling and resources permit, but clients that depend on the stopped service still lose access to its service. A separately authorized recovery manager would need to detect the failure, rebuild the service's state and arrange communication with its replacement. The microkernel architecture by itself supplies none of those application-specific recovery decisions.

A missing reply also does not reveal whether a service crashed, is delayed by process scheduling, or entered an infinite computation. For a read, retrying after recovery may be straightforward; for a modifying operation, absence of a reply does not prove that the modification never happened. Recovery must account for ambiguous completed operations and any required file-system crash consistency.

The boundary changes with authority. A deliberately shared writable memory page allows the service to change its shared contents; a security capability controlling another software thread permits the corresponding control operations. A device driver with uncontrolled DMA can bypass ordinary CPU memory protection. Thus service placement and actual permissions must be considered together. ^[ref]

For the microkernel file-read example, distinguish system call entry/exit, security capability lookup, payload transfer, process scheduling and context switches from the service's actual file lookup and read work. Entering a privileged processor privilege level and switching to a different address space are different events. A change of address space can affect the translation lookaside buffer; running different code and touching different computer memory can cause cache misses. Neither event implies that every translation lookaside buffer entry or cache line must be discarded.

A larger copied payload requires more transfer work; sharing memory pages can avoid copying that bulk data but introduces mapping and synchronization work. A service that requests another service adds another IPC exchange. If the cached file data were absent, communication with a device driver and the input-output wait would also matter. Blocking behind other requests affects observed response time even when the individual IPC operations are cheap.

These are cost sources, not timings or a universal ranking against a monolithic kernel. Implementation and CPU details matter: the construction analysis separates protected entry costs, address space switching, translation lookaside buffer effects and memory cache effects. ^[ref]

A microkernel using security capabilities to control access to operating-system kernel objects, including IPC endpoints and software threads. An IPC endpoint connects cooperating user-space services; its security capabilities are distinct from the service-defined handle 9 in the microkernel file-read example. ^[ref]

Formal verification uses machine-checked formal proofs to establish functional correctness: the C implementation of verified seL4 configurations conforms to its formal specification. Binary correctness connects compiled machine code to that C behaviour where supported. Separate security formal proofs cover integrity, resource-access availability and confidentiality under their configuration requirements; these properties are not all available in every configuration. For example, the documented confidentiality theorem separates scheduler security domains rather than arbitrary pairs of software threads. ^[ref]

The documented assumptions include correct hardware, selected handwritten machine-interface code, correct boot state, translation lookaside buffer and memory cache management, and noninterfering or trusted DMA. The confidentiality result covers information channels in its model; the assumptions page explicitly excludes timing channels from that coverage. ^[ref]

Return to READ(7, 9, 0, 4). Correct IPC delivery does not establish that the service looked up the right file, returned LIME, or ever replied. A service can produce an incorrect payload while the operating-system kernel exactly follows its formal specification. The formal proofs therefore do not automatically prove all user-space services, nor turn the resource-access availability theorem into a guarantee that this file read completes. The verified foundation supports further algorithm correctness and system reasoning; the service protocol and its recovery obligations still need their own justification.

// Destination: computer-science.bigb

Allows a running kernel execution to be involuntarily suspended for another schedulable context. Interrupt delivery and voluntary blocking are separate mechanisms.

// Target: computer-science.bigb