The Operating System: the Kernel, Processes, Files and Permissions

In this lesson#

This lesson is built from a few short chapters, in the same order as the slides:

1. Chapter 1 — what an operating system is and what the kernel is
2. Chapter 2 — a program versus a process, and memory management
3. Chapter 3 — the file system, users and permissions

Chapter 1: What an Operating System Is and What the Kernel Is#

What will we learn?#

In the previous lesson we saw that the operating system “finds the file, creates a process and loads the program into it”. Now we will get to know it properly:

• What an operating system (OS for short) is.
• What it manages, and why.
• What the kernel is — the core of the operating system.
• How our code “asks” it for things, through a system call.

Why do we need this?#

Every print, every file you open, every internet connection and, in the backend and databases topics, every server — everything goes through the operating system. When you get an error like PermissionError or Address already in use, it comes from the operating system. To understand the error, you need to understand who sent it and why.

In addition, most of the world’s servers run on Linux. Even if you work on Windows, you will meet Linux at work, so it is important to understand the ideas shared by all operating systems.

Explanation from the basics#

What is an operating system?#

An operating system is the first program that starts when you turn on a computer, and the last one to shut down. Its job: to manage the computer’s resources and let other programs use them safely and fairly.

Examples: Windows, macOS, Linux (Ubuntu, Debian…), Android, iOS.

The five main jobs#

What is the kernel?#

The kernel is the central and most protected part of the operating system. It is loaded into RAM first and stays there the whole time. It is what really carries out the five jobs above.

Everything else, such as the desktop, the file explorer, the terminal and the browser, are ordinary programs that run “on top of” the kernel.

Kernel mode versus user mode#

The processor can work in two modes:

• Kernel mode — full access to all the hardware and all the memory. Only the kernel runs like this.
• User mode — limited access. All our programs, including Python, run like this.

Why? If every program could access the hardware directly, one bug in one program could bring down the whole computer, and a virus could read your bank’s memory. The separation protects the system.

System call — asking the kernel#

So how does a program in user mode write to a file or send data to the network? It asks the kernel. That request is called a system call.

Your Python code:       print("Hi")
        │
Python interpreter:     "I need to write 'Hi' to the screen"
        │   System Call (write)
        ▼
Kernel (Kernel Mode):   checks permissions → talks to the screen driver
        │
        ▼
Hardware:               the text appears

Common operating systems#

They all do the same five jobs — processes, memory, files, users and permissions, hardware — and they all have a kernel. The differences are mainly in the interface, the terminal commands and the folder structure (for example C:\ on Windows versus / on Mac and Linux).

Why does it matter to developers? Code you write on Windows will often end up running on a Linux server in the cloud. So it is worth knowing the differences — and writing code that does not depend on one operating system.

Common mistake#

The mistake: trying to touch system files “just to check”.

A regular user’s program tries, for example, to delete a file that belongs to the operating system (in the C:\Windows\System32 folder). What happens:

1. The program runs in user mode, so it does not access the disk itself — it sends a system call to the kernel.
2. The kernel checks the permissions, and sees that the regular user has no write permission for this file.
3. The request is refused, and the program gets a permission error (“Access is denied” / PermissionError).

And that is good: permissions protect the computer from users’ mistakes and from malicious software. A permission error does not come from your program — it comes from the operating system, which refused.

Summary#

• An operating system manages the computer’s resources: processes, memory, files, users and permissions, and hardware.
• The kernel is its protected heart and runs in kernel mode. Our programs run in user mode.
• A program asks the kernel for a service through a system call: writing to the screen, opening a file, communication.
• Permission errors come from the operating system — it is the one that checks and refuses.
• Windows, macOS and Linux on computers and servers, Android and iOS on phones — all with the same basic jobs.

Official documentation#

• User mode and kernel mode

Chapter 2: A Program Versus a Process, and Memory Management#

What will we learn?#

• The difference between a program and a process.
• How the operating system runs dozens of processes “at once” — scheduling.
• How it manages RAM — memory management.
• How to see and stop processes from the terminal.

Why do we need this?#

• In the backend and databases topics we will run a server. A server is a process that does not end. If you run it twice, the second one will fail, because the first one is still running and holding its address. Without understanding processes, this looks like magic.
• When a program “freezes”, you need to know how to find the process and stop it.
• A memory leak, that is, a program whose memory only grows, is a bug that is hunted with process-monitoring tools.

Explanation from the basics#

Program — the recipe. Process — the cooking.#

• Program = a file on storage that contains instructions. It is static: it does nothing, it just lies on the disk. Examples: python.exe, chrome.exe, hello.py.
• Process = a running instance of a program. It is dynamic: it has memory in RAM, a PID, and a state (running / waiting / ending).

An important point: one program can run several processes in parallel. Opened 3 VS Code windows? There is one program and 3 processes (in practice more, because modern browsers and editors split themselves into several processes).

What does every process have?#

Scheduling — how does everyone run “at once”?#

You may have 8 cores, but 200 processes are running on the computer. How?

The scheduler, part of the kernel, gives every process a very short time slice on the CPU (milliseconds), and then moves on to the next process. It happens so fast that it looks as if they all run at the same time.

Time →   | P1 | P2 | P3 | P1 | P2 | P3 | P1 | ...
          each slice ≈ a few milliseconds

A process that is waiting (for example on time.sleep or for an answer from the network) gets no CPU time at all until the wait is over. That is why a server waiting for requests uses almost no CPU.

Memory management#

The operating system does three main things with memory:

1. Allocation — when a process is created, and every time it needs more memory (for example a list that grows), the OS gives it more RAM.
2. Isolation — every process sees only its own memory. One process cannot read or change another process’s variables. This prevents a bug in one app from corrupting another app.
3. Release — when a process ends, all its RAM is returned to the system.

And if RAM runs out? The operating system moves unused parts to the disk (swap on Linux/Mac, page file on Windows). That prevents a crash, but the computer becomes very slow, because the disk is slow.

The process tree — who started whom#

Every process is created by another process — its parent. When you run a program from the terminal, the terminal is the parent:

explorer.exe          PID 4120
└── pwsh.exe          PID 9920   (Terminal)
    └── python.exe    PID 18044  (our program)

Every process has its own PID, and each also has a PPID — the parent’s PID. When you close the terminal, the processes it started usually close too.

Commands for viewing and stopping processes#

You can see processes in Task Manager, and also from the terminal. The instructor will show the commands on the screen:

Windows (PowerShell):

Get-Process                    # list all processes
Get-Process python             # only processes named python
Stop-Process -Id 12345         # stop the process with PID 12345

Mac / Linux (Bash / Zsh):

ps aux                  # list all processes
ps aux | grep python    # filter lines that contain "python"
kill 12345              # stop the process with PID 12345

Common mistake#

The mistake: thinking that Ctrl + C caused an “error”. When you stop a running Python program with Ctrl + C, something like this appears:

^C
Traceback (most recent call last):
  ...
KeyboardInterrupt

• Ctrl + C sends the process a request to stop — it is not a bug.
• KeyboardInterrupt says: “you stopped me in the middle”.
• Any value that was only in the process’s RAM — is lost, because its memory was released.

Summary#

• A program is a static file on the disk. A process is a running instance of it, with a PID, memory and a state.
• One program can run several processes, each with its own isolated memory.
• The scheduler shares CPU time in small slices, so everything looks parallel.
• Memory management: allocation, isolation and release. When RAM runs out, swap / the page file is used, and that is slow.
• Every process has a parent. Stop a program with Ctrl + C, and a stuck process with Stop-Process / kill by PID.

Official documentation#

• About Processes and Threads
• Virtual Address Space

Chapter 3: The File System, Users and Permissions#

What will we learn?#

• The file system — how the operating system organizes the disk.
• Paths — absolute versus relative, and what . and .. are.
• Users — who is logged in, and who the “powerful” user is.
• Permissions — who may do what with each file.

Why do we need this?#

• In the Python topic we will read and write files from Python. The most common error there is FileNotFoundError, and the cause is almost always a wrong path.
• In the databases topic we will run a database server and connect to it — and many tools there are configured with paths to files and folders.
• When installing tools you sometimes get “Access denied” or “Permission denied”. It is worth understanding why this happens, rather than solving every problem with “run as administrator”.

Explanation from the basics#

The file system#

The disk itself is a huge sequence of bytes. The file system is the operating-system layer that turns that sequence into files and folders with names, and keeps extra information (metadata) for each file: size, modification date, owner and permissions.

The structure is a tree: there is a root, folders come out of it, and each folder holds more folders and files.

Windows                         Mac / Linux
C:\                             /
├── Users\                      ├── home/            (Mac: /Users/)
│   └── dana\                   │   └── dana/
│       ├── Desktop\            │       ├── Desktop/
│       └── dev\                │       └── dev/
│           └── kodkod6\        │           └── kodkod6/
├── Program Files\              ├── usr/
└── Windows\                    └── etc/

Important differences:

The home directory is the user’s private folder. In it, and in its subfolders, you have full permissions. That is why kodkod6 goes inside it.

Absolute path versus relative path#

• Absolute path — starts from the root. Works from anywhere. C:\Users\dana\dev\kodkod6\week-1\hello.py
• Relative path — starts from the current directory. Short, but depends on where you are standing. week-1\hello.py (if you are standing in kodkod6)

Two special shortcuts:

• . (dot) = the current directory.
• .. (two dots) = the parent folder, one folder up.

Standing in: C:\Users\dana\dev\kodkod6\week-1

.              → C:\Users\dana\dev\kodkod6\week-1
..             → C:\Users\dana\dev\kodkod6
..\week-2      → C:\Users\dana\dev\kodkod6\week-2
..\..          → C:\Users\dana\dev

Users#

An operating system supports several users. Each has a name, a password, a home folder and permissions. There are also “special” users:

• Administrator (Windows) / root (Linux/Mac) — a superuser with full permissions over the whole system.
• System users — users that are not people. Services such as a server or a database run under them, so even if they are broken into, the damage is limited.

In normal work you work as a regular user, and raise permissions only for a specific action: Run as Administrator on Windows, sudo on Linux/Mac.

Permissions#

Every file and folder has permissions that decide who may do what. The three basic actions:

On Linux/Mac permissions are shown as a string like -rwxr-xr--, split into three groups: owner, group and others:

-  rwx  r-x  r--
│   │    │    └── others: read only
│   │    └─────── group:  read + execute
│   └──────────── owner:  read + write + execute
└──────────────── type: "-" = file, "d" = directory

On Windows the permissions are called an ACL and appear under right-click → Properties → Security. The principle is the same.

Why do we need permissions? So that one user cannot read another user’s private files, so that a faulty program or a virus cannot change system files, and so that a server that is broken into cannot touch the whole computer.

Common mistake#

Mistake 1: “the file was not found” — but it exists.

Standing in:  C:\dev\kodkod6
Looking for:  data.txt
Searched:     C:\dev\kodkod6\data.txt
Real file:    C:\dev\kodkod6\week-1\data.txt

• A relative path is checked from the current directory — from where the terminal is standing — and not from the folder where the file is.
• Ran it from another folder? The program looks somewhere else, and does not find it.
• The fix: first move to the right folder (cd), or use an absolute path.

Mistake 2: working as administrator / root “so there won’t be problems”. This bypasses the protection permissions give: every mistake, or malicious program, gets access to the whole computer. Work as a regular user, and use superuser permissions only when you really need them.

Mistake 3: thinking a permission error is a bug in the program. When the operating system returns “Access is denied” / PermissionError, it checked and found that the current user has no permission for the action. The fix is to check who owns the file and what its permissions are — not to change the program.

Summary#

• The file system organizes the disk as a tree of folders and files, starting from the root (C:\ or /).
• An absolute path starts from the root. A relative path starts from the current directory. . = here, .. = up.
• Relative paths are measured from where you ran the program, not from the file’s location.
• Every user has a home folder and permissions. Administrator / root are superusers, used only when needed.
• Basic permissions: read / write / execute. A permission error = the operating system refused.

Official documentation#

• Naming Files, Paths, and Namespaces
• chmod — Linux manual page

Practice#

Comprehension questions on the lesson material — no code.

Exercise 1: The five jobs#

List the five main jobs of the operating system, and give an example of each from what happens on your computer.

Exercise 2: What happens when print prints?#

A Python program runs print("Hi"). Describe the path from the program until the text appears on the screen. Use the terms user mode, system call, kernel and driver.

Exercise 3: Program versus process#

1. What is the difference between a program and a process?
2. You opened three Chrome windows. How many programs are there, and at least how many processes?
3. The computer has 8 cores, but 300 processes are running. How is that possible?

Exercise 4: Paths#

The terminal stands in C:\Users\dana\dev\kodkod6\week-1. Which absolute path does each of the relative paths translate to?

.
..
..\week-2
..\..

Exercise 5: Reading permissions#

This is a line from ls -l on a Linux computer:

-rw-r-----   dana   grades.txt

1. Is it a file or a folder?
2. What is the owner (dana) allowed to do, what is the group allowed to do, and what is everyone else allowed to do?
3. Another user, not in the group, tries to read the file. What happens, and who decides?