/Teaching/Operating Systems/Tutorials/Allgemeines

Allgemeines

Where do I get the zsh together with an appropriate configuration?

Enter: sudo apt-get install zsh && wget -O ~/.zshrc http://git.grml.org/f/grml-etc-core/etc/zsh/zshrc && chsh -s /usr/bin/zsh

Where do I find the manuals for the X86 or ARM-Architecture?

Intel Manuals are available here.

Regarding ARM-Manuals there are diffrences depending on the version. The ARMv5-Manual is a good starting point for SWEB-Development. However the ARMv8-Manual will be useful as well, depending on the architecture you will choose in SWEB.

Do we HAVE to output to debug/kprintfd? Or is it enough to comment our code well?

You have to have at least some debug-output, but there is no requirement for how many debug-messages you need to generate. Please use the debug(…)-function, so you can control the output via debug flags. You can create your own flags, so you can have more fine-tuned control over which debug-output gets displayed. Obviously comments in the code are usually also a good idea, but a well-structured debug-output cannot be replaced by a comment.

Compiling + Linking

If I add Assembler Code into the projekt or edit existing parts, will those files be compiled with Make automatically or do I have to do this manually? What about C/C++?

As long as you add your files to already existing folders (which should contain a Makefile) and then run cmake, they will be integrated into sweb. The detection works via file endings: .cpp for C++, .c for C, .S for Assembly with AT&T-like Syntax (GNU assembler).

Multithreading

How do I acquire a new address for a new userstack and how can I request memory for this?

They don’t “get” anything on their own. Since all threads of a process have to share the same virtual address space, all stacks of all threads have to fit in there. Where you put the stacks of the new threads is left for you to decide. It is then your job to insert this design into sweb.

On a 32bit system the function parameters are passed via stack, how does it work for 64bit?

On 64bit systems the first parameters are passed via the following registers in order: RDI, RSI, RDX, RCX, R8, R9 (Unix calling convention). Any remaining arguments will be passed via stack.

Which kind of threads should be forked by us? Should we only fork UserThreads? Or any kind of thread? What about if a non-UserThread Thread calls fork?

Since fork will be implemented as a syscall, only UserThreads will call it. We will not fork Kernel-Threads. The details of how you fork will depend on your design and implementation, and it is part of your assignment to decide how to best go about it.

Syscalls

If a userthread is currently handling a syscall in the kernelspace and it calls yield(), where does the userthread continues, after it got resumed by the scheduler?

In the kernelspace after the yield().

Is the syscall ‘exit’ called everytime? Even if the program just reaches it’s last line of code?

If a program has reached the end of its code, then this program is terminated without the syscall exit. exit() is called in any case. If you look at the file userspace/libc/src/nonstd.c you will notice that it calls the main function first and the syscall exit.

Kernel (in general)

We need a system time!

IRQ0 is issued periodically. You can count how many IRQ0s occured (we call that ticks). Based on the frequency of the IRQ0 you get your system time. Sometimes you need more accurate timing. You can use the timestamp counter (rdtsc) for that or the RealTimeClock if you need the bios set date and time.

What is the frequency of the IRQ0 (timer interrupt)?

The default frequency is 18.2065Hz (that is approximately 54,9254ms between each IRQ0).

Userprogramme

The Posix-Norm specifies void* for pointers. Can we use the type “pointer” instead, like it is defined in SWEB?

arch/x86/include/types.h is not used in Userspace-libc, just in the Kernel. The simplest approch would be to use the provided Types of the libc-Part and not to violate POSIX. In the Kernel however, you can feel free to use the predifined types or add your own ones, as long as it is compatible with the libc-page and is documented.

Is there a possibility to use Classes in the Userspace?

No, currently only C programs are supported in user space.

Where in MemoryManagement is the classification into Data, Text, Stack and Heap? I have looked through PageManager, but it does not differentiate.

The PageManager only has one task: Managing the free physical pages. That has nothing to do with the sections of an executable nor the ELF format. It is marked in the executable at which linear address which section (Data, Text, BSS) should be put. The Loader reads those infos, gets a Page from PageManager, creates the mapping of the desired address to the just received physical adress and then writes the ByteCode on the specificed linear address range.

Userspace Locks

Are we allowed to use Syscalls when dealing with Userspace Locks?

Userspace locks should be implemented with as few Syscalls as possible. More exact, you may not use any other Syscall except yield when dealing with userspace locks. Consider handling, putting a thread to sleep, or waking up a thread in userspace without any syscalls..

Paging

The MMU does the address-translation, for which it needs the paging structure. It knows where that is by putting the starting address of the table into the register. But how does it know the structure itself?

Correct, the register is called CR3. However this does not point to the pageTABLE but first to the PageMappingLevel4, whose entries point to pageDirectoryPageTables and so on. The structure is determined by the CPU, which is also how it knows it. The OS then needs to adhere to that structure, you cannot change the structure of the CPU. You will find the barebones struct in paging-definitions.h in arch/x86/include/...

Ich habe mittlerweile verstanden, dass der vierte GB des virtuellen Speichers ein 1:1 Mapping des physikalischen Speichers ist. Man verwendet also ‘get3GBAddress’, um direkt auf den physikalischen Speicher zuzugreifen. Aber: Bietet die MMU dieses Mapping von Haus aus an?

die Umsetzung der linearen Adressen in physikalische Adressen macht natürlich schon der Prozessor. SWEB kümmert sich nur darum, dass alle neuen page Directories so initialisiert werden, dass der Bereich zwischen 3 und 4 GB auf die physikalischen Seiten zwischen 0 und 1 GB zeigen.

Assume 5 processes share a common physical page. As soon as this page is modified, the dirty flag is set in the active process PTE, right? The other 4 processes do not notice any modification.

Exactly!

Wie wird festgestellt, dass es sich um einen PageFault handelt?

Alle Speicherzugriffe werden über die MMU abgewickelt, die im PD und im PT die physikalische Adresse der Seite raussucht. Wenn im PDE (page-dir-entry) schon das present-Bit 0 ist, dann ist die PT nicht im physikalischen Speicher vorhanden. Wenn die PT aber vorhanden ist, dann wird erst im PTE nachgeschaut, ob das present-Bit gesetzt ist. Für das Auslösen eines PageFaults ist es egal, ob im PDE oder im PTE das present-Bit 0 ist.

Where are the already loaded Pages accessed? And where exactly is the Accessed Bit set?

On every access (read or write) the accessed bit will be set in the corresponding PTE automatically by the hardware (CPU/MMU).

Swapping

Wo ist die Größe des Blockdevices, das wir zum swapen verwenden sollen einzustellen? Ist es möglich die Devicegröße eines Blockdevices in sweb abzufragen?

Dazu gibt es die beiden BDRequest Typen BD_GET_BLK_SIZE und BD_GET_NUM_BLOCKS. Wenn du diese Requests an das Device schickst und die Ergebnisse multiplizierst hast du die Groesse des Devices in Bytes.

Are we allowed to change the Page Manager (PM) Class? Is it possible to add code and members without breaking everything?

Yes, you are allowed to add new members to the PM. HINT: Consider creating some Classes (f.e Swap-Management, PageTable, Inverse PageTable, ..) and just add these as members instead of coding everything into the PMn)

Kann die PageTable der ausgelagerten Seite auch ausgelagert sein? (bei Behandlung im PageFaultHandler)

Die PT auszulagern macht sowieso nur Sinn, wenn alle PageFrames dahinter auch ausgelagert sind (du greifst auf die PT mindestens genauso oft zu wie auf die dahinterliegenden PageFrames). Also ja, sie KANN ausgelagert sein, wenn ihr das Auslagern von PageTables implementiert habt. Dann müsst ihr halt PT UND PageFrame einlagern.

There are many descriptions of WorkingSet-*-algorithm on the internet, which one shall we implement?

All descriptions have something in common:

    • Each process has a working set size N (defined by a number or by time)
    • Each process has a number of mapped pages M
    • Each page has a process internal timestamp (process time(!), not real time) T
    • The N youngest pages are in the working set, the remaining M-N pages are not
    • Any page in no working set can be swapped
    • The working set size for all processes is decreased upon certain occasions (for example: when the number of free pages reaches zero)
    • The working set size for a process is increased upon certain occasions (for example: when a pagefault for this process occurs)

Apart from that, implement it as you like.

File System

Do I have to care for locking? For example one or more threads have a file open and another threads write to the same file.

No, you don’t. This is already implemented.

I’ve created a new test program, but actually I can not run it. It seems that the program is not even copied to the SWEB image.

Does your test-program have a long filename? Minix, the used file-system, restricts filenames to max 30 chars. So if your file has a name longer than 30 chars (including the extension!) it can not be copied to the image.