KFS-5
Screenshots
VGA:
Serial output:
Scheduler
The kernel now has a cooperative scheduler.
We are now a multiprocess and multithreading capable OS.
It is still single core for now, and the scheduler might need to be highly modified/rewritten
when we will want to support SMP, or preemption, but it works fine for now.
The schedule queue does not implement any fancy priority system, and is just a dumb FIFO.
Userspace
KFS-4 introduced a demo ring3 userspace and syscalls, with a single test program running in userspace.
Now that we are multiprocessed, we can finally become a true microkernel, and move all modules/drivers to separate userspace processes.
ELF Loading
Our userspace services are simple ELF files copied in memory by grub at boot (we use multiboot2 modules for that), and then loaded and started by the kernel after it has completed its initialization stage.
IPC
We now support Inter Process Communication between processes.
Our IPC architecture is a rendez-vous system between processes. A server accepts connexions on a 'port', and is put to sleep until a client connects to this port.
The kernel then creates a 'session', a two-ended half-duplex communication channel between two processes, and gives one end of the channel to each process.
The client can then make requests to the server through this channel, and then receive its answer on the same channel.
The IPC makes a heavy use of 'handles', that are just like unix file descriptors, but that can point on more resources than just files (thread/session/shared memory/...).
Syscalls
Implemented a bunch of syscalls, based on necessity:
- svcCreateThread
- svcStartThread
- svcSleepThread
- svcExitThread
- svcExitProcess
- svcMapFramebuffer
- svcCreateSharedMemory
- svcMapSharedMemory
- svcUnmapSharedMemory
- svcQueryMemory
- svcQueryPhysicalAddress
- svcSetHeapSize
- svcMapMmioRegion
- svcCreatePort
- svcManagePort
- svcConnectToPort
- svcConnectToNamedPort
- svcAcceptSession
- svcCreateSession
- svcWaitSynchronization
- svcSendSyncRequestWithUserBuffer
- svcReplyAndReceive
- svcReplyAndReceiveWithUserBuffer
- svcCreateIrqEvent
- svcCloseHandle
- svcOutputDebugString
Services
As stated above, we now are a microkernel, and we migrated a lot of kernel features to distinct userspace services:
Vi
The Visual Interface service.
This service is called to display windows at given coordinates on the screen. Every client asks to create a window, which is just a shared memory, modifies it, and then asks again vi to draw it (refresh screen).
Shell
The read-eval loop of keyboard inputs was moved to kfs_shell.
It's a very basic interactive program that is displayed at the top of the screen, and which is solely used to display memes and run debug commands.
Clock
A small window running at the bottom of the screen that displays the current time and date.
This is composed of both a driver for the Real Time Clock, and the program that displays it to the screen.
This should be split in a service and a program in the future.
Libraries
Libuser
A light libc, that exposes wrapper functions around syscalls, IPC calls, and a whole bunch of low-level helper functions.
All services have it in their dependencies.
In the far future this might end up becoming our rust libstd, or at least be used in the glue layer in libstd.
Libkern
A library that exposes constants shared between the kernel and userspace. This mainly are syscall numbers, and kernel error values.
This should only be useful to libuser, which wraps syscalls, and matches raw errors to more rich types.
Libutils
A messy crate with various utilities shared between the user and kernel code.
Should probably be further split into several useful libraries.
Other updates
Paging
Re-wrote the FrameAllocator and Paging completely, to have better management and bookkeeping of userspace memory.
Both were made a lot more architecture agnostic, to prepare for aarch64 port.
The KernelMemory is still shared between all processes, page tables being copied at context switch time, however each process has its own 3GiB of virtual space, which is no longer managed directly in the page tables, but instead in a BTreeMap of mappings present for every process.
This enables us to save more meta-data about each mapping.
In particular, we can now do shared memory between processes, where we represent it as an Arc of the physical regions, which will be freed only when its last user unmaps it (or dies).
Note that KernelMemory (4th GiB) is still bookkeeped directly in the page tables themselves.
mkisofs
A tool to create a bootable KFS iso image.
We used to depend on grub-mkrescue, and that made compilation impossible on mac OS or Windows.
We now create the iso ourselves, and can be built on any platform.
Maintained as a separate repo.
New member
The core team (@roblabla and @Orycterope) was expanded to include a third developer, say hi to @Thog, to form the deadly triade of KFS devs, coming to get you on a deep synthwave soundtrack.
Workflow
Pull Requests and peer-review
We improved our workflow a lot. We now only submit patches by PRs, and force ourselves to go through peer-review before merging any branch into master. This has greatly helped opening conversation and constructive criticism in the last few months, and also avoid deeply broken implementations to make it to the master branch.
Tests
We started writing some unit tests for a few functions. They are compiled for i686 linux, and run on the host machine.
This meant writing a lot of mockup code and conditional compilation, but it works.
For now there's only a few tests, and a big effort should be made to write some more, but at least the infrastructure is there, and it works.
We still have no integration tests though, see #31 .
Code cleanliness
We now make heavy use of the clippy lint, to make sure our code always matches a certain standard of quality, and avoid a lot of beginners mistakes.
Updating all of our project to match the standard was a real pain in the a**, see #88
Documentation
We use a combination of rust and clippy's lints in every crate:
#![deny(missing_docs)]
#![deny(clippy::missing_docs_in_private_items)]to make sure that every module, function, structure, and structure's field is documented, otherwise our code doesn't compile.
This ensures us that every part of the OS is fully documented, and easily understandable (looking at you, linux ರ_ರ).
Our documentation is automatically uploaded to our github pages thanks to cargo-travis
CI
We started using Travis for Code Integration. For a branch to be merged, it must pass compilation for both debug and release, the iso image must be built successfully, doc must be generated successfully, and all unit tests must pass. That way master is always 100% clean.
Issue tracking
We started using (and fell in love with) todo[bot] to automatically open issues for us, by finding "TODO" comments in our code. You can see we use it pretty much all the time, it's impressive to see how our issue count has exploded in only a few months. For an exemple, see #202.
Dependencies
We now use dependabot to automatically create a PR every time one of our dependencies is updated, so we're always up to date. See #196 for exemple.