Skip to content
GrygiriiS edited this page May 6, 2024 · 8 revisions

XEN sources

repo: https://github.com/xen-troops/xen.git

branch: xen-4.19-xt0.1

XEN build

Follow instructions in INSTALL document.

make XEN_TARGET_ARCH=arm64 CROSS_COMPILE=<path>/zephyr-sdk-0.16.3/aarch64-zephyr-elf/bin/aarch64-zephyr-elf- arm64_defconfig
make XEN_TARGET_ARCH=arm64 CROSS_COMPILE=<path>/zephyr-sdk-0.16.3/aarch64-zephyr-elf/bin/aarch64-zephyr-elf- DESTDIR=build

XEN boot (no domains)

Copy xen binary to RPI5 boot partition.

In RPI5 config.txt file select xen binary to boot instead of standard kernel.

[all]
enable_uart=1
kernel=xen
uart_2ndstage=1

Note

Note that boot uses FDT prepared by RPI5 firmware and placed at ${fdt_addr}.

XEN boot log

NOTICE:  BL31: v2.6(release):v2.6-239-g2a9ede0bd
NOTICE:  BL31: Built : 14:26:57, Jun 22 2023
 Xen 4.19-unstable
(XEN) Xen version 4.19-unstable (grygorii@) (aarch64-zephyr-elf-gcc (Zephyr SDK 0.16.3) 12.2.0) debug=y Thu Apr 11 15:33:36 EEST 2024
(XEN) Latest ChangeSet: Fri Jan 5 10:46:44 2024 +0200 git:8ff1f6708a
(XEN) build-id: 81400877508840d2162ff03548bb83af12be760b
(XEN) Processor: 00000000414fd0b1: "ARM Limited", variant: 0x4, part 0xd0b,rev 0x1
(XEN) 64-bit Execution:
(XEN)   Processor Features: 1100000010111112 0000000000000010
(XEN)     Exception Levels: EL3:64 EL2:64 EL1:64 EL0:64+32
(XEN)     Extensions: FloatingPoint AdvancedSIMD
(XEN)   Debug Features: 0000000010305408 0000000000000000
(XEN)   Auxiliary Features: 0000000000000000 0000000000000000
(XEN)   Memory Model Features: 0000000000101122 0000000010212122
(XEN)   ISA Features:  0000100010211120 0000000000100001
(XEN) 32-bit Execution:
(XEN)   Processor Features: 0000000010010131:0000000000010000
(XEN)     Instruction Sets: AArch32 A32 Thumb Thumb-2 Jazelle
(XEN)     Extensions: GenericTimer
(XEN)   Debug Features: 0000000004010088
(XEN)   Auxiliary Features: 0000000000000000
(XEN)   Memory Model Features: 0000000010201105 0000000040000000
(XEN)                          0000000001260000 0000000002122211
(XEN)   ISA Features: 0000000002101110 0000000013112111 0000000021232042
(XEN)                 0000000001112131 0000000000010142 0000000001011121
(XEN) Using SMC Calling Convention v1.2
(XEN) Using PSCI v1.1
(XEN) SMP: Allowing 4 CPUs
(XEN) Generic Timer IRQ: phys=30 hyp=26 virt=27 Freq: 54000 KHz
(XEN) GICv2 initialization:
(XEN)         gic_dist_addr=000000107fff9000
(XEN)         gic_cpu_addr=000000107fffa000
(XEN)         gic_hyp_addr=000000107fffc000
(XEN)         gic_vcpu_addr=000000107fffe000
(XEN)         gic_maintenance_irq=25
(XEN) GICv2: 320 lines, 4 cpus, secure (IID 0200143b).
(XEN) XSM Framework v1.0.1 initialized
(XEN) Initialising XSM SILO mode
(XEN) Initialized GSX IRQ
(XEN) Using scheduler: SMP Credit Scheduler rev2 (credit2)
(XEN) Initializing Credit2 scheduler
(XEN)  load_precision_shift: 18
(XEN)  load_window_shift: 30
(XEN)  underload_balance_tolerance: 0
(XEN)  overload_balance_tolerance: -3
(XEN)  runqueues arrangement: socket
(XEN)  cap enforcement granularity: 10ms
(XEN) load tracking window length 1073741824 ns
(XEN) Allocated console ring of 32 KiB.
(XEN) CPU0: Guest atomics will try 17 times before pausing the domain
(XEN) Bringing up CPU1
(XEN) CPU1: Guest atomics will try 15 times before pausing the domain
(XEN) CPU 1 booted.
(XEN) Bringing up CPU2
(XEN) CPU2: Guest atomics will try 15 times before pausing the domain
(XEN) CPU 2 booted.
(XEN) Bringing up CPU3
(XEN) CPU3: Guest atomics will try 16 times before pausing the domain
(XEN) Brought up 4 CPUs
(XEN) CPU 3 booted.
(XEN) I/O virtualisation disabled
(XEN) P2M: 40-bit IPA with 40-bit PA and 16-bit VMID
(XEN) P2M: 3 levels with order-1 root, VTCR 0x00000000800a3558
(XEN) Scheduling granularity: cpu, 1 CPU per sched-resource
(XEN) Initializing Credit2 scheduler
(XEN)  load_precision_shift: 18
(XEN)  load_window_shift: 30
(XEN)  underload_balance_tolerance: 0
(XEN)  overload_balance_tolerance: -3
(XEN)  runqueues arrangement: socket
(XEN)  cap enforcement granularity: 10ms
(XEN) load tracking window length 1073741824 ns
(XEN) Adding cpu 0 to runqueue 0
(XEN)  First cpu on runqueue, activating
(XEN) Adding cpu 1 to runqueue 0
(XEN) Adding cpu 2 to runqueue 0
(XEN) Adding cpu 3 to runqueue 0
(XEN) alternatives: Patching with alt table 00000a00002e5b58 -> 00000a00002e6cc8
(XEN) CPU0 will use 24 loops workaround on exception entry
(XEN) CPU1 will use 24 loops workaround on exception entry
(XEN) CPU3 will use 24 loops workaround on exception entry
(XEN) CPU2 will use 24 loops workaround on exception entry
(XEN) *** LOADING DOMAIN 0 ***
(XEN) Missing kernel boot module?
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) Could not set up DOM0 guest OS (rc = -2)
(XEN) ****************************************
(XEN)
(XEN) Reboot in five seconds...

XEN HACK to enable vuart "arm,sbsa-uart" by default

XEN provides support for emulated vuart console which can be used for debug purposes in some cases when HVC console not working, for example. In case of Zephyr Dom0/U the XEN "vuart" is not implemented yet, so below HACK need to be applied to enable vuart "arm,sbsa-uart" by default 0001-HACK-force-enable-arm-sbsa-uart-vuart-for-domus.patch:

diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
index 26dfabeecb..01590b3a86 100644
--- a/xen/arch/arm/domain.c
+++ b/xen/arch/arm/domain.c
@@ -745,6 +745,9 @@ int arch_domain_create(struct domain *d,
     if ( (rc = domain_io_init(d, count + MAX_IO_HANDLER)) != 0 )
         goto fail;
 
+    if ( !is_hardware_domain(d) ) {
+       config->arch.nr_spis = MAX(config->arch.nr_spis, GUEST_VPL011_SPI - 32 + 1);
+    }
     if ( (rc = domain_vgic_init(d, config->arch.nr_spis)) != 0 )
         goto fail;
 
@@ -787,6 +790,12 @@ int arch_domain_create(struct domain *d,
 
     d->arch.vgsx_osid = config->arch.vgsx_osid;
 
+    if ( !is_hardware_domain(d) ) {
+               rc = domain_vpl011_init(d, NULL);
+                         if ( rc < 0 )
+                                 return rc;
+    }
+
     return 0;
 
 fail:

Then the Zephyr DomUs can start using vuart "arm,sbsa-uart" by adding corresponding node to the device-tree:


    zephyr,console = &uart0;

    uart0: serial@22000000 {
           compatible = "arm,sbsa-uart";
           reg = <0x0 0x22000000 0x0 0x1000>;
           interrupts = <GIC_SPI 32 IRQ_TYPE_LEVEL IRQ_DEFAULT_PRIORITY>;
           interrupt-names = "irq1";
           current-speed = <115200>;
    };

XEN enable early printk

To enable early console enable below Kconfig options:

CONFIG_EARLY_UART_CHOICE_PL011=y
CONFIG_EARLY_UART_PL011=y
CONFIG_EARLY_PRINTK=y
CONFIG_EARLY_UART_BASE_ADDRESS=0x107d001000
CONFIG_EARLY_UART_PL011_BAUD_RATE=0
CONFIG_EARLY_UART_PL011_MMIO32=y