Add p8z77-m_pro board

[ThePlexus]

The P8Z77-M Pro
This board is a better choice over the P8H61 for a cost effective Heads + QubesOS desktop with ME neuter+disable compatibility. The P8H61 ecosystem was complex with multiple variants (some not even having a TPM header, and others having RamInit issues with some memory sticks), while less feature rich than the P8Z77 family. The P8H61s that were compatible still required some ME sections (FCRS,EFFS) to be whitelisted in order to post, which introduced unknowns. The P8H61s also needed a larger flash chip to work with heads than the manufacturer supplied 4M, which added complexity for the average user.

The P8Z77-M Pro is able to offer more SATA connectors (2x 6Gb, 4x 3Gb, 2x eSATA) as well as more full size expansion ports. The board has a PS/2 keyboard port as recommended for QubesOS. The board comes with 8M flash chip as standard.

The i7-3770 is the best CPU available for the board, with VT-x & VT-d both present

This has been a daily driver Desktop QubesOS machine for me for some time.

ME & ROM
The board supports Intel LGA1155, which allow for ME removal (both neuter+disable work), ME region resize/shrinking (aka 'maximized' board), as well as VSCC table modification..

The blob download script uses the manufacturer supplied ME and IFD and performs the necessary hashing. The download script also removes the VSCC table by overwriting a NULL at the VSCC length table and FF bytes at the VSCC identifier table - using a printf with dd. The download script also resizes the rom layout and minimizes ME while maximizing space.

The P8Z77-M Pro comes as standard with an 8Mb Flash chip, which means that no modification is needed to replace the chip is order to use heads as we shrink ME and 'maximize' this board by default, leaving just 335396 bytes available.

The P8Z77-M Pro has both TPM1 and TPM2 modules available, though at time of writing only the TPM1 module would be usable with heads until the TPM2 work is completed. All testing was done with a TPM1 module

Test platform
BOARD: Asus P8Z77-M Pro
RAM: 32Gb - 4x TimeTec DDRL3 75TT16NUL2R8-8G
CPU: Intel i7 3770
TPM: Modules tested: Asus branded TPM 1.02H & Foxconn TPM Krypton Rev 1.0

[tlaurion]

EDITED

CircleCI config in tree would also need two new entrees, one to call blob download and extraction script, and then another one to build the board on each commit to track regressions (most importantly telling us that a build fails because lack of space as defined under CBFS_SIZE = IDD BIOS region, as soon as possible).

Can you please look and make sure that ME modules untrimmed cannot be relocalised to give some more free space to reduce me size, the me ifd region and then recalculate ifd bios region accordingly with increased available BIOS region also mirrored into coreboot CBFS_SIZE?

I'm worried this board might be short lived if that is not figure out today. As you might have saw, we now pack bash inside of Heads on recent master.

[tlaurion]

@ThePlexus also, board name would require tpm1 suffix since TPM daughter board is replaceable and tpm2 support just landed. (EDIT: where toolstack for tpm2 being bigger, I highly doubt that 8mb spi chip will he able to provide tpm2 support).

But since both are possible, the board name should make it clear that this is for tpm1

[tlaurion]

Glad to see another board being integrated to Heads :)

[ThePlexus]

Can you please look and make sure that ME modules untrimmed cannot be relocalised to give some more free space to reduce me size, the me ifd region and then recalculate ifd bios region accordingly with increased available BIOS region also mirrored into coreboot CBFS_SIZE?

I'm worried this board might be short lived if that is not figure out today.

Sure - this is done. Not quite a meg, but not far off. Im just waiting to see if this has any effect on the system abiltiy to stay up, but it posts and gets into heads just fine

TOTP
(empty) 0x6e5600 null 932836 none
HOTP
(empty) 0x6e7400 null 925156 none

As you might have saw, we now pack bash inside of Heads on recent master.

Yes ive been following the TPM2 work closely. Just have not had time to get involved. Ultimately this board is a DIP8 so if we ever get to the stage where 8Mb isnt enough, I already have a procedure that works for migrating to 16M W25q128

[tlaurion]

Unfortunately, the download extract script relies on ifdtool, which doesn't exist until at least previous board was previously compiled, having extracted at least one coreboot version archive and ifdtool having been compiled for host, just before stitching everything back together to create a ROM. CircleCI builds are currently reusing past caches which is why it currently works in prep steps, but building clean would not.

@rbreslow had the same chicken egg problem, since he was the first to rely on cbfstool as part of extracting mrc.bin from peppy firmware image (see blobs/haswell script), with cbfstool. He resolved that issue by making the board config have blobs requirements, which called haswll/blobs script for the coreboot version specified under board config to use coreboot version tarball extracted per board config, which builds cbfstool for the host prior of using it.

A similar thing here for ifdtool would be needed if we depend on ifdtool.

[tlaurion]

(To test that hypothesis for yourself, go under CircleCI project setting, and define a variable called CACHE_VERSION with random value of your choice (I like to use date stamps):

Then trigger a rebuild for your pipeline. We expect things to fail really early in prep step since no coreboot tarball has been extracted, even less ifdtool being compiled for that versiob.

[tlaurion]

@ThePlexus so basically, you could add and ifd.bin statement inside of the board statement just like the t440p is doing for mrc.bin at https://github.com/osresearch/heads/blob/96440b928acb06de5b925ea12014c9c280b23165/boards/t440p-maximized/t440p-maximized.config#L38-L48

This triggers all other Makefile logic to have coreboot directory extracted, where the script makes cbfstool prior of using it (replace with ifdtool) https://github.com/osresearch/heads/blob/96440b928acb06de5b925ea12014c9c280b23165/blobs/haswell/obtain-mrc#L30-L35

[ThePlexus]

ifdtool dependency removed, script updated. just used me_cleaner and dd instead of me_cleaner and ifdtool

[ThePlexus]

mrc.bin from peppy firmware image

Was it not possible to extract just using offset and dd?

[tlaurion]

The same story will happen for me_cleaner here which is found under extracted coreboot build dir from previous build of the board, only difference here is that me_cleaner doesn't need to be built first but just present.

[tlaurion]

@ThePlexus I see, you use osresearch's CircleCI instance because you haven't create your own.
Currently changing CACHE_VERSION so that everything builds clean so you will see what i'm talking about.

[tlaurion]

@ThePlexus restarted pipeline without reusing cache is happening at https://app.circleci.com/pipelines/github/osresearch/heads/543/workflows/6f864890-ced6-49b9-89ed-9e02bbd37e2d

[tlaurion]

@ThePlexus failure happens after 42s at https://app.circleci.com/pipelines/github/osresearch/heads/543/workflows/6f864890-ced6-49b9-89ed-9e02bbd37e2d/jobs/8131, sepcific log (error) at blob run under https://app.circleci.com/pipelines/github/osresearch/heads/543/workflows/6f864890-ced6-49b9-89ed-9e02bbd37e2d/jobs/8131/parallel-runs/0/steps/0-111

[tlaurion]

@ThePlexus

Applied logic in question with some minimum visibility improvements and htop board including maximized board, just changing name after incorporating everything maximized under hotp here under branch https://github.com/tlaurion/heads/tree/ThePlexus_p8z77-m_pro_build_test_clean

commit: 970dad8
(patch : https://github.com/osresearch/heads/commit/970dad8b962dcd1f908b4ca3a079b6ab2cb95b3e.patch )

Pipeline building this commit from clean (no cache) at https://app.circleci.com/pipelines/github/tlaurion/heads/1556/workflows/04d36601-09cc-4506-8c4d-b409a4dbc273

[ThePlexus]

osresearch's CircleCI instance because you haven't create your own.

no I dont have cicleci currently.

@ThePlexus

Applied logic in question with some minimum visibility improvements ...

I can see the logic what you mean,I would also think the xx30 approach would have worked too, by adding this to the YAML (as heads ships with a me_cleaner.sh in the codebase already, thus its a not really a dependency on coreboot being extracted)

-m $(readlink -f ./blobs/xx30/me_cleaner.py)

However, I dont much mind what the approach is, ive applied your recommendations from your patch and pushed

I also checked out the NIC - the reason it didn't come up for me was because I was using ubuntu without the driver. I have used the gbe.bin generated into heads, booting into QubesOS. the NIC comes up and it uses MAC ff:ff:ff:ff:ff:ff. I assume thats because that is what is at 0x1000 in gbe.bin. I will test tomorrow or in the next few days setting the same MAC as your generated gbe.bin from xx30. So I am happy that there is no problem with the gbe.bin as extracted and using this with QubesOS. The only improvement would be to see if I can change the MAC at some point, buyt that shouldn't be a blocker as its more cosmetic to 'match' the MAC in xx30.

Are there any other blockers to pushing this board to master?

[tlaurion]

@ThePlexus I think its OK as of now if build from circlrci flashed works for you and is built by ci.

On reusing x230 blobs dir me_cleaner, goal is to get away of it at some point, having in board configs a base for newcomers to replicate instead of building on top of hacks.

Let me know when circleci artifact flashed works for you and I will be liberal for the commits here, since new board and not touching anything else.

[tlaurion]

@ThePlexus

• That would be https://output.circle-artifacts.com/output/job/028d7620-2b40-4377-8056-c58f9bcdc83d/artifacts/0/build/x86/p8z77-m_pro-tpm1-hotp-maximized/heads-p8z77-m_pro-tpm1-hotp-maximized-v0.2.0-1494-gc205c40.rom

[ThePlexus]

@tlaurion ill await the next artefact to test. There is no gbe.bin on this board, it was a path copy error on my part when moving in the download bios script. For some PBKAC reason I got my copy of the P8Z77-V script i made, not P8Z77-M, the former has gbe.bin and the latter does not. To add to cvonfusing, using the P8Z77-V BIOS for ME, IFD and GBE will allow the P8Z77-M to boot and function normally, it will just have PCI 03:00.0 missing (ethernet card)

The version building now uses the correct vendor BIOS, does not use GBE.bin and gives us 4k back. Not a lot - but was to those of us who use to code ZX81's back in the day ;)

[tlaurion]

@ThePlexus

The version building now uses the correct vendor BIOS, does not use GBE.bin and gives us 4k back.
Now i'm confused. Without gbe.bin, and with e1000/e1000e/igb driven ethernet, ethernet will not be useable. I went through that in my first steps creating the first maximized builds to understand this.

So is not having ethernet functional satisfactory? You set the entry point here for that board support.
You can test this directly from Heads through network-recovery-init script, which ephemerally randomizes mac adress if module loads properly, checking that at least one duplicated section in gbe is valid, otherwise you will see in dmesg the same errors I went through in other issues because gbe.bin was invalid and or not present.

gbe.bin is configuration blob for intel ethernet and if not found, pci device is found, driver will attempt to be loaded but will fail in OS. For other maximized builds where GBE can be generated, it was fixed to 00:de:ad:c0:ff:ee and some reminder for users to set under OS MAC randomization through NetworkManager or other means. Tails will do it automatically, QubesOS does it for wifi but not ethernet. Only other way around this is extraction script in blobs dir for other boards, so that users can extract their original gbe.bin dropped in directory (and not everriden if present, not if matching hashes of script which is not reran if expected blobs in board dir makefile is found) to keep their original mac.

[ThePlexus]

@ThePlexus

The version building now uses the correct vendor BIOS, does not use GBE.bin and gives us 4k back.
Now i'm confused. Without gbe.bin, and with e1000/e1000e/igb driven ethernet, ethernet will not be useable. I went through that in my first steps creating the first maximized builds to understand this.

So is not having ethernet functional satisfactory? You set the entry point here for that board support. You can test this directly from Heads through network-recovery-init script, which ephemerally randomizes mac adress if module loads properly, checking that at least one duplicated section in gbe is valid, otherwise you will see in dmesg the same errors I went through in other issues because gbe.bin was invalid and or not present.

gbe.bin is configuration blob for intel ethernet and if not found, pci device is found, driver will attempt to be loaded but will fail in OS.

@tlaurion sorry for confusion, there is * no * gbe.bin in this platform. At all.

The error was i used a download_BIOS_clean.sh script I made for a different test platform and copied that into the heads folder before I made the initial PR. There has been a knock on effect of confusion (including on my part where I couldnt get Ethernet working, then confused the USB adapter I had)

The vendor ROM for this board does also not contain gbe.bin

$ ~/bios/coreboot/util/ifdtool/ifdtool --layout layout.txt P8Z77-M-PRO-ASUS-2203.ROM 
File P8Z77-M-PRO-ASUS-2203.ROM is 8388608 bytes
Wrote layout to layout.txt
$ cat layout.txt 
00000000:00000fff fd
00180000:007fffff bios
00001000:0017ffff me

I can confirm with the new script that Ethernet works fine. Herei s heads after POST

and heres the PCI device in heads (10ec:8168 on 03:00.0)

And here it is in a ubuntu OS usb boot, working as intended

I will grab the artefacts and test, of course.

[ThePlexus]

I am testing with this artefact now https://output.circle-artifacts.com/output/job/ba9e1720-6dc1-4b2b-99fa-3c68f72609cc/artifacts/0/build/x86/p8z77-m_pro-tpm1-hotp-maximized/heads-p8z77-m_pro-tpm1-hotp-maximized-v0.2.0-1495-g89adc07.rom

[ThePlexus]

Looks good .

gbe.bin is configuration blob for intel ethernet

This platform uses Realtek RTL8111

[ThePlexus]

@tlaurion as to why adding an intel gbe.bin to the bios stops realtek RTL8111 coming up on the PCI bus remains a mystery that I suspect only the ASUS designers would understand.

[tlaurion]

@ThePlexus why have you deleted circleci config?

[ThePlexus]

@tlaurion i was doing some action in a side branch in git web admin - some how one of the deletes made it to my master branch. Have restored.

Ideally I want to move to branches for different pull requests in future, which I will move to once the items in this PR are merged to heads master.

Are there any items left to do for this? as I have some videos to finish and I it will be awkward guiding people to download from different repos and/or applying patches to heads.

[tlaurion]

Well commit log now looks a bit hideous :/

[tlaurion]

Edit: can you squash unneeded commits in a specific branch?

[tlaurion]

Otherwise I can co-aurhor your commits cleaning things up

[tlaurion]

Kinda late here

[ThePlexus]

@tlaurion to be fair .. this is on me - i do use cli normally and only use branches for pulls where needed. I just didnt expect to have two things 'in flight' for heads at the same time so used master of my fork (#1357 to fix OEM reownership). So I tried to make a new branch, without CLI while in bed with a partner sleeping next to me - hence clicky on web is not as noisy as mashing keys.

anyway, lesson learned. Ill tidy this up to one nice neat pull.