SDK 2.0 compatibility

[tjkirch]

Panic sent out this email about the upcoming SDK 2.0 update, which links to this detail page.

I tested my simple crankstart-based game with SDK 2.0 - it ran fine in the simulator, but it crashes on my Playdate, saying CAPI handler function wasn't located in loaded data. I'm wondering what's required for compatibility.

Here are some random thoughts:

• Build requirements: pdc now needs to know about ELF relocations via a pdex.elf file. I don't see it generating one automatically. I'm not knowledgeable in this area; is this a concern, or is Rust doing automatically what Panic suggests in their updated compiler/linker flags? I saw someone mention emit-relocs for tock.
• Similar for updated linker requirements.
• Once those basics are figured out: SDK 2.0 games aren't compatible with 1.x Playdates, and SDK 1.x games aren't compatible with 2.0 Playdates. Can it just be up to the crankstart user to pick an SDK and crankstart will work (possibly via detection) or will there need to be a flag? Maybe 1.x compatibility just won't matter?

[rtsuk]

@boozook tells me he's going to take a look at the build and linking issues.

On the subject of picking an SDK, I'd recommend just switching to 2.0 to start, and back-fit a 1.0 switch if there's anyone ends up needing it.

[boozook]

Same Issue for me. We're unable to test on the device because latest released OS version is 1.13.7 without new loader. We just can hope :))
official note
They will change elf loader in the future OS 2.0.

Playdates on OS 1.x will not be able to launch your 2.0 build, but will display a dialog asking the user to upgrade their Playdate’s OS. If OS 2.0 is not available to the public, the user will not be able to run your game.

I'll try to get "normal execution" of new build (for new loader with relloc support) on the rev1 device with OS 1.13.7 on board to get message like "The Playdate OS is too old to run this game".
I'm working on it...

[boozook]

Update.
Actually nothing interesting. I've build a correct binary (as I suppose) with rust and rust-lld, but after pdc there's nothing (only 130 bytes of nothing).

My test "foo" binary contains the following...

I'm sure it should work because now in linker-map eventHandlerShim as ENTRY explicitly.
And I have one exactly as needed, same as in the setup.c.

__bss_end__
__bss_start__
__data_end__
__data_start__
__etext
eventHandlerShim

foo:	file format elf32-littlearm

Program Header:
    LOAD off    0x00000080 vaddr 0x00000000 paddr 0x00000000 align 2**4
         filesz 0x00001774 memsz 0x00001778 flags rwx
   STACK off    0x00000000 vaddr 0x00000000 paddr 0x00000000 align 2**64
         filesz 0x00000000 memsz 0x00000000 flags rw-

Dynamic Section:

Sections:
Idx Name            Size     VMA      LMA      Type
  0                 00000000 00000000 00000000 
  1 .text           00001774 00000000 00000000 TEXT
  2 .data           00000000 00001774 00001774 TEXT
  3 .bss            00000004 00001774 00001774 BSS
  4 .rel.text       00000598 00000000 00000000 
  5 .ARM.attributes 00000047 00000000 00000000 
  6 .comment        00000013 00000000 00000000 
  7 .symtab         000007a0 00000000 00000000 
  8 .shstrtab       0000004f 00000000 00000000 
  9 .strtab         00000f71 00000000 00000000 

🤷‍♂️

[boozook]

One more update. I've compiled one of examples in the SDK/C_API/Examples...
Just to compare sections in the resulting elf. And you know what? When I run it on the device, I see "no capi handler"!
Well, I'm not sure that its possible right now if I can't run example built with pdc2.0-beta on my hardware with OSv1.13.7.
All ideas are welcome.

[tjkirch]

One more update. I've compiled one of examples in the SDK/C_API/Examples... Just to compare sections in the resulting elf. And you know what? When I run it on the device, I see "no capi handler"! Well, I'm not sure that its possible right now if I can't run example built with pdc2.0-beta on my hardware with OSv1.13.7. All ideas are welcome.

@boozook Interesting - I was able to get the expected "Update Needed" message on my 1.13.7 device. I just unpacked the 2.0 beta2 SDK to my $PLAYDATE_SDK_PATH, did cd $PLAYDATE_SDK_PATH/C_API/Examples/Hello World, then make all, and copied HelloWorld.pdx to my Playdate. Is there any chance you had the path variable pointing to a 1.x SDK, or something like that?

Two things surprised me, though - the pdx had no ELF relocations file, and it ran on both 1.x and 2.x simulators, which I didn't think was supposed to work. I'd have to guess that the simulator is special-cased to run 1.x and 2.x games.

[boozook]

@tjkirch I've reported about it on the dev forum. There's video with reproduction. Link

[tjkirch]

@boozook Do you think there's any way others could help with your investigation? I'm not experienced in this area, but I'm worried about the SDK 2 release obsoleting all crankstart games, so I'd like to do what I can!

[tjkirch]

Panic just announced that SDK 2.0 is going live in a few days, on June 12th, 2023. They make it easy to update, so most users will probably do so quickly, and then crankstart games will no longer work. We definitely need to figure this out.

[rtsuk]

Has anyone shipped a game using crankstart?

[tjkirch]

Has anyone shipped a game using crankstart?

I'm not sure, but I'd like to. I've spent a long time on mine, and I'd be quite sad to have to rebuild it in Lua.

[boozook]

I have an MVP-like build script "rust build executable bin then give it to pdc". I not sure that it works, so I just need to test it with latest beta. Or better test on real hardware, I have one, but so have to wait OS 2.0 release. 🤷🏻‍♂️

[tjkirch]

I have an MVP-like build script "rust build executable bin then give it to pdc". I not sure that it works, so I just need to test it with latest beta. Or better test on real hardware, I have one, but so have to wait OS 2.0 release. 🤷🏻‍♂️

Nice, thank you!!

If you're interested, they requested beta testers for new releases, and you obviously have a very good case. You might be able to try 2.0 on-device sooner. https://devforum.play.date/t/now-accepting-applications-for-beta-testers/11123

[boozook]

@tjkirch thank you, I didn’t know that.

Playdate OS/SDK Beta Tester Sign Up
Your response has been recorded

[boozook]

I'm working on it... 😐

[scratchminer]

Here's what I've figured out so far in terms of the pdex.bin format:

0x0: constant Playdate PDX
0xC: flags (all 0)
0x10: some kind of MD5 hash?
0x20: BSS start address
0x24: BSS end address
0x28 - 0x30: unknown, but clearly two uint32s

(everything after this is compressed)

.text section contents
.data section contents
relocation table: offsets are uint32 and relative to the compression boundary

[boozook]

Sad news for now. I tried really many options and didn't understand where I was wrong. So I have no idea for now.

There is dummy isolated example (/playground).

@scratchminer, @tjkirch, do you have any ideas. Seems to I need help.

[boozook]

Hm... Probably I have another option. Tomorrow I'll try to use only gcc linking. Will see...

[boozook]

Okay. Almost all my previous solutions did not works because of bug in the pdc.
Now building works fine. Testing relocations and fix will be here soon.

[boozook]

Status update:

1. Simple build lib with rustc+arm-gcc works
2. I have some trouble with alloc, just strange cstrings behaviour, solvable.
   Sims I close to finish.

UPD: For 2. - I've just forgot to build alloc, so -Z build-std=core,alloc solve this problem. ❤️‍🔥

[boozook]

Probably I'm done.
Please test and review.

[boozook]

Also we're need some validity tests for produced output to detect regression on future changes.

[boozook]

I need feedback to merge crank#42.

[boozook]

I would close this because there's a latest fix in #48.