[Feature Bounty - $350] MacOS Supported Version of BG3 SE

[whispy]

What: A bounty that will be paid out when BG3 SE works on MacOS with the Mac port of BG3.

• The MacOS version of BG3 SE should not cause major crashes or bugs.
• It should support all (or, at least, most) existing mods that work with the Windows version of BG3 SE.
• It should work on Apple Silicon (M1/M2/etc) processors. (Intel processor support is not required)

Amount: $350

Payment Method: Your choice (I am happy to post the bounty to an escrow service, but I don't know them. If you let me know a preferred one, I will add it there.)

Contributions to raise the bounty amount are welcome!

Please let me know if you have any questions. Thanks!

[ychin]

What are some common BG3 mods that use this script extender? I'm new to this repo. Is the point of this repo to basically add an API for Lua scripting so you can write scriptable mods for BG3? (I'm on a vanilla playthrough on my Mac still 😅)

[NextdoorPsycho]

What: A bounty that will be paid out when BG3 SE works on MacOS with the Mac port of BG3.

• The MacOS version of BG3 SE should not cause major crashes or bugs.

• It should support all (or, at least, most) existing mods that work with the Windows version of BG3 SE.

• It should work on Apple Silicon (M1/M2/etc) processors. (Intel processor support is not required)

Amount: $350

Payment Method: Your choice (I am happy to post the bounty to an escrow service, but I don't know them. If you let me know a preferred one, I will add it there.)

Contributions to raise the bounty amount are welcome!

Please let me know if you have any questions. Thanks!

I'd pitch in $150 easily. Have an m3 max chip if needed to test as well

[RnClank42]

What are some common BG3 mods that use this script extender? I'm new to this repo. Is the point of this repo to basically add an API for Lua scripting so you can write scriptable mods for BG3? (I'm on a vanilla playthrough on my Mac still 😅)

Biggest ones I can think of are 5E Spells (https://www.nexusmods.com/baldursgate3/mods/125) and Appearance Edit Enhanced (https://www.nexusmods.com/baldursgate3/mods/899). Other smaller'ish' ones are the custom class and subclass mods that rely on 5E spells.

[brightmoonjones]

What are some common BG3 mods that use this script extender? I'm new to this repo. Is the point of this repo to basically add an API for Lua scripting so you can write scriptable mods for BG3? (I'm on a vanilla playthrough on my Mac still 😅)

Mods that allow you to get cut dialogue/encounters. Polyamory mods, etc.

[brightmoonjones]

What: A bounty that will be paid out when BG3 SE works on MacOS with the Mac port of BG3.

• The MacOS version of BG3 SE should not cause major crashes or bugs.
• It should support all (or, at least, most) existing mods that work with the Windows version of BG3 SE.
• It should work on Apple Silicon (M1/M2/etc) processors. (Intel processor support is not required)

Amount: $350

Payment Method: Your choice (I am happy to post the bounty to an escrow service, but I don't know them. If you let me know a preferred one, I will add it there.)

Contributions to raise the bounty amount are welcome!

Please let me know if you have any questions. Thanks!

I could pitch in $50. I would really love for there to be a Mac version. I'm on a 2020 Intel iMac.

[tsunamaru]

I could pitch in $100, but I would like to add one mandatory requirement: the code should be merged to master branch (so we don't end with some bg3se fork with breaking changes that will quickly become outdated). @whispy what do you think?

Also I'm on M1 Max macbook and can help with testing.

Counting previous contributions, total bounty is now at $650

[ychin]

I'm interested in taking a look at porting this but I have only taken a cursory look at the code so not sure how much work or how feasible it is yet. A big part of it would be how similar the macOS build is to the Windows build. If they have the same code structure then we don't need to reverse engineer the game again and can just focus on porting. Otherwise it may require finding new entry-points to hook in the mods and that could be a lot of work.

Seems like from Norbyte/ositools#43 that @Norbyte just doesn't have Macs and generally don't have the resources to do this but probably would be receptive to this idea at least?

Also, does anyone know if @Norbyte is a commercial entity? Reason I'm asking is that the license for this repository (sorry, boring topic I know) is mostly an MIT license with an additional clause to prevent commercialization. It doesn't really stop us from having a bounty (since the resulting code will still be open source and public) but just wondering about it in general.

Otherwise I may start to take a look during the holiday seasons (I'm still busy playing BG3 as I mentioned, so there are conflicting priorities here… 😅).

Edit: If I do work on this I'll probably just set up a personal GitHub sponsor profile for this. Other than that I do have Macs to test this with (I maintain a macOS open source app) so hopefully won't need testing help until later on.

[Norbyte]

Seems like from Norbyte/ositools#43 that @Norbyte just doesn't have Macs and generally don't have the resources to do this but probably would be receptive to this idea at least?

My comment at the DOS2 SE issue is mostly valid for BG3 as well. One thing that should be noted though, is that porting to a different OS is not just a once-off thing.

Every time the game is updated adjustments must be made in the SE to follow the changes Larian did to the binary. These aren't necessarily because so many changes were done in the code itself, but rather compilers taking different inlining or optimization decisions, different outcomes caused by PGO, etc. Because of this, updating SE after big patch is often a significant effort (days of work).

If you support multiple platforms, every time a game update is made you have to perform the same remapping work on both platforms, test if everything works on every platform, etc. In addition, any time you add a new feature to the SE you need to map it on every supported platform.

In essence, support for a new platform means:

• Adding initial support (essentially a once-off but large task)
• Updates for the platform after game patches
• Adding platform-specific support for each new SE feature

Also, does anyone know if @Norbyte is a commercial entity? Reason I'm asking is that the license for this repository (sorry, boring topic I know) is mostly an MIT license with an additional clause to prevent commercialization. It doesn't really stop us from having a bounty (since the resulting code will still be open source and public) but just wondering about it in general.

He is very much not a commercial entity, just a dude with free time on his hands :P
The commercial clause was added in response to a specific incident.

[ychin]

Do you have examples of what such multi-day tasks look like that need to be made to make SE work with a game update? I'm curious to see what is involved.

Other than that yes, maintenance costs is a frequently ignored cost by anyone other than the maintainer since that person usually ends up having to bear the responsibility. I think if I implement this for macOS I wouldn't mind taking on some of the burden of keeping it up to date as I don't see how someone who doesn't have a Mac can realistically do this to begin with.

Given that macOS now has a native port though I think it's at least a worthwhile goal as it's a first party platform for the game now.

[Norbyte]

For example, these are the changes that needed to be applied after Patch 4 landed:
54f89ab
224aee8

[ssmoss]

I would be willing to chip in towards the bounty as well. I would also be willing to help out in testing with my Intel Mac if needed. I see the two commits you linked, for those who are new to this process, are you able to share a little more on the steps required to generate those diffs? Is it something that could be partially scripted to lower the maintenance burden of supporting more than one platform?

Thanks for all your hard work on this!

[brightmoonjones]

Just coming back to reiterate my support/willingness to chip in for this to be made! :)

[Thievsly]

I'd love to pitch in $20 more for this feature. Think that brings the total bounty to $670 if all parties are still interested in porting the Script Extender for Apple Silicone.

[hannalovesham]

I would also pitch in!! I just want Appearance Edit Enhanced so badly

[Foul-Tarnished]

Norbyte/ositools#43 (comment)

Multiple months of work, how much $ would that cost ?
plus a mac mini

[NextdoorPsycho]

I don't think it would be months the complexity comes in the mappingsand just knowing the format. Once you have that it's game over. I'd also gladly donate a shoddy Mac mini. But I don't think he'd accept a Po box delivery for a mac

[ychin]

@Norbyte I'm still looking into this on the side. I have some questions about how you decompiled the app. Seems like the manifest for the decompiled locations are in https://github.com/Norbyte/bg3se/blob/main/BG3Extender/GameHooks/BinaryMappings.xml, where functions like esv::SavegamePostVisit_DisableStoryMerge are mapped to specific bit patterns. How did you do the decompilation (or do you prefer to keep it private)? I tried to dump the binaries, or just straight up grep'ing the files, and couldn't find say the string SavegamePostVisit anywhere, so I'm wondering what technique you used (or is that a name you just made up?). That said I'm only looking at the macOS binary. Maybe I need to set up a Windows machine to look up the Windows binaries to see if maybe it exists there.

Otherwise, currently I'm looking into a good way to replace the Microsoft Detour usage with a macOS one. From the looks of it there are existing packages to do similar things but they are either unmaintained/barebones, or way over-complicated designed for other purposes. As such I'm debating just implementing it myself as at the basics it's essentially just replacing a couple instructions with a jump and then adding a trampoline function. But this won't be useful if we don't have the entry points for these functions mapped out and decompiled.

[ychin]

Actually, my example was kind of not a good one because it's clear that esv::SavegamePostVisit_DisableStoryMerge is just an internal name for describing a particular patch. That said, the general question still stands. I'm kind of curious how the reverse engineering was done and how various names/symbols (e.g. RPGStats) were obtained. I'm happy to hop on a private communication channel as well.

Otherwise I'm not sure if porting to the Mac is doable by a third party like me as I'm new to this engine and it seems like a significant of reverse engineering was necessarily and I'm not sure if it's possible for me to retrace the steps on macOS (either for ARM or x86) without understanding how the process works in general to get to what it is right now.

Either way I would agree that this seems like a decent amount of work to port.

[tsunamaru]

@ychin if you're asking what tool to use for reverse-engineering, you may look at IDA Pro, NSA Ghidra or radare2. As far as I know IDA Pro is what so-called "industry standard", although license cost is quite expensive; Ghidra and radare2 both free and open source.

But I guess you're more curious how applications are reverse engineered in general, so that is whole different topic. You may want to take some course online before digging in (google: "reverse engineering for beginners" or something similar).

[ychin]

I guess I'm more asking specifically about what the workflow for reverse engineering BG3 was. I can figure out how to use the tools. The job of a Mac port isn't to write a scripting interface like bg3se from scratch. It's to port the existing mappings to Mac, so I'm not trying to reproduce the months/years of work this project + ositools has done, but to figure out the process that went into each entry in say https://github.com/Norbyte/bg3se/blob/main/BG3Extender/GameHooks/BinaryMappings.xml, so we can generate a corresponding entry for macOS x86 / macOS ARM.

But sure, there's a probably tradeoff about whether it's just easier to spend the bounty money on buying a Mac mini for @Norbyte instead which may just be faster.

[henryhoo]

Hey! I am a mac user and have a few mods that I would like to use that depend on SE. I am willing to help porting this for Mac. @ychin Look like you already started some work, let me know how I can help.

Based on the threads look like it would be helpful if @Norbyte can provide some high level steps (or some code pointers) how to achieve this in Mac. (n00b in the reverse-engineering world, but happy to get my hands dirty :) )

[ychin]

Sure @henryhoo I would definitely be interested to collaborate if we progress further. I haven't really started writing any code so to speak, but mostly researching 1) how to properly detour functions in macOS since this repo uses the Microsoft Detour library which is Windows-specific, and 2) how to retrace the reverse engineering done on the Windows/x86 side, as those rely on very explicit bit pattern matching (along with things like fixed string lookups).

FWIW I haven't done that much reverse engineering before as well, but I did used to be a game engine/graphics programmer so I do have some experience working with game engine code, and I maintain a Mac app on GitHub so I have an Apple Developer account.

[whispy]

Sure @henryhoo I would definitely be interested to collaborate if we progress further. I haven't really started writing any code so to speak, but mostly researching 1) how to properly detour functions in macOS since this repo uses the Microsoft Detour library which is Windows-specific, and 2) how to retrace the reverse engineering done on the Windows/x86 side, as those rely on very explicit bit pattern matching (along with things like fixed string lookups).

FWIW I haven't done that much reverse engineering before as well, but I did used to be a game engine/graphics programmer so I do have some experience working with game engine code, and I maintain a Mac app on GitHub so I have an Apple Developer account.

@ychin, I am happy you're looking into this! Some options for hooking/detouring on MacOS are Mach Override (may be obsolete) and DYLD_INTERPOSE. There are probably other options, but those are just the ones that come to mind.

[whispy]

And osxinj too

[ychin]

@whispy I think the libraries you mentioned are there to support overriding loading code from dynamic libraries, similar to LD_PRELOAD in Linux? This would work for dynamically linked functions, but the functions we are detouring are static functions I think. The reverse engineering is identifying the spots where those static functions are called, and then use the detour library to dynamically route the function to your own. It does that by modifying the executable memory to do a jmp to your own function at the beginning of the original function, while saving the original implementation in a trampoline function that you could call. The reason I'm asking about the reverse engineering is because it's pretty much a manual work where you identify which parts of the binary are calling these statically linked function and then dynamically patching their implementation. This would work differently in macOS versus Windows, and also in ARM code.

But this is also why I'm confused at names like ecl__GameStateMachine__Update, as I am trying to understand if names like these are a made-up name, or actually dumped from the game's debug symbols (that I cannot find). Since these functions are statically linked it's not like they are neatly exported in a DLL/dylib.

Correct me if I'm wrong though.

[Sirgilly]

Id chip in $100 towards it as well as a bounty - would be a great thing

[ToasterGoblins]

Is there a place where I can follow this project, or will updates be sent here? don’t really have to money to support, but as a Mac player I would really love to keep an eye on this project as it progresses!

[ychin]

I think this is the issue to track it. I think I and maybe @henryhoo are the people who expressed interests in porting this to Mac. I looked into this last month, but it's a little hard to make progress without any support from the maintainer due to how the detour bindings work. Since it's all done via reverse engineering, what we would need to do isn't just reverse engineering BG3 on the Mac, but also to replicate the steps done in the Windows version of this to provide the same hooks and bindings to the mods.

FWIW, the bounty is just a nice bonus. I was going to recommend the bounty be split among people who work on it plus the maintainer of this project anyway since he would be doing a lot of the review work, maintenance, etc.

Obviously this is @Norbyte's project though, so it's up to him if he has the time to support this which will be some extra effort for sure.

[bpengu1n]

I'd love to help with this effort - I've got some good RE experience under my belt, and would love to assist however I can.

[denimcouch]

I am also interested in helping out!
I have an M1 Max MBP to help test should this pick up momentum.

Having the SE on mac would be so nice 😍

[NextdoorPsycho]

does anyone have any status on this? / Anything holding them up?

[tenarofatuan]

I would also be able to contribute $100 to the bounty. I'm on an M2 mac mini and could help test as well if it would be useful. I think that brings the bounty up to $870.

[coldrain96]

Up

[NextdoorPsycho]

@Norbyte I Assume you own BG3, so the cost of the game is not a problem for you if someone had a spare mac and was interested in lending it to you, or giving it to you. Would that be possible / Something you are interested in. Because at least from what i can read and see, the main problem is NOT that you have no interest, but that you just literally don't have a mac. if i could send it to you / lend / donate it. id not be opposed to it. (its not a powerhouse by any means, but its a dinky m1 air). Let me know!

That being said, you would need the time and effort to do so, so i dont think its JUST the mac as the issue. But If it is. I can help!

[tenarofatuan]

Now that patch 7 is out for everyone... is anyone working on this?

[NextdoorPsycho]

I dont think so