[Question] Hosted file import

[scrubbless]

There are a lot of issues with the content and it seems that while the official app is updated (and breaks Valkyrie) from time to time, that no substantial content is added. The fixes are getting quite complicated and we don't seem to have the deployment resources to continue to fix the app to the official application releases.

Since the game is officially out of print, I am wondering if it is time to create some additional functionality to import 3rd party source content, community members could host the content which would remain static but Valkyrie could have an optional feature to import from that source. Something like a content package function, where Valkyrie could export its current (working) contents to a ZIP file that could be shared between community members and uploaded. This will ensure the continued running of the application without having to rely on the official application or the expertise of Valkyrie developers.

I know there are IP issues and the like, but I feel like we need to push Valkyrie into a "legacy release" so that it can stand the test of time as willing developers begin to shift focus to other projects.

What are your thoughts?

[vidimus]

Absolutely! IP is the only backward. It could also open the doors for custom contents and mods.

[seinsinnes]

Yes, I like this idea.

I had started looking at how to make the import work with the latest MoM app update. Since I’m not familiar unity file structures, I’ve been going off https://github.com/Perfare/AssetStudio which can read the asset files of the latest MoM app but integration may be a bit of work.

I think a downloadable blob of importable assets would be a more robust solution. I’m not sure what IP issues may arise but I would hate to see the community decline due to lack of technical skill and attention to keep Valkyrie functioning with MoM.

[scrubbless]

@NPBruce @BenRQ

Would be interested to hear your thoughts?

[NPBruce]

I am happy for someone to host the extracted version, or unextracted that Valkyrie can find, but I don't want to include those files on this repository. Lets keep it at arms length.

[scrubbless]

I am happy with that direction, so really what we need in Valkyrie is an external source import function (select a ZIP file type deal).

Then we can work on a content packaging system, do we want an "export" button to produce a ZIP file of content to import into other machines?
Should we consider a separate app/project?
Would this help with the Android 11 problems?

[NPBruce]

I'd suggest the package is just an installer that installs the export in the right place for you. Should be doable for any platform.

[seinsinnes]

I know that the consensus looks like going with "the installer containing the exported assets" but since I'd already started trying to make the import work with MoM 1.9.3 I wanted to see how doable it was.

I've managed to get it working using the AssetStudio code (Unity_Studio is still there but it isn't doing much)
The changes are here if anyone is curious:
https://github.com/seinsinnes/valkyrie/tree/dev/FixImportFromMoM_1.9.3

It seems to be importing all the textures for tiles, tokens, cards, etc fine (as well as audio clips and text). I've only tested on Mac and MoM so far. Obviously these are pretty extensive changes in the importing code.

There is still a glitch with text/localisation but I expect that to be a minor fix.

[BenRQ]

I would agree that getting a fixed package backup is much welcome, so we can always play the game.
And it's better to do it now while the project is still active.

I remember the difficulty of integrating new MoM version was mostly finding the right id and place for the sprites as Unity was using a specific map tile, which was cut at precise locations to provide all the pictures.
If this issue is solved, integrating new version may be a lot faster too and would be much welcome.

[seinsinnes]

I've fixed the localisation problem with my importing from 1.9.3 branch.
Perhaps I'll make a pull request but I've only built and tested it on my mac so it will need testing on other platforms.

[vidimus]

Great, @seinsinnes . Is there a way to get it? I would like to try it on android and windows.

And by the way, how did you make to run it on a mac?

[seinsinnes]

I've attached the generated binaries to the pull request #1622
I've tested the mac binary and the windows one in a vm but not android or linux yet.

[seinsinnes]

@vidimus Sorry, I'm not sure what the 'it' you're referring to running on the mac is.
If you mean the build process.... The building of a mac build on mac seemed fairly straightforward with correct unity version and visual studio installed.

Haven't tried building for all the platforms directly on the mac.

For generating the full set of binaries I used a windows vm and the build.bat with some modifications. Primarily the change being pointing the build process to the newer msbuild packaged with visual studio. The msbuild in framework v4.0.30319 doesn't support all the language features used by the AssetStudio code.

[scrubbless]

Hi @seinsinnes
Sounds interesting, if you have a windows or android build (.exe or .apk) i'd be happy to test it out on windows.

I gather that you are working on fixing the import function so that it it works with the lasted official app? I assume you've not suggested any additional functionality for offline imports?

[seinsinnes]

Hi @scrubbless,
I haven't done any work on the hosted file import.
The pull request for the fix for importing from MoM 1.9.3 is here #1622

The binaries built from that branch are here:
Valkyrie-android-2.5.9a.apk
valkyrie-linux-2.5.9a.tar.gz
valkyrie-macos-2.5.9a.tar.gz
valkyrie-windows-2.5.9a.7z
valkyrie-windows-2.5.9a.exe
valkyrie-windows-2.5.9a.zip

I haven't tested the android binary so I am curious whether that is working.

[Salva-OA]

In Windows, after the installation of Valkyrie 2.5.9, do not shows unable to locate, but after a big lapse of time shows the initial message "reimport content"; like it can read the folder but does not complete the reading of files.
This is with version 1.9 of mdm since I could not find the console on Steam to do a downgrade of the version to 1.8 or other.

[seinsinnes]

"reimport content"
Is what shows after import has been successful.

It just gives you the option of reimporting if you need to for any reason.

[scrubbless]

Unfortunately this isn't compatible with my Pixel 7, I have a Galaxy S9+ kicking around, I'll try installing on that

[seinsinnes]

Oh, did Valkyrie 2.5.8 work previously on that phone?
Is this incompatibility introduced by my changes?

[Salva-OA]

Thanks, its true, it worked for me in windows

[scrubbless]

I have tried to the Android build on my S9+ and it just goes to a black screen after 'unity' appears. I checked and it has storage permissions.

[seinsinnes]

Right, thanks, I might have to setup android emulation to debug what is going wrong here.

[seinsinnes]

Ok, I couldn't get it installed on real android devices until I rebuilt it against platform sdk version 29. The previous apk I made was built against platform sdk version 34. So there might've been a problem with my build environment.

This is the new apk:
Valkyrie-android-2.5.9b.apk

[seinsinnes]

That said, when I test it on the two devices I have (one android 11 and one android 13) I get an exception:
2023-08-04 16:52:06.577 30668-30704 Unity com.bruce.valkyrie E UnauthorizedAccessException: Access to the path '/storage/emulated/0/Android/obb/com.fantasyflightgames.mom' is denied. at System.IO.__Error.WinIOError (System.Int32 errorCode, System.String maybeFullPath) [0x00129] in <2fa7a6a452ca43df998f07fd1486c0df>:0 at System.IO.FileSystemEnumerableIterator1[TSource].HandleError (System.Int32 hr, System.String path) [0x00006] in <2fa7a6a452ca43df998f07fd1486c0df>:0
at System.IO.FileSystemEnumerableIterator1[TSource].CommonInit () [0x00054] in <2fa7a6a452ca43df998f07fd1486c0df>:0 at System.IO.FileSystemEnumerableIterator1[TSource]..ctor (System.String path, System.String originalUserPath, System.String searchPattern, System.IO.SearchOption searchOption, System.IO.SearchResultHandler1[TSource] resultHandler, System.Boolean checkHost) [0x000d6] in <2fa7a6a452ca43df998f07fd1486c0df>:0 at System.IO.FileSystemEnumerableFactory.CreateFileNameIterator (System.String path, System.String originalUserPath, System.String searchPattern, System.Boolean includeFiles

And a blank screen.

[seinsinnes]

I get the same behaviour with Valkyrie 2.5.8.
I don't have much experience developing for android. Which android versions don't block one app accessing another app's data?

It seems like only way of getting around being blocked by the operating system may in fact be hosting it separately rather than extracting it from the official app.

[seinsinnes]

The latest official MoM app also moved around the data install locations. I've fixed finding them:
Valkyrie-android-2.5.9b.apk
I've tested the apk and it is working on android 8,9,10 (I haven't tested lower versions).
It doesn't work on android 11+ due to scoped storage. I haven't found away around that other than having the user move data around themselves or the hosted file import solution.

[victorts1991]

How can I move the data myself? Do you have steps?

[seinsinnes]

@victorts1991 manually moving the data was more hypothetical but I've made a version which allows that:
Valkyrie-android-2.5.9c.apk

If you use that apk and can copy the folder Android/data/com.fantasyflightgames.mom to Valkyrie/com.fantasyflightgames.mom then the import will run successfully.

This works on my Android 13 device. To do the copy just requires a file browser with enough access to the storage.
I used https://play.google.com/store/apps/details?id=com.cxinventor.file.explorer&hl=en&gl=US
For some reason I've been unable to reproduce the same access to the app data in Valkyrie.

[victorts1991]

I downloaded your app and this opened and started the intro but when I go to start the scenario the app crashes

Sorry my english kkkk

[victorts1991]

He crash after intro text scenario finish

[seinsinnes]

@victorts1991 I'm sorry to hear.
This crash could be related to the import or not. I've personally only been working the import code recently.
Could you move this to a new issue and add which android version and device? Which scenario? Is it all scenarios?
Was the import achieved by manually copying Android/data/com.fantasyflightgames.mom?

[vidimus]

He crash after intro text scenario finish

Be sure you have all the needed expansions marked on your collection. That kind of behaviour is typical when the Valkyrie wants to load an asset from one expansion and the expansion is not selected on the collection.

[victorts1991]

I enabled all expansions and when you click to proceed, the app crashes on the screen below, in any scenario.

[victorts1991]

I connected the phone to Android Studio when I clicked and the app crashed, and looking at the logs looking for the word Valkyrie, these errors appeared below:
2023-08-17 13:05:03.744 29183-18779 ActivityManagerWrapper pid-29183 E getRecentTasks: mainTaskId=272 userId=0 baseIntent=Intent { act=android.intent.action.MAIN flag=271581184 cmp=ComponentInfo{com.bruce.valkyrie/com.unity3d.player.UnityPlayerActivity} }

 E  process_name_ptr:14398 com.google.process.gservices
2023-08-17 13:05:04.704 14398-14398 zygote64                pid-14398                            E  Failed to parse /product/etc/mfz.xml: Tinyxml2 error (3): Error=XML_ERROR_FILE_NOT_FOUND ErrorID=3 (0x3) Line number=0: filename=/product/etc/mfz.xml
2023-08-17 13:05:04.704 14398-14398 zygote64                pid-14398                            E  MemoryFreeze XMLFile not found
2023-08-17 13:05:04.704  1851-2080  ActivityTaskManager     pid-1851                             I  Config changes=20000480 {1.0 724mcc5mnc [pt_BR] ldltr sw392dp w392dp h822dp 440dpi nrml long hdr widecg port finger -keyb/v/h -nav/h winConfig={ mBounds=Rect(0, 0 - 1080, 2400) mAppBounds=Rect(0, 93 - 1080, 2356) mMaxBounds=Rect(0, 0 - 1080, 2400) mDisplayRotation=ROTATION_0 mWindowingMode=fullscreen mDisplayWindowingMode=fullscreen mActivityType=undefined mAlwaysOnTop=undefined mRotation=ROTATION_0 mInSplitScreen=false letterBoxed=false} as.2 s.5711 fontWeightAdjustment=0 themeChanged=0 themeChangedFlags=0 extraData = Bundle[{}]/d}
2023-08-17 13:05:04.707  5723-14456 MediaProvider           pid-5723                             I  Deleted 0 1 0 0 0 0 0 items on external_primary due to com.bruce.valkyrie
2023-08-17 13:05:04.710 14398-14398 ocess.gservices         pid-14398                            I  Using CollectorTypeCC GC.
2023-08-17 13:05:04.720 14398-14398 ocess.gservices         pid-14398                            E  Not starting debugger since process cannot load the jdwp agent.
2023-08-17 13:05:04.728  5723-6301  MediaProvider           pid-5723                             I  Deleted 0 1 0 0 0 0 0 items on external_primary due to com.bruce.valkyrie
2023-08-17 13:05:04.735 14398-14398 System.err              pid-14398                            W  android.system.ErrnoException: open failed: ENOENT (No such file or directory)
2023-08-17 13:05:04.735 14398-14398 System.err              pid-14398                            W  	at libcore.io.Linux.open(Native Method)
2023-08-17 13:05:04.735 14398-14398 System.err              pid-14398                            W  	at libcore.io.ForwardingOs.open(ForwardingOs.java:563)
2023-08-17 13:05:04.735 14398-14398 System.err              pid-14398                            W  	at libcore.io.BlockGuardOs.open(BlockGuardOs.java:274)
2023-08-17 13:05:04.735 14398-14398 System.err              pid-14398                            W  	at libcore.io.ForwardingOs.open(ForwardingOs.java:563)
2023-08-17 13:05:04.735 14398-14398 System.err              pid-14398                            W  	at android.app.ActivityThread$AndroidOs.open(ActivityThread.java:8288)
2023-08-17 13:05:04.735 14398-14398 System.err              pid-14398                            W  	at android.system.Os.open(Os.java:494)
2023-08-17 13:05:04.735 14398-14398 System.err              pid-14398                            W  	at android.os.perfdebug.PerfDebugMonitorImpl.monitorVersionControl(PerfDebugMonitorImpl.java:204)
2023-08-17 13:05:04.735 14398-14398 System.err              pid-14398                            W  	at android.os.perfdebug.PerfDebugMonitorImpl.prepareMonitor(PerfDebugMonitorImpl.java:185)
2023-08-17 13:05:04.735 14398-14398 System.err              pid-14398                            W  	at android.app.ActivityThread.main(ActivityThread.java:8375)
2023-08-17 13:05:04.735 14398-14398 System.err              pid-14398                            W  	at java.lang.reflect.Method.invoke(Native Method)
2023-08-17 13:05:04.735 14398-14398 System.err              pid-14398                            W  	at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:559)
2023-08-17 13:05:04.735 14398-14398 System.err              pid-14398                            W  	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:954)
2023-08-17 13:05:04.736 14398-14398 MessageMonitor          pid-14398                            I  Load libmiui_runtime
2023-08-17 13:05:04.754  1851-2080  WindowManager           pid-1851                             I  Override config changes=20000480 {1.0 724mcc5mnc [pt_BR] ldltr sw392dp w392dp h822dp 440dpi nrml long hdr widecg port finger -keyb/v/h -nav/h winConfig={ mBounds=Rect(0, 0 - 1080, 2400) mAppBounds=Rect(0, 93 - 1080, 2356) mMaxBounds=Rect(0, 0 - 1080, 2400) mDisplayRotation=ROTATION_0 mWindowingMode=fullscreen mDisplayWindowingMode=fullscreen mActivityType=undefined mAlwaysOnTop=undefined mRotation=ROTATION_0 mInSplitScreen=false letterBoxed=false} as.2 s.5711 fontWeightAdjustment=0 themeChanged=0 themeChangedFlags=0 extraData = Bundle[{}]/d} for displayId=0
2023-08-17 13:05:04.775   563-563   lowmemorykiller         pid-563                              I  Kill 'com.bruce.valkyrie' (12314), uid 10625, oom_score_adj 900 to free 360936kB rss, 133284kB swap; reason: cw

2023-08-17 13:05:05.063 1851-5478 InputManager-JNI pid-1851 W Input channel object 'c8ebab5 com.bruce.valkyrie/com.unity3d.player.UnityPlayerActivity (client)' was disposed without first being removed with the input manager!

W  Input channel object 'Letterbox_left_ActivityRecord{ca27b05 u0 com.bruce.valkyrie/com.unity3d.player.UnityPlayerActivity} t272} (client)' was disposed without first being removed with the input manager!
2023-08-17 13:05:05.113  1851-5946  InputManager-JNI        pid-1851                             W  Input channel object 'Letterbox_top_ActivityRecord{ca27b05 u0 com.bruce.valkyrie/com.unity3d.player.UnityPlayerActivity} t272} (client)' was disposed without first being removed with the input manager!
2023-08-17 13:05:05.113  4271-4519  VRI[RoundCornerBottom]  pid-4271                             D  vri.reportDrawFinished syncSeqId=92 android.view.ViewRootImpl.lambda$createSyncIfNeeded$4$android-view-ViewRootImpl:3944 android.view.ViewRootImpl$$ExternalSyntheticLambda1.run:6 android.os.Handler.handleCallback:942 android.os.Handler.dispatchMessage:99 android.os.Looper.loopOnce:211 
2023-08-17 13:05:05.113  1851-5946  InputManager-JNI        pid-1851                             W  Input channel object 'Letterbox_right_ActivityRecord{ca27b05 u0 com.bruce.valkyrie/com.unity3d.player.UnityPlayerActivity} t272} (client)' was disposed without first being removed with the input manager!
2023-08-17 13:05:05.113  1851-5946  InputManager-JNI        pid-1851                             W  Input channel object 'Letterbox_bottom_ActivityRecord{ca27b05 u0 com.bruce.valkyrie/com.unity3d.player.UnityPlayerActivity} t272} (client)' was disposed without first being removed with the input manager!

[seinsinnes]

This bit looks most problematic to me:

2023-08-17 13:05:04.775 563-563 lowmemorykiller pid-563 I Kill 'com.bruce.valkyrie' (12314), uid 10625, oom_score_adj 900 to free 360936kB rss, 133284kB swap; reason: cw

Looks like valkyrie was killed because the system ran out of memory.