Extract application version from web app store

[nicerloop]

The iTunes search API does not provide application version per platform, and makes mas-cli report a newer version which in fact is for another platform.

The applicaiton metadata are embedded as json in the html page from the web version of the App Store, in a <script type="fastboot/shoebox" id="shoebox-media-api-cache-apps"> tag.

This information provides the application version history per platform.

Example for application Broadcasts https://apps.apple.com/app/broadcasts/id1469995354 :

curl "https://apps.apple.com/app/broadcasts/id1469995354" | tr -d '\n' | sed -e's:.*<script type="fastboot/shoebox" id="shoebox-media-api-cache-apps">::' -e's:</script>.*::' | jq ' to_entries | .[0].value | fromjson | .d[0].attributes.platformAttributes | to_entries[] | { platform: .key , version : .value.versionHistory[0].versionDisplay } '

produces

{ "platform": "appletvos", "version": "2.3" } { "platform": "osx", "version": "2.2.7" } { "platform": "ios", "version": "2.3.3" }

which are correct regarding the different platforms.

I checked this extraction on several cases of erroneous versions reported and so far have not encountered a problem.

[nicerloop]

#336 (comment)

[rgoldberg]

The existing API call used to get the current version of an app is:

https://itunes.apple.com/lookup?id=<app-id>

This currently returns a small JSON object that includes the app's current version, but it doesn't limit the results to the Mac platform (either it's retrieving the results for the platform with the highest version, or it's always preferentially retrieving results for some platform other than the Mac).

The endpoint will, however, accept an additional query string parameter: platform.

The only values that I have found to work are: ipad, iphone & ipod. Other values that I've tried, like mac, iPhone, etc. result in an error. I haven't yet found any value for platform for Macs, Apple TVs, or Apple Watches.

We can either try to find the platform for Macs for the existing endpoint that returns simple JSON, if one exists & works. Or we can switch to scraping an app's App Store HTML page. The former seems cleaner to me, but might take some time, or might not even be possible.

[odebroqueville]

@rgoldberg Problem is that the data in the iTunes database isn't reliable. As an example, I suggest you search for the ID=1365531024 which corresponds to the Mac App Store app called 1Blocker. The results from iTunes show that the latest version is 5.2.5. This is also reported by the mas client which reports that there's an update available from 4.1.7 to 5.2.5. Yet updating doesn't change the version which remains at 4.1.7. If you visit the Mac App Store and search for 1Blocker, you'll also see that the latest version available is 4.1.7. Therefore, it appears that the information from https://apps.apple.com/app/broadcasts/id1365531024 is more reliable.

[rgoldberg]

@odebroqueville Unless I've missed something, the JSON returned for that ID should be for the iPhone version of the app, not the macOS version of the app. I'm trying to find the correct value for the platform query string param that will return the JSON for the macOS version of the app.

[odebroqueville]

@rgoldberg If you use the query https://itunes.apple.com/search?term=id1365531024&country=ch&entity=macSoftware it will result in a json containing:

"supportedDevices":["MacDesktop-MacDesktop", "iPhone5s-iPhone5s", "iPadAir-iPadAir", "iPadAirCellular-iPadAirCellular", "iPadMiniRetina-iPadMiniRetina", "iPadMiniRetinaCellular-iPadMiniRetinaCellular", "iPhone6-iPhone6", "iPhone6Plus-iPhone6Plus", "iPadAir2-iPadAir2", "iPadAir2Cellular-iPadAir2Cellular", "iPadMini3-iPadMini3", "iPadMini3Cellular-iPadMini3Cellular", "iPodTouchSixthGen-iPodTouchSixthGen", "iPhone6s-iPhone6s", "iPhone6sPlus-iPhone6sPlus", "iPadMini4-iPadMini4", "iPadMini4Cellular-iPadMini4Cellular", "iPadPro-iPadPro", "iPadProCellular-iPadProCellular", "iPadPro97-iPadPro97", "iPadPro97Cellular-iPadPro97Cellular", "iPhoneSE-iPhoneSE", "iPhone7-iPhone7", "iPhone7Plus-iPhone7Plus", "iPad611-iPad611", "iPad612-iPad612", "iPad71-iPad71", "iPad72-iPad72", "iPad73-iPad73", "iPad74-iPad74", "iPhone8-iPhone8", "iPhone8Plus-iPhone8Plus", "iPhoneX-iPhoneX", "iPad75-iPad75", "iPad76-iPad76", "iPhoneXS-iPhoneXS", "iPhoneXSMax-iPhoneXSMax", "iPhoneXR-iPhoneXR", "iPad812-iPad812", "iPad834-iPad834", "iPad856-iPad856", "iPad878-iPad878", "iPadMini5-iPadMini5", "iPadMini5Cellular-iPadMini5Cellular", "iPadAir3-iPadAir3", "iPadAir3Cellular-iPadAir3Cellular", "iPodTouchSeventhGen-iPodTouchSeventhGen", "iPhone11-iPhone11", "iPhone11Pro-iPhone11Pro", "iPadSeventhGen-iPadSeventhGen", "iPadSeventhGenCellular-iPadSeventhGenCellular", "iPhone11ProMax-iPhone11ProMax", "iPhoneSESecondGen-iPhoneSESecondGen", "iPadProSecondGen-iPadProSecondGen", "iPadProSecondGenCellular-iPadProSecondGenCellular", "iPadProFourthGen-iPadProFourthGen", "iPadProFourthGenCellular-iPadProFourthGenCellular", "iPhone12Mini-iPhone12Mini", "iPhone12-iPhone12", "iPhone12Pro-iPhone12Pro", "iPhone12ProMax-iPhone12ProMax", "iPadAir4-iPadAir4", "iPadAir4Cellular-iPadAir4Cellular", "iPadEighthGen-iPadEighthGen", "iPadEighthGenCellular-iPadEighthGenCellular", "iPadProThirdGen-iPadProThirdGen", "iPadProThirdGenCellular-iPadProThirdGenCellular", "iPadProFifthGen-iPadProFifthGen", "iPadProFifthGenCellular-iPadProFifthGenCellular", "iPhone13Pro-iPhone13Pro", "iPhone13ProMax-iPhone13ProMax", "iPhone13Mini-iPhone13Mini", "iPhone13-iPhone13", "iPadMiniSixthGen-iPadMiniSixthGen", "iPadMiniSixthGenCellular-iPadMiniSixthGenCellular", "iPadNinthGen-iPadNinthGen", "iPadNinthGenCellular-iPadNinthGenCellular", "iPhoneSEThirdGen-iPhoneSEThirdGen", "iPadAirFifthGen-iPadAirFifthGen", "iPadAirFifthGenCellular-iPadAirFifthGenCellular", "iPhone14-iPhone14", "iPhone14Plus-iPhone14Plus", "iPhone14Pro-iPhone14Pro", "iPhone14ProMax-iPhone14ProMax"],

and "version":"5.2.5"

as well as: "trackViewUrl":"https://apps.apple.com/ch/app/1blocker/id1365531024?uo=4" which is the website for all 3 versions of the app (Mac, iPad and iOS).

In the json file, I don't see any version 4.1.7.

P.S. I used the following documentation: https://developer.apple.com/library/archive/documentation/AudioVideo/Conceptual/iTuneSearchAPI/Searching.html#//apple_ref/doc/uid/TP40017632-CH5-SW1

[rgoldberg]

@odebroqueville Your URL didn't include the platform param. It's possible that that might make the difference (it's also possible that it won't make a difference). I'm asking if anyone knows all the possible values for that param, and which one corresponds to macOS. You keep providing information that doesn't answer my questions. Alternatively, maybe some other params will work.

[odebroqueville]

@rgoldberg where did you get the platform parameter from?

[rgoldberg]

@odebroqueville I think I might have just tried it. Maybe I saw it on some web page, but I don't think so. It is an accepted param, and the values listed in my comment above are accepted. Random param names are rejected, as are random values for platform. So it obviously passes some input checks.

[toobuntu]

#496

[nicerloop]

Probably superceded by #496.

[nihaals]

In case that PR isn't merged/an alternative one is it might make sense to keep this open and have it be closed by the PR being merged.

[nicerloop]

Updated snippet to get version by platform:
curl -sL "https://apps.apple.com/app/id$MASID" | tr -d '\n' | sed -e's:.*<script type="fastboot/shoebox" id="shoebox-media-api-cache-apps">::' -e's:</script>.*::' | jq ' to_entries | .[0].value | fromjson | .d[0].attributes.platformAttributes | to_entries[] | { platform: .key , version : .value.versionHistory[0].versionDisplay } '