🐛 `mas account` doesn't work on macOS 12 and later
lukasmalkmus opened this issue · 73 comments
Your Environment
mas version: 1.8.3- macOS version (
system_profiler SPSoftwareDataType -detailLevel mini): 12.0 (21A5522h)
mas Install Method
-
brew install mas(homebrew-core) - [] mas-cli/tap
- []
.pkginstaller from releases - [] Built from source
- Fork/branch: ? (e.g. mas-cli/main)
- Xcode version: 10.?
Describe the Bug
mas account says I'm not signed in, although I am:
Not signed in
Error: Not signed in
To Reproduce
Steps to reproduce the behavior:
- Run
mas account - Make sure App Store has a signed in Apple ID
- Run
mas accountagain
Expected Behavior
Getting no error and being able to use other mas commands that depend on being signed din.
Actual Behavior
Not signed in
Error: Not signed in
Screenshots, Terminal Output
If applicable, add screenshots to help explain your problem.
$ mas account
Not signed in
Error: Not signed inAdditional Context
It stopped working around 2 weeks ago I guess? Maybe it worked with the previous version?
Same here (with same macOS Monterey version). I noticed to it stopped working after it updated to Beta 7, mas had been working on previous betas without issue.
I can reproduce this. I am also running macOS Monterey beta 7.
@lukasmalkmus, @malob, do either or both of you see this when iCloud Private Relay is enabled? Is it reproducible with iCloud Private Relay disabled?
@chris-araman, unfortunately I’m now away from my macOS machine for a few days, so won’t be able to test this until Thursday.
I'm able to reproduce this with Private Relay disabled in macOS 12 beta 7.
Guessing this is due to an API change in Monterey I ran our update_headers script, and it appears class-dump can't find the binary.
↪ script/update_headers
/Users/phatblat/bin/class-dump
class-dump: Input file (/System/Library/PrivateFrameworks/CommerceKit.framework/CommerceKit) does not exist.
class-dump: Input file (/System/Library/PrivateFrameworks/StoreFoundation.framework/StoreFoundation) does not exist.
Dumper agrees that these frameworks are missing the binary. I wonder if Monterey has somehow hid them from our prying eyes 👀.
@phatblat, those binaries don't exist in the file system in Big Sur either, due to the dyld cache. There is some sample code from Apple (search for dsc_extractor.bundle) that can extract binaries from the cache.
That said, class-dump doesn't seem to be able to handle recent x86-64/x86-64h binaries extracted this way, and definitely doesn't handle arm64/arm64e binaries. Maybe Dumper does?
This is what I was trying to address for Apple Silicon Macs when I last contributed to class-dump. 😃
I ran across Extracting libraries from dyld_shared_cache and was impressed with his/her understanding of the process. Using https://github.com/zhuowei/dsc_extractor_badly I was able to get a CommerceKit x86_64 binary. Neither class-dump nor Dumper were able to work with it, but IDA Free is able to understand this file.
I'll try updating the headers manually and then see which new methods might return the active account.
I have the same problem on the latest macOS 12 beta. As I am running tests for my install scripts in the moment it would be very nice to see this fixed.
Thanks in advance.
Same problem with Release Candidate - macOS 12.0 Beta (21A5552a)
As Monterey has been released, it would be nice if mas supports it (fully).
(I understand it takes a lot of work to play around with private frameworks and fix code for macOS upgrades. Keep up the good work — mas is a very useful tool!)
I can confirm the same issue on macOS Monterey (12.0.1) (21A559)
$ mas version
1.8.3
$ mas account
Not signed in
Error: Not signed in
Same issue on macOS 12.0.1, 16" MacBook Pro 2021.
Same issue on macOS 12.0.1, 13" MacBook Pro 2020 M1
Same on macOS 12.0.1, 15.4" MacBook Pro 2018.
1.8.3
❯ mas account
Not signed in
Error: Not signed in```
Same here.
Hey Team,
This replicates on macOS 12.0.1 (21A559), MacBook Air (M1, 2020). Let me know if you need any specific tests/checks.
Same here
Repros on 12.1 Beta as well.
@chris-araman @phatblat
Any news if this issue is being adressed?
Is there a workarround?
How can we help?
Hi, all!
Any news if this issue is being addressed?
You're in the right place. Efforts toward resolving this issue will be tracked here. @phatblat and I are both volunteer maintainers with obligations apart from mas maintenance. We are not paid (though I do welcome sponsorship). We do this because we love open source and want to give back to the community. We are also mas users.
Is there a workarround?
We're not aware of a workaround at this time, other than using the App Store app, of course.
How can we help?
As you're probably aware, Apple has made some changes to the private frameworks mas uses to manage App Store apps on your Mac. Complicating things, the class-dump tool that was used by the original mas author to generate Objective-C headers from those private frameworks does not support extracting dylibs from a cache, and does not appear to support the x86_64h, arm64, or arm64e architectures. The original mas author is no longer involved in the mas or class-dump projects, and class-dump appears to have been abandoned.
A few things have to happen in order to resolve this issue:
- We need tooling to generate Objective-C headers from current
CommerceKit.frameworkandStoreFoundation.framework. @phatblat has already done some investigation toward this effort, but there's more to do. - This will probably require tooling to extract the binary modules of those frameworks from the dylib cache that was introduced in macOS 11 Big Sur. There is some sample
dsc_extractorcode available from Apple that seems to work well enough for this, but it isn't (yet?) maintained as a standalone tool. This snippet may have already been built into some third-party tools, so it might not be necessary to publish it as a standalone tool. - We may or may not need to find or build tooling to support extracting modern arm64/arm64e binaries or generating headers from them.
- We then need to figure out what has changed in these private frameworks in order to determine whether there might be a new path to
masfunctionality. This may involve making new calls into these frameworks, or dumping headers for some additional framework(s). This one is the trickiest to estimate effort involved. It could be a one-liner, or it could be very involved, or it could be infeasible.
tl;dr: This is going to take time and effort. We welcome any helpful contributions from subject matter experts!
That said, additional reports of mas errors on Monterey aren't necessary or helpful right now. Please do try to minimize the noise on this thread while we work to solve the problem. We appreciate it!
When you use dsc_extractor to dump a dylib from the shared cache, it doesn't really give you a proper dylib. Lots of things get rewritten during the linking process that generates the shared cache, and dsc_extractor doesn't undo most of that. So, you mostly get a mach-o dylib that, at best, superficially resembles the original, but is mostly broken internally. @phatblat, you probably found lots of broken xrefs in IDA.
I'm not sure what the specific issues are with class-dump but I wouldn't be surprised if that's part of it.
Afaik, IDA still doesn't support the new dyld shard cache format in Monterey (split across several slices), but I have tooling that allows me to re-join the slices. I can then load it up and disassemble individual dylibs cleanly, for the Apple Silicon version (not the intel version, yet).
I don't know much about class-dump but if there exists (or someone wants to write) an IDA python script that replicates what it does, I can give it a try and share its output. Or really any IDA script that might give you useful info.
Regarding dyld_shared_cache, here are a couple resources for informational purposes. Neither of these help move the ball down the field, but hopefully they offer a bit of insight into the problem space.
There's a project called DyldExtractor which unfortunately doesn't work on macOS shared caches; it's iOS only. But looking through it can give you some insight into how hard a problem it is to reconstruct the dylibs. It's a pure python project that does not use dsc_extractor. It parses the shared cache and attempts to reconstruct the dylibs as faithfully as possible.
https://github.com/arandomdev/DyldExtractor
And here's a project that essentially dlopen()s dsc_extractor from Xcode to extract individual dylibs from the shared cache. Since it relies on Xcode's dsc_extractor it does support the new split format, but that also means it spits out broken dylibs. This doesn't get you any farther that what @phatblat probably has done, but just linking here as a generally useful tool.
https://github.com/keith/dyld-shared-cache-extractor
cheers,
zach
...I have tooling that allows me to re-join the slices. I can then load it up and disassemble individual dylibs cleanly...
Any chance you'd want to open source that? 😁 What are you using for disassembly?
FWIW, even the NSA doesn't seem to be able to grock Monterey's dylib_shared_cache yet.
Unfortunately I can't share the tool since it's work-related
As for disassembly, I'm using IDA to load the reconstructed dyld_shared_cache, and then disassemble individual dylibs. So, happy to run any IDA python scripts against my IDB and share the results if that helps.
And yeah I filed that bug on Ghidra :-)
@zcutlip, if you're able to somehow generate headers for these frameworks on macOS 12.0.1 and either post a link to them here, or fork and drop them in a branch for review, it'd be greatly appreciated. Not sure if that's possible given your current tooling. I'm not yet familiar with IDA, let alone Python scripting for it, unfortunately.
I'll see what I can do. I'm not super familiar with the class-dumping side of it. If there's something already out there, particularly in python, I'll see if I can adapt it.
Still not fixed?
Still not fixed?
Please read first, this will take some time to fix!
Shouldn't it be marked as "not working" on Monterey on brew.sh? It's confusing and creates unnecessary noise. Sorry for my snark earlier.
Shouldn't it be marked as "not working" on Monterey on brew.sh?
Maybe file that bug with homebrew-core then?
https://github.com/Homebrew/homebrew-core
Ok, doing that now.
EDIT: no idea how to report this. New issues require outputs from Brew doctor, and this is not that kind of issue.
Just a quick update on my progress:
I'm giving this project a try:
https://github.com/ChiChou/IDA-ObjCExplorer
It was a bit broken on Monterey's dyld_shared_cache and the latest IDA, but I was able to hack on it a bit and git it going.
Unfortunately, the first thing it tries to resolve, it failed on. It parses __objc_protolist and for each entry, tries to resolve name, types, and impl. It wasn't able to resolve any names. The name pointers look reasonable, but are slightly outside of CommerceKit's mapped addresses. I'm guessing they point into another library or framework's segments? Not sure how protocols work really.
Anyway, if my theory is correct, then creating an IDB with all of CommerceKit's dependencies loaded may help. My original IDB only had CommerceKit itself. So, yesterday afternoon I created a new IDB and had IDA load all dependencies. For anyone unfamiliar with loading dyld_shared_cache in IDA, parsing a library and all its dependencies is non-trivial. The resulting IDB loaded about 500 dylibs and is about 17GB. I let IDA analyze all afternoon and through the night, and it's still going this morning. Hopefully analysis will finish soon and I can try the above plugin again.
In the mean time if anyone comes up with an easier/faster way, like a class-dump implementation that runs directly on the shared cache, my feelings won't be hurt. Happy to be preempted by better ideas.
Cheers,
Zach
I was looking at this tool
It extracts and apparently works with Hooper
https://github.com/keith/dyld-shared-cache-extractor
I saw this interesting comment regarding using Xcode13
keith/dyld-shared-cache-extractor#1 (comment)
Hopper
https://www.hopperapp.com
For sure, it's an interesting tool. But see my comment about its reliance on dsc_extractor and spitting out broken dylibs
#417 (comment)
cheers
I think I have a workaround for the issue preventing install, lucky, and I think upgrade from working on Monterey in #428. It doesn't exactly fix account, because macOS 12 appears to no longer expose this information, or at least not via the same API. However, I suspect most people following this issue are here for redownloads and upgrades. 🤗
Getting new headers generated will still be useful to see what's changed and to see if we can re-enable account on macOS 12, or possibly even purchase on macOS 10.15 and signin on macOS 10.13. I'm hopeful others with more expertise in that area will be able to meet that goal! 🍀
And here's a project that essentially dlopen()s dsc_extractor from Xcode
#417 (comment)
For sure, it's an interesting tool.
It also look as if needs to be complied using Xcode 13 to read the new cache format.
Mas is compiled using Xcode 11.5 ?
I saw this interesting comment regarding using Xcode13
keith/dyld-shared-cache-extractor#1 (comment)
@andrewcrook, building mas requires Swift 5.3 or later, which is included with Xcode 12 or later. That said, whatever tools used to extract dylibs or generate headers from them don't necessarily need to be built with the same Xcode release as mas.
Managed to get the function list out of IDA free using the "Copy All" context menu entry.
This still needs additional manual processing to generate usable Objective-C headers.
Just a thought: do these frameworks exist on iOS? DyldExtractor (linked above) does a pretty good job of extracting dylibs from iOS's shared cache. They're not actually linkable/usable at run-time but they're very complete for static analysis purposes. If class-dump generally works on iOS libraries, and the frameworks you need exist on iOS, would that be an angle worth looking at? Happy to give it at try.
In other news: IDA FINALLY finished analyzing my 17GB IDB tonight. Just manually poking around, it seems like obj-c xrefs that weren't resolving in the "single-module" IDB are now resolving in the "I pulled on the thread until the whole sweater unravelled" IDB. It's a bit late for me to go back down the rabbit hole, but I'll have a look in the morning, and see what there is to see.
Cheers
@zcutlip That's a good question. I haven't ever looked for these private frameworks on iOS, but that did have an app store first so these low-level frameworks may be used on both platforms.
I haven't ever used class-dump on iOS frameworks. Work I do on that platform is to ship to the App Store so we steer clear of Apple's private stuff.
@phatblat Okay, good to know. I'll try to take a look in parallel to my macOS efforts
Looks like iOS has iTunesStore (private) and StoreKit (public, on all Apple platforms). The former may have been the basis for CommerceKit and StoreFoundation.
#428 has been merged, v1.8.5 released, and bottles have been published. This resolves the issue many of us were having on Monterey with install, lucky, and upgrade.
For folks using brew bundle with mas dependencies (as I do), you may need to wait for corresponding changes to Homebrew-bundle.
I will leave this issue open, as the symptom of mas account not working on Monterey remains.
Slower progress than I'd hoped, but getting closer. I was able to get the IDA plugin I'm working on to mostly resolve all the protocol and class structures.
Where it's currently choking is class & instance method names. It's able to find the type strings as well as the actual method IMPL
IDA can clearly work these out, as @phatblat as demonstrated. I'll see if there's a way to either ask IDA "what's the Obj-C method name for this IMPL?" or somehow parse @phatblat's CSV data.
Regarding typing, this plugin isn't currently rendering that, but I think the type encoding is well understood, and we're finding the type strings. So shouldn't be hard to fix up the output to look like a real objc header.
IDA output attached below
ida-output.txt
cheers
In the previously output, the methods like UNKNOWN_0x1a2b9f210 are actually known, but we just can't programmatically resolve the method names, so I'm putting in a placeholder.
Thanks, @zcutlip! Looking forward to seeing how this progresses.
Here's an example in case anyone has any ideas. This is the class method list for CKDialogController. It only has one entry, which should be for +[CKDialogController sharedDialogController]
ommerceKit:__objc_methlist:00000001A2938E98 __OBJC_$_CLASS_METHODS_CKDialogController DCW 0xF
CommerceKit:__objc_methlist:00000001A2938E98 ; DATA XREF: CommerceKit:__objc_class_ro:00000001DC1EAB20↓o
CommerceKit:__objc_methlist:00000001A2938E9A DCW 0xC000
CommerceKit:__objc_methlist:00000001A2938E9C DCB 1
CommerceKit:__objc_methlist:00000001A2938E9D DCB 0
CommerceKit:__objc_methlist:00000001A2938E9E DCB 0
CommerceKit:__objc_methlist:00000001A2938E9F DCB 0
CommerceKit:__objc_methlist:00000001A2938EA0 DCD 0xE10DC5 ; signed offset to method name, except not really?
CommerceKit:__objc_methlist:00000001A2938EA4 DCD 0x1896D ; signed offset to type string
CommerceKit:__objc_methlist:00000001A2938EA8 DCD -0x3D6F8 ; signed offset to IMP
CommerceKit:__objc_methlist:00000001A2938EAC DCD 0
The address of the entry is 0x1a2938ea0
To get the type string: 0x1a2938ea0 + 0x1896D = 0x1a2951811, which points to "@16@0:8"
To get the method impl: 0x1a2938ea0 + (-0x3D6F8) = 0x1A28FB7B0, which is the address of __CKDialogController_sharedDialogController_
But computing the name pointer: 0x1a2938ea0 + 0xe10dc5 gives a nonsense location.
The actual location of "sharedDialogController" is 0x1A29476D7
CommerceKit:__objc_methname:00000001A29476D7 aShareddialogco_0 DCB "sharedDialogController"
Okay, think I figured something out. The dyld_shared_cache has table of "preoptimized" strings for all the libraries bundled in the cache. This is located in the libobjc.A:__OBJC_RO segment. It starts out with a table of what appear to be class names, from all across the shared cache. Following that there appears to be a table of selector strings.
ibobjc.A:__OBJC_RO:00000001CA535943 unk_1CA535943 DCB 0xF0 ; DATA XREF: getMethodNoSuper_nolock(objc_class *,objc_selector *)+8C↑o
libobjc.A:__OBJC_RO:00000001CA535943 ; fixupMethodList(method_list_t *,bool,bool)+A8↑o ...
libobjc.A:__OBJC_RO:00000001CA535944 DCB 0x9F
libobjc.A:__OBJC_RO:00000001CA535945 DCB 0xA4
libobjc.A:__OBJC_RO:00000001CA535946 DCB 0xAF
libobjc.A:__OBJC_RO:00000001CA535947 DCB 0
libobjc.A:__OBJC_RO:00000001CA535948 aRetain DCB "retain",0 ; DATA XREF: _objc_retain_0:loc_18019836C↑o
libobjc.A:__OBJC_RO:00000001CA535948 ; _objc_retain_0+70↑o ...
libobjc.A:__OBJC_RO:00000001CA53594F aRelease DCB "release",0 ; DATA XREF: _objc_release_0:loc_18019A7C8↑o
libobjc.A:__OBJC_RO:00000001CA53594F ; _objc_release_0+8C↑o ...
libobjc.A:__OBJC_RO:00000001CA535957 aRetaincount DCB "retainCount",0 ; DATA XREF: -[OS_xpc_int64 retainCount]+28↑o
libobjc.A:__OBJC_RO:00000001CA535957 ; -[OS_xpc_int64 retainCount]+2C↑o ...
libobjc.A:__OBJC_RO:00000001CA535963 aAutorelease_0 DCB "autorelease",0 ; DATA XREF: _objc_autorelease:loc_18019DC30↑o
libobjc.A:__OBJC_RO:00000001CA535963 ; _objc_autorelease+54↑o ...
libobjc.A:__OBJC_RO:00000001CA53596F aTryretain DCB "_tryRetain",0 ; DATA XREF: -[NSObject retainWeakReference]↑o
libobjc.A:__OBJC_RO:00000001CA53596F ; -[NSObject retainWeakReference]+4↑o ...
libobjc.A:__OBJC_RO:00000001CA53597A aIsdeallocating DCB "_isDeallocating",0 ; DATA XREF: -[NSObject allowsWeakReference]+C↑o
libobjc.A:__OBJC_RO:00000001CA53597A ; -[NSObject allowsWeakReference]+10↑o ...
libobjc.A:__OBJC_RO:00000001CA53598A aCopy_0 DCB "copy",0 ; DATA XREF: _objc_setAssociatedObject:loc_1801A5B34↑o
libobjc.A:__OBJC_RO:00000001CA53598A ; _objc_setAssociatedObject+B4↑o ...
libobjc.A:__OBJC_RO:00000001CA53598F aCopywithzone DCB "copyWithZone:",0 ; DATA XREF: -[NSObject copy]↑o
libobjc.A:__OBJC_RO:00000001CA53598F ; -[NSObject copy]+4↑o ...
I believe the 0xE10DC5 offset is from the start of that selector string table, which gets us to 0x1cb346708:
ibobjc.A:__OBJC_RO:00000001CB346708 aShareddialogco DCB "sharedDialogController",0
libobjc.A:__OBJC_RO:00000001CB346708 ; DATA XREF: CommerceKit:__objc_selrefs:00000001D6F78788↓o
We'll see if that theory holds true for all the selector strings
Okay, got method selector strings resolved. Still CommerceKit. Haven't gotten to StoreKit yet.
ida-output.txt
Remaining todo:
- Resolve type strings to actual parameter and return types
- Properties
- ivars
- ObjcExplore has a TODO for class methods for protocols, but I haven't seen any so far in IDA. Are they uncommon?
- what else?
I'm working on resolving encoded type strings for the methods, and I thought I'd lean on the class-dump project for this, since it knows how to do that. Does anyone have experience with the formatType utility that's part of that project?
./build/Release/formatType
formatType 4.0 (mas fork)
Usage: formatType [options] <input file>
where options are:
-m format method (default is to format ivars)It looks like it takes as input a text file with one encoded type string per line, ignoring // lines and empty lines.
But I'm getting weird output. For example:
v48@0:8@16@24@32@?40
becomes:
(void)v48@0:(id /* CDUnknownBlockType */)arg1 8@16@24@32@?40My IDA objective-C plugin seems to be working well. However transforming the encoded type strings into actual method signatures is currently a blocker for me.
What I could do instead is generate something machine parseable like JSON with all the class and protocol pieces such as class names, class & instance selector strings, ivar names, type strings, etc. Then if someone is able to decode those type strings, it should be fairly easy to injest the JSON and spit out proper header files.
Would that be of use?
For what it's worth, class-dump has the logic to do the type string decoding (assuming the encoding hasn't changed recently) I'm just not sure how to exercise that capability separately from its actual mach-o parsing and actual class dumping.
Yeah, I think that could work. We might be able to extract the string decoding logic from class-dump to its own command.
Alternatively, do you think it should be possible to teach our fork of class-dump to read your JSON? That way it could still write headers out as it did from Mach-O binaries.
Thanks all that are working here to help mas work on Monterey! For anyone finding their way over here from https://github.com/geerlingguy/ansible-collection-mac/blob/master/roles/mas/README.md, I did a quick workaround to get mas account to satisfy the ansible task that was using it ... you need to be logged in, and your config needs to just have empty strings for your mas account info, but then you can create this simple script to trick the ansible task:
#!/usr/bin/env bash
if [[ $1 == "account" ]]
then
exit 0
fi
mas.bin $@
Rename your mas binary to mas.bin and save this script as mas in the same location as mas.bin.
Sorry for being slow on this. Actual job-related work has taken over. I'm taking the rest of the week off for the holiday, so I hope to at least get some JSON generated soon. Then I'll see if we can teach class-dump to ingest that
No need to apologize. Your expertise, efforts, and time are appreciate.
Okay, got method selector strings resolved. Still CommerceKit. Haven't gotten to StoreKit yet. ida-output.txt
Remaining todo:
- Resolve type strings to actual parameter and return types
- Properties
- ivars
- ObjcExplore has a TODO for class methods for protocols, but I haven't seen any so far in IDA. Are they uncommon?
- what else?
sorry for jumping in, but have you tried my tool ipsw: ipsw dyld macho DSC StoreKit --objc ? Add the -V flag to try and decode the types.
I'm familiar with your tool, but wasn't aware of it's dsc/obj-c capabilities. Giving it at try now. Thanks!
I'm familiar with your tool, but wasn't aware of it's dsc/obj-c capabilities. Giving it at try now. Thanks!
If you have bat installed (brew install bat) you can pipe the output of my tool to: bat -l m --tabs 0 -p --theme Nord --wrap=never --pager "less -S" to make it look amazing.
Yeah that's really cool. This gets us really close. I'll see if I can clean up the output to be a proper header file.
Very nice work on ipsw
Okay, making progress:
I'm dumping JSON from IDA pro, and I've written some python that shells out to classdump's formatType utility, to convert encoded type strings into method signatures.
I've attached a gist with JSON for CommerceKit as well as C-header formatted output from post-processing the JSON. There's definitely stuff missing, like ivars and properties, and probably a few other things. Also I'm not sure what magic class dump does to organize it into individual header files. For now it's all one big .h file.
Someone maybe take a look at let me know if we're getting close
https://gist.github.com/zcutlip/c74ceec2e818c630e655ee4c8f408878
Cheers
Oh, I think I get the organization. It's supposed to be one .h per protocol or class, is that right?
I'm now writing out individual .h files per protocol & class, but I'm now realizing there's a lot more to it. I'm currently dumping about 90-ish header files from CommerceKit alone, because there are lots of class and protocol structures in the binary that don't need to be dumped
I studied what class-dump does, and it's quite sophisticated. It actually builds a dependency graph and then is able to do things like add forward declarations and imports as well as work out what protocols & classes it doesn't need to dump. I don't think I'll be able to get to that level of refinement.
My proposals are:
- Dump out JSON and let others work on a tooling to ingest it (either by modifying class-dump or some other way), or
- Dump header files as best I can, and let others refine those header files and discard what they don't need
Here's an example of one of the header files I dumped, CKAuthenticationSettings.h:
@interface CKAuthenticationSettings : NSObject
{
}
// Class methods
+ (_Bool)supportsSecureCoding
// Instance methods
- (id)copyWithZone:(struct _NSZone *)arg1
- (id)init
- (void)encodeWithCoder:(id)arg1
- (id)initWithCoder:(id)arg1
- (void).cxx_destruct
- (id)_password
- (_Bool)showHelp
- (_Bool)createSession
- (void)setShowHelp:(_Bool)arg1
- (id)suggestedUsername
- (void)setSuggestedUsername:(id)arg1
- (id)authenticateArguments
- (void)setAuthenticateArguments:(id)arg1
- (id)createAccountArguments
- (void)setCreateAccountArguments:(id)arg1
- (_Bool)forceAccount
- (void)setForceAccount:(_Bool)arg1
- (void)setCreateSession:(_Bool)arg1
- (void)set_password:(id)arg1
- (_Bool)_loginToiCloud
- (void)set_loginToiCloud:(_Bool)arg1
@end /* CKAuthenticationSettings */I still need to do ivars and a couple other things, but that shouldn't be too bad.
Cheers
Someone maybe take a look at let me know if we're getting close
I haven't found time to dive into this gist yet, but I'm hoping to do so this week. Hopefully there are enough breadcrumbs here to piece together a plan for the original issue reported here, among others.
It's supposed to be one .h per protocol or class, is that right?
Yep!
My proposals are:
- Dump out JSON and let others work on a tooling to ingest it (either by modifying class-dump or some other way), or
- Dump header files as best I can, and let others refine those header files and discard what they don't need
Option 1 would probably be best in the long run if you were able to publish your IDA scripts as open source. That way, we could try to integrate the functionality directly into class-dump. If you aren't able to do so, let's go for the quicker, stop-gap progress of option 2.
Great work on this, @zcutlip. I really appreciate the time and effort you're putting into this, and I'm sure others here do too.
ObjcExplore has a TODO for class methods for protocols, but I haven't seen any so far in IDA. Are they uncommon?
It looks like several of the interfaces used by mas have class methods used to expose shared singletons.
I took a look at your progress, and I agree we're getting close. Your to-do list looks accurate. I haven't noticed anything else we'd need from a class dumper.
(To be clear, though, Apple still may not have left enough surface area to fix the account, signin, or purchase commands.)
I'll absolutely share the IDA plugin. I'm in the process of doing the following:
- Cleaning up the plugin so it's easier to use, gives you a choice of frameworks/dylibs to dump, etc.
- adding ivars, properties, etc. to the JSON
- rolling up a zip file with all the JSON and all the individual header files I'm able to generate
At that point, based on feedback,I can revise the JSON output to include any additional details as required
Got ivars going. Here's a sample
/*
* CKAccountStore.h
* ./classdump.py ../build/Release/formatType ./CommerceKit/ ../../IDA-ObjCExplorer/CommerceKit.json
*/
@interface CKAccountStore
{
CKStoreClient *_storeClient _storeClient;
}
// Class methods
+ (id)accountStoreForStoreClient:(id)arg1;
+ (id)sharedAccountStore;
// Instance methods
- (void).cxx_destruct;
- (id)accounts;
- (id)primaryStoreAccount;
- (void)addAccount:(id)arg1;
- (id)primaryAccount;
- (id)accountWithAppleID:(id)arg1;
- (_Bool)isDemoModeEnabled;
- (id)initWithStoreClient:(id)arg1;
- (id)demoAccount;
- (id)storeClient;
- (id)storeAccountForDSID:(id)arg1;
- (id)addAccountObserver:(id)arg1;
- (id)_initWithStoreClient:(id)arg1;
- (id)knownAccounts;
- (void)signOutWithCompletionHandler:(id /* CDUnknownBlockType */)arg1;
- (void)removeAccountObserver:(id)arg1;
- (id)storeAccountForAppleID:(id)arg1;
- (void)getEligibilityForService:(long long)arg1 completionBlock:(id /* CDUnknownBlockType */)arg2;
- (_Bool)primaryAccountIsPresentAndSignedIn;
- (void)signOut;
- (id)addPrimaryAccountObserverWithBlock:(id /* CDUnknownBlockType */)arg1;
- (void)removePrimaryAccountObserver:(id)arg1;
- (id)accountForDSID:(id)arg1;
- (void)signIn;
- (void)signInWithSuggestedAppleID:(id)arg1 allowChangeOfAppleID:(_Bool)arg2 completionHandler:(id /* CDUnknownBlockType */)arg3;
- (void)viewAccount;
- (id)eligibilityForService:(long long)arg1;
- (void)getPasswordSettingsWithCompletionBlock:(id /* CDUnknownBlockType */)arg1;
- (void)updatePasswordSettings:(id)arg1 completionBlock:(id /* CDUnknownBlockType */)arg2;
- (void)setTouchIDStateForAccount:(id)arg1 state:(long long)arg2 completionBlock:(id /* CDUnknownBlockType */)arg3;
- (void)getTouchIDStateForAccount:(id)arg1 completionBlock:(id /* CDUnknownBlockType */)arg2;
@end /* CKAccountStore */
I haven't looked at properties yet, but ivars weren't too bad. Hopefully properties will be equally straightforward.
I'm attaching a zip file of all the headers I've spit out so far, along with the JSON I spit out from IDA.
Cheers
EDIT: Note, many/most of the headers in the zip file are superfluous, but as mentioned previously, I'm not sure of a way to programmatically filter out the ones that don't need to be generated. So for now they're all in there, and we can work on a better way next.
I don't see CKDownloadDirectory or CKDownloadQueueObserver in the .zip. Are they visible to your script? What about those types mas uses from StoreFoundation?
I haven't started StoreFoundation yet. I have a separate IDB for it. I wanted to get the overall process working first,
I'll check on CKDownloadDirectory & CKDownloadQueueObserver in my CommerceKit IDB and figure out why they aren't being dumped.
It appears CKDownloadDirectory is just a standard C function exported by CommerceCore.framework (see attached screenshot). I won't be able to get normal exported functions by walking the objective-C structures. I'll have to walk the exports to get those.
As far as CKDownloadQueueObserver, I don't find any reference to it in the IDB. The closest thing there is is DownloadQueueObserver, for which there is a header file in the zip file attached above. Not sure if that's related or not.
CKDownloadQueueObserver is passed as arg1 to CKDownloadQueue.addObserver and removeObserver, and it looks like they're held in downloadQueueObservers. Maybe your work on properties will expose how to find that type. Perhaps it isn't exported because it's a callback/delegate protocol meant to be implemented by the caller.
Forgot to actually attach screenshot previously:
I should add that I'm happy to share my IDB if it's helpful. It's quite large though, around 18GB, so I'm not sure the best way to do that.
@zcutlip, thanks for the offer. I think we could rebuild an IDB if needed. Tooling to generate new headers will be more valuable than a one-time snapshot in the long run.