Objective-C support

[burtonageo]

With the Objective-C crate, the machinery to send messages is available in stable Rust. From there, it should be possible to parse Objective-C headers, and generate Rust bindings for them.

This would greatly help with integrating Rust into existing projects, especially iOS and Mac apps.

Some ideas about how this could be done:

• Classes should be generated as unit-structs which wrap id. Class methods and properties
  can generate an anonymous impl, which calls to msg_send!. Note that Objective-C has no
  concept of 'const' methods, so every generated method will take &mut self.
• Inheritance can be implemented with auto-deref. If you have 2 classes A and B, where B is
  inherited from A, then we can impl Deref<Target=A> for B { .. } and impl DerefMut<Target=A> for B. Because Objective-C only has single inheritance, there should be no ambiguity about which
  class to deref to. Base classes won't need this.
• Protocols can be cleanly mapped to traits. The way this works is that each trait will have a hidden
  method which gets the object pointer, which will be auto implemented on the struct which wraps
  the object which implements this protocol. The protocol's methods will then use this method to
  provide default implementations which call msg_send! on the pointer returned by this method.
• There can be multiple impl blocks on a type in a crate, so extensions can just open up another
  impl block.

A probably-incomplete example which ties these ideas together:

@protocol Adder <NSObject>
- (int)addNumber:(int)a toOtherNumber:(int)b;
@end

@interface Foo: NSObject
- (instancetype) initWithFirstNumber:(int)firstNumber;
@property(nonatomic, readonly) firstNumber;
@end

@interface Bar: Foo <Adder>
@property(nonatomic, readwrite) int someProperty;
- (instancetype) initWithFirstNumber:(int)firstNumber
                     AndSecondNumber:(int)secondNumber;
- (void) frobnicate;
- (NSString*) getDescriptionFor:(NSObject*)object atIndex:(size_t)index;
- (int) addNumber:(int)a toOtherNumber:(int)b;
@end

Could generate:

#[macro_use]
extern crate objc;

pub trait Adder: NSObjectProtocol {
    //  Gets the `self` pointer which will be used for calling `msg_send!`
    #[doc(hidden)]
    fn self_ptr(&self) -> id;

    // All functions which call objc methods are unsafe to call.
    unsafe fn addNumber_toOtherNumber_(a: libc::c_int, b: libc::c_int) -> libc::c_int {
        msg_send![self, addNumber:a toOtherNumber:b]
    }
}

pub struct Foo(id);

impl Deref for Foo {
    type Target = NSObject;
    fn deref(&self) -> &Self::Target {
        // Both `self` and `NSObject` wrap an `id` pointer, so this should be safe
        unsafe { ::core::mem::transmute(self.0) }
    }
}

impl DerefMut for Foo {
    fn deref_mut(&mut self) -> &mut Self::Target {
        unsafe { ::core::mem::transmute(self.0) }
    }
}

impl Foo {
    // An alloc method is generated automatically.
    pub unsafe fn alloc() -> Self {
        msg_send![class("Foo"), alloc]
    }

    // Objc's `instancetype` resolves to the `Self` type
    pub unsafe fn initWithFirstNumber_(&mut self, firstNumber: libc::c_int) -> Self {
        msg_send![self.0, initWithFirstNumber:firstNumber]
    }

    // Properties resolve to method calls, with the form:
    // * getter = `propertyName`
    // * setter = `setPropertyName:`
    pub unsafe fn firstNumber(&mut self) -> libc::c_int {
        msg_send![self.0, firstNumber]
    }
}

pub struct Bar(id);

impl Deref for Bar {
    type Target = Foo;
    fn deref(&self) -> &Self::Target {
        unsafe { ::core::mem::transmute(self.0) }
    }
}

impl DerefMut for Bar {
    fn deref_mut(&mut self) -> &mut Self::Target {
        unsafe { ::core::mem::transmute(self.0) }
    }
}

// NSObjectProtocol implementation through inheritance.
impl Adder for Bar {
    fn self_ptr(&self) -> id { self.0 }
}

impl Bar {
    pub unsafe fn alloc() -> Self {
        msg_send![class("Bar"), alloc]
    }

    pub unsafe fn initWithFirstNumber_andSecondNumber_(&mut self,
                  firstNumber: libc::c_int, secondNumber: libc::c_int) -> Self {
        msg_send![self.0, initWithFirstNumber:firstNumber andSecondNumber:secondNumber]
    }

    pub unsafe fn someProperty(&mut self) -> libc::c_int {
        msg_send![self.0, someProperty]
    }

    pub unsafe fn setSomeProperty_(&mut self, someProperty: libc::c_int) {
        msg_send![self.0, setSomeProperty:someProperty]
    }

    pub unsafe fn frobnicate(&mut self) {
        // ...
    }

    pub unsafe fn getDescriptionFor_atIndex_(&mut object: NSObject, atIndex: libc::size_t)
                                             -> NSString {
        // ...
    }

    // The `- (int) addNumber:(int)a toOtherNumber:(int)b;` declaration in `Bar` is ignored because
    // it is a protocol method which is already implemented.
}

Note that these bindings are not the most ergonomic to use: each method is unsafe, each variable has to be declared mut to actually call any methods on it, and null pointer checking has to be performed manually (it may be worth adding an is_null method to NSObject, or to a new trait which is automatically implemented by objc base objects). However, they take a lot of the tedium out of writing bindings manually. Furthermore, it could be possible to refine the ergonomics further:

• Methods which return objects could return Option<_>, with the None case matching to a nil.
  If we can parse the the new nullability attributes,
  we could probably just return the object if it was declared _Nonnull in the header.
• We could search the name of the class to see if the substring Mutable was present to decide
  whether methods should pass &self or &mut self, or perhaps it might be possible to add a
  custom attribute, e.g. __attribute__((rust_objc_mutating)) to method declarations. Of course,
  Apple might also decide to add their own mutability attributes to Clang later.
• We could decide that the bindings we generate are safe enough that we could enclose the
  msg_send! in an unsafe block and just generate safe fns.
• There is no mention of generics in this report, but they could be added as a PhantomData<T> in
  the struct. To maintain optional dynamism, this would require a new trait (e.g. AnyObject) which
  all objc objects implemented, and the T generic would require. With this in place, NSArray could
  be declared as struct NSArray<T: AnyObject = NSObject>(id, PhantomData<T>).
• Various Foundation protocols map fairly cleanly on to traits from std. There is a clear
  relation NSCopying and Clone, and every NSObject-derived class can implement
  std::hash::Hash courtesy of the hash method, and each NSObjecttype can be PartialEq
  with the isEqual: method.

[burtonageo]

I would be happy to be the primary driver behind this feature, and any help/comments on the API of
the bindings which would be generated, or any mentorship when implementing this feature would be
very appreciated.

[emilio]

Hi, thanks for opening this!

I agree this would be nice to have, so if you want to implement this, please do! I'll try to help answer your questions as much as I can :)

To be fair, I wouldn't worry too much about the code generation, if we get the parsing right it'd be straight-forward to do.

Seems like a good way to start would be adapting our IR to support the equivalent to: CXType_ObjCId, CXType_ObjCClass, CXType_ObjCSel, CXType_ObjCInterface and ``CXType_ObjCObjectPointer`, then also handle the different cursor kinds too.

FWIW, this is the ast generated by your example:

(ObjCClassRef Protocol ObjCInterface
)
(ObjCProtocolDecl Adder Invalid
    (ObjCInstanceMethodDecl addNumber:toOtherNumber: Invalid
        (ParmDecl a Int
        )
        (ParmDecl b Int
        )
    )
)
(ObjCInterfaceDecl Foo ObjCInterface
    (ObjCInstanceMethodDecl initWithFirstNumber: Invalid
        (TypeRef instancetype Typedef
        )
        (ParmDecl firstNumber Int
        )
    )
    (ObjCPropertyDecl firstNumber Int
    )
    (ObjCInstanceMethodDecl firstNumber Invalid
    )
)
(ObjCInterfaceDecl Bar ObjCInterface
    (FirstRef Foo ObjCInterface
    )
    (ObjCClassRef Foo ObjCInterface
    )
    (ObjCProtocolRef Adder Invalid
    )
    (ObjCPropertyDecl someProperty Int
    )
    (ObjCInstanceMethodDecl initWithFirstNumber:AndSecondNumber: Invalid
        (TypeRef instancetype Typedef
        )
        (ParmDecl firstNumber Int
        )
        (ParmDecl secondNumber Int
        )
    )
    (ObjCInstanceMethodDecl frobnicate Invalid
    )
    (ObjCInstanceMethodDecl getDescriptionFor:atIndex: Invalid
        (ParmDecl object ObjCId
        )
        (ParmDecl index ObjCId
        )
    )
    (ObjCInstanceMethodDecl addNumber:toOtherNumber: Invalid
        (ParmDecl a Int
        )
        (ParmDecl b Int
        )
    )
    (ObjCInstanceMethodDecl someProperty Invalid
    )
    (ObjCInstanceMethodDecl setSomeProperty: Invalid
        (ParmDecl someProperty Int
        )
    )
)

And there are a bunch of objective C related methods in the cursor docs.

Let me know if you need any help. Thanks for this again! :)

[metajack]

cc @nox who has been working on related things.

[burtonageo]

Okay, I've had a look at the structure of the code. It looks like (from a 1000 metre view) that the main entry point creates a BindgenContext, which parses the initial file and turns it into a TranslationUnit. Items are then parsed from the header and added to the BindgenContext, which then writes out the bindings.

Therefore, extending the parser to support objc should mostly require expanding out the Type variant of Item.

One issue I've come across so far is that bindgen won't parse objective-C declarations without passing in -x objective-c to the clang args. I've found an old blog post, which seems to suggest that it's a bug in the clang parser, but I'm not sure at the moment.

[emilio]

Yep, that sounds accurate, you probably need to add more TypeKinds, and potentially more ItemKinds.

IIRC with clang 3.9 it parses the file automatically as objc if it ends with .m, but you're right by default it parses .h files as C. We support passing flags to libclang though, so this should not be a problem.

[nox]

We started getting some Objective-C support, didn't we?

[emilio]

Right, it's in progress though.

[simlay]

So over the last year I've been adding more and more of this features and it's getting to a good place I think.

Anyway, I'd like to summarize what the features are really getting to:

• An initial implementation of objective-c generics for #1259 #1702 added objective-c template generation that builds traits for those templates. It's still a bit unclear how to use those appropriately.
• Higher level objective-c support. #1722 added struct wrapped ids that represent the objective-c class instances and a trait which has the getters, setters, and methods for that class. Take for example an objective-c class Foo, you end up with struct Foo(id) and trait IFoo (I for "interface") then impl IFoo for Foo {}.
• Objective-c inheritance support #1750 took Higher level objective-c support. #1722 and added the basic inheritance structures. Now if class Foo inherits Bar now, the rust bindings will generate impl IFoo for Foo {} and impl IBar for Foo {}
• Initial objective-c docs #1751 updated the docs!
• Added Bindgen names to objective-c pointer return types for #1835. #1847 added using the bindgen names and leveraged `repr(transparent) in the function input and return types. I think this is a pretty spectacular addition.
• Added Objc category inheritance #1784 isn't merged yet but it completes the inheritance generation for objective-c categories for the structs. I'll admit I'm not a huge fan of the names for the categories but I don't know what I'd make it.

I spent a bunch of time the other day trying to take advantage of the nullability attributes in clang and hopefully return Option<_>, with the None case matching to a nil but I couldn't figure out how to get the dumb attribute out of clang.

The other thing I spent a bunch of time exploring was how to determine "safety" for functions but I just don't think there's a way to make it "safe" with any assurance. One way to do this, is to add a "safe list" concept that's specified by the user. I just think that if you'd make it too easy for a user to add safe stuff it could get ignored. Thoughts?