Add header translation #264

See full history in 1c4c875.

Add initial header translation

Closer to usable

Mostly works with NSCursor

Mostly works with NSAlert.h

Refactor a bit

AppKit is now parse-able

handle reserved keywords

Handle protocols somewhat

Handle the few remaining entity kinds

Works with Foundation

Cleanup

Refactor

Refactor Method to (almost) be PartialEq

Parse more things

Parse NSConsumed

Verify memory management

More work

Fix reserved keywords

Refactor statements

Add initial availability

Prepare RustType

Split RustType up in parse and ToToken part

Add icrate

Add config files

Temporarily disable protocol generation

Generate files

Add initial generated files for Foundation

Skip "index" header

Add basic imports

Allow skipping methods

Properly emit `unsafe`

Make classes public

Rename objc feature flag

Improve imports somewhat

Further import improvements

Handle basic typedefs

Improve generics handling

Improve pointers to objects

Refactor RustType::TypeDef

Mostly support generics

Refactor config setup

Small fixes

Support nested generics

Comment out a bit of debug logging

Emit all files

Parse sized integer types

Parse typedefs that map to other typedefs

Appease clippy

Add `const`-parsing for RustType::Id

Parse Objective-C in/out/inout/bycopy/byref/oneway qualifiers

Fix `id` being emitted when it actually specifies a protocol

Make AppKit work again

Parse all qualifiers, in particular lifetime qualifiers

More consistent ObjCObjectPointer parsing

Validate some lifetime attributes

Fix out parameters (except for NSError)

Assuming we find a good solution to #277

Refactor Stmt objc declaration parsing

Clean up how return types work

Refactor property parsing

Fixes their order to be the same as in the source file

Add support for functions taking NSError as an out parameter

Assuming we do #276

Change icrate directory layout

Refactor slightly

Refactor file handling to allow for multiple frameworks simultaneously

Put method output inside an extern_methods! call

We'll want this no matter what, since it'll allow us to extend things with availability attributes in the future.

Use extern_methods! functionality

To cut down on the amount of code, which should make things easier to review and understand.

This uses features which are not actually yet done, see #244.

Not happy with the formatting either, but not sure how to fix that?

Manually fix the formatting of attribute macros in extern_methods!

Add AppKit bindings

Speed things up by optionally formatting at the end instead

Prepare for parsing more than one SDK

Specify a minimum deployment target

Document SDK situation

Parse headers on iOS as well

Refactor stmt parsing a bit

Remove Stmt::FileImport and Stmt::ItemImport

These are not nearly enough to make imports work well anyhow, so I'll rip it out and find a better solution

Do preprocessing step explicitly as the first thing

Refactor so that file writing is done using plain Display

Allows us to vastly improve the speed, as well as allowing us to make the output much prettier wrt. newlines and such in the future (proc_macro2 / quote output is not really meant to be consumed by human eyes)

Improve whitespace in generated files and add warning header

Don't crash on unions

Add initial enum parsing

Add initial enum expr parsing

Add very simple enum output

Fix duplicate enum check

Improve enum expr parsing

This should make it easier for things to work on 32-bit platforms

Add static variable parsing

Add a bit of WIP code

Add function parsing

Fix generic struct generation

Make &Class as return type static

Trim unnecessary parentheses

Fix generics default parameter

Remove methods that are both instance and class methods

For now, until we can solve this more generally

Skip protocols that are also classes

Improve imports setups

Bump recursion limit

Add MacTypes.h type translation

Fix int64_t type translation

Make statics public

Fix init methods

Make __inner_extern_class allowing trailing comma in generics

Attempt to improve Rust's parsing speed of icrate

Custom NSObject

TMP import

Remove NSProxy

Temporarily remove out parameter setup

Add struct support

Add partial support for "related result types"

Refactor typedef parsing a bit

Output remaining typedefs

Fix Option<Sel> and *mut bool

Fix almost all remaining type errors in Foundation

Skip statics whoose value we cannot find

Fix anonymous enum types

Fix AppKit duplicate methods

Add CoreData

Properly fix imports

Add `abstract` keyword

Put enum and static declarations behind a macro

Add proper IncompleteArray parsing

Refactor type parsing

Make NSError** handling happen in all the places that it does with Swift

Refactor Ty a bit more

Make Display for RustType always sound

Add support for function pointers

Add support for block pointers

Add extern functions

Emit protocol information

We can't parse it yet though, see #250

Make CoreData compile

Make AppKit compile

Add support for the AuthenticationServices framework

Do clang < v13 workarounds without modifying sources

Refactor Foundation fixes

crates/header-translator/Cargo.toml

@@ -0,0 +1,14 @@
+[package]
+name = "header-translator"
+version = "0.1.0"
+edition = "2021"
+publish = false
+
+repository = "https://github.com/madsmtm/objc2"
+license = "Zlib OR Apache-2.0 OR MIT"
+
+[dependencies]
+clang = { version = "2.0", features = ["runtime", "clang_10_0"] }
+toml = "0.5.9"
+serde = { version = "1.0.144", features = ["derive"] }
+apple-sdk = { version = "0.2.0", default-features = false }

crates/header-translator/README.md

@@ -0,0 +1,11 @@
+# Objective-C header translator
+
+For use in making `icrate`.
+
+```console
+cargo run --bin header-translator -- /Applications/Xcode.app/Contents/Developer
+```
+
+## SDKs
+
+We do not redistribute the relevant SDKs, to hopefully avoid a license violation. You can download the SDKs yourself (they're bundled in XCode) from [Apple's website](https://developer.apple.com/download/all/?q=xcode) (requires an Apple ID).

crates/header-translator/framework-includes.h

@@ -0,0 +1,17 @@
+// Workaround for clang < 13, only used in NSBundle.h
+#define NS_FORMAT_ARGUMENT(A)
+
+// Workaround for clang < 13
+#define _Nullable_result _Nullable
+
+#include <TargetConditionals.h>
+
+#import <Foundation/Foundation.h>
+
+#import <CoreData/CoreData.h>
+
+#if TARGET_OS_OSX
+#import <AppKit/AppKit.h>
+#endif
+
+#import <AuthenticationServices/AuthenticationServices.h>

crates/header-translator/src/availability.rs

@@ -0,0 +1,15 @@
+use clang::PlatformAvailability;
+
+#[derive(Debug, Clone)]
+pub struct Availability {
+    #[allow(dead_code)]
+    inner: Vec<PlatformAvailability>,
+}
+
+impl Availability {
+    pub fn parse(availability: Vec<PlatformAvailability>) -> Self {
+        Self {
+            inner: availability,
+        }
+    }
+}

crates/header-translator/src/config.rs

@@ -0,0 +1,108 @@
+use std::collections::HashMap;
+use std::fs;
+use std::io::Result;
+use std::path::Path;
+
+use serde::Deserialize;
+
+#[derive(Deserialize, Debug, Clone, PartialEq, Eq)]
+#[serde(deny_unknown_fields)]
+pub struct Config {
+    #[serde(rename = "class")]
+    #[serde(default)]
+    pub class_data: HashMap<String, ClassData>,
+    #[serde(rename = "protocol")]
+    #[serde(default)]
+    pub protocol_data: HashMap<String, ClassData>,
+    #[serde(rename = "struct")]
+    #[serde(default)]
+    pub struct_data: HashMap<String, StructData>,
+    #[serde(rename = "enum")]
+    #[serde(default)]
+    pub enum_data: HashMap<String, EnumData>,
+    #[serde(rename = "fn")]
+    #[serde(default)]
+    pub fns: HashMap<String, FnData>,
+    #[serde(rename = "static")]
+    #[serde(default)]
+    pub statics: HashMap<String, StaticData>,
+    #[serde(rename = "typedef")]
+    #[serde(default)]
+    pub typedef_data: HashMap<String, TypedefData>,
+    #[serde(default)]
+    pub imports: Vec<String>,
+}
+
+#[derive(Deserialize, Debug, Default, Clone, PartialEq, Eq)]
+#[serde(deny_unknown_fields)]
+pub struct ClassData {
+    #[serde(default)]
+    pub skipped: bool,
+    #[serde(rename = "superclass-name")]
+    #[serde(default)]
+    pub new_superclass_name: Option<String>,
+    #[serde(default)]
+    pub methods: HashMap<String, MethodData>,
+    #[serde(default)]
+    pub properties: HashMap<String, MethodData>,
+}
+
+#[derive(Deserialize, Debug, Default, Clone, PartialEq, Eq)]
+#[serde(deny_unknown_fields)]
+pub struct StructData {
+    #[serde(default)]
+    pub skipped: bool,
+}
+
+#[derive(Deserialize, Debug, Default, Clone, PartialEq, Eq)]
+#[serde(deny_unknown_fields)]
+pub struct EnumData {
+    #[serde(default)]
+    pub skipped: bool,
+    #[serde(rename = "use-value")]
+    #[serde(default)]
+    pub use_value: bool,
+    #[serde(default)]
+    pub constants: HashMap<String, StructData>,
+}
+
+#[derive(Deserialize, Debug, Clone, Copy, PartialEq, Eq)]
+#[serde(deny_unknown_fields)]
+pub struct MethodData {
+    #[serde(rename = "unsafe")]
+    #[serde(default = "unsafe_default")]
+    pub unsafe_: bool,
+    #[serde(default = "skipped_default")]
+    pub skipped: bool,
+    // TODO: mutating
+}
+
+// TODO
+pub type FnData = StructData;
+pub type StaticData = StructData;
+pub type TypedefData = StructData;
+
+fn unsafe_default() -> bool {
+    true
+}
+
+fn skipped_default() -> bool {
+    false
+}
+
+impl Default for MethodData {
+    fn default() -> Self {
+        Self {
+            unsafe_: unsafe_default(),
+            skipped: skipped_default(),
+        }
+    }
+}
+
+impl Config {
+    pub fn from_file(file: &Path) -> Result<Self> {
+        let s = fs::read_to_string(file)?;
+
+        Ok(toml::from_str(&s)?)
+    }
+}

crates/header-translator/src/expr.rs

@@ -0,0 +1,130 @@
+use std::fmt;
+use std::fmt::Write;
+
+use clang::token::TokenKind;
+use clang::{Entity, EntityKind, EntityVisitResult};
+
+use crate::unexposed_macro::UnexposedMacro;
+
+#[derive(Clone, Debug, PartialEq)]
+pub struct Expr {
+    s: String,
+}
+
+impl Expr {
+    pub fn from_val((signed, unsigned): (i64, u64), is_signed: bool) -> Self {
+        let s = if is_signed {
+            format!("{}", signed)
+        } else {
+            format!("{}", unsigned)
+        };
+        Expr { s }
+    }
+
+    pub fn parse_enum_constant(entity: &Entity<'_>) -> Option<Self> {
+        let mut declaration_references = Vec::new();
+
+        entity.visit_children(|entity, _parent| {
+            if let EntityKind::DeclRefExpr = entity.get_kind() {
+                let name = entity.get_name().expect("expr decl ref name");
+                declaration_references.push(name);
+            }
+            EntityVisitResult::Recurse
+        });
+
+        let mut res = None;
+
+        entity.visit_children(|entity, _parent| {
+            match entity.get_kind() {
+                EntityKind::UnexposedAttr => {
+                    if let Some(macro_) = UnexposedMacro::parse(&entity) {
+                        panic!("parsed macro in expr: {macro_:?}, {entity:?}");
+                    }
+                }
+                _ => {
+                    if res.is_none() {
+                        res = Self::parse(&entity, &declaration_references);
+                    } else {
+                        panic!("found multiple expressions where one was expected");
+                    }
+                }
+            }
+            EntityVisitResult::Continue
+        });
+
+        res
+    }
+
+    pub fn parse_var(entity: &Entity<'_>) -> Option<Self> {
+        Self::parse(entity, &[])
+    }
+
+    fn parse(entity: &Entity<'_>, declaration_references: &[String]) -> Option<Self> {
+        let range = entity.get_range().expect("expr range");
+        let tokens = range.tokenize();
+
+        if tokens.is_empty() {
+            // TODO: Find a better way to parse macros
+            return None;
+        }
+
+        let mut s = String::new();
+
+        for token in &tokens {
+            match (token.get_kind(), token.get_spelling()) {
+                (TokenKind::Identifier, ident) => {
+                    if declaration_references.contains(&ident) {
+                        // TODO: Handle these specially when we need to
+                    }
+                    write!(s, "{}", ident).unwrap();
+                }
+                (TokenKind::Literal, lit) => {
+                    let lit = lit
+                        .trim_end_matches("UL")
+                        .trim_end_matches("L")
+                        .trim_end_matches("u")
+                        .trim_end_matches("U");
+                    let lit = lit.replace("0X", "0x");
+                    write!(s, "{}", lit).unwrap();
+                }
+                (TokenKind::Punctuation, punct) => {
+                    match &*punct {
+                        // These have the same semantics in C and Rust
+                        "(" | ")" | "<<" | "-" | "+" | "|" | "&" | "^" => {
+                            write!(s, "{}", punct).unwrap()
+                        }
+                        // Bitwise not
+                        "~" => write!(s, "!").unwrap(),
+                        punct => panic!("unknown expr punctuation {punct}"),
+                    }
+                }
+                (kind, spelling) => panic!("unknown expr token {kind:?}/{spelling}"),
+            }
+        }
+
+        // Trim casts
+        s = s
+            .trim_start_matches("(NSBoxType)")
+            .trim_start_matches("(NSBezelStyle)")
+            .trim_start_matches("(NSEventSubtype)")
+            .trim_start_matches("(NSWindowButton)")
+            .trim_start_matches("(NSExpressionType)")
+            .to_string();
+
+        // Trim unnecessary parentheses
+        if s.starts_with('(')
+            && s.ends_with(')')
+            && s.chars().filter(|&c| c == '(' || c == ')').count() == 2
+        {
+            s = s.trim_start_matches("(").trim_end_matches(")").to_string();
+        }
+
+        Some(Self { s })
+    }
+}
+
+impl fmt::Display for Expr {
+    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
+        write!(f, "{}", self.s)
+    }
+}