support ARM mac #303

pebble8888 · 2020-06-25T00:29:04Z

XVim2 has Intel assemby code.

sobri909 · 2020-06-25T14:31:08Z

I'm about 30 years out of date on ARM assembly 😂 The last time I worked with it was on Acorn. But I'd be happy to contribute towards an Apple Transition Developer Kit!

rudedogg · 2020-06-26T03:44:05Z

Is this the assembly?

https://github.com/XVimProject/XVim2/blob/5b1033b4ee9db5c4821f31d9f8f2688247e47db6/XVim2/Xcode/SourceEditorViewWrapper.s

pebble8888 · 2020-06-26T06:20:47Z

@rudedogg Yes.

jspavlick · 2020-11-18T05:55:16Z

Is this still a problem? A friend of mine with one of the developer preview Mac mini's was able to run make and it compiled successfully. However he didn't code sign Xcode so he can't confirm fully if XVim2 runs or not.

jrsharp · 2020-11-18T17:22:44Z

Yes, this still appears to be an issue. I was able to build XVim2 successfully for Xcode 12.2, but Xcode does not seem to like installing/running Xcode plugins compiled for x86-64 only (using Rosetta2), as XVim2 currently excludes arm64. Initial attempt to build for arm64 failed -- will have to dig into it further.

(This was done on a MacBook Pro 13 w/ Apple M1 on Big Sur 11.0.1)

jspavlick · 2020-11-18T18:01:22Z

Is there anyway I can help (maybe by trying to cross-compile)? I have an Intel based MBP running 10.15.7 Catalina.

jrsharp · 2020-11-18T20:43:16Z

Well, the issue seems to be well understood, as identified above -- replacing the x86 asm in SourceEditorViewWrapper.s. In the short term, Xcode can be forced to be opened using Rosetta2, and I can confirm XVim2 functions when using Rosetta2.

jspavlick · 2020-11-18T20:51:39Z

Yeah, I saw the x86 assembly. But I was wondering maybe there is/are other problems in addition to that.

That's good news in terms of a (temporary) workaround. That's enough for me to order an M1 machine.

rookiezn · 2020-12-11T06:47:42Z

Yeah, I saw the x86 assembly. But I was wondering maybe there is/are other problems in addition to that.

That's good news in terms of a (temporary) workaround. That's enough for me to order an M1 machine.

@jspavlick What's the workaround to use XVim2 on ARM mac? Thanks.

jspavlick · 2020-12-11T06:50:05Z

@rookiezn I don't know. I don't have my M1 mac yet. As jrsharp eluded to, if you force Xcode to run using Rosetta2 (it's a universal binary, right? So it will run the x86 version through emulation) then it will work, but presumably everything (compiling etc) will be slower?

webcpu · 2020-12-12T08:03:45Z

There is a workaround. You can run Xcode using rosetta. Xcode is an universal application.

Close Xcode
Go to Applications and right click Xcode.
Click "Get info" menu item in Context menu.
Tick "Open using rosetta"
Run Xcode
Please notes that please make sure you have installed Xvim2.

It's unfortunately that the performance of Xcode is worse using Rosetta, especially building a big project.

r-plus · 2020-12-13T08:29:00Z

hmm, its assemby code is for call swift only interface (have not @objc attribute) instance method that implemented in xcode from xvim(objc), right?

webcpu · 2020-12-13T09:25:49Z

It's possible to convert Intel assembly to Objective-C/C using decompiler. https://www.hopperapp.com/

r-plus · 2020-12-15T14:21:02Z

I found one more related issue dependent intel arch.

XVim2 project imported rd_route project https://github.com/rodionovd/rd_route for getting swift function (property/method) pointer that not supporting arm arch.
We must fix it or change the dependent source/project for it.

NOTE: yes, using function function_ptr_from_name from it is for invoke swift method.
It is same purpose of SourceEditorViewWrapper.s assemble code.
If we found any other solution for it, we can remove both of code.

Refs:
https://github.com/XVimProject/XVim2/blob/master/XVim2/Helper/rd_route.c

iT-Boyer · 2020-12-17T14:41:44Z

There is a workaround. You can run Xcode using rosetta. Xcode is an universal application.

Close Xcode

Go to Applications and right click Xcode.

Click "Get info" menu item in Context menu.

Tick "Open using rosetta"

Run Xcode
Please notes that please make sure you have installed Xvim2.

It's unfortunately that the performance of Xcode is worse using Rosetta, especially building a big project.

I do this five steps and run XCode ，see Edit->Xvim2 -> about：

-------- Debug Info -------
XVim2 revision : 540f9b8cdbbc2a440cfd91f551981e801fc08f17
OS Version : Version 11.1 (Build 20C69)
Xcode Version : 12.3

--- .xvimrc ---
N/A
--------------

but , xvim2 not work in code editer .

webcpu · 2020-12-17T18:36:07Z

There is a workaround. You can run Xcode using rosetta. Xcode is an universal application.

Close Xcode

Go to Applications and right click Xcode.

Click "Get info" menu item in Context menu.

Tick "Open using rosetta"

Run Xcode
Please notes that please make sure you have installed Xvim2.

It's unfortunately that the performance of Xcode is worse using Rosetta, especially building a big project.

I do this five steps and run XCode ，see Edit->Xvim2 -> about：
-------- Debug Info -------
XVim2 revision : 540f9b8cdbbc2a440cfd91f551981e801fc08f17
OS Version : Version 11.1 (Build 20C69)
Xcode Version : 12.3

--- .xvimrc ---
N/A
--------------
but , xvim2 not work in code editer .

I have tested it on Xcode 12.2, it works. Because Xcode 12.3 isn't usable on my Mac, I haven't tested it on Xcode 12.3.

r-plus · 2020-12-18T00:16:14Z

Do you add Xcode 12.3 UUID?

iT-Boyer · 2020-12-18T12:12:09Z

YES，i set UUID by this command：

find ~/Library/Application\ Support/Developer/Shared/Xcode/Plug-ins -name Info.plist -maxdepth 3 | xargs -I{} defaults write {} DVTPlugInCompatibilityUUIDs -array-add `defaults read /Applications/Xcode.app/Contents/Info DVTPlugInCompatibilityUUID`

Now , when the mouse is focused on the editor, Xcode cannot respond for a long time.

smolck · 2020-12-29T18:09:49Z

Hello! So, my understanding from reading through this issue is that in order for XVim to work natively on M1, this assembly code needs to be removed/changed to work on ARM somehow, and rd_route also needs to be swapped out for something else or removed. Is that correct?

If so, I have a few questions; for starters, what does that x86 assembly do exactly/why is it used? Could it be replaced by Objective-C or C code?

Also, a quick glance over rd_route's README seems to suggest that it isn't very safe to use; I presume it can be used safely, but regardless I guess it could/should be replaced for something else? Perhaps a refactor could render it unnecessary?

I'm happy to help with this where I can, even if that just means testing out a PR or PRs on my machine to verify it works on M1.

Also, I can confirm that running Xcode under Rosetta does allow XVim to work, but I suspect running Xcode under Rosetta is something many M1 users (myself included) would rather avoid, considering the advantages to running Xcode natively on the M1.

jspavlick · 2020-12-29T20:51:00Z

@smolck I believe your assessment is correct.

I do not know what the x86 assembly is doing. I am by no means an assembly expert. However, @r-plus is seeming to indicate that the assembly code is doing the same thing as rd_route, or somehow calling it? That doesn't seem to be the case to me, but I am no expert.

XVim2 is only using function_ptr_from_name from rd_route. If you read around in rd_route.c you can kind of get an idea of what it's doing. If you read around in SourceEditorViewWrapper.swift and SourceEditorDataSourceWrapper.swiift you can kind of get an idea on how it's being used.

I predict that converting the x86 assembly to ARM will be easier than porting the functionality of function_ptr_to_name but I don't know. (Unless r-plus is correct [as is my understanding of his earlier comment] and they're doing the same thing?)

My M1 Mac is on the way, and when it gets here I'll be happy to play with function_ptr_to_name as well as help in any other way I can.

In the meanwhile, I have posted a $500 bug bounty on this for anyone who is more competent than I am.

smolck · 2020-12-29T22:06:22Z

XVim2 is only using function_ptr_from_name from rd_route. If you read around in rd_route.c you can kind of get an idea of what it's doing. If you read around in SourceEditorViewWrapper.swift and SourceEditorDataSourceWrapper.swiift you can kind of get an idea on how it's being used.

Just looking at that first file, this is where function_ptr_from_name is used:

XVim2/XVim2/Xcode/SourceEditorViewWrapper.swift

Lines 102 to 106 in b723d4e

    
           private let fpSetCursorStyle = function_ptr_from_name("_$s12SourceEditor0aB4ViewC11cursorStyleAA0ab6CursorE0Ovs", nil) 
        
           private let fpGetCursorStyle = function_ptr_from_name("_$s12SourceEditor0aB4ViewC11cursorStyleAA0ab6CursorE0Ovg", nil) 
        
           private let fpGetDataSource = function_ptr_from_name("_$s12SourceEditor0aB4ViewC04dataA0AA0ab4DataA0Cvg", nil) 
        
           private let fpSetSelectedRangeWithModifiers = function_ptr_from_name("_$s12SourceEditor0aB4ViewC16setSelectedRange_9modifiersySnyAA0aB8PositionVG_AA0aB18SelectionModifiersVtF", nil) 
        
           private let fpAddSelectedRangeWithModifiers = function_ptr_from_name("_$s12SourceEditor0aB4ViewC16addSelectedRange_9modifiers15scrollPlacement12alwaysScrollySnyAA0aB8PositionVG_AA0aB18SelectionModifiersVAA0kI0OSgSbtF", nil)

Those seem to just be pointers to Objective-C functions, right? For example, fpSetCursorStyle appears to reference this Objective-C function (or a similar one somewhere else in the code). In that case couldn't either (a) some sort of interop be done between Swift & Objective-C to remove the need for function_ptr_from_name (asking because I'm not super familiar with Swift or Objective-C), or (b) SourceEditorViewWrapper.swift just be rewritten in Objective-C so no interop is necessary?

As for (b), I imagine that might be the easier way of doing this, unless there's a serious reason why SourceEditorViewWrapper was written in Swift instead of Objective-C, or unless that class/file has to be written in Swift for some reason.

Again, though, I'm not familiar with this codebase (and hardly know anything about Swift/Objective-C), so I might be wrong here. (And even if I'm not and those are valid solutions, I don't think I currently have the understanding/knowledge necessary to implement the fix.)

r-plus · 2020-12-30T04:18:38Z

Sorry for confusion.

Both parts (assembly code and rd_route) are necessary for invoke Swift method implemented in Xcode.app by Apple.

getting Swift function (method) pointer by function_ptr_from_name
Then call that pointer with argument by XVim assembly code

NOTE: Usually, pure Swift function (not have objc interface and not public level access control) have not dynamic callable system out side of original module (in this case, Xcode).

jspavlick · 2020-12-30T06:34:39Z

Yeah, after studying the code a little more, I think @r-plus is completely correct.

Let me explain what I think is going on @smolck. SourceEditorViewWrapper wraps a SourceEditorView which is from some Apple internal framework. It is a Swift subclass of NSView. It's mangled name is _TtC15IDESourceEditor19IDESourceEditorView. It has methods on it like SourceEditor.SourceEditorView.setSelectedRange(_: SourceEditor.SourceEditorRange, modifiers: SourceEditor.SourceEditorSelectionModifiers) -> ()

SourceEditorViewWrapper wants to invoke these methods. When everything was Objective-C, you could just do something like [object performSelector:NSSelectorFromString(@"_someCrazyPrivateMethod")]; but in Swift land (unless the method is annotated with @objc) you cannot do this. (Note: even if SourceEditorViewWrapper was written in Objective-C, it still has this problem)

The workaround (which seems to be quite brilliant) is to get a function pointer that points to the function/method in question (I guess it's sitting in a v-table somewhere?) and then "manually" invoke the function as a method, by remembering to pass self as an argument. This manual invocation is what the x86 assembly code does I THINK.

jspavlick · 2020-12-30T06:43:56Z

@smolck could you please save this file as main.c and then run gcc main.c && ./a.out on your M1 mac and tell us what happens? https://pastebin.com/vKc8RG2K I stripped everything out of rd_route.c except function_ptr_from_name let's see if it works without any modification. It works on my Intel machine, for whatever little it's worth. 😅

qiudaomao · 2020-12-30T06:45:55Z

@jspavlick
You are right, the magic SourceEditorViewWrapper.s does is to implement the swift function call ABI.

Save registers
Place all arguments in the right registers or stack
call the function by symbol address
restore the saved registers
return

Need rewrite the whole procedure by ARM, de-compile some swift arm binaries should helping known how swift call a symbol in ARM.

jspavlick · 2020-12-30T06:57:53Z

@jspavlick

You are right, the magic SourceEditorViewWrapper.s does is to implement the swift function call ABI.

Save registers

Place all arguments in the right registers or stack

call the function by symbol address

restore the saved registers

return

Need rewrite the whole procedure by ARM, de-compile some swift arm binaries should helping known how swift call a symbol in ARM.

And I think that's totally doable. But what do you think the chances are we get function_ptr_from_name also working on ARM?

r-plus · 2020-12-30T08:27:08Z

@jspavlick yes, completely correct.

I’m thinking fishhook https://github.com/facebook/fishhook implementation is maybe helpful for getting function pointer on ARM. (looks like not have function like rd_route, so need work to fit for XVim)

smolck · 2021-01-12T01:11:19Z

but we'll have to find the equivalent knowledge for arm64.

That's the problem I'm having right now; for some reason there appears to be very little documentation on x64 assembly vs arm assembly, and it's quite frustrating.

However, I did find this write up which seems pretty useful: https://developer.apple.com/documentation/xcode/writing_armv6_code_for_ios

At the bottom of that there's a section regarding the prolog/epilog of a function, and some code as an example. My understanding from looking over the code there/reading a bit (I think I'll properly read through the whole thing later) is that the equivalent to leaq -240(%rsp), %rsp would be sub sp, sp, #240, but I'm not sure if that's correct. But perhaps specifically this code from that document might be useful:

push   {r4-r7, lr}     // Save Lr, R7, R4-R6.
mov    r6, r11         // Move high registers to low registers, so
mov    r5, r10         //  they can be saved. (Skip this part if
mov    r4, r8          //  the routine doesn’t use R8, R10, or R11.)
push   {r4-r6)         // Save R8, R10, R11 (now in R4-R6).
add    r7, sp, #24     // Adjust R7 to point to saved R7.
sub    sp, #36         // Allocate space for local storage.

If nothing else, it seems like a useful example of arm assembly.

r-plus · 2021-01-13T16:21:23Z

I've been learning x64 assembly for the last few days before learning arm64 assembly.
Now I totally understood current assembly code what doing.

made this pull request for adding many comment and refactoring. #352

r-plus · 2021-01-14T15:53:01Z

Hey guys!
I'm glad to announce start point to support arm64 architecture!

This is the branch to building universal binary. (Thank you @JugglerShu for commit permission)
It works basically operation in my test.
https://github.com/XVimProject/XVim2/tree/feature/support_arm

Let me know to this issue if you have the problem in arm, but not problem in intel.

known issues for arm

~~Xcode will crash when type Ctrl+v to enter Visual Block mode then type j key. I'm not sure how to resolve this.~~ This crash is only happen on Debug build of XVim. Crash free on Release build :)

jspavlick · 2021-01-14T19:21:30Z

OMG!

r-plus · 2021-01-15T12:33:49Z

ah, first known issue I wrote is only for Debug build.
uum, that's curious... but okey, mark as resolved.

r-plus · 2021-01-15T13:23:07Z

Wow, I found test case for XVim inside edit menu.

This is result on ARM.

and intel (Rosetta) result

Same result.
Hmm, is it release able quality...?

jspavlick · 2021-01-15T13:31:51Z

Are these issues present on Intel? I can't seem to find how to run these tests in Edit.

r-plus · 2021-01-15T13:34:49Z

not yet checked about it.

tests are here.
Edit menu (in menu bar) -> XVim -> Test categories -> All

jspavlick · 2021-01-15T13:54:20Z

Am I crazy? I'm on master b723d4e

r-plus · 2021-01-15T14:25:24Z

Oh, its menu will show only in debug mode. debug mode will add file logging AFAIK.
please did this.

echo "set debug" >> ~/.xvimrc

https://github.com/XVimProject/XVim2/blob/master/Documents/Contributing.md#debug-with-log

jspavlick · 2021-01-15T15:11:54Z

Got it!

Oh wow, Intel performs terribly! Can someone else with an Intel machine confirm this? This seems insane?

I ran it a second time and got this:

r-plus · 2021-01-15T16:04:56Z

In my intel mac (MBP Late 2016), same three failure result I previously pasted.

jspavlick · 2021-01-15T17:06:39Z

Why does mine fail so badly? Maybe we can ignore it?

smolck · 2021-01-15T17:21:35Z

@jspavlick Is there anything other than set debug in you .xvimrc?

For me, I ran the tests on the latest from the feature/support_arm branch, and this was the result:

Running it under Rosetta appeared to give the same thing.

Just from a quick test moving around the editor, it works well! Great work @r-plus!

r-plus · 2021-01-16T04:37:26Z

Test result will be changed by Xcode preference.

dw failed two cases I pasted are passed when I disable the Xcode preference Including white-space only lines (Xcode default is off)

After disabled it, only one failing case, same as @smolck's.

r-plus · 2021-01-16T04:52:50Z

looks like last failing test case O<ESC> is known broken test case commented as FIXME: expected text
https://github.com/XVimProject/XVim2/blob/master/XVim2/UnitTest/XVimTester%2BIssues.m#L87

initial text is "" empty string, then type O<ESC>, expected <CR>.
Where came from aaa bbb ccc string...?

Actually, Shift+o<ESC> is correctly works well :)

r-plus · 2021-01-16T05:31:30Z

fixed blank text initializing for unit test and test case expected result for fitting Xcode 12. a8a3e88

finally, all passed.

jspavlick · 2021-01-16T05:53:22Z

@jspavlick Is there anything other than set debug in you .xvimrc?

For me, I ran the tests on the latest from the feature/support_arm branch, and this was the result:

Running it under Rosetta appeared to give the same thing.

Just from a quick test moving around the editor, it works well! Great work @r-plus!

after emptying my .xvimrc and switching to feature/support_arm a8a3e88 I still get this. Maybe we should just ignore my sad machine and test results 😅

r-plus · 2021-01-16T07:22:53Z

@jspavlick This unit test expected Xcode default preference.
Looking your results, Expected and Actual differences are space width.
Are you changing indent preference?

jspavlick · 2021-01-16T08:00:21Z

Good thinking, @r-plus ! After changing my indentation from 2 to 4 in Xcode, everything passes!

jspavlick · 2021-01-18T05:01:58Z

I got my M1 Mac Mini (finally) and have set it up. Wow, this thing is amazing. Product > Archive is 3.7x faster than my Mid 2014 MBP, and I simply cannot get this thing to make fan noise. It's dead silent.

I got XVim2 installed and working. All 463 tests pass. And Xvim seems to work perfectly from what I can tell. Very good job @r-plus !

I did get one crash while running the tests, but I can't replicate it. Probably not a big deal. I am also experiencing this issue: #340

jhirsh · 2021-01-22T16:27:52Z

feature/support-arm also passes on my M1 MBA.

@r-plus, thank you for tackling this! 🦾

jspavlick · 2021-01-22T20:41:49Z

@r-plus when are we merging this into master? Go claim your bounty!

r-plus · 2021-01-23T02:54:28Z

@jspavlick I'm wondering if it's okay to self-merging pull request. (I’m new face of committer for this repo)

pebble8888 · 2021-01-23T03:06:31Z

@r-plus
I think you can merge your PR and close this issue.

r-plus · 2021-01-23T06:41:18Z

okey, thanks! I'll merge it.

r-plus · 2021-01-23T07:05:58Z

I just merge it to develop and master branches.
Close this issue, thanks to all supported great vimmers.

pebble8888 added the help wanted label Jun 25, 2020

pebble8888 mentioned this issue Aug 3, 2020

Build failed with Xcode 12 beta 3 #316

Closed

pebble8888 mentioned this issue Nov 26, 2020

no work for xcode12.2 on Big Sur 11.0.1 and M1 Mac mini #342

Closed

r-plus mentioned this issue Jan 16, 2021

Add support arm64 architecture. #354

Merged

pebble8888 added the Bounty label Jan 23, 2021

r-plus closed this as completed Jan 23, 2021

support ARM mac #303

support ARM mac #303

Comments

pebble8888 commented Jun 25, 2020

sobri909 commented Jun 25, 2020

rudedogg commented Jun 26, 2020

pebble8888 commented Jun 26, 2020

jspavlick commented Nov 18, 2020

jrsharp commented Nov 18, 2020 • edited Loading

jspavlick commented Nov 18, 2020

jrsharp commented Nov 18, 2020

jspavlick commented Nov 18, 2020

rookiezn commented Dec 11, 2020

jspavlick commented Dec 11, 2020 • edited Loading

webcpu commented Dec 12, 2020 • edited Loading

r-plus commented Dec 13, 2020 • edited Loading

webcpu commented Dec 13, 2020

r-plus commented Dec 15, 2020 • edited Loading

iT-Boyer commented Dec 17, 2020

webcpu commented Dec 17, 2020

r-plus commented Dec 18, 2020

iT-Boyer commented Dec 18, 2020

smolck commented Dec 29, 2020

jspavlick commented Dec 29, 2020 • edited Loading

smolck commented Dec 29, 2020

r-plus commented Dec 30, 2020 • edited Loading

jspavlick commented Dec 30, 2020 • edited Loading

jspavlick commented Dec 30, 2020

qiudaomao commented Dec 30, 2020

jspavlick commented Dec 30, 2020

r-plus commented Dec 30, 2020

smolck commented Jan 12, 2021

r-plus commented Jan 13, 2021 • edited Loading

r-plus commented Jan 14, 2021 • edited Loading

known issues for arm

jspavlick commented Jan 14, 2021

r-plus commented Jan 15, 2021

r-plus commented Jan 15, 2021

jspavlick commented Jan 15, 2021

r-plus commented Jan 15, 2021

jspavlick commented Jan 15, 2021

r-plus commented Jan 15, 2021 • edited Loading

jspavlick commented Jan 15, 2021

r-plus commented Jan 15, 2021

jspavlick commented Jan 15, 2021

smolck commented Jan 15, 2021 • edited Loading

r-plus commented Jan 16, 2021 • edited Loading

r-plus commented Jan 16, 2021

r-plus commented Jan 16, 2021 • edited Loading

jspavlick commented Jan 16, 2021

r-plus commented Jan 16, 2021

jspavlick commented Jan 16, 2021

jspavlick commented Jan 18, 2021

jhirsh commented Jan 22, 2021

jspavlick commented Jan 22, 2021

r-plus commented Jan 23, 2021

pebble8888 commented Jan 23, 2021

r-plus commented Jan 23, 2021

r-plus commented Jan 23, 2021

jrsharp commented Nov 18, 2020 •

edited

Loading

jspavlick commented Dec 11, 2020 •

edited

Loading

webcpu commented Dec 12, 2020 •

edited

Loading

r-plus commented Dec 13, 2020 •

edited

Loading

r-plus commented Dec 15, 2020 •

edited

Loading

jspavlick commented Dec 29, 2020 •

edited

Loading

r-plus commented Dec 30, 2020 •

edited

Loading

jspavlick commented Dec 30, 2020 •

edited

Loading

r-plus commented Jan 13, 2021 •

edited

Loading

r-plus commented Jan 14, 2021 •

edited

Loading

r-plus commented Jan 15, 2021 •

edited

Loading

smolck commented Jan 15, 2021 •

edited

Loading

r-plus commented Jan 16, 2021 •

edited

Loading

r-plus commented Jan 16, 2021 •

edited

Loading