Update Minio for local dev & e2e

[wiardvanrij]

Since #3967 we are using a fork. Which was absolutely necessary, however it will now sort of shoot us in the foot for future features.

For example: #5101

but also I'm running on a M1 laptop and local dev is not working due to:

# github.com/shirou/gopsutil/cpu
../../../go/pkg/mod/github.com/shirou/[email protected]+incompatible/cpu/cpu_darwin_cgo.go:13:5: warning: 'TARGET_OS_MAC' is not defined, evaluates to 0 [-Wundef-prefix=TARGET_OS_]
# github.com/shirou/gopsutil/disk
disk_darwin.c:28:2: warning: 'IOMasterPort' is deprecated: first deprecated in macOS 12.0 [-Wdeprecated-declarations]
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/IOKit.framework/Headers/IOKitLib.h:132:1: note: 'IOMasterPort' has been explicitly marked deprecated here
# github.com/shirou/gopsutil/process
../../../go/pkg/mod/github.com/shirou/[email protected]+incompatible/process/process_darwin.go:576:34: undefined: KinfoProc
../../../go/pkg/mod/github.com/shirou/[email protected]+incompatible/process/process_darwin.go:577:8: undefined: KinfoProc
../../../go/pkg/mod/github.com/shirou/[email protected]+incompatible/process/process_darwin.go:590:32: undefined: KinfoProc
../../../go/pkg/mod/github.com/shirou/[email protected]+incompatible/process/process_darwin.go:594:62: undefined: KinfoProc

Main problem that needs to be resolved is the race that caused the memory increase upstream.

Also, two things here that we need to split. One is that we use minio in the quickstart and e2e, but also as client. I think we could update the one via bingo for e2e and solve local dev issues on M1 machines?

[stale]

Hello 👋 Looks like there was no activity on this issue for the last two months.
Do you mind updating us on the status? Is this still reproducible or needed? If yes, just comment on this PR or push a commit. Thanks! 🤗
If there will be no activity in the next two weeks, this issue will be closed (we can always reopen an issue if we need!). Alternatively, use remind command if you wish to be reminded at some point in future.

[stale]

Closing for now as promised, let us know if you need this to be reopened! 🤗

[wiardvanrij]

Still relevant

[B0go]

Hi! I am trying to build a Thanos development environment on my M1 MacBook Pro so I can work on task #4961, and I am also seeing this error locally:

(re)installing /Users/bogo/Developer/Go/bin/minio-v0.0.0-20200527010300-cccf2de129da
# github.com/shirou/gopsutil/disk
disk_darwin.c:28:2: warning: 'IOMasterPort' is deprecated: first deprecated in macOS 12.0 [-Wdeprecated-declarations]
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/IOKit.framework/Headers/IOKitLib.h:132:1: note: 'IOMasterPort' has been explicitly marked deprecated here
# github.com/shirou/gopsutil/process
../../Go/pkg/mod/github.com/shirou/[email protected]+incompatible/process/process_darwin.go:576:34: undefined: KinfoProc
../../Go/pkg/mod/github.com/shirou/[email protected]+incompatible/process/process_darwin.go:577:8: undefined: KinfoProc
../../Go/pkg/mod/github.com/shirou/[email protected]+incompatible/process/process_darwin.go:590:32: undefined: KinfoProc
../../Go/pkg/mod/github.com/shirou/[email protected]+incompatible/process/process_darwin.go:594:62: undefined: KinfoProc
make: *** [/Users/bogo/Developer/Go/bin/minio-v0.0.0-20200527010300-cccf2de129da] Error 2
bogo@Bogos-MacBook-Pro ~/D/thanos (main) [2]>

Is there any known alternative for local development? I am thinking about using a Docker image or maybe a VM. I am also available to help solving this but I am not sure how complex the fix can be

[wiardvanrij]

Yeah I believe that you can upgrade/change the bingo one to unblock yourself. I haven't had the time to check this yet (:

[B0go]

Hi @wiardvanrij, I will try this approach!

I need to understand how bingo works and figure out if upgrading Minio's version already solves this incompatibility with Apple M1, and once I get there, I will open a PR :)

[B0go]

I did some tests with different versions of github.com/minio/minio and found out that minio/minio@91130e8 is the first release commit that does not present the undefined: KinfoProc error as stated above. After digging deeper, I noticed that this release upgraded github.com/shirou/gopsutil from version v2.20.3 to v2.20.9, which includes this Pull Request shirou/gopsutil#917 adding support for ARM64 and Apple Silicon to gopsutil!

I plan on creating a Pull Request upgrading the version used for Quickstart and e2e tests to the exact release stated above, do you think it makes sense @wiardvanrij? I am also considering creating a new issue to handle this upgrade and avoid polluting the more complex problem related to the race condition

[wiardvanrij]

Yeah it's absolutely fine to create a MR to fix the development minio. You can reference this issue in your MR.
I'd rather create a new one to see how we can move to upstream minio

[B0go]

@wiardvanrij I created this other issue for moving back to the upstream: #5393 please can you update this issue's title to match the local development apple silicon related problem?

[stale]

Hello 👋 Looks like there was no activity on this issue for the last two months.
Do you mind updating us on the status? Is this still reproducible or needed? If yes, just comment on this PR or push a commit. Thanks! 🤗
If there will be no activity in the next two weeks, this issue will be closed (we can always reopen an issue if we need!). Alternatively, use remind command if you wish to be reminded at some point in future.

[wiardvanrij]

This still should be fixed now!