[Bug]: test_runtime.py tests fail on mac

[neubig]

Is there an existing issue for the same bug?

• I have checked the troubleshooting document at https://opendevin.github.io/OpenDevin/modules/usage/troubleshooting
• I have checked the existing issues.

Describe the bug

Currently, on the main branch, test_runtime.py tests fail on mac. To reproduce:

poetry run pytest ./tests/unit/test_runtime.py

I've attached the full log below, but It seems like the most relevant error is:

08:48:49 - opendevin:INFO�[0m: runtime_build.py:202 - Error pulling image od_runtime:ubuntu_tag_22.04, building it from scratch

which causes the test to time-out?

Current OpenDevin version

commit 5a5713009f7a0b3fd2f61f43bf14452bf1b92ad5

Installation and Configuration

make build

Model and Agent

No response

Operating System

MacOS 14.1.2 (23B92)

Reproduction Steps

No response

Logs, Errors, Screenshots, and Additional Context

test_runtime.log

[tobitege]

I've asked @SmartManoj to apply a fix since he's got a PR for the same file:
#2989

Apparently I have to reason again.

[xingyaoww]

This should be fixed by #2993 - Let's reopen if it didn't

[neubig]

I have confirmed that the issue is not fixed by #2993... It couldn't be, as github workflows do not affect when you run tests locally.

I also think that in general disabling tests is not a great way to fix test errors...

[xingyaoww]

Ohhh
08:48:49 - opendevin:INFO�[0m: runtime_build.py:202 - Error pulling image od_runtime:ubuntu_tag_22.04, building it from scratch
I think this is not really an "error" locally..

self = <_UnixSelectorEventLoop running=False closed=False debug=False>
fut = None, sock = <socket.socket [closed] fd=-1, family=2, type=1, proto=6>
address = ('127.0.0.1', 61683)

    def _sock_connect_cb(self, fut, sock, address):
        if fut.done():
            return
    
        try:
            err = sock.getsockopt(socket.SOL_SOCKET, socket.SO_ERROR)
            if err != 0:
                # Jump to any except clause below.
>               raise OSError(err, f'Connect call failed {address}')
E               ConnectionRefusedError: [Errno 61] Connect call failed ('127.0.0.1', 61683)

I guess the real error is probably around the host networking? Let me see if i can push a fix

[xingyaoww]

@neubig what is your local docker desktop version - i remember only the latest supports host networking

If you have time, also feel free to try setting use_host_network=true to see if address the issue.. I should probably hard code it in the config for testing..

https://github.com/OpenDevin/OpenDevin/blob/f70c5afb6e9763fbaf84d2e4f4c7c5d4fc73bffb/opendevin/core/config.py#L523-L527

[neubig]

Ugh, it seems like the root cause may have been that my docker client was frozen. I just realized this when I tried to open docker to check the version. Let me try to revive it and if it works after that I'll close this issue.

[neubig]

Hmm, that did not solve the issue. I also updated to the latest docker desktop version (below) and it's still not working.

Version 4.32.0 (157355)
Engine: 27.0.3
Compose: v2.28.1-desktop.1
Credential Helper: v0.8.2
Kubernetes: v1.29.2

[xingyaoww]

@neubig You'd need to enable that feature as suggested here: https://docs.docker.com/network/drivers/host/#docker-desktop
Anyway - i added a PR #3005 to fix this, feel free to check that :D