adb_shell.exceptions.InvalidCommandError in `connect()` with Android emulator #81

mal-mel · 2020-02-28T15:53:18Z

Hello, Jeff, i need help)
When i running my code:

from adb_shell import adb_device
from adb_shell.auth.sign_pythonrsa import PythonRSASigner
import os


class Controller:
    def __init__(self, host: str, port: int):
        self.host, self.port = host, port
        self.device, self._signer = None, None

    def connect_to_device(self, timeout: int):
        self.device = adb_device.AdbDeviceTcp(self.host, self.port, default_timeout_s=timeout)
        self._make_sign("~/.android/adbkey")
        self.device.connect(auth_timeout_s=0.1, rsa_keys=[self._signer])

    def _make_sign(self, path: str):
        with open(os.path.expanduser(path)) as file:
            rsa_key = file.read()
        self._signer = PythonRSASigner("", rsa_key)

    def shell(self, command: str) -> str:
        output = self.device.shell(command, decode=True)
        return output


if __name__ == '__main__':
    adb_controller = Controller("localhost", 5556)
    adb_controller.connect_to_device(timeout=10)
    print(adb_controller.shell("echo work!"))

An emulator is running on 5556 port.
I get an error adb_shell.exceptions.InvalidCommandError: ('Unknown command: 72646e41', 1919184449, (543451503, 1936617283)) or something like this, I would like to know why this is happening.

The text was updated successfully, but these errors were encountered:

JeffLIrion · 2020-02-29T02:26:55Z

def from_int(n):
    return ''.join(chr((n >> (i * 8)) % 256) for i in range(4)).encode('utf-8')

That is the opposite of

adb_shell/tests/test_adb_device.py

Lines 25 to 26 in 825294d

    
           def to_int(cmd): 
        
               return sum(c << (i * 8) for i, c in enumerate(bytearray(cmd)))

I don't know what 72646e41 is, but the following 3 numbers translate to Andr + oid + Cons.

Your exception is occurring here:

adb_shell/adb_shell/adb_device.py

Line 847 in 825294d

    
           raise exceptions.InvalidCommandError('Unknown command: %x' % cmd, cmd, (arg0, arg1))

If you want more info, you need to turn on debug logging.

mal-mel · 2020-02-29T12:29:17Z

Ok, but how do I make it work?)

JeffLIrion · 2020-02-29T13:04:53Z

More info is needed to see what's going on.

import logging

logging.getLogger().setLevel(logging.DEBUG)

Post a log and I'll take a look.

mal-mel · 2020-02-29T13:26:34Z

2020-02-29 16:25:44,905 : [adb_device.py.921]: DEBUG : bulk_write: b'CNXN\x00\x00\x00\x01\x00\x10\x00\x00\x13\x00\x00\x00c\x05\x00\x00\xbc\xb1\xa7\xb1'
2020-02-29 16:25:44,905 : [adb_device.py.923]: DEBUG : bulk_write: b'host::9da23738bc43\x00'
2020-02-29 16:25:44,906 : [adb_device.py.784]: DEBUG : bulk_read(24): b'Android Console: Authent'

JeffLIrion · 2020-02-29T14:56:21Z

Try this with debug logging enabled and post a log.

adb_controller = Controller("localhost", 5556)
try:
    adb_controller.connect_to_device(timeout=10)
except:
    pass
print(adb_controller.device._handle.bulk_read(1000, 5))

mal-mel · 2020-02-29T15:22:50Z

This is output:

b"ication required\r\nAndroid Console: type 'auth <auth_token>' to authenticate\r\nAndroid Console: you can find your <auth_token> in \r\n'/root/.emulator_console_auth_token'\r\nOK\r\n"
Traceback (most recent call last):
  File "test.py", line 62, in <module>
    print(adb_controller.shell("echo work!"))
  File "test.py", line 26, in shell
    output = self.device.shell(command, decode=True)
  File "/usr/local/lib/python3.6/dist-packages/adb_shell/adb_device.py", line 426, in shell
    return self._service(b'shell', command.encode('utf8'), timeout_s, total_timeout_s, decode)
  File "/usr/local/lib/python3.6/dist-packages/adb_shell/adb_device.py", line 402, in _service
    return b''.join(self._streaming_command(service, command, adb_info)).decode('utf8')
  File "/usr/local/lib/python3.6/dist-packages/adb_shell/adb_device.py", line 953, in _streaming_command
    self._open(b'%s:%s' % (service, command), adb_info)
  File "/usr/local/lib/python3.6/dist-packages/adb_shell/adb_device.py", line 738, in _open
    _, adb_info.remote_id, their_local_id, _ = self._read([constants.OKAY], adb_info)
  File "/usr/local/lib/python3.6/dist-packages/adb_shell/adb_device.py", line 783, in _read
    msg = self._handle.bulk_read(constants.MESSAGE_SIZE, adb_info.timeout_s)
  File "/usr/local/lib/python3.6/dist-packages/adb_shell/handle/tcp_handle.py", line 127, in bulk_read
    raise TcpTimeoutException(msg)
adb_shell.exceptions.TcpTimeoutException: Reading from localhost:5556 timed out (10 seconds)

Log:

2020-02-29 18:21:50,678 : [adb_device.py.921]: DEBUG : bulk_write: b'CNXN\x00\x00\x00\x01\x00\x10\x00\x00\x13\x00\x00\x00c\x05\x00\x00\xbc\xb1\xa7\xb1'
2020-02-29 18:21:50,684 : [adb_device.py.923]: DEBUG : bulk_write: b'host::9da23738bc43\x00'
2020-02-29 18:21:50,685 : [adb_device.py.784]: DEBUG : bulk_read(24): b'Android Console: Authent'
2020-02-29 18:21:50,685 : [adb_device.py.921]: DEBUG : bulk_write: b'OPEN\x01\x00\x00\x00\x00\x00\x00\x00\x11\x00\x00\x00\xf5\x05\x00\x00\xb0\xaf\xba\xb1'
2020-02-29 18:21:50,685 : [adb_device.py.923]: DEBUG : bulk_write: b'shell:echo work!\x00'

JeffLIrion · 2020-02-29T15:25:07Z

That output is not from the same code I posted.

mal-mel · 2020-02-29T15:27:12Z

Ok, i commented print(adb_controller.shell("echo work!"))
output:

b"ication required\r\nAndroid Console: type 'auth <auth_token>' to authenticate\r\nAndroid Console: you can find your <auth_token> in \r\n'/root/.emulator_console_auth_token'\r\nOK\r\n"

JeffLIrion · 2020-02-29T15:42:30Z

OK, I see now.

It looks like it's saying that it will only work if you use the key /root/.emulator_console_auth_token.

This is the reply you get when you try to connect:

Android Console: Authentication required
Android Console: type 'auth <auth_token>' to authenticate
Android Console: you can find your <auth_token> in 
'/root/.emulator_console_auth_token'
OK

Have you connected to the emulator using the official ADB binary?

mal-mel · 2020-02-29T16:16:32Z

Yes, the official adb connect to emulator successfully

JeffLIrion · 2020-02-29T16:21:59Z

Do you have a file /root/.emulator_console_auth_token?

If so, try using that in your connect_to_device function.

JeffLIrion · 2020-03-01T14:52:43Z

Have you had a chance to try using the key /root/.emulator_console_auth_token (if it exists)?

mal-mel · 2020-03-02T10:27:06Z

Hello, Jeff, sorry for the silence, there was no access to the computer
Yes, i have /root/.emulator_console_auth_token. Should I use it instead of ~ / .android / adbkey?

mal-mel · 2020-03-02T10:58:12Z

If i connect to Android console manually and enter auth <my_token> all work fine (i have access to the Android Console, but my goal is android shell). How i to do this with your lib?

JeffLIrion · 2020-03-02T13:33:46Z

Hello, Jeff, sorry for the silence, there was no access to the computer
Yes, i have /root/.emulator_console_auth_token. Should I use it instead of ~ / .android / adbkey?

Yes, please try that.

mal-mel · 2020-03-02T13:58:36Z

Yes, I tried to make a signature with this key, but nothing

JeffLIrion · 2020-03-02T14:02:52Z

Can you post the output from this, as you did before.

adb_controller = Controller("localhost", 5556)
try:
    adb_controller.connect_to_device(timeout=10)
except:
    pass
print(adb_controller.device._handle.bulk_read(1000, 5))

mal-mel · 2020-03-02T15:28:31Z

Traceback (most recent call last):
  File "test.py", line 36, in <module>
    print(adb_controller.device._handle.bulk_read(1000, 5))
  File "/usr/local/lib/python3.6/dist-packages/adb_shell/handle/tcp_handle.py", line 122, in bulk_read
    readable, _, _ = select.select([self._connection], [], [], timeout)
TypeError: argument must be an int, or have a fileno() method.

JeffLIrion · 2020-03-02T15:43:01Z

I need debug logs to see the data that's being sent and read.

import logging
import os

from adb_shell import adb_device
from adb_shell.auth.sign_pythonrsa import PythonRSASigner


logging.getLogger().setLevel(logging.DEBUG)


class Controller:
    def __init__(self, host: str, port: int):
        self.host, self.port = host, port
        self.device, self._signer = None, None

    def connect_to_device(self, timeout: int):
        self.device = adb_device.AdbDeviceTcp(self.host, self.port, default_timeout_s=timeout)
        self._make_sign("/root/.emulator_console_auth_token")
        self.device.connect(auth_timeout_s=0.1, rsa_keys=[self._signer])

    def _make_sign(self, path: str):
        with open(os.path.expanduser(path)) as file:
            rsa_key = file.read()
        self._signer = PythonRSASigner("", rsa_key)

    def shell(self, command: str) -> str:
        output = self.device.shell(command, decode=True)
        return output


if __name__ == '__main__':
    adb_controller = Controller("localhost", 5556)
    try:
        adb_controller.connect_to_device(timeout=10)
    except:
        pass
    print(adb_controller.device._handle.bulk_read(1000, 5))

kunzhipeng · 2020-04-22T13:38:12Z

I met the same problem after i ctrl+c a running "shell command".

print device1.shell('busybox ping www.baidu.com', timeout_s=10, total_timeout_s=10)
Traceback (most recent call last):
File "", line 1, in
File "C:\Python27\lib\site-packages\adb_shell\adb_device.py", line 460, in shell
return self._service(b'shell', command.encode('utf8'), timeout_s, total_timeout_s, decode)
File "C:\Python27\lib\site-packages\adb_shell\adb_device.py", line 404, in _service
return b''.join(self._streaming_command(service, command, adb_info)).decode('utf8')
File "C:\Python27\lib\site-packages\adb_shell\adb_device.py", line 1013, in _streaming_command
for data in self._read_until_close(adb_info):
File "C:\Python27\lib\site-packages\adb_shell\adb_device.py", line 955, in _read_until_close
cmd, data = self._read_until([constants.CLSE, constants.WRTE], adb_info)
File "C:\Python27\lib\site-packages\adb_shell\adb_device.py", line 908, in _read_until
cmd, remote_id2, local_id2, data = self._read(expected_cmds, adb_info)
File "C:\Python27\lib\site-packages\adb_shell\adb_device.py", line 841, in _read
msg = self._handle.bulk_read(constants.MESSAGE_SIZE, adb_info.timeout_s)
File "C:\Python27\lib\site-packages\adb_shell\handle\tcp_handle.py", line 122, in bulk_read
readable, _, _ = select.select([self._connection], [], [], timeout)
KeyboardInterrupt
print device1.shell('busybox ping www.baidu.com')
Traceback (most recent call last):
File "", line 1, in
File "C:\Python27\lib\site-packages\adb_shell\adb_device.py", line 460, in shell
return self._service(b'shell', command.encode('utf8'), timeout_s, total_timeout_s, decode)
File "C:\Python27\lib\site-packages\adb_shell\adb_device.py", line 404, in _service
return b''.join(self._streaming_command(service, command, adb_info)).decode('utf8')
File "C:\Python27\lib\site-packages\adb_shell\adb_device.py", line 1011, in _streaming_command
self._open(b'%s:%s' % (service, command), adb_info)
File "C:\Python27\lib\site-packages\adb_shell\adb_device.py", line 796, in _open
_, adb_info.remote_id, their_local_id, _ = self._read([constants.OKAY], adb_info)
File "C:\Python27\lib\site-packages\adb_shell\adb_device.py", line 847, in _read
raise exceptions.InvalidCommandError('Unknown command: %x' % cmd, cmd, (arg0, arg1))
adb_shell.exceptions.InvalidCommandError: ('Unknown command: 62203436', 1646277686, (1936028793, 1869768224))

I also noticed the timeout seems notworking. Following command stuck all the time:
print device1.shell('busybox ping www.baidu.com', timeout_s=10, total_timeout_s=10)

kunzhipeng · 2020-04-22T13:49:16Z

I can reproduce the "adb_shell.exceptions.InvalidCommandError" problem like following:

Run:
print device1.shell('busybox ping www.baidu.com')
Press “Ctrl + C”, then run:
print device1.shell('echo hi')

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python27\lib\site-packages\adb_shell\adb_device.py", line 460, in shell
    return self._service(b'shell', command.encode('utf8'), timeout_s, total_timeout_s, decode)
  File "C:\Python27\lib\site-packages\adb_shell\adb_device.py", line 404, in _service
    return b''.join(self._streaming_command(service, command, adb_info)).decode('utf8')
  File "C:\Python27\lib\site-packages\adb_shell\adb_device.py", line 1011, in _streaming_command
    self._open(b'%s:%s' % (service, command), adb_info)
  File "C:\Python27\lib\site-packages\adb_shell\adb_device.py", line 796, in _open
    _, adb_info.remote_id, their_local_id, _ = self._read([constants.OKAY], adb_info)
  File "C:\Python27\lib\site-packages\adb_shell\adb_device.py", line 847, in _read
    raise exceptions.InvalidCommandError('Unknown command: %x' % cmd, cmd, (arg0, arg1))
adb_shell.exceptions.InvalidCommandError: ('Unknown command: 62203436', 1646277686, (1936028793, 1869768224))

JeffLIrion · 2020-04-22T15:15:39Z

@kunzhipeng your issue is different. I changed the title of this issue to reflect that it is about an error when trying to connect to an Android emulator.

I think the Ctrl+C issue you're describing here is even different than the new issue you created. I think the problem is that you're stopping the current task on the host computer -- i.e., running that she'll command via adb_shell in Python -- but one or both of the following scenarios occurs:

the command you sent is still running on the Android device
the command has stopped on the Android device, but there is still more output from that command for adb_shell to read

Either way, when you try to run a new shell command, it expects a response to the new command but receives a response from the old command, hence the error.

I'm not sure how to handle that other than closing the connection and reconnecting.

combacsa · 2020-05-09T15:44:13Z

Glad that I'm facing the exactly same issue here. I'm using Mac OS X Cata...blah, and using good old telnet I was able to do somethings, but facing InvalidCommandError: ('Unknown command: 72646e41', 1919184449, (543451503, 1936617283)).

combacsa · 2020-05-09T15:52:10Z

I replaced this line self._make_sign("/Users/khbyun/.android/adbkey") and debugging log I got is:

b"ication required\r\nAndroid Console: type 'auth <auth_token>' to authenticate\r\nAndroid Console: you can find your <auth_token> in \r\n'/Users/khbyun/.emulator_console_auth_token'\r\nOK\r\n"

combacsa · 2020-05-09T15:54:07Z

So @JeffLIrion is the above log message enough for you to debug?

JeffLIrion · 2020-05-09T16:19:42Z

@combacsa try using the key /Users/khbyun/.emulator_console_auth_token, if it exists.

combacsa · 2020-05-09T16:20:39Z

@JeffLIrion I tried with it but it failed with error, like it's not a permitable file for a RSA... exact log message is:

combacsa · 2020-05-09T16:21:00Z

ValueError: No PEM start marker "b'-----BEGIN PRIVATE KEY-----'" found

combacsa · 2020-05-09T16:21:27Z

(I'm kinda in a hurry, since I'm creating a code for a birthday present now...)

combacsa · 2020-05-09T16:24:12Z

Interestingly, when I just use telnet, My workflow goes like this:

Command:
telnet localhost 5554

Output:

Trying ::1...
Connected to localhost.
Escape character is '^]'.
Android Console: Authentication required
Android Console: type 'auth <auth_token>' to authenticate
Android Console: you can find your <auth_token> in
'/Users/khbyun/.emulator_console_auth_token'
OK

Then I just type the contents from '/Users/khbyun/.emulator_console_auth_token' like

auth <BLAHBLAH>

then

Android Console: type 'help' for a list of commands
OK

So, there could be a sudden change happend in adb's socket protocol?

JeffLIrion · 2020-05-09T16:58:52Z

ValueError: No PEM start marker "b'-----BEGIN PRIVATE KEY-----'" found

Try making a backup of the key and then pasting that as the first line in the original.

-----BEGIN PRIVATE KEY-----
<The rest of the file>

kolayne · 2020-08-22T10:59:16Z

+1

Python 3.8.2 (default, Jul 16 2020, 14:00:26) 
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from adb_shell.adb_device import AdbDeviceTcp, AdbDeviceUsb
>>> from adb_shell.auth.sign_pythonrsa import PythonRSASigner
>>> import os.path as op
>>> with open(op.expanduser('~/.android/adbkey')) as f:
...     priv = f.read()
... 
>>> signer = PythonRSASigner('', priv)
>>> device2 = AdbDeviceTcp('localhost', 5554)
>>> device2.connect(rsa_keys=[signer])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.8/dist-packages/adb_shell/adb_device.py", line 219, in connect
    cmd, arg0, arg1, banner = self._read([constants.AUTH, constants.CNXN], adb_info)
  File "/usr/local/lib/python3.8/dist-packages/adb_shell/adb_device.py", line 758, in _read
    raise exceptions.InvalidCommandError('Unknown command: %x' % cmd, cmd, (arg0, arg1))
adb_shell.exceptions.InvalidCommandError: ('Unknown command: 72646e41', 1919184449, (543451503, 1936617283))
>>>

JeffLIrion · 2020-08-22T19:25:08Z

I just published a new release. It doesn't fix the problem, but the exception message will contain more information. Although I'm sure it will just be the same info in this post: #81 (comment)

(Actually, it will only be the first part of it: "Android Console: Authent")

This isn't a use case that I plan to support, but if someone wants to contribute a fix they are welcome to do so.

barkside · 2023-05-22T13:59:33Z

Hi Jeff! I'm getting this error too.
Following on from comments, using emulator_console_auth_token with BEGIN PRIVATE KEY and END_PRIVATE_KEY fails with pyasn1.error.SubstrateUnderrunError: 105<10 at <TagSet object, tags 192:32:1> when decoding the rsa private key.

In my case emulator_console_auth_token is tiny "4ekw5I+6eOro0wvG" -

barkside · 2023-05-22T14:03:12Z

If I empty the auth token file I get:
Unknown command: 1919184449 = 'b'Andr'' (arg0 = 543451503, arg1 = 1936617283, msg = 'b"Android Console: type 'h"')

DEBUG:adb_shell.adb_device:bulk_write(24): b'CNXN\x00\x00\x00\x01\x00\x00\x10\x00\x10\x00\x00\x00K\x04\x00\x00\xbc\xb1\xa7\xb1'
DEBUG:adb_shell.adb_device:bulk_write(16): b'host::C-RB-L480\x00'
DEBUG:adb_shell.adb_device:bulk_read(24): b"Android Console: type 'h"

JeffLIrion changed the title ~~adb_shell.exceptions.InvalidCommandError~~ adb_shell.exceptions.InvalidCommandError with Android emulator Apr 22, 2020

JeffLIrion changed the title ~~adb_shell.exceptions.InvalidCommandError with Android emulator~~ adb_shell.exceptions.InvalidCommandError in connect() with Android emulator Apr 22, 2020

JeffLIrion mentioned this issue Nov 16, 2020

ADB Push fails: "Unable to unpack ADB command" #133

Closed

adb_shell.exceptions.InvalidCommandError in connect() with Android emulator #81

adb_shell.exceptions.InvalidCommandError in connect() with Android emulator #81

Comments

mal-mel commented Feb 28, 2020 • edited Loading

JeffLIrion commented Feb 29, 2020

mal-mel commented Feb 29, 2020

JeffLIrion commented Feb 29, 2020

mal-mel commented Feb 29, 2020 • edited Loading

JeffLIrion commented Feb 29, 2020 • edited Loading

mal-mel commented Feb 29, 2020

JeffLIrion commented Feb 29, 2020

mal-mel commented Feb 29, 2020

JeffLIrion commented Feb 29, 2020

mal-mel commented Feb 29, 2020

JeffLIrion commented Feb 29, 2020

JeffLIrion commented Mar 1, 2020

mal-mel commented Mar 2, 2020

mal-mel commented Mar 2, 2020 • edited Loading

JeffLIrion commented Mar 2, 2020

mal-mel commented Mar 2, 2020

JeffLIrion commented Mar 2, 2020

mal-mel commented Mar 2, 2020

JeffLIrion commented Mar 2, 2020

kunzhipeng commented Apr 22, 2020

kunzhipeng commented Apr 22, 2020 • edited by JeffLIrion Loading

JeffLIrion commented Apr 22, 2020

combacsa commented May 9, 2020

combacsa commented May 9, 2020

combacsa commented May 9, 2020

JeffLIrion commented May 9, 2020

combacsa commented May 9, 2020

combacsa commented May 9, 2020

combacsa commented May 9, 2020

combacsa commented May 9, 2020

JeffLIrion commented May 9, 2020

kolayne commented Aug 22, 2020

JeffLIrion commented Aug 22, 2020

barkside commented May 22, 2023

barkside commented May 22, 2023 • edited Loading

adb_shell.exceptions.InvalidCommandError in `connect()` with Android emulator #81

adb_shell.exceptions.InvalidCommandError in `connect()` with Android emulator #81

mal-mel commented Feb 28, 2020 •

edited

Loading

mal-mel commented Feb 29, 2020 •

edited

Loading

JeffLIrion commented Feb 29, 2020 •

edited

Loading

mal-mel commented Mar 2, 2020 •

edited

Loading

kunzhipeng commented Apr 22, 2020 •

edited by JeffLIrion

Loading

barkside commented May 22, 2023 •

edited

Loading