Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"Alexa Media Player failed to login" when adding integration for amazon.co.uk: Not logged in: An exception of type ContentTypeError occurred #2469

Open
deosrc opened this issue Aug 18, 2024 · 33 comments
Assignees
Labels
alexapy Issue relates to the API

Comments

@deosrc
Copy link

deosrc commented Aug 18, 2024

IMPORTANT: Please search the issues, including closed issues, and the FAQ before opening a new issue. The template is mandatory; failure to use it will result in issue closure.

Describe the bug

When adding the AMP integration using the amazon.co.uk region, login to amazon is successful but then the integration states:

image

To Reproduce

  1. Remove the AMP integration
  2. Re-add the AMP integration setting the amazon region to amazon.co.uk
  3. Login to amazon through the proxy page
  4. Once the proxy page closes, the error is displayed

Expected behavior

Should be able to add the integration for the amazon.co.uk region.

Screenshots

image

System details

  • Home-assistant (version): 2024.8.1
  • alexa_media (version from const.py or HA startup): 4.12.7
  • alexapy (version from pip show alexapy or HA startup): Not sure how to find this in HAOS. Doesn't seem to be in the logs?
  • Amazon 2FA is enabled (y/n). <!---We will not debug login issues if unanswered--->: y

Logs
Please provide logs.

Apologies if this is excessively redacted.

2024-08-18 16:17:20.261 DEBUG (MainThread) [alexapy.alexalogin] 5 cookies successfully exchanged for refresh token for domain .amazon.co.uk
2024-08-18 16:17:20.263 DEBUG (MainThread) [alexapy.alexalogin] Attempting to discover CSRF token
2024-08-18 16:17:20.383 DEBUG (MainThread) [alexapy.alexalogin] CSRF token not found from /spa/index.html
2024-08-18 16:17:20.626 DEBUG (MainThread) [alexapy.alexalogin] CSRF token not found from /api/devices-v2/device?cached=false
2024-08-18 16:17:20.684 DEBUG (MainThread) [alexapy.alexalogin] CSRF token not found from /templates/oobe/d-device-pick.handlebars
2024-08-18 16:17:20.728 DEBUG (MainThread) [alexapy.alexalogin] No csrf token found

2024-08-18 16:17:21.124 DEBUG (MainThread) [alexapy.alexalogin] GET: 
https://alexa.amazon.com/api/bootstrap returned 200:OK with response <CIMultiDictProxy('Content-Type': 'application/json', 'Content-Length': '172', 'Connection': 'keep-alive', 'Server': 'Server', 'Date': 'Sun, 18 Aug 2024 15:17:21 GMT', 'Vary': 'Content-Type,Accept-Encoding,User-Agent', 'x-amz-rid': '29EZT64H7XHEE5M0K0ER', 'Set-Cookie': 'csrf=-701784766; expires=Fri, 18-Aug-2034 15:17:20 GMT; Domain=.amazon.com; Path=/', 'x-amzn-RequestId': '29EZT64H7XHEE5M0K0ER', 'x-amzn-alt-domain': 'https://alexa.amazon.co.uk/logout', 'X-Cache': 'Miss from cloudfront', 'Via': '1.1 ce6dd6cf8e3b2bf695c066e441ad568e.cloudfront.net (CloudFront)', 'X-Amz-Cf-Pop': 'LHR61-P6', 'X-Amz-Cf-Id': 'l0vWnUXdmXysduXb9GVKjiCRjCUb0_4RDTxQiw8OZTSXDrilcU50bA==')>

2024-08-18 16:17:21.167 DEBUG (MainThread) [alexapy.alexalogin] GET: 
https://alexa.amazon.co.uk/api/bootstrap returned 401:Unauthorized with response <CIMultiDictProxy('Content-Length': '0', 'Connection': 'keep-alive', 'Server': 'Server', 'Date': 'Sun, 18 Aug 2024 15:17:21 GMT', 'Strict-Transport-Security': 'max-age=0; includeSubDomains;', 'x-amz-rid': 'GS2055T72KC6WFXSRBD8', 'loginUrl': 'https://www.amazon.co.uk/ap/signin?showRmrMe=1&openid.return_to=https%3A%2F%2Falexa.amazon.co.uk%2Flogin&openid.identity=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0%2Fidentifier_select&openid.assoc_handle=amzn_dp_project_dee_uk&openid.mode=checkid_setup&openid.claimed_id=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0%2Fidentifier_select&openid.ns=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0&', 'x-amzn-RequestId': 'GS2055T72KC6WFXSRBD8', 'Vary': 'Content-Type,Accept-Encoding,User-Agent', 'X-Cache': 'Error from cloudfront', 'Via': '1.1 6cd1874959ee48e26855209aa18a4014.cloudfront.net (CloudFront)', 'X-Amz-Cf-Pop': 'LHR50-P7', 'X-Amz-Cf-Id': 'a_kCzoW7gXbKbd_13B1UiHWlFsEnNgovCnc9TFi6p1N7mxi0Kq7pjw==')>

2024-08-18 16:17:21.168 DEBUG (MainThread) [alexapy.alexalogin] Not logged in: An exception of type ContentTypeError occurred. Arguments:
(RequestInfo(url=URL('https://alexa.amazon.co.uk/api/bootstrap'), method='GET', headers=<CIMultiDictProxy('Host': 'alexa.amazon.co.uk', 'User-Agent': 'AmazonWebView/Amazon Alexa/2.2.556530.0/iOS/16.6/iPhone', 'Accept': '*/*', 'Accept-Language': '*', 'DNT': '1', 'Upgrade-Insecure-Requests': '1', 'Accept-Encoding': 'gzip, deflate, br', 'Cookie': 'at-main=<REDATED>; lc-main=en_GB; sess-at-main="\\"<REDACTED>\\""; session-id=258-2185812-7023702; session-id-time=2354714240l; session-token=<REDACTED>; ubid-acbuk=130-0156096-0538242; ubid-main=130-0156096-0538242; x-main=<REDACTED>')>, real_url=URL('https://alexa.amazon.co.uk/api/bootstrap')), ())

Additional context

I have found a partial workaround to add the amazon.co.uk config entry, but it does not result in a functioning AMP integration. Instead, the integration prompts to be reconfigured (possibly one of the other issues such as #2453), and has some strange behaviour possibly as a result of the same account being registered in two config entries (conflicting pickle file?)

The workaround to add the config entry:

  1. Add the AMP integration but using the amazon.com region.
  2. Once logged in to amazon, the AMP integration will be added, but will fail to start.
    image
  3. Click the 3 dot menu for the failing entry and select "System options"
  4. Turn off both options and save:
    image
  5. Click "Add entry" and complete the setup this time for amazon.co.uk region
    image
  6. The integration should add successfully, but will then prompt to be re-configured:
    image
  7. Remove the amazon.com entry.

Edit: Removing the entry seemingly removes both the amazon.com and amazon.co.uk entries. Possibly a conflicting ID (email?) causing some strange behaviour.

@danielbrunt57
Copy link
Collaborator

danielbrunt57 commented Aug 18, 2024

Could you try this:

  • Delete the integration's config entry
  • Delete /config/,storage/alexa_media.<your_email>.pickle
  • In HACS, delete the custom_component and restart HA
  • In HACS, install Alexa Media Player custom component v4.12.8
  • Restart HA
  • in ssh/terminal session:
    • docker exec -it homeassistant bash
    • cd /usr/local/lib/python3.12/http
    • cd ../site-packages/alexapy
    • This adds alexapy merge request !380 which contains the corresponding async def delete_cookiefile(self) which AMP 4.12.8 will call to delete the pickle file when removing the config entry.
  • Restart HA
  • Clear browser cookies & cached files
  • Try adding the integration with:
    • Seconds between scans 360 - with http2push active, this effectively becomes 1 hour (3600 seconds)
    • For now, do not enable Include devices connected via Echo
  • Verify at this point that the integration is successfully configured
  • Three dots System options
    • Disable Enable newly added entities but leave Enable polling for updates enabled

@danielbrunt57 danielbrunt57 self-assigned this Aug 18, 2024
@danielbrunt57 danielbrunt57 added the alexapy Issue relates to the API label Aug 18, 2024
@deosrc
Copy link
Author

deosrc commented Aug 18, 2024

@danielbrunt57

I have:

  1. Removed all AMP config entries.
  2. Checked /config/.storage, pickle file was not present
  3. Updated AMP to 4.12.8 and restarted
  4. Installed and configured advanced SSH addon to gain access to docker in HassOS
  5. Replaced the contents of http/cookies.py and site-packages/alexapy/alexalogin.py within the homeassistant docker container.
  6. Restarted HA
  7. Launched chromium and cleared all browsing data, then restarted the browser.
  8. Login to HA using IP address (to avoid any differences between default URL in AMP and alternate URLs such as homeassistant.local)
  9. Attempt to add AMP integration using amazon.co.uk region and 360 seconds between scans
    • Same result after Amazon login and OTP entry. Failed to login.
  10. Attempt workaround which still seems to be working.

I forgot to re-enable debug logging when first trying this, but enabling it afterwards and attempting to add the AMP integration yielded mostly the same 401 Unauthorized log messages.

I'm not sure if there are more URLs now showing 401 or if I just missed these before. It looks like the same log section so perhaps related to the changes:

2024-08-18 21:35:31.988 DEBUG (MainThread) [alexapy.alexalogin] 5 cookies successfully exchanged for refresh token for domain .amazon.co.uk
2024-08-18 21:35:31.990 DEBUG (MainThread) [alexapy.alexalogin] Attempting to discover CSRF token
2024-08-18 21:35:32.100 DEBUG (MainThread) [alexapy.alexalogin] CSRF token not found from /spa/index.html
2024-08-18 21:35:32.178 DEBUG (MainThread) [alexapy.alexalogin] Unable to load page for csrf: <ClientResponse(https://alexa.amazon.co.uk/api/language) [401 Unauthorized]>
<CIMultiDictProxy('Content-Length': '0', 'Connection': 'keep-alive', 'Server': 'Server', 'Date': 'Sun, 18 Aug 2024 20:35:32 GMT', 'x-amz-rid': 'WRNKE3AZMZ9DXJHAX25T', 'loginUrl': 'https://www.amazon.co.uk/ap/signin?showRmrMe=1&openid.return_to=https%3A%2F%2Falexa.amazon.co.uk%2Flogin&openid.identity=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0%2Fidentifier_select&openid.assoc_handle=amzn_dp_project_dee_uk&openid.mode=checkid_setup&openid.claimed_id=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0%2Fidentifier_select&openid.ns=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0&', 'x-amzn-RequestId': 'WRNKE3AZMZ9DXJHAX25T', 'Vary': 'Content-Type,Accept-Encoding,User-Agent', 'Strict-Transport-Security': 'max-age=0; includeSubDomains;', 'X-Cache': 'Error from cloudfront', 'Via': '1.1 3d8e500d44b557879a1086daf1dc3aaa.cloudfront.net (CloudFront)', 'X-Amz-Cf-Pop': 'LHR50-P7', 'X-Amz-Cf-Id': 'HZXUb3Z6ORWiaDjGmhw8Zv8WJkPGC8h5tHRw_AzLcKlr13nXlC_qrA==')>

2024-08-18 21:35:32.294 DEBUG (MainThread) [alexapy.alexalogin] CSRF token not found from /api/devices-v2/device?cached=false
2024-08-18 21:35:32.354 DEBUG (MainThread) [alexapy.alexalogin] CSRF token not found from /templates/oobe/d-device-pick.handlebars
2024-08-18 21:35:32.398 DEBUG (MainThread) [alexapy.alexalogin] Unable to load page for csrf: <ClientResponse(https://alexa.amazon.co.uk/api/strings) [401 Unauthorized]>
<CIMultiDictProxy('Content-Length': '0', 'Connection': 'keep-alive', 'Server': 'Server', 'Date': 'Sun, 18 Aug 2024 20:35:32 GMT', 'x-amz-rid': 'RHX33MM85WX8E01P5DYE', 'loginUrl': 'https://www.amazon.co.uk/ap/signin?showRmrMe=1&openid.return_to=https%3A%2F%2Falexa.amazon.co.uk%2Flogin&openid.identity=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0%2Fidentifier_select&openid.assoc_handle=amzn_dp_project_dee_uk&openid.mode=checkid_setup&openid.claimed_id=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0%2Fidentifier_select&openid.ns=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0&', 'x-amzn-RequestId': 'RHX33MM85WX8E01P5DYE', 'Vary': 'Content-Type,Accept-Encoding,User-Agent', 'Strict-Transport-Security': 'max-age=0; includeSubDomains;', 'X-Cache': 'Error from cloudfront', 'Via': '1.1 e852f14db89debadcfac3bde6e826efe.cloudfront.net (CloudFront)', 'X-Amz-Cf-Pop': 'LHR50-P7', 'X-Amz-Cf-Id': 'ZZHYEbLe3Z4yrJguiY_mAIR03ftuSWd4OmLx4ppkR3AMWHh9l_Fvmw==')>

2024-08-18 21:35:32.398 DEBUG (MainThread) [alexapy.alexalogin] No csrf token found

From the details of the log message I have managed to reproduce this in RestFox (Postman equivalent). I built up the request from the log message for the request to https://alexa.amazon.co.uk/api/bootstrap which is failing and it returns the same 401 Unauthorized response with no body content. I only have to change the TLD from .co.uk to .com and it returns a 200 OK instead, just as the logs from HA indicate. There is a header X-Cache: Error from cloudfront appearing the response which may be relevant.

@deosrc
Copy link
Author

deosrc commented Aug 18, 2024

@danielbrunt57 This seems promising so feel free to leave this issue as is for now and I'll keep looking into it when I get chance. Summary below for info and for my own reference. Any guidance is of course welcome.


It occurred to me that the login is a proxy, so the cookies would probably be similar to the normal amazon website. I tried https://alexa.amazon.co.uk/api/bootstrap in my browser where I am already logged in and received the correct response.

I've then transferred the browser cookies into the RestFox request I had created above, and received the 200 OK response expected from amazon.co.uk. I then started comparing the cookies from the browser request with the cookies from the log mesesages in HA. There were more in the browser so I started removing the cookies one by one that were missing from the HA request. As soon as I removed the x-acbuk cookie, the request started failing.

To double check, I returned all of the cookies and it once again returned 200 OK, I then removed only the x-acbuk cookie, and the request failed. This seems a promising leads on a potential cause.

I logged in to amazon.co.uk again to grab the set-cookie header. There doesn't look to be anything obvious to make this cookie different, but I'm not familiar enough with it to know.

set-cookie: x-acbuk=<REDACTED>; Domain=.amazon.co.uk; Expires=Mon, 18-Aug-2025 21:26:21 GMT; Path=/; Secure

⚠️ For the following discussion of the logs, it should be noted that I have not reverted the changes to the python files, and have not yet restarted HA (to clear the logs) since performing various other steps adding and removing the AMP config. I can't be sure which attempt these messages were generated from. This is far from conclusive and could be a dead-end. I just wanted to post an update before signing off given this seems a promising atm.

I have 2 log messages shortly before the failing /api/bootstrap call showing the x-acbuk cookie. The one below is from "registration" which is first, it is then shortly followed by a similar one for "Exchange cookie json". The value looks to be different in each, but I need to confirm this.

(formatted for clarity and easier redaction)

2024-08-18 21:35:31.435 DEBUG (MainThread) [alexapy.alexalogin] Received registration data:
{
    "success": {
        // ...
        "tokens": {
            "website_cookies": [
                // ...
                {
                    "Path": "/",
                    "Secure": "true",
                    "Value": "\"<REDACTED>\"",
                    "Expires": "13 Aug 2044 20:35:31 GMT",
                    "Domain": ".amazon.co.uk",
                    "HttpOnly": "false",
                    "Name": "x-acbuk"
                },
                // ...
            ],
            // ...
        },
        //...
    }
}

There is another log message shortly after this which may (or may not) be related given there are 5 entries in "website_cookies".

2024-08-18 21:35:31.988 DEBUG (MainThread) [alexapy.alexalogin] 5 cookies successfully exchanged for refresh token for domain .amazon.co.uk

The code for the log message above looks to be here: https://gitlab.com/keatontaylor/alexapy/-/blob/dev/alexapy/alexalogin.py?ref_type=heads#L1187

This log message seems a little confusing as the code looks to be exchanging a refresh token for cookies, rather than cookies for a refresh token?

Between this log message and the failing api/bootstrap call are only attempts to determine the CSRF token which is never found.

@danielbrunt57
Copy link
Collaborator

@chrisvblemos Could you step in on this since it appears you have far more knowledge re: cookies, tokens and CSFR (based on alexapy merge request !378) than I do as mine is next to nil...

@danielbrunt57
Copy link
Collaborator

danielbrunt57 commented Aug 19, 2024

I just had this comment on an alexapy merge request (that I cancellled) from alandtse and I think it could very well relate to your issue...

Alan Tse
@alandtse
· 1 hour ago
Maintainer
The more likely cause of auth errors is the fact only like 5 cookies are saved now instead of every cookie that we found. That was the other change on that release.

@chrisvblemos
Copy link
Contributor

I'll take a look later if I can. But I think @deosrc is on the right track. My fix to alexapy here was made based on the same observations. since he is using the latest version with this fix, further investigation is needed to find out why aiohttp is not using the x-acbuk here.

@jleinenbach
Copy link
Contributor

jleinenbach commented Aug 19, 2024

amazon.de: As it works for me: Do we use the same minimum aiohttp version?
image

There were cookie fixes:
https://github.com/aio-libs/aiohttp/releases

Esp. this for my version 3.10.0:

Implemented filter_cookies() with domain-matching and path-matching on the keys, instead of testing every single cookie.
This may break existing cookies that have been saved with CookieJar.save(). Cookies can be migrated with this script::
import pickle
with file_path.open("rb") as f:
    cookies = pickle.load(f)

morsels = [(name, m) for c in cookies.values() for name, m in c.items()]
cookies.clear()
for name, m in morsels:
    cookies[(m["domain"], m["path"].rstrip("/"))][name] = m

with file_path.open("wb") as f:
    pickle.dump(cookies, f, pickle.HIGHEST_PROTOCOL)

@deosrc
Copy link
Author

deosrc commented Aug 19, 2024

@jleinenbach pip show aiohttp is showing as v3.10.3 for me. I believe it was updated in HA 2024.8.2 which I have updated to but the issue is still present for me.

Are you using amazon.de for the region? Based on the x-acbuk cookie name, this may be an amazon.co.uk specific issue. amazon.com doesn't seem to have the same problem.


I made some adjustments to the logging in alexalogin.py using the steps mentioned by @danielbrunt57 to try get some more diagnostics (adjusting some existing messages and adding output of filter_cookies("f{self._prefix}{self.url}") in various places) but I'm not seeing them take effect. I deleted the __pycache__ to be sure but it isn't being regenerated. pip show alexapy seems to indicate it is the correct location so I'm not sure what is happening.

I don't have any other version of alexalogin.py in the alexapy directory. The steps I'm using:

  1. SSH to HassOS Advanced SSH addon
  2. docker exec -it homeassistant bash
  3. cd /usr/local/lib/python3.12/site-packages/alexapy
  4. vi alexalogin.py
  5. Edit file and then exit using :wq
  6. Restart HA

The container shows as having restarted based on the uptime, and the changes are still there after reconnect so I'm not sure why they aren't being used. Any ideas?


Other things I've tried:

  • Reproducing the exchange token request in RestFox, all seems to work as expected and the x-acbuk token is one of the 5 cookies returned.

  • Writing a short python script (locally, not in HA container) to check filter_cookies is returning the cookie for what should be the inputs being used. Also seems to return as expected.

    import aiohttp
    import asyncio
    from http.cookies import SimpleCookie
    from yarl import URL
    
    async def main():
    
        async with aiohttp.ClientSession() as session:
            raw_cookie = SimpleCookie()
            cookie_name = "x-acbuk"
            cookie_value = "\"<REDACTED>\""
            raw_cookie[cookie_name] = (
                cookie_value
                if not (
                    cookie_value.startswith('"') and cookie_value.endswith('"')
                )
                # Strings are returned within quotations, strip them
                else cookie_value[1:-1]
            )
            raw_cookie[cookie_name]["domain"] = ".amazon.co.uk"
            raw_cookie[cookie_name]["path"] = "/"
            raw_cookie[cookie_name]["secure"] = True
            raw_cookie[cookie_name]["expires"] = "14 Aug 2044 15:57:18 GMT"
            raw_cookie[cookie_name]["httpOnly"] = False
    
            session.cookie_jar.update_cookies(raw_cookie, URL(".amazon.co.uk"))
            cookies = session.cookie_jar.filter_cookies(URL("https://alexa.amazon.co.uk/api/bootstrap"))
            print("Test") # Debugger breakpoint here...
    
    asyncio.run(main())

If I can't get the alexalogin.py changes to take effect, next thing I can think to try is to expand the test script to a full request to /api/bootstrap and see if it uses the cookies.

@danielbrunt57
Copy link
Collaborator

5. Replaced the contents of... and site-packages/alexapy/alexalogin.py within the homeassistant docker container.

async def delete_cookiefile(self) is now merged in alexapy 1.29.0 so you can just edit AMP's manifest.json from 1.28.2 to 1.29.0.

@danielbrunt57
Copy link
Collaborator

My aiohttp is also 3.10.3

@chrisvblemos
Copy link
Contributor

chrisvblemos commented Aug 19, 2024

I don't have any other version of alexalogin.py in the alexapy directory. The steps I'm using:

1. SSH to HassOS Advanced SSH addon

2. `docker exec -it homeassistant bash`

3. `cd /usr/local/lib/python3.12/site-packages/alexapy`

4. `vi alexalogin.py`

5. Edit file and then exit using `:wq`

6. Restart HA

The container shows as having restarted based on the uptime, and the changes are still there after reconnect so I'm not sure why they aren't being used. Any ideas?

HA doesn't reinstall packages on restart. Also, making changes directly to the locally stored python packages is not recommended.

@deosrc
Fork and then clone or just download the alexapy repo inside your /config path while in SSH (this folder is mapped to the homeassistant docker). Make your changes as desired. Then, attach to the docker as you did and run pip install -e /config/path/to/alexapy. Restart. Repeat these steps every time you make a change to alexapy.

@danielbrunt57 danielbrunt57 changed the title "Alexa Media Player failed to login" when adding integration for amazon.co.uk "Alexa Media Player failed to login" when adding integration for amazon.co.uk: Not logged in: An exception of type ContentTypeError occurred Aug 20, 2024
@emufan
Copy link

emufan commented Aug 20, 2024

I have the same (?) problems with amazon.de. At least the error messages are looking similar.

2024-08-20 10:56:50.146 DEBUG (MainThread) [alexapy.alexalogin] 5 cookies successfully exchanged for refresh token for domain .amazon.de
2024-08-20 10:56:50.147 DEBUG (MainThread) [alexapy.alexalogin] Attempting to discover CSRF token
2024-08-20 10:56:50.254 DEBUG (MainThread) [alexapy.alexalogin] CSRF token not found from /spa/index.html
2024-08-20 10:56:50.459 DEBUG (MainThread) [alexapy.alexalogin] CSRF token not found from /api/devices-v2/device?cached=false
2024-08-20 10:56:50.518 DEBUG (MainThread) [alexapy.alexalogin] CSRF token not found from /templates/oobe/d-device-pick.handlebars
2024-08-20 10:56:50.568 DEBUG (MainThread) [alexapy.alexalogin] No csrf token found
2024-08-20 10:56:50.866 DEBUG (MainThread) [alexapy.alexalogin] GET:
https://alexa.amazon.com/api/bootstrap returned 200:OK with response <CIMultiDictProxy('Content-Type': 'application/json', 'Content-Length': '171', 'Connection': 'keep-alive', 'Server': 'Server', 'Date': 'Tue, 20 Aug 2024 08:56:50 GMT', 'x-amz-rid': '6KKHP4FZ41X4BCB9HTB6', 'Set-Cookie': 'csrf=30060786; expires=Sun, 20-Aug-2034 08:56:50 GMT; Domain=.amazon.com; Path=/', 'x-amzn-RequestId': '6KKHP4FZ41X4BCB9HTB6', 'x-amzn-alt-domain': 'https://alexa.amazon.de/logout', 'Vary': 'Content-Type,Accept-Encoding,User-Agent', 'X-Cache': 'Miss from cloudfront', 'Via': '1.1 918126cde448a82c2757fd37058ea3ec.cloudfront.net (CloudFront)', 'X-Amz-Cf-Pop': 'HAM50-C1', 'X-Amz-Cf-Id': 'iO606tajoBCwq-Th8kkZq5uCE5wd6oikuf2eMvoQSlz_thiO78utvg==')>
2024-08-20 10:56:50.919 DEBUG (MainThread) [alexapy.alexalogin] GET:
https://alexa.amazon.de/api/bootstrap returned 401:Unauthorized with response <CIMultiDictProxy('Content-Length': '0', 'Connection': 'keep-alive', 'Server': 'Server', 'Date': 'Tue, 20 Aug 2024 08:56:50 GMT', 'Strict-Transport-Security': 'max-age=0; includeSubDomains;', 'x-amz-rid': 'Y477TQPNJ78YCCZWW972', 'loginUrl': 'https://www.amazon.de/ap/signin?showRmrMe=1&openid.return_to=https%3A%2F%2Falexa.amazon.de%2Flogin&openid.identity=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0%2Fidentifier_select&openid.assoc_handle=amzn_dp_project_dee_de&openid.mode=checkid_setup&openid.claimed_id=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0%2Fidentifier_select&openid.ns=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0&', 'x-amzn-RequestId': 'Y477TQPNJ78YCCZWW972', 'Vary': 'Content-Type,Accept-Encoding,User-Agent', 'X-Cache': 'Error from cloudfront', 'Via': '1.1 eabf0052502240e2b09c2e962490cabc.cloudfront.net (CloudFront)', 'X-Amz-Cf-Pop': 'HAM50-P1', 'X-Amz-Cf-Id': '-PQKFYqt386Wb4iwH8OjLJsELQZkdVXwXQyUcQujr8-7eTfUZivdVQ==')>
2024-08-20 10:56:50.920 DEBUG (MainThread) [alexapy.alexalogin] Not logged in: An exception of type ContentTypeError occurred. Arguments:
(RequestInfo(url=URL('https://alexa.amazon.de/api/bootstrap'), method='GET', headers=<CIMultiDictProxy('Host': 'alexa.amazon.de', 'User-Agent': 'AmazonWebView/Amazon Alexa/2.2.556530.0/iOS/16.6/iPhone', 'Accept': '*/*', 'Accept-Language': '*', 'DNT': '1', 'Upgrade-Insecure-Requests': '1', 'Accept-Encoding': 'gzip, deflate, br', 'Cookie': 'at-main=foo; lc-main=de_DE; sess-at-main="\\"b9BhEQzsyFcjYE/PA63XLpaDdONUfxzIgr+6vpYGL8E=\\""; session-id=257-9085617-4147239; session-id-time=2354864210l; session-token=bar; sid="\\"NPowU5s/wYMuNgqu4a7Dfg==|Sw777im+40ufbBsvCgpED22f/VD0l8vNEXo7L+h+hAE=\\""; ubid-acbde=135-3594605-4904817; ubid-main=135-3594605-4904817; x-main="\\"XDSi40t1MtVOVBV8nTfUOpNBgcWl?PaQ\\""')>, real_url=URL('https://alexa.amazon.de/api/bootstrap')), ())
async def delete_cookiefile(self) is now merged in alexapy 1.29.0 so you can just edit AMP's manifest.json from 1.28.2 to 1.29.0.

tried this already as well.

HA 2024.8.2
AMP v4.12.10

@deosrc
Copy link
Author

deosrc commented Aug 20, 2024

I'm not sure how, but AMP is working again for me. Entity updates, announcements, everything.

My container may be in a bit of a mess at the moment. As @chrisvblemos correctly points out, editing packages inside the container is not a good idea.

⚠️ Before making these changes, be aware that they are in the /cofing directory so will persist across updates. If they break HA, updating/re-creating the container will likely not fix it. If you are unsure, wait for an AMP update to be pushed out.

I believe my active changes are:

  1. Updated /config/custom_components/alexa_media/manifest.json to use a fork of alexapy with some logging changes:

    "requirements": ["alexapy@git+https://gitlab.com/deosrc1/alexapy.git@bugfix/unauthorized", "packaging>=20.3", "wrapt>=1.14.0"],
  2. Move/rename/remove the alexapy* directories from /config/deps/lib/python3.12/site-packages so that the dependency re-downloads (see additional)

  3. Restart HA

  4. Attempt to add the integration

There shouldn't be anything in that branch to affect the functionality other than adjusting some logging, but the integration added successfully, and everything seems to be working again. Perhaps it was some of the other recent changes which have now been applied from my steps above.

I'll try clean up my container and check to see if it is still working.


Additional

@chrisvblemos Thanks for the info. It put me on track to learn more about the internals and debugging of HA.

HA doesn't reinstall packages on restart.

I was still seeing some strange behaviour from the packages. It seems this might not be true: https://developers.home-assistant.io/docs/creating_integration_manifest/#requirements

The requirements seems to be checked when adding an integration. At one point when I had an invalid environments setup, attempting to add the AMP integration failed because it couldn't grab the pip package. There may be checks to see if the package is already installed before attempting. I haven't looked into it too closely.

Also, making changes directly to the locally stored python packages is not recommended.

Absolutely. My python knowledge and especially setting up HA container for debugging a pip package is fairly limited. I attempted setting up debugpy after this comment, but the breakpoints weren't being hit. Possibly because they are in a pip module? Even setting justMyCode to false and the path mappings didn't seem to work. It's possible I just had a configuration issue somewhere though.

I'm aware I'm making a mess of my container at the moment. It will at least destroy any (except within /config) when I update to the next version. I'll need to be careful until then.

Fork and then clone or just download the alexapy repo inside your /config path while in SSH (this folder is mapped to the homeassistant docker). Make your changes as desired. Then, attach to the docker as you did and run pip install -e /config/path/to/alexapy. Restart. Repeat these steps every time you make a change to alexapy.

👍 This seems a much better way, unfortunately see the above about how HA does requirements differently.

@chrisvblemos
Copy link
Contributor

@deosrc

Is there any chance you weren't using the latest alexapy version before using your fork? Not because you did something wrong, but to check if HA is not automatically updating the dependency (I don't think this is the case, but better safe than sorry).

Also, could you check if everything works if you roll back alexapy to this change? If you want to keep your way of using branches in the manifest.json, create a new branch and do the rollback. This is to see if any of the latest changes are in fact fixing this issue.

@deosrc
Copy link
Author

deosrc commented Aug 20, 2024

Is there any chance you weren't using the latest alexapy version before using your fork? Not because you did something wrong, but to check if HA is not automatically updating the dependency (I don't think this is the case, but better safe than sorry).

I'm honestly not sure. The alexa media version in the initial post is definitely accurate (although I've since upgraded to .8) , but I didn't know how to check the alexapy version at the time.

Also, could you check if everything works if you roll back alexapy to this change? If you want to keep your way of using branches in the manifest.json, create a new branch and do the rollback. This is to see if any of the latest changes are in fact fixing this issue.

I think I've worked out how the dependencies work now so should be able to target a specific alexapy commit to try this out and report back.

Given my deps folder also includes some older versions of python though, I'm thinking it might be worth creating a backup of the folder, and letting HA reacquire the dependencies. I should also look at re-creating the HA container to make sure I'm working with a clean container.

@alandtse
Copy link
Owner

The requirements seems to be checked when adding an integration. At one point when I had an invalid environments setup, attempting to add the AMP integration failed because it couldn't grab the pip package. There may be checks to see if the package is already installed before attempting. I haven't looked into it too closely.

This is correct. HA will only check component dependencies when adding the integration. HA upgrades will wipe out all custom dependencies, so forcing an add (even if not completed), is the best way to restore dependencies. I believe that was in the wiki at some point (or should be put there).

@philtarry
Copy link

Same issue here with amazon.co.uk

is there a simple workaround?

@arachnetech
Copy link

My Alexa Media player has been failing to log in too. I've just tried deleting and re-adding the integration, and even clearing all my 2FA settings, but I still get the 'Alexa Media Player failed to log in' error. I updated to HA 2024.8.3 (I was previously on 8.1) and tried again, but I still can't make it work.

Finding this open issue, hopefully this is a general issue with the integration which will be fixed in a future update? Or is this only affecting isolated accounts?

@LindsayReid
Copy link

LindsayReid commented Sep 1, 2024 via email

@jleinenbach
Copy link
Contributor

jleinenbach commented Sep 1, 2024

Try this:

  1. Upgrade HA 2024.8.1 and 2024.8.2 to HA 2024.8.3 if not already done
  2. Upgrade Python's cookie.py and check if it was successful: Alexa Media Reauthentication Required - continuously appear: Login error detected; not contacting API #2453 (comment)
  3. Install PythonScriptPro with HACS: https://github.com/AlexxIT/PythonScriptsPro/tree/master
  4. add this to your configuration.yaml
python_script:
  requirements:
    - alexapy>=1.29.0
    - aiohttp>=3.10.5
    - aiofiles>=24.1.0
  1. Deactivate all automations that use alexa_media.update_last_called
  2. Go to developer-tools/yaml (HA WebGUI) and check your configuration
  3. Reboot
  4. Reconfigure AMP: Set the scan interval from 60 to 90, make sure to add the / at the end of your URL if it is missing.

@andrea-git
Copy link

I'm pretty sure it's affecting thousands!

On Sun, 1 Sept 2024 at 11:09, David Miller @.> wrote: My Alexa Media player has been failing to log in too. I've just tried deleting and re-adding the integration, and even clearing all my 2FA settings, but I still get the 'Alexa Media Player failed to log in' error. I updated to HA 2024.8.3 (I was previously on 8.1) and tried again, but I still can't make it work. Finding this open issue, hopefully this is a general issue with the integration which will be fixed in a future update? Or is this only affecting isolated accounts? — Reply to this email directly, view it on GitHub <#2469 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABFWRDXL7DFW4LLQ7JKWVLDZULRUHAVCNFSM6AAAAABMWPTCBWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMRTGI3DQMRZHA . You are receiving this because you are subscribed to this thread.Message ID: @.>

The only thing "sure" (for now) you can do is disable system options, like this:

image

@LindsayReid
Copy link

LindsayReid commented Sep 2, 2024 via email

@cb12tre
Copy link

cb12tre commented Oct 1, 2024

Due to the ongoing issues I encountered after installing Home Assistant 2024.8 and 2024.9, I decided to restore an old backup of version 2024.7.4.
I can confirm that this way, Alexa Media Player works without any problems.

@philtarry
Copy link

Any progress or eta regarding this issue?

@danielbrunt57
Copy link
Collaborator

danielbrunt57 commented Oct 29, 2024

@deosrc

Removing the entry seemingly removes both the amazon.com and amazon.co.uk entries. Possibly a conflicting ID (email?) causing some strange behaviour.

I believe that could cause problems inside the code. Two config entries with the same email could be confused.
The required way to configure it is to specify the amazon domain account where the echo devices are registered. Anything else and you're playing with fire. Mine were originally registered in amazon.com but I moved a few months back to my amazon.ca account (same email) and had to remove/re-add the AMP config entry.
Perhaps you might need to reset your password for your amazon.co.uk account, and maybe the 2SV app secret?

@philtarry A lot of code has changed since August and there is still a pending MR for alexapy: !398

@deosrc
Copy link
Author

deosrc commented Oct 29, 2024

@deosrc

Removing the entry seemingly removes both the amazon.com and amazon.co.uk entries. Possibly a conflicting ID (email?) causing some strange behaviour.

I believe that could cause problems inside the code. Two config entries with the same email could be confused.

It's a while since I've looked at this but I figured that was happening. I was surprised it allowed it and didn't throw errors during the setup, but I guess it's not something you would expect people to do under normal circimstamces.

I've been running without issues for a while now and across updates of both HA and AMP, so problem seems to be solved for me. Unfortunately I'm not sure what eventually fixed it.

@electricbears
Copy link

Did anyone happen to find a way to resolve this? I know there's been a some updates to the integration since this was reported but I'm still having the issue with a UK based Amazon account. Same as others have mentioned, I can auth to amazon.com fine but obviously don't see any devices from that, amazon.co.uk won't auth at all.

@deosrc
Copy link
Author

deosrc commented Dec 19, 2024

@electricbears I'm not sure what change it was that fixed it for me but AMP is working fine on my HA with the latest version (and last few versions).

If you've not done so already, it might be worth removing the integration entirely, even uninstalling it from HACS, and then re-install and add the integration again. If you still encounter issues, there has been some updates to the logging so if you can turn on debug logging and provide the logs, it might highlight something about your setup (I'd recommend checking the logs for any sensitive data before posting).

@danielbrunt57
Copy link
Collaborator

Are you seeing this in debug logs?
static GET: https://alexa.amazon.de/api/phoenix returned 429:Too Many Requests:application/json

@philtarry
Copy link

philtarry commented Dec 23, 2024 via email

@danielbrunt57
Copy link
Collaborator

Need to see debug log trace for alexapy, authcaptureproxy and alexa_media...

@deosrc
Copy link
Author

deosrc commented Dec 24, 2024

Can anyone still experiencing this issue provide logs please? Logging has been added/improved since I opened this issue (I made some of the changes myself). Without logs it is almost impossible to help.

https://www.home-assistant.io/docs/configuration/troubleshooting/#debug-logs-and-diagnostics

If your logs do not specifically have the ContentTypeError for an amazon.co.uk account, I would suggest opening a new issue.

@danielbrunt57
Copy link
Collaborator

I'd like to add that you should also enable debug logging for authcaptureproxy in addition to alexapy and custom_components/alexa_media.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
alexapy Issue relates to the API
Projects
None yet
Development

No branches or pull requests