Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

authentication error #121

Open
vzlgdu opened this issue May 15, 2023 · 89 comments
Open

authentication error #121

vzlgdu opened this issue May 15, 2023 · 89 comments

Comments

@vzlgdu
Copy link

vzlgdu commented May 15, 2023

Hi, starting from this morning I'm receiving authentication errore in my python scritp. It's running with no problem starting from many months ago...

What's happening?

Thanks

Guido

@twrecked
Copy link
Owner

Which version are you on?

@vzlgdu
Copy link
Author

vzlgdu commented May 15, 2023

0.8.0a5

@twrecked
Copy link
Owner

Can you try the latest release. Arlo changed their back end and broke our login code.

@vzlgdu
Copy link
Author

vzlgdu commented May 15, 2023

With the latest release I receive this errors:

body-error=JSONDecodeError
body-error=JSONDecodeError
body-error=JSONDecodeError
authentication failed

@twrecked
Copy link
Owner

Can you try these 2 things:

  • Add mqtt_hostname_check=False to your PyArlo options.
  • Remove your aarlo pickle files.

If that doesn't work can you turn on verbose debug, I need more information on where it's failing.

@njaouen
Copy link

njaouen commented May 18, 2023

Hi vzlgdu,
Any update on your side ?

@m0urs
Copy link

m0urs commented May 18, 2023

Seems that I do have the same issue here also. Can no longer login with latest code. Enclosed my encrypted debug.log. It would be great if you have any chance to look at it and maybe find out what the issue could be. Let me know if I should test something. Thanks a lot. P.S: I already set "mqtt_hostname_check=False" but with no success.

Looks a bit like a Cloudflare issue? Think we had something similar also in the past?

debug.log

@twrecked
Copy link
Owner

It's definitely a cloud flare thing. If you leave for a few hours it might recover. I think the old code made cloudflare angry and they blocked it temporarily.

@twrecked
Copy link
Owner

You connection is stopping straight away. A couple of things to try:

  • change the user agent, try using user_agent='linux'
  • try logging in from a website using the pyaarlo account

I did notice I'm sending in an extra header - the transaction id one - but I don't think that is causing the issue, I can still connect from here with it.

@vzlgdu
Copy link
Author

vzlgdu commented May 22, 2023

Hi, sorry for the delay... with your suggest (mqtt_hostname_check=False) it was ok for some days. Today I received the same error.

Adding "user_agent='linux'" the result is the same.

Accessing from web interface doesn't work and ends with "timeout request"

Accessing from ios app everything work.

@m0urs
Copy link

m0urs commented May 22, 2023

Even after waiting some days I am still unable to login from pyarlo. I can login via browser from the same IP address. Maybe we need to set some more values toegether with the request. Here is a sample what my browser is sending:

:authority: ocapi-app.arlo.com
:method: POST
:path: /api/startAuth
:scheme: https
accept: application/json, text/plain, /
accept-encoding: gzip, deflate, br
accept-language: de
authorization:XXXXXXXXXXXXXXXXXXXXXXX
content-length: 184
content-type: application/json
cookie: _XXXXXXXXXXXXXXXXXXXXXXXXXXX
dnt: 1
origin: https://my.arlo.com
referer: https://my.arlo.com/
sec-ch-ua: "Microsoft Edge";v="113", "Chromium";v="113", "Not-A.Brand";v="24"
sec-ch-ua-mobile: ?0
sec-ch-ua-platform: "Windows"
sec-fetch-dest: empty
sec-fetch-mode: cors
sec-fetch-site: same-site
source: arloCamWeb
user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36 Edg/113.0.1774.42
x-user-device-automation-name: QlJPV1NFUg==
x-user-device-id: 615cdfd1-0a03-4a91-91b3-12d21aeba713
x-user-device-type: BROWSER

What about these ones:

sec-ch-ua: "Microsoft Edge";v="113", "Chromium";v="113", "Not-A.Brand";v="24"
sec-ch-ua-mobile: ?0
sec-ch-ua-platform: "Windows"
sec-fetch-dest: empty
sec-fetch-mode: cors
sec-fetch-site: same-site

@vzlgdu
Copy link
Author

vzlgdu commented May 23, 2023

Hi, I'm trying some test (now accessing using amail as 2fa) and I'm receiving this error:

/usr/local/lib/python3.7/dist-packages/urllib3/connection.py:362: SubjectAltNameWarning: Certificate for myapi.arlo.com has no subjectAltName, falling back to check for a commonName for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See urllib3/urllib3#497 for details.)
SubjectAltNameWarning
request-error=ValueError
session start failed
body-error=JSONDecodeError
body-error=JSONDecodeError
body-error=JSONDecodeError
authentication failed

I'm using this in my script:

import ssl
ssl.SSLContext.verify_mode = ssl.VerifyMode.CERT_OPTIONAL

This are my logon string:

arlo = pyaarlo.PyArlo( username='xxx',password='yyy',
tfa_type='email',tfa_source='imap',
synchronous_mode=True,save_state=False,
tfa_host='imap.gmail.com',
tfa_username='xxx',
tfa_password='zzz')

Thanks

@m0urs
Copy link

m0urs commented May 30, 2023

Do we have any new information? Is it working again for somebody? I still have the same issue with Cloudflare although I did not try it now for severral days. So it does not seem to be reset by its own :-(

@twrecked
Copy link
Owner

Sorry, it's working for me here - both standalone and on home assistant.

Did you remove the session files? Or are you still stuck at the cloud flare piece?

@m0urs
Copy link

m0urs commented May 30, 2023

Yes, the error seems still to be the Cloudflare error as far as I can see in the debug file. Where should I find the session / pickle files? Did not find anything which looks like that? And, @vzlgdu , is it working again for you?

@twrecked
Copy link
Owner

Look for a file with this name session.pickle. It's where we store the authentication credentials.

@m0urs
Copy link

m0urs commented May 30, 2023

There is no such file. Maybe because I am not currently authenticated?

 $ grep -ir "session.pickle" .
./pyaarlo/cfg.py:        return self.storage_dir + "/session.pickle"
Binary file ./pyaarlo/__pycache__/cfg.cpython-38.pyc matches
Binary file ./pyaarlo/backup/__pycache__/cfg.cpython-38.pyc matches

@twrecked
Copy link
Owner

So. A few more things to try/check:

  • Are you on the latest cloudscraper release?
pip3 install --update cloudscaper
  • You could try a custom user_agent. Add this to the PyArlo, note the ! at the front. (I copied this from above, make sure it's correct.)
  user_agent='!Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36 Edg/113.0.1774.42'

@ninjafella
Copy link

Hello there,

I'm having the same issue of having cloudflare blocking me. I've tried using different 'user_agents' and making sure my cloudscraper was up to date but no luck. I've attached my logs bellow.

Whats perculiar is that home assistant using aarlo can connect fine and i'm having no issues with it on there.

arlo_debug.log

@m0urs
Copy link

m0urs commented May 31, 2023

 user_agent='!Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36 Edg/113.0.1774.42'

I updated cloudscraper and tried this user agent, but with no luck :-( I am using pyarlo with my own script, not sure if there is an issue with my script using pyarlo (although it worked fine since some weeks ago): https://github.com/m0urs/arlo-fhem/blob/master/arlo-fhem.py

@shissam
Copy link

shissam commented May 31, 2023

 user_agent='!Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36 Edg/113.0.1774.42'

I updated cloudscraper and tried this user agent, but with no luck :-( I am using pyarlo with my own script, not sure if there is an issue with my script using pyarlo (although it worked fine since some weeks ago): https://github.com/m0urs/arlo-fhem/blob/master/arlo-fhem.py

overnight - my script continued to fail for approx 7 hours, after manual intervention (removed session.pickle and aarlo.pickle) it failed once, and then was successful.

I was comparing my script to your script and noted that ours are pretty much the same (args to PyArlo specifically using the sse backend and my agent is still linux). There is one other thing that my system has... in /etc/hosts (based on recommendations from past issues dealing with cloudfare) i have this entry:

104.18.30.98 ocapi-app.arlo.com
#104.18.31.98 ocapi-app.arlo.com

also cloudscraper is

pip list|grep -i scrap
cloudscraper      1.2.58 

@m0urs
Copy link

m0urs commented May 31, 2023

"ocapi-app.arlo.com" is already resolved here to 104.18.30.98 so an additional hosts entry should not do anything else. My cloudscraper is now updated to 1.2.71. I think it was 1.2.58 before updating this morning.

However I still have no pickle files. Where should these files be created??

@shissam
Copy link

shissam commented May 31, 2023

"ocapi-app.arlo.com" is already resolved here to 104.18.30.98 so an additional hosts entry should not do anything else. My cloudscraper is now updated to 1.2.71. I think it was 1.2.58 before updating this morning.

However I still have no pickle files. Where should these files be created??

the command you want to run to manually find those pickle files (and remove them) is:

sudo find / -mount -iname \*.pickle -type f

these files are created in a default location relative to the current working dir for the process unless specified on the PyArlo call (see storage_dir=)

@shissam
Copy link

shissam commented May 31, 2023

"ocapi-app.arlo.com" is already resolved here to 104.18.30.98 so an additional hosts entry should not do anything else. My cloudscraper is now updated to 1.2.71. I think it was 1.2.58 before updating this morning.
However I still have no pickle files. Where should these files be created??

the command you want to run to manually find those pickle files (and remove them) is:

sudo find / -mount -iname \*.pickle -type f

these files are created in a default location relative to the current working dir for the process unless specified on the PyArlo call (see storage_dir=)

to be clear remove both pickle files, mine are:

~/iot/aarlo/session.pickle
~/iot/aarlo/aarlo.pickle

@m0urs
Copy link

m0urs commented May 31, 2023

I am unable to find these files ... so I guess it will not be created as Cloudflare does not let me go so far and refuses with the error before?

I did a downgrade to cloudscraper 1.2.58 but this did not change anything.

I still get this Cloudflare issue:

You do not have access to ocapi-app.arlo.com.

The site owner may have set restrictions that prevent you from accessing the site.

...

Error code: 1020

Ray ID: 7cff7a4e2b7a37d8

Country: DE

Data center: fra12

IP: 176.199.xx.yy

Timestamp: 2023-05-31 13:14:03 UTC

@twrecked
Copy link
Owner

Also look in /tmp/.aarlo but I suspect you are seeing other problems.

@m0urs
Copy link

m0urs commented May 31, 2023

Also look in /tmp/.aarlo but I suspect you are seeing other problems.

Thanks, that was the right location :-) I did delete both files there but still no login possible. And the files are NOT recreated ....

@shissam
Copy link

shissam commented May 31, 2023

in my world... those files persist between logins, so i'd expect those files would be around from the last successful login. for those files to not be on your system anywhere would lead me to think there was never a successful login -- but that cannot certainly be the case for you. so "strange"

one other thought would be to move back to the mqtt backend - but perhaps that would not make a difference.

if it were me, i'd first try the mqtt backend, if that failed, i'd jump to another ocapi-app.arlo.com, and if that failed, I'd remove the ocapi-app.arlo.com completely from my /etc/hosts.

i'll also say this - i had this same problem on 18 MAY, and my resolution on that day was to git pull from pyaarlo which brought me up to the current version of pyaarlo. So far, in all cases, removing those pickle files was the common denominator in resolving my issues then and this morning.

@shissam
Copy link

shissam commented Jul 5, 2023

Does adding the ecdhCurve parameter get you legacy app to work outside of the HA container!

I'll give it a try here in an hour or two - and report back, my suspicion is unlikely - but I have been surprised way too many times to rule it out. The cloudscraper README.md reports

--
# Some servers require the use of a more complex ecdh curve than the default "prime256v1"
# It may can solve handshake failure
scraper = cloudscraper.create_scraper(ecdhCurve='secp384r1')
```python

But there did not appear to be any code in `cloudscraper` itself actually employing that tactics on its own.

@shissam
Copy link

shissam commented Jul 5, 2023

Does adding the ecdhCurve parameter get you legacy app to work outside of the HA container!

I'll give it a try here in an hour or two - and report back, my suspicion is unlikely - but I have been surprised way too many times to rule it out. The cloudscraper README.md reports

--
# Some servers require the use of a more complex ecdh curve than the default "prime256v1"
# It may can solve handshake failure
scraper = cloudscraper.create_scraper(ecdhCurve='secp384r1')
```python

But there did not appear to be any code in `cloudscraper` itself actually employing that tactics on its own.

@m0urs, uuuuugggggghhhhh!, no it made no difference:

$ python3 arlo-mon2FA.py 
init login config
reading login config
grabing secrets
reading /home/pi/.ssh/arlo-mon2FA.secrets
logging in
2023-07-05 13:49:42,199 - pyaarlo - INFO - pyarlo 0.8.0b9 starting...
2023-07-05 13:49:42,331 - pyaarlo - WARNING - body-error=JSONDecodeError
2023-07-05 13:49:43,355 - pyaarlo - WARNING - body-error=JSONDecodeError
2023-07-05 13:49:44,387 - pyaarlo - WARNING - body-error=JSONDecodeError
2023-07-05 13:49:45,389 - pyaarlo - ERROR - authentication failed
Error: 'PyArlo' object has no attribute '_bases'
monitoring bases
not successfully connected
$ grep cloudscraper.create_scraper ../pyaarlo/pyaarlo/backend.py 
            self._session = cloudscraper.create_scraper(ecdhCurve='secp384r1')
$ ls -ltr ../pyaarlo/pyaarlo/__pycache__/backend.cpython-37.pyc 
-rw-r--r-- 1 pi pi 27351 Jul  5 13:49 ../pyaarlo/pyaarlo/__pycache__/backend.cpython-37.pyc

@twrecked
Copy link
Owner

twrecked commented Jul 5, 2023

@shissam I also found out the delay option in cloudscraper doesn't do anything...

But I think you are onto something with the versioning, my pyaarlo works for me and these are my versions.

** python is version 3.10
** cloudscraper=1.2.71
** urllib3=2.0.3
** requests=2.31.0
** requests-toolbelt=1.0.0

This is Ubuntu 22..04. I have some PIs somewhere around here, I'll try one of them later.

@shissam
Copy link

shissam commented Jul 5, 2023

@shissam I also found out the delay option in cloudscraper doesn't do anything...

But I think you are onto something with the versioning, my pyaarlo works for me and these are my versions.

** python is version 3.10 ** cloudscraper=1.2.71 ** urllib3=2.0.3 ** requests=2.31.0 ** requests-toolbelt=1.0.0

This is Ubuntu 22..04. I have some PIs somewhere around here, I'll try one of them later.

@twrecked I got lazy. I created a python venv under/on my host raspbian OS using Python 3.7.3.

I simply (re-)cloned pyaarlo 0.8.0b9 in the venv working folder, upgraded pip, and ran my python pyaarlo app while using pip install repeatedly to resolve packages and specifying the same version of the package as stated in the HA container env.

It worked. I believe these steps can reproduce the env:

Commands used to set up the venv:

$ mkdir pyaarlo-dbg
$ python3 -m venv pyaarlo-dbg/asis-venv
$ source pyaarlo-dbg/asis-venv/bin/activate
$ ~/pyaarlo-dbg/asis-venv/bin/python3 -m pip install --upgrade pip
$ ~/pyaarlo-dbg/asis-venv/bin/python3 -m pip install 'cloudscraper==1.2.71'
$ ~/pyaarlo-dbg/asis-venv/bin/python3 -m pip install 'paho-mqtt==1.6.1'
$ ~/pyaarlo-dbg/asis-venv/bin/python3 -m pip install unidecode
$ ~/pyaarlo-dbg/asis-venv/bin/python3 -m pip install 'cryptography==40.0.2'

Here are the packages installed in the venv and the corresponding version from the Host OS:

                   Venv       Host (raspbian)
Package            Version    Version
------------------ --------   -------
certifi            2023.5.7   2018.8.24
cffi               1.15.1     1.15.0
charset-normalizer 3.1.0      (not listed by pip3)
cloudscraper       1.2.71     1.2.71
cryptography       40.0.2     2.6.1
idna               3.4        2.6
paho-mqtt          1.6.1      1.6.1
pip                23.1.2     18.1
pkg_resources      0.0.0      (not listed by pip3)
pycparser          2.21       2.21
pyparsing          3.1.0      3.0.7
requests           2.31.0     2.21.0
requests-toolbelt  1.0.0      0.9.1
setuptools         40.8.0     40.8.0
Unidecode          1.3.6      1.3.2
urllib3            2.0.3      1.24

One or some combinations of these may be the culprit. That is unless this mixture is sensitive to python 3.7.3 which I am still using on the Host OS

@twrecked
Copy link
Owner

twrecked commented Jul 5, 2023

Thanks for trying that. cloudscraper relies on requests. And requests seems to use urllib3. So that's 2 to try. But I could also see cryptography causing issues.

I could add some versions to the requirements.txt file, that might help, right now there are non specified.

@shissam
Copy link

shissam commented Jul 5, 2023

Thanks for trying that. cloudscraper relies on requests. And requests seems to use urllib3. So that's 2 to try. But I could also see cryptography causing issues.

I could add some versions to the requirements.txt file, that might help, right now there are non specified.

i am by far "not" a python expert (nor do I play one on TV) - but there is one thing that is nagging me - say, if I upgrade from cloudscraper 1.2.58 to 1.2.71 (for example) would a pip3 upgrade, upgrade all the sub-dependencies? I could swear in recent weeks I upgraded cloudscraper from 1.2.58 to 1.2.71 and in that process, those old version number were there and are still there (as shown in that table).

i'll see about upgrading those sub-dependencies one by one and see which one gets me back - I'll have to get to that in the next day or so and i'll report back if no one else does.

@m0urs
Copy link

m0urs commented Jul 6, 2023

Ok, as I had the same versions as @twrecked I just upgraded all to the most current version, just to give it a try. That is what I now have installed:

Python 3.10.12

Package            Version
------------------ --------
asn1crypto         1.5.1
certifi            2023.5.7
cffi               1.15.1
chardet            5.1.0
charset-normalizer 3.1.0
click              8.1.3
cloudscraper       1.2.71
cryptography       41.0.1
Cython             0.29.36
gevent             22.10.2
greenlet           2.0.2
idna               3.4
mock               5.0.2
paho-mqtt          1.6.1
pip                23.1.2
pycparser          2.21
pycryptodome       3.18.0
pyparsing          3.0.9
pysam              0.21.0
python-dateutil    2.8.2
requests           2.31.0
requests-toolbelt  1.0.0
setuptools         68.0.0
six                1.16.0
Unidecode          1.3.6
urllib3            2.0.3
websocket          0.2.1
wheel              0.40.0
zipp               3.15.0
zope.event         5.0
zope.interface     6.0

However @shissam:

I CANNOT connect if I do NOT add the parameter "ecdhCurve='secp384r1'" to the clouscraper call. I tried it several times. Without that parameter, the login fails with the known Cloudflare message, with that parameter the login works without a problem ...

But unfortunately, I still cannot set my modes for the cameras anymore :-( So hopefully @twrecked you would be able to find out what is going wrong here? I think we had a similar issue some time ago already.

@shissam
Copy link

shissam commented Jul 6, 2023

Ok, as I had the same versions as @twrecked I just upgraded all to the most current version, just to give it a try. That is what I now have installed:

Python 3.10.12

Package            Version
------------------ --------
asn1crypto         1.5.1
certifi            2023.5.7
cffi               1.15.1
chardet            5.1.0
charset-normalizer 3.1.0
click              8.1.3
cloudscraper       1.2.71
cryptography       41.0.1
Cython             0.29.36
gevent             22.10.2
greenlet           2.0.2
idna               3.4
mock               5.0.2
paho-mqtt          1.6.1
pip                23.1.2
pycparser          2.21
pycryptodome       3.18.0
pyparsing          3.0.9
pysam              0.21.0
python-dateutil    2.8.2
requests           2.31.0
requests-toolbelt  1.0.0
setuptools         68.0.0
six                1.16.0
Unidecode          1.3.6
urllib3            2.0.3
websocket          0.2.1
wheel              0.40.0
zipp               3.15.0
zope.event         5.0
zope.interface     6.0

However @shissam:

I CANNOT connect if I do NOT add the parameter "ecdhCurve='secp384r1'" to the clouscraper call. I tried it several times. Without that parameter, the login fails with the known Cloudflare message, with that parameter the login works without a problem ...

But unfortunately, I still cannot set my modes for the cameras anymore :-( So hopefully @twrecked you would be able to find out what is going wrong here? I think we had a similar issue some time ago already.

@m0urs - I too am unable to set base mode - this error is mentioned above in this issue report as well as over on the HA issue thread at twrecked/hass-aarlo#767 this is what I am seeing when I set base.mode = 'disarmed'

base: name=(redacted),state=available,mode=Armed Day Home,sched=True,schednam=schedule.1
['disarmed', 'armed', 'armed day home', 'armed home with driveway alert', 'armed night home', 'schedule.1']
disarming...
2023-07-05 16:03:41,217 - pyaarlo - WARNING - attempt 1: error in response when setting mode=
{'data': {'error': 'AUTO-5050',
          'message': 'Failed to update Automation Definitions',
          'reason': 'Unable to upload active automation. Please try again'},
 'success': False}

@m0urs
Copy link

m0urs commented Jul 6, 2023

YEs, that is the same error I see here. I uploaded a debug log under #121 (comment) So it seems that is a bigger problem :-(

@twrecked
Copy link
Owner

twrecked commented Jul 6, 2023

I'm seeing that as well. I'll take a look.

@twrecked
Copy link
Owner

twrecked commented Jul 6, 2023

I'll push a fix later, I have to head into work, but for now, un-comment the "SchemaVersion": "1" line.

I took it out to make sure the headers matched for the sse testing and forgot to add it back.

@herveaurel
Copy link

I'll push a fix later, I have to head into work, but for now, un-comment the "SchemaVersion": "1" line.

I took it out to make sure the headers matched for the sse testing and forgot to add it back.

Yes THKS !!!! ✌🏻🎉

@m0urs
Copy link

m0urs commented Jul 6, 2023

I'll push a fix later, I have to head into work, but for now, un-comment the "SchemaVersion": "1"

Thank you so much! Works again :-)

@shissam
Copy link

shissam commented Jul 6, 2023

I'll push a fix later, I have to head into work, but for now, un-comment the "SchemaVersion": "1" line.

I took it out to make sure the headers matched for the sse testing and forgot to add it back.

yep - did the complete git pull and my redeploy ... all appears good and mode setting is back!

@twrecked thank you for all you do!

@bjia56
Copy link

bjia56 commented Jul 6, 2023

@twrecked fwiw, on the Scrypted Arlo plugin side, I am working on a Cloudflare fix using this package: https://pypi.org/project/curl-cffi

From my personal testing, Cloudflare is picky about the initial login connection, but once a session token has been returned, further API calls are less sensitive and seem to go through ok. I'm doing initial testing with using curl-cffi only for the initial auth request, then switching to cloudscraper for future API and SSE connections using the token returned from the auth. Preliminary results are promising.

Unfortunately there are no 32 bit raspberry pi builds of curl-cffi, so not sure if that's a concern on the HA side.

@twrecked
Copy link
Owner

twrecked commented Jul 6, 2023

@bjia56 Thanks, I'll definitely take a look at that package. Might be nice to offer it as an option.

And pyaarlo already uses a non cloudscraper socket if it detects a valid cached authentication token (which are good for 14 days) and for the sse connection. In fact, sse doesn't seem compatible with cloudscraper, it just hangs.

It easy to modify pyaalo to ditch the cloudscraper after authentication as well, so I might try that.

@shissam
Copy link

shissam commented Jul 18, 2023

@twrecked as noted above, pyaarlo "0.8.0b10 worked (and was working since July 6 until this morning Jul 18) -- I did notice that hass-arlo's issue twrecked/hass-aarlo#778 were also noting issues requiring @m0urs detected change "ecdhCurve='secp384r1'" and removing "Source": "arloCamWeb", from the header in backend.py

I made those two changes this morning and I am back in again. I can only suspect that changes on cloudflare may not necessarily roll out to all their zones/servers at the same time and I may have been "benefiting" from older servers. After seeing the hass-arlo reports, I suspected it was only a matter of time before I had to make those changes.

diff --git a/pyaarlo/backend.py b/pyaarlo/backend.py
index 9561f7b..221ef03 100644
--- a/pyaarlo/backend.py
+++ b/pyaarlo/backend.py
@@ -688,7 +688,7 @@ class ArloBackEnd(object):
             # "Sec-Fetch-Dest": "empty",
             # "Sec-Fetch-Mode": "cors",
             # "Sec-Fetch-Site": "same-site",
-            "Source": "arloCamWeb",
+            # "Source": "arloCamWeb",
             "User-Agent": self._user_agent,
             "X-User-Device-Automation-name": "QlJPV1NFUg==",
             "X-User-Device-Id": self._user_device_id,
@@ -896,7 +896,8 @@ class ArloBackEnd(object):
         # If token looks invalid we'll try the whole process.
         get_new_session = days_until(self._expires_in) < 2
         if get_new_session:
-            self._session = cloudscraper.create_scraper()
+            # self._session = cloudscraper.create_scraper()
+            self._session = cloudscraper.create_scraper(ecdhCurve='secp384r1')
             self.debug("oldish session, getting a new one")
             if not self._auth():
                 return False

I should’ve re-tried without removing "Source": "arloCamWeb”,— but I am using everyone else’s success.

@twrecked
Copy link
Owner

twrecked commented Aug 3, 2023

And I've just update and checked in that success :)

Can somebody double check and make sure the master branch is working ok?

@shissam
Copy link

shissam commented Aug 3, 2023

@twrecked recent pull for pyarlo 0.8.0b11 is working as I expected for these changes. Thank you!

@tdeckers
Copy link

tdeckers commented Aug 3, 2023

Confirmed.. that latest version iss now working fine. Thanks!

@vinnygambiny
Copy link

I try the 0.8.0b11 and it's not working for me 😞

@shissam
Copy link

shissam commented Sep 6, 2023

@twrecked recent pull for pyarlo 0.8.0b11 is working as I expected for these changes. Thank you!

@twrecked so, it happened again JSON decode errors. I even tried a most recent pull of pyaarlo this morning (commit 4a0763fc...) still got pyaarlo - WARNING - body-error=JSONDecodeError (really no surprise there given the last change).

the fix for me today was that of @Molnigt over at hass-arlo #778 where cloudscraper.create_scraper() was modified to remove all customizations (i.e., no args for ecdhCurve ).

call this a silly hypothesis, but I am continuing to suspect that cloudflare continues to run different productions in places.

have fun!

@meavydev
Copy link

It just failed for me. It worked 1.5 hours ago and now it isn't (latest b11 build).
With debugging on I see access token revoked and then the JSONDecodeError.

@twrecked
Copy link
Owner

@shissam I agree with your hypothesis, and it's part of the reason I'm loathed to keep changing this code about. Be we do seem to have a few simple things to try:

  • change the create_scrapper parameter
  • change the Source: line
  • remove the session file

If all that fails wait before trying again.

I'll also take my monthly look at the headers in Chrome to see if the web requests have changed.

@meavydev
Copy link

It is working again for me, so thanks.

@meavydev
Copy link

Blimey... short lived working.
Back to JSON Decode error with 0.8.0.A14.

@meavydev
Copy link

meavydev commented Nov 4, 2023

Ah my fault.
I'm not a Python expert and hadn't realised that pip was updating one location and then the webapp was using a different one.
Now both are set to 0.8.0.1 and it seems to be working again, so hopefully it will stay working this time...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests