busio.I2C issues on ESP32-S3: LC709203F #6311
Comments
|
Could this be related? |
jepler
changed the title
|
I cannot reproduce this with import time
import bitbangio
import board
import busio
from adafruit_lc709203f import LC709203F, PackSize
uart = board.UART()
battery_monitor = LC709203F(board.I2C())
battery_monitor.pack_size = PackSize.MAH400
while True:
print("Battery Percent: {:.2f} %".format(battery_monitor.cell_percent))
print("Battery Voltage: {:.2f} V".format(battery_monitor.cell_voltage))
uart.write(b"Battery Percent: {:.2f} %\r\n".format(battery_monitor.cell_percent))
uart.write(b"Battery Voltage: {:.2f} V\r\n".format(battery_monitor.cell_voltage))
time.sleep(2) |
|
Tested further. These all work with #6366. Did not try with 7.3.0-beta.1
ESP32-S3 Feather works sometimes for a while at 10kHz I2C clock. At 100kHz, seems to fail immediately. Saleae trace with working reads, and then last is failure: |
dhalbert
changed the title
dhalbert
changed the title
|
We are also seeing issues with MCP960x, which does clock stretching and repeated start. Before #6366, I see EDIT: MCP960x problem is a sensor problem, not related to this. Fixed for MCP960x by adafruit/Adafruit_CircuitPython_MCP9600#20. |
|
For LC709203F, I am seeing bad handling of a clock stretch. The top trace is ESP-S3, the bottom trace is ESP-S2. The first four I2C transactions are the same. Then there is a clock stretch, which works on the S2, but not the S3. Note that SDA goes high after a relatively short time in the top (S3, bad) trace.
|
dhalbert
changed the title
|
I am going to ignore the S2 bitbangio issue for now. The LC709203F does clock stretching for several ms at a time, which seems to be an issue for the S3. I cannot reproduce the problem using https://github.com/adafruit/Adafruit_LC709203F/tree/master/examples/LC709203F_demo and removing the delay in |
|
I exercised this problem in a different way (causes a crash) in a simple ESP-IDF program on several S3 boards, and reported it comprehensively in espressif/esp-idf#8894 (comment). |
|
Pushing this to 7.x.x because it seems like an ESP-IDF issue. |
jepler
added
the
third-party
|
I tested with the tip of https://github.com/espressif/esp-idf/tree/release/v4.4, which is later than 4.4.2, released two weeks ago, and I am still seeing this problem, unfortunately. |
|
Tested again after #7023, which updates ESP-IDF to a very recent |
|
Just throwing my 2 cents in, I have tried two STEMMA-QT boards on my new Feather ESP32-S3 and both report the same issue as well (output for my BNO055 9DoF) import board
import adafruit_bno055
i2c = board.STEMMA_I2C()
sensor = adafruit_bno055.BNO055_I2C(i2c)Occasionally, after a reset of the Feather, it will get past board.STEMMA_I2C and fail at the adafruit_bno055: |
|
Recently I found a hack on a website (sorry - I forgot the name) which works fine in my environment: # Der following code to scan the I2C-Adresses is necessary
i2c = board.I2C()
while not i2c.try_lock():
pass
running = True
try:
while running:
print(
"I2C addresses found:",
[hex(device_address) for device_address in i2c.scan()],
)
# time.sleep(2)
running = False
finally: # unlock the i2c bus when ctrl-c'ing out of the loop
i2c.unlock()
# Create sensor object, using the board's default I2C bus.
battery_monitor = LC709203F(board.I2C())
# Update to match the mAh of your battery for more accurate readings.
# Can be MAH100, MAH200, MAH400, MAH500, MAH1000, MAH2000, MAH3000.
# Choose the closest match. Include "PackSize." before it, as shown.
battery_monitor.pack_size = PackSize.MAH2000
# print("LC709203F thermistor test")
# print("Make sure a thermistor is connected to the board!")
# print("Make sure LiPoly battery is plugged into the board!")
# check your NTC thermistor datasheet for the appropriate B-Constant
battery_monitor.thermistor_bconstant = 3950
battery_monitor.thermistor_enable = True
# print("IC version:", hex(battery_monitor.ic_version))
# print("Battery Percent: {:.2f} %".format(battery_monitor.cell_percent))
# print("Battery Voltage: {:.2f} V".format(battery_monitor.cell_voltage)) |
|
Sali @BeatArnet and many thanks! This worked for me 👏 I also formatted your example a bit: # See: https://github.com/adafruit/Adafruit_CircuitPython_LC709203F
# Ensure to have "adafruit_lc709203f" installed (circup install adafruit_lc709203f)
import time
import board
from adafruit_lc709203f import LC709203F, PackSize
BATTERY_MON_ADDRESS = 0x0B # Address for ESP32-s3 tft Feather
def hack_for_i2c_issue():
i2c = board.I2C()
while not i2c.try_lock():
pass
running = True
try:
while running:
print("I2C addresses found:",
[hex(device_address) for device_address in i2c.scan()]
)
# time.sleep(2)
running = False
return i2c
finally: # unlock the i2c bus when ctrl-c'ing out of the loop
i2c.unlock()
def main() -> None:
try:
print("Searching for battery monitor...")
# There is a known issue with the ESP32-S3:
# https://github.com/adafruit/circuitpython/issues/6311
# Therefore this will not work -> board.I2C()
i2c = hack_for_i2c_issue()
battery_monitor = LC709203F(i2c)
# Update to match the mAh of your battery for more accurate readings.
# Can be MAH100, MAH200, MAH400, MAH500, MAH1000, MAH2000, MAH3000.
# Choose the closest match. Include "PackSize." before it, as shown.
battery_monitor.pack_size = PackSize.MAH100 # type: ignore
while True:
print(f"Battery Percent: {battery_monitor.cell_percent:.2f} %")
print(f"Battery Voltage: {battery_monitor.cell_voltage:.2f} V")
time.sleep(2)
except Exception as e:
print(f"Error reading battery: {e}")
print("All done.")
if __name__ == "__main__":
main() |
|
Thanks a lot for the formatting! |
|
I have reproduced this, and am working on some changes to the https://github.com/adafruit/Adafruit_CircuitPython_LC709203F library to do retries and delays in various places to minimize the throwing of errors. It was interesting that the Arduino library seemed to work OK, but upon further inspection, it appears that is due to the Arduino library and example program largely ignoring many errors that occur intermittently (!). Those errors cause intermittent bad data values but the Arduino code simply plows ahead. I think are deifnitely issues with S3 I2C, but I think we can work around them with library changes |
|
Just for your information, I also had my share of issues with the build in I2C library. I am happy to help testing.... |
|
Working fine: from adafruit_lc709203f import LC709203F, PackSize
BATTERY_MON_ADDRESS = 0x0B # Address for ESP32-s3 tft Feather
def hack_for_i2c_issue():
i2c = board.I2C()
while not i2c.try_lock():
pass
running = True
try:
while running:
print("I2C addresses found:",
[hex(device_address) for device_address in i2c.scan()]
)
# time.sleep(2)
running = False
return i2c
finally: # unlock the i2c bus when ctrl-c'ing out of the loop
i2c.unlock()
try:
print("Searching for battery monitor...")
i2c = hack_for_i2c_issue()
battery_monitor = LC709203F(i2c)
battery_monitor.pack_size = PackSize.MAH2000 # type: ignore
print(f"Battery Percent: {battery_monitor.cell_percent:.2f} %")
print(f"Battery Voltage: {battery_monitor.cell_voltage:.2f} V")
time.sleep(1)Produces sporadic errors: from adafruit_lc709203f import LC709203F
i2c = board.I2C()
battery_monitor = LC709203F(i2c)
battery_monitor.pack_size = PackSize.MAH2000 # type: ignore |
|
@BeatArnet Thanks -- these are single-reading examples, but I'd assume you're reading in a loop? Could you post the whole program(s)? Uploading in a .zip is fine if large. |
|
btw, if you upgraded the library, make sure the only copy is in |
|
I used the library "adafruit-circuitpython-bundle-8.x-mpy-20230105.zip" |
@BeatArnet I would like to track this down to read about this technique in context. Do you remember if the author did the original coding in CircuitPython, MicroPython, regular Python (e.g., on RPi), or Arduino, or something else? Thanks. |
|
Sorry, I couldn't find the author of the code. But I did not even have to adapt the code.
|
|
Are you saying that the code you took is exactly as you wrote in #6311 (comment) ? If so, then I will do a websearch for that. |
|
Same code, but I did not find on the Web…
|
|
Came here from Google, brand new ESP32-S3 here, just hit this problem. Should I just wrap the call in a try/catch? Would that work? |
|
I would suggest to people that they attempt to add an additional pull-up resistor in order to solve this issue. I've used my analog scope to monitor the I2C bus in the default configuration and the rise time for the I2C signals are very much out of spec (rise time is 2.5us & spec max is 1us). While the documentation for the FeatherS3 indicates a 5k pull-up, looking at the schematic seems to show that it's actually 10k (though I could be reading it wrong). In any case, the I2C bus behaves much more reliably if you add a 2k resistor between the 3.3V pin and each of the I2C pins. |
|
@cherrydev Interesting. The LC709203F datasheet shows 10k as an example, so theoretically it should be OK. It works fine on ESP32-S2, but perhaps there is something hw or sw-related on S3 that causes the slow rise time. Have you looked at the rise time on an S2 board? |
|
@dhalbert I do not have an S2 board to test it on. In the past, though, I've found a lot of I2C problems to go away with stronger pull-up, so it might be worth a try. While 10k might theoretically be fine, it also depends on details of the host's I2C circuit, including capacitance, which I only know how to measure with the board unpowered, which might not give an accurate measurement due to the ESP's GPIO MUX. I can test an original ESP32 using a 10k pullup and see if I get similarly slow rise times. |
|
Got an error on the S3 using the LC709203F, might be related?
Error in my script is currently line 407 in this script As suggested by anecdata I've now added into my S3 weather station script a try/except around it. i2c = board.I2C()
battery_monitor = LC709203F(board.I2C())
battery_monitor.thermistor_bconstant = 3950
battery_monitor.thermistor_enable = True
try:
vbat_label.text = f"{battery_monitor.cell_voltage:.2f}"
except OSError as e:
print("OSError:", e)It generally takes about 10-12 hours for the read failure to occur while polling every 15 minutes. I'm not using any power mode switching code, just straight to DisplayIO TFT. Will let this run for a while and report back. |
|
@DJDevon3 Yes, this seems like the same issue. |
|
Ran it with the try/except for 24 hours successfully. The device rebooted when I unplugged my multi-card reader. Windows must have gone around and touched every other USB device because the S3 weather station rebooted. That reset the clock back to 0 for me in looking into this issue. 24 hours without a single error is definitely an improvement as before it was about 10-12 hours reliably failing two days in a row. Granted I've only had the feather weather station up and running for 2 days since having it shutoff for months. There is nothing in REPL showing the OSError print. Seems like a simple try/except might be a good workaround or I simply didn't run into it for the past 24 hours? ===============================
Failed to get data, retrying
Sending request failed
USB Sense: True
Attempting to GET Weather!
===============================The only error I got over the past 24 hours was a wifi failure which failed gracefully, recovered, and reconnected. Not a single OSError so I don't think it even triggered. If it shows up it'll be directly below USB Sense: True. Not a single entry in my logs. Because I run this every single day right in front of my main monitor, polling every 15 minutes, I'll definitely pickup any errors. Keeping Mu open to track the serial output. Will keep an eye on it this week and report back if I have more data to contribute. |
|
Running 5 days straight without a single error using try/except. try:
vbat_label.text = f"{battery_monitor.cell_voltage:.2f}"
except OSError as e:
print("OSError:", e)That's all that is required now as far as I can tell. |
|
I retested with CircuitPython 9.0.0-alpha.2, which is upgraded to ESP-IDF v5.1. |
|
Sorry to bring this back up. I am getting OSError(s) from this device in the latest Circuit Python.
I am seeing 1-2 errors per minute if I read at 1hz from the device. I added retry code to the read and write functions in the library and that mostly fixes it, but I occasionally see >10 errors when reading, which causes a crash. Do I need to update something to help with this? |
|
@ilikecake could you post a simple test program that causes the errors? Why do ten errors in a row cause a crash? |
|
Sorry, I should have been more specific, the code crashes after 10 errors because I told it to retry 10 times and then raise the error if that doesn't fix it. When looking at this a bit more, I am actually reading from the device once ever .2 seconds, or 5Hz. This is still not that fast, but might be faster than the device is capable of? I slowed it down to 1Hz, and the errors still occur, but less often. I still see the errors once every few min. A pared down version of the code that causes the errors is: #Test code to make the LC709203F device cause errors.
import board
import time
from adafruit_lc709203f import LC709203F, PackSize
#Initialize Hardware
i2c = board.STEMMA_I2C()
battery_monitor = LC709203F(i2c)
battery_monitor.pack_size = PackSize.MAH2000
now = time.localtime()
OldMin = now.tm_min
while True:
now = time.localtime()
pct = battery_monitor.cell_percent
volt = battery_monitor.cell_voltage
if now.tm_min != OldMin:
OldMin = now.tm_min
print("Battery: ", pct, "% at ", volt, "V")
time.sleep(.2)I modified the read and write functions in the library as below:
def _read_word(self, command: int) -> int:
self._buf[0] = LC709203F_I2CADDR_DEFAULT * 2 # write byte
self._buf[1] = command # command / register
self._buf[2] = self._buf[0] | 0x1 # read byte
for x in range(LC709203F_I2C_RETRY_COUNT):
try:
with self.i2c_device as i2c:
i2c.write_then_readinto(
self._buf,
self._buf,
out_start=1,
out_end=2,
in_start=3,
in_end=7,
)
crc8 = self._generate_crc(self._buf[0:5])
if crc8 != self._buf[5]:
raise OSError("CRC failure on reading word")
return (self._buf[4] << 8) | self._buf[3]
except OSError as exception:
print("OSError in read: ", x, "/10: ", exception)
if x == (LC709203F_I2C_RETRY_COUNT - 1):
# Retry count reached
raise exception
# Code should never reach this point
return None
def _write_word(self, command: int, data: int) -> None:
self._buf[0] = LC709203F_I2CADDR_DEFAULT * 2 # write byte
self._buf[1] = command # command / register
self._buf[2] = data & 0xFF
self._buf[3] = (data >> 8) & 0xFF
self._buf[4] = self._generate_crc(self._buf[0:4])
for x in range(LC709203F_I2C_RETRY_COUNT):
try:
with self.i2c_device as i2c:
i2c.write(self._buf[1:5])
return
except OSError as exception:
print("OSError in write: ", x, "/10: ", exception)
if x == (LC709203F_I2C_RETRY_COUNT - 1):
# Retry count reached
raise exception |
|
Your retry logic is retrying as fast as possible instead of delaying between retries. Previous testing in this issue has mostly been using the the library and its example program, I think: https://github.com/adafruit/Adafruit_CircuitPython_LC709203F/blob/main/examples/lc709203f_simpletest.py. That does a single try, and waits one second before trying again. |
|
True. In the case of my program (a more complicated version of what I shared above), I have a bunch of other things going on and I don't want the sensor read functions to block for 1 second or it will mess up the timing of the rest of the code. It appears that adding in the 10 retries as I have shown above catches most issues. I coupled that with a try/except in the @property
def cell_voltage(self) -> float:
"""Returns floating point voltage"""
try:
return self._read_word(LC709203F_CMD_CELLVOLTAGE) / 1000
except OSError:
return None
@property
def cell_percent(self) -> float:
"""Returns percentage of cell capacity"""
try:
return self._read_word(LC709203F_CMD_CELLITE) / 10
except OSError:
return NoneThis should mean that if 10 retries are not enough, the functions will return I will let the code run for a while with the latest fixes and see if it still crashes after a while. TBH, I am probably making it much worse by sampling as fast as I am, but it does appear to have issues, even at slower sample rates. |
CircuitPython version
Code/REPL
Behavior
Fails on line 9 with the following:
Description
frequencyandtimeouthave no effect on the behavior. It was already crashing consistently after a short period of time withbusioandboard.I2C(), or straight up not seeing the chip (even though I2C scan sees it). So we wanted to try bitbang I2C.bitbangiocrashed more quickly thanbusiodid.After chatting with @dhalbert about it, as per request, I am filing a bug here.
Additional information
No response
The text was updated successfully, but these errors were encountered: