Cannot effectively scan with OSX, RasPi, or Feather Sense

[alexwhittemore]

This might be related to #11 but I can't say for sure - the symptoms don't totally overlap. The gist is, ble.start_scan(adafruit_ble_broadcastnet.AdafruitSensorMeasurement) will never return any advertisements to process, tested on Mac and RasPi. If, on the other hand, I ble.start_scan(), plenty of the relevant ads ARE getting scanned.

I'm running the following code on a Feather Sense:

import time
import microcontroller
import adafruit_ble_broadcastnet

from adafruit_ble import BLERadio
from adafruit_ble.advertising.adafruit import AdafruitColor
import random

ble = BLERadio()

print("This is BroadcastNet sensor:", adafruit_ble_broadcastnet.device_address)

while True:
   measurement = adafruit_ble_broadcastnet.AdafruitSensorMeasurement()
   measurement.temperature = (
      microcontroller.cpu.temperature  # pylint: disable=no-member
   )
   print(measurement)
   print(measurement.__repr__())
   #adafruit_ble_broadcastnet.broadcast(measurement, broadcast_time=3)
   ble.start_advertising(measurement)
   time.sleep(10)
   ble.stop_advertising()

On the RasPi, I'm using the following to scan:

import time
from adafruit_ble.advertising.standard import ManufacturerDataField
import adafruit_ble
import adafruit_ble_broadcastnet



ble = adafruit_ble.BLERadio()
bridge_address = adafruit_ble_broadcastnet.device_address
print("This is BroadcastNet bridge:", bridge_address)
print()

print("scanning")
print()
sequence_numbers = {}
# By providing Advertisement as well we include everything, not just specific advertisements.

# ====== Switch between these two lines for A/B testing ======
# for measurement in ble.start_scan(adafruit_ble_broadcastnet.AdafruitSensorMeasurement, interval=0.5):
for measurement in ble.start_scan(interval=0.5): 
    reversed_address = [measurement.address.address_bytes[i] for i in range(5, -1, -1)]
    sensor_address = "{:02x}{:02x}{:02x}{:02x}{:02x}{:02x}".format(*reversed_address)
    if sensor_address == "f5bfcae0c159": ## On RasPi, the self-identified MAC address gets returned, so this looks sane.
        print(measurement.__repr__())

print("scan done")

On Mac, I'm using the following to implement the same test:

"""This example bridges from BLE to Adafruit IO on a Raspberry Pi, but without actually interacting with AIO at all"""
import time
from adafruit_ble.advertising.standard import ManufacturerDataField
import adafruit_ble
import adafruit_ble_broadcastnet

ble = adafruit_ble.BLERadio()
bridge_address = adafruit_ble_broadcastnet.device_address
print("This is BroadcastNet bridge:", bridge_address)
print()

print("scanning")
print()
sequence_numbers = {}

# ==== Again, switch between these two lines for testing ====
# for measurement in ble.start_scan(adafruit_ble_broadcastnet.AdafruitSensorMeasurement, interval=0.5):
for measurement in ble.start_scan(interval=0.5): 
    # Bleak on macOS doesn't deal in 6-byte MACs, so this obnoxious UUID is how things work.
    # Had to figure this out by waiting for an ad with the right complete_name, then checking this value.
    # Don't want to filter by name alone as some ads might be surfaced without complete_name.
    if measurement.address.string == "3A8F6F02-DA05-41C9-83D4-835F626674DC":
        print(measurement.__repr__())
print("scan done")

The result of this test:

# Ad as printed by the Feather:
Advertisement(data=b"\x0a\xff\x22\x08\x06\x04\x0a\x00\x00\xae\x41")

# Ad as it comes through on Mac in a generic scan:
Advertisement(data=b"\x0a\xff\x22\x08\x06\x04\x0a\x00\x00\xae\x41\x0e\x09\x43\x49\x52\x43\x55\x49\x54\x50\x59\x63\x31\x35\x39")

# Ad as it comes through on the Pi in a generic scan (these two ads repeat in alternation - I guess the Pi reports the 
# add, then reports again if a scan_response comes through, where Mac waits to avoid duplication.)
Advertisement(data=b"\x0a\xff\x22\x08\x06\x04\x0a\x00\x00\xae\x41")
Advertisement(data=b"\x02\x0a\x00\x0e\x09\x43\x49\x52\x43\x55\x49\x54\x50\x59\x63\x31\x35\x39")

# As ads come through on the Feather Sense in a generic scan, which looks identical to the Pi. Note that the reported temperature changed, because this is a new test at a different time (0xAE vs 0xB6).
Advertisement(data=b"\x0a\xff\x22\x08\x06\x04\x0a\x00\x00\xb6\x41")
Advertisement(data=b"\x02\x0a\x00\x0e\x09\x43\x49\x52\x43\x55\x49\x54\x50\x59\x63\x31\x35\x39")

I'm not quite sure what to make of this, but it looks like the check of match_prefixes is failing for some non-obvious reason, given that the ad data payload looks pretty matchy to me in the Mac case, and in at least the shorter Pi case.

EDIT: I just tested with a second Feather Sense running the RasPi version of the gateway code, and got matching results to the Pi. I noticed at the same time that the manufacturer data appears to be included in the first advertisement but not the second, which I guess is normal - the scan_response doesn't duplicate the original manufacturer data payload.

[alexwhittemore]

I do notice that AdafruitColor defines its match_prefixes differently from AdafruitSensorMeasurement
AdafruitColor
https://github.com/adafruit/Adafruit_CircuitPython_BLE/blob/8bb1210eebed4846dae7e76ff1db86a010b132bc/adafruit_ble/advertising/adafruit.py#L52-L60

# AdafruitColor
match_prefixes = (
        struct.pack(
            "<BHBH",
            _MANUFACTURING_DATA_ADT,
            _ADAFRUIT_COMPANY_ID,
            struct.calcsize("<HI"),
            _COLOR_DATA_ID,
        ),
    )

Adafruit_CircuitPython_BLE_BroadcastNet/adafruit_ble_broadcastnet.py

Lines 81 to 83 in 82899ad

	match_prefixes = (
	struct.pack("<BBH", 3, _MANUFACTURING_DATA_ADT, _ADAFRUIT_COMPANY_ID),
	)

Is that "3" out of order for matching?

[alexwhittemore]

Also maybe relevant: if it's supposed to be

match_prefixes = ( 
     struct.pack("<BBH", _MANUFACTURING_DATA_ADT, _ADAFRUIT_COMPANY_ID, 3), 
 ) 

where the 3 = 0x03 = 0x0003 from sequence_number = ManufacturerDataField(0x0003, "<B") and the idea here is to match everything up to the sequence number field specifier, assuming that anything with a sequence number field is an AdafruitSensorMeasurement... if I've got all that right, it's probably relevant that my advertisement .__repr__()s above don't have any sequence number in them?

[alexwhittemore]

Definitely some competing factors here - match_prefixes is out of order (3 should indeed be on the end) AND advertisements that fit entirely inside one ad packet don't seem to get assigned a sequence ID (therefore, the first unique byte isn't 3). I'm not sure if that second part is by design?

No - I just figured out: advertisements ALWAYS get assigned an ascending%256 sequence_id, even if the broadcast is only one submeasurement. The issue is that ManufacturerDataFields appear to get packed in the order of lifetime assignment, so since sequence_id gets defined last minute JUST before broadcast, it's not at the FRONT of the struct unless you specifically define it on your own when creating the AdafruitSensorMeasurement.

In other words,

# This works, since sequence_number gets put out front.
measurement = adafruit_ble_broadcastnet.AdafruitSensorMeasurement()
measurement.sequence_number = 0
measurement.battery_voltage = 1234
measurement.temperature = 55.55

vs

# This fails, since sequence_number gets tagged on the end of the struct and doesn't get caught as a prefix match.
measurement = adafruit_ble_broadcastnet.AdafruitSensorMeasurement()
#measurement.sequence_number = 0
measurement.battery_voltage = 1234
measurement.temperature = 55.55