MediaPlayer in voice_assistant

[KVinS]

Hello, I tried to add this component to my voice_assistant, but get many warnings:

03:43:45 | [W] | [component:170] | Component i2s_audio.microphone cleared Warning flag
03:43:45 | [W] | [component:157] | Component i2s_audio.microphone set Warning flag: unspecified
03:43:45 | [W] | [component:170] | Component i2s_audio.microphone cleared Warning flag
03:43:45 | [W] | [component:157] | Component i2s_audio.microphone set Warning flag: unspecified
03:43:45 | [W] | [component:170] | Component i2s_audio.microphone cleared Warning flag
03:43:45 | [W] | [component:157] | Component i2s_audio.microphone set Warning flag: unspecified

(By itself, the microphone works.)

My code:

external_components:
  - source:
      type: git
      url: https://github.com/gnumpi/esphome_audio
      ref: main
      #type: local
      #path: /Users/siekmann/Privat/Projects/espHome/esphome_audio/esphome/components
    components: [ adf_pipeline, i2s_audio ]

substitutions:
  name: esphome-smart-black-speaker
  friendly_name: Black speaker

esphome:
  name: ${name}
  friendly_name: ${friendly_name}
  min_version: 2024.6.0
  name_add_mac_suffix: false
  platformio_options:
    board_build.flash_mode: dio
  project:
    name: esphome.web
    version: dev
  on_boot:
    - light.turn_on:
        id: led_ww
        red: 100%
        brightness: 60%
        effect: fast pulse

esp32:
  board: esp32-s3-devkitc-1
  framework:
    type: esp-idf
    sdkconfig_options:
      CONFIG_ESP32S3_DEFAULT_CPU_FREQ_240: "y"
      CONFIG_ESP32S3_DATA_CACHE_64KB: "y"
      CONFIG_ESP32S3_DATA_CACHE_LINE_64B: "y"
      CONFIG_AUDIO_BOARD_CUSTOM: "y"
      CONFIG_ESP32_S3_BOX_BOARD: "y"

psram:
  mode: quad  # quad for N8R2 and octal for N16R8
  speed: 80MHz


# Enable logging
logger:


# Allow Over-The-Air updates
ota:
  platform: esphome

# Allow provisioning Wi-Fi via serial
improv_serial:

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password
  # Enable fallback hotspot (captive portal) in case wifi connection fails
  ap:
    ssid: "Esp32-S3-Wake-Word"
    password: "LJfUrdJk3svP"

captive_portal:

esp32_improv:
  authorizer: none

# To have a "next url" for improv serial
web_server:
  port: 80

# Enable Home Assistant API
api:
  on_client_connected:
        then:
          - delay: 50ms
          - light.turn_off: led_ww
          - micro_wake_word.start:
  on_client_disconnected:
        then:
          - voice_assistant.stop: 

button:
  - platform: restart
    name: "Restart"
    id: but_rest

switch:
  - platform: template
    id: mute
    name: mute
    optimistic: true
    on_turn_on: 
      - micro_wake_word.stop:
      - voice_assistant.stop:
      - light.turn_on:
          id: led_ww           
          red: 100%
          green: 0%
          blue: 0%
          brightness: 60%
          effect: fast pulse 
      - delay: 2s
      - light.turn_off:
          id: led_ww
      - light.turn_on:
          id: led_ww           
          red: 100%
          green: 0%
          blue: 0%
          brightness: 30%
    on_turn_off:
      - micro_wake_word.start:
      - light.turn_on:
          id: led_ww           
          red: 0%
          green: 100%
          blue: 0%
          brightness: 60%
          effect: fast pulse 
      - delay: 2s
      - light.turn_off:
          id: led_ww 

          
# Audio and Voice Assistant Config          
i2s_audio:
  - id: i2s_in
    i2s_lrclk_pin: GPIO3  #WS 
    i2s_bclk_pin: GPIO2 #SCK
  - id: i2s_speaker
    i2s_lrclk_pin: GPIO6  #LRC 
    i2s_bclk_pin: GPIO7 #BLCK

adf_pipeline:
  - platform: i2s_audio
    type: audio_out
    id: adf_i2s_out
    i2s_audio_id: i2s_speaker
    i2s_dout_pin: GPIO8
    fixed_settings: false
    adf_alc: true

microphone:
  - platform: i2s_audio
    id: va_mic
    adc_type: external
    i2s_din_pin: GPIO4 #SD pin on the INMP441
    channel: left
    pdm: false
    i2s_audio_id: i2s_in
    bits_per_sample: 32 bit


media_player:
  - platform: adf_pipeline
    id: adf_media_player
    name: m5stack-atom-echo
    keep_pipeline_alive: false
    internal: false
    pipeline:
      - self
      - adf_i2s_out

micro_wake_word:
  on_wake_word_detected:
    then:
    - light.turn_on:
        id: led_ww           
        red: 30%
        green: 30%
        blue: 70%
        brightness: 60%
        effect: fast pulse
    - voice_assistant.start:
        wake_word: !lambda return wake_word;
  models:
    - model: hey_mycroft

voice_assistant:
  id: va
  microphone: va_mic
  noise_suppression_level: 2.0
  volume_multiplier: 5.0
  media_player: adf_media_player
  on_stt_end:
       then: 
         - light.turn_off: led_ww
  on_error:
          - micro_wake_word.start:
  on_tts_end:
    then:
      - light.turn_on:
          id: led_ww           
          red: 50%
          blue: 50%
          brightness: 100%
  on_idle:
    then:
      - light.turn_on:
          id: led_ww           
          green: 50%
          brightness: 100%
  on_end:
        then:
          - light.turn_on:
              id: led_ww           
              red: 50%
              brightness: 100%
          - wait_until:
              not:
                media_player.is_playing:
          - light.turn_on:
              id: led_ww           
              red: 0%
              green: 50%
              blue: 50%
              brightness: 30%
          - micro_wake_word.start:
          - light.turn_off: led_ww


light:
  - platform: esp32_rmt_led_strip
    id: led_ww
    rgb_order: GRB
    pin: GPIO21
    num_leds: 1
    rmt_channel: 0
    chipset: ws2812
    name: "on board light"
    effects:
      - pulse:
      - pulse:
          name: "Fast Pulse"
          transition_length: 0.5s
          update_interval: 0.5s
          min_brightness: 0%
          max_brightness: 100%

Is it also expected that the radio or mp3 files (14kb) doesn't work from HA?

04:03:26 | [D] | [adf_i2s_out:127] | Set final i2s settings: 22050
04:03:26 | [D] | [esp-idf:000] | �[0;32mI (224034) AUDIO_THREAD: The http task allocate stack on external memory
04:04:02 | [D] | [media_player:061] | 'm5stack-atom-echo' - Setting
04:04:02 | [D] | [media_player:068] | Media URL: https://icecast.lider-fm.by/liderfm
04:04:02 | [D] | [adf_media_player:030] | Got control call in state 1
04:04:02 | [D] | [esp_adf_pipeline:050] | Starting request, current state UNINITIALIZED
04:04:02 | [D] | [esp-idf:000] | �[0;32mI (259957) MP3_DECODER: MP3 init
04:04:02 | [D] | [esp-idf:000] | �[0;32mI (259960) I2S: DMA Malloc info, datalen=blocksize=512, dma_buf_count=4
04:04:02 | [D] | [i2s_audio:072] | Installing driver : yes
04:04:02 | [D] | [esp_adf_pipeline:358] | pipeline tag 0, http
04:04:02 | [D] | [esp_adf_pipeline:358] | pipeline tag 1, decoder
04:04:02 | [D] | [esp_adf_pipeline:358] | pipeline tag 2, i2s_out
04:04:02 | [D] | [esp-idf:000] | �[0;32mI (259965) AUDIO_PIPELINE: link el->rb, el:0x3de051c0, tag:http, rb:0x3de056c8
04:04:02 | [D] | [esp-idf:000] | �[0;32mI (259966) AUDIO_PIPELINE: link el->rb, el:0x3de05364, tag:decoder, rb:0x3de06708
04:04:02 | [D] | [esp_adf_pipeline:370] | Setting up event listener.
04:04:02 | [D] | [esp_adf_pipeline:302] | State changed from UNINITIALIZED to PREPARING
04:04:02 | [I] | [adf_media_player:135] | got new pipeline state: 1
04:04:02 | [D] | [adf_i2s_out:127] | Set final i2s settings: 22050
04:04:02 | [D] | [esp-idf:000] | �[0;32mI (259974) AUDIO_THREAD: The http task allocate stack on external memory

Also, for some reason the log output disappeared and the command only works once. the second time it just blinks its LED endlessly

[formatBCE]

I don't see the pipeline: section for your microphone.

Regarding music: I use these with Music Assistant and they work... Also can play music directly from HA...

[KVinS]

I don't see the pipeline: section for your microphone.

I use a library that listens for a keyword in a loop:

micro_wake_word:
  on_wake_word_detected:

but it doesn't return to this cycle (

[formatBCE]

I don't see the pipeline: section for your microphone.

I use a library that listens for a keyword in a loop:

micro_wake_word:
  on_wake_word_detected:

but it doesn't return to this cycle (

Nothing to do with incorrect YAML. Add pipeline to your mic. It's missing.

[aarongorka]

Seeing something similar, I think I narrowed it down to when you're using a platform: adf_pipeline media player but you use a platform: i2s_audio microphone. I did this because I couldn't get any results from adf_pipeline microphone, but i2s_audio works fine.

This issue seems to be fixed on dev-next branch, or at least it doesn't spam warnings any more.

i2s_audio:
  - id: "my_i2s_speaker"
    i2s_lrclk_pin: GPIO21 # LRC
    i2s_bclk_pin: GPIO22
  - id: "my_i2s_microphone"
    i2s_lrclk_pin: GPIO16 # WS
    i2s_bclk_pin: GPIO4 # SCK

# # MAX98357
# speaker:
#   - platform: i2s_audio
#     id: "my_speaker"
#     dac_type: external
#     i2s_dout_pin: GPIO23
#     i2s_audio_id: my_i2s_speaker

# INMP441
microphone:
  - platform: i2s_audio
    id: my_microphone
    adc_type: external
    i2s_din_pin: GPIO0 # SD
    i2s_audio_id: my_i2s_microphone
    pdm: False
    # channel: left
    channel: right
    # bits_per_sample: 24bit
    bits_per_sample: 32bit
    sample_rate: 16000

external_components:
  - source:
      type: git
      url: https://github.com/gnumpi/esphome_audio
      ref: main
      # ref: dev-next
    components: [ adf_pipeline, i2s_audio ]

adf_pipeline:
  - platform: i2s_audio
    type: audio_out
    id: adf_i2s_out
    i2s_audio_id: my_i2s_speaker
    i2s_dout_pin: GPIO23
#   - platform: i2s_audio
#     type: audio_in
#     id: adf_i2s_in
#     i2s_audio_id: my_i2s_microphone
#     i2s_din_pin: GPIO4
#     pdm: False
#     channel: left
#     # sample_rate: 16000
#     # bits_per_sample: 32bit

# microphone:
#   - platform: adf_pipeline
#     id: my_microphone
#     # gain_log2: 3
#     # keep_pipeline_alive: True
#     pipeline:
#       - adf_i2s_in
#       - self

media_player:
  - platform: adf_pipeline
    id: my_media_player
    name: I2S Media Player
    keep_pipeline_alive: true
    internal: false
    pipeline:
      - self
      - adf_i2s_out