From 458e5a176ad7af19f1607235c652ff96be03a972 Mon Sep 17 00:00:00 2001 From: Carlos Pereira Atencio Date: Mon, 15 May 2023 11:07:25 +0100 Subject: [PATCH 1/7] docs: Add initial proposal for recording & playback API. --- docs/audio.rst | 71 ++++++++++++++++++++++++-- docs/microphone.rst | 118 +++++++++++++++++++++++++++++++++++++++++--- 2 files changed, 180 insertions(+), 9 deletions(-) diff --git a/docs/audio.rst b/docs/audio.rst index c607101b4..5ed607891 100644 --- a/docs/audio.rst +++ b/docs/audio.rst @@ -12,18 +12,25 @@ a speaker to pin 0 and GND on the edge connector to hear the sounds. The ``audio`` module can be imported as ``import audio`` or accessed via the ``microbit`` module as ``microbit.audio``. -There are three different kinds of audio sources that can be played using the +There are four different kinds of audio sources that can be played using the :py:meth:`audio.play` function: 1. `Built in sounds <#built-in-sounds-v2>`_ (**V2**), e.g. ``audio.play(Sound.HAPPY)`` + 2. `Sound Effects <#sound-effects-v2>`_ (**V2**), a way to create custom sounds by configuring its parameters:: my_effect = audio.SoundEffect(freq_start=400, freq_end=2500, duration=500) audio.play(my_effect) -3. `Audio Frames <#audioframe>`_, an iterable (like a list or a generator) +3. `AudioBuffer <#audiobuffer>`_ (**V2**), a generic buffer for audio that can + be used to record sound from the micro:bit V2 built-in microphone:: + + my_audio_buffer = microphone.record() + audio.play(my_audio_buffer) + +4. `Audio Frames <#audioframe>`_, an iterable (like a list or a generator) of Audio Frames, which are lists of 32 samples with values from 0 to 255:: square_wave = audio.AudioFrame() @@ -40,13 +47,16 @@ Functions Play the audio source to completion. - :param source: There are three types of data that can be used as a source: + :param source: There are four types of data that can be used as a source: - ``Sound``: The ``microbit`` module contains a list of built-in sounds, e.g. ``audio.play(Sound.TWINKLE)``. A full list can be found in the `Built in sounds <#built-in-sounds-v2>`_ section. - ``SoundEffect``: A sound effect, or an iterable of sound effects, created via the :py:meth:`audio.SoundEffect` class + - ``AudioBuffer``: An audio buffer, or an iterable of audio buffers, + created via the :py:meth:`audio.AudioBuffer` class or + :doc:`microphone.record() ` function - ``AudioFrame``: An iterable of ``AudioFrame`` instances as described in the `AudioFrame Technical Details <#id2>`_ section @@ -215,6 +225,61 @@ Sound Effects Example .. include:: ../examples/soundeffects.py :code: python + +Audio Buffer **V2** +=================== + +.. py:class:: + AudioBuffer(duration=3000, rate=11000) + + Create a buffer to contain audio data and its sampling rate. + + The sampling rate is configured via constructor or instance attribute, + and is used by the ``microphone.record_into()`` and + ``audio.play()`` functions to configure the recording and playback rates. + + For audio recording, reducing the number of samples recorded per second + will reduce the size of the data buffer, but also reduce the sound quality. + And increasing the sampling rate increases the buffer size and sound + quality. + + For audio playback, reducing the sampling rate compared with the recording + rate, will slow down the audio. And increasing the playback rate + will speed it up. + + The size of the buffer will be determined by the samples per second + and the ``duration`` configured when creating a new instance. + + :param duration: Indicates in milliseconds, how much sound data the buffer + can contained at the configured ``data_rate``. + :param rate: Sampling rate of for the data in the buffer. This value is + used for recording and playback, and can be edited as an attribute. + + .. py:function:: copy() + + :returns: A copy of the Audio Buffer. + + .. py:attribute:: rate + + The sampling rate for the data inside the buffer. + TODO: Indicate range of valid values here. + +Audio Buffer Example +-------------------- + +:: + + my_buffer = audio.AudioBuffer(duration=5000) + microphone.record_into(my_buffer) + audio.play(my_buffer) + + # A smaller buffer can be generated with the same duration by using + # a lower sampling rate + smaller_buffer = audio.AudioBuffer(duration=5000, rate=5500) + microphone.record_into(my_buffer) + audio.play(my_buffer) + + AudioFrame ========== diff --git a/docs/microphone.rst b/docs/microphone.rst index 88a847031..fe654886a 100644 --- a/docs/microphone.rst +++ b/docs/microphone.rst @@ -4,9 +4,9 @@ Microphone **V2** .. py:module:: microbit.microphone This object lets you access the built-in microphone available on the -micro:bit **V2**. It can be used to respond to sound. The microphone input -is located on the front of the board alongside a microphone activity LED, -which is lit when the microphone is in use. +micro:bit **V2**. It can be used to record and respond to sound. +The microphone input is located on the front of the board alongside a +microphone activity LED, which is lit when the microphone is in use. .. image:: microphone.png :width: 300px @@ -28,6 +28,25 @@ accessible via variables in ``microbit.SoundEvent``: - ``microbit.SoundEvent.LOUD``: Represents the transition of sound events, from ``quiet`` to ``loud`` like clapping or shouting. +Recording +========= + +TODO: +* Describe the feature. +* Indicate how the sampling rate relates to recording quality. +* Indicate how changing the sampling rate on the fly affects playback speed. +* What happens if the user changes the sampling rate while recording? + +:: + + from microbit import * + + while True: + if button_a.is_pressed(): + my_recording = microphone.record(duration=5000) + audio.play(my_recording) + sleep(200) + Functions ========= @@ -91,11 +110,60 @@ Functions :return: A representation of the sound pressure level in the range 0 to 255. +.. py:function:: record(duration=3000, rate=11000, wait=True) + + Record sound for the amount of time indicated by ``duration`` at the + sampling rate indicated by ``rate``. + + The amount of memory consumed is directly related to the length of the + recording and the sampling rate. The higher these values, the more memory + it will use. + + A lower sampling rate will reduce memory consumption and sound quality. + + If there isn't enough memory available a ``MemoryError`` will be raised. + + :param duration: How much time to record in milliseconds. + :param rate: Number of samples to capture per second. + :param wait: When set to ``True`` it blocks until the recording is + done, if it is set to ``False`` it will run in the background. + :returns: An ``AudioBuffer``, configured at the provided ``duration`` + and ``rate``, with the sound data. + +.. py:function:: record_into(buffer, rate=11000, wait=True) + + Record sound into an existing ``AudioBuffer``. + + :param buffer: An ``AudioBuffer`` to record the microphone sound. + :param rate: Number of samples to capture per second. + :param wait: When set to ``True`` it blocks until the recording is + done, if it is set to ``False`` it will run in the background. -Example -======= +.. py:function:: is_recording() -An example that runs through some of the functions of the microphone API:: + :returns: ``True`` if the microphone is currently recording sound, or + ``False`` otherwise. + +.. py:function:: stop_recording() + + Stops an a recording running in the background. + +.. py:function:: set_sensitivity(gain) + + Configure the microphone sensitivity to one of these three levels: + ``microphone.SENSITIVITY_LOW``, ``microphone.SENSITIVITY_MEDIUM``, + ``microphone.SENSITIVITY_HIGH``. + + These constants correspond to a number, and any values between these + constants are valid arguments + + :param gain: Microphone gain. + +Examples +======== + +An example that runs through some of the functions of the microphone +Sound Events API:: # Basic test for microphone. This test should update the display when # Button A is pressed and a loud or quiet sound *is* heard, printing the @@ -143,3 +211,41 @@ An example that runs through some of the functions of the microphone API:: display.clear() print(sound) sleep(500) + + +An example of recording and playback with a display animation:: + + from microbit import * + + talk_open = Image( + "09090:" + "00000:" + "09990:" + "90009:" + "09990" + ) + talk_closed = Image( + "09090:" + "00000:" + "00000:" + "99999:" + "00000" + ) + + my_recording = audio.AudioBuffer(duration=5000, rate=5500) + + while True: + if button_a.is_pressed(): + microphone.record_into(my_recording, rate=5500, wait=False) + display.show([talk_open, talk_closed], loop=True, wait=False, delay=150) + while button_a.is_pressed(): + sleep(50) + display.show(mouth_open, loop=False) # workaround issue #150 + display.clear() + if button_b.is_pressed(): + audio.play(my_recording, wait=False) + while audio.is_playing(): + x = accelerometer.get_x() + my_recording.rate = scale(x, (-1000, 1000), (2250, 11000)) + sleep(50) + sleep(100) From 1407c6b7cc2f1070aad9504de5bb9523c6e95c56 Mon Sep 17 00:00:00 2001 From: Carlos Pereira Atencio Date: Tue, 22 Aug 2023 15:48:39 +0100 Subject: [PATCH 2/7] docs: Update recording & playback based on review. --- docs/audio.rst | 113 +++++++++++++------------------------------- docs/microphone.rst | 82 +++++++++++++++++++++++--------- 2 files changed, 93 insertions(+), 102 deletions(-) diff --git a/docs/audio.rst b/docs/audio.rst index 5ed607891..250c69898 100644 --- a/docs/audio.rst +++ b/docs/audio.rst @@ -12,7 +12,7 @@ a speaker to pin 0 and GND on the edge connector to hear the sounds. The ``audio`` module can be imported as ``import audio`` or accessed via the ``microbit`` module as ``microbit.audio``. -There are four different kinds of audio sources that can be played using the +There are three different kinds of audio sources that can be played using the :py:meth:`audio.play` function: 1. `Built in sounds <#built-in-sounds-v2>`_ (**V2**), @@ -24,14 +24,9 @@ There are four different kinds of audio sources that can be played using the my_effect = audio.SoundEffect(freq_start=400, freq_end=2500, duration=500) audio.play(my_effect) -3. `AudioBuffer <#audiobuffer>`_ (**V2**), a generic buffer for audio that can - be used to record sound from the micro:bit V2 built-in microphone:: - - my_audio_buffer = microphone.record() - audio.play(my_audio_buffer) - -4. `Audio Frames <#audioframe>`_, an iterable (like a list or a generator) - of Audio Frames, which are lists of 32 samples with values from 0 to 255:: +3. `Audio Frames <#audioframe>`_, an instance or an iterable (like a list or + generator) of Audio Frames, which are lists of samples with values + from 0 to 255:: square_wave = audio.AudioFrame() for i in range(16): @@ -47,18 +42,16 @@ Functions Play the audio source to completion. - :param source: There are four types of data that can be used as a source: + :param source: There are three types of data that can be used as a source: - ``Sound``: The ``microbit`` module contains a list of built-in sounds, e.g. ``audio.play(Sound.TWINKLE)``. A full list can be found in the `Built in sounds <#built-in-sounds-v2>`_ section. - ``SoundEffect``: A sound effect, or an iterable of sound effects, created via the :py:meth:`audio.SoundEffect` class - - ``AudioBuffer``: An audio buffer, or an iterable of audio buffers, - created via the :py:meth:`audio.AudioBuffer` class or - :doc:`microphone.record() ` function - - ``AudioFrame``: An iterable of ``AudioFrame`` instances as described - in the `AudioFrame Technical Details <#id2>`_ section + - ``AudioFrame``: An instance or an iterable of ``AudioFrame`` + instances as described in the + `AudioFrame Technical Details <#technical-details>`_ section :param wait: If ``wait`` is ``True``, this function will block until the source is exhausted. @@ -79,6 +72,13 @@ Functions Stops all audio playback. +.. py:function:: set_rate(sample_rate) + + Changes the sampling rate of ``AudioFrame`` playback. + The default playback rate is 7812 samples per second. + Decreasing the playback sampling rate results in slowed down sound, and + increasing it speeds it up. + Built-in sounds **V2** ====================== @@ -226,70 +226,16 @@ Sound Effects Example :code: python -Audio Buffer **V2** -=================== - -.. py:class:: - AudioBuffer(duration=3000, rate=11000) - - Create a buffer to contain audio data and its sampling rate. - - The sampling rate is configured via constructor or instance attribute, - and is used by the ``microphone.record_into()`` and - ``audio.play()`` functions to configure the recording and playback rates. - - For audio recording, reducing the number of samples recorded per second - will reduce the size of the data buffer, but also reduce the sound quality. - And increasing the sampling rate increases the buffer size and sound - quality. - - For audio playback, reducing the sampling rate compared with the recording - rate, will slow down the audio. And increasing the playback rate - will speed it up. - - The size of the buffer will be determined by the samples per second - and the ``duration`` configured when creating a new instance. - - :param duration: Indicates in milliseconds, how much sound data the buffer - can contained at the configured ``data_rate``. - :param rate: Sampling rate of for the data in the buffer. This value is - used for recording and playback, and can be edited as an attribute. - - .. py:function:: copy() - - :returns: A copy of the Audio Buffer. - - .. py:attribute:: rate - - The sampling rate for the data inside the buffer. - TODO: Indicate range of valid values here. - -Audio Buffer Example --------------------- - -:: - - my_buffer = audio.AudioBuffer(duration=5000) - microphone.record_into(my_buffer) - audio.play(my_buffer) - - # A smaller buffer can be generated with the same duration by using - # a lower sampling rate - smaller_buffer = audio.AudioBuffer(duration=5000, rate=5500) - microphone.record_into(my_buffer) - audio.play(my_buffer) - - AudioFrame ========== .. py:class:: - AudioFrame + AudioFrame(size=32) - An ``AudioFrame`` object is a list of 32 samples each of which is an unsigned byte - (whole number between 0 and 255). + An ``AudioFrame`` object is a list of samples each of which is an unsigned + byte (whole number between 0 and 255). - It takes just over 4 ms to play a single frame. + :param size: How many samples to contain in this instance. .. py:function:: copyfrom(other) @@ -305,13 +251,19 @@ Technical Details You don't need to understand this section to use the ``audio`` module. It is just here in case you wanted to know how it works. -The ``audio`` module can consumes an iterable (sequence, like list or tuple, or -generator) of ``AudioFrame`` instances, each 32 samples at 7812.5 Hz, and uses -linear interpolation to output a PWM signal at 32.5 kHz, which gives tolerable -sound quality. +The ``audio.play()`` function can consume an instance or iterable +(sequence, like list or tuple, or generator) of ``AudioFrame`` instances. +Its default playback rate is 7812 Hz, and uses linear interpolation to output +a PWM signal at 32.5 kHz. -The function ``play`` fully copies all data from each ``AudioFrame`` before it -calls ``next()`` for the next frame, so a sound source can use the same +Each ``AudioFrame`` instance is 32 samples by default, but it can be +configured to a different size via constructor. + +So, for example, playing 32 samples at 7812 Hz takes just over 4 milliseconds +(1/7812.5 * 32 = 0.004096 = 4096 microseconds). + +The function ``play()`` fully copies all data from each ``AudioFrame`` before +it calls ``next()`` for the next frame, so a sound source can use the same ``AudioFrame`` repeatedly. The ``audio`` module has an internal 64 sample buffer from which it reads @@ -325,5 +277,8 @@ the buffer. This means that a sound source has under 4ms to compute the next AudioFrame Example ------------------ +Creating and populating ``AudioFrame`` iterables and generators with +different sound waveforms: + .. include:: ../examples/waveforms.py :code: python diff --git a/docs/microphone.rst b/docs/microphone.rst index fe654886a..706a531b9 100644 --- a/docs/microphone.rst +++ b/docs/microphone.rst @@ -31,20 +31,51 @@ accessible via variables in ``microbit.SoundEvent``: Recording ========= -TODO: -* Describe the feature. -* Indicate how the sampling rate relates to recording quality. -* Indicate how changing the sampling rate on the fly affects playback speed. -* What happens if the user changes the sampling rate while recording? +The microphone can record audio into an :doc:`AudioFrame