What if you could capture sound in an image? What if you could add another dimension to a still photograph? What if you could make a picture come alive?
This is what this projects aims to do. This is a project where the pictures are made more tangible by encoding the sound of the surroundings where the picture was taken into the image. Read more about how this works in my blog post Visualizing Sound In The Wild.
To understand how the algorithm works and make the experience more tangible I create an app using Streamlit. The app is hosted by Streamlit Share and Heroku.
Users can visualize sound on any image of their choosing. The app supports two methods of visualizing sound:
- Visualizing Sound in Real Time - Use your own voice to visualize sound.
- Visualizing Sound Youtube - Use audio from any YouTube video to visualize sound.
This app also doubles as a way to extract and download audio from any YouTube video.
Below are some projects that demonstrates how sound can be visualized in an image. The idea of visualizing sound is not limited to capturing the source of the sound but can be extended to capture any meaningful sound and visualize it in the picture. Think of it like tagging sound in an image instead of a location, but in a more visual way.
A complete list of projects can be found here and in the YouTube playlist Visualize Sound Projects.
Being the tallest waterfall in Oregon, listen in on 923 gallons of water falling over a height of 620 feet every second. The sound from the waterfall varies the brightness of the waterfall making the image come alive.
Get exclusive access to the original HD picture along with the sound of Multnomah Falls used to make this art.
An early morning hike on the Mirror Lake trail in the woods of Mount Hood, Oregon. Somewhere among the trees, I was taking a break and lying down in the snow. It was here, I captured this picture and recorded the sound of chirping birds. The sound from the birds vary the brightness of the sky making the image come alive.
Get exclusive access to the original HD picture along with the sound of the chirping birds used to make this art.
This is a video of me with an existing Devil Horns Snapchat filter, but with my own way of encoding the music into the picture. See how the music Some Nights is encoded into the picture and makes the picture come alive. Does this look cool or what? Would you want this as an Instagram or a Snapchat filter?
If you want to make any changes to the code or were directed here cause the hosted apps on Streamlit Share and Heroku are not working, here is a step by step guide on how to run the app locally.
Clone the repository locally.
git clone https://github.com/anirudhtopiwala/Visualize_Sound_In_The_Wild.git
Activate your virtual environment or create a new one. You can create a virtual environment by running:
python3 -m venv ${ENV_NAME}
source ${ENV_NAME}/bin/activate
Once your environment is active, install the dependencies and required packages by going to the repository directory and running:
pip install -r requirements.txt
Start the app by running:
streamlit run web_app.py
The advantage of running the app locally is that you get to visualize sound in real time and can also change the code to process youtube videos for a longer duration of time. This is not possible on the hosted apps because of limited resources.
