Video Fingerprints

Video Fingerprints

There can be multiple reasons you may want to generate video fingerprints from media files. I've blogged about it in detail here.

In short, instead of fingerprinting audio content, you may want to fingerprint and search video content. Starting with version 8.0.0 you can use SoundFingerprinting to generate video only or both audio and video fingerprints from a media file. It is up to you to decide what type of fingerprints to generate from a media file.

Below is an example of how you can generate audio and video fingerprints from an mp4 file, save them to storage, and later query to detect both audio and video matches.

Requirements: SoundFingerprinting >= v8.0.0, SoundFingerprinting.Emy >= v8.0.0.

// save fingerprints in Emy Community Edition https://hub.docker.com/repository/docker/addictedcs/soundfingerprinting.emy
private readonly IModelService modelService = EmyModelService.NewInstance("localhost", 3399);

// use FFmpeg to extract samples and frames from files: https://github.com/AddictedCS/soundfingerprinting/wiki/Audio-Services
private readonly IMediaService mediaService = new FFmpegAudioService();

public async Task GenerateAndStoreAudioVideoFingerprints()
{
    string path = "video.mp4";
    var avHashes = await FingerprintCommandBuilder.Instance
        .BuildFingerprintCommand()
        .From(path, MediaType.Audio | MediaType.Video)
        .UsingServices(mediaService)
        .Hash();

    var track = new TrackInfo("1234", "BWM", "", MediaType.Audio | MediaType.Video);
    modelService.Insert(track, avHashes);
}

Notice how media type is specified in both fingerprint creation and track definition: MediaType.Audio | MediaType.Video. The resulting object is of AVHashes type, and will contain both Audio and Video hashes that will be inserted in Emy storage.

Conversely, you can generate Video fingerprints only, by specifying MediaType.Video in the builder.

After fingerprints have been generated and stored, you can later query the storage with both audio and video fingerprints generated from a query file.

public async Task GenerateAudioVideoQueryFingerprintsFromFile()
{
    string path = "query.mp4";

    var avResults = await QueryCommandBuilder.Instance
        .BuildQueryCommand()
        .From(path, MediaType.Audio | MediaType.Video)
        .UsingServices(modelService, mediaService)
        .Query();

    foreach (var entry in avResults.ResultEntries)
    {
        var (audio, video) = entry;
        Console.WriteLine($"Matched track {entry.TrackId} with audio coverage {audio?.TrackRelativeCoverage} and video coverage {video?.TrackRelativeCoverage}");
    }
}