Video Cutting With Captioners, Video Classifiers, Audio Classifier, Audio Categorizer

audio analysis

audio classification

auto edit

transcription

video auto edit

video understanding

This article discusses various methods and tools for converting audio files to MIDI format, including COCA, audio classifiers, and taggers. It compares different audio-to-MIDI converter tools such as Polyphonic_track, audioToMidiConverter, PitchToMIDI, Tony, MusicTranscription, pYIN, Spleeter, and Musisep to transcribe polyphonic audio into MIDI format.

Published

May 10, 2022

you can cut based on video highlights, usually generated by counting “replay overlaps”, avaliable from youtube and bilibili, again needs supervised learning to recognize patterns and emit signals which we want

COCA using vit and palm for video captioning

audio classifier tutorial

audio tagger visualize how audio classifier works

need to identify sounds like dog bark and gun shots, sobs, laughs. Open sourced.

May use sound analyzers.

audio2midi:

https://gist.github.com/natowi/d26c7e97443ec97e8032fb7e7596f0b0

Recurrent Neural Network for generating piano MIDI-files from audio (MP3, WAV, etc.)

https://github.com/BShakhovsky/PolyphonicPianoTranscription

A python program which performs an FFT on an audio file and produces a MIDI file from the results

https://github.com/NFJones/audio-to-midi

Extract the melody from an audio file and export to MIDI

https://github.com/justinsalamon/audio_to_midi_melodia

Performs pitch detection on a polyphonic audio source and outputs to MIDI

https://github.com/corbanbrook/spectrotune

Program to detect pitch from wav files and write in time quantized MIDI

https://github.com/vaibhavnayel/Audio-to-MIDI-converter

A CNN which converts piano audio to a simplified MIDI format

https://github.com/hartmetzls/audio_to_midi

An application of vocal melody extraction.

https://github.com/bill317996/Audio-to-midi

Transcribes polyphonic piano pieces from audio (MP3, WAV, etc.) into MIDI-files

https://github.com/BShakhovsky/PianoAudioToMidi

Polyphonic pitch tracking in real time using machine learning algorithms

https://github.com/jaym910/polyphonic_track

Audio to MIDI converter

https://github.com/sbaeunker/audioToMidiConverter

Explore Transcribing Techniques to auto convert audio to midi

https://github.com/Goldspear/audio2midi

PitchToMIDI

https://github.com/KatoIppei/PitchToMIDI See releases

Piano & Drums

https://github.com/magenta/magenta/tree/master/magenta/models/onsets_frames_transcription

Tony: a tool for melody transcription

https://www.sonicvisualiser.org/tony/ https://github.com/sonic-visualiser/tony https://code.soundsoftware.ac.uk/projects/tony (https://github.com/mikulas-mrva/tony2max)

MusicTranscription

https://github.com/ClaraBing/CS229-MusicTranscription

pYIN

https://code.soundsoftware.ac.uk/projects/pyin https://github.com/ronggong/pypYIN (python)

Onsets and Frames Transcription (Piano & Drums)

https://github.com/magenta/magenta/tree/master/magenta/models/onsets_frames_transcription https://piano-scribe.glitch.me/

WaoN

https://sourceforge.net/projects/waon/

audio2midi conversion works great with prior source separation https://github.com/deezer/spleeter or others like https://github.com/rgcda/Musisep