Step 7 of 8

Analysis & Mastering

Working out what is in the audio, and making it sound like something else.

Everything on this page is offline work, not part of the real-time path: hand it a file, get back chords, notes or a mastered copy. Run it off the UI thread — some of it takes minutes. The matcher is the one that reaches back into the live path: it can hand its result to a running chain as settings rather than render a file, though the analysis behind it is still offline work.

Chord Detection

Detects musical chords, key, and tempo from audio files. Namespace: OwnaudioNET.Features.OwnChordDetect

var (chords, key, bpm) = ChordDetect.DetectFromFile("song.mp3", intervalSecond: 1.0f);

foreach (var chord in chords)
    Console.WriteLine($"{chord.StartTime:F1}s-{chord.EndTime:F1}s  {chord.ChordName} ({chord.Confidence:F2})");

Console.WriteLine($"Key: {key}  BPM: {bpm}");

// With MT3 an analysis takes minutes, so pass a progress callback (0..1 over the whole run)
var (chords2, key2, bpm2) = ChordDetect.DetectFromFile("song.mp3", mt3, 1.0f,
    p => Console.Write($"\rAnalyzing: {p:P0}"));

// Mixes all files before analysis
var (chords, key, bpm) = ChordDetect.DetectFromFiles(
    new[] { "guitar.wav", "piano.wav" },
    intervalSecond: 0.5f);

// Continuous detection from an active note stream. The detector is kept between
// calls on purpose — a fresh one every frame would wipe the stability history.
var (chord, stability) = ChordDetect.DetectRealtime(
    notes,                   // List<Note>
    DetectionMode.Optimized,
    buffersize: 5);          // how many recent frames vote on the answer

if (stability > 0.6f)
    Console.WriteLine($"Chord: {chord}  Stability: {stability:P0}");

intervalSecond is only a fallback window size. When tempo detection succeeds the window is derived from the BPM instead (quarter note below 100 BPM, half note up to 150, whole note above) and the argument is ignored.

Return Types

Transcription backends

Chord analysis never touches audio directly — it works from a List<Note>. Whatever produces those notes sits behind INoteTranscriber, so it can be swapped:

Every offline entry point takes an optional transcriber. Audio is decoded straight into whatever sample rate that backend asks for.

// Default — identical to the calls above
var (chords, key, bpm) = ChordDetect.DetectFromFile("song.mp3");

// MT3 instead
using var mt3 = new Mt3Transcriber(Mt3ModelPaths.FromDirectory("/models/mt3"));
var (chords2, key2, bpm2) = ChordDetect.DetectFromFile("song.mp3", mt3);

Note

The transcription output, in OwnaudioNET.Features.Extensions. Sorted by start time.

MT3 Transcription OwnAudioSharp.Mt3

Multi-track music transcription. Where BasicPitch hears pitches, MT3 hears instruments: a sequence-to-sequence transformer that emits MIDI-like events with a program number attached, so a bass line and a piano voicing stay separate instead of collapsing into one smear on the chromagram. For chord detection that separation is usually worth more than the extra pitch accuracy.

Getting the model files

The model weights are not in the package — they are about 290 MB, far too much to put in everyone's NuGet download. Grab the four files below, put them all in one folder, and hand that folder's path to Mt3ModelPaths.FromDirectory(). Keep the file names as they are; that is what the helper looks for.

mkdir -p ~/models/mt3 && cd ~/models/mt3

BASE=https://huggingface.co/ModernMube/HTDemucs_onnx/resolve/main/mt3-onnx
for f in mt3_encoder.onnx mt3_decoder_init.onnx mt3_decoder_step.onnx vocab.json; do
  curl -L -o "$f" "$BASE/$f?download=true"
done

// The folder you downloaded the four files into
using var transcriber = new Mt3Transcriber(
    Mt3ModelPaths.FromDirectory("/Users/me/models/mt3"));

If you would rather export your own weights from a different MT3-family checkpoint, the export scripts live in tools/mt3/ in the repository and produce exactly these four files. Mind the checkpoint's own licence in that case — YourMT3, the usual source of PyTorch MT3 weights, is GPL-3.0. This package contains none of its code and only reads exported weights at runtime, but what you may ship alongside those weights is between you and that licence.

using var transcriber = new Mt3Transcriber(
    Mt3ModelPaths.FromDirectory("/models/mt3"),
    threads: 0,        // 0 lets ONNX Runtime decide
    skipDrums: true);  // drops percussion natively

var (chords, key, bpm) = ChordDetect.DetectFromFile("song.wav", transcriber);

float[] samples = /* mono PCM at transcriber.PreferredSampleRate */;

var notes = transcriber.Transcribe(samples, transcriber.PreferredSampleRate,
    progress => Console.Write($"\r{progress:P0}"));

foreach (var n in notes.Where(n => !n.IsDrum))
    Console.WriteLine($"{n.StartTime:F2}s  pitch {n.Pitch}  program {n.Program}");

Mt3ModelPaths

Everything below the C# surface is Rust — the ONNX sessions, the greedy decode loop, the MT3 event codec and the note state machine all live in ownaudio_mt3_ffi. The managed side only marshals a float buffer in and a note array out.

Auto-Mastering (Matchering)

Point it at a reference track you like the sound of, and it moves your mix's spectral and dynamic character towards it. Namespace: OwnaudioNET.Features.Matchering. Everything below is an instance method on AudioAnalyzer — default-construct one and keep it, it caches its FFT setup and its preset targets.

var analyzer = new AudioAnalyzer();

AudioSpectrum spectrum = analyzer.AnalyzeAudioFile("track.wav");

analyzer.ProcessEQMatching(
    sourceFile: "my-mix.wav",
    targetFile: "reference.wav",   // the "sound" to match
    outputFile: "mastered.wav");

analyzer.ProcessWithEnhancedPreset(
    sourceFile: "my-mix.wav",
    outputFile: "for-club.wav",
    system:     PlaybackSystem.ClubPA,
    eqOnlyMode: false);

analyzer.BatchProcessWithEnhancedPreset(
    sourceFiles:    new[] { "track1.wav", "track2.wav", "track3.wav" },
    baseSampleFile: "reference.wav",
    outputDirectory: "output/",
    system:         PlaybackSystem.StudioMonitors,
    fileNameSuffix: "_mastered");

PlaybackSystem Values

GetAvailablePresets() is static and hands back a copy of the whole table — name, description, the 30-band curve and the dynamics settings — which is what you want for a preset picker.

Matching a live chain instead of a file

The four methods above are offline renderers: file in, file out. To drive a real-time mastering chain you need the step before the render — the numbers the effects have to be set to. Three additions hand that over, without changing anything the offline path does.

var analyzer = new AudioAnalyzer();

// The mix as it currently sounds - no temp wav, no decode round trip
AudioSpectrum mix = analyzer.AnalyzeAudioBuffer(masterSum, 48000, 2);

// The sound to move towards: a reference file...
AudioSpectrum reference = analyzer.AnalyzeAudioFile("reference.wav");
// ...or a playback system preset, built entirely in memory
AudioSpectrum preset = analyzer.GetPresetTargetSpectrum(PlaybackSystem.ClubPA);

MatcheringProfile profile = analyzer.CalculateProfile(mix, reference, sampleRate: 48000);

// Against a preset there is an overload that also brings the preset's own AGC block
MatcheringProfile presetProfile = analyzer.CalculateProfile(mix, PlaybackSystem.ClubPA, sampleRate: 48000);

// The profile is 30 third-octave bands, so it needs the 30-band EQ —
// the same filter bank the offline renderer drives. Its band centres
// already sit on the profile's frequencies, so read them back per band.
var eq = new Equalizer30BandEffect(sampleRate: 48000f);

for (int band = 0; band < 30; band++)
    eq.SetBandGain(band, eq.GetBandFrequency(band), profile.QFactors[band], profile.BandGainsDb[band]);

compressor.Threshold = CompressorEffect.DbToLinear(profile.CompThresholdDb);
compressor.Ratio     = profile.CompRatio;
dynamicAmp.TargetRmsLevelDb = profile.TargetLoudness;

There is no audio in a MatcheringProfile, so it serializes into a project file as is.

fixedQ — solve against the Q that actually plays

CalculateProfile defaults fixedQ to AudioAnalyzer.NativeBandQ (4.318474), the constant Q of the native 30-band equalizer. The engine has no per-band Q parameter, so that is the only Q a real-time chain ever plays; handing the solver the optimized per-band Qs instead would solve for a filter bank that is not the one making the sound. Pass fixedQ: 0 to get the per-band Qs anyway — that is what the offline render uses, since it runs the managed EQ where the Q does get through.

cutOnly — headroom without a gain stage

The offline chain buys headroom with a pre-gain stage before the EQ. A real-time chain assembled from native effects has no such stage, so cutOnly: true (the default) subtracts the curve's maximum from all 30 bands: the loudest band lands on 0 dB, everything else goes negative, and the DynamicAmpEffect behind it brings the level back to TargetLoudness. The broadband offset is already out of the curve by then, so this is a pure level change and leaves the tonal shape alone.

Next