Experimental research · not a released app

Birdsong 3D

What does a nightingale's song look like? Birdsong 3D turns a recording into a three-dimensional map of sound that grows in time with the music – so you can see how calls repeat, vary and combine into phrases, and perhaps one day understand a little more about how birds communicate.

In the video: one minute of a common nightingale (Luscinia megarhynchos), exported straight from the program. Press play with sound on.

Common nightingale – 60 s in 3DColour: spectral centroid · size: loudness · labels: detected callsRecording: Sander Pieterse, xeno-canto XC138451 via Wikimedia Commons, CC BY-SA 4.0 (first 60 s). Video © Grzegorz Duszak, shared under the same licence.
How to read it

Every point is a moment of sound

  • A point is a slice of the recording about 11 milliseconds long (44.1 kHz, 480-sample hop). The cloud grows in sync with playback.
  • Position comes from the timbre of the sound: 40 MFCC coefficients reduced to three dimensions with PCA. Moments that sound alike land close together, so a repeated call keeps returning to the same region.
  • Two sets of coordinates. The map you see is fitted separately for each recording. For comparing recordings, every stored phrase also gets coordinates in one shared, frozen space – see below.
  • Colour is the spectral centroid – the “centre of gravity” of the frequencies, shown on the scale in kHz.
  • Size follows loudness on a decibel scale: quiet chirps stay small, the loudest peaks become large accents.
  • Calls, not just frames. Each call is found as a region of the spectrogram that stands out from the background noise in its own frequency band, so two birds singing at once at different pitches stay two separate calls.
  • Similar calls grouped. Whole calls are compared with their order of changes kept (dynamic time warping), and similar ones are grouped with HDBSCAN – a rising and a falling whistle are not the same thing.
  • A memory across recordings. Call types are stored in a persistent atlas, so the same type can be recognised again in the next recording without redrawing the map.
  • Phrases and motifs. Calls separated by short pauses form phrases; sequences that come back – like a cuckoo's “cu-coo” – are found as motifs and can be listened to one by one.
  • Live mode and export. The map can grow from a microphone in real time, and any recording can be exported as a 1080 × 1350 video with its sound.
New in October 2026

Which bird is it – and what does it sing?

The program now names the singer and keeps what it finds. Every recording adds to a library of phrases, sorted by species and stored only on the computer it runs on.

  • Species by name. BirdNET listens to the recording in three-second windows and names the species – by its scientific name, with a confidence score. The recognition runs on the computer itself; nothing is uploaded.
  • Several birds in one recording. Each phrase gets the species heard during its own few seconds, so a recording with many singers is not forced into one name. A stretch the model is unsure about stays “unrecognised” instead of getting a guess.
  • A library of phrases. Every phrase is filed under its species, together with when it was sung and in which recording – a growing table of what a nightingale, a blackbird or a cuckoo has sung so far.
  • The same phrase is kept once. A phrase that matches one already stored for that species is counted as a repeat rather than stored again. In a 17-second cuckoo recording, 12 detected phrases turned out to be 2 different ones: the familiar “cu-coo”, heard 11 times, and one variant.
  • Phrases as 3D shapes. Each stored phrase keeps its shape in the map – the position, size, loudness and frequency of every point – so that phrases can be compared as shapes as well as sounds.
  • One shared space for comparison. Next to its on-screen shape, every phrase is mapped through a single frozen projection, fitted once on sound pooled from all recordings in the library – eight so far. The same sound lands in the same place whichever recording it comes from, and its loudness does not move it. Microphones and background noise still do, so this is a tool for comparison, not a proof of sameness.
The program's species panel: Luscinia megarhynchos, confidence 89 %, 16 phrases detected, 16 different, 16 stored in the library.
The program's own panel for the nightingale recording above. A species was named with confidence in 30 of 39 three-second windows: Luscinia megarhynchos in 29 of them, at 89 % on average, and a chaffinch in one, at 45 % – the model's doubts are shown, not hidden. 16 phrases were found, all different; 14 were filed as nightingale and 2 left unrecognised.
Plans, not promises

Where the project is heading

Birdsong 3D began as a way to look at a single song. These are the next stages, in the order they are being worked on:

  • Comparing phrases. Searching the library for similar phrases in the shared space, grouping motifs, and following how the repertoire of a species differs from one recording to the next.
  • Speech in the same space. Adding human speech to the library and refitting the shared space on birds and voices together, so that both are measured on one scale.
  • A version in the browser. A way to upload a sound – bird song, an animal call, speech or music – and explore its 3D shape without installing anything.
  • A research library. Bird vocalisations organised by species, recording, phrase and call, with the methods and settings documented so that an analysis can be repeated.

None of this is available yet and there are no dates. Recordings stay private by default. BirdNET may be used for non-commercial purposes only, so anything offered beyond personal research would need its own arrangement for species recognition.

An idea for the future

Does a bird ever sound like a word?

A later experiment would compare bird song with human speech – not to “translate” birds, but to measure how close two sounds are acoustically. The phrase library described above is its first building block: a growing collection of phrases for each species, each kept with its acoustic features, its 3D shape and its place in one shared space.

  • A library of spoken sounds. Instead of recording every word by hand, speech synthesis can generate a large set of words and their forms, together with the building blocks of speech – phonemes and syllables such as /x/, /ɔ/ or /t͡ɕ/ in the Polish “chodź”.
  • Phrase against phrase. The song is split into phrases, and each phrase and each spoken sample becomes the same kind of acoustic fingerprint: how the pitch moves over time, rhythm, frequency modulation, energy across bands and harmonic structure.
  • Shape, not pitch. Birds sing far higher than people speak, so the pitch is normalised – shifted down by several octaves – and only the shape of the sound in time is compared.
  • Many languages. Polish is a natural first language to try: its rich inflection and consonants such as sz, cz, ś, ć, ż and dź give very distinctive acoustic patterns. The same search can then run for German, English, French and others, to see in which language the closest matches appear.

The result could look like this – the numbers are only an illustration:

Phrase 14  →  “chodź”    93 % acoustic similarity
Phrase 27  →  “ciszej”   88 %
Phrase 41  →  “tutaj”    84 %

A high score would mean the two sounds have a similar structure – never that the bird “said” the word or meant anything by it.

What it is – and is not

An open question, explored carefully

Birdsong 3D is a personal research experiment. It is not published and not for sale. It analyses the acoustic structure of song – repetitions, variations and sequences – and makes it visible. It names the species with BirdNET, but it does not translate what birds “say”, and it does not recognise individual birds.

Whether patterns in these shapes can help us understand bird communication is exactly the question the project explores. Detected “calls” and “phrases” are measurements of sound, not confirmed biological units, and similar shapes do not prove that two sounds mean the same thing. A species name is a model's estimate with a confidence score, not an expert identification, and two phrases count as “the same” when they pass the program's own acoustic comparison.

Built on Sound-Visual, an open-source audio visualizer by Oscar David Hospinal R. (MIT licence), and inspired by Seeing Birdsong by Lucio Arese, which maps short slices of bird song with 40 MFCCs and PCA. Birdsong 3D is not affiliated with either work; call detection and grouping, the persistent atlas, phrase and motif analysis, the phrase library, live mode and video export were added for this project.

Species identification uses BirdNET (Kahl, Wood, Eibl & Klinck, 2021, Ecological Informatics 61, 101236), developed by the K. Lisa Yang Center for Conservation Bioacoustics at the Cornell Lab of Ornithology and Chemnitz University of Technology. Its models are licensed CC BY-NC-SA 4.0 and are used here for non-commercial research only. Birdsong 3D is not affiliated with BirdNET.