Build a Deep Audio Classifier with Python and Tensorflow

Audio Data Preprocessing
📌 The process involves converting raw audio data into a numerical representation (waveform) using TensorFlow audio processing libraries, specifically `tf.io.read_file` and `tf.audio.decode_wave`.
🎵 The raw audio waveform (initially 44.1 kHz) is resampled to 16,000 Hz to reduce data size for processing.
📊 The key transformation is converting the waveform into a spectrogram using the Short-Time Fourier Transform ($\text{tf.signal.stft}$), allowing the use of Convolutional Neural Networks (CNNs) like an image.
✂️ Audio clips are padded with zeros ($\text{tf.zeros}$) at the start to ensure a consistent length of 48,000 samples for spectrogram creation, which results in a shape of $1491 \times 257 \times 1$.

Data Pipeline and Model Training
💾 TensorFlow `tf.data.Dataset` is used to build an efficient data pipeline, utilizing `list_files` to load file paths and `zip` to append binary labels (1 for Capuchin bird, 0 otherwise).
⚙️ The data pipeline uses the "MIXAB" sequence: Map (to create spectrograms via the $\text{pre-process}$ function), Cache, Shuffle, Batch (training in batches of 16), and Prefetch (8 examples) to eliminate CPU bottlenecks.
⚖️ The dataset is unbalanced, with 217 positive examples (Capuchin calls) and 593 negative examples (other sounds).
🧠 A Sequential CNN model is built with two $\text{Conv2D}$ layers (16 kernels, $3\times3$), followed by a $\text{Flatten}$ layer, a $\text{Dense}$ layer (128 units, ReLU activation), and a final $\text{Dense}$ output layer with sigmoid activation for binary classification. The model has approximately 770 million parameters.

Model Evaluation and Sliding Window Application
✅ The model was trained for 4 epochs and achieved 100% recall and precision on both training and validation partitions.
📈 Performance monitoring tracks loss, precision, and recall over epochs, saved via the model's $\text{history}$ object for plotting.
🔮 Predictions on the test set showed high accuracy; the confidence threshold for a positive detection was initially set to 0.5 but was increased to 0.99 to filter out low-confidence results and prevent overcounting of consecutive calls.
🧮 For 3-minute forest recordings (MP3 files), the audio is loaded, converted to 16 kHz mono, sliced into non-overlapping 48,000 sample windows, and processed through the trained model to generate predictions for density counting.

Key Points & Insights
➡️ The primary objective is to classify 3-second Capuchin bird calls and then use this trained model to count call density in longer (3-minute) forest recordings.
➡️ To handle longer audio files, the technique involves sliding window classification where the large clip is segmented into 48,000 sample windows for model inference.
➡️ Consecutive positive detections must be aggregated (grouped as one call) to accurately represent the true count of bird calls in a segment.
➡️ The final output involves compiling the total call count per recording into a CSV file (`capuchin_bird_results.csv`) with columns for the recording file name and the total number of Capuchin calls.

📸 Video summarized with SummaryTube.com on Feb 09, 2026, 22:36 UTC