As any parent or caregiver knows, a baby's cry is one of the most urgent, raw, and stressful sounds in the world — an evolutionary marvel designed to demand immediate attention. But for sleep-deprived parents, decoding why a baby is crying can feel like an impossible guessing game.

What if we could use machine learning to translate these cries, turning acoustic waves into actionable, reassuring insights for parents?

Wail, built by Doisk, is an end-to-end deep learning AI model that classifies infant vocalizations. By transforming audio signals into 2D spectrogram images and leveraging state-of-the-art computer vision models, the production-ready model achieves 81% test accuracy across 9 cry categories.

1. Feature engineering: turning sound into vision

Simple 1D audio waveforms contain high-frequency oscillations that neural networks struggle to interpret directly. Instead of processing raw audio, Wail translates sound into images using librosa to extract Mel-spectrograms — a logarithmic frequency scale designed to mimic how the human ear actually perceives pitch, making it exceptionally suited for capturing the nuances of baby vocalizations.

Preprocessing pipeline

  • 1
    Compute spectrogram — Mel-spectrogram with n_mels=128, FFT window n_fft=2048, hop length hop_length=512.
  • 2
    Convert to decibels — Log-scale the power spectrogram via power_to_db with a top threshold of 80 dB.
  • 3
    Temporal alignment — Pad or truncate audio frames to a fixed width of 128 frames, ensuring consistent 128×128 dimensions.
  • 4
    Normalize to pixels — Rescale the decibel matrix from [-80.0, 0.0] to [0.0, 1.0], then multiply by 255.0 to produce a grayscale image representation of the vocalization.

2. Model selection: deep residual learning with ResNet-50

Deploying a model on a web server requires balancing prediction accuracy with latency. ResNet-50 was selected as our neural network backbone for three primary reasons:

  • Pretrained weights — Initialized with ImageNet weights, allowing the 50-layer network to leverage deep hierarchical visual features (edges, textures, spatial hierarchies) immediately.
  • Residual learning — Utilizes identity skip connections to bypass layers, mitigating the vanishing gradient problem and enabling efficient optimization of a deep architecture.
  • Transfer learning — The ResNet-50 feature extractor backbone is frozen, while a custom dense head is appended to classify features, adding:
    • Global average pooling to reduce spatial dimensions
    • Dropout (35%) to prevent overfitting on the specialized dataset
    • Softmax classification layer outputting probability scores across 9 classes

3. Test results

Evaluating a baby cry classifier requires a high degree of precision — misclassifying a cry of pain as silence or laughter could lead to a highly frustrating and unsafe user experience.

1,848
Unique test samples
81.0%
Overall test accuracy
9
Classification classes
Category Precision Recall F1-score Samples
Belly pain 1.00 0.89 0.94 310
Burping 0.94 0.91 0.93 253
Cold / hot 0.87 0.61 0.72 190
Discomfort 0.77 0.76 0.76 252
Hungry 0.55 0.60 0.57 218
Laugh 0.62 0.76 0.69 139
Noise (background) 0.96 1.00 0.98 199
Silence 0.88 0.96 0.92 146
Tired 0.64 0.78 0.71 141
Overall accuracy — — 0.81 1,848
Macro average 0.80 0.81 0.80 1,848
Weighted average 0.82 0.81 0.81 1,848
Key takeaways
  • Exceptional pain detection — The model achieves 100% precision on belly pain cries (94% F1). When a baby is in acute distress, the classifier is highly reliable at isolating it.
  • Superb noise and silence handling — Background noise (0.98 F1) and silence (0.92 F1) are classified with near-perfect accuracy, ensuring domestic ambient sound won't trigger false cry alerts.
  • The hunger challenge — At 57% F1, hunger is the most challenging category. Acoustically, infant hunger cries often mimic general discomfort or tiredness before escalating, causing classification overlap. Improving this is a prime focus for the next release.

4. Production-ready serving and edge safety

Building a model is only half the battle. Two serving guardrails are implemented inside Wail to make it even more reliable in the real world.

Voice activity detection

A fast VAD filter runs before the deep learning model. If the audio signal amplitude falls below active acoustic thresholds, the system short-circuits and returns a "silence" result immediately — preventing wasted compute and garbage classifications on dead air.

Confidence thresholding

In safety-critical or emotionally sensitive domains, knowing what you don't know is crucial. If the highest softmax probability falls below the 50% threshold, the model avoids guessing and classifies the signal as "unknown", ensuring no misleading recommendations are surfaced to parents.

Conclusion and what's next

Wail bridges the gap between deep signal processing and parental peace of mind. By combining powerful computer vision backbones with robust serving guardrails, the system accurately understands baby needs in real time.

Roadmap

We are working on Wail-2.2, with an aim of achieving better generalization, increasing the amount of training data, and improving accuracy.