arunrl.com honeycomb

Speech recognition for non-standard speech.

Starting with autistic children who have speech difficulties: learning their individual speech patterns and building around how they naturally communicate.

Listen continuously. Train from examples.
01 / live transcriptionListen and transcribe

Start listening once. See words appear as the child speaks, without pressing a button for every phrase.

02 / trainingCollect examples and train

Record a longer prompt-and-response session without stopping between attempts. Extract and review labelled clips, then fine-tune the model.

Training workflow · select a step for details

Recording. Record an adult prompting words or phrases and the child responding in one longer session. You do not need to stop and save each attempt as a separate training recording.

Personalization needs both a model that learns the speaker’s voice and a way to collect examples that fits how they communicate. For a child who does not speak on cue, the recording workflow becomes part of the recognition problem.

#Why this is a different challenge

I built Honeycomb for non-standard speech, starting with my daughter, who is autistic and has speech difficulties. Conventional speech recognition struggled to understand her. For our family, that meant everyday needs, choices and ideas could be missed.

A person’s speech can have individual sounds, rhythms, pace and vocabulary that a general-purpose model does not recognize reliably. Personalization offers a way to learn those patterns from examples of that person’s own speech.

But collecting those examples introduces another challenge. A typical training workflow asks someone to say a word or phrase, wait while it is recorded, and repeat it several times. That assumes the speaker can and will produce speech on demand.

For autistic children with speech difficulties who do not speak on cue, that assumption can make the workflow unusable. Attention, willingness to repeat a phrase, and the timing of natural expression all affect whether useful examples can be collected. In our experience, bringing out a phone and preparing to record could change the moment or stop the communication altogether.

The problem reaches beyond adapting a model. Data collection, the listening interface and parent review all need to fit the child’s natural way of communicating. This is the perspective I arrived at through building and using the prototypes at home, and later developed while exploring Clay Oz.

The child’s job is to communicate. The technology’s job is to learn.

#Remove the ritual

The first prototype, built using Claude Code, fine-tuned OpenAI’s Whisper on our own labelled recordings. Corrections became new examples. It used tap-to-talk, with an optional enrolled-speaker filter to leave out adult prompts. The model learned from examples, while using it still depended on an adult preparing the device before each expression.

The next version moved to live transcription: press once, then let speech happen. Text appears as audio chunks are processed. A built-in training page made it possible to record, label and fine-tune without running scripts.

I also removed runtime speaker filtering. With the conversation visible as text, that complexity was no longer necessary for our workflow. This was a lesson from using it at home, rather than evidence that every family needs the same interface.

#Learn from longer recordings

Live transcription reduced friction during use. Preparing clean training examples was still laborious. The next step was to extract responses from longer recordings in which an adult prompts a word and the child attempts it.

That work is now part of Honeycomb. The extractor automates a first pass, then gives the reviewer timelines and checks to refine it.

Read the adult’s prompt.

Whisper identifies the requested word and the order of prompts. Its transcript of the child’s speech is unreliable, and its word timestamps are too imprecise to use as exact cut points.

Find candidate responses.

Energy marks activity; pYIN estimates pitch. In the original recordings, pitch helped distinguish the adult voices from the child’s. The draft selects responses after a prompt and before the next adult utterance.

Review the boundaries.

Inspect pitch and transcript timelines, adjust cuts, and check for adult speech at the edges. Pitch ranges can overlap, so the threshold alone cannot identify a speaker reliably.

Make examples ready for training.

Trim silence, remove DC offset, normalize, add short fades and write labelled WAV clips with provenance. After review and listening, import the clips into the existing training workflow.

Speaker separation has returned here for a different task: preparing training data. The live transcription app continues without runtime speaker filtering.

#What the evidence says

The October 2 exploration produced 68 curated clips from 40 of 42 recordings. The automatic draft matched 60 of those 68 final clips in that batch. The remaining decisions required timeline inspection, especially around a higher-pitched adult voice, quiet responses and short fragments.

These are observations from one batch, not a recognition-accuracy benchmark. Final boundaries were curated from pitch, energy and transcript evidence. No one listened to the clips during that original processing session; spot-listening remained outstanding.

This is a semi-automatic, offline tool for prompt-and-response recordings. It does not yet turn arbitrary everyday conversation into trustworthy training data on its own. A prompt tells us the intended target; a reviewer still needs to decide whether the response is a useful example.

The extractor uses heuristics for data preparation. Whisper recognition remains learned through fine-tuning. New examples need evaluation on separate recordings to establish whether they actually improve recognition.

The public repository contains the tools and methodology. Raw audio, extracted speech and private batch manifests are excluded.

#Where this goes next

While exploring Clay Oz, I came to describe the broader goal as personalized speech recognition built around natural communication: capture useful moments, let a parent confirm meaning, and improve the model over time. I am no longer pursuing Clay Oz as a company. That thinking continues to guide Honeycomb as a personal, open-source project.

The direction remains useful: less staged data collection, lighter review, and better ways to measure whether personalization helps outside the training set. Getting from prompted recordings to natural, unprompted speech is still an open problem.

I would welcome collaborators interested in speech differences, accessible interfaces, robust sample extraction or evaluation. The starting point is the code and its limits.

Explore Honeycomb on GitHub ↗ · Try the extractor ↗