Pipecat Integration — Examples

Voice bot with enhanced input

A complete Pipecat bot that enhances the user's audio before it reaches STT.

import os

from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.worker import PipelineParams, PipelineWorker
from pipecat.processors.aggregators.llm_context import LLMContext
from pipecat.processors.aggregators.llm_response_universal import (
    LLMContextAggregatorPair,
    LLMUserAggregatorParams,
)
from pipecat.services.cartesia.tts import CartesiaTTSService
from pipecat.services.deepgram.stt import DeepgramSTTService
from pipecat.services.openai.llm import OpenAILLMService
from pipecat.transports.base_transport import TransportParams

from pipecat_hecttor import HecttorFilter

transport_params = TransportParams(
    audio_in_enabled=True,
    audio_out_enabled=True,
    audio_in_filter=HecttorFilter(),  # enhance user audio before VAD/STT
)

# ... create the transport for your environment with transport_params ...

stt = DeepgramSTTService(api_key=os.environ["DEEPGRAM_API_KEY"])
llm = OpenAILLMService(api_key=os.environ["OPENAI_API_KEY"])
tts = CartesiaTTSService(api_key=os.environ["CARTESIA_API_KEY"])

context = LLMContext()
user_aggregator, assistant_aggregator = LLMContextAggregatorPair(
    context,
    user_params=LLMUserAggregatorParams(vad_analyzer=SileroVADAnalyzer()),
)

pipeline = Pipeline(
    [
        transport.input(),      # user audio — filtered by HecttorFilter
        stt,
        user_aggregator,
        llm,
        tts,
        transport.output(),
        assistant_aggregator,
    ]
)

worker = PipelineWorker(pipeline, params=PipelineParams(enable_metrics=True))

Tuning the enhancer

The defaults (voice isolation, model-default weight) are the right starting point for transcription pipelines. Two knobs are worth trying:

Model — the default is a voice-isolation model, which isolates the primary speaker in addition to removing noise. If you want all voices to come through (multi-speaker rooms, side-conversations that should be transcribed), switch to a pure noise-cancellation model. Available models use different architectures — try them to find which gives the best transcription results for your audio.
Enhancer weight — the wet/dry blend. Lower it if enhancement sounds too aggressive for your input; at 1.0 the output is fully enhanced.

hecttor_filter = HecttorFilter(
    model_name="your_model",  # pure noise cancellation, keep all speakers
    enhancer_weight=0.8,      # blend 20% of the original signal back in
)

Model names and their default blend weights are provided during onboarding. Compare candidates by measuring WER, not by ear — see Evaluations for the protocol.

Enabling and disabling at runtime

Enhancement can be bypassed mid-session without tearing down the pipeline — useful for A/B comparison or a user-facing toggle. Pipecat's standard FilterEnableFrame controls any input audio filter; disabling passes frames through untouched.

from pipecat.frames.frames import FilterEnableFrame

await worker.queue_frame(FilterEnableFrame(False))  # bypass (raw passthrough)
await worker.queue_frame(FilterEnableFrame(True))   # re-enable

Running the same audio through your ASR with the filter enabled and disabled is the basis of an offline A/B evaluation — see Evaluations.

Pipecat Integration — Examples

Voice bot with enhanced input

Tuning the enhancer

Enabling and disabling at runtime

On this page