Skip to content

Text to Audio Pipeline

The TransformersSharp.TextToAudioPipeline class provides a high-level interface for generating audio from text using pre-trained models from the Hugging Face Transformers library. It simplifies the process of converting text into audio by handling preprocessing, model inference, and decoding.

What is a Text to Audio Pipeline?

A text-to-audio pipeline is designed to generate audio representations of input text. It is commonly used for tasks like:

  • Text-to-speech (TTS) systems
  • Audio content generation
  • Assistive technologies

Key Features of Text to Audio Pipelines:

  • Pre-trained Models: Leverages state-of-the-art models like Bark and others.
  • High-Quality Audio: Generates audio with high fidelity and naturalness.
  • Customizable Parameters: Allows fine-tuning of generation parameters.

Using the TextToAudioPipeline Class

The TextToAudioPipeline class in TransformersSharp provides methods to generate audio from text. Below are examples of how to use it.

Generating Audio from Text

using TransformersSharp.Pipelines;

var pipeline = TextToAudioPipeline.FromModel("suno/bark-small");
var text = "Hello, this is a test.";
var audioResult = pipeline.Generate(text);
Console.WriteLine($"Audio Length: {audioResult.Audio.Length}, Sampling Rate: {audioResult.SamplingRate}");

Equivalent Python Code:

from transformers import pipeline

pipeline = pipeline("text-to-audio", model="suno/bark-small")
text = "Hello, this is a test."
audio_result = pipeline(text)
print(f"Audio Length: {len(audio_result['audio'])}, Sampling Rate: {audio_result['sampling_rate']}")

Accessing the Audio Data

The TextToAudioPipeline class provides the generated audio as a ReadOnlySpan2D<float> (a 2 channel wave) for efficient processing. The sampling rate is also returned for playback or further processing.

Example: Accessing Audio Data

using TransformersSharp.Pipelines;

var pipeline = TextToAudioPipeline.FromModel("suno/bark-small");
var text = "Hello, this is a test.";
var audioResult = pipeline.Generate(text);

// Access audio data
foreach (var sample in audioResult.Audio)
{
    Console.WriteLine(sample);
}

// Access sampling rate
Console.WriteLine($"Sampling Rate: {audioResult.SamplingRate}");

Equivalent Python Code:

from transformers import pipeline

pipeline = pipeline("text-to-audio", model="suno/bark-small")
text = "Hello, this is a test."
audio_result = pipeline(text)

# Access audio data
for sample in audio_result['audio']:
    print(sample)

# Access sampling rate
print(f"Sampling Rate: {audio_result['sampling_rate']}")