Skip to main content

​
Multimodal AI

A modality is a form of input or output. A multimodal workflow combines more than one form, such as a text question about an uploaded image or a spoken request with a text response. In B-Bot Hub, modality support depends on the selected model, provider, Agent configuration, and Distribution Channel. Configure and test the complete combination before relying on it.

​
Common Modalities

Text

Instructions, questions, formatted responses, and code.

Voice

Speech can be transcribed as input, and text can be synthesized as audio where configured.

Images

Supported models may inspect an image supplied with a question or produce an image through a configured capability.

Files

Supported files can provide context or become outputs in workflows that include file handling.

​
Input and Output Are Separate

An Agent that accepts an image does not necessarily generate images. Voice input and audio output also require separate configuration. For each workflow, document:
  • Accepted input types
  • Expected output types
  • Size, format, and language constraints
  • Required provider keys
  • Channel limitations
  • Fallback behavior when processing fails

​
Example Patterns

​
Image and Text

The user uploads an image and asks a specific question. The Agent returns a text description or analysis. The result should be treated as an interpretation, especially for small text, measurements, or safety-critical images.

​
File and Text

The user supplies a document or dataset with instructions. The Agent extracts or analyzes supported content. For important work, confirm that the complete file was processed and verify the result against the source.

​
Voice and Text

Speech is transcribed, then processed as a user message. Transcription errors can change meaning, so show or confirm critical details before acting.

​
Text and Audio

A text response is converted to speech. Provide a text alternative and test pronunciation, language, and accessibility needs.

​
Design Checklist

1

Confirm Support

Verify that the model, provider, and Distribution Channel support the required input and output types.
2

Minimize Data

Send only the content needed for the task. Remove unnecessary personal or confidential information.
3

Define Failure Behavior

Tell users when an input cannot be processed and provide a safe alternative, such as text entry or a different file format.
4

Test Real Inputs

Test representative accents, images, document layouts, file sizes, devices, and network conditions.
5

Review Sensitive Results

Require qualified review for medical, legal, financial, safety, identity, or accessibility decisions.

​
Limitations

  • Speech recognition can mishear names, numbers, and specialist terms.
  • Image analysis can miss details or infer objects that are not present.
  • Document extraction can lose layout, tables, annotations, or scanned text.
  • Generated files may be malformed or contain incorrect data.
  • Provider behavior, limits, and pricing can change.
Do not use an AI Agent as the sole authority for identity verification, diagnosis, compliance, or other high-impact decisions based on multimodal input.

Configure Modalities

Review the current B-Bot Hub input and output settings.

Provider Keys

Configure credentials for supported providers.

Chat

Learn how users provide text, voice, images, and files in chat.

DeepAgents

Use workspace files in multi-step workflows.