Input and Output Modalities
Modalities define the additional input and output types associated with an agent’s main model.
- Open a saved agent.
- Open Models and modalities.
- Choose the main model first.
- Select Add under input or output modalities.
- Choose an option, review its configuration, and add it.
The Hub saves modality changes to the selected model. It allows one entry of each input type and replaces an existing output entry of the same type.
Speech-to-text options shown in the current Hub are:
- Local Whisper Tiny with WebGPU
- Local Whisper Base with WebGPU
- OpenAI Whisper
- Groq Whisper Large V3
- Use the main model, if supported
You can enable automatic transcription when recording stops. The configured audio formats are WebM, MP3, and WAV.
Camera, images, audio, and documents
The current editor also offers:
- Camera / video input with main-model, OpenAI, Anthropic, or Google vision choices
- Image upload for JPEG, PNG, GIF, and WebP
- Audio upload for MP3, WAV, WebM, and OGG
- Document upload for PDF, DOC, DOCX, and TXT
Image, audio, and document entries expose Base64 and URL-support toggles. These settings describe the modality configuration; successful handling still depends on the selected model and runtime.
Current output options
Text-to-speech
Text-to-speech choices shown in the current Hub are:
- OpenAI TTS-1
- OpenAI TTS-1-HD
- Google Gemini 2.5 Flash Preview TTS
- Google Gemini 2.5 Pro Preview TTS
- Local Pocket TTS, marked experimental
The configuration includes Auto-play responses and Stream audio while generating toggles.
Image generation
Image output choices shown in the current Hub are:
- OpenAI DALL·E 3
- OpenAI GPT Image 1
- Stability SD3
- Google Imagen 3
You can enable automatic generation when the agent mentions an image.
Compatibility guidance
A modality appearing in the editor does not guarantee that every main model, browser, device, provider key, or deployment supports it.
- Select a main model that accepts the configured input type.
- Add the provider key required by a remote STT, TTS, vision, or image model.
- Local WebGPU options require compatible browser and device support.
- Browser microphone, camera, and autoplay policies can affect chat behavior.
- Treat preview and experimental model names as changeable.
- Test the complete path with the same agent, model, browser, and deployment that users will use.
For credential management, see Keys. For profile and company images, see AI image generation.
