Mabyduck

Mabyduck: Build Better GenAI for Images With Human Feedback

Mabyduck is an online platform designed to help developers and researchers evaluate generative AI models. The platform enables users to create and run robust human-in-the-loop studies, leveraging subjective ratings and real-time analytics to assess the quality of audio-visual model outputs. Mabyduck streamlines the evaluation process for various types of image data, supporting better decision-making in AI development.

Key Features

• Self-serve Platform: Design studies easily using 13+ configurable experiment types tailored for audio, video, or image data.

• Real-Time Analytics: Instantly visualize and compare results across different rater groups for actionable insights.

• Flexible Rater Pools: Choose between crowdsourced participants and expert raters, tailoring studies to fit accuracy and budget requirements.

• Custom Rubrics and Leaderboards: Create private leaderboards to track internal model performance, or publish public rubrics to share results with the community.

• Active Selection Strategies: Optimize rating tasks with strategies that reduce rater workload while maintaining evaluation quality.

• High-Quality Raters: Access rigorously screened and hardware-qualified raters for reliable and repeatable metrics.

• Localization: Experiments and rater pools are supported in 8 languages, including English, French, Spanish, German, Vietnamese, Hindi, Mandarin, and Polish.

• Data Integrity: Automatic and manual quality controls ensure dependable data throughout every study.

• Technical Support: Dedicated rater support ensures smooth experiments and uninterrupted data collection.

Use Cases

• Evaluating the subjective quality of generative image models using human raters

• Benchmarking internal and public models with custom rubrics and leaderboards

• Comparing outputs from multiple AI models with different rater or language pools

• Optimizing evaluation costs using active sampling and pre-screened raters

• Running localized or multilingual model evaluation studies

Getting Started

• Website: https://www.mabyduck.com/

Mabyduck empowers research teams to conduct scalable, reliable evaluations of generative AI and related audio-visual models, making it easier to track quality, iterate on development, and showcase model improvements with trustable metrics and analytics.