Introduction to MosaicVoice AI QA Overview and Scoring Approaches
MosaicVoice is an AI-powered platform designed to help call centers evaluate customer interactions more efficiently, accurately, and at scale. It automates the process of call scoring by analyzing transcripts and providing structured insights based on predefined scorecards.
The platform supports two primary methods of scoring calls: keyword-based logic and AI-based evaluation.
Keyword-Based Scoring
In keyword-based scoring, MosaicVoice uses a rule-based approach to identify whether specific words or phrases appear in the call transcript. This method offers a wide range of configurable options to tailor scoring logic to the needs of different teams and call types.
Common filtering and targeting options include:
- Call Disposition Filtering: Score only calls tagged with specific outcomes (e.g., "Interested", "Escalated").
- Duration Requirements: Restrict scoring to calls over a certain length.
- Speaker-Based Logic: Ensure a phrase is spoken by the agent, not the customer.
- Nested Conditions: Combine multiple filters to determine whether a scorecard item should be evaluated (e.g., only score “closing statement” if the call was longer than 3 minutes and marked as a “successful sale").
This approach is ideal when criteria can be explicitly defined with high confidence using specific phrases or logic triggers.
AI-Based Scoring (Focus of This Guide)
The second — and often more powerful — method of evaluation is AI-based scoring, which leverages large language models to assess calls in a more nuanced, flexible, and context-aware manner.
To perform AI-based scoring, MosaicVoice relies on four key inputs:
AI System Prompt
The AI system prompt defines the role the model is being asked to play. In MosaicVoice, this is typically a quality assurance (QA) reviewer responsible for scoring customer service calls.
This prompt is preconfigured by MosaicVoice but can be personalized to match your industry or use case. For example, you can specify:
- The type of call center (e.g., healthcare, hospitality, collections)
- The nature of the calls (inbound vs. outbound)
- Customer expectations or tone requirements
Customizing the system prompt helps the model better align with your operational context.
AI User Prompt
The AI user prompt is a hidden configuration layer available only to super admins. It gives the model detailed technical instructions about:
- The format of the AI’s response (e.g., JSON, plain text, true/false, explanation)
- The type of reasoning or analysis it should apply
- Whether it should provide supporting evidence for its answers
This component ensures that the model outputs are structured correctly for MosaicVoice’s backend and user interface. It’s not something users typically modify.
The Transcript
The transcript is generated automatically by MosaicVoice when a call is made. It is the primary data source the AI uses to evaluate performance.
Important notes:
- MosaicVoice does not use audio for AI scoring — only the transcript is analyzed.
- Any judgment about tone, sentiment, or empathy is based on text cues, not vocal inflection.
- Transcripts often include metadata such as timestamps, call direction, disposition, and more — this additional context can aid the AI in making more accurate evaluations.
The QA Scorecard
The scorecard contains the list of items the AI will evaluate — for example, “Did the agent verify the customer’s identity?” or “Did the agent demonstrate empathy during the interaction?”
Each scorecard item functions as a prompt to the AI, and the way these items are written directly affects how well the model can understand and evaluate them.
Best Practices for Writing Scorecard Items
- Structure each item as a clear Yes/No/NA question
- All prompts should be phrased as questions.
- The affirmative response (i.e., "Yes") should indicate that the agent successfully met the requirement.
- When MosaicVoice evaluates a scorecard item, the AI returns:
- Yes, No, or Not Applicable,
- Along with a two-sentence rationale explaining its decision.
- Therefore, it's essential to structure each question clearly so the answer can logically and confidently be Yes, No, or NA.
- Continue using “Apply If Disposition” filters where appropriate
- MosaicVoice’s AI is context-aware and will generally avoid scoring questions that aren’t relevant (e.g., skipping a “hold time” question if no hold occurred).
- However, the existing “Apply If Disposition” feature remains a reliable way to pre-filter calls and ensure only applicable scorecard items are evaluated.
- Use this feature to reduce noise and maximize scoring relevance.
- Be explicit when multiple criteria must be met
- If a scorecard item requires several components to be true, spell them out clearly.
- For example, if an agent must do three things during a greeting — such as say their name, the business name, and offer assistance — structure the item like:
“During the greeting, did the agent (1) say their name AND (2) say the company name AND (3) offer assistance?”
- Use numbering and "AND" or "OR" in all caps to make it explicit that all conditions must be met for the item to be scored as “Yes.”
- Add examples when clarity is needed
- If a scorecard question might be interpreted in multiple ways, consider including a short example directly in the prompt to clarify the expected behavior.
- For instance, if you're scoring for a specific type of greeting, you can write:
“Did the agent greet the customer appropriately (e.g., ‘Thank you for calling, this is Alex at CarePoint Medical, how can I help you today?’)?”
- Examples can help guide the AI toward your intended interpretation, especially for more subjective or stylistic items.
- Treat scorecard items as flexible prompts, not rigid questions
- Think of AI-scored items more like prompts than traditional form questions — you're not limited to a short sentence.
- Use formatting tools like ALL CAPS, *asterisks*, or bullet points to emphasize critical elements.
- You can include brief notes or guidance within the item itself to help the AI interpret it correctly.
- For example:
“Did the agent clearly verify the caller’s identity? > REQUIREMENTS: (1) Ask for full name AND (2) Ask for date of birth or another identifier.”
- Avoid relying on specific proper nouns when possible
- Don’t depend on the AI finding exact brand or location names (e.g., “Ritz-Carlton,” “Mount Sinai Hospital”), as these may be mis-transcribed.
- Instead, phrase the prompt more generally:
“Did the agent mention the hotel name during the call introduction?”
- This avoids false negatives caused by transcription inconsistencies and increases scoring accuracy.
- Plan to iterate and refine
- MosaicVoice is highly flexible and gets better the more you tune it.
- Don’t expect your first version of the scorecard to be perfect — revise items over time based on how the AI performs.
- Test your prompts, review AI rationales, and adjust wording or structure to improve consistency and accuracy.
- Be explicit and overcommunicate when in doubt
- The AI will follow your guidance — so provide as much clarity as you can up front.
- When expectations are complex or nuanced, write them out plainly. It’s better to over-explain than to leave room for ambiguity.