A Scorecard for AI-Assisted Employee Coaching at Credit Unions

Credit unions evaluating AI-assisted employee coaching should require a six-lane scorecard before a pilot begins: work quality, member outcomes, employee development, work distribution, fairness and accessibility, and control performance. Keep those measures separate. Do not roll them into one opaque score that automatically affects pay, discipline, promotion, scheduling or termination.

This is narrower than a general AI workforce plan. A workforce plan maps tasks, permissions and training. A coaching scorecard answers a different question: once a tool reviews calls, drafts feedback, highlights process gaps or recommends practice, how will HR and operations leaders know whether it is helping—and where does its authority stop?

The measurement problem is real. The U.S. Government Accountability Office’s September 2, 2025 digital-surveillance report, reissued December 10, 2025, found that monitoring technology can support operations but may rely on flawed productivity benchmarks, miss important offline work or be used for purposes beyond its design. GAO also found that transparency, collection practices and the way employers use productivity measures can shape worker effects.

For credit unions, the right unit of analysis is the complete service task—not the easiest activity for software to count. The following lanes turn that principle into a pilot scorecard.

1. Work quality

Measure whether coached employees produce more accurate, policy-consistent work. Depending on the workflow, useful measures can include source accuracy, required disclosures, authentication steps, documentation completeness, supervisor corrections and avoidable rework.

Pair automated flags with human quality sampling. A falling flag rate is not improvement if the model stopped detecting a class of error, the knowledge base changed or employees learned to phrase interactions for the scoring system. Record the tool version, rubric version, sampled cases and reviewer agreement.

2. Member outcomes

Connect coaching to the member’s result: first-contact resolution, repeat contacts, complaints, transfers, abandonment, error correction and use of an equivalent human route. In a lending, collections or fraud workflow, add the outcome measures that belong to that service rather than treating shorter handling time as a universal benefit.

A coaching prompt that makes calls faster but increases repeat contact has moved work, not removed it. This is why the scorecard should align with the credit-union contact-center AI quality framework, which tests the full service and escalation path rather than one generated answer.

3. Employee development

Measure whether employees learn, not merely whether they comply with the prompt. Track time to demonstrated proficiency, whether a corrected behavior persists in later sampled work, appropriate use of overrides, confidence in escalation and the ability to explain the governing rule without the tool.

Usage is not mastery. A high acceptance rate may reflect good guidance, but it can also reflect automation bias or pressure to follow the recommendation. Supervisors need examples of accepted, rejected and revised suggestions to calibrate whether employees are exercising judgment.

4. Work distribution

Measure the whole workload: case mix, after-contact work, queue transfers, exception volume, supervisor review time and time spent helping colleagues. GAO noted that monitoring tools can undercount activities such as research, reading and helping others. In a credit union, those “invisible” tasks often carry member context or control value.

Compare similar roles and task types. Do not rank a specialist handling complex exceptions against a colleague receiving routine cases. Baseline the distribution before the pilot so leaders can see whether the tool changed the work employees receive.

5. Fairness and accessibility

Review false flags, coaching frequency, overrides and outcomes across relevant job contexts and groups, subject to legal and privacy review. Test accents, language patterns, assistive-technology use, approved accommodations, different shifts and channels. A speech model that misreads an accent or a productivity measure that ignores an accommodation can create a distorted performance record.

The EEOC’s current AI fact sheet says federal employment-discrimination laws apply to new technologies as they do to other employment practices. It specifically identifies monitoring performance, assessing productivity, setting wages and promotion or termination decisions as activities where AI may appear.

6. Control performance

Track whether the control system works: data-access exceptions, unsupported feedback, employee corrections, contested results, time to resolve an appeal, model or rubric changes, incidents and stop-condition activations. The NCUA’s AI resources emphasize that AI can extend beyond traditional vendor-risk practices and may require attention to algorithmic decisions, privacy, resilience, model risk and ongoing monitoring.

The NIST AI Risk Management Framework remains a useful voluntary structure for governing, mapping, measuring and managing risk, although NIST says version 1.0 is being revised. For a coaching pilot, that means naming the owner, intended use, affected employees, evaluation method, intervention authority and retirement or rollback conditions.

Keep coaching and employment decisions in separate lanes

The scorecard should support development conversations and pilot decisions. It should not silently become an employment-decision engine. If leaders later want to use any coaching signal for compensation, promotion, discipline, scheduling or termination, treat that as a new use case with separate HR, legal, privacy, accessibility, validation and employee-notice review.

At minimum, the coaching lane needs four limits:

• no automatic consequential employment action;
• an employee route to see and correct material source data or feedback;
• documented supervisor review that considers task context and approved accommodations; and
• a prohibition on reusing the data for a new purpose without a fresh review.

Run one decision-ready pilot

Choose one bounded role and workflow. Establish a pre-pilot baseline with enough volume to represent ordinary and difficult work. Define each metric’s source, owner, review frequency and threshold. Sample both apparently strong and weak results. Review results by task and context instead of relying only on averages.

Then set a scheduled continue, revise or stop decision. Continue only when quality or development improves without material deterioration in member outcomes, workload balance, fairness or controls. Revise when the signal is useful but the rubric, data or workflow boundary is wrong. Stop when the tool cannot produce reliable feedback, employees cannot contest material errors or the system begins influencing decisions beyond its approved coaching purpose.

This scorecard complements the six-step AI workforce plan for credit unions: the plan prepares people and tasks for AI assistance; the scorecard tests whether coaching actually improves the work. The executive decision is not whether an AI coach can produce feedback. It is whether the credit union can prove that the feedback helps, explain what it measures and prevent a development tool from becoming an unreviewed employment system.