How to Rank AI Opportunities When Stakeholders Pick Different Winners

When a founder asks everyone to rank the next AI opportunity, the meeting can produce four reasonable answers. That does not mean the stakeholders are confused. They may be optimizing for different consequences.

The mistake is to turn that disagreement into a popularity contest. First decide who owns the outcome, which constraints can veto a candidate, and what evidence would change the decision. Then aggregate the scores that remain.

The result: a forced average can create a winner that nobody actually chose

When stakeholder preferences cycle, there is no unique winner in the rankings alone. A decision needs an explicit owner, a declared tie-breaker, or new evidence.

I ran two small deterministic calculations before writing this post. They are analytical fixtures, not survey data or client results. The first shows what happens when three stakeholders each prefer a different opportunity. The second shows why a veto must be applied before a score is averaged.

Fixture	Result before constraints	Result after the decision rule	What it proves
Three stakeholders, three opportunities	Rank-sum points tie A, B, and C at 3. Pairwise majority cycles: A beats B, B beats C, C beats A.	No winner is declared. The next decision is to define decision rights or collect evidence.	Averages and votes cannot manufacture an intrinsic winner from a preference cycle.
Four roles, four opportunities	Equal averaging picks D at 4.25. Declared role weights also put D first at 4.15.	D is vetoed because its first release has no reversible read-only slice. C becomes the highest-scoring eligible option at 3.50.	A non-negotiable action boundary is not a low score. It is an eligibility condition.

The exact inputs and formulas are recorded in the research artifact for this post. The wider AI opportunity pillar gives the cluster context. The important limit is just as exact: these fixtures show what the rules do. They do not show how often real teams produce cycles, or that C is generally better than D.

Why different winners usually mean different objectives

Different stakeholder winners are usually a sign that the group has not agreed on the object being ranked. One person is ranking potential revenue. Another is ranking time saved this quarter. A third is ranking control risk. The scores can all be rational because the question is underspecified.

Microsoft's business-envisioning guidance starts with the problem, business objective, measurement of success, and accountability before it asks teams to prioritize use cases. It then compares business value, user experience, and technical viability. Microsoft's AI use-case guidance supports the order of operations, but it does not provide the role weights or veto rule in this article.

I have seen the same ambiguity while teaching product managers to move from writing specifications to building and shipping products. The recurring failure was often not the model. It was that nobody could say what done meant. That is a locked qualitative observation from Marius Manolachi's teaching record, not a measured rate or a study.

Before asking who wins, write one sentence that every score is meant to estimate:

For this decision, we are choosing the smallest AI-enabled change that can improve [named outcome] for [named owner or user] by [reviewable measure], while keeping [forbidden consequence] under [named control].

If the group cannot complete that sentence, do not debate the ranking yet. You are still defining the opportunity.

Set decision rights before collecting scores

The accountable owner decides which eligible option best serves the agreed outcome. Other stakeholders supply evidence, constraints, or specialist review. Equal speaking time does not require equal decision weight.

Use these roles:

Role	Contribution	What this role cannot do alone
Outcome owner	Defines success, carries the operational result, and makes the final choice among eligible options.	Ignore a declared safety, legal, or data veto.
Resource sponsor	States the budget, time horizon, and strategic constraints.	Redefine the operational outcome after seeing the scores.
Operator or domain expert	Tests workflow fit, review effort, and exception handling.	Approve a consequence they do not own.
Risk or control reviewer	Identifies unacceptable actions, data exposure, and required oversight.	Select the business opportunity solely because it is safest.
Affected user or customer representative	Describes the experience and harms the current workflow creates.	Turn a preference into validated demand without evidence.

NIST's AI Risk Management Framework calls for the business context and value to be defined, organizational risk tolerances to be documented, and relevant actors to understand system knowledge limits and human oversight. The NIST AI RMF Core gives this role separation a useful boundary: risk and oversight must be explicit before a team treats a score as permission.

Write the decision rights down before the meeting. A compact record is enough:

Outcome owner:
Decision date:
Business outcome:
Success measure:
Resource sponsor:
Required reviewers:
Non-negotiable vetoes:
Tie-breaker if eligible options remain close:
Evidence that could change the choice:

The tie-breaker is important. Without it, the group often discovers its real decision rule only after the scores disagree. That is when the loudest person, the newest demo, or the highest total silently becomes the rule.

Apply vetoes before you calculate a winner

A veto answers whether an opportunity may enter the ranking. A score answers how attractive it is after it is eligible. Do not put an unacceptable action into a 0-to-5 column and let strong upside compensate for it.

Use four pre-score gates:

1. Named owner. One person can accept the outcome, inspect failures, and pause the work.
2. Checkable result. The team can define pass, edit, reject, or another observable outcome.
3. Data boundary. The team knows what information the first release may access and whether the chosen tool is approved for it.
4. Reversible first slice. The pilot can start as retrieval, classification, drafting, recommendation, or another bounded action before it sends a consequential message or changes a record.

If a gate fails, route the candidate to preparation or rejection. Do not score it as a weak project.

This is where AI opportunity ranking differs from a generic feature vote. A theoretical benefit cannot compensate for an action the organization has not agreed to authorize. NIST frames risk tolerance as contextual and use-case specific, and says risk-management resources should be allocated based on assessed and prioritized risk. The NIST AI RMF Core supports using the organization's own threshold rather than pretending there is one universal AI-risk number.

The gate also protects the score from a common category error. “Automate invoice approval” and “extract invoice fields for review” may be discussed as one opportunity, but their action boundaries are different. They should not share one score.

Score independently with a shared rubric, then declare the weights

Once a candidate passes the gates, ask stakeholders to score it independently against the same anchored criteria. Discuss the reasons after the first scores are recorded.

The UK Government's options-appraisal guidance recommends individual consideration before group discussion when multiple stakeholders are involved, because this can reduce group-think. It also recommends pre-agreed weightings for calculating option scores. The GOV.UK options-appraisal guidance supports the procedure. The Green Book supplementary guidance on multi-criteria decision analysis gives the broader MCDA context and warns that the method has pitfalls.

Use a 0-to-5 scale with visible anchors. The numbers are a comparison aid, not a probability of success.

Criterion	0	3	5
Outcome value	No material outcome	Noticeable improvement for a named owner	Important outcome with a credible path to value
Evidence quality	Mostly an opinion	Some examples or a usable baseline	Repeated, traceable evidence and a clear unknown
Workflow fit	No stable workflow	Stable enough to test with known exceptions	Clear inputs, outputs, owner, and review path
Feasibility	Critical dependency unknown	Small pilot is possible with preparation	Data, access, and integration boundary are ready
Controllability	Wrong output creates unacceptable action	Review can contain most errors	First slice is reversible, reviewable, and easy to stop
Learning value	Pilot would teach little	Pilot tests one meaningful assumption	Pilot resolves a decision that changes the next investment

The shared rubric creates comparable measurements. It does not require identical judgments. That is the point of recording the reasons for a 1 beside the reasons for a 5.

For the role aggregation, use a formula agreed before the results are seen:

roleWeighted(candidate) =
  ownerWeight * ownerRating
  + sponsorWeight * sponsorRating
  + operatorWeight * operatorRating
  + riskWeight * riskRating

The weights express decision rights, not a claim that one role is more intelligent. If the outcome owner carries the consequence, that role may receive more weight on the eligible-choice score. A risk reviewer may have less preference weight but still retain a veto over unacceptable actions.

Do not change the weights to rescue a favored candidate. If a weight change is reasonable, record the reason and rerun every candidate through the new rule.

Diagnose a cycle instead of hiding it in an average

A preference cycle means the group has no unique winner under the information and tie-breaker currently available. The right next move is a targeted evidence sprint, not another round of persuasion.

In Fixture 1, the rankings were:

Stakeholder	1st	2nd	3rd
Operator	A	B	C
Product	B	C	A
Finance	C	A	B

The pairwise result is:

Comparison	Votes
A over B	2
B over C	2
C over A	2

Rank-sum gives A = 3, B = 3, and C = 3. A majority vote cannot break the cycle because each candidate beats one other candidate and loses to another. Choosing A, B, or C requires a rule that is not present in the rankings.

Use these triggers for an evidence sprint. They are operational rules for this artifact, not industry thresholds:

• The pairwise rankings cycle.
• The weighted winner changes when a declared, defensible role weight changes.
• The top two eligible candidates are less than 0.5 apart on a 0-to-5 weighted score.
• A stakeholder raises a concrete objection that the scorecard does not measure.
• The candidate depends on an unknown baseline, data permission, review burden, or exception rate.

The sprint should answer the disputed question, not gather more opinions.

Disagreement	Evidence task	Stop condition
“This will save more time.”	Observe the current workflow and measure elapsed time, waiting, correction, and exception work.	The baseline is stable enough to compare, or the opportunity is too irregular to rank.
“Operators will not trust it.”	Run a blind comparison or rehearsal with real operators on a safe sample.	Reviewers can name acceptable output and failure handling, or the role is redesigned.
“The data is not ready.”	Inspect representative inputs, permissions, missing fields, and source-of-truth ownership.	A bounded source set is available, or the candidate returns to preparation.
“The risk is too high.”	Define the action boundary, human approval, escalation, and rollback path with the control owner.	A reversible first slice exists, or the candidate is rejected.

NIST's Measure Playbook says the purpose and context affect which metrics are appropriate, calls for documented test sets and metrics, and recommends testing prototypes with end users early and continuously. The NIST measurement guidance supports this move from argument to evidence. The SBA gives small businesses a simpler version of the same practical advice: start small and test whether the tool adds value. SBA's AI guidance is not a ranking formula, but it is a sensible constraint on the next step.

The four-candidate fixture shows how a veto changes the result

Fixture 2 makes the procedure concrete. Four roles scored four candidates. Each role preferred a different one:

Role	A	B	C	D	Role's winner
Operations	5	3	2	4	A
Product	3	2	5	4	C
Finance	2	5	3	4	B
Risk	1	2	3	5	D

Equal averaging produces A = 2.75, B = 3.00, C = 3.25, and D = 4.25. Declared role weights of Operations 0.30, Product 0.40, Finance 0.15, and Risk 0.15 produce A = 3.15, B = 2.75, C = 3.50, and D = 4.15.

The fixture then applies one eligibility rule: D's first release would trigger an external customer action, and the candidate has no reversible read-only slice. D is not given a zero for controllability while remaining in the race. It is removed until the action boundary changes. C is the highest-scoring eligible candidate at 3.50.

That conclusion is deliberately narrow. The experiment does not say “always choose C.” It says:

Rank preferences only after eligibility is established. If a candidate fails a non-negotiable action or data boundary, change the candidate or remove it. Do not let a high average turn an unapproved action into an approved one.

This also explains why a risk reviewer can have a different winner without being the person who chooses the business priority. Risk can identify what must not happen. The outcome owner chooses among the candidates that remain acceptable.

Keep opportunities in separate queues when one ranking would be dishonest

Not every disagreement should end in one ordered list. Split the portfolio when the candidates have different decision horizons, evidence maturity, or consequence classes.

Use at least three queues:

Queue	Meaning	Next action
Eligible to rank	Owner, outcome, data boundary, review method, and reversible slice are defined.	Score independently, aggregate with declared weights, and choose or test the leader.
Evidence needed	The opportunity may matter, but a key assumption is unverified.	Run the smallest evidence task with an owner and stop condition.
Do not automate this slice	The first action is not governable, reversible, or proportionate to the evidence.	Narrow the action, keep it human-owned, or reject it.

This is useful when a strategic platform idea competes with a small operational improvement. A single score can make the platform look valuable because its upside is large, while the operational idea has better evidence and a shorter path to learning. They may both deserve attention, but not the same decision meeting.

The parent page, AI opportunity pillar, is the right place for the wider opportunity curriculum. For the general small-business portfolio decision after a candidate becomes eligible, see How to Prioritize AI Use Cases in a Small Business. This article adds the missing step for the meeting where the stakeholders still disagree.

Move from ranking to a reversible pilot

Once one eligible candidate has a defensible lead, record the dissent and test the assumption that matters most. A score is a decision aid. It is not evidence that the workflow will work.

Use this handoff:

1. Record the decision. Save the candidate card, role scores, weights, vetoes, dissenting reasons, and the condition that would reopen the choice.
2. Define the smallest slice. Prefer retrieval, extraction, classification, drafting, or recommendation before an external action. Name the human reviewer and escalation path.
3. Write the baseline. Measure the current task in the terms the owner cares about: elapsed time, correction effort, queue delay, missed information, or another fit-for-purpose outcome.
4. Run a contained comparison. Use representative cases, preserve failures, and compare the AI-assisted path with the current path. Do not replace the baseline with a satisfaction score alone.
5. Re-rank only after learning. Continue, narrow, redesign, pause, or reject. If the evidence changes the opportunity card, change the score rather than defending the old winner.

If your team needs help turning a disagreement into a testable AI opportunity, Marius Manolachi works as an AI consultant and AI tutor. The useful engagement is the one that leaves the team able to make the next ranking and run the next evidence sprint itself. Learn about Marius's AI consulting and tutoring work.