Panel · Template

Prioritisation Panel.

LivePrioritisation Panel

Put a list of items in front of specialists who each rank every item alone, in their own shuffled order, and get an aggregated ranking, a cut line, where the seats disagree, and how stable the line is when the weights or the items change.

You getA prioritisation report of about four pages (Standard)

DepthsLight (three seats), Standard (four) or Full (five)

Written byAI specialists who rank in separate, shuffled orders

SourcesEvery ranking reason tied to a stated criterion

Add it to your library to reuse from New → From template. Edit the saved copy in your library on the service form.

When to use it

You have more candidates than you can fund, sequence or build, and you want a ranking from informed specialists that shows where they disagree and whether the cut line holds.

Who it is for. Leads who must choose what comes first: evidence, access, development, safety, business development, digital health and brand leads sequencing studies, markets, assets, signals, partners or features.

Why a ranking needs its criteria first

Reviews of multiple criteria decision analysis in health report that criteria are often neither exhaustive nor mutually exclusive, and that the final ranking can depend on the scale chosen. With AI rankers there is a second hazard: position bias, where the order of the list changes the result. This template fixes the criteria and the scale before any item is ranked, gives each seat the items in its own shuffled order, and reports how stable the cut line is when the weights or the items change.^{3Marsh, K. et al. Multiple Criteria Decision Analysis for Health Care Decision Making: Emerging Good Practices: Report 2 of the ISPOR MCDA Emerging Good Practices Task Force. Value in Health, 2016;19(2):125–137.,2Permutation Self-Consistency Improves Listwise Ranking in Large Language Models. NAACL 2024.}

What you get

The question, the criteria and the items. What is being prioritized and for which decision, the criteria with one sentence each and the scale used, and the numbered items. Near-duplicate items are clustered before ranking.^{3Marsh, K. et al. Multiple Criteria Decision Analysis for Health Care Decision Making: Emerging Good Practices: Report 2 of the ISPOR MCDA Emerging Good Practices Task Force. Value in Health, 2016;19(2):125–137.}

The cut line and the top group. The answer first: which items fall above the line, how many, and the stated rule for where the line sits.^{1Cochrane Methods Priority Setting. James Lind Alliance Priority Setting Partnerships.}

How each seat ranked. Each seat’s full rank order, its top three and bottom three with a reason tied to a criterion. Each seat received the items in its own random order.^{2Permutation Self-Consistency Improves Listwise Ranking in Large Language Models. NAACL 2024.,4CASRAI. Nominal group technique guide.}

The aggregated ranking, and where the seats disagree. The ranks combined by a stated rule, with agreement reported apart from the ranking. A table shows the items on which the seats split most.^{8Free-MAD: consensus-free multi-agent debate. arXiv:2509.11035.}

Value versus effort, sensitivity, and limits. Items placed by value and effort using proxies, labeled as proxies; how the cut line moves if a criterion weight or an item changes; and what would change the ranking.^{5Cochrane Methods Priority Setting. Child Health and Nutrition Research Initiative (CHNRI) approach to research priority setting.,6Evidence and Value: Impact on DEcisionMaking (EVIDEM): core model and contextual tool for health decisions.,7Value of Information Analytical Methods: Report 2 of the ISPOR Value of Information Analysis Emerging Good Practices Task Force. Value in Health, 2020;23(3).}

What is in it, by length

Section	Brief	Standard	Comprehensive
The question, criteria and itemsThe decision, the criteria, the scale and the items	●	●	●
The cut line and the top groupWhich items are above the line and why	●	●	●
How each seat rankedEach seat’s rank order and reasons	●	●	●
The aggregated rankingThe combined ranking and the rule used	●	●	●
Where the seats disagreeThe items on which the seats split most	–	●	●
Value versus effortItems placed by value and effort	–	●	●
Sensitivity of the cut lineHow the line moves when inputs change	–	●	●
Limits and what would change the rankingThe limits of the exercise and the evidence that would change it	●	●	●

A Light run has three seats and about two pages and stays readable with ten items. Standard has four seats and adds the disagreement, value versus effort and sensitivity sections. Full has five seats and adds a rebuttal round.

Choosing a length

Choose Standard for most lists. Choose Full when the cut line sits between items that seats rank very differently and you want to see which rankings survive challenge.

The standards and practice behind it

The structure and the checks come from published work on priority setting, multi-criteria decision analysis and ranking by language models. These are the references we drew on and what we took from each. None of their authors has reviewed the template.

What we took	From	Where it shows
Cut a long list in stages, rank individually first, and agree a final top group	The James Lind Alliance Priority Setting Partnership method, as summarized by the Cochrane priority-setting methods group^{1Cochrane Methods Priority Setting. James Lind Alliance Priority Setting Partnerships.}	Independent ranking before any shared view, and a stated cut line
Shuffle the candidate list several times and aggregate, to cancel order effects in language-model ranking	Permutation Self-Consistency Improves Listwise Ranking in Large Language Models (NAACL, 2024)^{2Permutation Self-Consistency Improves Listwise Ranking in Large Language Models. NAACL 2024.}	Each seat ranks in its own random order
Define criteria, scales and weights before scoring, and use a checklist for designing and reviewing the analysis	The ISPOR MCDA Emerging Good Practices Task Force, Report 2 (Marsh and colleagues, 2016)^{3Marsh, K. et al. Multiple Criteria Decision Analysis for Health Care Decision Making: Emerging Good Practices: Report 2 of the ISPOR MCDA Emerging Good Practices Task Force. Value in Health, 2016;19(2):125–137.}	The criteria and scale stated before any ranking
Rank silently first, clarify meaning only, then vote	The nominal group technique, as described in the CASRAI guide^{4CASRAI. Nominal group technique guide.}	Independent first ranks and clarification limited to meaning
Score options against a short set of criteria and say when weights are equal	The Child Health and Nutrition Research Initiative (CHNRI) approach to research priority setting^{5Cochrane Methods Priority Setting. Child Health and Nutrition Research Initiative (CHNRI) approach to research priority setting.}	A short criteria list and a statement of the weights used
Use a core value framework for health decisions that draws on criteria and evidence	EVIDEM, the framework used in health technology assessment^{6Evidence and Value: Impact on DEcisionMaking (EVIDEM): core model and contextual tool for health decisions.}	The criteria drawn from established decision frameworks
Rank evidence gaps by the value of the evidence they would provide, and treat proxies as proxies	ISPOR Value of Information Analysis Emerging Good Practices Task Force, Report 2^{7Value of Information Analytical Methods: Report 2 of the ISPOR Value of Information Analysis Emerging Good Practices Task Force. Value in Health, 2020;23(3).}	The value versus effort section, with proxies labeled
Expect agents that share a base model to drift toward the majority and amplify shared errors	Free-MAD: consensus-free multi-agent debate (arXiv)^{8Free-MAD: consensus-free multi-agent debate. arXiv:2509.11035.}	Agreement reported apart from the ranking, and not read as independent confirmation

What it will not do

It is a simulation by AI specialists and not a panel of real experts. Seats that share a base model are not independent confirmations of each other.

It does not compute a value-of-information analysis. That needs a decision model. It ranks by proxies and says so.

It does not set the weights for you. If none are supplied, the criteria are weighted equally and the report says so.

References

[1] Cochrane Methods Priority Setting. James Lind Alliance Priority Setting Partnerships. Read in the template research: the stages, the interim cut to fewer than 30 questions and the final workshop. The JLA Guidebook itself was not retrieved.
[2] Permutation Self-Consistency Improves Listwise Ranking in Large Language Models. NAACL 2024. Abstract read in the template research.
[3] Marsh, K. et al. Multiple Criteria Decision Analysis for Health Care Decision Making: Emerging Good Practices: Report 2 of the ISPOR MCDA Emerging Good Practices Task Force. Value in Health, 2016;19(2):125–137. Abstract only; the full text was behind a login.
[4] CASRAI. Nominal group technique guide. Known from search results; the guide was not read in full. Published accounts differ in detail.
[5] Cochrane Methods Priority Setting. Child Health and Nutrition Research Initiative (CHNRI) approach to research priority setting. Known from search results; the page was not opened.
[6] Evidence and Value: Impact on DEcisionMaking (EVIDEM): core model and contextual tool for health decisions. Known from search results; the paper was not read in full.
[7] Value of Information Analytical Methods: Report 2 of the ISPOR Value of Information Analysis Emerging Good Practices Task Force. Value in Health, 2020;23(3). Known from search results; the report was not read in full.
[8] Free-MAD: consensus-free multi-agent debate. arXiv:2509.11035. Known from search results as a study of conformity in multi-agent debate; the paper was not read in full.

Links checked 9 October 2026

These are the sources the template drew on. None of their authors or issuers has reviewed it.

About the Panel service

Each panelist answers in their own voice, then the discussion is drawn together. You get the question and who was in the room, every position in full, what they agreed on, where they split, and what follows. Disagreements are written down rather than smoothed into a consensus nobody actually holds, because the split is usually the part you needed.

Use cases for the Prioritisation Panel

8 ready-to-run use cases. Each opens its own page with the question, the specialists, the standards and the references.

EGLiveEvidence Gap PrioritisationOf these 14 open evidence gaps in our dossier, which should we close first to support payer submissions?Prioritisation PanelMarket Access

Open Open →

LMLiveLaunch Market SequenceRank these 12 candidate countries for launch sequence, given our access and pricing criteria.Prioritisation PanelMarket Access

Open Open →

PALivePipeline Asset RankingRank these 11 pipeline assets for investment, against our criteria of unmet need, evidence plan and regulatory path.Prioritisation PanelCorporate Development

Open Open →

PSLivePost-Marketing Study TriageWhich of these 13 proposed post-marketing studies should we fund first?Prioritisation PanelMedical Affairs

Open Open →

SSLiveSafety Signal Investigation OrderRank these 10 open safety signals by how soon we should investigate them.Prioritisation PanelPharmacovigilance

Open Open →

PTLivePartnership Target RankingRank these 12 candidate partners for a co-development deal against our fit criteria.Prioritisation PanelInnovation

Open Open →

DFLiveDigital Feature PrioritisationOf these 15 candidate features for our digital companion, which should the team build first?Prioritisation PanelInnovation

Open Open →

PILivePolicy Issue PrioritiesRank the policy issues on your agenda for engagement, with criteria fixed first, the stage of each issue recorded and the cut line tested for sensitivity.Prioritisation PanelPublic Affairs

Open Open →

To edit a saved template, open New → Panel → From template, or the library rail on the service form.

When to use it

You have more candidates than you can fund, sequence or build, and you want a ranking from informed specialists that shows where they disagree and whether the cut line holds.

Why a ranking needs its criteria first

What you get

What is in it, by length

Section	Brief	Standard	Comprehensive
The question, criteria and itemsThe decision, the criteria, the scale and the items	●	●	●
The cut line and the top groupWhich items are above the line and why	●	●	●
How each seat rankedEach seat’s rank order and reasons	●	●	●
The aggregated rankingThe combined ranking and the rule used	●	●	●
Where the seats disagreeThe items on which the seats split most	–	●	●
Value versus effortItems placed by value and effort	–	●	●
Sensitivity of the cut lineHow the line moves when inputs change	–	●	●
Limits and what would change the rankingThe limits of the exercise and the evidence that would change it	●	●	●

The standards and practice behind it

What we took	From	Where it shows
Cut a long list in stages, rank individually first, and agree a final top group	The James Lind Alliance Priority Setting Partnership method, as summarized by the Cochrane priority-setting methods group^{1Cochrane Methods Priority Setting. James Lind Alliance Priority Setting Partnerships.}	Independent ranking before any shared view, and a stated cut line
Shuffle the candidate list several times and aggregate, to cancel order effects in language-model ranking	Permutation Self-Consistency Improves Listwise Ranking in Large Language Models (NAACL, 2024)^{2Permutation Self-Consistency Improves Listwise Ranking in Large Language Models. NAACL 2024.}	Each seat ranks in its own random order
Define criteria, scales and weights before scoring, and use a checklist for designing and reviewing the analysis	The ISPOR MCDA Emerging Good Practices Task Force, Report 2 (Marsh and colleagues, 2016)^{3Marsh, K. et al. Multiple Criteria Decision Analysis for Health Care Decision Making: Emerging Good Practices: Report 2 of the ISPOR MCDA Emerging Good Practices Task Force. Value in Health, 2016;19(2):125–137.}	The criteria and scale stated before any ranking
Rank silently first, clarify meaning only, then vote	The nominal group technique, as described in the CASRAI guide^{4CASRAI. Nominal group technique guide.}	Independent first ranks and clarification limited to meaning
Score options against a short set of criteria and say when weights are equal	The Child Health and Nutrition Research Initiative (CHNRI) approach to research priority setting^{5Cochrane Methods Priority Setting. Child Health and Nutrition Research Initiative (CHNRI) approach to research priority setting.}	A short criteria list and a statement of the weights used
Use a core value framework for health decisions that draws on criteria and evidence	EVIDEM, the framework used in health technology assessment^{6Evidence and Value: Impact on DEcisionMaking (EVIDEM): core model and contextual tool for health decisions.}	The criteria drawn from established decision frameworks
Rank evidence gaps by the value of the evidence they would provide, and treat proxies as proxies	ISPOR Value of Information Analysis Emerging Good Practices Task Force, Report 2^{7Value of Information Analytical Methods: Report 2 of the ISPOR Value of Information Analysis Emerging Good Practices Task Force. Value in Health, 2020;23(3).}	The value versus effort section, with proxies labeled
Expect agents that share a base model to drift toward the majority and amplify shared errors	Free-MAD: consensus-free multi-agent debate (arXiv)^{8Free-MAD: consensus-free multi-agent debate. arXiv:2509.11035.}	Agreement reported apart from the ranking, and not read as independent confirmation

What it will not do

It is a simulation by AI specialists and not a panel of real experts. Seats that share a base model are not independent confirmations of each other.

It does not compute a value-of-information analysis. That needs a decision model. It ranks by proxies and says so.

It does not set the weights for you. If none are supplied, the criteria are weighted equally and the report says so.