Brief · Use case · Regulatory
AI Screening Accuracy Evidence Review.
LiveEvidence ReviewReview the evidence on the diagnostic accuracy of AI diabetic retinopathy screening against specialist grading, with certainty rated.
AI Screening Accuracy Evidence Review is a ready-made use case for Regulatory. It ships under the Evidence Review template.
For regulatory and clinical leads. You get the PICO, the main studies with accuracy results and risk of bias, certainty per outcome with the reason and what the evidence does not cover. Studies are cited or marked as gaps.
Add it to your library to reuse from New → From template. Edit the saved copy in your library on the service form.
The decision
What does the published evidence show on the diagnostic accuracy of AI-based screening for diabetic retinopathy compared with specialist grading?
What is reviewed
This is a diagnostic accuracy question, so the outcomes are sensitivity and specificity and the review reports both rather than one summary.
| Evidence | What it holds | What the review does with it |
|---|---|---|
| Pivotal trial | Abràmoff and colleagues (2018) enrolled 900 adults with diabetes at ten primary care sites, graded against the Wisconsin Fundus Photograph Reading Center. Reported sensitivity was 87.2 percent, specificity 90.7 percent and imageability 96.1 percent, as given in press coverage of the presentation. The study was funded by the company that made the system.1 | The paper itself was not read; figures come from news coverage. |
| Meta-analyses | A 2023 meta-analysis of prospective studies reported pooled sensitivity of 0.880 and specificity of 0.912. A 2024 analysis of 34 real-world studies concluded AI screening has acceptable performance.2,3 | Pooled values reported as found; one interval in an abstract looks garbled and is flagged. |
| Head-to-head and real-world studies | A Polish study of 758 patients compared two systems: one had sensitivity 99.3 percent with specificity 68.9 percent, the other 89.4 and 94.8 percent. Positive predictive value was 48.1 percent against 96.4 percent.4 | Shows how operating points differ between systems. |
What the review does
Report both halves of accuracy. A system can catch nearly all disease and still send many healthy people to a specialist. The review shows sensitivity, specificity and positive predictive value together, so the reader sees the trade-off.4
Pivotal and real-world results differ. Performance varies with the algorithm, the setting, the patient mix and image quality. The review separates results from controlled studies from those in routine care.2,3
Independence of the evidence. The pivotal trial was funded by the maker of the system. The review notes funding for each study and does not treat it as a reason to discard the result.1
Recommended length
Standard
The question is narrow and the studies are few enough to tabulate. Standard carries the PICO, the studies, the effect, the certainty and the open question in about 1,300 words.
You can pick another length on the form. The other two stay available.
Assumptions the review checks
The review states the assumptions its question and its rating rest on, and says what each gap limits. Examples for this question:
- Reading-center grading is the right reference for the question.
- Results for one system apply to the system the reader has in mind.
- The studies’ patient mix resembles the screening population of interest.
Standards, guidance and sources
Evidence
The decision practice behind the memo itself is on the template page.
What gets flagged
Any figure taken from news coverage, any pooled interval that cannot be read, and any study funded by the maker of the system.
What arrives
- The question and the bottom line
- How the evidence was found
- The evidence
- What it shows
- How strong it is
- Where studies disagree
- Where the evidence is thin
- What the evidence does not answer
What it will not do
It is a rapid review, not a systematic one. It reports the effects the studies reported, does not pool them into a new estimate, and labels every certainty rating as analyst judgment.
It does not recommend a system or set a regulatory strategy.
It is not a systematic review. It uses a bounded search, says so, and recommends a full literature review when completeness matters.
It does not pool studies into a new estimate. It reports the effects the studies reported.
It does not compute a missing interval or plot a missing effect as zero. Both are marked as gaps.
References
- [1] Abràmoff, M. D. et al. Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices. 2018. Figures as reported by Healio. News coverage of the presentation; the paper was not read.
- [2] Performance of artificial intelligence in diabetic retinopathy screening: a systematic review and meta-analysis of prospective studies. 2023. Pooled figures as given in search excerpts; one interval in the abstract appears garbled. Check the full text.
- [3] Joseph, S. et al. Diagnostic test accuracy of artificial intelligence in screening for referable diabetic retinopathy in real-world settings: a systematic review and meta-analysis. American Journal of Ophthalmology, 2024. Link returned by search for the 2023–2024 real-world meta-analyses; check that it is the paper cited.
- [4] Diagnostic accuracy of automated diabetic retinopathy image assessment software: IDx-DR and RetCAD. Figures as given in search excerpts; the paper was not read in full.
Links checked 9 October 2026
These are the sources the template drew on. None of their authors or issuers has reviewed it.
About the Brief service
A Brief is a written answer from one specialist, at the depth you asked for. It arrives with its sources, and it closes by saying plainly what would change the conclusion. Nothing is hidden behind a summary, and anything the expert could not source is marked as a gap rather than filled in.
To edit a saved template, open New → Brief → From template, or the library rail on the service form.