Artificial intelligence for diagnosis and Gleason grading of prostate cancer: the PANDA challenge Wouter Bulten, Kimmo Kartasalo, Po-Hsuan Cameron Chen · et al. and the PANDA challenge consortium — Nature Medicine, 13 January 2022 · Open Access (CC BY 4.0 article; dataset CC BY-SA-NC 4.0) 10.1038/s41591-021-01620-2
TL;DR — The Prostate cANcer graDe Assessment (PANDA) challenge released 10,616 digitized, de-identified hematoxylin and eosin (H&E)-stained whole-slide prostate biopsy images (WSIs) — the largest public prostate-biopsy dataset to date — and ran the largest histopathology competition to date (1,290 developers, 1,010 teams, 65 countries). A diverse set of submitted deep-learning algorithms reached pathologist-level Gleason grading, agreeing with expert uropathologists at a quadratically weighted κ of 0.862 (United States external set) and 0.868 (European external set). This data-lake folder lands everything openly available (the Open Access paper, Supplementary Information, Source Data, the analysis-code snapshot and the preregistration); the primary whole-slide images themselves are distributed only through a gated Kaggle competition and are documented but not downloaded here.
0. Summary
Biological premise. Gleason grading of prostate biopsies yields key prognostic information for prostate-cancer patients and drives treatment planning. Pathologists classify tumor tissue into Gleason growth patterns and, from their distribution, assign one of five International Society of Urological Pathology (ISUP) grade groups (GGs 1–5). This assessment is inherently subjective, with considerable inter- and intra-observer variability, leading to both under- and over-grading. Artificial intelligence (AI) has shown promise for automating Gleason grading, but prior results were siloed to individual studies and lacked multinational, independently blinded validation — a central barrier to clinical implementation.
To address this, the authors organized the PANDA challenge: they compiled and publicly released a large European (EU) development cohort of prostate biopsies for AI development, hosted a global competition on the Kaggle platform, then independently and blindly reproduced and validated the top algorithms on separate United States and European cohorts against reference standards set by intercontinental panels of expert uropathologists. The competition design isolated algorithm developers from the independent evaluation, minimizing information leakage and positive bias.
The main finding is that AI algorithms obtained from a competition setup could detect and grade prostate tumors at pathologist-level concordance with expert reference standards, generalizing across different patient populations, laboratories, scanner vendors and reference standards. The result warrants prospective clinical-trial evaluation of AI-based Gleason grading.
Every landed artifact in this folder is either the peer-reviewed report of this challenge, its publisher-hosted supplements and source data, or the openly released analysis code and preregistration; the biopsy imagery it describes is a whole-slide histopathology readout of human prostate needle biopsies collected retrospectively across six sites in three countries.
Assay and processing. The primary data modality is brightfield whole-slide imaging (WSI) of H&E-stained prostate needle biopsies, digitized with four scanner models from three vendors (3DHISTECH, Hamamatsu Photonics, Leica Biosystems) and exported with the open-source ASAP software (v1.9) before upload to Kaggle. Each biopsy carries a case-level label: ISUP grade group (0/benign–5) and Gleason score; the Radboud subset additionally ships pixel-level Gleason/benign segmentation masks. Competing teams trained deep-learning models (predominantly patch-based convolutional neural networks with weak, case-level supervision, iterative label denoising and diverse model ensembles) built on PyTorch and TensorFlow inside the Kaggle Docker environment. For the independent validation phase, algorithms were reproduced on two systems (Google Cloud and the Puhti cluster via Singularity v3.8.3) and scored with analysis scripts written in Python (v3.8) using scipy 1.5.4, pandas 1.1.4, mlxtend 0.18.0, numpy 1.19.4, scikit-learn 0.23.2, matplotlib 3.3.2, jupyterlab 2.2.9 and notebook 6.1.5. The completeness tiers for this imaging-histology dataset are: Tier 1 (must-have) — whole-slide images + case-level grade labels; Tier 2 (recommended) — pixel-level tissue masks + per-site metadata; Tier 3 (optional) — held-out test/validation splits and per-algorithm prediction outputs. In this folder, Tier-1/Tier-2 imagery is gated (not downloaded); the tabular Source Data, code and documentation that describe and score them are present.
Sources.
prostate-cancer-grade-assessment(Kaggle competition) — the PANDA challenge development set: 10,616 WSIs (train_images/), pixel-level masks (train_label_masks/) and case-level labels (train.csv). Access-gated; see this dataset'sinput/kaggle/README_KAGGLE.md.10.5281/zenodo.5592578— PANDA Challenge Analysis Code snapshot (GitHubDIAGNijmegen/panda-challengev1.0.0), including the official getting-started notebook.10.5281/zenodo.3715938— PANDA challenge preregistration (study design document).- Code: DIAGNijmegen/panda-challenge (MIT/Apache-style research license in the archive; analysis + evaluation code and example submissions).
- Springer Nature supplements & source data: Supplementary Information (
41591_2021_1620_MOESM1_ESM.pdf), Reporting Summary (…MOESM2_ESM.pdf), Source Data Fig. 2 (…MOESM3_ESM.xlsx), Source Data Extended Data Fig. 7 (…MOESM4_ESM.xlsx). - Challenge hub: panda.grand-challenge.org.
- PMIDs: 35027755 · PMCID PMC8799467.
| Disease | Prostate cancer (acinar adenocarcinoma), graded by International Society of Urological Pathology (ISUP) grade group |
| Assay / modality | Whole-slide imaging (WSI) of hematoxylin and eosin (H&E)-stained prostate needle biopsies |
| Organisms | Human (Homo sapiens) |
| Model systems | Retrospective clinical prostate needle-biopsy specimens (no cell lines / animal models) |
| Repositories | Kaggle competition (primary images, gated); Zenodo (code + preregistration); Springer Nature (supplements + source data) |
| Accessions | Kaggle prostate-cancer-grade-assessment; Zenodo 10.5281/zenodo.5592578, 10.5281/zenodo.3715938 |
| Access | Article Open Access (CC BY 4.0); image + tabular dataset gated (Kaggle rules + CC BY-SA-NC 4.0) — authenticated kagglehub access works (user nhthongphan) but downloads return HTTP 403 until the competition rules are accepted; code + preregistration open |
| Number of studies / cohorts | 5 dataset roles (development, tuning, internal validation, US external validation, EU external validation) across 6 sites |
| Number of files (landed) | 9 |
| Total size (landed) | 17,783,403 bytes (≈ 17.78 MB) |
| Biological samples | Development set = 10,616 biopsies (from 2,113 patients); full study = 12,625 biopsies / 3,500 cases across all roles. "Sample" = one digitized prostate biopsy (one image_id / one WSI). Primary images are gated and not on disk. |
| Instrument | 3DHISTECH, Hamamatsu Photonics and Leica Biosystems whole-slide scanners (4 models, 3 vendors); slides exported with ASAP v1.9 |
| Reference resources | Kaggle Docker image; PyTorch; TensorFlow |
| Code license | Open research-use license bundled in the Zenodo/GitHub code archive |
1. Datasets — sample sources, protocols, experiment design and metadata
Landed sources at a glance
| Source | Access | Files |
|---|---|---|
Kaggle competition prostate-cancer-grade-assessment (primary WSIs + train.csv) | Gated — auth works (kagglehub, user nhthongphan) but rules not accepted → HTTP 403; documented, not downloaded | 1 (README only) |
Zenodo 10.5281/zenodo.5592578 — analysis code + getting-started notebook | Open (CC BY) | 1 |
Zenodo 10.5281/zenodo.3715938 — preregistration | Open (CC BY) | 1 |
| Springer Nature supplements & source data | Open (CC BY) | 4 |
| Paper (Open Access, EuropePMC) | Open (CC BY) | 2 |
The cohorts at a glance (per Table 1 of the paper; "sample" = one biopsy/WSI)
| Role | Source site(s) | Organism | Cases | Biopsies |
|---|---|---|---|---|
| EU development set | Radboud University Medical Center (Netherlands) + Karolinska Institutet (Sweden) | Human | 2,113 | 10,616 |
| EU tuning set | Radboud + Karolinska | Human | 105 | 393 |
| EU internal validation set | Radboud + Karolinska | Human | 211 | 545 |
| US external validation set | 2 medical laboratories + 1 tertiary teaching hospital (USA) | Human | 741 | 741 |
| EU external validation set | Karolinska University Hospital (Sweden) | Human | 330 | 330 |
| Total (all roles) | 6 sites, 3 countries | Human | 3,500 | 12,625 |
1.1 Sample origins and model-system lineage
- Roots (clinical specimens). Retrospectively collected, de-identified prostate needle biopsies with associated diagnoses from routine pathology reports.
- Development / tuning / internal validation → Radboud University Medical Center (Nijmegen, Netherlands) and Karolinska Institutet (Stockholm, Sweden). All biopsies from a given patient were assigned to either development or validation to keep the internal validation set independent.
- US external validation → two independent US medical laboratories (CA/UT) and a tertiary teaching hospital (CA), with immunohistochemistry available to aid tumor identification.
- EU external validation → Karolinska University Hospital (Stockholm, Sweden); histology and scanning performed by laboratories distinct from those that produced the development data.
- De-identification. Specimens were de-identified; for the Swedish set informed consent was provided, and for the other sets consent was waived for retrospective use of de-identified specimens (IRB 2016-2275 Radboud; Stockholm ethics permits; Advarra Pro00038251).
- Where clinicopathology lives. Case-level ISUP grade group and Gleason score sit in the gated
train.csv; per-site cohort composition and ISUP-grade breakdowns are in the paper's Table 1 and Supplementary Tables 7–8 (input/supplement/41591_2021_1620_MOESM1_ESM.pdf).
1.2 Shared wet-lab protocol (bench → scanner) — covers all six sites
- Obtain prostate needle-biopsy specimens in routine clinical practice; formalin-fix, paraffin-embed and section.
- Stain sections with hematoxylin and eosin (H&E).
- Digitize each slide on a whole-slide scanner — four models across three vendors (3DHISTECH, Hamamatsu Photonics, Leica Biosystems); scanner details in Supplementary Table 1.
- Export slides with the open-source ASAP software (v1.9) to a multi-resolution
.tiffpyramid. - Establish the reference standard: training labels from routine clinical pathology reports (Dutch) or a single uropathologist following clinical workflow (Swedish); validation-set reference standards by consensus of 3–4 uropathologists (internal) or a 6-member US/Canadian panel (US external) or a single uropathologist (EU external).
1.3 Shared data-processing protocol (images → scores)
Whole-slide .tiff → patch sampling → CNN ensemble → case-level ISUP grade prediction → quadratically weighted κ agreement with the uropathologist reference standard.
| Step | Tool (version) | Fingerprint in the files |
|---|---|---|
| Slide export | ASAP (1.9) | multi-resolution .tiff WSIs in train_images/ (gated) |
| Model training | PyTorch / TensorFlow (Kaggle Docker image) | team algorithm containers (not in this folder) |
| Reproduction | Singularity (3.8.3) on Puhti; native Docker on Google Cloud | prediction CSVs (per-team submission.csv in the code archive) |
| Scoring | Python 3.8; scikit-learn 0.23.2; scipy 1.5.4; pandas 1.1.4; numpy 1.19.4; mlxtend 0.18.0 | src/evaluation/metrics.py, src/compute-metrics-all-teams.py in input/zenodo/DIAGNijmegen_panda-challenge-v1.0.0.zip |
prostate-cancer-grade-assessment (Kaggle) — PANDA challenge development set
Kaggle competition · gated · nothing downloaded — authenticated kagglehub access works but every file returns HTTP 403 until the competition rules are accepted (see input/kaggle/README_KAGGLE.md)
- Sample source. 10,616 de-identified H&E prostate needle-biopsy WSIs from Radboud University Medical Center and Karolinska Institutet.
- Samples and retention. 10,616 biopsies from 2,113 patients (development set). Full study spans 12,625 biopsies. Sample = one
image_id/ WSI. - Experiment design. Observational: publicly released development set for community algorithm training, with held-out tuning/internal/external validation sets used for independent, blinded scoring.
- Metadata available.
train.csvcolumns:image_id,data_provider(radboud/karolinska),isup_grade(0–5),gleason_score; pixel-level masks intrain_label_masks/(Radboud = Gleason-pattern classes; Karolinska = benign/cancer). Column names verified from the getting-started notebook shipped in the Zenodo code archive. - Files (annotated).
| File | What it is |
|---|---|
train.csv | case-level labels (10,616 rows) — gated |
train_images/<image_id>.tiff | one multi-resolution H&E WSI per biopsy — gated (hundreds of GB) |
train_label_masks/<image_id>_mask.tiff | pixel-level Gleason/benign masks — gated |
sample_submission.csv | submission format template — gated |
input/kaggle/README_KAGGLE.md | landed here: how to accept rules + download via the Kaggle API |
- How this experiment became a dataset. Retrospective biopsies → H&E → whole-slide scan → ASAP export → Kaggle release under CC BY-SA-NC 4.0 for the PANDA competition.
- Completeness (imaging-histology). Tier-1 imagery (WSIs) and Tier-1 labels are gated, so absent from
input/; Tier-2 masks likewise gated. The processed scoring layer (per-team predictions, evaluation code) is present via the Zenodo code archive, and cohort-level composition is quoted from the paper. No image bytes were fabricated.
Zenodo 10.5281/zenodo.5592578 — PANDA Challenge Analysis Code
Zenodo record · 1 file · 2,615,318 bytes (≈ 2.62 MB)
- Sample source. Software snapshot of GitHub
DIAGNijmegen/panda-challengeat tag v1.0.0 (commit2c13b53). - Files (annotated).
| File | What it is |
|---|---|
DIAGNijmegen_panda-challenge-v1.0.0.zip | 33-entry repo snapshot: evaluation code (src/evaluation/metrics.py, sampling.py), all-teams metric script, example submissions/reference CSVs, requirements.txt, and notebooks/getting-started-with-the-panda-dataset.ipynb (official data-loading example) |
- Completeness. Analysis/evaluation code and worked example present; team model weights are not in this archive (they live with the individual teams, per the Supplementary algorithm descriptions).
Zenodo 10.5281/zenodo.3715938 — PANDA challenge preregistration
Zenodo record · 1 file · 2,651,727 bytes (≈ 2.65 MB)
- Sample source. The preregistered study-design document defining the competition and validation phases before results were known.
- Files (annotated).
| File | What it is |
|---|---|
ThePANDAchallenge_ProstatecANcergraDeAssessmentusingtheGleasongradingsystem.pdf | preregistration PDF (study design, endpoints, validation plan) |
1.4 Publication-level metadata (Springer Nature supplements & source data)
Anchored to this dataset's input/supplement/ folder:
| File | What it is |
|---|---|
41591_2021_1620_MOESM1_ESM.pdf | Supplementary Information — Supplementary Methods 1–7, Supplementary Tables 1–9, algorithm descriptions |
41591_2021_1620_MOESM2_ESM.pdf | Nature Research Reporting Summary |
41591_2021_1620_MOESM3_ESM.xlsx | Source Data Fig. 2 — daily competition scores (94 rows × 5 columns: competition day; highest/median tuning and internal-validation κ) |
41591_2021_1620_MOESM4_ESM.xlsx | Source Data Extended Data Fig. 7 — per-algorithm quadratically weighted κ vs prior-work validation subsets (48 rows × 4 columns) |
The Open Access article and its extracted text are in this dataset's input/_paper/ folder.
2. Analyses this atlas supports
- Reproduce challenge scoring / leaderboard metrics. Start from
input/zenodo/DIAGNijmegen_panda-challenge-v1.0.0.zip→src/evaluation/metrics.py+src/compute-metrics-all-teams.pywith the examplesubmission.csv/referenceCSVs to recompute quadratically weighted κ, sensitivity and specificity exactly as reported. - Learn the data schema before requesting access. Read
notebooks/getting-started-with-the-panda-dataset.ipynb(in the code archive) to see howtrain.csvlabels map totrain_images/*.tiffandtrain_label_masks/*.tiffviaopenslide, so a downstream user can plan storage and preprocessing before pulling the gated archive. - Benchmark new Gleason-grading models. Once the Kaggle images are obtained (see
input/kaggle/README_KAGGLE.md), train on the 10,616-biopsy development set and evaluate against the ISUP-grade reference standard, reusing the challenge's κ-based evaluation harness for comparability with the 15 consortium algorithms. - Re-plot the paper's quantitative figures. Use
input/supplement/41591_2021_1620_MOESM3_ESM.xlsx(Fig. 2 competition progression) and…MOESM4_ESM.xlsx(Extended Data Fig. 7 vs prior work) directly. - Reproduce the figures / code. Clone DIAGNijmegen/panda-challenge (or unzip the Zenodo snapshot), install
requirements.txt, and run the evaluation module; the Supplementary algorithm descriptions ininput/supplement/41591_2021_1620_MOESM1_ESM.pdfpoint to the individual teams' released model code.
Citation
Bulten, W., Kartasalo, K., Chen, P-H. C., Ström, P., Pinckaers, H., Nagpal, K., Cai, Y., Steiner, D. F., van Boven, H., Vink, R., Hulsbergen-van de Kaa, C., van der Laak, J., Amin, M. B., Evans, A. J., van der Kwast, T., Allan, R., Humphrey, P. A., Grönberg, H., Samaratunga, H., Delahunt, B., Tsuzuki, T., Häkkinen, T., Egevad, L., Demkin, M., Dane, S., Tan, F., Valkonen, M., Corrado, G. S., Peng, L., Mermel, C. H., Ruusuvuori, P., Litjens, G., Eklund, M. & the PANDA challenge consortium. Artificial intelligence for diagnosis and Gleason grading of prostate cancer: the PANDA challenge. Nature Medicine 28, 154–163 (2022). 10.1038/s41591-021-01620-2
About this dataset
This Whole-slide histopathology imaging (H&E prostate biopsies); AI grading challenge (Gleason/ISUP) dataset provides single-cell gene expression from human prostate samples, with disease state recorded as Prostate cancer (acinar adenocarcinoma). 10x Genomics published it on January 13, 2022. The data were processed with Python 3.8; ASAP 1.9; Singularity 3.8.3; PyTorch; TensorFlow scipy 1.5.4; pandas 1.1.4; mlxtend 0.18.0; numpy 1.19.4; scikit-learn 0.23.2; matplotlib 3.3.2; jupyterlab 2.2.9; notebook 6.1.5, and the 0 output files total 17 MB.
Analyze it in Drylab
Open this dataset in Drylab and ask a question in plain language. The AI agent loads the files, runs the analysis in a ready sandbox and explains each result. A typical first pass:
- Load the filtered feature-barcode matrix (HDF5) and run quality control on counts, genes and mitochondrial reads per cell.
- Normalize, select variable genes, then cluster the cells and draw a UMAP.
- Annotate clusters with marker genes and compare the cell-type mix between conditions.
- Run differential expression between cell types or samples to characterize prostate cancer (acinar adenocarcinoma) biology.