Aim/Introduction: The design and validation of a reliable radiomic signature is challenging when few patients are available because the disease is rare, the imaging protocol is specific and/or the classes are unbalanced. In this context, we propose an approach to identify a signature and estimate its reliability. Materials and Methods: 84 patients with a clinical and MRI suspicion of recurrent glioblastoma were retrospectively included. Each patient underwent a 18F-FDOPA PET-CT scan. For each patient, the suspicious lesion was segmented and 49 radiomic features were calculated using LIFEx [1]. Our goal was to distinguish between tumor recurrence and radiation-induced necrosis as confirmed on pathological data, or on a 3-month clinical/imaging follow-up. A screening procedure was developed to identify a signature using leave-one-out (LOO) cross-validation. The procedure involved: 1) selection of all features with a p-value of univariate Wilcoxon test lower than alpha varying from 0.005 to 0.10 for (N-1) learning patients, 2) based on these features, selection of only one feature among correlated features using a Pearson correlation cutoff R varying from 0.95 to 0.50, 3) building of a radiomic signature involving the resulting features using a linear discriminant analysis, 4) test of the model on the Nth patient, 5) characterization of the model performance using the Youden Index (Y=sensitivity+specificity-1). The final model selection was based on the consistency of Y values as a function of alpha and R, on the consistency of selected features between the different models, and favored models involving a low number of features. To test the reliability of the selected signature, we repeated the process by excluding one patient using a jackknife procedure (ie, 84 LOO of 83 patients each). We compared the results with those obtained with the same alpha and R when randomly assigning a label to each patient (sham task). Results: 61 patients had tumor recurrence and 23 had radiation necrosis. Visual interpretation yielded Y equal to 0.35 (Lizarraga scale, Se=100%, Sp=35%). Using LOO, 10/28 radiomic models had Y>0.35. The largest Y (Y=0.49, Se=62%, Sp=87%) were obtained with 4 features on average, reflecting the volume, sphericity and heterogeneity (GLCM_Correlation, GLCM_Contrast) of the lesion uptake. Using the jackknife procedure, Y was 0.47±0.06 (range: [0.32;0.56]), significantly higher (Wilcoxon p < 0.05) than for the sham task (Y=0.06±0.18, range: [-0.52;0.47]). Conclusion: The proposed systematic screening procedure enabled the identification of a parsimonious radiomic signature from scarce data. References: [1] Nioche et al. Cancer Res 2018.