IEC International ISO Standard IS0/IEC 19795-10 First edition Information technology - 2024-10 Biometric performance testing and reporting - Part 10: Quantifying biometric system performance variation across demographic groups Technologies de I'information - Essais et rapports de performance biométriques - Partie 10: Quantification de la variation des performances du systeme biométrique selon les groupes demographiques Reference number ISO/IEC19795-10:2024(en) @ISO/IEC 2024 IS0/IEC 19795-10:2024(en) COPYRIGHT PROTECTED DOCUMENT @IS0/IEC2024 All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication may be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying, or posting on the internet or an intranet, without prior written permission. Permission can be requested from either Iso at the address below orIso'smemberbody inthecountryoftherequester. ISO copyright office CP 401 : Ch. de Blandonnet 8 CH-1214 Vernier, Geneva Phone:+41227490111 Email: [email protected] Website: www.iso.org Published in Switzerland IS0/IEC 2024 - All rights reserved ii IS0/IEC 19795-10:2024(en) Contents Page Foreword. V Introduction ..vi Scope. .1 1 2 Normative references 3 Terms and definitions. 2 Conformance 4 5 Planning the evaluation 5.1 Identifying the scope of the evaluation .4 5.2 Demographic variables 5 5.2.1 Ground truth requirements .5 5.2.2 Categorical demographic variables 5 5.2.3 Continuous demographic variables 7 Other demographic variables. 5.2.4 8 Executing the evaluation. 8 6 6.1 Generation of mated comparison and identification trials. 8 6.2 Generation of non-mated comparison and identification trials 8 6.2.1 General 8 6.2.2 Verification (1:1) 8 6.2.3 Identification (1:N) 8 9 6.3 Selection of a threshold. 6.4 Calculating differential performance based on categorical variables for two specific demographic groups. .9 6.4.1 General 9 6.4.2 Differential performance between two groups based on mathematical difference. .9 Differential performance between two groups based on mathematical ratio. 6.4.3 .10 6.5 Calculating differential performance based on categorical variables for more than two .10 groups 6.5.1 .10 General Differential performance for more than two groups based on the largest error 6.5.2 .10 rate relative to the geometric mean. 6.5.3 Differential performance for more than two groups based on the Gini coefficient. .11 6.6 Calculating differential performance in identification trials .11 6.7 Calculating demographic differentials for failure-to-enrol rate, failure-to-acquire rate and transaction duration. 12 Calculating demographic differentials for continuous variables 12 6.8 6.9 13 Comparisonscoredifferentialmeasures 6.10 Calculating uncertainty .14 6.10.1 Uncertainty in demographic differentials. 14 6.10.2 Sampling the target population .14 15 6.10.3 Samplesizereguirements 16 7 Reporting the evaluation results 7.1 Reporting the experimental design .16 7.2 Reporting the target application .16 7.3 Reporting the test population.. .16 7.4 Reporting differential performance .17 7.4.1 Reporting differential performance on previously collected datasets. .17 7.4.2 Reporting differential performance for two or more groups .17 7.4.3 Reporting differential performance against a benchmark. .18 7.4.4 .18 Reportingerrortrade-offmetrics Reporting threshold management policy. .18 7.4.5 7.5 Reporting comparison score differential measures .18 7.6 Reporting exception handling. 19 @ IS0/IEC 2024 - All rights reserved iii