Evaluation and Comparison of Statistical Models for Estimating Large-Scale MOS from Small Subjective Subsets : WestminsterResearch

Publication dates
Title	Evaluation and Comparison of Statistical Models for Estimating Large-Scale MOS from Small Subjective Subsets
Authors	Javidi, K. and Martini, M.G.
Type	Conference paper
Abstract	The rapid growth of multimedia products and services creates strong competition in terms of content quality and user acceptance. While objective quality assessment methods provide useful technical insights, subjective evaluation remains essential for capturing true human perception. Most subjective quality studies are conducted in controlled laboratory environments with a number of viewers adequate to enable statistically significant results. This presents two main limitations: it is difficult and time expensive to involve a large number of participants in subjective tests; not all subjects have the same accuracy in assessing the quality of the content. To address these limitations, an alternative approach is to collect ratings from a smaller group of subjects and apply statistical models to estimate the mean opinion score (MOS) as provided by a large group of viewers. In this paper, several statistical models for MOS estimation are evaluated using small subsets of a subjective dataset. Specifically, EPMOS, based on ordinal logistic regression, is examined alongside the standardized ITU-T P.910, ITU-T P.913, and ITU-R BT.500 models. The estimated results from each model are compared with the reference MOS computed from the full dataset. The results demonstrate that EPMOS provides estimation errors and correlations with the reference scores that closely match the best performance across models, even if it does not always achieve the absolute optimum. The complete set of results is made publicly available upon publication of this paper.
Keywords	Subjective quality assessment, MOS, estimation population MOS, ITU-T P.910, ITU-T P.913, ITU-R BT.500.
Year	2026
Conference	18th International Conference on Quality of Multimedia Experience (QoMEX)
Publisher	IEEE
Accepted author manuscript	QOMEX_2026.pdf File Access Level Open (open metadata and files)
Published	29 Jul 2026
Digital Object Identifier (DOI)	https://doi.org/10.1109/QoMEX69967.2026.11618375

Related outputs

Preprint: Comparison of point-cloud construction strategies for the lossless compression of event camera data
Martini, M.G. and Javidi, K. 2025. Preprint: Comparison of point-cloud construction strategies for the lossless compression of event camera data. TechRxiv. https://doi.org/10.36227/techrxiv.174535660.05715604/v1

Subjective quality assessment study of KULF-TT53 dataset on a commercial light field display
Javidi, K. and Martini, M.G. 2025. Subjective quality assessment study of KULF-TT53 dataset on a commercial light field display. Visual Communications and Image Processing. Austria 01 - 04 Dec 2025 IEEE . https://doi.org/10.1109/VCIP67698.2025.11396857

Investigation of the separation zone for split-domain light field visualization
Javidi, K., Martini, M.G. and Kara, P.A. 2024. Investigation of the separation zone for split-domain light field visualization. 28th International Conference on Information Technology (IT). Zabljak, Montenegro 21 - 24 Apr 2024 IEEE . https://doi.org/10.1109/IT61232.2024.10475755

A Light-Field Video Dataset of Scenes with Moving Objects Captured with a Plenoptic Video Camera
Kamran Javidi and Maria G. Martini 2024. A Light-Field Video Dataset of Scenes with Moving Objects Captured with a Plenoptic Video Camera. Electronics. 13 (11) 2223. https://doi.org/10.3390/electronics13112223

KULF-TT53: A Display-Specific Turntable-Based Light Field Dataset for Subjective Quality Assessment
Kamran Javidi, Maria G. Martini and Peter A. Kara 2023. KULF-TT53: A Display-Specific Turntable-Based Light Field Dataset for Subjective Quality Assessment. Electronics. 12 (23) 4868. https://doi.org/10.3390/electronics12234868

A reduced-reference perceptual image and video quality metric based on edge preservation
Martini, M.G., Villarini, B. and Fiorucci, F. 2012. A reduced-reference perceptual image and video quality metric based on edge preservation. EURASIP Journal on Advances in Signal Processing. 2012 (66) 66. https://doi.org/10.1186/1687-6180-2012-66

Image quality assessment based on edge preservation
Martini, M.G., Hewage, C. and Villarini, B. 2012. Image quality assessment based on edge preservation. Signal Processing: Image Communication. 27 (8), pp. 875-882. https://doi.org/10.1016/j.image.2012.01.012

Reduced-Reference Image Quality Assessment Based on Edge Preservation
Martini, M.G., Villarini, B. and Fiorucci, F. 2011. Reduced-Reference Image Quality Assessment Based on Edge Preservation. 7th International ICST Mobile Multimedia Communications Conference. Cagliari, Italy 05 Sep 2011 Springer. https://doi.org/10.1007/978-3-642-30419-4_3

Permalink - https://westminsterresearch.westminster.ac.uk/item/x675q/evaluation-and-comparison-of-statistical-models-for-estimating-large-scale-mos-from-small-subjective-subsets

Evaluation and Comparison of Statistical Models for Estimating Large-Scale MOS from Small Subjective Subsets

Related outputs

Share this

Usage statistics

Export as