User:Supremum of tilt/sandbox

In other documents, it is also claimed that at minimum 10 subjects are needed to obtain meaningful averaged ratings.

Test environment
Subjective quality tests can be done in any environment. However, due to possible influence factors from heterogeneous contexts, it is typically advised to perform tests in a neutral environment, such as a dedicated laboratory room. Such a room may be sound-proofed, with walls painted in neutral grey, and using properly calibrated light sources. Several recommendations specify these conditions. Controlled environments have been shown to result in lower variability in the obtained scores. To facilitate the process of evaluating the subjective quality it can be helpful to use some professional tools designed to meet this challenge. For instance, MSU Perceptual Video Quality Tool, developed for evaluating the subjective quality of video and image files, allows organizations to create tests with the use of collected data, offer observers to pass them, and finally process the results.

Crowdsourcing
Crowdsourcing has recently been used for subjective video quality evaluation, and more generally, in the context of Quality of Experience. Here, viewers give ratings using their own computer, at home, rather than taking part in a subjective quality test in laboratory rooms. While this method allows for obtaining more results than in traditional subjective tests at lower costs, the validity and reliability of the gathered responses must be carefully checked. Two examples of crowdsourcing platforms that can be used to evaluate the subjective quality of videos and images are Amazon Mechanical Turk and Subjectify.us. During experiments both websites are capable of displaying data to paid participants pairwise and then processing responses in a specified manner, providing final scores and detailed report.

Application
Most recently published no-reference image and video quality assessment models, which are capable of determining the quality of processing data, relying solely on the distorted input without having the pristine one, need the learning procedure to be conducted to get the quality score by using machine learning for regression. So to train the model and ensure the high correlation between predicted score and human opinion there is obvious need in subjective quality evaluation. Moreover, while the most popular distortion in video quality assessment problem has always been compression, it is becoming more and more relevant to use subjective video quality evaluation to also develop authentic distortions recognition tools.