Gloss-Bridge: A Method to Reduce the Visual Perception Gap between Real-World Glossiness and PBR Reflectance Properties in Virtual Reality

Abstract

With the rapid development of virtual reality (VR) technology, digital materials have become more realistic and controllable, offering new opportunities for material perception research. However, material parameters for physically based rendering (PBR), which are widely used in industry, are often derived from design experience and lack interpretability. This study aims to investigate the visual perception difference of material glossiness between real and virtual environments. A compensation method is proposed to bridge the glossiness perception gap between the real-world and PBR. Two psychophysical experiments are conducted. Experiment 1 measures the psychological perception of materials with different glossiness in the real world and VR using a two-alternative forced-choice (2AFC) design. Experiment 2 establishes a connection between multiple parameters in PBR and the psychologically perceived glossiness obtained in Experiment 1 through a material matching adjustment experiment. Finally, a material glossiness mapping method for VR real-time rendering, Gloss-Bridge, is introduced. This method maps the physical quantity (measured by gloss meter) to various PBR parameters. By using physical parameters instead of empirical parameters, it eliminates tedious hyper-parameter adjustments. This achievement provides a design basis for the production of VR materials based on subjective perception and physical parameters.

1. Introduction

“How to digitize material information to render more realistic virtual materials” has been a hot topic in computer graphics. To address this issue, previous researchers have utilized the Bidirectional Reflectance Distribution Function (BRDF) to describe the relationship between incident light and reflected light during the interaction between light rays and material surfaces, thereby characterizing the reflection characteristics of an opaque surface [1]. Hsia et al. [2] designed a gonioreflectometer to measure the data of the material BRDF. Subsequently, a large body of work has been focused on optimizing this process [3,4,5,6]. Nowadays, researchers use multi-spectral polarized reflection meter systems [7] and free-form surface mirrors [8] to accurately measure and characterize material information in reality.

In the VR and gaming industries, real-time rendering is in high demand. However, BRDF data necessitates complicated calculations for rendering and consists of an enormous amount of data. Therefore, in real-time rendering systems, the estimation of BRDF for materials is implemented and saved as numerous textures. BRDF estimation is a crucial aspect of computer vision research, and early studies frequently employed photometric stereo based on the Lambertian reflection model to estimate material BRDF [9]. Yet, as an ideal reflection surface model, the Lambertian reflection model cannot accurately describe natural materials. Thus, non-Lambertian models have continued to be the focus of subsequent research. Tominaga et al. [10] estimated the material reflectance based on the Phong model [11] using RGB pictures under a single-point light source illumination. In recent years, with the advent of convolutional neural networks, numerous studies have presented outstanding deep learning-based material reflectance estimate techniques [12,13,14].

Due to the advancements in Physically-Based Rendering (PBR) technology, today’s video game and film industries are now capable of creating extremely realistic visual effects. In 1979, Whitted [15] published a paper introducing the idea of physics-based rendering, which opened up research in the field. The paper mentioned the use of ray tracing to achieve global illumination, which produced astonishing rendered images at the time. Following Whitted’s research, Cook and Torrance [16,17] proposed the micro-surface reflection model, which signifies a breakthrough in the development of physically-based rendering technology. This model is able to render the surface effects of metal materials more accurately than any previous method. Nowadays, the Cook-Torrance rendering framework has become the industry standard.

Existing PBR frameworks contain a large number of empirical parameters, and the majority of designers believe in the phrase “If it looks right, it is right” [18]. The production of virtual materials in VR often relies on the developer’s experience and aesthetic judgment rather than explicit quantitative criteria. This results in VR photorealistic visuals that are nevertheless distinguishable from actual images viewed with the naked eye. Researchers suggest establishing a mapping model between physical quantities and psychological quantities based on people’s perceptions as a viable approach.

In the study of the perception of material qualities. Sharan et al. [19,20] built the Flickr Material Database (FMD) by downloading 1000 images of different materials (each of the 10 classes of materials contains 100 photos, such as cloth, leaves, wood, etc.), which laid the framework for the investigation of material recognition. The study demonstrates that individuals can recognize materials in immediate visual stimulus and discern between actual and fake materials with remarkable accuracy in 40 ms. Sharan et al. [21] observe in a subsequent crowdsourcing investigation with 2500 individuals that the accurate recognition rate of materials is approximately 85%, supporting the discovery that observers are adept at detecting materials from 2d screens. Fleming et al. [22] use a subset of the same database as a stimulus for their experiment. The scores of nine subjective material qualities (glossiness, hardness, roughness, etc.) are 90% accurate in predicting the perceived categories of materials.

In the aforementioned research, glossiness is often selected as the primary target of the investigation, and some scholars have conducted more comprehensive studies specifically focused on glossiness. Initially, researchers investigated the elements that influence the perception of glossiness from the local picture characteristics of near-field stimuli. Marlow et al. [23] discovered that high light coverage, edge sharpness, and contrast have a direct effect on perceived glossiness intensity. Marlow et al. [24] demonstrated the effect of visual attributes on glossiness by manipulating the perceived glossiness by altering these aspects. According to Bingham et al.’s research on VR visual perception [25], the vergence-accommodation conflict may be a significant element influencing VR perception. Jamiy et al. [26] have emphasized the significance of this component since this physiological conflict might have a significant impact on the perception of a variety of reflection qualities.

In this work, psychophysical experiments are utilized to establish the mapping relationship between glossiness physical quantities and virtual material empirical parameters, producing experimental scenarios in VR. This not only allows designers to alter material properties faster but also provides adequate validity for visual perception research in VR. The article is organized based on the paper’s three primary components: Section 2 introduces a psychophysical experiment that uses the 2AFC design to measure the association of materials with different glossiness levels in both reality and VR; Section 3 introduces a material matching adjustment experiment that measures how much glossiness is psychologically equivalent under different PBR parameter levels. Section 4 develops the Gloss-Bridge mapping model from physical glossiness to PBR parameters based on the results of the two experiments and introduces the Gloss-Bridge mapping model. Section 5 summarizes the content of the research.

2. Experiment 1

This study evaluates the psychometric scale of gloss perception in both real and virtual environments. It establishes a mathematical model for comparing and analyzing the cross-platform perceptual differences of reflective properties.

2.1. Methodology

The paper proposes an experiment that employs a two-alternative forced-choice (2AFC) design to investigate how humans perceive changes in glossiness with physical glossiness. The experiment is designed with two groups: the Virtual group and the Real group. Each participant is required to undergo both sets of experiments, once in a virtual environment and once in a real environment, in order to ensure comprehensive data collection. The data collected from the experiment are analyzed using the maximum likelihood difference scaling method (MLDS) to obtain a functional expression of how humans perceive differences in glossiness. The 2AFC experiment presents two pairs of stimuli (totaling four stimulus samples) to each participant simultaneously. Participants are required to report which pair has a more significant within-pair difference. Based on the data reported by the participants, MLDS is used to estimate the corresponding psychological scale.

2.1.1. Participants

The experiment involves 20 participants who are unaware of its purpose, consisting of 8 males and 12 females. All participants have normal or corrected-to-normal vision and no color vision deficiencies.

2.1.2. Appliances

The author captures the illumination environment using a SONY Alpha7RⅡ full-frame camera with a SIGMA 15–35 mm lens (set to 15 mm), which has a horizontal field of view of 100.4° and a vertical field of view of 77.3°. The computer that runs the experimental program has an AMD Ryzen Thread Ripper 3990X CPU (manufactured at Gigabyte, Taiwan) and an NVIDIA GeForce RTX 3090 GPU (manufactured at Gigabyte, Taiwan), ensuring an average rendering framerate higher than 90 fps. We use an HTC Vive Pro head-mounted display with a resolution of 2880 × 1600 to present stimuli.

2.1.3. Stimuli

For this study, we produced two types of material stimuli for assessment, one set for the Virtual group and the other for the Real group. The materials for the Real group consist of nine rectangular aluminum plates with black paint applied to the front, combined with differing amounts of 9-micron silica matting agent. Prior to spraying, the paint, matting agent, and dispersant are placed in a centrifuge tube and shaken for 30 min at a speed of 4000 rpm to create a range of paint surfaces with varying gloss levels. Once the paint surface is fully dry and solid, we use a 3NHYG60S gloss meter to measure the 60° glossiness of all stimuli.

During the official experiment, it is important for the experimenter to arrange the Real group in accordance with the sample number for consistency. To prevent subjects from viewing the sample numbers, labels should not be attached. Moreover, to prevent the index from giving the subjects incorrect psychological cues about their perception, we used word names to represent the numbers allocated to the Real materials. This approach, therefore, achieves a double-blind design. Table 1 presents the index and corresponding measured glossiness of the stimuli in the Real group.

We employ the Unity Engine (version 2020.3) Legacy Pipeline Standard Shader to render the stimuli of the Virtual group. As shown in Figure 1, the albedo parameter maintains a constant HSV of (0.0, 0.0, and 0.0), corresponding to the pure black of the Real group, while the metallic parameter is fixed at 0.0. The Smoothness parameter varies across 9 levels as experimental factors for the Virtual group, ranging from 0 to 1 in increments of 0.125. The gloss levels of the Virtual group do not match exactly those of the Real group, but they all attempt to span the maximum gloss range under the current experimental grouping environment conditions. We retain the remaining parameters of the shader at the default rendering state and do not input any special values.

Consistent lighting conditions are applied to both the Virtual and Real groups, and lighting information for the Real group experiment is captured and stored in a 360° HDR panoramic image which is used as environmental lighting for the Virtual group. To create the panoramic image, a camera is positioned within the Real group’s experimental environment, and photos are taken in all directions and combined to form a panoramic image. One-third of overlapping views are used for each shooting direction to ensure the quality of the panoramic image synthesis, resulting in a total of 31 different shooting directions. Each direction includes five different exposure settings (−4 EV, −2 EV, +0 EV, +2 EV, and +4 EV) to capture lighting information and produce a total of 155 environmental photos. After shooting, color correction is performed using Adobe Lightroom and the Meino 24-color card. Finally, PTGuiPro software is used to stitch the photos together and create a complete 360° HDR panoramic image. The output produces a 24 K HD result, which can be seen in Figure 1 (with a black point of 0.0 and a white point of 0.2).

2.1.4. Procedural

During the Virtual group experiment, participants wear HTC Vive Pro head-mounted displays and observe four stimuli, each of which contains nine levels of glossiness. Before beginning the experiment, participants’ pupillary distances are measured, and both the hardware and rendering settings of the VR displays are adjusted accordingly. Their task is to report which pair of materials among the four stimuli appear more different in terms of glossiness. The experiment consists of 504 trials ($C_9^4=126$ in each block repeated 4 times), which are presented to the participants in random order. There are also three rest stages during the experiment, in which participants are allowed to freely rotate the materials and move their heads without any restrictions. On average, the experiment takes 50 min for each participant to complete.

For the Real group, the experimenter manually presents the stimuli to the subjects in a pre-determined order. The physical stimuli-switching process is time-consuming, so each subject only finishes 126 non-repeating tasks of 2AFC, which is a method for measuring the sensitivity of an observer to some stimulus. The researchers use MLDS, a data analysis package for R language developed by Knoblauch et al. [27], to analyze the experimental data.

2.2. Results and Discussion

2.2.1. MLDS Analysis

The results, as shown in Figure 2a, are consistent with the intuitive guess that the larger the index number difference, i.e., the greater the gloss difference, the higher the probability of subjects choosing that pair of materials. Consequently, the blocks located further from the diagonal in the figure correspond to a higher probability of choice. By observing a series of material pairs with equal serial number differences (the series of blocks parallel to the diagonal corresponding to material pairs), such as only comparing all material pairs marked with black dots (the index number difference is 1), it is found that the higher overall gloss (8 and 9 have higher overall gloss than 1 and 2), the higher probability of being selected, indicating that subjects are more likely to perceive greater gloss differences even though these material pairs have similar gloss differences on physical axes. Figure 2b shows the data of the Virtual group in a similar way. The results of the Virtual group are generally the same as those of the Real group.

After conducting MLDS analysis, the glossiness psychological scale image for the Real group is shown in Figure 3a. From an overall perspective, the relative perception intensity increases monotonically with actual glossiness; however, the relationship between them does not change linearly, especially in the 20GU-30GU interval, where lower glossiness perception intensity occurred than in slightly higher glossiness situations. This phenomenon can also be observed in Figure 2a, where the combination of (3, 4) exhibits a lower probability compared to other combinations with a difference of 1 in their corresponding index values. This shows that people are not sensitive to glossiness perception in this interval, and it is difficult to reflect perception differences in data by adding some error disturbance. However, in the high glossiness interval, perception intensity rises very quickly, and the same increase in glossiness brings a relatively larger change in perceived glossiness.

Figure 3b depicts the data analysis outcome of the Virtual group. Notably, the perception in the medium gloss range is not particularly sensitive, whereas the perception outcome in the high gloss range is more influenced by the actual gloss. However, the characteristics of the extremely low gloss range (i.e., approximately 0.0–0.3) in the Virtual group are more salient, similar to the changes in the high gloss range, with a steeper incline. The nonlinear relationship of the virtual group is relatively clearer, exhibiting a growth pattern similar to that of a quadratic function, but it can also be divided into three ranges: low, medium, and high, each with its unique characteristic change. Within the low-gloss range (i.e., from 0.0 to approximately 0.3), the perceived glossiness increases at a faster rate, then transition to the medium-gloss stage, and the sensitivity to glossiness alterations visually diminishes. The glossiness within the range of 0.3 to 0.6 is compressed in perception. After 0.6, the perceptual response to glossiness becomes sensitive again, and minor alterations in glossiness can result in a notable perceptual response.

2.2.2. Psychological Scale Fitting

In order to accurately compare the gloss perception differences between real and virtual materials, it is necessary to obtain the mathematical expressions of the gloss psychometric scales in both environments. Therefore, linear regression is executed using the analysis data based on MLDS.

The fitting results of the real group are presented in Figure 4a. In the low-gloss area (between approximately 0 and 30 GU), the slope of the psychometric scale is initially large and gradually tends to ease. Subjects are able to perceive large areas covered by highlights in the mirror reflection direction when observing stimuli in this range. At this stage, subjects rely solely on judging the glossiness based on the brightness of highlights. It is noteworthy that surfaces with higher glossiness are characterized by a higher overall brightness. In the intermediate range from 30 GU to 65 GU, the perceived glossiness changes of the subjects appear to be more linear, increasing at a constant and low slope. This stage can be regarded as a transition between the two nonlinear change areas at both ends. The basis for the subjects to judge the glossiness changes from the overall brightness of the material surface in the mirror reflection direction to the clarity of the reflected image. Due to the mixed cue, it becomes difficult for observers to decide which type of visual information is more credible. Therefore, the weight distribution of each cue becomes relatively balanced in this area. In the high-gloss area (above 65 GU), the change in perceived glossiness begins to proceed in a nonlinear mode again. The material surface in this range of glossiness can already reflect clear images, and subjects pay more attention to judging the level of glossiness based on the clarity of reflected images or the sharpness of highlight edges.

Figure 4b displays the fitting results of the Virtual group. There is a significant difference in the final form of the psychological scale between the virtual group and the real group, with the overall three-stage change observed in the Real group not appearing in the Virtual group. The change in relative perception glossiness of the Virtual group is very consistent from the beginning to the end, which is in contrast to the Real group.

The linear regression equation for the Real group is presented in Equation (1), where x represents the real-world glossiness, and y represents the relative perception glossiness. The fitted model ($p<2.2\times {10}^{-16}$) has very strong explanatory power for the original data ($r^2=0.9619$). The gray band-shaped area in the figure is a confidence interval with a confidence level of 99%. The overall change shows a very obvious three-stage growth, which is consistent with Obein et al.’s [28] similar results.

(1)$\mathrm{y}=\frac{1}{10000}\left(0.0235{\mathrm{x}}^3-2.817{\mathrm{x}}^2+184.027\mathrm{x}-977.183\right),$

The linear regression equation for the Virtual group is presented in Formula (2), where x represents the PBR smoothness value, and y represents the relative perception glossiness. The regression results ($p<2.2\times {10}^{-16}$) also have a high degree of interpretability, with all fitted items showing a very high level of significance ($r^2=0.9619$). Based on Equation (2), it can be observed that as PBR smoothness increases, the impact of the cubic term with equal weight becomes more significant. On the other hand, the linear term only has a significant effect at very low gloss levels, indicating that it approximates linear changes only in the low glossiness area. In the high glossiness area, the influence of higher-order terms in perceived glossiness becomes more significant and enhances the performance of high-gloss materials to some extent.

(2)$y=0.4536x^3+0.5432x+0.006953,$

3. Experiment 2

This study aimed to adapt the perceptual difference model of material reflectance properties for PBR workflows. To achieve this goal, the impact of several parameters commonly found in PBR materials (including albedo, smoothness, metallic and normal distribution) is investigated in virtual reality. Regression analysis is conducted to quantify the effects of these parameters on the perception of reflectance properties. Furthermore, the researchers extended the psychometric scale fitting results of Experiment 1 to the PBR rendering framework.

3.1. Methodology

3.1.1. Participants

The experiment involves 20 participants who are unaware of its purpose, consisting of 8 males and 12 females. All participants have normal or corrected-to-normal vision and no color vision deficiencies.

3.1.2. Appliances

The computer that runs the experimental program has an AMD Ryzen Thread Ripper 3990X CPU (manufactured at Gigabyte, Taiwan) and an NVIDIA GeForce RTX 3090 GPU (manufactured at Gigabyte, Taiwan), ensuring an average rendering framerate higher than 90 fps. We use an HTC Vive Pro head-mounted display with a resolution of 2880 × 1600 to present stimuli.

3.1.3. Stimuli

The stimulus model is presented in Figure 5. To render the stimuli, a standard shader from the Unity engine Legacy pipeline was used. The shader is based on the Cook-Torrance framework and can be widely applied in VR applications. The albedo, smoothness, metallic, and normal distribution parameters of the shader were varied across different levels according to the experimental design.

The albedo parameter is input in the form of HSV, and previous studies have shown that varying hues do not significantly affect the perception of glossiness [29]. Therefore, both hue and saturation parameters that represent chroma purity are not varied. The reflectance level is set at three levels on the brightness axis, with HSV values of (0.0, 0.0, and 0.0), (0.0, 0.0, and 0.5), and (0.0, 0.0, and 1.0). The metallic parameter is set at three levels: 0.0, 0.5, and 1.0. To ensure resolution accuracy for the smoothness parameter, which is a psychophysical measure, five levels are set at 0.1, 0.3, 0.5, 0.7, and 0.9. The normal distribution parameter is set at three levels: 0.0, 0.5, and 1.0.

Surface normals are obtained through high-poly baking using the method proposed by Ho et al. [30]. The surface of material model one is used as the base (with a maximum width of 5.5 m) to construct a virtual material library and extract perceptual dimensions. A total of 10,000 points are randomly generated on the model surface using Perlin noise, with a minimum distance of 0.035 m between each point. A small ball with a diameter of 0.04 m is then generated at each point to create a uniformly uneven surface, as shown in Figure 5. The normal information of the high-poly models is subsequently baked into a 4K normal map according to the DirectX 3D normal storage specification. In the formal experiment, the stimulus models are placed 50 cm in front of the camera and are scaled down so that they occupy no more than a 75° field of view.

3.1.4. Procedural

The entire experiment is conducted in a VR environment. Before beginning the experiment, participants’ pupillary distances are measured, and both the hardware and rendering settings of the VR displays are adjusted accordingly. Participants sit at a desk with a head-mounted VR display and operate a keyboard with their left hand and a mouse with their right hand.

The present study adopts a 4-factor design, comprising of 3 levels of albedo, 3 levels of metallic, 3 levels of normal distribution, and 5 levels of smoothness, respectively. Each trial is duplicated and presented in a completely randomized order. Specifically, in each trial, two material models are presented, with the reference material appearing on the left side, and its material parameters are controlled by the experimental application program. The adjustable material is presented on the right side, with its smoothness parameter manually adjusted by the subject. At the commencement of each trial, the smoothness parameter of the adjustable material is pre-set to 0.5.

In each trial, the subject’s task is to adjust the smoothness parameter of the adjustable material using the mouse wheel until the glossiness of the adjustable material is visually consistent with that of the reference material. The subject then proceeds to the next trial by pressing the space bar. To provide the subject with richer visual information, the W, A, S, and D keys can be used to rotate both the reference and adjustable material models synchronously from any angle during the adjustment process. However, since the relative position of the two stimulus models and the subject is different, it is not possible to form an identical near-field stimulus image even with synchronous rotation. This is to avoid the potential of the subject unconsciously transforming the perceptual task into a “find image difference” task. Each subject completes all experimental tasks in an environment without time pressure, with each subject carrying out a total of 270 trials, and a break period is set after every 90 trials.

3.2. Results and Discussion

3.2.1. Differential Distribution Analysis

To examine the perceived glossiness and actual glossiness differences of the subjects under different combinations of PBR material parameters, statistical analysis is conducted on the differences between the smoothness of the adjustable material and the corresponding reference material smoothness. This is conducted in order to obtain the distribution of perceptual differences.

As depicted in Figure 6, it can be observed that there exists a strong correlation between the perceived difference in smoothness and the reference smoothness. As the reference smoothness increases, the variability of perceived smoothness differences among individuals decreases, while there is relatively greater uncertainty in the perceived differences within the range of low smoothness. Therefore, predictions of differences in smoothness need to be more cautious in the low smoothness range.

Furthermore, when all parameters are set at their mid-range values, the perceived differences in smoothness are relatively small. In the different distribution of each 3 × 3 matrix, the absolute difference of the center data point is small, indicating that the combination of 0.5 albedos and 0.5 metallic produces materials with smaller perceived differences in smoothness.

It can also be observed that as any parameter increases towards extreme values, the corresponding absolute perceived difference also increases, which may be related to the weighting of various material parameters in the process of smoothness perception. The authors suggest that this may be because moving any parameter towards its extreme value increases the visual feature corresponding to that parameter, resulting in a more distinctive stimulus signal that strengthens the perception weighting of the visual system towards the parameter and its derived stimuli. However, when the increase in weighting is imbalanced with the evaluation error of that parameter, parameters with larger weightings will further amplify perceived differences in smoothness. The main directions in which the perceived differences increase are along the principal diagonal pointing towards “low glossiness-high reflectance” and “high glossiness-low reflectance”.

By observing the diagonal direction, it can be found that the combination of “high glossiness-high reflectance” and “low glossiness-low reflectance” can to some extent suppress the amplification effect of perceived differences caused by a single prominent parameter. This indirectly verifies that when multiple parameters move towards their extreme values in the same direction, the relatively small relative changes in the perceived weighting of each parameter do not amplify the differences.

3.2.2. Multiple Regression Analysis

With the aim of developing a relationship model between perceived differences and various material properties, researchers conduct a variance analysis on experimental data to identify key influencing elements. On the basis of the results of the variance analysis, a regression model is established.

Under the confidence condition of p < $0.05$, only albedo (p < $2\times {10}^{-16}$), metallic (p < 0.00471), and reference smoothness (p <$2\times {10}^{-16}$) showed significance, while normal distribution did not show significance (p < 0.06284). It can be seen that the smoothness intensity of the micro-surface of the material does not have a significant impact on perceived glossiness differences, but the values of the remaining parameters of PBR materials will have a significant impact on the perceived glossiness.

After conducting an analysis of variance (ANOVA), a stepwise method is used to adjust the full combination model of albedo, metallic, and reference smoothness, with the Akaike Information Criterion (AIC) [31] set as the criterion for obtaining a suitable regression model. During each step of the process, researchers delete the factor in the full combination model that has the greatest impact on AIC. They recalculate the AIC value of the model until they reduce the AIC value of the regression model to a lower level. The final multiple linear regression model for perceived differences is displayed in Table 2. The intercept term has a small and insignificant effect; therefore, it is subsequently removed from the model.

The dependent variable of the regression model is the perceived smoothness (matching value). Subtracting the reference value from the matching value yields the perceived difference. As shown in Figure 7, the perceived differences in glossiness generally decrease monotonically with the variation of the reference smoothness parameter. When the value of the reference smoothness is low, individuals in the virtual reality (VR) environment often overestimate the glossiness of the presented materials. However, at high reference smoothness values, individuals tend to underestimate the glossiness of the materials in the VR environment.

However, the specific direction of the difference change is influenced by the combined effect of metallic and reflectance and is not entirely parallel to the reference smoothness axis. As observed in Figure 7, there exists only a certain linear relationship between the perceived difference and the reference smoothness, with the albedo being around 0.35. When the albedo and metallic approach their respective extremes, the linear relationship between the perceived difference and the variation of the reference smoothness will be affected to some extent. Furthermore, the degree of this effect is inversely proportional to the distance of the albedo and metallic from their respective extremes.

Moreover, the accuracy of the regression results under different parameters is examined. As shown in Figure 8, the length of the confidence interval corresponding to the center region of the image is generally small. This signifies that, under the same confidence level, the perceived glossiness differences predicted by this model have greater stability. However, the length of the confidence interval increases as each parameter approaches its respective extreme. This suggests that when the material parameters are closer to the extremes, it becomes more challenging to predict the perceived differences in glossiness, and the perceived differences become less stable. This result is consistent with the analysis in Section 3.2.1.

4. Gloss-Bridge: The Glossiness Mapping Method

In Experiment 1, we obtain psychophysical scales for glossiness perception in both real and virtual environments. However, the independent variable in the real environment is the glossiness units (GU) measured by an instrument, whereas, in the virtual environment, the independent variable is the dimensionless smoothness parameter of PBR materials. To compare the differences in glossiness perception between VR materials and real-world materials, it is necessary to convert the smoothness parameter of PBR materials into glossiness units (GU).

ISO 7668:1986 [32] provides a strict definition for specular glossiness. Under specified conditions of the light source and receiver angles, the ratio of the reflected light flux of the sample in the specular reflection direction to that of the standard glass sample in the same direction is defined as the specular glossiness. A black glass standard sample with a refractive index of 1.567 is used, and its glossiness is set as 100 (glossiness units). Therefore, the formula for calculating the glossiness units of a certain test sample is shown in Formula (3). In the formula, $\mathsf{\Phi }$ represents the measured luminous flux (lm), and $\theta$ represents the selected measurement angle (which can be chosen from 20°, 45°, 60°, and 85°). When the unit solid angle is determined, the formula can be equivalent to the ratio of calculated emissivity.

(3)$GU=\frac{{\mathsf{\Phi }}_{test}^{\theta }}{{\mathsf{\Phi }}_{standard}^{\theta }}\times 100$

For the Real Group stimuli in Experiment 1, a measurement is taken using a 60-degree incidence angle chosen through the application of the Fresnel equations. This enables us to calculate the corresponding reflectance of the standard glass sample as 0.100056032172. We then utilize the rendering equation to calculate the reflectance radiance of the standard sample as well as each smoothness material in the virtual reality platform. To simplify the calculation process, we fix the incident radiance at 1. The rendering equation is displayed in Equation (4), where $\mathrm{l}$ represents the unit vector of the incident light direction and is set to $\left(1,\frac{\mathsf{\pi }}{3},0\right)$. Furthermore, the integration domain is set to $\mathsf{\theta }\in \left[\frac{\mathsf{\pi }}{3}-\mathsf{ϵ},\frac{\mathsf{\pi }}{3}+\mathsf{ϵ}\right]\mathsf{\varphi }\in \left[\mathsf{\pi }-\mathsf{ϵ},\mathsf{\pi }+\mathsf{ϵ}\right]$, and to ensure that the contribution of specular reflection is taken into account, a small value of $\mathsf{ϵ}=0.05$ is chosen.

(4)$L_o={{\displaystyle \int }}_{\mathsf{\Omega }}{f}_r\times \mathit{n}\cdot \mathit{l}d\omega ={{\displaystyle \int }}_{\frac{\pi }{3}-ϵ}^{\frac{\pi }{3}+ϵ}{{\displaystyle \int }}_{\pi -ϵ}^{\pi +ϵ}{f}_r\times \mathit{n}\cdot \mathit{l}\times sin\theta \mathrm{d}\theta \mathrm{d}\varphi$

In Experiment 1, the VR group uses the BRDF of the Unity Standard Material shader for rendering which is divided into a diffuse term $f_{df}$ and a specular term $f_{spec}$. An analysis of the source codes reveals the calculation formulas, presented in Equations (5)–(13).

(5)$f_{df}=\left(1+\left(f_{d90}-1\right){\left(1-\mathit{n}\cdot \mathit{l}\right)}^5\right)\times \left(1+\left(f_{d90}-1\right){\left(1-\mathit{n}\cdot \mathit{v}\right)}^5\right)$

(6)$f_{d90}=0.5+2\times \mathit{l}\cdot \mathit{n}\times \mathit{l}\cdot \mathit{h}\times \left(1-smoothness\right)$

(7)$f_{spec}=F\cdot D\cdot V$

(8)$F=f_0+\left(1-f_0\right)\times {\left(1-\mathit{h}\cdot \mathit{l}\right)}^5$

(9)$D=\frac{{\alpha }^2}{\pi \times \left({\left(\left(\mathit{n}\cdot \mathit{h}\times {\alpha }^2-\mathit{n}\cdot \mathit{h}\right)\times \mathit{n}\cdot \mathit{h}+1\right)}^2\right)}$

(10)$V=\frac{0.5}{{\lambda }_v+{\lambda }_l}$

(11)${\lambda }_v=\mathit{n}\cdot \mathit{l}\times \left(\mathit{n}\cdot \mathit{v}\times \left(1-\alpha \right)+\alpha \right)$

(12)${\lambda }_l=\mathit{n}\cdot \mathit{v}\times \left(\mathit{n}\cdot \mathit{l}\times \left(1-\alpha \right)+\alpha \right)$

(13)$\alpha ={\left(1-smoothness\right)}^2$

The calculation of the complete BRDF is a highly complex process, and finding the antiderivative can be a challenging task. To address this issue, we utilize MATLAB as an aid to perform numerical integration. By substituting various smoothness parameters, we can obtain the corresponding outgoing radiance of the virtual material. We set the reflection coefficient of the glass standard and the smoothness to 1, and the albedo map to black. This allows us to calculate the outgoing radiance of the glass standard. After plugging the result back into Equation (3), we are then able to obtain the glossiness after conversion for all smoothness values. The relationship between smoothness and glossiness is illustrated in Figure 9.

Upon clear observation of Figure 9, it becomes apparent that the actual glossiness and smoothness parameters are non-linearly correlated. Smoothness within a wider range (0–0.6) only leads to minor changes in glossiness. However, it is only when the smoothness value reaches approximately 0.6 that a corresponding significant increase in glossiness can be observed. Importantly, the glossiness further increases until it reaches 185 Glossiness (GU) at relatively high smoothness values of about 0.95.

After converting the glossiness psychometric scale of materials in virtual reality environments to glossiness units, it is displayed alongside the psychometric scale of real materials, as shown in Figure 10. When presenting stimuli of materials with low glossiness units to subjects, the material in the VR platform induces stronger glossiness stimulation, indicating that the reflection-related properties of the material are relatively high. As the target glossiness unit increases, the relative perceived intensity deviation also increases, reaching a maximum of 43.83% at 30 GU. However, the deviation in relative perceived intensity decreases with an increase in the ideal glossiness, until a value of 100 GU is reached, at which point the deviation in relative perceived intensity decreases to approximately 0%.

The model for correcting the perceived intensity deviation of glossiness has been ultimately constructed based on the results of two experiments. Firstly, from the findings of Experiment 1, the PBR smoothness parameters can be converted to psychological deviation values on the corresponding glossiness psychometric scale when the glossiness is known. Subsequently, the perceived smoothness deviation value can be calculated based on a combination of PBR parameters.

The calculation method for the main deviation value is depicted in Figure 10. Assuming that a glossiness unit is measured at a data point on the material and the value is x0, the perception intensity obtained by the observer when the actual glossiness unit is x0 is represented by point A(x0, y0). A horizontal line y = y0 is then drawn, and the intersection point B(x1, y0) of the psychometric scale in the VR environment is identified. The glossiness that corresponds to the same perceived intensity stimulation in the VR environment can be obtained. By applying Equation (13), the corresponding PBR smoothness value can be converted. To enable real-time operation, the results can be pre-calculated and swiftly converted through a lookup table (LUT).

Based on the multiple linear regression model in the results of Experiment 2, the perceived differences under the combination of PBR parameters can be further calculated. Once the difference value has been obtained, the increment can be added to the corresponding parameter to achieve the presentation of material glossiness in the VR scene that is as close as possible to naked-eye observation in the real world.

5. Conclusions

This study investigates the differences in visual perception of virtual material reflection properties in a VR environment, using smoothness as a representative attribute. The article includes two experiments and the construction of mapping methods.

In the first experiment, data is collected using a 2AFC-designed experiment to facilitate the calculation of perception differences, and the psychometric scales of glossiness for people in virtual and real environments are obtained. In the second experiment, a material matching adjustment experiment is performed under the Cook-Torrance framework, exploring how each important parameter (albedo, metallic, smoothness, normal distribution) of PBR material in a virtual reality environment affects the material surface glossiness and extending the result of the psychometric scale construction experiment to the PBR rendering. Finally, the rendering equation is solved using numerical integration methods. The PBR material smoothness parameter is converted into a physical glossiness (GU) on the glossiness dimension of real material so that the glossiness perception between the real-world and VR can be compared on the same dimension. An accurate mathematical mapping curve is obtained to describe the difference through calculation, and corresponding design suggestions based on the results are given.

It is worth noting that the experiments in this study were conducted using a VR head-mounted display, and the resulting findings are solely applicable to the optimization of VR-related applications. With the emergence of wearable display devices featuring higher resolutions, wider fields of view, and a broader range of brightness, the outcomes obtained in this study have certain limitations. Nonetheless, since Glossiness (GU) and PBR materials are widely used technologies in related fields, the experimental methods and mapping models employed in this article can be readily transferred and applied to other devices.

The human visual system operates in an extremely complex manner, and the technology used for graphics rendering is rapidly evolving. This study focuses on mapping the visual perception of reflective properties, and while significant progress has been made in this area, there is still a need for further exploration of the depth and breadth of material perception differences in the context of VR. Based on the methodology of this study, there are several promising research directions that are worth exploring in the future, including:

• (1). Developing quantitative methods based on psychology to measure other PBR material attributes (such as subsurface scattering intensity, transparency, etc.);

• (2). Investigating the impact of non-visual cues on material visual properties and their effect on VR immersion;

• (3). Examining the influence of perceptual constancy on material perception in VR;

• (4). Studying the representation of materials and its impact on perception, driven by emerging rendering technologies such as NERF.

Author Contributions

W.X. conceived the experiments, developed the prototype system, performed the Experiment 2, and wrote the paper; T.T. develops the plotting module, performed the Experiment 1 and reviewed the manuscript; C.X. conceived and supervised the experiments, and reviewed the manuscript. All authors have read and agreed to the published version of the manuscript.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

Not applicable.

Acknowledgments

The author would like to thank the contributors to this article and all the participants in the experiment.

Conflicts of Interest

The authors declare no conflict of interest.

Footnotes

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Figures and Tables

Figure 1. The stimuli within the Virtual group exhibit PBR smoothness parameters that are uniformly distributed between 0 and 1.

Figure 2. The original data before MLDS analysis. (a) shows the original data of the Real group. (b) shows the data of the Virtual group. The glossiness intensity (index number) of the stimuli is sorted from low to high. The different color brightness in the figure represents the probability of subjects choosing the stimulus pair composed of horizontal and vertical coordinate intensities. Different marker represents the index number difference between different stimulus pairs.

Figure 3. The results of the psychological scale were obtained by MLDS analysis. (a) The relationship between real-world glossiness and relative perceived glossiness. (b) The relationship between PBR smoothness parameters and relative perceived glossiness. The black dashed line in the figure is a reference, indicating the relative perception intensity that theoretically corresponds to no psychological perception influence.

Figure 4. The results of the polynomial regression analysis. (a) depicts the relationship between gloss units (GU) measured in the physical world and the psychophysical measure of glossiness. (b) depicts the relationship between the PBR smoothness parameter in VR and the psychophysical measure. The black “x” markers signify the original experimental data obtained from distinct participants.

Figure 5. The stimuli presented to the participants in VR during Experiment 2: a natural block model with an irregular shape. This model is generated by using random Perlin noise to perturb the radial displacement of the spherical vertices in Blender. The left side of the screen displayed a material stimulus used for reference, while the right side displayed stimuli that the users could adjust.

Figure 6. The results of differential distribution. The opacity of the circular data points in the figure indicates the standard deviation of the perceived differences under the current parameter combination. A higher opacity indicates a larger degree of discrepancy in the perceived differences at the corresponding point. The radius of each circular data point represents the absolute value of the smoothness difference, with a larger radius indicating a larger absolute value of the difference.

Figure 7. Contour map of glossiness perception difference.

Figure 8. The results of differential distribution.

Figure 9. The mapping curve between PBR smoothness and physical glossiness (GU).

Figure 10. The mapping curve between physical glossiness (GU) and relative perceived glossiness.

Table 1

The parameters of the stimuli in the Real group.

Index	Color	Glossiness Measured at 60°/GU
1	black	5.5
2	black	10.9
3	black	21.4
4	black	23.2
5	black	45.1
6	black	54.5
7	black	62.7
8	black	77.7
9	black	90.6

Table 2

The results of multiple linear regression.

Terms	Coefficients	p Value
albedo	0.078241	$4.01\times {10}^{-12}$
metallic	0.037660	$0.000342$
reference smoothness	0.996924	<$2\times {10}^{-16}$
Albedo × metallic	−0.107515	$2.29\times {10}^{-10}$
albedo × reference smoothness	−0.134260	$5.16\times {10}^{-12}$
metallic × reference smoothness	−0.081839	$9.38\times {10}^{-06}$
albedo × metallic × reference smoothness	0.240802	$4.51\times {10}^{-16}$

References

1. Nicodemus, F.E. Directional Reflectance and Emissivity of an Opaque Surface. Appl. Opt.; 1965; 4, pp. 767-775. [DOI: https://dx.doi.org/10.1364/AO.4.000767]

2. Hsia, J.J.; Richmond, J.C. Bidirectional Reflectometry. Part I.: A High Resolution Laser Bidirectional Reflectometer With Results on Several Optical Coatings. J. Res. Natl. Bur. Stand. Sect. A Phys. Chem.; 1976; 80, pp. 189-205. [DOI: https://dx.doi.org/10.6028/jres.080A.021]

3. Rabal, A.M.; Ferrero, A.; Campos, J.; Fontecha, J.L.; Pons, A.; Rubiño, A.M.; Corróns, A. Automatic go-nio-spectrophotometer for the absolute measurement of the spectral BRDF at in-and out-of-plane and retroreflection geometries. Metrologia; 2012; 49, 213. [DOI: https://dx.doi.org/10.1088/0026-1394/49/3/213]

4. Baribeau, R.; Neil, W.S.; Côté, É. Development of a robot-based gonioreflectometer for spectral BRDF measurement. J. Mod. Opt.; 2009; 56, pp. 1497-1503. [DOI: https://dx.doi.org/10.1080/09500340903045702]

5. Thomas, J.J. Proceedings of the 19th Annual Conference on Computer Graphics and Interactive Techniques; Chicago, IL, USA, 26–31 July 1992; ACM: New York, NY, USA, 1992.

6. Proceedings of the Sixteenth Eurographics Conference on Rendering Techniques; Konstanz, Germany, 29 June–1 July 2005; Bala, K.; Dutré, P. Eurographics Association: Eindhoven, The Netherlands, 2005.

7. Riviere, N.; Ceolato, R.; Hespel, L. Multispectral polarized BRDF: Design of a highly resolved reflectometer and development of a data inversion method. Opt. Appl.; 2012; 42, pp. 7-22.

8. Mukaigawa, Y.; Sumino, K.; Yagi, Y. Multiplexed illumination for measuring BRDF using an ellipsoidal mirror and a projector. Proceedings of the Computer Vision–ACCV 2007: 8th Asian Conference on Computer Vision; Tokyo, Japan, 18–22 November 2007; Proceedings, Part II 8 Springer: Berlin/Heidelberg, Germany, 2007; pp. 246-257.

9. Woodham, R.J. Photometric Method for Determining Surface Orientation from Multiple Images. Opt. Eng.; 1980; 19, 191139. [DOI: https://dx.doi.org/10.1117/12.7972479]

10. Tominaga, S.; Tanaka, N. Estimating reflection parameters from a single color image. IEEE Comput. Graph. Appl.; 2000; 20, pp. 58-66. [DOI: https://dx.doi.org/10.1109/38.865881]

11. Phong, B.T. Illumination for computer generated pictures. Commun. ACM; 1975; 18, pp. 311-317. [DOI: https://dx.doi.org/10.1145/360825.360839]

12. Deschaintre, V.; Aittala, M.; Durand, F.; Drettakis, G.; Bousseau, A. Single-image svbrdf capture with a render-ing-aware deep network. ACM Trans. Graph. (ToG); 2018; 37, pp. 1-15. [DOI: https://dx.doi.org/10.1145/3197517.3201378]

13. Li, Z.; Sunkavalli, K.; Chandraker, M. Materials for masses: SVBRDF acquisition with a single mobile phone image. Proceedings of the European Conference on Computer Vision (ECCV); München, Germany, 8–14 September 2018; pp. 72-87.

14. Zhang, J.; Meuret, Y.; Wang, X.; Smet, K.A.G. Improved and Robust Spectral Reflectance Estimation. Leukos; 2021; 17, pp. 359-379. [DOI: https://dx.doi.org/10.1080/15502724.2020.1798246]

15. Whitted, T. An improved illumination model for shaded display. Proceedings of the 6th Annual Conference on Computer Graphics and Interactive Techniques; Chicago, IL, USA, 8–10 August 1979; 14.

16. Cook, R.L.; Torrance, K.E. A reflectance model for computer graphics. ACM Siggraph Comput. Graph.; 1981; 15, pp. 307-316. [DOI: https://dx.doi.org/10.1145/965161.806819]

17. Cook, R.L.; Torrance, K.E. A Reflectance Model for Computer Graphics. ACM Trans. Graph.; 1982; 1, pp. 7-24. [DOI: https://dx.doi.org/10.1145/357290.357293]

18. Dunn, F.; Parberry, I. 3D Math Primer for Graphics and Game Development; CRC Press: Boca Ration, FL, USA, 2011.

19. Sharan, L.; Rosenholtz, R.; Adelson, E. Material perception: What can you see in a brief glance?. J. Vis.; 2009; 9, 784. [DOI: https://dx.doi.org/10.1167/9.8.784]

20. Sharan, L.; Rosenholtz, R.; Adelson, E.H. Accuracy and speed of material categorization in real-world images. J. Vis.; 2014; 14, 12. [DOI: https://dx.doi.org/10.1167/14.9.12]

21. Sharan, L.; Liu, C.; Rosenholtz, R.; Adelson, E.H. Recognizing Materials Using Perceptually Inspired Features. Int. J. Comput. Vis.; 2013; 103, pp. 348-371. [DOI: https://dx.doi.org/10.1007/s11263-013-0609-0] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/23914070]

22. Fleming, R.W.; Wiebel, C.; Gegenfurtner, K. Perceptual qualities and material classes. J. Vis.; 2013; 13, 9. [DOI: https://dx.doi.org/10.1167/13.8.9]

23. Marlow, P.J.; Kim, J.; Anderson, B.L. The Perception and Misperception of Specular Surface Reflectance. Curr. Biol.; 2012; 22, pp. 1909-1913. [DOI: https://dx.doi.org/10.1016/j.cub.2012.08.009]

24. Marlow, P.J.; Anderson, B.L. Generative constraints on image cues for perceived gloss. J. Vis.; 2013; 13, 2. [DOI: https://dx.doi.org/10.1167/13.14.2]

25. Bingham, G.P.; Bradley, A.; Bailey, M.; Vinner, R. Accommodation, occlusion, and disparity matching are used to guide reaching: A comparison of actual versus virtual environments. J. Exp. Psychol. Hum. Percept. Perform.; 2001; 27, 1314. [DOI: https://dx.doi.org/10.1037/0096-1523.27.6.1314]

26. El Jamiy, F.; Marsh, R. Survey on depth perception in head mounted displays: Distance estimation in virtual reality, augmented reality, and mixed reality. IET Image Process.; 2019; 13, pp. 707-712. [DOI: https://dx.doi.org/10.1049/iet-ipr.2018.5920]

27. Knoblauch, K.; Maloney, L.T. MLDS: Maximum Likelihood Difference Scaling in R. J. Stat. Softw.; 2008; 25, pp. 1-26. [DOI: https://dx.doi.org/10.18637/jss.v025.i02]

28. Obein, G.; Knoblauch, K.; Viéot, F. Difference scaling of gloss: Nonlinearity, binocularity, and constancy. J. Vis.; 2004; 4, pp. 4-20. [DOI: https://dx.doi.org/10.1167/4.9.4]

29. Mohammadalizadeh, A.; Ameri, F.; Moradian, S.; Mirjalili, F. Effect of color on some geometric attributes of visual appearance of non-effect coatings. J. Coatings Technol. Res.; 2020; 17, pp. 949-961. [DOI: https://dx.doi.org/10.1007/s11998-019-00315-1]

30. Ho, Y.-X.; Landy, M.S.; Maloney, L.T. Conjoint Measurement of Gloss and Surface Texture. Psychol. Sci.; 2008; 19, pp. 196-204. [DOI: https://dx.doi.org/10.1111/j.1467-9280.2008.02067.x] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/18271869]

31. Akaike, H. A new look at the statistical model identification. IEEE Trans. Autom. Control; 1974; 19, pp. 716-723. [DOI: https://dx.doi.org/10.1109/TAC.1974.1100705]

32. ISO 7668:1986 Anodizing of Aluminium and Its Alloys. Measurement of Specular Reflectance and Specular Gloss at Angles of 20°, 45°, 60° or 85°; International Organization for Standardization: Geneva, Switzerland, 1986.