Dataset: Modelling surface color discrimination under different lighting environments using image chromatic statistics and convolutional neural networks

Associated publication

[1] Samuel Ponting*, Takuma Morimoto*, Hannah E. Smithson, “Modelling surface color discrimination under different lighting environments using image chromatic statistics and convolutional neural networks”, *equal contribution, bioRxiv, https://doi.org/10.1101/2022.11.02.514864

[2] Takuma Morimoto, and Hannah E. Smithson, “Discrimination of spectral reflectance under complex environmental illumination,” Journal of the Optical Society of America A, 35, 4, B244-B255 (2018) https://doi.org/10.1364/JOSAA.35.00B244

Datasets contain 2 folders and 1 mat file.

(Folder 1) Stimuli

(Folder 2) Psychophysics_data

(Mat file) stimulusMagnitudeToMacLeodBoynton.mat

Details are described below.

----------------------------------------------------------------------------------------------------------------------------------------

(Folder 1) Stimuli

Overview of datasets

This Image dataset includes 57,600 images (2 gloss levels * 3 environments * 100 stimulus magnitudes * 8 hue directions * 12 camera angles from 0 to 330 degree in 30 degree step) in .mat format.

The half of images were used in psychophysical experiment (camera angles: 0, 60, 120, 180, 240, 300 degrees).

Other half images were used for testing chromatic statistics models and CNN-based models [1] (camera angles: 30, 90, 150, 210, 270, 330 degrees).

Each image file

Filename denotes a condition name and the camera angle as formatted in a following way.

stim_”environment” _”glossiness”_”hueAngle”_”magnitude”_”cameraAngle”.mat

e.g. “stim_en1_glossy_hue45_n45_cameraAngle90.mat”

Stimulus magnitude 100 is a maximum saturation, and 1 corresponds to equal energy white (which was used as a distractor object).

Each image file contains two valuables : MacLeodBoynton, XYZ

Each variable contains an image of 128*128*3 pixels (height*width*channel).

MacLeod-Boynton: MacLeod-Boynton chromaticity image (1st channel: L/(L+M), 2nd channel: S/(L+M), and 3rd channel L+M)

XYZ: XYZ coordinates calculated based on 2-degree CIE 1931 xyz color matching function (1st channel: X, 2nd channel: Y, and 3rd channel Z)

Luminance and L+M are both relative (normalised by the maximum luminance across all 57,600 images).

----------------------------------------------------------------------------------------------------------------------------------------

(Folder 2) Psychophysics_data

Filename denotes the condition and observers formatted in a following way.

data_”environment” _”specularities”_”sessionNumber”_”obsever”.mat

e.g. data_en2_matte_session4_JH.mat or .csv

Each file includes following variables:

(Variable 1) threshold

Thresholds are stored in MacLeod-Boynton (MB) chromaticity coordinates for all 8 hue directions (from 0 to 315 degree in 45 degree step).

MacLeod-Boynton chromaticity coordinates were calculated in a following way.   

These scalings are in accordance with description in CVRL main site (Chromaticity coordinates tab ).

First of all, L, M, and S cone signals were calculated based on Stockman & Sharpe cone fundamentals (energy in linear scale available at at http://www.cvrl.org).

Each sensitivity curve was normalised to have 1.0 at the peak.

Then, MB coordinates were calculated using equation (1-3).

L/(L+M) = Lw*L/(Lw*L+Mw*M) - (1)

S/(L+M) = Sw*S/(Lw*L+Mw*M) - (2)

L+M = Lw*L+Mw*M - (3)

where Lw = 0.689903; Mw = 0.348322;Sw = 1.93540.

L, M and S denote L-cone, M-cone, S-cone excitations, respectively.

Under this calculation, equal energy white becomes L/(L+M) = 0.7078 and S/(L+M) = 1.

(Variable 2) staircase

Since we ran 8 interleaved staircase (for 8 hue angles), information about 8 staircases are stored in this single variable.

(staircase(1) corresponds to 0 degree, and staircase(8) corresponds to 315 degree)

There are 5 fields:

(i) groundtruth,    (ii) response,    (iii) correct, (iv) magnitude, (v) cameraAngle

For each trial, the location of objects was defined in a following way.

| 1 3 |

| 2 4 |

And each field stores following information for all trials in the staircase.

(i) groundtruth

Location of the target object

(ii) response

Location that the participant chose

(iii) correct

If the response was correct (1) or incorrect    (0)

(iv) magnitude

Stimulus magnitude of target object in each trial from 1 to 100 (1 for equal energy white and 100 for maximum saturation).

(v) Camera angle

Camera angles assigned for four objects in each trial.

This data and (i) groundtruth allow reconstruct of the exact image for each trial.

----------------------------------------------------------------------------------------------------------------------------------------

(Mat file) stimulusMagnitudeToMacLeodBoynton.mat

This file stores a variable ‘stimulusMagnitudeToMacLeodBoynton’ (8*100*2) which describes correspondence map between stimulus magnitude and MacLeod-Boynton chromaticity.

1st channel: hue direction from 0 degree to 315 degree, 45 degree step

2nd channel: magnitude from 1 to 100

3rd channel: MacLeod-Boynton coordinate, 1 being L/(L+M) and 2 being S/(L+M)