[LS2N_IPI_DIBR_Images] Subjectives scores for DIBR algorithms using ACR-HR & Pair comparison

Overview

This database contains 96 still images and their associated subjective scores.

Three different multiview plus depth (MVD) sequences are considered in this database. The sequences are Book Arrival (1024x768, 16 cameras with 6.5cm spac-ing), Lovebird1 (1024x768, 12 cameras with 3.5 cm spacing) and Newspaper (1024x768, 9 cameras with 5 cm spacing). Seven Depth Image Based Rendering (DIBR) algorithms processed the three sequences to generate, for each sequence, four new viewpoints.

These seven DIBR algorithms are labeled from A1 to A7. The test was conducted in an ITU conforming test environment. For the subjective assessments, the stimuli were displayed on a TVLogic LVM401W, and according to ITU-T BT.500. See the paper for more details.

From the generated views (96 video sequences in total), 96 still images are extracted. They are referred to as "key frames" in the papers above. However, the extraction is random : only one frame was selected out of each synthesized view viewpoint. The frame number is fixed for a given viewpoint. This database contains only the extracted still images and the video sequences will be available subsequently in a different database.

We provide 2 spreadsheets with the individual scores for Absolute Category Rating (ACR) and Pair Comparison and the Mean Opinion Score (MOS) for each of the 96 images.

We used Absolute Category Rating (ACR) and pair comparison as test methodologies.

Data provided

Images

On the FTP server, we provide the images in the directory "Images". These videos are PNG files.

SRC

Name

Frame number and point of view

01
Book Arrival

Left point of view : frame 54 of the view 8Right point of view : frame 60 of the view 10Target center point of view : view 9

02
Lovebird1

Left point of view : frame 104 of the view 4Right point of view : frame 112 of the view 8Target center point of view : view 7

03
Newspaper

Left point of view : frame 136 of the view 4Right point of view : frame 104 of the view 6Target center point of view : view 5

Subjective scores

The subjective scores of ACR and pair comparison subjective experiment are provided. An Excel file is provided for each of the methodologies.

In the ACR file, an image is associated to a line and an observer to a column. You have access to all individual vote from each observer to each image. The last column is the Mean Opinion Score (MOS).

In the pair comparison file, the first sheet, explain the data organization as follow :

In the "Individual_PC_Scores" sheet, there are the results for each observer and each image. The observers see a pair only one time. The results in the matrix is "1" if the line algorithm is prefered to the column algorithm. "0" if the column algorithm is prefered to the line algorithm.

In the "Sum_Matrix" sheet, the individual scores are sum across observers for each image. Higher the number is, higher the column algorithm is prefered to the line algorithm. You can see the sum of the values of pair comparison between the algorithms i and j and between j and i is always 43. There are 43 observers for this experiment and all observers see all pairs.

In the "TM_Results" sheet, there are the results of Thurstone Mosteller analysis. These results were computed with the implementation of "The law of comparative judgment : Theory and implementation by R. E. Vasquez-Espinosa and Richard W. Conners" in Matlab

Hypothetical Reference Circuits (HRC)

The Processed Video Sequences (PVS) were created using the following Hypothetical Reference Circuits (HRC). You can find details of the algorithms in the associated publication.

From the left point of view of each source content, 2 images are generated using the algorithm of the table. The first one is the center point of view and the second one is the right point of view. From the right point of view, the center point of view is generated and the left one. For each algorithm and for each source content, there are 4 images. So, for each image content, there are 8 versions x 4 images = 32 images.

Algorithm name
Algorithm number
Associated paper

Fehn cropped
A1
A. Telea, “An image inpainting technique based on the fast marching method,”Journal of Graphics, GPU, and Game Tools, vol. 9, no. 1, pp. 23–34, 2004

Fehn interpolated
A2
A. Telea, “An image inpainting technique based on the fast marching method,”Journal of Graphics, GPU, and Game Tools, vol. 9, no. 1, pp. 23–34, 2004

Holes
A7

ICIP TMM
A6
M. Koppel, P. Ndjiki-Nya, D. Doshkov, H. Lakshman, P. Merkle,K. Muller, and T. Wiegand, “Temporally consistent handling of disocclusions with texture synthesis for depth-image-based rendering,” in Proc. of IEEE ICIP, Hong Kong, China, Sep. 2010

ICME
A5
P. Ndjiki-Nya, M. Koppel, D. Doshkov, H. Lakshman, P. Merkle,K. Muller, and T. Wiegand, “Depth image based rendering with advanced texture synthesis,” in Proc. of IEEE ICME, Singapore, Jul. 2010

MPEG_VSRS
A3
Y. Mori, N. Fukushima, T. Yendo, T. Fujii, and M. Tanimoto, “View generation with 3D warping using depth information for FTV,” Elsevier Signal Processing: Image Communication, vol. 24, pp. 65–72, 2009

Mueller
A4
K. Muller, A. Smolic, K. Dix, P. Merkle, P. Kauff, and T. Wiegand, “View synthesis for advanced 3D video systems,”EURASIP Journal on Image and Video Processing, 2008, Article ID 438148

Original

Experiment information

ACR test

PC test

Display
LCD TvLogic LVM401

Luminance
0.1-180cd/m²

Observation distance
4H

Resolution
1024x768

Methodology
ACR
pair comparison

Duration
20 minutes
40 minutes

Number of videos
96

Rejection method
VQEG MM tesplan
no rejection

Number of observers
43 (0 rejected)
43