Replay of digitally-recorded holograms using Grid computing

Holography has found application in the analysis of particulates in many fields, from snowflakes to plankton. Digital recording of holograms (with a CCD or CMOS sensor) has many advantages over traditional methods such as glass plates coated with a silver-halide emulsion, especially for the portability and robustness of field equipment, and the resulting digital holograms can be reconstructed by a computer code such as HoloReco [1] and either viewed on screen or passed onward to other software for automated analysis.The hologram is a recording of a volume, and so typically one reconstructs a series of slices in depth and then extracts further information (such as the presence of any particles) from those images. The numerical reconstruction is computationally expensive, and we have previously suggested using a Grid computing framework to replay the images and store them for further analysis [2].This spreadsheet contains data about Grid submission of hologram-replay tasks, to assess the utility of Grid computing for this application. The sample hologram and replay software were uploaded to a Grid Storage Element (SE) at Brunel University. Jobs were then submitted to run on the Grid, with each pulling down the executable and sample hologram, reconstructing some number of slices, and uploading the results back to the SE at Brunel University.For the initial tests in 2005 (tabs Test01 and Test02), varying numbers of Grid jobs were submitted with each returning only a single slice. From 2007 on, each Grid job returned a pre-determined number of slices such that the number of Grid jobs needed to complete the full task (usually 2200 slices) was reduced as each did more work.The rate at which completed slices were made available (by being uploaded to the Grid storage) was compared with serial replay on single machines (references "F", "D" and "y"; tabs Test02 and 07test02).For more details see [3] and [4]. This data has also been incorporated into [5] and other works. Tabs Test01 and Test02 include the data used for [3].The scripts for Grid submission, and sample holograms, are included with HoloReco [1] and described in more detail in [4].GridResults-2005.xls is an early version of the results spreadsheet, before some of the charts got corrupted, and Grid-HoloReplay.zip contains some of the initial raw data still in text format. Acknowledgements:Henry Nebrensky submitted the jobs to the Grid, monitored their progress and tabulated the results. Initial Grid deployments were tested on the Worldwide-LCG Grid courtesy of the CMS VO.Tests from 2007 onwards were made within the London Tier 2 of the UK GridPP Grid, under the LondonGrid VO.References1. "HoloReco" digital hologram reconstruction code, DOI: 10.17633/rd.brunel.4570327 (2008)2. J.J. Nebrensky, P.R. Hobson and P.C. Fryer: "Grid computing for the numerical reconstruction of digital holograms" Proceedings of SPIE 5775 pp. 285-296, DOI: 10.1117/12.610677 (2005)3. J.J. Nebrensky and P.R. Hobson: "The reconstruction of digital holograms on a computational grid" Proceedings of SPIE 6252 p. 62521I, DOI: 10.1117/12.677160 (2006)4. J.J. Nebrensky and P.R. Hobson: "Replay of digitally-recorded holograms using a computational Grid" Brunel University https://bura.brunel.ac.uk/handle/2438/3443 (2009)5. I.D. Reid, J.J. Nebrensky and P.R. Hobson: "Challenges in using GPUs for the reconstruction of digital hologram images" Journal of Physics: Conference Series, 368 012025, DOI: 10.1088/1742-6596/368/1/012025 (2012)全息术已在诸多领域的微粒分析中得到应用，覆盖从雪花到浮游生物的诸多研究对象。采用电荷耦合器件（Charge-Coupled Device, CCD）或互补金属氧化物半导体（Complementary Metal-Oxide Semiconductor, CMOS）传感器的数字全息图记录方案，相较于传统的卤化银乳剂涂布玻璃板方法具备诸多优势，尤其凸显在野外设备的便携性与鲁棒性层面，且所得数字全息图可通过诸如HoloReco[1]这类计算机代码完成重建，既可直接在屏幕上查看，也可导入其他软件以实现自动化分析。 全息图本质是对体积光场的记录，因此通常需要重建一系列深度切片，再从这些切片图像中提取进一步的分析信息（例如是否存在目标微粒）。数值重建的计算成本高昂，我们此前曾提出利用网格计算框架完成图像重建并存储结果以供后续分析[2]。 本电子表格包含全息图重建任务的网格提交数据，旨在评估网格计算在该应用场景中的实用价值。示例全息图与重建软件已上传至布鲁内尔大学的网格存储元素（Grid Storage Element, SE）。随后将作业提交至网格运行，每个作业都会下载可执行程序与示例全息图，重建指定数量的切片，并将结果上传回布鲁内尔大学的网格存储元素。 2005年的初始测试（对应工作表Test01与Test02）中，共提交了不同数量的网格作业，且每个作业仅重建单个切片。2007年起，每个网格作业可重建预设数量的切片，因此完成全部任务（通常为2200个切片）所需的网格作业数量大幅减少，因单个作业承担了更多计算量。 本研究将完成切片的上传速率（即上传至网格存储的速率）与单台机器上的串行重建速率进行了对比（参考文献“F”、“D”与“y”；对应工作表Test02与07test02）。 更多细节可参见文献[3]与[4]。本数据集亦被纳入文献[5]及其他学术成果中。工作表Test01与Test02包含了文献[3]所使用的核心数据。 网格提交脚本与示例全息图已随HoloReco[1]一同提供，详细说明可参见文献[4]。GridResults-2005.xls是本结果电子表格的早期版本，部分图表在此版本中已出现损坏；Grid-HoloReplay.zip则包含了部分初始的原始文本格式数据。 致谢： 亨利·内布伦斯基（Henry Nebrensky）负责提交网格作业、监控作业进度并整理统计结果。初始网格部署测试依托CMS虚拟组织（Virtual Organization, VO）的全球LCG网格完成。2007年起的相关测试则在英国GridPP网格的伦敦二级节点（London Tier 2）内开展，隶属于LondonGrid虚拟组织。 参考文献 1. 《HoloReco数字全息重建代码》，DOI: 10.17633/rd.brunel.4570327 (2008) 2. J.J. Nebrensky、P.R. Hobson与P.C. Fryer：《利用网格计算实现数字全息图的数值重建》，载于《光学工程进展（Proceedings of SPIE）》第5775卷，第285-296页，DOI: 10.1117/12.610677 (2005) 3. J.J. Nebrensky与P.R. Hobson：《在计算网格上重建数字全息图》，载于《光学工程进展（Proceedings of SPIE）》第6252卷，第62521I页，DOI: 10.1117/12.677160 (2006) 4. J.J. Nebrensky与P.R. Hobson：《利用计算网格重建数字记录全息图》，布鲁内尔大学，https://bura.brunel.ac.uk/handle/2438/3443 (2009) 5. I.D. Reid、J.J. Nebrensky与P.R. Hobson：《利用图形处理器（Graphics Processing Unit, GPU）重建数字全息图像的挑战》，载于《物理学杂志：会议系列》，368卷，012025，DOI: 10.1088/1742-6596/368/1/012025 (2012)