Neighborhood Mapping dataset.

Children’s dietary habits are influenced by complex factors within their home, school and neighborhood environments. Identifying such influencers and assessing their effects is traditionally based on self-reported data which can be prone to recall bias. We developed a culturally acceptable machine-learning-based data-collection system to objectively capture school-children’s exposure to food (including food items, food advertisements, and food outlets) in two urban Arab centers: Greater Beirut, in Lebanon, and Greater Tunis, in Tunisia. Our machine-learning-based system consists of 1) a wearable camera that captures continuous footage of children’s environment during a typical school day, 2) a machine learning model that automatically identifies images related to food from the collected data and discards any other footage, 3) a second machine learning model that classifies food-related images into images that contain actual food items, images that contain food advertisements, and images that contain food outlets, and 4) a third machine learning model that classifies images that contain food items into two classes, corresponding to whether the food items are being consumed by the child wearing the camera or whether they are consumed by others. This manuscript reports on a user-centered design study to assess the acceptability of using wearable cameras to capture food exposure among school children in Greater Beirut and Greater Tunis. We then describe how we trained our first machine learning model to detect food exposure images using data collected from the Web and utilizing the latest trends in deep learning for computer vision. Next, we describe how we trained our other machine learning models to classify food-related images into their respective categories using a combination of public data and data acquired via crowdsourcing. Finally, we describe how the different components of our system were packed together and deployed in a real-world case study and we report on its performance.

儿童的饮食习惯受到家庭、学校与社区环境中诸多复杂因素的共同影响。传统上，识别此类影响因素并评估其效应的研究多基于自我报告数据，而这类数据极易出现回忆偏倚（recall bias）。我们开发了一套文化适配性良好的基于机器学习的数据采集系统，用于客观捕捉黎巴嫩贝鲁特大区与突尼斯突尼斯大区两座阿拉伯城市城区中，学龄儿童所接触的食物相关场景，涵盖食品、食品广告与食品门店。该基于机器学习的系统包含四大模块：1）可穿戴相机（wearable camera），可在典型上学日持续采集儿童所处环境的影像片段；2）机器学习模型，可自动从采集到的影像数据中识别与食物相关的画面，并过滤掉其余无关影像；3）第二个机器学习模型，可将食物相关影像划分为三类：包含实体食品的影像、包含食品广告的影像，以及包含食品门店的影像；4）第三个机器学习模型，可将包含实体食品的影像进一步分为两类，分别对应相机佩戴儿童正在食用该食品，以及该食品由他人食用的场景。本论文报告了一项以用户为中心的设计研究，旨在评估在贝鲁特大区与突尼斯大区的学龄儿童中使用可穿戴相机采集食物接触场景的可接受性。随后，我们详述了如何利用网络采集的数据，并结合计算机视觉（computer vision）领域最新的深度学习技术，训练首个用于检测食物接触影像的机器学习模型。接着，我们介绍了如何结合公开数据集与通过众包（crowdsourcing）获取的自有数据，训练其余机器学习模型以将食物相关影像归类至对应类别。最后，我们阐述了该系统各组件如何整合封装并部署于真实世界案例研究，并报告了其运行性能。