類神經網路模型應用於食品熱量與營養成份分析

Abstract

現代人追求生活品質，注重視身體的健康。然而，根據世界衛生組織2018年公布之2016年全球十大死因死亡人數統計中，慢性病死因佔前十大死因的一半。透過良好的飲食習慣能預防慢性疾病與肥胖。瞭解飲食習慣的典型方法，是紀錄三餐並且分析卡路里與營養成份。因此本研究提出一套利用食物影像分析與估算熱量與營養成份的系統，讓使用者快速的瞭解每餐所攝取的熱量與營養，進而達到均衡營養的目的。系統啟動後會讀入食物影像，將食物影像調整成特定比例後輸入Mask R-CNN。Mask R-CNN首先利用ResNet101-FPN架構擷取低階至高階的食物特徵，再將各階食物特徵皆輸入RPN(Region proposal network)架構獲得影像中食物區塊。使用RoAlign技術固定食物區塊的尺寸後輸入Mask R-CNN head偵測食物種類、食物預測框與食物遮罩。接著系統會利用食物遮罩得到食物在影像中所佔之面積，將其在影像中所佔的像素數量輸入線性迴歸方程式得到食物重量估測。得到食物重量之後，結合衛生福利部與美國農業部之食品營養資料庫，標示出所估測之食物熱量與營養成份。本研究所辨識的食物類別共有16個，分別為沙拉、水果、吐司、蛋、香腸、雞肉、培根、法式吐司、歐姆蛋、薯餅、鬆餅、火腿、漢堡排、三明治、薯條以及漢堡。結合Ville Cafe Dataset與Food-256 Dataset，共有36013張影像、58013個食物。其中使用1278張影像、6096個食物作為訓練集，686張影像、3680個食物當作測試集。Ville Cafe Dataset與Food-256 Dataset結合之食物辨識正確率為99.86%，IoU為97.17%。食物重量估算實驗類別為沙拉、水果、吐司、香腸、培根、火腿、漢堡排與薯條等非複合型食物估算重量。其中每類食物分別使用40、40、44、40、41、49、40與40筆資料，共320筆資料做線性迴歸運算。實驗結果中，平均絕對誤差為8.22，平均相對誤差為0.13。

Alternate abstract:

For the past few decays, obesity has become a serious problem in modern life. Obesity associates with many chronic diseases, which are the leading causes of death, including diabetes, heart disease, stroke and cancer. The most effective way to prevent obesity is through food control, i.e., knowing the food ingestion including the nutrient and calorie. To assist in understanding the food ingestion of each meal, this thesis develops a food recognition system that can analyze the food composition based on the provided image. This thesis also proposes a new-collected dataset Ville Cafe Dataset for food recognition.

The system is developed based on a Mask R-CNN network with a postprocessing mechanism. Mask R-CNN is composed by a Mask R-CNN backbone, a RoIAlign layer, and a Mask R-CNN head. The Mask R-CNN backbone first applies a ResNet101-FPN structure to extract different levels of features. These features are then fed to RPN to locate food regions, or Region of Interests (RoIs), in image. RoIAlign layer resizes the RoIs using bilinear interpolation method and fed to the Mask R-CNN head. The Mask R-CNN head then classify the food category, detect food bounding boxes, and food masks. After obtaining the regions and the categories of each kind of food, the system estimates weight of food using a linear regression model. This thesis also proposes a postprocessing mechanism, which modifies the extracted bounding boxes and masks, to provide a better result on both analytics and visualization.

To estimate the calories and nutrients accurately, the system considers dataset provided the Ministry of Health and Welfare and the United States Department of Agriculture (USDA). According to these informations, the system then shows the estimated calories and nutrients based on the computed food weight and the analysis results.

To estimate the effort, this experiment applied two datasets in the experiments, the Food-256 dataset and the Ville Cafe Dataset. The Ville Cafe Dataset contains 16 categories with 35842 images for each category. This experiment first train our model on the training set, which is the mixture of Food 256 and Ville Cafe, to recognize 16 categories of food, including salad, fruit, toast, egg, sausage, chicken cutlet, bacon, french toast, omelette, hash browns, pancake, ham, hamburger, sandwich and french fries. The training set contains 1278 of food images, 6096 of food items. As for testing, there are 686 food images and 3680 food items being used for evaluation. The food recognition accuracy of the mixture of Ville Cafe Dataset and Food-256 Dataset is 99.86%, and the IoU is 97.17%. As for the food weight estimation experiment includes eight categories: salad, fruit, toast, sausage, bacon, ham, hamburger and french fries. Each of the categories uses 40, 40, 44, 40, 41, 49, 40 and 40 data respectively, a total of 320 data, for linear regression model. In the experimental results, the average absolute error is 8.22, and the average relative error is 0.13.