Knowledge graph enhanced large language models for food computing
Loading...
Authors
Mohbat, Fnu
Issue Date
2025-08
Type
Electronic thesis
Thesis
Thesis
Language
en_US
Keywords
Computer science
Alternative Title
Abstract
Recent advances in large language models (LLMs) and the increasing availability of food-related data have led to significant progress in applying LLMs to food understanding. These developments have enabled Natural Language Processing (NLP) methods to address various food computing tasks, including food recognition, personalized recipe recommendation, and the generation of cooking guidelines. Despite the impressive performance and multi-modal adaptability of LLMs, domain-specific training remains essential for effective application. LLMs are still prone to hallucinations, outdated responses, and struggle in logical and numerical reasoning, limiting their utility in specialized domains. This thesis investigates supervised fine-tuning and instruction tuning of LLMs for text generation, and their use as a text processing engine for recommendation systems. Specifically, we explore their fine-tuning for recipe generation, nutritional estimation, and recipe data processing for recommendation tasks. We evaluate state-of-the-art small LLMs in the context of recipe generation and introduce LLaVA-Chef, a novel model trained using a multi-stage approach on a diverse dataset of recipe prompts. LLaVA-Chef significantly outperforms previous models, generating more detailed and accurate recipes with precise ingredient mentions — often surpassing the quality of human-authored recipes. Although prior research has highlighted hallucination issues in LLMs and explored incorporating contextual knowledge to improve factual accuracy, integration of food-specific knowledge graphs (KGs) with LLMs remains underexplored. To address this, we propose KERL, a unified system that leverages food KGs and LLMs to provide personalized food recommendations and generates recipes with associated micro-nutritional information. Given a natural language question, KERL extracts entities, retrieves subgraphs from the KG, which are then fed into the LLM as context to select the recipes that satisfy the constraints. Next, our system generates the cooking steps and nutritional information for each recipe. To evaluate our approach, we also develop a benchmark dataset by curating recipe related questions, combined with constraints and personal preferences. Through extensive experiments, we show that our proposed KG-augmented LLM significantly outperforms existing approaches, offering a complete and coherent solution for food recommendation, recipe generation, and nutritional analysis.
Description
August2025
School of Science
School of Science
Full Citation
Publisher
Rensselaer Polytechnic Institute, Troy, NY