EaT-PIM: Substituting Entities in Procedural Instructions Using Flow Graphs and Embeddings

Shirai, Sola
Kim, HyeongSik
Thumbnail Image
Other Contributors
Issue Date
Semantic Web , Ingredient Substitution , Graph Embedding , Flow Graph , Procedural Instructions , Natural Language Processing
Terms of Use
Full Citation
Shirai, S.S., Kim, H. (2022). EaT-PIM: Substituting Entities in Procedural Instructions Using Flow Graphs and Embeddings. In: , et al. The Semantic Web – ISWC 2022. ISWC 2022. Lecture Notes in Computer Science, vol 13489. Springer, Cham.
When cooking, it can sometimes be desirable to substitute ingredients for purposes such as avoiding allergens, replacing a missing ingredient, or exploring new flavors. More generally, the problem of substituting entities used in procedural instructions is challenging as it requires an understanding of how entities and actions in the instructions interact to produce the final result. To support the task of automatically identifying viable substitutions, we introduce a methodology to (1) parse instructions, using NLP tools and domain-specific ontologies, to generate flow graph representations, (2) train a novel embedding model which captures flow and interaction of entities in each step of the instructions, and (3) utilize the embeddings to identify plausible substitutions. Our embedding strategy aggregates nodes and dynamically computes intermediate results within the flow graphs, which requires learning embeddings for fewer nodes than typical graph embedding models. Our rule-based flow graph generation method shows comparable performance to machine learning-based work, while our embedding model outperforms baselines on a link-prediction task for ingredients in recipes.
Springer, Cham