Multimodal Cooking Assistant
Written by Sebastian Feuerstack
October 4th, 2015

StarCook-LogoCooking can well be called one of the most basic and important activities in many households. However, this every day activity has until now only been supported through a few service ideas such as cooking-TV-shows, web recipe data bases and of course countless cookbooks.

The Smart Cooking Assistant combines all of these services adding up to a unique multimodal cooking experience and numerous possibilities for the creation of smart cooking services as well as kitchen appliance support services. Performance and usability are augmented by personalization, multimodality, audio video support context awareness and proactive access to third party information. As a result, the Smart Cooking Assistant facilitates comfortable, practical cooking with a high degree of security and entertainment.

Copyright 2007 Next Generation Media

The 4 Star Cooking Assistant is composed of two independent modules – the Recipe Finder on the one hand, and the Smart Cooking Aid on the other hand. Both of them are hosted on a home server and accessible through any audio visual terminal device.

The Recipe Finder module has access to internal and external recipe databases from where it extracts those recipes that fit the actual needs of the user. For the extraction it uses all context data available, such as user preferences, level of difficulty, ingredient supplies, cooking history, kitchen inventory, number of people to join the meal, diet and nutrition information and any wishes explicitly specified by the user. Recipes matching the information are then presented to the user in the Smart Cooking Assistant GUI in an overview.

In the Cooking Aid module, the recipe is processed and presented to the user step by step, supported by bidirectional multimodality. Any cooking step may be rendered on a GUI or read out by the Smart Cooking Assistant. It may furthermore be supported by movie clips for entertainment or a detailed demonstration of how exactly to proceed. Via the Home Server, the SCA may also monitor and communicate with kitchen devices e.g. to preheat the oven or switch it off at the right moment.


For operating the SCA a multimodal UI is provided to boost comfort. User interaction may be performed through a touch screen terminal device or through the voice interface. A simple command like “done” causes the SCA to proceed with reading out and rendering the next cooking step.

Besides enhancing comfort and security in the kitchen the SCA is a great application to assist children in the kitchen and give cooking lessons. Through context aware appliance control it may avoid any disaster in the kitchen, through audio video support it may provide enough entertainment or background information to make cooking a fun experience. The 4 Star Cooking Assistant pursues the development of cookbook design from nutrition to lifestyle and transfers it to the 21st century.

Last Updated 10:57 pm