Housed strawberries and other features of machine translation: to take another step towards automated kitchens, you need to teach the technique to understand recipes written in human language and in human logic. To do this, they need to be transformed. Under the cut text is a reflection on the problem that was born over a cup of coffee during the discussion of a related topic.
 
 

 

 

Automated kitchens. The required steps are

 
Smart kitchen appliances facilitate the cooking process, but the person still does most of the work: washes, cuts and puts products into them, mixes, etc. And as I would like to simply say: "A pot, varie," and that it all washed itself, cut, mixed, and the user only had to wait for the sound signal and go to the trapeznichat.
 
Now, automatic kitchens only appear on an industrial scale, but it will not reach the consumer level for a very long time. To do this, you need to solve several groups of tasks, and these solutions should be available.
 
 
The first group of tasks is to allow the kitchen to act autonomously from the user.
 
This task, for example, can be solved by introducing into the kitchen system of manipulators , the conveyor ;
 
 
The second group of tasks is to introduce computer vision and a system of sensors into the kitchen to automatically determine the degree of readiness of a dish or part of it.
 
 
The third group of tasks is the management interface. The task can be implemented through an application on smartphones, as well as voice assistants.
 
 
And, finally, the fourth group is the task of the kitchen with the logic of behavior. How to teach the kitchen to cook? For example, a robot cook is taught by a person - a professional cook. Putting on the sensors, he slowly shows how to. The machine then reproduces its actions, to the smallest detail, down to the trembling of the hands. Of course, when processing extra traffic should be removed, but if the programmer misses something, the robot cook will stop one day to scratch its nonexistent nose.
 
 
In addition, in order to teach the robot a recipe, the movements need to be repeated several times. Even if we teach the robot individual movements, which he can then combine, then there will be a lot of time for such training. Not to mention strict compliance with the prescription and, apparently, calibrated ingredients: to enable the robot to pick out the eyes from the potato - a separate complex engineering problem. And even after cooking, you still have to wash the stove and dishes.
 
 

The recipe is the code

 
If we return to the state of affairs now - i.e. take into account the economic component of the issue, the difficulties of development, etc., it becomes clear that in the foreseeable future we can only teach smart home appliances to read the recipe and apply it. To do this, you need to convert the recipe to a code. But the classic Java or C program is hardly suitable - they can describe the recipe, but if the user wants to make any changes to it, then he will either have to contact tech support, or sign up for programming courses. In addition, the classic code is too formal. For example, if the ingredients say "dried mushrooms," then fresh will not be cooked. In addition, the fact that recipes of national cuisines are represented in different natural languages ​​complicates the task.
 
 
And I would like to give the user the opportunity to find on the web an interesting recipe for him, download it, and robokuhne - correctly interpret it and apply it. The maximum task is the ability of the kitchen itself, based on the culinary preferences of the user, to search and offer options. In a good way, cooking recipes should not only include a list of raw ingredients and ways of cooking, but also the environment in which foods grow. The dish refers to the national cuisine by many criteria, including specific cooking methods, used dishes, tools, etc. Transformation of all these data into a proper digital form or, most likely, its interpretation will be a difficult task.
 
 
So, which languages ​​can be suitable for making recipes?
 
 

RDF

 
The existing technologies of formalized representation of knowledge fall into several groups of structures.
 
 
These include the Semantic Web approach with its technologies. OWL and RDF . Online collections of related datasets using Semantic Web tools are also known as Linked Data. Recently on Habré appeared article , dedicated to this concept, so we will not focus on it. Many digitization projects are built around the idea of ​​using some higher ontology , which can be extended by onto engineers in a specific field of knowledge.
 
 
Consider the use and application of these tools for the digitization of national cuisines. The main idea of ​​RDF is that the Internet departs from the storage of information perceived only by a person, becoming a worldwide network of interacting processes. As the name suggests, RDF is the basis for expressing information about resources. First of all - about web documents and various organizations. Its formalism is based on the idea of ​​static classes and properties. The question arises: how reasonable to treat the recipe as an entity, and not as a complex process with arguments, time calculation, incoming subprocesses, etc. For the recipe, this is clearly not enough.
 
 

Schema.org

 
The second approach can be considered the initiative Schema.org - joint activity of the Internet community to create, maintain and promote schemes for structuring data on the Internet. This initiative aims to provide a standardized vocabulary for common metadata published on web resources. Web resources related to cooking can use the metadata fields of the class "Recipe", which are stored at . Below is an example of the recipe code provided by Google:
 
 
 
The dictionary and format adopted by Schema.org , primarily focused on the presentation of high-level metadata in Web documents. However, a correctly normalized semantic graph requires a much more explicit notion of concepts. Most of the string values ​​in the fields Schema.org These are natural texts that require human cognitive interpretation. Such text can not be directly read by digital systems without special processing tools natural language , which are often subject to errors. As an illustration, a machine translation of Google to one element of a line from the list of ingredients of the above recipe example is suitable:
 
 
'Fresh strawberries, quartered'
 
Translation into Russian gives a completely incorrect semantic meaning:
 
 

    'Fresh strawberries, quartered'    

 
with "quartered" used in the sense of "housing", as in "Our troops were quartered in Boston", rather than "Cut into four parts".
 
 
The lack of clearly defined object roles and the explicit identification of methods makes the task of machine translation a recipe much more difficult.
 
 
For our futuristic task for digital recipes - a scenario in which a robotic machine could follow instructions for preparing a dish, a simple line of text can not be a solution.
 
 
Consider a typical description of the ingredient:
 
 

    '8 Granny Smith apples - peeled, cored and sliced'    

 
 
 
It is obvious that this text string contains a lot of classifying information: a raw ingredient as a class, a specific apple variety, quantity in pieces, a list of methods that should be applied to each part to properly use the ingredient: removing seeds, peel, cutting the object into parts of a certain form.
 
 
Formalism Schema.org is not expressive enough to record the cooking process in a certain program code. To properly digitize recipes for robotic cooking, we need to separate the description of the ingredient from the logic
 
 

The third approach is cognitive modeling of prescription

 
The traditional way of composing a recipe is to start with the components and operations that need to be performed. This order and style descriptions are known as imperative or procedural. A declarative or functional style of describing the logic of a process usually starts from the top of the execution pyramid, the expected useful result that we want to achieve.
 
 
Consider the following simplified representation of the classic Russian recipe for mushroom soup. Arrows denote subprocesses (sometimes alternative) necessary for the parent process.
 
 
 
 
Cognitive interpretation can lead to a dead end even at the moment when we try to define the recipe. The term "recipe" has several contextual meanings. It can be defined in a general sense as a way of obtaining the desired result. When used in the context of cooking, it means a set of instructions for preparing a culinary dish. Thus, this concept can be considered as an object with certain properties, such as the necessary ingredients and time. As an alternative, it can be viewed as a technological process that has some initial data, goes through a series of steps that must be performed, and leads to certain results. The recipe also includes the time it takes to complete the steps, and a description of the necessary utensils.
 
 
Since 200? in order to develop the optimal formalism for expressing the complex semantics of natural language, the project was created. Knowdy , focused on the management of graph data.
 
 

The project Knowdy

 
Knowdy Is an open source software project for a research group of linguists in St. Petersburg, which develops a super-fast graph database that allows you to directly and efficiently work with conceptual graphs, bypassing any intermediate views, such as SQL tables. The database engine is implemented in C and can be used both as a network service and as a stand-alone library for the embedded environment.
 
 
After several years of research and development, a team of research researchers came up with a special data format for Knowdy DB, called GSL (an acronym for General Semantics Language). GSL is optimized for compact storage of conceptual graphs. It is used for storing data, sending messages and exchanging information. This format is not too wordy, like XML, and a bit more compact than JSON. The language takes some peculiarities of Lisp S-expressions, but with a large modification of the semantics, since one should keep in mind that the charts are not lists. The parenthesized description in GSL is of particular importance, allowing users to express not only multi-level groupings, but also CRUD operations within the database storage system.
 
 
In the GSL description, the process is encoded as a function of the first class, which can be named or can be anonymous, supports inheritance from the base function, has arguments and subprocesses that can work in parallel. The processes below describe some logic of the same recipe of Russian mushroom soup.
 
 

    {! proc prepare mushroom soup mix
[_gloss {ru подготовка заправки
грибного супа}]
Unknown macro: {is cooking by boiling}
(arg cut-mushrooms
{run prepare mushroom mix})
(arg cut-potatoes
{run prepare potato mix})
(arg cut-onions
{run prepare onion mix})
{run _put
[_gloss {ru Все ингредиенты выложить
в контейнер и перемешать.}]
{obj _all}
{into_loc container}}}
(proc prepare mushroom mix
[_gloss {ru подготовка
грибной заправки}]
(arg clean-mushrooms
{run clean mushrooms})
{run_cut
.[_gloss {ru Нарезать грибы
мелкими кубиками.}].
{obj clean-mushrooms}
{form slice {size 1.5 {unit cm}}}})  
 
The third approach is used in R4S. It allows us to present a recipe with the required degree of formalism, to describe both the essence and the action. The recipe, written in this way in the database, can be quickly loaded and applied by technology. In our development instruments, where recipes are supported, they are recorded in the GSL format. The plans are to train the neural network for recording recipes of the peoples of the world, which will simplify the exchange of recipes and give the technological reserve for full-fledged automated kitchens.
 
 
We often come across seemingly trivial problems, the embodiment of which is not so simple. What language do you think can be used to record recipes, on the one hand accessible to users from around the world, on the other hand - understood and interpreted by machines?