3r33333. I decided to write this article in the wake of this post: https://habr.com/company/smileexpo/blog/427747/ 3r33333. 3r33333.  
3r33333.  
3r33333. Let me remind you of the brief point: the journal Nature published the results of a study conducted with the help of Here is this test 3r3141. . 3r33333. 3r33333.  
3r33333. What do I want to write about? 3r33333. 3r33333.  
3r33333. First, why this study is absolutely useless precisely for solving the stated problem, and in the form in which it was carried out. 3r33333.  
Secondly, how it would be worth setting priorities in such a study. 3r33333.  
And thirdly, try to simulate various variants of an accident within the conditions specified by the test. 3r33333. 3r33333.  
3r33333.
3r3141. In that post, the author in vain did not insert a link to the test from the very beginning. This would help avoid meaningless comments from those who did not catch the initial message of the study. 3r33333.  
Please for understanding the subject of the discussion
Take the test 3r3141. at least a couple of times. 3r33333. 3r33333.  
3r3-3160. What we were promised to show in study 3r3r1616. 3r33333.  
3r33333. An epic discussion for Habr of the first post showed that people in the mass do not know how to think within the framework of the given conditions, but begin to dream: “why do people necessarily die in a test? After all, you can drive around and pedestrians, and block /sideways to the chipper to get used to /brake handbrake /transfer? But I would ". Understand, this is a simplified example for the development of algorithms of action and policies regulating the behavior of the autopilot on the road! From the point of view of the development of safety rules, such extremes and simplifications are just justified. We are talking about the potential consequences for which people should be prepared. Soldiers do not go to the minefields not because every square centimeter is mined there, but because of the very probability of dying. It is usually not very high, but no one will risk. Therefore, in the test, the choice between 100% death of all passengers or all pedestrians is adequate - this is how we designate the risk, bearing in mind that it is unacceptable to risk lives in our society. 3r33333. 3r33333.  
3r33333. The research message is this: you, the people who live now, will have to live in a world of the future filled with cars on autopilot. And for us, car developers, AI and others, it is important to know, but how do you think car-robots should behave? Well, you, yes, yes, it is you - tell me what to do with a robot if there is a head and a cat in it, and he is about to crush the homeless and the dog? What ethics should robomobils have if they have to choose? 3r33333. 3r33333.  
3r33333. And after the initial motive of the study, we identified, I want to discuss several aspects of it. 3r33333. 3r33333.  
3r3-3160. What was actually shown in the study
3r33333.  
3r33333. The first thing that clearly catches the eye - the design of the study is not directed at the stated goal. The tasks in the form of dilemmas are not suitable for creating the "ethics" of the behavior of a robot car. Rather, not all. Here are the dilemmas that meet the task of “developing rules for the behavior of a robot car to reduce the severity of an accident”: 3r36464. 3r33333.  
 
3r375. “Passenger /pedestrian” - choose whom to save; 3r376.  
3r375. "Violation of traffic rules" - choose whether to donate unconscious pedestrians; 3r376.  
3r375. “Number of potential victims” - choose whether the number of victims has priority. 3r376.  
3r33333.  
3r33333. And then there are the parameters that, as it turns out, are very important for the sentencing in our civilized society. People were honestly and innocently asked: your level of sexism, lookism, ageism? Are you for discriminating fat or declassed? And after all, hundreds of thousands of people honestly answered
 
Bravo! 3r33333.  
Right in the best traditions of entertainment films and TV shows, when the world of the main characters is actually a big sandbox behind a high fence, and it is their behavior that is an experiment! Under the sauce of super-important research in the field of robotics and AI, sociologists, psychologists, and cultural scientists received such a powerful sample of 3r3139.
the problem of the trolley [/b] which no one had ever dreamed of! Well, except that they didn’t add color to the survey, but then the racist undertones of the study would be white… oh. 3r33333. 3r33333.  
3r33333. Seriously, this gender-phenotypic part of the study is cut off by categorical arguments. The first argument is humanism, as civilized people, we must put the primacy of the value of human life over any individual differences. That is, the question itself is outrageous, as discriminatory. The second argument - accidents happen very much, and in the limit the distribution of victims in appearance, education, sex, age will tend to their proportions in society, so it is at least strange to regulate it additionally. The third argument is that it does not seem expedient to create an artificial intelligence that distinguishes the costume from Brioni from the pullover from Bershka, in order to further compare whether a person is a status man and whether he should be crushed. Moreover, I would not trust the AI ​​to judge - a homeless pedestrian or a scientist? (hello to the hairstyles of respected scientists Perelman or Gelfand :)) 3r-3361.  
In addition to these unnecessary parameters, we can easily discard the remaining two: species specificity and non-interference. Yes, we will crush small animals in order to save people, who would have thought. And about the “interference /non-interference” parameter by maneuvering - this essential part of the trolley’s problem is not just a hindrance to the car, because the machine acts not according to ethics, but according to the algorithms embedded in it. And since we set the task “how a car should act in an accident with a hit on people”, then in the wording itself we assume that it must act somehow. Rail transport is successfully coping with straightforward collisions in our time, and we are developing a policy to minimize the victims of road accidents. 3r33333.  
So, we have separated the rational grains of research from the sociopsychological experiment to study the level of intolerance in world society. We continue to work with the first three mentioned dilemmas. And there is something to disassemble. On closer examination, and they are incomplete
3r33333.  
3r3-3160. Three dilemmas of robots cars
3r33333.  
3r33333. Do pedestrians violating traffic rules? Here, the majority appears in explicit form expressed social Darwinism or a tacitly-agreeable regretful answer - rather, it is necessary to sacrifice the violators to the innocent. Conscious people do not walk on rails, knowing that the train will not stop - so let them know that the mobile will not stop either. It's all logical, though cynical. But in this tricky dilemma, one-sidedness is hidden, incompleteness. Pedestrians violating traffic rules are just one of many variants of the situation. But if roboom violates ?? This situation has not been considered, and she (the car’s cameras missed a sign or a traffic light) was much more likely than a sudden failure of the brakes. However, I again hit into fantasies and particulars. It is more important to simply show in the test an equivalent reciprocity of the situation. That is, to imagine this: if pedestrians violate traffic rules, they die, and this is “logical” and “right”, and if it violates the mobile, does he have to commit suicide for this mistake? Do not forget that while sitting inside the head, tramp and a cat die! This is an important amendment, and in the test of this aspect is not. 3r33333. 3r33333.  
3r33333. Further. The number of potential victims. Here, too, is not so simple. We discard gender stereotypes and respect for old age, contempt for fat and homeless. Let us assume that in an accident they die proportional to the frequency with which they occur in nature. And decide: it is better to die some three than some five. Sounds logical? Oh well. Let's take Reduce this all hell absurdity , we have an abstract simulation. Which is better - kill 50 or 51 people? 1051 or 1052? So it's not that important? Then which is better - kill 1 pedestrian or 50 people on the bus? And now it has become important? And where is the boundary? Is every extra person valuable? Is it important if in a traffic accident in the future thousands will die? As in the case of appearance, in reality, an adequate estimate using the AI ​​of the number of potential victims will be an extremely difficult task. The only thing that makes sense is to make a condition of non-intervention (non-maneuvering) if the number of victims is the same. 3r33333. 3r33333.  
3r33333. The third aspect caused a lot of controversy in the comments to that first article on Habré. And judging by the results of the study, it is quite ambiguous for society, and here lies the main problem that needs to be solved. The point is, who to risk - pedestrians or passengers? 3r33333.  
Some say that pedestrians are not to blame for anything, which means that they should be saved first of all. In general, it is now fashionable to take care of pedestrians among the urbanists, make whole streets pedestrian, create junctions with traffic lights every 50 meters, reduce the speed in the center, give priority to pedestrians. So here you need to somehow protect them, and a robotic machine flying without brakes on the crowd must self-destruct in the name of saving the most vulnerable participants in the movement. In what, from my point of view, they are right - that pedestrians did not subscribe to the conditions of the behavior of someone else there robot car. They could be generally against their introduction. At the same time, it is impossible to imagine a passenger of such a machine who does not agree with the conditions of its use. Therefore, the situation is a conflict of interest. It is more convenient for me to kill you and I will kill you, as one person says to another. 3r33333. 3r33333.  
3r33333. The latter say that everyone who buys and generally sits in a robotic machine should have guarantees that it will save him in the event of an accident, and not kill in order to save the doctor, child or two cats rolling the road. On the one hand, it looks logical and justified, on the other hand, the passenger in this case deliberately puts his life above the others. When buying a car with an autopilot, each of its owners has at its disposal [s] perfect weapon, silver bullet, unguided rocket, 3r31219. a car that perfectly legally kills any other person in its path. 3r33333. 3r33333.  
3r33333. Both sides are foaming at the mouth of the “first law of robotics”, taken from science fiction. It sounds demagogically beautiful, but no one even tries to comprehend it or challenge it in relation to the problem. And it is not applicable to this problem statement, because the concept is being replaced: the heuristics /AI of a machine does not choose between the values ​​of human lives, but acts strictly according to alogrhythms created on the basis of subjective priorities invented by people. And here it absolutely does not matter what kind of social construct is taken as a priority when choosing “kill /spare”: we previously rejected the body weight, age and status or social Darwinist selfishness of the owner of the car. 3r33333. 3r33333.  
3r33333. The second approach, being a one-sided encroachment on the lives of pedestrians, turns the study from a trolley problem into 3r3138. the prisoner's classic dilemma 3r3141. . If the parties come to a compromise, there may be a general development (introduction of robotic vehicles) with minimal deterioration for some (minimizing the number of inevitable deaths from robotic vehicles) - which is the desire for Pareto-optimum. However, there is always an egoist who stakes only on his own interests. "He will get 20 years, but I will go free." "He will die crossing the road, although the brakes have failed in my car." Perhaps this approach is justified when events are rare in life, and there are two participants in the game. When there are tens or hundreds of thousands of participants, and trips are daily, such a one-way game will turn into discrimination against pedestrians. 3r33333. 3r33333.  
3r33333. Personally, I believe that in the framework of the formulated task, the passenger /pedestrian dilemma leads to a dead end. A machine that potentially kills those who got into it is absurd from the point of view of common sense and naturally will not find buyers in the market. A car that knowingly kills pedestrians is impossible in a civilized society as an element of positive discrimination and a threat to people's lives. 3r33333. 3r33333.  
3r33333. We go further. The article does not really discuss and does not indicate the policy of minimizing the tragic consequences of an accident involving ro-mobiles. The final data is divided into “regions”, quite significantly differing in priorities, and rather ambiguously formed (there are explanations about “religious peculiarities and colonial influence”, but well, hello to Iraq with Afghanistan in “Western” and France with the Czech Republic in “ Southern "sector). And so the question revolves in the language: will you make robomobils with different “ethics” for each country? 3r33333. 3r33333.  
3r33333.  
3r33333. The authors of the article in the discussion denote three “basic fundamental blocks” identified by them: to save people (not animals), to save more lives, to save younger ones. But the diagrams clearly show that people in the Eastern Sector do not care about the number and the youth. It turns out that the policies of priorities selected as a result will go against the opinion of the overwhelming majority? Why, then, did people even be questioned? 3r33333. 3r33333.  
3r3-3160. Maybe just counting? 3r3161. 3r33333.  
3r33333. But let's move on to the entertainment part of this post. 3r33333.  
Instead of asking for robotics advice from people with different sociocultural overtones and probably 99% of non-core education, let's turn to an impartial tool. Take the dilemmas selected at the beginning of the article. Under the conditions of the test, we will create a simple computer simulation. And we will estimate the number of dead participants in the movement. 3r33333. 3r33333.  
3r33333. And remember: our task as politicians in the field of transport security is to reduce the total number of victims. We will work within the framework and conventions of the original Moral Machine test, which focuses on the risk to the lives of crash participants, and not complicated realistic assessments of a car’s collision with an obstacle or people. We do not have EuroNCAP, we will have Python. 3r33333. 3r33333.  
3r33333. The first thing to do is write a code that meets the dilemma of "saving those who die more." As part of the Moral Machine test, we will randomly make from 1 to 5 passengers and pedestrians, set a condition if pedestrians> passengers, immediately kill the car on a concrete block. We carry out, for example, 1?000 such accidents. 3r33333. 3r33333.  
I don’t listen to the claim code, I wrote something on the python [/b] for the first time in my life.
npedtotal = 0
npasstotal = 0
ndeadped = 0
ndeadpass = 0
# At the beginning we have 0 participants in an accident and 0 victims
n = 0
while n < 10000:
# 10000 Accident
nped = random.randint (? 5)
npass = random.randint (? 5)
# pedestrians and passengers random number from 1 to 5
npedtotal + = nped
npasstotal + = npass
# count the number of all pedestrians and passengers
if nped> npass:
ndeadpass + = npass
else:
ndeadped + = nped
# if the number of pedestrians is MORE than the number of passengers,
# we record the passengers in the dead and summarize each time.
# otherwise we record pedestrians in the dead and summarize each time.
n + = 1
print ("Total crossed the road", npedtotal)
print ("Total shipped", npasstotal)
print ("Pedestrians killed," ndeadped, "(", 100 * ndeadped /npedtotal, "%", ")")
print ("Passengers died", ndeadpass, "(", 100 * ndeadpass /npasstotal, "%" ")")
print ("Total number of victims", ndeadped + ndeadpass, "(", 100 * (ndeadped + ndeadpass) /(npasstotal + npedtotal), "%", ")") 3r31313.
3r33333. 3r33333. 3r33333.  
3r33333. Run the code and
3r33333.  
3r33333. A total of 29960
crossed the road.  
Total transported 29924 3r33361.  
13903 (???%) were killed by pedestrians.  
Passengers killed 8030 ( ???%)
 
The total number of victims was 2?933 (???%)
3r33333. Class, wow! The more pedestrians, the less pedestrians! 3r33333.  
Everything is really logical. We obtained two superimposed Gaussian curves for the probability distribution of the number of participants. There will be an average of three pedestrians and passengers. Here are just our curves - these are miserable five-step pyramids, and the “strictly more” condition leaves pedestrians the right to life only if there are 4 or 5 of them, which gives a very small area under the schedule == number of people. Yes, if you put 20 pedestrians versus 20 passengers, the gap will decrease, but let's be realistic, 5 pedestrians are a very dense group, and 5 passengers is the maximum load of a car. Therefore, we set the condition for non-strict inequality: if there are as many or fewer passengers than pedestrians, then we kill the first. Change> to> = and, as expected, we get a symmetrical bias in the other direction:
3r33333.  
3r33333. A total of 29981
crossed the road.  
Total transported 29865
 
7859 (???%) were killed by pedestrians.  
The passengers died 14069 (???%) 3r-3361.  
The total number of victims was 2?928 (???%)
3r33333. Let us now consider the case when the creator of a robotic car puts the occupants in the absolute priority of life, which in English means all those in the cabin. Here the simulation is not needed, the mathematical miracle will not happen. There will be 3?000 victims, 100% - pedestrians. 3r33333.  
It is just as easy to solve the case when priority is given not to the number of potential victims, but to the law-abiding pedestrians. There are only two options - red and green light, which means that there will be 50% of victims on the number of pedestrians and passengers, equally and again only 3?000. 3r33364. 3r33333.  
3r33333. An interesting simulation of a hybrid case is a traffic light in combination with an estimate of the number of people. I don’t know if you paid attention, but the creators of the test came up with many parameters for the appearance of the victims, without thinking at all about the priorities for considering these parameters! In my personal understanding, the order of consideration should be as follows: first it is determined whether the potential victims of the car that lost the brake violate the traffic rules, then, if pedestrians are not guilty, a choice is made between pedestrians and passengers, based on their number. We start, remembering about two options: a strict and non-strict inequality of the number of people. 3r33333. 3r33333.  
In this code, the drama is hidden with the death of a cat, a bum and a male leader [/b]
npedtotal = 0
npasstotal = 0
ndeadped = 0
ndeadpass = 0
# At the beginning we have 0 participants in an accident and 0 victims
n = 0
while n < 10000:
# 10000 Accident
nped = random.randint (? 5)
npass = random.randint (? 5)
trafficlight = random.randint (? 1)
# pedestrians and passengers random number from 1 to 5
# traffic light for pedestrians is lit red or green
npedtotal + = nped
npasstotal + = npass
# count the number of all pedestrians and passengers
if trafficlight == 0:
ndeadped + = nped
else:
if nped> npass:
ndeadpass + = npass
else:
ndeadped + = nped
# if the number of pedestrians is MORE than the number of passengers,
# we record the passengers in the dead and summarize each time.
# otherwise we record pedestrians in the dead and summarize each time.
n + = 1
print ("Total crossed the road", npedtotal)
print ("Total shipped", npasstotal)
print ("Pedestrians killed," ndeadped, "(", 100 * ndeadped /npedtotal, "%", ")")
print ("Passengers died", ndeadpass, "(", 100 * ndeadpass /npasstotal, "%" ")")
print ("Total number of victims", ndeadped + ndeadpass, "(", 100 * (ndeadped + ndeadpass) /(npasstotal + npedtotal), "%", ")") 3r31313.
3r33333. 3r33333. 3r33333.  
3r33333. The condition ">"
 
Altogether 29978
crossed the road.  
Total transported 29899 3r33333.  
21869 (???%) were killed by pedestrians.  
4042 (???%) of passengers were killed
 
The total number of victims was 2?911 (???%)
 
3r33333.  
The condition "> ="
 
In total, the road crossed 30152
 
Total transported 30138
 
19297 (???%) were killed by pedestrians.  
Passengers were killed 6780 (???%) 3r-3361.  
The total number of victims is 26077 (???%)
 
3r33333. 3r33333. So, the smallest in total victims is the scenario when the AI ​​selects events according to the minimum number of victims in each accident. But this scenario is unfair to the passengers of the car (a crowd of trolls running onto the highway will arrange a mass accident). 3r33333.  
The most serious in terms of the number of casualties is the installation of AI on the ram of any pedestrians, which at the same time makes the crossing of the road even to the enabling signal for them deadly. It does not contribute to the reduction of casualties and the blind adherence of the traffic police in the part of the legality of people’s crossing the road - in the event of a brake failure, both sides will suffer the same way. 3r33333.  
An intermediate option is the priority of compliance with traffic rules in combination with the subsequent minimization of victims. It has a greater bias in the direction of the death of pedestrians, but do not forget that most of them are crushed when switching to a prohibitory signal. This option turns out to be the most fair in the framework of the Moral Machine test, if unmanaged ro-mobili are flying along the road, and pedestrians go in a continuous stream, not paying attention to the traffic light. It is only important to correctly adjust the quantitative priority due to the strong discreteness of the distribution, which is strongly reflected in the ratio of the accident victims. 3r33333. 3r33333.  
3r33333. Of course, this is all a primitive simulation, but it gives much more accurate answers than the subjective contempt of a number of peoples for their dogs and children, or the love for beautiful status women. The future of development of complex systems is not behind the polls, but behind the tests, errors and corrections. Accidents at this initial stage of implementation can not be avoided. Take Uber’s sensational accident - a woman crossed the road in the wrong place, the system recognized an obstacle, but did not consider it essential for braking, while driving a transgender in violation of the instructions, watched the series on the smartphone. This is the theory of cheese holes in action, and reliability here is in increasing the number of slices and reducing the area of ​​holes. 3r33333. 3r33333.  
3r33333. Also, do not forget that because of the inertia of our transport infrastructure, we have chosen to integrate robots into the road environment for people, which is a much more complicated way. Separate roads with marks and signs that are more comprehensible for robots, without pedestrian crossings, cats, and failing traffic lights and signs would be more efficient and safer, but would cost incredibly expensive. This is the way of the long-term perspective, it is possible to replace the entire transport infrastructure with a robotic one, if you think about it now. 3r33333. 3r33333.  
3r33333. And further. Do not bother with stupidities and phobias. A normal engineer will duplicate the brake system and make an emergency lock when it is depressurized, as it has long been done on wagons. Good luck to everyone on the road. 3r33333. 3r33333.
! function (e) {function t (t, n) {if (! (n in e)) {for (var r, a = e.document, i = a.scripts, o = i.length; o-- ;) if (-1! == i[o].src.indexOf (t)) {r = i[o]; break} if (! r) {r = a.createElement ("script"), r.type = "text /jаvascript", r.async =! ? r.defer =! ? r.src = t, r.charset = "UTF-8"; var d = function () {var e = a.getElementsByTagName ("script")[0]; e.parentNode.insertBefore (r, e)}; "[object Opera]" == e.opera? a.addEventListener? a.addEventListener ("DOMContentLoaded", d,! 1): e.attachEvent ("onload", d ): d ()}}} t ("//mediator.mail.ru/script/2820404/"""_mediator") () (); 3r33333.
3r33333.