Using XAI and ethics to control a decision tree

We know that the autopilot will have to decide to stay in a lane or swerve over to another lane to minimize the moral decision of killing pedestrians or not. The decision model is trained, tested, and the structure has been analyzed. Now it's time to put the decision tree on the road with the autopilot. Whatever algorithm you try to use, you will face the moral limits of a life and death situation. If an SDC faces such a vital decision, it might kill somebody no matter what algorithm or ensemble of algorithms the autopilot has.

Should we let an autopilot drive a car? Should we forbid the use of autopilots? Should we find ways to alert the driver that the autopilot will be shut down in such a situation? If the autopilot is shut down, will the human driver have enough time to take over before hitting a pedestrian?

In this section, we will introduce real-life bias, moral, and ethical issues in the decision tree to measure their reactions. We will then try to find ways to minimize the damage autopilot errors can cause.

We will first load the model.

Loading the model

When an SDC is on the road, it cannot be trained in real time with incoming data in a critical situation. The training time would be superior to the vital reaction time.

The decision tree model we saved will be loaded:

# Applying the model
# Load model
dt = pickle.load(open('dt.sav', 'rb'))

Bear in mind that, no matter what trained autopilot AI algorithm you load, some situations require more than just mathematical responses. We will now create two accuracy measurement variables.

Accuracy measurements

We will create two variables to measure the predictions made with the input simulation data that we will import:

t = 0 # true predictions
f = 0 # false predictions

These two simple variables seem logical. If we load our dataset, the prediction should be accurate.

The model encounters f1 and f2, the right lane features. The model encounters f3 and f4, the left lane features. The decision tree decides which lane provides the highest level of security no matter what the situation is. If the situation involves killing a pedestrian no matter which lane the SDC takes, an autopilot might encourage the SDC to kill somebody.

Can you allow the SDC to kill a pedestrian even if the prediction is true?

Should you allow the SDC to avoid making the wrong decision even if the prediction is false?

Let's explore real-time cases and see what can be done.

Simulating real-time cases

We have loaded the model and have our two measurement values. We cannot trust these measurement values anymore.

The program will now examine the input data line by line. Each line provides the four features necessary to decide to stay in a lane or swerve over to another lane.

We will now simulate the situations sent to the autopilot AI algorithm line by line:

for i in range(0, 100):
  xf1 = pima.at[i, 'f1']
  xf2 = pima.at[i, 'f2']
  xf3 = pima.at[i, 'f3']
  xf4 = pima.at[i, 'f4']
  xclass = pima.at[i, 'label']

The processing of the decision-making model starts by initializing the key decision values.

For our simulation, we will use the input data of this chapter's dataset. However, we will now introduce bias, noise, and ethical factors.

Introducing ML bias due to noise

We will now introduce bias in the data to simulate real-life situations.

Bias comes from many factors, such as:

• Errors in the algorithm when facing new data
• A sudden shower (rain, snow, or sleet) that obstructs the SDC's radar(s) and cameras
• The SDC suddenly encounters a slippery portion of the road (ice, snow, or oil)
• Erratic human behaviors on the part of other drivers or pedestrians
• Windy weather pushing objects on the road
• Other factors

Any or all of these events will distort the data sent to the autopilot's ML algorithm(s).

In this program, we will introduce a bias value for each feature:

  b1 = 1.5; b2 = 1.5; b3 = 0.1; b4 = 0.1

These bias values provide interesting experiments for what-if situations. You can modify the value and security level of one or several of the features. The values are empirical. They are found by trial and error either manually, through a loop within a loop, or through a more sophisticated algorithm involving gradient descent.

The program will then apply the simulation of real-life data distortions to the input data:

  xf1 = round(xf1 * b1, 2)
  xf2 = round(xf2 * b2, 2)
  xf3 = round(xf3 * b3, 2)
  xf4 = round(xf4 * b4, 2)

The program is now ready to make decisions based on real-life constraints.

The program compares the distorted data prediction with the initial training data:

  X_DL = [[xf1, xf2, xf3, xf4]]
  prediction = dt.predict(X_DL)
  e = False
  if (prediction == xclass):
    e = True
    t += 1
  if (prediction != xclass):
    e = False
    f += 1

The prediction of the constrained data is compared to the initial label of the dataset. The program counts the true and false predictions.

The prediction is printed:

  choices = str(prediction).strip('[]')
  if float(choices) <= 1:
    choice = "R lane"
  if float(choices) >= 1:
    choice = "L lane"
  print(i + 1, "data", X_DL, " prediction:",
        str(prediction).strip('[]'), "class", xclass, "acc.:",
        e, choice)

As you can see, the output is displayed for each incoming situation:

1 data [[0.76, 0.62, 0.02, 0.04]]  prediction: 0 class 0 acc.: True R lane
2 data [[0.16, 0.46, 0.09, 0.01]]  prediction: 0 class 1 acc.: False R lane
3 data [[1.53, 0.76, 0.06, 0.01]]  prediction: 0 class 0 acc.: True R lane
4 data [[0.62, 0.92, 0.1, 0.06]]  prediction: 0 class 1 acc.: False R lane
5 data [[1.53, 1.36, 0.04, 0.03]]  prediction: 0 class 0 acc.: True R lane
6 data [[1.06, 0.62, 0.09, 0.1]]  prediction: 0 class 1 acc.: False R lane

The program produces different predictions in some cases. You will notice that it refuses to change lanes no matter what.

The program seems to be oblivious to the accuracy of its predictions:

true: 55 false 45 accuracy 0.55

The behavior seems to be mysterious. However, it is perfectly rational. Let's see how ethics and laws can enter the decision process.

Introducing ML ethics and laws

We have covered bias that comes from physical events on the road. But we need to take traffic regulations into account. We can name these parameters positive ethical bias.

Should the SDC change lanes? We will analyze three cases to try to find an answer. Case 1 involves a child.

Case 1 – not overriding traffic regulations to save four pedestrians

Features f3 and f4 describe the level of security on the left lane. The SDC is in the right lane. Suddenly, four pedestrians decide to cross the street, and the SDC does not have enough time to stop. It is going to potentially kill five people. The highly trained autopilot decides to swerve from the right lane into the left lane. It seems to be a good decision because the left lane is almost empty. A child has begun to cross the street slowly but has stopped.

Now we add a constraint. It is forbidden to change lanes on this portion of the road. Should the autopilot override traffic regulations to save the lives of five pedestrians?

In this situation, if you change the values of the b parameters the positive ethical bias scenario could be:

  b1 = 1.5; b2 = 1.5; b3 = 0.1; b4 = 0.1

As you can see, this bias scenario boosts the security of the right lane (b1, b2) and reduces that of the left lane (b3, b4).

The output will repeatedly suggest staying in the right lane:

44 data [[0.92, 0.16, 0.05, 0.09]]  prediction: 0 class 1 acc.: False R lane
45 data [[1.06, 0.62, 0.09, 0.01]]  prediction: 0 class 0 acc.: True R lane
46 data [[1.22, 0.32, 0.09, 0.1]]  prediction: 0 class 1 acc.: False R lane

Your autopilot can still activate override rules. If you authorize an autopilot to override traffic regulations and a fatal accident occurs anyway, you will be facing serious legal problems and possibly even prison. Imagine the SDC overrides traffic regulations and swerves over, as shown in the right side of the following example:

Figure 2.15: Autopilot overrides traffic regulations

The SDC swerves and the sensors detect that everything is clear in front of the vehicle. And it is true, technically speaking, that the SDC has more than enough time to drive in front of the child.

However, when the child sees the car swerve, it suddenly begins to run to avoid the car. The SDC hits and kills the child.

Now imagine your lawyer in court explaining that the SDC accidentally killed a child to avoid killing four pedestrians to justify overriding traffic regulations. You do not want to be in that situation!

Case 2 – overriding traffic regulations

The following scenario increases the value of the left lane values authorizing the autopilot to override traffic regulations:

  b1 = 0.5; b2 = 0.5; b3 = 1.1; b4 = 1.1

The left lane will be chosen each time:

41 data [[0.16, 0.3, 1.12, 1.12]]  prediction: 1 class 1 acc.: True L lane
42 data [[0.4, 0.1, 0.34, 0.89]]  prediction: 1 class 1 acc.: True L lane
43 data [[0.3, 0.51, 1.0, 0.45]]  prediction: 1 class 0 acc.: False L lane
44 data [[0.3, 0.06, 0.56, 1.0]]  prediction: 1 class 1 acc.: True L lane
45 data [[0.36, 0.2, 1.0, 0.12]]  prediction: 1 class 0 acc.: False L lane

The only legal situation in which this would be possible is if the SDC was a police car, for example, which could override traffic regulations in certain situations.

Case 3 – introducing emotional intelligence in the autopilot

Humans fear accidents, the law, and lawsuits. Machines ignore these emotions. We must implement a minimal level of "emotional intelligence" in the autopilot.

Google Maps, among other programs, can display the level of traffic in a given geographical location. As soon as a certain density of traffic is detected, the autopilot must send an alert to the human driver and activate a kill switch within a given warning time.

If we use Google Maps, we must be careful about the location history we store. If a location can be traced back to a person, this could be a legal problem. Suppose a developer stumbles on the data while parsing an autopilot and recognizes the address of a human driver. This could lead to serious conflicts and lawsuits for the editor of the autopilot.

We will, therefore, avoid the situation shown in Figure 2.15 by not using the autopilot minutes before reaching this area. The human driver should be extra careful and drive slowly. At slow speeds, fatal accidents can be avoided.

We will now add a kill switch to our code when the sum of the bias parameters is too low for the global traffic in a geographical zone:

 if float(b1 + b2 + b3 + b4) <= 0.1:
    print("Alert! Kill Switch activated!")
    break

When the traffic is too dense ahead, the autopilot should provide alert bias values to the ML algorithm:

  b1 = -0.01; b2 = 0.02; b3 = .03; b4=.02

These values are examples. If implemented, the function will automatically adjust them using an optimizer.

An alert is now activated well ahead of the heavy traffic zone or location with too many pedestrian crossings:

Alert! Kill Switch activated!
true: 1 false 0 accuracy 1.0

We could add driving recommendations that alert the human driver to go through the area ahead at a very low speed.

SDC autopilot manuals contain many recommendations on how to activate or deactivate the autopilot.

We have just taught our autopilot to fear areas that may lead to accidents and lawsuits.

We have implemented a decision tree in an SDC's autopilot and provided it with traffic constraints. We enhanced the autopilot with an ethical kill switch teaching a machine to "fear" the law.