(9th-January-2021)
• Utilitarianism is a system of normative ethics that says we should choose actions that maximize benefit and minimize suffering. For a mathematical agent, benefit and suffering are defined by a utility function.
• The principal criticism of utilitarian ethics is that they can allow morally bad actions such as lying and stealing when those actions have good consequences. In contrast, rule-based ethical systems focus on the intrinsic morality of actions. However, following a set of rules may lead to ambiguous situations, such as the ambiguities in Asimov's laws of robotics. The way to resolve these ambiguities is to recognize that environments may present agents with situations where all choices of actions involve breaking rules. A utilitarian system can provide a means of resolving such ambiguous situations by defining the utility value of each human history according to the number and severity of rules that the agent breaks by its actions in the history. Thus utility functions can express rule-based ethics.
• As noted previously, the environment model for current AI systems like the Google car is mostly designed by human engineers. Future AI systems that exceed human intelligence will need to learn environment models through their own exploration. Marcus Hutter (2005) had the insight that AI model learning is mathematically similar to Ray Solomonoff's work on sequence prediction (Solomonoff 1964). We assume that the agent's observations of the environment can be generated by computer programs and design the agent to search for those programs. The programs constitute the AI's environment model and generate the probability distribution used to predict observations.
Comments