Before we move beyond zero-sum games, it’s worth taking a deeper look
at the idea of utilities. As I mentioned before, in a game, the scores in
the matrix are given by something called a utility function.
Utility is an idea for how to mathematically describe preferences in terms
of a game or lottery. For a game to be valid (that is, for a game to have a meaningful analysis and solution), there must be a valid utility function that
describes the players’ preferences.
But what do we have to do to make a valid utility function? It’s
simple, but as usual, we’ll make it all formal and explicit.
The easiest way to talk about utility and games is to think of games
as a lottery. A lottery is a game with one player, with a variable payoff. You can think of it as a game where you only have one strategy available, and
you know a probability distribution for the other player’s strategy. So you
can compute a value for the game.
- Universality: Every possible outcome has a utility value
associated with it; this includes both specific final outcomes,
and indirect/meta outcomes (that is, situations where
an outcome from one lottery/game is a ticket for another lottery/game.
That outcome will have as its value the expected utility
for the lottery). - Comparability: utility values are always comparable;
for a particular player with utility function u,
given any two choices A and B, they’ll either prefer one over the
other (u(A) > u(B) or u(B)>u(A)), or they’ll be indifferent (u(A) = A(B)).
There are no possible utility values that can’t be compared. - Consistency: utility comparisons are consistent. Consistency
requires that utility values be reflexive (if u(A)>u(B), then u(B)<u(A), and if u(A)=u(B) then u(B)=u(A)), and transitive (if u(A)<u(B) and u(B)<U(C) then u(A)<u(C)). - Rationality: Players are rational actors. This means that
players will always make choices that maximize their expected payoff. So
given a lottery with expected payoff 5, and a winning of 4, a player will
always choose to play. On the other hand, people won’t play when the
expected payoff is small – there’s no intrinsic attraction to playing
beyond the expected payoff encoded in the utility function. (So, for example,
for a gambler who enjoys gambling, his pleasure at playing the game
is encoded in the utility value of every outcome that involves playing,
win or lose.) Even given a probabilistic situation,
where the two choices are a definite payoff of 4, or a lottery with
an expected payoff of 4, the player will be indifferent.
Let’s take an example to show how we can build a consistent utility
function. Suppose we’ve got the following
sitaution. We’ve got three lotteries – A, B, and C, where:
- In lottery A, the prizes are an apple (with probability 0.3),
a pear (probability 0.2), a ticket for lottery B (probability 0.2)
or nothing. - In lottery B, the prizes are a pear (probability 0.1), a banana
(probability 0.5), or a ticket for lottery C (probability 0.4). - In lottery C, the prizes are an apple (probability 0.4), a pear
(probability 0.3), or nothing (probability 0.3).
The player can choose either a pear or a ticket for lottery A. How can
we decide what they should do?
We’ll start by describing what the player prefers in the concrete prizes. They
surely don’t like nothing as a prize – so we’ll give that utility 0. And they like apples twice as much as pears, and bananas three times as much as pears. So
we’ll assign u(pear)=1, u(apple)=2, u(banana)=3.
So what’s a ticket for lottery A worth?
Since one prize of A is a ticket for B, and one prize for B is a ticket for C,
we need to figure out the utility values of tickets for B and C. We’ll do C first, since it doesn’t depend on the other two. The utility value of a lottery is the weighted average of its outcome utilities. So, u(c)=0.4*2 + 0.3*1 + 0.3*0 =
1.1. So a ticket for lottery C is worth just a little bit more than
a pear – given a choice between a ticket for C and a pear, the player will choose the ticket; given a choice between a ticket for C and an apple, the player
will choose the apple.
Now, we can figure out the utility value of a ticket for lottery B. u(B) =
0.1*1 + 0.5*3 + 0.4*1.1 = 2.04. So a ticket for lottery B is worth a lot more than a ticket for lottery C, and it’s also worth more than either an apple or a pear.
Now, finally, we can compute the value of a ticket for A. u(A) = 0.3*2 + 0.2*1 + 0.2*2.04 + 0.3*0 = 0.6+0.2+0.408=1.208. So according to the utility function, the player will choose a ticket for lottery A.
There’s a couple of properties of utility functions that can seem strange
at first. Given a utility function u, you can define another utility
function such that ∀x:v(x)=u(x)+c (where C is a constant). In every
possible game, the two utility functions are effectively equivalent: they’ll result in exactly the same strategic choices. Depending on how you look at it, this can be either entirely obvious, or it can seem very strange. If you look at
the example above, where we defined initial utility values in terms of the magnitude of preference – that is, “I like an apple twice as much as a pear, so I’ll say that an apple equals 2 pears”, it seems strange. But the real driving
thing in the utility function is the simple difference between utilities – and the simple difference is exactly the same – so the strategies will be the same.
Similarly, utility functions have equivalent outcomes when multiplied by a constant. Again, it’s the difference between things that are important, and if the distance between outcomes is varied consistently – either by adding or multiplying, then the strategic outcomes will be the same.
It’s all very simple and very rational. Applying it is a lot harder – because
people aren’t always rational. People will gamble for the fun of gambling
depending on their mood; an apple may be worth more than a pear today, and
less tomorrow; a person may prefer an apple rather than a pear, and be indifferent to an apple versus a banana, but prefer a pear to a banana. People just
don’t necessarily define the values of their choices in ways that fit
with the utility function requirements of consistency, comparability, and
transitivity.
As we’ll see in later posts, you can model some interesting economic phenomena using game theory and utility functions. But those models are tricky to apply in the real world, because real-world decision making is often too complex to be able to accurately model with a comprehensible, analyzable utility function. This leads to some really intense debates between political perspectives, ranging from
people who find the whole idea of describing human interactions in terms of utility functions to be morally obscene; to people who believe that there’s no moral problem but that human behavior is too complex to describe as utility function; to people who believe that utility functions are the ideal way (both mathematically and ethically) to model behavior. Interestingly, the lines between these viewpoints don’t correspond to the traditional political divide. Most people seem to expect that the “morally obscene” group should correspond to
the political left, and that the “utility functions are perfect” should correspond to the right. But you can find libertarians and free-market uber-conservatives on the morally obscene side, and marxists on the “utility functions are ideal” side. It’s pretty interesting.
ah, utility. econ major lost me in those classes — the teacher refused to make all axioms explicit, and when i asked why should it be anywhere near commonsense that utilities are always comparable & transitive, he told me to leave the classroom immediately. :-)(now i’m a liberal arts major, look what they did to me).great post as usual, Mark. you have a gift for exposing maths simply (intuitively even) but without ever letting go of the rigor. since this is applied maths, however, and can therefore, informally, be quite easily subverted into a mere rhetorical figure, i think it would be nice to point out a couple of drawbacks and quirks of mathematical utilitarism.i think the foremost problem here is that people forget to state their conclusions as “if preferences are universal, comparable and consistent and the agents are rational…”. i remember reading a footnote somewhere (iirc, in Mas-Collel et al., Microeconomics) that referred to a couple of everyday examples on the contrary. another two fallacies that came up a lot in econ classes but which are also more specific to economics were, first, to expose a choice between good x1 and the bundle of “all possible goods you could afford with the same money” while tacitly deriving conclusions from the economical proposition of homogeneous goods. another — this one made my teeth cringe — was to assume long term (or even short term) consistency. consistency is only possible instantly. the moment you took whatever action or made whatever decision, your preferences are no longer supposed to remain as they were.these are the fallacies i can think of right now. there are a bunch more, of course. economics, game theory very much included, are used as a rhetorical device every day because people don’t state their axioms clearly enough (as if they were commonsense) and then don’t keep to them. (and as if the trivialization of classical systems wasn’t bad enough, they manage to bend fuzzy and paraconsistent systems as well).again, excelent post, Mark. but for the sake of keeping the rigor, i’d like to see, before we move on, a post on cases in which mathematical utilitarism doesn’t apply, or on usual economical & game-theoretical fallacies (they’re probably as serious if not more dangerous than those bayesian proofs of god you hate so much). 🙂
Count me among those who find human behavior to be too complicated. More specifically, there are so many terms (and second-order terms, and third-order, and…) to ever make any concrete calculation of utility values. I’d draw the analogy with nonrenormalizable quantum field theories and their completely unsummable series, but QFTs at least have observable effects we can measure. Humans are just ornery!
yup, Mas-Colell, Microeconomic Theory. (do notice i mis-spelt the author’s name and got the title wrong, in my previous comment). footnote in page 7: “if we continue in this fashion, letting the paint colors get progressively lighter with each successive choice experiment, she may express indifference at each step” — colours aren’t always comparable.(sure, colours sound like a most useless counter-example for these axioms of preference, but then again, people tend to apply utilitarian theories to every possible thing without first checking at any level comparability and consistency).
This is nitpicky, but you have described Von Neumann-Morgenstern Utility or Expected Utility(better than lots of textbooks I might add) which makes use of lotteries to create cardinal utilities and there are other conceptions of utility, that maintain that utility is ordinal, that are more common outside of game theory in economics.
I agree with Bruno, however, there are some awful inconsistencies in the use of all theories of utility, and the theories should be treated as little more than fascinating mathematical curiosities. As a means of desciribing and explaining human behaviour they are just garbage.
Count me as someone who lands on the “Utility functions are ideal” side of things.
I don’t think they can describe human behavior, but that’s because humans don’t always act rationally – even ignoring situations like altruism, there’s still spite and simply doing things against our intentions.
A partway argument can be seen in the work of Ken Binmore (UCL).
Basically, when running game theory experiments on undergrads (a plentiful and renewable resource), he found that their behavior didn’t follow game theoretical predictions.
When running the same games (on a new crop of undergrads) with the payoffs significantly higher (maximum about 50 pounds instead of 5), they suddenly became a *lot* more rational.
Other experiments also suggest that game theory has decent predictive power in situations where the payoffs matter, and they have time to learn the game.
Tht said, one core element of the examples given is that they assume the players are risk neutral. So the palyer is equally happy either getting $100 or getting a lottery ticket that is worth $100,000 but only pays off .1% of the time.
This makes the formulation of condition 4 (rationality) slightly incorrect (the final example, between a payoff of 4 or an expected payoff of 4, is wrong). A player will always select a strategy that maximizes her expected utility. However, the expected utility is not necessarily the expected payoff.
In my graduate micro theory course, the professor used the following to show the issues of comparability and transitivity:
Suppose you have a bottle of Sam Adams, and you replace one drop of beer with one drop of water. You probably could not detect the difference, and would thus be indifferent. This would be true as you continued this process. However, at some point, you would have a Bud Light, and clearly you are not indifferent between Sam Adams and Bud Light, even though you were indifferent between each pair.
I love reading this blog but like a few readers commented before, you left a few things out of the description.
So, like 2 guys have pointed out already, you’ve described vonNeumann-Morgenstern utilities, that is, preferences over lotteries. Originally, utility is used in basic rational choice theory to assign values to choices from an arbitrary set.
Where do the utility numbers come from?
Suppose we have some possible set of outcomes X, it can be finite or infinite. We’re looking to be able to represent an individual’s ordering of these outcomes as a real number.
This is ordinal utility. That is, it represents only an ordering of choice, not “strength of” ordering. In this case u(A) = 10 and u(B) = 5 it means A is preferred to B, not A is “twice as good as B”, in fact, replacing u(A) with any number greater than 5 yields the same preferences.
There is a proof by Debreu of the Representation Theorem that says that if you have an infinite set of choices, say X with some kind of metric (to tell you how ‘different’ two choices are) and preferences obey the following conditions:
1) COMPLETENESS For each a, b in X either a is preferred to b or b to a.
2) TRANSITIVITY If a is preferred to b and b is preferred to c then a is preferred to c.
3) CONTINUITY If a is preferred to b then each other choice that is ‘arbitrarily similar’ to a is preferred to each choice ‘arbitrarily similar’ to b
If these 3 conditions are satisfied then there is a function which can assign to each choice a real number such that if a is preferred to b then u(a) >= u(b) and is in agreement with the preference relation on X for each member of X.
It’s a pretty interesting result, and certainly much less controversial (though bordering on tautological) than vNM utility.
I took utility functions in economics. I agree with what you have said. I was wondering what your opinion was on applying calculus to utility functions. Measurements of utility are supposed use ordinal numbers. With ordinal numbers the difference between any two ordinal numbers is meaningless. A derivative is defined as a quotient with both parts having a subtraction sign. I think a derivative of a utility function contradicts the concept of a utility function but it is used nevertheless in economics. I asked my prof (years ago) about this and he never really answered my question. He has to teach the material as part of the course even if the math stinks. What is your opinion Mark?
Reply to Peter:
There’s no reason for calculus not to work on a ‘well behaved’ utility function if indeed the properties I posed about hold on the preference relation on the space. A derivative (total, partial or as a linear operator) represents a local rate of change, which is why we define it as a limit, and thus does not impose any cardinality and leaves utility as a strictly ordinal relation. Indeed, you can see this by composing the original utility function with any monotonic transformation and comparing the locations of the maxima and minima.
Late to the party, but:
Ah, so there are different types of utility functions!
Good, because the posted one strikes me as removed from how we use money – as commenters said or indicated, we would like to avoid risks, and the actual amount of capital matters, as real life players wants to avoid bankruptcy.
That said, and being a layman here, I would deem the utility [sic] of utility functions comes from predicting market (player) behavior, so “experiments also suggest that game theory has decent predictive power in situations where the payoffs matter, and they have time to learn the game” sounds promising.