The Ideal Government is a Hyper-Rational Evangelist

The singular goal of government, implicit until now, is to design and protect a structure where there is no distinction between the rational and the hyper-rational.

Definitions

• Rational Action: A single agent's action that optimizes that agent's utility function under some policy.
• Hyper-Rational Action: A single agent's action that optimizes all agent utility functions, so long as everyone agrees to follow the same strategy.
• Agent: Someone who has the ability to perform actions in the world.
• Utility Function: A way to measure how "good" the state-of-affairs is and represent it as a single value.
• Policy: A decision framework an agent follows when performing actions that could lead to multiple different state-of-affairs down the line.

Policies can be confusing, so for further clarification lets consider an agent with three (3) actions available.

If an agent follows a "best case" policy, they would choose the action that has the best possible world after it, even if it leaves open the possibility of a bad world ahead.

If an agent follows the "best average" policy, they might also choose action 1. After all, a perfect world brings up the average quite a bit. However, the action may also follow a "maximin" policy, where they want to maximize their worst-case scenario. For example, they may choose action 2 here since, worst case, it leads to an okay world.

Given each of these policies, the "rational" decision is obvious - choose whatever action best follows your policy.

Hyper-Rationality

What happens if two different actors have different opinions on what makes the world "good"? In this case, two agents may have conflicting notions of rationality. The canonical example is the Prisoner's Dilemma.

For each player, the rational (maximin) approach is to defect - worst case you still have a payoff of "-5". However, cooperation maximizes the payoff when combining the players - a more globally optimal solution. If no one talks, no one goes to jail.

In the classic Prisoner's Dilemma style, you don't know if your partner is going to cooperate, so you should always defect. After all, your best case is still no jail time. How do we fix this? We introduce a government (the mob). They will kill you if turn into a rat. Now the payoff structure looks like:

And this is exactly the role that an ideal government should fill. For example, fraud is illegal and punished because otherwise defrauding your peers would be far too rational a thing to do.