Nonrecursive dynamic incentives: A rate of convergence approach

Abstract

In repeated principal‐agent problems and games, more outcomes are implementable when performance signals are privately observed by a principal or mediator with commitment power than when the same signals are publicly observed and form the basis of a recursive equilibrium. We investigate the gains from nonrecursive equilibria (e.g., “review strategies”) based on privately observed signals. Under a pairwise identification condition, we find that the gains from nonrecursive equilibria are “small”: their inefficiency is of the same 1 − δ power order as that of recursive equilibria. Thus, while private strategies or monitoring can outperform public ones for a fixed discount factor, they cannot accelerate the power rate of convergence to the efficient payoff frontier when the folk theorem holds. An implication is that the gains from withholding performance feedback from agents are small when the parties are patient.

Introduction

Most analysis of repeated moral hazard problems and games focuses on contracts and equilibria that are recursive in the players' continuation values. This approach is without loss in single-agent problems with public performance signals (Spear and Srivastava (1987)). It is also without loss in repeated games with imperfect public monitoring, if attention is restricted to equilibria in pure strategies or in strategies that depend only on the public signals (Abreu, Pearce, and Stacchetti (1990), Fudenberg, Levine, and Maskin (1994)). In contrast, in single-agent problems where the principal privately observes performance, or in repeated games where signals are privately observed by a mediator, more payoffs are implementable as compared to the case where the same signals are publicly observed, as concealing signals reduces the players' available deviations. Similarly, focusing on pure or public equilibria in repeated games with public monitoring is not without loss (Kandori and Obara (2006)). Yet, characterizing the equilibrium payoff set with private signals or strategies is intractable, precisely because this set lacks a tractable recursive structure (Kandori (2002)). The extent of the possible gains from nonrecursive equilibria based on private signals or strategies over recursive equilibria based on public signals is thus an open question, which forms the subject of the current paper.

We consider discounted repeated games where in each period players take actions a and a signal y is drawn from a distribution $p(y|a)$ with full support. We compare the equilibrium payoff sets in a version of the game with public monitoring, where the signal y is publicly observed and attention is restricted to equilibria in public strategies, and a version with private monitoring, where the signal y is observed only by a principal (or mediator) with commitment power, who privately recommends actions to the players. We call these two versions of the game the public game and the blind game. By the revelation principle, for any discount factor δ, the equilibrium payoff set is weakly larger in the blind game than the public game. Our question is, how much larger?

For any fixed discount factor $\delta <1$, this question is difficult to answer in any generality, because characterizing equilibrium payoffs in the blind game is intractable. We instead adopt a rate of convergence approach: under standard identification conditions that ensure that efficiency is attainable in the $\delta \to 1$ limit, how quickly does inefficiency vanish as $\delta \to 1$ in the most efficient equilibrium in the public game as compared to the blind game?

Our main result is that inefficiency is of the same power order of $1-\delta$ in both games. Thus, while private strategies or monitoring can outperform public ones for a fixed discount factor, they cannot accelerate the rate of convergence to the efficient payoff frontier when the folk theorem holds, except possibly for log terms in $1-\delta$. In this sense, the gains from nonrecursive equilibria are small when players are patient.

Our results have implications for the design of principal-agent relationships. An important design variable in such relationships is the amount of performance feedback provided to the agent. While providing feedback can have practical benefits that are not captured by our model, a benefit of withholding feedback is that this facilitates nonrecursive contracting by making the game blind rather than public. However, our results show that this benefit of withholding feedback is small when the parties are patient.

The high-level intuition for our results is that, as compared to a recursive contract where the agents' continuation values are revealed in every period, pooling information across periods improves monitoring precision but also necessitates larger rewards and punishments, which reduces the scope for providing incentives by transferring future surplus between the agents than destroying it through mutual punishments. Our analysis shows that these two effects essentially cancel out, so that little is gained by pooling information.

A subtlety in our results is that, while inefficiency is always of the same power order in the public game and the blind game, this order depends on the curvature of the boundary of the feasible payoff set. If the boundary is smooth with positive curvature (as in Green and Porter (1984), Spear and Srivastava (1987), Sannikov (2007, 2008), or Sadzik and Stacchetti (2015)), inefficiency is of order $1-\delta$.¹ In this case, the first-order inefficiency associated with small continuation payoff movements along the payoff boundary is zero. We show that this implies that inefficiency in the public and blind games differs only by a constant factor: i.e., the rates of convergence are identical. Moreover, for a class of smooth principal-agent models (similar to Spear and Srivastava (1987), or Sannikov (2008)), inefficiency in the public and blind games is identical up to a first-order approximation.

In contrast, if the boundary of the feasible payoff set is kinked (e.g., if there are finitely many actions), inefficiency is of power order ${(1-\delta )}^{1/2}$.² In this case, the first-order inefficiency associated with small continuation payoff movements is positive. We show that this greater inefficiency leads to a greater value of withholding feedback: now, inefficiency in the public and blind games can differ by a log factor in $1-\delta$. Thus, while the value of withholding feedback is always “small” (no improvement in the power rate of convergence), it is somewhat less small in the kinked case (where there can be a log-factor improvement) than in the smooth case (where there is at most a constant-factor improvement, with no first-order improvement whatsoever in standard principal-agent models).

Methodologically, we develop a new technique for bounding equilibrium payoffs in repeated games with private monitoring. The starting point is that continuation payoff rewards or punishments incur an efficiency loss related to the curvature of the boundary of the feasible payoff set, while providing incentives that are proportional to a likelihood ratio difference $(p(y|a)-p(y|a^{\prime }))/p(y|a)$. Since the likelihood ratio difference is a martingale increment (as the expected likelihood ratio difference under $p(\cdot |a)$ equals 0), large deviations theory can be used to bound the cumulative likelihood ratio difference over any number of periods. This bound connects the inefficiency and “incentive strength” of any strategy profile, so that any equilibrium where players do not take myopic best responses must incur a certain amount of inefficiency, regardless of whether signals are public or private.

Relation to the literature

Our finding that the gains from nonrecursive equilibria are small contrasts with two strands of prior literature that find large gains. These strands share the feature that continuation value transfers are impossible with public strategies. This feature reduces the efficiency of public strategies and thereby generates large gains from nonrecursive private strategies.

First, Holmström and Milgrom (1987) study a dynamic principal-agent model where the agent exerts effort over T periods and consumption occurs at the end of the game. The value of withholding feedback is large: without feedback, first-best profit can be approximated as $T\to \mathrm{\infty }$ using a “review strategy” that aggregates signals over T periods and harshly penalizes the agent if a count of signal realizations falls below a threshold;³ with feedback, optimal contracts are linear in the count of signal realizations, and profits are bounded away from the first best for all T. The key difference from our setup is that Holmström and Milgrom's model is not a repeated game (as consumption only occurs once), so efficiency cannot be improved by transferring continuation payoffs over time.⁴

Second, several papers study principal-agent problems or games that, while repeated, do not permit continuation value transfers. Abreu, Milgrom, and Pearce (1991) and Sannikov and Skrzypacz (2007) consider settings without pairwise identifiability, while Matsushima (2004) and Fuchs (2007) restrict attention to block belief-free equilibria. These settings preclude continuation value transfers, and consequently these papers all find that efficiency is attained as $\delta \to 1$ only when feedback is withheld.⁵

In past work (Sugaya and Wolitzky, 2017, 2018), we showed that the value of withholding feedback (“maintaining privacy”) is large in some specific repeated and dynamic games when δ is bounded away from 1. For example, our 2018 paper examined how maintaining privacy can help sustain multimarket collusion. In contrast, the current paper shows that the value of privacy in repeated games is small when δ is close to 1.

We also relate to the broader literature on feedback in dynamic agency and games. We consider complete information repeated games without payoff-relevant state variables, so feedback concerns only past performance, which is payoff-irrelevant in the continuation game. In contrast, most of the literature on feedback in dynamic agency considers dynamic games with additional state variables, such as an agent's ability (Ederer (2010), Smolin (2021)), other agents' progress in a tournament (Gershkov and Perry (2009), Aoyagi (2010), Ely, Georgiadis, and Rayo (2025)), whether a project has been completed (Halac, Kartik, and Liu (2017), Ely, Georgiadis, Khorasani, and Rayo (2023)), or the evolution of an exogenous state (Ely and Szydlowski (2020), Orlov, Skrzypacz, and Zryumov (2020), Ball (2023)). An exception is Lizzeri, Meyer, and Persico (2002), who examine optimal two-period agency contracts with and without a “midterm review.”

We also contribute to the literature on review strategies, introduced by Rubinstein (1979), Rubinstein and Yaari (1983), and Radner (1985), and developed by Abreu, Milgrom, and Pearce (1991) and Matsushima (2001, 2004). These papers show that review strategies can support efficient outcomes when $\delta \to 1$ (or when there is no discounting at all). In contrast, we show that review strategies cannot greatly outperform recursive contracts when δ is close to 1.

Methodologically, the closest papers are Hörner and Takahashi (2016), who build on Fudenberg, Levine, and Maskin's recursive methods to show that inefficiency is of order ${(1-\delta )}^{1/2}$ in repeated finite-action games with public monitoring; and Sugaya and Wolitzky (2023), who bound the strength of players' equilibrium incentives in repeated finite-action games with private monitoring.⁶ Rather than bounding incentives, the current paper derives a tradeoff between efficiency and incentives (expressed though Lagrange multipliers, e.g., program (4) below) and uses it to characterize the rate of convergence. In addition, the arguments in our 2023 paper are based on variance decomposition, while the current paper requires more precise estimates from martingale large deviations theory.

Finally, our exact characterization of first-order inefficiency in repeated principal-agent models relates to Sannikov (2008) and Sadzik and Stacchetti (2015), who derive similar results under public monitoring in continuous time or “frequent action” models. Here, our main contribution is showing that withholding feedback leaves first-order inefficiency unchanged.

Outline

The paper is organized as follows. Section 2 describes the model. Section 3 gives an informal overview of our results. Section 4 establishes general upper bounds on equilibrium efficiency. Section 5 establishes that these bounds are attainable in public equilibria (excepting a log factor in the finite-action case). Combining these results implies that the gains from nonrecursive equilibria are small. Section 6 gives a stronger result for principal-agent problems. Section 7 discusses extensions.

Preliminaries

This section introduces our model of repeated games with public monitoring and blind repeated games.

A stage game $G=(I,A,u)$ consists of a finite set of players $I=\{1,\mathrm{\dots },N\}$, a product set of actions $A={\times }_{i\in I}{A}_i$, and a payoff function $u_i:A\to \mathbb{R}$ for each $i\in I$. We assume that each $A_i$ is a nonempty, compact metric space and each $u_i$ is continuous.⁷ By the Debreu–Fan–Glicksberg theorem, the stage game admits a Nash equilibrium in mixed actions.

We fix some basic notation: the sets of stage-game Nash and correlated equilibria are ${\mathrm{\Sigma }}^{\mathrm{NE}}\subseteq {\times }_{i\in I}\mathrm{\Delta }(A_i)$ and ${\mathrm{\Sigma }}^{\mathrm{CE}}\subseteq \mathrm{\Delta }(A)$; the feasible payoff set is $F=\mathrm{co}({\{u(a)\}}_{a\in A})\subseteq {\mathbb{R}}^N$ with boundary bnd(F); the sets of stage-game Nash and correlated equilibrium payoffs are $V^{\mathrm{NE}}=\{v:v=u(\alpha )\text{ for some }\alpha \in {\mathrm{\Sigma }}^{\mathrm{NE}}\}$ and $V^{\mathrm{CE}}=\{v:v=u(\alpha )\text{ for some }\alpha \in {\mathrm{\Sigma }}^{\mathrm{CE}}\}$; the Euclidean metric and norm on ${\mathbb{R}}^N$ are $d(\cdot ,\cdot )$ and $\Vert \cdot \Vert$; and the set of unit vectors (or directions) in ${\mathbb{R}}^N$ is $\mathrm{\Lambda }=\{\lambda \in {\mathbb{R}}^N:\Vert \lambda \Vert =1\}$.

Recall that a payoff vector $v\in F$ is an exposed point of F if there exists $\lambda \in \mathrm{\Lambda }$ such that v uniquely maximizes $\lambda \cdot w$ over $w\in F$: that is, if the set of exposing directions ${\mathrm{\Lambda }}_v=\{\lambda \in \mathrm{\Lambda }:v={\mathrm{argmax}}_{w\in F}\lambda \cdot w\}$ is nonempty. If $|A|$ is finite, the sets of exposed and extreme points of F coincide; if $|A|$ is infinite, the exposed points are a dense subset of the extreme points.⁸ An example at the end of Section 4.1 motivates focusing on exposed points.

A monitoring structure $(Y,p)$ consists of a set of possible signal realizations Y and a family of conditional probability distributions $p(y|a)$. We assume that either Y is finite and y is drawn according to a probability mass function $p(y|a)$, or Y is measurable and y is drawn according to a density $p(y|a)$: we use the same notation $p(y|a)$ for both cases. We assume $p(y|a)>0$ for all $y\in Y$, $a\in A$. This full support assumption is crucial and, in particular, excludes perfect monitoring.

We also require that the monitoring structure satisfies the following key assumption. $1$

Assumption

There exists a number $K>0$ such that, for any $a\in A$, $i\in I$, and $a_i^{\prime }\in A_i$, we have 1${\mathbb{E}}^{y\sim p(\cdot |a)}\left[\exp \right(\theta \frac{{\displaystyle p(y|a)-p(y|a_i^{\prime },a_{-i})}}{{\displaystyle p(y|a)}}\left)\right]\le \exp \left(\frac{{\displaystyle {\theta }^{2}K}}{{\displaystyle 2}}\right)\text{for all }\theta \in \mathbb{R}.$

Assumption 1 says that the likelihood ratio difference between $p(\cdot |a)$ and $p(\cdot |a_i^{\prime },a_{-i})$ has a sub-Gaussian distribution, where the number K is called a variance proxy.⁹ For example, Assumption 1 holds if Y is finite, or if $Y\subseteq {\mathbb{R}}^n$ and $y=g(a)+\epsilon$, where $g:A\to Y$ is a deterministic function and ε has a multivariate normal distribution with covariance matrix independent of a.¹⁰ As we will see, Assumption 1 lets us apply results from large deviations theory to bound the power of tail tests that aggregate signals over many periods.

In a repeated game with public monitoring, in each period $t\in \mathbb{N}$, each player i takes an action $a_i$, and then a signal y is drawn according to $p(y|{(a_i)}_i)$ and is publicly observed. A history for player i at the beginning of period t takes the form $h_i^t={(a_{i,t^{\prime }},y_{t^{\prime }})}_{t^{\prime }=1}^{t-1}$. A strategy ${\sigma }_i$ for player i maps histories $h_i^t$ to distributions over actions $a_{i,t}$. A strategy for player i is public if it depends on $h_i^t$ only through its public component $y^t={(y_{t^{\prime }})}_{t^{\prime }=1}^{t-1}$. Players choose strategies to maximize discounted expected payoffs, with common discount factor $\delta \in [0,1)$. A perfect public equilibrium (PPE) is a profile of public strategies that, beginning at any period t and any public history $y^t$, forms a Nash equilibrium from that period on. We denote the repeated game with public monitoring with stage game G, monitoring structure $(Y,p)$, and discount factor δ by ${\mathrm{\Gamma }}^P(\delta )$, and we denote the corresponding set of PPE payoff vectors by $E^P(\delta )\subseteq {\mathbb{R}}^N$. Thus, $E^P(\delta )$ is the set of attainable payoffs in a PPE where signals are publicly observed.

In a blind repeated game, the players do not observe signals directly but are assisted by a mediator with commitment power.¹¹ In each period $t\in \mathbb{N}$, (i) the mediator privately recommends an action $r_i\in A_i$ to each player i, (ii) each player i takes an action $a_i$, and (iii) a signal y is drawn according to $p(y|{(a_i)}_i)$ and is observed only by the mediator. A history for the mediator at the beginning of period t takes the form $h_0^t={({(r_{i,t^{\prime }})}_i,y_{t^{\prime }})}_{t^{\prime }=1}^{t-1}$, while a history for player i just before she takes an action in period t takes the form $h_i^t=({(r_{i,t^{\prime }},a_{i,t^{\prime }})}_{t^{\prime }=1}^{t-1},r_{i,t})$. A strategy ${\sigma }_0$ for the mediator maps histories $h_0^t$ to distributions over recommendation profiles ${(r_{i,t})}_i$, while a strategy ${\sigma }_i$ for player i maps histories $h_i^t$ to distributions over actions $a_{i,t}$. We denote the blind repeated game with stage game G, monitoring structure $(Y,p)$, and discount factor δ by ${\mathrm{\Gamma }}^B(\delta )$, and we denote the corresponding set of Nash equilibrium payoff vectors (taking the union over all possible mediator strategies) by $E^B(\delta )\subseteq {\mathbb{R}}^N$. Thus, $E^B(\delta )$ is the set of attainable payoffs in a Nash equilibrium where signals are privately observed by a mediator.¹²

By standard arguments (similar to Forges (1986)), any Nash equilibrium outcome $\mu \in \mathrm{\Delta }({(A\times Y)}^{\mathrm{\infty }})$ in ${\mathrm{\Gamma }}^P(\delta )$ (i.e., any equilibrium distribution over infinite paths of action profiles and signals) can also be implemented by a Nash equilibrium in ${\mathrm{\Gamma }}^B(\delta )$ where the players follow the mediator's recommendations on path. Since PPE is a refinement of Nash equilibrium, it follows that $E^P(\delta )\subseteq E^B(\delta )$. Our goal is to evaluate the advantage of arbitrary equilibria based on private signals over recursive equilibria based on public signals: that is, to assess the size of the set $E^B(\delta )\backslash E^P(\delta )$.

$1$

Remark

The model is easily adapted to allow a player with commitment power (such as a principal with full commitment power in a principal-agent model) or one or more players with perfectly observed actions (such as a principal who offers contracts each period in a relational contracting model). A player with commitment power is treated like any other player, except that no incentive constraints are imposed on her strategy. For example, in a principal-agent model, ${\mathrm{\Sigma }}^{\mathrm{NE}}$ is the set of mixed action profiles where the agent does not have a profitable deviation. Moreover, it suffices to impose full support (and sub-Gaussianity) only for the agent, so that $\mathrm{supp}p(\cdot |a)=\mathrm{supp}p(\cdot |a^{\prime })$ for all a, $a^{\prime }$ that agree on the principal's action. Similarly, to extend our results to the case where some players' actions are perfectly observed, let $I^{\ast }\subseteq I$ be the set of players with observable actions, and assume that deviations by players $i\in I\backslash I^{\ast }$ do not affect the support of p, so that $\mathrm{supp}p(\cdot |a)=\mathrm{supp}p(\cdot |a_i^{\prime },a_{-i})$ for all $a\in A$, $i\in I\backslash I^{\ast }$, and $a_i^{\prime }\in A_i$. Then Theorem 1 below applies for any $v\in \exp (F)$ that cannot be attained by an action profile distribution α such that $g_i(s_i,\alpha )=0$ for each player $i\in I\backslash I^{\ast }$ and each manipulation $s_i$ (where these objects are defined in Section 4.2), while Theorem 2 applies verbatim.

Overview of results

We first provide an informal overview of our results. We focus on two leading cases: finite stage games and games where the boundary of the feasible payoff set has positive curvature.

Finite games

With a finite stage game, Hörner and Takahashi (2016) showed that the minimum distance between any PPE payoff vector and any extreme, nonstatic Nash payoff vector is of order ${(1-\delta )}^{1/2}$. This result relies on Fudenberg, Levine, and Maskin's (1994) recursive characterization of PPE and is generalized by our Theorem 2.

In contrast, our main result concerns arbitrary Nash equilibrium payoffs in the blind game. There is a wide range of nonrecursive equilibria in the blind game. A leading example of these equilibria is given by review strategies (Radner (1985), Abreu, Milgrom, and Pearce (1991), Matsushima (2004)), which aggregate signals over T periods—during which the players take constant actions—before adjusting the players' actions. Heuristically, an optimal review strategy pools information for $T=O({(1-\delta )}^{\mathrm{-1}})$ periods and then applies a penalty if the number of “good signals” over these periods falls short of a cutoff. Call the number of standard deviations by which the number of good signals falls short of its mean the score. Since the number of good signals, normalized by $T^{-1/2}$, is approximately normally distributed, for any cutoff score z the probability that a single signal is pivotal is $O(T^{-1/2}\varphi (z))=O({(1-\delta )}^{1/2}\varphi (z))$.¹³ As stage game payoffs are $O(1-\delta )$, incentive compatibility requires that the pivot probability is at least $O(1)$, so that z is at most $O({(-\log (1-\delta ))}^{1/2})$, which in turn implies that the review strategy's “false positive rate” (and hence its minimum inefficiency) is $\mathrm{\Phi }(-z)=O({((1-\delta )/(-\log (1-\delta )))}^{1/2})$.¹⁴ Thus, review strategies improve on PPE by at most a factor of ${(-\log (1-\delta ))}^{1/2}$. Our Theorem 1 implies that this factor is unimprovable. Thus, combining Theorems 1 and 2 shows that withholding feedback accelerates convergence to efficiency by at most a factor of ${(-\log (1-\delta ))}^{1/2}$. Moreover, our Proposition 1 constructs an equilibrium that attains this factor in a one-sided prisoner's dilemma, which shows that this result is tight.

Positive curvature

Now consider an infinite stage game where the boundary of the feasible payoff set has positive curvature. In this case, Theorem 2 shows that PPE in the public game can attain inefficiency of order $1-\delta$. As we explain following the statement of Theorem 2, this improved efficiency relative to finite games is obtained because a smooth set of equilibrium payoffs can approximate a smooth set of feasible payoffs more closely than a kinked set of feasible payoffs. Conversely, Theorem 1 shows that arbitrary Nash equilibria in the blind game cannot attain inefficiency of order less than $1-\delta$. Thus, in the positive curvature case, withholding feedback does not accelerate convergence to efficiency. Moreover, Theorem 3 shows that, in principal-agent problems, withholding feedback does not reduce first-order inefficiency.

Maximum efficiency with arbitrary strategies

Main result

Our main result gives an upper bound for the rate of convergence of $E^B(\delta )$ toward an exposed point $v\in \exp (F)$ that is not attainable as a static correlated equilibrium. As discussed above, the bound depends on the order of curvature of the boundary of F at v.

$1$

Definition

Fix an exposed point $v\in \exp (F)$. For any $\beta \ge 1$, the boundary of F has max-curvature of order at least β at v if, for all $\lambda \in {\mathrm{\Lambda }}_v$, there exists $\eta >0$ such that $\lambda \cdot (v-w)\ge \eta d{(v,w)}^{\beta }\text{for all }w\in \mathrm{bnd}(F).$ The boundary of F has max-curvature of order β at v if $\beta =\inf \{\tilde{\beta }:\mathrm{bnd}(F)\text{ has }\max \text{-curvature of order at least }\tilde{\beta }\text{ at }v\}.$

This definition says that moving away from an exposed point v in the convex set F entails an efficiency loss of order at least β, relative to Pareto weights λ: heuristically, $\mathrm{bnd}(F)$ is approximated by a power function of degree β at v. Note that $\beta \ge 1$ because F is convex, and $\beta <\mathrm{\infty }$ by the definition of an exposed point. To understand the definition, the key cases to consider are $\beta =1$, $\beta =2$, and the limit case $\beta =\mathrm{\infty }$.¹⁵

• The $\beta =1$ case arises when the stage game G is finite. This implies a first-order efficiency loss from moving away from any extreme point. This case is studied by Abreu, Pearce, and Stacchetti (1990), Fudenberg, Levine, and Maskin (1994), Hörner and Takahashi (2016), and Sugaya and Wolitzky (2023).
• The $\beta =2$ case arises when the boundary of F has positive curvature. This case is studied by Green and Porter (1984), Spear and Srivastava (1987), Sannikov (2007, 2008), and Sadzik and Stacchetti (2015). More generally, if $\beta \le 2$ then the boundary of F has nonzero curvature: its curvature is positive but finite if $\beta =2$ and is infinite if $\beta <2$.
• The $\beta =\mathrm{\infty }$ case arises in the limit where the boundary of F is linear at v. This limit case occurs in repeated games with transferable utility, as in Athey and Bagwell (2001), Levin (2003), and Goldlücke and Kranz (2012).

The following is our main result.

$1$

Theorem

Fix an exposed point $v\in \exp (F)\backslash V^{\mathrm{CE}}$ where $\mathrm{bnd}(F)$ has max-curvature of order β, and fix a direction $\lambda \in {\mathrm{\Lambda }}_v$. Then there exists $c>0$ such that $\begin{array}{c}\lambda \cdot (v-w)\ge c\zeta (\delta )\text{for all }\delta <1\text{ and }w\in E^B(\delta ),\text{where}\\ \zeta (\delta )=\{\begin{array}{cc}{\left(\frac{{\displaystyle 1-\delta }}{{\displaystyle \max \{-\log (1-\delta ),1\}}}\right)}^{1/2}\hfill & \text{if }\beta =1,\hfill \\ {(1-\delta )}^{\max \{\beta /2,\beta -1\}}\hfill & \text{if }\beta >1.\hfill \end{array}\end{array}$

The key implications of Theorem 1 are as follows:

• For Pareto weights where welfare is maximized at a kink in bnd(F), equilibrium inefficiency in the blind game is at least $O({((1-\delta )/(-\log (1-\delta )))}^{1/2})$.
• For Pareto weights where welfare is maximized at a point where bnd(F) has positive curvature, equilibrium inefficiency in the blind game is at least $O(1-\delta )$.

We will see that both of these bounds—as well as the ${(1-\delta )}^{\beta /2}$ bound for $\beta \in (1,2]$—are tight in the blind game. Moreover, the bound in the kinked case remains tight up to log-factor slack in the public game, while the bound in the $\beta \in (1,2]$ case remains tight up to constant-factor slack in the public game. These results imply that the gains from non-recursive equilibria are small at any point of nonzero curvature when players are patient.¹⁶

We outline the proof of Theorem 1 in the next subsection. The basic logic is that if a repeated game Nash equilibrium gives payoffs close to $v\in \exp (F)$, then the stage game payoff must be close to v almost all the time along the equilibrium path of play. Since signals have full support, this implies that continuation payoffs usually remain close to v even after low-probability signal realizations, and hence that equilibrium continuation play does not vary much with the signal realizations. But then, if $v\notin V^{\mathrm{CE}}$, we can conclude that δ must be so high that even small variations in continuation play can provide strong incentives.¹⁷

We mention a couple technical aspects of the statement of Theorem 1. First, generically, the condition $v\in \exp (F)\backslash V^{\mathrm{CE}}$ is equivalent to $v\in \exp (F)\backslash V^{\mathrm{NE}}$: since v is extremal, the distinction only matters in the nongeneric case where v is attained at two different pure action profiles. Second, we focus on exposed points because the condition $\lambda \in {\mathrm{\Lambda }}_v$ (i.e., $v={\mathrm{argmax}}_{w\in F}\lambda \cdot w$) cannot be weakened to $v\in {\mathrm{argmax}}_{w\in F}\lambda \cdot w$. To see this, consider the stage game Here, the point $v=(1,1)$ is exposed and is not attainable as a static CE; bnd(F) has curvature of order 1 (i.e., a kink) at v; and $v\in {\mathrm{argmax}}_{w\in F}\lambda \cdot w$ for $\lambda =(0,1)$. But the point $w=(0.5,1)$ is attained by the static NE $(C,\frac{1}{2}L+\frac{1}{2}R)$ (so $w\in E^B(\delta )$ for all $\delta \in [0,1)$) and satisfies $\lambda \cdot w=\lambda \cdot v$, so the conclusion of Theorem 1 fails.

Proof sketch for Theorem 1

We sketch the proof of Theorem 1, deferring the details to the Appendix. Fix any $v\in \exp (F)\backslash V^{\mathrm{CE}}$ and $\lambda \in {\mathrm{\Lambda }}_v$. We wish to derive a lower bound for $\lambda \cdot (v-w)$—the inefficiency of w in direction λ—which holds for any $w\in E^B(\delta )$.

We introduce some notation. Note that any outcome $\mu \in \mathrm{\Delta }({(A\times Y)}^{\mathrm{\infty }})$ defines a marginal distribution over period-t action profiles, ${\alpha }_t^{\mu }\in \mathrm{\Delta }(A)$, as well as an occupation measure over action profiles, ${\alpha }^{\mu }\in \mathrm{\Delta }(A)$, defined as ${\alpha }^{\mu }=(1-\delta )\sum _{t=1}^{\mathrm{\infty }}{\delta }^{t-1}{\alpha }_t^{\mu }.$ The players' ex ante payoffs under μ are determined by ${\alpha }^{\mu }$, as by linearity of u, $(1-\delta )\sum _{t=1}^{\mathrm{\infty }}{\delta }^{t-1}u\left({\alpha }_t^{\mu }\right)=u((1-\delta )\sum _{t=1}^{\mathrm{\infty }}{\delta }^{t-1}{\alpha }_t^{\mu })=u\left({\alpha }^{\mu }\right).$ Thus, letting ${\mathcal{M}}^B(\delta )$ be the set of Nash equilibrium outcomes μ in the blind game ${\mathrm{\Gamma }}^B(\delta )$, we wish to derive a lower bound for $\underset{\mu \in {\mathcal{M}}^B(\delta )}{\inf }\lambda \cdot (v-u({\alpha }^{\mu }\left)\right).$

Now, for each player i, let $S_i$ denote the set of functions $s_i:A_i\to A_i$, which we call manipulations. For any $i\in I$, $\alpha \in \mathrm{\Delta }(A)$, and $s_i\in S_i$, define the deviation gain $g_i(s_i,\alpha )=\sum _{a\in A}\alpha (a)\left(u_i\right(s_i(a_i),a_{-i})-u_i(a)).$ The interpretation is: if a recommended action profile a is drawn according to α and player i takes $s_i(a_i)$ when recommended $a_i$ rather than obeying the recommendation, her expected payoff gain is $g_i(s_i,\alpha )$. Finally, for any complete history of play $h={(a_t,y_t)}_{t=1}^{\mathrm{\infty }}$ and any player i and manipulation $s_i$, let ${û}_{i,t}(h)=u_i(a_t)-v_i\text{and}{\ell }_{i,t}(s_i,h)=\frac{{\displaystyle p(y_t|a_t)-p(y_t|s_i(a_{i,t}),a_{-i,t})}}{{\displaystyle p(y_t|a_t)}}.$ That is, ${û}_t(h)$ is the difference between player i's realized period t payoff at history h and $v_i$—which we will call player i's period t reward at history h—and ${\ell }_t(h)$ is the realized likelihood ratio difference of the period t signal $y_t$ at the period t action profile $a_t$, as compared to the action profile $(s_i(a_{i,t}),a_{-i,t})$ that results when player i manipulates according to $s_i$.

A simple necessary condition for an outcome μ to be consistent with equilibrium play (Lemma 6 in the Appendix) is that, for each player i, manipulation $s_i$, and period t, we have 2$g_i(s_i,{\alpha }_t^{\mu })\le {\mathbb{E}}^{\mu }[{\ell }_{i,t}(s_i,h)\sum _{t^{\prime }=t+1}^{\mathrm{\infty }}{\delta }^{t^{\prime }-t}{û}_{i,t^{\prime }}(h)].$ This inequality holds because, if it were violated, player i could gain by obeying her recommendation in every period other than t, while manipulating according to $s_i$ in period t. Given this inequality, since $\mathrm{bnd}(F)$ has max-curvature of order β at v, we have 3$\begin{array}{rl}& \underset{\mu \in {\mathcal{M}}^B(\delta )}{\inf }\lambda \cdot (v-u({\alpha }^{\mu }\left)\right)\\ & \ge \underset{\mu \in \mathrm{\Delta }({(A\times Y)}^{\mathrm{\infty }})}{\inf }\underset{i\in I,s_i\in S_i}{\sup }\underset{\begin{array}{c}\hfill {({û}_{i,t}(h))}_{t,h}\in [-{ū}_i,{ū}_i]\hfill \\ \hfill \text{such that (2)}\hfill \end{array}}{\inf }{\mathbb{E}}^{\mu }[(1-\delta )\sum _{t=1}^{\mathrm{\infty }}{\delta }^{t-1}\eta |{û}_{i,t}(h){|}^{\beta }],\end{array}$ where ${ū}_i$ is the range of $u_i$. Intuitively, the program (3) minimizes the maximum over players i and manipulations $s_i$ of the ${\beta }^{th}$ moment of the deviation of player i's stage game payoff from $v_i$, subject to the incentive constraint (2).

To prove the theorem, it remains to bound (3) as a function of δ and β. To do so, consider the inner problem where μ is fixed and $(i,s_i)\in {\mathrm{argmax}}_{i,s_i}{g}_i(s_i,{\alpha }^{\mu })$. Let $(1-\delta ){\delta }^{t-1}{\xi }_t$ denote the Lagrange multiplier on the period t incentive constraint, and form the Lagrangian 4$\underset{{({\xi }_t)}_t\ge 0}{\sup }\underset{{({û}_t(h))}_{t,h}\in [-ū,ū]}{\inf }(1-\delta )\sum _{t=1}^{\mathrm{\infty }}{\delta }^{t-1}{\mathbb{E}}^{\mu }\left[\eta \right|{û}_t(h){|}^{\beta }+{\xi }_t(g_t^{\mu }-{\ell }_t(h)\sum _{t^{\prime }=t+1}^{\mathrm{\infty }}{\delta }^{t^{\prime }-t}{û}_{t^{\prime }}(h))],$ where we have simplified notation by letting ${û}_t(h)={û}_{i,t}(h)$, $g_t^{\mu }=g_i(s_i,{\alpha }_t^{\mu })$, and ${\ell }_t(h)={\ell }_{i,t}(s_i,h)$. The Lagrangian expresses a tradeoff between efficiency and incentives: to maximize efficiency, the reward ${û}_t(h)$ must minimize the sum of the inefficiency resulting from the curvature of bnd(F) (i.e., $\eta |{û}_t(h){|}^{\beta }$) and an incentive cost in each earlier period $\tilde{t}<t$ (i.e., $-{\xi }_{\tilde{t}}{\ell }_{\tilde{t}}(h){û}_t(h)$). Moreover, if we take ${\xi }_t$ to be constant across periods (i.e., ${\xi }_t=\xi$ ∀t)—which suffices to bound the Lagrangian, by weak duality—we can reverse the order of summation between t and $t^{\prime }$ (and also note that ${û}_1(h)=0$ for all h at the optimum) to rewrite the Lagrangian as $\underset{\xi \ge 0}{\sup }\underset{{({û}_t(h))}_{t\ge 2,h}\in [-ū,ū]}{\inf }(1-\delta )\sum _{t=2}^{\mathrm{\infty }}{\delta }^{t-1}{\mathbb{E}}^{\mu }\left[\eta \right|{û}_t(h){|}^{\beta }-\xi {\mathcal{L}}_{t-1}(h){û}_t(h)]+\xi g^{\mu },$ where ${\mathcal{L}}_t(h)={\sum }_{t^{\prime }=1}^t{\ell }_{t^{\prime }}(h)$ and $g^{\mu }=g_i(s_i,{\alpha }^{\mu })=(1-\delta ){\sum }_{t=1}^{\mathrm{\infty }}{\delta }^{t-1}{g}_t^{\mu }$. Thus, to bound the Lagrangian, it suffices to bound the probability that $|{\mathcal{L}}_t(h)|$ is large. Since ${\mathcal{L}}_t$ is a martingale with sub-Gaussian increments (by Assumption 1), the required bound follows from the large deviations theory (Lemma 9). Intuitively, the bound say that sequences of signals with large cumulative likelihood ratio differences—which are highly informative when they occur—also occur with low equilibrium probability, and hence do not provide a large amount of information on average. This bound can be used to show that if we take $\xi =\zeta (\delta )$ then the value of the Lagrangian—and hence inefficiency under outcome μ—is not much smaller than $\zeta (\delta )g^{\mu }$. Finally, since $v\in \exp (F)\backslash V^{\mathrm{CE}}$, if $u({\alpha }^{\mu })$ is close to v then $g^{\mu }={\max }_{i,s_i}{g}_i(s_i,{\alpha }^{\mu })$ is bounded away from zero (Lemma 4), which yields the desired bound.

Tightness of the efficiency bound in the kinked case

We will see in the next section that inefficiency of order ${(1-\delta )}^{\beta /2}$ is attainable when $\beta \in [1,2]$ under public monitoring. This implies that the lower bound on inefficiency in Theorem 1 cannot be improved when $\beta \in (1,2]$ (the smooth, nonzero curvature case). Here, we show that in the kinked case ($\beta =1$) inefficiency of order ${((1-\delta )/-\log (1-\delta ))}^{1/2}$ is sometimes attainable in the blind game. This shows that the lower bound on inefficiency in Theorem 1 also cannot be improved when $\beta =1$. Consequently, withholding feedback can accelerate the rate of convergence by at most a factor of ${(-\log (1-\delta ))}^{-1/2}$ in the kinked case.

We consider a one-sided prisoner's dilemma, where the payoff matrix is and the monitoring structure is given by $Y={\{0,1\}}^2$ and $p((y_1,y_2)|(a_1,a_2))=p_1(y_1|a_1)p_2(y_2|a_2)$, with $p_1(1|C)=p_2(1|L)=1/2$ and $p_1(1|D)=p_2(1|R)=1/4$. The following result shows that the rate of convergence toward the efficient payoff vector $(2,2)$ is at least ${((1-\delta )/-\log (1-\delta ))}^{1/2}$.

$1$

Proposition

In the one-sided prisoner's dilemma, there exists $c>0$ such that, for any sufficiently large $\delta <1$, there exists $v\in E^B(\delta )$ satisfying $v_1=v_2>2-c{\left(\frac{{\displaystyle 1-\delta }}{{\displaystyle -\log (1-\delta )}}\right)}^{1/2}.$

The proof constructs a review strategy with inefficiency of order ${((1-\delta )/(-\log (1-\delta )))}^{1/2}$, as sketched in Section 3.1.

Attainable efficiency with public strategies

We now show that the maximum efficiency levels identified in Theorem 1 are attainable under public monitoring in the smooth, nonzero curvature case, and are attainable up to a log factor in the kinked case. To this end, denote the set of feasible and strictly individually rational payoffs by $F^{\ast }=\{v\in F:v_i>{\underset{\_}{v}}_i:={\min }_{{\alpha }_{-i}\in {\times }_{j\ne i}\mathrm{\Delta }(A_j)}{\max }_{a_i\in A_i}{u}_i(a_i,{\alpha }_{-i})\forall i\}$. For $v\in \mathrm{bnd}(F^{\ast })$, define ${\mathrm{\Lambda }}_v^{\ast }=\{\lambda \in \mathrm{\Lambda }:v\in {\mathrm{argmax}}_{w\in F^{\ast }}\lambda \cdot w\}$.

The following definition is a counterpart of Definition 1, adjusted to apply to all boundary points rather than only exposed points. It says that moving away from v along the boundary of $F^{\ast }$ entails an efficiency loss of order at most β, relative to Pareto weights λ.¹⁸

$2$

Definition

Fix a boundary point $v\in \mathrm{bnd}(F^{\ast })$. For any $\beta \ge 1$, the boundary of $F^{\ast }$ has min-curvature of order at most β at v if, for all $\lambda \in {\mathrm{\Lambda }}_v^{\ast }$, there exists $k>0$ such that $\lambda \cdot (v-w)<kd{(v,w)}^{\beta }\text{for all }w\in \mathrm{bnd}\left(F^{\ast }\right).$ The boundary of $F^{\ast }$ has min-curvature of order β at v if $\beta =\sup \{\tilde{\beta }:\mathrm{bnd}(F^{\ast }\left)\text{ has }\min \text{-curvature of order at most }\tilde{\beta }\text{ at }v\right\}.$

Note that, at any exposed point $v\in \mathrm{bnd}(F^{\ast })$, the min-curvature of $\mathrm{bnd}(F^{\ast })$ is at least 1 and at most the max-curvature.

The following assumption generalizes standard identification conditions for the public-monitoring folk theorem to the case where action sets can be infinite. $2$

Assumption

There exists $\overline{x}>0$ such that the following conditions hold:

• i.For each i, there exists a minmax profile against i, ${\alpha }^i\in {\times }_{j\ne i}\mathrm{\Delta }(A_j)\times A_i$, and $x_j:Y\to [-\overline{x},\overline{x}]$ for each $j\ne i$, such that 5$a_j\in \underset{a_j^{\prime }}{\mathrm{argmax}}{u}_j(a_j^{\prime },{\alpha }_{-j})+\mathbb{E}[x_j(y)|a_j^{\prime },{\alpha }_{-j}]\text{for all }j\ne i\text{and}{a}_j\in \mathrm{supp}({\alpha }_j).$
• ii.For each $a\in A$, $c\in \{-1,+1\}$, and $(i,j)$ with $i\ne j$, there exists $x_i:Y\to [-\overline{x},\overline{x}]$ such that 6$a_i\in \underset{a_i^{\prime }}{\mathrm{argmax}}{u}_i(a_i^{\prime },a_{-i})+\mathbb{E}[x_i(y)|a_i^{\prime },a_{-i}]\text{and}$ 7$a_j\in \underset{a_j^{\prime }}{\mathrm{argmax}}\mathbb{E}[c{x}_i(y)|a_j^{\prime },a_{-j}].$

Intuitively, Assumption 2 says that, when payoff transfers of magnitude at most $\overline{x}$ are available, players −i can be incentivized to minmax player i, and player i can be incentivized to take a given action $a_i$ via transfers from player j without affecting player j's incentive to take a given action $a_j$.¹⁹

We consider the rate of convergence of $E^P(\delta )$ toward a strictly individually rational payoff vector $v\in \mathrm{bnd}(F^{\ast })$. For finite stage games, Hörner and Takahashi (2016) show that this rate equals ${(1-\delta )}^{1/2}$. Thus, withholding feedback can accelerate the rate of convergence by at most a factor of ${(-\log (1-\delta ))}^{-1/2}$ in finite-action games. We now show that whenever the boundary of $F^{\ast }$ has nonzero curvature ($\beta \le 2$), the rate equals ${(1-\delta )}^{\beta /2}$. (We discuss the zero curvature case below.) Thus, withholding feedback cannot accelerate the rate of convergence in smooth games with nonzero curvature.

We require the standard assumption that $\dim F^{\ast }=N$ and further exclude payoff vectors where some player obtains her maximum feasible payoff.²⁰

$2$

Theorem

Assume that Assumption 2 holds and $\dim F^{\ast }=N$, and fix any $v\in \mathrm{bnd}(F^{\ast })$, satisfying $v_i<{\max }_a{u}_i(a)$ for all i, where $\mathrm{bnd}(F^{\ast })$ has min-curvature of order $\beta \ge 1$. Then there exists $c>0$ such that $d(v,E^P(\delta ))\le c{(1-\delta )}^{\min \{\beta ,2\}/2}$ for any sufficiently large $\delta <1$.

Theorem 2 builds on Fudenberg, Levine, and Maskin (1994), Hörner and Takahashi (2016), and Sugaya and Wolitzky (2023). As these authors showed, a given level of inefficiency relative to an exposed point v and a direction $\lambda \in {\mathrm{\Lambda }}_v$ is attainable under public monitoring if it equals the distance in direction λ between v and a self-generating ball $B\subseteq F$. We thus seek a self-generating ball $B\subseteq F$ at distance $O({(1-\delta )}^{\min \{\beta ,2\}/2})$ to v in direction $\lambda \in {\mathrm{\Lambda }}_v$. To this end, let $d=d(B,v)$ be the desired distance, and (without loss) let $u=v-d\lambda$ be the closest point to v in B (see Figure 1). Consider decomposing u into an instantaneous payoff v and continuation payoffs ${(w(y))}_y$ that lie on the translated tangent hyperplane H with normal vector λ passing through the point $\mathbb{E}[w(y)]=v-((1-\delta )/\delta )d\lambda$. Under Assumption 2, the continuation payoffs ${(w(y))}_y$ can be chosen to enforce v on $H\cap B$ if the diameter of $H\cap B$, which we denote by x, is of order $1-\delta$. At the same time, denoting the radius of B by r, the Pythagorean theorem gives ${(x/2)}^2+{(r-((1-\delta )/\delta )d)}^2=r^2$, and hence $x=O(\sqrt{(1-\delta )rd})$. It follows that the product rd is of order $1-\delta$, and hence $r=O({(1-\delta )}^{1-\min \{\beta ,2\}/2})$. Finally, for a point v where the (max-)curvature of $\mathrm{bnd}(F)$ equals β, a ball B with radius $r=O({(1-\delta )}^{1-\min \{\beta ,2\}/2})$ and center $v-(r+d)\lambda$, where $d=O({(1-\delta )}^{\min \{\beta ,2\}/2})$, lies entirely within F. For example, if $\beta =1$ then r and d are both $O({(1-\delta )}^{1/2})$, and thus shrink at the same rate as $\delta \to 1$; while if $\beta \ge 2$ then $r=O(1)$ and $d=O(1-\delta )$, so B simply shifts toward v as $\delta \to 1$.²¹

[IMAGE OMITTED. SEE PDF]

In light of Theorem 1, when $\beta >2$ one might hope to find conditions under which $d(v,E^P(\delta ))=O({(1-\delta )}^{\beta -1})$. While this may be possible, we do not pursue such a result here. The difficulty is that the corresponding ball B would have radius r of at least $O({(1-\delta )}^{2-\beta })$ (as rd must be at least $O(1-\delta )$). While such a ball can satisfy the self-generation condition $B\subseteq F$ in a neighborhood of v, its radius explodes as $\delta \to 1$ (when $\beta >2$), so it must violate self-generation at some point far from v. Therefore, any conditions that ensure that $d(v,E^P(\delta ))$ is less than $O(1-\delta )$ must involve the global geometry of the feasible payoff set. Investigating such conditions is left for future work.

We finally mention a class of infinite games where Assumption 2(ii) holds.²² Say that the game is linear-concave if (i) for each i, $A_i$ is a compact interval $[{\underset{\_}{A}}_i,{Ā}_i]\subseteq \mathbb{R}$, and $u_i(a_i,a_{-i})$ is differentiable and concave in $a_i$ for every $a_{-i}$ with a bounded derivative: there exists $\kappa >0$ such that $|\partial u_i(a_i,a_{-i})/\partial a_i|\le \kappa$ for all i, a; and (ii) the public signal is a D-dimensional real variable, $Y={\times }_{d=1}^D{Y}^d\subseteq {\mathbb{R}}^D$, and ${\mu }^d(a)=\mathbb{E}[y^d|a]$ is a linear function of a for each dimension d. In a linear-concave game, let $M^i(a)={(\frac{d}{d{a}_i}{\mu }^d(â){|}_{â=a})}_d$ be a D-dimensional vector representing the sensitivity of the mean public signal to player i's action. Say that a linear-concave game satisfies pairwise identifiability if for any a and $i\ne j$, $M^i(a)\ne 0$, and the spans of $M^i(a)$ and $M^j(a)$ intersect only at the origin.²³

$2$

Proposition

In any linear-concave game satisfying pairwise identifiability, Assumption 2(ii) holds.

Proof of Theorem 2

We recall a key definition and lemma from Abreu, Pearce, and Stacchetti (1990).

$3$

Definition

A bounded set $W\subseteq {\mathbb{R}}^N$ is self-generating if for all $\hat{v}\in W$, there exist $\alpha \in {\times }_i\mathrm{\Delta }(A_i)$ and $w:Y\to {\mathbb{R}}^N$ satisfying

• Promise keeping (PK)$\hat{v}=(1-\delta )u(\alpha )+\delta {\int }_{y}w(y)p(y|\alpha )dy$.
• Incentive compatibility (IC)$\mathrm{supp}({\alpha }_i)\subseteq {\mathrm{argmax}}_{a_i}(1-\delta )u_i(a_i,{\alpha }_{-i})+\delta {\int }_y{w}_i(y)p(y|a_i,{\alpha }_{-i})dy$ for all i.
• Self-generation (SG)$w(y)\in W$ for all y.

When (PK), (IC), and (SG) hold, we say that the pair $(\alpha ,w)$ decomposes $\hat{v}$ on W.

$1$

Lemma

Any bounded, self-generating set W is contained in $E^P(\delta )$.

It thus suffices to find a bounded, self-generating set W such that $d(v,W)=O({(1-\delta )}^{{\beta }^{\ast }/2})$, where ${\beta }^{\ast }=\min \{\beta ,2\}$. To do so, we first establish a sufficient condition for a ball B to be self-generating. This condition builds on Fudenberg and Levine (1994) and Sugaya and Wolitzky (2023).²⁴

$4$

Definition

The maximum score in direction $\lambda \in \mathrm{\Lambda }$ with reward bound $\overline{x}>0$ is $k(\lambda ,\overline{x}):=\underset{\alpha \in {\times }_i\mathrm{\Delta }(A_i),x:Y\to {\mathbb{R}}^N}{\sup }\lambda \cdot (u(\alpha )+{\int }_{y}x(y)p(y|\alpha )dy),\text{subject to:}$

• 1.(IC): $\mathrm{supp}({\alpha }_i)\subseteq {\mathrm{argmax}}_{a_i}{u}_i(a_i,{\alpha }_{-i})+{\int }_y{x}_i(y)p(y|a_i,{\alpha }_{-i})dy$ for all i.
• 2.Half-space decomposability with reward bound $\overline{x}$ (HS$\overline{x}$): $\lambda \cdot x(y)\le 0$ and $\Vert x(y)\Vert \le \overline{x}$ for all y.

$2$

Lemma

For any $\overline{x}>{\max }_{u,u^{\prime }\in F}\Vert u-u^{\prime }\Vert$ and $\epsilon >0$, if a ball B of radius r satisfies 8$k(\lambda ,\overline{x})\ge \underset{v^{\prime }\in B}{\max }\lambda \cdot v^{\prime }+\epsilon \text{ \hspace{1em}for all }\lambda \in \mathrm{\Lambda },\text{and}$ 9${\overline{x}}^2\le \frac{{\displaystyle \delta }}{{\displaystyle 1-\delta }}\frac{{\displaystyle \epsilon r}}{{\displaystyle 36}},$ then B is self-generating.

We then show that there exists B with $d(v,B)=O({(1-\delta )}^{{\beta }^{\ast }/2})$ that satisfies the sufficient condition for self-generation just given.

$3$

Lemma

There exist $\overline{x}>{\max }_{u,u^{\prime }\in F}\Vert u-u^{\prime }\Vert$, $c>0$ and $\overline{\delta }<1$ such that, for any $\delta >\overline{\delta }$, there exist $\epsilon >0$ and a ball B of radius r satisfying (8), (9), and $d(v,B)\le c{(1-\delta )}^{{\beta }^{\ast }/2}$.

The proof of Lemma 3 uses Assumption 2 and the assumptions that $\dim F^{\ast }=N$, $v_i\in ({\underset{\_}{v}}_i,{\max }_a{u}_i(a))$ for all i, and $\mathrm{bnd}(F^{\ast })$ has min-curvature of order $\beta \ge 1$ at v. The logic is similar to that accompanying Figure 1.

The proofs of Lemmas 2 and 3 are deferred to the Appendix. Given these lemmas, taking $\overline{x}$, c, and $\overline{\delta }$ as in Lemma 3 establishes Theorem 2.

A stronger result for the principal-agent problem

In this section, we establish that withholding feedback in a standard repeated principal-agent problem leaves unchanged not only the rate of convergence to efficiency (the order of inefficiency in $1-\delta$), but also the exact level of first-order inefficiency (the constant multiplying $1-\delta$). This stronger result also has the virtue of identifying the precise features of the stage game and the monitoring structure that determine the level of first-order inefficiency.

Consider a standard repeated principal-agent problem in discrete time. In each period t, an agent chooses an effort level a from a compact interval $A=[0,Ā]$, and a signal y is then drawn according to a pmf or pdf $p(y|a)$. Assume that $p(y|a)$ is twice continuously differentiable in a, with first and second derivatives $p_a(y|a)$ and $p_{aa}(y|a)$. A contract specifies, for each period t, a recommended effort level $r_t\in A$ as a function of the history of past recommendations and signals ${(r_{t^{\prime }},y_{t^{\prime }})}_{t^{\prime }=1}^{t-1}$, as well as the agent's current consumption $c_t\ge 0$ as a function of ${(r_{t^{\prime }},y_{t^{\prime }})}_{t^{\prime }=1}^t$. In the public game, the agent chooses her period t action $a_t$ as a function of $({(r_{t^{\prime }},a_{t^{\prime }},y_{t^{\prime }})}_{t=1}^{t-1},r_t)$; in the blind game, she chooses $a_t$ as a function of $({(r_{t^{\prime }},a_{t^{\prime }})}_{t=1}^{t-1},r_t)$ only. The agent's payoff in period t is $u(c_t)-\psi (a_t)$, where the consumption utility u is twice continuously differentiable on ${\mathbb{R}}_{+}$ with $u(0)=0$, $u^{\prime }>0$, $u^{\prime \prime }<0$, ${\lim }_{c\to \mathrm{\infty }}{u}^{\prime }(c)=0$, and 10$\underset{c\in [0,\mathrm{\infty })}{\sup }\frac{{\displaystyle u^{\prime \prime }(c)}}{{\displaystyle {(u^{\prime }(c))}^3}}<0,$ and the effort cost ψ is twice continuously differentiable on A with $\psi (0)={\psi }^{\prime }(0)=0$ and ${\psi }^{\prime \prime }>0$. (We discuss the role of condition (10) below.) The principal's payoff in period t is $a_t-c_t$.²⁵ The parties have the same discount factor $\delta \in [0,1)$.

We remark that our public game is nearly identical to the model of Spear and Srivastava (1987) (although unlike them we do not require a monotone likelihood ratio) or a discrete-time version of the baseline model of Sannikov (2008) (although we allow much more general monitoring structures, as discussed below).²⁶ However, those papers do not consider the possibility of withholding feedback from the agent.

For any effort level $a\in A$, the score of the signal y is $\nu (y|a)=\frac{{\displaystyle p_a(y|a)}}{{\displaystyle p(y|a)}}\text{for all }y\in Y,$ and the Fisher information—the variance of the score—is $\mathcal{I}(a)={\int }_y\frac{{\displaystyle p_a{(y|a)}^2}}{{\displaystyle p(y|a)}}dy.$ We require the following technical assumption, which implies that the Fisher information is finite, strictly positive, and Lipschitz continuous in a; the distribution of the score is sub-Gaussian; and a second-order condition holds. $3$

Assumption

The following hold:

• i.For all $a\in A$, there exists $\mathrm{\Delta }>0$ such that ${\int }_y\frac{{\displaystyle \underset{ã\in [a,a+\mathrm{\Delta }]}{\max }{p}_a{(y|ã)}^2}}{{\displaystyle p(y|a)}}dy<\mathrm{\infty }.$
• ii.$\mathcal{I}(a)$ is strictly positive and Lipschitz continuous on A.
• iii.The score $\nu (y|a)$ is sub-Gaussian with variance proxy $\mathcal{I}(a)$: ${\int }_y\exp (\theta \nu (y|a))p(y|a)dy\le \exp \left(\frac{{\displaystyle {\theta }^2\mathcal{I}(a)}}{{\displaystyle 2}}\right)\text{for all }\theta \in \mathbb{R}.$
• iv.There exists $\hat{K}$ such that, for all $a,â\in A$, we have 11${\int }_y\nu (y|a)p_{aa}(y|â)dy\le 0\text{and}$ 12${\int }_y\frac{{\displaystyle p_{aa}{(y|â)}^2}}{{\displaystyle p(y|a)}}dy\le \hat{K}.$

For example, Assumption 3 is satisfied if Y is finite, or if $Y\subseteq {\mathbb{R}}^n$ and $y=g(a)+\epsilon$ for a deterministic function $g:A\to Y$ with a bounded gradient and multivariate normal noise ε with covariance independent of a. Note that Assumption 3(iii) strengthens Assumption 1.

For any $w\in [-\psi (Ā),ū)$, where $ū={\lim }_{c\to \mathrm{\infty }}u(c)\in {\mathbb{R}}_{+}\cup \{\mathrm{\infty }\}$, let $\overline{F}(w)$ be the first-best payoff for the principal when the agent's payoff equals w, which is given by $\overline{F}(w)=\underset{a\in A}{\max }a-u^{\mathrm{-1}}(w+\psi (a)).$ Let $ā(w)$ be the maximizer (which is unique, as the maximand is strictly concave), and let $\overline{c}(w)=u^{\mathrm{-1}}(w+\psi (ā(w)))$ be the corresponding consumption for the agent. Note that $\overline{F}$ is twice continuously differentiable, and ā and $\overline{c}$ are continuously differentiable. In addition, since ${\psi }^{\prime }(0)=0$ and $u^{\prime }>0$, we have $ā(w)>0$ for all $w\in [-\psi (Ā),ū)$.

Finally, let $F_{\delta }^B(w)$ (resp., $F_{\delta }^P(w)$) denote the maximum payoff for the principal over all $v\in E^B(\delta )$ (resp., $v\in E^P(\delta )$) where the agent's payoff is w. That is, $F_{\delta }^B(w)$ is the principal's second-best payoff in the blind game, while $F_{\delta }^P(w)$ is her second-best payoff in the public game. Recall that $E^B(\delta )\supseteq E^P(\delta )$, so $F_{\delta }^B(w)\ge F_{\delta }^P(w)$. Nonetheless, we show that $F_{\delta }^B(w)$ and $F_{\delta }^P(w)$ agree up to a first-order approximation as $\delta \to 1$.

$3$

Theorem

For any $\delta <1$ and $w\in (0,ū)$, we have 13$\begin{array}{rl}{F}_{\delta }^B(w)& =\overline{F}(w)+\frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}\frac{{\displaystyle {\psi }^{\prime }{(ā(w))}^2}}{{\displaystyle \mathcal{I}(ā(w))}}\frac{{\displaystyle {\overline{F}}^{\prime \prime }(w)}}{{\displaystyle 2}}+o(1-\delta )\text{and}\\ F_{\delta }^P(w)& =\overline{F}(w)+\frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}\frac{{\displaystyle {\psi }^{\prime }{(ā(w))}^2}}{{\displaystyle \mathcal{I}(ā(w))}}\frac{{\displaystyle {\overline{F}}^{\prime \prime }(w)}}{{\displaystyle 2}}+o(1-\delta ),\end{array}$ where in each equation $o(1-\delta )$ stands for a (different) function satisfying ${\lim }_{\delta \to 1}o(1-\delta )/(1-\delta )=0$.

Theorem 3 shows that, whether or not the agent receives feedback, the first-order inefficiency of an optimal contract is precisely 14$\frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}\frac{{\displaystyle {\psi }^{\prime }{(ā(w))}^2}}{{\displaystyle \mathcal{I}(ā(w))}}\frac{{\displaystyle {\overline{F}}^{\prime \prime }(w)}}{{\displaystyle 2}}.$ In the public monitoring case, this result is similar to Theorem 5 of Sannikov (2008) and Corollary 1 of Sadzik and Stacchetti (2015), but we consider a discrete-time game with a general monitoring structure, while Sannikov considers a continuous-time game with Brownian noise, and Sadzik and Stacchetti consider the “frequent action limit” of a class of monitoring structures that converge to Brownian noise. However, the key point of Theorem 3 is that the first-order inefficiency is exactly the same under private monitoring. Thus, even if the principal can conceal the agent's past performance in a standard repeated principal-agent problem, she can do little better than to fully reveal it and utilize a public contract.

A rough intuition for Theorem 3 is that, with high probability, the agent's continuation payoff is approximately constant for a long time under an optimal contract, so there is little information about the continuation payoff to conceal, and thus little to gain from concealing it.

The proof of Theorem 3 is facilitated by the principal's ability to commit to delivering any feasible promised continuation value for the agent. It may be possible to generalize Theorem 3 to smooth games with 1-dimensional actions and product structure monitoring (as considered by Sannikov (2007)), but this would require constructing equilibria that attain specific continuation payoff vectors far from the initial target vector. This possibility is left for future research.

We finally comment on the role of condition (10). This condition implies that the second-order efficiency loss from varying the agent's utility is uniformly bounded away from zero. With CRRA utility $u(c)=c^{1-\gamma }/(1-\gamma )$, it holds if and only if $\gamma \ge 1/2$. Without this condition, review strategies with infrequent, large rewards may yield a first-order improvement over (14) if $u^{\prime }(c)$ converges to 0 sufficiently slowly as $c\to \mathrm{\infty }$.

Proof sketch for Theorem 3

We first bound $F_{\delta }^B(w)$ from above. Fix a period t and a small constant $\mathrm{\Delta }>0$, and consider the manipulation where, whenever the agent is recommended effort a in period t, she instead takes effort $a-({\psi }^{\prime }(a)/\mathcal{I}(a))\mathrm{\Delta }$. (The agent thus shades her effort more after recommendations where effort is more costly or less detectable.) For this manipulation to be unprofitable for all t and $\mathrm{\Delta }>0$, we must have 15${\mathbb{E}}^{\mu }[\frac{{\displaystyle {\psi }^{\prime }{(a_t)}^2}}{{\displaystyle \mathcal{I}(a_t)}}-\frac{{\displaystyle {\psi }^{\prime }(a_t)\nu (y_t|a_t)}}{{\displaystyle \mathcal{I}(a_t)}}\sum _{t^{\prime }=t+1}^{\mathrm{\infty }}{\delta }^{t^{\prime }-t}{û}_{t^{\prime }}]\le 0\text{for all }t,$ where μ is the equilibrium occupation measure and ${û}_{t^{\prime }}=u(c_{t^{\prime }})-\psi (a_{t^{\prime }})-w$ is the deviation of the agent's period $t^{\prime }$ utility from w (see Lemma 22 in the Appendix). Letting $(1-\delta ){\delta }^{t-1}{\xi }_t$ denote the Lagrange multiplier on this relaxed period t incentive constraint (as in the proof Theorem 1) and letting ${\xi }_t=\frac{1-\delta }{\delta }{\overline{F}}^{\prime \prime }(w)$ for all t, the inner problem in the Lagrangian (4) becomes $\underset{{({û}_t(h))}_{t,h}}{\inf }(1-\delta )\sum _{t=1}^{\mathrm{\infty }}{\delta }^{t-1}{\mathbb{E}}^{\mu }\left[\begin{array}{c}\hfill \overline{F}(w+{û}_t(h))-\overline{F}(w)\hfill \\ \hfill +\frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}{\overline{F}}^{\prime \prime }(w)(\frac{{\displaystyle {\psi }^{\prime }{(a_t)}^2}}{{\displaystyle \mathcal{I}(a_t)}}-\frac{{\displaystyle {\psi }^{\prime }(a_t)\nu (y_t|a_t)}}{{\displaystyle \mathcal{I}(a_t)}}\sum _{t^{\prime }=t+1}^{\mathrm{\infty }}{\delta }^{t^{\prime }-t}{û}_{t^{\prime }}(h))\hfill \end{array}\right].$ Taking the Taylor approximation $\overline{F}(w+{û}_t(h))-\overline{F}(w)\approx \frac{{\overline{F}}^{\prime \prime }(w)}{2}{û}_t{(h)}^2$ and rearranging, this equals $\frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}\frac{{\displaystyle {\overline{F}}^{\prime \prime }(w)}}{{\displaystyle 2}}\underset{{({û}_t(h))}_{t,h}}{\inf }\sum _{t=1}^{\mathrm{\infty }}{\mathbb{E}}^{\mu }\left[\begin{array}{c}\hfill {\delta }^t{û}_t{(h)}^2-2\frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}\frac{{\displaystyle {\psi }^{\prime }(a_t)\nu (y_t|a_t)}}{{\displaystyle \mathcal{I}(a_t)}}\sum _{t^{\prime }=t+1}^{\mathrm{\infty }}{\delta }^{t^{\prime }}{û}_{t^{\prime }}(h)\hfill \\ \hfill +2(1-\delta )\sum _{t=1}^{\mathrm{\infty }}{\delta }^{t-1}\frac{{\displaystyle {\psi }^{\prime }{(a_t)}^2}}{{\displaystyle \mathcal{I}(a_t)}}\hfill \end{array}\right].$ The FOC for ${û}_t$ is ${û}_t(h)=\frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}\sum _{t^{\prime }=1}^{t-1}\frac{{\displaystyle {\psi }^{\prime }(a_{t^{\prime }})\nu (y_{t^{\prime }}|a_{t^{\prime }})}}{{\displaystyle \mathcal{I}(a_{t^{\prime }})}}\text{for all }t\ge 2\text{ and }h.$ ²⁷ Substituting the FOC into the Lagrangian (together with ${û}_1(h)=0$ for all h) and simplifying (see equation (40) in the Appendix) gives $\frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}{\mathbb{E}}^{{\alpha }^{\mu }}\left[\frac{{\displaystyle {\psi }^{\prime }{(a)}^2}}{{\displaystyle \mathcal{I}(a)}}\right]\frac{{\displaystyle {\overline{F}}^{\prime \prime }(w)}}{{\displaystyle 2}}.$ Finally, to attain inefficiency of order $1-\delta$, we must have $|{\alpha }^{\mu }-a(\overline{w})|\le O(1-\delta )$. Inefficiency is thus no less than $\begin{array}{rl}& \frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}\frac{{\displaystyle {\psi }^{\prime }{(ā(w))}^2}}{{\displaystyle \mathcal{I}(ā(w))}}\frac{{\displaystyle {\overline{F}}^{\prime \prime }(w)}}{{\displaystyle 2}}+\frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}\left({\mathbb{E}}^{{\alpha }^{\mu }}\right[\frac{{\displaystyle {\psi }^{\prime }{(a)}^2}}{{\displaystyle \mathcal{I}(a)}}]-\frac{{\displaystyle {\psi }^{\prime }{(ā(w))}^2}}{{\displaystyle \mathcal{I}(ā(w))}})\frac{{\displaystyle {\overline{F}}^{\prime \prime }(w)}}{{\displaystyle 2}}\\ & =\frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}\frac{{\displaystyle {\psi }^{\prime }{(ā(w))}^2}}{{\displaystyle \mathcal{I}(ā(w))}}\frac{{\displaystyle {\overline{F}}^{\prime \prime }(w)}}{{\displaystyle 2}}+o(1-\delta ).\end{array}$

We next bound $F_{\delta }^P(w)$ from below. Given a continuation payoff $w_t$ for the agent, suppose the principal implements first-best effort $ā(w_t)$ by offering the corresponding first-best consumption $\overline{c}(w_t)$ and providing incentives entirely by varying the continuation payoff $w_{t+1}$ while making it a martingale: $\mathbb{E}[w_{t+1}|w_t]=w_t$. The Taylor approximation of inefficiency is then equal to $\sum _{t=1}^{\mathrm{\infty }}{\mathbb{E}}^{\mu }\left[{(w_{t+1}-w_t)}^2\right]\frac{{\displaystyle {\overline{F}}^{\prime \prime }(w_t)}}{{\displaystyle 2}}.$ To bound the variance ${\mathbb{E}}^{\mu }[{(w_{t+1}-w_t)}^2]$, note that the agent's incentive constraint is $ā(w_t)\in \underset{a}{\mathrm{argmax}}(1-\delta )\left(u\right(\overline{c}(w_t))-\psi (a))+\delta \int p(y|a)(w_{t+1}(y)-w_t)dy,$ with FOC $(1-\delta ){\psi }^{\prime }(ā(w_t))+\delta \int p_a(y|ā(w_t)\left)\right(w_{t+1}(y)-w_t)dy=0.$ To minimize variance subject to the agent's FOC, the principal takes $w_{t+1}(y)-w_t$ proportional to $\nu (y|ā(w_t))$, which gives variance ${\mathbb{E}}^{\mu }\left[{(w_{t+1}-w_t)}^2\right]={\left(\frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}\right)}^2\frac{{\displaystyle {\psi }^{\prime }{(ā(w_t))}^2}}{{\displaystyle \mathcal{I}(ā(w_t))}}.$ ²⁸ The resulting ex ante inefficiency thus equals $\begin{array}{rl}& {\mathbb{E}}^{\mu }\left[\sum _t{\delta }^{t-1}{\left(\frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}\right)}^2\frac{{\displaystyle {\psi }^{\prime }{(ā(w_t))}^2}}{{\displaystyle I(ā(w_t))}}\frac{{\displaystyle {\overline{F}}^{\prime \prime }(w_t)}}{{\displaystyle 2}}\right]\\ & =\frac{{\displaystyle 1-\delta }}{{\displaystyle {\delta }^2}}\frac{{\displaystyle {\psi }^{\prime }{(ā(w))}^2}}{{\displaystyle \mathcal{I}(ā(w))}}\frac{{\displaystyle {\overline{F}}^{\prime \prime }(w)}}{{\displaystyle 2}}\\ & +\sum _t{\delta }^{t-1}{\left(\frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}\right)}^2{\mathbb{E}}^{\mu }[\frac{{\displaystyle {\psi }^{\prime }{(ā(w_t))}^2}}{{\displaystyle I(ā(w_t))}}\frac{{\displaystyle {\overline{F}}^{\prime \prime }(w_t)}}{{\displaystyle 2}}-\frac{{\displaystyle {\psi }^{\prime }{(ā(w))}^2}}{{\displaystyle \mathcal{I}(ā(w))}}\frac{{\displaystyle {\overline{F}}^{\prime \prime }(w)}}{{\displaystyle 2}}]\\ & =\frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}\frac{{\displaystyle {\psi }^{\prime }{(ā(w))}^2}}{{\displaystyle \mathcal{I}(ā(w))}}\frac{{\displaystyle {\overline{F}}^{\prime \prime }(w)}}{{\displaystyle 2}}+o(1-\delta ),\end{array}$ where the last line follows as ${\mathbb{E}}^{\mu }[{(w_{t+1}-w_t)}^2]=O({(1-\delta )}^2)$, and hence $|w_t-w|\le O(\sqrt{t}(1-\delta ))$ with high probability, so in the second line both the sum from $t=1$ to ${(1-\delta )}^{\mathrm{-1}}$ and the sum from $t={(1-\delta )}^{\mathrm{-1}}$ to ∞ are $o(1-\delta )$ (see the proof of Lemma 18).

Discussion

The low-discounting/low-monitoring double limit

This paper focuses on the rate at which inefficiency vanishes as $\delta \to 1$ for a fixed monitoring structure. In contrast, in Sugaya and Wolitzky (2023) we showed that in the double limit where simultaneously $\delta \to 1$ and monitoring precision degrades, efficiency depends on a ratio of discounting and monitoring precision. This double limit arises, for example, in the “frequent action limit” considered by Abreu, Milgrom, and Pearce (1991), Fudenberg and Levine (2007), Sannikov and Skrzypacz (2010), and Sadzik and Stacchetti (2015), where signals are parameterized by an underlying continuous-time process, actions and signal observations occur simultaneously every Δ units of time, and the analysis concerns the $\mathrm{\Delta }\to 0$ limit.

The results of the current paper extend to the low-discounting/low-monitoring double limit. To see this, maintain the assumption that the monitoring structure is sub-Gaussian with variance proxy K, but now view K as a variable that varies simultaneously with the discount factor. Since K proxies the variance of the likelihood ratio difference, a lower value for K corresponds to less precise monitoring, so the low-discounting/low-monitoring double limit arises when $K\to 0$ and $\delta \to 1$ simultaneously. For example, in the standard frequent action limit, discounting and monitoring vanish at the same rate, so $(1-\delta )/K$ remains constant as $K\to 0$ and $\delta \to 1$.

From this more general perspective, it can be shown (by nearly the same proof) that Theorem 1 holds verbatim with $(1-\delta )/K$ in place of $1-\delta$. Conversely, Theorem 2 also holds with $(1-\delta )/K$ in place of $1-\delta$, under the condition that $\overline{x}$ in Assumption 2 can be taken to be of order $K^{-1/2}$. For example, this condition holds with finite signals with $p(y|a)$ bounded away from zero, or with Gaussian signals.²⁹

Summary and directions for future research

This paper has used a rate-of-convergence approach to analyze the gains from nonrecursive equilibria in repeated agency problems and games with patient players. The main result is that these gains are “small”: (i) in finite-action games, nonrecursive equilibria reduce inefficiency by at most a log factor; (ii) in smooth games, nonrecursive equilibria reduce inefficiency by at most a constant factor; and (iii) in smooth principal-agent problems, non-recursive equilibria do not reduce first-order inefficiency at all. The key force underlying these results is that, while pooling information across periods improves monitoring precision, it also entails larger rewards and punishments, which reduce the scope for providing incentives by transferring future surplus between the players rather than destroying it.

A basic lesson of our analysis is that the value of withholding feedback in dynamic agency is very different in a one-off production process that unfolds gradually over time (as in Holmström and Milgrom (1987)) as compared to a repeated interaction. Since continuation payoff transfers are impossible in one-shot interactions, the monitoring benefit of withholding feedback dominates, so withholding feedback can be very valuable. But in repeated interactions, this benefit is offset by the cost of using larger rewards and punishments, which limit continuation payoff transfers.

We mention some possible extensions of our results. First, as discussed in Section 5, characterizing rates of convergence toward extreme points with curvature of order $\beta >2$ is a challenging open question involving nonlocal properties of the feasible payoff set. Second, as discussed in Section 6, it may be possible to generalize Theorem 3 from smooth agency problems to smooth games. Third, it would be interesting to relax the assumption that the likelihood ratio difference is sub-Gaussian. This could result in a faster rate of convergence, because rare but highly informative signals would become more common, and such signals become more useful as δ increases. Fourth, it would be interesting to extend our results to irreducible stochastic games with observable states, where one can investigate the rate of convergence to extreme points of the limit feasible payoff set. Fifth, one could consider stochastic games where the state is only observed by the principal. In this setting, withholding information has costs as well as benefits, which are interesting to compare. Finally, the rate of convergence to efficiency as discounting vanishes may be a useful lens for analyzing a range of other questions about long-run economic relationships, beyond the value of withholding performance feedback.

A

Appendix

Proof of Theorem 1

We first bound a player's deviation gain at any $\alpha \in \mathrm{\Delta }(A)$ that attains payoffs close to v.

$4$

Lemma

There exist $\epsilon >0$ and $\gamma >0$ such that, for all $\alpha \in \mathrm{\Delta }(A)$ satisfying $\lambda \cdot (v-u(\alpha ))<\epsilon$, there exist a player i and a manipulation $s_i$ such that $g_i(\alpha ,s_i)>\gamma$.

$$

Proof

Since $v\in \exp (F)\backslash V^{\mathrm{CE}}$, for all $\alpha \in \mathrm{\Delta }(A)$ such that $v=u(\alpha )$, there exist i and $s_i$ such that $g_i(s_i,\alpha )>0$. Let $\gamma =\frac{{\displaystyle 1}}{{\displaystyle 2}}\underset{\alpha \in \mathrm{\Delta }(A):v=u(\alpha )}{\inf }\underset{i,s_i}{\sup }{g}_i(s_i,\alpha ).$ Note that $\gamma >0$. To see this, note that $g_i(\mathrm{Id},\alpha )=0$ for all i, α, so $\gamma \ge 0$, and suppose toward a contradiction that there exists a sequence ${\alpha }^n$ such that $v=u({\alpha }^n)$ for all n and ${\sup }_{i,s_i}{g}_i(s_i,{\alpha }^n)\to 0$. Since $\mathrm{\Delta }(A)$ is weak*-compact by Alaoglu's theorem, taking a subsequence if necessary, ${\alpha }^n\to \alpha \in \mathrm{\Delta }(A)$. Moreover, since each $u_i$ is continuous, $u(\alpha )=v$; and since each $A_i$ is compact, by the maximum theorem, ${\sup }_{s_i}{g}_i(s_i,\alpha )={\lim }_n{\sup }_{s_i}{g}_i(s_i,{\alpha }^n)=0$ for all i, contradicting $v\notin V^{\mathrm{CE}}$.

Now suppose that for all $\epsilon >0$ there exists ${\alpha }^{\epsilon }\in \mathrm{\Delta }(A)$ satisfying $\lambda \cdot (v-u({\alpha }^{\epsilon }))<\epsilon$ and $g_i(s_i,{\alpha }^{\epsilon })<\gamma$ for all i, $s_i$. Taking a subsequence if necessary, ${\alpha }^{\epsilon }\to \alpha \in \mathrm{\Delta }(A)$. Moreover, we have $u(\alpha )={\lim }_{\epsilon }u({\alpha }^{\epsilon })=v$ (since $u({\alpha }^{\epsilon })\in F$ and $v\in \exp (F)$), and ${\sup }_{s_i}{g}_i(s_i,\alpha )={\lim }_{\epsilon }{\sup }_{s_i}{g}_i(s_i,{\alpha }^{\epsilon })\le \gamma$ for all i (by the maximum theorem), so ${\sup }_{i,s_i}{g}_i(s_i,\alpha )\le \gamma$, contradicting the definition of γ. □

Fix such ε and γ. Next, for any outcome μ and period T, define the occupation measure over the first T periods by ${\alpha }^{\mu ,T}=((1-\delta )/(1-{\delta }^T)){\sum }_{t=1}^T{\delta }^{t-1}{\alpha }_t^{\mu }$, and define $T(\delta )=\lceil (\log 2)/(-\log \delta )\rceil$. We first bound $\lambda \cdot (v-u({\alpha }^{\mu }))$ for any μ where all players' deviation gains over the first $T(\delta )$ periods are small.

$5$

Lemma

For any outcome μ where $g_i(s_i,{\alpha }^{\mu ,T(\delta )})\le \gamma$ for all players i and manipulations $s_i$, we have $\lambda \cdot (v-u({\alpha }^{\mu }))\ge \epsilon /2$.

$$

Proof

Since ${\delta }^T\le 1/2$ by construction, we have $\begin{array}{rl}\lambda \cdot (v-u({\alpha }^{\mu }\left)\right)& =(1-\delta )\sum _{t=1}^{\mathrm{\infty }}{\delta }^{t-1}\lambda \cdot (v-u({\alpha }_t^{\mu }\left)\right)\\ & \ge (1-\delta )\sum _{t=1}^T{\delta }^{t-1}\lambda \cdot (v-u({\alpha }_t^{\mu }\left)\right)\\ & =(1-{\delta }^T)\lambda \cdot (v-u({\alpha }^T\left)\right)\ge \frac{{\displaystyle \lambda }}{{\displaystyle 2}}\cdot (v-u({\alpha }^T\left)\right).\end{array}$ By construction of $(\epsilon ,\gamma )$, if $\lambda \cdot (v-u({\alpha }^T))<\epsilon$ then ${\sup }_{i,s_i}{g}_i({\alpha }^T,s_i)>\gamma$. Hence, ${\sup }_{i,s_i}{g}_i({\alpha }^T,s_i)\le \gamma$ implies $\lambda \cdot (v-u({\alpha }^T))\ge \epsilon$, as desired. □

We next establish the incentive constraint, (2).

$6$

Lemma

For any equilibrium outcome $\mu \in {\mathcal{M}}^B(\delta )$, player i, manipulation $s_i$, and period t, we have $g_i(s_i,{\alpha }_t^{\mu })\le {\mathbb{E}}^{\mu }[{\ell }_{i,t}(s_i,h){\sum }_{t^{\prime }=t+1}^{\mathrm{\infty }}{\delta }^{t^{\prime }-t}{û}_{t^{\prime }}(h)]$.

$$

Proof

For any sequence of action profiles ${(a_t)}_{t=1}^{\mathrm{\infty }}$ and any period t, let $w_t(h)={\sum }_{t^{\prime }=t}^{\mathrm{\infty }}{\delta }^{t^{\prime }-t}{u}_i(a_{t^{\prime }})$. Since μ is an equilibrium outcome, for every $t\in \mathbb{N}$ we have $g_i({\alpha }_t^{\mu },s_i)\le {\int }_{h^t,a_t,y_t}(p(y_t|a_t)-p(y_t|s_i(a_{i,t}),a_{-i,t}\left)\right)\delta \mathbb{E}[w_{t+1}(h)|h^t,a_t,y_t\left]d\mu \right(h^t,a_t)d{y}_t.$ This holds because, if she follows her recommendation in every period $t^{\prime }\ne t$ while manipulating according to $s_i$ in period t, player i obtains an expected continuation payoff of ${\int }_{h^t,a_t,y_t}p(y_t|s_i(a_{i,t}),a_{-i,t})\mathbb{E}[w_{t+1}(h)|h^t,a_t,y_t]d\mu (h^t,a_t)d{y}_t$ in period $t+1$, and this deviation must be unprofitable. The lemma follows as $\begin{array}{rl}& {\int }_{h^t,a_t,y_t}(p(y_t|a_t)-p(y_t|s_i(a_{i,t}),a_{-i,t}\left)\right)\delta \mathbb{E}[w_{t+1}(h)|h^t,a_t,y_t\left]d\mu \right(h^t,a_t)d{y}_t\\ & ={\int }_{h^t,a_t,y_t}p(y_t|a_t){\ell }_{i,t}(s_i,h)\delta \mathbb{E}[w_{t+1}(h)|h^t,a_t,y_t]d\mu (h^t,a_t)d{y}_t\\ & ={\int }_h{\ell }_{i,t}(s_i,h)\sum _{t^{\prime }=t+1}^{\mathrm{\infty }}{\delta }^{t^{\prime }-t}{û}_{t^{\prime }}(h)d\mu (h),\end{array}$ where the last line follows by iterated expectation. □

We now come to our key lemma, which bounds (4)—and hence $\lambda \cdot (v-u({\alpha }^{\mu }))$—for any μ where some player's deviation gain over the first $T(\delta )$ periods is large. $7$

Lemma

There exists $\overline{c}>0$ such that, for any outcome μ, player i, and manipulation $s_i$, and discount factor $\delta <1$ satisfying $g_i(s_i,{\alpha }^{\mu ,T(\delta )})>\gamma$, we have $\begin{array}{cc}\hfill & \underset{{({\xi }_t)}_t\ge 0}{\sup }\underset{{({û}_t(h))}_{t,h}\in [-ū,ū]}{\inf }(1-\delta )\sum _{t=1}^{\mathrm{\infty }}{\delta }^{t-1}{\mathbb{E}}^{\mu }\left[\eta \right|{û}_t(h){|}^{\beta }+{\xi }_t(g_t^{\mu }-{\ell }_t(h)\sum _{t^{\prime }=t+1}^{\mathrm{\infty }}{\delta }^{t^{\prime }-t}{û}_{t^{\prime }}(h))]\hfill \\ \hfill & \ge \overline{c}\zeta (\delta ).\hfill \end{array}$

Together, Lemmas 5, 6, and 7 imply that $\lambda \cdot (v-u({\alpha }^{\mu }))\ge \max \{\epsilon /2,\overline{c}\zeta (\delta )\}\ge \max \{\epsilon /2,\overline{c}\}\zeta (\delta )$ for all $\delta <1$ and $\mu \in {\mathcal{M}}^B(\delta )$. Theorem 1 therefore holds with $c=\min \{\epsilon /2,\overline{c}\}$.

It thus remains to prove Lemma 7. To this end, let ${\xi }_t=\xi \ge 0$ if $t\le T(\delta )$, and ${\xi }_t=0$ otherwise. Letting $T=T(\delta )$ to ease notation, we then have $\begin{array}{rl}(1-\delta )\sum _{t=1}^{\mathrm{\infty }}{\delta }^{t-1}{\xi }_t{\ell }_t(h)\sum _{t^{\prime }=t+1}^{\mathrm{\infty }}{\delta }^{t^{\prime }-t}{û}_{t^{\prime }}(h)& =(1-\delta )\xi \sum _{t=1}^T{\ell }_t(h)\sum _{t^{\prime }=t+1}^{\mathrm{\infty }}{\delta }^{t^{\prime }-1}{û}_{t^{\prime }}(h)\\ & =(1-\delta )\xi \sum _{t=2}^{\mathrm{\infty }}{\delta }^{t-1}{\mathcal{L}}_{\min \{t-1,T\}}{û}_t(h),\text{and}\\ (1-\delta )\sum _{t=1}^{\mathrm{\infty }}{\delta }^{t-1}{\xi }_t{g}_t^{\mu }& =\xi (1-\delta )\sum _{t=1}^T{\delta }^{t-1}{g}_t^{\mu }=\xi (1-{\delta }^T)g_i(s_i,{\alpha }^{\mu ,T})\ge \frac{{\displaystyle \xi \gamma }}{{\displaystyle 2}}.\end{array}$ In total, we see that (4) is no less than 16$\underset{\xi \ge 0}{\sup }(\frac{{\displaystyle \xi \gamma }}{{\displaystyle 2}}+\underset{{({û}_t(h))}_{t\ge 2,h}\in [-ū,ū]}{\inf }(1-\delta )\sum _{t=2}^{\mathrm{\infty }}{\delta }^{t-1}{\mathbb{E}}^{\mu }[\eta |{û}_t(h){|}^{\beta }-\xi {\mathcal{L}}_{\min \{t-1,T\}}{û}_t(h)]).$ The following lemma thus establishes Lemma 7.

$8$

Lemma

For each $\beta \ge 1$, there exists $\overline{c}>0$ such that, for any μ and δ, the value of (16) is no less than $\overline{c}\zeta (\delta )$.

In turn, Lemma 8 relies on the following large deviations bound for martingales.

$9$

Lemma

Let ${(X_t)}_{t\ge 1}$ be a sequence of martingale increments adapted to a filtration ${(H_t)}_{t\ge 0}$, so that $\mathbb{E}[X_t|H_{t-1}]=0$, and let ${({\omega }_t)}_{t\ge 1}$ be a stochastic process adapted to the same filtration satisfying $\mathbb{E}[\exp (\theta X_t)|H_{t-1}]\le \exp ({\theta }^2{\omega }_t/2)$ for all $t\ge 1$ and $\theta \in \mathbb{R}$. Let $S_T={\sum }_{t=1}^T{X}_t$ and $W_T={\sum }_{t=1}^T{\omega }_t$. For all $T\ge 1$, we have $\mathbb{E}[\exp (\theta S_T)]\le \exp ({\theta }^2{W}_T/2)$, and hence (i) $\mathrm{Pr}(|S_T|\ge x)\le 2\exp (-x^2/(2W_T))$ for all $x\ge 0$, and (ii) $\mathbb{E}[|S_T{|}^{\phi }]\le 2{(\phi W_T/e)}^{\phi /2}$ for all $\phi \ge 0$.

$$

Proof

By iterated expectation, $\mathbb{E}[\exp (\theta S_T)]=\mathbb{E}[\exp (\theta S_{T-1})\mathbb{E}[\exp (\theta X_T)|H_{T-1}]]\le \mathbb{E}[\exp (\theta S_{T-1})\left]\exp \right({\theta }^2{\omega }_T/2).$ Recursively applying the same argument gives $\mathbb{E}[\exp (\theta S_T)]\le \exp ({\theta }^2{W}_T/2)$. Applying the Chernoff bound then gives (i) and (ii); see, e.g., Lemmas 1.3 and 1.4 of Buldygin and Kozachenko (2000). □

$8$

Proof of Lemma

We consider separately the cases where $\beta =1$ and $\beta >1$.

Case 1: When $\beta =1$, the minimand in (16) is linear in ${û}_t(h)$. Minimizing over ${û}_t(h)\in [-ū,ū]$, we see that (16) equals 17$\underset{\xi \ge 0}{\sup }(\frac{{\displaystyle \xi \gamma }}{{\displaystyle 2}}+(1-\delta )\sum _{t=2}^{\mathrm{\infty }}{\delta }^{t-1}{\int }_{h:\xi |{\mathrm{\Sigma }}_{t^{\prime }=1}^{\min \{t-1,T\}}{\ell }_{t^{\prime }}(h)|\ge \eta }(\eta -\xi |{\mathcal{L}}_{\min \{t-1,T\}}|)ūd\mu (h)).$ Note that $\mathbb{E}[{\ell }_t|{(a_{t^{\prime }},y_{t^{\prime }})}_{t^{\prime }=1}^{t-1},a_t]=0$ and $\mathbb{E}[\exp (\theta {\ell }_t)|{(a_{t^{\prime }},y_{t^{\prime }})}_{t^{\prime }=1}^{t-1},a_t]\le \exp ({\theta }^{2}K/2)$ for all t, h, and θ, by (1). Hence, by Lemma 9, ${\mathcal{L}}_{\min \{t-1,T\}}$ is sub-Gaussian with variance proxy $K\min \{t-1,T\}$, and thus satisfies $\mathrm{Pr}(\xi |{\mathcal{L}}_{\min \{t-1,T\}}|\ge x)\le 2\left(\exp \right(-\frac{{\displaystyle x^2}}{{\displaystyle 2{\xi }^{2}K\min \{t-1,T\}}}\left)\right).$ We thus have $\begin{array}{rl}& {\int }_{h:\xi {\mathcal{L}}_{\min \{t-1,T\}}\ge \eta }(\eta -\xi |{\mathcal{L}}_{\min \{t-1,T\}}|)d\mu (h)\\ & ={\mathrm{Pr}}^{\mu }(\xi |{\mathcal{L}}_{\min \{t-1,T\}}|\ge \eta )\eta -{\mathbb{E}}^{\mu }\left[\mathbf{1}\right\{\xi |{\mathcal{L}}_{\min \{t-1,T\}}|\ge \eta \}|\xi {\mathcal{L}}_{\min \{t-1,T\}}|]\\ & =-{\int }_{x\ge \eta }{\mathrm{Pr}}^{\mu }(\xi |{\mathcal{L}}_{\min \{t-1,T\}}|\ge x)dx\ge -2{\int }_{x\ge \eta }\exp (-\frac{{\displaystyle x^2}}{{\displaystyle 2K{\xi }^2\min \{t-1,T\}}})dx,\end{array}$ where the second equality is by integration by parts. Now note that 18$\begin{array}{rl}& {\int }_{x\ge \eta }\exp (-\frac{{\displaystyle x^2}}{{\displaystyle 2K{\xi }^2\min \{t-1,T\}}})dx\\ & =\sqrt{2{\xi }^{2}K}\sqrt{\min \{t-1,T\}}{\int }_{y\ge \frac{\eta }{\sqrt{2{\xi }^{2}K}\sqrt{\min \{t-1,T\}}}}\exp (-y^2)dy\\ & \le \frac{{\displaystyle 2{\xi }^{2}K}}{{\displaystyle \eta }}\min \{t-1,T\}\exp (-\frac{{\displaystyle {\eta }^2}}{{\displaystyle 2{\xi }^{2}K\min \{t-1,T\}}})\\ & \le \frac{{\displaystyle 2{\xi }^{2}KT}}{{\displaystyle \eta }}\exp (-\frac{{\displaystyle {\eta }^2}}{{\displaystyle 2{\xi }^{2}KT}}),\end{array}$ where the first inequality uses the Mills ratio inequality $\varphi (-x)/\mathrm{\Phi }(-x)\ge x$ for $x\ge 0$. Hence, (17) is no less than $\begin{array}{cc}\hfill & \underset{\xi \ge 0}{\sup }(\frac{{\displaystyle \xi \gamma }}{{\displaystyle 2}}-\frac{{\displaystyle 4ū{\xi }^{2}K}}{{\displaystyle \eta }}(1-\delta )\sum _{t=2}^{\mathrm{\infty }}{\delta }^{t-1}T\exp (-\frac{{\displaystyle {\eta }^2}}{{\displaystyle 2{\xi }^{2}KT}}\left)\right)\hfill \\ \hfill & \ge \underset{\xi \ge 0}{\sup }\xi (\frac{{\displaystyle \gamma }}{{\displaystyle 2}}-\frac{{\displaystyle 4ū\xi KT}}{{\displaystyle \eta }}\exp (-\frac{{\displaystyle {\eta }^2}}{{\displaystyle 2{\xi }^{2}KT}}\left)\right).\hfill \end{array}$ Finally, letting ${\xi }^{\ast }=\eta {\left(KT\max \right\{\log \left(2^{8}KT{ū}^2{\gamma }^{\mathrm{-2}}\right),1\left\}\right)}^{-1/2},$ we have $\begin{array}{rl}& \underset{\xi \ge 0}{\sup }\xi (\frac{{\displaystyle \gamma }}{{\displaystyle 2}}-\frac{{\displaystyle 4ū\xi KT}}{{\displaystyle \eta }}\exp (-\frac{{\displaystyle {\eta }^2}}{{\displaystyle 2{\xi }^{2}KT}}\left)\right)\\ & \ge {\xi }^{\ast }(\frac{{\displaystyle \gamma }}{{\displaystyle 2}}-4ū\sqrt{KT}{\left(\max \right\{\log \left(2^{8}KT{ū}^2{\gamma }^{\mathrm{-2}}\right),1\left\}\right)}^{-1/2}\\ & \times \exp (-\frac{{\displaystyle 1}}{{\displaystyle 2}}\max \{\log \left(2^{8}KT{ū}^2{\gamma }^{\mathrm{-2}}\right),1\left\}\right))\\ & \ge {\xi }^{\ast }(\frac{{\displaystyle \gamma }}{{\displaystyle 2}}-4ū\sqrt{KT}\exp (-\frac{{\displaystyle 1}}{{\displaystyle 2}}\log \left(2^{8}KT{ū}^2{\gamma }^{\mathrm{-2}}\right)\left)\right)\\ & =\frac{{\displaystyle {\xi }^{\ast }\gamma }}{{\displaystyle 4}}=\frac{{\displaystyle \eta \gamma }}{{\displaystyle 4}}{\left(KT\max \right\{\log \left(2^{8}KT{ū}^2{\gamma }^{\mathrm{-2}}\right),1\left\}\right)}^{-1/2}\\ & \ge \frac{{\displaystyle \eta \gamma }}{{\displaystyle 4}}{((4K\log 2)\max \{\log (2^9(\log 2)K{ū}^2{\gamma }^{\mathrm{-2}}),1\left\}\right)}^{-1/2}\zeta (\delta ),\end{array}$ where the last inequality follows because $T\le \lceil (\log 2)/(1-\delta )\rceil \le 2(\log 2)/(1-\delta )$ and $\max \{-\log xy,1\}\le 2\max \{-\log x,1\}\max \{-\log y,1\}$ for all $x,y\in \mathbb{R}$. This is a constant multiple of $\zeta (\delta )$, as desired.

Case 2: When $\beta >1$, the minimand in (16) is convex in ${û}_t(h)$. Relaxing the constraint ${û}_t(h)\in [-ū,ū]$ and minimizing over ${û}_t(h)\in \mathbb{R}$ gives ${û}_t(h)={\left(\frac{{\displaystyle \xi }}{{\displaystyle \eta \beta }}\right)}^{\frac{1}{\beta -1}}\mathrm{sign}({\mathcal{L}}_{\min \{t-1,T\}})|{\mathcal{L}}_{\min \{t-1,T\}}{|}^{\frac{1}{\beta -1}}\text{for all }t\ge 2.$ Hence, substituting for ${û}_t(h)$, (16) is no less than 19$\underset{\xi \ge 0}{\sup }(\frac{{\displaystyle \xi \gamma }}{{\displaystyle 2}}-{\xi }^{\frac{\beta }{\beta -1}}{\left(\frac{{\displaystyle 1}}{{\displaystyle \eta }}\frac{{\displaystyle {\beta }^{\beta -1}-1}}{{\displaystyle {\beta }^{\beta }}}\right)}^{\frac{1}{\beta -1}}(1-\delta )\sum _{t=2}^{\mathrm{\infty }}{\delta }^{t-1}\mathbb{E}[|{\mathcal{L}}_{\min \{t-1,T\}}{|}^{\frac{\beta }{\beta -1}}]).$ By Lemma 9, ${\mathcal{L}}_{\min \{t-1,T\}}$ is sub-Gaussian with variance proxy $K\min \{t-1,T\}\le K(t-1)$, and thus satisfies $\begin{array}{rl}{\mathbb{E}}^{\mu }[|{\mathcal{L}}_{\min \{t-1,T\}}{|}^{\frac{\beta }{\beta -1}}]& \le 2{\left(\frac{{\displaystyle \beta }}{{\displaystyle e(\beta -1)}}\right)}^{\frac{\beta }{2(\beta -1)}}{(K(t-1))}^{\frac{\beta }{2(\beta -1)}}\\ & \le 2{\left(\frac{{\displaystyle \beta }}{{\displaystyle e(\beta -1)}}\right)}^{\frac{\beta }{2(\beta -1)}}{K}^{\frac{\beta }{2(\beta -1)}}{(t-1)}^{\max \{\frac{\beta }{2(\beta -1)},1\}}.\end{array}$ Next, for any $\vartheta \ge 1$, we let $k(\vartheta )\ge 1$ satisfy 20$\sum _{t=1}^{\mathrm{\infty }}{\delta }^t{t}^{\vartheta }\le \frac{{\displaystyle k(\vartheta )}}{{\displaystyle {(1-\delta )}^{\vartheta +1}}}\text{for all }\delta .$ (The existence of such $k(\vartheta )$ follows from the standard fact that ${\sum }_{t=1}^{\mathrm{\infty }}{\delta }^t{t}^{\vartheta }=\mathrm{\Gamma }(\vartheta +1){(1-\delta )}^{-(\vartheta +1)}+O({(1-\delta )}^{-\vartheta })$; see, e.g., Wood (1992, equation (6.4)).) With this definition, we have $\begin{array}{rl}& (1-\delta )\sum _{t=2}^{\mathrm{\infty }}{\delta }^{t-1}\mathbb{E}[|{\mathcal{L}}_{\min \{t-1,T\}}{|}^{\frac{\beta }{\beta -1}}]\\ & \le 2{\left(\frac{{\displaystyle \beta }}{{\displaystyle e(\beta -1)}}\right)}^{\frac{\beta }{2(\beta -1)}}{K}^{\frac{\beta }{2(\beta -1)}}k\left(\max \right\{\frac{{\displaystyle \beta }}{{\displaystyle 2(\beta -1)}},1\left\}\right){(1-\delta )}^{-\max \{\frac{\beta }{2(\beta -1)},1\}}\\ & \le 2{\left(\frac{{\displaystyle \beta }}{{\displaystyle e(\beta -1)}}\right)}^{\frac{\beta }{2(\beta -1)}}{K}^{\frac{\beta }{2(\beta -1)}}k\left(\max \right\{\frac{{\displaystyle \beta }}{{\displaystyle 2(\beta -1)}},1\left\}\right){(1-\delta )}^{-\max \{\frac{\beta }{2(\beta -1)},1\}}.\end{array}$ Thus, (19) is no less than $\begin{array}{cc}\hfill & \underset{\xi \ge 0}{\sup }\xi (\frac{{\displaystyle \gamma }}{{\displaystyle 2}}-2{\left(\frac{{\displaystyle \beta }}{{\displaystyle e(\beta -1)}}\right)}^{\frac{\beta }{2(\beta -1)}}{\left(\frac{{\displaystyle {\beta }^{\beta -1}-1}}{{\displaystyle \eta {\beta }^{\beta }}}\right)}^{\frac{1}{\beta -1}}\hfill \\ \hfill & \times K^{\frac{\beta }{2(\beta -1)}}k\left(\max \right\{\frac{{\displaystyle \beta }}{{\displaystyle 2(\beta -1)}},1\left\}\right){\left(\frac{{\displaystyle \xi }}{{\displaystyle {(1-\delta )}^{\max \{\beta /2,\beta -1\}}}}\right)}^{\frac{1}{\beta -1}}).\hfill \end{array}$ Since the coefficient of ${(\xi /{(1-\delta )}^{\max \{\beta /2,(\beta -1)\}})}^{\frac{1}{\beta -1}}$ is independent of δ, there exists $ĉ>0$ such that if $\xi =4ĉ{(1-\delta )}^{\max \{\beta /2,\beta -1\}}$ then the resulting value is no less than $\xi \gamma /4$, which is again a constant multiple of $\zeta (\delta )$. □

B

Appendix

Proof of Proposition 1

Consider a review strategy where the game is divided into blocks of T consecutive periods. Let $T=\lfloor \rho /(1-\delta )\rfloor$, where $\rho >0$ is a small number to be determined: note that $\rho \approx 1-{\delta }^T$ when $\delta \approx 1$. In the first block, the players are prescribed $(C,L)$ in every period. At the end of the first block—as well any subsequent block where $(C,L)$ is prescribed—the mediator records the summary statistic $E=\mathbf{1}\{\frac{{\displaystyle 1}}{{\displaystyle \sqrt{T}}}\sum _{t=1}^T(2y_{1,t}-1)\le -\sqrt{-\log (1-\delta )}\}.$ (Here, periods are numbered from the start of the block and $y_{1,t}\in \{0,1\}$ is the signal of player 1's action in the $t^{\text{th}}$ period of the block.³⁰) If $E=0$, the players “pass the review” and $(C,L)$ is prescribed in the next block. If $E=1$, then with some probability $q\in [0,1]$ (which also remains to be determined), the players fail the review and $(D,R)$ is prescribed forever. With the complementary probability $1-q$, the players pass the review anyway and $(C,L)$ is prescribed in the next block.

We show that there exists $c>0$ and $\overline{\delta }<1$ such that, for any $\delta >\overline{\delta }$, the parameters ρ and q can be chosen so that this strategy profile is an equilibrium that yields payoff $v>2-c{((1-\delta )/(-\log (1-\delta )))}^{1/2}$ for each player.³¹

Let p be the probability that $E=1$ when player 1 takes C throughout a block; let $p_1$ be the probability that $E=1$ when player 1 takes D once and takes C $T-1$ times; and let $p_T$ be the probability that $E=1$ when player 1 takes D throughout. Observe that v is given by PK$v=(1-{\delta }^T)2+{\delta }^T(1-pq)v⟺v=2-\frac{{\displaystyle {\delta }^{T}pqv}}{{\displaystyle 1-{\delta }^T}}.$ At the same time, the incentive conditions that player 1 prefers to take C throughout a block where $(C,L)$ is prescribed, rather than taking D in period 1 only, or always taking D, are IC₁$1-\delta \le {\delta }^T(p_1-p)qv\text{and}$ IC_T$1-{\delta }^T\le {\delta }^T(p_T-p)qv.$ Conditions (IC₁) and (IC_T) are obviously necessary for the review strategy to be an equilibrium; moreover, as shown by Matsushima (2004, p. 846), they are also sufficient.³² It thus suffices to find $c>0$, $\overline{\delta }<1$, ρ, and q such that, for any $\delta >\overline{\delta }$, (PK), (IC₁), and (IC_T) hold, and $v>2-c{((1-\delta )/(-\log (1-\delta )))}^{1/2}$.

Define 21$v^{\ast }=2-\frac{{\displaystyle 1-\delta }}{{\displaystyle 1-{\delta }^T}}\frac{{\displaystyle p}}{{\displaystyle p_1-p}}\text{and}q={(\frac{{\displaystyle 2{\delta }^T}}{{\displaystyle 1-\delta }}(p_1-p)-\frac{{\displaystyle {\delta }^T}}{{\displaystyle 1-{\delta }^T}}p)}^{\mathrm{-1}}.$ With these definitions, (PK) and (IC₁) hold with equality, with $v=v^{\ast }$. We show that 22$\underset{\delta \to 1}{\lim }\frac{{\displaystyle \frac{{\displaystyle 1-\delta }}{{\displaystyle 1-{\delta }^T}}\frac{{\displaystyle p}}{{\displaystyle p_1-p}}}}{{\displaystyle \sqrt{\frac{{\displaystyle 1-\delta }}{{\displaystyle -\log (1-\delta )}}}}}<\frac{{\displaystyle 5\sqrt{\rho }{e}^{\rho }}}{{\displaystyle e^{\rho }-1}}\text{for all }\rho >0,$ 23$\underset{\rho \to 0}{\lim }\underset{\delta \to 1}{\lim }\frac{{\displaystyle 2{\delta }^T(p_1-p)}}{{\displaystyle 1-\delta }}-\frac{{\displaystyle {\delta }^{T}p}}{{\displaystyle 1-{\delta }^T}}>1,\text{and}$ 24$\underset{\rho \to 0}{\lim }\underset{\delta \to 1}{\lim }\frac{{\displaystyle {\delta }^{T}q(p_T-p)}}{{\displaystyle 1-{\delta }^T}}>1.$ Given these inequalities, the proof is completed by first taking $\rho >0$ and ${\overline{\delta }}_1>0$ such that the inequalities in (23) and (24) hold for ρ and all $\delta >{\overline{\delta }}_1$, then taking ${\overline{\delta }}_2>0$ such that the inequality in (22) holds for ρ for all $\delta >{\overline{\delta }}_2$, and finally taking $c=5\sqrt{\rho }{e}^{\rho }/(e^{\rho }-1)$ and $\overline{\delta }=\max \{{\overline{\delta }}_1,{\overline{\delta }}_2\}$.

We now establish (22)–(24). Let $k=\lfloor (\sqrt{T}/2)(\sqrt{T}-\sqrt{-\log (1-\delta )})\rfloor$. Note that 25$\begin{array}{rl}p& =\mathrm{Pr}(\sum _{t=2}^T{y}_{1,t}<k)+\frac{{\displaystyle 1}}{{\displaystyle 2}}\mathrm{Pr}(\sum _{t=2}^T{y}_{1,t}=k)<\mathrm{Pr}(\sum _{t=2}^T{y}_{1,t}\le k),\text{and}\\ p_1-p& =\frac{{\displaystyle 1}}{{\displaystyle 4}}\mathrm{Pr}(\sum _{t=2}^T{y}_{1,t}=k)\end{array}$ $=\frac{{\displaystyle (T-1)!}}{{\displaystyle k!(T-1-k)!}}{\left(\frac{{\displaystyle 1}}{{\displaystyle 2}}\right)}^{T+1}\ge \frac{{\displaystyle (T)!}}{{\displaystyle k!(T-k)!}}{\left(\frac{{\displaystyle 1}}{{\displaystyle 2}}\right)}^{T+2},$ where the last inequality holds because $k\le T/2$.

We first establish (22). Recall that the $y_{1,t}$ are independent Bernoulli random variables. As shown by Zhu, Li, and Hayashi (2022, Theorem 2.1), $\frac{{\displaystyle \mathrm{Pr}(\sum _{t=2}^T{y}_{1,t}\le k)}}{{\displaystyle \mathrm{Pr}(\sum _{t=2}^T{y}_{1,t}=k)}}\le k+1-\frac{{\displaystyle T}}{{\displaystyle 2}}+\sqrt{{(k-1-\frac{{\displaystyle T}}{{\displaystyle 2}})}^2+2k}.$ Since $\begin{array}{cc}\hfill \frac{{\displaystyle p}}{{\displaystyle p_1-p}}& <4\frac{{\displaystyle \mathrm{Pr}(\sum _{t=2}^T{y}_{1,t}\le k)}}{{\displaystyle \mathrm{Pr}(\sum _{t=2}^T{y}_{1,t}=k)}}\text{and}\hfill \\ \hfill \underset{\delta \to 1}{\lim }\frac{{\displaystyle \frac{{\displaystyle 1-\delta }}{{\displaystyle 1-{\delta }^T}}(k+1-\frac{{\displaystyle T}}{{\displaystyle 2}}+\sqrt{{(k-1-\frac{{\displaystyle T}}{{\displaystyle 2}})}^2+2k})}}{{\displaystyle \sqrt{\frac{{\displaystyle 1-\delta }}{{\displaystyle -\log (1-\delta )}}}}}& =\frac{{\displaystyle \sqrt{\rho }{e}^{\rho }}}{{\displaystyle e^{\rho }-1}},\hfill \end{array}$ where the second line follows by l'Hopital's rule, we have $\underset{\delta \to 1}{\lim }\frac{{\displaystyle \frac{{\displaystyle 1-\delta }}{{\displaystyle 1-{\delta }^T}}\frac{{\displaystyle p}}{{\displaystyle p_1-p}}}}{{\displaystyle \sqrt{\frac{{\displaystyle 1-\delta }}{{\displaystyle -\log (1-\delta )}}}}}\le \underset{\delta \to 1}{\lim }\frac{{\displaystyle \frac{{\displaystyle 1-\delta }}{{\displaystyle 1-{\delta }^T}}4\frac{{\displaystyle \mathrm{Pr}(\sum _{t=2}^T{y}_{1,t}\le k)}}{{\displaystyle \mathrm{Pr}(\sum _{t=2}^T{y}_{1,t}=k)}}}}{{\displaystyle \sqrt{\frac{{\displaystyle 1-\delta }}{{\displaystyle -\log (1-\delta )}}}}}=\frac{{\displaystyle 4\sqrt{\rho }{e}^{\rho }}}{{\displaystyle e^{\rho }-1}}<\frac{{\displaystyle 5\sqrt{\rho }{e}^{\rho }}}{{\displaystyle e^{\rho }-1}},$ which establishes (22).

We next establish (23). Applying Stirling's formula to (25), we have 26$p_1-p\ge \frac{{\displaystyle \sqrt{2\pi (T-1)}}}{{\displaystyle 4e^2\sqrt{k(T-1-k)}}}{\left(\frac{{\displaystyle T-1}}{{\displaystyle 2k}}\right)}^k{\left(\frac{{\displaystyle T-1}}{{\displaystyle 2(T-1-k)}}\right)}^{T-1-k}.$ Therefore, $\begin{array}{cc}\hfill \underset{\rho \to 0}{\lim }\underset{\delta \to 1}{\lim }\frac{{\displaystyle {\delta }^T(p_1-p)}}{{\displaystyle 1-\delta }}& \ge \underset{\rho \to 0}{\lim }\underset{\delta \to 1}{\lim }\frac{{\displaystyle {\delta }^T}}{{\displaystyle 1-\delta }}\frac{{\displaystyle \sqrt{2\pi (T-1)}}}{{\displaystyle 4e^2\sqrt{k(T-1-k)}}}{\left(\frac{{\displaystyle T-1}}{{\displaystyle 2k}}\right)}^k{\left(\frac{{\displaystyle T-1}}{{\displaystyle 2(T-1-k)}}\right)}^{T-1-k}\hfill \\ \hfill & =\mathrm{\infty }.\hfill \end{array}$ On the other hand, ${\lim }_{\rho \to 0}{\lim }_{\delta \to 1}\frac{{\delta }^{T}p}{1-{\delta }^T}\le {\lim }_{\delta \to 1}\frac{{\delta }^T}{1-{\delta }^T}={\lim }_{\rho \to 0}\frac{e^{-\rho }}{1-e^{-\rho }}=\mathrm{\infty }$, which establishes (23).

Finally, we establish (24). We will show that ${\lim }_{\delta \to 1}p=0$ and ${\lim }_{\delta \to 1}{p}_T=1$. Hence, for sufficiently large δ, $p_T-p\ge 1/2$. This implies (24), as we have $\begin{array}{rl}& \underset{\rho \to 0}{\lim }\underset{\delta \to 1}{\lim }\frac{{\displaystyle {\delta }^{T}q(p_T-p)}}{{\displaystyle 1-{\delta }^T}}\\ & =\underset{\rho \to 0}{\lim }\underset{\delta \to 1}{\lim }\frac{{\displaystyle {\delta }^T}}{{\displaystyle 1-{\delta }^T}}\frac{{\displaystyle 1}}{{\displaystyle {\delta }^T}}\frac{{\displaystyle p_T-p}}{{\displaystyle 2\frac{{\displaystyle p_1-p}}{{\displaystyle 1-\delta }}-\frac{{\displaystyle p}}{{\displaystyle 1-{\delta }^T}}}}\text{by (21)}\\ & \ge \underset{\rho \to 0}{\lim }\underset{\delta \to 1}{\lim }\frac{{\displaystyle 1}}{{\displaystyle 1-{\delta }^T}}\frac{{\displaystyle \frac{{\displaystyle 1}}{{\displaystyle 2}}}}{{\displaystyle 2\frac{{\displaystyle p_1-p}}{{\displaystyle 1-\delta }}}}\\ & \ge \underset{\rho \to 0}{\lim }\underset{\delta \to 1}{\lim }\frac{{\displaystyle 1-\delta }}{{\displaystyle 1-{\delta }^T}}{\left(\frac{{\displaystyle \sqrt{e(T-1)}}}{{\displaystyle 2\pi \sqrt{k(T-1-k)}}}{\left(\frac{{\displaystyle T-1}}{{\displaystyle 2k}}\right)}^k{\left(\frac{{\displaystyle T-1}}{{\displaystyle 2(T-1-k)}}\right)}^{T-1-k}\right)}^{\mathrm{-1}}=\mathrm{\infty },\end{array}$ where the second inequality follows by applying Stirling's formula to (25).

It remains to show that ${\lim }_{\delta \to 1}p=0$ and ${\lim }_{\delta \to 1}{p}_T=1$. Note that the random variable $2y_{1,t}-1$ has zero mean and unit variance when player 1 takes C. Thus, by the Berry–Esseen theorem, there exists an absolute constant $C_0$ such that $\begin{array}{rl}p& ={\mathrm{Pr}}^{\text{player 1 takes }C}(\frac{{\displaystyle 1}}{{\displaystyle \sqrt{T}}}\sum _{t=1}^T(2y_{1,t}-1)\le -\sqrt{-\log (1-\delta )})\\ & \le \mathrm{\Phi }(-\sqrt{-\log (1-\delta )})+C_0\left(\frac{{\displaystyle {\mathbb{E}}^{\text{player 1 takes }C}[|2y_{1,t}-1{|}^3]}}{{\displaystyle \sqrt{T}}}\right)\stackrel{\delta \to 1}{⟶}0.\end{array}$ On the other hand, $(4y_{1,t}-1)/\sqrt{3}$ has zero mean and unit variance when player 1 takes D. Thus, again by Berry–Esseen, $\begin{array}{rl}{p}_T& ={\mathrm{Pr}}^{\text{player 1 takes }D}(\frac{{\displaystyle 1}}{{\displaystyle \sqrt{T}}}\sum _{t=1}^T(2y_{1,t}-1)\le -\sqrt{-\log (1-\delta )})\\ & ={\mathrm{Pr}}^{\text{player 1 takes }D}(\frac{{\displaystyle 1}}{{\displaystyle \sqrt{T}}}\sum _{t=1}^T\frac{{\displaystyle 4y_{1,t}-1}}{{\displaystyle \sqrt{3}}}\le \frac{{\displaystyle \sqrt{T}-2\sqrt{-\log (1-\delta )}}}{{\displaystyle \sqrt{3}}})\\ & \ge \mathrm{\Phi }\left(\frac{{\displaystyle \sqrt{T}-2\sqrt{-\log (1-\delta )}}}{{\displaystyle \sqrt{3}}}\right)+C_0\left(\frac{{\displaystyle {\mathbb{E}}^{\text{player 1 takes }D}\left[\right|(4y_{1,t}-1)/\sqrt{3}{|}^3]}}{{\displaystyle \sqrt{T}}}\right)\stackrel{\delta \to 1}{⟶}1,\end{array}$ completing the proof.

C

Appendix

Proof of Proposition 2

To define $\overline{x}$, we first observe that for each pair of players $i\ne j$ and each action profile a, we can take ${(x_i^j(d;a))}_d$ such that (i) ${\sum }_d{x}_i^j(d;a)y^d$ has mean 0 and bounded Euclidean norm; (ii) rewards ${\sum }_d{x}_i^j(d;a)y^d$ induce player i to take $a_i$ when her opponents take $a_{-i}$; and (iii) $\mathbb{E}[{\sum }_d{x}_i^j(d;a)y^d|a]$ is independent of player j's action.

$10$

Lemma

There exists $\hat{x}$ such that, for each pair of players $i\ne j$ and action profile $a\in A$, there exist ${(x_i^j(d;a))}_d$ such that $\mathbb{E}[{\sum }_d{x}_i^j(d;a)y^d|a]=0$, $\frac{d}{d{a}_i}\mathbb{E}[{\sum }_d{x}_i^j(d;a)y^d|a]=1$, $\frac{d}{d{a}_j}\mathbb{E}[{\sum }_d{x}_i^j(d;a)y^d|a]=0$, and $|{\sum }_d{x}_i^j(d;a)y^d|\le \hat{x}$ for all y.

$$

Proof

For each a and $(i,j)$, let $f^{ij}(a)$ be the value of the program $\begin{array}{rl}& \underset{b\in {\mathbb{R}}^D}{\inf }|b|\text{subject to}\\ \sum _d{b}_d\frac{{\displaystyle d}}{{\displaystyle d{a}_i}}\mu (a_i,a_{-i})& =1\text{, or equivalently }b{M}_i(a)=1,\\ \sum _d{b}_d\frac{{\displaystyle d}}{{\displaystyle d{a}_j}}\mu (a_i,a_{-i})& =0\text{, or equivalently }b{M}_j(a)=0.\end{array}$ (Here, b is a row vector while $M_i(a)$ and $M_j(a)$ are column vectors.)

Since $A\ni a$ is compact and N is finite, it suffices to prove that, for each $(i,j)$, (i) $f^{ij}(a)<\mathrm{\infty }$ for all a, and (ii) $f^{ij}(a)$ is upper semicontinuous.

We first prove (i). As in Lemma 1 of Sannikov (2007), pairwise identifiability implies that the columns of $[M^i(a);M^j(a)]$ are linearly independent, so there exists $L(a)$ such that $[M^i(a);M^j(a);L(a)]$ is a D-dimensional invertible matrix. For $Q(a)=[M_i(a);0;0]{[M^i(a);M^j(a);L(a)]}^{\mathrm{-1}},$ we have $Q(a)M^i(a)=M^i(a)$ and $Q(a)M^j(a)=0$. Moreover, since $M^i(a)$ is non-degenerate, there exists $\overline{b}$ such that $\overline{b}{M}^i(a)=1$. Since $b=\overline{b}Q(a)$ satisfies the constraints, we have $f^{ij}(a)<\mathrm{\infty }$.

We next prove (ii). Fix any a and ${\eta }_0$. There exists b such that $|b|\le f^{ij}(a)+\frac{{\eta }_0}{2}$ and b satisfies $b{M}_i(a)=1$ and $b{M}_j(a)=0$. Take $L(a)$ as in the proof of (i). Taking ${\eta }_1>0$ sufficiently small, we can guarantee that $[M^i(a^{\prime });M^j(a^{\prime });L(a)]$ is a D-dimensional invertible matrix for each $a^{\prime }$ with $|a-a^{\prime }|\le {\eta }_1$. Define a D-dimensional vector ${\mathrm{\Delta }}_{a^{\prime }}$ by ${\mathrm{\Delta }}_{a^{\prime }}=\left[b\right(M_i\left(a^{\prime }\right)-M_i(a)),b(M_j\left(a^{\prime }\right)-M_j(a)),0]{\left[M^i\right(a^{\prime });M^j(a^{\prime });L(a)]}^{\mathrm{-1}}.$ By definition, $\begin{array}{rl}(b+{\mathrm{\Delta }}_{a^{\prime }})M_i\left(a^{\prime }\right)& =b{M}_i\left(a^{\prime }\right)-b\left(M_i\right(a^{\prime })-M_i(a))=b{M}_i(a)=1,\\ (b+{\mathrm{\Delta }}_{a^{\prime }})M_j\left(a^{\prime }\right)& =b{M}_j\left(a^{\prime }\right)-b\left(M_j\right(a^{\prime })-M_j(a))=b{M}_j(a)=0.\end{array}$ Thus, $b-{\mathrm{\Delta }}_{a^{\prime }}$ satisfies the constraint for $a^{\prime }$, and hence $f^{ij}(a^{\prime })\le |b|+|{\mathrm{\Delta }}_{a^{\prime }}|$. Since $\lim {\sup }_{{\eta }_1\to 0}{\sup }_{a^{\prime }:|a-a^{\prime }|\le {\eta }_1}|{\mathrm{\Delta }}_{a^{\prime }}|=0$, for sufficiently small ${\eta }_1>0$, we have $f^{ij}(a^{\prime })\le |b|+|{\mathrm{\Delta }}_{a^{\prime }}|\le |b|+\frac{1}{2}{\eta }_0\le f^{ij}(a)+{\eta }_0$ for all $a^{\prime }$ with $|a-a^{\prime }|\le {\eta }_1$, establishing upper semicontinuity. □

Given Lemma 10, Assumption 2(ii) holds with $\overline{x}=ū\hat{x}$. To see why, for any i and a, let $\partial u_i=\frac{\partial }{\partial a_i^{\prime }}{u}_i(a_i^{\prime },a_{-i}){|}_{a_i^{\prime }=a_i}$ and $x_i(y)=-\partial u_i{\sum }_d{x}_i^j(d;a^{\lambda })y^d$. Then $\begin{array}{rl}\frac{{\displaystyle \partial }}{{\displaystyle \partial a_i^{\prime }}}\left(u_i\right(a_i^{\prime },a_{-i})+\mathbb{E}[x_i(y)|a_i^{\prime },a_{-i}]){|}_{a_i^{\prime }=a_i}& =0\text{and}\\ \frac{{\displaystyle \partial }}{{\displaystyle \partial a_j^{\prime }}}\mathbb{E}[c{x}_i(y)|a_j^{\prime },a_{-i}]{|}_{a_j^{\prime }=a_j}& =0\text{for all }j\ne i.\end{array}$ Since $u_i$ is concave in $a_i$, $\mathbb{E}[x_i(y)|a_i^{\prime },a_{-i}]$ is linear in $a_i^{\prime }$, and $\mathbb{E}[c{x}_i(y)|a_j^{\prime },a_{-j}]$ is linear in $a_j^{\prime }$, we have (6) and (7). Moreover, since $|\partial u_i|\le ū$, we have $|x_i(y)|\le ū\hat{x}$ for all i, y.

D

Appendix

Proof of Lemma 2

The proof is similar to (but simpler than) the proof of Lemma 6 of Sugaya and Wolitzky (2023). To show that B is self-generating, it suffices to show that the extreme points of any ball $B^{\prime }\subseteq B$ of radius $r/2$ are decomposable on $B^{\prime }$.

$11$

Lemma

$Sugaya\; and\; Wolitzky\; (2023,\; Lemma\; 10)$

Suppose that for any ball $B^{\prime }\subseteq B$ with radius $r/2$ and any direction $\lambda \in \mathrm{\Lambda }$, the point $\hat{v}={\mathrm{argmax}}_{v^{\prime }\in B^{\prime }}\lambda \cdot v^{\prime }$ is decomposable on $B^{\prime }$. Then B is self-generating.

We thus fix a ball $B^{\prime }\subseteq B$ of radius $r/2$ and a direction $\lambda \in \mathrm{\Lambda }$, and let $\hat{v}={\mathrm{argmax}}_{v^{\prime }\in B^{\prime }}\lambda \cdot v^{\prime }$. We construct $(\alpha ,w)$ that decompose $\hat{v}$ on $B^{\prime }$.

Since $k(\lambda ,\overline{x})\ge {\max }_{v^{\prime }\in B}\lambda \cdot v^{\prime }+\epsilon$ by hypothesis, there exist α and $x:Y\to {\mathbb{R}}^N$ satisfying (IC), (HS$\overline{x}$), and 27$\lambda \cdot (u(\alpha )+{\int }_{y}x(y)p(y|\alpha )dy)\ge \underset{v^{\prime }\in B}{\max }\lambda \cdot v^{\prime }+\epsilon /2\ge \underset{v^{\prime }\in B^{\prime }}{\max }\lambda \cdot v^{\prime }+\epsilon /2.$

To construct w, for each y, let $w(y)=\hat{v}+\frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}(\hat{v}-u(\alpha )+x(y)-{\int }_{y^{\prime }}p(y^{\prime }|\alpha )x\left(y^{\prime }\right)d{y}^{\prime }).$ We show that $(\alpha ,w)$ decomposes $\hat{v}$ on $B^{\prime }$ by verifying (PK), (IC), and (SG).

(PK): This holds by construction: we have ${\int }_{y}w(y)p(y|\alpha )dy=(1/\delta )(\hat{v}-(1-\delta )u(\alpha ))$, and hence $(1-\delta )u(\alpha )+\delta {\int }_{y}w(y)p(y|\alpha )dy=\hat{v}$.

(IC): Setting aside the constant terms in $w(y)$, we see that an action $a_i$ maximizes $(1-\delta )u_i(a_i,{\alpha }_{-i})+\delta {\int }_y{w}_i(y)p(y|a_i,{\alpha }_{-i})dy$ if and only if it maximizes $u_i(a_i,{\alpha }_{-i})+{\int }_y{x}_i(y)p(y|a_i,{\alpha }_{-i})dy$, which follows from (IC).

(SG): We start with a simple geometric observation.

$12$

Lemma

$Sugaya\; and\; Wolitzky\; (2023,\; Lemma\; 11)$

For each $w\in {\mathbb{R}}^N$, we have $w\in B^{\prime }$ if $\lambda \cdot (\hat{v}-w)\ge 0$ and 28$d(\hat{v},w)\le \sqrt{(r/2)\lambda \cdot (\hat{v}-w)}.$

We thus show that, for each y, $w(y)$ satisfies $\lambda \cdot (\hat{v}-w(y))\ge 0$ and (28). Note that $\hat{v}-w(y)=\frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}(u(\alpha )+{\int }_{y^{\prime }}x(y^{\prime }\left)p\right(y^{\prime }|\alpha )d{y}^{\prime }-\hat{v}-x(y)).$ By (HS$\overline{x}$) and (27), we have $\lambda \cdot (\hat{v}-w(y))\ge (\delta /(1-\delta ))\epsilon /2$ and, therefore, 29$\sqrt{(r/2)\lambda \cdot (\hat{v}-w(y))}\ge \frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}\sqrt{\frac{{\displaystyle \delta }}{{\displaystyle 1-\delta }}\frac{{\displaystyle \epsilon r}}{{\displaystyle 4}}}.$ Similarly, we have 30$\begin{array}{rl}d(\hat{v},w)& \le \frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}\left(d\right(\hat{v},u(\alpha ))+d({\int }_{y^{\prime }}x\left(y^{\prime }\right)p(y^{\prime }|\alpha )d{y}^{\prime },x(y)))\\ & \le \frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}(\underset{u,u^{\prime }\in F}{\max }\Vert u-u^{\prime }\Vert +2\overline{x})\le \frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}3\overline{x}.\end{array}$ Comparing (29) and (30), we see that $w(y)$ satisfies (28) whenever $3\overline{x}\le \sqrt{(\delta /(1-\delta ))\epsilon r/4}$, which holds by (9).

E

Appendix

Proof of Lemma 3

Since $\dim F^{\ast }=N$ and $v_i<{\max }_a{u}_i(a)$ for all i, there exist $\overline{\eta }>0$ and $\overline{F}\subseteq F^{\ast }$ such that $\{w\in F^{\ast }:d(v,w)\le \overline{\eta }\}\subseteq \overline{F}$, $\dim \overline{F}=N$, and ${\underset{\_}{v}}_i<w_i<{\max }_a{u}_i(a)$ for all i and $w\in \overline{F}$. Fix any such $(\overline{\eta },\overline{F})$.

The following lemma is similar to Lemma 5 of Hörner and Takahashi (2016) or Lemma 7 and pp. 1750–1751 of Sugaya and Wolitzky (2023).

$13$

Lemma

There exists $\overline{x}>ū$ such that $k(\lambda ,\overline{x})\ge {\max }_{v^{\prime }\in \overline{F}}\lambda \cdot v^{\prime }$ for all $\lambda \in \mathrm{\Lambda }$.

$$

Proof

Let $\hat{x}>0$ satisfy the conditions of Assumption 2. For each i, since ${\underset{\_}{v}}_i<w_i<{\max }_a{u}_i(a)$ for all $w\in \overline{F}$, there exist ${\underset{\_}{\lambda }}_i>-1$ and ${\overline{\lambda }}_i<1$ such that (i) for all $\lambda \in \mathrm{\Lambda }$ with ${\lambda }_i\le {\underset{\_}{\lambda }}_i$, we have $\lambda \cdot u(\alpha )-{\sum }_{n\ne i}|{\lambda }_n|\hat{x}\ge {\max }_{w\in \overline{F}}\lambda \cdot w$ for all α satisfying $u_i(\alpha )={\underset{\_}{v}}_i$; and (ii) for all $\lambda \in \mathrm{\Lambda }$ with ${\lambda }_i\ge {\overline{\lambda }}_i$, we have $\lambda \cdot u(\alpha )-{\sum }_{n\ne i}|{\lambda }_n|\hat{x}\ge {\max }_{w\in \overline{F}}\lambda \cdot w$ for all α satisfying $u_i(\alpha )={\max }_a{u}_i(a)$. Given such ${({\underset{\_}{\lambda }}_i,{\overline{\lambda }}_i)}_i$, we define $\overline{x}=\sqrt{N}(2N+\underset{i}{\max }\frac{{\displaystyle \max \{|{\underset{\_}{\lambda }}_i|,{\overline{\lambda }}_i\}}}{{\displaystyle \sqrt{(1-\max {\{|{\underset{\_}{\lambda }}_i|,{\overline{\lambda }}_i\}}^2)/(N-1)}}})\hat{x}.$ For each λ, we now construct $(\alpha ,x(y))$ such that $\lambda \cdot (u(\alpha )+\mathbb{E}[x(y)|\alpha ])\ge {\max }_{w\in \overline{F}}\lambda \cdot w$ and (IC) and (HSx̄) hold. To do so, fix any $i\in \mathrm{argmax}|{\lambda }_i|$, and consider three cases.

(i) ${\lambda }_i\le {\underset{\_}{\lambda }}_i$. In this case, take a minmax profile ${\alpha }^i$ and ${({\hat{x}}_j(y))}_{j\ne i}$ that satisfy Assumption 2(i). Define $x_i(y)={\sum }_{n\ne i}|{\lambda }_n|\hat{x}/{\lambda }_i$ and $x_j(y)={\hat{x}}_j(y)$ for all y. (IC) holds for $j\ne i$ given Assumption 2(i), and (IC) holds for i since player i takes a best response in ${\alpha }^i$ and $x_i(y)$ is independent of y. (HSx̄) holds with $\overline{x}\ge 2N\hat{x}$ given Assumption 2(i). Finally, $\lambda \cdot u({\alpha }^i)+\mathbb{E}[\lambda \cdot x(y)]\ge \lambda \cdot u({\alpha }^i)-{\sum }_{n\ne i}|{\lambda }_n|\hat{x}\ge {\max }_{w\in \overline{F}}\lambda \cdot w$.

(ii) ${\lambda }_i\ge {\underset{\_}{\lambda }}_i$. In this case, take ${ā}^i\in \max u_i(a)$ and ${({\hat{x}}_j(y))}_{j\ne i}$ that satisfies (6) for player j. Define $x_i(y)={\sum }_{n\ne i}|{\lambda }_n|\hat{x}/{\lambda }_i$ and $x_j(y)={\hat{x}}_j(y)$ for all y. The argument is now the same as case (ii).

(iii) Otherwise, there exists $n\ne i$ such that $|{\lambda }_i|/|{\lambda }_n|\le \max \{|{\underset{\_}{\lambda }}_i|,{\overline{\lambda }}_i\}/\sqrt{(1-\max {\{|{\underset{\_}{\lambda }}_i|,{\overline{\lambda }}_i\}}^2)/(N-1)}$. Fix such n. Next, fix $a\in A$ such that $\lambda \cdot u(a)\ge {\max }_{w\in \overline{F}}\lambda \cdot w$. For player i, take $x^i(y)$ such that Assumption 2(ii) holds for a, $(i,n)$, and $c=\mathrm{sign}(-{\lambda }_n/{\lambda }_i)$. For player $j\ne i$, take $x^j(y)$ such that Assumption 2(ii) holds for a, $(j,i)$, and $c=\mathrm{sign}(-{\lambda }_j/{\lambda }_i)$. We then define $x_i(y)=x^i(y)-{\sum }_{j\ne i}\frac{{\lambda }_j}{{\lambda }_i}{x}^j(y)$, $x_n(y)=x^n(y)-\frac{{\lambda }_i}{{\lambda }_n}{x}^i(y)$, and $x_j(y)=x^j(y)$ for $j\ne i,n$. Then (IC) holds for player i since $a_i\in \arg {\max }_{a_i^{\prime }}u(a_i^{\prime },a_{-i})+\mathbb{E}[x^i(y)|a_i^{\prime },a_{-i}]$ and $a_i\in \arg {\max }_{a_i^{\prime }}\mathbb{E}[-({\lambda }_j/{\lambda }_i)x^j(y)|a_i^{\prime },a_{-i}]$ for all $j\ne i$ by Assumption 2(ii). (IC) holds for player n since $a_n\in \arg {\max }_{a_n^{\prime }}u(a_n^{\prime },a_{-n})+\mathbb{E}[x^n(y)|a_n^{\prime },a_{-n}]$ and $a_n\in \arg {\max }_{a_n^{\prime }}\mathbb{E}[-({\lambda }_i/{\lambda }_n)x^i(y)|a_n^{\prime },a_{-n}]$ by Assumption 2(ii). In addition, (IC) holds for player $j\ne i,n$ since $a_j\in \arg {\max }_{a_j^{\prime }}u(a_j^{\prime },a_{-j})+\mathbb{E}[x^j(y)|a_j^{\prime },a_{-j}]$ by Assumption 2(ii). Finally, (HSx̄) holds since $\lambda \cdot x(y)=0$ for all y and $\Vert x(y)\Vert \le \sqrt{N}{\sum }_j|x_j(y)|\le \sqrt{N}(2N+|{\lambda }_i/{\lambda }_n|)\hat{x}\le \overline{x}$. □

By Lemma 13, it suffices to find $c>0$ and $\overline{\delta }<1$ such that, for all $\delta >\overline{\delta }$, there exist $\epsilon >0$ and a ball B with radius $r>0$ such that 31$\underset{v^{\prime }\in \overline{F}}{\max }{\lambda }^{\prime }\cdot v^{\prime }\ge \underset{v^{\prime }\in B}{\max }{\lambda }^{\prime }\cdot v^{\prime }+\epsilon \text{for all }{\lambda }^{\prime }\in \mathrm{\Lambda },$ 32$r\epsilon \ge 36{\overline{x}}^2(1-\delta ),\text{and}$ 33$d(v,B)\le c{(1-\delta )}^{{\beta }^{\ast }/2}.$

If ${\beta }^{\ast }=1$ then, as in Lemma 3 of Hörner and Takahashi (2016), it suffices to take any $o\in \mathrm{int}(\overline{F})$ and any $\ell >0$ sufficiently large compared to $36{\overline{x}}^2$. Let $r={(1-\delta )}^{1/2}$, and take B to have radius r and center $(1-\ell r)v+\ell ro$.

For the rest of the proof, we assume that ${\beta }^{\ast }>1$. We first derive a geometric condition for $w\in F^{\ast }$, similar to Lemma 12.

$14$

Lemma

There exist $\lambda \in {\mathrm{\Lambda }}_v^{\ast }$, $\rho >0$, and $\kappa >0$ such that, if $d(v,w)<\rho$ and $\kappa d{(v,w)}^{{\beta }^{\ast }}<\lambda \cdot (v-w)$, then $w\in \mathrm{int}(F^{\ast })$.

$$

Proof

Since $F^{\ast }$ is full-dimensional and has min-curvature of order at most β at v, there exist $\overline{\epsilon }>0$ and $\kappa >0$ such that, for all $w\in \mathrm{bnd}(F^{\ast })$ satisfying $d(v,w)<\overline{\epsilon }$, we have $\lambda \cdot (v-w)<\kappa d{(v,w)}^{\beta }\le \kappa d{(v,w)}^{{\beta }^{\ast }}$ for all $\lambda \in {\mathrm{\Lambda }}_v^{\ast }$. Let $B_{{\epsilon }^{\prime }}(v)=\{w\in {\mathbb{R}}^N:d(v,w)={\epsilon }^{\prime }\}$. Since $F^{\ast }$ is full-dimensional, there exists $\lambda \in {\mathrm{\Lambda }}_v^{\ast }$, ${\epsilon }^{\prime }>0$, and $t>0$ such that $C:=B_{{\epsilon }^{\prime }}(v)-t\lambda \subseteq F^{\ast }$. Fix such λ, ${\epsilon }^{\prime }$, and t, and let $\epsilon =\min \{\overline{\epsilon },{\epsilon }^{\prime },t\}$.

Now fix any $\rho <\min \{\epsilon ,{(t/2\kappa )}^{1/{\beta }^{\ast }}\}$ and $d<\rho$, and let $G=\{w\in B_d(v):\lambda \cdot (v-w)\ge 2\kappa d^{{\beta }^{\ast }}\}$. We wish to show that $G\subseteq F^{\ast }$ (and in particular $G\subseteq \mathrm{int}(F^{\ast })$, since $G\cap \mathrm{bnd}(F^{\ast })=\mathrm{\varnothing }$).

To see this, let $W=B_d(v)\cap \mathrm{bnd}(F^{\ast })$, $H=\{w:\lambda \cdot (v-w)=\kappa d^{{\beta }^{\ast }}\}$, $H^{\prime }=\{w:\lambda \cdot (v-w)=t\}$, and $D=C\cap H^{\prime }$. Since $d<\rho <\min \{\epsilon ,{(t/\kappa )}^{1/{\beta }^{\ast }}\}$, G lies in between H and $H^{\prime }$. In addition, the projection of G onto H is a subset of the projection of W onto H, and the projection of G onto $H^{\prime }$ is a subset of D. Hence, we have $G\subseteq \mathrm{co}(W\cup D)$. Finally, since $W\subseteq F^{\ast }$ and $D\subseteq C\subseteq F^{\ast }$, and $F^{\ast }$ is convex, we have $\mathrm{co}(W\cup D)\subseteq F^{\ast }$, so $G\subseteq F^{\ast }$. □

$15$

Lemma

There exist $\overline{c}>0$, $\eta >0$, and $\overline{\delta }<1$ such that, for all $\delta >\overline{\delta }$, there exists a ball $B\subseteq \overline{F}$ of radius $r=\eta {(1-\delta )}^{1-{\beta }^{\ast }/2}$ satisfying $d(v,B)=\overline{c}{(1-\delta )}^{{\beta }^{\ast }/2}$.

$$

Proof

Fix $\lambda \in {\mathrm{\Lambda }}_v^{\ast }$, $\rho >0$, and $\kappa >0$ as in Lemma 14. Given $\overline{c}$ and η to be determined, let B be the ball with radius $r=\eta {(1-\delta )}^{1-{\beta }^{\ast }/2}$ and center $o=v-(r+d)\lambda$, where $d=\overline{c}{(1-\delta )}^{{\beta }^{\ast }/2}$, and take any $ŵ\in \partial B$. Let $x=\lambda \cdot (ŵ-o)$, so that xλ is the projection of $ŵ-o$ on λ. Then $\begin{array}{rl}{\Vert v-ŵ\Vert }^2& ={\Vert v-o-x\lambda \Vert }^2+{\Vert ŵ-o-x\lambda \Vert }^2={(r+d-x)}^2+r^2-x^2,\text{and}\\ \lambda \cdot (v-ŵ)& =r+d-x.\end{array}$

Recall that, by construction, $\{w\in F^{\ast }:d(v,w)\le \overline{\eta }\}\subseteq \overline{F}$. Since $d(v,w)\le d(v,o)+d(o,w)\le 2r+d$ for all $w\in B$, it suffices to show that $2r+d\le \overline{\eta }$ and $B\subseteq F^{\ast }$. By Lemma 14, if $d(v,w)<\rho$ and $\kappa d{(v,w)}^{{\beta }^{\ast }}\le \lambda \cdot (v-w)$, then $w\in F^{\ast }$. Since $x\in [-r,r]$, it suffices to find $\overline{c}$, η, and $\overline{\delta }$ such that, for all $\delta >\overline{\delta }$, we have 34$2r+d\le \overline{\eta },$ 35${((r+d)-x)}^2+r^2-x^2\le {\rho }^2\text{for all }x\in [-r,r],\text{and}$ 36$\underset{x\in [-r,r]}{\max }f(x,\delta ,{\beta }^{\ast })\le 0,$ where $f(x,\delta ,{\beta }^{\ast }):=\kappa {({(r+d-x)}^2+r^2-x^2)}^{{\beta }^{\ast }/2}-(r+d-x).$

We consider separately the cases where ${\beta }^{\ast }=2$ and ${\beta }^{\ast }\in (1,2)$. First, consider ${\beta }^{\ast }=2$. Let $\hat{\eta }>0$ be such that (35) holds whenever $\max \{r,d\}\le \hat{\eta }$, and let any $\overline{c}=1$ and $\eta =\min \{\hat{\eta },\overline{\eta }/4,\kappa /4\}$, so that $r=\eta$ and $d=1-\delta$. For sufficiently large δ, we have $2r+d\le \overline{\eta }$ and $d\le \hat{\eta }$, and hence (34) and (35) hold. In addition, since $f(x,\delta ,2)$ is linear in x when ${\beta }^{\ast }=2$, (36) holds whenever $f(r,\delta ,2)\le 0$ and $f(-r,\delta ,2)\le 0$. In turn, these inequalities hold for sufficiently large δ, since $f(r,\delta ,2)=d(\kappa d-1)$ and ${\lim }_{\delta \to 1}\kappa d-1<0$, and $f(-r,\delta ,{\beta }^{\ast })=(2r+d)(\kappa (2r+d)-1)$ and ${\lim }_{\delta \to 1}\kappa (2r+d)-1=2\kappa \eta -1<0$.

Next, consider ${\beta }^{\ast }\in (1,2)$. Let $\overline{c}=2^{2/(2-{\beta }^{\ast })}{\kappa }^{2/(2-{\beta }^{\ast })}{\beta }^{\ast {\beta }^{\ast }/(2-{\beta }^{\ast })}$ and $\eta =1$, so that $r={(1-\delta )}^{1-{\beta }^{\ast }/2}$ and $d=\overline{c}{(1-\delta )}^{{\beta }^{\ast }/2}$. Since $\max \{r,d\}\to 0$ as $\delta \to 1$, (34) and (35) hold for sufficiently large δ. In addition, $f(x,\delta ,{\beta }^{\ast })$ is concave in x and is maximized over $x\in [-r,r]$ at $x^{\ast }=\frac{{\displaystyle 2r^2+2dr+d^2-{(\kappa (r+d){\beta }^{\ast })}^{\frac{2}{2-{\beta }^{\ast }}}}}{{\displaystyle 2(r+d)}}.$ It thus suffices to show that $f(x^{\ast },\delta ,{\beta }^{\ast })\le 0$ for sufficiently large δ. By algebra, $f(x^{\ast },\delta ,{\beta }^{\ast })=-\frac{{\displaystyle 2r+d}}{{\displaystyle r+d}}\frac{{\displaystyle d}}{{\displaystyle 2}}+({\beta }^{\ast \frac{{\beta }^{\ast }}{2-{\beta }^{\ast }}}-\frac{{\displaystyle 1}}{{\displaystyle 2}}{\beta }^{\ast \frac{2}{2-{\beta }^{\ast }}}){\kappa }^{\frac{2}{2-{\beta }^{\ast }}}{(r+d)}^{\frac{{\beta }^{\ast }}{2-{\beta }^{\ast }}}.$ Finally, since $r={(1-\delta )}^{1-{\beta }^{\ast }/2}\ge \overline{c}{(1-\delta )}^{{\beta }^{\ast }/2}=d$ for sufficiently large δ, we have $\begin{array}{rl}f(x^{\ast },\delta ,{\beta }^{\ast })& \le -\frac{{\displaystyle d}}{{\displaystyle 2}}+2^{\frac{{\beta }^{\ast }}{2-{\beta }^{\ast }}}{\kappa }^{\frac{2}{2-{\beta }^{\ast }}}{\beta }^{\ast \frac{{\beta }^{\ast }}{2-{\beta }^{\ast }}}{r}^{\frac{{\beta }^{\ast }}{2-{\beta }^{\ast }}}\\ & =-\frac{{\displaystyle \overline{c}{(1-\delta )}^{\frac{{\beta }^{\ast }}{2}}}}{{\displaystyle 2}}+2^{\frac{{\beta }^{\ast }}{2-{\beta }^{\ast }}}{\kappa }^{\frac{2}{2-{\beta }^{\ast }}}{\beta }^{\ast \frac{{\beta }^{\ast }}{2-{\beta }^{\ast }}}{(1-\delta )}^{(1-\frac{{\beta }^{\ast }}{2})\frac{{\beta }^{\ast }}{2-{\beta }^{\ast }}}\\ & ={(1-\delta )}^{\frac{{\beta }^{\ast }}{2}}(-\frac{{\displaystyle \overline{c}}}{{\displaystyle 2}}+2^{\frac{{\beta }^{\ast }}{2-{\beta }^{\ast }}}{\kappa }^{\frac{2}{2-{\beta }^{\ast }}}{\beta }^{\ast \frac{{\beta }^{\ast }}{2-{\beta }^{\ast }}})=0.\end{array}$ □

We now complete the proof of Lemma 3. Take $\overline{c}$, η, $\overline{\delta }$, B, and r as in Lemma 15. Let $B^{\prime }$ be the radial contraction of B by a factor of $1-72{\overline{x}}^2{(1-\delta )}^{{\beta }^{\ast }/2}/(\eta r)$, and define $\epsilon =72{\overline{x}}^2{(1-\delta )}^{{\beta }^{\ast }/2}/\eta$ and $c=\overline{c}+72{\overline{x}}^2/\eta$. Since $d(v,B)=\overline{c}{(1-\delta )}^{{\beta }^{\ast }/2}$, we have $d(v,B^{\prime })=(\overline{c}+72{\overline{x}}^2/\eta ){(1-\delta )}^{{\beta }^{\ast }/2}=c{(1-\delta )}^{{\beta }^{\ast }/2}$, so (33) holds. Moreover, denoting the radius of $B^{\prime }$ by $r^{\prime }$, we have $\begin{array}{cc}\hfill r^{\prime }\epsilon & =(1-\frac{{\displaystyle 72{\overline{x}}^2{(1-\delta )}^{{\beta }^{\ast }/2}}}{{\displaystyle {\eta }^2{(1-\delta )}^{1-{\beta }^{\ast }/2}}})\eta {(1-\delta )}^{1-{\beta }^{\ast }/2}\times \frac{{\displaystyle 72{\overline{x}}^2{(1-\delta )}^{{\beta }^{\ast }/2}}}{{\displaystyle \eta }}\hfill \\ \hfill & =(1-\frac{{\displaystyle 72{\overline{x}}^2{(1-\delta )}^{{\beta }^{\ast }-1}}}{{\displaystyle {\eta }^2}})72{\overline{x}}^2(1-\delta ).\hfill \end{array}$ For sufficiently large δ, this is greater than $36{\overline{x}}^2(1-\delta )$, so (32) holds. Finally, since $B\subseteq \overline{F}$, for all ${\lambda }^{\prime }\in \mathrm{\Lambda }$ we have ${\max }_{v^{\prime }\in \overline{F}}{\lambda }^{\prime }\cdot v^{\prime }\ge {\max }_{v^{\prime }\in B}{\lambda }^{\prime }\cdot v^{\prime }={\max }_{v^{\prime }\in B^{\prime }}{\lambda }^{\prime }\cdot v^{\prime }+72{\overline{x}}^2{(1-\delta )}^{{\beta }^{\ast }/2}/\eta ={\max }_{v^{\prime }\in B^{\prime }}{\lambda }^{\prime }\cdot v^{\prime }+\epsilon$, so (31) holds.

F

Appendix

Proof of Theorem 3

We first show that first-order inefficiency in the blind game is no less than (14). Fix $\delta <1$, $w\in (0,ū)$, and a Nash equilibrium in the blind game where the agent's payoff is w. Let $\mu \in \mathrm{\Delta }({(A\times Y)}^{\mathrm{\infty }})$ and $\alpha \in \mathrm{\Delta }(A)$ be the corresponding outcome and occupation measure. Let ${û}_t=u(c_t)-\psi (a_t)-w$.

By feasibility, the principal's payoff is at most $(1-\delta )\sum _{t=1}^{\mathrm{\infty }}{\delta }^{t-1}{\mathbb{E}}^{\mu }[\overline{F}(w+{û}_t)].$ At the same time, incentive compatibility implies inequality (15) (see Lemma 22 below), and promise keeping implies 37$(1-\delta )\sum _{t=1}^{\mathrm{\infty }}{\delta }^{t-1}{\mathbb{E}}^{\mu }[{û}_t]=0.$

The following is the key lemma. $16$

Lemma

There exist $c,\epsilon >0$ such that, for any sufficiently large $\delta <1$, we have 38$\begin{array}{rl}& \underset{\begin{array}{c}\hfill {({û}_t(h))}_{t,h}\ge -(w+h(Ā))\hfill \\ \hfill \text{such that (15) and (37)}\hfill \end{array}}{\max }(1-\delta )\sum _{t=1}^{\mathrm{\infty }}{\delta }^{t-1}{\mathbb{E}}^{\mu }\left[\overline{F}\right(w+{û}_t(h)\left)\right]\\ & \le \overline{F}(w)+\frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}{\mathbb{E}}^{\alpha }\left[\frac{{\displaystyle {\psi }^{\prime }{(a)}^2}}{{\displaystyle \mathcal{I}(a)}}\right]\frac{{\displaystyle {\overline{F}}^{\prime \prime }(w)}}{{\displaystyle 2}}+c{(1-\delta )}^{1+\epsilon }.\end{array}$

We sketch the proof of Lemma 16, providing the details in the next section. Subtracting $\overline{F}(w)$ from both sides of (38), dividing both sides by $\frac{1-\delta }{\delta }\frac{{\overline{F}}^{\prime \prime }(w)}{2}$, and taking a second-order Taylor approximation of the LHS (where the first-order term is zero by (37)) give $\underset{{({û}_t(h))}_{t,h}\ge -(w+\psi (Ā))\text{ such that (15)}}{\inf }\sum _{t=1}^{\mathrm{\infty }}{\delta }^t{\mathbb{E}}^{\mu }\left[{û}_t{(h)}^2\right].$ To establish Lemma 16, it suffices to show that the value of this program exceeds ${\mathbb{E}}^{\alpha }[{\psi }^{\prime }{(a)}^2/\mathcal{I}(a)]$. To see this, take a common Lagrange multiplier of $2(1-\delta ){\delta }^{t-1}$ on (15) for each t. Then, by weak duality, the value of the program is no less than 39$\underset{{({û}_t(h))}_{t,h}}{\inf }\sum _{t=1}^{\mathrm{\infty }}{\mathbb{E}}^{\mu }[{\delta }^t{û}_t{(h)}^2-2\frac{{\displaystyle {\psi }^{\prime }(a_t)\nu (y_t|a_t)}}{{\displaystyle \mathcal{I}(a_t)}}\frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}\sum _{t^{\prime }=t+1}^{\mathrm{\infty }}{\delta }^{t^{\prime }}{û}_{t^{\prime }}(h)]+2{\mathbb{E}}^{\alpha }\left[\frac{{\displaystyle {\psi }^{\prime }{(a)}^2}}{{\displaystyle \mathcal{I}(a)}}\right].$ Taking the first-order condition for ${û}_t(h)$ and substituting into (39) as in the text gives $-{\left(\frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}\right)}^2\sum _{t=2}^{\mathrm{\infty }}{\delta }^t{\mathbb{E}}^{\mu }\left[{\left(\sum _{t^{\prime }=1}^{t-1}\frac{{\displaystyle {\psi }^{\prime }(a_{t^{\prime }})\nu (y_{t^{\prime }}|a_{t^{\prime }})}}{{\displaystyle \mathcal{I}(a_{t^{\prime }})}}\right)}^2\right]+2{\mathbb{E}}^{\alpha }\left[\frac{{\displaystyle {\psi }^{\prime }{(a)}^2}}{{\displaystyle \mathcal{I}(a)}}\right]={\mathbb{E}}^{\alpha }\left[\frac{{\displaystyle {\psi }^{\prime }{(a)}^2}}{{\displaystyle \mathcal{I}(a)}}\right],$ where the equality follows because, since ${({\psi }_t(a_t)v_t(y_t|a_t)/{\mathcal{I}}_t(a_t))}_t$ is a sequence of martingale increments, we have $\begin{array}{rl}{\mathbb{E}}^{\mu }\left[{\left(\sum _{t^{\prime }=1}^{t-1}\frac{{\displaystyle {\psi }^{\prime }(a_{t^{\prime }})\nu (y_{t^{\prime }}|a_{t^{\prime }})}}{{\displaystyle \mathcal{I}(a_{t^{\prime }})}}\right)}^2\right]& =\sum _{t^{\prime }=1}^{t-1}{\mathbb{E}}^{\mu }\left[{\left(\frac{{\displaystyle {\psi }^{\prime }(a_{t^{\prime }})\nu (y_{t^{\prime }}|a_{t^{\prime }})}}{{\displaystyle \mathcal{I}(a_{t^{\prime }})}}\right)}^2\right]\\ & =\sum _{t^{\prime }=1}^{t-1}{\mathbb{E}}^{{\alpha }_{t^{\prime }}}\left[{\left(\frac{{\displaystyle {\psi }^{\prime }(a)}}{{\displaystyle \mathcal{I}(a)}}\right)}^2\mathbb{E}\right[\nu {(y|a)}^2\left]\right]=\sum _{t^{\prime }=1}^{t-1}{\mathbb{E}}^{{\alpha }_{t^{\prime }}}\left[\frac{{\displaystyle {\psi }^{\prime }{(a)}^2}}{{\displaystyle \mathcal{I}(a)}}\right],\end{array}$ and hence 40$\begin{array}{rl}{\left(\frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}\right)}^2\sum _{t=2}^{\mathrm{\infty }}{\delta }^t{\mathbb{E}}^{\mu }\left[{\left(\sum _{t^{\prime }=1}^{t-1}\frac{{\displaystyle {\psi }^{\prime }(a_{t^{\prime }})\nu (y_{t^{\prime }}|a_{t^{\prime }})}}{{\displaystyle \mathcal{I}(a_{t^{\prime }})}}\right)}^2\right]& ={\left(\frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}\right)}^2\sum _{t=2}^{\mathrm{\infty }}{\delta }^t\sum _{t^{\prime }=1}^{t-1}{\mathbb{E}}^{{\alpha }_{t^{\prime }}}\left[\frac{{\displaystyle {\psi }^{\prime }{(a)}^2}}{{\displaystyle \mathcal{I}(a)}}\right]\\ & =(1-\delta )\sum _{t=1}^{\mathrm{\infty }}{\delta }^{t-1}{\mathbb{E}}^{{\alpha }_t}\left[\frac{{\displaystyle {\psi }^{\prime }{(a)}^2}}{{\displaystyle \mathcal{I}(a)}}\right]\\ & ={\mathbb{E}}^{\alpha }\left[\frac{{\displaystyle {\psi }^{\prime }{(a)}^2}}{{\displaystyle \mathcal{I}(a)}}\right].\end{array}$

At the same time, since $\overline{F}$ is strictly concave, there exists ${\mathcal{\epsilon }}_1>0$ such that the principal's payoff is at most $\overline{F}(w)-{\mathcal{\epsilon }}_1{\mathbb{E}}^{\alpha }[{(a-ā(w))}^2]$ (see Lemma 19 below). So, together with Lemma 16, the following lemma establishes that first-order inefficiency in the blind game is no less than (14).

$17$

Lemma

There exist $ĉ,\hat{\epsilon }>0$ such that, for any sufficiently large $\delta <1$, we have $\begin{array}{rl}& \underset{\alpha }{\max }\min \left\{\frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}{\mathbb{E}}^{\alpha }\right[\frac{{\displaystyle {\psi }^{\prime }{(a)}^2}}{{\displaystyle \mathcal{I}(a)}}]\frac{{\displaystyle {\overline{F}}^{\prime \prime }(w)}}{{\displaystyle 2}}+c{(1-\delta )}^{1+\epsilon },-{\mathcal{\epsilon }}_1{\mathbb{E}}^{\alpha }[{(a-ā(w))}^2\left]\right\}\\ & \le \frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}\frac{{\displaystyle {\psi }^{\prime }{(ā(w))}^2}}{{\displaystyle \mathcal{I}(ā(w))}}\frac{{\displaystyle {\overline{F}}^{\prime \prime }(w)}}{{\displaystyle 2}}+ĉ{(1-\delta )}^{1+\hat{\epsilon }}.\end{array}$

We now show that first-order inefficiency in the public game is no more than (14). The proof is constructive. As a first step, it is helpful to first construct a static contract that induces a target effort level $ā\in A$. In particular, if the agent is rewarded with a utility of $({\psi }^{\prime }(ā)/\mathcal{I}(ā))\nu (y|ā)$ following any signal realization y, she chooses a to maximize ${\int }_y\frac{{\displaystyle {\psi }^{\prime }(ā)}}{{\displaystyle \mathcal{I}(ā)}}\nu (y|ā)p(y|a)dy-\psi (a).$ The solution is $a=ā$, because $a=ā$ satisfies the first-order condition ${\int }_y\frac{{\displaystyle {\psi }^{\prime }(ā)}}{{\displaystyle \mathcal{I}(ā)}}\nu (y|ā)p_a(y|a)dy={\psi }^{\prime }(a),$ and the second-order condition holds by (11). Moreover, the expected reward equals zero.

Heuristically, the repeated game equilibrium is constructed by using the above reward to adjust the agent's continuation payoff $w_t$ after each period t (so the agent's continuation payoff is a martingale), while targeting effort level $ā(w_t)$ in each period t. This heuristic requires two adjustments, however. First, if the score $\nu (y|a)$ is unbounded, we must truncate the reward for extreme scores, and then further adjust the reward so the agent's first-order condition is exactly satisfied. Second, it is convenient to target zero effort once the agent's continuation payoff $w_t$ strays too far from its initial value w.

Formally, fix any $\epsilon >0$. Recursively, given the agent's promised continuation payoff $w_t(h^t)$ at history $h^t={(r_{t^{\prime }},y_{t^{\prime }})}_{t^{\prime }=1}^{t-1}$, we define the recommended period t action $r_t(h^t)$ and consumption level $c_t(h^t)$ (which is independent of the period t signal $y_t$), as well as the next period continuation payoff $w_{t+1}(h^t,y_t)$, as follows. First, say that a history $h^t$ is regular if $|w_t(h^t)-w|\le {(1-\delta )}^{1/2-\epsilon }$, and irregular otherwise. At a regular history, define $r_t(h^t)=ā(w_t(h^t))$, $c_t(h^t)=\overline{c}(w_t(h^t))$, and $w_{t+1}(h^t,y_t)=w_t\left(h^t\right)+\frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}{x}_{r_t(h^t)}(y),$ where, for each $ā\in A$, $x_{ā}(y)$ is an adjusted version of $({\psi }^{\prime }(ā)/\mathcal{I}(ā))\nu (y|ā)$ (constructed in Lemma 25 below) that satisfies 41$\begin{array}{rl}ā& \in \underset{a}{\mathrm{argmax}}{\int }_y{x}_{ā}(y)p(y|a)dy-\psi (a),\\ {\int }_y{x}_{ā}(y)p(y|ā)dy& =0,\end{array}$ 42${\int }_y{x}_{ā}{(y)}^{2}p(y|ā)dy=\frac{{\displaystyle {\psi }^{\prime }{(ā)}^2}}{{\displaystyle \mathcal{I}(ā)}}+O(1-\delta ),\text{and}$ $|x_{ā}(y)|\le {(1-\delta )}^{-1/4}.$ At an irregular history, define $r_t(h^t)=0$, $c_t(h^t)=u^{\mathrm{-1}}(w_t(h^t))$, and $w_{t+1}(h^t,y_t)=w_t(h^t)$ for all $y_t$. Note that the initial history $h^1$ is regular, and that if a history $h^t$ is irregular, then so is every subsequent history. Note also that, by construction, $|w_{t+1}(h^t,y_t)-w_t(h^t)|=O{(1-\delta )}^{3/4}$ for every regular history $h^t$ and signal $y_t$. Since $|w_t(h^t)-w|\le {(1-\delta )}^{1/2-\epsilon }$ for every regular history $h^t$, this implies that, for sufficiently large $\delta <1$, we have $w_t(h^t)\in [0,2w]$ for every history $h^t$.

The proof is completed by the following lemma, which shows that the first-order inefficiency of this equilibrium is no more than (14).

$18$

Lemma

There exist $\tilde{c},\tilde{\epsilon }>0$ such that, for any sufficiently large $\delta <1$, the principal's payoff in the above equilibrium is no less than $\overline{F}(w)+\frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}\frac{{\displaystyle {\psi }^{\prime }{(ā(w))}^2}}{{\displaystyle \mathcal{I}(ā(w))}}\frac{{\displaystyle {\overline{F}}^{\prime \prime }(w)}}{{\displaystyle 2}}-\tilde{c}{(1-\delta )}^{1+\tilde{\epsilon }}.$

Intuitively, since $w_t(h^t)$ is a martingale with volatility of order ${(1-\delta )}^2$ (by (41) and (42)), it is very unlikely that $w_t(h^t)$ moves more than a $O({(1-\delta )}^{1/2})$ distance away from w within a timeframe that has more than an $O(1-\delta )$ payoff impact. Consequently, the principal's payoff is almost entirely determined by her payoff at regular histories, and thus equals ${\mathbb{E}}^{\mu }[(1-\delta )\sum _{t=1}^{\mathrm{\infty }}{\delta }^{t-1}\overline{F}(w_t\left(h^t\right)\left)\right]+o(1-\delta ),$ where μ is the equilibrium outcome. Taking a second-order Taylor expansion around $w_t(h^t)=w$ and ignoring the remainder, this equals $\overline{F}(w)+{\mathbb{E}}^{\mu }[(1-\delta )\sum _{t=1}^{\mathrm{\infty }}{\delta }^{t-1}\frac{{\displaystyle {\overline{F}}^{\prime \prime }(w)}}{{\displaystyle 2}}{\left(w_t\right(h^t)-w)}^2].$ Since $w_t\left(h^t\right)=w+\frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}\sum _{t^{\prime }=1}^{t-1}{x}_{a_{t^{\prime }}}(y_{t^{\prime }})\text{for all regular histories }{h}^t$ and (42) holds, the same calculation as for (40) gives ${\mathbb{E}}^{\mu }[(1-\delta )\sum _{t=1}^{\mathrm{\infty }}{\delta }^{t-1}\frac{{\displaystyle {\overline{F}}^{\prime \prime }(w)}}{{\displaystyle 2}}{\left(w_t\right(h^t)-w)}^2]=\frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}{\mathbb{E}}^{\alpha }\left[\frac{{\displaystyle {\psi }^{\prime }(a)}}{{\displaystyle \mathcal{I}(a)}}\right]\frac{{\displaystyle {\overline{F}}^{\prime \prime }(w)}}{{\displaystyle 2}}+o(1-\delta ),$ where α is the equilibrium occupation measure. Finally, since $w_t(h^t)$ is close to w with high probability under α, we have ${\mathbb{E}}^{\alpha }[{\psi }^{\prime }(a)/\mathcal{I}(a)]={\psi }^{\prime }(ā(w))/\mathcal{I}(ā(w))+o(1)$, completing the proof.

G

Appendix

Omitted details for the proof of Theorem 3

We require some preliminary lemmas. The first two derive properties of the feasible payoff set.

$19$

Lemma

There exists ${\mathcal{\epsilon }}_1>0$ such that, for any $\alpha \in \mathrm{\Delta }(A)$, we have ${\mathbb{E}}^{\alpha }[a]-u^{\mathrm{-1}}(w+{\mathbb{E}}^{\alpha }[\psi (a)\left]\right)\le \overline{F}(w)-{\mathcal{\epsilon }}_1{\mathbb{E}}^{\alpha }\left[{(a-ā(w))}^2\right].$

$$

Proof

Since $\psi \in C^2$ and ${\psi }^{\prime \prime }>0$, there exists ${\mathcal{\epsilon }}_1>0$ such that, for all $a\in A$, we have $\psi (a)-\psi (ā(w))\ge {\psi }^{\prime }(ā(w))(a-ā(w))+{\mathcal{\epsilon }}_1{(a-ā(w))}^2.$ Thus, for any $\alpha \in \mathrm{\Delta }(A)$, we have $\begin{array}{rl}& \overline{F}(w)-{\mathbb{E}}^{\alpha }[a]+u^{\mathrm{-1}}(w+{\mathbb{E}}^{\alpha }[\psi (a)\left]\right)\\ & =ā(w)-u^{\mathrm{-1}}(w+\psi (ā(w)\left)\right)-{\mathbb{E}}^{\alpha }[a]+u^{\mathrm{-1}}(w+{\mathbb{E}}^{\alpha }[\psi (a)\left]\right)\\ & \ge ā(w)-{\mathbb{E}}^{\alpha }[a]+\frac{{\displaystyle {\mathbb{E}}^{\alpha }[\psi (a)]-\psi (ā(w))}}{{\displaystyle u^{\prime }(\overline{c}(w))}}\\ & \ge ā(w)-{\mathbb{E}}^{\alpha }[a]+\frac{{\displaystyle {\psi }^{\prime }(ā(w))({\mathbb{E}}^{\alpha }[a]-ā(w))+{\mathcal{\epsilon }}_1{\mathbb{E}}^{\alpha }\left[{(a-ā(w))}^2\right]}}{{\displaystyle u^{\prime }(\overline{c}(w))}}\ge \frac{{\displaystyle {\mathcal{\epsilon }}_1{\mathbb{E}}^{\alpha }\left[{(a-ā(w))}^2\right]}}{{\displaystyle u^{\prime }(\overline{c}(w))}},\end{array}$ where the first inequality is by Taylor expansion of $u^{\mathrm{-1}}(w+{\mathbb{E}}^{\alpha }[\psi (a)])-u^{\mathrm{-1}}(w+\psi (ā(w)))$ around $w+{\mathbb{E}}^{\alpha }[\psi (a)]=w+\psi (ā(w))=\overline{c}(w)$, the second inequality is by Taylor expansion of $\psi (a)-\psi (ā(w))$ around $a=ā(w)$, and the last equality is by $u^{\prime }(\overline{c}(w))={\psi }^{\prime }(ā(w))$ (by definition of $ā(w)$). □

$20$

Lemma

There exists ${\epsilon }_2>0$ such that ${\overline{F}}^{\prime \prime }(w)\le -{\epsilon }_2$ for all $w\ge -\psi (Ā)$.

$$

Proof

Differentiating the equality $u^{\prime }(\overline{c}(w))={\psi }^{\prime }(ā(w))$ with respect to w yields 43${ā}^{\prime }(w)=\frac{{\displaystyle u^{\prime \prime }(\overline{c}(w))}}{{\displaystyle u^{\prime }(ā(w))\left({\psi }^{\prime \prime }\right(ā(w))-u^{\prime \prime }(\overline{c}(w)\left)\right)}}.$ Since $\overline{F}(w)={\max }_{a\in A}a-u^{\mathrm{-1}}(w+\psi (a))$, by the envelope theorem we have ${\overline{F}}^{\prime }(w)=-1/u^{\prime }(\overline{c}(w))$, or equivalently ${\overline{F}}^{\prime }(w)=-1/u^{\prime }(w+\psi (ā(w)))$. Differentiating this equality respect to w and substituting (43) yield ${\overline{F}}^{\prime \prime }(w)=\frac{{\displaystyle u^{\prime \prime }(\overline{c}(w)){\psi }^{\prime \prime }(ā(w))}}{{\displaystyle u^{\prime }{(\overline{c}(w))}^3\left({\psi }^{\prime \prime }\right(ā(w))-u^{\prime \prime }(\overline{c}(w)\left)\right)}}.$ The lemma follows since ${\psi }^{\prime \prime }>0$, $u^{\prime \prime }<0$, and $u^{\prime \prime }(c)/u^{\prime }{(c)}^3$ is bounded away from zero by (10). □

The next lemma gives a key probability bound.

$21$

Lemma

For any $c>0$ and $\vartheta \ge 0$, there exists $\overline{\delta }<1$ such that, for any $\delta >\overline{\delta }$ and any sequence of nonnegative random variables ${(X_t)}_{t\ge 1}$, where $X_t$ is distributed according to a cdf $G_t$ satisfying $1-G_t(x)\le 2\exp (-c{x}^2/({(1-\delta )}^{2}t))$ for all t, we have $(1-\delta )\sum _{t=1}^{\mathrm{\infty }}{\delta }^{t-1}t{\int }_{x\ge {(1-\delta )}^{\frac{1}{2}-\epsilon }}{x}^{\vartheta }d{G}_t(x)\le {(1-\delta )}^2.$

$$

Proof

Let $T={(1-\delta )}^{\frac{1}{2}-\epsilon }$. It suffices to show that $\begin{array}{cc}\hfill & \underset{\delta \to 1}{\lim }{(1-\delta )}^{\mathrm{-1}}\sum _{t=1}^T{\delta }^{t-1}t{\int }_{x\ge {(1-\delta )}^{\frac{1}{2}-\epsilon }}{x}^{\vartheta }d{G}_t(x)\hfill \\ \hfill & =\underset{\delta \to 1}{\lim }{(1-\delta )}^{\mathrm{-1}}\sum _{t=T+1}^{\mathrm{\infty }}{\delta }^{t-1}t{\int }_{x\ge {(1-\delta )}^{\frac{1}{2}-\epsilon }}{x}^{\vartheta }d{G}_t(x)=0.\hfill \end{array}$ For each t, we have $\begin{array}{rl}& {\int }_{x\ge {(1-\delta )}^{\frac{1}{2}-\epsilon }}{x}^{\vartheta }d{G}_t(x)\\ & ={\int }_{x\ge {(1-\delta )}^{\frac{1}{2}-\epsilon }}\vartheta x^{\vartheta -1}(1-G_t(x))dx+{(1-\delta )}^{\vartheta (\frac{1}{2}-\epsilon )}(1-G_t({(1-\delta )}^{\frac{1}{2}-\epsilon }\left)\right)\\ & \le 2\vartheta {\int }_{x\ge {(1-\delta )}^{\frac{1}{2}-\epsilon }}{x}^{\vartheta -1}\exp (-\frac{{\displaystyle c{x}^2}}{{\displaystyle {(1-\delta )}^{2}t}})dx+2{(1-\delta )}^{\vartheta (\frac{1}{2}-\epsilon )}\exp (-\frac{{\displaystyle c{(1-\delta )}^{1-2\epsilon }}}{{\displaystyle {(1-\delta )}^{2}t}}),\end{array}$ where the equality is by integration by parts and ${\lim }_{x\to \mathrm{\infty }}{x}^{\vartheta }(1-G_t(x))=0$, and the inequality is by $1-G_t(x)\le 2\exp (-c{x}^2/({(1-\delta )}^{2}t))$. Note that ${\sum }_{t=1}^{T}t{\delta }^{t-1}\le {(1-\delta )}^{\mathrm{-2}}$ and, if $t\le T$, then ${(1-\delta )}^{1-2\epsilon }/({(1-\delta )}^{2}t)\ge {(1-\delta )}^{-\epsilon }$. Therefore, $\underset{\delta \to 1}{\lim }{(1-\delta )}^{\mathrm{-1}}\sum _{t=1}^{T}t{\delta }^{t-1}{(1-\delta )}^{\vartheta (\frac{1}{2}-\epsilon )}\exp (-\frac{{\displaystyle c{(1-\delta )}^{1-2\epsilon }}}{{\displaystyle {(1-\delta )}^{2}t}})=0.$ At the same time, since ${\sum }_{t=T+1}^{\mathrm{\infty }}t{\delta }^{t-1}=\frac{{\delta }^{T+1}(1+T(1-\delta ))}{\delta {(1-\delta )}^2}\le {(1-\delta )}^{-2-\epsilon }{\delta }^T\le {(1-\delta )}^{-2-\epsilon }\exp (-{(1-\delta )}^{-\epsilon })$, we have $\underset{\delta \to 1}{\lim }{(1-\delta )}^{\mathrm{-1}}\sum _{t=T+1}^{\mathrm{\infty }}t{\delta }^{t-1}{(1-\delta )}^{\vartheta (\frac{1}{2}-\epsilon )}\exp (-\frac{{\displaystyle c{(1-\delta )}^{1-2\epsilon }}}{{\displaystyle {(1-\delta )}^{2}t}})=0.$ It thus suffices to show that 44$\underset{\delta \to 1}{\lim }{(1-\delta )}^{\mathrm{-1}}\sum _{t=1}^{T}t{\delta }^{t-1}{\int }_{x\ge {(1-\delta )}^{\frac{1}{2}-\epsilon }}{x}^{\vartheta -1}\exp (-\frac{{\displaystyle c{x}^2}}{{\displaystyle {(1-\delta )}^{2}t}})dx=0,\text{and}$ 45$\underset{\delta \to 1}{\lim }{(1-\delta )}^{\mathrm{-1}}\sum _{t=T+1}^{\mathrm{\infty }}t{\delta }^{t-1}{\int }_{x\ge {(1-\delta )}^{\frac{1}{2}-\epsilon }}{x}^{\vartheta -1}\exp (-\frac{{\displaystyle c{x}^2}}{{\displaystyle {(1-\delta )}^{2}t}})dx=0.$

We first establish (44). Since ${\sum }_{t=1}^{T}t{\delta }^{t-1}\le {(1-\delta )}^{\mathrm{-2}}$ and ${(1-\delta )}^{2}T={(1-\delta )}^{1-\epsilon }$, it suffices to show that $\underset{\delta \to 1}{\lim }{(1-\delta )}^{\mathrm{-3}}{\int }_{x\ge {(1-\delta )}^{\frac{1}{2}-\epsilon }}{x}^{\vartheta -1}\exp (-\frac{{\displaystyle c{x}^2}}{{\displaystyle {(1-\delta )}^{1-\epsilon }}})dx=0.$ If $\vartheta \le 1$, then $\begin{array}{rl}& {(1-\delta )}^{\mathrm{-3}}{\int }_{x\ge {(1-\delta )}^{\frac{1}{2}-\epsilon }}{x}^{\vartheta -1}\exp (-\frac{{\displaystyle c{x}^2}}{{\displaystyle {(1-\delta )}^{1-\epsilon }}})dx\\ & \le {(1-\delta )}^{(\frac{1}{2}-\epsilon )(\vartheta -1)-3}{\int }_{x\ge {(1-\delta )}^{\frac{1}{2}-\epsilon }}\exp (-\frac{{\displaystyle c{x}^2}}{{\displaystyle {(1-\delta )}^{1-\epsilon }}})dx\\ & \le {(1-\delta )}^{(\frac{1}{2}-\epsilon )(\vartheta -1)-3}\frac{{\displaystyle \sqrt{\pi }}}{{\displaystyle c}}{(1-\delta )}^{1/2}\exp (-c{(1-\delta )}^{-\epsilon })\stackrel{\delta \to 1}{⟶}0,\end{array}$ where the second inequality follows by the same calculation as (18). If instead $\vartheta >1$ then, for sufficiently large δ, we have $\begin{array}{rl}& {(1-\delta )}^{\mathrm{-3}}{\int }_{x\ge {(1-\delta )}^{\frac{1}{2}-\epsilon }}{x}^{\vartheta -1}\exp (-\frac{{\displaystyle c{x}^2}}{{\displaystyle {(1-\delta )}^{1-\epsilon }}})dx\\ & =\frac{{\displaystyle 1}}{{\displaystyle 2}}{(1-\delta )}^{\mathrm{-3}}{\int }_{y\ge 0}{(y+{(1-\delta )}^{1-2\epsilon })}^{\vartheta -1}\exp (-\frac{{\displaystyle c(y+{(1-\delta )}^{1-2\epsilon })}}{{\displaystyle {(1-\delta )}^{1-\epsilon }}})dy\\ & \le \frac{{\displaystyle 1}}{{\displaystyle 2}}{(1-\delta )}^{-3+\vartheta (1-2\epsilon )}\exp (-c{(1-\delta )}^{-\epsilon }){\int }_{y\ge 0}\exp \left(\right(\frac{{\displaystyle \vartheta -1}}{{\displaystyle {(1-\delta )}^{1-2\epsilon }}}-\frac{{\displaystyle c}}{{\displaystyle {(1-\delta )}^{1-\epsilon }}}\left)y\right)dy\\ & =\frac{{\displaystyle 1}}{{\displaystyle 2}}{(1-\delta )}^{-3+\vartheta (1-2\epsilon )}\exp (-c{(1-\delta )}^{-\epsilon }){(-\frac{{\displaystyle \vartheta -1}}{{\displaystyle {(1-\delta )}^{1-2\epsilon }}}+\frac{{\displaystyle c}}{{\displaystyle {(1-\delta )}^{1-\epsilon }}})}^{\mathrm{-1}}\stackrel{\delta \to 1}{⟶}0,\end{array}$ where the second follows by integration by substitution (setting $y=x^2-{(1-\delta )}^{1-2\epsilon }$), the third line follows because $\begin{array}{rl}{(y+{(1-\delta )}^{1-2\epsilon })}^{\vartheta -1}& ={(1-\delta )}^{(1-2\epsilon )(\vartheta -1)}\exp ((\vartheta -1)\log (\frac{{\displaystyle y}}{{\displaystyle {(1-\delta )}^{1-2\epsilon }}}+1\left)\right)\\ & \le {(1-\delta )}^{(1-2\epsilon )(\vartheta -1)}\exp \left(\frac{{\displaystyle (\vartheta -1)y}}{{\displaystyle {(1-\delta )}^{1-2\epsilon }}}\right),\end{array}$ and the fourth line follows because $\frac{\vartheta -1}{{(1-\delta )}^{1-2\epsilon }}-\frac{c}{{(1-\delta )}^{1-\epsilon }}<0$ for sufficiently large δ.

We next establish (45). For any t, we have $\begin{array}{rl}{\int }_{x\ge 0}{x}^{\vartheta -1}\exp (-\frac{{\displaystyle c{x}^2}}{{\displaystyle {(1-\delta )}^{2}t}})dx& =\frac{{\displaystyle 1}}{{\displaystyle 2}}{\left(\frac{{\displaystyle {(1-\delta )}^{2}t}}{{\displaystyle c}}\right)}^{\frac{\vartheta }{2}}{\int }_{y\ge 0}{y}^{\frac{\vartheta }{2}-1}\exp (-y)dy\\ & =\frac{{\displaystyle 1}}{{\displaystyle 2}}{\left(\frac{{\displaystyle {(1-\delta )}^{2}t}}{{\displaystyle c}}\right)}^{\frac{\vartheta }{2}}\mathrm{\Gamma }\left(\frac{{\displaystyle \vartheta }}{{\displaystyle 2}}\right),\end{array}$ where the first line follows by setting $y=c{x}^2/({(1-\delta )}^{2}t)$, and the second line follows by the definition of the gamma function, Γ. Hence, there exist constants $c_1$, $c_2$ such that ${\int }_{x\ge 0}{x}^{\vartheta -1}\exp (-\frac{{\displaystyle c{x}^2}}{{\displaystyle {(1-\delta )}^{2}t}})dx\le c_1{(1-\delta )}^{2c_2}{t}^{c_2}\text{for all }t.$ We thus have $\begin{array}{rl}& {(1-\delta )}^{\mathrm{-1}}\sum _{t=T+1}^{\mathrm{\infty }}t{\delta }^{t-1}{\int }_{x\ge {(1-\delta )}^{\frac{1}{2}-\epsilon }}{x}^{\vartheta -1}\exp (-\frac{{\displaystyle c{x}^2}}{{\displaystyle {(1-\delta )}^{2}t}})dx\\ & \le {(1-\delta )}^{\mathrm{-1}}\sum _{t=T+1}^{\mathrm{\infty }}t{\delta }^{t-1}{\int }_{x\ge 0}{x}^{\vartheta -1}\exp (-\frac{{\displaystyle c{x}^2}}{{\displaystyle {(1-\delta )}^{2}t}})dx\\ & \le c_1{(1-\delta )}^{2c_2-1}\sum _{t=T+1}^{\mathrm{\infty }}{\delta }^{t-1}{t}^{c_2+1}\\ & \le c_1{(1-\delta )}^{2c_2-1}{\delta }^{T-1}{(T+1)}^{c_2+1}\sum _{t=1}^{\mathrm{\infty }}{\delta }^t{t}^{c_2}\\ & \le c_1{(1-\delta )}^{c_2-2}{\delta }^{T-1}{(T+1)}^{c_2+1}k(c_2)\stackrel{\delta \to 1}{⟶}0,\end{array}$ where k is defined in (20) and the limit follows recalling that ${\delta }^T\le \exp (-{(1-\delta )}^{-\epsilon })$. □

We now establish inequality (15).

$22$

Lemma

We have ${\mathbb{E}}^{{\alpha }_t}\left[\frac{{\displaystyle {\psi }^{\prime }{(a)}^2}}{{\displaystyle \mathcal{I}(a)}}\right]\le {\mathbb{E}}^{\mu }\left[\frac{{\displaystyle {\psi }^{\prime }(a_t)\nu (y_t|a_t)}}{{\displaystyle \mathcal{I}(a_t)}}\sum _{t^{\prime }=t+1}^{\mathrm{\infty }}{\delta }^{t^{\prime }-t}{û}_{t^{\prime }}\right]\text{for all }t.$

$$

Proof

Fix any t and $\epsilon >0$. Since ${\psi }^{\prime }(a)/\mathcal{I}(a)$ is bounded (uniformly in a) given Assumption 3(ii), there exists $\tilde{\mathrm{\Delta }}>0$ such that, for all $\mathrm{\Delta }<\tilde{\mathrm{\Delta }}$, we have $({\psi }^{\prime }(a)/\mathcal{I}(a))\mathrm{\Delta }\le \epsilon$ for all $a\in A$. Fix $\overline{\mathrm{\Delta }}<\tilde{\mathrm{\Delta }}$ such that, for all $a\in A$, ${\int }_y({\max }_{ã\in [a,a+\overline{\mathrm{\Delta }}]}{p}_a{(y|ã)}^2/p(y|a))dy<\mathrm{\infty }$. (Such $\overline{\mathrm{\Delta }}$ exists by Assumption 3(i).) Now, for any $\mathrm{\Delta }<\overline{\mathrm{\Delta }}$, consider the manipulation where, whenever the agent is recommended action $a\ge \epsilon$ in period t, she instead takes $a-({\psi }^{\prime }(a)/\mathcal{I}(a))\mathrm{\Delta }$. This manipulation is unprofitable for the agent if and only if $\begin{array}{rcl}& & {\mathbb{E}}^{{\alpha }_t}[\mathbf{1}\{a\ge \epsilon \}(\psi (a)-\psi (a-\frac{{\displaystyle {\psi }^{\prime }(a)}}{{\displaystyle \mathcal{I}(a)}}\mathrm{\Delta })\left)\right]\\ & & \le {\mathbb{E}}^{\mu }[\mathbf{1}\{a_t\ge \epsilon \}\frac{{\displaystyle p(y_t|a_t)-p(y_t|a_t-\frac{{\displaystyle {\psi }^{\prime }(a_t)}}{{\displaystyle \mathcal{I}(a_t)}}\mathrm{\Delta })}}{{\displaystyle p(y|a_t)}}\sum _{t^{\prime }=t+1}^{\mathrm{\infty }}{\delta }^{t^{\prime }-t}{û}_{t^{\prime }}].\end{array}$ Since this holds for all $\mathrm{\Delta }<\overline{\mathrm{\Delta }}$, we have $\begin{array}{rcl}& & \underset{\mathrm{\Delta }\to 0}{\lim }{\mathbb{E}}^{{\alpha }_t}[\mathbf{1}\{a\ge \epsilon \}\frac{{\displaystyle \psi (a)-\psi (a-\frac{{\displaystyle {\psi }^{\prime }(a)}}{{\displaystyle \mathcal{I}(a)}}\mathrm{\Delta })}}{{\displaystyle \mathrm{\Delta }}}]\\ & & \le \underset{\mathrm{\Delta }\to 0}{\lim }{\mathbb{E}}^{\mu }[\mathbf{1}\{a_t\ge \epsilon \}\frac{{\displaystyle p(y_t|a_t)-p(y_t|a_t-\frac{{\displaystyle {\psi }^{\prime }(a_t)}}{{\displaystyle \mathcal{I}(a_t)}}\mathrm{\Delta })}}{{\displaystyle \mathrm{\Delta }p(y_t|a_t)}}\sum _{t^{\prime }=t+1}^{\mathrm{\infty }}{\delta }^{t^{\prime }-t}{û}_{t^{\prime }}].\end{array}$ In this inequality, the LHS is bounded because ${\psi }^{\prime }{(a)}^2/\mathcal{I}(a)$ is bounded given Assumption 3(ii), and hence is equal to ${\mathbb{E}}^{{\alpha }_t}[\mathbf{1}\{a\ge \epsilon \}{\psi }^{\prime }{(a)}^2/\mathcal{I}(a)]$, by dominated convergence. As for the RHS, for each $t^{\prime }\ge t+1$, by Cauchy–Schwarz the corresponding term in the sum is bounded by ${\mathbb{E}}^{\mu }{[\mathbf{1}\{a_t\ge \epsilon \}{\left(\frac{{\displaystyle p(y_t|a_t)-p(y_t|a_t-\frac{{\displaystyle {\psi }^{\prime }(a_t)}}{{\displaystyle \mathcal{I}(a_t)}}\mathrm{\Delta })}}{{\displaystyle \mathrm{\Delta }p(y|a_t)}}\right)}^2]}^{1/2}\times {\mathbb{E}}^{\mu }{\left[{û}_{t^{\prime }}^2\right]}^{1/2}.$ This is finite because ${\int }_y({\max }_{ã\in [a,a+\overline{\mathrm{\Delta }}]}{p}_a{(y|ã)}^2/p(y|a))dy<\mathrm{\infty }$ (by Assumption 3(i)) and ${\mathbb{E}}^{\mu }[{û}_{t^{\prime }}^2]<\mathrm{\infty }$ for all $t^{\prime }$ (as otherwise the principal's expected payoff would equal −∞ by Lemma 20). Hence, the entire RHS is bounded, and hence is equal to ${\mathbb{E}}^{\mu }[\mathbf{1}\{a_t\ge \epsilon \}({\psi }^{\prime }(a_t)\nu (y_t|a_t)/\mathcal{I}(a_t)){\sum }_{t^{\prime }=t+1}^{\mathrm{\infty }}{\delta }^{t^{\prime }-t}{û}_{t^{\prime }}]$, by dominated convergence. In total, we have ${\mathbb{E}}^{{\alpha }_t}[\mathbf{1}\{a\ge \epsilon \}\frac{{\displaystyle {\psi }^{\prime }{(a)}^2}}{{\displaystyle \mathcal{I}(a)}}]\le {\mathbb{E}}^{\mu }[\mathbf{1}\{a_t\ge \epsilon \}\frac{{\displaystyle {\psi }^{\prime }(a_t)\nu (y_t|a_t)}}{{\displaystyle \mathcal{I}(a_t)}}\sum _{t^{\prime }=t+1}^{\mathrm{\infty }}{\delta }^{t^{\prime }-t}{û}_{t^{\prime }}].$ Since this holds for all $\epsilon >0$, and ${\psi }^{\prime }{(a)}^2/\mathcal{I}(a)$ and ${\psi }^{\prime }(a_t)\nu (y_t|a_t)/\mathcal{I}(a_t)$ are continuous, taking $\epsilon \to 0$ completes the proof. □

Now we prove our key lemmas, Lemmas 16 and 17. These complete the proof that inefficiency is at least (14) in the blind game.

G.1

Proof of Lemma 16

Multiplying both sides of (38) by $2\delta /((1-\delta ){\overline{F}}^{\prime \prime }(w))$ and using (37), it suffices to find $c,\epsilon >0$ such that $\underset{{({û}_t(h))}_{t,h}\ge -(w+\psi (Ā))\text{ such that }(\text{15})}{\min }\sum _{t=1}^{\mathrm{\infty }}{\delta }^t\frac{{\displaystyle 2{\mathbb{E}}^{\mu }\left[\tilde{F}\right({û}_t(h)\left)\right]}}{{\displaystyle {\overline{F}}^{\prime \prime }(w)}}\ge {\mathbb{E}}^{\alpha }\left[\frac{{\displaystyle {\psi }^{\prime }{(a)}^2}}{{\displaystyle \mathcal{I}(a)}}\right]-c{(1-\delta )}^{\epsilon },$ where $\tilde{F}(û)=\overline{F}(w+û)-\overline{F}(w)-{\overline{F}}^{\prime }(w)û.$ By weak duality, taking a Lagrange multiplier of $2(1-\delta ){\delta }^{t-1}$ on (15) for each t, the LHS is no less than 46$\begin{array}{cc}\hfill & \underset{{({û}_t(h))}_{t,h}\ge -(w+\psi (Ā))}{\min }2\sum _{t=1}^{\mathrm{\infty }}{\delta }^t{\mathbb{E}}^{\mu }[\frac{{\displaystyle \tilde{F}({û}_t(h))}}{{\displaystyle {\overline{F}}^{\prime \prime }(w)}}-\underset{:={\mathrm{\Omega }}_t^{\delta }(h)}{\underbrace{\left(\frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}\sum _{t^{\prime }=1}^{t-1}\frac{{\displaystyle {\psi }^{\prime }(a_{t^{\prime }})\nu (y_t|a_{t^{\prime }})}}{{\displaystyle \mathcal{I}(a_{t^{\prime }})}}\right)}}{û}_t(h)]\hfill \\ \hfill & +2{\mathbb{E}}^{\alpha }\left[\frac{{\displaystyle {\psi }^{\prime }{(a)}^2}}{{\displaystyle \mathcal{I}(a)}}\right].\hfill \end{array}$

It remains to bound (46). Since $\tilde{F}$ is concave, the first-order necessary and sufficient condition for ${û}_t(h)$ is ${û}_t(h)=\{\begin{array}{cc}{\tilde{F}}^{\prime -1}({\overline{F}}^{\prime \prime }(w){\mathrm{\Omega }}_t^{\delta }(h))\hfill & \text{if }{\tilde{F}}^{\prime -1}({\overline{F}}^{\prime \prime }(w){\mathrm{\Omega }}_t^{\delta }(h))\ge -(w+\psi (Ā)),\hfill \\ -(w+\psi (Ā))\hfill & \text{if }{\tilde{F}}^{\prime -1}({\overline{F}}^{\prime \prime }(w){\mathrm{\Omega }}_t^{\delta }(h))<-(w+\psi (Ā)).\hfill \end{array}$ Now fix any $\epsilon \in (0,1/8)$ and let 47${\overline{H}}^t=\{h\in {(A\times Y)}^{\mathrm{\infty }}:-{(1-\delta )}^{\frac{1}{2}-\epsilon }\le {\mathrm{\Omega }}_t^{\delta }(h)\le {(1-\delta )}^{\frac{1}{2}-\epsilon }\}\text{for all }t.$ We establish two further lemmas.

$23$

Lemma

There exists $ĉ>0$ such that, for any sufficiently large $\delta <1$, we have 48$2(\frac{{\displaystyle \tilde{F}({û}_t(h))}}{{\displaystyle {\overline{F}}^{\prime \prime }(w)}}-{\mathrm{\Omega }}_t^{\delta }(h){û}_t(h))\ge -{\mathrm{\Omega }}_t^{\delta }{(h)}^2-ĉ{(1-\delta )}^{1+\epsilon }\text{for all }t\text{ and }h\in {\overline{H}}_t.$

$$

Proof

For sufficiently large δ, we have ${û}_t(h)={\tilde{F}}^{\prime -1}({\overline{F}}^{\prime \prime }(w){\mathrm{\Omega }}_t^{\delta }(h))$ for all t and $h\in {\overline{H}}^t$. Since $\tilde{F}\in C^2$ and ${\mathrm{\Omega }}_t^{\delta }(h)$ is bounded (uniformly in δ) for $h\in {\overline{H}}^t$, by Taylor expansion of ${\tilde{F}}^{\prime -1}$ and $\tilde{F}○{\tilde{F}}^{\prime -1}$ around 0, there exists $ĉ>0$ such that, for any $\delta <1$ and $h\in {\overline{H}}^t$, we have $\begin{array}{c}|{û}_t(h)-({\tilde{F}}^{\prime -1}(0)-\frac{{\displaystyle {\overline{F}}^{\prime \prime }(w)}}{{\displaystyle {\tilde{F}}^{\prime \prime }○{\tilde{F}}^{\prime -1}(0)}}{\mathrm{\Omega }}_t^{\delta }(h)\left)\right|\le \frac{{\displaystyle ĉ}}{{\displaystyle 3}}{\mathrm{\Omega }}_t^{\delta }{(h)}^2\text{and}\\ |\frac{{\displaystyle 2\tilde{F}({û}_t(h))}}{{\displaystyle {\overline{F}}^{\prime \prime }(w)}}-\left(\begin{array}{c}\hfill \frac{{\displaystyle 2\tilde{F}○{\tilde{F}}^{\prime -1}(0)}}{{\displaystyle {\overline{F}}^{\prime \prime }(w)}}-\frac{{\displaystyle 2{\tilde{F}}^{\prime }○{\tilde{F}}^{\prime -1}(0)}}{{\displaystyle {\tilde{F}}^{\prime \prime }○{\tilde{F}}^{\prime -1}(0)}}{\mathrm{\Omega }}_t^{\delta }(h)\hfill \\ \hfill +\frac{{\displaystyle {\overline{F}}^{\prime \prime }(w)}}{{\displaystyle {\tilde{F}}^{\prime \prime }○{\tilde{F}}^{\prime -1}(0)}}(1-\frac{{\displaystyle {\tilde{F}}^{\prime }○{\tilde{F}}^{\prime -1}(0)\times {\tilde{F}}^{\prime \prime \prime }○{\tilde{F}}^{\prime -1}(0)}}{{\displaystyle {({\tilde{F}}^{\prime \prime }○{\tilde{F}}^{\prime -1}(0))}^2}}){\mathrm{\Omega }}_t^{\delta }{(h)}^2\hfill \end{array}\right)|\le \frac{{\displaystyle ĉ}}{{\displaystyle 3}}|{\mathrm{\Omega }}_t^{\delta }(h){|}^3.\end{array}$ Since ${\tilde{F}}^{\prime -1}(0)={\tilde{F}}^{\prime }○{\tilde{F}}^{\prime -1}(0)=0$ and ${\tilde{F}}^{\prime \prime }○{\tilde{F}}^{\prime -1}(0)={\overline{F}}^{\prime \prime }(w)$ by definition of $\tilde{F}$, these inequalities simplify to $\begin{array}{rl}|{û}_t(h)-{\mathrm{\Omega }}_t^{\delta }(h)|& \le \frac{{\displaystyle ĉ}}{{\displaystyle 3}}{\mathrm{\Omega }}_t^{\delta }{(h)}^2\text{and}\\ |\frac{{\displaystyle 2\tilde{F}({û}_t(h))}}{{\displaystyle {\overline{F}}^{\prime \prime }(w)}}-{\mathrm{\Omega }}_t^{\delta }{(h)}^2|& \le \frac{{\displaystyle ĉ}}{{\displaystyle 3}}|{\mathrm{\Omega }}_t^{\delta }(h){|}^3.\end{array}$ Multiplying the first inequality by ${\mathrm{\Omega }}_t^{\delta }(h)$ and applying the triangle inequality give $\begin{array}{rl}|{\mathrm{\Omega }}_t^{\delta }(h){û}_t(h)-{\mathrm{\Omega }}_t^{\delta }{(h)}^2|& \le \frac{{\displaystyle ĉ}}{{\displaystyle 3}}{\mathrm{\Omega }}_t^{\delta }{(h)}^3\text{and}\\ |\frac{{\displaystyle 2\tilde{F}({û}_t(h))}}{{\displaystyle {\overline{F}}^{\prime \prime }(w)}}-{\mathrm{\Omega }}_t^{\delta }(h){û}_t(h)|& \le \frac{{\displaystyle 2ĉ}}{{\displaystyle 3}}|{\mathrm{\Omega }}_t^{\delta }(h){|}^3.\end{array}$ We thus have $\begin{array}{rl}2(\frac{{\displaystyle \tilde{F}({û}_t(h))}}{{\displaystyle {\overline{F}}^{\prime \prime }(w)}}-{\mathrm{\Omega }}_t^{\delta }(h){û}_t(h))& \ge -{\mathrm{\Omega }}_t^{\delta }(h){û}_t(h)-\frac{{\displaystyle 2ĉ}}{{\displaystyle 3}}|{\mathrm{\Omega }}_t^{\delta }(h){|}^3\\ & \ge -{\mathrm{\Omega }}_t^{\delta }{(h)}^2-ĉ|{\mathrm{\Omega }}_t^{\delta }(h){|}^3\\ & \ge -{\mathrm{\Omega }}_t^{\delta }{(h)}^2-ĉ{(1-\delta )}^{\frac{3}{2}-3\epsilon }\\ & \ge -{\mathrm{\Omega }}_t^{\delta }{(h)}^2-ĉ{(1-\delta )}^{1+\epsilon },\end{array}$ where the third line follows by (47), and the fourth line follows because $\epsilon <1/8$. □

$24$

Lemma

For any sufficiently large $\delta <1$, we have 49$2\sum _{t=1}^{\mathrm{\infty }}{\delta }^t{\int }_{h\notin {\overline{H}}^t}(\frac{{\displaystyle \tilde{F}({û}_t(h))}}{{\displaystyle {\overline{F}}^{\prime \prime }(w)}}-{\mathrm{\Omega }}_t^{\delta }(h){û}_t(h))d\mu (h)\ge -{(1-\delta )}^{\epsilon }.$

$$

Proof

We first show that, for sufficiently large δ, 50$\sum _{t=1}^{\mathrm{\infty }}{\delta }^t{\int }_{h\notin {\overline{H}}^t}{\mathrm{\Omega }}_t^{\delta }{(h)}^{2}d\mu (h)\le {(1-\delta )}^{2\epsilon }.$ To see this, note that ${({\psi }^{\prime }(a_t)\nu (y_t|a_t)/\mathcal{I}(a_t))}_t$ is a sequence of martingale increments where, for all $\theta >0$, $\begin{array}{rl}{\mathbb{E}}^{\mu }\left[\exp \right(\theta \frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}\frac{{\displaystyle {\psi }^{\prime }(a_t)\nu (y_t|a_t)}}{{\displaystyle \mathcal{I}(a_t)}}\left)\right|h^t]& \le \underset{a\in A}{\max }{\mathbb{E}}^{y\sim p(y|a)}\left[\exp \right(\theta \frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}\frac{{\displaystyle {\psi }^{\prime }(a)\nu (y|a)}}{{\displaystyle \mathcal{I}(a)}}\left)\right]\\ & \le \exp \left(\frac{{\displaystyle {\theta }^2}}{{\displaystyle 2}}{\left(\frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}\right)}^2\underset{a\in A}{\max }\frac{{\displaystyle {\psi }^{\prime }{(a)}^2}}{{\displaystyle \mathcal{I}(a)}}\right),\end{array}$ where the second line follows from Assumption 3(iii), noting that ${\psi }^{\prime }{(a)}^2/\mathcal{I}(a)$ is bounded. Hence, by Lemma 9, there exists $\tilde{c}>0$ such that, for all t and $x\ge 0$, we have ${\mathrm{Pr}}^{\mu }\left(\right|{\mathrm{\Omega }}_t^{\delta }(h)|\ge x)={\mathrm{Pr}}^{\mu }\left(\right|\frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}\sum _{t^{\prime }=1}^{t-1}\frac{{\displaystyle {\psi }^{\prime }(a_{t^{\prime }})\nu (y_{t^{\prime }}|a_{t^{\prime }})}}{{\displaystyle \mathcal{I}(a_{t^{\prime }})}}|\ge x)\le 2\exp (-\frac{{\displaystyle \tilde{c}{x}^2}}{{\displaystyle {(1-\delta )}^{2}t}}).$ We can thus apply Lemma 21 to the sequence ${(|{\mathrm{\Omega }}^{\delta }{|}_t)}_{t\ge 1}$ to conclude that (50) holds for sufficiently large δ.

Thus, for sufficiently large δ, we have $\begin{array}{rl}& 2\sum _{t=1}^{\mathrm{\infty }}{\delta }^t{\int }_{h\notin {\overline{H}}^t}(\frac{{\displaystyle \tilde{F}({û}_t(h))}}{{\displaystyle {\overline{F}}^{\prime \prime }(w)}}-{\mathrm{\Omega }}_t^{\delta }(h){û}_t(h))d\mu (h)\\ & \ge 2\sum _{t=1}^{\mathrm{\infty }}{\delta }^t{\int }_{h\notin {\overline{H}}^t}(-\frac{{\displaystyle {\epsilon }_2{û}_t{(h)}^2}}{{\displaystyle 2{\overline{F}}^{\prime \prime }(w)}}-{\mathrm{\Omega }}_t^{\delta }(h){û}_t(h))d\mu (h)\\ & \ge \frac{{\displaystyle {\overline{F}}^{\prime \prime }(w)}}{{\displaystyle {\epsilon }_2}}\sum _{t=1}^{\mathrm{\infty }}{\delta }^t{\int }_{h\notin {\overline{H}}^t}{\mathrm{\Omega }}_t^{\delta }{(h)}^{2}d\mu (h)\ge \frac{{\displaystyle {\overline{F}}^{\prime \prime }(w)}}{{\displaystyle {\epsilon }_2}}{(1-\delta )}^{2\epsilon }\ge -{(1-\delta )}^{\epsilon },\end{array}$ where the first inequality follows by Lemma 20 (as taking a second-order Taylor expansion gives $\tilde{F}(x)\le -({\epsilon }_2/2)x^2$ for all $x\ge -w$), the second follows by minimizing over $x_t(h)$, the third follows by (50), and the fourth follows by ${\overline{F}}^{\prime \prime }<0$. □

By (48), (49), and ${\mathrm{\Omega }}_t^{\delta }{(h)}^2>0$, we see that (46) is no less than $-\sum _{t=2}^{\mathrm{\infty }}{\delta }^t{\mathbb{E}}^{\mu }\left[{\mathrm{\Omega }}_t^{\delta }{(h)}^2\right]+2{\mathbb{E}}^{\alpha }\left[\frac{{\displaystyle {\psi }^{\prime }{(a)}^2}}{{\displaystyle \mathcal{I}(a)}}\right]-\max \{ĉ,1\}{(1-\delta )}^{\epsilon }.$ Since ${({\psi }^{\prime }(a_t)\nu (y_t|a_t)/\mathcal{I}(a_t))}_t$ is a sequence of martingale increments, we have ${\mathbb{E}}_{\mu }\left[{\mathrm{\Omega }}_t^{\delta }{(h)}^2\right]={\left(\frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}\right)}^2\sum _{t^{\prime }=1}^{t-1}{\mathbb{E}}^{\mu }\left[{\left(\frac{{\displaystyle {\psi }^{\prime }(a_{t^{\prime }})\nu (y_{t^{\prime }}|a_{t^{\prime }})}}{{\displaystyle \mathcal{I}(a_{t^{\prime }})}}\right)}^2\right]={\left(\frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}\right)}^2\sum _{t^{\prime }=1}^{t-1}{\mathbb{E}}^{{\alpha }_{t^{\prime }}}\left[\frac{{\displaystyle {\psi }^{\prime }{(a)}^2}}{{\displaystyle \mathcal{I}(a)}}\right].$ Therefore, (46) is no less than $\begin{array}{rl}& -{\left(\frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}\right)}^2\sum _{t=2}^{\mathrm{\infty }}{\delta }^t\sum _{t^{\prime }=1}^{t-1}{\mathbb{E}}^{{\alpha }_{t^{\prime }}}\left[\frac{{\displaystyle {\psi }^{\prime }{(a)}^2}}{{\displaystyle \mathcal{I}(a)}}\right]+2{\mathbb{E}}^{\alpha }\left[\frac{{\displaystyle {\psi }^{\prime }{(a)}^2}}{{\displaystyle \mathcal{I}(a)}}\right]-\max \{ĉ,1\}{(1-\delta )}^{\epsilon }\\ & ={\mathbb{E}}^{\alpha }\left[\frac{{\displaystyle {\psi }^{\prime }{(a)}^2}}{{\displaystyle \mathcal{I}(a)}}\right]-\max \{ĉ,1\}{(1-\delta )}^{\epsilon }.\end{array}$ Taking $c=\max \{ĉ,1\}$ completes the proof.

G.2

Proof of Lemma 17

If α assigns probability 1 to $a=ā(w)$, then $\begin{array}{rl}& \min \left\{\frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}{\mathbb{E}}^{\alpha }\right[\frac{{\displaystyle {\psi }^{\prime }{(a)}^2}}{{\displaystyle \mathcal{I}(a)}}]\frac{{\displaystyle {\overline{F}}^{\prime \prime }(w)}}{{\displaystyle 2}}+c{(1-\delta )}^{1+\epsilon },-{\mathcal{\epsilon }}_1{\mathbb{E}}^{\alpha }[{(a-ā(w))}^2\left]\right\}\\ & \ge \frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}\frac{{\displaystyle {\psi }^{\prime }{(ā(w))}^2}}{{\displaystyle \mathcal{I}(ā(w))}}\frac{{\displaystyle {\overline{F}}^{\prime \prime }(w)}}{{\displaystyle 2}}.\end{array}$ Hence, the optimal α satisfies 51${\mathbb{E}}^{\alpha }\left[{(a-ā(w))}^2\right]\le \frac{{\displaystyle 1}}{{\displaystyle {\mathcal{\epsilon }}_1}}\frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}\frac{{\displaystyle {\psi }^{\prime }{(ā(w))}^2}}{{\displaystyle \mathcal{I}(ā(w))}}\frac{{\displaystyle -{\overline{F}}^{\prime \prime }(w)}}{{\displaystyle 2}}.$

Since ${\psi }^{\prime }{(a)}^2/\mathcal{I}(a)$ is Lipschitz continuous, there exists $\kappa >0$ such that $\begin{array}{rl}& \frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}{\mathbb{E}}^{\alpha }\left[\frac{{\displaystyle {\psi }^{\prime }{(a)}^2}}{{\displaystyle \mathcal{I}(a)}}\right]\frac{{\displaystyle {\overline{F}}^{\prime \prime }(w)}}{{\displaystyle 2}}+c{(1-\delta )}^{1+\epsilon }\\ & \le \frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}(\frac{{\displaystyle {\psi }^{\prime }{(ā(w))}^2}}{{\displaystyle \mathcal{I}(ā(w))}}+(\kappa {\mathbb{E}}^{\alpha }\left[\right|a-ā(w)\left|\right]\left)\right)\frac{{\displaystyle {\overline{F}}^{\prime \prime }(w)}}{{\displaystyle 2}}+c{(1-\delta )}^{1+\epsilon }.\end{array}$ Note that $\begin{array}{rl}\frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}\frac{{\displaystyle -{\overline{F}}^{\prime \prime }(w)}}{{\displaystyle 2}}\kappa {\mathbb{E}}^{\alpha }\left[\right|a-ā(w)\left|\right]& \le \frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}\frac{{\displaystyle -{\overline{F}}^{\prime \prime }(w)}}{{\displaystyle 2}}\kappa \sqrt{{\mathbb{E}}^{\alpha }\left[{(a-ā(w))}^2\right]}\\ & \le {\left(\frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}\frac{{\displaystyle -{\overline{F}}^{\prime \prime }(w)}}{{\displaystyle 2}}\right)}^{\frac{3}{2}}\kappa \sqrt{\frac{{\displaystyle {\psi }^{\prime }{(ā(w))}^2}}{{\displaystyle {\mathcal{\epsilon }}_1\mathcal{I}(ā(w))}}},\end{array}$ where the first inequality follows by Cauchy–Schwarz, and the second follows by (51). Since this expression is of order ${(1-\delta )}^{3/2}$, there exists $\tilde{c}>0$ such that, for sufficiently large δ, we have $\frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}{\mathbb{E}}^{\alpha }\left[\frac{{\displaystyle {\psi }^{\prime }{(a)}^2}}{{\displaystyle \mathcal{I}(a)}}\right]\frac{{\displaystyle {\overline{F}}^{\prime \prime }(w)}}{{\displaystyle 2}}+c{(1-\delta )}^{1+\epsilon }\le \frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}\frac{{\displaystyle {\psi }^{\prime }{(ā(w))}^2}}{{\displaystyle \mathcal{I}(ā(w))}}\frac{{\displaystyle {\overline{F}}^{\prime \prime }(w)}}{{\displaystyle 2}}+\tilde{c}{(1-\delta )}^{1+\epsilon },$ completing the proof.

G.3

Construction of $x_a(y)$

$25$

Lemma

There exists $Ī<\mathrm{\infty }$ such that, for any sufficiently large $\delta <1$ and any $ā\in A$, there exists $x_{ā}:Y\to [-{(1-\delta )}^{-1/4},{(1-\delta )}^{-1/4}]$ satisfying 52$ā\in \underset{a\in A}{\mathrm{argmax}}{\int }_y{x}_{ā}(y)p(y|a)dy-\psi (a),$ 53${\int }_y{x}_{ā}(y)p(y|ā)dy=0,$ 54${\int }_y{x}_{ā}{(y)}^{2}p(y|ā)dy\le \frac{{\displaystyle 1}}{{\displaystyle \delta }}\frac{{\displaystyle {\psi }^{\prime }{(ā)}^2}}{{\displaystyle \mathcal{I}(ā)}},\text{and}$ 55${\int }_y\exp (\theta x(y|ā))p(y|ā)dy\le \exp \left({\theta }^2Ī\right).$

$$

Proof

Define, in turn, 56${\phi }_{ā}=\frac{{\displaystyle \mathcal{I}(ā)}}{{\displaystyle \mathbb{E}\left[\mathbf{1}\right\{|\nu (y|ā)|\le {(1-\delta )}^{-\frac{1}{5}}\}\nu {(y|ā)}^2]}},$ 57${\epsilon }_{ā}=-{\phi }_{ā}\frac{{\displaystyle {\psi }^{\prime }(ā)}}{{\displaystyle \mathcal{I}(ā)}}\mathbb{E}\left[\mathbf{1}\right\{|\nu (y|ā)|\le {(1-\delta )}^{-\frac{1}{5}}\}\nu (y|ā)],\text{and}$ $x_{ā}(y)={\phi }_{ā}\frac{{\displaystyle {\psi }^{\prime }(ā)}}{{\displaystyle \mathcal{I}(ā)}}\mathbf{1}\left\{\right|\nu (y|ā)|\le {(1-\delta )}^{-\frac{1}{5}}\}\nu (y|ā)+{\epsilon }_{ā}.$ Note that ${\phi }_{ā}\ge 1$, since $\mathcal{I}(ā)=\mathbb{E}[\nu {(y|ā)}^2]$. We will prove that $x_{ā}(y)\in [-{(1-\delta )}^{-1/4},{(1-\delta )}^{-1/4}]$ for all y, and that (52)–(55) hold. We first establish that, for any sufficiently large $\delta <1$ and any $ā\in [0,Ā]$, we have 58$|{\phi }_{ā}-1|\le \exp (-{(1-\delta )}^{\frac{1}{4}})\text{and}$ 59$|{\epsilon }_{ā}|\le \frac{{\displaystyle {\psi }^{\prime }(ā)}}{{\displaystyle \sqrt{\mathcal{I}(ā)}}}\exp (-{(1-\delta )}^{-\frac{1}{5}}).$ Note that (58) and (59) immediately imply that $x_{ā}(y)\in [-{(1-\delta )}^{-1/4},{(1-\delta )}^{-1/4}]$.

For (58), note that $\begin{array}{rl}0& \le {\phi }_{ā}=\frac{{\displaystyle \mathcal{I}(ā)}}{{\displaystyle \mathcal{I}(ā)-\mathbb{E}\left[1\right\{|\nu (y|ā)|>{(1-\delta )}^{-\frac{1}{5}}\}\nu {(y|ā)}^2]}}\\ & \le \frac{{\displaystyle \mathcal{I}(ā)}}{{\displaystyle \mathcal{I}(ā)-\sqrt{\mathrm{Pr}\left(\right|\nu (y|ā)|>{(1-\delta )}^{-\frac{1}{5}}\left)\mathbb{E}\right[\nu {(y|ā)}^4]}}}\end{array}$ where the second line follows by Cauchy–Schwarz. By Assumption 3(iii), we have 60${\int }_y\exp (\theta \nu (y|ā))p(y|ā)dy\le \exp ({\theta }^2\mathcal{I}(ā)/2)\text{for all }ā\in A\text{ and }\theta \in \mathbb{R}.$ Since $\mathcal{I}(ā)$ is uniformly bounded in ā given Assumption 3(ii), there exists $c>0$ such that, for all $ā\in A$, ${\phi }_{ā}$ is bounded by $\frac{{\displaystyle \mathcal{I}(ā)}}{{\displaystyle \mathcal{I}(ā)-\exp (-\frac{{\displaystyle {(1-\delta )}^{-\frac{2}{5}}}}{{\displaystyle c}})16\mathrm{\Gamma }(2)\mathcal{I}(ā)}}=\frac{{\displaystyle 1}}{{\displaystyle 1-\exp (-\frac{{\displaystyle {(1-\delta )}^{-\frac{2}{5}}}}{{\displaystyle c}})16\mathrm{\Gamma }(2)}},$ which implies (58).

For (59), note that $\begin{array}{rl}|{\epsilon }_{ā}|& =\frac{{\displaystyle {\psi }^{\prime }(ā)}}{{\displaystyle \mathcal{I}(ā)}}\left|\mathbb{E}\right[{\phi }_{ā}\mathbf{1}\left\{\right|\nu (y|ā)|\le {(1-\delta )}^{-\frac{1}{5}}\}\nu (y|ā)\left]\right|\\ & \le \frac{{\displaystyle {\psi }^{\prime }(ā)}}{{\displaystyle \mathcal{I}(ā)}}\left(\begin{array}{c}\hfill \left|\mathbb{E}\right[\mathbf{1}\left\{\right|\nu (y|ā)|>{(1-\delta )}^{-\frac{1}{5}}\}\nu (y|ā)\left]\right|\hfill \\ \hfill +({\phi }_{ā}-1)\left|\mathbb{E}\right[\mathbf{1}\left\{\right|\nu (y|ā)|\le {(1-\delta )}^{-\frac{1}{5}}\}\nu (y|ā)\left]\right|\hfill \end{array}\right)\end{array}$ where the inequality is by $\mathbb{E}[\nu (y|ā)]=0$ and the triangle inequality. As above, applying Cauchy–Schwarz and Assumption 3(ii) and (iii), we have $\frac{{\displaystyle {\psi }^{\prime }(ā)}}{{\displaystyle \mathcal{I}(ā)}}\left|\mathbb{E}\right[\mathbf{1}\left\{\right|\nu (y|ā)|>{(1-\delta )}^{-\frac{1}{5}}\}\nu (y|ā)\left]\right|\le \exp (-\frac{{\displaystyle {(1-\delta )}^{-\frac{2}{5}}}}{{\displaystyle c}})\frac{{\displaystyle {\psi }^{\prime }(ā)}}{{\displaystyle \sqrt{\mathcal{I}(ā)}}}.$ Again applying Cauchy–Schwarz, together with (58), we have $\frac{{\displaystyle {\psi }^{\prime }(ā)}}{{\displaystyle \mathcal{I}(ā)}}({\phi }_{ā}-1)\left|\mathbb{E}\right[\mathbf{1}\left\{\right|\nu (y|ā)|\le {(1-\delta )}^{-\frac{1}{5}}\}\nu (y|ā)\left]\right|\le \frac{{\displaystyle {\psi }^{\prime }(ā)}}{{\displaystyle \sqrt{\mathcal{I}(ā)}}}\exp (-{(1-\delta )}^{\frac{1}{4}}).$ Taking $\delta <1$ sufficiently large so that $\exp (-{(1-\delta )}^{-2/5}/c)+\exp (-{(1-\delta )}^{1/4})\le \exp (-{(1-\delta )}^{1/5})$, we have (59).

We now establish (52)–(55). Note that (53) follows directly from (57). For (52), for any $a\ne ā$, we have 61$\begin{array}{cc}\hfill & {\mathbb{E}}^{ā}[x_{ā}(y)]-\psi (ā)-\left({\mathbb{E}}^a\right[x_{ā}(y)]-\psi (a))\hfill \\ \hfill & =\psi (a)-\psi (ā)-{\phi }_{ā}\frac{{\displaystyle {\psi }^{\prime }(ā)}}{{\displaystyle \mathcal{I}(ā)}}{\int }_y\mathbf{1}\left\{\right|\nu (y|ā)|\le {(1-\delta )}^{-\frac{1}{5}}\}\nu (y|ā)\left(\begin{array}{c}\hfill p(y|a)\hfill \\ \hfill -p(y|ā)\hfill \end{array}\right)dy.\hfill \end{array}$ We bound the second line as follows. For any $\gamma \in (0,{\inf }_{a\in Ā}{\psi }^{\prime \prime }(a))$ and any $ā\in A$, we have $\psi (a)-\psi (ā)\ge {\psi }^{\prime }(a)(a-ā)+\frac{{\displaystyle \gamma }}{{\displaystyle 2}}{(a-ā)}^2.$ Taking a second-order Taylor expansion of $p(y|a)$ around $a=ā$, there exists $â\in A$ such that $\begin{array}{rl}{\phi }_{ā}\frac{{\displaystyle {\psi }^{\prime }(ā)}}{{\displaystyle \mathcal{I}(ā)}}{\int }_y\mathbf{1}\left\{\right|\nu (y|ā)|& \le {(1-\delta )}^{-\frac{1}{5}}\}\nu (y|ā)(p(y|a)-p(y|ā))dy\\ & =(a-ā){\phi }_{ā}\frac{{\displaystyle {\psi }^{\prime }(ā)}}{{\displaystyle \mathcal{I}(ā)}}{\int }_y\mathbf{1}\left\{\right|\nu (y|ā)|\le {(1-\delta )}^{-\frac{1}{5}}\}\nu {(y|ā)}^{2}p(y|a)dy\\ & +\frac{{\displaystyle {(a-ā)}^2}}{{\displaystyle 2}}{\phi }_{ā}\frac{{\displaystyle {\psi }^{\prime }(ā)}}{{\displaystyle \mathcal{I}(ā)}}{\int }_y\mathbf{1}\left\{\right|\nu (y|ā)|\le {(1-\delta )}^{-\frac{1}{5}}\}\nu (y|ā)p_{aa}(y|â)dy.\end{array}$ Substituting (56), (61) is no less than ${(a-ā)}^2/2$ multiplied by $\gamma -{\phi }_{ā}\frac{{\displaystyle {\psi }^{\prime }(ā)}}{{\displaystyle \mathcal{I}(ā)}}{\int }_y\mathbf{1}\left\{\right|\nu (y|ā)|\le {(1-\delta )}^{-\frac{1}{5}}\}\nu (y|ā)p_{aa}(y|â)dy.$ It remains to show that, for any sufficiently large $\delta <1$ and any ā, â, this expression is nonnegative. Since ${\psi }^{\prime }(ā)/\mathcal{I}(ā)$ is bounded, by (11) it suffices to show that $\underset{\delta \to 1}{\lim }\underset{ā,â}{\sup }{\int }_y\left({\phi }_{ā}\mathbf{1}\right\{|\nu (y|ā)|\le {(1-\delta )}^{-\frac{1}{5}}\}-1)\nu (y|ā)p_{aa}(y|â)dy=0.$ In turn, it suffices to show that both $\begin{array}{c}{\int }_y\mathbf{1}\left\{\right|\nu (y|ā)|>{(1-\delta )}^{-\frac{1}{5}}\}\nu (y|ā)p_{aa}(y|â)dy\text{and}\\ ({\phi }_{ā}-1){\int }_y\mathbf{1}\left\{\right|\nu (y|ā)|\le {(1-\delta )}^{-\frac{1}{5}}\}\nu (y|ā)p_{aa}(y|â)dy\end{array}$ converge to 0 as $\delta \to 1$, uniformly in $(ā,â)$.

By Cauchy–Schwarz, the first line is bounded by $\mathrm{Pr}{\left(\right|\nu (y|ā)|>{(1-\delta )}^{-\frac{1}{5}})}^{\frac{1}{4}}{({\int }_y\nu {(y|ā)}^{4}p(y|ā)dy)}^{\frac{1}{4}}{({\int }_y{\left(\frac{{\displaystyle p_{aa}(y|â)}}{{\displaystyle p(y|ā)}}\right)}^{2}p(y|ā)dy)}^{\frac{1}{2}}.$ By (60), the first term of this product converges to 0 uniformly in ā as $\delta \to 1$, and the second term is bounded uniformly in ā given Assumptions 2(i) and (ii). Moreover, (12) ensures the last term is bounded uniformly in $(ā,â)$. So, the entire product converges to 0 uniformly in $(ā,â)$.

Similarly, again by Cauchy–Schwarz, the second line is bounded by $({\phi }_{ā}-1){({\int }_y\nu {(y|ā)}^{2}p(y|ā)dy)}^{\frac{1}{2}}{({\int }_y{\left(\frac{{\displaystyle p_{aa}(y|â)}}{{\displaystyle p(y|ā)}}\right)}^{2}p(y|ā)dy)}^{\frac{1}{2}}.$ By (58), the first term of this product converges to 0 uniformly in ā as $\delta \to 1$; and, as above, the other terms are bounded uniformly in $(ā,â)$. The product thus converges to 0 uniformly in $(ā,â)$. This establishes (52).

We next establish (54). By construction, we have $\begin{array}{rl}\mathbb{E}\left[x_{ā}{(y)}^2\right]-\frac{{\displaystyle 1}}{{\displaystyle \delta }}\frac{{\displaystyle {\psi }^{\prime }{(ā)}^2}}{{\displaystyle \mathcal{I}(ā)}}& ={\phi }_{ā}^2{\left(\frac{{\displaystyle {\psi }^{\prime }(ā)}}{{\displaystyle \mathcal{I}(ā)}}\right)}^2\mathbb{E}\left[\mathbf{1}\right\{|\nu (y|ā)|\le {(1-\delta )}^{-\frac{1}{5}}\}\nu {(y|ā)}^2]\\ & +2{\epsilon }_{ā}{\phi }_{ā}\frac{{\displaystyle {\psi }^{\prime }(ā)}}{{\displaystyle \mathcal{I}(ā)}}\mathbb{E}\left[\mathbf{1}\right\{|\nu (y|ā)|\le {(1-\delta )}^{-\frac{1}{5}}\}\nu (y|ā)]+{\epsilon }_{ā}^2-\frac{{\displaystyle 1}}{{\displaystyle \delta }}\frac{{\displaystyle {\psi }^{\prime }{(ā)}^2}}{{\displaystyle \mathcal{I}(ā)}}.\end{array}$ By (56), ${\left(\frac{{\displaystyle {\psi }^{\prime }(ā)}}{{\displaystyle \mathcal{I}(ā)}}\right)}^2{\phi }_{ā}\mathbb{E}\left[\mathbf{1}\right\{|\nu (y|ā)|\le {(1-\delta )}^{-\frac{1}{5}}\}\nu {(y|ā)}^2]=\frac{{\displaystyle {\psi }^{\prime }{(ā)}^2}}{{\displaystyle \mathcal{I}(ā)}}.$ Thus, the above expression equals $\begin{array}{rl}& {\phi }_{ā}({\phi }_{ā}-1){\left(\frac{{\displaystyle {\psi }^{\prime }(ā)}}{{\displaystyle \mathcal{I}(ā)}}\right)}^2\mathbb{E}\left[1\right\{|\nu (y|ā)|\le {(1-\delta )}^{-\frac{1}{5}}\}\nu {(y|ā)}^2]\\ & +2{\epsilon }_{ā}{\phi }_{ā}\frac{{\displaystyle {\psi }^{\prime }(ā)}}{{\displaystyle \mathcal{I}(ā)}}\mathbb{E}\left[1\right\{|\nu (y|ā)|\le {(1-\delta )}^{-\frac{1}{5}}\}\nu (y|ā)]+{\epsilon }_{ā}^2-\frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}\frac{{\displaystyle {\psi }^{\prime }{(ā)}^2}}{{\displaystyle \mathcal{I}(ā)}}.\end{array}$ Together with $\begin{array}{rl}{\left(\frac{{\displaystyle {\psi }^{\prime }(ā)}}{{\displaystyle \mathcal{I}(ā)}}\right)}^2\mathbb{E}\left[1\right\{|\nu (y|ā)|\le {(1-\delta )}^{-\frac{1}{5}}\}\nu {(y|ā)}^2]& \le \frac{{\displaystyle {\psi }^{\prime }{(ā)}^2}}{{\displaystyle \mathcal{I}(ā)}},\\ \frac{{\displaystyle {\psi }^{\prime }(ā)}}{{\displaystyle \mathcal{I}(ā)}}\mathbb{E}\left[1\right\{|\nu (y|ā)|\le {(1-\delta )}^{-\frac{1}{5}}\}\nu (y|ā)]& \le \frac{{\displaystyle {\psi }^{\prime }(ā)}}{{\displaystyle \mathcal{I}(ā)}}\sqrt{\mathbb{E}\left[\nu {(y|ā)}^2\right]}=\frac{{\displaystyle {\psi }^{\prime }(ā)}}{{\displaystyle \sqrt{\mathcal{I}(ā)}}},\end{array}$ (58), and (59), this in turn is bounded by ${\psi }^{\prime }{(ā)}^2/\mathcal{I}(ā)$ multiplied by ${\phi }_{ā}\exp (-{(1-\delta )}^{\frac{1}{4}})+2{\phi }_{ā}\exp (-{(1-\delta )}^{-\frac{1}{5}})+\exp (-2{(1-\delta )}^{-\frac{1}{5}})-\frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }},$ which is nonpositive uniformly in ā for sufficiently large $\delta <1$.

We finally establish (55). It suffices to show that, for any $I\ge {\sup }_{a\in A}{\psi }^{\prime }{(ā)}^2/\mathcal{I}(ā)$, we have $\mathrm{Pr}\left(\right|x_{ā}(y)|\ge \lambda )\le 2\exp (-{\lambda }^2/(4I)).$ This is immediate if $\lambda \le \sqrt{(\log 2)(4I)}$. Next, for any sufficiently large δ, $ā\in A$, and $\lambda \ge \sqrt{(\log 2)(4I)}$, we have $\lambda -{\epsilon }_{ā}>0$ and $-\lambda -{\epsilon }_{ā}<0$ by (59). Hence, we have $\begin{array}{rl}\mathrm{Pr}\left(\right|x_{ā}(y)|\ge \lambda )& =\mathrm{Pr}\left(\frac{{\displaystyle {\psi }^{\prime }(ā)}}{{\displaystyle \mathcal{I}(ā)}}{\phi }_{ā}\mathbf{1}\right\{|\nu (y|ā)|\le {(1-\delta )}^{-\frac{1}{5}}\}\nu (y|ā)\le -\lambda -{\epsilon }_{ā})\\ & +\mathrm{Pr}\left(\frac{{\displaystyle {\psi }^{\prime }(ā)}}{{\displaystyle \mathcal{I}(ā)}}{\phi }_{ā}\mathbf{1}\right\{|\nu (y|ā)|\le {(1-\delta )}^{-\frac{1}{5}}\}\nu (y|ā)\ge \lambda -{\epsilon }_{ā})\\ & \le \mathrm{Pr}(\frac{{\displaystyle {\psi }^{\prime }(ā)}}{{\displaystyle \mathcal{I}(ā)}}\nu (y|ā)\le \frac{{\displaystyle -\lambda +{\epsilon }_{ā}}}{{\displaystyle {\phi }_{ā}}})+\mathrm{Pr}(\frac{{\displaystyle {\psi }^{\prime }(ā)}}{{\displaystyle \mathcal{I}(ā)}}\nu (y|ā)\ge \frac{{\displaystyle \lambda -{\epsilon }_{ā}}}{{\displaystyle {\phi }_{ā}}}).\end{array}$ Since $\nu (y|ā)$ is sub-Gaussian with variance-proxy $\mathcal{I}(ā)$ by Assumption 3(iii), we have $\mathrm{Pr}(\frac{{\displaystyle {\psi }^{\prime }(ā)}}{{\displaystyle \mathcal{I}(ā)}}\nu (y|ā)\le \frac{{\displaystyle -\lambda +{\epsilon }_{ā}}}{{\displaystyle {\phi }_{ā}}})+\mathrm{Pr}(\frac{{\displaystyle {\psi }^{\prime }(ā)}}{{\displaystyle \mathcal{I}(ā)}}\nu (y|ā)\ge \frac{{\displaystyle \lambda -{\epsilon }_{ā}}}{{\displaystyle {\phi }_{ā}}})\le 2\exp (-\frac{{\displaystyle {\left(\frac{{\displaystyle \lambda -{\epsilon }_{ā}}}{{\displaystyle {\phi }_{ā}}}\right)}^2}}{{\displaystyle 2\frac{{\displaystyle {\psi }^{\prime }{(ā)}^2}}{{\displaystyle \mathcal{I}(ā)}}}}).$ Finally, note that $\left(\frac{{\displaystyle {\left(\frac{{\displaystyle \lambda -{\epsilon }_{ā}}}{{\displaystyle {\phi }_{ā}}}\right)}^2}}{{\displaystyle 2\frac{{\displaystyle {\psi }^{\prime }{(ā)}^2}}{{\displaystyle \mathcal{I}(ā)}}}}\right)/\left(\frac{{\displaystyle {\lambda }^2}}{{\displaystyle 4I}}\right)\ge 2(1-\frac{{\displaystyle 2{\epsilon }_{ā}}}{{\displaystyle \lambda }})\frac{{\displaystyle 1}}{{\displaystyle {\phi }_{ā}^2}}\frac{{\displaystyle 1}}{{\displaystyle \frac{{\displaystyle {\psi }^{\prime }{(ā)}^2}}{{\displaystyle \mathcal{I}(ā)}}/I}}\ge 2(1-\frac{{\displaystyle 2{\epsilon }_{ā}}}{{\displaystyle \sqrt{(\log 2)(4I)}}})\frac{{\displaystyle 1}}{{\displaystyle {\phi }_{ā}^2}},$ which is greater than one uniformly in ā for sufficiently large δ, by (58) and (59). We thus have $\mathrm{Pr}(\frac{{\displaystyle {\psi }^{\prime }(ā)}}{{\displaystyle \mathcal{I}(ā)}}\nu (y|ā)\le \frac{{\displaystyle -\lambda -{\epsilon }_{ā}}}{{\displaystyle {\phi }_{ā}}})+\mathrm{Pr}(\frac{{\displaystyle {\psi }^{\prime }(ā)}}{{\displaystyle \mathcal{I}(ā)}}\nu (y|ā)\ge \frac{{\displaystyle \lambda -{\epsilon }_{ā}}}{{\displaystyle {\phi }_{ā}}})\le 2\exp (-\frac{{\displaystyle {\lambda }^2}}{{\displaystyle 4I}}),$ as desired. □

G.4

Equilibrium verification

We verify that the contract defined in the main Appendix, with $x_a(y)$ defined as in Lemma 25, satisfies incentive compatibility and promise keeping, and hence is an equilibrium.

$26$

Lemma

For each $h^t$, we have 62$a_t\left(h^t\right)\in \underset{a}{\mathrm{argmax}}(1-\delta )\left(u\right(c_t\left(h^t\right))-\psi (a))+\delta \mathbb{E}\left[w_{t+1}\right(h^{t+1})|h^t,a],$ 63$w_t\left(h^t\right)=(1-\delta )\left(u\right(c_t\left(h^t\right))-\psi (a_t\left(h^t\right)\left)\right)+\delta \mathbb{E}\left[w_{t+1}\right(h^{t+1})|h^t,a_t\left(h^t\right)].$

$$

Proof

The conclusion is immediate if $h^t$ is irregular. If $h^t$ is regular, then (62) follows from (52) since $\begin{array}{cc}\hfill & \underset{a}{\mathrm{argmax}}(1-\delta )\left(u\right(c_t\left(h^t\right))-\psi (a))+\delta \mathbb{E}\left[w_{t+1}\right(h^{t+1})|h^t,a]\hfill \\ \hfill & =\underset{a}{\mathrm{argmax}}\mathbb{E}[x_{ā(w_t(h^t))}(y_t)|h^t,a]-\psi (a).\hfill \end{array}$ Moreover, (63) holds as $u(\overline{c}(w_t(h^t)))-\psi (ā(w_t(h^t)))=w_t(h^t)$ by definition of $\overline{c}$ and ā, and $\mathbb{E}[w_{t+1}(h^{t+1})|h^t,ā(w_t(h^t))]=w_t(h^t)$ by (53). □

G.5

Proof of Lemma 18

Let ${Ĥ}^t=\{h^t:|w_t(h^t)-w|\le {(1-\delta )}^{1/2-\epsilon }\}$ be the set of regular period t histories. Let $x_{a_t}^{h^t}(y_t)=\{\begin{array}{cc}{x}_{a_t}(y_t)\hfill & \text{if }{h}^t\in {Ĥ}^t,\hfill \\ 0\hfill & \text{if }{h}^t\notin {Ĥ}^t,\hfill \end{array}\text{and}X\left(h^t\right)=\frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}\sum _{t^{\prime }=1}^{t-1}{x}_{a_{t^{\prime }}}^{h^{t^{\prime }}}(y_{t^{\prime }}),$ where in the latter definition $h^{t^{\prime }}$ is the period $t^{\prime }$ truncation of $h^t$. Note that $w_t(h^t)=w+X(h^t)$ for all t and $h^t$, and that $h^t\in {Ĥ}^t$ if and only if $|X(h^t)|\le {(1-\delta )}^{1/2-\epsilon }$.

We first bound the weight on irregular histories under the equilibrium outcome μ.

$27$

Lemma

For any sufficiently large $\delta <1$, we have $(1-\delta ){\sum }_{t=1}^{\mathrm{\infty }}{\delta }^{t-1}{\mathrm{Pr}}^{\mu }(h^t\notin {Ĥ}^t)\le {(1-\delta )}^2$.

$$

Proof

Note that $x_{a_t}^{h^t}(y_t)$ is a sequence of martingale increments. Moreover, by (55), ${\mathbb{E}}^{\mu }\left[\exp \right(\theta \frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}{x}_{a_t}^{h^t}(y_t))|h^t,a_t]\le \exp ({\theta }^2{\left(\frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}\right)}^2Ī)\text{for all }\theta ,t,a_t.$ Therefore, by Lemma 9, ${\mathrm{Pr}}^{\mu }\left(X\right(h^t)>x)\le 2\exp (-\frac{{\displaystyle x^2}}{{\displaystyle 2{\left(\frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}\right)}^2Īt}})\text{for all }x>0.$ We can now apply Lemma 21 with $\vartheta =0$ to conclude that, for any sufficiently large δ, $(1-\delta )\sum _{t=1}^{\mathrm{\infty }}{\delta }^{t-1}{\mathrm{Pr}}^{\mu }(h^t\notin {Ĥ}^t)=(1-\delta )\sum _{t=1}^{\mathrm{\infty }}{\delta }^{t-1}{\mathrm{Pr}}^{\mu }\left(\right|X\left(h^t\right)|>{(1-\delta )}^{1/2-\epsilon })\le {(1-\delta )}^2.$ □

Recall that $\epsilon <1/8$. By Taylor expansion, since $3(1/2-\epsilon )>1+\epsilon$ and $|w_t(h^t)-w|\le {(1-\delta )}^{1/2-\epsilon }$ for all $h^t\in {Ĥ}^t$, for any sufficiently large $\delta <1$ and any $h^t\in {Ĥ}^t$, we have $\begin{array}{rl}{a}_t\left(h^t\right)-c_t\left(h^t\right)& =ā\left(w_t\right(h^t\left)\right)-\overline{c}\left(w_t\right(h^t\left)\right)\\ & =\overline{F}\left(w_t\right(h^t\left)\right)\\ & \ge \overline{F}(w)+{\overline{F}}^{\prime }(w)\left(w_t\right(h^t)-w)+\frac{{\displaystyle {\overline{F}}^{\prime \prime }(w)}}{{\displaystyle 2}}{\left(w_t\right(h^t)-w)}^2-{(1-\delta )}^{1+\epsilon }\\ & =\overline{F}(w)+{\overline{F}}^{\prime }(w)\left(w_t\right(h^t)-w)+\frac{{\displaystyle {\overline{F}}^{\prime \prime }(w)}}{{\displaystyle 2}}X{\left(h^t\right)}^2-{(1-\delta )}^{1+\epsilon }.\end{array}$

At the same time, since $w_t(h^t)\in [0,2w]$, $a_t(h^t)-c_t(h^t)$ and $\overline{F}(w)+{\overline{F}}^{\prime }(w)(w_t(h^t)-w)$ are bounded, and ${\overline{F}}^{\prime \prime }(w)\le 0$, there exists $c_1>0$ such that, for any δ and $h^t\notin {Ĥ}^t$, we have $a_t\left(h^t\right)-c_t\left(h^t\right)\ge \overline{F}(w)+{\overline{F}}^{\prime }(w)\left(w_t\right(h^t)-w)+\frac{{\displaystyle {\overline{F}}^{\prime \prime }(w)}}{{\displaystyle 2}}X{\left(h^t\right)}^2-c_1.$

Combining these bounds, we have 64$\begin{array}{rl}& (1-\delta )\sum _{t=1}^{\mathrm{\infty }}{\delta }^{t-1}{\mathbb{E}}^{\mu }\left[a_t\right(h^t)-c_t(h^t\left)\right]\\ & \ge \overline{F}(w)+\frac{{\displaystyle {\overline{F}}^{\prime \prime }(w)}}{{\displaystyle 2}}(1-\delta )\sum _{t=2}^{\mathrm{\infty }}{\delta }^{t-1}{\int }_{h}X{\left(h^t\right)}^{2}d\mu (h)\\ & -{(1-\delta )}^{1+\epsilon }-c_1(1-\delta )\sum _{t=1}^{\mathrm{\infty }}{\delta }^{t-1}{\mathrm{Pr}}^{\mu }(h_t\notin {Ĥ}^t).\end{array}$ Moreover, since ${(x_{a_t}(y_t))}_t$ is a sequence of martingale increments with variance bounded by (54), we have $\begin{array}{rl}& (1-\delta )\sum _{t=2}^{\mathrm{\infty }}{\delta }^{t-1}\sum _h\mu (h){(\frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}\sum _{t^{\prime }=1}^{t-1}{x}_{a_{t^{\prime }}}(y_{t^{\prime }}))}^2\\ & =\frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}((1-\delta )\sum _{t=1}^{\mathrm{\infty }}{\delta }^{t-1}{\mathbb{E}}^{a\sim {\alpha }_t}[{\mathbb{E}}^{y\sim p(y|a)}\left[x_a{(y)}^2\right]\left]\right)\le \frac{{\displaystyle 1-\delta }}{{\displaystyle {\delta }^2}}{\mathbb{E}}^{\alpha }\left[\frac{{\displaystyle {\psi }^{\prime }{(a)}^2}}{{\displaystyle \mathcal{I}(a)}}\right].\end{array}$ Together with Lemma 27, (64) now implies that the principal's payoff is no less than $\overline{F}(w)+\frac{{\displaystyle {\overline{F}}^{\prime \prime }(w)}}{{\displaystyle 2}}\frac{{\displaystyle 1-\delta }}{{\displaystyle {\delta }^2}}{\mathbb{E}}^{\alpha }\left[\frac{{\displaystyle {\psi }^{\prime }{(a)}^2}}{{\displaystyle \mathcal{I}(a)}}\right]-{(1-\delta )}^{1+\epsilon }-{(1-\delta )}^2.$

It remains to bound ${\mathbb{E}}^{\alpha }[{\psi }^{\prime }{(a)}^2/\mathcal{I}(a)]$. Since ${\psi }^{\prime }{(a)}^2/\mathcal{I}(a)$ is Lipschitz continuous, there exists $\kappa >0$ such that ${\mathbb{E}}^{\alpha }\left[\frac{{\displaystyle {\psi }^{\prime }{(a)}^2}}{{\displaystyle \mathcal{I}(a)}}\right]-\frac{{\displaystyle {\psi }^{\prime }{(ā(w))}^2}}{{\displaystyle \mathcal{I}(ā(w))}}\ge -\kappa {\mathbb{E}}^{\alpha }\left[\right|a-ā(w)\left|\right].$ Since $ā(w)$ is continuously differentiable and $w\in [0,2w]$, there exists $c_2$ such that, for any t and $h^t\in {Ĥ}^t$, $\left|a_t\right(h^t)-ā(w)|=\left|ā\right(w_t\left(h^t\right))-ā(w)|\le c_2\left|w_t\right(h^t)-w|\le c_2{(1-\delta )}^{1/2-\epsilon }.$ Since $|a_t(h^t)-ā(w)|\le Ā$ for all t and $h^t$, we have $\begin{array}{rl}{\mathbb{E}}^{\alpha }\left[\right|a-ā(w)\left|\right]& \le (1-\delta )\sum _t{\delta }^{t-1}\left({\int }_{h^t\in {Ĥ}^t}{c}_2\right|w_t\left(h^t\right)-w\left|d\mu \right(h^t)+{\mathrm{Pr}}^{\mu }(h_t\notin {Ĥ}^t\left)Ā\right)\\ & \le c_2{(1-\delta )}^{1/2-\epsilon }+2Ā(1-\delta )\sum _t{\delta }^{t-1}{\mathrm{Pr}}^{\mu }(h_t\notin {Ĥ}^t)\\ & \le c_2{(1-\delta )}^{1/2-\epsilon }+2Ā{(1-\delta )}^2,\end{array}$ where the third inequality follows from Lemma 27. We thus have ${\mathbb{E}}^{\alpha }\left[\frac{{\displaystyle {\psi }^{\prime }{(a)}^2}}{{\displaystyle \mathcal{I}(a)}}\right]-\frac{{\displaystyle {\psi }^{\prime }{(ā(w))}^2}}{{\displaystyle \mathcal{I}(ā(w))}}\ge -\kappa c_2{(1-\delta )}^{\frac{1}{2}-\epsilon }-2\kappa Ā{(1-\delta )}^2.$ Therefore, by (64) (and using $(1-\delta )/{\delta }^2=(1-\delta )/\delta +O({(1-\delta )}^2)$), there exists $c_3$ such that, for any sufficiently large $\delta <1$, the principal's payoff is no less than $\overline{F}(w)+\frac{{\displaystyle 1-\delta }}{{\displaystyle \delta }}\frac{{\displaystyle {\psi }^{\prime }{(ā(w))}^2}}{{\displaystyle \mathcal{I}(ā(w))}}\frac{{\displaystyle {\overline{F}}^{\prime \prime }(w)}}{{\displaystyle 2}}-c_3{(1-\delta )}^{1+\epsilon },$ completing the proof.

References

Abreu, Dilip, Paul Milgrom, and David Pearce (1991), “Information and timing in repeated partnerships.” Econometrica, 59 (6), 1713–1733.

Abreu, Dilip, David Pearce, and Ennio Stacchetti (1990), “Toward a theory of discounted repeated games with imperfect monitoring.” Econometrica, 58 (5), 1041–1063.

Aliprantis, Charalambos and Kim Border (2006), Infinite Dimensional Analysis: A Hitchhiker's Guide. Springer Science & Business Media.

Aoyagi, Masaki (2010), “Information feedback in a dynamic tournament.” Games and Economic Behavior, 70 (2), 242–260.

Athey, Susan and Kyle Bagwell (2001), “Optimal collusion with private information.” RAND Journal of Economics, 32 (3), 428–465.

Ball, Ian (2023), “Dynamic information provision: Rewarding the past and guiding the future.” Econometrica, 91 (4), 1363–1391.

Buldygin, Valerii and Yu Kozachenko (2000), Metric Characterization of Random Variables and Random Processes, Vol. 188. American Mathematical Society.

Ederer, Florian (2010), “Feedback and motivation in dynamic tournaments.” Journal of Economics & Management Strategy, 19 (3), 733–769.

Ely, Jeffrey, George Georgiadis, Sina Khorasani, and Luis Rayo (2023), “Optimal feedback in contests.” Review of Economic Studies, 90 (5), 2370–2394.

Ely, Jeffrey, George Georgiadis, and Luis Rayo (2025), “Feedback design in dynamic moral hazard.” Econometrica, 93 (2), 597–621.

Ely, Jeffrey C. and Martin Szydlowski (2020), “Moving the goalposts.” Journal of Political Economy, 128 (2), 468–506.

Forges, Francoise (1986), “An approach to communication equilibria.” Econometrica, 54 (6), 1375–1385.

Frick, Mira, Ryota Iijima, and Yuhta Ishii (2024), “Monitoring with rich data.” Working Paper.

Fuchs, William (2007), “Contracting with repeated moral hazard and private evaluations.” American Economic Review, 97 (4), 1432–1448.

Fudenberg, Drew, David Levine, and Eric Maskin (1994), “The folk theorem with imperfect public information.” Econometrica, 62 (5), 997–1039.

Fudenberg, Drew and David Levine (1994), “Efficiency and observability with long‐run and short‐run players.” Journal of Economic Theory, 62 (1), 103–135.

Fudenberg, Drew and David Levine (2007), “Continuous time limits of repeated games with imperfect public monitoring.” Review of Economic Dynamics, 10 (2), 173–192.

Gershkov, Alex and Motty Perry (2009), “Tournaments with midterm reviews.” Games and Economic Behavior, 66 (1), 162–190.

Goldlücke, Susanne and Sebastian Kranz (2012), “Infinitely repeated games with public monitoring and monetary transfers.” Journal of Economic Theory, 147 (3), 1191–1221.

Green, Edward J. and Robert H. Porter (1984), “Noncooperative collusion under imperfect price information.” Econometrica, 52 (1), 87–100.

Halac, Marina, Navin Kartik, and Qingmin Liu (2017), “Contests for experimentation.” Journal of Political Economy, 125 (5), 1523–1569.

Holmström, Bengt and Paul Milgrom (1987), “Aggregation and linearity in the provision of intertemporal incentives.” Econometrica, 55 (2), 303–328.

Hörner, Johannes and Satoru Takahashi (2016), “How fast do equilibrium payoff sets converge in repeated games?” Journal of Economic Theory, 165, 332–359.

Kandori, Michihiro (2002), “Introduction to repeated games with private monitoring.” Journal of Economic Theory, 102 (1), 1–15.

Kandori, Michihiro and Hitoshi Matsushima (1998), “Private observation, communication and collusion.” Econometrica, 66 (3), 627–652.

Kandori, Michihiro and Ichiro Obara (2006), “Efficiency in repeated games revisited: The role of private strategies.” Econometrica, 74 (2), 499–519.

Levin, Jonathan (2003), “Relational incentive contracts.” American Economic Review, 93 (3), 835–857.

Lizzeri, Alessandro, Margaret Meyer, and Nicola Persico (2002), “The incentive effects of interim performance evaluations.” Working Paper.

Madrigale, Vicente (1986), “On the non‐existence of efficient equilibria of repeated principal agent games with discounting.” Working Paper.

Matsushima, Hitoshi (2001), “Multimarket contact, imperfect monitoring, and implicit collusion.” Journal of Economic Theory, 98 (1), 158–178.

Matsushima, Hitoshi (2004), “Repeated games with private monitoring: Two players.” Econometrica, 72 (3), 823–852.

Meng, Delong (2021), “On the value of repetition for communication games.” Games and Economic Behavior, 127, 227–246.

Mirrlees, James A. (1975), “The theory of moral hazard and unobservable behaviour: Part I.” Working Paper. Published in Review of Economic Studies, 66 (1) (1999), 3–21.

Orlov, Dmitry, Andrzej Skrzypacz, and Pavel Zryumov (2020), “Persuading the principal to wait.” Journal of Political Economy, 128 (7), 2542–2578.

Radner, Roy (1985), “Repeated principal‐agent games with discounting.” Econometrica, 53 (5), 1173–1198.

Rahman, David (2014), “The power of communication.” American Economic Review, 104 (11), 3737–3751.

Rubinstein, Ariel (1979), “An optimal conviction policy for offenses that may have been committed by accident.” In Applied Game Theory (Schotter Brams and Schwodiauer, eds.), 406–413, Physical‐Verlag, Heidleberg.

Rubinstein, Ariel and Menahem E. Yaari (1983), “Repeated insurance contracts and moral jazard.” Journal of Economic Theory, 30 (1), 74–97.

Sadzik, Tomasz and Ennio Stacchetti (2015), “Agency models with frequent actions.” Econometrica, 83 (1), 193–237.

Sannikov, Yuliy (2007), “Games with imperfectly observable actions in continuous time.” Econometrica, 75 (5), 1285–1329.

Sannikov, Yuliy (2008), “A continuous‐time version of the principal‐agent problem.” Review of Economic Studies, 75 (3), 957–984.

Sannikov, Yuliy and Andrzej Skrzypacz (2007), “Impossibility of collusion under imperfect monitoring with flexible production.” American Economic Review, 97 (5), 1794–1823.

Sannikov, Yuliy and Andrzej Skrzypacz (2010), “The role of information in repeated games with frequent actions.” Econometrica, 78 (3), 847–882.

Smolin, Alex (2021), “Dynamic evaluation design.” American Economic Journal: Microeconomics, 13 (4), 300–331.

Spear, Stephen E. and Sanjay Srivastava (1987), “On repeated moral hazard with discounting.” Review of Economic Studies, 54 (4), 599–617.

Sugaya, Takuo (2022), “Folk theorem in repeated games with private monitoring.” Review of Economic Studies, 89 (4), 2201–2256.

Sugaya, Takuo and Alexander Wolitzky (2017), “Bounding equilibrium payoffs in repeated games with private monitoring.” Theoretical Economics, 12, 691–729.

Sugaya, Takuo and Alexander Wolitzky (2018), “Maintaining privacy in cartels.” Journal of Political Economy, 126 (6), 2569–2607.

Sugaya, Takuo and Alexander Wolitzky (2023), “Monitoring vs. Discounting in repeated games.” Econometrica, 91 (5), 1727–1761.

Wood, David C. (1992), “The computation of polylogarithms.” Working Paper.

Zhu, Huangjun, Zihao Li, and Masahito Hayashi (2022), “Nearly tight universal bounds for the binomial tail probabilities.” Working Paper.