LONG TIME BEHAVIOR AND TURNPIKE SOLUTIONS IN MILDLY NON-MONOTONE MEAN FIELD GAMES

We consider mean field game systems in time-horizon (0, T ), where the individual cost functional depends locally on the density distribution of the agents, and the Hamiltonian is locally uniformly convex. We show that, even if the coupling cost functions are mildly non-monotone, then the system is still well posed due to the effect of individual noise. The rate of anti-monotonicity (i.e. the aggregation rate of the cost functions) which can be afforded depends on the intensity of the diffusion and on global bounds of solutions. We give applications to either the case of globally Lipschitz Hamiltonians or the case of quadratic Hamiltonians and couplings having mild growth. Under similar conditions, we investigate the long time behavior of solutions and we give a complete description of the ergodic and long term properties of the system. In particular we prove: (i) the turnpike property of solutions in the finite (long) horizon (0, T ), (ii) the convergence of the system from (0, T ) towards (0,∞), (iii) the vanishing discount limit of the infinite horizon problem and the long time convergence towards the ergodic stationary solution. This way we extend previous results which were known only for the case of monotone and smoothing couplings; our approach is self-contained and does not need the use of the linearized system or of the master equation. Mathematics Subject Classification. 49J20, 35B40. Received November 11, 2020. Accepted July 3, 2021. Dedicated to Enrique Zuazua with friendship


Introduction
The theory of mean field games was initiated by Lasry and Lions since 2006 [24][25][26] in order to describe Nash equilibrium configurations in multi-agents strategic interactions. Similar ideas appeared in [21], at the same time. Mostly peculiar to the approach suggested by Lasry and Lions is to describe the equilibria through solutions of a forward-backward system of PDEs involving a Hamilton-Jacobi-Bellman equation for the value function of a single agent and a Kolmogorov-Fokker-Planck equation for the distribution law of the whole population. The simplest form of this kind of system is the following u(x, T ) = G(x, m(T )) where u(t, x) represents the optimal value for a player at time t in state x, while m(t, x) represents the density of the distribution law of the controlled dynamical state. The typical interpretation of the system is that any single player controls his/her dynamical state (in a standard probability space where a given d-dimensional Brownian motion is defined) where κ > 0 and L, F, G represent different costs depending on the control process {α τ } as well as on the measures {m τ }. Here m enters as an exogeneous datum and stays fixed in the agents'optimization; so defining the Hamiltonian H(x, p) := sup q [−q · p − L(x, q)], under suitable convexity and smoothness conditions the value function satisfy the first equation in (1.1) and α t = −H p (X t , Du(t, X t )) is the optimal feedback control. Assuming consistency with the agents' rational anticipations, at equilibrium it happens that the distribution law of the optimal controlled process coincides with the family of measures m t used in the individual optimization. Thus m satisfies the second equation as the law of the optimal process, where α corresponds to the optimal feedback strategy. We refer to [6] for an extended introduction to mean field games systems. In all the above presentation, the state space could be differently chosen, together with possibly boundary effects (reflection or absorption effects at the boundary, for instance) but we will assume here the simplest, yet instructive case of periodic setting. This means that x belongs to the flat torus This is a typical setting to investigate ergodic properties and long time behavior of the controlled dynamics. This kind of question is very natural in control theory and quite popular in applications. Many economic models, for instance, expect that if the time horizon is long then the optimal strategies are nearly stationary for most of the time. This is referred to as the turnpike property of optimal control problems, according to a terminology introduced by P.A. Samuelson in 1949 (see [15]). Even if the turnpike theory is a longstanding topic in optimal control, it has attracted an increasing renovated interest in the last years from both theoretical and applicative viewpoint: it is impossible here to mention all contributions in this direction, we refer e.g. to [14,30,32,33] and references therein.
When coming at mean field game systems as in (1.1), the long time behavior was investigated in several papers under the assumption that the cost functions F, G are nondecreasing in m. It is well established in the theory that this monotonicity condition gives uniqueness and stability of solutions; under the same condition the turnpike property and the convergence of solutions towards the stationary ergodic state have been first proved in [3,4] for quadratic Hamiltonian (i.e. H(x, p) = |p| 2 ). Milder statements (time average convergence) or stronger statements (exponential pointwise decay estimates) were obtained according to different sets of assumptions. The case of discrete time, finite states system was analyzed in [17]. Later on, the long time behavior was completely described in [5] in case of smoothing couplings and uniformly convex Hamiltonian, and in [29] for the case of local couplings and globally Lipschitz Hamiltonian. We also point out that the turnpike pattern is clearly shown in many numerical simulations, see e.g. [1].
The purpose of this paper is twofold. On one hand we wish to show that the monotonicity of the couplings F, G can be relaxed to some extent, using the diffusive character of the equations. Otherwise said, we show that the Brownian noise in the individual dynamics can compensate, to some extent, the lack of monotonicity so that mildly aggregative cost functions F, G may not affect the uniqueness of solutions and their stability, even in long time. This was already suggested by P.-L. Lions in the early stages of the theory ( [27]) although we could not find any further development of this issue in the literature. On another hand, still in the context of monotone or mildly non-monotone couplings, we revisit both the long time behavior of solutions and the ergodic limits in order to clarify the full picture: turnpike estimates for solutions in (0, T ), pointwise limit of the system as T → ∞, vanishing discount limit in the infinite horizon problem.
The above issues are somehow related. In fact, the complete characterization of the long time behavior, namely the convergence of u T (t) − λ(T − t) and m T (t) at any time t, was previously obtained in [5] as a byproduct of the long time convergence of the master equation. The master equation is an infinite dimensional equation (defined on time, space and the Wasserstein space of probability measures) which gives the value function u as a feedback of the measure m. This equation is not easy to handle and plays a similar role as the Riccati equation for the feedback operator in control systems. Establishing the long time behavior of the master equation allows one to completely describe the behavior of the system but is actually a hard result which requires much stronger conditions, starting from the smoothness of the functions F, G. In addition, what is more relevant here, when the couplings are not monotone, the master equation can not be properly used, since no satisfactory notion of solution has been developed so far outside the monotone case. Therefore, motivated by a setting of mildly nonmonotone cost functions F, G, we refine and develop some of the arguments introduced in [5] in order to study the long time convergence -as well as the vanishing discount limit -without any use of the master equation. The approach that we propose here is actually simpler and only relying on the PDE system; this seems to be much more flexible and it is actually promising for many other situations where the master equation still looks untractable (e.g. the case of state constraint problems, congestion models, etc...).
Let us mention that other nonmonotone mean field games have been considered in several works previously: for example, second-order problems have been analyzed in [9,12,13], while [7,18] deal with first-order systems. The works by Ambrose (see [2] and references therein) show the existence of solutions to second-order systems under smallness conditions on the coupling, though these conditions seem to be depending on the time horizon T . Tran considers in [31] a setting which is nonmonotone in a broader sense: the coupling in the system is increasing, but the Hamiltonian is not convex. Though he does not address the long-time analysis, his uniqueness results are closer to ours, as he obtains uniqueness under smallness conditions on H, which are independent of the time horizon.
Let us now summarize a bit more precisely our results, with reference to the content of the next Sections.
-In Section 3, we first show that, for given L ∞ -bounds in (0, T ) on m and Du, there is some γ (depending on the diffusion constant κ, but not on T ) such that if F + γm and G + γm are nondecreasing then the system (1.1) has a unique solution. See Theorem 3.1.
A similar result is also proved for the stationary ergodic problem We suggest two sets of assumptions under which this kind of result can be applied, assuming for simplicity that the final cost is a given function u T (x): (A) the case of globally Lipschitz (and locally uniformly convex) Hamiltonian. This corresponds to typical control problems with smooth, uniformly convex, Lagrangian cost and controls in a compact set. Thanks to the global Lipschitz bound of H, in this case the growth of F can be arbitrary, for instance one can take F = −γ m α . Then the system (1.1) is well-posed if γ is not too large (see also Rem. 3.6). (B) the case of superlinear, uniformly convex, Hamiltonian, with quadratic-like growth, and cost function F which satisfies 0 ≥ F (x, m) ≥ −γ m α with α < 2 d . The threshold 2 d for the growth of the coupling is not new in the context of mean field game systems with quadratic Hamiltonian (see e.g. [12,13,19]), and we comment this issue in Remark 3.10.
Let us stress that, in the aforementioned results, the admissible threshold γ of anti-monotonicity depends on the diffusivity constant κ and on the initial datum (through m 0 ∞ ). This latter fact explains very well why the master equation is hardly usable, in this context; indeed, there is no general feedback policy of u in the space of probability measures (unless we reduce to the case of monotone couplings).
-In Section 4 we show that, in the same context given above, the solutions (u T , m T ) of (1.1) satisfy an exponential turnpike property. More precisely, if Du T ∞ is bounded in (1, T − 1) independently of T , then there exists ω > 0 and M (independent of T ) such that where (ū,m) is a stationary state. See Theorem 4.1 and Corollary 4.4.
In particular, this result applies to the examples (A), (B) mentioned above. It is to be noted that this result is independent of initial and terminal conditions, as is customary in turnpike theory.
-In Section 5, we describe the convergence of u T (t), m T (t) at any time scale t. This is now influenced by both initial and terminal conditions (which we assume to be fixed). Namely, we prove that, given m 0 ∈ L ∞ (T d ), u(T ) ∈ W 1,∞ (T d ) and, for instance, a globally Lipschitz, locally uniformly convex, Hamiltonian, there exists γ > 0 such that, if F + γm is nondecreasing, then Notice that the convergence is not just for subsequences, but for the whole sequence (u T , m T ). See Theorem 5.3. -In Section 6, we describe the vanishing discount limit for the (discounted) infinite horizon problem Under similar conditions as before, we prove that the solution (u δ , m δ ) satisfies is again one particular solution of (1.3). Once more, the convergence occurs for the whole sequence; we prove that actually the particular solution selected in the limit satisfies where θ is itself a specific ergodic constant of a linearized problem. See Proposition 6.4 and Theorem 6.5.
In the end, the results in Section 6 establish the commutation property between the limit as t → ∞ and the limit as δ → 0 in the discounted infinite horizon problem. Let us point out thatm is the unique invariant measure of problem (1.3); indeed, in this problem µ is uniquely determined (while v is unique up to addition of constants) and satisfies To conclude, we recall that the results described in the above items and contained in Sections 4-6 were previously proved in [5] for the case of smoothing and monotone couplings F, G, using the long time convergence of the master equation. Even if we rely on many ideas contained in [5], we develop here a simpler program to achieve those results, which avoids both the use of the master equation and the explicit use of the linearized mean field game system. The main benefit of this approach is that it is much less demanding on the functions F, G (say, much cheaper in terms of the required smoothness) and more case-sensitive in terms of initial conditions. We give evidence of this fact by extending the results of [5] to the case of couplings which are local and even possibly non-monotone. Let us point out that even the case of non local mildly non-monotone couplings could be dealt with in a similar way but we did not pursue this extension here for the sake of simplicity.

Standing assumptions
Let x belong to the flat torus T d . We denote by P(T d ) the space of probability measures on T d . Throughout the whole paper, we suppose that H(x, ·) is locally Lipschitz continuous and locally uniformly convex on R d . Namely, p → H(x, p) is a C 2 function which satisfies and The couplings F, G are real valued functions defined on T d × [0, ∞); we suppose, as a standing condition, that they are locally bounded and Lipschitz continuous with respect to m: and similarly for G(x, ·): The dependence of H, F, G with respect to x is only assumed to be measurable; as it is commonly said, they are Carathéodory functions (measurable in x for any p, or m, accordingly), and the above conditions (2.1)-(2.4) are meant to hold almost everywhere for x ∈ T d . We stress that H, F could depend on t as well (in a measurable way) without additional difficulty, unless for results in which the long time behavior is concerned, where this dependence could change drastically the picture, of course.

Uniqueness for the MFG system
In a first result, we show that the MFG system admits a unique solution if the rate of anti-monotonicity does not exceed some threshold, depending on the global L ∞ -bounds of m, Du.
To this purpose, we consider the set of (classical) solutions (u, m) to (1.1) which satisfy for some M, U > 0.
Theorem 3.1. Let us set There exists γ > 0, only depending on κ, M, U (in particular through the constants then (1.1) admits at most one solution (u, m) in X.
Proof. Let (u 1 , m 1 ) and (u 2 , m 2 ) be two solutions to (1.1) which belong to the set X. Convexity of H and the usual duality identity give Integrating on (0, T ), using (2.2) and the monotonicity assumption on F (m), G(m), we have The equation for ρ := m 1 − m 2 reads with ρ(0) = 0. Thus Lemma A.2 applies and yields, by Lipschitz regularity of H p for some constant C only depending on κ and L U given by (2.1). Similarly, using (A.3) in Lemma A.2 we have Plugging those informations into (3.2) we obtain and therefore Du 1 = Du 2 whenever γ < α U (2C β 2 U M) −1 (note that m 2 is bounded away from zero on (0, T ) by the strong maximum principle). The equalities u 1 = u 2 and m 1 = m 2 then follow by uniqueness of solutions of the Fokker-Planck equation and of the (backward) Bellman equation.
Remark 3.2. The constant γ does not depend directly on the time horizon T , but only on M, U. In particular, if those bounds are independent of T , then so is γ as well. Note also that γ depends on the diffusion coefficient κ, and it must vanish as κ vanishes. Indeed, using bifurcation arguments as in [9], it is possible to prove the existence of multiple solutions (having comparable Lipschitz bounds) for large T and arbitrarily small anti-monotonicity degree, i.e. F ≈ −κ. Therefore, the constant C in the previous proof must explode as κ → 0.
There exists γ 0 > 0, only depending on κ,M,Ū (in particular through the constants LŪ, αŪ, βŪ in (2.1)-(2.2)), such that if Proof. The proof follows the lines of Theorem 3.1. If (λ 1 , u 1 , m 1 ) and (λ 2 , u 2 , m 2 ) are two solutions to (1.2) which belong to X, we consider the duality between the equations of u 1 − u 2 and m 1 − m 2 . Since m 1 − m 2 has zero average, the term with λ 1 − λ 2 disappears if integrated against m 1 − m 2 . Then one gets Using Corollary 1.3 of [5] for the equation of m 1 − m 2 , we have, for some C only depending on κ, LŪ, Plugging this information into (3.3) gives that Du 1 − Du 2 = 0 if γ 0 is sufficiently small, only depending on κ, LŪ, αŪ, βŪ. Since Du 1 = Du 2 it follows that m 1 = m 2 (from the second equation) and u 1 = u 2 from the prescribed normalization condition. Finally, the first equation gives λ 1 = λ 2 .
We now give two examples of applications of the previous results, namely two settings where the global bounds (3.1) are proved to hold.

(A) Globally Lipschitz Hamiltonians
The simplest case where a global bound in ensured for m, Du is when the Hamiltonian is globally Lipschitz. This is for instance the case when the set of controls of the individual agents lies in a compact set. For simplicity, we assume here that the final datum G is m-independent, i.e. (3.4) , and in addition suppose that H(x, p) satisfies Assume also that G satisfies (3.4).  Proof. Since H p (x, Du) ∞ ≤ L due to (3.5), by standard parabolic regularity we know that there exists M 0 , only depending on κ, L, m 0 ∞ such that Using (3.4) and parabolic regularity, there exists a constant U 0 , depending on κ, L, u T W 1,∞ (T d ) and on the regularity of F (x, m) for |m| ≤ M 0 , such that Since the bounds above are true for all solutions, by Theorem 3.1 there exists γ 0 , depending on κ, M 0 , U 0 , such that if (3.6) holds then there is a unique solution of the MFG system. This concludes the proof of the statement, because M 0 , U 0 only depend on κ, L, m 0 ∞ , u T (x) W 1,∞ (T d ) and on the local behavior of F, H on related compact sets.
Remark 3.6. It is possible to quantify a bit more precisely the dependence of γ 0 on the functions F, H. Assume that H satisfies (3.5) and for some c 0 > 0. Let for simplicity G = 0, and suppose that F (x, m) −γ m p , with |F m (x, m)| γ m p−1 for some γ > 0.
If L is given by (3.5), then one has for a constant C only depending on κ, L (and the dimension d). Therefore F (x, m) ∞ γ M p , hence for a possibly different constant C still depending only on κ, L, d. Coming back to the proof of Theorem 3.1, Hence we can estimate for some C only depending on κ, L, d. It is also easy to check that C → 0 as κ → 0, so we have One may guess at this point that γ 0 vanishes as the diffusion κ vanishes because norms of solutions are uncontrolled. In fact this seems related to subtler issues, involving the deterioration of the exponential decay of the heat semigroup as κ → 0. Indeed, as we already observed in Remark 3.2, one can find multiple solutions with controlled norms whenever F ≈ −κ. Hence, even though one assumes bounds U, M on solutions, it is mandatory to require smaller γ 0 as κ becomes smaller. Note also that bifurcation methods allow to construct solutions that are periodic in time; therefore, not only uniqueness fails, but also the turnpike property that will be addressed in the next section, at least for selected families of solutions. See also [18] for the failure of long time stabilization (due to existence of traveling waves) in deterministic mean field games with anti-monotone couplings.
In a similar way, using the elliptic regularity, there is existence and uniqueness of solutions for the stationary problem, provided the rate of anti-monotonicity of F is not too large. We skip the proof which follows the same lines as in Corollary 3.5.

(B) Quadratic Hamiltonians and couplings with mild growth
Our second example includes the case of superlinear Hamiltonians, having quadratic-like growth in the gradient. Namely, we assume that H ∈ C 1 (T d × R d ) is nonnegative and satisfies, for some c 0 > 0: for all x, y ∈ T d , p, q ∈ R d . In order to have global bounds, we need here to restrict the growth of the coupling term. Thus, we suppose that F satisfies, for some α < 2 d and c F > 0, Note that it is sufficient that F be bounded from above. Then, one can assume that it is nonpositive by adding a term Ct, for suitable C, to u. We also assume here that the final datum G is more regular, i.e. 3)) such that if then the MFG system (1.1) has a unique solution.
We divide the proof of the estimate (3.1) in several steps. The uniqueness statement is then a straightforward consequence of Theorem 3.1.
Step 1. Estimates on the oscillation of u(t). Denote, as usual, The estimate will be a consequence of the following "oscillation decay" inequality Indeed, given (3.11), by induction which is (3.10). We now turn to (3.11). It will be sufficient to prove it for T − k = 0, being the case T − k = 0 identical; it suffices indeed to perform a time-shift, which is allowed by the following crucial observation: since To obtain the oscillation estimate, we will argue by duality. For a review of the so-called adjoint method and its application to mean field games, see [19] and the recent developments in [11]. For a smooth probability density ρ 0 ∈ C ∞ (T d ) let ρ be the classical solution to the Fokker-Planck equation Then, as p = p p−1 < d d−2 , by means of Lemma A.3 there exists C ε depending on ε, d, α (but independent of ρ 0 ) such that for ε > 0 that will be chosen below; the second inequality is just a consequence of the third assumption (3.7) on H.
We may now add a constant to u so that max T d u(x, 1) = 0. Note that by the maximum principle, u(x, t) ≤ 0 for all t ≤ 1, x ∈ T d . Using the duality between the equations of u and ρ, and estimate (3.14), we obtain Testing the equation ofρ by u and the equation of u byρ and integrating by parts we obtain Denote for simplicity Applying now the first and the fourth assumption in (3.7) to the left-hand side of this resulting inequality yields Using now Young's and Hölder's inequalities, there exists c 1 (only depending on c 0 ) such that as a consequence of the second assumption in (3.7). Finally, we plug in (3.14) and (3.16), and by an appropriate choice of ε small we obtain Choosing now a sequence of ρ 0 converging (weak-*) to δ x0 we obtain the desired estimate Step 2. Estimates on a C β -norm of u. We claim that there exists β ∈ (0, 1) and C > 0 depending on c, c F , α, d such that for all n ∈ N (3.17) and the inequality can be extended up to t = T when n = 0.
can be obtained by duality as in Step 1 (and the argument is even simpler). Note that z(T − n) ∞ ≤ C by (3.10), independently on T, n. We then proceed assuming without loss of generality that [T − (n + 2), T − n] = [−1, 1]. Since z(1) ≤ 0 and F ≤ 0, we have z ≤ 0 on T d × (−∞, 1] by the maximum principle. Let now ρ be as in (3.13). Arguing as before by duality (see equations (3.14)-(3.16)), there exists a constant C depending on c 0 , c F , z(1) ∞ (but not depending on ρ 0 ) such that Varying ρ 0 yields a bound on z(τ ) ∞ for τ = 0. Then, varying τ ∈ [−1, 1) in the initial condition ρ(x, τ ) = ρ 0 (x) for ρ allows to extend such L ∞ -bounds for z to the whole cylinder Once sup-bounds on u(·, t) − max T d u(T − n) are established, using the uniform integrability of F (m) in (3.12), a control on a Hölder semi-norm follows by standard results for quasi-linear parabolic equations with quadratic growth in the gradient, see e.g. Theorem V.1.1 of [22].
Step 3. Estimates on the L q (L p )-norm of |Du| 2 . Let q > 1. We claim that there exists C > 0 depending on c 0 , c F , α, d, q such that for all n ∈ N We prove the inequality in the case (T − (n + 1), T − n) = (0, 1) and T ≥ 2; constants below will not depend on T nor n, so the validity of (3.18) for t ∈ [0, T − 1] will be a straightforward consequence. Some comments regarding the interval t ∈ [T − 1, T ], that is for t close to the time-horizon will be made below.
In the interval [T − 1, T ] there is no need to localize in time with the term (2 − t) and normalize the sup-norm, i.e. it is sufficient to perform the very same argument withũ(x, t) = u(x, t) (and use that u(T ) is C 2 ).
Step 4. Estimates on the sup-norm of Du and m. By the assumptions on H p , which has linear growth in |p|, the previous estimate (3.18) reads for any q > 1 and for some p > d/2. Hence, m solves a linear equation in divergence form with drift H p (x, Du), that in turn satisfy the previous integrability condition. Since (3.17)), we can apply ( [22], Thm. V.3.1) in order to get a bound for Du at time T − (n + 2). Since this bound is independent of T , and thanks to (3.9), we conclude that a uniform bound holds up to t = T : A similar result also holds for the stationary ergodic problem, as well.
Moreover, there existM,Ū (only depending on κ, α and the constants c 0 , c F ) such that any solution of (1.2) satisfies Finally, there exists γ 0 > 0 (only depending on κ, α and the constants c 0 , c F ) such that, if F (x, s) + γ 0 s is nondecreasing, then the solution (λ,m,ū) is unique.
Proof. We first prove the second assertion, namely the a priori estimate. Let (λ,m,ū) be any solution of (1.2). We start with bounds onλ (that are somehow related to oscillation estimates in the previous part). First, Hence, testing the equation ofm byū and the equation ofū bym and integrating by parts we obtain which is clearly bounded from below by a positive constant depending on c F , c 0 , C, σ. Therefore,λ is bounded only in terms of κ, c 0 , c F , α. Moreover, since m = 1, by (3.8) Thus, we obtain bounds on Note that the above procedure holds also if one replaces F by its truncation = F (x, min{m, M}), since it depends only on the upper bound (3.8) on |F |. Therefore, one may consider a classical solution of the system which exists, e.g, by results in [8] (F is globally bounded). Since m ≤ M, then F (x, m) = F (x, m), and we have the existence statement of a solution to the original ergodic problem.
In view of the a priori bounds found above, the uniqueness statement is a direct consequence of Theorem 3.4.
Remark 3.10. On the growth assumption α < 2 d . We have seen in this section that a condition of mild growth of F (x, ·) guarantees the existence of solutions to the MFG systems (both the ergodic and the evolutive one). It is worth noting that the existence of a triple (ū,m,λ) to the ergodic MFG system has been established under the weaker growth assumption α < 2 d−2 and additional smallness constraints on c F , while non-existence of solutions might even arise in the regime α > 2 d−2 , see [8]. Concerning the evolutive MFG system (1.1), existence of solutions for arbitrarily large time horizon T may fail already when α > 2 d in general [12] (so the study of the long time behavior becomes much more delicate, being the MFG system even more sensitive to the data). Still, smallness assumptions on c F are sufficient to recover existence for all T > 0, as described in [12]. These assumptions may then guarantee uniqueness also, and the turnpike property, but such an analysis is a bit beyond the scopes of this work.

The exponential turnpike estimate
In this section we prove that, if the anti-monotonicity of the coupling F (x, m) is sufficiently small, then we can prove the existence of solutions of (1.1) satisfying the turnpike property. The strategy we adopt follows Section 1.3.6 of [6], through the construction of a solution via a fixed point in a suitable weighted space. Then there exists γ > 0 only depending on L, κ (and on the functions F, H), such that if F (x, s) + γs is nondecreasing then any solution (u T , m T ) of problem (1.1) satisfies for some ω, M > 0 (independent of T ), where (λ,ū,m) is given by Proposition 3.7.
Remark 4.2. According to the Theorem, there is a threshold of anti-monotonicity, namely there exists somê γ > 0 such that if F (x, s) + γs is nondecreasing with γ <γ, then any solution enjoys the turnpike estimate.
Of course, we haveγ ≤ γ 0 given by Proposition 3.7 (indeed, a unique stationary state is used here). This valuê γ depends on F, H through the constant L in (3.5) and through assumptions (2.3), (2.2), in the sense that it depends on the constants c K , K , α K for a value of K only depending on L, κ. No special effort is devoted, in the proof below, to catch a refined estimate ofγ; as it will be clear, several arguments will need γ to be smaller than generic constants appearing in the global (in time) estimates of the solutions.
The proof of Theorem 4.1 will rely on the application of Schaefer's fixed point theorem ( [16], Thm. 11.3). The key-point is given in the following lemma which contains an a priori estimate on a sort of linearization of the mean field game system. We recall that, for any Let h(x, p) be differentiable with respect to p, and assume that h(x, p), h p (x, p), f (x, s) and B(x, p) are all Carathéodory functions which satisfy the following growth conditions for some constants 0 , C 1 , C 2 and for every s ∈ R, x ∈ T d and p ∈ R d such that |p| ≤ K: where we assume that, for any (t, x) ∈ Q T , we have |Dv(t, x)| ≤ K and for some c 0 , K > 0.
Proof. For T > 0, σ ∈ [0, 1], µ 0 ∈ L 2 (T d ) with T d µ 0 = 0, and v T ∈ L 2 (T d ), let (µ, v) be the solution of system (4.5). We first prove that there exists a constant c, independent of σ, T, µ 0 , v T , such that (4.8) To start with, we observe that, due to (4.6) and (4. (4.9) Integrating and using Lemma A.2 we get, for a constant C only depending on κ, 0 (given by (4.2)): where we used (4.4) and σ ≤ 1. If γC C 2 2 < c 0 we deduce the bound Since the Hamilton-Jacobi equation implies (using Lem. A.1 and (4.3)) coming back to (4.11) we deduce (for possibly different c) A similar estimate follows for sup [0,T ] ṽ(t) 2 , using again Lemma A.1. This allows us to conclude estimate (4.8). In addition, the inequalities above also show that We claim now that this implies the existence of τ such that, for every T > 2τ we have In fact, using (4.12), we know that there exist points ξ τ ∈ [0, τ /2] and η τ ∈ [T − τ /2, T ] such that (4.14) Estimating once more µ through Lemma A.2, and then using (4.9) (integrated in the interval (ξ τ , η τ )), thanks to (4.14) we get Using the global bound for ṽ(t) 2 and (4.14) we estimate last two terms. In the end, choosing γ sufficiently small we deduce and in turn what we estimate in between gives (as we said before, for possibly different c) Using once more Lemma A.2 we have, for all t ∈ (ξ τ , η τ ) and so (4.14) and (4.16) yield ) .
Now we complete the proof of Theorem 4.1.
Proof of Theorem 4.1.
Step 1 (definition of the fixed point mapping) We will first prove the result assuming m 0 ∈ P(T d ) ∩ C 0,α (T d ).
We set X = C 0 ([0, T ]; L 2 0 (T d )), where L 2 0 (T d ) denotes the subspace of L 2 (T d ) made of functions with zero average. Then we introduce the following norm in X: where ω > 0 will be chosen later. We have that (X, |||u||| X ) is a Banach space and ||| · ||| is equivalent to the standard norm u = sup [0,T ] u(t) L 2 (T d ) .
By Proposition 3.7, there exists some γ 0 such that, if F (x, s) + γ 0 s is nondecreasing, then there exists a unique (λ,m,ū) solution of the stationary ergodic problem (1.2).
We define the operator Φ on X as follows: given µ ∈ X, let (v, ρ) be the solution to the system Then we set ρ = Φµ. We observe that the existence and uniqueness of (v, ρ) is well known because H satisfies (3.5). We point out that µ is a fixed point of Φ if and only if m :=m + µ and u :=ū +λ(T − t) + v yield a solution of (1.1).
Step 2. (a priori estimates and existence of a fixed point) We first observe that, due to (3.5), there exists R, only depending on L, κ and m 0 ∞ , m ∞ , such that Therefore, up to replacing µ with min(µ, R) in the first equation of (4.19), we can assume that F (x, ·) is globally bounded and Lipschitz, thanks to (2.3) used with K = m ∞ + R. Using again (3.5) and the global bound of the right-hand side, we deduce (e.g. by Lemma A.1) that a global bound holds for ṽ(t) 2 . Hence, by (local) regularizing effect of parabolic equations, there exists a constant K > 0 (only depending on L, κ and R) such that The continuity of the operator Φ in C 0 ([0, T ]; L 2 (T d )) (hence in X) is a routine stability argument for parabolic equations, due to the Lipschitz character of H and the boundedness of F . In addition, sincem, m 0 ∈ C 0,α , by standard regularity results (see [22], Chap. V, Thms. 1.1 and 2.1) we have that the range of Φ is bounded in C α/2,α (Q T ), in particular its closure is compact in X. Thus, Φ is a compact and continuous operator. In order to apply Schaefer 's fixed point theorem, we are left to prove the following claim: there exists a constant M > 0 such that |||µ||| ≤ M for every µ ∈ X and every σ ∈ [0, 1] such that µ = σΦ(µ). , where we also used that F (x, s) + γs is nondecreasing. In addition, since µ = σΦ(µ) implies µ(t, x) ≥ −σm(x), we also have, for some constant c 0 , where we used the local uniform convexity of H and thatm > 0. Therefore, condition (4.6) holds too. Applying Lemma 4.3 we deduce that there exists a constant c (independent of σ, T ) such that Since ṽ 2 is uniformly bounded, this yields The estimate extends to (0, T ) because µ is also uniformly bounded, thus we proved that |||µ||| ≤ M for some M independent of σ, T . This proves (4.22) and concludes the fixed point argument. Eventually, one can upgrade the estimate in (1, T − 1) to the L ∞ -norm of µ(t) and Dv(t) by the regularizing effect in the two equations. This latter argument is already developed e.g. in [29].
Step 3. (conclusion of the general case) Assume now that m 0 ∈ P(T d ) and (u, m) is any solution of (1.1). Due to (3.5), by standard regularizing effect we have that m(t) ∈ C 0,α (T d ) for some α ∈ (0, 1), and that m(t) ∞ ≤ c (t − t 0 ) −d/2 , for every t ∈ (t 0 , t 0 + 1) and every t 0 > 0. Therefore, m is uniformly bounded (globally in time) in the interval (1, T ). Similarly, again by (3.5) and parabolic regularity, we have Du(t) ∞ ≤ C, t ∈ (0, T − 1], for some C only depending on L (the Lipschitz bound of H) and the global bounds of m,ũ. Therefore, (u, m) satisfies (3.1) in the interval (1, T − 1), for some M, U only depending on L. By Theorem 3.1, there exists γ L such that problem (1.1) admits at most one solution in (1, T − 1) if γ ≤ γ L . This means that (u, m) coincides with the solution built in Step 1 in the interval (1, T − 1), with initial-terminal conditions given by m(1) and u(T − 1). Hence (u, m) satisfies the exponential estimate (4.1).
We now observe that, as a consequence of the previous result, any globally bounded solution has a stationary attractor, provided the anti-monotonicity constant is sufficiently small.
for some ω, M > 0 (independent of T ) and some (ū,m) solution of the ergodic problem (1.2).
Proof. We first build a globally Lipschitz extension of the Hamiltonian function H(x, p). Namely, we consider a functionH(x, p) which still satisfies (2.2) and such that for some number C U only depending on U (eventually through some constants in (2.1)-(2.2) depending on U). An example of a similar extension can be built as follows. First of all we take a cut-off function ζ ∈ C 2 c (R d ) such that ζ ≡ 1 for |p| ≤ 2 and ζ ≡ 0 for |p| > 3; then we take a C 2 real function ϕ(r) such that ϕ(r) ≡ 0 if |r| ≤ 3 2 , ϕ is increasing, locally uniformly convex for r ∈ ( 3 2 , +∞) and globally Lipschitz continuous. Then the functionH (x, p) := H(x, p)ζ p U + Cϕ |p| U satisfies the required properties for a convenient choice of C (which will only depend on U). Now, we can apply Theorem 4.1 to the MFG system so there exists some γ > 0 such that if s → F (x, s) + γs is nondecreasing, then the ergodic stationary problem (corresponding toH(x, p)) has a unique solution (m,ū) and any solution of (4.24) satisfies (4.1). This value of γ depends onH, so it actually depends on U and on the functions F, H. Now, since Du T ∞ ≤ U, we havẽ H(x, Du T ) = H(x, Du T ) so the result applies to the given solution of the original problem (1.1). Therefore, (u T , m T ) satisfies (4.23). It remains only to show that (m,ū) is solution with H(x, p) rather than withH(x, p). But this must necessarily be true, because (4.23) implies, e.g., that Du T x, T 2 → Dū(x), so Dū ∞ ≤ U. This also proves that (m,ū) does not depend on the extensionH(x, p) which was constructed.
Remark 4.5. The previous result is new even in case that F (x, ·) is nondecreasing. It means that any solution (u T , m T ) such that Du T is uniformly bounded exhibits a turnpike behavior in long horizon and approximates (for most of time) a stationary solution (m,ū). So far this shows that a global (in time) gradient bound for u T is sufficient for the turnpike property to hold.  Then, there exists some γ, only depending on κ, m 0 ∞ , and on F, H (through constants appearing in the assumptions above, and the local bounds induced by M, U) such that if s → F (x, s) + γs is nondecreasing we have for some ω, M > 0 (independent of T ) and some (ū,m) solution of the ergodic problem (1.2).

The stationary feedback and the convergence of u T
In this section we are going to see how the exponential turnpike estimate established so far implies the convergence of u T (t) −λ(T − t), as well as of m T (t), for any fixed t > 0.
From now on, and throughout the rest of the paper, we will consider only the case (A) discussed above, which involves globally Lipschitz and locally uniformly convex Hamiltonians. Similar results can be proven for case (B) involving uniformly convex Hamiltonians and costs with mild growth, in view of the estimates obtained in Section 3, but they will not be stated explicitly for brevity.
We are now interested in a convergence for any t > 0, so we are going to require that (4.1) holds in the whole We start by deducing the following global bound as a corollary of (5.1).
Proof. Due to estimate (5.1), and the local Lipschitz character of F , we have that Now we establish a uniqueness result for the limiting problem of the MFG system as T → ∞; namely, we consider the problem Proof. We first observe that T d µ(t) = T d m 0 for every t, so µ(t) L 1 (T d ) is uniformly bounded; since H p is uniformly bounded by L, this implies that µ ∈ L ∞ ((0, ∞) × T d ) and its norm is universally bounded by a constant only depending on d, κ, L, m 0 ∞ . This provides with a uniform bound for F (x, µ) and in turn, as a consequence of Lemma A.1, this implies thatṽ(t) and Dv(t) are bounded in L 2 (T d ), uniformly in time. By local parabolic regularity we deduce that Dv ∈ L ∞ ((0, ∞) × T d ) as well and its norm is bounded uniformly in time, for all solutions. We will therefore use the upper and lower bounds of H pp for |p| varying in a compact set which is the same for all solutions (µ, v).
On account of the above ingredients, we follow a standard argument for the proof of uniqueness. Let (µ 1 , v 1 ), (µ 2 , v 2 ) be two solutions of (5.3). Consider a function ξ R (t) := ξ(t/R), where ξ is a C 1 function such that ξ ≡ 1 in (0, 1) and with support in (−1, 2). Using ψ = (µ 1 − µ 2 )ξ R as test function in the equation of v 1 − v 2 , we get as usual: which implies, using the uniform convexity of H: for some constant c > 0. Integrating we get where we used that F (x, s) + γs is monotone. Notice that Dv 1 − Dv 2 ∈ L 2 ((0, ∞) × T d ) by assumption. Then, by Lemma A.2 (see (A.4) with δ = 0), we deduce for some constant C depending on H p ∞ , on the L ∞ bound of µ 2 and on the local upper bound of H pp (all being estimated only in terms of κ, L, m 0 ∞ ). Hence we deduce Using Poincaré-Wirtinger inequality and the fact that µ 1 − µ 2 is uniformly bounded in L 2 (T d ), due to the properties of ξ R (t) we get for a sufficiently small γ, only depending on m 0 ∞ , κ, L (eventually trough F, H). Hence Dv 1 = Dv 2 and, from the Fokker-Planck equation, this yields µ 1 = µ 2 . Finally, this implies (v 1 − v 2 ) t = 0, hence v 1 (t, x) = v 2 (t, x) + K for some constant K ∈ R, and for every (t, x).
We now establish the convergence of u T −λ(T − t). There exists γ > 0, only depending on m 0 ∞ , κ, L (and on the functions F, H), such that if F (x, s) + γs is nondecreasing, then there exists a solution (v, µ) of (5.3) such that In particular, there exists a constantc ∈ R such that (v, µ) is the unique solution of (5.3) such that lim t→∞ v(t) =c.
Proof. We set v T := u T (x, t) −λ(T − t). Since m 0 ∈ L ∞ (T d ) and u(T ) ∈ W 1,∞ (Ω), we know that the exponential estimate (4.1) holds for t ∈ [0, T ]. Hence, from Corollary 5.1, we have that v T ∞ and Dv T ∞ are bounded uniformly with respect to T ; and by parabolic regularity, this readily implies that v T is locally (in time) relatively compact in the uniform topology. Similarly, m T is locally bounded in Hölder (time-space) norms, and it is relatively compact in the uniform topology (locally in time). Then, it is straightforward to see that (v T , m T ) converges, up to subsequences, to a solution (v, µ) of (5.3). We only need to prove that the limit function v is the same for all subsequences.
To this purpose, we develop an idea from [5]: for T, T we consider v T , v T and we define the shifted functions Both (v T ,μ T ) and (v T ,μ T ) are solutions of MFG systems in the interval (−τ, 0); the usual estimate gives where we used that F (x, s) + γs is monotone and the uniform bound from below of H pp on compact subsets. Integrating and using the final condition, we have which yields, thanks to (5.1), where, as usual, we denote generically by c possibly different constants (which may vary from line to line) independent of T, T . Applying Lemma A.2 to the equation ofμ T −μ T , we have Thus for γ sufficiently small we conclude that A similar estimate is then deduced for sup t∈[−τ,0] μ T (t) −μ T (t) 2 , from the above estimates.
With a bootstrap argument, using the global L ∞ bounds for D(v T − v T ) and for m T − m T , the previous L 2 bound can be updated into a L ∞ bound: is a Cauchy sequence and admits a limit as t → ∞. The value of this limit, sayc, fully characterizes the function v due to Lemma 5.2.
Remark 5.4. In the above result, we have assumed that the final cost G is independent of m. The reason is that if G is just a Lipschitz function of the density m(T ), then we are not able to show a bound for Du T up to t = T . Otherwise, should we have a global bound for Du in the whole (0, T ), then the same conclusion would hold for G satisfying (2.4) and G(x, m) + γm monotone with γ small.
We stress, in particular, that the convergence result of Theorem 5.3 remains true for smoothing couplings at final time, say for instance if G = G(x, m) is a Lipschitz continuous mapping from T d × P(T d ) (endowed with the Wasserstein distance d 1 ) such that G(x, m) W 1,∞ (T d ) is bounded uniformly in P(T d ) and for γ sufficiently small.
Remark 5.5. It would be possible to characterize the limit of u T −λ(T − t) in terms of a stationary feedback operator defined on the current measure µ(t), which is the unique solution of (5.3) (see Lem. 5.2). Namely, one can define an operatorÊ such that for any t > 0.
In the present setting,Ê could only be defined as an unbounded operator in (L 2 (T d )) + , with a domain which includes the set of bounded functions. Nevertheless,Ê could still be characterized thanks to the well-posedness of the MFG system. We stress that this kind of feedback (which plays a similar role as the Riccati stationary operator in other control problems) would coincide with a solution of the stationary ergodic master equation introduced in [5] for monotone and smooth mean field game systems. This may suggest alternative ways to look at the master equation of mean field games whenever it cannot be defined as a smooth function on the space of probability measures.

The discounted problem
We now deduce the existence of a solution to the infinite horizon MFG system. As before, we start by assuming that p → H(x, p) is a C 2 function which satisfies (2.2) and (3.5), and that F (x, m) satisfies (2.3) and for some γ > 0. It is possible to prove that, under the above assumptions, there exist δ 0 , γ 0 > 0 such that if γ < γ 0 and δ < δ 0 then the stationary problem admits a unique solution (ū δ ,m δ ), which is smooth. Here γ 0 only depends on κ, H, F , i.e. it depends on the constant L in (3.5) and on the constants c K , K , α K , β K for a K only depending on κ, L. While the existence of (ū δ ,m δ ) can be proved with a usual fixed point method, the uniqueness argument is similar as the one we used in Theorem 3.1 and requires smallness of δ as well.
Remark 6.2. We stress that the smallness condition required on γ in Theorem 6.1 does not depend on the parameter δ. Indeed, the purpose of this result is to provide an exponential decay in time which is uniform for δ sufficiently small, in order to apply it to the vanishing discount limit studied in the next Section. Such a uniform decay rate is obtained in the next Lemma, where we choose δ ≤ δ 0 in the final step; no effort is made here to quantify δ 0 (in fact a possible study for arbitrary large δ would even be possible but is beyond our goals here).
The proof of Theorem 6.1 will follow from a fixed point argument similarly as in Theorem 4.1. The crucial step consists in obtaining a priori estimates on the system: B(x, p) · p ≥ C −1 2 |p| 2 , |B(x, p)| ≤ C 2 |p| .
Step 1. In this first step, we show that there exists c such that and To show the above two estimates, we observe that the duality between the two equations implies where we used (6.7) and (6.9). Hence we get for some constant C independent of δ. Hence we get, using σ ≤ 1: Since µ, v are globally bounded, we have that last term vanishes as t → ∞. We deduce that, for γ so that c 0 − γ C C 2 2 > 0, we have where, we recall,ṽ = v − v . Here and after we denote by c possibly different constants which only depend on κ, 0 , C 1 , C 2 , c 0 . By using Lemma A.1 for the equation of v (with horizon T → ∞) we have where C only depends on κ, 0 and we also used (6.7). If we take here t = 0 and we estimate the right-hand side with (6.13), then we obtain an estimate for ṽ(0) 2 , which can be used in (6.14) to deduce that In turn again from (6.13) this concludes the proof of (6.11). Now, since we have, from Lemma A.2, we also deduce from (6.11) that ∀t > 0 and in turn now (6.15) implies as well: From Lemma A.1, the above two estimates imply a similar one for Dv(t) 2 , so (6.10) is proved.
Step 2. We first deduce from (6.11) that there exist ξ τ ∈ (0, τ ), η τ ∈ (2τ, 3τ ) such that Observe that, from (6.12), we have where we used the global bound forṽ(t) and (6.17). Using now (6.13) to estimate last integral in the right-side we get We use (6.17) and we take γ sufficiently small (independent of δ) so we conclude that ητ ξτ e −δs Therefore, for every t ∈ [τ, 2τ ] we estimate µ as Using the estimate for the last integral and the fact that η τ ≤ 3τ , we conclude that In particular, there exist τ 0 (and correspondingly, δ 0 ) such that Iterating this estimate we conclude that there exists ω > 0 such that Finally, using (6.15) we deduce a similar exponential decay for ṽ(t) 2 , and then for Dv(t) 2 as well.
We are now ready to prove Theorem 6.1 Proof of Theorem 6.1. Let us set X = L ∞ ((0, ∞); L 2 0 (T d )) where, we recall, L 2 0 (T d ) denotes L 2 functions with zero average.
We define the operator Φ ε on X as follows: given µ ∈ X, let (v, ρ) be the solution to the system then we set ρ = Φ ε µ. Here ε > 0 is a parameter used in a first step for compactness issues. We notice that µ is a fixed point of Φ 0 if and only if (v +ū δ , µ +m δ ) solves the MFG system (6.3).
In fact, if m := ρ +m δ , then m solves the evolution equation Since H p is bounded by L, and since m 0 ∈ L ∞ (T d ), T d m(t) = 1 for every t, there exists a constant M 0 such that m(t) ∞ ≤ M 0 for every t > 0. The same applies tom δ . We deduce that the range of Φ ε is contained in a uniform ball in L ∞ ((0, ∞) × T d ). Hence, there is no loss of generality in restricting the domain of Φ ε to functions which are uniformly bounded (it would be enough to replace µ with a suitable truncation in the definition of the operator); in particular, due to (2.3), the function F can be treated as uniformly Lipschitz in the µ-variable.
We also observe that the uniform bound of µ implies a uniform bound for Dv. Indeed, one can first proceed as in (6.15) to deduce that ṽ(t) 2 is uniformly bounded, then by Lemma A.1 and the regularizing effect of the equation it follows that for some constant K > 0. We now show the following properties of Φ ε : (i) Φ ε is continuous and compact.
To show this fact, let µ n be bounded in X. By decay properties of the equation of v, and by Lipschitz continuity of F , we have Since In particular, ρ n (t) is uniformly small in L 2 (T d ) for t large. Since ρ n (t) is (locally in time) relatively compact for the uniform topology, we deduce that it is compact in L ∞ ((0, ∞); L 2 (T d )). The continuity is easily proved in a similar way.

Vanishing discount limit
Now we wish to close the chain of implications by showing what happens in the vanishing discount limit. The preliminary result which is needed is the asymptotic behavior of problem (6.2). Proposition 6.4. Assume that H satisfies (2.2) and (3.5), and that F (x, m) satisfies (2.3), (6.1) and is differentiable with respect to m. There exists γ 0 , only depending on H, F such that, if γ < γ 0 in (6.1), then the sequence (ū δ ,m δ ) solution of (6.2) has the following asymptotic behavior as δ → 0: where (λ,ū,m) is the unique solution of (1.2) and θ is the unique constant such that the following ergodic stationary problem admits a solution (w, ρ): in Ω −κ∆ρ − div(ρ H p (x, Dū)) − div(m H pp (x, Dū)Dw) = 0 in Ω. (6.22) We admit for a while the above result and we proceed with the proof of the vanishing discount limit of the infinite horizon problem. Theorem 6.5. Under the assumptions of Proposition 6.4, let (u δ , m δ ) be the solution of (6.3). As δ → 0, we have where the constant θ is the unique ergodic constant of problem (6.22). Moreover, we have Proof. Using (6.4), and assumption (2.3), we know that there exists a constant C such that We deduce that, for a convenient constant M , the functionsū δ ± M e −ωt are, respectively, super and subsolution of the equation Using the comparison principle (in the class of bounded solutions) for the viscous Hamilton-Jacobi equation, we obtain thatū Due to (6.24) and Proposition 6.4, we deduce that v δ is uniformly bounded in (0, ∞) × T d . From estimate (6.4), we also know that Dv δ is uniformly bounded, and so is m δ as well. By local compactness and stability of the MFG system, there exists a subsequence -not relabeled -and a couple (v, µ) such that v δ converges to v, m δ converges to µ (locally uniformly) and (v, µ) is a solution of (5.3). Notice that Dv − Dū ∈ L 2 ((0, ∞) × T d ) as a consequence of (6.21). Finally, (6.24) and Proposition 6.4 imply that Therefore (v, µ) is the unique solution of (5.3) satisfying (6.25). We deduce that the whole sequence (v δ , m δ ) converges and this concludes the proof. We point out that Theorem 6.1, Proposition 6.4 and Theorem 6.5 establish that the two limits, for time going to infinity and discount factor going to zero, actually commute.
We are only left with the proof of Proposition 6.4, which is similar to Proposition 6.5 of [5].
Proof of Proposition 6.4. Let (ū δ ,m δ ) be solutions of (6.2). We set Using again (6.29) in the last two terms, we see that there exists γ 0 > 0 such that if γ < γ 0 , and δ is sufficiently small, we have that Dw δ , and in turn µ δ , are bounded in L 2 (T d ). Now, using that f δ , h δ , B δ grow at most linearly with respect to µ δ and w δ , respectively, we can use a bootstrap regularity argument and we conclude that µ δ , Dw δ are bounded in L ∞ (T d ).
From the bound of B δ (x, Dw δ ) and h p , we can now deduce that µ δ is bounded in C 0,α (T d ) for some α > 0, hence it is relatively compact in the uniform topology. Similarly, there exists w such that, up to subsequences, w δ − w δ converges to w uniformly and in W 1,p (T d ), for any p < ∞. Since δ w δ ∞ is bounded, overall we conclude that, for some constant θ ∈ R and some subsequence (not relabeled), we have δw δ → θ, w δ − w δ → w, µ δ → µ, uniformly in T d , (6.30) where (θ, w, µ) is a solution of (6.22). We only need to prove the uniqueness for this limit problem. This is the usual argument we just used in (6.26). In problem (6.22) we observe that F m (x,m) ≥ −γ due to (6.1) and that H pp (x, Dū) is bounded from below and from above. Therefore, if (θ,ŵ,μ) is any other solution, we have where c 0 , C 0 denote the bounds of H pp (x, Dū) from below and from above, and we estimated µ −μ as we did before. Thus, if γ is sufficiently small, we deduce that Dw = Dŵ, and the uniqueness follows (first of w, then of µ by the second equation, and finally of θ from the first equation). The uniqueness of the limit also implies the convergence of the whole sequence w δ , µ δ . Finally, we find that δw δ → θ, i.e.ū δ − λ δ −ū → θ. So the Proposition is proved.

Appendix A.
We collect here global in time decay estimates of both viscous Hamilton-Jacobi and Fokker-Planck equations, which we used throughout the paper.
Proof. Estimate (A.3) is already proved in the Appendix of [4]. So we only need to prove (A.4). For a fixed t ∈ (t 0 , T ), we start with considering the solution v of where ν > 0 and C > 0 depends on κ, V ∞ only. Throughout the proof, the value of C may increase, but will be always independent of t. Still using Lemma A.1 we have Dv(s) 2 2 ≤ C ṽ(s + 1) 2 2 + ṽ 2 L 2 ((s,s+1)×T d ) ∀s ≤ t − 1.
Moreover, G and D x G are bounded in L 1 ((0, T ); L p (T d )) and L 1 (0, T ); L p γ p γ−p +1 (T d ) respectively, provided that that is equivalent to The first inequality is true by the standing assumptions. Pick then γ < 2 so that p < d d−γ < d d−2 . Then, ρ(0) L 1 (T d ) G L 1 ((0,T );L p (T d )) + C ρV .