Mean-Field Linear-Quadratic Stochastic Differential Games in an Infinite Horizon

This paper is concerned with two-person mean-field linear-quadratic non-zero sum stochastic differential games in an infinite horizon. Both open-loop and closed-loop Nash equilibria are introduced. Existence of an open-loop Nash equilibrium is characterized by the solvability of a system of mean-field forward-backward stochastic differential equations in an infinite horizon and the convexity of the cost functionals, and the closed-loop representation of an open-loop Nash equilibrium is given through the solution to a system of two coupled non-symmetric algebraic Riccati equations. The existence of a closed-loop Nash equilibrium is characterized by the solvability of a system of two coupled symmetric algebraic Riccati equations. Two-person mean-field linear-quadratic zero-sum stochastic differential games in an infinite time horizon are also considered. Both the existence of open-loop and closed-loop saddle points are characterized by the solvability of a system of two coupled generalized algebraic Riccati equations with static stabilizing solutions. Mean-field linear-quadratic stochastic optimal control problems in an infinite horizon are discussed as well, for which it is proved that the open-loop solvability and closed-loop solvability are equivalent.

Note that U ad (x) depends on the initial state x. The corresponding X(·) ≡ X(· ; x, u 1 (·), u 2 (·)) is called an admissible state process for the initial state x. Then we can loosely formulate the following problem.
One may feel that cost functional (1.2) could be a little more general by including terms like q i (s), E[X(s)] . However, it is not hard to see that (as long as the integrals exist) which can be absorbed by replacing q i (·) by q i (·) + E[q i (·)]. Likewise, terms like ρ ij (s), E[u j (s)] are not necessarily included.
LQ stochastic control problems and two-person differential games have a long history. We do not want to make a lengthy survey on the literature here. Instead, we briefly mention the following early works [3,9,10,22,8,6,4,2,1,12], and refer the readers to the recent books by Sun-Yong [19,20] for more details and references cited therein.
The major novelty of this paper includes the following: (i) A general theory of two-person non-zero sum stochastic LQ differential game with mean-fields over [0, ∞), which is a complementary to the theory found in [18] for the problem in finite horizons, has been established.
(ii) The equivalence among the solvability of a system of coupled algebraic Riccati equations, open-loop solvability, and closed-loop solvability of stochastic LQ optimal control problem with mean-fields in [0, ∞), has been proved, which extends the same result found in [17] for the problem without mean-fields. Such a result will play an important role in studying closed-loop Nash equilibria in the current paper.
(iii) Two-person zero-sum stochastic LQ differential game with mean-fields over [0, ∞) has been discussed. On one hand, this is complementary to [7,21,17]. On the other hand, it has a special feature which makes it essentially different from general non-zero sum problems, that is the closed-loop representation of an open-loop saddle point coincides with a closed-loop saddle point, provided both exist.
The rest of the paper is organized as follows. In Section 2, we present some preliminary results about mean-field LQ stochastic optimal control problems in an infinite horizon. Section 3 aims to give results on mean-field LQ non-zero sum stochastic differential games, including open-loop Nash equilibria and their closed-loop representation, and closed-loop Nash equilibria with algebraic Riccati equations. In Section 4, the open-loop and closed-loop saddle points for mean-field LQ zero-sum stochastic differential games are investigated. Some examples are presented in Section 5. Section 6 is devoted to a proof of Theorem 2.9, the equivalence among open-loop, closed-loop solvability of mean-field LQ control problem, and solvability of a system of algebraic Riccati equations. Finally, some concluding remarks are given in Section 7.

Preliminaries
Throughout this paper, besides the notation introduced in the previous section, we will let I be the identity matrix or operator whose size can be determined from the context. When there is no confusion, we use · , · for inner products in possibly different Hilbert spaces, and denote by | · | the norm induced by · , · . See [13] for some basic properties of the pseudo-inverse. We define the inner product in L 2 F (H) by so that L 2 F (H) is a Hilbert space. We now consider the following controlled linear MF-SDE over [0, ∞): J(x; u(·)) = E ∞ 0 Q S ⊤ S R X(t) u(t) , X(t) u(t) + 2 q(t) ρ(t) , X(t) u(t) where A,Ā, C,C ∈ R n×n , B,B, D,D ∈ R n×m , Q,Q ∈ S n , S,S ∈ R m×n , R,R ∈ S m are given constant matrices, and b(·), σ(·), q(·) ∈ L 2 F (R n ), ρ(·) ∈ L 2 F (R m ) are stochastic processes. For any initial state x ∈ R n and control u(·) ∈ L 2 F (R m ), (2.1) admits a unique adapted solution X(·) ≡ X(· ; x, u(·)) ∈ X loc [0, ∞). We define the admissible control set as (2.3) U ad (x) = u(·) ∈ L 2 F (R m ) X(·) ≡ X(·; x, u(·)) ∈ X [0, ∞) .
In general, U ad (x) depends on x ∈ R n . Let us pose the following problem.
For any given x ∈ R n , the set U ad (x) of admissible controls is either empty or a convex set in L 2 F (R m ), since the state equation (2.1) is linear. To investigate Problem (MF-SLQ), we should find conditions so that the set U ad (x) is at least non-empty and hopefully it admits an accessible characterization. For this target, let us first look at the following uncontrolled linear system on [0, ∞): (2. 5) dX(t) = AX(t) +ĀE[X(t)] + b(t) dt + CX(t) +CE[X(t)] + σ(t) dW (t), t 0, X(0) = x.
According to [7], and via a similar argument proving Theorem 3.3 of [21], we have the following result.
From the above, we can easily show that under (H1), U ad (x) = L 2 F (R m ) which is independent of x. Hence, hereafter, once (H1) is assumed, we will denote U ad (x) = U ad . Now, we introduce the following definitions concerning Problem (MF-SLQ). and Problem (MF-SLQ) is said to be finite if it is finite at all x ∈ R n .
If an open-loop optimal control (uniquely) exists for x ∈ R n , Problem (MF-SLQ) is said to be (uniquely) is called a closed-loop strategy of Problem (MF-SLQ). The solution X(·) ≡ X Θ,v (·) of (2.8) is called the closed-loop state process corresponding to (x, Θ, v(·)). The control u(·) defined by (2.7) is called the outcome of (Θ, v(·)), or a closed-loop control for the initial state x ∈ R n .
On the other hand, from Proposition 2.4, we know that under (H1), the set U ad (x) = U ad of admissible controls consists of closed-loop controls for all x, and that (2.12) is equivalent to the following condition: In general, an open-loop optimal control depends on the initial state x ∈ R n , whereas a closed-loop strategy is required to be independent of x. From (2.13), we see that the outcome of a closed-loop optimal strategy (Θ * , v * (·)) is an open-loop optimal control for the initial state X * (0). Hence, for Problem (MF-SLQ), the closed-loop solvability implies the open-loop solvability. The converse is also true for stochastic LQ optimal control problems in an infinite horizon without mean fields. That is to say, the open-loop and closed-loop solvabilities are equivalent (see [17]  Note that for any Θ ≡ (Θ,Θ) ∈ S [A,Ā, C,C; B,B, D,D] and v(·) ∈ L 2 F (R m ), we have the stabilized system (2.8). The cost functional (2.2) becomes We see that Q Θ , S Θ depend on Θ,Q Θ ,S Θ , q Θ depend on Θ ≡ (Θ,Θ), and R,R, ρ are unchanged. Similar to (2.15), we will denote Definition 2.7. The following is called a system of generalized algebraic Riccati equations (ARE, for short): with the unknown (P, P ) ∈ S n × S n . A solution pair (P, P ) to (2.19) is said to be static stabilizing if there exists a pair (θ,θ) Definition 2.8. Let (H1) hold. Open-loop optimal controls u * (· ; x) ∈ U ad of Problem (MF-SLQ), parameterized by x ∈ R n , admits a closed-loop representation, if there exists a pair (Θ * , v * (·)) ∈ S [A,Ā, C,C; B,B, D,D] × L 2 F (R m ) such that for any initial state x ∈ R n , (2.14) holds, which is an open-loop optimal control of Problem (MF-SLQ) for x, where X * (·) ≡ X Θ * ,v * (·) ∈ X [0, ∞) is the solution to the closed-loop system (2.8) corresponding to (Θ * , v * (·)).
We now state the following result which is an extension of a similar result in [17] for which no mean-field terms presented, and will play an important role in the following section while we studying closed-loop Nash equilibria by means of a system of coupled algebraic Riccati equations. The proof will be postponed to Section 6. Theorem 2.9. Let (H1) hold. Then the following statements are equivalent: (iii) The system (2.19) admits a static stabilizing solution pair (P, P ) ∈ S n × S n , the BSDE on [0, ∞): and the ODE on [0, ∞): admits a solutionη(·) ∈ L 2 (R n ) such that In the above case, the closed-loop optimal strategy (Θ * , v * (·)) ≡ (Θ * ,Θ * , v * (·)) is given by Each open-loop optimal control u * (·) for the initial state x ∈ R n admits a closed-loop representation (2.14), where X * (·) ∈ X [0, ∞) is the solution to the closed-loop system (2.8) under (Θ * , v * (·)). Further, the value function is given by

Mean-Field LQ Non-Zero Sum Stochastic Differential Games
We now return to our Problem (MF-SDG).

Notions of Nash equilibria
To simplify the notation, let m = m 1 + m 2 and denote (for i = 1, 2) Then the state equation (1.1) becomes which is of the same form as (2.1), and the cost functionals are, for i = 1, 2, In order the game to make sense, we make a convention that both players at least want to keep the state X(·) ≡ X(· ; x, u 1 (·), u 2 (·)) in X [0, ∞) so that both cost functionals are well-defined. To guarantee this, similar to Problem (MF-SLQ), we introduce the following assumption. 8 Although (H2) looks exactly the same as (H1), the meaning is different. Hypothesis (H2) provides the possibility for both players to cooperatively make both cost functionals finite. In fact, under (H2), for any x ∈ R n , by Proposition 2.4, the following set of all admissible control pairs is non-empty: Further, for any u 2 (·) ∈ L 2 F (R m2 ), making use of Proposition 2.4 again, the following holds: Likewise, for any u 1 (·) ∈ L 2 F (R m1 ), one has We now give the following definition.
On the other hand, if Θ * , v * (·) is a closed-loop Nash equilibrium of Problem (MF-SDG), we may consider the following stabilized state equation (recall (2.9)): Then by (3.11), it is easy to see that (v * 1 (·), v * 2 (·)) is an open-loop Nash equilibrium of the corresponding mean-field LQ two-person non-zero sum stochastic differential games.
From the above, we see that Problems (MF-SDG) and (MF-SLQ) are essentially different in a certain sense, and we can only say that Problem (MF-SLQ) is a formal special case of Problem (MF-SDG).

Open-loop Nash equalibria and their closed-loop representation
In this section, we discuss the open-loop Nash equilibria for Problem (MF-SDG) in terms of MF-FBSDEs. We first have the following result. Theorem 3.3. Let (H2) hold, and x ∈ R n . Then u * (·) ≡ (u * 1 (·), u * 2 (·)) ∈ U ad (x) is an open-loop Nash equilibrium of Problem (MF-SDG) for x if and only if the following two conditions hold: to the following MF-FBSDE: satisfies the following stationarity condition: a.e. t ∈ [0, ∞), a.s., i = 1, 2.
Note that by the definition of Θ * andΘ * (in (3.38)), we have and we can rewrite (3.33) as We may further write (3.40) and (3.41) in component forms: In the above, the coefficient matrices of the equations for Θ * andΘ * are not symmetric in general (even if P 1 , P 2 , P 1 , P 2 are all symmetric). Hence, the equations for P i , P i , i = 1, 2 and for P ⊤ i , P ⊤ i , i = 1, 2 are different. Consequently, we do not expect P i , P i , i = 1, 2 to be symmetric in general.

Closed-loop Nash equalibria and symmetric algebraic Riccati equations
We now look at closed-loop Nash equilibria for Problem (MF-SDG). First, we present the following result, which is a consequence of Theorem 3.3.
Proof. By the observation we made at the end of subsection 3.1, we see that (Θ * , v * (·)) is a closed-loop Nash equilibrium of Problem (MF-SDG) if and only if v * (·) is an open-loop Nash equilibrium of the problem for any initial state x ∈ R n , with the stabilized state equation (3.15) and cost functionals (3.16) which we rewrite here in details for convenience (we suppress some t): , Thus by Theorem 3.3, (Θ * , v * (·)) is a closed-loop Nash equilibrium of Problem (MF-SDG) if and only if for for i = 1, 2, satisfies the following stationarity condition: and for i = 1, 2, the following convexity condition holds: is the solution to the following controlled homogeneous MF-SDE on [0, ∞): Since (Θ * , v * (·)) is independent of x and (3.45)-(3.48) hold for all x ∈ R n , by subtracting equations corresponding to x and 0, the latter from the former, we see that for any x ∈ R n , the following MF-FBSDE: admits an adapted solution satisfying It follows, again from Theorem 3.3, that (Θ * , 0) is a closed-loop Nash equilibrium of Problem (MF-SDG) 0 . The proof is complete. Now, we give a necessary condition for the existence of closed-loop Nash equilibria of Problem (MF-SDG).
Proposition 3.5. Let (H2) hold, and let (Θ * , v * (·)) be a closed-loop Nash equilibrium of Problem (MF-SDG). Then for i = 1, 2, the following system of coupled AREs admits a solution pair (P i , P i ) ∈ S n × S n : , and the following conditions are satisfied: , t 0, X 0 1 (0) = 0, and cost functional: It is easy to see that (Θ * 1 , 0) is a closed-loop optimal strategy for the above mean-field LQ stochastic optimal control problem. Thanks to Theorem 2.9, the following system of coupled AREs: admit a static stabilizing solution pair (P 1 , P 1 ) ∈ S n × S n , satisfying Similarly, for any u 2 (·) ∈ L 2 F (R m2 ), we can consider the state equation: and cost functional: where A 2 ,Ā 2 , C 2 ,C 2 and Q 2 ,Q 2 , S 22 ,S 22 are defined similar to (3.52). In the same spirit as above, we can see is a closed-loop optimal strategy for the above mean-field LQ stochastic optimal control problem. Making use of Theorem 2.9 again, the following system of coupled AREs admit a static stabilizing solution pair (P 2 , P 2 ) ∈ S n × S n , satisfying Similarly, which implies (3.50). By (3.50), (3.53) and (3.55), we have (by a straightforward calculation) In the same way, we have This yields (3.49). The proof is complete. Now, we are ready to present the main result of this subsection, which characterizes the closed-loop Nash equilibrium of Problem (MF-SDG).
Before the end of this section, let us rewrite the system of two coupled AREs (3.49) in a more compact form so that one can see an interesting feature of it. We define (different from that in (3.26)) Note that (3.50) is equivalent to (recalling the notation introduced in (3.21) and (3.24)) If we assume both Σ andΣ are invertible, then we have On the other hand, (3.49) can be written as Clearly, both equations in (3.73) are symmetric with solutions P, P ∈ S 2n . Recall that system of AREs
To summarize, we have the following result.
In  (i) The following system of two coupled generalized AREs: admits a static stabilizing solution (P c , P c ) ∈ S n × S n such that (ii) The following BSDE on [0, ∞): admits a solution (η c (·), ζ c (·)) ∈ X [0, ∞) × L 2 F (R n ), and the following ODE: (4.20) admits a solutionη c (·) ∈ L 2 (R n ). In the above case, the closed-loop saddle point is given by ) be a closed-loop saddle point of Problem (MF-SDG) 0 . System of coupled algebraic equations (3.49) for (P 1 , P 1 ) ≡ (−P 2 , − P 2 ) = (P c , P c ) becomes This system is solvable if and only if Finally, from (3.58) we have , for some ν(·) ∈ L 2 F (R m ). It follows from (3.60) that we get (4.26) , for someν(·) ∈ L 2 (R m ). Combining the above two expressions leads to (4.22). The proof is complete.
Comparing Theorems 4.3 and 4.4, we have the following result. Note that general non-zero sum differential game does not have such a result (see [18]).
Finally, we have the following corollary for Problem (MF-SLQ), since it is a special case of Problem (MF-SDG) 0 when m 2 = 0.

Examples
In this section, we present some examples illustrating the results in the previous sections.
The following example shows that for the closed-loop saddle points of the mean-field LQ zero-sum stochastic differential game, the system of generalized AREs may only admit non-static stabilizing solutions even if the system [A,Ā, C,C; B,B, D,D] is MF-L 2 -stabilizable, which leads to the non-existence of a closed-loop saddle point.
Example 5.1. We consider one example for Problem (MF-SDG) 0 . Consider the following one-dimensional state equation with the cost functional In this example, Due to the fact that P 0 > 0, hence, we obtain that Note that Σ c =Σ c is invertible for all (P c , P c ) ∈ S 1 × S 1 with Then the corresponding system of generalized AREs (4.17) reads Thus, P c = − 1 3 , P c = 2 and Also, the range condition hold automatically since Σ c andΣ c are invertible. However, we have The following example tells us that for the closed-loop saddle points of the mean-field LQ zero-sum stochastic differential game, it may admit uncountably many closed-loop saddle points.
Example 5.2. We consider one example for Problem (MF-SDG) 0 . Consider the following one-dimensional state equation with the cost functional Then the corresponding system of generalized AREs (4.17) reads which admits a unique solution P c = −1; and which admits a solution P c = 0. Thus, Hence, hold. By Theorem 4.4, we see that Problem (MF-SDG) 0 admits a closed-loop saddle point if the condition (5.9) holds. But, However, we can choose θ,θ = 0 in (4.21), such that (I − Σ † c Σ c )θ, (I −Σ † cΣ c )θ is a stabilizer of the system [A, C; B, D]. Thus, Problem (MF-SDG) 0 may still admit uncountably many closed-loop saddle points.
The following example shows that for the closed-loop saddle points of the mean-field LQ zero-sum stochastic differential game, it may happen that the system [A,Ā, C,C; B,B, D,D] has more than one (uncountably many) MF-L 2 -stabilizer, while the closed-loop saddle point is unique. Example 5.3. We consider one example for Problem (MF-SDG) 0 . Consider the following one-dimensional state equation with the cost functional Then the corresponding generalized ARE (4.17) for P c reads , which has three solutions: All of them satisfy the range condition However, only P c = 1 satisfies Now putting P c = 1 into the following generalized ARE (4.17) to resolve P c : which has a unique solution P c = 8. Due to the fact thatΣ c = R + D ⊤ P c D = 3 1 1 −1 is invertible, it is easy to verify that the range condition holds, and We can see that (Θ * 1 , Θ * 2 ) = (1, −3) satisfies (5.17) and hence is a stabilizer of the system [A, C; B, D]. By Theorem 4.4, the above problem admits a unique closed-loop saddle point.
On the other hand, by verifying (5.17), we see that is the closed-loop saddle point of the problem. The following example shows that for the closed-loop saddle points of the mean-field LQ zero-sum stochastic differential game, it may happen that the system of generalized AREs may only admit non-static stabilizing solutions and the system [A,Ā, C,C; B,B, D,D] is not MF-L 2 -stabilizable. Thus no closed-loop saddle points exist.
Example 5.4. We give one example of Problem (MF-SDG) 0 . Consider the following two-dimensional state equation with the cost functional (5.24) In this example, Note that the corresponding system of generalized AREs (4.17) reads Then, solving (5.26) yields Thus, Also, the range condition hold automatically since Σ c andΣ c are invertible. However, we have However, substituting the above Θ * andΘ * of (5.28) into (5.25), we no longer obtain the result of P 0 > 0 andP 0 > 0, which conflicts with (5.25). Therefore, (Θ * ,Θ * ) is not an MF-L 2 -stabilizer of the system The following example shows that for the mean-field LQ zero-sum stochastic differential game, it may happen that the closed-loop saddle point uniquely exists, which coincides with the closed-loop representation of the open-loop saddle point. Moreover, the system of generalized AREs may only admit static stabilizing solutions and the system [A,Ā, C,C; B,B, D,D] is MF-L 2 -stabilizable.
Example 5.5. We give one example of Problem (MF-SDG) 0 . Consider the following two-dimensional state equation with the cost functional (5.30) In this example, First, let us study its open-loop saddle point. According to (4.10), we have the following coupled AREs Then, solving (5.31) yields The above solutions satisfy (4.7) and (4.8). Further, since Σ o andΣ o are invertible, we have Substituting the above Θ * andΘ * of (5.32) into (5.33), we have Note that the corresponding backward systems (4.11)-(4.14) reads where X * (·) ∈ X [0, ∞) is the solution to (5.29) corresponding to u * (·). Next, we consider the closed-loop saddle point. Note that the corresponding system of generalized AREs (4.17) reads

35
Then, solving (5.39) yields Thus, Also, the range condition hold automatically since Σ c andΣ c are invertible. However, we have In this case, the closed-loop saddle point is given by (5.41) and Since P = P c and P = P c , we get Σ = Σ c ,Σ =Σ c and η o (·) = η c (·), ζ o (·) = ζ c (·),η o (·) =η c (·). This implies that the closed-loop representation of the open-loop saddle point coincides with the closed-loop saddle point.
The following example shows that for the mean-field LQ non-zero sum stochastic differential game, it may happen that the closed-loop representations of open-loop Nash equilibria are different from the closed-loop Nash equilibria, although the solutions to the system of algebraic Riccati equations of the open-loop Nash equilibria are symmetric.
Example 5.6. We present one example of Problem (MF-SDG). Consider the following two-dimensional state equation with the cost functional In this example, First, let us study the open-loop Nash equilibria. Note that we recall the system of algebraic equations (3.42)-(3.43) as Then, solving (5.47)-(5.48) yields which are symmetric. Since are invertible, it follows from (5.48) that we have Also, it follows from (3.35) and (3.36) that we have From (3.38), we have By Theorem 3.3, the problem admits an open-loop Nash equilibrium u * (·) for any initial state x ∈ R n . And it follows from (3.39) that it has the following closed-loop representation: with (Θ * ,Θ * , v * (·)) given by ( For i = 1, 2, we recall the system of algebraic equations (3.49)-(3.51) as , with the following conditions: , a.e. t ∈ [0, ∞), a.s., Hence, by Theorem 3.6, the problem admits a closed-loop Nash equilibrium. From this example, we see that the solution to the system of algebraic equations (3.42)-(3.43), which comes from the open-loop Nash equilibrium, may be symmetric, but they are different from the solution to the system of algebraic equations (3.49)-(3.51), which comes from the closed-loop Nash equilibrium. It is obvious that the closed-loop representation of the open-loop Nash equilibrium is different from the closed-loop Nash equilibrium.
The following example shows that for the closed-loop representation of open-loop Nash equilibria of the mean-field LQ non-zero sum stochastic differential game, it may happen that the solutions to the system of algebraic equations may be asymmetric In this example, ,Q 1 = Then, solving (5.65)-(5.66) yields are invertible, it follows from (5.66) that Also, it follows from (3.35) and (3.36) that we have and From (3.38), we have By Theorem 3.3, the problem admits an open-loop Nash equilibrium for any initial state x ∈ R n . And it follows from (3.39) that it has the following closed-loop representation: with (Θ * ,Θ * , v * (·)) given by ( For i = 1, 2, we recall the system of algebraic equations (3.49)-(3.51) as with the following conditions: Hence, by Theorem 3.6, the problem admits a closed-loop Nash equilibrium. From this example, we see that the solutions to the system of algebraic equations (3.42)-(3.43), which come from the open-loop saddle point, may be asymmetric, but the solutions to the system of algebraic equations (3.49)-(3.51), which come from the closed-loop saddle point, are still symmetric. It is obvious that the closed-loop representation of the open-loop Nash equilibrium is different from the closed-loop Nash equilibrium.

Proof of Theorem 2.9
This section is denoted to a proof of Theorem 2.9.
(ii) ⇒ (i) is obvious from the statements after Definition 2.6, and (iii) ⇒ (ii) is direct from Corollary 4.6.
With the help of representation (6.1), we have the following results concerning with the open-loop solvability of Problem (MF-SLQ) whose proof is classical. Lemma 6.2. Suppose system [A,Ā, C,C] is L 2 -globally integrable. We have the following results: is an open-loop optimal control for the initial state x if and only if M 2 u * + M 1 x + u = 0.
(ii) If there exists a constant δ > 0 such that M 2 δI, or equivalently, We need the following lemma about Problem (MF-SLQ) 0 . Lemma 6.3. Suppose system [A,Ā, C,C] is L 2 -globally integrable and (6.3) holds for some δ > 0. Then the following system of coupled AREs admits a solution pair (P, P ) ∈ S n × S n , and (Θ,Θ) ∈ R m×n × R m×n defined by is the solution to the following closed-loop system: Moreover, the value function is given by Proof. For T > 0, we consider the state equation on [0, T ]: and the cost functional We claim that (6.9) J 0 T (0; u(·)) δE T 0 |u(t)| 2 dt, ∀u(·) ∈ L 2 F (R m ), for some δ > 0.
To prove this, choose any u(·) ∈ L 2 F (R m ) and let X T (·) be the corresponding solution to the above state equation with initial state x. Define the zero-extension of u(·) as follows: T (x; u(·)) = J 0 (x;v(·)). In particular, taking x = 0, by (6.3) we get This proves our claim. The fact (6.9) allows us to apply Theorems 4.2, 4.4 and 5.2 of [14] to conclude that for any T > 0, the following system of two coupled differential Riccati equations Further, if we define then the unique open-loop optimal control u * T (·) is given by where X * T (·) ∈ L 2 F ([0, T ]; R n ) is the solution to the following MF-SDE:
We pose the following deterministic LQ optimal control problem: Problem (DLQ). For any initial state x ∈ R n , to find a control w * (·) ∈ L 2 (R m ) such that J (x; w * (·)) = inf w(·)∈L 2 (R m )J (x; w(·)), and have the following lemma.
The following additional lemma is a direct consequence of Proposition 5.2, (iii) of [17].
Lemma 6.5. If Problem (DLQ) is open-loop solvable, then there exists a U * (·) ∈ L 2 (R m×n ) such that for any x ∈ R n , U * (·) x is an open-loop optimal control for the initial state x.
With the above two lemmas in hand, we can continue to consider the limitation of (6.11) when ε → 0, without (6.3). For any ε > 0, consider the state equation (6.15) and the cost functional Denote the above problem by Problem (DLQ) ε , and the corresponding value function byV ε (·). Since Problem (MF-SLQ) 0 is open-loop solvable, by (6.20) and Lemma 6.1, we must havē where M is a bounded self-adjoint linear operator from L 2 (R m ) to itself. That is to say, Problem (DLQ) ε is uniquely open-loop solvable, for each ε > 0. Consequently, by Lemma 6.4, ARE (6.11) admits a unique solution P ε ∈ S n such thatV ε (x) = P ε x, x , ∀x ∈ R n . Moreover,Θ ε ∈ R m×n is a stabilizer of system A; B , and the unique open-loop optimal control w * ε (·; x) of Problem (DLQ) ε at x is given by whereΨ ε (·) ∈ L 2 (R n×n ) is the solution to the following closed-loop system: Now let U * (·) ∈ L 2 (R m×n ) be a deterministic function with the property in Lemma 6.5. By the definition of value function, we have for any x ∈ R n and ε > 0, =J ε (x; w * ε (·; x)) =V ε (x) J ε (x; U * (·)x) =V (x) + ε  It is clear thatV ε1 (x) V ε2 (x), for any 0 < ε 1 ε 2 , ∀x ∈ R n . Thus P ε1 P ε2 for any 0 < ε 1 ε 2 . Then, noting (6.23), P ≡ lim ε→0 P ε exists andV (x) = P x, x .
In the end, we give the proof of (i) ⇒ (iii) for the general case S [A,Ā, C,C; B,B, D,D] = ∅, that is, under (H1). Take Θ ≡ (Θ,Θ) ∈ S [A,Ā, C,C; B,B, D,D], and consider the state equation (2.8) with the cost functional J Θ (x; v(·)) by (2.16). By Proposition 3.6 and Theorem 3.7 of [7], system [A Θ ,Ā Θ , C Θ ,C Θ ] is L 2 -globally integrable. We denote by Problem (MF-SLQ) Θ the corresponding mean-field LQ stochastic optimal control problem. The following lemma lists some facts about it, whose proofs are straightforward consequences of Proposition 2.4. Lemma 6.6. We have the following statements.
is an open-loop optimal control of Problem (MF-SLQ).

Concluding Remarks
In this paper, we have established a theory of linear-quadratic two-person (non-zero sum) differential games with mean-field in the infinite horizon [0, ∞). The case of two-person zero-sum, which is also new, has been treated as a special case. Our results recover several existing ones in the literature for infinite horizon problems, including LQ optimal control problems with mean-field ( [7]), two-person zero-sum LQ stochastic differential games (without mean-field) ( [21]), and LQ optimal control problem (without mean-field), giving the equivalence between the open-loop solvability and the closed-loop solvability ( [17]). The finite horizon version of the relevant results can found in [23,16,15,11,18].