Second order necessary conditions for optimal control problems of evolution equations involving final point equality constraints

. We establish some second order necessary conditions for optimal control problems of evolution equations involving ﬁnal point equality and inequality constraints. Compared with the existing works, the main diﬀerence is due to the presence of end-point equality constraints. With such constraints, we cannot simply use the variational techniques since perturbations of a given control may be no longer admissible. We also cannot use the Ekeland’s variational principle, which is a ﬁrst order variational principle, to obtain second order necessary conditions. Instead, we combine some inverse mapping theorems on metric spaces and second order linearization of data to obtain our results


Introduction
Let H be a separable Hilbert space and A : D(A) → H an infinitesimal generator of a C 0 -semigroup {e At } t≥0 on H. Let H 1 be a separable Banach space and U ⊂ H 1 be a nonempty bounded closed subset.Let T > 0. Put U = u : [0, T ] → U u(•) is Lebesgue measurable and define a distance on U as follows: Consider the following control system: x t (t) = Ax(t) + f (t, x(t), u(t)), t ∈ (0, T ], x(0) = x 0 , u ∈ U, where x 0 ∈ H is given and the map f : [0, T ] × H × H 1 → H satisfies the following conditions: (H1) For all (x, u) ∈ H × H 1 , f (•, x, u) is Lebesgue measurable, for all (t, x) ∈ [0, T ] × H, f (t, x, •) is continuous and there exists C > 0 such that (H2) For every r > 0, there exists k r (•) ∈ L 1 (0, T ) such that for a.e.t ∈ [0, T ] Here and in what follows, C denotes a generic constant which may change from line to line.Under the assumptions (H1) and (H2), to each u ∈ U there corresponds a unique (mild) solution x ∈ C([0, T ]; H) of the system (1.1) called a trajectory of (1.1).
It is one of the important issues in optimal control theory to characterize local minimizers.Similarly to the Calculus, the most usual way to do this is to find necessary conditions satisfied by local minimizers.Since the milestone in [17], first-order necessary conditions are studied extensively in the literature for different kinds of control systems, such as systems governed by ordinary differential equations (e.g.[17]), systems governed by partial differential equations (e.g.[13]), systems governed by stochastic differential equations (e.g.[21]), systems governed by stochastic evolution equations (e.g.[15]), etc.
Second order necessary conditions for controlled partial differential equations are studied extensively in the literature (e.g.[3][4][5][18][19][20]).However, as far as we know, there are no published results on second order necessary conditions with end-point equality constraints.The main purpose of this paper is to investigate such kind of problems.Second order necessary conditions have the same purpose as the first order conditions -to eliminate some candidates for optimality that the first order conditions did not throw out.Sufficient conditions are usually obtained by strengthening the obtained second order necessary conditions.This is planned as a future direction of our research.
The main difficulties we face are as follows: -The interior of the tangent space to constraints defined by equalities is, in general, empty.Then one cannot simply use the variational technique to obtain second order necessary conditions since perturbations of the optimal control may be not admissible.-Unlike for the first order necessary condition, it is unclear whether one can use the Ekeland variational principle, which is a first order variational principle, to derive second order necessary conditions.-Unlike in the finite dimensional case, in the Hilbert spaces, in general, two convex sets having an empty intersection can not be separated by a linear functional: at least one of them has to have interior points.
When there are several constraints, this requires all but one of them to have a nonempty interior.In the presence of the end point equality constraints we face two sets with empty interiors, the second one coming from the control system itself, that is usually seen as a second equality constraint.
In the literature these difficulties are usually handled by reducing the optimal control problem to an abstract infinite dimensional mathematical programming one.For this aim it is usually requested that control sets are described via inequality and equality constraints verifying some gradients independence assumptions.In particular, if the set of controls is a union of such sets, instead of their intersection, such methodology does not work anymore.In contrast, in our paper the control set U is an arbitrary nonempty closed subset of a Banach space.
Usually, once such new abstract optimisation problem is stated, some abstract constraints qualification assumptions (Robinson's like conditions) allow to write a Lagrange multiplier rule.The main difficulty is then to return back to the original problem and to translate the derived multiplier rule in terms of the original problem.For this reason very often the authors restrict their attention to very simple sets U , impose some structural assumptions on optimal controls and assume that optimal controls are piecewise continuous.Such assumptions are very strong, because usually, in the presence of constraints, the classical existence results guarantee only measurability of optimal controls with no particular structure.
In the difference with this approach, we do not make reduction of the optimal control of evolution system to an abstract mathematical programming problem.Instead we linearize twice the control system and the constraints in the original state space and apply the separation theorems to the linearized problem.This allows us to work with measurable controls by the methods of variational analysis.Another advantage of our approach is that it brings also sufficient conditions for the normality of the derived necessary conditions.
To overcome the difficulties mentioned above, we borrow some ideas from [8], where an optimal control problem was investigated in the finite dimensional setting.Let us underline that usually the second order tangents to reachable sets of control systems do have an empty interior.Same for the second order tangents to the sets defined by the equality constraints.The separation theorems can not be applied then in the usual way to get second order conditions like in [10,12].We exploit instead the idea from [8] saying that under a surjectivity type assumption imposed on the linearized control system, the intersection of the two second order tangents is the second order tangent to the reachable set of the control system with a final point equality constraint.For this aim a non-standard inverse mapping theorem on a metric space is applied.We remain then with just one set having an empty interior and the application of the separation theorem becomes then possible.Roughly speaking, the inverse mapping theorem on a metric space allows us to consider equality constraints as a part of control system because its second order linearization incorporates the second order linearization of the end-point equality constraints.More details can be found in Section 4.
The rest of this paper is organized as follows: in Section 2, we present our main results, whose proofs are given in Section 4. Section 3 is devoted to some preliminaries.At last, in Section 5, we provide two illustrative examples.

Statement of the main results
To begin with, let us introduce some notations.Let Z be a Banach space with the norm | • | Z .For any subset K ⊂ Z, denote by intK, clK and ∂K the interior, closure and boundary of K, respectively.A subset K is called a cone if αz ∈ K for every α ≥ 0 and z ∈ K.For a general K, denote by co K the smallest closed convex set containing K. Define the distance between a point z ∈ Z and K as and the adjacent cone The Hamiltonian and the terminal Lagrange function are defined respectively by Clearly, for every x ∈ ∂K j , we have g j (x) = 0. Let (x, ū) be a local minimizer of (1.5).Put When f (t, •, •), g j (j = 0, . . ., r) and h are C1 , the first-order necessary condition for local minimizers is as follows: Theorem 2.3.Assume (H1), that the maps f (t, •, •), g j (j = 0, . . ., r) and h are continuously Fréchet differentiable for a.e.t ∈ [0, T ] and Then there exist α = (α 0 , α 1 , . . ., α r ) ∈ R r+1 + and β ∈ H 2 , not vanishing simultaneously, satisfying such that for the (mild we have where H x and H u are the Fréchet derivatives of H with respect to x and u, respectively. When U is convex and f is affine with respect to u, Theorem 2.3 is just the Pontryagin minimum principle for the local minimizer (x, ū).When U is not convex, generally speaking, the conclusion of Theorem 2.3 is weaker than the Pontryagin maximum principle derived by the needle variation technique (e.g., [13]) for the so called strong local minimizers, i.e. when local means that trajectories stay nearby x (instead of controls nearby ū).Note that under our assumptions any strong local minimizer is also a weak local minimizer.However, in general the converse is not true.Thus the first order results for local minimizers, in general, can not be deduced from those known for strong local minimizers.In the recent publication [11], in the finite dimensional setting, the authors derived the second order conditions for local minimizers together with the maximum principle using a relaxation theorem.We do not have yet such analogue for the infinite dimensional context.Theorem 2.3 can be proved by the separation theorem for convex sets.Since the main purpose of this paper is to study second order necessary conditions, we shall derive the first-order condition as a consequence of the second-order one and, so, for C2 data.
To study second order conditions we introduce the set C(x, ū) of all critical pairs (y, u) ∈ Ξ solving the linear system (2.6) such that g 0,x x(T ) y(T ) ≤ 0, (2.8) .9) and The last inequality strengthens the inclusion u(t) ∈ T b U (ū(t)) and is very useful in the second order analysis involving such stronger tangents.Thus the critical set C(x, ū) can be seen as the set of all the solutions to the strengthened linearized system (2.6) that satisfy the linearized final point constraints (2.9), (2.10) and is critical in the sense of (2.8).Our proofs below imply that for any (y, u) ∈ C(x, ū) we have g 0,x x(T ) y(T ) = 0 and, consequently, the word critical is inherited from the classical Calculus.For any (α, β, p) ∈ Λ(x, ū), u : [0, T ] → H 1 and t ∈ [0, T ], define where U (ū(t), u(t)), a.e.t ∈ (0, T ], where f xx [t] = f xx (t, x(t), ū(t)) and f xu [t], f uu [t] are similarly defined.Denote by R L (2) the reachable set at time T of (2.13), which depends on the choice of (y, u).It is well known that the set cl R L (2) is convex.This can be easily deduced from Corollary 3.9 below.Define the convex sets 2) ) . (2.15) Clearly, both Θ and Θ depend on the choice of (y, u).We impose the following additional assumption.
(H4) There is a closed subspace H of H such that Θ ⊂ H and int H Θ = ∅, where int H Θ denotes the interior of Θ in H. Remark 2.4.(H4) is introduced to apply the Hahn-Banach theorem to separate two convex sets.Indeed, it is well known that to separate two convex sets in an infinite dimensional Hilbert space, one of them should have an interior point.Usually, Θ does not satisfy this requirement.To remedy that difficulty, we assume that int H Θ = ∅.Under this condition, for any nonempty convex subset K of H having an empty intersections with Θ one can find a linear functional to separate Θ from K.More details can be found in the proof of Theorem 2.5.
On the other hand, if there is no state constraint (1.4), then (2.17) and (2.18) can be omitted because in this case h would not be involved in our proofs.Since R L is convex and (2.7) is linear in y, (2.17) can be verified by a separation theorem whenever cl h x (x(T ))(R L ) has a nonempty interior.
Recall that if α 0 > 0, then the necessary condition in Theorem 2.5 is called normal.In such case, by normalizing, one can set α 0 = 1.On the other hand, in the abnormal case i.e. when α 0 = 0, the necessary condition is independent of the cost functional.Hence, it is important to show the normality of the multiplier.We present a second-order sufficient condition for normality below.To this end, let us introduce the following linear control system Denote by R the reachable set of (2.20) at time T .Clearly, R is convex.Theorem 2.7.Under all the assumptions of Theorem 2.5 suppose that (2.21) then the conclusion of Theorem 2.5 is valid with α 0 = 1.
If there is no w T as in Theorem 2.7, then the second-order condition holds true in the abnormal form.More generally, we have the following result.
For the sake of completeness we also provide a result without assumption (2.17).
Let X ⊂ H be a separable reflexive Banach space with the norm Gateaux differentiable away from zero.Further suppose that X is dense in H and that the embedding from X to H is continuous.
Several assumptions are in order.
(S1) {e At } t≥0 is a C 0 -semigroup on X and for every t > 0, there is a constant C = C(t) > 0 such that for all η ∈ H, |e At η| X ≤ C|η| H . Further, there exists a constant C > 0 such that for all t ∈ [0, T ) and η ∈ H, measurable in the first variable, continuous in the third variable, and (2.26) By (S1)-(S2), it is easy to prove that for any x 0 ∈ H and u ∈ U there exists a unique mild solution x(•) ∈ C([0, T ]; H) ∩ L 2 (0, T ; X) to the system (1.1) and x(t) ∈ X for every t ∈ (0, T ].Furthermore, for any trajectory-control pair (y, u) of (2.6) we have y ∈ C([0, T ]; X).Similar statement is also valid for trajectories of (2.7).Thus the system in (2.13) is well defined.Moreover R, R L , R L (2) are subsets of X.
Define the convex sets and where cl X denotes the closure in X.
We impose the following assumption.
(S5) There is a closed subspace X of X such that Θ X ⊂ X and int X Θ X = ∅, where int X Θ X denotes the interior of Θ X in X.

Some results of set-valued analysis
For readers' convenience, we collect some basic facts from set-valued analysis.More information can be found in [1].Definition 3.1.Let (M, Σ) be a measurable space, Z a separable Banach space and F : M Z a set-valued map.For any ω ∈ M, F (ω) is called the value of F at ω.The map F is called measurable if where B(Z) is the Borel σ-algebra on Z. Definition 3.2.Let (M, Σ, µ) be a complete σ-finite measure space, Y and Z two complete separable metric spaces.We call g : M × Y → Z a Carathéodory function if for every ω ∈ M, g(ω, •) is continuous and for every y ∈ Y , g(•, y) is measurable. is measurable.
Next, we recall the notion of measurable selection for a set-valued map.
Definition 3.4.Let (M, Σ) be a measurable space and Z a complete separable metric space.Let F be a setvalued map from M to Z.A measurable map f : A sufficient condition for the existence of a measurable selection is as follows.
Recall that for a measure space (M, Σ, µ), the integral of a set-valued map ω Ψ(ω) from M into a Banach space Z is defined by where M ψ(ω)dω is the Bochner integral of ψ(•) over M. We say that Ψ is integrably bounded, if there is a (Lebesgue) integrable k : M → R + such that Ψ(ω) ∈ k(ω)B Z (1) for all ω ∈ M, where B Z (1) stands for the closed unit ball in Z. Lemma 3.7.Let (M, Σ, µ) be a complete measure space with µ(M) < ∞, Y be a separable metric space, and Z be a separable Banach space.Consider a measurable set-valued map S : M Y with closed values and a Carathéodory function Fix a Bochner integrable selection ψ(ω) ∈ cl g(ω, S(ω)) and let δ > 0. Then for By Lemma 3.6, there exists a measurable selection u(ω is measurable, it is the pointwise limit of simple measurable maps {u n } n≥1 .Using that g is Carathéodory we deduce that g(•, u(•)) is pointwise limit of simple measurable maps.Since g(ω, Y ) is integrably bounded, by the Lebesgue dominated convergence theorem, g(•, u(•)) is Bochner integrable.This implies that .
By arbitrariness of δ > 0, we have Corollary 3.9.Suppose that all the assumptions of Lemma 3.7 hold and µ is non-atomic.Then As an immediate corollary of Lemma 2.10 of [12], we have the following result.Lemma 3.10.Let Σ be the σ-algebra of all Lebesgue measurable sets contained in [0, T ], U ⊂ H 1 be closed and u ∈ U. Then the set-valued map t C U (u(t)) is Σ measurable.
We shall also need the following lemma.

Some inverse mapping theorems
The aim of this section is to prove the following two inverse mapping like theorems.
Theorem 3.12.Assume (H1), (H3) and let (x, ū) be a trajectory-control pair of (1.1) with h(x(T )) = 0.If (2.17) holds true, then there exist ε > 0 and c > 0 such that for every trajectory-control pair (x, u) of (1.1) satisfying |u − ū| L 1 (0,T :H1) < ε we can find a trajectory-control pair (x, ũ) of (1.1) satisfying Theorem 3.13.Assume (S1)-(S4) and let (x, ū) be a trajectory-control pair of (1.1) with h(x(T )) = 0.If (2.17) holds true, then there exist ε > 0 and C > 0 such that for every trajectory-control pair (x, u) of (1.1) with |u − ū| L 1 (0,T :H1) < ε we can find a trajectory-control pair (x, ũ) of (1.1) satisfying By the classical theory of evolution equations (e.g.[13], Chap.2, Subsect.5.2), we know that under assumptions (H1) and (H2), to every control u ∈ U there corresponds a unique solution x(•, u) of the control system (1.1).Furthermore, using the Gronwall lemma, it is not difficult to check that the mapping U u → x(•; u) ∈ C([0, T ]; H) is continuous.However, since U is only a metric space, one cannot differentiate the map U u → h(x(T, u)) to get the above inverse mapping theorems.Instead we replace derivatives by variations in the following way: Recall that for a family of subsets {Γ δ } δ>0 of a Banach space Z, the upper set limit is defined by Define the continuous map G : U → H 2 by G(u) = h(x(T, u)).For δ > 0 and u ∈ U, set The first-order contingent variation of G at u ∈ U is defined by Usually, it is difficult to compute the whole set G (1) (u).However, as we show below, the set . This is sufficient for our purposes.
Denote by B H2 the closed unit ball in H 2 .The following result is an immediate consequence of Theorem 3.2 of [7].
Before proving Lemma 3.15, we recall the following result.
Then u j,δ ∈ B δ (u) and e A(t−s) f (s, x(s, u j ), u j ) − f (s, x(s, u), u j ) ds e A(t−s) f (s, x(s, u), u j ) − f (s, x(s, u), u(s)) ds. (3.1) From (H3), we deduce that e A(t−s) Since t ∈ T , there exists k > 0 such that t ∈ T k .Since t is a Lebesgue density point of T k , it follows from (H1) that Combining (3.1), (3.2) and (3.4), we have Consider the following equation: and define the family of operators Φ(•, t; u) on H as follows: Then we have that Therefore, Since j ∈ N is arbitrary, {u j } ∞ j=1 is dense in U , and f is continuous with respect to the third variable, it follows from (3.7) that for a.e.t ∈ [0, T ], Consequently, for every measurable selection v(t) ∈ f (t, x(t, u), U ) − f (t, x(t, u), u(t)) and for a.e.t ∈ [0, T ], we have This implies that Denote by R L,u the reachable set at time T of the following control system: Hence, we only need to show that By the continuity of the map U u → x(•, u) ∈ C([0, T ]; H) we know that Using the Gronwall lemma it is not difficult to show that Let ũ ∈ U. From (3.11), (3.12) and (H3) we deduce that for each ε > 0, there exists δ = δ(ε) > 0 independent from ũ such that for any u ∈ U satisfying d(u, ū) < δ, Observe next that by Lemmas 3.5 and 3.6 for every measurable selection Therefore, by Lemma 3.7, applied with S(t) = co F (t) and g(t, ξ This and Corollary 3.9 applied with S ≡ U and g(t, u Therefore cl h x (x(T ))R L = cl h x (x(T ))R L,ū and the set cl h x (x(T ))R L,ū is convex.The same argument implies that cl h x (x(T ))R L,u is convex for any u ∈ U.By (2.17), for some ρ > 0 we have Fix ε ∈ (0, ρ 2 ) and let δ = δ(ε) be as above.We claim that Otherwise, by the separation theorem, there exists q 1 ∈ H 2 with |q 1 | H2 = 1 such that sup e∈hx(x(T ;u))(R L,u ) e, q 1 H2 < ρ − 2ε. (3.16) e, q 1 H2 + ε.
To prove Theorem 3.13, we need to modify the assumptions of Lemma 3.15.
Proof.By (S1)-(S2), for every u ∈ U and t ∈ (0, T ], x(t, u) ∈ X.Then one can mimic the proof of Lemma 3.15 to deduce Lemma 3.17.The proof of Theorem 3.13 is the same as the one of Theorem 3.12.

Tangents to trajectories satisfying end-point equality constraints
The aim of this section is to prove the following two lemmas.
t ∈ [0, T ]; Then for any ρ ∈ (0, ρ 0 ], there exists a measurable In particular, if v ∈ L 2 (0, T ; H 1 ) and c ∈ L 2 (0, T ), then lim ρ→0+ |v ρ − v| L 2 (0,T ;H1) = 0. (3.17) Then a ρ is measurable.Further, by Lemmas 3.5 and 3.6, there exists a measurable y ρ : [0, T ] → U such that Similarly, there exists a measurable z ρ : [0, T ] → U such that This, together with (3.17), implies that Consequently, Then we have that and Then we have that for some C > 0, Then by the choice of y and w, h(x δ (T )) = o(δ 2 ).By Theorem 3.12, for every δ > 0, there exists a trajectorycontrol pair (x δ , u δ ) of (1.1) such that and h(x δ (T )) = 0. Thus, Similarly to the proof of (3.22), we can get that This, together with (3.27), implies that To prove the last statement, consider trajectory-control pairs (w n , v n ) of (2.13) such that lim Fix n and let (x δ , ũδ ) be as at the beginning of the proof with w replaced by w n .In particular, lim On the other hand, setting we know that lim n→∞ ε n = 0.By the choice of y, for all δ > 0 sufficiently small, By Theorem 3.12 there exists c > 0 independent of n and δn > 0, such that for every δ ∈ (0, δn ], we can find a trajectory-control pair (x δ , u δ ) of (1.1) such that and h(x δ (T )) = 0.Then, by the Gronwall inequality, for a constant C > 0 independent of n, we have Taking δn smaller and keeping the same notation, we may assume that for all δ ∈ (0, δn ].Then for all δ ∈ (0, δn ].Consider next any δ n ∈ (0, δn ] such that δ n → 0+ and define (x n , u n ) = (x δn , u δn ).Then

The proof is complete
The proof of Lemma 3.19 is very similar to that of Lemma 3.18.One only needs to use the smoothing effect to compensate the loss of the regularity of the derivatives of f , h, etc.Indeed, for any x 0 ∈ H, the solution x(t) ∈ X for t > 0. Hence, for t > 0, the terms such as |f x (t, x(t), u)| L(X;H) , |f xx (t, x(t), u)| L(X,X;H) make sense.We omit the details.

Proofs of the main results
We borrow some ideas from [8,10] to prove Theorem 2.5.The key point is to find "admissible perturbations" of the control (perturbation of the local minimizer which still fulfill the endpoints constraints) using Lemma 3.18, the second order linearization of our control system and second order linearizations of the end-point constraints.Then we write a variational inequality satisfied by these perturbations and derive a dual expression of this variational inequality by using the separation theorem.To get the above variational inequality we need a Mangasarian-Fromovitz like constraint qualification.When it fails to hold, again, by using the same kind of arguments based on the separation theorem the necessary conditions follow in the abnormal form.Proof of Theorem 2.5.By (2.18), the mapping h x (x(T )) is surjective.Thus, h x (x(T )) * is injective and Fix (y, u) ∈ C(x, ū) with V 2 (ū, u) ∩ L 2 (0, T ; H 1 ) = ∅.By Lemma 3.11, we obtain that Put and observe that if for some This implies that g ,x (x(T )) = 0, g ,xx (x(T ))(y(T ), y(T )) ≥ 0.
From (4.3), we see that if g j,x (x(T )) = 0, then Q j = H.Put In what follows, for convenience, if I = ∅, then we set j∈I Q j = H.The remaining proof is quite long and is divided into three steps.It is well known that the set cl R L( 2) is convex (it can be verified similarly to the end of the proof of Lemma 3.15, where we have shown that cl (h (2.14) for the definition of Θ).Let (x n , u n ) and δ n → 0+ be as in the last statement of Lemma 3.18.If j / ∈ I g , then for all large n, x n (T ) ∈ Q j and therefore g j (x n (T )) ≤ 0. Also if for some j ∈ I g we have g j,x (x(T ))(y(T )) < 0, then for all large n Finally, if g j,x (x(T ))(y(T )) = 0 for some j ∈ I g , then for all large n Consequently, (x n , u n ) is admissible whenever n is sufficiently large.It follows from the inequality g 0,x (x(T ))(y(T )) ≤ 0 that If g 0,x (x(T )) = 0, set α 0 = 1, α j = 0 for j = 0 and β = 0, p(•) = 0.Then, by the above inequality, the conclusion of Theorem 2.5 holds with this choice of multipliers.From now on we assume that g 0,x (x(T )) = 0.If I = ∅, let I = {j 1 , . . ., j γ }, where 1 ≤ γ ≤ r.The above string of inequalities implies that the convex set does not contain 0. By (H4), we know that Γ has interior points in the product space γ j=0 E j × H, where E j = H for all j = 0, . . ., γ.By the separation theorem for convex sets, there exists a nonzero bounded linear functional ρ on Define an extension ρ of ρ as follows: where P H denotes the orthogonal projection from H to H. Then we have By the Riesz representation theorem, there exist ζ, ζ j ∈ H (j ∈ I ∪ {0}), not vanishing simultaneously, such that for every implying that ζ ∈ ker h x (x(T )) ⊥ .Hence, by (4.1), we get that Set β = −q 1 .Similarly, by the definition of Q j , for every j ∈ I ∪ {0}, we have Consequently, We next observe that for all large λ > 0, −λg j,x (x(T )) ∈ Q j .By (4.6), we have that for some c > 0 Recalling that |g j,x (x(T ))| 2 H > 0, we deduce that λ j ≤ 0. By the very definition of Q j , for every j ∈ I ∪ {0}, Therefore, Set By (4.5), Since ζ * and {ζ * j } j∈I∪{0} do not vanish simultaneously, we have (α, β) = 0. Denote by p(•) the solution to the adjoint equation (2.2) with the final datum p T = p 1 .It follows from (4.6) that inf κ∈R L(2) p(T )(κ) is bounded from below.Since R is a cone, we deduce from (4.2) and (4.6) that inf κ∈R p(T )(κ) ≥ 0.
Let (z, π) be a trajectory-control pair of (2.20).Then From Lemmas 3.10 and 3.5, using that C U (ū(t)) is a cone, we deduce that Step 2. Let (w, v) be a trajectory-control pair of (2.13).From (4.6) and the above calculations we obtain On the other hand, Next, for every ṽ ∈ V 2 (ū, u), there exists Then, by an argument similar to end of Section 5 of [10], we can obtain the inequality (2.19).
Step 3. Assume that does not contain zero.Applying the same separation arguments as in Step 1 and the analysis of Step 2 to this new set Γ and I ∪ {0} replaced by I we obtain (α, β, p) ∈ Λ(x, ū) with α 0 = 0 satisfying (2.19).
The proof of Theorem 2.10 is almost the same as the one of Theorem 2.5.One only needs to use the fact that for all t ∈ (0, T ], x(t), y(t), w(t) ∈ X.We only focus on the differences in the proof.
Step 2. Let (w, v) be a trajectory-control pair of (2.13).From (4.15) and the above calculations we obtain On the other hand, by assumptions (S1)-(S4), The proof ends in the same way as the one of Theorem 2.5.
Proof of Theorem 2.7.By our assumption, if j ∈ I g , then Q j = ∅.From the proof of Theorem 2.5 we know that if g 0,x (x(T )) = 0, then the conclusion of Theorem 2.5 holds with α 0 = 1, β = 0, α j = 0 for all j = 0 and p(•) = 0. Now we consider the case of g 0,x (x(T )) = 0. From the proof of Theorem 2.5, we only need to show that ζ 0 = 0. Otherwise, let w T be as in the assumptions of our theorem.From (4.5) and (4.6), taking the closure of R L (2) in H we have that This and the choice of w T yield ζ j = 0 for every j ∈ I. Consequently, This and (4.6) imply that Then, by the same arguments as in the proof of Theorem 2.5, we can get the conclusion.
Proof of Theorem 2.9.By Theorem 2.5, we only need to deal with the case when 0 is on the boundary of cl h x (x(T ))(R L ).By the assumption of Theorem 2.9 on cl h x (x(T ))(R L ) and the separation theorem, there exists a nonzero q 1 ∈ H 2 such that min Hence, This completes the proof.

Two examples
In this section, we present two illustrative examples for our assumptions to be verified.

A controlled hyperbolic equation
Let D ⊂ R 3 be a bounded domain with a C 2 boundary ∂D.Let D 0 be a nonempty open subset of D. Put Consider the following controlled hyperbolic equation: where and the constraints are hj (ξ)x(T, ξ)dξ = 0, j = 1, . . ., k.

K
(z, v) is closed and may be empty for instance when v / ∈ T b K (z).Let Y 1 , Y 2 and Z be Banach spaces.Denote by L(Y 1 ; Z) the Banach space of all bounded linear operators from Y 1 to Z and by L(Y 1 , Y 2 ; Z) the Banach space of all bounded bilinear operators from Y 1 × Y 2 to Z.When Y 1 = Z, we replace L(Y 1 ; Z) by L(Y 1 ) for simplicity.
Lemma 3.3.([1], Thm.8.2.8)Let (M, Σ, µ) be a complete σ-finite measure space, Y and Z two complete separable metric spaces, and S : M Y a measurable set-valued map with closed values.Let g : M × Y → Z be a Carathéodory function.Then the set-valued map defined by M ω cl g(ω, S(ω))

Lemma 3 .
16. ([2], Thm.2.8) Let Y and Z be separable metric spaces and consider a Carathéodory map F : [0, T ] × Y → Z.Then, for any ε > 0 there exists a compact set T ε ⊂ [0, T ] with the Lebesgue measure of ([0, T ] \ T ε ) smaller than ε and such that the restriction of F to T ε × Y is continuous.Proof of Lemma 3.15.For u ∈ U, let x(•, u) denote the corresponding solution of (1.1).By (H1) the set {x(•, u)} u∈U is bounded in C([0, T ]; H).We borrow some ideas from the proof of Proposition 2.9 in[2].ChoosingY = H × H 1 , Z = H and F = f , by Lemma 3.16, for every k ∈ N, there exists a compact T 1,k ⊂ [0, T ] such that f is continuous on T 1,k × H × H 1and the Lebesgue measure of [0, T ] \ T 1,k is smaller than 1 k .Fix any u ∈ U.By Lemma 3.16 again, for every k ∈ N, there exists a compact T 2,k ⊂ [0, T ] such that u is continuous on T 2,k and the Lebesgue measure of [0, T ] \ T 2,k is smaller than 1 k .Let T k = T 1,k ∩ T 2,k and denote by T k the set of Lebesgue density points in T k .Then the Lebesgue measure of the set T = ∞ k=1 T k is equal to T .Define M = max w∈U |w| H1 + 1 and consider a dense subset {u j } ∞ j=1 of U .Let t ∈ T ∩ (0, T ] and δ ∈ (0, 2M t).For a fixed j ∈ N, define the control