72:: that the optimal control solution in this case is the same as would be obtained in the absence of the additive disturbances. This property is applicable to all centralized systems with linear equations of evolution, quadratic cost function, and noise entering the model only additively; the quadratic assumption allows for the optimal control laws, which follow the certainty-equivalence property, to be linear functions of the observations of the controllers.
958:
In the literature, there are two types of MPCs for stochastic systems; Robust model predictive control and
Stochastic Model Predictive Control (SMPC). Robust model predictive control is a more conservative method which considers the worst scenario in the optimization procedure. However, this method,
99:
In the discrete-time case with uncertainty about the parameter values in the transition matrix (giving the effect of current values of the state variables on their own evolution) and/or the control response matrix of the state equation, but still with a linear state equation and quadratic objective
913:
matrices. But if they are so correlated, then the optimal control solution for each period contains an additional additive constant vector. If an additive constant vector appears in the state equation, then again the optimal control solution for each period contains an additional additive constant
975:
chosen at any time, the determinants of the change in wealth are usually the stochastic returns to assets and the interest rate on the risk-free asset. The field of stochastic control has developed greatly since the 1970s, particularly in its applications to finance. Robert Merton used stochastic
91:
In a discrete-time context, the decision-maker observes the state variable, possibly with observational noise, in each time period. The objective may be to optimize the sum of expected values of a nonlinear (possibly quadratic) objective function over all the time periods from the present to the
100:
function, a
Riccati equation can still be obtained for iterating backward to each period's solution even though certainty equivalence does not apply. The discrete-time case of a non-quadratic loss function but only additive disturbances can also be handled, albeit with more complications.
92:
final period of concern, or to optimize the value of the objective function as of the final period only. At each time period new observations are made, and the control variables are to be adjusted optimally. Finding the optimal solution for the present time may involve iterating a
47:
affects the evolution and observation of the state variables. Stochastic control aims to design the time path of the controlled variables that performs the desired control task with minimum cost, somehow defined, despite the presence of this noise. The context may be either
959:
similar to other robust controls, deteriorates the overall controller's performance and also is applicable only for systems with bounded uncertainties. The alternative method, SMPC, considers soft constraints which limit the risk of violation by a probabilistic inequality.
941:
If the model is in continuous time, the controller knows the state of the system at each instant of time. The objective is to maximize either an integral of, for example, a concave function of a state variable over a horizon from time zero (the present) to a terminal time
889:
616:
68:. Here the model is linear, the objective function is the expected value of a quadratic form, and the disturbances are purely additive. A basic result for discrete-time centralized systems with only additive uncertainty is the
1027:), dynamic programming is used. There is no certainty equivalence as in the older literature, because the coefficients of the control variables—that is, the returns received by the chosen shares of assets—are stochastic.
232:
343:
664:
971:
context, the state variable in the stochastic differential equation is usually wealth or net worth, and the controls are the shares placed at each time in the various assets. Given the
79:
of the model, or decentralization of control—causes the certainty equivalence property not to hold. For example, its failure to hold for decentralized control was demonstrated in
905:
The optimal control solution is unaffected if zero-mean, i.i.d. additive shocks also appear in the state equation, so long as they are uncorrelated with the parameters in the
656:
902:
matrices is the expected value and variance of each element of each matrix and the covariances among elements of the same matrix and among elements across matrices.
458:
39:
that deals with the existence of uncertainty either in observations or in the noise that drives the evolution of the system. The system designer assumes, in a
997:
894:
which is known as the discrete-time dynamic
Riccati equation of this problem. The only information needed regarding the unknown parameters in the
1005:
1127:(1976). "Optimal Stabilization Policies for Stochastic Linear Systems: The Case of Correlated Multiplicative and Additive disturbances".
1036:
1023:
is the main tool of analysis. In the case where the maximization is an integral of a concave function of utility over an horizon (0,
93:
950:. As time evolves, new observations are continuously made and the control variables are continuously adjusted in optimal fashion.
114:
884:{\displaystyle X_{t-1}=Q+\mathrm {E} \left-\mathrm {E} \left\left^{-1}\mathrm {E} \left(B^{\mathsf {T}}X_{t}A\right),}
1402:
1279:
1105:
65:
266:
80:
1009:
1431:
981:
1421:
75:
Any deviation from the above assumptions—a nonlinear state equation, a non-quadratic objective function,
108:
A typical specification of the discrete-time stochastic linear quadratic control problem is to minimize
1164:
Mitchell, Douglas W. (1990). "Tractable Risk
Sensitive Control Based on Approximate Expected Utility".
1129:
1074:
1193:
1019:, is subject to stochastic processes on the components of wealth. In this case, in continuous time
44:
1426:
985:
1051:
76:
20:
1056:
628:
1235:
1220:
Hashemian; Armaou (2017). "Stochastic MPC Design for a Two-Component
Granulation Process".
1124:
611:{\displaystyle u_{t}^{*}=-\left^{-1}\mathrm {E} \left(B^{\mathsf {T}}X_{t}A\right)y_{t-1},}
40:
8:
1372:(1991). "A Simplified Treatment of the Theory of Optimal Regulation of Brownian Motion".
446:
distributed through time, so the expected value operations need not be time-conditional.
1239:
1225:
1202:
1146:
1041:
434:) are known symmetric positive definite cost matrices. We assume that each element of
1398:
1385:
1348:
1275:
1177:
1101:
977:
378:
1191:
Turnovsky, Stephen (1974). "The stability properties of optimal economic policies".
1020:
1381:
1338:
1173:
1138:
1001:
972:
449:
253:
1269:
993:
53:
31:
1343:
1326:
1015:
The maximization, say of the expected logarithm of net worth at a terminal date
1046:
242:
36:
1415:
1369:
1352:
49:
933:
is characterized by removing the time subscripts from its dynamic equation.
1093:
1206:
1150:
64:
An extremely well-studied formulation in stochastic control is that of
925:
goes to infinity, can be found by iterating the dynamic equation for
1142:
1230:
921:(if it exists), relevant for the infinite-horizon problem in which
992:
literature. Influential mathematical textbook treatments were by
989:
968:
452:
can be used to obtain the optimal control solution at each time,
1395:
Stochastic
Controls : Hamiltonian Systems and HJB Equations
1327:"Blockchain Token Economics: A Mean-Field-Type Game Perspective"
946:, or a concave function of a state variable at some future date
96:
backwards in time from the last period to the present period.
443:
621:
with the symmetric positive definite cost-to-go matrix
1312:
Stochastic
Optimal Control and the US Financial Crisis
953:
1324:
667:
631:
461:
269:
117:
227:{\displaystyle \mathrm {E} _{1}\sum _{t=1}^{S}\left}
1297:
Controlled Markov
Processes and Viscosity Solutions
260:is the time horizon, subject to the state equation
883:
650:
610:
337:
226:
1413:
1098:Analysis and Control of Dynamic Economic Systems
1219:
338:{\displaystyle y_{t}=A_{t}y_{t-1}+B_{t}u_{t},}
43:-driven fashion, that random noise with known
1267:
1294:
1271:Deterministic and Stochastic Optimal Control
1119:
1117:
356:Ă— 1 vector of observable state variables,
1342:
1229:
1190:
1123:
1037:Backward stochastic differential equation
1392:
1374:Journal of Economic Dynamics and Control
1325:Barreiro-Gomez, J.; Tembine, H. (2019).
1163:
1114:
59:
1414:
1252:
1088:
1086:
1084:
1082:
854:
796:
752:
709:
565:
505:
200:
167:
77:noise in the multiplicative parameters
1393:Yong, Jiongmin; Zhou, Xun Yu (1999).
1368:
1309:
1303:
917:The steady-state characterization of
1092:
929:repeatedly until it converges; then
1079:
1004:. These techniques were applied by
967:In a continuous time approach in a
954:Stochastic model predictive control
410:matrix of control multipliers, and
13:
1362:
936:
839:
783:
737:
694:
550:
490:
120:
14:
1443:
364:Ă— 1 vector of control variables,
66:linear quadratic Gaussian control
1268:Fleming, W.; Rishel, R. (1975).
625:evolving backwards in time from
86:
1318:
1295:Fleming, W.; Soner, M. (2006).
1288:
1261:
1246:
1213:
1184:
1157:
1068:
821:
787:
402:realization of the stochastic
70:certainty equivalence property
1:
1062:
962:
444:independently and identically
81:Witsenhausen's counterexample
16:Probabilistic optimal control
1386:10.1016/0165-1889(91)90037-2
1178:10.1016/0264-9993(90)90018-Y
252:, superscript T indicates a
7:
1344:10.1109/ACCESS.2019.2917517
1075:Definition from Answers.com
1030:
1010:financial crisis of 2007–08
450:Induction backwards in time
10:
1448:
1130:Review of Economic Studies
988:changed the nature of the
980:of safe and risky assets.
103:
18:
1194:American Economic Review
245:operator conditional on
45:probability distribution
1255:Continuous Time Finance
1253:Merton, Robert (1990).
651:{\displaystyle X_{S}=Q}
387:state transition matrix
94:matrix Riccati equation
1397:. New York: Springer.
1052:Multiplier uncertainty
885:
652:
612:
339:
228:
150:
21:Stochastic programming
1310:Stein, J. L. (2012).
1057:Stochastic scheduling
1000:, and by Fleming and
886:
653:
613:
340:
229:
130:
60:Certainty equivalence
1432:Stochastic processes
665:
629:
459:
267:
115:
41:Bayesian probability
1314:. Springer-Science.
1240:2017arXiv170404710H
1100:. New York: Wiley.
476:
377:realization of the
205:
172:
1422:Stochastic control
1166:Economic Modelling
1125:Turnovsky, Stephen
1042:Stochastic process
978:optimal portfolios
881:
648:
608:
462:
335:
224:
189:
156:
35:is a sub field of
26:Stochastic control
976:control to study
1439:
1408:
1389:
1357:
1356:
1346:
1322:
1316:
1315:
1307:
1301:
1300:
1292:
1286:
1285:
1265:
1259:
1258:
1250:
1244:
1243:
1233:
1222:IEEE Proceedings
1217:
1211:
1210:
1188:
1182:
1181:
1161:
1155:
1154:
1121:
1112:
1111:
1094:Chow, Gregory P.
1090:
1077:
1072:
973:asset allocation
890:
888:
887:
882:
877:
873:
869:
868:
859:
858:
857:
842:
837:
836:
828:
824:
811:
810:
801:
800:
799:
786:
775:
771:
767:
766:
757:
756:
755:
740:
732:
728:
724:
723:
714:
713:
712:
697:
683:
682:
657:
655:
654:
649:
641:
640:
617:
615:
614:
609:
604:
603:
588:
584:
580:
579:
570:
569:
568:
553:
548:
547:
539:
535:
534:
530:
520:
519:
510:
509:
508:
493:
475:
470:
344:
342:
341:
336:
331:
330:
321:
320:
308:
307:
292:
291:
279:
278:
254:matrix transpose
233:
231:
230:
225:
223:
219:
218:
217:
204:
203:
197:
185:
184:
171:
170:
164:
149:
144:
129:
128:
123:
1447:
1446:
1442:
1441:
1440:
1438:
1437:
1436:
1412:
1411:
1405:
1365:
1363:Further reading
1360:
1337:: 64603–64613.
1323:
1319:
1308:
1304:
1293:
1289:
1282:
1266:
1262:
1251:
1247:
1218:
1214:
1189:
1185:
1162:
1158:
1143:10.2307/2296614
1122:
1115:
1108:
1091:
1080:
1073:
1069:
1065:
1033:
965:
956:
939:
937:Continuous time
864:
860:
853:
852:
848:
847:
843:
838:
829:
806:
802:
795:
794:
790:
782:
781:
777:
776:
762:
758:
751:
750:
746:
745:
741:
736:
719:
715:
708:
707:
703:
702:
698:
693:
672:
668:
666:
663:
662:
636:
632:
630:
627:
626:
593:
589:
575:
571:
564:
563:
559:
558:
554:
549:
540:
515:
511:
504:
503:
499:
498:
494:
489:
488:
484:
483:
471:
466:
460:
457:
456:
397:
372:
326:
322:
316:
312:
297:
293:
287:
283:
274:
270:
268:
265:
264:
251:
240:
213:
209:
199:
198:
193:
180:
176:
166:
165:
160:
155:
151:
145:
134:
124:
119:
118:
116:
113:
112:
106:
89:
62:
54:continuous time
32:optimal control
23:
17:
12:
11:
5:
1445:
1435:
1434:
1429:
1427:Control theory
1424:
1410:
1409:
1403:
1390:
1380:(4): 657–673.
1370:Dixit, Avinash
1364:
1361:
1359:
1358:
1317:
1302:
1287:
1280:
1260:
1245:
1212:
1201:(1): 136–148.
1183:
1172:(2): 161–164.
1156:
1113:
1106:
1078:
1066:
1064:
1061:
1060:
1059:
1054:
1049:
1047:Control theory
1044:
1039:
1032:
1029:
1021:ItĂ´'s equation
964:
961:
955:
952:
938:
935:
892:
891:
880:
876:
872:
867:
863:
856:
851:
846:
841:
835:
832:
827:
823:
820:
817:
814:
809:
805:
798:
793:
789:
785:
780:
774:
770:
765:
761:
754:
749:
744:
739:
735:
731:
727:
722:
718:
711:
706:
701:
696:
692:
689:
686:
681:
678:
675:
671:
647:
644:
639:
635:
619:
618:
607:
602:
599:
596:
592:
587:
583:
578:
574:
567:
562:
557:
552:
546:
543:
538:
533:
529:
526:
523:
518:
514:
507:
502:
497:
492:
487:
482:
479:
474:
469:
465:
393:
368:
346:
345:
334:
329:
325:
319:
315:
311:
306:
303:
300:
296:
290:
286:
282:
277:
273:
249:
243:expected value
238:
235:
234:
222:
216:
212:
208:
202:
196:
192:
188:
183:
179:
175:
169:
163:
159:
154:
148:
143:
140:
137:
133:
127:
122:
105:
102:
88:
85:
61:
58:
37:control theory
15:
9:
6:
4:
3:
2:
1444:
1433:
1430:
1428:
1425:
1423:
1420:
1419:
1417:
1406:
1404:0-387-98723-1
1400:
1396:
1391:
1387:
1383:
1379:
1375:
1371:
1367:
1366:
1354:
1350:
1345:
1340:
1336:
1332:
1328:
1321:
1313:
1306:
1298:
1291:
1283:
1281:0-387-90155-8
1277:
1273:
1272:
1264:
1256:
1249:
1241:
1237:
1232:
1227:
1224:: 4386–4391.
1223:
1216:
1208:
1204:
1200:
1196:
1195:
1187:
1179:
1175:
1171:
1167:
1160:
1152:
1148:
1144:
1140:
1137:(1): 191–94.
1136:
1132:
1131:
1126:
1120:
1118:
1109:
1107:0-471-15616-7
1103:
1099:
1095:
1089:
1087:
1085:
1083:
1076:
1071:
1067:
1058:
1055:
1053:
1050:
1048:
1045:
1043:
1040:
1038:
1035:
1034:
1028:
1026:
1022:
1018:
1013:
1011:
1007:
1003:
999:
995:
991:
987:
986:Black–Scholes
983:
979:
974:
970:
960:
951:
949:
945:
934:
932:
928:
924:
920:
915:
912:
908:
903:
901:
897:
878:
874:
870:
865:
861:
849:
844:
833:
830:
825:
818:
815:
812:
807:
803:
791:
778:
772:
768:
763:
759:
747:
742:
733:
729:
725:
720:
716:
704:
699:
690:
687:
684:
679:
676:
673:
669:
661:
660:
659:
658:according to
645:
642:
637:
633:
624:
605:
600:
597:
594:
590:
585:
581:
576:
572:
560:
555:
544:
541:
536:
531:
527:
524:
521:
516:
512:
500:
495:
485:
480:
477:
472:
467:
463:
455:
454:
453:
451:
447:
445:
441:
437:
433:
429:
425:
421:
417:
413:
409:
405:
401:
396:
392:
388:
386:
382:
376:
371:
367:
363:
359:
355:
351:
332:
327:
323:
317:
313:
309:
304:
301:
298:
294:
288:
284:
280:
275:
271:
263:
262:
261:
259:
255:
248:
244:
220:
214:
210:
206:
194:
190:
186:
181:
177:
173:
161:
157:
152:
146:
141:
138:
135:
131:
125:
111:
110:
109:
101:
97:
95:
87:Discrete time
84:
82:
78:
73:
71:
67:
57:
55:
51:
50:discrete time
46:
42:
38:
34:
33:
27:
22:
1394:
1377:
1373:
1334:
1330:
1320:
1311:
1305:
1296:
1290:
1270:
1263:
1257:. Blackwell.
1254:
1248:
1221:
1215:
1198:
1192:
1186:
1169:
1165:
1159:
1134:
1128:
1097:
1070:
1024:
1016:
1014:
984:and that of
966:
957:
947:
943:
940:
930:
926:
922:
918:
916:
910:
906:
904:
899:
895:
893:
622:
620:
448:
439:
435:
431:
427:
423:
419:
415:
411:
407:
403:
399:
398:is the time
394:
390:
384:
380:
374:
373:is the time
369:
365:
361:
357:
353:
349:
347:
257:
246:
236:
107:
98:
90:
74:
69:
63:
29:
25:
24:
1331:IEEE Access
1299:. Springer.
442:is jointly
379:stochastic
30:stochastic
1416:Categories
1231:1704.04710
1063:References
963:In finance
19:See also:
1353:2169-3536
831:−
734:−
677:−
598:−
542:−
481:−
473:∗
302:−
132:∑
1096:(1976).
1031:See also
982:His work
914:vector.
1236:Bibcode
1207:1814888
1151:2296614
1008:to the
994:Fleming
990:finance
969:finance
241:is the
237:where E
104:Example
1401:
1351:
1278:
1205:
1149:
1104:
998:Rishel
422:) and
352:is an
348:where
256:, and
1226:arXiv
1203:JSTOR
1147:JSTOR
1006:Stein
1002:Soner
360:is a
1399:ISBN
1349:ISSN
1276:ISBN
1102:ISBN
996:and
909:and
898:and
438:and
1382:doi
1339:doi
1174:doi
1139:doi
52:or
28:or
1418::
1378:15
1376:.
1347:.
1333:.
1329:.
1274:.
1234:.
1199:64
1197:.
1168:.
1145:.
1135:43
1133:.
1116:^
1081:^
1012:.
430:Ă—
418:Ă—
406:Ă—
389:,
383:Ă—
83:.
56:.
1407:.
1388:.
1384::
1355:.
1341::
1335:7
1284:.
1242:.
1238::
1228::
1209:.
1180:.
1176::
1170:7
1153:.
1141::
1110:.
1025:T
1017:T
948:T
944:T
931:X
927:X
923:S
919:X
911:B
907:A
900:B
896:A
879:,
875:)
871:A
866:t
862:X
855:T
850:B
845:(
840:E
834:1
826:]
822:)
819:R
816:+
813:B
808:t
804:X
797:T
792:B
788:(
784:E
779:[
773:]
769:B
764:t
760:X
753:T
748:A
743:[
738:E
730:]
726:A
721:t
717:X
710:T
705:A
700:[
695:E
691:+
688:Q
685:=
680:1
674:t
670:X
646:Q
643:=
638:S
634:X
623:X
606:,
601:1
595:t
591:y
586:)
582:A
577:t
573:X
566:T
561:B
556:(
551:E
545:1
537:]
532:)
528:R
525:+
522:B
517:t
513:X
506:T
501:B
496:(
491:E
486:[
478:=
468:t
464:u
440:B
436:A
432:k
428:k
426:(
424:R
420:n
416:n
414:(
412:Q
408:k
404:n
400:t
395:t
391:B
385:n
381:n
375:t
370:t
366:A
362:k
358:u
354:n
350:y
333:,
328:t
324:u
318:t
314:B
310:+
305:1
299:t
295:y
289:t
285:A
281:=
276:t
272:y
258:S
250:0
247:y
239:1
221:]
215:t
211:u
207:R
201:T
195:t
191:u
187:+
182:t
178:y
174:Q
168:T
162:t
158:y
153:[
147:S
142:1
139:=
136:t
126:1
121:E
Text is available under the Creative Commons Attribution-ShareAlike License. Additional terms may apply.