Conjugate gradient method

4730: 4075: 4725:{\displaystyle {\begin{aligned}&\mathbf {r} _{0}:=\mathbf {b} -\mathbf {Ax} _{0}\\&{\hbox{if }}\mathbf {r} _{0}{\text{ is sufficiently small, then return }}\mathbf {x} _{0}{\text{ as the result}}\\&\mathbf {p} _{0}:=\mathbf {r} _{0}\\&k:=0\\&{\text{repeat}}\\&\qquad \alpha _{k}:={\frac {\mathbf {r} _{k}^{\mathsf {T}}\mathbf {r} _{k}}{\mathbf {p} _{k}^{\mathsf {T}}\mathbf {Ap} _{k}}}\\&\qquad \mathbf {x} _{k+1}:=\mathbf {x} _{k}+\alpha _{k}\mathbf {p} _{k}\\&\qquad \mathbf {r} _{k+1}:=\mathbf {r} _{k}-\alpha _{k}\mathbf {Ap} _{k}\\&\qquad {\hbox{if }}\mathbf {r} _{k+1}{\text{ is sufficiently small, then exit loop}}\\&\qquad \beta _{k}:={\frac {\mathbf {r} _{k+1}^{\mathsf {T}}\mathbf {r} _{k+1}}{\mathbf {r} _{k}^{\mathsf {T}}\mathbf {r} _{k}}}\\&\qquad \mathbf {p} _{k+1}:=\mathbf {r} _{k+1}+\beta _{k}\mathbf {p} _{k}\\&\qquad k:=k+1\\&{\text{end repeat}}\\&{\text{return }}\mathbf {x} _{k+1}{\text{ as the result}}\end{aligned}}} 10627: 10168: 10622:{\displaystyle {\begin{aligned}\left\|\mathbf {e} _{k}\right\|_{\mathbf {A} }&=\min _{p\in \Pi _{k}^{*}}\left\|p(\mathbf {A} )\mathbf {e} _{0}\right\|_{\mathbf {A} }\\&\leq \min _{p\in \Pi _{k}^{*}}\,\max _{\lambda \in \sigma (\mathbf {A} )}|p(\lambda )|\ \left\|\mathbf {e} _{0}\right\|_{\mathbf {A} }\\&\leq 2\left({\frac {{\sqrt {\kappa (\mathbf {A} )}}-1}{{\sqrt {\kappa (\mathbf {A} )}}+1}}\right)^{k}\ \left\|\mathbf {e} _{0}\right\|_{\mathbf {A} }\\&\leq 2\exp \left({\frac {-2k}{\sqrt {\kappa (\mathbf {A} )}}}\right)\ \left\|\mathbf {e} _{0}\right\|_{\mathbf {A} }\,,\end{aligned}}} 14537: 9302: 20: 1565: 8117: 14291: 3271: 6803: 8807: 9039: 1281: 7829: 648: 8550: 2883: 2990: 14532:{\displaystyle W={\begin{bmatrix}t&{\sqrt {t}}&&&&\\{\sqrt {t}}&1+t&{\sqrt {t}}&&&\\&{\sqrt {t}}&1+t&{\sqrt {t}}&&\\&&{\sqrt {t}}&\ddots &\ddots &\\&&&\ddots &&\\&&&&&{\sqrt {t}}\\&&&&{\sqrt {t}}&1+t\end{bmatrix}},\quad b={\begin{bmatrix}1\\0\\\vdots \\0\end{bmatrix}}} 6542: 8577: 9297:{\displaystyle \alpha _{1}={\frac {\mathbf {r} _{1}^{\mathsf {T}}\mathbf {r} _{1}}{\mathbf {p} _{1}^{\mathsf {T}}\mathbf {Ap} _{1}}}\approx {\frac {{\begin{bmatrix}-0.2810&0.7492\end{bmatrix}}{\begin{bmatrix}-0.2810\\0.7492\end{bmatrix}}}{{\begin{bmatrix}-0.3511&0.7229\end{bmatrix}}{\begin{bmatrix}4&1\\1&3\end{bmatrix}}{\begin{bmatrix}-0.3511\\0.7229\end{bmatrix}}}}=0.4122.} 5773: 1560:{\displaystyle \mathbf {p} _{k}^{\mathsf {T}}\mathbf {b} =\mathbf {p} _{k}^{\mathsf {T}}\mathbf {A} \mathbf {x} _{*}=\sum _{i=1}^{n}\alpha _{i}\mathbf {p} _{k}^{\mathsf {T}}\mathbf {A} \mathbf {p} _{i}=\sum _{i=1}^{n}\alpha _{i}\left\langle \mathbf {p} _{k},\mathbf {p} _{i}\right\rangle _{\mathbf {A} }=\alpha _{k}\left\langle \mathbf {p} _{k},\mathbf {p} _{k}\right\rangle _{\mathbf {A} }} 6467: 8112:{\displaystyle \alpha _{0}={\frac {\mathbf {r} _{0}^{\mathsf {T}}\mathbf {r} _{0}}{\mathbf {p} _{0}^{\mathsf {T}}\mathbf {Ap} _{0}}}={\frac {{\begin{bmatrix}-8&-3\end{bmatrix}}{\begin{bmatrix}-8\\-3\end{bmatrix}}}{{\begin{bmatrix}-8&-3\end{bmatrix}}{\begin{bmatrix}4&1\\1&3\end{bmatrix}}{\begin{bmatrix}-8\\-3\end{bmatrix}}}}={\frac {73}{331}}\approx 0.2205} 488: 9487: 9003: 8305: 2593: 8330: 2685: 3266:{\displaystyle {\begin{aligned}f(\mathbf {x} _{k+1})&=f(\mathbf {x} _{k}+\alpha _{k}\mathbf {p} _{k})=:g(\alpha _{k})\\g'(\alpha _{k})&{\overset {!}{=}}0\quad \Rightarrow \quad \alpha _{k}={\frac {\mathbf {p} _{k}^{\mathsf {T}}(\mathbf {b} -\mathbf {Ax} _{k})}{\mathbf {p} _{k}^{\mathsf {T}}\mathbf {A} \mathbf {p} _{k}}}\,.\end{aligned}}} 7783: 11055: 6798:{\displaystyle \mathbf {p} _{k}^{\mathsf {T}}\mathbf {A} \mathbf {p} _{k}=(\mathbf {r} _{k}+\beta _{k-1}\mathbf {p} _{k-1})^{\mathsf {T}}\mathbf {A} \mathbf {p} _{k}={\frac {1}{\alpha _{k}}}\mathbf {r} _{k}^{\mathsf {T}}(\mathbf {r} _{k}-\mathbf {r} _{k+1})={\frac {1}{\alpha _{k}}}\mathbf {r} _{k}^{\mathsf {T}}\mathbf {r} _{k}} 8802:{\displaystyle \beta _{0}={\frac {\mathbf {r} _{1}^{\mathsf {T}}\mathbf {r} _{1}}{\mathbf {r} _{0}^{\mathsf {T}}\mathbf {r} _{0}}}\approx {\frac {{\begin{bmatrix}-0.2810&0.7492\end{bmatrix}}{\begin{bmatrix}-0.2810\\0.7492\end{bmatrix}}}{{\begin{bmatrix}-8&-3\end{bmatrix}}{\begin{bmatrix}-8\\-3\end{bmatrix}}}}=0.0088.} 2010: 5597: 6263: 1201: 13525: 7391: 3977: 13382:

In numerically challenging applications, sophisticated preconditioners are used, which may lead to variable preconditioning, changing between iterations. Even if the preconditioner is symmetric positive-definite on every iteration, the fact that it may change makes the arguments above invalid, and in

6059: 2432:

must be conjugate to each other. A practical way to enforce this is by requiring that the next search direction be built out of the current residual and all previous search directions. The conjugation constraint is an orthonormal-type constraint and hence the algorithm can be viewed as an example of

643:{\displaystyle \mathbf {u} ^{\mathsf {T}}\mathbf {A} \mathbf {v} =\langle \mathbf {u} ,\mathbf {v} \rangle _{\mathbf {A} }:=\langle \mathbf {A} \mathbf {u} ,\mathbf {v} \rangle =\langle \mathbf {u} ,\mathbf {A} ^{\mathsf {T}}\mathbf {v} \rangle =\langle \mathbf {u} ,\mathbf {A} \mathbf {v} \rangle .} 9533:

it produces the exact solution after a finite number of iterations, which is not larger than the size of the matrix. In practice, the exact solution is never obtained since the conjugate gradient method is unstable with respect to even small perturbations, e.g., most directions are not in practice

3281:

The above algorithm gives the most straightforward explanation of the conjugate gradient method. Seemingly, the algorithm as stated requires storage of all previous searching directions and residue vectors, as well as many matrix–vector multiplications, and thus can be computationally expensive.

9327: 11136:

If initialized randomly, the first stage of iterations is often the fastest, as the error is eliminated within the Krylov subspace that initially reflects a smaller effective condition number. The second stage of convergence is typically well defined by the theoretical convergence bound with

1680: 7569: 8834: 8135: 10814: 8545:{\displaystyle \mathbf {r} _{1}=\mathbf {r} _{0}-\alpha _{0}\mathbf {A} \mathbf {p} _{0}={\begin{bmatrix}-8\\-3\end{bmatrix}}-{\frac {73}{331}}{\begin{bmatrix}4&1\\1&3\end{bmatrix}}{\begin{bmatrix}-8\\-3\end{bmatrix}}\approx {\begin{bmatrix}-0.2810\\0.7492\end{bmatrix}}.} 13647: 11973: 2878:{\displaystyle \alpha _{k}={\frac {\mathbf {p} _{k}^{\mathsf {T}}(\mathbf {b} -\mathbf {Ax} _{k})}{\mathbf {p} _{k}^{\mathsf {T}}\mathbf {A} \mathbf {p} _{k}}}={\frac {\mathbf {p} _{k}^{\mathsf {T}}\mathbf {r} _{k}}{\mathbf {p} _{k}^{\mathsf {T}}\mathbf {A} \mathbf {p} _{k}}},} 2443: 11601: 7612: 370:

problems. Despite differences in their approaches, these derivations share a common topic—proving the orthogonality of the residuals and conjugacy of the search directions. These two properties are crucial to developing the well-known succinct formulation of the method.

12216: 10924: 12305: 12550:

would take more time/computational resources than solving the conjugate gradient algorithm itself. As an example, let's say that we are using a preconditioner coming from incomplete Cholesky factorization. The resulting matrix is the lower triangular matrix

6246: 5768:{\displaystyle \alpha _{k}={\frac {\mathbf {r} _{k}^{\mathsf {T}}\mathbf {r} _{k}}{\mathbf {r} _{k}^{\mathsf {T}}\mathbf {A} \mathbf {p} _{k}}}={\frac {\mathbf {r} _{k}^{\mathsf {T}}\mathbf {r} _{k}}{\mathbf {p} _{k}^{\mathsf {T}}\mathbf {Ap} _{k}}}} 12454:

has to be symmetric positive-definite and fixed, i.e., cannot change from iteration to iteration. If any of these assumptions on the preconditioner is violated, the behavior of the preconditioned conjugate gradient method may become unpredictable.

1895: 3793: 6462:{\displaystyle \mathbf {r} _{k+1}^{\mathsf {T}}\mathbf {A} \mathbf {p} _{k}={\frac {1}{\alpha _{k}}}\mathbf {r} _{k+1}^{\mathsf {T}}(\mathbf {r} _{k}-\mathbf {r} _{k+1})=-{\frac {1}{\alpha _{k}}}\mathbf {r} _{k+1}^{\mathsf {T}}\mathbf {r} _{k+1}} 1057: 13393: 7258: 12861: 6145: 5859: 9971: 5945: 12782: 11860: 9482:{\displaystyle \mathbf {x} _{2}=\mathbf {x} _{1}+\alpha _{1}\mathbf {p} _{1}\approx {\begin{bmatrix}0.2356\\0.3384\end{bmatrix}}+0.4122{\begin{bmatrix}-0.3511\\0.7229\end{bmatrix}}={\begin{bmatrix}0.0909\\0.6364\end{bmatrix}}.} 12055: 11756: 5506: 5135: 11677: 11419: 5002: 11192:

and the spectral distribution of the error. In the last stage, the smallest attainable accuracy is reached and the convergence stalls or the method may even start diverging. In typical scientific computing applications in

2674: 2147: 12408: 4917:, i.e., can be used as a simple implementation of a restart of the conjugate gradient iterations. Restarts could slow down convergence, but may improve stability if the conjugate gradient method misbehaves, e.g., due to 1576: 8998:{\displaystyle \mathbf {p} _{1}=\mathbf {r} _{1}+\beta _{0}\mathbf {p} _{0}\approx {\begin{bmatrix}-0.2810\\0.7492\end{bmatrix}}+0.0088{\begin{bmatrix}-8\\-3\end{bmatrix}}={\begin{bmatrix}-0.3511\\0.7229\end{bmatrix}}.} 8300:{\displaystyle \mathbf {x} _{1}=\mathbf {x} _{0}+\alpha _{0}\mathbf {p} _{0}={\begin{bmatrix}2\\1\end{bmatrix}}+{\frac {73}{331}}{\begin{bmatrix}-8\\-3\end{bmatrix}}\approx {\begin{bmatrix}0.2356\\0.3384\end{bmatrix}}.} 3531: 908: 13146: 7474: 10708: 12310:

The Cholesky decomposition of the preconditioner must be used to keep the symmetry (and positive definiteness) of the system. However, this decomposition does not need to be computed, and it is sufficient to know

452: 5424: 5350: 5202: 14704: 13720: 13191: 10160: 7455: 9570:

to the exact solution and may reach the required tolerance after a relatively small (compared to the problem size) number of iterations. The improvement is typically linear and its speed is determined by the

2588:{\displaystyle \mathbf {p} _{k}=\mathbf {r} _{k}-\sum _{i<k}{\frac {\mathbf {p} _{i}^{\mathsf {T}}\mathbf {A} \mathbf {r} _{k}}{\mathbf {p} _{i}^{\mathsf {T}}\mathbf {A} \mathbf {p} _{i}}}\mathbf {p} _{i}} 11362: 5057: 3392: 2307: 27:

with optimal step size (in green) and conjugate vector (in red) for minimizing a quadratic function associated with a given linear system. Conjugate gradient, assuming exact arithmetic, converges in at most

13540: 11866: 9785: 13921: 804: 11503: 2074: 11115: 7778:{\displaystyle \mathbf {r} _{0}={\begin{bmatrix}1\\2\end{bmatrix}}-{\begin{bmatrix}4&1\\1&3\end{bmatrix}}{\begin{bmatrix}2\\1\end{bmatrix}}={\begin{bmatrix}-8\\-3\end{bmatrix}}=\mathbf {p} _{0}.} 12932: 12698: 12615: 11050:{\displaystyle {\frac {{\sqrt {\kappa (\mathbf {A} )}}-1}{{\sqrt {\kappa (\mathbf {A} )}}+1}}\approx 1-{\frac {2}{\sqrt {\kappa (\mathbf {A} )}}}\quad {\text{for}}\quad \kappa (\mathbf {A} )\gg 1\,.} 12119: 9831: 14212: 12968: 4012: 165: 10173: 4080: 2995: 12227: 11464: 10072: 13982: 1273: 12445: 11282: 11170: 9725: 14850:(2004). "Konrad Zuse und die ERMETH: Ein weltweiter Architektur-Vergleich" [Konrad Zuse and the ERMETH: A worldwide comparison of architectures]. In Hellige, Hans Dieter (ed.). 13277: 13246: 12652: 1237: 1049: 10897: 10696: 10661: 9862: 358:

The conjugate gradient method can be derived from several different perspectives, including specialization of the conjugate direction method for optimization, and variation of the

14824:

On the Extension of the Davidon–Broyden Class of Rank One, Quasi-Newton Minimization Methods to an Infinite Dimensional Hilbert Space with Applications to Optimal Control Problems

2005:{\displaystyle f(\mathbf {x} )={\tfrac {1}{2}}\mathbf {x} ^{\mathsf {T}}\mathbf {A} \mathbf {x} -\mathbf {x} ^{\mathsf {T}}\mathbf {b} ,\qquad \mathbf {x} \in \mathbf {R} ^{n}\,.} 6505: 6155: 5901: 5560: 4882: 3788: 12526: 12341: 11245: 10863: 5256: 10101: 9568: 6534: 5930: 5589: 4915: 4814: 4781: 4063: 3728: 3677: 3648: 3615: 3586: 3472: 3421: 3338: 3309: 2915: 2430: 2388: 2339: 1806: 1777: 1016: 987: 342: 13783: 10837: 13372: 13350: 13328: 13299: 13215: 13084: 13058: 13036: 13014: 12992: 12885: 12571: 12548: 12494: 11304: 5224: 4847: 4034: 3750: 3699: 3443: 1196:{\displaystyle \mathbf {x} _{*}=\sum _{i=1}^{n}\alpha _{i}\mathbf {p} _{i}\Rightarrow \mathbf {A} \mathbf {x} _{*}=\sum _{i=1}^{n}\alpha _{i}\mathbf {A} \mathbf {p} _{i}.} 849: 739: 717: 695: 673: 477: 402: 313: 283: 238: 190: 14036: 14009: 5283: 2983: 2942: 1740: 9682: 9650: 9601: 216: 12089: 11197:

for matrices of large sizes, the conjugate gradient method uses a stopping criterion with a tolerance that terminates the iterations during the first or second stage.

10001: 2598:(see the picture at the top of the article for the effect of the conjugacy constraint on convergence). Following this direction, the next optimal location is given by 14286: 13520:{\displaystyle \beta _{k}:={\frac {\mathbf {r} _{k+1}^{\mathsf {T}}\left(\mathbf {z} _{k+1}-\mathbf {z} _{k}\right)}{\mathbf {r} _{k}^{\mathsf {T}}\mathbf {z} _{k}}}} 10917: 3557: 934: 11492: 7386:{\displaystyle \mathbf {A} \mathbf {x} ={\begin{bmatrix}4&1\\1&3\end{bmatrix}}{\begin{bmatrix}x_{1}\\x_{2}\end{bmatrix}}={\begin{bmatrix}1\\2\end{bmatrix}},} 15076:

Barrett, Richard; Berry, Michael; Chan, Tony F.; Demmel, James; Donato, June; Dongarra, Jack; Eijkhout, Victor; Pozo, Roldan; Romine, Charles; van der Vorst, Henk.

13656:, as it allows for variable preconditioning. The flexible version is also shown to be robust even if the preconditioner is not symmetric positive definite (SPD). 3972:{\displaystyle x_{k}=\mathrm {argmin} _{y\in \mathbb {R} ^{n}}{\left\{(x-y)^{\top }A(x-y):y\in \operatorname {span} \left\{b,Ab,\ldots ,A^{k-1}b\right\}\right\}}} 12791: 6070: 5781: 14586: 14557: 11190: 10025: 9621: 2359: 1713: 954: 827: 14179:

The conjugate gradient method with a trivial modification is extendable to solving, given complex-valued matrix A and vector b, the system of linear equations

9878: 6054:{\displaystyle \beta _{k}=-{\frac {\mathbf {r} _{k+1}^{\mathsf {T}}\mathbf {A} \mathbf {p} _{k}}{\mathbf {p} _{k}^{\mathsf {T}}\mathbf {A} \mathbf {p} _{k}}}} 14079: 13815:

methods is a bit cheaper compared to that for the conjugate gradient methods. However, the latter converge faster, unless a (highly) variable and/or non-SPD

12707: 11784: 14591: 9513:. If exact arithmetic were to be used in this example instead of limited-precision, then the exact solution would theoretically have been reached after 15574: 11979: 11683: 5433: 5062: 15502:

Gérard Meurant: "Detection and correction of silent errors in the conjugate gradient algorithm", Numerical Algorithms, vol.92 (2023), pp.869-891. url=

11607: 11368: 4932: 1675:{\displaystyle \alpha _{k}={\frac {\langle \mathbf {p} _{k},\mathbf {b} \rangle }{\langle \mathbf {p} _{k},\mathbf {p} _{k}\rangle _{\mathbf {A} }}}.} 14882: 14148:) and so the rate of convergence of CGNR may be slow and the quality of the approximate solution may be sensitive to roundoff errors. Finding a good 2604: 2093: 14125:

explicitly in memory but only to perform the matrix–vector and transpose matrix–vector multiplications. Therefore, CGNR is particularly useful when

13838: 12346: 15395: 7564:{\displaystyle \mathbf {x} ={\begin{bmatrix}{\frac {1}{11}}\\\\{\frac {7}{11}}\end{bmatrix}}\approx {\begin{bmatrix}0.0909\\\\0.6364\end{bmatrix}}} 3477: 854: 10809:{\displaystyle k={\tfrac {1}{2}}{\sqrt {\kappa (\mathbf {A} )}}\log \left(\left\|\mathbf {e} _{0}\right\|_{\mathbf {A} }\varepsilon ^{-1}\right)} 13091: 653:

Two vectors are conjugate if and only if they are orthogonal with respect to this inner product. Being conjugate is a symmetric relation: if

410: 5362: 5288: 5140: 15695: 13662: 13152: 10106: 7402: 353: 13642:{\displaystyle \beta _{k}:={\frac {\mathbf {r} _{k+1}^{\mathsf {T}}\mathbf {z} _{k+1}}{\mathbf {r} _{k}^{\mathsf {T}}\mathbf {z} _{k}}}} 11968:{\displaystyle \beta _{k}:={\frac {\mathbf {r} _{k+1}^{\mathsf {T}}\mathbf {z} _{k+1}}{\mathbf {r} _{k}^{\mathsf {T}}\mathbf {z} _{k}}}} 1808:. So, we want to regard the conjugate gradient method as an iterative method. This also allows us to approximately solve systems where 11596:{\displaystyle \alpha _{k}:={\frac {\mathbf {r} _{k}^{\mathsf {T}}\mathbf {z} _{k}}{\mathbf {p} _{k}^{\mathsf {T}}\mathbf {Ap} _{k}}}} 11312: 5007: 3343: 2254: 15720: 13930: 9730: 747: 2025: 13652:

may dramatically improve the convergence in this case. This version of the preconditioned conjugate gradient method can be called

11071: 14758: 14242:

The advantages and disadvantages of the conjugate gradient methods are summarized in the lecture notes by Nemirovsky and BenTal.

14706:

In words, during the CG process, the error grows exponentially, until it suddenly becomes zero as the unique solution is found.

12892: 12658: 15715: 15567: 12211:{\displaystyle \mathbf {E} ^{-1}\mathbf {A} (\mathbf {E} ^{-1})^{\mathsf {T}}\mathbf {\hat {x}} =\mathbf {E} ^{-1}\mathbf {b} } 11194: 14733: 12578: 15518: 15493: 15464: 15445: 15426: 15112: 15052: 15027: 14927: 11124:

is assumed in the convergence theorem, but the convergence bound is commonly valid in practice as theoretically explained by

9790: 14182: 12938: 12300:{\displaystyle \mathbf {EE} ^{\mathsf {T}}=\mathbf {M} ,\qquad \mathbf {\hat {x}} =\mathbf {E} ^{\mathsf {T}}\mathbf {x} .} 3982: 135: 15284:

Knyazev, Andrew V.; Lashuk, Ilya (2008). "Steepest Descent and Conjugate Gradient Methods with Variable Preconditioning".

14748: 14133:

since these operations are usually extremely efficient. However the downside of forming the normal equations is that the

13737: 13531: 13384: 4743: 115: 11425: 10033: 15741: 15674: 15560: 14859: 14722: 12463: 15535: 1242: 15705: 13302: 13061: 12413: 12113:

The above formulation is equivalent to applying the regular conjugate gradient method to the preconditioned system

11250: 1868:

we search for the solution and in each iteration we need a metric to tell us whether we are closer to the solution

9691: 15632: 15159:

Golub, Gene H.; Ye, Qiang (1999). "Inexact Preconditioned Conjugate Gradient Method with Inner-Outer Iteration".

11140: 13383:

practical tests leads to a significant slow down of the convergence of the algorithm presented above. Using the

13251: 13220: 12624: 1209: 1021: 4885: 4784: 8310:

This result completes the first iteration, the result being an "improved" approximate solution to the system,

15545: 13659:

The implementation of the flexible version requires storing an extra vector. For a fixed SPD preconditioner,

2241: 2225: 81: 15664: 13744:

in this case, in particular, it does not converge slower than the locally optimal steepest descent method.

10871: 10670: 10635: 9836: 6833: 6241:{\displaystyle \mathbf {A} \mathbf {p} _{k}={\frac {1}{\alpha _{k}}}(\mathbf {r} _{k}-\mathbf {r} _{k+1}),} 65: 15540: 14716: 111: 6475: 5871: 5530: 4852: 3758: 76:

systems that are too large to be handled by a direct implementation or other direct methods such as the

15618: 12499: 12314: 11218: 10842: 5229: 127: 61: 14993: 10077: 9544: 6510: 5906: 5565: 4891: 4790: 4757: 4039: 3704: 3653: 3624: 3591: 3562: 3448: 3397: 3314: 3285: 2891: 2406: 2364: 2315: 1782: 1753: 992: 963: 318: 15746: 15669: 14728: 88: 15583: 15210: 15173: 14219: 13755: 10819: 9529:

The conjugate gradient method can theoretically be viewed as a direct method, as in the absence of

5204:

mathematically equivalent. The former is used in the algorithm to avoid an extra multiplication by

2434: 253: 14738: 13355: 13333: 13311: 13282: 13198: 13067: 13041: 13019: 12997: 12975: 12868: 12554: 12531: 12477: 11287: 5359:

A norm of the residual is typically used for stopping criteria. The norm of the explicit residual

5207: 4819: 4017: 3733: 3682: 3426: 832: 722: 700: 678: 656: 460: 385: 296: 266: 221: 173: 14014: 13987: 13822: 11065: 5261: 2961: 2920: 1718: 15019: 15013: 9658: 9626: 9577: 15353: 15205: 15168: 957: 195: 77: 14787: 13752:

In both the original and the preconditioned conjugate gradient methods one only needs to set

12061: 9979: 15628: 14945:"Universality for the Conjugate Gradient and MINRES Algorithms on Sample Covariance Matrices" 10902: 3536: 913: 15018:(2nd ed.). Philadelphia, Pa.: Society for Industrial and Applied Mathematics. pp. 14253: 11470: 7396:

we will perform two steps of the conjugate gradient method beginning with the initial guess

5426:

provides a guaranteed level of accuracy both in exact arithmetic and in the presence of the

15623: 12856:{\displaystyle \mathbf {z} =(\mathbf {L} ^{-1})^{\mathsf {T}}\mathbf {L} ^{-1}\mathbf {r} } 10664: 9541:, the conjugate gradient method monotonically (in the energy norm) improves approximations 6140:{\displaystyle \mathbf {r} _{k+1}=\mathbf {r} _{k}-\alpha _{k}\mathbf {A} \mathbf {p} _{k}} 5854:{\displaystyle \mathbf {r} _{k+1}=\mathbf {p} _{k+1}-\mathbf {\beta } _{k}\mathbf {p} _{k}} 1779:

carefully, then we may not need all of them to obtain a good approximation to the solution

8: 15602: 14227: 14223: 480: 92: 14944: 9966:{\displaystyle \Pi _{k}^{*}:=\left\lbrace \ p\in \Pi _{k}\ :\ p(0)=1\ \right\rbrace \,,} 15482: 15476: 15415: 15335: 15311: 15293: 15266: 15246: 14956: 14542: 13736:

The mathematical explanation of the better convergence behavior of the method with the

11306:, a preconditioned conjugate gradient method can be used. It takes the following form: 11175: 10010: 9606: 2344: 1887: 1698: 939: 812: 57: 14562: 15514: 15489: 15460: 15441: 15422: 15389: 15108: 15058: 15048: 15023: 14974: 14923: 14886: 14855: 13924: 11172:, but may be super-linear, depending on a distribution of the spectrum of the matrix 363: 359: 15315: 15270: 15235:"Nonsymmetric Preconditioning for Conjugate Gradient and Steepest Descent Methods 1" 14156: 12777:{\displaystyle \mathbf {M} ^{-1}=(\mathbf {L} ^{-1})^{\mathsf {T}}\mathbf {L} ^{-1}} 11855:{\displaystyle \mathrm {Solve} \ \mathbf {M} \mathbf {z} _{k+1}:=\mathbf {r} _{k+1}} 15638: 15303: 15256: 15215: 15178: 15141: 15079:

Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods

14966: 14915: 14827: 14802: 14168: 14134: 14055: 13832: 13812: 13790: 10699: 9572: 9538: 2391: 2213:. The other vectors in the basis will be conjugate to the gradient, hence the name 241: 69: 24: 19: 15679: 15331: 14779: 14753: 14743: 13828: 13730: 12050:{\displaystyle \mathbf {p} _{k+1}:=\mathbf {z} _{k+1}+\beta _{k}\mathbf {p} _{k}} 11751:{\displaystyle \mathbf {r} _{k+1}:=\mathbf {r} _{k}-\alpha _{k}\mathbf {Ap} _{k}} 11212: 11121: 10004: 9685: 9530: 5509: 5501:{\displaystyle \mathbf {r} _{k+1}:=\mathbf {r} _{k}-\alpha _{k}\mathbf {Ap} _{k}} 5427: 5353: 5130:{\displaystyle \mathbf {r} _{k+1}:=\mathbf {r} _{k}-\alpha _{k}\mathbf {Ap} _{k}} 4918: 3618: 96: 15261: 15234: 15077: 11672:{\displaystyle \mathbf {x} _{k+1}:=\mathbf {x} _{k}+\alpha _{k}\mathbf {p} _{k}} 11414:{\displaystyle {\textrm {Solve:}}\mathbf {M} \mathbf {z} _{0}:=\mathbf {r} _{0}} 11060:

This limit shows a faster convergence rate compared to the iterative methods of

4997:{\displaystyle \mathbf {x} _{k+1}:=\mathbf {x} _{k}+\alpha _{k}\mathbf {p} _{k}} 15597: 14907: 14847: 14783: 14149: 13816: 12459: 11206: 11125: 2669:{\displaystyle \mathbf {x} _{k+1}=\mathbf {x} _{k}+\alpha _{k}\mathbf {p} _{k}} 2142:{\displaystyle \nabla f(\mathbf {x} )=\mathbf {A} \mathbf {x} -\mathbf {b} \,.} 2016: 100: 15219: 15182: 13823:

Conjugate gradient method as optimal feedback controller for double integrator

12403:{\displaystyle \mathbf {E} ^{-1}\mathbf {A} (\mathbf {E} ^{-1})^{\mathsf {T}}} 121: 15735: 15643: 15503: 15062: 14978: 14130: 11215:

is necessary to ensure fast convergence of the conjugate gradient method. If

11061: 104: 73: 15371: 15129: 14919: 14832: 9534:

conjugate, due to a degenerative nature of generating the Krylov subspaces.

5430:, where convergence naturally stagnates. In contrast, the implicit residual 3526:{\displaystyle \mathbf {p} _{i}^{\mathsf {T}}\mathbf {A} \mathbf {p} _{j}=0} 903:{\displaystyle \mathbf {p} _{i}^{\mathsf {T}}\mathbf {A} \mathbf {p} _{j}=0} 15552: 14995:

An Introduction to the Conjugate Gradient Method Without the Agonizing Pain

14588:. Solving it by conjugate gradient descent gives us rather bad convergence: 14174: 6817:

are conjugated and again that the residuals are orthogonal. This gives the

1877:(that is unknown to us). This metric comes from the fact that the solution 13141:{\displaystyle \mathbf {z} =(\mathbf {L} ^{-1})^{\mathsf {T}}\mathbf {a} } 3650:

form the orthogonal basis with respect to the standard inner product, and

14807: 13786: 5285:. The latter may be more accurate, substituting the explicit calculation 290: 45: 13377: 12474:

It is importart to keep in mind that we don't want to invert the matrix

4069:. It is a different formulation of the exact procedure described above. 15370:

Pennington, Fabian Pedregosa, Courtney Paquette, Tom Trogdon, Jeffrey.

6847:

Return the solution to `A * x = b` using the conjugate gradient method.

3679:

form the orthogonal basis with respect to the inner product induced by

367: 15307: 14970: 5508:

is known to keep getting smaller in amplitude well below the level of

87:

The conjugate gradient method can also be used to solve unconstrained

15659: 15298: 15233:

Bouwmeester, Henricus; Dougherty, Andrew; Knyazev, Andrew V. (2015).

14231: 14215: 447:{\displaystyle \mathbf {u} ^{\mathsf {T}}\mathbf {A} \mathbf {v} =0.} 53: 15145: 8564:

that will eventually be used to determine the next search direction

5512:

and thus cannot be used to determine the stagnation of convergence.

5419:{\displaystyle \mathbf {r} _{k+1}:=\mathbf {b} -\mathbf {Ax} _{k+1}} 5356:

accumulation, and is thus recommended for an occasional evaluation.

5345:{\displaystyle \mathbf {r} _{k+1}:=\mathbf {b} -\mathbf {Ax} _{k+1}} 5197:{\displaystyle \mathbf {r} _{k+1}:=\mathbf {b} -\mathbf {Ax} _{k+1}} 15339: 14961: 14699:{\displaystyle \|b-Wx_{k}\|^{2}=(1/t)^{k},\quad \|b-Wx_{n}\|^{2}=0} 13715:{\displaystyle \mathbf {r} _{k+1}^{\mathsf {T}}\mathbf {z} _{k}=0,} 13186:{\displaystyle \mathbf {a} =\mathbf {L} ^{\mathsf {T}}\mathbf {z} } 10155:{\displaystyle \mathbf {e} _{k}:=\mathbf {x} _{k}-\mathbf {x} _{*}} 7788:

Since this is the first iteration, we will use the residual vector

7450:{\displaystyle \mathbf {x} _{0}={\begin{bmatrix}2\\1\end{bmatrix}}} 15251: 13831:. In this approach, the conjugate gradient method falls out as an 13747: 2087:)=0) solves the initial problem follows from its first derivative 479:

is symmetric and positive-definite, the left-hand side defines an

315:

is known as well. We denote the unique solution of this system by

4036:

is a real, symmetric, positive-definite matrix. The input vector

11357:{\displaystyle \mathbf {r} _{0}:=\mathbf {b} -\mathbf {Ax} _{0}} 5052:{\displaystyle \mathbf {r} _{k}:=\mathbf {b} -\mathbf {Ax} _{k}} 3387:{\displaystyle \mathbf {r} _{i}^{\mathsf {T}}\mathbf {r} _{j}=0} 2302:{\displaystyle \mathbf {r} _{k}=\mathbf {b} -\mathbf {Ax} _{k}.} 15710: 15700: 11200: 9780:{\displaystyle \mathbf {M} ^{-1}(\mathbf {Ax} -\mathbf {b} )=0} 6831: 4735:

This is the most commonly used algorithm. The same formula for

14155:

Several algorithms have been proposed (e.g., CGLS, LSQR). The

13916:{\displaystyle u=k(x,v):=-\gamma _{a}\nabla f(x)-\gamma _{b}v} 799:{\displaystyle P=\{\mathbf {p} _{1},\dots ,\mathbf {p} _{n}\}} 15334:, "An Optimal Control Theory for Accelerated Optimization," 15107:(4th ed.). Johns Hopkins University Press. sec. 11.5.2. 14046:

The conjugate gradient method can be applied to an arbitrary

14041: 2069:{\displaystyle \mathbf {H} (f(\mathbf {x} ))=\mathbf {A} \,,} 1812:

is so large that the direct method would take too much time.

114:

provides a generalization to non-symmetric matrices. Various

14852:

Geschichten der Informatik. Visionen, Paradigmen, Leitmotive

14159:

algorithm purportedly has the best numerical stability when

11110:{\displaystyle \approx 1-{\frac {2}{\kappa (\mathbf {A} )}}} 9499:, is a "better" approximation to the system's solution than 80:. Large sparse systems often arise when numerically solving 15455:

Golub, Gene H.; Van Loan, Charles F. (2013). "Chapter 11".

15232: 14788:"Methods of Conjugate Gradients for Solving Linear Systems" 122:

Description of the problem addressed by conjugate gradients

68:. The conjugate gradient method is often implemented as an 12927:{\displaystyle \mathbf {a} =\mathbf {L} ^{-1}\mathbf {r} } 12693:{\displaystyle \mathbf {z} =\mathbf {M} ^{-1}\mathbf {r} } 8317:. We may now move on and compute the next residual vector 6536:

are orthogonal by design. The denominator is rewritten as

15511:

Error Norm Estimation in the Conjugate Gradient Algorithm

15130:"Block Preconditioning for the Conjugate Gradient Method" 5059:, which both hold in exact arithmetic, make the formulas 3559:. This can be regarded that as the algorithm progresses, 1685:

This gives the following method for solving the equation

14175:

Conjugate gradient method for complex Hermitian matrices

13827:

The conjugate gradient method can also be derived using

12610:{\displaystyle \mathbf {M} =\mathbf {LL} ^{\mathsf {T}}} 7460:

in order to find an approximate solution to the system.

1715:

conjugate directions, and then compute the coefficients

15045:

Iterative solution of large sparse systems of equations

14795:

Journal of Research of the National Bureau of Standards

14778: 12469: 10162:. Now, the rate of convergence can be approximated as 3282:

However, a closer analysis of the algorithm shows that

2888:

where the last equality follows from the definition of

2015:

The existence of a unique minimizer is apparent as its

15351: 14565: 14494: 14306: 14256: 14226:. The trivial modification is simply substituting the 13729:

are equivalent in exact arithmetic, i.e., without the

11143: 10719: 10074:

be the iterative approximations of the exact solution

9826:{\displaystyle \kappa (\mathbf {M} ^{-1}\mathbf {A} )} 9455: 9423: 9391: 9261: 9225: 9198: 9167: 9140: 8968: 8933: 8898: 8763: 8733: 8702: 8675: 8555:

Our next step in the process is to compute the scalar

8515: 8480: 8444: 8399: 8273: 8238: 8199: 8060: 8024: 7994: 7960: 7930: 7730: 7701: 7665: 7636: 7537: 7491: 7426: 7359: 7316: 7280: 4454: 4128: 1917: 14594: 14545: 14294: 14207:{\displaystyle \mathbf {A} \mathbf {x} =\mathbf {b} } 14185: 14152:

is often an important part of using the CGNR method.

14118:

As an iterative method, it is not necessary to form

14017: 13990: 13933: 13841: 13758: 13665: 13543: 13396: 13378:

The flexible preconditioned conjugate gradient method

13358: 13336: 13314: 13285: 13254: 13223: 13201: 13155: 13094: 13070: 13044: 13022: 13000: 12978: 12963:{\displaystyle \mathbf {r} =\mathbf {L} \mathbf {a} } 12941: 12895: 12871: 12794: 12710: 12661: 12627: 12581: 12557: 12534: 12502: 12480: 12416: 12349: 12317: 12230: 12122: 12064: 11982: 11869: 11787: 11686: 11610: 11506: 11473: 11428: 11371: 11315: 11290: 11253: 11221: 11178: 11074: 10927: 10905: 10874: 10845: 10822: 10711: 10673: 10638: 10171: 10109: 10080: 10036: 10013: 9982: 9881: 9839: 9793: 9733: 9694: 9661: 9629: 9609: 9580: 9547: 9330: 9042: 8837: 8580: 8333: 8138: 7832: 7615: 7477: 7405: 7261: 6545: 6513: 6478: 6266: 6158: 6073: 5948: 5909: 5874: 5784: 5600: 5568: 5533: 5436: 5365: 5291: 5264: 5232: 5210: 5143: 5065: 5010: 4935: 4894: 4855: 4822: 4793: 4760: 4078: 4042: 4020: 4007:{\displaystyle \mathbf {A} \mathbf {x} =\mathbf {b} } 3985: 3796: 3761: 3736: 3707: 3685: 3656: 3627: 3594: 3565: 3539: 3480: 3451: 3429: 3400: 3346: 3317: 3288: 2993: 2964: 2944:

can be derived if one substitutes the expression for

2923: 2894: 2688: 2607: 2446: 2409: 2367: 2347: 2318: 2257: 2096: 2028: 2019:

of second derivatives is symmetric positive-definite

1898: 1785: 1756: 1721: 1701: 1579: 1284: 1245: 1212: 1060: 1024: 995: 966: 942: 916: 857: 835: 815: 750: 725: 703: 681: 659: 491: 463: 413: 388: 321: 299: 269: 224: 198: 176: 160:{\displaystyle \mathbf {A} \mathbf {x} =\mathbf {b} } 138: 15372:"Random Matrix Theory and Machine Learning Tutorial" 15196:

Notay, Yvan (2000). "Flexible Conjugate Gradients".

15134:

SIAM Journal on Scientific and Statistical Computing

7574:Our first step is to calculate the residual vector 15481: 15475: 15414: 15127: 15085:(2nd ed.). Philadelphia, PA: SIAM. p. 13 15075: 14698: 14580: 14551: 14531: 14280: 14206: 14030: 14003: 13976: 13915: 13777: 13714: 13641: 13519: 13366: 13344: 13322: 13293: 13271: 13240: 13209: 13185: 13140: 13078: 13052: 13030: 13008: 12986: 12962: 12926: 12879: 12855: 12776: 12692: 12646: 12609: 12565: 12542: 12520: 12488: 12439: 12402: 12335: 12299: 12210: 12083: 12049: 11967: 11854: 11750: 11671: 11595: 11486: 11459:{\displaystyle \mathbf {p} _{0}:=\mathbf {z} _{0}} 11458: 11413: 11356: 11298: 11276: 11239: 11184: 11164: 11109: 11049: 10911: 10891: 10857: 10831: 10808: 10690: 10655: 10621: 10154: 10095: 10066: 10019: 9995: 9965: 9856: 9825: 9779: 9719: 9676: 9644: 9615: 9595: 9562: 9481: 9296: 8997: 8801: 8544: 8299: 8111: 7777: 7563: 7449: 7385: 6797: 6528: 6499: 6461: 6240: 6139: 6053: 5924: 5895: 5853: 5767: 5583: 5554: 5500: 5418: 5344: 5277: 5250: 5218: 5196: 5129: 5051: 4996: 4909: 4876: 4841: 4808: 4775: 4724: 4057: 4028: 4006: 3971: 3782: 3744: 3722: 3693: 3671: 3642: 3609: 3580: 3551: 3525: 3466: 3437: 3415: 3386: 3332: 3303: 3265: 2977: 2936: 2909: 2877: 2668: 2587: 2424: 2382: 2353: 2333: 2301: 2141: 2068: 2004: 1800: 1771: 1734: 1707: 1674: 1559: 1267: 1231: 1195: 1043: 1010: 981: 948: 928: 902: 843: 821: 798: 733: 711: 689: 667: 642: 471: 446: 396: 336: 307: 277: 232: 210: 184: 159: 14559:is invertible, there exists a unique solution to 13785:in order to make them locally optimal, using the 12266: 12179: 10067:{\displaystyle \left(\mathbf {x} _{k}\right)_{k}} 7213:# Update squared residual norm for next iteration 5352:for the implicit one by the recursion subject to 15733: 15459:(4th ed.). Johns Hopkins University Press. 15286:SIAM Journal on Matrix Analysis and Applications 15005: 14943:Paquette, Elliot; Trogdon, Thomas (March 2023). 10325: 10296: 10213: 9688:is commonly used to replace the original system 5515: 4924: 2228:provided by this initial step of the algorithm. 347: 118:seek minima of nonlinear optimization problems. 14942: 14826:(PhD thesis). North Carolina State University. 14237: 13977:{\displaystyle {\dot {x}}=v,\quad {\dot {v}}=u} 13748:Vs. the locally optimal steepest descent method 2403:. Here, however, we insist that the directions 1833:(we can assume without loss of generality that 15128:Concus, P.; Golub, G. H.; Meurant, G. (1985). 14949:Communications on Pure and Applied Mathematics 13308:Using this method, there is no need to invert 7588:. This residual is computed from the formula 4148: is sufficiently small, then return 2394:method would require to move in the direction 1886:is also the unique minimizer of the following 1268:{\displaystyle \mathbf {p} _{k}^{\mathsf {T}}} 36:is the size of the matrix of the system (here 15568: 15454: 15283: 15103:Golub, Gene H.; Van Loan, Charles F. (2013). 15102: 12440:{\displaystyle \mathbf {M} ^{-1}\mathbf {A} } 11277:{\displaystyle \mathbf {M} ^{-1}\mathbf {A} } 15582: 15508: 15413:Atkinson, Kendell A. (1988). "Section 8.9". 15394:: CS1 maint: multiple names: authors list ( 14912:Iterative Methods for Solving Linear Systems 14854:(in German). Berlin: Springer. p. 185. 14681: 14658: 14618: 14595: 14214:for the complex-valued vector x, where A is 11201:The preconditioned conjugate gradient method 11165:{\textstyle {\sqrt {\kappa (\mathbf {A} )}}} 9720:{\displaystyle \mathbf {Ax} -\mathbf {b} =0} 3979:The algorithm is detailed below for solving 2152:This suggests taking the first basis vector 1655: 1624: 1619: 1596: 793: 757: 634: 613: 607: 577: 571: 550: 536: 519: 15484:Iterative methods for sparse linear systems 15438:Nonlinear Programming: Analysis and Methods 15015:Iterative methods for sparse linear systems 13301:is easy and computationally cheap by using 13060:is easy and computationally cheap by using 12528:for use it in the process, since inverting 10816:iterations suffices to reduce the error to 9314:using the same method as that used to find 8821:, we can compute the next search direction 4480: is sufficiently small, then exit loop 829:mutually conjugate vectors with respect to 354:Derivation of the conjugate gradient method 15575: 15561: 15504:https://doi.org/10.1007/s11075-022-01380-1 14836:– via NASA Technical Reports Server. 14088:conjugate gradient on the normal equations 14042:Conjugate gradient on the normal equations 13272:{\displaystyle \mathbf {L} ^{\mathsf {T}}} 13241:{\displaystyle \mathbf {L} ^{\mathsf {T}}} 12647:{\displaystyle \mathbf {Mz} =\mathbf {r} } 4065:can be an approximate initial solution or 3276: 1232:{\displaystyle \mathbf {Ax} =\mathbf {b} } 1044:{\displaystyle \mathbf {Ax} =\mathbf {b} } 15297: 15260: 15250: 15209: 15172: 15042: 14960: 14906: 14831: 14806: 14245: 13793:methods. With this substitution, vectors 12080: 11483: 11043: 10611: 10323: 9959: 9524: 3840: 3255: 2135: 2062: 1998: 1745: 969: 15412: 14991: 14821: 11131: 18: 15706:Basic Linear Algebra Subprograms (BLAS) 15354:"Optimization III: Convex Optimization" 15158: 15047:(2nd ed.). Switzerland: Springer. 14846: 13805:, so there is no need to store vectors 13352:explicitly at all, and we still obtain 9024:using the same method as that used for 6970:# Compute initial squared residual norm 2437:. This gives the following expression: 15734: 15435: 15369: 14881: 13685: 13618: 13579: 13496: 13432: 13263: 13232: 13172: 13127: 12827: 12753: 12601: 12394: 12283: 12242: 12167: 11944: 11905: 11569: 11536: 11195:double-precision floating-point format 9867: 9105: 9072: 8643: 8610: 7895: 7862: 7468:For reference, the exact solution is 6777: 6699: 6643: 6559: 6435: 6348: 6286: 6025: 5987: 5741: 5708: 5663: 5630: 4563: 4524: 4292: 4259: 3755:That is, if the CG method starts with 3494: 3360: 3229: 3179: 2846: 2813: 2768: 2718: 2547: 2509: 2159:to be the negative of the gradient of 1963: 1936: 1395: 1325: 1298: 1259: 871: 596: 500: 422: 15556: 15509:Meurant, Gerard; Tichy, Petr (2024). 15421:(2nd ed.). John Wiley and Sons. 15417:An introduction to numerical analysis 15327: 15325: 15195: 10892:{\displaystyle \kappa (\mathbf {A} )} 10691:{\displaystyle \kappa (\mathbf {A} )} 10656:{\displaystyle \sigma (\mathbf {A} )} 9857:{\displaystyle \kappa (\mathbf {A} )} 5591:. The denominator is simplified from 3730:can be regarded as the projection of 15473: 15198:SIAM Journal on Scientific Computing 15161:SIAM Journal on Scientific Computing 15011: 14902: 14900: 14898: 14877: 14875: 14873: 14871: 12573:, and the preconditioner matrix is: 12470:Using the preconditioner in practice 7239: 6949:# Initialize search direction vector 4742:is also used in the Fletcher–Reeves 116:nonlinear conjugate gradient methods 15345: 14759:Sparse matrix–vector multiplication 14749:Nonlinear conjugate gradient method 11284:has a better condition number than 11247:is symmetric positive-definite and 7811:will change in further iterations. 4744:nonlinear conjugate gradient method 1750:If we choose the conjugate vectors 23:A comparison of the convergence of 16:Mathematical optimization algorithm 13: 15406: 15322: 13882: 12865:Let's take an intermediary vector 11801: 11798: 11795: 11792: 11789: 10906: 10307: 10224: 9984: 9915: 9883: 9872:Define a subset of polynomials as 6821:in the algorithm after cancelling 6500:{\displaystyle \mathbf {r} _{k+1}} 5896:{\displaystyle \mathbf {p} _{k+1}} 5555:{\displaystyle \mathbf {r} _{k+1}} 4877:{\displaystyle \mathbf {x} _{k+1}} 3875: 3827: 3824: 3821: 3818: 3815: 3812: 3783:{\displaystyle \mathbf {x} _{0}=0} 2958:and minimizing it with respect to 2097: 989:, and we may express the solution 14: 15758: 15528: 14895: 14868: 14723:Conjugate gradient squared method 13811:. Thus, every iteration of these 13279:is upper triangular, solving for 13038:is lower triangular, solving for 12521:{\displaystyle \mathbf {M} ^{-1}} 12464:incomplete Cholesky factorization 12336:{\displaystyle \mathbf {M} ^{-1}} 11240:{\displaystyle \mathbf {M} ^{-1}} 10858:{\displaystyle \varepsilon >0} 6808:using that the search directions 5251:{\displaystyle \mathbf {Ap} _{k}} 2188:. Starting with an initial guess 374:We say that two non-zero vectors 107:, and extensively researched it. 15536:"Conjugate gradients, method of" 15376:random-matrix-learning.github.io 14739:Iterative method: Linear systems 14200: 14192: 14187: 13693: 13668: 13626: 13607: 13587: 13562: 13504: 13485: 13466: 13445: 13415: 13360: 13338: 13316: 13287: 13257: 13226: 13203: 13179: 13166: 13157: 13134: 13108: 13096: 13072: 13046: 13024: 13002: 12980: 12956: 12951: 12943: 12920: 12906: 12897: 12873: 12849: 12835: 12808: 12796: 12761: 12734: 12713: 12686: 12672: 12663: 12640: 12632: 12629: 12595: 12592: 12583: 12559: 12536: 12505: 12482: 12433: 12419: 12375: 12366: 12352: 12320: 12290: 12277: 12263: 12252: 12236: 12233: 12204: 12190: 12176: 12148: 12139: 12125: 12037: 12006: 11985: 11952: 11933: 11913: 11888: 11836: 11815: 11809: 11738: 11735: 11710: 11689: 11659: 11634: 11613: 11580: 11577: 11558: 11544: 11525: 11446: 11431: 11401: 11386: 11380: 11344: 11341: 11332: 11318: 11292: 11270: 11256: 11224: 11153: 11097: 11030: 11006: 10966: 10940: 10882: 10782: 10766: 10739: 10681: 10646: 10605: 10589: 10565: 10515: 10499: 10461: 10435: 10399: 10383: 10342: 10279: 10262: 10253: 10199: 10183: 10142: 10127: 10112: 10096:{\displaystyle \mathbf {x} _{*}} 10083: 10044: 9847: 9816: 9802: 9764: 9756: 9753: 9736: 9707: 9699: 9696: 9652:is, the slower the improvement. 9563:{\displaystyle \mathbf {x} _{k}} 9550: 9521:being the order of the system). 9373: 9348: 9333: 9116: 9113: 9094: 9080: 9061: 8880: 8855: 8840: 8651: 8632: 8618: 8599: 8381: 8375: 8351: 8336: 8181: 8156: 8141: 7906: 7903: 7884: 7870: 7851: 7795:as our initial search direction 7762: 7618: 7479: 7408: 7268: 7263: 7168:# Update search direction vector 6785: 6766: 6725: 6710: 6688: 6656: 6650: 6621: 6590: 6572: 6566: 6548: 6529:{\displaystyle \mathbf {r} _{k}} 6516: 6481: 6443: 6418: 6374: 6359: 6331: 6299: 6293: 6269: 6216: 6201: 6166: 6160: 6127: 6121: 6097: 6076: 6038: 6032: 6014: 6000: 5994: 5970: 5925:{\displaystyle \mathbf {p} _{k}} 5912: 5877: 5841: 5808: 5787: 5752: 5749: 5730: 5716: 5697: 5676: 5670: 5652: 5638: 5619: 5584:{\displaystyle \mathbf {r} _{k}} 5571: 5536: 5488: 5485: 5460: 5439: 5400: 5397: 5388: 5368: 5326: 5323: 5314: 5294: 5258:is already computed to evaluate 5238: 5235: 5212: 5178: 5175: 5166: 5146: 5117: 5114: 5089: 5068: 5039: 5036: 5027: 5013: 4984: 4959: 4938: 4910:{\displaystyle \mathbf {x} _{k}} 4897: 4858: 4809:{\displaystyle \mathbf {x} _{0}} 4796: 4776:{\displaystyle \mathbf {x} _{1}} 4763: 4697: 4644: 4613: 4592: 4571: 4552: 4532: 4507: 4462: 4437: 4434: 4409: 4388: 4370: 4345: 4324: 4303: 4300: 4281: 4267: 4248: 4190: 4175: 4153: 4136: 4112: 4109: 4100: 4086: 4058:{\displaystyle \mathbf {x} _{0}} 4045: 4022: 4000: 3992: 3987: 3764: 3738: 3723:{\displaystyle \mathbf {x} _{k}} 3710: 3687: 3672:{\displaystyle \mathbf {p} _{i}} 3659: 3643:{\displaystyle \mathbf {r} _{i}} 3630: 3610:{\displaystyle \mathbf {r} _{i}} 3597: 3581:{\displaystyle \mathbf {p} _{i}} 3568: 3507: 3501: 3483: 3467:{\displaystyle \mathbf {p} _{j}} 3454: 3431: 3416:{\displaystyle \mathbf {p} _{i}} 3403: 3368: 3349: 3333:{\displaystyle \mathbf {r} _{j}} 3320: 3304:{\displaystyle \mathbf {r} _{i}} 3291: 3242: 3236: 3218: 3201: 3198: 3189: 3168: 3065: 3040: 3006: 2910:{\displaystyle \mathbf {r} _{k}} 2897: 2859: 2853: 2835: 2821: 2802: 2781: 2775: 2757: 2740: 2737: 2728: 2707: 2656: 2631: 2610: 2575: 2560: 2554: 2536: 2522: 2516: 2498: 2464: 2449: 2425:{\displaystyle \mathbf {p} _{k}} 2412: 2383:{\displaystyle \mathbf {x} _{k}} 2370: 2334:{\displaystyle \mathbf {r} _{k}} 2321: 2286: 2283: 2274: 2260: 2131: 2123: 2118: 2107: 2058: 2044: 2030: 1988: 1979: 1970: 1957: 1948: 1943: 1930: 1906: 1846:, otherwise consider the system 1815:We denote the initial guess for 1801:{\displaystyle \mathbf {x} _{*}} 1788: 1772:{\displaystyle \mathbf {p} _{k}} 1759: 1660: 1644: 1629: 1615: 1601: 1551: 1534: 1519: 1492: 1475: 1460: 1408: 1402: 1384: 1338: 1332: 1314: 1305: 1287: 1248: 1225: 1217: 1214: 1180: 1174: 1129: 1123: 1109: 1063: 1037: 1029: 1026: 1011:{\displaystyle \mathbf {x} _{*}} 998: 982:{\displaystyle \mathbb {R} ^{n}} 884: 878: 860: 837: 783: 762: 727: 705: 683: 661: 630: 625: 617: 603: 590: 581: 567: 559: 554: 541: 531: 523: 512: 507: 494: 465: 434: 429: 416: 390: 337:{\displaystyle \mathbf {x} _{*}} 324: 301: 271: 263:> 0 for all non-zero vectors 226: 178: 153: 145: 140: 15363: 15352:Nemirovsky and Ben-Tal (2023). 15277: 15226: 15189: 15152: 15121: 15096: 15069: 14657: 14482: 14222:, and the symbol ' denotes the 13955: 13799:are always the same as vectors 12259: 11022: 11016: 10868:Note, the important limit when 4659: 4589: 4488: 4452: 4385: 4321: 4229: 3149: 3145: 2435:Gram-Schmidt orthonormalization 1977: 382:are conjugate (with respect to 95:. It is commonly attributed to 64:, namely those whose matrix is 15036: 14985: 14936: 14840: 14815: 14772: 14645: 14630: 14275: 14263: 13894: 13888: 13863: 13851: 13740:formula is that the method is 13122: 13103: 12822: 12803: 12748: 12729: 12458:An example of a commonly used 12389: 12370: 12162: 12143: 11157: 11149: 11101: 11093: 11034: 11026: 11010: 11002: 10970: 10962: 10944: 10936: 10886: 10878: 10776: 10761: 10743: 10735: 10685: 10677: 10650: 10642: 10599: 10584: 10569: 10561: 10509: 10494: 10465: 10457: 10439: 10431: 10393: 10378: 10369: 10365: 10359: 10352: 10346: 10338: 10273: 10257: 10249: 10242: 10193: 10178: 9942: 9936: 9851: 9843: 9820: 9797: 9768: 9749: 9671: 9665: 9639: 9633: 9590: 9584: 7606:, and in our case is equal to 6741: 6705: 6638: 6585: 6390: 6354: 6232: 6196: 3895: 3883: 3871: 3858: 3211: 3185: 3146: 3125: 3112: 3097: 3084: 3075: 3035: 3022: 3001: 2750: 2724: 2111: 2103: 2051: 2048: 2040: 2034: 1910: 1902: 1119: 82:partial differential equations 1: 14992:Shewchuk, Jonathan R (1994). 14765: 14038:are variable feedback gains. 13778:{\displaystyle \beta _{k}:=0} 10832:{\displaystyle 2\varepsilon } 5516:Computation of alpha and beta 4925:Explicit residual calculation 2079:and that the minimizer (use D 1206:Left-multiplying the problem 348:Derivation as a direct method 126:Suppose we want to solve the 15043:Hackbusch, W. (2016-06-21). 14888:Introduction to Optimization 14238:Advantages and disadvantages 13367:{\displaystyle \mathbf {z} } 13345:{\displaystyle \mathbf {L} } 13323:{\displaystyle \mathbf {M} } 13294:{\displaystyle \mathbf {z} } 13210:{\displaystyle \mathbf {a} } 13079:{\displaystyle \mathbf {a} } 13053:{\displaystyle \mathbf {a} } 13031:{\displaystyle \mathbf {L} } 13009:{\displaystyle \mathbf {L} } 12987:{\displaystyle \mathbf {r} } 12880:{\displaystyle \mathbf {a} } 12566:{\displaystyle \mathbf {L} } 12543:{\displaystyle \mathbf {M} } 12489:{\displaystyle \mathbf {M} } 11299:{\displaystyle \mathbf {A} } 6925:# Initialize residual vector 6844:conjugate_gradient!(A, b, x) 6834:Julia (programming language) 5219:{\displaystyle \mathbf {A} } 4842:{\displaystyle \beta _{k}=0} 4029:{\displaystyle \mathbf {A} } 3745:{\displaystyle \mathbf {x} } 3694:{\displaystyle \mathbf {A} } 3438:{\displaystyle \mathbf {A} } 2341:is the negative gradient of 844:{\displaystyle \mathbf {A} } 734:{\displaystyle \mathbf {u} } 712:{\displaystyle \mathbf {v} } 690:{\displaystyle \mathbf {v} } 668:{\displaystyle \mathbf {u} } 472:{\displaystyle \mathbf {A} } 397:{\displaystyle \mathbf {A} } 308:{\displaystyle \mathbf {b} } 278:{\displaystyle \mathbf {x} } 233:{\displaystyle \mathbf {A} } 185:{\displaystyle \mathbf {x} } 7: 15541:Encyclopedia of Mathematics 15474:Saad, Yousef (2003-04-01). 15262:10.1016/j.procs.2015.05.241 14734:Gaussian belief propagation 14717:Biconjugate gradient method 14709: 14064:and right-hand side vector 14031:{\displaystyle \gamma _{b}} 14004:{\displaystyle \gamma _{a}} 13833:optimal feedback controller 12496:explicitly in order to get 10103:, and define the errors as 7463: 7244:Consider the linear system 7030:# Iterate until convergence 5278:{\displaystyle \alpha _{k}} 4749: 2978:{\displaystyle \alpha _{k}} 2937:{\displaystyle \alpha _{k}} 1735:{\displaystyle \alpha _{k}} 112:biconjugate gradient method 103:, who programmed it on the 62:systems of linear equations 10: 15763: 15619:System of linear equations 14250:This example is from Let 14163:is ill-conditioned, i.e., 13086:in the original equation: 12450:The preconditioner matrix 11204: 9677:{\displaystyle \kappa (A)} 9645:{\displaystyle \kappa (A)} 9596:{\displaystyle \kappa (A)} 9008:We now compute the scalar 7814:We now compute the scalar 7802:; the method of selecting 351: 128:system of linear equations 84:or optimization problems. 15688: 15670:Cache-oblivious algorithm 15652: 15611: 15590: 15436:Avriel, Mordecai (2003). 15239:Procedia Computer Science 15220:10.1137/S1064827599362314 15183:10.1137/S1064827597323415 14729:Conjugate residual method 14054:matrix by applying it to 12410:has the same spectrum as 9017:using our newly acquired 2215:conjugate gradient method 211:{\displaystyle n\times n} 50:conjugate gradient method 15742:Numerical linear algebra 15721:General purpose software 15584:Numerical linear algebra 14822:Straeter, T. A. (1971). 14220:positive-definite matrix 13925:double integrator system 12084:{\displaystyle k:=k+1\,} 9996:{\displaystyle \Pi _{k}} 6838: 3752:on the Krylov subspace. 1861:instead). Starting with 14920:10.1137/1.9781611970937 14281:{\textstyle t\in (0,1)} 12619:Then we have to solve: 12343:. It can be shown that 10912:{\displaystyle \infty } 8828:using the relationship 8812:Now, using this scalar 7823:using the relationship 3552:{\displaystyle i\neq j} 3277:The resulting algorithm 929:{\displaystyle i\neq j} 15488:(2nd ed.). SIAM. 14700: 14582: 14553: 14533: 14282: 14246:A pathological example 14208: 14032: 14005: 13978: 13917: 13829:optimal control theory 13779: 13716: 13643: 13521: 13368: 13346: 13324: 13295: 13273: 13242: 13211: 13187: 13142: 13080: 13064:. Then, we substitute 13054: 13032: 13010: 12988: 12964: 12928: 12881: 12857: 12778: 12694: 12648: 12611: 12567: 12544: 12522: 12490: 12441: 12404: 12337: 12301: 12212: 12085: 12051: 11969: 11856: 11772:is sufficiently small 11752: 11673: 11597: 11488: 11487:{\displaystyle k:=0\,} 11460: 11415: 11358: 11300: 11278: 11241: 11186: 11166: 11111: 11051: 10913: 10893: 10859: 10833: 10810: 10692: 10657: 10623: 10156: 10097: 10068: 10021: 9997: 9967: 9858: 9827: 9781: 9721: 9678: 9646: 9617: 9597: 9564: 9525:Convergence properties 9483: 9298: 8999: 8803: 8546: 8301: 8113: 7779: 7565: 7451: 7387: 6799: 6530: 6501: 6463: 6242: 6141: 6055: 5926: 5897: 5855: 5769: 5585: 5556: 5502: 5420: 5346: 5279: 5252: 5220: 5198: 5131: 5053: 4998: 4911: 4878: 4843: 4810: 4777: 4726: 4059: 4030: 4008: 3973: 3784: 3746: 3724: 3695: 3673: 3644: 3611: 3582: 3553: 3527: 3468: 3439: 3417: 3388: 3334: 3305: 3267: 2979: 2938: 2911: 2879: 2670: 2589: 2426: 2384: 2355: 2335: 2303: 2143: 2070: 2006: 1802: 1773: 1746:As an iterative method 1736: 1709: 1676: 1561: 1441: 1371: 1269: 1233: 1197: 1162: 1096: 1045: 1012: 983: 950: 930: 904: 845: 823: 800: 735: 713: 691: 669: 644: 473: 448: 398: 338: 309: 279: 234: 212: 186: 161: 78:Cholesky decomposition 41: 15716:Specialized libraries 15629:Matrix multiplication 15624:Matrix decompositions 15012:Saad, Yousef (2003). 14701: 14583: 14554: 14534: 14283: 14209: 14080:positive-semidefinite 14033: 14006: 13979: 13918: 13780: 13722:so both formulas for 13717: 13644: 13522: 13369: 13347: 13325: 13303:backward substitution 13296: 13274: 13243: 13212: 13188: 13143: 13081: 13055: 13033: 13011: 12989: 12965: 12929: 12882: 12858: 12779: 12695: 12649: 12612: 12568: 12545: 12523: 12491: 12442: 12405: 12338: 12302: 12213: 12086: 12052: 11970: 11857: 11753: 11674: 11598: 11489: 11461: 11416: 11359: 11301: 11279: 11242: 11187: 11167: 11132:Practical convergence 11112: 11052: 10914: 10894: 10860: 10834: 10811: 10693: 10658: 10624: 10157: 10098: 10069: 10022: 9998: 9968: 9859: 9828: 9782: 9722: 9679: 9647: 9618: 9603:of the system matrix 9598: 9565: 9484: 9299: 9000: 8804: 8547: 8302: 8114: 7780: 7566: 7452: 7388: 6800: 6531: 6502: 6464: 6243: 6142: 6056: 5927: 5898: 5856: 5770: 5586: 5557: 5503: 5421: 5347: 5280: 5253: 5221: 5199: 5132: 5054: 4999: 4912: 4879: 4849:would similarly make 4844: 4811: 4778: 4727: 4060: 4031: 4009: 3974: 3785: 3747: 3725: 3696: 3674: 3645: 3612: 3583: 3554: 3528: 3469: 3440: 3418: 3389: 3335: 3306: 3268: 2980: 2939: 2917:. The expression for 2912: 2880: 2671: 2590: 2427: 2385: 2356: 2336: 2304: 2195:, this means we take 2144: 2071: 2007: 1803: 1774: 1737: 1710: 1695:: find a sequence of 1677: 1562: 1421: 1351: 1270: 1234: 1198: 1142: 1076: 1046: 1013: 984: 951: 931: 905: 846: 824: 801: 736: 714: 692: 670: 645: 474: 449: 399: 339: 310: 280: 235: 213: 187: 162: 66:positive-semidefinite 22: 15440:. Dover Publishing. 14808:10.6028/jres.049.044 14592: 14563: 14543: 14292: 14254: 14183: 14015: 13988: 13931: 13839: 13819:is used, see above. 13756: 13663: 13541: 13394: 13356: 13334: 13312: 13283: 13252: 13221: 13199: 13153: 13092: 13068: 13062:forward substitution 13042: 13020: 12998: 12976: 12939: 12893: 12869: 12792: 12708: 12659: 12625: 12579: 12555: 12532: 12500: 12478: 12414: 12347: 12315: 12228: 12120: 12062: 11980: 11867: 11785: 11684: 11608: 11504: 11471: 11426: 11369: 11313: 11288: 11251: 11219: 11176: 11141: 11072: 10925: 10903: 10872: 10843: 10820: 10709: 10671: 10636: 10169: 10107: 10078: 10034: 10011: 9980: 9879: 9837: 9791: 9731: 9692: 9659: 9627: 9607: 9578: 9545: 9328: 9040: 8835: 8578: 8331: 8136: 7830: 7613: 7475: 7403: 7259: 6543: 6511: 6476: 6264: 6156: 6071: 5946: 5907: 5872: 5868:is chosen such that 5782: 5598: 5566: 5531: 5527:is chosen such that 5434: 5363: 5289: 5262: 5230: 5208: 5141: 5063: 5008: 4933: 4892: 4853: 4820: 4791: 4758: 4076: 4040: 4018: 3983: 3794: 3759: 3734: 3705: 3683: 3654: 3625: 3592: 3563: 3537: 3478: 3449: 3427: 3398: 3344: 3315: 3286: 2991: 2962: 2921: 2892: 2686: 2605: 2444: 2407: 2365: 2345: 2316: 2255: 2094: 2026: 1896: 1783: 1754: 1719: 1699: 1577: 1282: 1243: 1210: 1058: 1022: 993: 964: 940: 914: 855: 833: 813: 748: 723: 701: 679: 657: 489: 461: 411: 386: 319: 297: 267: 222: 196: 174: 136: 15603:Numerical stability 15457:Matrix Computations 15105:Matrix Computations 14780:Hestenes, Magnus R. 14228:conjugate transpose 14224:conjugate transpose 14218:(i.e., A' = A) and 13690: 13623: 13584: 13501: 13437: 11949: 11910: 11574: 11541: 10320: 10237: 9896: 9868:Convergence theorem 9110: 9077: 8648: 8615: 8122:We can now compute 7900: 7867: 6856:conjugate_gradient! 6782: 6704: 6564: 6440: 6353: 6291: 6030: 5992: 5746: 5713: 5668: 5635: 4783:is computed by the 4715: as the result 4568: 4529: 4297: 4264: 4165: as the result 3499: 3365: 3234: 3184: 2851: 2818: 2773: 2723: 2552: 2514: 2312:As observed above, 1400: 1330: 1303: 1264: 876: 93:energy minimization 70:iterative algorithm 14696: 14578: 14549: 14529: 14523: 14473: 14278: 14204: 14028: 14001: 13974: 13913: 13775: 13712: 13666: 13639: 13605: 13560: 13517: 13483: 13413: 13364: 13342: 13320: 13291: 13269: 13238: 13207: 13183: 13138: 13076: 13050: 13028: 13006: 12984: 12960: 12924: 12877: 12853: 12774: 12690: 12644: 12607: 12563: 12540: 12518: 12486: 12437: 12400: 12333: 12297: 12208: 12081: 12047: 11965: 11931: 11886: 11852: 11748: 11669: 11593: 11556: 11523: 11484: 11456: 11411: 11354: 11296: 11274: 11237: 11182: 11162: 11107: 11047: 10909: 10889: 10855: 10829: 10806: 10728: 10688: 10653: 10619: 10617: 10350: 10322: 10306: 10239: 10223: 10152: 10093: 10064: 10017: 10007:of maximal degree 9993: 9963: 9882: 9854: 9823: 9777: 9717: 9674: 9642: 9613: 9593: 9560: 9479: 9470: 9441: 9406: 9294: 9279: 9250: 9214: 9185: 9156: 9092: 9059: 8995: 8986: 8954: 8916: 8799: 8784: 8752: 8720: 8691: 8630: 8597: 8542: 8533: 8501: 8469: 8420: 8324:using the formula 8297: 8288: 8259: 8214: 8129:using the formula 8109: 8081: 8049: 8013: 7981: 7949: 7882: 7849: 7775: 7751: 7716: 7690: 7651: 7561: 7555: 7523: 7447: 7441: 7383: 7374: 7345: 7305: 7147:A_search_direction 7090:A_search_direction 7045:A_search_direction 6850:""" 6841:""" 6795: 6764: 6686: 6546: 6526: 6497: 6459: 6416: 6329: 6267: 6238: 6137: 6051: 6012: 5968: 5922: 5893: 5851: 5765: 5728: 5695: 5650: 5617: 5581: 5552: 5520:In the algorithm, 5498: 5416: 5342: 5275: 5248: 5216: 5194: 5127: 5049: 4994: 4907: 4874: 4839: 4806: 4787:method applied to 4773: 4722: 4720: 4550: 4505: 4458: 4279: 4246: 4132: 4055: 4026: 4004: 3969: 3780: 3742: 3720: 3691: 3669: 3640: 3607: 3578: 3549: 3523: 3481: 3464: 3435: 3413: 3384: 3347: 3330: 3301: 3263: 3261: 3216: 3166: 2975: 2934: 2907: 2875: 2833: 2800: 2755: 2705: 2666: 2585: 2534: 2496: 2492: 2422: 2380: 2351: 2331: 2299: 2174:. The gradient of 2139: 2066: 2002: 1926: 1888:quadratic function 1798: 1769: 1732: 1705: 1672: 1557: 1382: 1312: 1285: 1265: 1246: 1229: 1193: 1041: 1008: 979: 946: 926: 900: 858: 841: 819: 796: 731: 709: 687: 665: 640: 469: 444: 394: 334: 305: 275: 230: 208: 192:, where the known 182: 157: 58:numerical solution 42: 15729: 15728: 15520:978-1-61197-785-1 15495:978-0-89871-534-7 15466:978-1-4214-0794-4 15447:978-0-486-43227-4 15428:978-0-471-50023-0 15308:10.1137/060675290 15114:978-1-4214-0794-4 15054:978-3-319-28483-5 15029:978-0-89871-534-7 14971:10.1002/cpa.22081 14929:978-0-89871-396-1 14786:(December 1952). 14581:{\textstyle Wx=b} 14552:{\displaystyle W} 14458: 14443: 14404: 14389: 14369: 14354: 14334: 14319: 13965: 13943: 13637: 13515: 12269: 12182: 11963: 11807: 11591: 11376: 11185:{\displaystyle A} 11160: 11105: 11020: 11014: 11013: 10982: 10973: 10947: 10746: 10727: 10581: 10573: 10572: 10491: 10477: 10468: 10442: 10375: 10324: 10295: 10212: 10020:{\displaystyle k} 9953: 9932: 9926: 9907: 9616:{\displaystyle A} 9307:Finally, we find 9286: 9127: 8791: 8662: 8437: 8231: 8101: 8088: 7917: 7519: 7502: 7240:Numerical example 7123:# Update residual 7096:# Update solution 6762: 6684: 6414: 6327: 6250:the numerator of 6194: 6150:and equivalently 6049: 5763: 5687: 5562:is orthogonal to 5226:since the vector 4716: 4693: 4683: 4582: 4481: 4457: 4314: 4222: 4166: 4149: 4131: 3394:, for i ≠ j. And 3311:is orthogonal to 3253: 3140: 2870: 2792: 2571: 2477: 2354:{\displaystyle f} 1925: 1708:{\displaystyle n} 1667: 949:{\displaystyle P} 822:{\displaystyle n} 254:positive-definite 91:problems such as 15754: 15747:Gradient methods 15639:Matrix splitting 15577: 15570: 15563: 15554: 15553: 15549: 15524: 15499: 15487: 15479: 15470: 15451: 15432: 15420: 15400: 15399: 15393: 15385: 15383: 15382: 15367: 15361: 15360: 15358: 15349: 15343: 15329: 15320: 15319: 15301: 15281: 15275: 15274: 15264: 15254: 15230: 15224: 15223: 15213: 15204:(4): 1444–1460. 15193: 15187: 15186: 15176: 15156: 15150: 15149: 15125: 15119: 15118: 15100: 15094: 15093: 15091: 15090: 15084: 15073: 15067: 15066: 15040: 15034: 15033: 15009: 15003: 15002: 15000: 14989: 14983: 14982: 14964: 14955:(5): 1085–1136. 14940: 14934: 14933: 14904: 14893: 14892: 14879: 14866: 14865: 14844: 14838: 14837: 14835: 14833:2060/19710026200 14819: 14813: 14812: 14810: 14792: 14776: 14705: 14703: 14702: 14697: 14689: 14688: 14679: 14678: 14653: 14652: 14640: 14626: 14625: 14616: 14615: 14587: 14585: 14584: 14579: 14558: 14556: 14555: 14550: 14538: 14536: 14535: 14530: 14528: 14527: 14478: 14477: 14459: 14454: 14451: 14450: 14449: 14448: 14444: 14439: 14436: 14435: 14434: 14433: 14432: 14429: 14428: 14422: 14421: 14420: 14417: 14405: 14400: 14397: 14396: 14393: 14392: 14390: 14385: 14370: 14365: 14362: 14359: 14358: 14357: 14355: 14350: 14335: 14330: 14325: 14324: 14323: 14322: 14320: 14315: 14287: 14285: 14284: 14279: 14213: 14211: 14210: 14205: 14203: 14195: 14190: 14169:condition number 14144:) is equal to κ( 14135:condition number 14086:. The result is 14056:normal equations 14037: 14035: 14034: 14029: 14027: 14026: 14010: 14008: 14007: 14002: 14000: 13999: 13983: 13981: 13980: 13975: 13967: 13966: 13958: 13945: 13944: 13936: 13922: 13920: 13919: 13914: 13909: 13908: 13881: 13880: 13813:steepest descent 13810: 13804: 13798: 13791:steepest descent 13784: 13782: 13781: 13776: 13768: 13767: 13728: 13721: 13719: 13718: 13713: 13702: 13701: 13696: 13689: 13688: 13682: 13671: 13648: 13646: 13645: 13640: 13638: 13636: 13635: 13634: 13629: 13622: 13621: 13615: 13610: 13603: 13602: 13601: 13590: 13583: 13582: 13576: 13565: 13558: 13553: 13552: 13526: 13524: 13523: 13518: 13516: 13514: 13513: 13512: 13507: 13500: 13499: 13493: 13488: 13481: 13480: 13476: 13475: 13474: 13469: 13460: 13459: 13448: 13436: 13435: 13429: 13418: 13411: 13406: 13405: 13373: 13371: 13370: 13365: 13363: 13351: 13349: 13348: 13343: 13341: 13329: 13327: 13326: 13321: 13319: 13300: 13298: 13297: 13292: 13290: 13278: 13276: 13275: 13270: 13268: 13267: 13266: 13260: 13247: 13245: 13244: 13239: 13237: 13236: 13235: 13229: 13216: 13214: 13213: 13208: 13206: 13192: 13190: 13189: 13184: 13182: 13177: 13176: 13175: 13169: 13160: 13147: 13145: 13144: 13139: 13137: 13132: 13131: 13130: 13120: 13119: 13111: 13099: 13085: 13083: 13082: 13077: 13075: 13059: 13057: 13056: 13051: 13049: 13037: 13035: 13034: 13029: 13027: 13015: 13013: 13012: 13007: 13005: 12993: 12991: 12990: 12985: 12983: 12969: 12967: 12966: 12961: 12959: 12954: 12946: 12933: 12931: 12930: 12925: 12923: 12918: 12917: 12909: 12900: 12886: 12884: 12883: 12878: 12876: 12862: 12860: 12859: 12854: 12852: 12847: 12846: 12838: 12832: 12831: 12830: 12820: 12819: 12811: 12799: 12783: 12781: 12780: 12775: 12773: 12772: 12764: 12758: 12757: 12756: 12746: 12745: 12737: 12725: 12724: 12716: 12699: 12697: 12696: 12691: 12689: 12684: 12683: 12675: 12666: 12653: 12651: 12650: 12645: 12643: 12635: 12616: 12614: 12613: 12608: 12606: 12605: 12604: 12598: 12586: 12572: 12570: 12569: 12564: 12562: 12549: 12547: 12546: 12541: 12539: 12527: 12525: 12524: 12519: 12517: 12516: 12508: 12495: 12493: 12492: 12487: 12485: 12446: 12444: 12443: 12438: 12436: 12431: 12430: 12422: 12409: 12407: 12406: 12401: 12399: 12398: 12397: 12387: 12386: 12378: 12369: 12364: 12363: 12355: 12342: 12340: 12339: 12334: 12332: 12331: 12323: 12306: 12304: 12303: 12298: 12293: 12288: 12287: 12286: 12280: 12271: 12270: 12262: 12255: 12247: 12246: 12245: 12239: 12217: 12215: 12214: 12209: 12207: 12202: 12201: 12193: 12184: 12183: 12175: 12172: 12171: 12170: 12160: 12159: 12151: 12142: 12137: 12136: 12128: 12090: 12088: 12087: 12082: 12056: 12054: 12053: 12048: 12046: 12045: 12040: 12034: 12033: 12021: 12020: 12009: 12000: 11999: 11988: 11974: 11972: 11971: 11966: 11964: 11962: 11961: 11960: 11955: 11948: 11947: 11941: 11936: 11929: 11928: 11927: 11916: 11909: 11908: 11902: 11891: 11884: 11879: 11878: 11861: 11859: 11858: 11853: 11851: 11850: 11839: 11830: 11829: 11818: 11812: 11805: 11804: 11757: 11755: 11754: 11749: 11747: 11746: 11741: 11732: 11731: 11719: 11718: 11713: 11704: 11703: 11692: 11678: 11676: 11675: 11670: 11668: 11667: 11662: 11656: 11655: 11643: 11642: 11637: 11628: 11627: 11616: 11602: 11600: 11599: 11594: 11592: 11590: 11589: 11588: 11583: 11573: 11572: 11566: 11561: 11554: 11553: 11552: 11547: 11540: 11539: 11533: 11528: 11521: 11516: 11515: 11493: 11491: 11490: 11485: 11465: 11463: 11462: 11457: 11455: 11454: 11449: 11440: 11439: 11434: 11420: 11418: 11417: 11412: 11410: 11409: 11404: 11395: 11394: 11389: 11383: 11378: 11377: 11374: 11363: 11361: 11360: 11355: 11353: 11352: 11347: 11335: 11327: 11326: 11321: 11305: 11303: 11302: 11297: 11295: 11283: 11281: 11280: 11275: 11273: 11268: 11267: 11259: 11246: 11244: 11243: 11238: 11236: 11235: 11227: 11191: 11189: 11188: 11183: 11171: 11169: 11168: 11163: 11161: 11156: 11145: 11116: 11114: 11113: 11108: 11106: 11104: 11100: 11085: 11056: 11054: 11053: 11048: 11033: 11021: 11018: 11015: 11009: 10998: 10994: 10983: 10981: 10974: 10969: 10958: 10955: 10948: 10943: 10932: 10929: 10918: 10916: 10915: 10910: 10898: 10896: 10895: 10890: 10885: 10864: 10862: 10861: 10856: 10838: 10836: 10835: 10830: 10815: 10813: 10812: 10807: 10805: 10801: 10800: 10799: 10787: 10786: 10785: 10779: 10775: 10774: 10769: 10747: 10742: 10731: 10729: 10720: 10700:condition number 10697: 10695: 10694: 10689: 10684: 10662: 10660: 10659: 10654: 10649: 10628: 10626: 10625: 10620: 10618: 10610: 10609: 10608: 10602: 10598: 10597: 10592: 10579: 10578: 10574: 10568: 10557: 10556: 10545: 10524: 10520: 10519: 10518: 10512: 10508: 10507: 10502: 10489: 10488: 10487: 10482: 10478: 10476: 10469: 10464: 10453: 10450: 10443: 10438: 10427: 10424: 10408: 10404: 10403: 10402: 10396: 10392: 10391: 10386: 10373: 10372: 10355: 10349: 10345: 10321: 10319: 10314: 10288: 10284: 10283: 10282: 10276: 10272: 10271: 10270: 10265: 10256: 10238: 10236: 10231: 10204: 10203: 10202: 10196: 10192: 10191: 10186: 10161: 10159: 10158: 10153: 10151: 10150: 10145: 10136: 10135: 10130: 10121: 10120: 10115: 10102: 10100: 10099: 10094: 10092: 10091: 10086: 10073: 10071: 10070: 10065: 10063: 10062: 10057: 10053: 10052: 10047: 10026: 10024: 10023: 10018: 10002: 10000: 9999: 9994: 9992: 9991: 9972: 9970: 9969: 9964: 9958: 9954: 9951: 9930: 9924: 9923: 9922: 9905: 9895: 9890: 9863: 9861: 9860: 9855: 9850: 9833:is smaller than 9832: 9830: 9829: 9824: 9819: 9814: 9813: 9805: 9786: 9784: 9783: 9778: 9767: 9759: 9748: 9747: 9739: 9726: 9724: 9723: 9718: 9710: 9702: 9683: 9681: 9680: 9675: 9651: 9649: 9648: 9643: 9622: 9620: 9619: 9614: 9602: 9600: 9599: 9594: 9573:condition number 9569: 9567: 9566: 9561: 9559: 9558: 9553: 9539:iterative method 9517:= 2 iterations ( 9488: 9486: 9485: 9480: 9475: 9474: 9446: 9445: 9411: 9410: 9382: 9381: 9376: 9370: 9369: 9357: 9356: 9351: 9342: 9341: 9336: 9303: 9301: 9300: 9295: 9287: 9285: 9284: 9283: 9255: 9254: 9219: 9218: 9191: 9190: 9189: 9161: 9160: 9133: 9128: 9126: 9125: 9124: 9119: 9109: 9108: 9102: 9097: 9090: 9089: 9088: 9083: 9076: 9075: 9069: 9064: 9057: 9052: 9051: 9032: 9016: 9004: 9002: 9001: 8996: 8991: 8990: 8959: 8958: 8921: 8920: 8889: 8888: 8883: 8877: 8876: 8864: 8863: 8858: 8849: 8848: 8843: 8820: 8808: 8806: 8805: 8800: 8792: 8790: 8789: 8788: 8757: 8756: 8726: 8725: 8724: 8696: 8695: 8668: 8663: 8661: 8660: 8659: 8654: 8647: 8646: 8640: 8635: 8628: 8627: 8626: 8621: 8614: 8613: 8607: 8602: 8595: 8590: 8589: 8563: 8551: 8549: 8548: 8543: 8538: 8537: 8506: 8505: 8474: 8473: 8438: 8430: 8425: 8424: 8390: 8389: 8384: 8378: 8373: 8372: 8360: 8359: 8354: 8345: 8344: 8339: 8306: 8304: 8303: 8298: 8293: 8292: 8264: 8263: 8232: 8224: 8219: 8218: 8190: 8189: 8184: 8178: 8177: 8165: 8164: 8159: 8150: 8149: 8144: 8118: 8116: 8115: 8110: 8102: 8094: 8089: 8087: 8086: 8085: 8054: 8053: 8018: 8017: 7987: 7986: 7985: 7954: 7953: 7923: 7918: 7916: 7915: 7914: 7909: 7899: 7898: 7892: 7887: 7880: 7879: 7878: 7873: 7866: 7865: 7859: 7854: 7847: 7842: 7841: 7822: 7784: 7782: 7781: 7776: 7771: 7770: 7765: 7756: 7755: 7721: 7720: 7695: 7694: 7656: 7655: 7627: 7626: 7621: 7581:associated with 7570: 7568: 7567: 7562: 7560: 7559: 7546: 7528: 7527: 7520: 7512: 7507: 7503: 7495: 7482: 7456: 7454: 7453: 7448: 7446: 7445: 7417: 7416: 7411: 7392: 7390: 7389: 7384: 7379: 7378: 7350: 7349: 7342: 7341: 7328: 7327: 7310: 7309: 7271: 7266: 7235: 7232: 7229: 7226: 7223: 7220: 7217: 7214: 7211: 7210:search_direction 7208: 7205: 7202: 7199: 7196: 7193: 7190: 7187: 7184: 7181: 7178: 7175: 7174:search_direction 7172: 7169: 7166: 7163: 7160: 7157: 7154: 7151: 7148: 7145: 7142: 7139: 7136: 7133: 7130: 7127: 7124: 7121: 7120:search_direction 7118: 7115: 7112: 7109: 7106: 7103: 7100: 7097: 7094: 7091: 7088: 7085: 7082: 7081:search_direction 7079: 7076: 7073: 7070: 7067: 7064: 7061: 7058: 7057:search_direction 7055: 7052: 7049: 7046: 7043: 7040: 7037: 7034: 7031: 7028: 7025: 7022: 7019: 7016: 7013: 7010: 7007: 7004: 7001: 6998: 6995: 6992: 6989: 6986: 6983: 6980: 6977: 6974: 6971: 6968: 6965: 6962: 6959: 6956: 6953: 6952:search_direction 6950: 6947: 6944: 6941: 6938: 6935: 6932: 6929: 6926: 6923: 6920: 6917: 6914: 6911: 6908: 6905: 6902: 6899: 6896: 6893: 6890: 6887: 6884: 6881: 6878: 6875: 6872: 6869: 6866: 6863: 6860: 6857: 6854: 6851: 6848: 6845: 6842: 6832:Example code in 6827: 6820: 6804: 6802: 6801: 6796: 6794: 6793: 6788: 6781: 6780: 6774: 6769: 6763: 6761: 6760: 6748: 6740: 6739: 6728: 6719: 6718: 6713: 6703: 6702: 6696: 6691: 6685: 6683: 6682: 6670: 6665: 6664: 6659: 6653: 6648: 6647: 6646: 6636: 6635: 6624: 6618: 6617: 6599: 6598: 6593: 6581: 6580: 6575: 6569: 6563: 6562: 6556: 6551: 6535: 6533: 6532: 6527: 6525: 6524: 6519: 6506: 6504: 6503: 6498: 6496: 6495: 6484: 6468: 6466: 6465: 6460: 6458: 6457: 6446: 6439: 6438: 6432: 6421: 6415: 6413: 6412: 6400: 6389: 6388: 6377: 6368: 6367: 6362: 6352: 6351: 6345: 6334: 6328: 6326: 6325: 6313: 6308: 6307: 6302: 6296: 6290: 6289: 6283: 6272: 6257:is rewritten as 6256: 6247: 6245: 6244: 6239: 6231: 6230: 6219: 6210: 6209: 6204: 6195: 6193: 6192: 6180: 6175: 6174: 6169: 6163: 6146: 6144: 6143: 6138: 6136: 6135: 6130: 6124: 6119: 6118: 6106: 6105: 6100: 6091: 6090: 6079: 6060: 6058: 6057: 6052: 6050: 6048: 6047: 6046: 6041: 6035: 6029: 6028: 6022: 6017: 6010: 6009: 6008: 6003: 5997: 5991: 5990: 5984: 5973: 5966: 5958: 5957: 5938: 5931: 5929: 5928: 5923: 5921: 5920: 5915: 5903:is conjugate to 5902: 5900: 5899: 5894: 5892: 5891: 5880: 5867: 5860: 5858: 5857: 5852: 5850: 5849: 5844: 5838: 5837: 5832: 5823: 5822: 5811: 5802: 5801: 5790: 5774: 5772: 5771: 5766: 5764: 5762: 5761: 5760: 5755: 5745: 5744: 5738: 5733: 5726: 5725: 5724: 5719: 5712: 5711: 5705: 5700: 5693: 5688: 5686: 5685: 5684: 5679: 5673: 5667: 5666: 5660: 5655: 5648: 5647: 5646: 5641: 5634: 5633: 5627: 5622: 5615: 5610: 5609: 5590: 5588: 5587: 5582: 5580: 5579: 5574: 5561: 5559: 5558: 5553: 5551: 5550: 5539: 5526: 5507: 5505: 5504: 5499: 5497: 5496: 5491: 5482: 5481: 5469: 5468: 5463: 5454: 5453: 5442: 5425: 5423: 5422: 5417: 5415: 5414: 5403: 5391: 5383: 5382: 5371: 5351: 5349: 5348: 5343: 5341: 5340: 5329: 5317: 5309: 5308: 5297: 5284: 5282: 5281: 5276: 5274: 5273: 5257: 5255: 5254: 5249: 5247: 5246: 5241: 5225: 5223: 5222: 5217: 5215: 5203: 5201: 5200: 5195: 5193: 5192: 5181: 5169: 5161: 5160: 5149: 5136: 5134: 5133: 5128: 5126: 5125: 5120: 5111: 5110: 5098: 5097: 5092: 5083: 5082: 5071: 5058: 5056: 5055: 5050: 5048: 5047: 5042: 5030: 5022: 5021: 5016: 5003: 5001: 5000: 4995: 4993: 4992: 4987: 4981: 4980: 4968: 4967: 4962: 4953: 4952: 4941: 4916: 4914: 4913: 4908: 4906: 4905: 4900: 4886:gradient descent 4884:computed by the 4883: 4881: 4880: 4875: 4873: 4872: 4861: 4848: 4846: 4845: 4840: 4832: 4831: 4815: 4813: 4812: 4807: 4805: 4804: 4799: 4785:gradient descent 4782: 4780: 4779: 4774: 4772: 4771: 4766: 4741: 4731: 4729: 4728: 4723: 4721: 4717: 4714: 4712: 4711: 4700: 4694: 4691: 4688: 4684: 4681: 4678: 4657: 4653: 4652: 4647: 4641: 4640: 4628: 4627: 4616: 4607: 4606: 4595: 4587: 4583: 4581: 4580: 4579: 4574: 4567: 4566: 4560: 4555: 4548: 4547: 4546: 4535: 4528: 4527: 4521: 4510: 4503: 4498: 4497: 4486: 4482: 4479: 4477: 4476: 4465: 4459: 4455: 4450: 4446: 4445: 4440: 4431: 4430: 4418: 4417: 4412: 4403: 4402: 4391: 4383: 4379: 4378: 4373: 4367: 4366: 4354: 4353: 4348: 4339: 4338: 4327: 4319: 4315: 4313: 4312: 4311: 4306: 4296: 4295: 4289: 4284: 4277: 4276: 4275: 4270: 4263: 4262: 4256: 4251: 4244: 4239: 4238: 4227: 4223: 4220: 4217: 4203: 4199: 4198: 4193: 4184: 4183: 4178: 4171: 4167: 4164: 4162: 4161: 4156: 4150: 4147: 4145: 4144: 4139: 4133: 4129: 4125: 4121: 4120: 4115: 4103: 4095: 4094: 4089: 4082: 4064: 4062: 4061: 4056: 4054: 4053: 4048: 4035: 4033: 4032: 4027: 4025: 4013: 4011: 4010: 4005: 4003: 3995: 3990: 3978: 3976: 3975: 3970: 3968: 3967: 3963: 3962: 3958: 3954: 3953: 3879: 3878: 3851: 3850: 3849: 3848: 3843: 3830: 3806: 3805: 3789: 3787: 3786: 3781: 3773: 3772: 3767: 3751: 3749: 3748: 3743: 3741: 3729: 3727: 3726: 3721: 3719: 3718: 3713: 3700: 3698: 3697: 3692: 3690: 3678: 3676: 3675: 3670: 3668: 3667: 3662: 3649: 3647: 3646: 3641: 3639: 3638: 3633: 3616: 3614: 3613: 3608: 3606: 3605: 3600: 3587: 3585: 3584: 3579: 3577: 3576: 3571: 3558: 3556: 3555: 3550: 3532: 3530: 3529: 3524: 3516: 3515: 3510: 3504: 3498: 3497: 3491: 3486: 3473: 3471: 3470: 3465: 3463: 3462: 3457: 3444: 3442: 3441: 3436: 3434: 3422: 3420: 3419: 3414: 3412: 3411: 3406: 3393: 3391: 3390: 3385: 3377: 3376: 3371: 3364: 3363: 3357: 3352: 3339: 3337: 3336: 3331: 3329: 3328: 3323: 3310: 3308: 3307: 3302: 3300: 3299: 3294: 3272: 3270: 3269: 3264: 3262: 3254: 3252: 3251: 3250: 3245: 3239: 3233: 3232: 3226: 3221: 3214: 3210: 3209: 3204: 3192: 3183: 3182: 3176: 3171: 3164: 3159: 3158: 3141: 3133: 3124: 3123: 3111: 3096: 3095: 3074: 3073: 3068: 3062: 3061: 3049: 3048: 3043: 3021: 3020: 3009: 2984: 2982: 2981: 2976: 2974: 2973: 2943: 2941: 2940: 2935: 2933: 2932: 2916: 2914: 2913: 2908: 2906: 2905: 2900: 2884: 2882: 2881: 2876: 2871: 2869: 2868: 2867: 2862: 2856: 2850: 2849: 2843: 2838: 2831: 2830: 2829: 2824: 2817: 2816: 2810: 2805: 2798: 2793: 2791: 2790: 2789: 2784: 2778: 2772: 2771: 2765: 2760: 2753: 2749: 2748: 2743: 2731: 2722: 2721: 2715: 2710: 2703: 2698: 2697: 2675: 2673: 2672: 2667: 2665: 2664: 2659: 2653: 2652: 2640: 2639: 2634: 2625: 2624: 2613: 2594: 2592: 2591: 2586: 2584: 2583: 2578: 2572: 2570: 2569: 2568: 2563: 2557: 2551: 2550: 2544: 2539: 2532: 2531: 2530: 2525: 2519: 2513: 2512: 2506: 2501: 2494: 2491: 2473: 2472: 2467: 2458: 2457: 2452: 2431: 2429: 2428: 2423: 2421: 2420: 2415: 2392:gradient descent 2389: 2387: 2386: 2381: 2379: 2378: 2373: 2360: 2358: 2357: 2352: 2340: 2338: 2337: 2332: 2330: 2329: 2324: 2308: 2306: 2305: 2300: 2295: 2294: 2289: 2277: 2269: 2268: 2263: 2187: 2148: 2146: 2145: 2140: 2134: 2126: 2121: 2110: 2075: 2073: 2072: 2067: 2061: 2047: 2033: 2011: 2009: 2008: 2003: 1997: 1996: 1991: 1982: 1973: 1968: 1967: 1966: 1960: 1951: 1946: 1941: 1940: 1939: 1933: 1927: 1918: 1909: 1885: 1876: 1845: 1832: 1823: 1807: 1805: 1804: 1799: 1797: 1796: 1791: 1778: 1776: 1775: 1770: 1768: 1767: 1762: 1741: 1739: 1738: 1733: 1731: 1730: 1714: 1712: 1711: 1706: 1694: 1681: 1679: 1678: 1673: 1668: 1666: 1665: 1664: 1663: 1653: 1652: 1647: 1638: 1637: 1632: 1622: 1618: 1610: 1609: 1604: 1594: 1589: 1588: 1566: 1564: 1563: 1558: 1556: 1555: 1554: 1548: 1544: 1543: 1542: 1537: 1528: 1527: 1522: 1510: 1509: 1497: 1496: 1495: 1489: 1485: 1484: 1483: 1478: 1469: 1468: 1463: 1451: 1450: 1440: 1435: 1417: 1416: 1411: 1405: 1399: 1398: 1392: 1387: 1381: 1380: 1370: 1365: 1347: 1346: 1341: 1335: 1329: 1328: 1322: 1317: 1308: 1302: 1301: 1295: 1290: 1274: 1272: 1271: 1266: 1263: 1262: 1256: 1251: 1239:with the vector 1238: 1236: 1235: 1230: 1228: 1220: 1202: 1200: 1199: 1194: 1189: 1188: 1183: 1177: 1172: 1171: 1161: 1156: 1138: 1137: 1132: 1126: 1118: 1117: 1112: 1106: 1105: 1095: 1090: 1072: 1071: 1066: 1050: 1048: 1047: 1042: 1040: 1032: 1017: 1015: 1014: 1009: 1007: 1006: 1001: 988: 986: 985: 980: 978: 977: 972: 955: 953: 952: 947: 935: 933: 932: 927: 909: 907: 906: 901: 893: 892: 887: 881: 875: 874: 868: 863: 850: 848: 847: 842: 840: 828: 826: 825: 820: 805: 803: 802: 797: 792: 791: 786: 771: 770: 765: 740: 738: 737: 732: 730: 719:is conjugate to 718: 716: 715: 710: 708: 696: 694: 693: 688: 686: 675:is conjugate to 674: 672: 671: 666: 664: 649: 647: 646: 641: 633: 628: 620: 606: 601: 600: 599: 593: 584: 570: 562: 557: 546: 545: 544: 534: 526: 515: 510: 505: 504: 503: 497: 478: 476: 475: 470: 468: 453: 451: 450: 445: 437: 432: 427: 426: 425: 419: 403: 401: 400: 395: 393: 343: 341: 340: 335: 333: 332: 327: 314: 312: 311: 306: 304: 284: 282: 281: 276: 274: 239: 237: 236: 231: 229: 217: 215: 214: 209: 191: 189: 188: 183: 181: 166: 164: 163: 158: 156: 148: 143: 72:, applicable to 40: = 2). 25:gradient descent 15762: 15761: 15757: 15756: 15755: 15753: 15752: 15751: 15732: 15731: 15730: 15725: 15684: 15680:Multiprocessing 15648: 15644:Sparse problems 15607: 15586: 15581: 15534: 15531: 15521: 15496: 15467: 15448: 15429: 15409: 15407:Further reading 15404: 15403: 15387: 15386: 15380: 15378: 15368: 15364: 15356: 15350: 15346: 15330: 15323: 15282: 15278: 15231: 15227: 15194: 15190: 15157: 15153: 15146:10.1137/0906018 15126: 15122: 15115: 15101: 15097: 15088: 15086: 15082: 15074: 15070: 15055: 15041: 15037: 15030: 15010: 15006: 14998: 14990: 14986: 14941: 14937: 14930: 14908:Greenbaum, Anne 14905: 14896: 14880: 14869: 14862: 14848:Speiser, Ambros 14845: 14841: 14820: 14816: 14790: 14784:Stiefel, Eduard 14777: 14773: 14768: 14763: 14754:Preconditioning 14744:Krylov subspace 14712: 14684: 14680: 14674: 14670: 14648: 14644: 14636: 14621: 14617: 14611: 14607: 14593: 14590: 14589: 14564: 14561: 14560: 14544: 14541: 14540: 14522: 14521: 14515: 14514: 14508: 14507: 14501: 14500: 14490: 14489: 14472: 14471: 14460: 14453: 14446: 14445: 14438: 14430: 14427: 14418: 14416: 14411: 14406: 14399: 14394: 14391: 14384: 14382: 14371: 14364: 14360: 14356: 14349: 14347: 14336: 14329: 14326: 14321: 14314: 14312: 14302: 14301: 14293: 14290: 14289: 14255: 14252: 14251: 14248: 14240: 14199: 14191: 14186: 14184: 14181: 14180: 14177: 14082:matrix for any 14078:is a symmetric 14044: 14022: 14018: 14016: 14013: 14012: 13995: 13991: 13989: 13986: 13985: 13984:The quantities 13957: 13956: 13935: 13934: 13932: 13929: 13928: 13904: 13900: 13876: 13872: 13840: 13837: 13836: 13825: 13806: 13800: 13794: 13763: 13759: 13757: 13754: 13753: 13750: 13742:locally optimal 13731:round-off error 13727: 13723: 13697: 13692: 13691: 13684: 13683: 13672: 13667: 13664: 13661: 13660: 13630: 13625: 13624: 13617: 13616: 13611: 13606: 13604: 13591: 13586: 13585: 13578: 13577: 13566: 13561: 13559: 13557: 13548: 13544: 13542: 13539: 13538: 13532:Fletcher–Reeves 13530:instead of the 13508: 13503: 13502: 13495: 13494: 13489: 13484: 13482: 13470: 13465: 13464: 13449: 13444: 13443: 13442: 13438: 13431: 13430: 13419: 13414: 13412: 13410: 13401: 13397: 13395: 13392: 13391: 13380: 13359: 13357: 13354: 13353: 13337: 13335: 13332: 13331: 13315: 13313: 13310: 13309: 13286: 13284: 13281: 13280: 13262: 13261: 13256: 13255: 13253: 13250: 13249: 13248:are known, and 13231: 13230: 13225: 13224: 13222: 13219: 13218: 13202: 13200: 13197: 13196: 13178: 13171: 13170: 13165: 13164: 13156: 13154: 13151: 13150: 13133: 13126: 13125: 13121: 13112: 13107: 13106: 13095: 13093: 13090: 13089: 13071: 13069: 13066: 13065: 13045: 13043: 13040: 13039: 13023: 13021: 13018: 13017: 13016:and known, and 13001: 12999: 12996: 12995: 12979: 12977: 12974: 12973: 12955: 12950: 12942: 12940: 12937: 12936: 12919: 12910: 12905: 12904: 12896: 12894: 12891: 12890: 12872: 12870: 12867: 12866: 12848: 12839: 12834: 12833: 12826: 12825: 12821: 12812: 12807: 12806: 12795: 12793: 12790: 12789: 12765: 12760: 12759: 12752: 12751: 12747: 12738: 12733: 12732: 12717: 12712: 12711: 12709: 12706: 12705: 12685: 12676: 12671: 12670: 12662: 12660: 12657: 12656: 12639: 12628: 12626: 12623: 12622: 12600: 12599: 12591: 12590: 12582: 12580: 12577: 12576: 12558: 12556: 12553: 12552: 12535: 12533: 12530: 12529: 12509: 12504: 12503: 12501: 12498: 12497: 12481: 12479: 12476: 12475: 12472: 12432: 12423: 12418: 12417: 12415: 12412: 12411: 12393: 12392: 12388: 12379: 12374: 12373: 12365: 12356: 12351: 12350: 12348: 12345: 12344: 12324: 12319: 12318: 12316: 12313: 12312: 12289: 12282: 12281: 12276: 12275: 12261: 12260: 12251: 12241: 12240: 12232: 12231: 12229: 12226: 12225: 12203: 12194: 12189: 12188: 12174: 12173: 12166: 12165: 12161: 12152: 12147: 12146: 12138: 12129: 12124: 12123: 12121: 12118: 12117: 12109: 12063: 12060: 12059: 12041: 12036: 12035: 12029: 12025: 12010: 12005: 12004: 11989: 11984: 11983: 11981: 11978: 11977: 11956: 11951: 11950: 11943: 11942: 11937: 11932: 11930: 11917: 11912: 11911: 11904: 11903: 11892: 11887: 11885: 11883: 11874: 11870: 11868: 11865: 11864: 11840: 11835: 11834: 11819: 11814: 11813: 11808: 11788: 11786: 11783: 11782: 11771: 11742: 11734: 11733: 11727: 11723: 11714: 11709: 11708: 11693: 11688: 11687: 11685: 11682: 11681: 11663: 11658: 11657: 11651: 11647: 11638: 11633: 11632: 11617: 11612: 11611: 11609: 11606: 11605: 11584: 11576: 11575: 11568: 11567: 11562: 11557: 11555: 11548: 11543: 11542: 11535: 11534: 11529: 11524: 11522: 11520: 11511: 11507: 11505: 11502: 11501: 11472: 11469: 11468: 11450: 11445: 11444: 11435: 11430: 11429: 11427: 11424: 11423: 11405: 11400: 11399: 11390: 11385: 11384: 11379: 11373: 11372: 11370: 11367: 11366: 11348: 11340: 11339: 11331: 11322: 11317: 11316: 11314: 11311: 11310: 11291: 11289: 11286: 11285: 11269: 11260: 11255: 11254: 11252: 11249: 11248: 11228: 11223: 11222: 11220: 11217: 11216: 11213:preconditioning 11211:In most cases, 11209: 11203: 11177: 11174: 11173: 11152: 11144: 11142: 11139: 11138: 11134: 11122:round-off error 11096: 11089: 11084: 11073: 11070: 11069: 11068:which scale as 11029: 11017: 11005: 10993: 10965: 10957: 10956: 10939: 10931: 10930: 10928: 10926: 10923: 10922: 10904: 10901: 10900: 10881: 10873: 10870: 10869: 10844: 10841: 10840: 10821: 10818: 10817: 10792: 10788: 10781: 10780: 10770: 10765: 10764: 10760: 10759: 10758: 10754: 10738: 10730: 10718: 10710: 10707: 10706: 10680: 10672: 10669: 10668: 10645: 10637: 10634: 10633: 10616: 10615: 10604: 10603: 10593: 10588: 10587: 10583: 10582: 10564: 10546: 10544: 10540: 10522: 10521: 10514: 10513: 10503: 10498: 10497: 10493: 10492: 10483: 10460: 10452: 10451: 10434: 10426: 10425: 10423: 10419: 10418: 10406: 10405: 10398: 10397: 10387: 10382: 10381: 10377: 10376: 10368: 10351: 10341: 10328: 10315: 10310: 10299: 10286: 10285: 10278: 10277: 10266: 10261: 10260: 10252: 10245: 10241: 10240: 10232: 10227: 10216: 10205: 10198: 10197: 10187: 10182: 10181: 10177: 10176: 10172: 10170: 10167: 10166: 10146: 10141: 10140: 10131: 10126: 10125: 10116: 10111: 10110: 10108: 10105: 10104: 10087: 10082: 10081: 10079: 10076: 10075: 10058: 10048: 10043: 10042: 10038: 10037: 10035: 10032: 10031: 10012: 10009: 10008: 9987: 9983: 9981: 9978: 9977: 9918: 9914: 9904: 9900: 9891: 9886: 9880: 9877: 9876: 9870: 9846: 9838: 9835: 9834: 9815: 9806: 9801: 9800: 9792: 9789: 9788: 9763: 9752: 9740: 9735: 9734: 9732: 9729: 9728: 9706: 9695: 9693: 9690: 9689: 9686:preconditioning 9660: 9657: 9656: 9628: 9625: 9624: 9608: 9605: 9604: 9579: 9576: 9575: 9554: 9549: 9548: 9546: 9543: 9542: 9531:round-off error 9527: 9512: 9505: 9498: 9469: 9468: 9462: 9461: 9451: 9450: 9440: 9439: 9433: 9432: 9419: 9418: 9405: 9404: 9398: 9397: 9387: 9386: 9377: 9372: 9371: 9365: 9361: 9352: 9347: 9346: 9337: 9332: 9331: 9329: 9326: 9325: 9320: 9313: 9278: 9277: 9271: 9270: 9257: 9256: 9249: 9248: 9243: 9237: 9236: 9231: 9221: 9220: 9213: 9212: 9207: 9194: 9193: 9192: 9184: 9183: 9177: 9176: 9163: 9162: 9155: 9154: 9149: 9136: 9135: 9134: 9132: 9120: 9112: 9111: 9104: 9103: 9098: 9093: 9091: 9084: 9079: 9078: 9071: 9070: 9065: 9060: 9058: 9056: 9047: 9043: 9041: 9038: 9037: 9031: 9025: 9023: 9015: 9009: 8985: 8984: 8978: 8977: 8964: 8963: 8953: 8952: 8943: 8942: 8929: 8928: 8915: 8914: 8908: 8907: 8894: 8893: 8884: 8879: 8878: 8872: 8868: 8859: 8854: 8853: 8844: 8839: 8838: 8836: 8833: 8832: 8827: 8819: 8813: 8783: 8782: 8773: 8772: 8759: 8758: 8751: 8750: 8742: 8729: 8728: 8727: 8719: 8718: 8712: 8711: 8698: 8697: 8690: 8689: 8684: 8671: 8670: 8669: 8667: 8655: 8650: 8649: 8642: 8641: 8636: 8631: 8629: 8622: 8617: 8616: 8609: 8608: 8603: 8598: 8596: 8594: 8585: 8581: 8579: 8576: 8575: 8570: 8562: 8556: 8532: 8531: 8525: 8524: 8511: 8510: 8500: 8499: 8490: 8489: 8476: 8475: 8468: 8467: 8462: 8456: 8455: 8450: 8440: 8439: 8429: 8419: 8418: 8409: 8408: 8395: 8394: 8385: 8380: 8379: 8374: 8368: 8364: 8355: 8350: 8349: 8340: 8335: 8334: 8332: 8329: 8328: 8323: 8316: 8287: 8286: 8280: 8279: 8269: 8268: 8258: 8257: 8248: 8247: 8234: 8233: 8223: 8213: 8212: 8206: 8205: 8195: 8194: 8185: 8180: 8179: 8173: 8169: 8160: 8155: 8154: 8145: 8140: 8139: 8137: 8134: 8133: 8128: 8093: 8080: 8079: 8070: 8069: 8056: 8055: 8048: 8047: 8042: 8036: 8035: 8030: 8020: 8019: 8012: 8011: 8003: 7990: 7989: 7988: 7980: 7979: 7970: 7969: 7956: 7955: 7948: 7947: 7939: 7926: 7925: 7924: 7922: 7910: 7902: 7901: 7894: 7893: 7888: 7883: 7881: 7874: 7869: 7868: 7861: 7860: 7855: 7850: 7848: 7846: 7837: 7833: 7831: 7828: 7827: 7821: 7815: 7810: 7801: 7794: 7766: 7761: 7760: 7750: 7749: 7740: 7739: 7726: 7725: 7715: 7714: 7708: 7707: 7697: 7696: 7689: 7688: 7683: 7677: 7676: 7671: 7661: 7660: 7650: 7649: 7643: 7642: 7632: 7631: 7622: 7617: 7616: 7614: 7611: 7610: 7605: 7594: 7587: 7580: 7554: 7553: 7547: 7544: 7543: 7533: 7532: 7522: 7521: 7511: 7508: 7505: 7504: 7494: 7487: 7486: 7478: 7476: 7473: 7472: 7466: 7440: 7439: 7433: 7432: 7422: 7421: 7412: 7407: 7406: 7404: 7401: 7400: 7373: 7372: 7366: 7365: 7355: 7354: 7344: 7343: 7337: 7333: 7330: 7329: 7323: 7319: 7312: 7311: 7304: 7303: 7298: 7292: 7291: 7286: 7276: 7275: 7267: 7262: 7260: 7257: 7256: 7242: 7237: 7236: 7233: 7230: 7227: 7224: 7221: 7218: 7215: 7212: 7209: 7206: 7203: 7200: 7197: 7194: 7191: 7188: 7185: 7182: 7179: 7176: 7173: 7170: 7167: 7164: 7161: 7158: 7155: 7152: 7149: 7146: 7143: 7140: 7137: 7134: 7131: 7128: 7125: 7122: 7119: 7116: 7113: 7110: 7107: 7104: 7101: 7098: 7095: 7092: 7089: 7086: 7083: 7080: 7077: 7074: 7071: 7068: 7065: 7062: 7059: 7056: 7053: 7050: 7047: 7044: 7041: 7038: 7035: 7032: 7029: 7026: 7023: 7020: 7017: 7014: 7011: 7008: 7005: 7002: 6999: 6996: 6993: 6990: 6987: 6984: 6981: 6978: 6975: 6972: 6969: 6966: 6963: 6960: 6957: 6954: 6951: 6948: 6945: 6942: 6939: 6936: 6933: 6930: 6927: 6924: 6921: 6918: 6915: 6912: 6909: 6906: 6903: 6900: 6897: 6894: 6891: 6888: 6885: 6882: 6879: 6876: 6873: 6870: 6867: 6864: 6861: 6858: 6855: 6852: 6849: 6846: 6843: 6840: 6837: 6826: 6822: 6818: 6816: 6789: 6784: 6783: 6776: 6775: 6770: 6765: 6756: 6752: 6747: 6729: 6724: 6723: 6714: 6709: 6708: 6698: 6697: 6692: 6687: 6678: 6674: 6669: 6660: 6655: 6654: 6649: 6642: 6641: 6637: 6625: 6620: 6619: 6607: 6603: 6594: 6589: 6588: 6576: 6571: 6570: 6565: 6558: 6557: 6552: 6547: 6544: 6541: 6540: 6520: 6515: 6514: 6512: 6509: 6508: 6485: 6480: 6479: 6477: 6474: 6473: 6447: 6442: 6441: 6434: 6433: 6422: 6417: 6408: 6404: 6399: 6378: 6373: 6372: 6363: 6358: 6357: 6347: 6346: 6335: 6330: 6321: 6317: 6312: 6303: 6298: 6297: 6292: 6285: 6284: 6273: 6268: 6265: 6262: 6261: 6255: 6251: 6220: 6215: 6214: 6205: 6200: 6199: 6188: 6184: 6179: 6170: 6165: 6164: 6159: 6157: 6154: 6153: 6131: 6126: 6125: 6120: 6114: 6110: 6101: 6096: 6095: 6080: 6075: 6074: 6072: 6069: 6068: 6042: 6037: 6036: 6031: 6024: 6023: 6018: 6013: 6011: 6004: 5999: 5998: 5993: 5986: 5985: 5974: 5969: 5967: 5965: 5953: 5949: 5947: 5944: 5943: 5937: 5933: 5916: 5911: 5910: 5908: 5905: 5904: 5881: 5876: 5875: 5873: 5870: 5869: 5866: 5862: 5845: 5840: 5839: 5833: 5828: 5827: 5812: 5807: 5806: 5791: 5786: 5785: 5783: 5780: 5779: 5756: 5748: 5747: 5740: 5739: 5734: 5729: 5727: 5720: 5715: 5714: 5707: 5706: 5701: 5696: 5694: 5692: 5680: 5675: 5674: 5669: 5662: 5661: 5656: 5651: 5649: 5642: 5637: 5636: 5629: 5628: 5623: 5618: 5616: 5614: 5605: 5601: 5599: 5596: 5595: 5575: 5570: 5569: 5567: 5564: 5563: 5540: 5535: 5534: 5532: 5529: 5528: 5525: 5521: 5518: 5510:rounding errors 5492: 5484: 5483: 5477: 5473: 5464: 5459: 5458: 5443: 5438: 5437: 5435: 5432: 5431: 5428:rounding errors 5404: 5396: 5395: 5387: 5372: 5367: 5366: 5364: 5361: 5360: 5354:round-off error 5330: 5322: 5321: 5313: 5298: 5293: 5292: 5290: 5287: 5286: 5269: 5265: 5263: 5260: 5259: 5242: 5234: 5233: 5231: 5228: 5227: 5211: 5209: 5206: 5205: 5182: 5174: 5173: 5165: 5150: 5145: 5144: 5142: 5139: 5138: 5121: 5113: 5112: 5106: 5102: 5093: 5088: 5087: 5072: 5067: 5066: 5064: 5061: 5060: 5043: 5035: 5034: 5026: 5017: 5012: 5011: 5009: 5006: 5005: 4988: 4983: 4982: 4976: 4972: 4963: 4958: 4957: 4942: 4937: 4936: 4934: 4931: 4930: 4927: 4919:round-off error 4901: 4896: 4895: 4893: 4890: 4889: 4862: 4857: 4856: 4854: 4851: 4850: 4827: 4823: 4821: 4818: 4817: 4800: 4795: 4794: 4792: 4789: 4788: 4767: 4762: 4761: 4759: 4756: 4755: 4752: 4740: 4736: 4719: 4718: 4713: 4701: 4696: 4695: 4690: 4686: 4685: 4680: 4676: 4675: 4655: 4654: 4648: 4643: 4642: 4636: 4632: 4617: 4612: 4611: 4596: 4591: 4590: 4585: 4584: 4575: 4570: 4569: 4562: 4561: 4556: 4551: 4549: 4536: 4531: 4530: 4523: 4522: 4511: 4506: 4504: 4502: 4493: 4489: 4484: 4483: 4478: 4466: 4461: 4460: 4453: 4448: 4447: 4441: 4433: 4432: 4426: 4422: 4413: 4408: 4407: 4392: 4387: 4386: 4381: 4380: 4374: 4369: 4368: 4362: 4358: 4349: 4344: 4343: 4328: 4323: 4322: 4317: 4316: 4307: 4299: 4298: 4291: 4290: 4285: 4280: 4278: 4271: 4266: 4265: 4258: 4257: 4252: 4247: 4245: 4243: 4234: 4230: 4225: 4224: 4219: 4215: 4214: 4201: 4200: 4194: 4189: 4188: 4179: 4174: 4173: 4169: 4168: 4163: 4157: 4152: 4151: 4146: 4140: 4135: 4134: 4127: 4123: 4122: 4116: 4108: 4107: 4099: 4090: 4085: 4084: 4079: 4077: 4074: 4073: 4049: 4044: 4043: 4041: 4038: 4037: 4021: 4019: 4016: 4015: 3999: 3991: 3986: 3984: 3981: 3980: 3943: 3939: 3917: 3913: 3874: 3870: 3857: 3853: 3852: 3844: 3839: 3838: 3831: 3811: 3810: 3801: 3797: 3795: 3792: 3791: 3768: 3763: 3762: 3760: 3757: 3756: 3737: 3735: 3732: 3731: 3714: 3709: 3708: 3706: 3703: 3702: 3686: 3684: 3681: 3680: 3663: 3658: 3657: 3655: 3652: 3651: 3634: 3629: 3628: 3626: 3623: 3622: 3619:Krylov subspace 3601: 3596: 3595: 3593: 3590: 3589: 3572: 3567: 3566: 3564: 3561: 3560: 3538: 3535: 3534: 3511: 3506: 3505: 3500: 3493: 3492: 3487: 3482: 3479: 3476: 3475: 3458: 3453: 3452: 3450: 3447: 3446: 3445:-orthogonal to 3430: 3428: 3425: 3424: 3407: 3402: 3401: 3399: 3396: 3395: 3372: 3367: 3366: 3359: 3358: 3353: 3348: 3345: 3342: 3341: 3324: 3319: 3318: 3316: 3313: 3312: 3295: 3290: 3289: 3287: 3284: 3283: 3279: 3260: 3259: 3246: 3241: 3240: 3235: 3228: 3227: 3222: 3217: 3215: 3205: 3197: 3196: 3188: 3178: 3177: 3172: 3167: 3165: 3163: 3154: 3150: 3132: 3128: 3119: 3115: 3104: 3101: 3100: 3091: 3087: 3069: 3064: 3063: 3057: 3053: 3044: 3039: 3038: 3025: 3010: 3005: 3004: 2994: 2992: 2989: 2988: 2969: 2965: 2963: 2960: 2959: 2953: 2928: 2924: 2922: 2919: 2918: 2901: 2896: 2895: 2893: 2890: 2889: 2863: 2858: 2857: 2852: 2845: 2844: 2839: 2834: 2832: 2825: 2820: 2819: 2812: 2811: 2806: 2801: 2799: 2797: 2785: 2780: 2779: 2774: 2767: 2766: 2761: 2756: 2754: 2744: 2736: 2735: 2727: 2717: 2716: 2711: 2706: 2704: 2702: 2693: 2689: 2687: 2684: 2683: 2660: 2655: 2654: 2648: 2644: 2635: 2630: 2629: 2614: 2609: 2608: 2606: 2603: 2602: 2579: 2574: 2573: 2564: 2559: 2558: 2553: 2546: 2545: 2540: 2535: 2533: 2526: 2521: 2520: 2515: 2508: 2507: 2502: 2497: 2495: 2493: 2481: 2468: 2463: 2462: 2453: 2448: 2447: 2445: 2442: 2441: 2416: 2411: 2410: 2408: 2405: 2404: 2402: 2374: 2369: 2368: 2366: 2363: 2362: 2346: 2343: 2342: 2325: 2320: 2319: 2317: 2314: 2313: 2290: 2282: 2281: 2273: 2264: 2259: 2258: 2256: 2253: 2252: 2239: 2223: 2212: 2201: 2194: 2179: 2173: 2158: 2130: 2122: 2117: 2106: 2095: 2092: 2091: 2057: 2043: 2029: 2027: 2024: 2023: 1992: 1987: 1986: 1978: 1969: 1962: 1961: 1956: 1955: 1947: 1942: 1935: 1934: 1929: 1928: 1916: 1905: 1897: 1894: 1893: 1884: 1878: 1875: 1869: 1867: 1860: 1840: 1834: 1831: 1825: 1822: 1816: 1792: 1787: 1786: 1784: 1781: 1780: 1763: 1758: 1757: 1755: 1752: 1751: 1748: 1726: 1722: 1720: 1717: 1716: 1700: 1697: 1696: 1686: 1659: 1658: 1654: 1648: 1643: 1642: 1633: 1628: 1627: 1623: 1614: 1605: 1600: 1599: 1595: 1593: 1584: 1580: 1578: 1575: 1574: 1550: 1549: 1538: 1533: 1532: 1523: 1518: 1517: 1516: 1512: 1511: 1505: 1501: 1491: 1490: 1479: 1474: 1473: 1464: 1459: 1458: 1457: 1453: 1452: 1446: 1442: 1436: 1425: 1412: 1407: 1406: 1401: 1394: 1393: 1388: 1383: 1376: 1372: 1366: 1355: 1342: 1337: 1336: 1331: 1324: 1323: 1318: 1313: 1304: 1297: 1296: 1291: 1286: 1283: 1280: 1279: 1258: 1257: 1252: 1247: 1244: 1241: 1240: 1224: 1213: 1211: 1208: 1207: 1184: 1179: 1178: 1173: 1167: 1163: 1157: 1146: 1133: 1128: 1127: 1122: 1113: 1108: 1107: 1101: 1097: 1091: 1080: 1067: 1062: 1061: 1059: 1056: 1055: 1051:in this basis: 1036: 1025: 1023: 1020: 1019: 1002: 997: 996: 994: 991: 990: 973: 968: 967: 965: 962: 961: 941: 938: 937: 915: 912: 911: 888: 883: 882: 877: 870: 869: 864: 859: 856: 853: 852: 836: 834: 831: 830: 814: 811: 810: 787: 782: 781: 766: 761: 760: 749: 746: 745: 741:. Suppose that 726: 724: 721: 720: 704: 702: 699: 698: 682: 680: 677: 676: 660: 658: 655: 654: 629: 624: 616: 602: 595: 594: 589: 588: 580: 566: 558: 553: 540: 539: 535: 530: 522: 511: 506: 499: 498: 493: 492: 490: 487: 486: 464: 462: 459: 458: 433: 428: 421: 420: 415: 414: 412: 409: 408: 389: 387: 384: 383: 356: 350: 328: 323: 322: 320: 317: 316: 300: 298: 295: 294: 270: 268: 265: 264: 225: 223: 220: 219: 197: 194: 193: 177: 175: 172: 171: 170:for the vector 152: 144: 139: 137: 134: 133: 124: 97:Magnus Hestenes 17: 12: 11: 5: 15760: 15750: 15749: 15744: 15727: 15726: 15724: 15723: 15718: 15713: 15708: 15703: 15698: 15692: 15690: 15686: 15685: 15683: 15682: 15677: 15672: 15667: 15662: 15656: 15654: 15650: 15649: 15647: 15646: 15641: 15636: 15626: 15621: 15615: 15613: 15609: 15608: 15606: 15605: 15600: 15598:Floating point 15594: 15592: 15588: 15587: 15580: 15579: 15572: 15565: 15557: 15551: 15550: 15530: 15529:External links 15527: 15526: 15525: 15519: 15506: 15500: 15494: 15471: 15465: 15452: 15446: 15433: 15427: 15408: 15405: 15402: 15401: 15362: 15344: 15321: 15276: 15225: 15211:10.1.1.35.7473 15188: 15174:10.1.1.56.1755 15151: 15140:(1): 220–252. 15120: 15113: 15095: 15068: 15053: 15035: 15028: 15004: 14984: 14935: 14928: 14894: 14867: 14860: 14839: 14814: 14770: 14769: 14767: 14764: 14762: 14761: 14756: 14751: 14746: 14741: 14736: 14731: 14726: 14720: 14713: 14711: 14708: 14695: 14692: 14687: 14683: 14677: 14673: 14669: 14666: 14663: 14660: 14656: 14651: 14647: 14643: 14639: 14635: 14632: 14629: 14624: 14620: 14614: 14610: 14606: 14603: 14600: 14597: 14577: 14574: 14571: 14568: 14548: 14526: 14520: 14517: 14516: 14513: 14510: 14509: 14506: 14503: 14502: 14499: 14496: 14495: 14493: 14488: 14485: 14481: 14476: 14470: 14467: 14464: 14461: 14457: 14452: 14447: 14442: 14437: 14431: 14426: 14423: 14419: 14415: 14412: 14410: 14407: 14403: 14398: 14395: 14388: 14383: 14381: 14378: 14375: 14372: 14368: 14363: 14361: 14353: 14348: 14346: 14343: 14340: 14337: 14333: 14328: 14327: 14318: 14313: 14311: 14308: 14307: 14305: 14300: 14297: 14277: 14274: 14271: 14268: 14265: 14262: 14259: 14247: 14244: 14239: 14236: 14202: 14198: 14194: 14189: 14176: 14173: 14150:preconditioner 14116: 14115: 14043: 14040: 14025: 14021: 13998: 13994: 13973: 13970: 13964: 13961: 13954: 13951: 13948: 13942: 13939: 13912: 13907: 13903: 13899: 13896: 13893: 13890: 13887: 13884: 13879: 13875: 13871: 13868: 13865: 13862: 13859: 13856: 13853: 13850: 13847: 13844: 13824: 13821: 13817:preconditioner 13774: 13771: 13766: 13762: 13749: 13746: 13725: 13711: 13708: 13705: 13700: 13695: 13687: 13681: 13678: 13675: 13670: 13650: 13649: 13633: 13628: 13620: 13614: 13609: 13600: 13597: 13594: 13589: 13581: 13575: 13572: 13569: 13564: 13556: 13551: 13547: 13528: 13527: 13511: 13506: 13498: 13492: 13487: 13479: 13473: 13468: 13463: 13458: 13455: 13452: 13447: 13441: 13434: 13428: 13425: 13422: 13417: 13409: 13404: 13400: 13379: 13376: 13362: 13340: 13318: 13289: 13265: 13259: 13234: 13228: 13205: 13181: 13174: 13168: 13163: 13159: 13136: 13129: 13124: 13118: 13115: 13110: 13105: 13102: 13098: 13074: 13048: 13026: 13004: 12982: 12958: 12953: 12949: 12945: 12922: 12916: 12913: 12908: 12903: 12899: 12875: 12851: 12845: 12842: 12837: 12829: 12824: 12818: 12815: 12810: 12805: 12802: 12798: 12771: 12768: 12763: 12755: 12750: 12744: 12741: 12736: 12731: 12728: 12723: 12720: 12715: 12688: 12682: 12679: 12674: 12669: 12665: 12642: 12638: 12634: 12631: 12603: 12597: 12594: 12589: 12585: 12561: 12538: 12515: 12512: 12507: 12484: 12471: 12468: 12460:preconditioner 12435: 12429: 12426: 12421: 12396: 12391: 12385: 12382: 12377: 12372: 12368: 12362: 12359: 12354: 12330: 12327: 12322: 12308: 12307: 12296: 12292: 12285: 12279: 12274: 12268: 12265: 12258: 12254: 12250: 12244: 12238: 12235: 12219: 12218: 12206: 12200: 12197: 12192: 12187: 12181: 12178: 12169: 12164: 12158: 12155: 12150: 12145: 12141: 12135: 12132: 12127: 12111: 12110: 12104: 12100:The result is 12098: 12093: 12092: 12091: 12079: 12076: 12073: 12070: 12067: 12057: 12044: 12039: 12032: 12028: 12024: 12019: 12016: 12013: 12008: 12003: 11998: 11995: 11992: 11987: 11975: 11959: 11954: 11946: 11940: 11935: 11926: 11923: 11920: 11915: 11907: 11901: 11898: 11895: 11890: 11882: 11877: 11873: 11862: 11849: 11846: 11843: 11838: 11833: 11828: 11825: 11822: 11817: 11811: 11803: 11800: 11797: 11794: 11791: 11780: 11766: 11758: 11745: 11740: 11737: 11730: 11726: 11722: 11717: 11712: 11707: 11702: 11699: 11696: 11691: 11679: 11666: 11661: 11654: 11650: 11646: 11641: 11636: 11631: 11626: 11623: 11620: 11615: 11603: 11587: 11582: 11579: 11571: 11565: 11560: 11551: 11546: 11538: 11532: 11527: 11519: 11514: 11510: 11494: 11482: 11479: 11476: 11466: 11453: 11448: 11443: 11438: 11433: 11421: 11408: 11403: 11398: 11393: 11388: 11382: 11364: 11351: 11346: 11343: 11338: 11334: 11330: 11325: 11320: 11294: 11272: 11266: 11263: 11258: 11234: 11231: 11226: 11207:Preconditioner 11202: 11199: 11181: 11159: 11155: 11151: 11148: 11133: 11130: 11126:Anne Greenbaum 11103: 11099: 11095: 11092: 11088: 11083: 11080: 11077: 11058: 11057: 11046: 11042: 11039: 11036: 11032: 11028: 11025: 11012: 11008: 11004: 11001: 10997: 10992: 10989: 10986: 10980: 10977: 10972: 10968: 10964: 10961: 10954: 10951: 10946: 10942: 10938: 10935: 10908: 10888: 10884: 10880: 10877: 10854: 10851: 10848: 10828: 10825: 10804: 10798: 10795: 10791: 10784: 10778: 10773: 10768: 10763: 10757: 10753: 10750: 10745: 10741: 10737: 10734: 10726: 10723: 10717: 10714: 10687: 10683: 10679: 10676: 10652: 10648: 10644: 10641: 10630: 10629: 10614: 10607: 10601: 10596: 10591: 10586: 10577: 10571: 10567: 10563: 10560: 10555: 10552: 10549: 10543: 10539: 10536: 10533: 10530: 10527: 10525: 10523: 10517: 10511: 10506: 10501: 10496: 10486: 10481: 10475: 10472: 10467: 10463: 10459: 10456: 10449: 10446: 10441: 10437: 10433: 10430: 10422: 10417: 10414: 10411: 10409: 10407: 10401: 10395: 10390: 10385: 10380: 10371: 10367: 10364: 10361: 10358: 10354: 10348: 10344: 10340: 10337: 10334: 10331: 10327: 10318: 10313: 10309: 10305: 10302: 10298: 10294: 10291: 10289: 10287: 10281: 10275: 10269: 10264: 10259: 10255: 10251: 10248: 10244: 10235: 10230: 10226: 10222: 10219: 10215: 10211: 10208: 10206: 10201: 10195: 10190: 10185: 10180: 10175: 10174: 10149: 10144: 10139: 10134: 10129: 10124: 10119: 10114: 10090: 10085: 10061: 10056: 10051: 10046: 10041: 10016: 10003:is the set of 9990: 9986: 9974: 9973: 9962: 9957: 9950: 9947: 9944: 9941: 9938: 9935: 9929: 9921: 9917: 9913: 9910: 9903: 9899: 9894: 9889: 9885: 9869: 9866: 9853: 9849: 9845: 9842: 9822: 9818: 9812: 9809: 9804: 9799: 9796: 9776: 9773: 9770: 9766: 9762: 9758: 9755: 9751: 9746: 9743: 9738: 9716: 9713: 9709: 9705: 9701: 9698: 9673: 9670: 9667: 9664: 9641: 9638: 9635: 9632: 9612: 9592: 9589: 9586: 9583: 9557: 9552: 9526: 9523: 9510: 9503: 9496: 9490: 9489: 9478: 9473: 9467: 9464: 9463: 9460: 9457: 9456: 9454: 9449: 9444: 9438: 9435: 9434: 9431: 9428: 9425: 9424: 9422: 9417: 9414: 9409: 9403: 9400: 9399: 9396: 9393: 9392: 9390: 9385: 9380: 9375: 9368: 9364: 9360: 9355: 9350: 9345: 9340: 9335: 9318: 9311: 9305: 9304: 9293: 9290: 9282: 9276: 9273: 9272: 9269: 9266: 9263: 9262: 9260: 9253: 9247: 9244: 9242: 9239: 9238: 9235: 9232: 9230: 9227: 9226: 9224: 9217: 9211: 9208: 9206: 9203: 9200: 9199: 9197: 9188: 9182: 9179: 9178: 9175: 9172: 9169: 9168: 9166: 9159: 9153: 9150: 9148: 9145: 9142: 9141: 9139: 9131: 9123: 9118: 9115: 9107: 9101: 9096: 9087: 9082: 9074: 9068: 9063: 9055: 9050: 9046: 9029: 9021: 9013: 9006: 9005: 8994: 8989: 8983: 8980: 8979: 8976: 8973: 8970: 8969: 8967: 8962: 8957: 8951: 8948: 8945: 8944: 8941: 8938: 8935: 8934: 8932: 8927: 8924: 8919: 8913: 8910: 8909: 8906: 8903: 8900: 8899: 8897: 8892: 8887: 8882: 8875: 8871: 8867: 8862: 8857: 8852: 8847: 8842: 8825: 8817: 8810: 8809: 8798: 8795: 8787: 8781: 8778: 8775: 8774: 8771: 8768: 8765: 8764: 8762: 8755: 8749: 8746: 8743: 8741: 8738: 8735: 8734: 8732: 8723: 8717: 8714: 8713: 8710: 8707: 8704: 8703: 8701: 8694: 8688: 8685: 8683: 8680: 8677: 8676: 8674: 8666: 8658: 8653: 8645: 8639: 8634: 8625: 8620: 8612: 8606: 8601: 8593: 8588: 8584: 8568: 8560: 8553: 8552: 8541: 8536: 8530: 8527: 8526: 8523: 8520: 8517: 8516: 8514: 8509: 8504: 8498: 8495: 8492: 8491: 8488: 8485: 8482: 8481: 8479: 8472: 8466: 8463: 8461: 8458: 8457: 8454: 8451: 8449: 8446: 8445: 8443: 8436: 8433: 8428: 8423: 8417: 8414: 8411: 8410: 8407: 8404: 8401: 8400: 8398: 8393: 8388: 8383: 8377: 8371: 8367: 8363: 8358: 8353: 8348: 8343: 8338: 8321: 8314: 8308: 8307: 8296: 8291: 8285: 8282: 8281: 8278: 8275: 8274: 8272: 8267: 8262: 8256: 8253: 8250: 8249: 8246: 8243: 8240: 8239: 8237: 8230: 8227: 8222: 8217: 8211: 8208: 8207: 8204: 8201: 8200: 8198: 8193: 8188: 8183: 8176: 8172: 8168: 8163: 8158: 8153: 8148: 8143: 8126: 8120: 8119: 8108: 8105: 8100: 8097: 8092: 8084: 8078: 8075: 8072: 8071: 8068: 8065: 8062: 8061: 8059: 8052: 8046: 8043: 8041: 8038: 8037: 8034: 8031: 8029: 8026: 8025: 8023: 8016: 8010: 8007: 8004: 8002: 7999: 7996: 7995: 7993: 7984: 7978: 7975: 7972: 7971: 7968: 7965: 7962: 7961: 7959: 7952: 7946: 7943: 7940: 7938: 7935: 7932: 7931: 7929: 7921: 7913: 7908: 7905: 7897: 7891: 7886: 7877: 7872: 7864: 7858: 7853: 7845: 7840: 7836: 7819: 7806: 7799: 7792: 7786: 7785: 7774: 7769: 7764: 7759: 7754: 7748: 7745: 7742: 7741: 7738: 7735: 7732: 7731: 7729: 7724: 7719: 7713: 7710: 7709: 7706: 7703: 7702: 7700: 7693: 7687: 7684: 7682: 7679: 7678: 7675: 7672: 7670: 7667: 7666: 7664: 7659: 7654: 7648: 7645: 7644: 7641: 7638: 7637: 7635: 7630: 7625: 7620: 7603: 7592: 7585: 7578: 7572: 7571: 7558: 7552: 7549: 7548: 7545: 7542: 7539: 7538: 7536: 7531: 7526: 7518: 7515: 7510: 7509: 7506: 7501: 7498: 7493: 7492: 7490: 7485: 7481: 7465: 7462: 7458: 7457: 7444: 7438: 7435: 7434: 7431: 7428: 7427: 7425: 7420: 7415: 7410: 7394: 7393: 7382: 7377: 7371: 7368: 7367: 7364: 7361: 7360: 7358: 7353: 7348: 7340: 7336: 7332: 7331: 7326: 7322: 7318: 7317: 7315: 7308: 7302: 7299: 7297: 7294: 7293: 7290: 7287: 7285: 7282: 7281: 7279: 7274: 7270: 7265: 7241: 7238: 7222:new_resid_norm 7216:old_resid_norm 7195:old_resid_norm 7189:new_resid_norm 7150:new_resid_norm 7066:old_resid_norm 7036:old_resid_norm 7012:old_resid_norm 6892:AbstractVector 6880:AbstractVector 6868:AbstractMatrix 6839: 6836: 6830: 6824: 6812: 6806: 6805: 6792: 6787: 6779: 6773: 6768: 6759: 6755: 6751: 6746: 6743: 6738: 6735: 6732: 6727: 6722: 6717: 6712: 6707: 6701: 6695: 6690: 6681: 6677: 6673: 6668: 6663: 6658: 6652: 6645: 6640: 6634: 6631: 6628: 6623: 6616: 6613: 6610: 6606: 6602: 6597: 6592: 6587: 6584: 6579: 6574: 6568: 6561: 6555: 6550: 6523: 6518: 6494: 6491: 6488: 6483: 6470: 6469: 6456: 6453: 6450: 6445: 6437: 6431: 6428: 6425: 6420: 6411: 6407: 6403: 6398: 6395: 6392: 6387: 6384: 6381: 6376: 6371: 6366: 6361: 6356: 6350: 6344: 6341: 6338: 6333: 6324: 6320: 6316: 6311: 6306: 6301: 6295: 6288: 6282: 6279: 6276: 6271: 6253: 6237: 6234: 6229: 6226: 6223: 6218: 6213: 6208: 6203: 6198: 6191: 6187: 6183: 6178: 6173: 6168: 6162: 6148: 6147: 6134: 6129: 6123: 6117: 6113: 6109: 6104: 6099: 6094: 6089: 6086: 6083: 6078: 6062: 6061: 6045: 6040: 6034: 6027: 6021: 6016: 6007: 6002: 5996: 5989: 5983: 5980: 5977: 5972: 5964: 5961: 5956: 5952: 5935: 5919: 5914: 5890: 5887: 5884: 5879: 5864: 5848: 5843: 5836: 5831: 5826: 5821: 5818: 5815: 5810: 5805: 5800: 5797: 5794: 5789: 5776: 5775: 5759: 5754: 5751: 5743: 5737: 5732: 5723: 5718: 5710: 5704: 5699: 5691: 5683: 5678: 5672: 5665: 5659: 5654: 5645: 5640: 5632: 5626: 5621: 5613: 5608: 5604: 5578: 5573: 5549: 5546: 5543: 5538: 5523: 5517: 5514: 5495: 5490: 5487: 5480: 5476: 5472: 5467: 5462: 5457: 5452: 5449: 5446: 5441: 5413: 5410: 5407: 5402: 5399: 5394: 5390: 5386: 5381: 5378: 5375: 5370: 5339: 5336: 5333: 5328: 5325: 5320: 5316: 5312: 5307: 5304: 5301: 5296: 5272: 5268: 5245: 5240: 5237: 5214: 5191: 5188: 5185: 5180: 5177: 5172: 5168: 5164: 5159: 5156: 5153: 5148: 5124: 5119: 5116: 5109: 5105: 5101: 5096: 5091: 5086: 5081: 5078: 5075: 5070: 5046: 5041: 5038: 5033: 5029: 5025: 5020: 5015: 4991: 4986: 4979: 4975: 4971: 4966: 4961: 4956: 4951: 4948: 4945: 4940: 4926: 4923: 4904: 4899: 4871: 4868: 4865: 4860: 4838: 4835: 4830: 4826: 4803: 4798: 4770: 4765: 4751: 4748: 4738: 4733: 4732: 4710: 4707: 4704: 4699: 4689: 4687: 4679: 4677: 4674: 4671: 4668: 4665: 4662: 4658: 4656: 4651: 4646: 4639: 4635: 4631: 4626: 4623: 4620: 4615: 4610: 4605: 4602: 4599: 4594: 4588: 4586: 4578: 4573: 4565: 4559: 4554: 4545: 4542: 4539: 4534: 4526: 4520: 4517: 4514: 4509: 4501: 4496: 4492: 4487: 4485: 4475: 4472: 4469: 4464: 4451: 4449: 4444: 4439: 4436: 4429: 4425: 4421: 4416: 4411: 4406: 4401: 4398: 4395: 4390: 4384: 4382: 4377: 4372: 4365: 4361: 4357: 4352: 4347: 4342: 4337: 4334: 4331: 4326: 4320: 4318: 4310: 4305: 4302: 4294: 4288: 4283: 4274: 4269: 4261: 4255: 4250: 4242: 4237: 4233: 4228: 4226: 4218: 4216: 4213: 4210: 4207: 4204: 4202: 4197: 4192: 4187: 4182: 4177: 4172: 4170: 4160: 4155: 4143: 4138: 4126: 4124: 4119: 4114: 4111: 4106: 4102: 4098: 4093: 4088: 4083: 4081: 4052: 4047: 4024: 4002: 3998: 3994: 3989: 3966: 3961: 3957: 3952: 3949: 3946: 3942: 3938: 3935: 3932: 3929: 3926: 3923: 3920: 3916: 3912: 3909: 3906: 3903: 3900: 3897: 3894: 3891: 3888: 3885: 3882: 3877: 3873: 3869: 3866: 3863: 3860: 3856: 3847: 3842: 3837: 3834: 3829: 3826: 3823: 3820: 3817: 3814: 3809: 3804: 3800: 3779: 3776: 3771: 3766: 3740: 3717: 3712: 3689: 3666: 3661: 3637: 3632: 3617:span the same 3604: 3599: 3575: 3570: 3548: 3545: 3542: 3522: 3519: 3514: 3509: 3503: 3496: 3490: 3485: 3461: 3456: 3433: 3410: 3405: 3383: 3380: 3375: 3370: 3362: 3356: 3351: 3327: 3322: 3298: 3293: 3278: 3275: 3274: 3273: 3258: 3249: 3244: 3238: 3231: 3225: 3220: 3213: 3208: 3203: 3200: 3195: 3191: 3187: 3181: 3175: 3170: 3162: 3157: 3153: 3148: 3144: 3139: 3136: 3131: 3129: 3127: 3122: 3118: 3114: 3110: 3107: 3103: 3102: 3099: 3094: 3090: 3086: 3083: 3080: 3077: 3072: 3067: 3060: 3056: 3052: 3047: 3042: 3037: 3034: 3031: 3028: 3026: 3024: 3019: 3016: 3013: 3008: 3003: 3000: 2997: 2996: 2972: 2968: 2948: 2931: 2927: 2904: 2899: 2886: 2885: 2874: 2866: 2861: 2855: 2848: 2842: 2837: 2828: 2823: 2815: 2809: 2804: 2796: 2788: 2783: 2777: 2770: 2764: 2759: 2752: 2747: 2742: 2739: 2734: 2730: 2726: 2720: 2714: 2709: 2701: 2696: 2692: 2677: 2676: 2663: 2658: 2651: 2647: 2643: 2638: 2633: 2628: 2623: 2620: 2617: 2612: 2596: 2595: 2582: 2577: 2567: 2562: 2556: 2549: 2543: 2538: 2529: 2524: 2518: 2511: 2505: 2500: 2490: 2487: 2484: 2480: 2476: 2471: 2466: 2461: 2456: 2451: 2419: 2414: 2398: 2377: 2372: 2350: 2328: 2323: 2310: 2309: 2298: 2293: 2288: 2285: 2280: 2276: 2272: 2267: 2262: 2235: 2221: 2210: 2199: 2192: 2171: 2156: 2150: 2149: 2138: 2133: 2129: 2125: 2120: 2116: 2113: 2109: 2105: 2102: 2099: 2077: 2076: 2065: 2060: 2056: 2053: 2050: 2046: 2042: 2039: 2036: 2032: 2017:Hessian matrix 2013: 2012: 2001: 1995: 1990: 1985: 1981: 1976: 1972: 1965: 1959: 1954: 1950: 1945: 1938: 1932: 1924: 1921: 1915: 1912: 1908: 1904: 1901: 1882: 1873: 1865: 1858: 1838: 1829: 1820: 1795: 1790: 1766: 1761: 1747: 1744: 1729: 1725: 1704: 1683: 1682: 1671: 1662: 1657: 1651: 1646: 1641: 1636: 1631: 1626: 1621: 1617: 1613: 1608: 1603: 1598: 1592: 1587: 1583: 1568: 1567: 1553: 1547: 1541: 1536: 1531: 1526: 1521: 1515: 1508: 1504: 1500: 1494: 1488: 1482: 1477: 1472: 1467: 1462: 1456: 1449: 1445: 1439: 1434: 1431: 1428: 1424: 1420: 1415: 1410: 1404: 1397: 1391: 1386: 1379: 1375: 1369: 1364: 1361: 1358: 1354: 1350: 1345: 1340: 1334: 1327: 1321: 1316: 1311: 1307: 1300: 1294: 1289: 1261: 1255: 1250: 1227: 1223: 1219: 1216: 1204: 1203: 1192: 1187: 1182: 1176: 1170: 1166: 1160: 1155: 1152: 1149: 1145: 1141: 1136: 1131: 1125: 1121: 1116: 1111: 1104: 1100: 1094: 1089: 1086: 1083: 1079: 1075: 1070: 1065: 1039: 1035: 1031: 1028: 1005: 1000: 976: 971: 945: 925: 922: 919: 899: 896: 891: 886: 880: 873: 867: 862: 839: 818: 807: 806: 795: 790: 785: 780: 777: 774: 769: 764: 759: 756: 753: 729: 707: 685: 663: 651: 650: 639: 636: 632: 627: 623: 619: 615: 612: 609: 605: 598: 592: 587: 583: 579: 576: 573: 569: 565: 561: 556: 552: 549: 543: 538: 533: 529: 525: 521: 518: 514: 509: 502: 496: 467: 455: 454: 443: 440: 436: 431: 424: 418: 392: 366:iteration for 352:Main article: 349: 346: 331: 326: 303: 273: 228: 207: 204: 201: 180: 168: 167: 155: 151: 147: 142: 123: 120: 101:Eduard Stiefel 60:of particular 15: 9: 6: 4: 3: 2: 15759: 15748: 15745: 15743: 15740: 15739: 15737: 15722: 15719: 15717: 15714: 15712: 15709: 15707: 15704: 15702: 15699: 15697: 15694: 15693: 15691: 15687: 15681: 15678: 15676: 15673: 15671: 15668: 15666: 15663: 15661: 15658: 15657: 15655: 15651: 15645: 15642: 15640: 15637: 15634: 15630: 15627: 15625: 15622: 15620: 15617: 15616: 15614: 15610: 15604: 15601: 15599: 15596: 15595: 15593: 15589: 15585: 15578: 15573: 15571: 15566: 15564: 15559: 15558: 15555: 15547: 15543: 15542: 15537: 15533: 15532: 15522: 15516: 15512: 15507: 15505: 15501: 15497: 15491: 15486: 15485: 15478: 15472: 15468: 15462: 15458: 15453: 15449: 15443: 15439: 15434: 15430: 15424: 15419: 15418: 15411: 15410: 15397: 15391: 15377: 15373: 15366: 15355: 15348: 15341: 15337: 15333: 15328: 15326: 15317: 15313: 15309: 15305: 15300: 15295: 15291: 15287: 15280: 15272: 15268: 15263: 15258: 15253: 15248: 15244: 15240: 15236: 15229: 15221: 15217: 15212: 15207: 15203: 15199: 15192: 15184: 15180: 15175: 15170: 15166: 15162: 15155: 15147: 15143: 15139: 15135: 15131: 15124: 15116: 15110: 15106: 15099: 15081: 15080: 15072: 15064: 15060: 15056: 15050: 15046: 15039: 15031: 15025: 15021: 15017: 15016: 15008: 14997: 14996: 14988: 14980: 14976: 14972: 14968: 14963: 14958: 14954: 14950: 14946: 14939: 14931: 14925: 14921: 14917: 14913: 14909: 14903: 14901: 14899: 14890: 14889: 14884: 14883:Polyak, Boris 14878: 14876: 14874: 14872: 14863: 14861:3-540-00217-0 14857: 14853: 14849: 14843: 14834: 14829: 14825: 14818: 14809: 14804: 14800: 14796: 14789: 14785: 14781: 14775: 14771: 14760: 14757: 14755: 14752: 14750: 14747: 14745: 14742: 14740: 14737: 14735: 14732: 14730: 14727: 14724: 14721: 14718: 14715: 14714: 14707: 14693: 14690: 14685: 14675: 14671: 14667: 14664: 14661: 14654: 14649: 14641: 14637: 14633: 14627: 14622: 14612: 14608: 14604: 14601: 14598: 14575: 14572: 14569: 14566: 14546: 14524: 14518: 14511: 14504: 14497: 14491: 14486: 14483: 14479: 14474: 14468: 14465: 14462: 14455: 14440: 14424: 14413: 14408: 14401: 14386: 14379: 14376: 14373: 14366: 14351: 14344: 14341: 14338: 14331: 14316: 14309: 14303: 14298: 14295: 14272: 14269: 14266: 14260: 14257: 14243: 14235: 14233: 14230:for the real 14229: 14225: 14221: 14217: 14196: 14172: 14170: 14166: 14162: 14158: 14153: 14151: 14147: 14143: 14140: 14136: 14132: 14131:sparse matrix 14128: 14124: 14121: 14114: 14111: 14107: 14104: 14101: 14100: 14099: 14097: 14093: 14089: 14085: 14081: 14077: 14074: 14070: 14067: 14063: 14060: 14057: 14053: 14049: 14039: 14023: 14019: 13996: 13992: 13971: 13968: 13962: 13959: 13952: 13949: 13946: 13940: 13937: 13926: 13910: 13905: 13901: 13897: 13891: 13885: 13877: 13873: 13869: 13866: 13860: 13857: 13854: 13848: 13845: 13842: 13834: 13830: 13820: 13818: 13814: 13809: 13803: 13797: 13792: 13788: 13772: 13769: 13764: 13760: 13745: 13743: 13739: 13738:Polak–Ribière 13734: 13732: 13709: 13706: 13703: 13698: 13679: 13676: 13673: 13657: 13655: 13631: 13612: 13598: 13595: 13592: 13573: 13570: 13567: 13554: 13549: 13545: 13537: 13536: 13535: 13533: 13509: 13490: 13477: 13471: 13461: 13456: 13453: 13450: 13439: 13426: 13423: 13420: 13407: 13402: 13398: 13390: 13389: 13388: 13386: 13385:Polak–Ribière 13375: 13306: 13304: 13193: 13161: 13148: 13116: 13113: 13100: 13087: 13063: 12970: 12947: 12934: 12914: 12911: 12901: 12888: 12863: 12843: 12840: 12816: 12813: 12800: 12787: 12784: 12769: 12766: 12742: 12739: 12726: 12721: 12718: 12703: 12700: 12680: 12677: 12667: 12654: 12636: 12620: 12617: 12587: 12574: 12513: 12510: 12467: 12465: 12461: 12456: 12453: 12448: 12427: 12424: 12383: 12380: 12360: 12357: 12328: 12325: 12294: 12272: 12256: 12248: 12224: 12223: 12222: 12198: 12195: 12185: 12156: 12153: 12133: 12130: 12116: 12115: 12114: 12107: 12103: 12099: 12097: 12094: 12077: 12074: 12071: 12068: 12065: 12058: 12042: 12030: 12026: 12022: 12017: 12014: 12011: 12001: 11996: 11993: 11990: 11976: 11957: 11938: 11924: 11921: 11918: 11899: 11896: 11893: 11880: 11875: 11871: 11863: 11847: 11844: 11841: 11831: 11826: 11823: 11820: 11781: 11779: 11775: 11769: 11765: 11762: 11759: 11743: 11728: 11724: 11720: 11715: 11705: 11700: 11697: 11694: 11680: 11664: 11652: 11648: 11644: 11639: 11629: 11624: 11621: 11618: 11604: 11585: 11563: 11549: 11530: 11517: 11512: 11508: 11500: 11499: 11498: 11495: 11480: 11477: 11474: 11467: 11451: 11441: 11436: 11422: 11406: 11396: 11391: 11365: 11349: 11336: 11328: 11323: 11309: 11308: 11307: 11264: 11261: 11232: 11229: 11214: 11208: 11198: 11196: 11179: 11146: 11129: 11127: 11123: 11118: 11090: 11086: 11081: 11078: 11075: 11067: 11063: 11044: 11040: 11037: 11023: 10999: 10995: 10990: 10987: 10984: 10978: 10975: 10959: 10952: 10949: 10933: 10921: 10920: 10919: 10875: 10866: 10852: 10849: 10846: 10826: 10823: 10802: 10796: 10793: 10789: 10771: 10755: 10751: 10748: 10732: 10724: 10721: 10715: 10712: 10703: 10701: 10674: 10666: 10639: 10612: 10594: 10575: 10558: 10553: 10550: 10547: 10541: 10537: 10534: 10531: 10528: 10526: 10504: 10484: 10479: 10473: 10470: 10454: 10447: 10444: 10428: 10420: 10415: 10412: 10410: 10388: 10362: 10356: 10335: 10332: 10329: 10316: 10311: 10303: 10300: 10292: 10290: 10267: 10246: 10233: 10228: 10220: 10217: 10209: 10207: 10188: 10165: 10164: 10163: 10147: 10137: 10132: 10122: 10117: 10088: 10059: 10054: 10049: 10039: 10028: 10014: 10006: 9988: 9960: 9955: 9948: 9945: 9939: 9933: 9927: 9919: 9911: 9908: 9901: 9897: 9892: 9887: 9875: 9874: 9873: 9865: 9864:, see below. 9840: 9810: 9807: 9794: 9774: 9771: 9760: 9744: 9741: 9714: 9711: 9703: 9687: 9668: 9662: 9653: 9636: 9630: 9623:: the larger 9610: 9587: 9581: 9574: 9555: 9540: 9535: 9532: 9522: 9520: 9516: 9509: 9502: 9495: 9476: 9471: 9465: 9458: 9452: 9447: 9442: 9436: 9429: 9426: 9420: 9415: 9412: 9407: 9401: 9394: 9388: 9383: 9378: 9366: 9362: 9358: 9353: 9343: 9338: 9324: 9323: 9322: 9317: 9310: 9291: 9288: 9280: 9274: 9267: 9264: 9258: 9251: 9245: 9240: 9233: 9228: 9222: 9215: 9209: 9204: 9201: 9195: 9186: 9180: 9173: 9170: 9164: 9157: 9151: 9146: 9143: 9137: 9129: 9121: 9099: 9085: 9066: 9053: 9048: 9044: 9036: 9035: 9034: 9028: 9020: 9012: 8992: 8987: 8981: 8974: 8971: 8965: 8960: 8955: 8949: 8946: 8939: 8936: 8930: 8925: 8922: 8917: 8911: 8904: 8901: 8895: 8890: 8885: 8873: 8869: 8865: 8860: 8850: 8845: 8831: 8830: 8829: 8824: 8816: 8796: 8793: 8785: 8779: 8776: 8769: 8766: 8760: 8753: 8747: 8744: 8739: 8736: 8730: 8721: 8715: 8708: 8705: 8699: 8692: 8686: 8681: 8678: 8672: 8664: 8656: 8637: 8623: 8604: 8591: 8586: 8582: 8574: 8573: 8572: 8567: 8559: 8539: 8534: 8528: 8521: 8518: 8512: 8507: 8502: 8496: 8493: 8486: 8483: 8477: 8470: 8464: 8459: 8452: 8447: 8441: 8434: 8431: 8426: 8421: 8415: 8412: 8405: 8402: 8396: 8391: 8386: 8369: 8365: 8361: 8356: 8346: 8341: 8327: 8326: 8325: 8320: 8313: 8294: 8289: 8283: 8276: 8270: 8265: 8260: 8254: 8251: 8244: 8241: 8235: 8228: 8225: 8220: 8215: 8209: 8202: 8196: 8191: 8186: 8174: 8170: 8166: 8161: 8151: 8146: 8132: 8131: 8130: 8125: 8106: 8103: 8098: 8095: 8090: 8082: 8076: 8073: 8066: 8063: 8057: 8050: 8044: 8039: 8032: 8027: 8021: 8014: 8008: 8005: 8000: 7997: 7991: 7982: 7976: 7973: 7966: 7963: 7957: 7950: 7944: 7941: 7936: 7933: 7927: 7919: 7911: 7889: 7875: 7856: 7843: 7838: 7834: 7826: 7825: 7824: 7818: 7812: 7809: 7805: 7798: 7791: 7772: 7767: 7757: 7752: 7746: 7743: 7736: 7733: 7727: 7722: 7717: 7711: 7704: 7698: 7691: 7685: 7680: 7673: 7668: 7662: 7657: 7652: 7646: 7639: 7633: 7628: 7623: 7609: 7608: 7607: 7602: 7598: 7591: 7584: 7577: 7556: 7550: 7540: 7534: 7529: 7524: 7516: 7513: 7499: 7496: 7488: 7483: 7471: 7470: 7469: 7461: 7442: 7436: 7429: 7423: 7418: 7413: 7399: 7398: 7397: 7380: 7375: 7369: 7362: 7356: 7351: 7346: 7338: 7334: 7324: 7320: 7313: 7306: 7300: 7295: 7288: 7283: 7277: 7272: 7255: 7254: 7253: 7251: 7247: 6835: 6829: 6815: 6811: 6790: 6771: 6757: 6753: 6749: 6744: 6736: 6733: 6730: 6720: 6715: 6693: 6679: 6675: 6671: 6666: 6661: 6632: 6629: 6626: 6614: 6611: 6608: 6604: 6600: 6595: 6582: 6577: 6553: 6539: 6538: 6537: 6521: 6492: 6489: 6486: 6454: 6451: 6448: 6429: 6426: 6423: 6409: 6405: 6401: 6396: 6393: 6385: 6382: 6379: 6369: 6364: 6342: 6339: 6336: 6322: 6318: 6314: 6309: 6304: 6280: 6277: 6274: 6260: 6259: 6258: 6248: 6235: 6227: 6224: 6221: 6211: 6206: 6189: 6185: 6181: 6176: 6171: 6151: 6132: 6115: 6111: 6107: 6102: 6092: 6087: 6084: 6081: 6067: 6066: 6065: 6043: 6019: 6005: 5981: 5978: 5975: 5962: 5959: 5954: 5950: 5942: 5941: 5940: 5932:. Initially, 5917: 5888: 5885: 5882: 5846: 5834: 5829: 5824: 5819: 5816: 5813: 5803: 5798: 5795: 5792: 5757: 5735: 5721: 5702: 5689: 5681: 5657: 5643: 5624: 5611: 5606: 5602: 5594: 5593: 5592: 5576: 5547: 5544: 5541: 5513: 5511: 5493: 5478: 5474: 5470: 5465: 5455: 5450: 5447: 5444: 5429: 5411: 5408: 5405: 5392: 5384: 5379: 5376: 5373: 5357: 5355: 5337: 5334: 5331: 5318: 5310: 5305: 5302: 5299: 5270: 5266: 5243: 5189: 5186: 5183: 5170: 5162: 5157: 5154: 5151: 5122: 5107: 5103: 5099: 5094: 5084: 5079: 5076: 5073: 5044: 5031: 5023: 5018: 4989: 4977: 4973: 4969: 4964: 4954: 4949: 4946: 4943: 4929:The formulas 4922: 4920: 4902: 4887: 4869: 4866: 4863: 4836: 4833: 4828: 4824: 4801: 4786: 4768: 4754:We note that 4747: 4745: 4708: 4705: 4702: 4672: 4669: 4666: 4663: 4660: 4649: 4637: 4633: 4629: 4624: 4621: 4618: 4608: 4603: 4600: 4597: 4576: 4557: 4543: 4540: 4537: 4518: 4515: 4512: 4499: 4494: 4490: 4473: 4470: 4467: 4442: 4427: 4423: 4419: 4414: 4404: 4399: 4396: 4393: 4375: 4363: 4359: 4355: 4350: 4340: 4335: 4332: 4329: 4308: 4286: 4272: 4253: 4240: 4235: 4231: 4211: 4208: 4205: 4195: 4185: 4180: 4158: 4141: 4117: 4104: 4096: 4091: 4072: 4071: 4070: 4068: 4050: 3996: 3964: 3959: 3955: 3950: 3947: 3944: 3940: 3936: 3933: 3930: 3927: 3924: 3921: 3918: 3914: 3910: 3907: 3904: 3901: 3898: 3892: 3889: 3886: 3880: 3867: 3864: 3861: 3854: 3845: 3835: 3832: 3807: 3802: 3798: 3777: 3774: 3769: 3753: 3715: 3701:. Therefore, 3664: 3635: 3620: 3602: 3573: 3546: 3543: 3540: 3520: 3517: 3512: 3488: 3459: 3408: 3381: 3378: 3373: 3354: 3325: 3296: 3256: 3247: 3223: 3206: 3193: 3173: 3160: 3155: 3151: 3142: 3137: 3134: 3130: 3120: 3116: 3108: 3105: 3092: 3088: 3081: 3078: 3070: 3058: 3054: 3050: 3045: 3032: 3029: 3027: 3017: 3014: 3011: 2998: 2987: 2986: 2985: 2970: 2966: 2957: 2951: 2947: 2929: 2925: 2902: 2872: 2864: 2840: 2826: 2807: 2794: 2786: 2762: 2745: 2732: 2712: 2699: 2694: 2690: 2682: 2681: 2680: 2661: 2649: 2645: 2641: 2636: 2626: 2621: 2618: 2615: 2601: 2600: 2599: 2580: 2565: 2541: 2527: 2503: 2488: 2485: 2482: 2478: 2474: 2469: 2459: 2454: 2440: 2439: 2438: 2436: 2417: 2401: 2397: 2393: 2375: 2348: 2326: 2296: 2291: 2278: 2270: 2265: 2251: 2250: 2249: 2247: 2243: 2238: 2234: 2229: 2227: 2220: 2217:. Note that 2216: 2209: 2205: 2198: 2191: 2186: 2182: 2177: 2170: 2166: 2162: 2155: 2136: 2127: 2114: 2100: 2090: 2089: 2088: 2086: 2082: 2063: 2054: 2037: 2022: 2021: 2020: 2018: 1999: 1993: 1983: 1974: 1952: 1922: 1919: 1913: 1899: 1892: 1891: 1890: 1889: 1881: 1872: 1864: 1857: 1853: 1849: 1844: 1837: 1828: 1819: 1813: 1811: 1793: 1764: 1743: 1727: 1723: 1702: 1693: 1689: 1669: 1649: 1639: 1634: 1611: 1606: 1590: 1585: 1581: 1573: 1572: 1571: 1545: 1539: 1529: 1524: 1513: 1506: 1502: 1498: 1486: 1480: 1470: 1465: 1454: 1447: 1443: 1437: 1432: 1429: 1426: 1422: 1418: 1413: 1389: 1377: 1373: 1367: 1362: 1359: 1356: 1352: 1348: 1343: 1319: 1309: 1292: 1278: 1277: 1276: 1253: 1221: 1190: 1185: 1168: 1164: 1158: 1153: 1150: 1147: 1143: 1139: 1134: 1114: 1102: 1098: 1092: 1087: 1084: 1081: 1077: 1073: 1068: 1054: 1053: 1052: 1033: 1003: 974: 959: 943: 923: 920: 917: 897: 894: 889: 865: 816: 788: 778: 775: 772: 767: 754: 751: 744: 743: 742: 637: 621: 610: 585: 574: 563: 547: 527: 516: 485: 484: 483: 482: 481:inner product 441: 438: 407: 406: 405: 381: 377: 372: 369: 365: 361: 355: 345: 329: 292: 288: 262: 259: 255: 251: 247: 243: 205: 202: 199: 149: 132: 131: 130: 129: 119: 117: 113: 108: 106: 102: 98: 94: 90: 85: 83: 79: 75: 71: 67: 63: 59: 55: 51: 47: 39: 35: 32:steps, where 31: 26: 21: 15591:Key concepts 15539: 15510: 15483: 15456: 15437: 15416: 15379:. Retrieved 15375: 15365: 15347: 15299:math/0605767 15289: 15285: 15279: 15242: 15238: 15228: 15201: 15197: 15191: 15164: 15160: 15154: 15137: 15133: 15123: 15104: 15098: 15087:. Retrieved 15078: 15071: 15044: 15038: 15014: 15007: 14994: 14987: 14952: 14948: 14938: 14911: 14887: 14851: 14842: 14823: 14817: 14798: 14794: 14774: 14288:, and define 14249: 14241: 14234:everywhere. 14178: 14167:has a large 14164: 14160: 14154: 14145: 14141: 14138: 14126: 14122: 14119: 14117: 14112: 14109: 14105: 14102: 14095: 14091: 14087: 14083: 14075: 14072: 14068: 14065: 14061: 14058: 14051: 14047: 14045: 13826: 13807: 13801: 13795: 13751: 13741: 13735: 13658: 13653: 13651: 13529: 13381: 13307: 13194: 13149: 13088: 12971: 12935: 12889: 12864: 12788: 12785: 12704: 12701: 12655: 12621: 12618: 12575: 12473: 12457: 12451: 12449: 12309: 12220: 12112: 12105: 12101: 12095: 11777: 11773: 11767: 11763: 11760: 11496: 11210: 11135: 11119: 11066:Gauss–Seidel 11059: 10867: 10704: 10698:denotes the 10663:denotes the 10631: 10029: 9975: 9871: 9654: 9536: 9528: 9518: 9514: 9507: 9500: 9493: 9492:The result, 9491: 9315: 9308: 9306: 9026: 9018: 9010: 9007: 8822: 8814: 8811: 8565: 8557: 8554: 8318: 8311: 8309: 8123: 8121: 7816: 7813: 7807: 7803: 7796: 7789: 7787: 7600: 7596: 7589: 7582: 7575: 7573: 7467: 7459: 7395: 7249: 7245: 7243: 6813: 6809: 6807: 6471: 6249: 6152: 6149: 6063: 5777: 5519: 5358: 4928: 4888:method from 4753: 4734: 4692:return 4066: 3754: 3280: 2955: 2949: 2945: 2887: 2678: 2597: 2399: 2395: 2311: 2245: 2236: 2232: 2230: 2224:is also the 2218: 2214: 2207: 2203: 2196: 2189: 2184: 2180: 2175: 2168: 2164: 2160: 2153: 2151: 2084: 2080: 2078: 2014: 1879: 1870: 1862: 1855: 1851: 1847: 1842: 1835: 1826: 1817: 1814: 1809: 1749: 1691: 1687: 1684: 1569: 1205: 809:is a set of 808: 652: 456: 379: 375: 373: 357: 286: 260: 257: 249: 245: 169: 125: 109: 89:optimization 86: 49: 43: 37: 33: 29: 15477:"Chapter 6" 15332:Ross, I. M. 15292:(4): 1267. 15245:: 276–285. 15167:(4): 1305. 13787:line search 10705:This shows 10005:polynomials 9684:is large, 46:mathematics 15736:Categories 15633:algorithms 15381:2023-12-05 15340:1902.09004 15089:2020-03-31 14962:2007.00640 14801:(6): 409. 14766:References 12096:end repeat 11776:exit loop 11205:See also: 9787:such that 4816:. Setting 4682:end repeat 368:eigenvalue 15660:CPU cache 15546:EMS Press 15252:1212.6680 15206:CiteSeerX 15169:CiteSeerX 15063:952572240 14979:0010-3640 14682:‖ 14665:− 14659:‖ 14619:‖ 14602:− 14596:‖ 14512:⋮ 14425:⋱ 14414:⋱ 14409:⋱ 14261:∈ 14232:transpose 14216:Hermitian 14071:, since 14020:γ 13993:γ 13963:˙ 13941:˙ 13902:γ 13898:− 13883:∇ 13874:γ 13870:− 13761:β 13546:β 13462:− 13399:β 13114:− 12912:− 12841:− 12814:− 12767:− 12740:− 12719:− 12678:− 12511:− 12425:− 12381:− 12358:− 12326:− 12267:^ 12196:− 12180:^ 12154:− 12131:− 12027:β 11872:β 11725:α 11721:− 11649:α 11509:α 11337:− 11262:− 11230:− 11147:κ 11091:κ 11082:− 11076:≈ 11038:≫ 11024:κ 11000:κ 10991:− 10985:≈ 10960:κ 10950:− 10934:κ 10907:∞ 10899:tends to 10876:κ 10847:ε 10827:ε 10794:− 10790:ε 10752:⁡ 10733:κ 10675:κ 10640:σ 10559:κ 10548:− 10538:⁡ 10529:≤ 10455:κ 10445:− 10429:κ 10413:≤ 10363:λ 10336:σ 10333:∈ 10330:λ 10317:∗ 10308:Π 10304:∈ 10293:≤ 10234:∗ 10225:Π 10221:∈ 10148:∗ 10138:− 10089:∗ 9985:Π 9916:Π 9912:∈ 9893:∗ 9884:Π 9841:κ 9808:− 9795:κ 9761:− 9742:− 9704:− 9663:κ 9631:κ 9582:κ 9427:− 9384:≈ 9363:α 9265:− 9202:− 9171:− 9144:− 9130:≈ 9045:α 8972:− 8947:− 8937:− 8902:− 8891:≈ 8870:β 8777:− 8767:− 8745:− 8737:− 8706:− 8679:− 8665:≈ 8583:β 8519:− 8508:≈ 8494:− 8484:− 8427:− 8413:− 8403:− 8366:α 8362:− 8266:≈ 8252:− 8242:− 8171:α 8104:≈ 8074:− 8064:− 8006:− 7998:− 7974:− 7964:− 7942:− 7934:− 7835:α 7744:− 7734:− 7658:− 7530:≈ 7252:given by 7141:step_size 7114:step_size 7060:step_size 6754:α 6721:− 6676:α 6630:− 6612:− 6605:β 6406:α 6397:− 6370:− 6319:α 6212:− 6186:α 6112:α 6108:− 5963:− 5951:β 5830:β 5825:− 5603:α 5475:α 5471:− 5393:− 5319:− 5267:α 5171:− 5104:α 5100:− 5032:− 4974:α 4825:β 4634:β 4491:β 4424:α 4420:− 4360:α 4232:α 4105:− 3948:− 3934:… 3911:⁡ 3905:∈ 3890:− 3876:⊤ 3865:− 3836:∈ 3544:≠ 3474:, i.e. 3194:− 3152:α 3147:⇒ 3117:α 3089:α 3055:α 2967:α 2926:α 2733:− 2691:α 2646:α 2479:∑ 2475:− 2390:, so the 2279:− 2248:th step: 2128:− 2098:∇ 1984:∈ 1953:− 1794:∗ 1724:α 1656:⟩ 1625:⟨ 1620:⟩ 1597:⟨ 1582:α 1503:α 1444:α 1423:∑ 1374:α 1353:∑ 1344:∗ 1165:α 1144:∑ 1135:∗ 1120:⇒ 1099:α 1078:∑ 1069:∗ 1004:∗ 921:≠ 776:… 635:⟩ 614:⟨ 608:⟩ 578:⟨ 572:⟩ 551:⟨ 537:⟩ 520:⟨ 330:∗ 242:symmetric 203:× 54:algorithm 15689:Software 15653:Hardware 15612:Problems 15513:. SIAM. 15390:cite web 15316:17614913 15271:51978658 14910:(1997). 14885:(1987). 14710:See also 13923:for the 13654:flexible 13534:formula 13387:formula 10839:for any 10777:‖ 10762:‖ 10665:spectrum 10600:‖ 10585:‖ 10510:‖ 10495:‖ 10394:‖ 10379:‖ 10274:‖ 10243:‖ 10194:‖ 10179:‖ 7464:Solution 7180:residual 7162:residual 7135:residual 7129:residual 7024:residual 6964:residual 6928:residual 6853:function 6472:because 4750:Restarts 4456:if 4130:if 3621:. Where 3109:′ 2242:residual 2226:residual 1546:⟩ 1514:⟨ 1487:⟩ 1455:⟨ 956:forms a 936:. Then 910:for all 56:for the 15548:, 2001 15342:, 2019. 12462:is the 12221:where 9292:0.4122. 8797:0.0088. 3340:, i.e. 2244:at the 2240:be the 2178:equals 1570:and so 1275:yields 851:, i.e. 697:, then 364:Lanczos 360:Arnoldi 289:), and 244:(i.e., 218:matrix 15711:LAPACK 15701:MATLAB 15517: 15492: 15463: 15444: 15425: 15314: 15269: 15208: 15171: 15111: 15061: 15051: 15026: 14977: 14926: 14858: 14719:(BiCG) 14539:Since 13195:Since 12972:Since 12786:Then: 11806: 11778:end if 11497:repeat 11375:Solve: 11062:Jacobi 10667:, and 10632:where 10580: 10490: 10374: 9976:where 9952: 9931: 9925: 9906: 9537:As an 9466:0.6364 9459:0.0909 9437:0.7229 9430:0.3511 9416:0.4122 9402:0.3384 9395:0.2356 9275:0.7229 9268:0.3511 9210:0.7229 9205:0.3511 9181:0.7492 9174:0.2810 9152:0.7492 9147:0.2810 8982:0.7229 8975:0.3511 8926:0.0088 8912:0.7492 8905:0.2810 8716:0.7492 8709:0.2810 8687:0.7492 8682:0.2810 8529:0.7492 8522:0.2810 8284:0.3384 8277:0.2356 8107:0.2205 7551:0.6364 7541:0.0909 7228:return 6910:eltype 6064:using 5861:. The 5778:since 4221:repeat 4014:where 3790:, then 3533:, for 2679:with 457:Since 293:, and 256:(i.e. 74:sparse 52:is an 48:, the 15696:ATLAS 15357:(PDF) 15336:arXiv 15312:S2CID 15294:arXiv 15267:S2CID 15247:arXiv 15083:(PDF) 14999:(PDF) 14957:arXiv 14791:(PDF) 14725:(CGS) 14129:is a 12702:But: 9727:with 7084:' 7033:while 2954:into 958:basis 404:) if 15675:SIMD 15515:ISBN 15490:ISBN 15461:ISBN 15442:ISBN 15423:ISBN 15396:link 15109:ISBN 15059:OCLC 15049:ISBN 15024:ISBN 14975:ISSN 14924:ISBN 14856:ISBN 14157:LSQR 14096:CGNR 14050:-by- 14011:and 13217:and 12994:and 11774:then 10850:> 10030:Let 9506:and 7156:norm 7039:> 7018:norm 6988:sqrt 6973:norm 6958:copy 6507:and 5137:and 5004:and 3908:span 3588:and 2486:< 2231:Let 960:for 378:and 291:real 110:The 99:and 15665:TLB 15304:doi 15257:doi 15216:doi 15179:doi 15142:doi 15020:195 14967:doi 14916:doi 14828:hdl 14803:doi 14108:= 14098:). 14094:or 14092:CGN 13330:or 11120:No 11064:or 11019:for 10749:log 10535:exp 10326:max 10297:min 10214:min 9655:If 8435:331 8229:331 8099:331 7234:end 7225:end 7042:tol 6994:sum 6904:eps 6898:tol 5939:is 3423:is 2361:at 2163:at 1824:by 1018:of 285:in 252:), 240:is 44:In 15738:: 15544:, 15538:, 15480:. 15392:}} 15388:{{ 15374:. 15324:^ 15310:. 15302:. 15290:29 15288:. 15265:. 15255:. 15243:51 15241:. 15237:. 15214:. 15202:22 15200:. 15177:. 15165:21 15163:. 15136:. 15132:. 15057:. 15022:. 14973:. 14965:. 14953:76 14951:. 14947:. 14922:. 14914:. 14897:^ 14870:^ 14799:49 14797:. 14793:. 14782:; 14171:. 14137:κ( 14106:Ax 13867::= 13789:, 13770::= 13733:. 13555::= 13408::= 13374:. 13305:. 12887:: 12466:. 12447:. 12108:+1 12069::= 12002::= 11881::= 11832::= 11770:+1 11761:if 11706::= 11630::= 11518::= 11478::= 11442::= 11397::= 11329::= 11128:. 11117:. 10865:. 10702:. 10123::= 10027:. 9898::= 9321:. 9033:. 8571:. 8432:73 8226:73 8096:73 7601:Ax 7599:- 7595:= 7517:11 7500:11 7248:= 7246:Ax 7171:@. 7126:@. 7099:@. 7009:)) 7003:.^ 6919:)) 6889::: 6877::: 6865::: 6828:. 5456::= 5385::= 5311::= 5163::= 5085::= 5024::= 4955::= 4921:. 4746:. 4664::= 4609::= 4500::= 4405::= 4341::= 4241::= 4209::= 4186::= 4097::= 3079:=: 2952:+1 2208:Ax 2206:− 2202:= 2183:− 2181:Ax 2167:= 1856:Ax 1854:− 1850:= 1848:Az 1841:= 1742:. 1690:= 1688:Ax 548::= 442:0. 344:. 261:Ax 248:= 105:Z4 15635:) 15631:( 15576:e 15569:t 15562:v 15523:. 15498:. 15469:. 15450:. 15431:. 15398:) 15384:. 15359:. 15338:: 15318:. 15306:: 15296:: 15273:. 15259:: 15249:: 15222:. 15218:: 15185:. 15181:: 15148:. 15144:: 15138:6 15117:. 15092:. 15065:. 15032:. 15001:. 14981:. 14969:: 14959:: 14932:. 14918:: 14891:. 14864:. 14830:: 14811:. 14805:: 14694:0 14691:= 14686:2 14676:n 14672:x 14668:W 14662:b 14655:, 14650:k 14646:) 14642:t 14638:/ 14634:1 14631:( 14628:= 14623:2 14613:k 14609:x 14605:W 14599:b 14576:b 14573:= 14570:x 14567:W 14547:W 14525:] 14519:0 14505:0 14498:1 14492:[ 14487:= 14484:b 14480:, 14475:] 14469:t 14466:+ 14463:1 14456:t 14441:t 14402:t 14387:t 14380:t 14377:+ 14374:1 14367:t 14352:t 14345:t 14342:+ 14339:1 14332:t 14317:t 14310:t 14304:[ 14299:= 14296:W 14276:) 14273:1 14270:, 14267:0 14264:( 14258:t 14201:b 14197:= 14193:x 14188:A 14165:A 14161:A 14146:A 14142:A 14139:A 14127:A 14123:A 14120:A 14113:b 14110:A 14103:A 14090:( 14084:A 14076:A 14073:A 14069:b 14066:A 14062:A 14059:A 14052:m 14048:n 14024:b 13997:a 13972:u 13969:= 13960:v 13953:, 13950:v 13947:= 13938:x 13927:, 13911:v 13906:b 13895:) 13892:x 13889:( 13886:f 13878:a 13864:) 13861:v 13858:, 13855:x 13852:( 13849:k 13846:= 13843:u 13835:, 13808:p 13802:z 13796:p 13773:0 13765:k 13726:k 13724:β 13710:, 13707:0 13704:= 13699:k 13694:z 13686:T 13680:1 13677:+ 13674:k 13669:r 13632:k 13627:z 13619:T 13613:k 13608:r 13599:1 13596:+ 13593:k 13588:z 13580:T 13574:1 13571:+ 13568:k 13563:r 13550:k 13510:k 13505:z 13497:T 13491:k 13486:r 13478:) 13472:k 13467:z 13457:1 13454:+ 13451:k 13446:z 13440:( 13433:T 13427:1 13424:+ 13421:k 13416:r 13403:k 13361:z 13339:L 13317:M 13288:z 13264:T 13258:L 13233:T 13227:L 13204:a 13180:z 13173:T 13167:L 13162:= 13158:a 13135:a 13128:T 13123:) 13117:1 13109:L 13104:( 13101:= 13097:z 13073:a 13047:a 13025:L 13003:L 12981:r 12957:a 12952:L 12948:= 12944:r 12921:r 12915:1 12907:L 12902:= 12898:a 12874:a 12850:r 12844:1 12836:L 12828:T 12823:) 12817:1 12809:L 12804:( 12801:= 12797:z 12770:1 12762:L 12754:T 12749:) 12743:1 12735:L 12730:( 12727:= 12722:1 12714:M 12687:r 12681:1 12673:M 12668:= 12664:z 12641:r 12637:= 12633:z 12630:M 12602:T 12596:L 12593:L 12588:= 12584:M 12560:L 12537:M 12514:1 12506:M 12483:M 12452:M 12434:A 12428:1 12420:M 12395:T 12390:) 12384:1 12376:E 12371:( 12367:A 12361:1 12353:E 12329:1 12321:M 12295:. 12291:x 12284:T 12278:E 12273:= 12264:x 12257:, 12253:M 12249:= 12243:T 12237:E 12234:E 12205:b 12199:1 12191:E 12186:= 12177:x 12168:T 12163:) 12157:1 12149:E 12144:( 12140:A 12134:1 12126:E 12106:k 12102:x 12078:1 12075:+ 12072:k 12066:k 12043:k 12038:p 12031:k 12023:+ 12018:1 12015:+ 12012:k 12007:z 11997:1 11994:+ 11991:k 11986:p 11958:k 11953:z 11945:T 11939:k 11934:r 11925:1 11922:+ 11919:k 11914:z 11906:T 11900:1 11897:+ 11894:k 11889:r 11876:k 11848:1 11845:+ 11842:k 11837:r 11827:1 11824:+ 11821:k 11816:z 11810:M 11802:e 11799:v 11796:l 11793:o 11790:S 11768:k 11764:r 11744:k 11739:p 11736:A 11729:k 11716:k 11711:r 11701:1 11698:+ 11695:k 11690:r 11665:k 11660:p 11653:k 11645:+ 11640:k 11635:x 11625:1 11622:+ 11619:k 11614:x 11586:k 11581:p 11578:A 11570:T 11564:k 11559:p 11550:k 11545:z 11537:T 11531:k 11526:r 11513:k 11481:0 11475:k 11452:0 11447:z 11437:0 11432:p 11407:0 11402:r 11392:0 11387:z 11381:M 11350:0 11345:x 11342:A 11333:b 11324:0 11319:r 11293:A 11271:A 11265:1 11257:M 11233:1 11225:M 11180:A 11158:) 11154:A 11150:( 11102:) 11098:A 11094:( 11087:2 11079:1 11045:. 11041:1 11035:) 11031:A 11027:( 11011:) 11007:A 11003:( 10996:2 10988:1 10979:1 10976:+ 10971:) 10967:A 10963:( 10953:1 10945:) 10941:A 10937:( 10887:) 10883:A 10879:( 10853:0 10824:2 10803:) 10797:1 10783:A 10772:0 10767:e 10756:( 10744:) 10740:A 10736:( 10725:2 10722:1 10716:= 10713:k 10686:) 10682:A 10678:( 10651:) 10647:A 10643:( 10613:, 10606:A 10595:0 10590:e 10576:) 10570:) 10566:A 10562:( 10554:k 10551:2 10542:( 10532:2 10516:A 10505:0 10500:e 10485:k 10480:) 10474:1 10471:+ 10466:) 10462:A 10458:( 10448:1 10440:) 10436:A 10432:( 10421:( 10416:2 10400:A 10389:0 10384:e 10370:| 10366:) 10360:( 10357:p 10353:| 10347:) 10343:A 10339:( 10312:k 10301:p 10280:A 10268:0 10263:e 10258:) 10254:A 10250:( 10247:p 10229:k 10218:p 10210:= 10200:A 10189:k 10184:e 10143:x 10133:k 10128:x 10118:k 10113:e 10084:x 10060:k 10055:) 10050:k 10045:x 10040:( 10015:k 9989:k 9961:, 9956:} 9949:1 9946:= 9943:) 9940:0 9937:( 9934:p 9928:: 9920:k 9909:p 9902:{ 9888:k 9852:) 9848:A 9844:( 9821:) 9817:A 9811:1 9803:M 9798:( 9775:0 9772:= 9769:) 9765:b 9757:x 9754:A 9750:( 9745:1 9737:M 9715:0 9712:= 9708:b 9700:x 9697:A 9672:) 9669:A 9666:( 9640:) 9637:A 9634:( 9611:A 9591:) 9588:A 9585:( 9556:k 9551:x 9519:n 9515:n 9511:0 9508:x 9504:1 9501:x 9497:2 9494:x 9477:. 9472:] 9453:[ 9448:= 9443:] 9421:[ 9413:+ 9408:] 9389:[ 9379:1 9374:p 9367:1 9359:+ 9354:1 9349:x 9344:= 9339:2 9334:x 9319:1 9316:x 9312:2 9309:x 9289:= 9281:] 9259:[ 9252:] 9246:3 9241:1 9234:1 9229:4 9223:[ 9216:] 9196:[ 9187:] 9165:[ 9158:] 9138:[ 9122:1 9117:p 9114:A 9106:T 9100:1 9095:p 9086:1 9081:r 9073:T 9067:1 9062:r 9054:= 9049:1 9030:0 9027:α 9022:1 9019:p 9014:1 9011:α 8993:. 8988:] 8966:[ 8961:= 8956:] 8950:3 8940:8 8931:[ 8923:+ 8918:] 8896:[ 8886:0 8881:p 8874:0 8866:+ 8861:1 8856:r 8851:= 8846:1 8841:p 8826:1 8823:p 8818:0 8815:β 8794:= 8786:] 8780:3 8770:8 8761:[ 8754:] 8748:3 8740:8 8731:[ 8722:] 8700:[ 8693:] 8673:[ 8657:0 8652:r 8644:T 8638:0 8633:r 8624:1 8619:r 8611:T 8605:1 8600:r 8592:= 8587:0 8569:1 8566:p 8561:0 8558:β 8540:. 8535:] 8513:[ 8503:] 8497:3 8487:8 8478:[ 8471:] 8465:3 8460:1 8453:1 8448:4 8442:[ 8422:] 8416:3 8406:8 8397:[ 8392:= 8387:0 8382:p 8376:A 8370:0 8357:0 8352:r 8347:= 8342:1 8337:r 8322:1 8319:r 8315:1 8312:x 8295:. 8290:] 8271:[ 8261:] 8255:3 8245:8 8236:[ 8221:+ 8216:] 8210:1 8203:2 8197:[ 8192:= 8187:0 8182:p 8175:0 8167:+ 8162:0 8157:x 8152:= 8147:1 8142:x 8127:1 8124:x 8091:= 8083:] 8077:3 8067:8 8058:[ 8051:] 8045:3 8040:1 8033:1 8028:4 8022:[ 8015:] 8009:3 8001:8 7992:[ 7983:] 7977:3 7967:8 7958:[ 7951:] 7945:3 7937:8 7928:[ 7920:= 7912:0 7907:p 7904:A 7896:T 7890:0 7885:p 7876:0 7871:r 7863:T 7857:0 7852:r 7844:= 7839:0 7820:0 7817:α 7808:k 7804:p 7800:0 7797:p 7793:0 7790:r 7773:. 7768:0 7763:p 7758:= 7753:] 7747:3 7737:8 7728:[ 7723:= 7718:] 7712:1 7705:2 7699:[ 7692:] 7686:3 7681:1 7674:1 7669:4 7663:[ 7653:] 7647:2 7640:1 7634:[ 7629:= 7624:0 7619:r 7604:0 7597:b 7593:0 7590:r 7586:0 7583:x 7579:0 7576:r 7557:] 7535:[ 7525:] 7514:7 7497:1 7489:[ 7484:= 7480:x 7443:] 7437:1 7430:2 7424:[ 7419:= 7414:0 7409:x 7381:, 7376:] 7370:2 7363:1 7357:[ 7352:= 7347:] 7339:2 7335:x 7325:1 7321:x 7314:[ 7307:] 7301:3 7296:1 7289:1 7284:4 7278:[ 7273:= 7269:x 7264:A 7250:b 7231:x 7219:= 7207:* 7204:2 7201:^ 7198:) 7192:/ 7186:( 7183:+ 7177:= 7165:) 7159:( 7153:= 7144:* 7138:- 7132:= 7117:* 7111:+ 7108:x 7105:= 7102:x 7093:) 7087:* 7078:( 7075:/ 7072:2 7069:^ 7063:= 7054:* 7051:A 7048:= 7027:) 7021:( 7015:= 7006:2 7000:x 6997:( 6991:( 6985:= 6982:) 6979:x 6976:( 6967:) 6961:( 6955:= 6946:x 6943:* 6940:A 6937:- 6934:b 6931:= 6922:) 6916:b 6913:( 6907:( 6901:= 6895:; 6886:x 6883:, 6874:b 6871:, 6862:A 6859:( 6825:k 6823:α 6819:β 6814:k 6810:p 6791:k 6786:r 6778:T 6772:k 6767:r 6758:k 6750:1 6745:= 6742:) 6737:1 6734:+ 6731:k 6726:r 6716:k 6711:r 6706:( 6700:T 6694:k 6689:r 6680:k 6672:1 6667:= 6662:k 6657:p 6651:A 6644:T 6639:) 6633:1 6627:k 6622:p 6615:1 6609:k 6601:+ 6596:k 6591:r 6586:( 6583:= 6578:k 6573:p 6567:A 6560:T 6554:k 6549:p 6522:k 6517:r 6493:1 6490:+ 6487:k 6482:r 6455:1 6452:+ 6449:k 6444:r 6436:T 6430:1 6427:+ 6424:k 6419:r 6410:k 6402:1 6394:= 6391:) 6386:1 6383:+ 6380:k 6375:r 6365:k 6360:r 6355:( 6349:T 6343:1 6340:+ 6337:k 6332:r 6323:k 6315:1 6310:= 6305:k 6300:p 6294:A 6287:T 6281:1 6278:+ 6275:k 6270:r 6254:k 6252:β 6236:, 6233:) 6228:1 6225:+ 6222:k 6217:r 6207:k 6202:r 6197:( 6190:k 6182:1 6177:= 6172:k 6167:p 6161:A 6133:k 6128:p 6122:A 6116:k 6103:k 6098:r 6093:= 6088:1 6085:+ 6082:k 6077:r 6044:k 6039:p 6033:A 6026:T 6020:k 6015:p 6006:k 6001:p 5995:A 5988:T 5982:1 5979:+ 5976:k 5971:r 5960:= 5955:k 5936:k 5934:β 5918:k 5913:p 5889:1 5886:+ 5883:k 5878:p 5865:k 5863:β 5847:k 5842:p 5835:k 5820:1 5817:+ 5814:k 5809:p 5804:= 5799:1 5796:+ 5793:k 5788:r 5758:k 5753:p 5750:A 5742:T 5736:k 5731:p 5722:k 5717:r 5709:T 5703:k 5698:r 5690:= 5682:k 5677:p 5671:A 5664:T 5658:k 5653:r 5644:k 5639:r 5631:T 5625:k 5620:r 5612:= 5607:k 5577:k 5572:r 5548:1 5545:+ 5542:k 5537:r 5524:k 5522:α 5494:k 5489:p 5486:A 5479:k 5466:k 5461:r 5451:1 5448:+ 5445:k 5440:r 5412:1 5409:+ 5406:k 5401:x 5398:A 5389:b 5380:1 5377:+ 5374:k 5369:r 5338:1 5335:+ 5332:k 5327:x 5324:A 5315:b 5306:1 5303:+ 5300:k 5295:r 5271:k 5244:k 5239:p 5236:A 5213:A 5190:1 5187:+ 5184:k 5179:x 5176:A 5167:b 5158:1 5155:+ 5152:k 5147:r 5123:k 5118:p 5115:A 5108:k 5095:k 5090:r 5080:1 5077:+ 5074:k 5069:r 5045:k 5040:x 5037:A 5028:b 5019:k 5014:r 4990:k 4985:p 4978:k 4970:+ 4965:k 4960:x 4950:1 4947:+ 4944:k 4939:x 4903:k 4898:x 4870:1 4867:+ 4864:k 4859:x 4837:0 4834:= 4829:k 4802:0 4797:x 4769:1 4764:x 4739:k 4737:β 4709:1 4706:+ 4703:k 4698:x 4673:1 4670:+ 4667:k 4661:k 4650:k 4645:p 4638:k 4630:+ 4625:1 4622:+ 4619:k 4614:r 4604:1 4601:+ 4598:k 4593:p 4577:k 4572:r 4564:T 4558:k 4553:r 4544:1 4541:+ 4538:k 4533:r 4525:T 4519:1 4516:+ 4513:k 4508:r 4495:k 4474:1 4471:+ 4468:k 4463:r 4443:k 4438:p 4435:A 4428:k 4415:k 4410:r 4400:1 4397:+ 4394:k 4389:r 4376:k 4371:p 4364:k 4356:+ 4351:k 4346:x 4336:1 4333:+ 4330:k 4325:x 4309:k 4304:p 4301:A 4293:T 4287:k 4282:p 4273:k 4268:r 4260:T 4254:k 4249:r 4236:k 4212:0 4206:k 4196:0 4191:r 4181:0 4176:p 4159:0 4154:x 4142:0 4137:r 4118:0 4113:x 4110:A 4101:b 4092:0 4087:r 4067:0 4051:0 4046:x 4023:A 4001:b 3997:= 3993:x 3988:A 3965:} 3960:} 3956:b 3951:1 3945:k 3941:A 3937:, 3931:, 3928:b 3925:A 3922:, 3919:b 3915:{ 3902:y 3899:: 3896:) 3893:y 3887:x 3884:( 3881:A 3872:) 3868:y 3862:x 3859:( 3855:{ 3846:n 3841:R 3833:y 3828:n 3825:i 3822:m 3819:g 3816:r 3813:a 3808:= 3803:k 3799:x 3778:0 3775:= 3770:0 3765:x 3739:x 3716:k 3711:x 3688:A 3665:i 3660:p 3636:i 3631:r 3603:i 3598:r 3574:i 3569:p 3547:j 3541:i 3521:0 3518:= 3513:j 3508:p 3502:A 3495:T 3489:i 3484:p 3460:j 3455:p 3432:A 3409:i 3404:p 3382:0 3379:= 3374:j 3369:r 3361:T 3355:i 3350:r 3326:j 3321:r 3297:i 3292:r 3257:. 3248:k 3243:p 3237:A 3230:T 3224:k 3219:p 3212:) 3207:k 3202:x 3199:A 3190:b 3186:( 3180:T 3174:k 3169:p 3161:= 3156:k 3143:0 3138:! 3135:= 3126:) 3121:k 3113:( 3106:g 3098:) 3093:k 3085:( 3082:g 3076:) 3071:k 3066:p 3059:k 3051:+ 3046:k 3041:x 3036:( 3033:f 3030:= 3023:) 3018:1 3015:+ 3012:k 3007:x 3002:( 2999:f 2971:k 2956:f 2950:k 2946:x 2930:k 2903:k 2898:r 2873:, 2865:k 2860:p 2854:A 2847:T 2841:k 2836:p 2827:k 2822:r 2814:T 2808:k 2803:p 2795:= 2787:k 2782:p 2776:A 2769:T 2763:k 2758:p 2751:) 2746:k 2741:x 2738:A 2729:b 2725:( 2719:T 2713:k 2708:p 2700:= 2695:k 2662:k 2657:p 2650:k 2642:+ 2637:k 2632:x 2627:= 2622:1 2619:+ 2616:k 2611:x 2581:i 2576:p 2566:i 2561:p 2555:A 2548:T 2542:i 2537:p 2528:k 2523:r 2517:A 2510:T 2504:i 2499:p 2489:k 2483:i 2470:k 2465:r 2460:= 2455:k 2450:p 2418:k 2413:p 2400:k 2396:r 2376:k 2371:x 2349:f 2327:k 2322:r 2297:. 2292:k 2287:x 2284:A 2275:b 2271:= 2266:k 2261:r 2246:k 2237:k 2233:r 2222:0 2219:p 2211:0 2204:b 2200:0 2197:p 2193:0 2190:x 2185:b 2176:f 2172:0 2169:x 2165:x 2161:f 2157:0 2154:p 2137:. 2132:b 2124:x 2119:A 2115:= 2112:) 2108:x 2104:( 2101:f 2085:x 2083:( 2081:f 2064:, 2059:A 2055:= 2052:) 2049:) 2045:x 2041:( 2038:f 2035:( 2031:H 2000:. 1994:n 1989:R 1980:x 1975:, 1971:b 1964:T 1958:x 1949:x 1944:A 1937:T 1931:x 1923:2 1920:1 1914:= 1911:) 1907:x 1903:( 1900:f 1883:∗ 1880:x 1874:∗ 1871:x 1866:0 1863:x 1859:0 1852:b 1843:0 1839:0 1836:x 1830:0 1827:x 1821:∗ 1818:x 1810:n 1789:x 1765:k 1760:p 1728:k 1703:n 1692:b 1670:. 1661:A 1650:k 1645:p 1640:, 1635:k 1630:p 1616:b 1612:, 1607:k 1602:p 1591:= 1586:k 1552:A 1540:k 1535:p 1530:, 1525:k 1520:p 1507:k 1499:= 1493:A 1481:i 1476:p 1471:, 1466:k 1461:p 1448:i 1438:n 1433:1 1430:= 1427:i 1419:= 1414:i 1409:p 1403:A 1396:T 1390:k 1385:p 1378:i 1368:n 1363:1 1360:= 1357:i 1349:= 1339:x 1333:A 1326:T 1320:k 1315:p 1310:= 1306:b 1299:T 1293:k 1288:p 1260:T 1254:k 1249:p 1226:b 1222:= 1218:x 1215:A 1191:. 1186:i 1181:p 1175:A 1169:i 1159:n 1154:1 1151:= 1148:i 1140:= 1130:x 1124:A 1115:i 1110:p 1103:i 1093:n 1088:1 1085:= 1082:i 1074:= 1064:x 1038:b 1034:= 1030:x 1027:A 999:x 975:n 970:R 944:P 924:j 918:i 898:0 895:= 890:j 885:p 879:A 872:T 866:i 861:p 838:A 817:n 794:} 789:n 784:p 779:, 773:, 768:1 763:p 758:{ 755:= 752:P 728:u 706:v 684:v 662:u 638:. 631:v 626:A 622:, 618:u 611:= 604:v 597:T 591:A 586:, 582:u 575:= 568:v 564:, 560:u 555:A 542:A 532:v 528:, 524:u 517:= 513:v 508:A 501:T 495:u 466:A 439:= 435:v 430:A 423:T 417:u 391:A 380:v 376:u 362:/ 325:x 302:b 287:R 272:x 258:x 250:A 246:A 227:A 206:n 200:n 179:x 154:b 150:= 146:x 141:A 38:n 34:n 30:n

Text is available under the Creative Commons Attribution-ShareAlike License. Additional terms may apply.

Knowledge

Conjugate gradient method

Index