Preface ........................................................ ix
1 Introduction ................................................. 1
1.1 Classification of Parameter Estimation and Inverse
Problems ................................................ 1
1.2 Examples of Parameter Estimation Problems ............... 4
1.3 Examples of Inverse Problems ............................ 8
1.4 Discretizing Integral Equations ........................ 14
1.5 Why Inverse Problems Are Difficult ..................... 19
1.6 Exercises .............................................. 22
1.7 Notes and Further Reading .............................. 23
2 Linear Regression ........................................... 25
2.1 Introduction to Linear Regression ...................... 25
2.2 Statistical Aspects of Least Squares ................... 27
2.3 An Alternative View of the 95% Confidence Ellipsoid .... 37
2.4 Unknown Measurement Standard Deviations ................ 38
2.5 Li Regression .......................................... 42
2.6 Monte Carlo Error Propagation .......................... 47
2.7 Exercises .............................................. 49
2.8 Notes and Further Reading .............................. 52
3 Rank Deficiency and Ill-Conditioning ........................ 55
3.1 The SVD and the Generalized Inverse .................... 55
3.2 Covariance and Resolution of the Generalized Inverse
Solution ............................................... 62
3.3 Instability of the Generalized Inverse Solution ........ 64
3.4 A Rank Deficient Tomography Problem .................... 68
3.5 Discrete Ill-Posed Problems ............................ 74
3.6 Exercises .............................................. 87
3.7 Notes and Further Reading .............................. 91
4 Tikhonov Regularization ..................................... 93
4.1 Selecting Good Solutions to Ill-Posed Problems ......... 93
4.2 SVD Implementation of Tikhonov Regularization .......... 95
4.3 Resolution, Bias, and Uncertainty in the Tikhonov
Solution ............................................... 99
4.4 Higher-Order Tikhonov Regularization .................. 103
4.5 Resolution in Higher-Order Tikhonov Regularization .... 111
4.6 The TGSVD Method ...................................... 113
4.7 Generalized Cross-Validation .......................... 115
4.8 Error Bounds .......................................... 119
4.9 Exercises ............................................. 124
4.10 Notes and Further Reading ............................. 127
5 Discretizing Problems Using Basis Functions ................ 129
5.1 Discretization by Expansion of the Model .............. 129
5.2 Using Representers as Basis Functions ................. 133
5.3 The Method of Backus and Gilbert ...................... 134
5.4 Exercises ............................................. 139
5.5 Notes and Further Reading ............................. 140
6 Iterative Methods .......................................... 141
6.1 Introduction .......................................... 141
6.2 Iterative Methods for Tomography Problems ............. 142
6.3 The Conjugate Gradient Method ......................... 150
6.4 The CGLS Method ....................................... 155
6.5 Resolution Analysis for Iterative Methods ............. 160
6.6 Exercises ............................................. 166
6.7 Notes and Further Reading ............................. 168
7 Additional Regularization Techniques ....................... 169
7.1 Using Bounds as Constraints ........................... 169
7.2 Sparsity Regularization ............................... 174
7.3 Using IRLS to Solve L1 Regularized Problems ........... 176
7.4 Total Variation ....................................... 186
7.5 Exercises ............................................. 191
7.6 Notes and Further Reading ............................. 192
8 Fourier Techniques ......................................... 193
8.1 Linear Systems in the Time and Frequency Domains ...... 193
8.2 Linear Systems in Discrete Time ....................... 199
8.3 Water Level Regularization ............................ 204
8.4 Tikhonov Regularization in the Frequency Domain ....... 208
8.5 Exercises ............................................. 214
8.6 Notes and Further Reading ............................. 215
9 Nonlinear Regression ....................................... 217
9.1 Introduction to Nonlinear Regression .................. 217
9.2 Newton's Method for Solving Nonlinear Equations ....... 217
9.3 The Gauss-Newton and Levenberg-Marquardt Methods for
Solving Nonlinear Least Squares Problems .............. 220
9.4 Statistical Aspects of Nonlinear Least Squares ........ 224
9.5 Implementation Issues ................................. 228
9.6 Exercises ............................................. 234
9.7 Notes and Further Reading ............................. 237
10 Nonlinear Inverse Problems ................................. 239
10.1 Regularizing Nonlinear Least Squares Problems ......... 239
10.2 Occam's Inversion ..................................... 244
10.3 Model Resolution in Nonlinear Inverse Problems ........ 248
10.4 Exercises ............................................. 251
10.5 Notes and Further Reading ............................. 252
11 Bayesian Methods ........................................... 253
11.1 Review of the Classical Approach ...................... 253
11.2 The Bayesian Approach ................................. 255
11.3 The Multivariate Normal Case .......................... 260
11.4 The Markov Chain Monte Carlo Method ................... 269
11.5 Analyzing MCMC Output ................................. 273
11.6 Exercises ............................................. 278
11.7 Notes and Further Reading ............................. 280
12 Epilogue ................................................... 281
Appendix A. Review of Linear Algebra .......................... 283
A.1 Systems of Linear Equations ........................... 283
A.2 Matrix and Vector Algebra ............................. 286
A.3 Linear Independence ................................... 292
A.4 Subspaces of Rn ....................................... 293
A.5 Orthogonality and the Dot Product ..................... 298
A.6 Eigenvalues and Eigenvectors .......................... 302
A.7 Vector and Matrix Norms ............................... 304
A.8 The Condition Number of a Linear System ............... 306
A.9 The QR Factorization .................................. 308
A.10 Complex Matrices and Vectors .......................... 310
A.11 Linear Algebra in Spaces of Functions ................. 311
A.12 Exercises ............................................. 312
A.13 Notes and Further Reading ............................. 314
Appendix B Review of Probability and Statistics .............. 315
B.1 Probability and Random Variables ...................... 315
B.2 Expected Value and Variance ........................... 321
B.3 Joint Distributions ................................... 323
B.4 Conditional Probability .............................. 326
B.5 The Multivariate Normal Distribution .................. 329
B.6 The Central Limit Theorem ............................. 330
B.7 Testing for Normality ................................. 330
B.8 Estimating Means and Confidence Intervals ............. 332
B.9 Hypothesis Tests ...................................... 334
B.10 Exercises ............................................. 336
B.11 Notes and Further Reading ............................. 337
Appendix C Review of Vector Calculus ......................... 339
C.1 The Gradient, Hessian, and Jacobian ................... 339
C.2 Taylor's Theorem ...................................... 341
C.3 Lagrange Multipliers .................................. 341
C.4 Exercises ............................................. 344
C.5 Notes and Further Reading ............................. 345
Appendix D Glossary of Notation .............................. 347
Bibliography .................................................. 349
Index ......................................................... 355
|
