4. Process Modeling 4.1. Introduction to Process Modeling 4.1.4. What are the some of the different statistical methods for model building? 4.1.4.2. Nonlinear Least Sum of Squares Regression
Extension of Linear Least Squares Regression	Nonlinear least sum of squares regression extends linear least squares regression for use with a much larger and more general class of functions. Almost any function that can be written down in closed form can be incorporated in a nonlinear regression model. Unlike linear regression, there are very few limitations on the way parameters can be used in the functional part of a nonlinear regression model. The way in which the unknown parameters in the function are estimated, however, is conceptually the same as it is in linear least sum of squares regression.
Definition of a Nonlinear Least Squares Model	As the name suggests, a nonlinear model is any model of the basic form . in which the functional part of the model is not linear with respect to the unknown parameters, , and the method of least squares is used to estimate the values of the unknown parameters.
	Due to the way in which the unknown parameters of the function are usually estimated, however, it is often much easier to work with models for which meet two additional criteria the function is smooth with respect to the unknown parameters, and the least squares criterion that is used to obtain the parameter estimates has a unique solution. These last two criteria are not essential parts of the defintion of a nonlinear least squares model, but are of practical importance.
Examples of Nonlinear Models	Some examples of nonlinear models include:
Advantages of Nonlinear Least Squares	The biggest advantage of nonlinear least squares regression over many other techniques is the broad range of functions that can be fit. Although many scientfic and engineering processes can be described well using linear models, or other relatively simple types of models, there are many other processes that are inherently nonlinear. For example, the strengthening of concrete as it cures is a nonlinear process. Research on concrete strength shows that the strength increases quickly at first and then levels off, or approaches an asymptote in mathematical terms, over time. Linear models do not describe processes that asymptote well because for all linear functions the function value always approaches plus or minus infinity as the explanatory variables go to the extremes. There are many types of nonlinear models, on the other hand, that describe the asymptotic behavior of a process well. Like the asymptotic behavior of some processes, other features of physical processes can often be expressed using nonlinear models more easily than with simpler model types.
	Being a "least sum of squares" procedure, nonlinear least squares shares some of the same advantages (and disadvantages) that linear least squares regression has over other methods. One shared advantage is efficient use of data. Nonlinear regression can produce good estimates of the unknown parameters in the model with relatively small datasets. Another advantage that nonlinear least squares shares with linear least squares is a fairly well developed theory for computing confidence, prediction and calibration intervals to answer scientific and engineering questions. In most cases the probabilistic interpretation of the intervals produced by nonlinear regression are only approximately correct, but these intervals still work very well in practice.
Disadvantages of Nonlinear Least Squares	The major cost of moving to nonlinear least squares regression from simpler modeling techniques like linear least squares is the need to use iterative optimization procedures to compute the parameter estimates. With functions that are linear in the parameters the least squares the estimates of the parameters can always be obtained analytically, while that is generally not the case with nonlinear models. The use of iterative procedures requires the user to provide starting values for the unknown parameters before the software can begin the optimization. The starting values must be reasonably close to the as yet unknown parameter estimates or the optimization procedure will not converge. Bad starting values can also cause the software to converge to a local minimum rather than the global minimum that defines the least squares estimates.
	Disadvantages shared with linear least squares procedures include a strong sensitivity to outliers. Just as in a linear least squares analysis, the presence of one or two outliers in the data can seriously affect the results of a nonlinear analysis. In addition, unfortunately, there are fewer model validation tools for the detection of outliers in nonlinear regression than there are for linear regression.