Statistical Learning (SL) problems often have to do with understanding the relationships between two elements, X and Y. These relationships are expressed in the form of Y being a function of X i.e., Y = f(X). Therefore, SL problems often imply solving for f or estimating f. In estimating f, we can use parametric or non-parametric methods.

Parametric methods begin by assuming a specific functional form for f, and turns the problem of estimating f into a task of estimating a set of parameters. Therefore the steps involved in parametric method of estimating f are basically:

1. Making an assumption about the functional form of f and specifying the associated model, be it linear or non-linear.
2. Fitting the training data to the specified model.

As a shortcoming, many functional specifications of f will not match the true form of f. But, there are specifications that can be used to try to increase model flexibility to better fit f, often with more parameters. However, beware that more parameters can impose an overfitting risk.

Non-parametric methods, as you may already be thinking from the word ‘Non-parametric’, imply estimating f without relying on parameters. Here, we estimate f by trying different estimates of f based on how close they fit the data. This means, we do not assume or follow a specific form or shape of f. However, to get the best estimate, we often rely on large observation sets. This is not necessarily the case in parametric methods.

Also here with non-parametric methods, we can be at risk of overfitting depending on our fitting strategy. For instance, when using a thin-plate spline, setting a very low level of smoothness that generates a perfect data-fitting f plane can create an undesirable case of overfitting. This is because the fit obtained, being too exact, will not yield accurate estimates of the response when different set of observations are used.

There is yet much to learn about parametric and non-parametric methods. This note, which is based on content from chapter 2 of ISL only provide a basic introduction to these methods.

Speaking of parametricism as used in the blog title, parametricism is a style of architecture. A style of contemporary architecture illustrated, for example, by the Mercedes Benz Museum in Stuttgart. From my perspective as an architecturer layman, these parametric structures tend to have rounded forms compared to the more boxy or linear shaped buildings. It’s as if the design flows, taking advantage of different parameter forms to develop continuous, smooth, and adaptive buildings. These go beyond using linear parametric models to rather more flexible ones.