Highly Efficient Hierarchical Online Nonlinear Regression Using Second Order Methods
| Autor: | Ibrahim Delibalta, Burak C. Civek, Suleyman S. Kozat |
|---|---|
| Jazyk: | angličtina |
| Rok vydání: | 2017 |
| Předmět: |
FOS: Computer and information sciences
Mathematical optimization Hierarchical tree Computational complexity theory Statistical assumption 02 engineering and technology Overfitting Machine Learning (cs.LG) Piecewise linear function 020204 information systems 0202 electrical engineering electronic engineering information engineering Nonlinear regression Electrical and Electronic Engineering Segmented regression Mathematics Linear model 020206 networking & telecommunications Nonlinear system Computer Science - Learning Piecewise linear regression Control and Systems Engineering Newton method Online learning Signal Processing Computer Vision and Pattern Recognition Software |
| Zdroj: | Signal Processing |
| Popis: | Highly efficient sequential nonlinear regression algorithms are proposed.Piecewise linear models are used for the nonlinear modeling.Region boundaries are continuously updated according to the data statistics.Second order NewtonRaphson methods are used for the adaptation of boundaries. We introduce highly efficient online nonlinear regression algorithms that are suitable for real life applications. We process the data in a truly online manner such that no storage is needed, i.e., the data is discarded after being used. For nonlinear modeling we use a hierarchical piecewise linear approach based on the notion of decision trees where the space of the regressor vectors is adaptively partitioned based on the performance. As the first time in the literature, we learn both the piecewise linear partitioning of the regressor space as well as the linear models in each region using highly effective second order methods, i.e., NewtonRaphson Methods. Hence, we avoid the well known over fitting issues by using piecewise linear models, however, since both the region boundaries as well as the linear models in each region are trained using the second order methods, we achieve substantial performance compared to the state of the art. We demonstrate our gains over the well known benchmark data sets and provide performance results in an individual sequence manner guaranteed to hold without any statistical assumptions. Hence, the introduced algorithms address computational complexity issues widely encountered in real life applications while providing superior guaranteed performance in a strong deterministic sense. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|