BMRM

(Bundle Methods for Regularized Risk Minimization)

version 1.0 (alpha)

9 August 2007

Notice

A new version of BMRM with more complete features is coming soon. This version is mainly for experimental use.

Overview

BMRM is an open source, modular and scalable convex solver for many machine learning problems cast in the form of regularized risk minimization problem [1]. It is "modular" because the (problem-specific) loss function module is decoupled from the (regularization-specific) optimization module (e.g. quadratic programming or linear programming solvers), thus shorten the time to implement/prototype solutions to new problems. Besides, the decoupling leads to easier parallelization of the loss function computation. At the moment, BMRM can solve the following problems (and more problems soon),

  1. Binary classification
  2. Univariate regression
  3. Novelty detection (1-class SVM) [11]
  4. Quantile regression [12]
  5. Poisson regression [13]
  6. Ranking

along with either l1 or l2 regularizer. Also, users can add new loss function for problems with structured input and output variables.

Download

BMRM version 1.0

Installation

BMRM version 1.0 requires the following external (publicly available) libraries: Prior to the installation of BMRM, the above libraries must be installed properly. An example of installation of BMRM (including the required external libraries) on a linux machine is presented in the README file in the source code bundle.

Disclaimer

BMRM is licensed under Mozilla Public License version 1.1. The authors are not responsible for any implications from the use of the software.

FAQs, Suggestions, Comments, Bug Reports

(You are welcomed to expand this list ;-)

Contacts

Choon Hui Teo | Quoc Le | Alex Smola | SVN Vishwanathan | Markus Weimer

References

[1] C. H. Teo, Q. Le, A. J. Smola and S. V. N. Vishwanathan, A Scalable Modular Convex Solver for Regularized Risk Minimization, KDD, 2007. [pdf]
[2] K. P. Bennett and O. L. Mangasarian, Robust Linear Programming Discrimination of Two Linearly Inseparable Sets, Optimization Methods and Software, 1:23-24, 1992.
[3] O. Chappelle, Training a Support Vector Machine in the Primal, Neural Computation, 2007.
[4] M. Collins, R. E. Schapire and Y. Singer, Logistic regression, AdaBoost and Bregman distances, COLT, 2000.
[5] R. Cowell, A. David, S. Lauritzen and D. Spiegelhalter, Probabilistic Networks and Expert Systems, Springer, New York, 1999.
[6] T. Joachims, A Support Vector Method for Multivariate Performance Measures, ICML, 2005.
[7] T. Joachims, Training linear SVMs in linear time, KDD, 2006.
[8] V. Vapnik, S. Golowich and A. J. Smola, Support Vector Method for Function Approximation, Regression Estimation, and Signal Processing, NIPS, 1997.
[9] K.-R. Mueller, A. J. Smola, G. Raetsch, B. Schoelkopf, J. Kphlmorgen and V. Vapnik, Predicting Time Series with Support Vector Machines, ICANN, 1997.
[10] C. K. I. Williams, Prediction with Gaussian Processes: From Linear Regression to Linear Prediction and Beyond, M. I. Jordan, editor, Learning and Inference in Graphical Models, 1998.
[11] B. Schoelkopf, R. C. Williamson, A. J. Smola, J. Shawe-Taylor and J. Platt, Support Vector Method for Novelty Detection, NIPS, 2000.
[12] R. Koenker, Quantile Regression, Cambridge University Press, 2005.
[13] N. A. C. Cressie, Statistics for Spatial Data, John Wiley and Sons, New York, 1993.
[14] Q. Le and A. J. Smola, Direct Optimization of Ranking Measures, JMLR, submitted. 2007.
[15] R. Herbrich, T. Graepel and K. Obermayer, Large Margin Rank Boundries for Ordinal Regression, Advanced in Large Margin Classifiers, MIT Press, MA, 2000.

Last modified: 10 August 2007