Abstract

The real-world data is often susceptible to label noise, which might constrict the effectiveness of the existing state of the art algorithms for ordinal regression. Existing works on ordinal regression do not take label noise into account. We propose a theoretically grounded approach for class conditional label noise in ordinal regression problems. We present a deep learning implementation of two commonly used loss functions for ordinal regression that is both-1) robust to label noise, and 2) rank consistent for a good ranking rule. We verify these properties of the algorithm empirically and show robustness to label noise on real data and rank consistency. To the best of our knowledge, this is the first approach for robust ordinal regression models.

Abstract

In many real-life classification problems, we may not get exact class labels for training samples. One such example is bandit feedback in multiclass classification. In this setting, we only get to know whether our predicted label is correct or not. Due to which, we are left in uncertainty about the actual class label when we predict the wrong class. This paper proposes exact passive-aggressive online algorithms for multiclass classification under bandit feedback (EPABF). The proposed approach uses an exploration-exploitation strategy to guess the class label in every trial. To update the weights, we solve a quadratic optimization problem under multiple class separability constraints and find the exact solution. We do this by finding active constraints using the KKT conditions of the optimization problem. These constraints form a support set that determines the classes for which the weight vector needs to be updated. We propose three different variants of the weight update rule, which vary based on the aggressiveness to correct the mistake. These are called EPABF, EPABF-I, and EPABF-II. We also provide mistake bounds for the proposed EPABF, EPABF-I, and EPABF-II. Experiments demonstrated that our proposed algorithms perform better than other bandit feedback-based approaches and comparably to the full information approaches.

Abstract

Active learning is an important technique to reduce the number of labeled examples in supervised learning. Active learning for binary classification has been well addressed in machine learning. However, active learning of the reject option classifier remains unaddressed. In this paper, we propose novel algorithms for active learning of reject option classifiers. We develop an active learning algorithm using double ramp loss function. We provide mistake bounds for this algorithm. We also propose a new loss function called double sigmoid loss function for reject option and corresponding active learning algorithm. We offer a convergence guarantee for this algorithm. We provide extensive experimental results to show the effectiveness of the proposed algorithms. The proposed algorithms efficiently reduce the number of label examples required.

Abstract

In this paper, we address the problem of robust learning of multilabel classifiers when the training data has label noise. We consider learning algorithms in the risk-minimization framework. We define what we call symmetric label noise in multi-label settings which is a useful noise model for many random errors in the labeling of data. We prove that risk minimization is robust to symmetric label noise if the loss function satisfies some conditions. We show that Hamming loss and couple of surrogates of Hamming loss satisfy these sufficient conditions and hence are robust. By learning feedforward neural networks on some benchmark multi-label datasets, we provide empirical evidence to illustrate our theoretical results on robust learning of multi-label classifiers under label noise.

Abstract

In this paper, we propose online algorithms for multiclass classification using partial labels. We propose two variants of Perceptron called Avg Perceptron and Max Perceptron to deal with the partially labeled data. We also propose Avg Pegasos and Max Pegasos, which are extensions of the Pegasos algorithm. We also provide mistake bounds for Avg Perceptron and regret bound for Avg Pegasos. We show the effectiveness of the proposed approaches by experimenting on various datasets and comparing them with the standard Perceptron and Pegasos.

Abstract

This paper addresses the problem of multiclass classification with corrupted or noisy bandit feedback. In this setting, the learner may not receive true feedback. Instead, it receives feedback that has been flipped with some non-zero probability. We propose a novel approach to deal with noisy bandit feedback, based on the unbiased estimator technique. We further propose an approach that can efficiently estimate the noise rates, and thus providing an end-to-end framework. The proposed algorithm enjoys mistake bound of the order of O(√T). We provide a theoretical mistake bound for our proposal. We also carry out extensive experiments on several benchmark datasets to demonstrate that our proposed approach successfully learns the underlying classifier even using noisy bandit feedbacks.

Abstract

n this article, we propose exact passive-aggressive(PA) online algorithms for ordinal regression. The proposedalgorithms can be used even when we have interval labelsinstead of actual labels for example. The proposed algorithmssolve a convex optimization problem at every trial. We find anexact solution to those optimization problems to determine theupdated parameters. We propose a support class algorithm (SCA)that finds the active constraints using the Karush–Kuhn–Tucker(KKT) conditions of the optimization problems. These active con-straints form a support set, which determines the set of thresholdsthat need to be updated. We derive update rules for PA, PA-I, and PA-II. We show that the proposed algorithms maintainthe ordering of the thresholds after every trial. We provide themistake bounds of the proposed algorithms in both ideal andgeneral settings. We also show experimentally that the proposedalgorithms successfully learn accurate classifiers using intervallabels as well as exact labels. The proposed algorithms also dowell compared to other approaches.

Abstract

In this paper, we propose an online learning algorithm called PRIL for learning ranking classifiers using interval labeled data. We show the correctness of PRIL by showing that it preserves the orderings of the thresholds in successive trials. We show that the proposed algorithm converges in finite number of steps if there exists an ideal classifier. We also give the mistake bound for the general case and provide O(pT) regret bound for the proposed algorithm. We show the effectiveness of PRIL by comparing its performance with other approaches.

Abstract

In this paper, we propose a novel mixture of expert architecture for learning polyhedral classifiers. We learn the parameters of the classifier using an expectation maximization algorithm. We derive the generalization bounds of the proposed approach. Through extensive simulation study, we show that the proposed method performs comparably to other state-of-the-art approaches.

Abstract

In this paper, we propose an approach for learning sparse reject option classifiers using double ramp loss Ldr. We use DC programming to find the risk minimizer. The algorithm solves a sequence of linear programs to learn the reject option classifier. We show that the loss Ldr is Fisher consistent. We also show that the excess risk of loss Ld is upper bounded by excess risk of Ldr. We derive the generalization error bounds for the proposed approach. We show the effectiveness of the proposed approach by experimenting it on several real world datasets. The proposed approach not only performs comparable to the state of the art, it also successfully learns sparse classifiers.