Optimizing penalized and constrained loss functions with applications to large-scale internet media selection

The use of penalized loss functions in optimization procedures is prevalent throughout the statistical literature. However, penalized loss function methodology is less familiar outside the areas of mathematics and statistics, and even within statistics loss functions are typically studied on a problem-by-problem basis. In this thesis I present general loss function approaches for both penalized as well as penalized and constrained loss functions and apply these approaches to a problem outside statistics, namely the optimization of a large-scale Internet advertising campaign. ❧ Recent work in penalized loss functions has focused largely on linear regression methods with excellent results. Methods like the LASSO (Tibshirani 1996a) have demonstrated that penalization can yield gains in both prediction accuracy and algorithmic efficiency. In cases where linearity is not appropriate, however, the best approaches can be unclear. To this end I propose a generalized methodology, applicable to many loss function for which the first and second derivatives can be readily computed. This leverages the efficiency benefits seen in penalized linear regression approaches and applies them to problems of general nonlinear objectives. ❧ Unfortunately, as useful as penalization is in optimization, many problems require another step: constraints. Problems like portfolio optimization or monotone curve estimation have very natural constraints which need to be taken into account directly during the optimization. Because of this, I propose a further advance in the penalized methodology: PAC, or penalized and constrained regression. PAC is capable of incorporating constraints directly into the optimization, allowing for better parameter estimation when compared to unconstrained methods. Again, the use of penalization allows for efficient algorithms applicable to a wide variety of problems. ❧ One such problem comes from the field of marketing: the optimization of large-scale Internet advertising campaigns. I apply both approaches to the real-world problem encountered by firms and advertising agencies when attempting to select a set of websites to use for advertising and then allocating an advertising budget over those websites. Due to the sheer number of websites available for advertising, all of which represent a unique advertising opportunity with varying cost, traffic, etc., these selection and allocation questions can present daunting challenges to anyone wanting to advertise online. At the same time, online advertising is becoming increasingly necessary to businesses of all industries. To address this, I show how the efficient penalized approaches developed in the general case can be specifically applied to this problem. I carry out two case studies using real data from comScore, Inc., which exemplify campaign considerations faced by companies today. Further, I show how these methods significantly improve on the methods currently available. ❧ I also discuss future directions for the research in this thesis, particularly in the real-world application of Internet media campaigns. Though this research could be extended in a wide variety of directions, I focus on furthering the marketing application to extend to the most current advertising procedure, bidding on ad impressions in real time, as well as extending the penalized and constrained approach to include time-varying objective functions (for problems in which the behavior of one time period affects the optimization of a second time period).