Frank Hutter researches automated design and configuration of algorithms and machine learning systems. His work asks how experimental measurements of an algorithm's performance can guide the search for better settings, replacing some manual trial and error with an explicit computational procedure.[ref]
Hutter and colleagues developed ParamILS, an algorithm configuration framework described in 2009. It searches settings of a target algorithm and limits unpromising evaluation runs to use the experimental budget more effectively.[ref]
ParamILS: An Automatic Algorithm Configuration Framework by Frank Hutter and others. Abstract, local-search configuration and adaptive evaluation-time limits
A useful example is a solver with several search settings: choosing a setting changes the procedure used on the next problem instance, rather than changing that instance's answer. Performance measured on the tuning instances should therefore be checked on separate instances before claiming a general improvement.With Ilya Loshchilov, Hutter also introduced the decoupled weight decay mechanism underlying AdamW. Their analysis showed why using coupled L2 regularization and directly shrinking parameters cease to be equivalent under general adaptive gradient scaling.[ref]
Decoupled Weight Decay Regularization by Ilya Loshchilov and Frank Hutter. Section 2, propositions 1–2 and Algorithm 2
This contribution connects algorithm design to a precise update equation: an option called weight decay can describe different mathematical operations in different optimizers.
Frank Hutter, portrait linked from his Freiburg Machine Learning Lab profile
. Articles by others on the same topic
There are currently no matching articles.