Parameter estimation in nonlinear mixed effect models based on ordinary differential equations: An optimal control approach

We present a parameter estimation method for nonlinear mixed effectmodels based on ordinary differential equations (NLME-ODEs). The methodpresented here aims at regularizing the estimation problem in presenceof model misspecifications, practical identifiability issues and unknowninitial conditions. For doing so, we define our estimator as the minimizerof a cost function which incorporates a possible gap between the assumedmodel at the population level and the specific individual dynamic.The cost function computation leads to formulate and solve optimalcontrol problems at the subject level. This control theory approachallows to bypass the need to know or estimate initial conditions foreach subject and it regularizes the estimation problem in presenceof poorly identifiable parameters. Comparing to maximum likelihood,we show on simulation examples that our method improves estimationaccuracy in possibly partially observed systems with unknown initialconditions or poorly identifiable parameters with or without modelerror. We conclude this work with a real application on antibody concentrationdata after vaccination against Ebola virus coming from phase 1 trials.We use the estimated model discrepancy at the subject level to analyzethe presence of model misspecification.