Predicting and characterizing the state of the ocean is needed for various scientific, industrial, social, management, and recreational activities. Despite the tremendous progress in ocean modeling and simulation capabilities, the ocean models still suffer from different sources of uncertainties. To obtain accurate ocean state predictions, data assimilation (DA) is widely used to constrain the ocean model outputs with available observations. Ensemble Kalman filtering (EnKF) is a sequential DA approach that represents the distribution of the system state through an ensemble of ocean state samples. Different factors may limit the performance of an EnKF in realistic ocean applications, particularly the use of small ensembles and poorly known model error statistics, and also to less extent the strongly nonlinear variations and abrupt regime changes, and unsatisfied underlying assumptions such as the commonly used white observation noise assumption. The objective of this PhD thesis is to develop, implement and test efficient ensemble filtering schemes to enhance the performances of EnKFs in such challenging settings.

We resort to the one-step-ahead (OSA) smoothing formulation of the Bayesian filtering problem to introduce EnKFs involving a new update step with future observations (smoothing) between two successive analyses, thereby conditioning the ensemble sampling with more information. We show that this approach enhances the EnKFs performances by providing improved ensemble background statistics, and showcase its performance with realistic ocean DA and forecasting applications, namely a storm surge EnKF forecasting system and the Red Sea ensemble DA and forecasting system. We then derive new EnKF-based schemes accounting for time-correlated observation errors for efficient DA into the class of large dimensional DA problems where observation errors statistics are correlated in time, and further propose a new approach for online estimation of the parameters of the observation error time-correlations model concurrently with the state. We also exploit the OSA-smoothing formulation to propose a new joint EnKF with OSA-smoothing which mitigates for the reported inconsistencies in the joint EnKF update for efficient DA into one-way-coupled systems.