The authors made a great attempt in the reviewing process in improving the understanding of the manuscript.
However, my major critic to the article is the presentation of the formalism and its implementation, which still can be improved. It would remarkably better the understanding if the authors could provide a short introduction in every section about what they got to describe and how it is embedded in the QND formalism. The same applies when concepts are introduced: to keep the reader on track, it would be beneficial to explain in short consequences/meaning/limits of definitions before jumping right into an explanation on the results.
Regarding the presentation of the formalism and the application of results, the authors should clearly distinguish between presentation of the formalism (case-independent) and the experimental setup. An idea not involving too many changes could be to mention first (in every section) the general mechanism and then to present how it has been realised in this particular case. For instance, the control vector (subjective choice) plus uncertainties, as well as EO products and their uncertainties, and the (uncertainties of the) target quantities belong to the particular experimental setup and are not unique for all possible applications of the formalism.
Due to several factors, there is a confusion about what actually has been done. At a superficial level I understood, that the authors investigate the impact of different EO products (more precisely their uncertainties) on a potential reduction of uncertainty for particular target quantities, which might be for instance the SIV as mean over a certain area for a particular point in time, given additional uncertainty knowledge about involved quantities, such as model parameters, initial or boundary conditions.
Having gone through the manuscript, it is for instance not clear, where the information about the EO products are applied. It appears, that the authors project a propagated model state (an „initially“ unperturbed and „initially“ perturbed) onto model equivalents of the EO products and, based on the difference between both (plus an estimate of the control vector uncertainty), the authors deduce the impact of EO products (observations) on the reduction of the uncertainties of model target quantities.
In a section where the authors describe the sea ice-ocean model, the authors performed a comparison between observations and model output, identified regions, where the model output is in good agreement with the observations and in which it is not. In the discussion of the results of the formalism, this to my understanding is not taking into account (e.g. in the discussion of Fig.14).
It appears to me that the authors used notions, that sound bigger than what actually has been done, which hampers the understanding of the method. To my guessing, neither PDFs are used throughout the entire framework (beside a guess of how strong a quantity should be perturbed by a single number), neither has there been applied an inverse of a model („backward propagation“ in step 1 of the formalism). I am moreover not sure, whether an ensemble has been run to represent the PDF of the uncertainties in the target quantities. Applied mechanisms and tools should be called by their name, and clarified accordingly.
To emphasize the confusion about the methodology that occurs while reading the manuscript, the following is a summary on what I understood or was confused about in each subsection. I hope, that this leads to an understanding, where the confusions arise, and thus to a changed, more clarified description of the formalism:
Section 2.1 QND formalism
According to Fig.1 including its explanations, the QND formalism propagates uncertainties in the observations to those in the target quantities (which is any information extracted from the simulations) taking into account model uncertainties via a user defined control vector.
It consists of two steps: 1) a model is backward propagated via inverse modeling. It is not clear, why an inverse model has to be applied here (and which concept is taken to derive the inverse of a model). The input for this step is uncertainties in the observations and prior control uncertainties. The used model in this step is a mapping from control variables to (model based) observational equivalents. The output is posterior control uncertainties C(x). 2) based on these posterior uncertainties of the control vector and on uncertainties in „the model“, a (possibly another) model is used for forward propagation to derive posterior uncertainties of target quantities. Eq. (1) - (3) are the formula representation of Fig.1.
All quantities are presented by PDFs, i.e. functions (a mapping from an uncountable set to [0,1])! It is unclear, how these PDFs are represented, particularly as input to and output of computer models. Gaussianity is presumed apparently for each. It should be clarified how the statistics are represented/used? In case that several samples are drawn from the distribution, how representative is this set of draws? It appears to me, that the PDF is only used to determine one single perturbation value that is added to an input value of the formalism. If this is the case, the above formalism is incorrect.
From Fig. 2 I deduce, that there is an ensemble of model runs involved in the QND formalism. However, this has not been mentioned at any point, neither how the ensemble has been created.
If the authors run an ensemble, how is it designed and what is the ensemble size? Is a sufficient ensemble spread guaranteed? Does the ensemble size ensure statistically significant results?
Moreover, basic descriptions in the graph of Fig. 2 and its caption are lacking.
– For instance, the notion of the x-axis is not clear. What is the meaning of „time = 2“?
– Why is there a bar of width ~1.75 y-axis units at „time = 2“ and of ~2 y-axis units at „time =4“ labelled with sigma(d1) and sigma(d2)?
– Where and how are the uncertainties in the observations fed in?
– How is the first step of the QND formalism (see Fig. 1) represented in this Figure?
– Does the graph in the end illustrate the development of two ensemble runs in the second step of the QND formalism (Fig.1) – or (more or less) 100 single experiments? If the latter, how does it fit to the claim, that the authors treat PDFs?
– What does the y-axis represent and what are the units? If only the development of the ensemble spread is of relevance, then state this. I would consider to remove the ticks on the y-axis, as they apparently have no meaning.
– Lines are depicted in lighter and in darker colors. I guess, that these lines represent a development of a quantitiy in each of the ensemble member runs from a QND formalism, where the first step has been skipped (light) and where it has not been skipped (dark). But this is not mentioned and should be described in the caption.
- It is not clear how the scheme in Fig. 1 can be brought into agreement with this figure. Both are supposed to be a schematic representation of the same formalism.
2.2 Target quanities
This section is supposed to discuss the output of the entire QND formalism, which in the present case are the uncertainties in SIV and SNV in the aforementioned 3 regions at time May 28, 2015.
In this section it is said, that the QND formalism performs a model run of a particular model from April 1 to April 30 and the QND formalism assimilates data (EO product uncertainties). I guess, that the assimilation phase is related to step 1 of the formalism, while the forcast step might be step 2 of the QND formalism. Accordingly I could make the assumption, that the backward propagation in Fig.1 uses a 4dVar method, but having read the entire manuscript I haven‘t found any indication for this. On the other hand, the graph in Fig. 4 indicates a forward model propagation; combining this with Fig. 2 it appears, that an ensemble based assimilation method has been used (which? Smoother? Filter?), but no indication for this has been found in the remaining of the manuscript.
It is also unclear how the EO product uncertainties are used to provide posterior control uncertainties. Maybe, I missed the according explanation in the manuscript. Comparing with Fig. 1 it is not clear to me, how the PDFs of the control vector uncertainties are provided as input (comparing Fig.1 and Fig.4).
The authors should mention how Fig.4 links to the QND formalism (if there is one to draw), and state which methods have been used following which goal (addressing the above questions).
If my guessings are correct, in the assimilation period the involved model is M, which projects from control variables to observational equivalents. In Fig.12 (rhs) on p.19 I find an indication on the structure of M (without that it is explicitly mentioned, that this is a representation of M), on how a circulation model might be involved (time integration and thus backward propagation) having a control vector as input. However, it is still unclear, how and why the model is backward propagated and how the observation information is fed into the model.
2.3 Sea ice-ocean model
This section introduces the applied sea ice-ocean model and assesses the model‘s sea ice state. It is said, that by using the MPIOM the operator M (step 1) simulates observation equivalents and in step 2 (operator N) the target quantities are computed using the MPIOM. At this stage it is unclear how this is done. It would be helpful to clarify the QND formalism and how step 1 and step 2 is performed, as well as the structure of M and N itself, in an earlier Section. No particular knowledge about the IO-CM is needed for that. This would also help to prevent the following confusions/questions:
The EO products used in this manuscript are SIT, SND, RFB, SIFB and LFB. Given the authors statement (MPIOM is used in step 1 within operator M) and recalling that model M applies a mapping from the model state onto individual data streams (eq.(1)), I presume at this stage of the manuscript, that equivalents of these quantities are also available in MPIOM. Having mentioned this, I guess, that the eq. (6) – (10) are implemented in MPIOM. Having read through the entire manuscript, I learned something different (Fig.12) – and needed to change my vague imagination of the formalism at page 19 (one example).
In the case that MPIOM is involved in step 1 of the formalism, the authors should mention how the inverse modeling is performed. It would be easier to follow the development of the ideas in the manuscript if the used notions (e.g. how M is structured) would have been explained beforehand.
2.4 Control vector
The control vector (whose uncertainties are provided as input in PDF form in the first and in the second step of the QND formalism) consists of initial and surface boundary conditions, observation operators etc.. Due to resource restrictions, the author partition the considered region and treat projections onto these partitions. The explain how standard deviations for the single parameters have been derived. To keep it concise I would remove mentioning the Jacobian and just say, that to every entry of the control vector there is a PDF assigned. A proper description of the entries of the control vector is missing. For instance: As there are boundary and initial values involved, the authors should precicely state how they are constructed, e.g. at which time and from which initialisation these are derived.
2.5 Data sets and observation operators
In this section the EO products are introduced. The left hand side of Figure 12 gives a sketch on how the uncertainties of the EO products are derived (which are input into step 1 of the QND formalism, which should be mentioned in the beginning of this section or at least somewhere in the text before Figure 12 is explained).
Sticking to that Figure, in this section the authors also explain, how they derive EO model equivalents, i.e. M(x), representations of RFB, SIFB, SIT – provided the model outputs. (The authors could explicity state that the entire right hand side of that Figure is an illustration of the operator M and indicate, that both, the MPIOM as well as the projections (eq. (8)-(10)) are found here.)
These model outputs (M(x)) are derived from the input control vector and a model run (is this the backward propagation, the inverse modelling?).
As the observation operator is being introduced here (and not in Section 2.2): Could the authors elaborate in this section on the model run (How the backward intergration takes place, in which mode the model is run (assimilation?), in which time period? Under which initial conditions and settings? (I.e. indicate that the model is initialised with the control vector as described in sectino 2.4) It would be easier if the authors would mention the configuration here and would make a link to the Fig.1 and related equations.
3. Target and observational Jacobians
In contrast to what has been introduced in Section 2.1, the authors compute the Jacobian not by representing the PDF but as follows:
To derive the Jacobian, the authors 1) determine a reference model equivalent of the EO products (M(x)) by performing what is being sketched on the rhs of Fig.12. If this is correct, the authors should mention this in the manuscript. 2) for each entry of the control vector, the authors do the same as in 1) with a perturbation of this entry by eps_i, resulting in M(x+p_i) with p_i=(0,…,0,eps_i,0,…,0). Is this the representation in PDF form?
How is the magnitude of eps_i chosen? Is the model run in forecast mode? From which time and in which period?
(Given, that the authors clearly prescribe the setting of the control vector in Section 2.4, the initial and boundary conditions and their uncertainties should be clear as well. And if the authors describe the operator M in Section 2.5, the above questions should be answered already.)
The prior uncertainty of the control vector entry, sigma_i, as given in Table 1, is then multiplied with the vector: (M(x_p)-M(x))/(eps_i). It appears, that in contrast to what the authors state in Section 2.1, sigma_i is not a PDF, but a single value. The authors should state, why it is sufficient to perform two runs to identify the PDF of the control vector uncertaintiies.
Comparing with Fig. 1: What is the posterior uncertainty of the control vector? Do the authors take the maximum over the entire column to estimate the posterior uncertainty of the control vector element?
I do not see, where uncertainties of the EO products have been applied so far. As far as I understand the mechanism by now is, that it provides an estimate on how changing the model variables and parameters lead to changes in the model, which are then projected onto model equivalents of the EO products. How do the authors derive posterior control vector uncertainties with this mechanism?
The authors mention, that they likewise compute the target Jacobians N‘. (N is a mapping from model and control vector uncertainties to target quanities. ) How does that fit to the different notions of forward and backward integration in step 2 and in step 1?
5. Sea ice and snow volume uncertainty
- when the authors describe how to derive sigma_mod,relative, do the authors initially perturb SIT or before the EO product equivalents are determined (i.e. in the model run?)?
- p.25 l.20: How are the posterior uncertainties determined?
Detailed remarks:
- p.1 l.10: replace ; by :
- abstract: The authors might consider to replace „increase of performance“ by „decrease of uncertainty“, as this is what application of information on different EO products effect to my understanding.
- p.2 l.10: model‘s initial state
- p.2 l.11f: can help to reduce […] to reduce
- p.2 l.14: more directly relate
- p.2 l.18: It is only one order of magnitude to my understanding.
- p.2 l.13: with other variables
- articles are missing at several places in the manuscript. Please check, e.g. p.3 l.3, l.8 etc
- p.3 l.4: on uncertainties of forecasts of snow...
- p.3 last paragraph in Introduction: correct the numbering of the Secions (e.g. Section 4 is missing)
- p.4: explain/sketch backward integration and purpose, forward propagation, representation of the PDF in the formalism
- p.4 l.14: Instead of „Here we refer to M as model“ I would write „It consists of a (sea ice-ocean) model run and a projection from the model onto the observational space.“ And later in that line „Let us first consider M to be linear, ...“ As well, later in the section mention similarly that/how the (sea ice-ocean) model is involved in N.
- p.4f: When mentioning M‘ and N‘ indicate that you detail them in Sections X and Y.
- p.4 eq (1): add „ ,“ and at end of eq (2) add „ .“
- p.5 l.19: simulation of […] is incorrect
- p.6 Fig2: needs lots of improvements, see in text above. Add „.“ at the end of the caption. In the caption: „the model M“ instead of „the model“.
- p.6 l.10f: It is not clear how the model (M? MPIOM?) is constrained.
- p.8: Fig.4: unclear why here and relation to Fig.1, see also above.
- p.8 Fig.4: Comparing with Fig.1 I presume, that by „SIV“ and „SNV“ the authors mean „uncertainties of SIV and SNV“.
-How do the illustrations of Fig.1, Fig.2 and Fig.4 fit together? It appears to me that these are three different descriptions of the same formalism.
- p.9 l.29: Jacobian
- p.9 l.29: I would remove „(introduced in Section 2.1)“, if the authors do not further explain in that section.
- p.9 l.33: The ocean initially is ...
- p.11 l.2: „In our target regions the misfit remains very small“ plus p.12 .13ff: In the study the authors analyse sensitivities for the entire Arctic (Fig14). Thus, possible of impacts larger misfits in remote regions should be discussed when the results are presented.
- p.11 Fig 6: Graphs (b) and (e) are not discussed. Either discuss or remove.
- p.12 l.7: Could the authors provide magnitudes for the bias?
- p.12 l.15: more than one member
- p.13 l.3 „As described in ...“ The discussion on the Jacobian is confusing here. I would remove the sentence.
- p.13 l.7f „In each of ...“: This is misleading, for instance for the reference run the authors do not add perturbation, of I understood correctly. I would mention something along the line: „Each of the entries is assigned a perturbation. For each forcing field we add a constant perturbation ...“
- p.14 l.1: I presume: The reason of choosing the diagonal form is, that the proper cross-correlations are unknown and the next natural choice is to neglect any cross-correlations. Furthermore, the effect might be, that only one control vector entry is perturbed in the definition of the Jacobian. One might add a statement to the text. It could be an idea to shift the discussion of the cross-correlation of the uncertainties to the introduction of the Jacobian.
-p.14 l3ff: When perturbing an initial state variable or a boundary variable: Is the statistics you derive space-dependent, and if so, is the perturbence you add to this initial/boundary vector inthe definition of the Jacobian also varying in space? Or is it a mean over the entire space?
- p.14 Fig.8, and p.16 Fig.9 caption: „a) The modelled...“
- Section 2.5: To simplify reading, the authors could also write „observation operator M“, at least the first time mentioned in this section.
- p.15, Name of entries 7, 29 and 30 should have the same font as the other entries.
- p.15: Meaning of entries 32-45: these should be regional averages of the mentioned values and not values themselves?
- p.15: Caption: l.2: f,i,p: Write in order of occurrence in the table.
- p.15: Caption: l.1: replace ; by , | l.4: magnitude: Does that mean the „mean“? If so, then also replace „value“ by „mean“ in the header of the table.
- p.15: Meaning of entries 19ff: coeff.
- p.19 caption, l.2: after rhs: I suppose to add: „(Model M)“
-p.19 caption l.3: I suggest to add „(from MPIOM)“ after „model variables“. In the mansucript there were mentioned too many „models“ to easily get confused, what the authors mean here.
- p.19: caption: clarify abbreviation „CS“ used in the graph.
- p.19ff: The first time you mention „retrieval chain“ add a note that this is used to determine the uncertainties in the EO products and refer to Fig.1, where these data find entry into the formalism and how.
- p.20, l.1f: „recall that...“ spatio-temporal coverage needed as well as the uncertainty ranges: I am not aware that the former has been mentioned before. I would remove „recall that“.
- p.20, l.5: Add „M‘ “ after „observational Jacobian“.
- p.20 l.11: remove „(Archimedes ...(1970))“.
- p.21 l.3: \rho_i and \rho_s
- p.21 Fig. 13: add to the titles „(m)“ and remove the label of the colorbar. It is redundant. The title of the lower left panel should be changed to „sea ice thickness random error (m)“. According changes should be made in the lower rigth panel. Moreover, replace „sea-ice“ by „sea ice“ in the titles of the upper two panels. Increase the fontsize of minimum and maximum values.
- p.21 l.9: The authors might add something along the line „derived via the method depicted on the left-hand side of Fig.12“ after „April 2015“.
- p.22 Caption Table 2: Add „(column 8)“ at the end.
- p.22 .15: Add „EO“: „For a given EO product...“
- p.22 l.21: It is unclear how eps_i has been chosen and whether the PDF has been represented via ensemble runs. Explain it in the Section about control vectors and refer to there.
- p.23 l.1: I would dedicate the Jacobian a separate line (extra equation).
- p.23 l.1: I presume that there is a tight relation between eps_i, sigma_i and the difference between M(x_p) and M(x). Could the authors elaborate a bit on that after introducing the Jacobian to prepare the reader for the analysis?
- p.23 l.4: ith component of the control vector, x_i,
- p,23 l.9: „The SIT sensitivy“ and p.24 l.5: Add information about the time (May 28?).
- p.23 l.26: refer to Fig.15 instead.
- p.23 l.32f: „The impact of ...“: Indicate which bar you are refering to.
- How is Eq. (1) implemented? How do the authors derive at C(x), C(x)^-1? Could the authors identify this, maybe in Section 3 or later?
- p.24 l.9: overlaps
- p.24 l.14: Consider to replace „high enough concentration“ by „sufficiently high SIC (sea ice concentrations)“
- p.24 l.17f: „The largest...“: This is not clear to me. If I look at the graph in Fig. 17, it appears that albsnm is strongest. For snow volume, the statement appearrs correct.
- Section 3: Before the authors start to explain the derivation of the Jabobian M‘, they could refer to the former section (2.5) and mention, that they detailed the construction of M (rhs of Fig. 12 plus equations (8)-(10) I presume) of the first step of the QND formalism there. It is not mentioned that clearly and leads to confusion, when the reader does not know how M is defined.
- Sectioning: It is not clear why the Jacobians are given a separate Section, while they are tightly linked to Section 2.4.
- Section 3: The authors might consider to separate the construction of the two Jacobians, i.e. using two subsections: one referring to step one of the algorithm and one to the second step.
- Section 4: Lots of information missing, if this section is kept. E.g. the restriction to the target regions and the focus on one day after May 28.
- Section 5: The section should be started with a short introduction/overview and not straight with the discussion of a special case.
- p.25 l.14: remove „and listed in the last but one row“.
- p.25 l.24: add „(yellow bar)“ after „SIT“
- p.25 l.25f: “only difference consists in the target Jacobians...“: How do you draw that conclusion? In the upcoming you explain that there is a difference in the target Jacobians, but there is no explanation, why there is no other difference elsewhere.
- p.26 Table 3, caption: Except for the first line, the authors only show posterior uncertainties. This is not written in the caption (where it is stated that for all columsn 4-9 there are prior and posterior uncertainties, which I do not see). Mention, what the prior uncertainty is (sigma(y0)?).
- p.29-32: enlargen font size. This cannot be read. Titles of the graphs are missing in Fig. 15 and Fig.17.
- p.30 and p.32: The plot is not easy to be read. The authors should use a grouping, which enables to read which bar belongs to which quantity.
- p.33: Improve labeling of the graph. Instead of using machine representation of the variable names, use the abbreviations used in the text, e.g. SIT instead of sit, etc.
- p.34 l.11: sea ice-ocean
- p.34 l.26 „cross-correlation for each of the products“ - as the autocorrelations are not set to zero.
- p.34 l.30: How can the mean values be that low?
- 4. Experimental Setup: This section carries very little information. And the most information about the particular setting is already formulated in the former sections, i.e. assimilation window, control vector, forecast window, initialisation. As well, the referred Table 3 contains results. I suggest to shift the content to the next section entirely.
- Discussion section:
In Section 2.3, the authors identified regions, where the differences between model and observations are not small. Recalling, that MPIOM runs are involved in both steps of the formalism, possible impacts of these inconsistencies need to be discussed when presenting the results, in particular, when remote effects are discussed. |
- Printer-friendly version
- Supplement
