API reference

Required:

• name required

  A name for the dataset, e.g. "temperature" — used internally to keep track of things. Letters and numbers only.

• source required

  The source of the observational data, e.g. "NOAA".

• model_variable required

  The name of the model variable to compare against the observations. Not currently enforced by the function but should always be supplied.

• obs_path required

  Path to a file or directory containing the observational data. If a directory, ensure it only contains files relevant to this variable — OceanVal recursively uses all NetCDF files inside it.

Optional:

• source_info

  Additional information about the source, e.g. publication details.

• short_name

  A short name for the observational variable, e.g. "temp".

• long_name

  A long name for the variable, e.g. "sea surface temperature".

• short_title

  A short title for plots, e.g. "Nitrate Concentration".

• vertical

  Whether to carry out vertical validation. Default False (surface only).

• start / end

  First / last year of observations to use. Default to all available years.

• obs_multiplier

  Multiplier applied to observational data, e.g. for unit conversion. Default 1.

• obs_adder

  Value added to observational data, e.g. 273.15 to convert Kelvin to Celsius. Default 0.

• binning

  Spatially bin data to a [lon_bin_size, lat_bin_size] resolution in degrees. Off by default.

• recipe

  A built-in point recipe to use instead of obs_path, e.g. {"temperature": "ices"}. The observations are downloaded during matchup for the years and lon_lim/lat_lim area being matched (or the model's own extent), and name, source and the labels default to the recipe's.

Required:

• name required

  A name for the dataset, e.g. "temperature".

• source required

  The source of the observational data, e.g. "CMEMS".

• model_variable required

  The name of the model variable to compare against the observations.

• obs_path required

  Path to the directory containing the observational data files.

• climatology required

  Whether the observational data is a climatology.

Optional:

• obs_variable

  Name of the variable in the observational files. If omitted, OceanVal assumes a single variable is present.

• source_info

  Additional information about the source, e.g. publication details.

• short_name

  A short name for the observational variable, e.g. "temp".

• long_name

  A long name for the variable, e.g. "sea surface temperature".

• short_title

  A short title for plots, e.g. "Nitrate Concentration".

• vertical

  Whether to carry out vertical validation. Default False (surface only).

• start / end

  First / last year of observations to use. Default to all available years.

• obs_multiplier

  Multiplier applied to observational data, e.g. for unit conversion. Default 1.

• obs_adder

  Value added to observational data, e.g. 273.15 to convert Kelvin to Celsius. Default 0.

• recipe

  A built-in recipe dict, e.g. {"temperature": "woa23"}, providing standard metadata and file locations automatically.

• thredds

  Whether obs_path is a remote OPeNDAP/THREDDS URL rather than a local file or directory. Default False.

• file_check

  Whether to check that obs_path exists and its variables are valid. Default True.

Required:

• sim_dir required

  Path to the directory containing the model simulation output files.

• start / end required

  First / last year of the simulation to use for validation.

• thickness vertical only

  "z_level" or the exact name of the variable holding the cell thickness, e.g. "e3t" (a variable that only has it in its name, such as O3_c_e3t, is not used) — required for vertical validation. The thickness can have a time dimension or not (a static thickness such as e3t_0), and its vertical coordinate needs a positive attribute ("down" for NEMO). A thickness file must hold only the thickness variable. With "z_level", the model’s levels must be depths (positive down); output on heights (positive up) needs the cell thickness instead.

Optional:

• cores

  Number of CPU cores to use for parallel processing. Default 6.

• lon_lim / lat_lim

  Longitude / latitude limits for the validation region, e.g. [-180, 180].

• n_dirs_down

  Directory levels to search down for output files. Default 2, assuming a YYYY/MM/ structure.

• overwrite / ask

  Whether to make again any matchup this run makes that is in out_dir already, and whether to confirm first. overwrite defaults to True; False keeps and uses those there, and a list overwrites only those it picks, each entry a variable or "variable/source": ["temperature/COBE2"]. Matchups of variables or sources this run does not match up are left in out_dir either way (see validate’s previous). ask defaults to True.

• cache

  Whether to cache intermediate results. Default False.

• exclude

  Strings that should not appear in any simulation file paths.

• require

  Strings that must appear in a simulation file path for it to be included — useful when multiple simulations share a directory.

• out_dir

  Directory to save matchup files in. Defaults to the execution directory.

• point_time_res

  Time resolution for point matchups. Default ["year", "month", "day"]; set to ["month", "day"] to compare climatological output with observations.

• n_check

  Number of files checked when identifying the file naming convention.

• as_missing

  A float or [min, max] range of values to treat as missing in the model output.

• strict_names

  Whether to strictly enforce variable naming conventions. Default True.

• fvcom

  Set to True if sim_dir contains raw FVCOM output. Each file is regridded with fvcom_preprocess before matching: surface values for comparisons with vertical=False, and 0–150 m every 5 m for vertical=True. Point comparisons use a 0.05° grid; gridded comparisons use twice the resolution of the observations. thickness is not needed, lon_lim / lat_lim default to the mesh extent, and the regridded files are deleted afterwards. Default False.

• live_validation

  Build an interim validation report, HTML only, as the matchups are made: each matchup’s page is added as soon as the matchup is made, and the summary is run again, so the results can be read long before the last matchup is made. True builds it with validate’s defaults; a dict gives validate’s report options — out_dir, lon_lim, lat_lim, subregions, fixed_scale, concise, transect, depth_bins and variables (pdf, word and zip are for the full report). Matchups variables leaves out are still made, but have no page in it. It is built in oceanval_interim_report in out_dir, starting at oceanval_report/_build/html/notebooks/summary.html, and matchup waits for it to be finished before returning. Build the full report with validate afterwards, as usual. Default None.

• time_shift

  Number of days to shift every model time by before matching up, e.g. -1 for output that dates each time mean at the end of the period it covers (00:00 on 1 February for a January mean): shifted back a day, each mean falls in the period it covers. By default (None) matchup checks whether the times of time means look like the ends of their periods and, if so, asks whether to shift them back a day; with ask=False it only says so, and leaves them as they are. 0 leaves the times as they are, without checking.

• data_store

  Whether to use your data store: the folder set in the oceanval window’s Preferences where the built-in observational data downloaded (WOA23, OC-CCI, NSBC, COBE2, GLODAP and ICES) is kept for later matchups, used as the Preferences say. Your own data is never kept in it. By default (None) it is used if one is set; False leaves it out for this run.

• simdir

  Directory holding the model output. Required.

• ndown

  How many directories below simdir the output files sit, the same idea as n_dirs_down on matchup. Required.

• out

  Path of the Python file to write. Required.

• domain

  Either "global" or "nwes". Where a variable has a recipe in both regions (e.g. temperature), the one for this domain is left live and the other is commented out as an alternative; a variable with a recipe only outside this domain still gets that one. Required.

• start / end

  First / last year of the simulation to validate. Passed straight through to the generated script's matchup() call, and used to pick the WOA23 decadal period for the temperature and salinity recipes. Required.

• ask

  Whether to ask you to confirm output that looks like FVCOM, and for the model variable of anything that could not be identified (press Enter to skip). Only asks when run from an interactive terminal. With gui=True, the model variables are filled in in the window instead. Default True.

• fvcom

  Whether simdir holds raw FVCOM output. If True, the variables are read with xarray, as CDO cannot read FVCOM's, and the script calls matchup with fvcom=True. Default None: the files are checked for FVCOM's unstructured mesh and, if they have one, you are asked to confirm — or, when OceanVal cannot ask, they are assumed to be FVCOM output and it says so.

• gui

  Whether to choose the recipes in a page in your web browser before the script is written. Each observational variable gets a row with the model variable identified for it, which you can change (a pop-out under the box lists every variable in the model output with its long_name; search it, and tick several to sum them as var1+var2+var3), and tick-boxes for the gridded and point datasets available for it, ticked where the script would otherwise use them. Every ticked dataset is written out live, so a variable can be validated against several. A ticked dataset can be given years of its own and, if its observations are resolved in depth (WOA23, NSBC and ICES), be set to Vertical to validate the full water column. Cancelling, in the page or with Ctrl+C, writes nothing and returns None. create_recipes waits for the page, so pass gui=False where no one can answer it, e.g. in a batch job. Default True.

• validate

  Whether the script builds the validation report, with validate, once matchup has run. With validate=False the call is still written, commented out, to run once the matchups are made. Default True.

• exclude / require

  A string or list of strings: leave out output files whose names contain any of exclude, and use only those whose names contain each of require, as matchup does. Only file names are checked, not directories. Both are passed straight through to the script's matchup() call. Default None.

• sim_dir required

  The directory the simulation’s output is in.

• variables required

  The name to give each variable, and the model variable holding it, several summed as "a+b", e.g. {"Chlorophyll": "P1_Chl+P2_Chl", "Temperature": "votemper"}.

• climatology

  For each variable to work out the climatology of, by its name in variables: "start" and "end" years, and "surface": True, "bottom": True (the deepest value that is not missing, from nctoolkit’s bottom method with choice="value") and/or "water_column", "mean" (the vertical mean, weighted by the cell thickness) or "integral" (the depth integral). The mean of each month of the year is worked out, and the annual mean; both are mapped.

• phenology

  For each variable to work out the phenology of: "start" and "end" years. The day of the year of the seasonal peak at the surface is found in each grid cell from the variable’s daily climatology, so it needs daily output.

• n_dirs_down

  How many directories below sim_dir the output files are. Default 2.

• exclude, require

  Leave out the files whose names contain any of the words in exclude, or lack any of those in require, as matchup does.

• thickness

  The cell thickness, which the water column needs: the thickness variable in the output (e.g. "e3t"), a file with it, or "z_level" for output on fixed depth levels.

• time_shift

  Days to move the model’s times by, e.g. -1 for time means dated at the end of the period they cover, as for matchup.

• out_dir

  Where oceanval_analysis is made. Default the current directory.

• cores required

  How many cores nctoolkit may use. There is no default: you must say how many.

• view

  Open the report in a browser once it is built. Default True.

• long_term

  For each variable to work out long-term time series of, in the form of climatology: "start" and "end" years and "surface", "bottom" and/or "water_column". The mean of each year (the mean of its months’ means) over the model’s domain, each region of subregions and each of stations. Years with output missing are the mean of the months there are, and the report marks them. A variable can also have "window", the years of a rolling mean of the annual means (1, the default, is none; each year is the mean of it and the years before it), and "show": "absolute" (the default), "absolute_change" or "percentage_change", each year’s change from the first year with a value.

• seasonal

  As long_term, for each month from start to end, and so the mean seasonal cycle of each. It needs monthly output, or output more often.

• transects

  For each variable to work out transects of, a list of straight lines to work the model’s values out along, as many as you like, for that variable on its own. Each is a dict of "start" and "end" ([lon, lat]), which must run north-south (the same longitude at both ends) or east-west (the same latitude at both ends), and at least one of "climatology" (the monthly climatology at the surface), "trends" (at the surface, a heat map of the position along the transect by year, with "how" of "absolute", the annual means, "absolute_change", each annual mean minus the first, or "percentage_change"), "vertical" (the annual mean through the water column, in a section; it needs thickness, which places each cell at its depth, or "z_level") and "vertical_trends" (the same "how", as heat maps at some of the section’s depths), each a dict of its own "start" and "end" years; "trends" and "vertical_trends" can also have a "window", the years of a rolling mean of the annual means, worked out with nctoolkit’s rolling_mean after the annual means (default 1: none; each year is then the last of its window, and the first rolling mean is what the changes are from). Years without all their months are left out of the trends. Needs nctoolkit 1.3.6 or later.

• map_changes

  For each variable to map the changes of, between a historical period and one or two future periods: a dict of "historical" (a dict of "start" and "end" years), "future" (a list of one or two such dicts), "plot" ("absolute" for the historical and future maps, "absolute_changes" for those and the maps of the changes, or "changes" for the historical map and the changes), "change" ("absolute" for future minus historical, or "percentage" for that as a percentage of the historical value) and "surface", "bottom" and/or "water_column", as in climatology, e.g. {"Temperature": {"historical": {"start": 1990, "end": 1999}, "future": [{"start": 2040, "end": 2049}], "plot": "absolute_changes", "change": "absolute", "surface": True}}. Each period is mapped as the mean of its years’ annual means, and all its years must be years the output has.

• stations

  Places to work the time series out at: the name of each and its [lon, lat], e.g. {"L4": [-4.217, 50.25]}. The nearest cell of the model’s grid with values is used, which must be within two grid cells.

• subregions

  "nwes", "global" or the path of your own regions .nc file, as in validate, for the regional summaries of the time series. The domain as a whole is always done. Default None.

• lon_lim, lat_lim

  Cut a box out of the model’s domain, [minimum, maximum] each, for the domain and region means of the time series. Stations are not cut by it. All spatial means use nctoolkit’s spatial_mean. If nctoolkit warns that the model’s grid has no cell bounds, so that the cell areas cannot be worked out, the mean is worked out again after regridding the grid to a regular lon-lat grid with to_latlon, by nearest neighbour and by bilinear interpolation (the bilinear values are used, and the report says how far the two differ), and the report says that the raw grid did not have enough information to calculate grid cell areas. The means of the regions in subregions are not worked out that way: their files are lon-lat grids already, so the raw model is regridded (bilinear) straight to the region file’s grid, and each region’s mean is taken there.

• as_missing

  A value, or [minimum, maximum], to treat as missing in the model output, as in matchup. Default None.

• figure_code

  Give each figure in the report a Copy code to reproduce figure button, which copies the Python code that draws it from the analysis’s saved results, to change and run in the conda environment OceanVal ran in. Default False.

• simulations required

  Two to five dicts, each with a "name" and "sim_dir" (the directory its output is in), and, if they differ, "n_dirs_down" (default 2), "exclude", "require", "time_shift", "thickness" and "as_missing", as in analyze.

• variables required

  The name to give each variable, and the model variable holding it in each simulation, several summed as "a+b", e.g. {"Temperature": {"Control": "votemper", "New mixing": "thetao"}}. A model variable on its own is the same in every simulation. The simulations must have it in the same units.

• climatology, phenology, long_term, seasonal, transects, map_changes, stations, subregions, lon_lim, lat_lim

  As in analyze. They are the same for every simulation, and the years must be years every simulation has.

• unify

  What to do if the simulations are on different grids. None (the default) leaves each on its own. {"lon": [west, east], "lat": [south, north], "res": degrees} puts all of them on a lon-lat grid of that box and resolution, with nctoolkit’s to_latlon. {"baseline": name} regrids the others to the grid of simulation name, with nctoolkit’s regrid. In both, a cell with no value in any one simulation (judged from its first time step) is missing in all of them.

• thickness, time_shift, as_missing

  As in analyze, for the simulations that do not give their own.

• out_dir

  Where oceanval_simulation_comparison is made. Default the current directory.

• cores required

  How many cores nctoolkit may use. There is no default: you must say how many.

• figure_code

  Give each figure in the report a Copy code to reproduce figure button. Default False.

• view

  Open the report in a browser once it is built. Default True.

• lon_lim / lat_lim

  Longitude / latitude limits for the validation region.

• subregions

  The sub-regions used for the regional summaries: "global", "nwes" (Northwest European Shelf), or a path to your own regions netCDF (.nc) file. The file needs one variable per region, with 1 for grid cells in the region and missing values elsewhere. Any 0 is set to missing and ignored in regional summaries. See Using your own regions.

• concise

  Whether to generate a concise HTML summary page. Default True.

• fixed_scale

  Whether to use a fixed colour scale for the seasonal plots, capping min/max to the 2nd/98th percentile. Default False.

• data_dir / out_dir

  Where matchup data is read from / the report is written to. Default to the current directory.

• pdf

  Whether to also generate PDF downloads of the report (a per-page PDF and a combined PDF of the whole report). Default False — generating them is slow, and most users only need the HTML report.

• word

  Whether to also generate Word versions of the report — one per page, plus a combined oceanval_report.docx — with figures, tables and editable Word equations. Default False. Requires pandoc.

• zip

  Whether to also bundle the report into a zip archive (oceanval_report.zip) in the output directory, containing just the files needed to view the complete report — the HTML pages, plus the PDFs and Word file if those were generated. Default False.

• transect

  A transect to validate the gridded matchups along, as a dict of its start and end, each [lon, lat]: {"start": [-30, 0], "end": [-30, 65]}. It must run north–south (the same longitude at both ends) or east–west (the same latitude at both ends). Each gridded matchup’s page then maps the transect, shows the monthly climatology of the surface values along it, and, for matchups made through the water column (vertical=True), a section of the annual mean along it, interpolated onto 30 evenly spaced depths from the shallowest to the deepest. Needs nctoolkit 1.3.6 or later. Default None. See Validating along a transect.

• depth_bins

  The depth ranges, in metres, that point matchups made through the water column (vertical=True) are summarised in, as a list of [min, max] pairs: [[0, 20], [20, 200], [200, None]]. A depth is in a bin if it is deeper than its min and no deeper than its max (the shallowest bin also takes its min). The deepest bin can have a max of None, for everything below its min. Bins can leave gaps, whose observations are left out, but cannot overlap. The same bins are used for every point dataset, and in the summary. Default None: 0–10, 10–30, 30–60, 60–100, 100–150, 150–300, 300–600, 600–1000 and >1000 m. See Choosing the depth bins.

• variables

  The matchups to validate, if not all of them. Each entry is a variable, for all its sources, or "variable/source", for one: ["temperature", "nitrate/WOA23"]. They are as named in the matchups (oceanval_matchups), not case sensitive, and each must match at least one matchup. The summary covers only the matchups validated. Default None: every matchup.

• previous

  Whether to validate the matchups in data_dir that the last matchup run there did not make: those of earlier runs into the same directory, which matchup leaves there when it matches up other variables or sources. Default True. False validates only the matchups the last run made or kept, as it recorded them; a list validates those and the earlier matchups it picks, as variables takes them: ["chlorophyll/OCCCI"]. variables then picks among the matchups validated. See Matchups from earlier runs.

• region

  Deprecated. Use subregions instead.

• model_dict required

  Dictionary mapping a short model name to the path of its validation output.

• view

  Open the comparison report in a browser once built. Default True.

• ask

  Ask for confirmation before replacing an existing comparison directory. Default True.

• pdf

  Whether to also generate PDF downloads of the comparison report. Default False.

• word

  Whether to also generate Word versions of the comparison report — one per page, plus a combined oceanval_report.docx. Default False. Requires pandoc.

• data_dir

  Directory containing oceanval_report. Defaults to the current directory.

• pdf

  Whether to also generate PDF downloads of the report. Default False.

Required:

• out_dir required

  Output directory for the processed data — an error is raised if not supplied.

Optional:

• variables

  List of NetCDF variable names to process.

• paths

  List of file paths to the FVCOM data files.

• lon_lim / lat_lim

  Minimum and maximum longitude / latitude for regridding.

• res

  Regridding resolution in degrees. Default 0.05, suitable for point matchups.

• model_res

  Approximate native resolution of the FVCOM mesh in degrees, used to build the land/sea mask by binning node positions. If None (default), the mask is built from the mesh triangles instead: a grid cell is sea if its centre lies inside the mesh.

• missing

  Value used to represent missing data in the FVCOM output.

• z_level

  If True, variables on sigma layers are interpolated onto fixed depth levels below the free surface before regridding. Depths above the top layer centre take the top layer value, depths between the bottom layer centre and the seabed take the bottom layer value, and depths below the seabed are missing. Default False (surface only).

• levels

  Depth levels in metres (positive down) to interpolate to. Supplying levels implies z_level=True. Defaults to the WOA23 standard depths, down to the maximum model depth.