read.odf: Read an ODF file

Description

ODF (Ocean Data Format) is a format developed at the Bedford Institute of Oceanography and also used at other Canadian Department of Fisheries and Oceans (DFO) facilities (see references 1 and 2). It can hold various types of time-series data, which includes a variety of instrument types. Thus, read.odf() is used by read.ctd.odf for CTD data, etc. As of mid-2018, read.odf() is still in development, with features being added as a project with DFO makes available more files.

Usage

read.odf(
  file,
  columns = NULL,
  header = "list",
  exclude = NULL,
  debug = getOption("oceDebug")
)

Arguments

file

the file containing the data.

columns

An optional list that can be used to convert unrecognized data names to resultant variable names. For example, columns=list(salinity=list(name="salt", unit=list(unit=expression(), scale="PSS-78")) states that a short-name of "salt" represents salinity, and that the unit is as indicated. This is passed to cnvName2oceName() or ODFNames2oceNames(), as appropriate, and takes precedence over the lookup table in that function.

header

An indication of whether, or how, to store the entire ODF file header in the metadata slot of the returned object. There are three choices for the header argument. (1) If it is NULL, then the ODF header is not stored in the metadata slot (although some of its contents are). (2) If it is "character", the header is stored within the metadata as a vector named header, comprising a character string for each line of the header within the ODF file. (3) If it is "list", then the metadata slot of the returned object will contain a list named header that has lists as its entries. (The sub-lists are in the form of key-value pairs.) The naming of list entries is patterned on that in the ODF header, except that unduplicateNames() is used to transform repeated names by adding numerical suffices. Note: on June 6, 2019, the default value of header was changed from NULL to "list"; in addition, the resultant list was made to contain every single item in the ODF header, with unduplicateNames() being used to append integers to distinguish between repeated names in the ODF format.

exclude

either a character value holding a regular expression that is used with grep() to remove lines from the header before processing, or NULL (the default), meaning not to exclude any such lines. The purpose of this argument is to solve problems with some files, which can have thousands of lines that indicate details that are may be of little value in processing. For example, some files have thousands of lines that would be excluded by using exclude="PROCESS='Nulled the .* value" in the function call.

debug

an integer specifying whether debugging information is to be printed during the processing. This is a general parameter that is used by many oce functions. Generally, setting debug=0 turns off the printing, while higher values suggest that more information be printed. If one function calls another, it usually reduces the value of debug first, so that a user can often obtain deeper debugging by specifying higher debug values.

Value

An '>oce object.

Metadata conventions

Some metadata items may be specific to certain instruments, and certain research groups. It can be important for analysts to be aware of the conventions used in datasets that are under study. For example, as of June 2018, adp objects created at the Bedford Institute of Oceanography may have a metadata item named depthOffBottom (called DEPTH_OFF_BOTTOM in ODF files), which is not typically present in ctd files. This item illustrates the renaming convention, from the CAMEL_CASE used in ODF files to the snakeCase used in oce. Bearing this conversion in mind, users should not find it difficult to understand the meaning of items that read.odf() stores within the metadata slot. Users should bear in mind that the whole ODF header is saved as a list by calling the function with header="list", after which e.g. str(rval[["header"]]) or View(rval[["header"]]) can be used to isolate any information of interest (but bear in mind that suffices are used to disambiguate sibling items of identical name in the ODF header).

Caution

ODF files do not store information on the temperature or salinity scale, and read.odf() assumes these to be ITS-90 and PSS-78, respectively. These scales may be incorrect for old data files. Note that the temperature scale can be converted from old scales using T90fromT68() and T90fromT48(), although the change will be in a fraction of a millidegree, which probably exceeds reasonable confidence in old data.

Details

Note that some elements of the metadata are particular to ODF objects, e.g. depthMin, depthMax and sounding, which are inferred from ODF items named MIN_DEPTH, MAX_DEPTH and SOUNDING, respectively. In addition, the more common metadata item waterDepth, which is used in ctd objects to refer to the total water depth, is set to sounding if that is finite, or to maxDepth otherwise.

The function ODFNames2oceNames() is used to translate data names from the ODF file to standard oce names, and handles conversion for a few non-standard units. The documentation of ODFNames2oceNames() should be consulted for more details.

References

For sources that describe the ODF format, see the documentation for the '>odf.

Examples

Run this code

# NOT RUN {
library(oce)
#
# 1. Read a CTD cast made on the Scotian Shelf. Note that the file's metadata
# states that conductivity is in S/m, but it is really conductivity ratio,
# so we must alter the unit before converting to a CTD object. Note that
# read.odf() on this data file produces a warning suggesting that the user
# repair the unit, using the method outlined here.
odf <- read.odf(system.file("extdata", "CTD_BCD2014666_008_1_DN.ODF.gz", package="oce"))
ctd <- as.ctd(odf) ## so we can e.g. extract potential temperature
ctd[["conductivityUnit"]] <- list(unit=expression(), scale="")
#
# 2. Make a CTD, and plot (with span to show NS)
plot(ctd, span=500)
#
# 3. Highlight bad data on TS diagram. (Note that the eos
# is specified, because we will extract practical-salinity and
# UNESCO-defined potential temperatures for the added points.)
plotTS(ctd, type="o", eos="unesco") # use a line to show loops
bad <- ctd[["QCFlag"]]!=0
points(ctd[['salinity']][bad],ctd[['theta']][bad],col='red',pch=20)

# }

Run the code above in your browser using DataLab