Data Formats

Intermagnet MQTT format

This section describes how to format topics and payloads when sending data over MQTT.

Intermagnet MQTT topics

Topics in MQTT are strings used to identify and route messages. Topics allow publishers to identify messages and subscribers to choose which types of message to receive. Topics for the Intermagnet MQTT service have been designed to allow the contents of the message to be easily identified. Observatories publishing data through the Intermagnet MQTT service must use the topic:

impf/<iaga-code>/<cadence>/<publication-level>/<elements-recorded>

Where:

  • “impf” stands for Intermagnet MQTT Payload Format and is included in the topic to allow alternative topic and payload formats to be used on the same MQTT brokers in the future.

  • “iaga-code” is the IAGA registered code of the observatory sending data.

  • “cadence” is the sample period for the data as an ISO8601 duration (for data with sampling rate of 1Hz or below) or the sample rate with the suffix “hz” (for data with sampling rate of 1Hz or above). Valid cadences are:

    • “pt1s” or “1hz” for 1-second data.

    • “pt1m” for 1-minute data.

  • “publication-level” is a number indicating the level of processing applied to the data:

    • 1 = The data is unprocessed and as recorded at the observatory with no changes made.

    • 2 = Some edits have been made such as gap filling and spike removal and a preliminary baseline added.

    • 3 = The data is at the level required for production of an initial bulletin or for quasi-definitive publication.

    • 4 = The data has been finalised and no further changes are intended.

  • “elements-recorded” lists the geomagnetic elements that will be in the message. All 4 elements must be specified, even if the payload includes fewer elements. “elements-recorded” must be one of:

    • “XYZS” - XYZ vector orientation with an independent Scalar F

    • “HDZS” - HDZ vector orientation with an independent Scalar F

    • “DIFS” - DIF vector orientation with an independent Scalar F

Note that topics are case-sensitive. All topic values for the Intermagnet MQTT service must be in lower case.

### Examples ###

The topic for Eskdalemuir observatory’s 1-minute “reported” HDZ data (straight from the observatory with no processing) would be:

impf/esk/pt1m/1/hdzs

The topic for Lerwick observatory’s 1-second quasi-definitive XYZS data would be either of the following:

impf/ler/pt1s/3/xyzs
impf/ler/1hz/3/xyzs

Intermagnet MQTT payload format

Intermagnet MQTT messages are formatted in JSON. A [JSON schema](ImagMQTTSchema.json) describes the valid contents of an MQTT message. Messages must consist of JSON documents conforming to the schema and no other content.

The JSON schema consists of three sections:

  1. Mandatory metadata

  2. Mandatory geomagnetic field data

  3. Optional extra metadata

The mandatory metadata consists of a single field which must be present in every JSON data message:

  1. “startDate”: The date and time of the first data sample, in ISO8601 format. The string is truncated to the appropriate precision for the data cadence (ie. all fields except milli-seconds for 1-second data, all fields except seconds and milli-seconds for 1-minute data).

The topic used to publish data to the MQTT broker includes 4 further pieces of metadata (IAGA code, cadence, publication level and geomagnetic orientation) that are also used to describe the data being transmitted.

The mandatory geomagnetic field data consists of anything between 1 and 4 arrays corresponding to the elements recorded (as described in the topic). Individual elements of the geomagnetic field may be sent together in a message or in separate messages, so that, for example, data from a vector instrument and a separate scalar instrument do not need to be combined into a single message to be sent. An entirely missing scalar element does not need to be sent at all. The arrays sent in a single message must all be the same length and contain only numbers or the value “null”, which is used to indicate a missing sample. Valid arrays are:

  • “geomagneticFieldX”: Magnetic field vector strength in nT, x component, geographic coordinates.

  • “geomagneticFieldY”: Magnetic field vector strength in nT, y component, geographic coordinates.

  • “geomagneticFieldZ”: Magnetic field vector strength in nT, z component, geographic coordinates.

  • “geomagneticFieldH”: Magnetic field vector strength in nT, in the horizontal plane along the magnetic meridian.

  • “geomagneticFieldD”: Angle between the magnetic vector and true north, in minutes of arc, positive east.

  • “geomagneticFieldI”: Angle between the magnetic vector and the horizontal, in minutes of arc, positive below the horizontal.

  • “geomagneticFieldF”: Geomagnetic field strength in nT, calculated from and consistent with XYZ or HDZ field elements.

  • “geomagneticFieldS”: Geomagnetic field strength in nT, measured by an independent scalar instrument.

Only arrays corresponding with the “elements-recorded” in the message topic may be used.

The optional extra metadata may be completely missing, or any part of parts may be included. The purpose of this metadata is to allow the data provider to supply metadata values that the Intermagnet data portal will use when creating IMF, IAGA-2002 and ImagCDF data files for users. The data portal stores a metadata file for each day and publication level of data. Thus it is only neccessary for a data provider to send one message per day / publication level, containing the optional metadata that they require to be distributed with their data. Subsquent messages for the same day / publication level do not require any optional metadata. If data provider’s follow this advice, it is best to provide the optional metadata along with the first data of the day, to ensure that data and metadata are always consistent.

If the optional metadata is not supplied, default values will be used by the portal (from static metadata that the Edinburgh GIN holds)).

The following optional metadata fields are available - the names in brackets after each of the fields describes which data distribution format(s) the metadata field is used in:

  • “ginCode”: (IMF)

  • “decbas”: (IMF)

  • “latitude”: (IMF, IAGA-2002, ImagCDF)

  • “longitude”: (IMF, IAGA-2002, ImagCDF)

  • “elevation”: (IAGA-2002, ImagCDF)

  • “institute”: (IAGA-2002, ImagCDF - called “Source of data” in IAGA-2002)

  • “name”: (IAGA-2002, ImagCDF - called “ObservatoryName” in ImagCDF)

  • “sensorOrientation”: (IAGA-2002, ImagCDF - called “VectorSensOrient in CDF)

  • “digitalSampling”: (IAGA-2002)

  • “dataIntervalType”: (IAGA-2002)

  • “publicationDate”: (IAGA-2002, ImagCDF)

  • “standardLevel”: (ImagCDF)

  • “standardName”: (ImagCDF)

  • “standardVersion”: (ImagCDF)

  • “partialStandDesc”: (ImagCDF)

  • “source”: (ImagCDF)

  • “termsOfUse”: (ImagCDF)

  • “uniqueIdentifier”: (ImagCDF)

  • “parentIdentifiers”: (ImagCDF)

  • “referenceLinks”: (ImagCDF)

  • “comments”: (IAGA-2002)

For details of each of these fields, see the [JSON schema](ImagMQTTSchema.json).

Data sets created for this schema can be checked at the JSON Schema verfier: https://json-schema.hyperjump.io/. Reference material for JSON Schema is here: https://json-schema.org/learn/getting-started-step-by-step. The Schema for Intermagnet MQTT JSON is [here](ImagMQTTSchema.json).

Example IMPF data

The first example is a minimum viable data file for 1-minute data. Three 3-component samples are presented along with mandatory metadata:

{
    "startDate": "2023-01-01T00:00",
    "geomagneticFieldX": [ 17595.02, null, 17594.99 ],
    "geomagneticFieldY": [ -329.19, -329.18, -329.21 ],
    "geomagneticFieldZ": [ 46702.70, 46703.01, 46703.24 ]
}

An expanded example shows how to add “comment” metadata, to set the “comments” section that will be included with the data when it is distributed from the Intermagnet web site in IAGA-2002 format. This example also shows how the startDate should be modified for 1-second data:

{
    "startDate": "2023-01-01T00:00:00",
    "comments": [ "This data file was created using INTERMAGNET data",
                  "from the Edinburgh GIN. These data were acquired",
                  "from an INTERMAGNET quasi-def data file.",
                  "Final data will be available on the INTERMAGNET DVD",
                  "Go to www.intermagnet.org for details on obtaining this product.",
                  "CONDITIONS OF USE: The conditions of use for data provided",
                  "through INTERMAGNET and acknowledgement templates can be found",
                  "at www.intermagnet.org" ],
    "geomagneticFieldS": [ 49000.00, 49000.21, 49000.34 ]
}

JSON Schema

The following schema can be used to verify that a JSON document conforms to the IMPF specification:

{
    "$schema": "https://json-schema.org/draft/2020-12/schema",

    "$id": "https://intermagnet.org/mqtt_exchange.schema.json",
    "title": "Intermagnet MQTT Exchange Format",
    "description": "A format for exchanging geomagnetic data in Intermagnet using MQTT",
    "type": "object",

    "properties": {
        "startDate": {
            "type": "string",
            "format": "datetime",
            "description": "Date and time of the first data sample, in ISO8601 format, to appropriate precision for the data cadence"
        },

        "ginCode": {
            "type": "string",
            "enum": [ "edi", "gol", "kyo", "ott", "par" ],
            "description": "Optional: Code for the GIN that this observatory reports to"
        },
        "decbas": {
            "type": "integer",
            "minimum": -10800,
            "maximum": 21600,
            "description": "Optional: DECBAS (declination baseline) field from IMF format"
        },
        "latitude": {
            "type": "number",
            "minimum": -90.0,
            "maximum": 90.0,
            "description": "Optional: Observatory location: latitude"
        },
        "longitude": {
            "type": "number",
            "minimum": -180.0,
            "maximum": 360.0,
            "description": "Optional: Observatory location: east longitude"
        },
        "elevation": {
            "type": "number",
            "minimum": -10000.0,
            "maximum": 10000.0,
            "description": "Optional: Observatory location: elevation in metres"
        },
        "institute": {
            "type": "string",
            "description": "Optional: Institute that the observatory belongs to"
        },
        "name": {
            "type": "string",
            "description": "Optional: Name of the observatory"
        },
        "sensorOrientation": {
            "type": "string",
            "description": "Optional: Orientation of the vector magnetic field sensor"
        },
        "digitalSampling": {
            "type": "string",
            "description": "Optional: Rate (in seconds) of the data sampling of the vector magnetic field sensor"
        },
        "dataIntervalType": {
            "type": "string",
            "description": "Optional: Mean or instantaneous time interval of the data"
        },
        "publicationDate": {
            "type": "string",
            "format": "date",
            "description": "Optional: Date of publication of the data, in ISO8601 format, date only (no time)"
        },
        "standardLevel": {
            "type": "string",
            "enum": [ "None", "Partial", "Full" ],
            "description": "Optional: Describe whether the data conforms to a standard"
        },
        "standardName": {
            "type": "string",
            "enum": [ "INTERMAGNET_1-Second", "INTERMAGNET_1-Minute", "INTERMAGNET_1-Minute_QD" ],
            "description": "Optional: The name of the standard this data conforms to (if any)"
        },
        "standardVersion": {
            "type": "string",
            "description": "Optional: The version of the standard this data conforms to (if any)"
        },
        "partialStandDesc": {
            "type": "string",
            "description": "Optional: A comma separated list describing the parts of the standard this data conforms to (if any), for use when the standard level is partial"
        },
        "source": {
            "type": "string",
            "enum": [ "Institute", "Intermagnet", "WDC" ],
            "description": "Optional: Where the data came from"
        },
        "termsOfUse": {
            "type": "string",
            "description": "Optional: The terms of use for the data"
        },
        "uniqueIdentifier": {
            "type": "string",
            "description": "Optional: A string that can be used to uniquely identify this data, e.g. a DOI"
        },
        "parentIdentifiers": {
            "type": "array",
            "items": { "type": "string" },
            "description": "Optional: The unique identifer(s) of the parent data set"
        },
        "referenceLinks": {
            "type": "array",
            "items": { "type": "string" },
            "description": "Optional: URLs pointing to information about the data, one URL per element"
        },
        "comments": {
            "type": "array",
            "items": { "type": "string" },
            "description": "Optional: Use these to record important information not contained in the defined fields"
        },

        "geomagneticFieldX": {
            "type": "array",
            "items": { "type": ["number", "null"] },
            "minimum": -99999.0,
            "maximum": 99999.0,
            "description": "Magnetic field vector strength in nT, x component, geographic coordinates"
        },
        "geomagneticFieldY": {
            "type": "array",
            "items": { "type": ["number", "null"] },
            "minimum": -99999.0,
            "maximum": 99999.0,
            "description": "Magnetic field vector strength in nT, y component, geographic coordinates"
        },
        "geomagneticFieldZ": {
            "type": "array",
            "items": { "type": ["number", "null"] },
            "minimum": -99999.0,
            "maximum": 99999.0,
            "description": "Magnetic field vector strength in nT, z component, geographic coordinates"
        },
        "geomagneticFieldH": {
            "type": "array",
            "items": { "type": ["number", "null"] },
            "minimum": -99999.0,
            "maximum": 99999.0,
            "description": "Magnetic field vector strength in nT, in the horizontal plane along the magnetic meridian"
        },
        "geomagneticFieldD": {
            "type": "array",
            "items": { "type": ["number", "null"] },
            "minimum": -180.0,
            "maximum": 99999.0,
            "description": "Angle between the magnetic vector and true north, in degrees of arc, positive east"
        },
        "geomagneticFieldI": {
            "type": "array",
            "items": { "type": ["number", "null"] },
            "minimum": -180.0,
            "maximum": 99999.0,
            "description": "Angle between the magnetic vector and the horizontal, in degrees of arc, positive below the horizontal"
        },
        "geomagneticFieldF": {
            "type": "array",
            "items": { "type": ["number", "null"] },
            "minimum": 0.0,
            "maximum": 99999.0,
            "description": "Geomagnetic field strength in nT, calculated from and consistent with XYZ or HDZ field elements"
        },
        "geomagneticFieldS": {
            "type": "array",
            "items": { "type": ["number", "null"] },
            "minimum": 0.0,
            "maximum": 99999.0,
            "description": "Geomagnetic field strength in nT, measured by an independent scalar instrument"
        }
    },

    "allOf": [
        { "required": [ "startDate" ] },
        { "oneOf":
            [
                { "allOf": [ { "required": [ "geomagneticFieldX", "geomagneticFieldY", "geomagneticFieldZ", "geomagneticFieldS" ] },
                            { "not": { "required": [ "geomagneticFieldH" ] } },
                            { "not": { "required": [ "geomagneticFieldD" ] } },
                            { "not": { "required": [ "geomagneticFieldI" ] } }
                ] },
                { "allOf": [ { "required": [ "geomagneticFieldX", "geomagneticFieldY", "geomagneticFieldZ" ] },
                            { "not": { "required": [ "geomagneticFieldH" ] } },
                            { "not": { "required": [ "geomagneticFieldD" ] } },
                            { "not": { "required": [ "geomagneticFieldI" ] } },
                            { "not": { "required": [ "geomagneticFieldS" ] } }
                ] },
                { "allOf": [ { "required": [ "geomagneticFieldH", "geomagneticFieldD", "geomagneticFieldZ", "geomagneticFieldS" ] },
                            { "not": { "required": [ "geomagneticFieldX" ] } },
                            { "not": { "required": [ "geomagneticFieldY" ] } },
                            { "not": { "required": [ "geomagneticFieldI" ] } }
                ] },
                { "allOf": [ { "required": [ "geomagneticFieldH", "geomagneticFieldD", "geomagneticFieldZ" ] },
                            { "not": { "required": [ "geomagneticFieldX" ] } },
                            { "not": { "required": [ "geomagneticFieldY" ] } },
                            { "not": { "required": [ "geomagneticFieldI" ] } },
                            { "not": { "required": [ "geomagneticFieldS" ] } }
                ] },
                { "allOf": [ { "required": [ "geomagneticFieldD", "geomagneticFieldI", "geomagneticFieldF", "geomagneticFieldS" ] },
                            { "not": { "required": [ "geomagneticFieldX" ] } },
                            { "not": { "required": [ "geomagneticFieldY" ] } },
                            { "not": { "required": [ "geomagneticFieldH" ] } }
                ] },
                { "allOf": [ { "required": [ "geomagneticFieldD", "geomagneticFieldI", "geomagneticFieldF" ] },
                            { "not": { "required": [ "geomagneticFieldX" ] } },
                            { "not": { "required": [ "geomagneticFieldY" ] } },
                            { "not": { "required": [ "geomagneticFieldH" ] } },
                            { "not": { "required": [ "geomagneticFieldS" ] } }
                ] },
                { "allOf": [ { "required": [ "geomagneticFieldS" ] },
                            { "not": { "required": [ "geomagneticFieldX" ] } },
                            { "not": { "required": [ "geomagneticFieldY" ] } },
                            { "not": { "required": [ "geomagneticFieldZ" ] } },
                            { "not": { "required": [ "geomagneticFieldH" ] } },
                            { "not": { "required": [ "geomagneticFieldD" ] } },
                            { "not": { "required": [ "geomagneticFieldI" ] } }
                ] }
            ]
        }
    ],
    "additionalProperties": false
}

IAGA2002 INTERMAGNET Exchange Format (Spreadsheet compatible)

This ASCII Exchange Format, adopted in August 2001 and revised in December 2001, July 2003 and in August 2011, is intended as a data exchange format for geomagnetic data (samples and means) from observatories and variometer stations at time intervals from millisecond up to and including monthly means. The format comprises:

  • Twelve (12) mandatory file header records.

  • Unlimited optional comment records.

  • One (1) mandatory data header record.

  • A series of data records.

  • Every record is 70 characters long plus the machine-dependent carriage return/line feed.

Pad records with spaces if needed. Data records report exactly 4 magnetic field elements (DHIF, DHZF, XYZF, or DHIG, DHZG, XYZG where G = DF).

Use missing data values (8’s or 9’s) if fewer than 4 elements are available.

The 12 Mandatory File Header Records

Mandatory header and optional comment records begin with a space character in column 1 and end with the vertical bar | (ASCII 124) in column 70. Content labels begin in column 2 and descriptions begin in column 25.

  • This format is designated IAGA-2002.

  • Source of Data is the name of the institute responsible for collecting the data.

  • Please spell the entire station name; do not use abbreviations. Capitalize the first letter.

  • The IAGA Code is the official IAGA 3-letter station code. It should be in capital letters and correspond to the IAGA list of magnetic observatories.

  • Location of the station is reported to the one thousandth degree in geodetic latitude (positive north) from -90 to 90 degrees and in geodetic longitude (positive east) -180 to 180 or 0 to 360 degrees. Geographic location is reported using WGS-84 datum

  • Report elevation in meters above mean sea level.

  • Reported refers to the magnetic field elements contained in the data record, in the order recorded in data record. Valid values are DHIF, DHZF, and XYZF (or DHIG, DHZG,and XYZG where G = DF). Use E/V instead of D/I for declination/inclination given in intensity units (ONLY if data type is variation).

  • Sensor Orientation is the physical orientation of the observing instruments, i.e. XYZF, HDZ.

  • Digital Sampling is the rate (in seconds) of the data sampling of the magnetic field sensor (instrument) or the digitizing interval for analogue data.

  • Data interval type is the mean or instantaneous time interval of the data. Common values include 1-minute (00:30-01:29), 1-minute (00:00-00:59), 1-hour (00-59), 1-day (00-23) and 1-month (01-31); the last day could also be 30, 29, or 28. There are many possible intervals, including a fraction of a second (instant value), averages by 1-second (501-1500), 1- second (0-1000), 10 second, or 2.5 minute. Define the type of mean and how values are centered in the comment section.

  • Data type is provisional (P), definitive(D), quasi-definitive (Q), or variation (V).

  • Publication date (optional) is the date when the data is published. This was added in June 2015. The inclusion of this line is (#13) optional but desirable for the modern data. The old data files cannot be modified (e.g. Publication date 2014-10-20).

The Optional Comment Header Records

Use these records to record important information concerning the data that is not contained in the defined fields. Types of information may include the type of means and how the mean values are centered, important gaps in the data record, or explanations of missing values (9 filled element field).

  • Every record begins with a space character in column 1 and # (hash or number sign) in column 2.

  • The end of each record is indicated with a vertical bar | (ASCII 124) in column 70.

  • For transmission of incomplete day files include two additional optional comment headers stating the start time and duration in seconds. These records must have the form:

    • Start Time hh:mm:ss

    • Duration-in-seconds sssss

  • If the data file contains absolute values (Data Type provisional or definitive), include the formula for computing the missing (non-reported) magnetic elements. For example, if an observatory reports XYZF, the comments should contain the formula for computing HDI (see sample data header and data record for 1-minute values).

  • If the data file contains variometer values without considering base line values (Data Type variation), formula and constants are to be given to convert the recorded magnetic elements into others or to convert angular units into nT or vice versa. The possible conversions are:

    • E = D * Ha * arc(1’)

    • V = I * Fa * arc(1’)

    • X = H * cos(Da) – E* sin(Da)

    • Y = H * sin(Da) + E* cos(Da)

    • Z = F * sin(Ia) + V * cos(Ia)

    • F = Z * sin(Ia) + H * cos(Ia)

    • H = X * cos(Da) + Y * sin(Da) = F * cos(Ia) – V * sin(Ia)

    • E = Y * cos(Da) – X * sin(Da)

    • V = Z * cos(Ia) – H * sin(Ia)

    X, Y, Z, H, F, E, V in nT; D, I in minutes of arc

    arc(1’) = 2.9089E-04 = 0.00029089

    Ha, Fa, Da, Ia : approximate values of H, F, D, I (eg the most recent annual mean value or a value from the IGRF)

    The formula and constants are to be given in additional optional comment header lines (see sample data header and data record for 1-second values).

The Mandatory Data Header Record

The mandatory data-header record contains column headers useful for multi-station analysis. Elements, both type and order, are indicated in the file-header field Reported.

  • The date and time headers are DATE, TIME, and DOY (day of year).

  • The magnetic element headers comprise 4 letters: 3 letters for the observatory IAGA code and 1 letter for the magnetic element reported. Valid values are H, D (or E), I (or V), X, Y, Z, F (or G). The column headers are space delimited.

  • The end of each record is indicated with a vertical bar | (ASCII 124) in column 70.

The Data Records

The data records contain the date, time, and magnetic field elements reported. Report data to the least significant digit. Indicate missing data with 99999 to the corresponding accuracy for a given component(e.g., 99999.00). If an element is not observed, please record (to the corresponding accuracy for a given component) 88888 in that field. The format for field elements is 4(1X, F9.2). Each record is exactly 70 characters long plus the machine dependent carriage return / line feed.

  • DATE is the calendar date in ISO YYYY-MM-DD format (4-digit year, month as 01-12, day as 01-31).

  • TIME is in ISO hh:mm:ss.sss format (hour as 0-24, minute as 0-59, second as 0-59 Note: if the hour is 24, the minute and second must be 0). Values beyond the time interval of the means being reported must be zero-filled. i.e. 14:01:00.000 for 1 minute data for the 14th hour, first minute. Describe the method of average, i.e., centered to the hour for hourly means, in the comments area.

  • DOY is the day of the year, from 1 - 365 (or 366 for leap years).

  • D and I are reported in angular units of minutes of arc to the precision of the instruments. F, H, X, Y, Z, E, G, and V are reported in nanotesla and a fraction of nT.

Sample of IAGA-2002 Format July 2003 Revision

Sample Data Header and Data Record for 1-minute Values

 Format                 IAGA-2002                                    |
 Source of Data         Danish Meteorological Institute              |
 Station Name           Narsarsuaq                                   |
 IAGA Code              NAQ                                          |
 Geodetic Latitude      61.160                                       |
 Geodetic Longitude     314.560                                      |
 Elevation              4                                            |
 Reported               XYZF                                         |
 Sensor Orientation     DIF                                          |
 Digital Sampling       0.01 seconds                                 |
 Data Interval Type     Filtered 1-minute (00:30 - 01:29)            |
 Data Type              Definitive                                   |
 # This area is where the data source or distributor can include     |
 # any additional information needed for proper use of data. For     |
 # example, the observers name and contact, notes on a change of     |
 # instrumentation, reasons for missing data values, definition of   |
 # observed values, geomagnetic location of the observatory, etc.    |
 # This area should also contain the formula for computing the non-  |
 # reported elements and components of the INTERMAGNET binary        |
 # format which do not fit elsewhere. These include:                 |
 # D-conversion:                                                     |
 # = H/3438*10000.                                                   |
 # which is word 8 in the INTERMAGNET binary format and is used      |
 # to convert variations of D in minutes of arc <-> nT. Please       |
 # note that all of the header records and comment records begin     |
 # with a space in column 1, end with a | (ASCII 124), and are       |
 # padded with spaces - never with tabs.                             |
 # H = squareroot(X*X + Y*Y), cos D = X/H, sin I = Z/F               |
DATE       TIME         DOY     NAQX      NAQY      NAQZ      NAQF   |
2001-03-13 00:00:00.000 072     10800.11 -6100.23   53381.51  54801.12
2001-03-13 00:01:00.000 072     10800.31 -6100.20   53381.51  54801.12
2001-03-13 00:02:00.000 072     10801.11 -6101.23   99999.00  54801.12
2001-03-13 00:03:00.000 072     10803.12 -6100.23   99999.00  54801.12

Sample Data Header and Data Record for Hourly Values

The header record for Data Interval Type might read:
Data Interval Type 1-hour (00 - 59). Note filler for non-reported element.
DATE       TIME         DOY     NAQX      NAQY      NAQZ      NAQF   |
2001-03-13 00:00:00.000 072     10800.11 -6100.23   53381.51  88888.00
2001-03-13 01:00:00.000 072     10800.31 -6100.20   53381.51  88888.00
2001-03-13 02:00:00.000 072     10801.11 -6101.23   53381.50  88888.00
2001-03-13 03:00:00.000 072     10803.12 -6100.23   99999.00  88888.00

Sample Data Header and Data Record for Monthly Values

The header record for Data Interval Type might read:
Data Interval Type 1-month (01 - 31). Note filler for non-reported element.
DATE       TIME         DOY     NAQX      NAQY      NAQZ      NAQF   |
2001-01-15 00:00:00.000 015     10800.11 -6100.23   53381.51  88888.00
2001-02-14 00:00:00.000 045     10800.31 -6100.20   53381.51  88888.00
2001-03-15 00:00:00.000 074     10801.11 -6101.23   53381.50  88888.00
2001-04-15 00:00:00.000 105     10803.12 -6100.23   99999.00  88888.00

Sample Data Header and Data Record for 1-second Values

 Format                 IAGA-2002                                    |
 Source of Data         Danish Meteorological Institute              |
 Station Name           Narsarsuaq                                   |
 IAGA CODE              NAQ                                          |
 Geodetic Latitude      61.160                                       |
 Geodetic Longitude     314.560                                      |
 Elevation              4                                            |
 Reported               HEZF                                         |
 Sensor Orientation     HEZF                                         |
 Digital Sampling       1 seconds                                    |
 Data Interval Type     1-second instantaneous                       |
 Data Type              Variation                                    |
 # This area should contain additional information needed            |
 # in order to transform the reported elements. For the data in this |
 # sample information about the declination at the observatory is    |
 # needed in order to tranform the variation data from HEZ to the    |
 # widely used XYZ orientation. Also information about the value of  |
 # horizontal field H can be useful to convert E-variations in nT to |
 # variations in minutes of arcs.                                    |
 # E = D * Ha * 0.00029089                                           |
 # Ha = 17123.45                                                     |
 # X = H * cos(Da) B E* sin(Da)                                      |
 # Y = H * sin(Da) + E* cos(Da)                                      |
 # Da = 312.89 minutes of arc                                        |
DATE       TIME         DOY     NAQH      NAQE      NAQZ      NAQF   |
2001-03-13 00:00:00.000 072       800.11   -100.23    381.51  54801.12
2001-03-13 00:00:01.000 072       800.31   -100.20    381.51  54802.32
2001-03-13 00:00:02.000 072       801.11   -101.23  99999.00  54803.22
2001-03-13 00:00:03.000 072       803.12   -100.23  99999.00  54803.43

Sample Data Header and Data Record for 5-Millisecond Values

The header record for Data Interval Type might read:
Data Interval Type 5-millisecond (instantaneous values). Note filler for missing element.
DATE       TIME         DOY     NAQX      NAQY      NAQZ      NAQF   |
2001-03-13 00:00:00.000 072     10800.11 -6100.23   53381.51  99999.00
2001-03-13 00:00:00.005 072     10800.31 -6100.20   53381.51  99999.00
2001-03-13 00:00:00.010 072     10801.11 -6101.23   53381.50  54801.12
2001-03-13 00:00:00.015 072     10803.12 -6100.23   99999.00  54801.12

INTERMAGNET Baseline Format : IBF

IBFV2.00 (2009 and after)

This file format is to be used to provide baselines for use in examining equipment and environmental quality (mainly thermal stability of the variometers). This format makes room for all published components, including a baseline to scalar data. After a one-line header, the first section contains the observed baseline values on the days when they are measured. Consequently the number of entries will depend upon the schedule for absolute measurements at the observatory. The second section contains adopted baseline values, representing each day of the year. The third section for comments is mandatory. This comment section must include a brief description of the baseline adoption method.

COMP_HHHHH_FFFFF_IDC_YEARCrLf
DDD_aaaaaa.aa_bbbbbb.bb_zzzzzz.zz_ssssss.ssCrLf
. . .
. . .
. . .
DDD_aaaaaa.aa_bbbbbb.bb_zzzzzz.zz_ssssss.ssCrLf
*CrLf
001_AAAAAA.AA_BBBBBB.BB_ZZZZZZ.ZZ_SSSSSS.SS_DDDD.DD_mCrLf
002_AAAAAA.AA_BBBBBB.BB_ZZZZZZ.ZZ_SSSSSS.SS_DDDD.DD_mCrLf
003_AAAAAA.AA_BBBBBB.BB_ZZZZZZ.ZZ_SSSSSS.SS_DDDD.DD_mCrLf
. . .
. . .
. . .
366_AAAAAA.AA_BBBBBB.BB_ZZZZZZ.ZZ_SSSSSS.SS_DDDD.DD_mCrLf
*CrLf
Comments:CrLf
.....................................................CrLf
.....................................................CrLf

Component values are coded as signed float numbers, right justified with format (1X,F9.2). DeltaF values (denoted as DDDD.DD) are coded as signed float numbers, right justified with format (1X,F7.2). The field widths must be maintained, either through zero-filling or space-filling in order to have records of fixed length: every line is 43 characters long in section one, and 53 characters long in section two excluding terminator CrLf or Lf. Comment lines should not exceed 53 characters long.

Field

Description

COMP

Order of components XYZF, DIF_, HDZF, UVZF (see Note 1).

HHHHH

Annual mean value of H component in nT.

FFFFF

Annual mean value of F component in nT.

IDC

Official IAGA three-letter station code in capital letters.

YEAR

4-digit Year: eg, 2009.

DDD

Day of the year.

aaaaaa.aa

Observed baseline of X, D, H or U in nT or minutes (absolute - variometer)

bbbbbb.bb

Observed baseline of Y, I, D or V in nT or minutes (absolute - variometer)

zzzzzz.zz

Observed baseline of Z or F in nT (absolute - variometer)

ssssss.ss

Observed baseline of Scalar F in nT (absolute - scalar, See Note 5)

AAAAAA.AA

Adopted baseline value of X, D, H or U in nT or minutes.

BBBBBB.BB

Adopted baseline value of Y, I, D or V in nT or minutes.

ZZZZZZ.ZZ

Adopted baseline value of Z, or F in nT.

SSSSSS.SS

Adopted baseline value of Scalar F in nT. (absolute - scalar, See Note 5)

DDDD.DD

Representative value of Delta F for this day (see Note 2).

m

Discontinuity marker (valid values c or d - see Note 3).

*

Section separator.

_

Space character.

CrLf

Indicates Carriage Return and Line Feed.

File name convention is IAGYEAR.BLV e.g. BFE2008.BLV where:

  • IAG = 3-letter observatory IAGA code

  • YEAR = 4-digit year

Note

  1. The codes XYZF, DIF_, HDZF, UVZF are the only supported codes of the components which must be listed in specified order within sections one and two.

  2. Delta F is defined as:

    Delta F = F(v) - F(s).

    Where F(v) represents the total field value calculated from the main observatory instrument (‘vector F’) and F(s) represents the total field from an independent instrument (‘scalar F’). Both F(v) and F(s) must be corrected to the location in the observatory where geomagnetic absolute observations are made.

  3. Discontinuity marker: c indicates that the baselines for all components form a continuous series with the previous day, d indicates a discontinuity in the baseline of one or more components (i.e. step) between the current day and the preceding one. This is to allow the baselines to be plotted at an appropriate scale. A discontinuity is defined as a known operational change such as an instrument re-alignment.

  4. Missing values must be replaced by 99999.00 for D, H, X, Y, Z, F and by 999.00 for Delta F. Components that are not observed, which may include Scalar F (SSSSSS.SS) and Delta F (DDDD.DD), must be coded 88888.00 and 888.00 respectively.

  5. The observed and adopted F baselines represent the DIFFERENCE in the F component between the absolute position and the continuously recording scalar instrument. Pier corrections are not applied to the data before computing the baselines.

IBFV1.20 (2008 and before)

This format is to be used to provide baselines for use in examining equipment performance and for inclusion on the INTERMAGNET DVD. The first section contains the observed baseline values on those days on which they were measured. Consequently the number of entries will depend upon the schedule for absolute measurements at that observatory. The second section contains adopted baseline values representing each day of the year. A comment section is also provided.

COMP_HHHHH_IDC_YEARCrLf
DDD_AAAAAAA_BBBBBBB_ZZZZZZZ CrLf.
. . . . .
. . . . .
. . . . .
DDD_AAAAAAA_BBBBBBB_ZZZZZZZ CrLf.
*
001_AAAAAAA_BBBBBBB_ZZZZZZZ_FFFFF CrLf.
002_AAAAAAA_BBBBBBB_ZZZZZZZ_FFFFF CrLf.
003_AAAAAAA_BBBBBBB_ZZZZZZZ_FFFFF CrLf.
...
366_AAAAAAA_BBBBBBB_ZZZZZZZ_FFFFF CrLf.
*
Comments:

Component values are coded as signed integers, right-justified with a field width of 7. Total field (Delta F) values are coded as signed integers, right-justified with a field width of 5. The field widths must be maintained, either through zero-filling or space-filling. The ‘+’ sign for positive values is optional.

Field

Description

COMP

Order of components HDZF, XYZF, DIF, UVZF

HHHHH

Annual mean value of H component in nT.

IDC

IAGA three-letter observatory ID code eg: BOU for Boulder, OTT for Ottawa, LER for Lerwick, etc.

YEAR

4-digit Year: for example, 1991.

DDD

Day of the year.

AAAAAAA

Signed value of H, D, U or X in 0.1 nT

BBBBBBB

Signed value of D, I, V or Y in 0.1 nT or 0.1 min of arc for D

ZZZZZZZ

Signed value of Z or F in 0.1 nT

FFFFF

Signed value of Delta F, the difference between calculated and observed value of F (by a proton magnetometer) in 0.1 nT

*

Section separator.

_

Space character.

CrLf

Indicates Carriage Return and Line Feed.

Missing values must be replaced by 999999 for D, H, X, Y, Z and by 9999 for F.

File name convention is IAGYR.BLV where:

  • IAG = 3-letter observatory IAGA code

  • YR = 2-digit year

IMAGCDF INTERMAGNET Exchange Format

This document describes how NASA’s Common Data Format ( CDF - http://cdf.gsfc.nasa.gov/ ) is used to store geomagnetic data. This format is called ImagCDF.

Design Details and CDF Concepts

General Design Details

Geomagnetic data is held in CDF variables, one variable per geomagnetic element. Additional variables hold time stamp data. A single time stamp variable may be used if all time series data uses the same sample rate. Alternatively one time stamp variable may be for the vector magnetometer data, a second variable for the scalar magnetometer data, if data from these two instruments are recorded at different sample rates.

Each variable (data or time stamp) has one or more records, an individual record holding a single sample value. The index numbers of the records in the data variables correspond with the index numbers in the appropriate time stamp variable’s records. A typical file might contain 6 variables: 3 geomagnetic elements (such as HDZ or XYZ); one scalar element (F); and two timing variables, one for the vector data, the other for the scalar. The format does not mandate that these variables are present – there may be fewer (for example only scalar data) or more (for example additional temperature data).

Metadata is held in CDF attributes in two ways: global attribute entries concern all the data in a file; variable attributes have entries that concern a single variable (e.g. the geomagnetic data ‘H’ variable). An entry holds an individual item of metadata. An attribute name (whether global or variable) must be unique, so when an attribute needs to be used more than once (e.g. the element type for a geomagnetic variable must be used once for each geomagnetic element), then multiple entries are created in a single attribute. Global attributes in ImagCDF will have only one entry (with two exceptions). Variable attributes may have an entry for each of the relevant variables in the file. For variable attributes, the multiple entries are indexed using the variable’s numeric identifier, so that the metadata ‘belongs’ to the variable. For example, in an ImagCDF file holding HDZF data, the attribute Units will have four entries (in this order): “nT”; “Degrees of arc”; “nT”; “nT”.

Data Types Used for Variables and Attributes

Real Numbers

ImagCDF uses double precision CDF_DOUBLE (8-byte) floating point numbers (to the IEEE 754 standard) for all numeric values. These numbers provide about 14.5 (decimal) digits of precision - http://flint.cs.yale.edu/cs422/doc/art-of-asm/pdf/CH14.PDF. In order to use floating point numbers successfully, the dynamic range of the quantity being represented (ratio of smallest to largest value) must be smaller than 14.5 digits. For geomagnetic field strengths, assume that the ratio of smallest difference to largest value that we need to represent in geomagnetic field data is 0.1pT in 80,000nT. This equates to 1 part in 80,000 x 1,000 x 10, or 1 part in 800,000,000, or 9 digits of precision, so is within the dynamic range available.

Dates / Times

All date / time values in ImagCDF are held as CDF TT2000 dates, which are based on 8-byte integers. TT2000 uses an epoch (midday on 1st January 2000) to store dates and times, has a precision of 1 nanosecond which gives a range in excess of ±280 years from the epoch date. The TT2000 type can correctly handle leap seconds.

Strings

Text data is held using CDF_CHAR variables in ImagCDF.

Compression

The CDF library allows the user to specify whether all, part or none of a CDF file is compressed at the time it is written. Once this choice has been made, the CDF library handles compression and decompression of the data in the file automatically (reading and any further writing to the file will decompress or compress as required). Choosing whether or not to compress a file is simply a matter of specifying which (if any) compression method to use when the file is created. ImagCDF allows any of the compressions provided by CDF.

Significant compression is achieved using the CDF compression option. A day file of four element minute data can occupy under 15Kb.

Where to Obtain the CDF Software

Before you can use any of NASA or INTERMAGNET’s tools for working with ImagCDF, you will need to download and install the CDF software from NASA: http://cdf.gsfc.nasa.gov/html/sw_and_docs.html . Software that has been written to work with CDF is likely to need the libraries that are installed. For details of other software that may be useful see section Tools to Look at CDF Data.

ImagCDF Data

All variables holding geomagnetic data have the following features:

  • Units used must be nT for geomagnetic field values, degrees for angles or celsius for temperatures.

  • Lengths of time series are arbitrary (e.g. a file may be used to store an entire day of data or a small fragment of a day down to a single sample).

Geomagnetic data is held in variables called GeomagneticField <E> where <E> represents the code for the geomagnetic element recorded - see section Valid Codes for Elements Recorded (V1.2) for a list of valid codes (in version 1.2). The variable has 0 dimensions, each consecutive record holding individual consecutive data samples in CDF double data type, starting at record 1. Missing data values are represented by a data sample that contains the same number as is present in the FILLVAL metadata attribute. The FILLVAL must exceed any valid geomagnetic field strength or angle. Typically this would be 99999.0. The variable attribute FIELDNAM must be set to “Geomagnetic Field Element <E>”.

An ImagCDF file must include a set of geomagnetic field variables that describe the vector field in a recognised orientation (such as ‘HDZ’, ‘XYZ’ or ‘DIF). All vector variables must have the same number of records. An ImagCDF file may also include an additional field element from an independent scalar instrument, with an element code of ‘G’ or ‘S’.

Temperature data is optional (unless otherwise stated in the data standard that the data conforms to, for example, temperature is mandatory for data that conforms to the INTERMAGNET 1-second data standard). The first temperature variable is called Temperature1, then Temperature2 and so on. The FIELDNAM attribute describes, in free text, for each temperature variable, the location at which the temperature was recorded.

Time stamps for the data are held in separate variables. Time stamps must represent a regular time series with no missing values in the series.

If all data in the CDF file (including non-geomagnetic data) has the same sample rate, a single time stamp variable called DataTimes should be used. If all geomagnetic data has the same sample rate (but other data has different sample rates), a time stamp variable called GeomagneticTimes should be used for the geomagnetic data, and time stamps for temperature data should be called TemperatureTimes (if all temperature data has the same sample rate) or Temperature1Times, Temperature2Times, … (for temperature data with different sample rates). If the geomagnetic vector and scalar data have different sample rates, time stamp variables GeomagneticVectorTimes and GeomagneticScalarTimes should be used (time stamp variables for temperature data should follow the rules already described).

All time stamp variables have 0 dimensions and must have the same number of records as the data variables that they apply to. Each record in a time stamp variable holds a CDF TT2000 epoch time. Time stamps must always refer to the start of each sample period (e.g. for minute data, the seconds and milliseconds will always be set to zero).

Whatever time stamp variables are used, every data variable must have an entry in the DEPEND_0 variable attribute containing the name of the time stamp variable that is to be used with that data variable.

Names for time stamp variables are:

Situation

Names

The same time stamps can be used for all data in the file (i.e. there is a single time stamp variable in the file)

DataTimes

The same time stamps can be used for all geomagnetic data, a single different time stamp is used for all temperature data

GeomagneticTimes, TemperatureTimes

Different time stamps for vector, scalar and temperature data

GeomagneticVectorTimes, GeomagneticScalarTimes, Temperature1Times, Temperature2Times, …

Additional variables and metadata may be carried in an ImagCDF (e.g. it may be convenient to include meteorological data in the same file). The format of these variables and metadata is left to the user to define. The CDF system means that these variables can be included without causing problems to software reading ImagCDF files. Software writing ImagCDF files should preserve additional variables and attributes read from an ImagCDF file.

ImagCDF Version 1.2

The descriptions in this section apply to version 1.2 of ImagCDF.

ImagCDF Global Attributes (V1.2)

The following attributes apply to all the data in an ImagCDF file. The “Entries” column shows whether the attribute has:

  • A single mandatory entry (number of entries is exactly 1)

  • A single optional entry (number of entries may be 0 or 1)

  • Multiple mandatory entries (number of entries is between 1 and N1)

  • Optional mandatory entries (number of entries is between 1 and N2)

Superscript numbers following the attribute name show:

  1. That the attribute is a recommended attribute for use with NASA’s CDF tools

  2. That the attribute is part of the ISTP/IACG guidelines - http://spdf.gsfc.nasa.gov/istp_guide/gattributes.html

Attributes that Describe the Data Format (V1.2)

These are ‘constant’ values that will be the same for all ImagCDF files. They allow ‘generic’ CDF programs to understand and process the data correctly.

Attribute Name

Type

Entries

Description

FormatDescription

String

1

Always set to “INTERMAGNET CDF Format”

FormatVersion

String

1

Set to the current version of the format – “1.2”

Title

String

1

Always set to “Geomagnetic time series data”
Attributes that Uniquely Identify the Data (V1.2)

The attributes in this section are sufficient, along with the start date and duration of the time series, to uniquely identify a piece of geomagnetic data.

Attribute Name

Type

Entries

Description

IagaCode

String

1

The IAGA code for the observatory

ElementsRecorded

String

1

A string consisting of single character codes, each describing one of the geomagnetic field elements that is recorded in this data file. This might typically be a three or four digit code such as HDZ, XYZG or DIFG. The codes in this attribute determine the names of the data variables (see the section on geomagnetic data). Valid codes are defined in section Valid Codes for Elements Recorded (V1.2)

PublicationLevel

String

1

Choose one of the following codes to describe the level that the data has been processed to:

  • 1: The data is unprocessed and as recorded at the observatory with no changes made.

  • 2: Some edits have been made such as gap filling and spike removal and a preliminary baseline added.

  • 3: The data is at the level required for production of an initial bulletin or for quasi-definitive publication.

  • 4: The data has been finalised and no further changes are intended

Only these values are allowed.

This field provides a quick description of the point the data has reached in the publication process. For detailed information on the standards that the data conforms to see section Attributes that Relate to Data Standards and Quality (V1.2)

PublicationDate

Date/time

1

Date and time on which the data was published. This attribute is used to distinguish multiple publications of the same data.
Attributes that Describe the Observatory (V1.2)

These attributes are available from other metadata systems (given an IAGA code), but are included for convenience of the user.

Attribute Name

Type

Entries

Description

ObservatoryName

String

1

The full name for the observatory

Latitude

Double

1

The latitude of the observing position in degrees, to WGS-84 datum

Longitude

Double

1

The longitude of the observing position in degrees, to WGS-84 datum

Elevation

Double

1

The height of the observing position in metres above sea level, to WGS-84 datum. Set to 99999.0 if not known.

Institution

String

1 or more

The name of the responsible institute.

VectorSensOrient

String

0 - 1

The orientation code of the sensor at the original recording of the vector data. A string consisting of single character codes, each describing one of the geomagnetic field elements that was recorded by the vector instrument. Valid codes are the same as for the ElementsRecorded attribute.
Attributes that Relate to Data Standards and Quality (V1.2)

These attributes describe the standards, if any, that the data meets.

Attribute Name

Type

Entries

Description

StandardLevel

String

1

Describe whether the data conforms to a standard. Choose from one of the following codes:

  • None: The data does not conform to any standards. When using this, the StandardName attribute does not need to be set.

  • Partial: The data partially conforms to the relevant standard for this data product.

  • Full: The data fully conforms to the relevant standard for this data product.

Only these values are allowed.

If StandardsLevel is set to Partial, then the PartialStandDesc attribute must also be set.

StandardName

String

0 - 1

The name of the relevant standard. See section Relevant Data Standards (V1.2)

StandardVersion

String

0 – 1

If the standard has a version, put its version number in this attribute.

PartialStandDesc

String

0 - 1

See section Relevant Data Standards (V1.2)
Attributes that Relate to Publication of the Data (V1.2)

These attributes are needed when that data is published.

Attribute Name

Type

Entries

Description

Source

String

1

Set to one of: “institute” (if the named institution provided the data- see section Attributes that Describe the Observatory (V1.2) for the institution); “INTERMAGNET” (if the data file has been created by INTERMAGNET from another data source); “WDC” (if the World Data Centre has created the file from another data source)

TermsOfUse

String

0 – 1

The terms of use for the data. This could be text describing the terms of use or a link to a web page. INTERMAGNET has a recommended wording for data provided through INTERMAGNET.

UniqueIdentifier

String

0 – 1

A string that can be used to uniquely identify this data. This could be a Digital object identifier or could be an identifier created according to local rules. Note this is optional and will not be present if this data is not covered by an identifier.

ParentIdentifiers

String

0 or more

The unique identifiers of the parent data sets (if any), one identifier per entry. The parent data set is the data set from which this data set’s values have been derived. E.g. if a one minute data set has been created from a one second data set, the one second data set is the parent. Another example is where definitive data is created from provisional data.

ReferenceLinks

String/URL

0 or more

URLs pointing to (e.g.) information about the data creator, information about the data repository… One URL per entry.
Valid Codes for Elements Recorded (V1.2)

  • ‘X’, ‘Y’, or ‘Z’ indicate that the variable holds the strength of the magnetic field vector in the standard geographic coordinates in nT.

  • ‘H’ indicates that the variable holds the strength of the magnetic field vector in the horizontal plane along the magnetic meridian in nT.

  • ‘D’ indicates that the variable holds the angle between the magnetic vector and true north, in degrees of arc, positive east.

  • ‘E’ indicates that the variable holds a field strength in the horizontal plane perpendicular to ‘H’ in nT. ‘E’ is only valid for data that is not baseline corrected.

  • ‘V’ indicates that the variable holds the field strength along the direction of the inclination.

  • ‘I’ indicates that the variable holds the angle between the magnetic vector and the horizontal plane, in degrees of arc, positive below the horizontal.

  • ‘F’ indicates that the variable holds the geomagnetic field strength in nT, calculated from and consistent with XYZ or HDZ field elements.

  • ‘S’ indicates that the variable holds the geomagnetic field strength in nT, measured by an independent scalar instrument

  • ‘G’ indicates that the variable holds delta- F values, defined as F(vector) –S(scalar) in nT. When calculating values for the G element, if F(vector) is missing, G is set to –S (scalar)

Other codes are allowed, but may lead to data not being understood.

Relevant Data Standards (V1.2)

Different geomagnetic data products have different standards associated with them. This table shows what standards are being referred to in the StandardsLevel attribute and describes what to put into the StandardName attribute in the case where StandardsLevel is set to Partial or Full. The table also shows what to put in the PartialStandDesc attribute in the case where the StandardsLevel attribute is set to Partial.

Data product

Relevant Standard

Contents of StandardName

One second definitive data

INTERMAGNET Definitive One-second Data Standard [1]

INTERMAGNET_1-Second

One minute definitive data

INTERMAGNET magnetic observatory – specifications [2]

INTERMAGNET_1-Minute

One minute quasi-definitive data

INTERMAGNET magnetic observatory – specifications [2],modified for baseline accuracy [3]

INTERMAGNET_1-Minute_QD

Hourly means

No relevant standard

Daily means

No relevant standard

Monthly means

No relevant standard

Annual means

No relevant standard

If a standard is met in full or not met at all, PartialStandDesc can be omitted. Where a standard is partially met (e.g. the time stamp accuracy is within tolerance, but the data is not baseline corrected), the name of the relevant standard should be put in the StandardName attribute, StandardLevel should be set to Partial and the PartialStandDesc attribute should be filled in with a comma separated list of the sub-sections from the standard that the data meets. E.g. if the data meets the time stamp accuracy and thermal stability sections of the 1-minute data standard, enter IMOM-01,IMOM-16 in PartialStandDesc.

Value to put in PartialStandDesc

Description

One-minute Definitive Data: General specifications

IMOM-01

Time-stamp accuracy (centered on the UTC minute): 5s

One-minute Definitive Data: Vector Magnetometer specifications

IMOM-11

Absolute Accuracy: ±5nT

IMOM-12

Resolution: 0.1nT

IMOM-13

Dynamic Range: ≥±4000nT High Lat., ≥±3000nT Mid/Equatorial Lat.

IMOM-14

Band pass: D.C. to 0.1Hz

IMOM-15

Minimum sampling rate: 1Hz

IMOM-16

Thermal stability: 0.25nT/°C

IMOM-17

Long term stability: 5nT/year

IMOM-18

Filtering to one-minute data: INTERMAGNET Gaussian

One-minute Definitive Data: Scalar Magnetometer specifications

IMOM-21

Resolution: 0.1nT

IMOM-22

Absolute Accuracy: ±1nT

IMOM-23

Minimum sampling rate: 0.033Hz (30 sec)

One-second Data: General Specifications

IMOS-01

Time-stamp accuracy (centered on the UTC second): 0.01s

IMOS-02

Phase response: Maximum group delay: ±0.01s

IMOS-03

Maximum filter width: 25 seconds

IMOS-04

Instrument amplitude range: ≥±4000nT High Lat., ≥±3000nT Mid/Equatorial Lat.

IMOS-05

Data resolution: 1pT

IMOS-06

Pass band: DC to 0.2Hz

One-second Data: Specifications in the Pass Band [DC to 8mHz (120s)]

IMOS-11

Noise level: ≤100pT RMS

Maximum offset error (cumulative error between absolute observations): ±2.5 nT

IMOS-13

Maximum component scaling plus linearity error: 0.25%

IMOS-14

Maximum component orthogonality error: 2mrad

IMOS-15

Maximum Z-component verticality error: 2mrad

One-second Data: Specifications in the Pass Band [8mHz (120s) to 0.2Hz]

IMOS-21

Noise level: ≤10pT/√Hz at 0.1 Hz

IMOS-22

Maximum gain/attenuation: 3dB

One-second Data: Specifications in the Stop Band [≥0.5 Hz]

IMOS-31

Minimum attenuation in the stop band (≥ 0.5Hz): 50dB

One-second Data: Auxiliary measurements:

IMOS-41

Compulsory full-scale scalar magnetometer measurements with a data resolution of 0.01nT at a minimum sample period of 30 seconds.

IMOS-42

Compulsory vector magnetometer temperature measurements with a resolution of 0.1°C at a minimum sample period of one minute.

ImagCDF Variable Attributes (V1.2)

The following attributes apply to individual variables - there is an attribute entry for each geomagnetic field element or temperature in an ImagCDF file and the value of that entry applies only to that field element or temperature. The “Entries” column shows whether the attribute is:

  • Mandatory (number of entries per variable is exactly 1)

  • Optional (number of entries per variable may be 0 or 1)

None of these attributes are required for the time stamp variables GeomagneticVectorTimes and GeomagneticScalarTimes.

Attribute Name

Type

Entries

Description

FIELDNAM [4]

String

1

Set to “Geomagnetic Field Element ” + the element code (e.g. H, D, Z,… - see section Valid Codes for Elements Recorded (V1.2) for a list of valid codes); or set to “Temperature ” + the name of the location where the temperature was recorded.

UNITS [4]

String

1

Must be one of “nT”, “Degrees of arc” or “Celsius”

FILLVAL [4]

Double

1

The value used to show that a data sample is missing. Set to 99999.0 for compatibility with other formats. The values must either less than VALIDMIN or greater than VALIDMAX.

VALIDMIN [4]

Double

1

The smallest allowed numeric value for the data in the corresponding variable.

VALIDMAX [4]

Double

1

The largest allowed numeric value for the data in the corresponding variable.

DEPEND_0 [5]

String

0 - 1

For geomagnetic data, set this to the name of the variable that holds time stamps for this time series. For records containing time stamps, do not set this variable.

DISPLAY_TYPE [5]

String

1

Set to “time_series”

LABLAXIS [5]

String

1

Set to the element code (as defined in section Valid Codes for Elements Recorded (V1.2) )

ImagCDF File Names (V1.2)

ImagCDF files are named using the convention:

[iaga-code]_[date-time]_[publication-level].cdf

Filenames are in lower case. Files may contain arbitrary amounts of data, however the amount of data is not coded into the filename.

Format of Date/Time Portion of Filename and Examples (V1.2)

The date/time portion of the filename is formatted differently for different data with different data intervals:

Data Interval

Date/time format

Example filename

Annual means

YYYY

esk_2000_4.cdf – final annual mean data from Eskdalemuir starting in the year 2000

Monthly means

YYYYMM

ott_201401_4.cdf – final monthly mean data from Ottawa starting in January 2014

Daily means

YYYYMMDD

gua_20100101_4.cdf – final daily mean data from Guam starting at the beginning of 2010

Hourly means

YYYYMMDD_HH

naq_20020201_00_4.cdf – final hourly mean data from NAQ in February 2002

Minute means

YYYYMMDD_HHMM

naq_20020120_0000_3.cdf – ‘bulletin’ or quasi-definitive minute mean data from NAQ for 20th January 2002 starting at midnight

Second

YYYYMMDD_HHMMSS

naq_20020120_012300_1.cdf – raw 1-second data from NAQ for 20th January 2002 starting at 01:23:00

  • YYYY = four digit year (i.e. 2002)

  • MM = two digit month (01 for January - 12 for December)

  • DD = two digit day of month (01-31)

  • HH = two digit hour (0-23)

  • MM = two digit minute (0-59)

  • SS = two digit second (0-59)

ImagCDF Version 1.3

The descriptions in this section apply to version 1.3 of ImagCDF.

Version 1.3 only changes the format of the ImagCDF filenames. The documentation for version 1.2 should be used for the contents of these files:

ImagCDF Global Attributes (V1.3)

The following changes have been made to global attributes since version 1.2:

  1. The “FormatVersion” global attribute should be set to “1.3”.

  2. An additional optional variable “LeapSecondLastUpdated” may be used to describe when the CDF Leap Second table was last updated. NASA CDF libraries specify this value as part of CDF file metadata, but some third party libraries do not set the value in the metadata. Where libraries are used that do not set this value, data providers are encouraged to use this global attribute. The contents of the attribute should be the same as the CDF file metadata, which is the date of the day following the last leap second, coded as an integer: (year * 10000) + (month * 100) + day.

See the global attributes documentation for version 1.2 for a decsription of the other global attributes.

ImagCDF Variable Attributes (V1.3)

There are no changes to the variable attributes in the format since version 1.2. See the varible attributes documentation for version 1.2.

ImagCDF File Names (V1.3)

Version 1.3 fixes some errors and omissions in the format of ImagCDF filenames.

ImagCDF files are named using the convention:

[iaga-code]_[date-time]_[cadence]_[publication-level].cdf

  • Iaga-code is the three letter IAGA code for the observatory that the data is from. This should match the IAGA code in section Attributes that Uniquely Identify the Data (V1.2).

  • Date-time describes the time coverage of the data (i.e. how much data is in the file) along with the date/time of the data in the file. The format for the date/time is described below.

  • Cadence refers to the data sample period and is formatted as an ISO 8601 duration. Some sample ISO 8601 durations are given below.

  • Publication-level is the PublicationLevel attribute from section Attributes that Uniquely Identify the Data (V1.2).

Filenames are in lower case. Files may contain any amount of data. The date/time string may be used to indicate the amount of data in the file - more information on this is given in the description of the date/time format.

ISO 8601 Duration Strings for Common Geomagnetic Sample Periods (V1.3)

Sample Period

ISO 8601 duration

1 second

PT1S

1 minute

PT1M

Hourly means

PT1H

Daily means

P1D

Monthly means

P1M

Annual means

P1Y

The table above is a set of examples. Other sample periods may be used provided that the sample period used represents the vector data and conforms to ISO 8601.

Format of Date/Time Portion of Filename and Examples (V1.3)

The date/time portion of the filename is formatted differently, depending on the amount of data the file contains (the data coverage).

Amount of data in file (coverage)

Date/time format

Example filename

One year

YYYY

esk_2000_pt1m_4.cdf – one year of final minute mean data from Eskdalemuir for the year 2000

One month

YYYYMM

ott_201401_pt1s_4.cdf – one month of final 1-second data from Ottawa for January 2014

One day

YYYYMMDD

gua_20100101_pt1m_3.cdf – one day of quasi-definitive minute mean data from Guam for 1st January 2010

One hour

YYYYMMDD_HH

naq_20020201_00_pt1m_2.cdf – one hour of provisional minute mean data from NAQ starting at midnight on 1st February 2002

One minute

YYYYMMDD_HHMM

naq_20020120_0000_pt1s_2.cdf – one minute of provisional 1-second data from NAQ starting at 00:00 on 20th of January 2002

One second

YYYYMMDD_HHMMSS

naq_20020120_012300_pt1s_1.cdf – one second of raw 1-second data from NAQ at 01:23:00 on 20th January 2002

  • YYYY = four digit year (i.e. 2002)

  • MM = two digit month (01 for January - 12 for December)

  • DD = two digit day of month (01-31)

  • HH = two digit hour (0-23)

  • MM = two digit minute (0-59)

  • SS = two digit second (0-59)

The start date/time of the data is expected to conform to the coverage: For a year of data the start date/time is expected to be the start of the first day of the year. For a day of data the start date/time is expected to be the start of the day. And so on for the other coverages listed in the table.

In the situations where the state date/time doesn’t conform to this rule, or the amount of data is not listed in the table, the data is considered to be a “fragment” (ie an incomplete part of a larger data coverage). The filename for data fragments should use the full date/time, the same as for data with a coverage of one second.

Tools to Look at CDF Data

Once you have some CDF data you will want to look at it. The CDF toolset (that is installed for you when you install CDF) provides programs to do this. The simplest way is (using a command shell or DOS prompt):

cdfdump [filename] | more

Which will display the entire contents of the file. The example file is formatted using cdfdump. Other tools from the CDF toolset that may be useful include cdfexport and cdfedit.

Autoplot is a useful tool that can plot the time series data in ImagCDF data files. You can download it from http://autoplot.org/.

A number of packages provide access to CDF data. These include Matlab and IDL. A list is maintained on the NASA CDF website: http://cdf.gsfc.nasa.gov/html/FAQ.html#cdfsw

Octave may be able to read CDF data:http://lists.gnu.org/archive/html/help-octave/2007-04/txtyRdYjKU85H.txt. The Wolfram Language can use CDF data via its NASACDF data format - http://reference.wolfram.com/language/guide/NumericalDataFormats.html.

Example Data File

The data in this example was extracted from a monthly file containing 1-second definitive data from Abisko observatory for January 2019. Only the first two data records are shown for each variable.

File Info
=========================================
CDF File:     abk_201901_PT1S_4.cdf
Version:      3.7.0

Common Data Format (CDF)
https://cdf.gsfc.nasa.gov
Space Physics Data Facility
NASA/Goddard Space Flight Center
Greenbelt, Maryland 20771 USA
(User support: gsfc-cdf-support@lists.nasa.gov)

Format:       SINGLE
Encoding:     IBMPC
Majority:     ROW
NumrVars:     0
NumzVars:     5
NumAttrs:     29 (21 global, 8 variable)
Compression:  GZIP.1
Checksum:     None
LeapSecondLastUpdated:     20170101

Global Attributes (21 attributes)
=========================================
FormatDescription (num:0) (1 entry):
        0 (CDF_CHAR/22):        "INTERMAGNET CDF format"
FormatVersion (num:1) (1 entry):
        0 (CDF_CHAR/3):         "1.3"
Title (num:2) (1 entry):
        0 (CDF_CHAR/28):        "Geomagnetic time series data"
IagaCode (num:3) (1 entry):
        0 (CDF_CHAR/3):         "ABK"
ElementsRecorded (num:4) (1 entry):
        0 (CDF_CHAR/4):         "XYZG"
PublicationLevel (num:5) (1 entry):
        0 (CDF_CHAR/1):         "4"
PublicationDate (1 entry):
     0 (CDF_TT2000/1):          2022-10-08T12:19:06.088000000
ObservatoryName (num:7) (1 entry):
        0 (CDF_CHAR/6):         "Abisko"
Latitude (num:8) (1 entry):
        0 (CDF_DOUBLE/1):       68.358
Longitude (num:9) (1 entry):
        0 (CDF_DOUBLE/1):       18.823
Elevation (num:10) (1 entry):
        0 (CDF_CHAR/5):         "380.0"
Institution (num:11) (1 entry):
        0 (CDF_CHAR/27):        "Geological Survey of Sweden"
VectorSensOrient (num:12) (1 entry):
        0 (CDF_CHAR/3):         "XYZ"
StandardLevel (num:13) (1 entry):
        0 (CDF_CHAR/7):         "Partial"
StandardName (num:14) (1 entry):
        0 (CDF_CHAR/20):        "INTERMAGNET_1-Second"
PartialStandDesc (num:15) (1 entry):
        0 (CDF_CHAR/87):        "IMOS-03,IMOS-04,IMOS-05,IMOS-06,IMOS-11,IMOS-12,IMOS-14,IMOS-15,IMOS-22,IMOS-41,IMOS-42"
Source (num:16) (1 entry):
        0 (CDF_CHAR/9):         "institute"
TermsOfUse (num:17) (1 entry):
        0 (CDF_CHAR/62):        "http://www.intermagnet.org/data-donnee/data-eng.php#conditions"
LeapSecondUpdated (num:18) (1 entry):
        0 (CDF_CHAR/8):         "20170101"
ParentIdentifiers (num:19) (1 entry):
        0 (CDF_CHAR/13):        "ABKsec_4_0001"
SamplingRate (num:20) (1 entry):
        0 (CDF_CHAR/7):         "1.0 sec"

Variable Attributes (8 attributes)
=========================================
DEPEND_0 (num:21)
DISPLAY_TYPE (num:22)
LABLAXIS (num:23)
FILLVAL (num:24)
VALIDMIN (num:25)
VALIDMAX (num:26)
FIELDNAM (num:27)
UNITS (num:28)

Variable Information (0 rVariable, 5 zVariables)
===========================================================
DataTimes             CDF_TT2000/1      0:[]    T/
GeomagneticFieldX     CDF_DOUBLE/1      0:[]    T/
GeomagneticFieldY     CDF_DOUBLE/1      0:[]    T/
GeomagneticFieldZ     CDF_DOUBLE/1      0:[]    T/
GeomagneticFieldG     CDF_DOUBLE/1      0:[]    T/

Variable (5 variables)
=========================================

DataTimes (No: 0) (Recs: 2678400) (Compression: GZIP.6 BlockingFactor: 8192)
----------------------
Data Type:           CDF_TT2000
Dimensionality:      0:[]       (T/)
Compression:         GZIP.6
Pad value:           0000-01-01T00:00:00.000000000
Written Records:     2678400/2678400(max)
Allocated Records:   2678400/2678400(max)
Blocking Factor:     8192 (records)
Attribute Entries:
Variable Data:
  Record # 1: 2019-01-01T00:00:00.000000000
  Record # 2: 2019-01-01T00:00:01.000000000
  ...

GeomagneticFieldX (No: 1) (Recs: 2678400) (Compression: GZIP.6 BlockingFactor: 8192)
-----------------
Data Type:           CDF_DOUBLE
Dimensionality:      0:[]       (T/)
Compression:         GZIP.6
Pad value:           -1.0e+30
Written Records:     2678400/2678400(max)
Allocated Records:   2678400/2678400(max)
Blocking Factor:     8192 (records)
Attribute Entries:
     DEPEND_0        (CDF_CHAR/22): "DataTimes"
     DISPLAY_TYPE    (CDF_CHAR/11): "time_series"
     LABLAXIS        (CDF_CHAR/1): "X"
     FILLVAL         (CDF_DOUBLE/1): nan
     VALIDMIN        (CDF_DOUBLE/1): -88880.0
     VALIDMAX        (CDF_DOUBLE/1): 88880.0
     FIELDNAM        (CDF_CHAR/27): "Geomagnetic Field Element X"
     UNITS           (CDF_CHAR/2): "nT"
Variable Data:
  Record # 1: 11283.1
  Record # 2: 11283.0
  ...

GeomagneticFieldY (No: 2) (Recs: 2678400) (Compression: GZIP.6 BlockingFactor: 8192)
-----------------
Data Type:           CDF_DOUBLE
Dimensionality:      0:[]     (T/)
Compression:         GZIP.6
Pad value:           -1.0e+30
Written Records:     2678400/2678400(max)
Allocated Records:   2678400/2678400(max)
Blocking Factor:     8192 (records)
Attribute Entries:
     DEPEND_0        (CDF_CHAR/22): "DataTimes"
     DISPLAY_TYPE    (CDF_CHAR/11): "time_series"
     LABLAXIS        (CDF_CHAR/1): "Y"
     FILLVAL         (CDF_DOUBLE/1): nan
     VALIDMIN        (CDF_DOUBLE/1): -88880.0
     VALIDMAX        (CDF_DOUBLE/1): 88880.0
     FIELDNAM        (CDF_CHAR/27): "Geomagnetic Field Element Y"
     UNITS           (CDF_CHAR/2): "nT"
Variable Data:
  Record # 1: 1806.72
  Record # 2: 1806.67
  ...

GeomagneticFieldZ (No: 3) (Recs: 2678400) (Compression: GZIP.6 BlockingFactor: 8192)
-----------------
Data Type:           CDF_DOUBLE
Dimensionality:      0:[]     (T/)
Compression:         GZIP.6
Pad value:           -1.0e+30
Written Records:     2678400/2678400(max)
Allocated Records:   2678400/2678400(max)
Blocking Factor:     8192 (records)
Attribute Entries:
     DEPEND_0        (CDF_CHAR/22): "DataTimes"
     DISPLAY_TYPE    (CDF_CHAR/11): "time_series"
     LABLAXIS        (CDF_CHAR/1): "Z"
     FILLVAL         (CDF_DOUBLE/1): nan
     VALIDMIN        (CDF_DOUBLE/1): -88880.0
     VALIDMAX        (CDF_DOUBLE/1): 88880.0
     FIELDNAM        (CDF_CHAR/27): "Geomagnetic Field Element Z"
     UNITS           (CDF_CHAR/2): "nT"
Variable Data:
  Record # 1: 52087.4
  Record # 2: 52087.4
  ...

GeomagneticFieldG (No: 4) (Recs: 2678400) (Compression: GZIP.6 BlockingFactor: 8192)
-----------------
Data Type:           CDF_DOUBLE
Dimensionality:      0:[]     (T/)
Compression:         GZIP.6
Pad value:           -1.0e+30
Written Records:     2678400/2678400(max)
Allocated Records:   2678400/2678400(max)
Blocking Factor:     8192 (records)
Attribute Entries:
     DEPEND_0        (CDF_CHAR/22): "DataTimes"
     DISPLAY_TYPE    (CDF_CHAR/11): "time_series"
     LABLAXIS        (CDF_CHAR/1): "G"
     FILLVAL         (CDF_DOUBLE/1): nan
     VALIDMIN        (CDF_DOUBLE/1): -88880.0
     VALIDMAX        (CDF_DOUBLE/1): 88880.0
     FIELDNAM        (CDF_CHAR/28): "Geomagnetic Field Element DF"
     UNITS           (CDF_CHAR/2): "nT"
Variable Data:
  Record # 1: 0.0254182
  Record # 2: 0.0189378
  ...

INTERMAGNET Gin Dissemination Format for Minute Values : IMFV

IMFV1.23

Magnetic data, with tenth-nanotesla resolution, are organized on a day file basis. One file contains 24 one-hour blocks, each containing 60 minutes worth of values. Blocks of 60 minutes of data are transmitted. Blocks are padded with 9’s if incomplete. Information is coded in ASCII.

File name: To remain compatible with all operating systems, the file name is limited to 8 characters and will contain the date and the three-letter code as an extension. eg: MAR1591.BOU for Boulder, March 15, 1991; and JUN2391.OTT for June 23, 1991 at Ottawa.

Description of the Header Block (64 characters including CrLf)

IDC_DDDDDDD_DOY_HH_COMP_T_GIN_COLALONG_DECBAS_RRRRRRRRRRRRRRRRCrLf

Field

Description

IDC

Indicates the IAGA three letter observatory identification (ID) code eg: BOU for Boulder, OTT for Ottawa, LER for Lerwick, etc.

DDDDDDD

Indicates the date, eg: FEB1591 for February 15, 1991.

DOY

Indicates day of the year (1-366)

HH

Indicates the Hour (0-23). The first line following the header will contain the values corresponding to minute 0 and 1 of this hour. The first value of the day file is hour 0 minute 0.

COMP

Order in which the components are listed, can be HDZF, HDZG, XYZF, XYZG. G represents the difference between a measured ‘Scalar’ F and a computed ‘vector’ F: G = F(vector) - F(scalar). All components excluding D must be in tenths of nT. D must be in hundredths of minutes, east. The F or G component should be included only if it is measured from a scalar instrument independent of the other 3 components otherwise it must be filled with 999999.

T

One-letter code for data type. R=Reported, A=Adjusted, Q=Quasi-definitive, D=Definitive data.

Reported data are defined as: the raw data obtained from the IMO, either by satellite, computer link, or other means. It will be formatted in either version IMFV2.8N (binary) or IMFV1.2N (ASCII,) without any RM (Reference Measurements), or other modifications applied to it.

Adjusted data are defined as: the Reported data with RM, spike removal, timeshifts, and/or other modifications applied to it. It is emphasized that only one (1) adjusted version of the data would be allowed, to be completed within 7 days of receipt of the Reported data to prevent the proliferation of multiple versions of the Adjusted data.

Quasi-definitive data are defined as data that have been corrected using provisional baselines. Produced soon after their acquisition, their accuracy is intended to be very close to that of an observatory’s definitive data product. 98% of the differences between quasi-definitive and definitive data (X, Y, Z) monthly mean values should be less than 5nT.

Definitive data are defined as the final adopted data values. Definitive data will only be distributed by the institution responsible for the observatory.

GIN

Three-letter code for GIN responsible for processing the station (IMO) data eg: EDI(Edinburgh), GOL(Golden), OTT(Ottawa), PAR(Paris).

COLALONG

Colatitude and east longitude of the observatory in tenths of degrees, to WGS-84 datum.

DECBAS

Baseline declination value in tenths of minutes East (0-216,000). Declination baseline values to be provided annually. If components are X,Y,Z then DECBAS=000000. The DECBAS value is used to allow declination data to exceed 166.66 degrees (which is the limit of the declination elements in each data record without any offset). If the declination is > 166 degrees at an observatory then a DECBAS value should be selected such that the minute samples lie between 0 and 166 degrees. The declination baseline DECBAS should be subtracted from the minute data samples before coding them.

RRR..RRR

Reserved 16 bytes of R-characters for future use.

_

Indicates a space character.

CrLf

Indicates a Carriage return, Line feed.

Missing values for a vector component must be coded as a space (for the sign bit) followed by six 9 digits: _999999.

Missing values for the scalar component must be coded as six 9 digits: 999999

Description of Data Space (64 characters per line including CrLf)

Component values are coded as signed integers, right-justified with a field width of 7. Total field (F) values are coded as unsigned integers, right-justified with a field width of 6. The field widths must be maintained, either through zero-filling or space-filling. The ‘+’ sign for positive values is optional.

Two (2) minutes of data are concatenated on the same line

AAAAAAA_BBBBBBB_CCCCCCC_FFFFFF__AAAAAAA_BBBBBBB_CCCCCCC_FFFFFFCrLf
(values for minute 0) (values for minute 1)
. .
. .
. .
AAAAAAA_BBBBBBB_CCCCCCC_FFFFFF__AAAAAAA_BBBBBBB_CCCCCCC_FFFFFFCrLf
(values for minute 58) (values for minute 59)

Field

Description

AAAAAAA

Indicates Component 1 data field (H,X, etc.).

BBBBBBB

Indicates Component 2 data field (D,Y, etc.).

CCCCCCC

Indicates Component 3 data field (Z,I, etc.).

FFFFFF

Indicates Total Field data field.

_

Indicates space character.

CrLf

Indicates Carriage Return and Line Feed.

Sample of missing values

_999999__999999__999999_999999___999999__999999__999999_999999CrLf

This example represents all components as missing for the first two minutes of the hour.

“_” indicates a space character.

IMFV1.22

Magnetic data, with tenth-nanotesla resolution, are organized on a day file basis. One file contains 24 one-hour blocks, each containing 60 minutes worth of values. Blocks of 60 minutes of data are transmitted. Blocks are padded with 9’s if incomplete. Information is coded in ASCII.

File name: To remain compatible with all operating systems, the file name is limited to 8 characters and will contain the date and the three-letter code as an extension. eg: MAR1591.BOU for Boulder, March 15, 1991; and JUN2391.OTT for June 23, 1991 at Ottawa.

Description of the Header Block (64 characters including CrLf)

IDC_DDDDDDD_DOY_HH_COMP_T_GIN_COLALONG_DECBAS_RRRRRRRRRRRRRRRRCrLf

Field

Description

IDC

Indicates the IAGA three letter observatory identification (ID) code eg: BOU for Boulder, OTT for Ottawa, LER for Lerwick, etc.

DDDDDDD

Indicates the date, eg: FEB1591 for February 15, 1991.

DOY

Indicates day of the year (1-366)

HH

Indicates the Hour (0-23). The first line following the header will contain the values corresponding to minute 0 and 1 of this hour. The first value of the day file is hour 0 minute 0.

COMP

Order in which the components are listed, can be HDZF, XYZF. All components excluding D must be in tenths of nT. D must be in hundredths of minutes, east. The F component should be included only if it is measured from a scalar instrument independent of the other 3 components otherwise it must be filled with 999999.

T

One-letter code for data type. R=Reported, A=Adjusted, D=Definitive data.

Reported data are defined as: the raw data obtained from the IMO, either by satellite, computer link, or other means. It will be formatted in either version IMFV2.8N (binary) or IMFV1.2N (ASCII,) without any RM (Reference Measurements), or other modifications applied to it.

Adjusted data are defined as: the Reported data with RM, spike removal, timeshifts, and/or other modifications applied to it. It is emphasized that only one (1) adjusted version of the data would be allowed, to be completed within 7 days of receipt of the Reported data to prevent the proliferation of multiple versions of the Adjusted data.

Definitive data are defined as the final adopted data values. Definitive data will only be distributed by the institution responsible for the observatory.

GIN

Three-letter code for GIN responsible for processing the station (IMO) data eg: EDI(Edinburgh), GOL(Golden), OTT(Ottawa), PAR(Paris).

COLALONG

Colatitude and east longitude of the observatory in tenths of degrees, to WGS-84 datum.

DECBAS

Baseline declination value in tenths of minutes East (0-216,000). Declination baseline values to be provided annually. If components are X,Y,Z then DECBAS=000000.

RRR..RRR

Reserved 16 bytes of R-characters for future use.

_

Indicates a space character.

CrLf

Indicates a Carriage return, Line feed.

Missing values for a vector component must be coded as a space (for the sign bit) followed by six 9 digits: _999999.

Missing values for the scalar component must be coded as six 9 digits: 999999

Description of Data Space (64 characters per line including CrLf)

Component values are coded as signed integers, right-justified with a field width of 7. Total field (F) values are coded as unsigned integers, right-justified with a field width of 6. The field widths must be maintained, either through zero-filling or space-filling. The ‘+’ sign for positive values is optional.

Two (2) minutes of data are concatenated on the same line

AAAAAAA_BBBBBBB_CCCCCCC_FFFFFF__AAAAAAA_BBBBBBB_CCCCCCC_FFFFFFCrLf
(values for minute 0) (values for minute 1)
. .
. .
. .
AAAAAAA_BBBBBBB_CCCCCCC_FFFFFF__AAAAAAA_BBBBBBB_CCCCCCC_FFFFFFCrLf
(values for minute 58) (values for minute 59)

Field

Description

AAAAAAA

Indicates Component 1 data field (H,X, etc.).

BBBBBBB

Indicates Component 2 data field (D,Y, etc.).

CCCCCCC

Indicates Component 3 data field (Z,I, etc.).

FFFFFF

Indicates Total Field data field.

_

Indicates space character.

CrLf

Indicates Carriage Return and Line Feed.

Sample of missing values

_999999__999999__999999_999999___999999__999999__999999_999999CrLf

This example represents all components as missing for the first two minutes of the hour.

“_” indicates a space character.