.. NeXus - Neutron and X-ray Common Data Format Copyright (C) 2008-2020 NeXus International Advisory Committee (NIAC) This library is free software; you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version. This library is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details. You should have received a copy of the GNU Lesser General Public License along with this library; if not, write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA For further information, see http://www.nexusformat.org Definitions of the basic data types and unit types allowed in NXDL instance files. A ``definition`` element is the ``group`` at the root of every NXDL specification. It may *only* appear at the root of an NXDL file and must only appear **once** for the NXDL to be *well-formed*. Used for allowed names of elements and attributes. Need to be restricted to valid program variable names. Note: This means no "-" or "." characters can be allowed and you cannot start with a number. HDF4 had a 64 character limit on names (possibly including NULL) and NeXus enforces this via the ``NX_MAXNAMELEN`` variable with a **64** character limit (which may be 63 on a practical basis if one considers a NULL terminating byte). (This data type is used internally in the NXDL schema to define a data type.) Used for allowed names of NX class types (e.g. NXdetector) not the instance (e.g. bank1) which is covered by validItemName. (This data type is used internally in the NXDL schema to define a data type.) This is a valid link target - currently it must be an absolute path made up of valid names with the ``/`` character delimiter. But we may want to consider allowing "``..``" (parent of directory) at some point. If the ``name`` attribute is helpful, then use it in the path with the syntax of *name:type* as in these examples:: /NXentry/NXinstrument/analyzer:NXcrystal/ef /NXentry/NXinstrument/monochromator:NXcrystal/ei /NX_other Must also consider use of ``name`` attribute in resolving ``link`` targets. (This data type is used internally in the NXDL schema to define a data type.) From the HDF5 documentation: *Note that relative path names in HDF5 do not employ the ``../`` notation, the UNIX notation indicating a parent directory, to indicate a parent group.* Thus, if we only consider the case of ``[name:]type``, the matching regular expression syntax is written: ``/[a-zA-Z_][\w_]*(:[a-zA-Z_][\w_]*)?)+``. Note that HDF5 also permits relative path names, such as: ``GroupA/GroupB/Dataset1`` but this is not permitted in the matching regular expression and not supported in NAPI. The presence of the ``deprecated`` attribute indicates to the data file validation process that an advisory message (specified as the content of the ``deprecated`` attribute) will be reported. Future versions of the NXDL file might not define (or even re-use) the component marked with this attribute. The value of the attribute will be printed in the documentation. Make it descriptive (limited to no line breaks). For example:: deprecated="as of release MAJOR.MINOR" Note: because ``deprecated`` is an attribute, the XML rules do not permit it to have any element content. A ``definition`` is the root element of every NXDL definition. It may *only* appear at the root of an NXDL file and must only appear **once** for the NXDL to be *well-formed*. The ``definitionType`` defines the documentation, attributes, fields, and groups that will be used as children of the ``definition`` element. Could contain these elements: * ``attribute`` * ``doc`` * ``field`` * ``group`` * ``link`` Note that a ``definition`` element also includes the definitions of the ``basicComponent`` data type. (The ``definitionType`` data type is used internally in the NXDL schema to define elements and attributes to be used by users in NXDL specifications.) Note that the first line of text in a ``doc`` element in a ``definition`` is used as a summary in the manual. Follow the pattern as shown in the base class NXDL files. Use a ``symbols`` list to define each of the mnemonics that represent the length of each dimension in a vector or array. In addition to an optional ``symbols`` list, a ``definition`` may contain any of the items allowed in a ``group``. The ``name`` of this NXDL file (without the file extensions). The name must be unique amongst all the NeXus base class, application, and contributed definitions. For the class to be adopted by the NIAC, the first two letters must be "``NX``" (in uppercase). Any other use must *not* begin with "``NX``" in any combination of upper or lower case. Must be ``type="group"`` The ``extends`` attribute allows this definition to *subclass* from another NXDL, otherwise ``extends="NXobject"`` should be used. The ``restricts`` attribute is a flag to the data validation. When ``restricts="1"``, any non-standard component found (and checked for validity aginst this NXDL specification) in a NeXus data file will be flagged as an error. If the ``restricts`` attribute is not present, any such situations will produce a warning. (2014-08-19: deprecated since switch to GitHub version control) The identifier string from the subversion revision control system. This reports the time stamp and the revision number of this file. NXDL ``base`` definitions define the dictionary of terms to use for these components. All terms in a ``base`` definition are optional. NXDL ``application`` definitions define what is required for a scientific interest. All terms in an ``application`` definition are required. NXDL ``contributed`` definitions may be considered either base or applications. Contributed definitions must indicate their intended use, either as a base class or as an application definition. Only validate known groups; do not not warn about unknowns. The ``ignoreExtraGroups`` attribute is a flag to the process of validating NeXus data files. By setting ``ignoreExtraGroups="true"``, presence of any undefined groups in this class will not generate warnings during validation. Normally, validation will check all the groups against their definition in the NeXus base classes and application definitions. Any items found that do not match the definition in the NXDL will generate a warning message. The ``ignoreExtraGroups`` attribute should be used sparingly! Only validate known fields; do not not warn about unknowns. The ``ignoreExtraFields`` attribute is a flag to the process of validating NeXus data files. By setting ``ignoreExtraFields="true"``, presence of any undefined fields in this class will not generate warnings during validation. Normally, validation will check all the fields against their definition in the NeXus base classes and application definitions. Any items found that do not match the definition in the NXDL will generate a warning message. The ``ignoreExtraFields`` attribute should be used sparingly! Only validate known attributes; do not not warn about unknowns. The ``ignoreExtraAttributes`` attribute is a flag to the process of validating NeXus data files. By setting ``ignoreExtraAttributes="true"``, presence of any undefined attributes in this class will not generate warnings during validation. Normally, validation will check all the attributes against their definition in the NeXus base classes and application definitions. Any items found that do not match the definition in the NXDL will generate a warning message. The ``ignoreExtraAttributes`` attribute should be used sparingly! Prescribes the allowed values for ``definition`` ``type`` attribute. (This data type is used internally in the NXDL schema to define a data type.) A ``choice`` element is used when a named group might take one of several possible NeXus base classes. Logically, it must have at least two group children. NeXus base class that could be used here. The group will take the ``@name`` attribute defined by the parent ``choice`` element so do not specify the ``@name`` attribute of the group here. The name to be applied to the selected child group. None of the child groups should define a ``@name`` attribute. A group element refers to the definition of an existing NX object or a locally-defined component. Could contain these elements: * ``attribute`` * ``doc`` * ``field`` * ``group`` * ``link`` Note that a ``group`` element also includes the definitions of the ``basicComponent`` data type. (The ``groupType`` data type is used internally in the NXDL schema to define elements and attributes to be used by users in NXDL specifications.) A ``group`` may be contained within another ``group``. The ``type`` attribute *must* contain the name of a NeXus base class, application definition, or contributed definition. A particular scientific application may expect a name of a ``group`` element. It is helpful but not required to specify the ``name`` attribute in the NXDL file. It is suggested to always specify a ``name`` to avoid ambiguity. It is also suggested to derive the ``name`` from the type, using an additional number suffix as necessary. For example, consider a data file with only one ``NXentry``. The suggested default ``name`` would be ``entry``. For a data file with two or more ``NXentry`` groups, the suggested names would be ``entry1``, ``entry2``, ... Alternatively, a scientific application such as small-angle scattering might require a different naming procedure; two different ``NXaperture`` groups might be given the names ``beam-defining slit`` and ``scatter slit``. Minimum number of times this ``group`` is allowed to be present within its parent group. Note each ``group`` must have a ``name`` attribute that is unique among all ``group`` and ``field`` declarations within a common parent group. A synomym for optional, but with the recommendation that this ``group'' be specificied. A synomym for minOccurs=0.. Maximum number of times this ``group`` is allowed to be present within its parent ``group``. Note each ``group`` must have a ``name`` attribute that is unique among all ``group`` and ``field`` declarations within a common parent ``group``. A ``groupGroup`` defines the allowed children of a ``group`` specification. Describe the purpose of this ``group``. This documentation will go into the manual. Use an ``attribute`` if additional information needs to be associated with a ``group``. Use a ``choice`` if a named group could be either of a defined list of base classes. A ``group`` may contain ``group``s. A ``group`` may contain ``field`` elements (datasets). Use a ``link`` to refer locally to information placed elsewhere else in the data storage hierarchy. The ``name`` attribute uniquely identifies the element in this ``group``. The ``target`` attribute (added automatically in a data file by the NAPI) identifies the original location of this data in the data storage hierarchy. In an NXDL specification, the ``target`` attribute indicates a link to be made by the software that writes the data file. The value, as written in the NXDL file, will be a suggestion of the path to the source of the link. For example:: The value of ``target`` is written using the NeXus class names since this is a suggestion and does not actually use the element names from a particular data file. A ``basicComponent`` defines the allowed name format and attributes common to all ``field`` and ``group`` specifications. (This data type is used internally in the NXDL schema to define elements and attributes to be used by users in NXDL specifications.) Describe this ``basicComponent`` and its use. This documentation will go into the manual. The ``name`` attribute is the identifier string for this entity. It is required that ``name`` must be unique within the enclosing ``group``. The rule (``validItemName``) is defined to only allow names that can be represented as valid variable names in most computer languages. A ``field`` declares a new element in the component being defined. A ``field`` is synonymous with the HDF4 SDS (Scientific Data Set) and the HDF5 *dataset* terms. Could contain these elements: * ``attribute`` * ``dimensions`` * ``doc`` * ``enumeration`` Note that a ``field`` element also includes the definitions of the ``basicComponent`` data type. (The ``fieldType`` data type is used internally in the NXDL schema to define elements and attributes to be used by users in NXDL specifications.) dimensions of a data element in a NeXus file attributes to be used with this field A field can specify which values are to be used String describing the engineering units. The string should be appropriate for the value and should conform to the NeXus rules for units. Conformance is not validated at this time. Descriptive name for this field (may include whitespace and engineering units). Often, the long_name (when defined) will be used as the axis label on a plot. Presence of the ``signal`` attribute means this field is an ordinate. Integer marking this field as plottable data (ordinates). The value indicates the priority of selection or interest. Some facilities only use ``signal=1`` while others use ``signal=2`` to indicate plottable data of secondary interest. Higher numbers are possible but not common and interpretation is not standard. A field with a ``signal`` attribute should not have an ``axis`` attribute. NOTE: Use of this attribute is discouraged. It is for legacy support. You should use the axes attribute on the NXdata group instead. Presence of the ``axes`` attribute means this field is an ordinate. This attribute contains a colon (or comma in legacy files) delimited list of the names of independent axes when plotting this field. Each name in this list must exist as a field in the same group. (Optionally, the list can be enclosed by square brackets but this is not common.) The regular expression for this rule is:: [A-Za-z_][\w_]*([ :][A-Za-z_][\w_]*)* NOTE: Use of this attribute is discouraged. It is for legacy support. You should use the axes attribute on the NXdata group instead. Presence of the ``axis`` attribute means this field is an abcissa. The attribute value is an integer indicating this field as an axis that is part of the data set. The data set is a field with the attribute ``signal=1`` in the same group. The value can range from 1 up to the number of independent axes (abcissae) in the data set. A value of ``axis=1``" indicates that this field contains the data for the first independent axis. For example, the X axis in an XY data set. A value of ``axis=2`` indicates that this field contains the data for the second independent axis. For example, the Y axis in a 2-D data set. A value of ``axis=3`` indicates that this field contains the data for the third independent axis. For example, the Z axis in a 3-D data set. A field with an ``axis`` attribute should not have a ``signal`` attribute. Integer indicating the priority of selection of this field for plotting (or visualization) as an axis. Presence of the ``primary`` attribute means this field is an abcissa. Defines the type of the element as allowed by NeXus. See :ref:`here<Design-DataTypes>` and :ref:`elsewhere<nxdl-types>` for the complete list of allowed types. Defines the minimum number of times this ``field`` may be used. Its value is confined to zero or greater. Must be less than or equal to the value for the "maxOccurs" attribute. A synomym for optional, but with the recommendation that this ``field'' be specificied. A synomym for minOccurs=0.. Defines the maximum number of times this element may be used. Its value is confined to zero or greater. Must be greater than or equal to the value for the "minOccurs" attribute. A value of "unbounded" is allowed. The ``stride`` and ``data_offset`` attributes are used together to index the array of data items in a multi-dimensional array. They may be used as an alternative method to address a data array that is not stored in the standard NeXus method of "C" order. The ``stride`` list chooses array locations from the data array with each value in the ``stride`` list determining how many elements to move in each dimension. Setting a value in the ``stride`` array to 1 moves to each element in that dimension of the data array, while setting a value of 2 in a location in the ``stride`` array moves to every other element in that dimension of the data array. A value in the ``stride`` list may be positive to move forward or negative to step backward. A value of zero will not step (and is of no particular use). See https://support.hdfgroup.org/HDF5/Tutor/phypereg.html or *4. Dataspace Selection Operations* in https://portal.hdfgroup.org/display/HDF5/Dataspaces The ``stride`` attribute contains a comma-separated list of integers. (In addition to the required comma delimiter, whitespace is also allowed to improve readability.) The number of items in the list is equal to the rank of the data being stored. The value of each item is the spacing of the data items in that subscript of the array. The ``stride`` and ``data_offset`` attributes are used together to index the array of data items in a multi-dimensional array. They may be used as an alternative method to address a data array that is not stored in the standard NeXus method of "C" order. The ``data_offset`` attribute determines the starting coordinates of the data array for each dimension. See https://support.hdfgroup.org/HDF5/Tutor/phypereg.html or *4. Dataspace Selection Operations* in https://portal.hdfgroup.org/display/HDF5/Dataspaces The ``data_offset`` attribute contains a comma-separated list of integers. (In addition to the required comma delimiter, whitespace is also allowed to improve readability.) The number of items in the list is equal to the rank of the data being stored. The value of each item is the offset in the array of the first data item of that subscript of the array. This instructs the consumer of the data what the last dimensions of the data are. It allows plotting software to work out the natural way of displaying the data. For example a single-element, energy-resolving, fluorescence detector with 512 bins should have ``interpretation="spectrum"``. If the detector is scanned over a 512 x 512 spatial grid, the data reported will be of dimensions: 512 x 512 x 512. In this example, the initial plotting representation should default to data of the same dimensions of a 512 x 512 pixel ``image`` detector where the images where taken at 512 different pressure values. In simple terms, the allowed values mean: * ``scaler`` = 0-D data to be plotted * ``spectrum`` = 1-D data to be plotted * ``image`` = 2-D data to be plotted * ``vertex`` = 3-D data to be plotted This interprets the name attribute as: * ``specified`` = use as specified * ``any`` = can be any name not already used in group Any new group or field may expect or require some common attributes. .. Could contain these elements: * ``doc`` * ``enumeration`` (This data type is used internally in the NXDL schema to define elements and attributes to be used by users in NXDL specifications.) Description of this ``attribute``. This documentation will go into the manual. An enumeration specifies the values to be used. dimensions of an attribute with data value(s) in a NeXus file Name of the attribute (unique within the enclosing group). Type of the attribute. For ``group`` specifications, the class name. For ``field`` or ``attribute`` specifications, the NXDL field type. Is this attribute *optional* (if **true**) or *required* (if **false**)? A ``nonNegativeUnbounded`` allows values including all positive integers, zero, and the string ``unbounded``. (This data type is used internally in the NXDL schema to define a data type.) A link to another item. Use a link to avoid needless repetition of information. (This data type is used internally in the NXDL schema to define elements and attributes to be used by users in NXDL specifications.) Declares the absolute HDF5 address of an existing field or group. The target attribute is added for NeXus to distinguish the HDF5 path to the original dataset. Could contain these elements: * ``doc`` Matching regular expression:: (/[a-zA-Z_][\w_]*(:[a-zA-Z_][\w_]*)?)+ For example, given a ``/entry/instrument/detector/polar_angle`` field, link it into the ``NXdata`` group (at ``/entry/data/polar_angle``). This would be the NeXus data file structure:: /: NeXus/HDF5 data file /entry:NXentry /data:NXdata /polar_angle:NX_NUMBER @target="/entry/instrument/detector/polar_angle" /instrument:NXinstrument /detector:NXdetector /polar_angle:NX_NUMBER @target="/entry/instrument/detector/polar_angle" Group attribute that provides a URL to a group in another file. More information is described in the *NeXus Programmers Reference*. http://manual.nexusformat.org/pdf/NeXusIntern.pdf NXDL allows for documentation on most elements using the ``doc`` element. The documentation is useful in several contexts. The documentation will be rendered in the manual. Documentation, is provided as tooltips by some XML editors when editing NXDL files. Simple documentation can be typed directly in the NXDL:: Descriptive name of sample This is suitable for basic descriptions that do not need extra formatting such as a bullet-list or a table. For more advanced control, use the rules of restructured text, such as in the :ref:`NXdetector` specification. Refer to examples in the NeXus base class NXDL files such as :ref:`NXdata`. Could contain these elements: * *any* (This data type is used internally in the NXDL schema to define elements and attributes to be used by users in NXDL specifications.) Note: For documentation of ``definition`` elements, the first line of text in a ``doc`` is used as a summary in the manual. Follow the pattern as shown in the base class NXDL files. Each ``symbol`` has a ``name`` and optional documentation. Please provide documentation that indicates what each symbol represents. For example:: number of reflecting surfaces number of wavelengths Describe the purpose of this list of ``symbols``. This documentation will go into the manual. When multiple ``field`` elements share the same dimensions, such as the dimension scales associated with plottable data in an ``NXdata`` group, the length of each dimension written in a NeXus data file should be something that can be tested by the data file validation process. Describe the purpose of the parent ``symbol``. This documentation will go into the manual. Mnemonic variable name for this array index symbol. An ``enumeration`` restricts the values allowed for a specification. Each value is specified using an ``item`` element, such as: ````. Could contain these elements: * ``doc`` * ``item`` (This data type is used internally in the NXDL schema to define elements and attributes to be used by users in NXDL specifications.) :: source operating mode for storage rings for storage rings One of the prescribed values. Use the ``value`` attribute. Defines the value of one selection for an ``enumeration`` list. Each enumerated item must have a value (it cannot have an empty text node). Individual items can be documented but this documentation might not be printed in the *NeXus Reference Guide*. The value of ``value`` of an ``enumItem`` is defined as an attribute rather than a name. dimensions of a data element in a NeXus file (This data type is used internally in the NXDL schema to define elements and attributes to be used by users in NXDL specifications.) Documentation might be necessary to describe how the parts of the ``dimensions`` element are to be used. Specify the parameters for each index of the ``dimensions`` element with a ``dim`` element. The number of ``dim`` entries should be equal to the ``rank`` of the array. For example, these terms describe a 2-D array with lengths (``nsurf``, ``nwl``): .. code-block:: xml :linenos: The ``value`` attribute is used by NXDL and also by the NeXus data file validation tools to associate and coordinate the same array length across multiple fields in a group. Number or symbol indicating which axis (subscript) is being described, ranging from 1 up to ``rank`` (rank of the data structure). For example, given an array ``A[i,j,k]``, ``index="1"`` would refer to the ``i`` axis (subscript). (``NXdata`` uses ``index="0"`` to indicate a situation when the specific index is not known *a priori*.) Integer length (number of values), or mnemonic symbol representing the length of this axis. Deprecated: 2016-11-23 telco (https://github.com/nexusformat/definitions/issues/330) The dimension specification is the same as that in the ``ref`` field, specified either by a relative path, such as ``polar_angle`` or ``../Qvec`` or absolute path, such as ``/entry/path/to/follow/to/ref/field``. Deprecated: 2016-11-23 telco (https://github.com/nexusformat/definitions/issues/330) The dimension specification is the same as the ``refindex`` axis within the ``ref`` field. Requires ``ref`` attribute to be present. The dimension specification is related to the ``refindex`` axis within the ``ref`` field by an offset of ``incr``. Requires ``ref`` and ``refindex`` attributes to be present. This dimension is required (true: default) or not required (false). The default value is ``true``. When ``required="false"`` is specified, all subsequent ``<dim`` nodes (with higher ``index`` value) **must** also have ``required="false"``. Rank (number of dimensions) of the data structure. Value could be either an unsigned integer or a symbol as defined in the *symbol* table of the NXDL file. For example: ``a[5]`` has ``rank="1"`` while ``b[8,5,6,4]`` has ``rank="4"``. See https://en.wikipedia.org/wiki/Rank_%28computer_programming%29 for more details.