Information technology — SEDRIS —
Part 3: Transmittal format binary encoding

4 Concepts

4.1 Introduction

4.1.1 Description

The abstract transmittal format is the standard interchange mechanism for transmittals as defined in Part 2 of ISO/IEC 18023. The abstract transmittal format is realized by specifying an encoding that maps the elements of the abstract transmittal format into specific elements in two or more files. This part of ISO/IEC 18023 defines a binary encoding that conforms to the abstract transmittal format. The name of this binary encoding is SEDRIS Transmittal Format (STF).

4.1.2 Topics

Table 4.1 lists the topics for this clause.

Table 4.1 — Topics

4 Concepts

4.1 Introduction

4.1.1 Description

4.1.2 Topics

4.2 Purpose

4.3 Transmittal structure

4.3.1 Overview

4.3.2 STF root file

4.3.3 STF object file

4.3.3.1 STF object file structure

4.3.3.2 STF block

4.3.3.3 Encoding of DRM objects

4.3.3.4 DRM object data and DRM object referencing

4.3.4 STF bulk data file

4.3.4.1 Overview

4.3.4.2 Data table data storage

4.3.4.2.1 Organization

4.3.4.2.2 Partitioning the data

4.3.4.2.3 Block data format and packing

4.2 Purpose

STF is a file format that defines the syntax of a binary-encoded transmittal.

4.3 Transmittal structure

4.3.1 Overview

A transmittal encoded using the techniques defined in this part of ISO/IEC 18023 consists of a set of files that together are termed an STF-encoded transmittal. An STF-encoded transmittal shall contain:

  1. exactly one STF-encoded transmittal root file,
  2. one or more STF-encoded transmittal DRM object files, and
  3. zero or more STF-encoded transmittal:
    1.  image data files,
    2. mesh face table data files, and/or
    3. data table data files.

    These files are termed bulk files and are specified in 4.3.4 STF bulk data file.

Each of these is described below.

4.3.2 STF root file

An STF root file contains information that pertains to the entire transmittal including support for ITR. It has the general structure shown in Figure 4.1:

root file header

master file list

ITR referenced transmittal list

ITR referenced DRM object list

published DRM object list

Figure 4.1 — STF root file structure

The root file header contains transmittal level information and fields to validate and identify the STF format and version of the transmittal.

The master file list specifies all the content files that are in this transmittal.

The ITR referenced transmittal list is a list of the names of all transmittals referenced from this transmittal.

The ITR referenced DRM object list contains an entry for every unique ITR reference within this transmittal. Each entry identifies the transmittal and published name of the DRM object being referenced.

The Published DRM object list is a list of DRM objects in the transmittal that may be referenced from other transmittals.

The exact form for each element in the STF root file is specified in 6.3.1 Root file content organization.

4.3.3 STF object file

4.3.3.1 STF object file structure

The STF object files contain all of the DRM objects in the transmittal including field data and relationships. Each STF object file has the structure depicted in Figure 4.2:

file header

block 0

block 1

● ● ●

block N < 4096

referenced file list

block list

compressed block list

Figure 4.2 — STF object file structure

The file header contains fields to validate the STF format and identify the version of the data as specified in 6.3.2.2 File header. It also contains organizational information.

Following the file header are from one to 4096 STF blocks as described in 4.3.3.2 STF block and specified in 6.3.3 STF block.

The referenced file list is a list of the STF object files and STF bulk data files that contain an DRM object that is referenced from an DRM object within this file. The referenced file list is specified in 6.3.2.3 Referenced file list.

The block list in which each entry specifies the storage size and location of each STF block in the file as specified in 6.3.2.4 Block list.

The compressed block list is a list in which each entry specifies the uncompressed size of each compressed STF block in the file as specified in 6.3.2.4 Block list. A value of zero is specified for STF blocks that are not compressed.

STF blocks that are compressed shall be compressed according to the GZIP specification (see RFC 1952).

4.3.3.2 STF block

Each STF object file contains up to 4096 STF blocks. Each STF block is then used to store and organize up to 256 DRM objects. Each DRM object is given an index from 0 to 255 to locate it within the STF block. The layout of the STF block is depicted in Figure 4.3:

STF block header

Object type list

Object data

Object pointer list

Figure 4.3 — STF block layout

STF block header specifies the number of objects in the STF block and the location of the object pointer table in the STF block as specified in 6.3.3.2 Block header.

The object type list specifies the DRM class or the special value for non-DRM objects of each object in the STF block as specified in 6.3.3.3 Object type list.

Object data contains object-specific data for each object as described in 4.3.3.3 Object data and object referencing and specified in 6.3.3.4 Object data.

The object pointer list specifies the location of each object in the STF block as specified in 6.3.3.5 Object pointer list. It also indicates which objects have been deleted from the transmittal.

4.3.3.3 Encoding of DRM objects

Each DRM object in a transmittal is encoded according to the requirements of its corresponding DRM class specified in Part 1 of ISO/IEC 18023. Table 4.2 specifies how each aspect of the DRM class specification is encoded for DRM objects of that DRM class.

Table 4.2 — Encoding of DRM objects

Property Object encoding

Class

See 6.3.3.3 Object type table

Superclass

None

Subclass

None

Definition

None

Class diagram

None

Inherited field elements

See 6.2.7 <object field>

Field elements

See 6.2.7 <object field>

Default field values

None

Associated to (one-way) (inherited)

See 6.2.5 <associate list> and <associate count>

Associated to (one-way)

See 6.2.5 <associate list> and <associate count>

Associated by (one-way) (inherited)

None

Associated by (one-way)

None

Associated with (two-way) (inherited)

See 6.2.5 <associate list> and <associate count>

Associated with (two-way)

See 6.2.5 <associate list> and <associate count>

Composed of (two-way) (inherited)

See 6.2.6 <component list> and <component count>

Composed of (two-way)

See 6.2.6 <component list> and <component count>

Composed of (two-way) (metadata) (inherited)

See 6.2.6 <component list> and <component count>

Composed of (two-way) (metadata)

See 6.2.6 <component list> and <component count>

Component of (two-way) (inherited)

See 6.2.4 <aggregate list> and <aggregate count>

Component of (two-way)

See 6.2.4 <aggregate list> and <aggregate count>

Constraints

None

Clarifications

None

Example(s)

None

This part of ISO/IEC 18023 implements the constructs specified in 5.2.9 DRM objects of Part 2 of ISO/IEC 18023 as specified in Table 4.3.

Table 4.3 — Mapping of abstract transmittal format constructs

Abstract transmittal format construct Transmittal format binary encoding construct

<FIELD>

<object field> (see 6.2.7 <object field>)

<DRM Object Reference>

<object reference> (see 6.2.8 Encoding of DRM object reference lists)

<IMAGE DATA>
(for DRM_Image)

IMAGE_DATA (see 6.3.4.2 IMAGE_DATA bulk data object)

<MESH FACE TABLE DATA>
(for DRM_Mesh_Face_Table>)

MESH_FACE_TABLE_DATA and MESH_FACE_TABLE_PARAMETER_DATA (see 6.3.4.3 MESH_FACE_TABLE_DATA and MESH_FACE_TABLE_PARAMETER_DATA bulk data objects>

<DATA TABLE DATA>
(for DRM_Data_Table)

DATA_TABLE_ROOT_DATA (see 6.3.4.4 DATA_TABLE_ROOT_DATA bulk data object),

DATA_TABLE_BLOCK_DATA (see 6.3.4.5 DATA_TABLE_BLOCK_DATA bulk data object), and

DATA_TABLE_BLOCK_PARAMETER_DATA (see 6.3.4.6 DATA_TABLE_BLOCK_PARAMETER_DATA bulk data object)

4.3.3.4 DRM object data and DRM object referencing

STF object data consists of the association, component, and aggregation relationships for the object as well as any DRM field data the formats of which are specified in 5 Encoding of data types. The overall format of STF object data is defined in 6.2 DRM object syntax and encoding. Each relationship is implemented by an object reference.

For the aggregation relationship, which is always a two-way relationship, STF stores each object’s parents and children, if any, separately. The parents, if any, are stored as aggregate object references. The children, if any, are stored as component object references. The associated objects, if any, are stored as associate object references.

To implement these DRM object references, STF uniquely identifies every DRM object in the transmittal. STF uses a set of three zero-based indices for this purpose. Every DRM object has a reference called an FBO that is comprised of a File index (F), a Block index (B), and an Object index (O). The file index is the index into the STF root file’s master file table. This table specifies in which file the DRM object is located. The block index is the index into the file’s block table that specifies in which block the DRM object is located within the file identified by F. The object index identifies the specific DRM object within the block identified by B.

The encoding of an FBO for referencing DRM objects from other DRM objects is specified in 6.2.8 Encoding of object reference lists. This technique for referencing objects is also used for referencing image data, data table data, and mesh face table data as described in 4.3.4.1 Overview. The encoding of items in the published DRM object table and for the reference to the root DRM object in the root file header is specified in 5.2.5.4 STF_FBO.

4.3.4 STF bulk data file

4.3.4.1 Overview

DRM_Image, DRM_Mesh_Face_Table, and DRM_Data_Table DRM objects have non-field data associated with the DRM object. Such data is stored in the STF bulk data files, as bulk data objects.

The STF bulk data files are structurally identical to the STF object files with the exception that bulk data objects are stored instead of DRM objects. An STF bulk data file has a file header, reference file table and block table of the same format as the STF object file (see 4.3.3 STF object file). The format of the block header, object type table and object pointer table is also the same.  However, the object types are limited to one of only six types of bulk data objects:

  1. IMAGE_DATA,
  2. MESH_FACE_TABLE_DATA,
  3. MESH_FACE_TABLE_PARAMETER_DATA,
  4. DATA_TABLE_ROOT_DATA,
  5. DATA_TABLE_BLOCK_DATA, or
  6. DATA_TABLE_BLOCK_PARAMETER_DATA.

The format of the bulk data objects is defined in 6.3.4 STF bulk data files.

The bulk data objects also use the same FBO referencing mechanism as follows:

  1. A DRM_Image DRM object contains component references to one or more IMAGE_DATA bulk data objects that are used to locate the image data for the DRM_Image DRM object.
  2. A DRM_Mesh_Face_Table DRM object contains component references to one or more MESH_FACE_TABLE_DATA bulk data object that are used to locate the mesh face table data for the DRM_Mesh_Face_Table DRM object.
  3. A DRM_Data_Table DRM object contains component references to one or more DATA_TABLE_ROOT_DATA bulk data objects.

While STF treats these as components, they do not represent component relationships as defined in the DRM.

4.3.4.2 Data table data storage

4.3.4.2.1 Organization

A DRM_Data_Table DRM object represents an n-dimensional grid of cells. Each cell may contain any number of elements. The number, size and extents of the cells of a DRM_Data_Table DRM object are defined by its ordered set of DRM_Axis components. The number, meaning, and data type of each of the elements of a cell are defined by the ordered DRM_Table_Property_Description components of the DRM_Data_Table DRM object. Detailed information about data tables may be found  in Part 1 of ISO/IEC 18023.

One DATA_TABLE_ROOT_DATA bulk data object exists for each element in the data table. The DATA_TABLE_ROOT_DATA bulk data object serves to organize the storage of the data values for the element. The data is stored in DATA_TABLE_BLOCK_DATA objects as components of the DATA_TABLE_ROOT_DATA objects. A DATA_TABLE_BLOCK_PARAMETER_DATA bulk data object is referenced as a link to this component relationship. The DATA_TABLE_BLOCK_PARAMETER_DATA bulk data object stores information about the type, extents, and storage parameters of the data contained in the DATA_TABLE_BLOCK_DATA bulk data object.

4.3.4.2.2 Partitioning the data

Each DATA_TABLE_BLOCK_DATA contains data for a given subextent of the data table. There is no set value on the number or size of the blocks or on the subextents partitioning the data. A DATA_TABLE_BLOCK_PARAMETER_DATA bulk data object defines the subextents and the number of cells for the corresponding DATA_TABLE_BLOCK_DATA bulk data object. For an n-dimensional data table, each DATA_TABLE_BLOCK_DATA bulk data object represents an n-dimensional subextent. This set of subextents shall completely partition the extents of the data table so that each element in the data table is in one and only one DATA_TABLE_BLOCK_DATA subextent.

The DATA_TABLE_BLOCK_DATA stores an n-dimensional array of values termed block data values. The format of the block data values is discussed in 4.3.4.2.3 Block data format and packing. The size of each dimension in this array is determined from the first_index and last_index values of the subextent’s corresponding dimension as specified in the corresponding DATA_TABLE_PARAMETER_DATA bulk data object.

EXAMPLE  Figure 4.4 depicts a data table with four subextents of varying sizes:

3D Data Table Example

Figure 4.4 — Three-dimensional data table example

This data table instance has been partitioned into four data blocks with the subextents shown in Table 4.4:

Table 4.4 — Three-dimensional data table example subextents

 

Axis 0
(first, last)

Axis 1
(first, last)

Axis 2
(first, last)

block data value array
dimensions
(axis0,axis1,axis2)

Block 0

0, 49

0, 29

0, 24

(50,30,25)

Block 1

50, 79

0, 29

0, 24

(30,30,25)

Block 2

0, 49

30, 49

0, 24

(50,20,25)

Block 3

50, 79

30, 49

0, 24

(30,20,25)

The block data values in the n-dimensional array for a specific block are sequenced such that the block data values along the last axis are stored in order, followed by the block data values of the penultimate axis continuing until the block data values of the first axis have been stored.

For the three-dimensional table example above, the ith element in block data value is specified for each block by:

Block 0:  (x ´ 30 ´ 25) + (y ´ 25) + z
Block 1:  (x ´ 30 ´ 25) + (y ´ 25) + z
Block 2:  (x ´ 20 ´ 25) + (y ´ 25) + z
Block 3:  (x ´ 20 ´ 25) + (y ´ 25) + z

where (x,y,z) is the coordinate of the block data value in the three-dimensional array.

4.3.4.2.3 Block data format and packing

The element values within the DATA_TABLE_BLOCK_DATA bulk data object may be packed or not packed. If not packed, the block data values represent the actual element values and are encoded based on their data type. Encoding of data types is specified in 5 Encoding of data types.

For integer and float data types, the element values may be packed. In this case, the block data values are encoded as bit fields representing unsigned numbers and are used to calculate the element values. The size of these bit fields for a given block may vary depending on the data type and the element values in the block. These block data value bit fields are encoded one after another with no padding and without respect to any word boundaries. This is depicted in Figure 4.5 for four block data values with a bit field size of five represented by aaaaa, bbbbb, ccccc, and ddddd:

7

6

5

4

3

2

1

0

a

a

a

a

a

b

b

b

7

6

5

4

3

2

1

0

b

b

c

c

c

c

c

d

7

6

5

4

3

2

1

0

d

d

d

d

Figure 4.5 — Block_data_value five-bit fields

The DATA_TABLE_BLOCK_PARAMETER_DATA bulk data object (see 6.3.4.6 DATA_TABLE_BLOCK_PARAMETER_DATA bulk data object) specifies the parameters needed to calculate the element values from the block data values. The packing parameters include a base_data_value, a scale_factor, a tolerance (for floating point values) and possibly a list of sentinel value mappings used to encode element values that represent  metadata values in the DRM_Data_Table DRM object.

The base_data_value packing parameter is of the same type as the element. For integer types, the base_data_value is compared to each of the mapped_sentinel_values and, if found, the corresponding sentinel_value is used.. For floating point types, the base_data_value is the largest multiple of the tolerance that is less than the minimum element value.

The tolerance packing parameter is only used for floating point types.  Element values are encoded as multiples of tolerance. The meaning of tolerance is the same as the EDCS_Value_Characteristic code EVC_TOLERANCE as specified in Table 6.33 of ISO/IEC 18025. Packing floating point data will necessarily result in a loss of precision for the element values that are being encoded. For this reason, if a tolerance is to be provided, it shall be specified in the transmittal  in the DRM_Property_Characteristic component of the DRM_Table_Property_Description DRM object corresponding to the element. If not specified, tolerance is undefined and floating point values are packed at full resolution.

The fields of a DRM_Property_Characteristic have an EDCS_Value_Characteristic code and a value representing a sentinel for that code. The concept of DRM_Property_Characteristic is described in 4.8.1.3 <DRM Property Characteristic> of Part 1 of ISO/IEC 18023.

EXAMPLE   Let the EDCS_Value_Characteristic code be EVC_MISSING with a corresponding sentinel value of -1. If an element has a value of -1 within a cell, the meaning is that the element data is missing for that cell.

Since sentinel values are usually values outside the normal range (e.g., -1 or 65535), it is more efficient to encode a mapped_sentinel value in the block data value bit field. The sentinel value mappings that are specified in the DATA_TABLE_BLOCK_PARAMETER_DATA bulk data object define each sentinel value with its corresponding mapped value. It is the mapped value that is stored in the block data value bit field. Since these mapped sentinel values cannot correspond to values that yield actual element values, an extra bit may need to be allocated when determining the size of the bit field for the block data value. There is no requirement that the sentinel of the DRM_Property_Characteristic DRM object be given a mapped value. Specifying a mapped value increases the efficiency of the packing.

For integer types, the block data value is compared to each sentinel in the list of sentinels. If found, the corresponding sentinel value is used.  If not found, element_value is computed as follows:

element_value = (block_data)value × 2scale_factor) + base_data_value

For floating point types, the block data value is compared to each sentinel in the list of sentinels. If found, the corresponding sentinel value is used. If not found, element_value is computed as follows:

element_value = (block_data_value × 2scale_factor) × tolerance + base_data_value

http://standards.iso.org/ittf/PubliclyAvailableStandards/ISO_IEC_18023-3_Ed1.html