Subject: SCI Data Transfer Formats standard approved

From: [email-masked] (Dave Gustavson)
Date: Thu, 15 Jul 1993 12:56:25 -0800

In its June 1993 meeting, the IEEE Standards Board approved:

IEEE Std 1596.5-1993 Data Transfer Formats Optimized for SCI. 
(The approved document was Draft 1.0 8Dec92, but with significant edits to
clarify the vendor-dependent formats listed in the appendix.)

Congratulations to the working group, and especially to working group
chairman David James!

This new standard defines a set of data types and formats that will work
efficiently on SCI for transferring data among heterogeneous processors in
a multiprocessor SCI system.

This work has attracted much interest, even beyond the SCI community. It
solves a difficult problem that must be faced in heterogeneous systems.

Over the years a great amount of effort has been invested in translating
data among dissimilar computers. Computer-bus bridges have incorporated
byte swappers to try to handle the big-endian/little-endian conversion.
Software and hardware have been used to convert floating point formats. 

It was always tempting to have the hardware swap byte addresses to preserve
full-bus-width integers, which seem to look the same on big- and
little-endian machines, and then not swap bytes when passing character
strings. 

But finally we understood that this problem cannot be solved by the
hardware (at least until some far-future day when we all use standardized
fully tagged self-describing data structures!). 

The magnitude of the problem became clearer during work on Futurebus+,
where we had to deal with multiple bus widths and their interfaces with
other standards like VME and SCI. When you observe data flowing along paths
of various widths through a connected system, you see how hardware
byte-swappers can arbitrarily scramble the data bytes of various number
formats such as long integer or floating point. Furthermore, the scrambling
may depend on the particular path used and on the state of the bridge
hardware at the time the data passed through!

Finally the solution became clear: first, keep the relative byte address of
each component of a data item fixed as it flows through the complex system.
(This is now referred to as the "address invariance" principle.) Thus,
character strings arrive unchanged, but other data items may have been
created with their bytes in inconvenient (but well-defined) places. 

Then provide the descriptive tools needed to tell the compiler what the
original format of the data was. (That is what this standard does.) 

The compiler knows the properties of the machine for which it is compiling,
and thus now has enough information to allow it to generate code to perform
the needed conversions before trying to do arithmetic on the foreign data.
For example, when the compiler loads a long integer into a register it may
swap bytes to convert from little-endian to big-endian significance, so
that the register will contain the correct arithmetic value for use in
calculations. Similarly, when an arithmetic result is stored back into a
structure that is declared with foreign data types the compiler ensures
that the conversions are done appropriately before the data are stored.

This capability is critical for work in heterogeneous multiprocessors, but
it is also useful for interpreting data tapes or disk files that were
written on a different machine as well.

The IEEE Std 1596.5-defined descriptors include type (character, integer,
floating), sizes, alignment, endian-ness, and atomic properties (can I be
certain this long integer is always changed as a unit, never by a series of
narrower loads and stores that might allow inconsistent data to be
momentarily visible to a sharing machine).

The standard also includes a C-code test suite that can be used to check
the degree of compliance of a given implementation.

The chairman is Dr. David V. James, MS 301-4G, Apple Computer, 20525
Mariani Avenue, Cupertino, CA  95014, 408-974-1321, fax 408-974-9793,
[email-masked].

Again, my hearty congratulations on a job well done!

David Gustavson, SCI (IEEE Std 1596-1992 Scalable Coherent Interface) chair

David B. Gustavson                                      phone 415/961-3539
SCI (ANSI/IEEE Std 1596 Scalable Coherent Interface) chairman
SLAC Computation Research Group, Stanford University      fax 415/961-3530
POB 4349, MS 88, Stanford, CA 94309                  [email-masked]