.if n \{\
.ND
.po 0
.pl 99i
.ll 7.8i
.nr LL 7.8i
.nh
.na
.de ts
.\"
..
.de PT
.\"
..
.de BT
.\"
..
.de ul
.\"
..  \}

.if t \{\
.de ts
.ds CH \\$1
.ds RH %
.ds LH Students of Prof. W. Kahan
.ds LF \\$3
.ds CF
.ds RF \\$2
.TL
\\$1
.AU
Students of Prof. W. Kahan
.. \}

.de p
.PP
..
.ts "UCBTEST README" "95/03/08" "readme 1.46"
.if n \{\
.ll 7.8i
.pl 99i
.nh
.na
.nr LL 7.8i
.de PT
.\"
..
.de BT
.\"
..
.de ul
.\"
..  \}
.AB
UCBTEST is a suite of programs for testing certain difficult
cases of IEEE 754 floating-point arithmetic.
While most of the source code is NOT public domain, the usage restrictions
are modest; please read and observe the copyright notices in the source
code modules that have them.
Please report any porting problems or logic errors
to fdlibm-comments@sunpro.sun.com.
This readme may be formatted with commands like "tbl readme | troff -ms".
.AE
.PP
UCBTEST is a suite of programs for testing certain difficult
cases of IEEE 754 floating-point arithmetic.
Some of the difficult test cases are obtained from number-theoretic
algorithms developed by Turing Award winner
Prof. W. Kahan, Department of Electrical Engineering and Computer Science,
University of California, Berkeley, as part of ongoing research into
test methods for computer arithmetic.
.p
After Prof. Kahan presented a
Floating-Point Indoctrination lecture
series at Sun Microsystems in 1988,
some lecture attendees and former graduate students of Prof. Kahan wrote
programs to test
the correctness of alleged IEEE 754 implementations of multiplication,
division, and square root.
The suite has subsequently expanded to include the following:
.PP
ucbmultest - by number-theoretic means,
generates multiplication test cases that are exactly halfway
and nearly halfway between representable numbers.
The test algorithm is in ucbtest/mul.c.
.PP
ucbdivtest - by number-theoretic means,
generates division test cases nearly halfway between representable
numbers.
Exactly halfway cases don't arise in
division.
The test algorithm is in ucbtest/div.c.    
.PP
ucbsqrtest - by number-theoretic means,
generates sqrt test cases nearly halfway between representable
numbers.    
Exactly halfway cases don't arise in sqrt.
The test algorithm is in ucbtest/sqr.c.
.PP
ucbpitest - attempts to determine a rational approximation A/B close to
a system's approximate value of \(*p used in
trigonometric argument reduction, and tests whether the system's
approximation is used consistently.   
ucbctest/pi.c is a complete translation into C
of ucbftest/pi.F, derived from the pirats program.
.PP
ucbpartest - attempts to determine general properties of computer arithmetic
by ostensibly portable means.   ucbftest/par.F and ucbctest/par.c are
complete translations from the original Basic program of Kahan.
After reworking by various collaborators,
ucbpartest's current pass/fail criterion
reflects one particular set of prejudices 
about x**0 and double rounding during underflow; 
others may prefer different choices.   
.PP
ucbfindpitest - determines an approximation to a system's approximate value of
\(*p used in trigonometric argument reduction,
suitable for use in ucbeef/local.h.
.PP
ucbeefscan - a postprocessor that summarizes ucbeeftest's detailed output.
.PP
ucbeeftest - programs for testing sin, cos, atan, exp, expm1, log, and log1p.
Random arguments test the functions for comparison to
independently computed values accurate to better than test function precision.
In UCBTEST, the program is compiled and run separately for each function.
The testing interface is thus
somewhat different from the original form of the program described in its
README.
Note the usage restrictions set forth in that 
README and source code.
.PP
ucblibtest -  hex test vectors for
library functions in single, double, and quad precision.
Rounding precision control is not yet implemented, so some x86 double
test cases misleadingly appear to fail by one ulp,
due to double rounding.
x86 and m68k double extended are not supported.   Considerable additional
work remains to be done on this program.
The source codes provided in UCBTEST may be helpful now,
but are subject to ongoing improvement.
.SH
Pentium flaw
.PP
An instance of the infamous Pentium FDIV flaw can be found in less than
a minute  
by compiling ucbdivtest in single precision with
-DNTESTS=1000000 -DSP etc.
That finding
is unexpected since the program was not designed to seek out
such gross errors inherent in a particular division method, but rather
looks for least-significant-bit errors independent of division method.
Running in double or extended precision,
ucbdivtest does not detect the Pentium bug
in a reasonable amount of time.
.SH
Porting
.PP
All the UCBTEST programs are intended for users with a good understanding
of IEEE 754 arithmetic and C.
Some sample *.sh scripts in ucbREADME/
compile and run all the programs,
for three floating-point types: IEEE 754 single, double, and double-extended
(including quad).
The test programs are designed to terminate with non-zero exit status in
case they find an error, unless compiled with -DNO_FAIL, which causes
zero exit status even in error cases.
At the end, the scripts summarize run-time failures indicated by UCBFAIL
in the output or by the existence of incomplete *.output.tmp files.
The criterion for ucblibtest is that all test vectors succeed; the
criterion for ucbeeftest is that no monotonicity or symmetry failures
occur and that the worst case errors are less than one ulp.
These summaries don't catch compile and link errors - those errors terminate
Makefiles and scripts unless make -k is used.
.PP
ucbmultest, ucbdivtest, ucbsqrtest, and ucblibtest were developed on 
SPARCstations running SunOS 4 or SunOS 5, and on x86 PC's running 
SunOS 5.   ucbdivtest and ucbsqrtest may be ported to other RISC/Unix
systems with little difficulty.   Systems with extended precision,
non-Unix systems, and ucbmultest
require more effort to provide the equivalents of
the Sun functions ieee_flags etc.
.PP
On SPARCstations, "int" and "long" and Fortran
"INTEGER" are 32 bits wide.
On DOS systems, "int" is usually 16 and "long" 32.
On some supercomputers, "int" and "long" might be 64.
In those cases, these definitions in libctest and ucbtest may require
modification:
.nf

libctest.h: #define INTEGERTYPE int
libctest.h: #define INTEGER INTEGERTYPE *

ucbtest.h: #define FINT	   int             /* C equivalent of Fortran integer */
ucbtest.h: #define INT32   int             /* 32 bit int */
ucbtest.h: #define UINT32  unsigned int    /* 32 bit unsigned */

.fi
.PP
Some Fortran compilers append underscores to subprogram identifiers -
in order to avoid name space
conflicts when C and Fortran programs are mixed - and
some do not.   Since UCBTEST depends on that underscore differentiation,
aix.sh for instance specifies "-DUNDERSCORE=_" in $cppopts, and
libftest/libftest.h and ucbftest/ucbftest.h provide definitions like
.nf

#if defined(__hpux) || defined(_AIX)
#define UNDERSCORE _
#else
#define UNDERSCORE
#endif

.fi
HPUX and AIX require the explicit underscore because they don't provide
it by default.
.SH
Misapplication
.PP
Misapplication often causes these programs to fail,
reporting hardware or software bugs where none exist.
If misapplied - especially after porting to a new
environment for the first time - they may report arithmetic
"errors" when nothing is
amiss.   Nor do "error-free" results prove correct arithmetic, although they
may "lend artistic verisimilitude to an otherwise bald and unconvincing"
proof.
Common misapplications include:
.IP precision
On extended-precision systems such as x86 and 68k,
the extended-precision precision control must be set to the precision
of the type of floating-point variable being tested.
On Sun systems, ieee_flags("set","precision",...) is useful.
.IP prototypes
Traditional C and ANSI-C differ in their treatment of float function
arguments and function results.
.IP "traditional C expression evaluation"
Traditional C may evaluate expressions involving floats by promoting
them first to double, causing two rounding errors where UCBTEST expects one. 
Compiling with "-fsingle"
is helpful with /bin/cc on SunOS 4.
.IP optimizers
Optimizing compilers that allocate variables to extended-precision registers
and fail to round them to storage precision,
even in response to source-code assignments,
may cause problems.   Certain variables have been declared
"volatile" in hope of avoiding those problems.
Particularly on PC's, compilers seem to have trouble correctly optimizing
code and libraries to work with rounding precision set to single
instead of extended.
Perhaps more
"volatile" declarations are required.   On GCC, -ffloat-store sometimes helps.
.IP "sqrt functions"
C compilers usually don't convert calls to sqrtf, sqrt, or sqrtl to
inline machine sqrt instructions by default.   
Thus testing those instructions, rather than the subroutine library,
may require extra steps such as coding
in assembly language.    Sun compilers invoked with "-xlibmil" provide the
necessary inlining to test SPARC or x86 sqrt instructions.
.SH
Source code
.PP
Some compile-time flags used to configure the tests include:
.KS
.TS
center ;
l l .
-DNTESTS	number of test cases to try
-DDEBUG	turn on debugging output
-DSUN_IEEE	ieee_flags, ieee_retrospective available
-DSUN_MATH	sunmath.h and libsunmath.a are available
-DSUN_LIBS	Fortran-callable versions available for
	Sun libm and libsunmath functions
-DNO_FUNCF	no ANSI-C reserved functions sqrtf, expf, etc.
-DNO_FAIL	do not abort on certain errors
-DSP	test IEEE single precision
-DDP	test IEEE double precision
-DQP	test IEEE extended precision
.TE
.KE
.PP
Several of the programs are set up so that the C infrastructure may be used to
test the arithmetic and libraries of any language.
Sample wrappers are provided for C and Fortran.
Outer main programs are in ucbctest/ and ucbftest/; they use inner main
programs and utility functions
from ucbtest/ to invoke the test functions in libctest/ or libftest/.
.PP
Makefiles are provided in the ucbctest/, ucbftest/, libctest/, and libftest/
directories.  
Disable .SILENT: targets for more debugging output; enable
them for less.
The Makefiles avoid infinite loops in ucblibtest,
but aux.sh gets into infinite loops in ucbpartest and ucbmultest in single
precision.   In order to continue UCBTEST,
such looping processes need to be terminated manually with a kill command.
.PP
Sample *.sh scripts for invoking the Makefiles are in 
ucbREADME/.   The scripts define -NTESTS=1000000; reduce that
to 1000 to simplify debugging; increase it to one billion
for a more comprehensive test.
ucb{mul,div,sqr,eef}test have been adjusted so their run times
are approximately comparable for comparable NTESTS.
ucb{pi,par,findpi,lib}test run times do not depend on NTESTS.
.PP
The scripts define which precisions and which languages are to be tested,
how each language is to be compiled, and whether (via make -k) 
script/compile/link errors terminate execution.
To create a script for a new platform, start by copying one of the existing
scripts for a similar system, or start with generic.sh and modify it 
according to the suggestions in its comments until it works acceptably.
.PP
Generally speaking, the source code uses -DSP, -DDP, and -DQP to distinguish
single, double, and extended precision.
-DQP refers flexibly to double-extended on x86 and quadruple on SPARC
on all the tests except
ucblibtest, where the inputs and output test vectors are specified in hex,
so -DQP means SPARC quadruple precision, and double-extended isn't supported.
The programs start out by setting extended rounding precision on x86
to match that of the nominal precision (SP, DP, QP), but because some of the
libm functions misbehave when invoked that way, ucb{pi,findpi,eef}test
prevent that by restoring the default_precision.
.PP
The source code file ucbtest/fpce.S is intended to provide a convenient
place to integrate assembly-language implementations of the functions
to set IEEE rounding direction and precision.
sunos5.gcc.sh depends on those to provide a substitute for ieee_flags in
Sun's unbundled libsunmath.a.
Unfortunately in Solaris 2,
there's no convenient way to set const _lib_version,
defined in <math.h>, to -1 in order to
get the same effect as Sun's -xlibmieee option; this causes some ucblibtest
vectors to fail with gcc.    That can be remedied by reconfiguring GCC
to link with values-Xi.o instead of values-X?.o (see config/svr4.h) where
values-Xi.c is:
.nf

#include <math.h>
const enum version _lib_version = libm_ieee;

.fi
.SH
Directory information
.PP
ucbctest/ and ucbftest/ contain the main programs for each test, in C and
Fortran respectively.
.PP
libctest/ and libftest/ contain the test modules which invoke each function
to be tested, in C and Fortran respectively.   Each function is in a 
separate module so that compilation failure on one will not affect other
tests.   On systems which lack some common library functions, it is necessary
to improvise; see for instance libctest/testexpm1*.c.
.PP
ucbtest/ contains the C logic for the programs that can be invoked from
main programs in C or Fortran.
.PP
ucbeef/ contains various source files invoked by ucbeeftest 
and ucbfindpitest.   See the separate README.
.PP
ucblib/ contains the input files for ucblibtest.
.PP
ucbREADME/ contains this readme and also readme.about.testing
that discusses the limitations of various testing approaches.
The various *.sh sample test scripts are here, along with logs of typical
executions.
.PP
The sc301.sh script for 
Sun's SC3.0.1 compilers on SunOS 5.3 passes
the largest number of successful tests of any of these sample scripts,
not surprisingly because predecessors of UCBTEST have been part of Sun's
compiler test regime for years.
.PP
ucbDOS contains Warren Ferguson's adapted source codes and compiled
executables for MSDOS.   See the separate readme.
.KS
.SH
Results Obtained During Development
.PP
The following results are recorded in *.log files obtained with corresponding
*.sh scripts.
In many cases the hardware, operating systems, compilers,
and libraries may not be current products, and the compilation options
may not be optimal.    
The sample *.sh scripts are only starting points and may require
modification when hardware, operating systems, compilers,
or libraries change.
Many of the anomalies in the logs have not been investigated; some 
may represent problems with the hardware, operating systems, compilers,
or libraries; problems
with the UCBTEST port may underlie other anomalies, corresponding to the
various types of "misapplication" discussed above.
Contributions of improved scripts and log files
are welcome.   
.PP
The table indicates the fraction of "UCBPASS tests" / "total tests"
possible with the tested languages and precisions, and when available,
the total time in minutes, either real or user+sys, 
for the script to create the log file.   
In most cases other jobs were
also running on the test systems, so the times are very approximate.
Some of the log files were produced by
early versions of the scripts that don't report
as much information as the current versions.  
.TS
center ;
c s s s s
l l l l .
RISC systems with 113-bit extended or none
=
 .sh	hardware	c	UCBPASS
 .log	op sys	fortran	minutes
=

sc301	SS10/41	3.0.1 cc -O -xlibmil -xlibmieee ...	77/84 
		... -xcg92 -fsingle -lsunmath -lm
sc301	SunOS 5.3	3.0.1 f77 -O -xlibmil -xcg92	210 user+sys

sunos4	SPARCclassic	/bin/cc -O /usr/lib/libm.il -fsingle -lm	56/56
sunos4	SunOS 4.1.3U1	1.0 f77 -O /usr/lib/libm.il	170 user+sys

apogee30	SPARC 2000, 60MHz	3.0 apcc -O -cg92 -Dlint -lm	38/56
apogee30	SunOS 5.4	3.0 apf77 -O -cg92	49 user+sys

hpux	HP 9000/712	/bin/c89 -O -lm -DNTESTS=10000	34/70
hpux	HP-UX 9.05	/usr/bin/f77 -O -DNTESTS=10000	60 real

irix	SGI ?	/usr/bin/cc -O -lm	24/28
irix	IRIX 5.2	-	60 real

mips.ultrix.gcc	DEC mips	gcc -O -lm	21/28
mips.ultrix.gcc	Ultrix 4.1	-	?

alpha.osf.gcc	DEC alpha	gcc -O -ffloat-store -lm	11/14	
alpha.osf.gcc	OSF 3.0	-	?
.TE
.KE
.KS
.TS
center ;
c s s s s
l l l l .
CISC systems with 64-bit extended
=
 .sh	hardware	c	UCBPASS
 .log	op sys	fortran	minutes
=

pc201	Pentium 90 fixed	2.0.1 cc -O sqrt.il -xlibmieee -xpentium -fsingle -lm	54/70
pc201.fix	SunOS 5.4	2.0.1 f77 -O sqrt.il -pentium	89 user+sys

pc201	Pentium 90 flawed	2.0.1 cc -O sqrt.il -xlibmieee -xpentium -fsingle -lm	52/70
pc201.flaw	SunOS 5.4	2.0.1 f77 -O sqrt.il -pentium	88 user+sys 

linux	486 66DX2	/usr/bin/cc -O -ffloat-store -lm	44/56	
linux	slackware 1.1.59	/usr/bin/f77 -O	170 user+sys	

sunos3.fpa	Sun-3/260+FPA	/bin/cc -O -ffpa /usr/lib/ffpa.il -fsingle -lm	41/56
sunos3.fpa	SunOS 3.5.2	/usr/bin/f77 -O -ffpa /usr/lib/ffpa.il	1300 user+sys

aux	Apple Quadra 800	/bin/cc -O -lm	40/56
aux	A/UX 3.0.2	/usr/bin/f77 -O	440 real

sunos5.gcc	Pentium 90 fixed	2.6.3 gcc -O -ffloat-store -lfdm	24/28
sunos5.gcc	SunOS 5.4	-	43 real	

sunos5.fdlibm.gcc	Pentium 90 fixed	2.6.3 gcc -O -ffloat-store -lfdm	24/28
sunos5.fdlibm.gcc	SunOS 5.4	-	32 user+sys	

m68.hp.netbsd1.0a	HP m68k	gcc -O -D_POSIX_SOURCE	24/28
m68.hp.netbsd1.0a	BSD 1.0a	-	?

sunos3	Sun-3/260	/bin/cc -O -f68881 /usr/lib/f68881.il -fsingle -lm	?/56
sunos3	SunOS 3.5.2	/usr/bin/f77 -O -f68881 /usr/lib/f68881.il	?
.TE
.KE
.KS
.SH
Source Code Authors; Translators, Porters, Maintainers
.PP
ucbdivtest - K.-C. Ng; M. Alemi, S. Ma, D. Priest.
.PP
ucbeeftest - Z. A. Liu inspired in part by W. J. Cody and W. Waite's ELEFUNT.
.PP
ucbfindpitest - Z. A. Liu.
.PP
ucblibtest - K.-C. Ng.
.PP
ucbmultest - B. Toy; M. Alemi, S. Ma, M. Mueller, K.-C. Ng, D. Priest.
.PP
ucbpartest - W. Kahan inspired in part by W. J. Cody's MACHAR; 
D. Feenberg, D. Gay, R. Karpkinski,
T. Quarles, T. Sumner, G. Taylor, B. Wichmann.
.PP
ucbpitest - W. Kahan; D. Priest.
.PP
ucbsqrtest - K.-C. Ng; M. Alemi, S. Ma, D. Priest.
.PP
Z. A. Liu generously permitted distribution, as part of UCBTEST,
of a version of his Berkeley
Elementary Function tests, according to the conditions set forth in the
comments of those source codes.
.PP
Sun Microsystems generously permitted distribution, as part of UCBTEST,
of versions of certain source codes and test vectors
developed at Sun, according to the conditions set forth in the header
comments of those source codes and test vectors.
.SH
Porting and Quality Acknowledgements
.PP
David Gay improved portability to non-Solaris workstations 
running various proprietary compilers.   
Stephen Moshier improved portability to non-Solaris workstations running GCC.
Warren Ferguson improved portability to MSDOS and contributed the
ucbDOS directory.
Intel lent systems equipped with flawed and unflawed Pentiums for testing
purposes.
.SH
Problems
.PP
Please report bugs and portability 
enhancements for other systems to fdlibm-comments@sunpro.sun.com.
.SH
Futures
.PP
Enhancements planned to be incorporated some day:
.IP *
Ng's libm test vectors - ucblibtest - are still in the process of 
conversion to the UCBTEST format, and the program's tests for rounding
modes and exceptions are not yet working portably.
.IP *
Kahan's tests for SRT-PLA-style dividers.
.IP *
Coonen's IEEE Test Vectors program and data.
.IP *
Hough's tests for correctly-rounded base conversion.
.KE
