C ALGORITHM 778, COLLECTED ALGORITHMS FROM ACM. C THIS WORK PUBLISHED IN TRANSACTIONS ON MATHEMATICAL SOFTWARE, C VOL. 23,NO. 4, December, 1997, P. 550--560. #! /bin/sh # This is a shell archive, meaning: # 1. Remove everything above the #! /bin/sh line. # 2. Save the resulting text in a file. # 3. Execute the file with /bin/sh (not csh) to create the files: # Doc # Drivers # Info # Src # This archive created: Fri Jul 31 08:51:37 1998 export PATH; PATH=/bin:$PATH if test ! -d 'Doc' then mkdir 'Doc' fi cd 'Doc' if test -f 'Makefile' then echo shar: will not over-write existing file "'Makefile'" else cat << \SHAR_EOF > 'Makefile' FC = epcf90 FFLAGS = -sloppy -C -d1 -g -temp=/tmp -u DRIVER1 = driver1.o DRIVER2 = driver2.o DRIVER3 = driver3.o ROUTINES = src.o blas1.o dpmeps.o linpack.o timer.o all : lbfgsb1 lbfgsb2 lbfgsb3 lbfgsb1 : $(DRIVER1) $(ROUTINES) $(FC) $(FFLAGS) $(DRIVER1) $(ROUTINES) -o x.lbfgsb1 lbfgsb2 : $(DRIVER2) $(ROUTINES) $(FC) $(FFLAGS) $(DRIVER2) $(ROUTINES) -o x.lbfgsb2 lbfgsb3 : $(DRIVER3) $(ROUTINES) $(FC) $(FFLAGS) $(DRIVER3) $(ROUTINES) -o x.lbfgsb3 SHAR_EOF fi # end of overwriting check if test -f 'Readme' then echo shar: will not over-write existing file "'Readme'" else cat << \SHAR_EOF > 'Readme' L-BFGS-B (version 2.3) April 1997 This directory contains the limited memory code for solving bound constrained optimization problems. A makefile is provided for Unix users. We recommend that the user read driver1.f, which gives a good idea of how the code works as well as directions on how to adapt the timing routine to particular systems. Two articles are also included [1] algorithm.ps - describes the algorithm [2] compact.ps - presents the compact form of matrices A description of the use of the code is appended to this file. Users interested in understanding the algorithm should read [1] and possibly also [2]. _______________________________________________________________________ For questions and help contact Ciyou Zhu or Jorge Nocedal ciyou@ece.nwu.edu nocedal@ece.nwu.edu _______________________________________________________________________ To run the code on a Unix machine simply type make This will create (among others) the file x.lbfgsb1* which is the executable file for the simplest driver in the package. One can execute the program by typing x.lbfgsb1 ************************************************************************ How to use L-BFGS-B ************************************************************************ The simplest way to use the code is to modify one of the drivers provided in the package. Most users will only need to make a few changes to the drivers to run their applications. L-BFGS-B is written in FORTRAN 77, in double precision. The user is required to calculate the function value f and its gradient g. In order to allow the user complete control over these computations, reverse communication is used. The routine setulb.f must be called repeatedly under the control of the variable task. The calling statement of L-BFGS-B is call setulb(n,m,x,l,u,nbd,f,g,factr,pgtol,wa,iwa,task,iprint, + csave,lsave,isave,dsave) Following is a description of all the parameters used in this call. c n is an INTEGER variable that must be set by the user to the c number of variables. It is not altered by the routine. c c m is an INTEGER variable that must be set by the user to the c number of corrections used in the limited memory matrix. c It is not altered by the routine. Values of m < 3 are c not recommended, and large values of m can result in excessive c computing time. The range 3 <= m <= 20 is recommended. c c x is a DOUBLE PRECISION array of length n. On initial entry c it must be set by the user to the values of the initial c estimate of the solution vector. Upon successful exit, it c contains the values of the variables at the best point c found (usually an approximate solution). c c l is a DOUBLE PRECISION array of length n that must be set by c the user to the values of the lower bounds on the variables. If c the i-th variable has no lower bound, l(i) need not be defined. c c u is a DOUBLE PRECISION array of length n that must be set by c the user to the values of the upper bounds on the variables. If c the i-th variable has no upper bound, u(i) need not be defined. c c nbd is an INTEGER array of dimension n that must be set by the c user to the type of bounds imposed on the variables: c nbd(i)=0 if x(i) is unbounded, c 1 if x(i) has only a lower bound, c 2 if x(i) has both lower and upper bounds, c 3 if x(i) has only an upper bound. c c f is a DOUBLE PRECISION variable. If the routine setulb returns c with task(1:2)= 'FG', then f must be set by the user to c contain the value of the function at the point x. c c g is a DOUBLE PRECISION array of length n. If the routine setulb c returns with taskb(1:2)= 'FG', then g must be set by the user to c contain the components of the gradient at the point x. c c factr is a DOUBLE PRECISION variable that must be set by the user. c It is a tolerance in the termination test for the algorithm. c The iteration will stop when c c (f^k - f^{k+1})/max{|f^k|,|f^{k+1}|,1} <= factr*epsmch c c where epsmch is the machine precision which is automatically c generated by the code. Typical values for factr on a computer c with 15 digits of accuracy in double precision are: c factr=1.d+12 for low accuracy; c 1.d+7 for moderate accuracy; c 1.d+1 for extremely high accuracy. c The user can suppress this termination test by setting factr=0. c c pgtol is a double precision variable. c On entry pgtol >= 0 is specified by the user. The iteration c will stop when c c max{|proj g_i | i = 1, ..., n} <= pgtol c c where pg_i is the ith component of the projected gradient. c The user can suppress this termination test by setting pgtol=0. c c wa is a DOUBLE PRECISION array of length c (2mmax + 4)nmax + 11mmax^2 + 8mmax used as workspace. c This array must not be altered by the user. c c iwa is an INTEGER array of length 3nmax used as c workspace. This array must not be altered by the user. c c task is a CHARACTER string of length 60. c On first entry, it must be set to 'START'. c On a return with task(1:2)='FG', the user must evaluate the c function f and gradient g at the returned value of x. c On a return with task(1:5)='NEW_X', an iteration of the c algorithm has concluded, and f and g contain f(x) and g(x) c respectively. The user can decide whether to continue or stop c the iteration. c When c task(1:4)='CONV', the termination test in L-BFGS-B has been c satisfied; c task(1:4)='ABNO', the routine has terminated abnormally c without being able to satisfy the termination conditions, c x contains the best approximation found, c f and g contain f(x) and g(x) respectively; c task(1:5)='ERROR', the routine has detected an error in the c input parameters; c On exit with task = 'CONV', 'ABNO' or 'ERROR', the variable task c contains additional information that the user can print. c This array should not be altered unless the user wants to c stop the run for some reason. See driver2 or driver3 c for a detailed explanation on how to stop the run c by assigning task(1:4)='STOP' in the driver. c c iprint is an INTEGER variable that must be set by the user. c It controls the frequency and type of output generated: c iprint<0 no output is generated; c iprint=0 print only one line at the last iteration; c 0100 print details of every iteration including x and g; c When iprint > 0, the file iterate.dat will be created to c summarize the iteration. c c csave is a CHARACTER working array of length 60. c c lsave is a LOGICAL working array of dimension 4. c On exit with task = 'NEW_X', the following information is c available: c lsave(1) = .true. the initial x did not satisfy the bounds; c lsave(2) = .true. the problem contains bounds; c lsave(3) = .true. each variable has upper and lower bounds. c c isave is an INTEGER working array of dimension 44. c On exit with task = 'NEW_X', it contains information that c the user may want to access: c isave(30) = the current iteration number; c isave(34) = the total number of function and gradient c evaluations; c isave(36) = the number of function value or gradient c evaluations in the current iteration; c isave(38) = the number of free variables in the current c iteration; c isave(39) = the number of active constraints at the current c iteration; c c see the subroutine setulb.f for a description of other c information contained in isave c c dsave is a DOUBLE PRECISION working array of dimension 29. c On exit with task = 'NEW_X', it contains information that c the user may want to access: c dsave(2) = the value of f at the previous iteration; c dsave(5) = the machine precision epsmch generated by the code; c dsave(13) = the infinity norm of the projected gradient; c c see the subroutine setulb.f for a description of other c information contained in dsave SHAR_EOF fi # end of overwriting check if test -f 'algorithm.ps' then echo shar: will not over-write existing file "'algorithm.ps'" else cat << \SHAR_EOF > 'algorithm.ps' %!PS-Adobe-2.0 %%Creator: dvips 5.47 Copyright 1986-91 Radical Eye Software %%Title: pp8.dvi %%Pages: 25 1 %%BoundingBox: 0 0 612 792 %%EndComments %%BeginProcSet: tex.pro /TeXDict 200 dict def TeXDict begin /N /def load def /B{bind def}N /S /exch load def /X{S N}B /TR /translate load N /isls false N /vsize 10 N /@rigin{ isls{[0 1 -1 0 0 0]concat}if 72 Resolution div 72 VResolution div neg scale Resolution VResolution vsize neg mul TR matrix currentmatrix dup dup 4 get round 4 exch put dup dup 5 get round 5 exch put setmatrix}N /@letter{/vsize 10 N}B /@landscape{/isls true N /vsize -1 N}B /@a4{/vsize 10.6929133858 N}B /@a3{ /vsize 15.5531 N}B /@ledger{/vsize 16 N}B /@legal{/vsize 13 N}B /@manualfeed{ statusdict /manualfeed true put}B /@copies{/#copies X}B /FMat[1 0 0 -1 0 0]N /FBB[0 0 0 0]N /nn 0 N /IE 0 N /ctr 0 N /df-tail{/nn 8 dict N nn begin /FontType 3 N /FontMatrix fntrx N /FontBBox FBB N string /base X array /BitMaps X /BuildChar{CharBuilder}N /Encoding IE N end dup{/foo setfont}2 array copy cvx N load 0 nn put /ctr 0 N[}B /df{/sf 1 N /fntrx FMat N df-tail} B /dfs{div /sf X /fntrx[sf 0 0 sf neg 0 0]N df-tail}B /E{pop nn dup definefont setfont}B /ch-width{ch-data dup length 5 sub get}B /ch-height{ch-data dup length 4 sub get}B /ch-xoff{128 ch-data dup length 3 sub get sub}B /ch-yoff{ ch-data dup length 2 sub get 127 sub}B /ch-dx{ch-data dup length 1 sub get}B /ch-image{ch-data dup type /stringtype ne{ctr get /ctr ctr 1 add N}if}B /id 0 N /rw 0 N /rc 0 N /gp 0 N /cp 0 N /G 0 N /sf 0 N /CharBuilder{save 3 1 roll S dup /base get 2 index get S /BitMaps get S get /ch-data X pop /ctr 0 N ch-dx 0 ch-xoff ch-yoff ch-height sub ch-xoff ch-width add ch-yoff setcachedevice ch-width ch-height true[1 0 0 -1 -.1 ch-xoff sub ch-yoff .1 add]{ch-image} imagemask restore}B /D{/cc X dup type /stringtype ne{]}if nn /base get cc ctr put nn /BitMaps get S ctr S sf 1 ne{dup dup length 1 sub dup 2 index S get sf div put}if put /ctr ctr 1 add N}B /I{cc 1 add D}B /bop{userdict /bop-hook known{bop-hook}if /SI save N @rigin 0 0 moveto}N /eop{clear SI restore showpage userdict /eop-hook known{eop-hook}if}N /@start{userdict /start-hook known{start-hook}if /VResolution X /Resolution X 1000 div /DVImag X /IE 256 array N 0 1 255{IE S 1 string dup 0 3 index put cvn put}for}N /p /show load N /RMat[1 0 0 -1 0 0]N /BDot 260 string N /rulex 0 N /ruley 0 N /v{/ruley X /rulex X V}B /V statusdict begin /product where{pop product dup length 7 ge{0 7 getinterval(Display)eq}{pop false}ifelse}{false}ifelse end{{gsave TR -.1 -.1 TR 1 1 scale rulex ruley false RMat{BDot}imagemask grestore}}{{gsave TR -.1 -.1 TR rulex ruley scale 1 1 false RMat{BDot}imagemask grestore}}ifelse B /a{ moveto}B /delta 0 N /tail{dup /delta X 0 rmoveto}B /M{S p delta add tail}B /b{ S p tail}B /c{-4 M}B /d{-3 M}B /e{-2 M}B /f{-1 M}B /g{0 M}B /h{1 M}B /i{2 M}B /j{3 M}B /k{4 M}B /w{0 rmoveto}B /l{p -4 w}B /m{p -3 w}B /n{p -2 w}B /o{p -1 w }B /q{p 1 w}B /r{p 2 w}B /s{p 3 w}B /t{p 4 w}B /x{0 S rmoveto}B /y{3 2 roll p a}B /bos{/SS save N}B /eos{clear SS restore}B end %%EndProcSet TeXDict begin 1000 300 300 @start /Fa 7 85 df<001F0000001F0000003F8000003B8000 003B8000007BC0000073C0000071C00000F1E00000E1E00000E0E00001E0F00001E0F00001C0F0 0003C0780003C078000380780007803C0007803C0007003C000FFFFE000FFFFE000FFFFE001E00 0F001E000F003C000F803C0007803C000780780007C0780003C0780003C0F00003E01B207F9F1E >65 D<001FC000FFF801FFFC03E03C07800C0F00001E00003E00003C00007C0000780000780000 780000F00000F00000F00000F00000F00000F00000F00000F000007800007800007800007C0000 3C00003E00001E00000F000207800E03E03E01FFFC00FFF0001FC017227DA01D>67 D69 D76 D78 D<003F000000FFC00003FFF00007E1F8000F807C001F003E001E001E003C000F003C000F007800 07807800078078000780F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F00003C0F0 0003C0F00003C0F80007C07800078078000780780007803C000F003C000F001E001E001F003E00 0F807C0007E1F80003FFF00000FFC000003F00001A227DA021>I84 D E /Fb 1 49 df<081C1C3838383070706060C0C0060D7E8D09>48 D E /Fc 4 50 df0 D<020002000200C218F2783AE00F800F803A E0F278C2180200020002000D0E7E8E12>3 D<060F0F0E1E1E1C3C383830707060E0C04008117F 910A>48 D<0F8007C019E01C202070301040184008C00C80048007800480070004800380048007 80048004C00C400860082030381010E01E600F8007C01E0E7E8D23>I E /Fd 17 114 df34 DI<00001C00003C0000F80001E00003C0000780000F00000E00001E0000 3C00003C00003C0000780000780000780000780000780000780000780000780000780000780000 780000780000780000780000780000780000780000780000780000780000780000780000780000 780000780000780000780000780000780000780000780000780000780000780000780000780000 780000780000F00000F00000F00001E00001E00003C0000380000700000E00001C0000780000E0 0000E000007800001C00000E000007000003800003C00001E00001E00000F00000F00000F00000 780000780000780000780000780000780000780000780000780000780000780000780000780000 780000780000780000780000780000780000780000780000780000780000780000780000780000 7800007800007800007800007800007800007800007800007800007800007800007800003C0000 3C00003C00001E00000E00000F000007800003C00001E00000F800003C00001C167C7B8121>40 D50 DI< E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E0 00E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000 E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E0 00E000E000E000E000E000E000E000E000E000E000E000E000FFFCFFFCFFFC0E4A73811C>I<00 1C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C 001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C00 1C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C 001C001C001C001C001C001C001C001C001C001C001C001CFFFCFFFCFFFC0E4A80811C>III<0018007800F001E003C00780 0F001F001E003E003C007C007C007800F800F800F800F800F800F800F800F800F800F800F800F8 00F800F800F800F800F800F800F800F800F800F800F8000D25707E25>I58 D<007C007C007C007C007C007C007C007C007C007C007C007C007C007C007C007C007C007C007C 007C007C007C007C007C00F800F800F800F001F001E003E003C0078007000E001C003800F000C0 00F00038001C000E000700078003C003E001E001F000F000F800F800F8007C007C007C007C007C 007C007C007C007C007C007C007C007C007C007C007C007C007C007C007C007C007C007C007C0E 4D798025>60 D62 D104 DI<00000000020000000006000000000C000000000C000000001800000000180000000030 00000000300000000060000000006000000000C000000000C00000000180000000018000000003 000000000300000000060000000006000000000C000000000C0000000018000000001800000000 30000000003000000000600008000060001C0000C0003C0000C000CE000180000E000180000E00 03000007000300000700060000038006000003800C000001C00C000001C018000001E018000000 E030000000E0300000007060000000706000000038C000000038C00000001D800000001D800000 001F000000000F000000000E000000000600000027327C812A>112 D<00000000020000000006 0000000006000000000C000000000C000000000C00000000180000000018000000001800000000 300000000030000000003000000000600000000060000000006000000000C000000000C0000000 00C000000001800000000180000000018000000003000000000300000000030000000006000000 00060000000006000000000C000000000C000000000C0000000018000000001800000000180000 00003000000000300000000030000000006000000000600008000060001C0000C0003C0000C000 7C0000C000DC000180008E000180000E000180000E000300000700030000070003000007000600 000380060000038006000003800C000003800C000001C00C000001C018000001C018000000E018 000000E030000000E030000000E03000000070600000007060000000706000000038C000000038 C000000038C00000001D800000001D800000001D800000001F000000000F000000000F00000000 0E00000000060000000006000000274B7C812A>I E /Fe 18 118 df<01E0033006100C100810 1818101830183010603060307FF06030C060C060C060C0C0C0C0C180C1004300660038000D177E 9610>18 D<60F0F070101020204040040A7D830A>59 D<1FFFFE381C0E201C04601C04401C0440 380480380400380000380000700000700000700000700000E00000E00000E00000E00001C00001 C00001C00001C00003C0007FFC0017177F9615>84 D<071018F0307060706060C060C060C06080 C080C480C4C1C446C838700E0E7E8D13>97 D<7C00180018001800180030003000300030006780 68C070406060C060C060C060C06080C080C08180C10046003C000B177E960F>I<07C00C201070 20706000C000C000C00080008000C010C02060C03F000C0E7E8D0F>I<003E000C000C000C000C 0018001800180018073018F0307060706060C060C060C06080C080C480C4C1C446C838700F177E 9612>I<000E0013003700360060006000600060006007FC00C000C000C000C000C00180018001 8001800180018003000300030003006600E600CC007800101D7E9611>102 D<0300038003000000000000000000000000001C002400460046008C000C001800180018003100 3100320032001C0009177F960C>105 D<00180038001000000000000000000000000001C00220 04300430086000600060006000C000C000C000C001800180018001806300E300C60078000D1D80 960E>I<1F0006000600060006000C000C000C000C00181C1866188E190C32003C003F00318060 C060C460C460C8C0C8C0700F177E9612>I<3E0C0C0C0C181818183030303060606060C0C8C8C8 D07007177E960B>I<383C1E0044C6630047028100460301008E0703000C0603000C0603000C06 0600180C0600180C0620180C0C20180C0C4030180440301807801B0E7F8D1F>I<383C0044C600 4702004602008E06000C06000C06000C0C00180C00180C40181840181880300880300F00120E7F 8D15>I<07C00C20101020186018C018C018C01880308030C060C0C061803E000D0E7E8D11>I<1C 3C22462382230346030603060306030C060C060C0C0C081A3019E018001800300030003000FC00 1014808D12>I<030003000600060006000600FFC00C000C000C00180018001800180030003080 3080310031001E000A147F930D>116 D<1C0200260600460600460600860C000C0C000C0C000C 0C001818001818801818801838800C5900078E00110E7F8D14>I E /Ff 19 121 df0 D<400004C0000C6000183000301800600C00C0 06018003030001860000CC0000780000300000300000780000CC000186000303000601800C00C0 180060300030600018C0000C40000416187A9623>2 D<03C00FF01FF83FFC7FFE7FFEFFFFFFFF FFFFFFFFFFFFFFFF7FFE7FFE3FFC1FF80FF003C010127D9317>15 D17 D<000000C0000003C000000F0000003C000000F0000003C00000070000001C00000078000001E0 0000078000001E00000078000000E0000000780000001E0000000780000001E000000078000000 1C0000000700000003C0000000F00000003C0000000F00000003C0000000C00000000000000000 00000000000000000000000000000000000000007FFFFF80FFFFFFC01A247C9C23>20 DI<00000004 00000000020000000002000000000100000000008000000000400000000020FFFFFFFFFCFFFFFF FFFC00000000200000000040000000008000000001000000000200000000020000000004002610 7D922D>33 D<03E0001F000FFC007FC01C7E00C020301F018010600F8300084007C600084003EC 00088001F800048001F800048000F8000480007C000480007E000480007E00044000DF00084001 8F8008400307C018200603E030100C01F8E00FF800FFC003E0001F0026147D932D>49 D<003FF800FFF803C0000700000C0000180000300000300000600000600000C00000C00000C000 00FFFFF8FFFFF8C00000C00000C000006000006000003000003000001800000C000007000003C0 0000FFF8003FF8151C7C981E>I<400001C0000360000660000660000630000C30000C30000C18 00181800181800180FFFF00FFFF00C00300600600600600600600300C00300C001818001818001 818000C30000C30000C300006600006600006600003C00003C00003C000018000018001821809F 19>56 D<00001E000F00FF0011810F0020C60F8060E40780407807C0407003E0407801E0207801 F0307800F0187800F80C780078047C003C067C0038027C0060027C0180027FFE00427C3C00467C 3C00447C3C00447C3C0038783C0000783C0000783C0000783C0000783C0000783C0040781E00E0 703C0020703C0020601E0810E01E3008C00FC0078003001E227EA021>60 D<00000010000000003000000000F000000000F000000000F000000001F000000001F000000002 F000000002F000000004F00000000CF000000008F000000018F000000010F000000020F0000000 20F000000040F8000000C0F8000000807800000180780000030078000002007800000600780000 0DFFF800000FFFF800001800780000300078000060007C0040E0007C00C0C0003C00E380003C00 FF00003E00FE00003F80FE00001F00780000000021237FA024>65 D<00007FFFE00003FFFFF000 0E3801E000303800C0006078000000E078000001C070000001807000000000F000000000F00000 0000E000000000E000000001E000000001C000000001C000000003FFF8000003FFF00000038000 0000070000000007000000000F000000000E000000000E000000001C000000001C000000003800 0000003800000030700000007870000000FCC00000007F800000003E0000000024207F9E21>70 D<000F0038006000E001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C0 038007001E00F8001E000700038001C001C001C001C001C001C001C001C001C001C001C001C001 C001C001C000E000600038000F102D7DA117>102 DI106 D<4020C030C030C030C030C030C030C030C030C030C030 C030C030C030C030C030C030C030C030C030C030C030C030C030C030C030C030C030C030C030C0 30C030C030C030C030C030C030C030C030C030C030C030C030C030C03040200C2E7BA117>I114 D<0F80184030207010E030E070E020E000E000600070003000180006 00198030C070606070E030E038E038E03860387030307018600CC0030000C00060007000300038 00382038703860384070206010C00F800D297D9F14>120 D E /Fg 50 123 df<007C0001C3000701810E01C11E00C11C00E23C00E27800E27800E47800E4F000E8F000F0F0 00F0F000E0F000E07000E07003E030046118383207C01C18147E931D>11 D<0000F800030600040600080300100300200300400700400700800700800601000E01000C0107 F80104700207D802001C02001C02001E04001E04001E04001E04001E08003C08003C08003C0800 781800701400F01400E01201C0218700207C002000002000004000004000004000004000008000 0080000080000018297F9F1A>I<03E0040FF0081FF8083FF810301C10400C10C0042080042000 024000024000028000028000028000030000030000030000020000020000020000060000060000 0600000C00000C00000C00000C00001800001800001800001000161E7F9318>I<001E00006300 00C38001C1800381800301C00701C00F01C00E01C01E03C01C03C03C03C03C03C03C03C0780780 7FFF807FFF80780780F00F00F00F00F00F00F00E00F01E00E01C00E03C00E03800E03000607000 60E00070C0003180001E000012207E9F15>18 D<07000001C00000E00000E00000F00000700000 7000007800003800003800003C00001C00001C00001E00000E00000E00000F00000700000F0000 378000638000C38001C3C00381C00701C00E01E01C00E03800E07000F0F00070E00070C0003815 207D9F1B>21 D<000F800038C000606000C07001C0700380780380780700780700780700780E00 F00E00F00E00F00E01E01C01C01C01C01E03801E0700390C0038F0003800003800007000007000 00700000700000E00000E00000E00000C00000151E7F9318>26 D<70F8F8F87005057C840D>58 D<70F8FCFC74040404080810102040060E7C840D>I<000001C00000078000001E000000780000 01E00000078000000E00000038000000F0000003C000000F0000003C000000F0000000F0000000 3C0000000F00000003C0000000F0000000380000000E0000000780000001E0000000780000001E 0000000780000001C01A1A7C9723>I<000100030003000600060006000C000C000C0018001800 1800300030003000600060006000C000C000C00180018001800300030003000600060006000C00 0C000C00180018001800300030003000600060006000C000C000C000102D7DA117>II<00000200000006000000 0E0000000E0000001E0000001F0000002F0000002F0000004F0000008F0000008F0000010F0000 010F0000020F0000040F0000040F0000080F8000080780001007800020078000200780007FFF80 0040078000800780018007800100078002000780020007C0040003C00C0003C01E0007C0FF807F FC1E207E9F22>65 D<00FFFFE0000F0078000F003C000F001C000F001E001E001E001E001E001E 001E001E001E003C003C003C003C003C0078003C00F0007803C0007FFF80007803C0007801E000 F000F000F000F000F000F000F0007001E000F001E000F001E000F001E000E003C001E003C003C0 03C0038003C00F0007801E00FFFFF0001F1F7E9E22>I<00FFFFE000000F007800000F001C0000 0F000E00000F000700001E000700001E000380001E000380001E000380003C000380003C000380 003C000380003C000380007800078000780007800078000780007800078000F0000F0000F0000F 0000F0000E0000F0001E0001E0001C0001E0003C0001E000380001E000700003C000E00003C001 C00003C003800003C007000007803C0000FFFFF00000211F7E9E26>68 D<00FFF9FFF0000F801F 00000F001E00000F001E00000F001E00001E003C00001E003C00001E003C00001E003C00003C00 7800003C007800003C007800003C007800007800F000007FFFF000007800F000007800F00000F0 01E00000F001E00000F001E00000F001E00001E003C00001E003C00001E003C00001E003C00003 C007800003C007800003C007800003C007800007C00F8000FFF8FFF800241F7E9E26>72 D<00FFFC000F80000F00000F00000F00001E00001E00001E00001E00003C00003C00003C00003C 0000780000780000780000780000F00000F00000F00000F00001E00001E00001E00001E00003C0 0003C00003C00003C00007C000FFFC00161F7F9E14>I<00FFFC00000F8000000F0000000F0000 000F0000001E0000001E0000001E0000001E0000003C0000003C0000003C0000003C0000007800 0000780000007800000078000000F0000000F0000000F0000000F0004001E0008001E0008001E0 018001E0010003C0030003C0030003C0060003C00E0007803C00FFFFFC001A1F7E9E1F>76 D<00FF00001FF0000F00003F00000B80003E00000B80005E00000B80005E0000138000BC000013 80013C00001380013C00001380023C000023800278000023800478000023800878000021C00878 000041C010F0000041C020F0000041C020F0000041C040F0000081C041E0000081C081E0000081 C101E0000081C101E0000100E203C0000100E203C0000100E403C0000100E803C0000200E80780 000200F00780000200F00780000600E00780000F00C00F8000FFE0C1FFF8002C1F7E9E2C>I<00 FF803FF0000F800780000F800200000BC00200000BC002000013C004000011E004000011E00400 0011E004000020F008000020F008000020F808000020780800004078100000403C100000403C10 0000403C100000801E200000801E200000801E200000800F200001000F400001000F4000010007 C000010007C00002000780000200038000020003800006000380000F00010000FFE0010000241F 7E9E25>I<0001FC0000070700001C01C0003000E000E0006001C0007003800070078000380700 00380E0000381E0000381C0000383C0000383C00003878000078780000787800007878000078F0 0000F0F00000F0F00000E0F00001E0F00001C0F00003C0700003807000070078000F0038001E00 38003C001C0070000E00E0000783800001FC00001D217E9F23>I<00FFFFC0000F0070000F0038 000F001C000F001E001E001E001E001E001E001E001E001E003C003C003C003C003C0078003C00 70007800E000780380007FFE000078000000F0000000F0000000F0000000F0000001E0000001E0 000001E0000001E0000003C0000003C0000003C0000003C0000007C00000FFFC00001F1F7E9E1D >I<00FFFF80000F01E0000F0070000F0038000F003C001E003C001E003C001E003C001E003C00 3C0078003C0078003C00F0003C01E00078038000780F00007FF80000781C0000F00E0000F00F00 00F0070000F0078001E00F0001E00F0001E00F0001E00F0003C01E0003C01E0203C01E0203C01E 0407C00E04FFFC0718000003E01F207E9E23>82 D<0007E0800018118000300B000060070000C0 070001C0030001800200038002000380020003800200038000000380000003C0000003F8000003 FF800001FFC00000FFE000003FF0000003F0000000F00000007000000070000000700020007000 20007000200060006000E0006000C0006001C00070018000E8030000C60E000081F8000019217D 9F1C>I87 D89 D<007FFFF800FC00F000E001E000C003 C0018007800100078003000F0002001E0002003C00040078000000F8000000F0000001E0000003 C00000078000000F0000000F0000001E0000003C00000078008000F0008001F0010001E0010003 C00300078002000F0006001E0004003E000C003C003C007800F800FFFFF8001D1F7D9E1F>I<00 F1800389C00707800E03801C03803C0380380700780700780700780700F00E00F00E00F00E00F0 0E10F01C20F01C20703C20705C40308C400F078014147E9318>97 D<07803F8007000700070007 000E000E000E000E001C001C001CF01D0C3A0E3C0E380F380F700F700F700F700FE01EE01EE01E E01CE03CE038607060E031C01F0010207E9F14>I<007C01C207010E0F1E0F1C0E3C0478007800 7800F000F000F000F000F00070017002300418380FC010147E9314>I<0000780003F800007000 00700000700000700000E00000E00000E00000E00001C00001C000F1C00389C00707800E03801C 03803C0380380700780700780700780700F00E00F00E00F00E00F00E10F01C20F01C20703C2070 5C40308C400F078015207E9F18>I<007C01C207010E011C013C013802780C7BF07C00F000F000 F000F0007000700170023004183807C010147E9315>I<00007C0000CE00019E00039E00030C00 0700000700000700000700000E00000E00000E0000FFF0000E00000E00001C00001C00001C0000 1C00001C0000380000380000380000380000380000700000700000700000700000700000E00000 E00000E00000E00000C00001C000318000798000F300006200003C000017297E9F16>I<001E30 00713800E0F001C0700380700780700700E00F00E00F00E00F00E01E01C01E01C01E01C01E01C0 1E03801E03800E07800E0B8006170001E700000700000700000E00000E00300E00781C00F03800 6070003FC000151D809316>I<00E001E001E000C000000000000000000000000000000E001300 23804380438043808700070007000E000E001C001C001C20384038403840388019000E000B1F7E 9E10>105 D<0000C00001E00001E00001C0000000000000000000000000000000000000000000 001E00006300004380008380010380010380020700000700000700000700000E00000E00000E00 000E00001C00001C00001C00001C0000380000380000380000380000700000700030700078E000 F1C0006380003E00001328819E13>I<01E0000FE00001C00001C00001C00001C0000380000380 000380000380000700000700000701E00706100E08700E10F00E20F00E40601C80001D00001E00 001FC000387000383800383800381C20703840703840703840701880E01880600F0014207E9F18 >I<03C01FC0038003800380038007000700070007000E000E000E000E001C001C001C001C0038 003800380038007000700070007100E200E200E200E200640038000A207E9F0E>I<1E07C07C00 231861860023A032030043C0340300438038038043803803808700700700070070070007007007 0007007007000E00E00E000E00E00E000E00E00E000E00E01C101C01C01C201C01C038201C01C0 38401C01C0184038038018801801800F0024147E9328>I<1E07802318C023A06043C070438070 4380708700E00700E00700E00700E00E01C00E01C00E01C00E03821C03841C07041C07081C0308 3803101801E017147E931B>I<03C1E004621804741C08781C08701E08701E10E01E00E01E00E0 1E00E01E01C03C01C03C01C03C01C0380380780380700380E003C1C0072380071E000700000700 000E00000E00000E00000E00001C00001C0000FFC000171D819317>112 D<00F0400388C00705800E03801C03803C0380380700780700780700780700F00E00F00E00F00E 00F00E00F01C00F01C00703C00705C0030B8000F38000038000038000070000070000070000070 0000E00000E0000FFE00121D7E9314>I<1E1E0023210023C38043C78043878043830087000007 00000700000700000E00000E00000E00000E00001C00001C00001C00001C000038000018000011 147E9315>I<007C018203010603060706060E00078007F803FC01FE001F00077007F006F006E0 04400820301FC010147E9315>I<00C000E001C001C001C001C003800380FFF803800700070007 0007000E000E000E000E001C001C001C001C10382038203820384018800F000D1C7F9B10>I<0F 00601180702180E021C0E041C0E04380E08381C00701C00701C00701C00E03800E03800E03800E 03840E07080C07080C07080E0F1006131003E1E016147E931A>I<0F01801183C02183E021C1E0 41C0E04380608380400700400700400700400E00800E00800E00800E01000E01000C02000E0400 0E040006180001E00013147E9316>I<0F006060118070F02180E0F821C0E07841C0E0384380E0 188381C0100701C0100701C0100701C0100E0380200E0380200E0380200E0380400E0380400E03 80800E078080060781000709860001F078001D147E9321>I<03C1C00C62201034701038F02038 F020386040700000700000700000700000E00000E00000E00000E02061C040F1C040F1C080E2C0 80446300383C0014147E931A>I<0F00601180702180E021C0E041C0E04380E08381C00701C007 01C00701C00E03800E03800E03800E03800E07000C07000C07000E0F00061E0003EE00000E0000 0E00001C0078180078380070700060600021C0001F0000141D7E9316>I<01E02003F04007F8C0 0C1F8008010000020000040000080000100000600000C000010000020000040080080100100300 3F060061FC0040F80080700013147E9315>I E /Fh 42 123 df<00700000F800007000007000 207020F07078F870F8FE73F83F27E007AF0000F80000F80007AF003F27E0FE73F8F870F8F07078 20702000700000700000F80000700015167DA41C>42 D<387CFEFEFE7C3807077C8610>46 D<00180000780001F800FFF800FFF80001F80001F80001F80001F80001F80001F80001F80001F8 0001F80001F80001F80001F80001F80001F80001F80001F80001F80001F80001F80001F80001F8 0001F80001F80001F80001F8007FFFE07FFFE013207C9F1C>49 D<03FC000FFF003C1FC07007E0 7C07F0FE03F0FE03F8FE03F8FE01F87C01F83803F80003F80003F00003F00007E00007C0000F80 001F00003E0000380000700000E01801C0180380180700180E00380FFFF01FFFF03FFFF07FFFF0 FFFFF0FFFFF015207D9F1C>I<00FE0007FFC00F07E01E03F03F03F03F81F83F81F83F81F81F03 F81F03F00003F00003E00007C0001F8001FE0001FF000007C00001F00001F80000FC0000FC3C00 FE7E00FEFF00FEFF00FEFF00FEFF00FC7E01FC7801F81E07F00FFFC001FE0017207E9F1C>I<00 00E00001E00003E00003E00007E0000FE0001FE0001FE00037E00077E000E7E001C7E00187E003 07E00707E00E07E00C07E01807E03807E07007E0E007E0FFFFFEFFFFFE0007E00007E00007E000 07E00007E00007E00007E000FFFE00FFFE17207E9F1C>I<1000201E01E01FFFC01FFF801FFF00 1FFE001FF8001BC00018000018000018000018000019FC001FFF001E0FC01807E01803E00003F0 0003F00003F80003F83803F87C03F8FE03F8FE03F8FC03F0FC03F07007E03007C01C1F800FFF00 03F80015207D9F1C>I<001F8000FFE003F07007C0F00F01F81F01F83E01F83E01F87E00F07C00 007C0000FC0800FC7FC0FCFFE0FD80F0FF00F8FE007CFE007CFC007EFC007EFC007EFC007E7C00 7E7C007E7C007E3C007C3E007C1E00F80F00F00783E003FFC000FF0017207E9F1C>I<60000078 00007FFFFE7FFFFE7FFFFC7FFFF87FFFF87FFFF0E00060E000C0C00180C00300C0030000060000 0C00001C0000180000380000780000780000F00000F00000F00001F00001F00001F00003F00003 F00003F00003F00003F00003F00003F00001E00017227DA11C>I66 D<0003FE0080001FFF818000FF01E38001F8003F8003E0001F8007C0000F 800F800007801F800007803F000003803F000003807F000001807E000001807E00000180FE0000 0000FE00000000FE00000000FE00000000FE00000000FE00000000FE00000000FE000000007E00 0000007E000001807F000001803F000001803F000003801F800003000F8000030007C000060003 F0000C0001F800380000FF00F000001FFFC0000003FE000021227DA128>I69 DI<0003FE0040001FFFC0C0007F00F1C001F8003FC003F0000FC007C00007C00FC000 03C01F800003C03F000001C03F000001C07F000000C07E000000C07E000000C0FE00000000FE00 000000FE00000000FE00000000FE00000000FE00000000FE00000000FE000FFFFC7E000FFFFC7F 00001FC07F00001FC03F00001FC03F00001FC01F80001FC00FC0001FC007E0001FC003F0001FC0 01FC003FC0007F80E7C0001FFFC3C00003FF00C026227DA12C>I73 D76 DII<0007FC0000003FFF800000FC07E00003F001F80007E000FC000FC0007E001F8000 3F001F80003F003F00001F803F00001F807F00001FC07E00000FC07E00000FC0FE00000FE0FE00 000FE0FE00000FE0FE00000FE0FE00000FE0FE00000FE0FE00000FE0FE00000FE0FE00000FE07E 00000FC07F00001FC07F00001FC03F00001F803F80003F801F80003F000FC0007E0007E000FC00 03F001F80000FC07E000003FFF80000007FC000023227DA12A>I<01FC0407FF8C1F03FC3C007C 7C003C78001C78001CF8000CF8000CFC000CFC0000FF0000FFE0007FFF007FFFC03FFFF01FFFF8 0FFFFC03FFFE003FFE0003FF00007F00003F00003FC0001FC0001FC0001FE0001EE0001EF0003C FC003CFF00F8C7FFE080FF8018227DA11F>83 D<7FFFFFFF807FFFFFFF807E03F80F807803F807 807003F803806003F80180E003F801C0E003F801C0C003F800C0C003F800C0C003F800C0C003F8 00C00003F800000003F800000003F800000003F800000003F800000003F800000003F800000003 F800000003F800000003F800000003F800000003F800000003F800000003F800000003F8000000 03F800000003F800000003F800000003F800000003F8000003FFFFF80003FFFFF80022227EA127 >I<07FC001FFF803F07C03F03E03F01E03F01F01E01F00001F00001F0003FF003FDF01FC1F03F 01F07E01F0FC01F0FC01F0FC01F0FC01F07E02F07E0CF81FF87F07E03F18167E951B>97 DI<00FF8007FFE00F83 F01F03F03E03F07E03F07C01E07C0000FC0000FC0000FC0000FC0000FC0000FC00007C00007E00 007E00003E00301F00600FC0E007FF8000FE0014167E9519>I<0001FE000001FE0000003E0000 003E0000003E0000003E0000003E0000003E0000003E0000003E0000003E0000003E0000003E00 01FC3E0007FFBE000F81FE001F007E003E003E007E003E007C003E00FC003E00FC003E00FC003E 00FC003E00FC003E00FC003E00FC003E00FC003E007C003E007C003E003E007E001E00FE000F83 BE0007FF3FC001FC3FC01A237EA21F>I<00FE0007FF800F87C01E01E03E01F07C00F07C00F8FC 00F8FC00F8FFFFF8FFFFF8FC0000FC0000FC00007C00007C00007E00003E00181F00300FC07003 FFC000FF0015167E951A>I<003F8000FFC001E3E003C7E007C7E00F87E00F83C00F80000F8000 0F80000F80000F80000F8000FFFC00FFFC000F80000F80000F80000F80000F80000F80000F8000 0F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F80007FF8007FF800 13237FA211>I<03FC1E0FFF7F1F0F8F3E07CF3C03C07C03E07C03E07C03E07C03E07C03E03C03 C03E07C01F0F801FFF0013FC003000003000003800003FFF801FFFF00FFFF81FFFFC3800FC7000 3EF0001EF0001EF0001EF0001E78003C7C007C3F01F80FFFE001FF0018217E951C>II<1C003E007F007F007F003E001C 000000000000000000000000000000FF00FF001F001F001F001F001F001F001F001F001F001F00 1F001F001F001F001F001F001F001F00FFE0FFE00B247EA310>I108 DII<00FE0007FFC00F83E01E00 F03E00F87C007C7C007C7C007CFC007EFC007EFC007EFC007EFC007EFC007EFC007E7C007C7C00 7C3E00F81F01F00F83E007FFC000FE0017167E951C>II114 D<0FF3003FFF00781F00600700E00300E00300F00300FC00007FE0007FF8003FFE000FFF0001FF 00000F80C00780C00380E00380E00380F00700FC0E00EFFC00C7F00011167E9516>I<01800001 80000180000180000380000380000780000780000F80003F8000FFFF00FFFF000F80000F80000F 80000F80000F80000F80000F80000F80000F80000F80000F80000F81800F81800F81800F81800F 81800F830007C30003FE0000F80011207F9F16>II120 DI<7FFFF0 7FFFF07C03E07007C0600FC0E01F80C01F00C03E00C07E0000FC0000F80001F00003F03007E030 07C0300F80701F80703F00603E00E07C03E0FFFFE0FFFFE014167E9519>I E /Fi 51 122 df<60F0F07010101020204080040B7D830B>44 DI<60 F0F06004047D830B>I<078018603030303060186018E01CE01CE01CE01CE01CE01CE01CE01CE0 1CE01CE01CE01C6018601870383030186007800E187E9713>48 D<03000700FF00070007000700 07000700070007000700070007000700070007000700070007000700070007000700FFF00C187D 9713>I<0F80106020304038803CC01CE01C401C003C003800380070006000C001800100020004 040804100430083FF87FF8FFF80E187E9713>I<0F8010E0207060787038203800780070007000 6000C00F8000E000700038003C003CE03CE03CC03C4038407030E00F800E187E9713>I<003000 30007000F000F001700370027004700C7008701070307020704070C070FFFF0070007000700070 0070007007FF10187F9713>I<30183FF03FE03FC02000200020002000200027C0386020300038 0018001C001C401CE01CE01C80184038403030E00F800E187E9713>I<01E006100C1818383038 300070006000E000E7C0E860F030F018E018E01CE01CE01C601C601C701830183030186007C00E 187E9713>I<40007FFE7FFC7FFC40088010801080200040004000800180018001000300030003 00030007000700070007000700070002000F197E9813>I<078018603030201860186018601870 103C303E600F8007C019F030F86038401CC00CC00CC00CC00C6008201018600FC00E187E9713> I<07801860303070306018E018E018E01CE01CE01C601C603C303C185C0F9C001C001800180038 70307060604021801F000E187E9713>I<60F0F060000000000000000060F0F070101010202040 8004177D8F0B>59 D<007F00000180C000060030000800080010000400203E020020E1020041C0 810043807100838070808700708087007080870070808700708087007080870070808380708043 80708041C0F10020E13100203E1E0010000000080000000600038001803E00007FE000191A7E99 1E>64 D<000C0000000C0000000C0000001E0000001E0000003F00000027000000270000004380 0000438000004380000081C0000081C0000081C0000100E0000100E00001FFE000020070000200 700006007800040038000400380008001C0008001C001C001E00FF00FFC01A1A7F991D>II<003F0201C0C603002E0E001E1C000E1C0006380006780002700002700002F00000F0 0000F00000F00000F00000F000007000027000027800023800041C00041C00080E000803003001 C0C0003F00171A7E991C>IIII<003F020001C0 C60003002E000E001E001C000E001C00060038000600780002007000020070000200F0000000F0 000000F0000000F0000000F0000000F001FFC070000E0070000E0078000E0038000E001C000E00 1C000E000E000E000300160001C06600003F82001A1A7E991E>I73 D76 D78 D<007F000001C1C000070070000E00 38001C001C003C001E0038000E0078000F0070000700F0000780F0000780F0000780F0000780F0 000780F0000780F0000780F000078078000F0078000F0038000E003C001E001C001C000E003800 0700700001C1C000007F0000191A7E991E>I82 D<0FC21836200E6006C006C002C002C002E00070007E003FE01F F807FC003E000E00070003800380038003C002C006E004D81887E0101A7E9915>I<7FFFFF0070 1C0700401C0100401C0100C01C0180801C0080801C0080801C0080001C0000001C0000001C0000 001C0000001C0000001C0000001C0000001C0000001C0000001C0000001C0000001C0000001C00 00001C0000001C0000001C0000001C000003FFE000191A7F991C>II<3F8070C0 70E020700070007007F01C7030707070E070E071E071E0F171FB1E3C10107E8F13>97 DI<07F80C1C381C30087000E000E000E000E000E000E0007000300438080C1807 E00E107F8F11>I<007E00000E00000E00000E00000E00000E00000E00000E00000E00000E0003 CE000C3E00380E00300E00700E00E00E00E00E00E00E00E00E00E00E00E00E00600E00700E0038 1E001C2E0007CFC0121A7F9915>I<07C01C3030187018600CE00CFFFCE000E000E000E0006000 300438080C1807E00E107F8F11>I<01F0031807380E100E000E000E000E000E000E00FFC00E00 0E000E000E000E000E000E000E000E000E000E000E000E000E007FE00D1A80990C>I<0FCE1873 30307038703870387038303018602FC02000600070003FF03FFC1FFE600FC003C003C003C00360 06381C07E010187F8F13>II<18003C003C001800000000000000000000000000 FC001C001C001C001C001C001C001C001C001C001C001C001C001C001C00FF80091A80990A>I< FC001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C 001C001C001C001C001C00FF80091A80990A>108 DII<07E01C38300C700E 6006E007E007E007E007E007E0076006700E381C1C3807E010107F8F13>II114 D<1F2060E04020C020C020F0007F003FC01FE000F080708030C030C020F0408F800C107F8F0F> I<0400040004000C000C001C003C00FFC01C001C001C001C001C001C001C001C001C201C201C20 1C201C200E4003800B177F960F>III I121 D E /Fj 2 51 df<0C003C00CC000C000C000C000C000C000C000C000C000C000C000C000C00FF 8009107E8F0F>49 D<1F00618040C08060C0600060006000C00180030006000C00102020207FC0 FFC00B107F8F0F>I E /Fk 9 62 df<0030000030000030000030000030000030000030000030 00003000003000003000FFFFFCFFFFFC0030000030000030000030000030000030000030000030 0000300000300000300016187E931B>43 D<07C018303018701C600C600CE00EE00EE00EE00EE0 0EE00EE00EE00EE00E600C600C701C30181C7007C00F157F9412>48 D<03000700FF0007000700 0700070007000700070007000700070007000700070007000700070007007FF00C157E9412>I< 0F8030E040708030C038E0384038003800700070006000C00180030006000C08080810183FF07F F0FFF00D157E9412>I<0FE030306018701C701C001C00180038006007E000300018000C000E00 0EE00EE00EC00C401830300FE00F157F9412>I<00300030007000F001F0017002700470087018 70107020704070C070FFFE0070007000700070007003FE0F157F9412>I<20303FE03FC0240020 002000200020002F8030E020700030003800384038E038E0388030406020C01F000D157E9412> I<01F00608080C181C301C70006000E000E3E0EC30F018F00CE00EE00EE00E600E600E300C3018 183007C00F157F9412>I61 D E /Fl 51 123 df<00003FE00000E01000018038000380780003007800 070030000700000007000000070000000E0000000E0000000E000000FFFFE0000E00E0001C01C0 001C01C0001C01C0001C01C0001C03800038038000380380003803800038070000380700007007 000070071000700E2000700E2000700E2000E00E2000E0064000E0038000E0000000C0000001C0 000001C000003180000079800000F3000000620000003C0000001D29829F1A>12 D<1C3C3C3C3C040408081020204080060E7D840E>44 D<7FF0FFE07FE00C037D8A10>I<70F8F8 F0E005057B840E>I<000F800030E000E07001C0700380300380380700380F00780F00780E0078 1E00781E00703C00F03C00F03C00F03C00F07801E07801E07801E07801C07003C0F003C0F00380 F00780F00700700700700E00701C003038001870000FC000151F7C9D17>48 D<000200020006000E003C00DC031C001C0038003800380038007000700070007000E000E000E0 00E001C001C001C001C003800380038003800780FFF80F1E7B9D17>I<070F1F1F0E0000000000 000000000070F8F8F0E008147B930E>58 D<00000200000006000000060000000E0000001E0000 001E0000003F0000002F0000004F0000004F0000008F0000010F0000010F0000020F0000020F00 00040F00000C0F0000080F0000100F0000100F0000200F80003FFF800040078000C00780008007 8001000780010007800200078002000780060007801E000F80FF807FF81D207E9F22>65 D<01FFFFC0001E00F0001E0078001E0038001E003C003C003C003C003C003C003C003C003C0078 007800780078007800F0007801E000F0078000FFFE0000F00F8000F003C001E001C001E001E001 E001E001E001E003C001E003C001E003C001E003C001C0078003C00780078007800F0007801E00 0F007800FFFFE0001E1F7D9E20>I<0000FE0200078186001C004C0038003C0060003C00C0001C 01C0001803800018070000180F0000181E0000101E0000103C0000003C00000078000000780000 007800000078000000F0000000F0000000F0000000F0000000F000008070000080700000807000 01003800010038000200180004000C001800060020000381C00000FE00001F217A9F21>I<01FF FF80001E00E0001E0070001E0038001E001C003C001C003C000E003C000E003C000E0078000E00 78000E0078000E0078000E00F0001E00F0001E00F0001E00F0001E01E0003C01E0003C01E0003C 01E0007803C0007003C0007003C000E003C001C0078001C00780038007800E0007801C000F0070 00FFFFC0001F1F7D9E22>I<01FFFFFE001E001C001E000C001E0004001E0004003C0004003C00 04003C0004003C00040078080800780800007808000078180000F0300000FFF00000F0300000F0 300001E0200001E0200001E0200001E0001003C0002003C0002003C0004003C000400780008007 80018007800100078007000F001F00FFFFFE001F1F7D9E1F>I<01FFF3FFE0001F003E00001E00 3C00001E003C00001E003C00003C007800003C007800003C007800003C007800007800F0000078 00F000007800F000007800F00000F001E00000FFFFE00000F001E00000F001E00001E003C00001 E003C00001E003C00001E003C00003C007800003C007800003C007800003C007800007800F0000 07800F000007800F000007800F00000F801F0000FFF1FFE000231F7D9E22>72 D<01FFF0001F00001E00001E00001E00003C00003C00003C00003C000078000078000078000078 0000F00000F00000F00000F00001E00001E00001E00001E00003C00003C00003C00003C0000780 000780000780000780000F8000FFF800141F7D9E12>I<001FFF0000F80000F00000F00000F000 01E00001E00001E00001E00003C00003C00003C00003C0000780000780000780000780000F0000 0F00000F00000F00001E00001E00301E00781E00F83C00F83C00F0780080700040E00021C0001F 000018207D9E18>I<01FFF03FE0001F000F80001E000E00001E000800001E001000003C002000 003C004000003C010000003C020000007804000000780800000078100000007830000000F0F000 0000F1F8000000F278000000F478000001E83C000001F03C000001E03C000001E01E000003C01E 000003C01E000003C00F000003C00F000007800F00000780078000078007800007800780000F80 07C000FFF03FF800231F7D9E23>I<01FFF800001F0000001E0000001E0000001E0000003C0000 003C0000003C0000003C00000078000000780000007800000078000000F0000000F0000000F000 0000F0000001E0000001E0000001E0000001E0008003C0010003C0010003C0030003C002000780 06000780060007800C0007801C000F007800FFFFF800191F7D9E1D>I<01FE00007FC0001E0000 FC00001E0000F80000170001780000170001780000270002F00000270004F00000270004F00000 270008F00000470009E00000470011E00000470021E00000470021E00000870043C00000838043 C00000838083C00000838083C00001038107800001038207800001038207800001038407800002 03840F00000203880F00000203900F00000203900F00000401E01E00000401E01E00000401C01E 00000C01801E00001C01803E0000FF8103FFC0002A1F7D9E29>I<01FF007FE0001F000F00001F 0004000017800400001780040000278008000023C008000023C008000023C008000041E0100000 41E010000041F010000040F010000080F0200000807820000080782000008078200001003C4000 01003C400001003C400001001E400002001E800002001E800002000F800002000F800004000F00 00040007000004000700000C000700001C00020000FF80020000231F7D9E22>I<0001FC000007 0700001C01C0003000E000E0006001C000700380007007800038070000380E0000381E0000381C 0000383C0000383C00003878000078780000787800007878000078F00000F0F00000F0F00000E0 F00001E0F00001C0F00003C0700003807000070078000F0038001E0038003C001C0070000E00E0 000783800001FC00001D217A9F23>I<01FFFF80001E00E0001E0070001E0038001E003C003C00 3C003C003C003C003C003C003C0078007800780078007800F0007800E000F003C000F00F0000FF FC0000F0000001E0000001E0000001E0000001E0000003C0000003C0000003C0000003C0000007 8000000780000007800000078000000F800000FFF000001E1F7D9E1F>I<01FFFF00001E03C000 1E00E0001E0070001E0078003C0078003C0078003C0078003C0078007800F0007800F0007801E0 007801C000F0070000F01E0000FFF00000F0380001E01C0001E01E0001E00E0001E00F0003C01E 0003C01E0003C01E0003C01E0007803C0007803C0807803C0807803C100F801C10FFF00C200000 07C01D207D9E21>82 D<0007E040001C18C0003005800060038000C0038001C001800180010003 80010003800100038001000380000003C0000003C0000003F8000001FF800001FFE000007FF000 001FF0000001F80000007800000078000000380000003800200038002000380020003000600070 00600060006000E0007000C000E8038000C606000081F800001A217D9F1A>I<0FFFFFF01E0780 E0180780201007802020078020200F0020600F0020400F0020400F0020801E0040001E0000001E 0000001E0000003C0000003C0000003C0000003C00000078000000780000007800000078000000 F0000000F0000000F0000000F0000001E0000001E0000001E0000001E0000003E00000FFFF0000 1C1F789E21>I<7FFC1FF807C003C00780010007800100078001000F0002000F0002000F000200 0F0002001E0004001E0004001E0004001E0004003C0008003C0008003C0008003C000800780010 00780010007800100078001000F0002000F0002000F0002000F0004000F0004000700080007001 000030020000380400000C18000007E000001D20779E22>I<007FFFF800FC00F000E001E000C0 03C0008007800180078001000F0001001E0001003C00020078000000F8000000F0000001E00000 03C00000078000000F0000000F0000001E0000003C00000078010000F0010001F0020001E00200 03C00600078004000F000C001E0008003E0018003C0038007801F000FFFFF0001D1F7D9E1C>90 D<00F1800389C00707800E03801C03803C0380380700780700780700780700F00E00F00E00F00E 00F00E20F01C40F01C40703C40705C40308C800F070013147C9317>97 D<07803F800700070007 0007000E000E000E000E001C001C001CF01D0C3A0E3C0E380F380F700F700F700F700FE01EE01E E01EE01CE03CE038607060E031C01F0010207B9F15>I<007E0001C1000300800E07801E07801C 07003C0200780000780000780000F00000F00000F00000F00000F0000070010070020030040018 380007C00011147C9315>I<0000780003F80000700000700000700000700000E00000E00000E0 0000E00001C00001C000F1C00389C00707800E03801C03803C0380380700780700780700780700 F00E00F00E00F00E00F00E20F01C40F01C40703C40705C40308C800F070015207C9F17>I<007C 01C207010E011C013C013802780C7BF07C00F000F000F000F0007000700170023804183807C010 147C9315>I<00007800019C00033C00033C000718000700000700000E00000E00000E00000E00 000E0001FFE0001C00001C00001C00001C00003800003800003800003800003800007000007000 00700000700000700000700000E00000E00000E00000E00000C00001C00001C000018000318000 7B0000F300006600003C00001629829F0E>I<003C6000E27001C1E00380E00700E00F00E00E01 C01E01C01E01C01E01C03C03803C03803C03803C03803C07003C07001C0F001C17000C2E0003CE 00000E00000E00001C00001C00301C00783800F0700060E0003F8000141D7E9315>I<01E0000F E00001C00001C00001C00001C000038000038000038000038000070000070000071E000763000E 81800F01C00E01C00E01C01C03801C03801C03801C0380380700380700380700380E10700E2070 0C20701C20700C40E00CC060070014207D9F17>I<00C001E001E001C000000000000000000000 000000000E003300230043804300470087000E000E000E001C001C001C00384038803080708031 0033001C000B1F7C9E0E>I<0001800003C00003C0000380000000000000000000000000000000 000000000000003C00004600008700008700010700010700020E00000E00000E00000E00001C00 001C00001C00001C0000380000380000380000380000700000700000700000700000E00000E000 30E00079C000F180006300003C00001228829E0E>I<03C01FC003800380038003800700070007 0007000E000E000E000E001C001C001C001C0038003800380038007000700070007100E200E200 E200E200640038000A207C9F0C>108 D<1C0F80F0002630C318004740640C004780680E004700 700E004700700E008E00E01C000E00E01C000E00E01C000E00E01C001C01C038001C01C038001C 01C038001C01C0708038038071003803806100380380E100380380620070070066003003003800 21147C9325>I<1C0F802630C04740604780604700704700708E00E00E00E00E00E00E00E01C01 C01C01C01C01C01C03843803883803083807083803107003303001C016147C931A>I<007C0001 C3000301800E01C01E01C01C01E03C01E07801E07801E07801E0F003C0F003C0F003C0F00780F0 0700700F00700E0030180018700007C00013147C9317>I<01C1E002621804741C04781C04701E 04701E08E01E00E01E00E01E00E01E01C03C01C03C01C03C01C0380380780380700380E003C1C0 072380071E000700000700000E00000E00000E00000E00001C00001C0000FFC000171D809317> I<00F0400388C00705800E03801C03803C0380380700780700780700780700F00E00F00E00F00E 00F00E00F01C00F01C00703C00705C0030B8000F38000038000038000070000070000070000070 0000E00000E0000FFE00121D7C9315>I<1C1E002661004783804787804707804703008E00000E 00000E00000E00001C00001C00001C00001C000038000038000038000038000070000030000011 147C9313>I<00FC030206010C030C070C060C000F800FF007F803FC003E000E700EF00CF00CE0 08401020601F8010147D9313>I<018001C0038003800380038007000700FFF007000E000E000E 000E001C001C001C001C003800380038003820704070407080708031001E000C1C7C9B0F>I<0E 00C03300E02301C04381C04301C04701C08703800E03800E03800E03801C07001C07001C07001C 07101C0E20180E20180E201C1E200C264007C38014147C9318>I<0E03803307802307C04383C0 4301C04700C08700800E00800E00800E00801C01001C01001C01001C02001C02001C04001C0400 1C08000E300003C00012147C9315>I<0E00C1C03300E3C02301C3E04381C1E04301C0E04701C0 60870380400E0380400E0380400E0380401C0700801C0700801C0700801C0701001C0701001C06 02001C0F02000C0F04000E13080003E1F0001B147C931E>I<0383800CC4401068E01071E02071 E02070C040E00000E00000E00000E00001C00001C00001C00001C040638080F38080F38100E581 0084C60078780013147D9315>I<0E00C03300E02301C04381C04301C04701C08703800E03800E 03800E03801C07001C07001C07001C07001C0E00180E00180E001C1E000C3C0007DC00001C0000 1C00003800F03800F07000E06000C0C0004380003E0000131D7C9316>I<01C04003E08007F180 0C1F00080200000400000800001000002000004000008000010000020000040100080200100200 3E0C0063FC0041F80080E00012147D9313>I E /Fm 86 124 df<00008000000001C000000001 C000000003E000000003E000000005F000000004F000000008F80000000878000000107C000000 103C000000203E000000201E000000401F000000400F000000800F80000080078000010007C000 010003C000020003E000020001E000040001F000040000F000080000F80008000078001000007C 001000003C002000003E002000001E007FFFFFFF007FFFFFFF00FFFFFFFF8021207E9F26>1 D<001F83E000F06E3001C078780380F8780300F030070070000700700007007000070070000700 70000700700007007000FFFFFF8007007000070070000700700007007000070070000700700007 007000070070000700700007007000070070000700700007007000070070000700700007007000 07007000070070007FE3FF001D20809F1B>11 D<003F0000E0C001C0C00381E00701E00701E007 0000070000070000070000070000070000FFFFE00700E00700E00700E00700E00700E00700E007 00E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E07FC3FE17 20809F19>I<001F81F80000F04F040001C07C06000380F80F000300F00F000700F00F00070070 000007007000000700700000070070000007007000000700700000FFFFFFFF0007007007000700 700700070070070007007007000700700700070070070007007007000700700700070070070007 007007000700700700070070070007007007000700700700070070070007007007000700700700 07007007007FE3FE3FF02420809F26>14 D<07070F1C383060C00808779F17>19 D22 D<7038F87CFC7EFC7E743A040204020402080408041008100820104020 0F0E7E9F17>34 D<70F8FCFC74040404080810102040060E7C9F0D>39 D<002000400080010002 0006000C000C00180018003000300030007000600060006000E000E000E000E000E000E000E000 E000E000E000E000E0006000600060007000300030003000180018000C000C0006000200010000 80004000200B2E7DA112>I<800040002000100008000C00060006000300030001800180018001 C000C000C000C000E000E000E000E000E000E000E000E000E000E000E000E000C000C000C001C0 01800180018003000300060006000C00080010002000400080000B2E7DA112>I<000600000006 000000060000000600000006000000060000000600000006000000060000000600000006000000 060000000600000006000000060000FFFFFFF0FFFFFFF000060000000600000006000000060000 000600000006000000060000000600000006000000060000000600000006000000060000000600 00000600001C207D9A23>43 D<70F8FCFC74040404080810102040060E7C840D>II<70F8F8F87005057C840D>I<000100030003000600060006000C000C000C00180018 001800300030003000600060006000C000C000C00180018001800300030003000600060006000C 000C000C00180018001800300030003000600060006000C000C000C000102D7DA117>I<03F000 0E1C001C0E00180600380700700380700380700380700380F003C0F003C0F003C0F003C0F003C0 F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0700380700380700380780780380700 1806001C0E000E1C0003F000121F7E9D17>I<018003800F80F380038003800380038003800380 03800380038003800380038003800380038003800380038003800380038003800380038007C0FF FE0F1E7C9D17>I<03F0000C1C00100E00200700400780800780F007C0F803C0F803C0F803C020 07C00007C0000780000780000F00000E00001C0000380000700000600000C00001800003000006 00400C00401800401000803FFF807FFF80FFFF80121E7E9D17>I<03F0000C1C00100E00200F00 780F80780780780780380F80000F80000F00000F00000E00001C0000380003F000003C00000E00 000F000007800007800007C02007C0F807C0F807C0F807C0F00780400780400F00200E001C3C00 03F000121F7E9D17>I<000600000600000E00000E00001E00002E00002E00004E00008E00008E 00010E00020E00020E00040E00080E00080E00100E00200E00200E00400E00C00E00FFFFF0000E 00000E00000E00000E00000E00000E00000E0000FFE0141E7F9D17>I<1803001FFE001FFC001F F8001FE00010000010000010000010000010000010000011F000161C00180E0010070010078000 03800003800003C00003C00003C07003C0F003C0F003C0E00380400380400700200600100E000C 380003E000121F7E9D17>I<007C000182000701000E03800C07801C0780380300380000780000 700000700000F1F000F21C00F40600F80700F80380F80380F003C0F003C0F003C0F003C0F003C0 7003C07003C07003803803803807001807000C0E00061C0001F000121F7E9D17>I<4000007FFF C07FFF807FFF804001008002008002008004000008000008000010000020000020000040000040 0000C00000C00001C0000180000380000380000380000380000780000780000780000780000780 00078000078000030000121F7D9D17>I<03F0000C0C0010060030030020018060018060018060 01807001807803003E03003F06001FC8000FF00003F80007FC000C7E00103F00300F8060038040 01C0C001C0C000C0C000C0C000C0C000806001802001001002000C0C0003F000121F7E9D17>I< 03F0000E18001C0C00380600380700700700700380F00380F00380F003C0F003C0F003C0F003C0 F003C07007C07007C03807C0180BC00E13C003E3C0000380000380000380000700300700780600 780E00700C002018001070000FC000121F7E9D17>I<70F8F8F8700000000000000000000070F8 F8F87005147C930D>I<70F8F8F8700000000000000000000070F0F8F878080808101010202040 051D7C930D>I<7FFFFFE0FFFFFFF0000000000000000000000000000000000000000000000000 0000000000000000FFFFFFF07FFFFFE01C0C7D9023>61 D<001F800000E0700001000800060006 000800010008000100100F00802030C0402060404040C0202041C01C2041C01C2081801C108380 1C1083801C1083801C1083801C1083801C1083801C1081801C1041C01C1041C01C1040C03C2020 605C202030CC40100F0780080000000800000006000070010001C000E01F00001FF0001C207D9F 23>64 D<000100000003800000038000000380000007C0000007C0000007C0000009E0000009E0 000009E0000010F0000010F0000010F00000207800002078000020780000403C0000403C000040 3C0000801E0000801E0000FFFE0001000F0001000F0001000F0002000780020007800200078004 0003C00E0003C01F0007E0FFC03FFE1F207F9F22>II<000FC0 40007030C001C009C0038005C0070003C00E0001C01E0000C01C0000C03C0000C07C0000407C00 004078000040F8000000F8000000F8000000F8000000F8000000F8000000F8000000F8000000F8 000000780000007C0000407C0000403C0000401C0000401E0000800E0000800700010003800200 01C0040000703800000FC0001A217D9F21>IIII<000FE0200078186000E004E0038002E007 0001E00F0000E01E0000601E0000603C0000603C0000207C00002078000020F8000000F8000000 F8000000F8000000F8000000F8000000F8000000F8007FFCF80003E0780001E07C0001E03C0001 E03C0001E01E0001E01E0001E00F0001E0070001E0038002E000E0046000781820000FE0001E21 7D9F24>III<0FFFC0007C00003C00003C00003C00003C00003C00003C00003C00003C 00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C 00203C00F83C00F83C00F83C00F0380040780040700030E0000F800012207E9E17>IIIII<001F800000F0F00001C0380007801E000F000F000E0007001E00 07803C0003C03C0003C07C0003E0780001E0780001E0F80001F0F80001F0F80001F0F80001F0F8 0001F0F80001F0F80001F0F80001F0F80001F0780001E07C0003E07C0003E03C0003C03C0003C0 1E0007800E0007000F000F0007801E0001C0380000F0F000001F80001C217D9F23>II<001F800000F0F00001C0380007801E000F000F000E0007001E0007803C 0003C03C0003C07C0003E07C0003E0780001E0F80001F0F80001F0F80001F0F80001F0F80001F0 F80001F0F80001F0F80001F0F80001F0780001E0780001E07C0003E03C0003C03C0F03C01E1087 800E2047000F204F0007A03E0001E0380000F0F010001FB0100000301000003830000038700000 3FF000001FE000001FE000000FC0000007801C297D9F23>II<07E0800C1980100780300380600180600180E00180E00080E00080E00080F00000F000 007800007F00003FF0001FFC000FFE0003FF00001F800007800003C00003C00001C08001C08001 C08001C08001C0C00180C00380E00300F00600CE0C0081F80012217D9F19>I<7FFFFFE0780F01 E0600F0060400F0020400F0020C00F0030800F0010800F0010800F0010800F0010000F0000000F 0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F000000 0F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000001F800007FFFE00 1C1F7E9E21>IIII<7FF83FF80FE00FC007C0070003C0020001E0040001F00C0000F00800007810 00007C1000003C2000003E4000001E4000000F8000000F8000000780000003C0000007E0000005 E0000009F0000018F8000010780000207C0000603C0000401E0000801F0001800F000100078002 0007C0070003C01F8007E0FFE01FFE1F1F7F9E22>II<7FFFF8 7C00F87000F06001E04001E0C003C0C003C0800780800F80800F00001E00001E00003C00003C00 00780000F80000F00001E00001E00003C00403C0040780040F80040F000C1E000C1E00083C0018 3C0018780038F801F8FFFFF8161F7D9E1C>II<08041008201020 1040204020804080408040B85CFC7EFC7E7C3E381C0F0E7B9F17>II<0C001E0033006180C0C080400A067A9E17>I<1FE000303000781800781C00300E00000E0000 0E00000E0000FE00078E001E0E00380E00780E00F00E10F00E10F00E10F01E10781E103867200F 83C014147E9317>97 D<0E0000FE00000E00000E00000E00000E00000E00000E00000E00000E00 000E00000E00000E3E000EC3800F01C00F00E00E00E00E00700E00700E00780E00780E00780E00 780E00780E00780E00700E00700E00E00F00E00D01C00CC300083E0015207F9F19>I<03F80E0C 1C1E381E380C70007000F000F000F000F000F000F00070007000380138011C020E0C03F010147E 9314>I<000380003F800003800003800003800003800003800003800003800003800003800003 8003E380061B801C0780380380380380700380700380F00380F00380F00380F00380F00380F003 807003807003803803803807801C07800E1B8003E3F815207E9F19>I<03F0000E1C001C0E0038 0700380700700700700380F00380F00380FFFF80F00000F00000F0000070000070000038008018 00800C010007060001F80011147F9314>I<007C00C6018F038F07060700070007000700070007 000700FFF007000700070007000700070007000700070007000700070007000700070007000700 07007FF01020809F0E>I<0000E003E3300E3C301C1C30380E00780F00780F00780F00780F0078 0F00380E001C1C001E380033E0002000002000003000003000003FFE001FFF800FFFC03001E060 0070C00030C00030C00030C000306000603000C01C038003FC00141F7F9417>I<0E0000FE0000 0E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E3E000E43000E8180 0F01C00F01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0 0E01C00E01C00E01C0FFE7FC16207F9F19>I<1C001E003E001E001C0000000000000000000000 00000E007E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E 00FFC00A1F809E0C>I<00E001F001F001F000E0000000000000000000000000007007F000F000 700070007000700070007000700070007000700070007000700070007000700070007000700070 00706070F060F0C061803F000C28829E0E>I<0E0000FE00000E00000E00000E00000E00000E00 000E00000E00000E00000E00000E00000E0FF00E03C00E03000E02000E04000E08000E10000E30 000E70000EF8000F38000E1C000E1E000E0E000E07000E07800E03800E03C00E03E0FFCFF81520 7F9F18>I<0E00FE000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E00 0E000E000E000E000E000E000E000E000E000E000E000E000E000E00FFE00B20809F0C>I<0E1F 01F000FE618618000E81C81C000F00F00E000F00F00E000E00E00E000E00E00E000E00E00E000E 00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E00 0E00E00E000E00E00E000E00E00E00FFE7FE7FE023147F9326>I<0E3E00FE43000E81800F01C0 0F01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0 0E01C00E01C0FFE7FC16147F9319>I<01F800070E001C03803801C03801C07000E07000E0F000 F0F000F0F000F0F000F0F000F0F000F07000E07000E03801C03801C01C0380070E0001F8001414 7F9317>I<0E3E00FEC3800F01C00F00E00E00E00E00F00E00700E00780E00780E00780E00780E 00780E00780E00700E00F00E00E00F01E00F01C00EC3000E3E000E00000E00000E00000E00000E 00000E00000E00000E0000FFE000151D7F9319>I<03E0800619801C05803C0780380380780380 700380F00380F00380F00380F00380F00380F003807003807803803803803807801C0B800E1380 03E380000380000380000380000380000380000380000380000380003FF8151D7E9318>I<0E78 FE8C0F1E0F1E0F0C0E000E000E000E000E000E000E000E000E000E000E000E000E000E00FFE00F 147F9312>I<1F9030704030C010C010C010E00078007F803FE00FF00070803880188018C018C0 18E030D0608F800D147E9312>I<020002000200060006000E000E003E00FFF80E000E000E000E 000E000E000E000E000E000E000E000E080E080E080E080E080610031001E00D1C7F9B12>I<0E 01C0FE1FC00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E 01C00E01C00E01C00E03C00603C0030DC001F1FC16147F9319>III<7FC3FC0F01E00701C00701800381 0001C20000E40000EC00007800003800003C00007C00004E000087000107000303800201C00601 E01E01E0FF07FE1714809318>II<3FFF380E200E201C40 384078407000E001E001C00380078007010E011E011C0338027006700EFFFE10147F9314>II E /Fn 50 124 df45 D<387CFEFEFE7C3807077C860F>I<00E00001E0000FE000FFE000F3E00003E00003E00003E000 03E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E000 03E00003E00003E00003E00003E00003E000FFFF80FFFF80111D7C9C1A>49 D<07F0001FFE00383F007C1F80FE0FC0FE0FC0FE0FE0FE07E07C07E03807E0000FE0000FC0000F C0001F80001F00003E0000780000F00000E00001C0000380600700600E00601C00E01FFFC03FFF C07FFFC0FFFFC0FFFFC0131D7D9C1A>I<01FC0007FF000E0F801E0FC03F07E03F07E03F07E03F 07E01E0FC0000FC0000F80001F0001FC0001FC00000F800007C00003E00003F00003F83803F87C 03F8FE03F8FE03F8FE03F0FC03F07807E03C0FC01FFF8003FC00151D7E9C1A>I<3803803FFF80 3FFF003FFE003FFC003FF0003F800030000030000030000030000033F80037FE003C1F00380F80 1007C00007C00007E00007E07807E0FC07E0FC07E0FC07E0FC07C0780FC0600F80381F001FFC00 07F000131D7D9C1A>53 D<387CFEFEFE7C38000000000000387CFEFEFE7C3807147C930F>58 D<0000E000000000E000000001F000000001F000000001F000000003F800000003F800000006FC 00000006FC0000000EFE0000000C7E0000000C7E000000183F000000183F000000303F80000030 1F800000701FC00000600FC00000600FC00000C007E00000FFFFE00001FFFFF000018003F00001 8003F000030001F800030001F800060001FC00060000FC000E0000FE00FFE00FFFE0FFE00FFFE0 231F7E9E28>65 DI<0007FC02003FFF0E00FE03DE03F000FE07 E0003E0FC0001E1F80001E3F00000E3F00000E7F0000067E0000067E000006FE000000FE000000 FE000000FE000000FE000000FE000000FE0000007E0000007E0000067F0000063F0000063F0000 0C1F80000C0FC0001807E0003803F0007000FE01C0003FFF800007FC001F1F7D9E26>IIII<0007FC0200003FFF 0E0000FE03DE0003F000FE0007E0003E000FC0001E001F80001E003F00000E003F00000E007F00 0006007E000006007E00000600FE00000000FE00000000FE00000000FE00000000FE00000000FE 003FFFE0FE003FFFE07E00007E007E00007E007F00007E003F00007E003F00007E001F80007E00 0FC0007E0007E0007E0003F000FE0000FE01FE00003FFF8E000007FC0600231F7D9E29>III76 DII<001FF80000FFFF0001F81F8007E007E00FC003F01F8001F81F 0000F83F0000FC7F0000FE7E00007E7E00007EFE00007FFE00007FFE00007FFE00007FFE00007F FE00007FFE00007FFE00007FFE00007F7E00007E7F0000FE7F0000FE3F0000FC3F8001FC1F8001 F80FC003F007E007E001F81F8000FFFF00001FF800201F7D9E27>II82 D<03FC080FFF381E03F83800F8700078700038F00038F00018F00018F800 00FC00007FC0007FFE003FFF801FFFE00FFFF007FFF000FFF80007F80000FC00007C00003CC000 3CC0003CC0003CE00038E00078F80070FE01E0E7FFC081FF00161F7D9E1D>I<7FFFFFFC7FFFFF FC7C07E07C7007E01C6007E00C6007E00CE007E00EC007E006C007E006C007E006C007E0060007 E0000007E0000007E0000007E0000007E0000007E0000007E0000007E0000007E0000007E00000 07E0000007E0000007E0000007E0000007E0000007E0000007E00003FFFFC003FFFFC01F1E7E9D 24>II89 D<3FFFFF803FFFFF803F803F003E007F0038007E003800FC 007001FC007001F8006003F0006007F0006007E000000FC000001FC000001F8000003F0000007F 0000007E000000FC000001FC018001F8018003F0018007F0018007E003800FC003801FC003001F 8007003F000F007F001F007E007F00FFFFFF00FFFFFF00191F7D9E20>I<07FC001FFF003F0F80 3F07C03F03E03F03E00C03E00003E0007FE007FBE01F03E03C03E07C03E0F803E0F803E0F803E0 FC05E07E0DE03FF8FE0FE07E17147F9319>97 DI<01FE0007FF801F0FC03E0FC03E0FC07C0FC07C0300FC0000FC0000FC0000FC0000 FC0000FC00007C00007E00003E00603F00C01F81C007FF0001FC0013147E9317>I<0007F80007 F80000F80000F80000F80000F80000F80000F80000F80000F80000F80000F801F8F80FFEF81F83 F83E01F87E00F87C00F87C00F8FC00F8FC00F8FC00F8FC00F8FC00F8FC00F87C00F87C00F87E00 F83E01F81F07F80FFEFF03F8FF18207E9F1D>I<01FE0007FF800F83C01E01E03E00F07C00F07C 00F8FC00F8FFFFF8FFFFF8FC0000FC0000FC00007C00007C00003E00181E00180F807007FFE000 FF8015147F9318>I<001F8000FFC001F3E003E7E003C7E007C7E007C3C007C00007C00007C000 07C00007C000FFFC00FFFC0007C00007C00007C00007C00007C00007C00007C00007C00007C000 07C00007C00007C00007C00007C00007C00007C0003FFC003FFC0013207F9F10>I<01FC3C07FF FE0F079E1E03DE3E03E03E03E03E03E03E03E03E03E01E03C00F07800FFF0009FC001800001800 001C00001FFF800FFFF007FFF81FFFFC3C007C70003EF0001EF0001EF0001E78003C78003C3F01 F80FFFE001FF00171E7F931A>II< 1C003E007F007F007F003E001C00000000000000000000000000FF00FF001F001F001F001F001F 001F001F001F001F001F001F001F001F001F001F001F00FFE0FFE00B217EA00E>I<0038007C00 FE00FE00FE007C003800000000000000000000000001FE01FE003E003E003E003E003E003E003E 003E003E003E003E003E003E003E003E003E003E003E003E003E303E783EFC3CFC7C78783FF01F C00F2A83A010>IIIII<01FF0007FFC01F83F03E00F83E00F87C007C7C007CFC007EFC007EFC007EFC007EFC 007EFC007E7C007C7C007C3E00F83E00F81F83F007FFC001FF0017147F931A>II114 D<0FE63FFE701E600EE006E006F800FFC07FF83FFC1FFE03FE001FC007C007E007F006F81EFFFC C7F010147E9315>I<01800180018003800380038007800F803F80FFFCFFFC0F800F800F800F80 0F800F800F800F800F800F800F860F860F860F860F8607CC03F801F00F1D7F9C14>II121 D<3FFFE03FFFE03C07C0380F80701F80603F00603E00607C0000F80001F80003F00003E06007C0 600F80601F80E03F00C03E01C07C03C0FFFFC0FFFFC013147F9317>II E /Fo 46 123 df<003F07E00001C09C18000380F018000701F03C000E01E03C000E 00E018000E00E000000E00E000000E00E000000E00E000000E00E00000FFFFFFFC000E00E01C00 0E00E01C000E00E01C000E00E01C000E00E01C000E00E01C000E00E01C000E00E01C000E00E01C 000E00E01C000E00E01C000E00E01C000E00E01C000E00E01C000E00E01C000E00E01C007FC7FC FF80211D809C23>14 D45 D<60F0F06004047C830C>I<000600000006 000000060000000F0000000F0000000F00000017800000178000001780000023C0000023C00000 23C0000041E0000041E0000041E0000080F0000080F0000180F8000100780001FFF80003007C00 02003C0002003C0006003E0004001E0004001E000C001F001E001F00FF80FFF01C1D7F9C1F>65 DI<001F808000E0618001801980070007800E0003801C0003801C 00018038000180780000807800008070000080F0000000F0000000F0000000F0000000F0000000 F0000000F0000000F0000000700000807800008078000080380000801C0001001C0001000E0002 00070004000180080000E03000001FC000191E7E9C1E>III< FFFFF80F00780F00180F00080F00080F000C0F00040F00040F02040F02000F02000F02000F0600 0FFE000F06000F02000F02000F02000F02000F00000F00000F00000F00000F00000F00000F0000 0F8000FFF800161C7E9B1B>I<001F808000E0618001801980070007800E0003801C0003801C00 018038000180780000807800008070000080F0000000F0000000F0000000F0000000F0000000F0 000000F000FFF0F0000F80700007807800078078000780380007801C0007801C0007800E000780 07000B800180118000E06080001F80001C1E7E9C21>III78 D<003F800000E0E0000380380007001C000E000E001C0007003C00 078038000380780003C0780003C0700001C0F00001E0F00001E0F00001E0F00001E0F00001E0F0 0001E0F00001E0F00001E0700001C0780003C0780003C0380003803C0007801C0007000E000E00 07001C000380380000E0E000003F80001B1E7E9C20>I82 D<07E0801C1980300580 700380600180E00180E00080E00080E00080F00000F800007C00007FC0003FF8001FFE0007FF00 00FF80000F800007C00003C00001C08001C08001C08001C0C00180C00180E00300D00200CC0C00 83F800121E7E9C17>I<7FFFFFC0700F01C0600F00C0400F0040400F0040C00F0020800F002080 0F0020800F0020000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000 000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000001F80 0003FFFC001B1C7F9B1E>IIII89 D<1FC000307000783800781C00301C0000 1C00001C0001FC000F1C00381C00701C00601C00E01C40E01C40E01C40603C40304E801F870012 127E9115>97 DI<07E00C301878307870306000E000E000 E000E000E000E00060007004300418080C3007C00E127E9112>I<003F00000700000700000700 00070000070000070000070000070000070000070003E7000C1700180F00300700700700600700 E00700E00700E00700E00700E00700E00700600700700700300700180F000C370007C7E0131D7E 9C17>I<03E00C301818300C700E6006E006FFFEE000E000E000E00060007002300218040C1803 E00F127F9112>I<00F8018C071E061E0E0C0E000E000E000E000E000E00FFE00E000E000E000E 000E000E000E000E000E000E000E000E000E000E000E000E007FE00F1D809C0D>I<00038003C4 C00C38C01C3880181800381C00381C00381C00381C001818001C38000C300013C0001000003000 001800001FF8001FFF001FFF803003806001C0C000C0C000C0C000C06001803003001C0E0007F8 00121C7F9215>II<18003C003C00180000000000000000 00000000000000FC001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C00 1C00FF80091D7F9C0C>I<00C001E001E000C000000000000000000000000000000FE000E000E0 00E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E060E0F0 C0F1C061803E000B25839C0D>IIIII<03F0000E1C00180600300300700380600180E001 C0E001C0E001C0E001C0E001C0E001C06001807003803003001806000E1C0003F00012127F9115 >II114 D<1F9030704030C010C010E010F8007F803FE00FF000F88038 8018C018C018E010D0608FC00D127F9110>I<04000400040004000C000C001C003C00FFE01C00 1C001C001C001C001C001C001C001C001C101C101C101C101C100C100E2003C00C1A7F9910>I< FC1F801C03801C03801C03801C03801C03801C03801C03801C03801C03801C03801C03801C0380 1C03801C07800C07800E1B8003E3F014127F9117>III<7F8FF00F03800F030007020003840001C80001D80000F00000700000780000F800 009C00010E00020E000607000403801E07C0FF0FF81512809116>II<7FFC7038 6038407040F040E041C003C0038007000F040E041C043C0C380870087038FFF80E127F9112>I E end %%EndProlog %%BeginSetup %%Feature: *Resolution 300 TeXDict begin %%EndSetup %%Page: 1 1 bop 634 120 a Fo(NOR)m(THWESTERN)13 b(UNIVERSITY)617 170 y(Departmen)o(t)h (of)f(Electrical)h(Engineering)753 220 y(and)f(Computer)g(Science)332 789 y Fn(A)k(LIMITED)g(MEMOR)l(Y)g(ALGORITHM)h(F)o(OR)f(BOUND)547 846 y(CONSTRAINED)h(OPTIMIZA)l(TION)933 1058 y Fm(b)o(y)289 1162 y Fl(R)o(ichar)n(d)e(H.)g(Byr)n(d)621 1145 y Fk(1)641 1162 y Fl(,)g(Peihuang)g(Lu)926 1145 y Fk(2)946 1162 y Fl(,)33 b(Jor)n(ge)16 b(No)n(c)n(e)n(dal)1271 1145 y Fk(2)1305 1162 y Fl(and)h(Ciyou)f(Zhu)1604 1145 y Fk(2)694 1573 y Fm(T)l(ec)o(hnical)h(Rep)q (ort)e(NAM-08)767 1724 y(Revised:)21 b(Ma)o(y)15 b(1994)p 0 2253 766 2 v 52 2280 a Fj(1)81 2296 y Fi(Computer)e(Science)h(Departmen)o(t,) f(Univ)o(ersit)o(y)h(of)e(Colorado)h(at)g(Boulder,)g(Boulder)h(Colorado)g (80309.)j(This)c(author)g(w)o(as)0 2342 y(supp)q(orted)i(b)o(y)e(NSF)g(gran)o (t)g(CCR-9101795,)h(AR)o(O)e(gran)o(t)i(D)o(AAL)e(03-91-G-0151,)i(and)g(AF)o (OSR)f(gran)o(t)g(AF)o(OSR-90-0109.)52 2371 y Fj(2)86 2387 y Fi(Departmen)o(t)k(of)g(Electrical)i(Engineering)g(and)f(Computer)f (Science,)i(North)o(w)o(estern)e(Univ)o(ersit)o(y)m(,)i(Ev)n(anston)f(Il)f (60208;)0 2433 y(no)q(cedal@eecs.n)o(wu.edu.)22 b(These)14 b(authors)h(w)o(ere)f(supp)q(orted)h(b)o(y)f(National)i(Science)f(F)m (oundation)h(Gran)o(ts)f(CCR-9101359)g(and)0 2479 y(ASC-9213149,)f(and)f(b)o (y)h(Departmen)o(t)f(of)g(Energy)h(Gran)o(t)f(DE-F)o(G02-87ER25047-A004.)945 2603 y Fm(1)p eop %%Page: 1 2 bop 332 120 a Fn(A)17 b(LIMITED)g(MEMOR)l(Y)g(ALGORITHM)h(F)o(OR)f(BOUND)547 177 y(CONSTRAINED)h(OPTIMIZA)l(TION)933 389 y Fm(b)o(y)344 493 y Fl(R)o(ichar)n(d)f(H.)f(Byr)n(d,)g(Peihuang)h(Lu,)f(Jor)n(ge)g(No)n(c)n (e)n(dal)f(and)h(Ciyou)h(Zhu)829 963 y Fm(ABSTRA)o(CT)114 1098 y Fo(An)h(algorithm)e(for)i(solving)g(large)g(nonlinear)g(optimization)e (problems)i(with)g(simple)f(b)q(ounds)i(is)f(de-)114 1148 y(scrib)q(ed.)32 b(It)18 b(is)h(based)f(on)g(the)h(gradien)o(t)f(pro)r(jection)h(metho)q(d)e (and)i(uses)g(a)f(limited)e(memory)g(BF)o(GS)114 1197 y(matrix)c(to)i(appro)o (ximate)e(the)i(Hessian)h(of)f(the)g(ob)r(jectiv)o(e)h(function.)j(It)c(is)g (sho)o(wn)g(ho)o(w)g(to)g(tak)o(e)g(adv)n(an-)114 1247 y(tage)f(of)f(the)h (form)e(of)h(the)i(limited)c(memory)h(appro)o(ximation)e(to)k(implemen)o(t)d (the)k(algorithm)c(e\016cien)o(tly)m(.)114 1297 y(The)k(results)h(of)e(n)o (umerical)g(tests)i(on)f(a)f(set)i(of)e(large)h(problems)f(are)h(rep)q (orted.)0 1442 y Fl(Key)20 b(wor)n(ds:)28 b Fm(b)q(ound)20 b(constrained)g(optimization,)h(limited)g(memory)e(metho)q(d,)h(nonlinear)h (optimization,)0 1499 y(quasi-Newton)16 b(metho)q(d,)f(large-scale)h (optimization.)0 1575 y Fl(A)o(bbr)n(eviate)n(d)f(title:)20 b Fm(A)c(Limited)g(Memory)f(Metho)q(d)0 1751 y Fh(1.)j(In)n(tro)r(duction.)0 1832 y Fm(In)f(this)g(pap)q(er)g(w)o(e)g(describ)q(e)h(a)e(limited)j(memory)d (quasi-Newton)h(algorithm)g(for)f(solving)h(large)g(nonlinear)0 1888 y(optimization)f(problems)g(with)g(simple)g(b)q(ounds)g(on)f(the)h(v)m (ariables.)21 b(W)l(e)15 b(write)g(this)h(problem)g(as)875 1990 y(min)8 b Fg(f)d Fm(\()p Fg(x)p Fm(\))773 b(\(1)p Fg(:)p Fm(1\))731 2093 y(sub)s(ject)20 b(to)40 b Fg(l)13 b Ff(\024)g Fg(x)f Ff(\024)h Fg(u;)645 b Fm(\(1)p Fg(:)p Fm(2\))0 2176 y(where)20 b Fg(f)26 b Fm(:)21 b Ff(<)251 2160 y Fe(n)295 2176 y Ff(!)g(<)f Fm(is)h(a)f(nonlinear)h(function)g(whose)f(gradien)o(t)g Fg(g)i Fm(is)e(a)o(v)m(ailable,)j(the)d(v)o(ectors)g Fg(l)g Fm(and)g Fg(u)0 2232 y Fm(represen)o(t)f(lo)o(w)o(er)f(and)h(upp)q(er)g(b)q (ounds)h(on)e(the)h(v)m(ariables,)h(and)f(the)f(n)o(um)o(b)q(er)h(of)f(v)m (ariables)i Fg(n)f Fm(is)g(assumed)0 2289 y(to)d(b)q(e)h(large.)23 b(The)17 b(algorithm)f(do)q(es)h(not)f(require)h(second)g(deriv)m(ativ)o(es)h (or)d(kno)o(wledge)i(of)f(the)h(structure)f(of)0 2345 y(the)f(ob)s(jectiv)o (e)h(function,)g(and)f(can)h(therefore)f(b)q(e)h(applied)h(when)f(the)g (Hessian)g(matrix)f(is)h(not)f(practical)h(to)0 2402 y(compute.)k(A)15 b(limited)i(memory)d(quasi-Newton)h(up)q(date)h(is)f(used)h(to)e(appro)o (ximate)g(the)h(Hessian)h(matrix)f(in)0 2458 y(suc)o(h)h(a)e(w)o(a)o(y)h (that)f(the)h(storage)f(required)j(is)e(linear)i(in)f Fg(n)p Fm(.)945 2603 y(1)p eop %%Page: 2 3 bop 71 120 a Fm(The)15 b(algorithm)h(describ)q(ed)i(in)e(this)g(pap)q(er)g (is)g(similar)h(to)e(the)h(algorithms)f(prop)q(osed)h(b)o(y)g(Conn,)f(Gould)0 177 y(and)f(T)l(oin)o(t)f([9)o(])g(and)h(Mor)o(\023)-21 b(e)11 b(and)j(T)l(oraldo)f([20)o(],)g(in)h(that)f(the)g(gradien)o(t)h(pro)s (jection)f(metho)q(d)h(is)f(used)h(to)f(deter-)0 233 y(mine)k(a)e(set)h(of)f (activ)o(e)h(constrain)o(ts)f(at)g(eac)o(h)h(iteration.)22 b(Our)16 b(algorithm)g(is)g(distinguished)i(from)d(the)h(these)0 289 y(metho)q(ds)d(b)o(y)g(our)f(use)i(of)e(line)j(searc)o(hes)e(\(as)f(opp)q (osed)h(to)f(trust)h(regions\))f(but)h(mainly)h(b)o(y)f(our)g(use)g(of)f (limited)0 346 y(memory)18 b(BF)o(GS)g(matrices)g(to)g(appro)o(ximate)g(the)g (Hessian)i(of)e(the)g(ob)s(jectiv)o(e)h(function.)30 b(The)19 b(prop)q(erties)0 402 y(of)e(these)h(limited)i(memory)d(matrices)h(ha)o(v)o (e)f(far)g(reac)o(hing)h(consequences)h(in)g(the)f(implemen)o(tation)h(of)e (the)0 459 y(metho)q(d,)c(as)g(will)h(b)q(e)g(discussed)g(later)f(on.)19 b(W)l(e)13 b(\014nd)h(that)e(b)o(y)h(making)g(use)g(of)g(the)g(compact)f (represen)o(tations)0 515 y(of)g(limited)j(memory)d(matrices)h(describ)q(ed)i (b)o(y)e(Byrd,)g(No)q(cedal)h(and)f(Sc)o(hnab)q(el)i([6)o(],)e(the)g (computational)g(cost)0 572 y(of)i(one)g(iteration)h(of)e(the)i(algorithm)f (can)g(b)q(e)h(k)o(ept)f(to)g(b)q(e)g(of)g(order)g Fg(n)p Fm(.)71 628 y(W)l(e)h(used)h(the)g(gradien)o(t)f(pro)s(jection)g(approac)o(h)g([16)o (],)g([17],)f([3])h(to)g(determine)h(the)g(activ)o(e)f(set,)g(b)q(ecause)0 685 y(recen)o(t)k(studies)i([7)o(],)f([5)o(])f(indicate)i(that)e(it)g(p)q (ossess)h(go)q(o)q(d)f(theoretical)h(prop)q(erties,)h(and)e(b)q(ecause)i(it)e (also)0 741 y(app)q(ears)c(to)f(b)q(e)i(e\016cien)o(t)f(on)g(man)o(y)g(large) f(problems)i([8)o(],)f([20)o(].)21 b(Ho)o(w)o(ev)o(er)15 b(some)h(of)f(the)h (main)g(comp)q(onen)o(ts)0 798 y(of)i(our)g(algorithm)h(could)h(b)q(e)f (useful)g(in)h(other)e(framew)o(orks,)g(as)g(long)h(as)f(limited)i(memory)e (matrices)h(are)0 854 y(used)d(to)e(appro)o(ximate)h(the)g(Hessian)h(of)f (the)g(ob)s(jectiv)o(e)h(function.)0 1006 y Fh(2.)i(Outline)g(of)h(the)f (algorithm.)71 1086 y Fm(A)o(t)c(the)h(b)q(eginning)i(of)e(eac)o(h)g (iteration,)g(the)g(curren)o(t)g(iterate)g Fg(x)1179 1093 y Fe(k)1200 1086 y Fm(,)g(the)g(function)h(v)m(alue)g Fg(f)1623 1093 y Fe(k)1645 1086 y Fm(,)e(the)h(gradien)o(t)0 1143 y Fg(g)22 1150 y Fe(k)43 1143 y Fm(,)f(and)h(a)f(p)q(ositiv)o(e)h(de\014nite)h(limited) g(memory)e(appro)o(ximation)g Fg(B)1187 1150 y Fe(k)1223 1143 y Fm(are)g(giv)o(en.)20 b(This)15 b(allo)o(ws)g(us)f(to)g(form)g(a)0 1199 y(quadratic)h(mo)q(del)i(of)d Fg(f)21 b Fm(at)14 b Fg(x)514 1206 y Fe(k)535 1199 y Fm(,)419 1315 y Fg(m)459 1322 y Fe(k)480 1315 y Fm(\()p Fg(x)p Fm(\))e(=)h Fg(f)5 b Fm(\()p Fg(x)673 1322 y Fe(k)694 1315 y Fm(\))10 b(+)g Fg(g)791 1296 y Fe(T)789 1326 y(k)818 1315 y Fm(\()p Fg(x)g Ff(\000)g Fg(x)943 1322 y Fe(k)965 1315 y Fm(\))f(+)1043 1284 y(1)p 1043 1304 23 2 v 1043 1346 a(2)1070 1315 y(\()p Fg(x)h Ff(\000)h Fg(x)1196 1322 y Fe(k)1217 1315 y Fm(\))1235 1296 y Fe(T)1262 1315 y Fg(B)1296 1322 y Fe(k)1318 1315 y Fm(\()p Fg(x)f Ff(\000)g Fg(x)1443 1322 y Fe(k)1464 1315 y Fm(\))p Fg(:)325 b Fm(\(2)p Fg(:)p Fm(1\))0 1427 y(Just)17 b(as)f(in)h(the)g(metho)q(d)f(studied)i(b)o(y) f(Conn,)f(Gould)h(and)g(T)l(oin)o(t)f([9])g(the)g(algorithm)h(appro)o (ximately)f(min-)0 1483 y(imizes)22 b Fg(m)182 1490 y Fe(k)204 1483 y Fm(\()p Fg(x)p Fm(\))e(sub)s(ject)h(to)f(the)i(b)q(ounds)g(giv)o(en)f (b)o(y)g(\(1.2\).)36 b(This)22 b(is)f(done)h(b)o(y)f(\014rst)f(using)i(the)f (gradien)o(t)0 1540 y(pro)s(jection)13 b(metho)q(d)g(to)g(\014nd)h(a)e(set)h (of)g(activ)o(e)g(b)q(ounds,)h(follo)o(w)o(ed)g(b)o(y)f(a)g(minimization)i (of)e Fg(m)1606 1547 y Fe(k)1640 1540 y Fm(treating)g(those)0 1596 y(b)q(ounds)j(as)f(equalit)o(y)h(constrain)o(ts.)71 1653 y(T)l(o)e(do)i(this,)f(w)o(e)g(\014rst)g(consider)h(the)f(piece-wise)i (linear)g(path)724 1752 y Fg(x)p Fm(\()p Fg(t)p Fm(\))c(=)g Fg(P)6 b Fm(\()p Fg(x)942 1759 y Fe(k)973 1752 y Ff(\000)11 b Fg(tg)1057 1759 y Fe(k)1078 1752 y Fg(;)d(l)q(;)g(u)p Fm(\))p Fg(;)0 1851 y Fm(obtained)16 b(b)o(y)f(pro)s(jecting)g(the)g(steep)q(est)h (descen)o(t)f(direction)i(on)o(to)d(the)h(feasible)i(region,)e(where)619 2005 y Fg(P)6 b Fm(\()p Fg(x;)i(l)q(;)g(u)p Fm(\))799 2012 y Fe(i)823 2005 y Fm(=)871 1906 y Fd(8)871 1944 y(>)871 1956 y(<)871 2031 y(>)871 2043 y(:)929 1948 y Fg(l)943 1955 y Fe(i)1011 1948 y Fm(if)15 b Fg(x)1078 1955 y Fe(i)1105 1948 y Fg(<)e(l)1167 1955 y Fe(i)929 2005 y Fg(x)955 2012 y Fe(i)1011 2005 y Fm(if)i Fg(x)1078 2012 y Fe(i)1105 2005 y Ff(2)e Fm([)p Fg(l)1175 2012 y Fe(i)1188 2005 y Fg(;)8 b(u)1235 2012 y Fe(i)1249 2005 y Fm(])929 2061 y Fg(u)955 2068 y Fe(i)1011 2061 y Fm(if)15 b Fg(x)1078 2068 y Fe(i)1105 2061 y Fg(>)e(u)1179 2068 y Fe(i)1193 2061 y Fm(.)1820 2005 y(\(2)p Fg(:)p Fm(2\))0 2159 y(W)l(e)j(then)h(compute)f (the)h(generalized)h(Cauc)o(h)o(y)e(p)q(oin)o(t)h Fg(x)995 2142 y Fe(c)1012 2159 y Fm(,)f(whic)o(h)h(is)g(de\014ned)h(as)e(the)g (\014rst)g(lo)q(cal)h(minimizer)0 2215 y(of)e(the)g(univ)m(ariate,)h (piece-wise)i(quadratic)786 2314 y Fg(q)806 2321 y Fe(k)828 2314 y Fm(\()p Fg(t)p Fm(\))12 b(=)h Fg(m)980 2321 y Fe(k)1001 2314 y Fm(\()p Fg(x)p Fm(\()p Fg(t)p Fm(\)\))p Fg(:)0 2413 y Fm(The)i(v)m(ariables)h(whose)f(v)m(alue)h(at)e Fg(x)613 2397 y Fe(c)645 2413 y Fm(is)h(at)f(lo)o(w)o(er)h(or)f(upp)q(er)i(b)q(ound,)f (comprising)h(the)f(activ)o(e)g(set)f Ff(A)p Fm(\()p Fg(x)1790 2397 y Fe(c)1808 2413 y Fm(\),)g(are)0 2470 y(held)i(\014xed.)k(W)l(e)14 b(then)h(consider)g(the)f(follo)o(wing)h(quadratic)g(problem)g(o)o(v)o(er)e (the)h(subspace)h(of)f(free)g(v)m(ariables,)945 2603 y(2)p eop %%Page: 3 4 bop 612 177 a Fm(min)10 b Ff(f)p Fg(m)760 184 y Fe(k)781 177 y Fm(\()p Fg(x)p Fm(\))i(:)23 b Fg(x)917 184 y Fe(i)943 177 y Fm(=)13 b Fg(x)1017 158 y Fe(c)1017 188 y(i)1035 177 y Fg(;)27 b Ff(8)p Fg(i)13 b Ff(2)g(A)p Fm(\()p Fg(x)1252 158 y Fe(c)1269 177 y Fm(\))p Ff(g)510 b Fm(\(2)p Fg(:)p Fm(3\))590 260 y(sub)s(ject)21 b(to)29 b Fg(l)838 267 y Fe(i)865 260 y Ff(\024)13 b Fg(x)939 267 y Fe(i)965 260 y Ff(\024)g Fg(u)1039 267 y Fe(i)1099 260 y Ff(8)p Fg(i)k(=)-28 b Ff(2)13 b(A)p Fm(\()p Fg(x)1275 241 y Fe(c)1293 260 y Fm(\))p Fg(:)496 b Fm(\(2)p Fg(:)p Fm(4\))0 343 y(W)l(e)15 b(\014rst)g(solv)o(e)h(or)e(appro)o(ximately)i(solv)o(e)f (\(2.3\),)f(ignoring)i(the)f(b)q(ounds)h(on)f(the)h(free)f(v)m(ariables,)h (whic)o(h)g(can)0 400 y(b)q(e)h(accomplished)h(either)g(b)o(y)e(direct)h(or)f (iterativ)o(e)h(metho)q(ds)g(on)f(the)h(subspace)g(of)f(free)g(v)m(ariables,) i(or)e(b)o(y)h(a)0 456 y(dual)h(approac)o(h,)e(handling)i(the)f(activ)o(e)g (b)q(ounds)g(in)h(\(2.3\))d(b)o(y)i(Lagrange)f(m)o(ultipliers.)26 b(When)17 b(an)g(iterativ)o(e)0 513 y(metho)q(d)d(is)h(emplo)o(y)o(ed)g(w)o (e)e(use)i Fg(x)578 496 y Fe(c)609 513 y Fm(as)f(the)g(starting)g(p)q(oin)o (t)g(for)g(this)g(iteration.)20 b(W)l(e)14 b(then)g(truncate)g(the)g(path)0 569 y(to)o(w)o(ard)g(the)h(solution)h(so)f(as)g(to)f(satisfy)h(the)g(b)q (ounds)h(\(2.4\).)71 626 y(After)f(an)g(appro)o(ximate)g(solution)20 b(\026)-26 b Fg(x)715 633 y Fe(k)q Fk(+1)797 626 y Fm(of)15 b(this)h(problem)g(has)g(b)q(een)g(obtained,)g(w)o(e)g(compute)f(the)h(new)0 682 y(iterate)f Fg(x)170 689 y Fe(k)q Fk(+1)252 682 y Fm(b)o(y)g(a)g(line)h (searc)o(h)f(along)h Fg(d)722 689 y Fe(k)755 682 y Fm(=)g(\026)-26 b Fg(x)829 689 y Fe(k)q Fk(+1)906 682 y Ff(\000)10 b Fg(x)977 689 y Fe(k)1014 682 y Fm(that)k(satis\014es)i(the)f(su\016cien)o(t)h (decrease)f(condition)670 784 y Fg(f)5 b Fm(\()p Fg(x)741 791 y Fe(k)q Fk(+1)807 784 y Fm(\))12 b Ff(\024)h Fg(f)5 b Fm(\()p Fg(x)956 791 y Fe(k)977 784 y Fm(\))10 b(+)h Fg(\013\025)1107 791 y Fe(k)1128 784 y Fg(g)1152 766 y Fe(T)1150 796 y(k)1178 784 y Fg(d)1202 791 y Fe(k)1223 784 y Fg(;)584 b Fm(\(2)p Fg(:)p Fm(5\))0 886 y(and)15 b(that)g(also)g(attempts)f(to)h(enforce)g(the)g(curv)m (ature)h(condition)763 989 y Ff(j)p Fg(g)800 970 y Fe(T)798 1000 y(k)q Fk(+1)863 989 y Fg(d)887 996 y Fe(k)908 989 y Ff(j)c(\024)h Fg(\014)r Ff(j)p Fg(g)1046 970 y Fe(T)1044 1000 y(k)1073 989 y Fg(d)1097 996 y Fe(k)1118 989 y Ff(j)p Fg(;)676 b Fm(\(2)p Fg(:)p Fm(6\))0 1091 y(where)12 b Fg(\025)155 1098 y Fe(k)187 1091 y Fm(is)f(the)h(steplength)g(and)f Fg(\013)p Fm(,)h Fg(\014)h Fm(are)e(parameters)g(that)f(ha)o(v)o(e)h(the)g(v)m(alues)i(10)1447 1074 y Fc(\000)p Fk(4)1505 1091 y Fm(and)e(0)p Fg(:)p Fm(9,)g(resp)q(ectiv)o (ely)l(,)0 1147 y(in)20 b(our)f(co)q(de.)33 b(The)19 b(line)i(searc)o(h,)f (whic)o(h)g(ensures)g(that)e(the)i(iterates)f(remain)h(in)g(the)f(feasible)i (region,)f(is)0 1204 y(describ)q(ed)d(in)f Ff(x)p Fm(6.)j(W)l(e)c(then)g(ev)m (aluate)h(the)f(gradien)o(t)g(at)f Fg(x)1025 1211 y Fe(k)q Fk(+1)1091 1204 y Fm(,)h(compute)g(a)g(new)g(limited)h(memory)f(Hessian)0 1260 y(appro)o(ximation)g Fg(B)337 1267 y Fe(k)q Fk(+1)419 1260 y Fm(and)h(b)q(egin)g(a)f(new)g(iteration.)71 1316 y(Because)d(in)g(our) f(algorithm)g(ev)o(ery)g(Hessian)h(appro)o(ximation)f Fg(B)1181 1323 y Fe(k)1214 1316 y Fm(is)h(p)q(ositiv)o(e)g(de\014nite,)h(the)f(appro)o (ximate)0 1373 y(solution)19 b(\026)-26 b Fg(x)198 1380 y Fe(k)q Fk(+1)279 1373 y Fm(of)15 b(the)g(quadratic)h(problem)f(\(2.3\)-\(2.4\))d (de\014nes)17 b(a)d(descen)o(t)i(direction)g Fg(d)1567 1380 y Fe(k)1601 1373 y Fm(=)g(\026)-26 b Fg(x)1675 1380 y Fe(k)q Fk(+1)1751 1373 y Ff(\000)11 b Fg(x)1823 1380 y Fe(k)1859 1373 y Fm(for)0 1429 y(the)16 b(ob)s(jectiv)o(e)f(function)h Fg(f)5 b Fm(.)21 b(T)l(o)15 b(see)h(this,)f(\014rst)g(note)h(that)e(the)i (generalized)h(Cauc)o(h)o(y)e(p)q(oin)o(t)h Fg(x)1669 1413 y Fe(c)1686 1429 y Fm(,)f(whic)o(h)i(is)f(a)0 1486 y(minimizer)h(of)e Fg(m)304 1493 y Fe(k)325 1486 y Fm(\()p Fg(x)p Fm(\))g(on)g(the)h(pro)s (jected)f(steep)q(est)g(descen)o(t)h(direction,)g(satis\014es)g Fg(m)1489 1493 y Fe(k)1510 1486 y Fm(\()p Fg(x)1554 1493 y Fe(k)1575 1486 y Fm(\))c Fg(>)h(m)1693 1493 y Fe(k)1715 1486 y Fm(\()p Fg(x)1759 1469 y Fe(c)1776 1486 y Fm(\))i(if)g(the)0 1542 y(pro)s(jected)d(gradien)o(t)h(is)g(nonzero.)19 b(Since)14 b(the)f(p)q(oin)o(t)j(\026)-26 b Fg(x)937 1549 y Fe(k)q Fk(+1)1016 1542 y Fm(is)13 b(on)g(a)f(path)h(from)f Fg(x)1390 1526 y Fe(c)1420 1542 y Fm(to)g(the)g(minimizer)j(of)d(\(2.3\),)0 1599 y(along)i(whic)o(h)g Fg(m)288 1606 y Fe(k)323 1599 y Fm(decreases,)g(the)g(v)m(alue)h(of)f Fg(m)816 1606 y Fe(k)851 1599 y Fm(at)i(\026)-26 b Fg(x)931 1606 y Fe(k)q Fk(+1)1011 1599 y Fm(is)14 b(no)g(larger)g(than)f(its)h(v)m (alue)h(at)e Fg(x)1607 1582 y Fe(c)1625 1599 y Fm(.)19 b(Therefore)14 b(w)o(e)0 1655 y(ha)o(v)o(e)290 1712 y Fg(f)5 b Fm(\()p Fg(x)361 1719 y Fe(k)382 1712 y Fm(\))12 b(=)h Fg(m)500 1719 y Fe(k)521 1712 y Fm(\()p Fg(x)565 1719 y Fe(k)586 1712 y Fm(\))g Fg(>)g(m)705 1719 y Fe(k)726 1712 y Fm(\()p Fg(x)770 1693 y Fe(c)787 1712 y Fm(\))f Ff(\025)h Fg(m)905 1719 y Fe(k)926 1712 y Fm(\()s(\026)-26 b Fg(x)970 1719 y Fe(k)q Fk(+1)1036 1712 y Fm(\))13 b(=)g Fg(f)5 b Fm(\()p Fg(x)1186 1719 y Fe(k)1207 1712 y Fm(\))10 b(+)g Fg(g)1304 1693 y Fe(T)1302 1723 y(k)1331 1712 y Fg(d)1355 1719 y Fe(k)1386 1712 y Fm(+)1436 1694 y Fk(1)p 1436 1701 18 2 v 1436 1727 a(2)1459 1712 y Fg(d)1483 1693 y Fe(T)1483 1723 y(k)1510 1712 y Fg(B)1544 1719 y Fe(k)1566 1712 y Fg(d)1590 1719 y Fe(k)1611 1712 y Fg(:)0 1795 y Fm(This)17 b(inequalit)o(y)g(implies)h(that)d Fg(g)594 1779 y Fe(T)592 1808 y(k)621 1795 y Fg(d)645 1802 y Fe(k)680 1795 y Fg(<)f Fm(0)i(if)g Fg(B)844 1802 y Fe(k)882 1795 y Fm(is)h(p)q(ositiv)o(e)g(de\014nite)g(and)f Fg(d)1373 1802 y Fe(k)1410 1795 y Fm(is)g(not)g(zero.)22 b(In)16 b(this)h(pap)q(er)0 1852 y(w)o(e)c(do)g(not)f(presen)o(t)i(an)o(y)e(con)o(v)o(ergence)i(analysis) f(or)g(study)g(the)g(p)q(ossibilit)o(y)i(of)e(zigzagging.)20 b(Ho)o(w)o(ev)o(er,)12 b(giv)o(en)0 1908 y(the)j(use)g(of)g(gradien)o(t)g (pro)s(jection)g(in)h(the)f(step)g(computation)g(w)o(e)g(b)q(eliev)o(e)i (analyses)e(similar)h(to)f(those)g(in)g([7])0 1964 y(and)g([9)o(])g(should)g (b)q(e)h(p)q(ossible,)g(and)f(that)f(zigzagging)h(should)h(only)f(b)q(e)h(a)e (problem)h(in)h(the)f(degenerate)g(case.)71 2021 y(The)j(Hessian)h(appro)o (ximations)f Fg(B)694 2028 y Fe(k)734 2021 y Fm(used)h(in)g(our)f(algorithm)h (are)f(limited)i(memory)e(BF)o(GS)f(matrices)0 2077 y(\(No)q(cedal)j([21)o(]) g(and)g(Byrd,)g(No)q(cedal)h(and)f(Sc)o(hnab)q(el)h([6]\).)32 b(Ev)o(en)20 b(though)g(these)f(matrices)h(do)g(not)f(tak)o(e)0 2134 y(adv)m(an)o(tage)12 b(of)h(the)f(structure)h(of)f(the)h(problem,)h (they)f(require)g(only)g(a)g(small)g(amoun)o(t)f(of)h(storage)e(and,)i(as)g (w)o(e)0 2190 y(will)i(sho)o(w,)e(allo)o(w)h(the)f(computation)h(of)f(the)g (generalized)j(Cauc)o(h)o(y)d(p)q(oin)o(t)h(and)g(the)f(subspace)h (minimization)0 2247 y(to)20 b(b)q(e)h(p)q(erformed)f(in)h Fg(O)q Fm(\()p Fg(n)p Fm(\))f(op)q(erations.)36 b(The)20 b(new)h(algorithm)f (therefore)g(has)h(similar)g(computational)0 2303 y(demands)d(as)g(the)g (limited)i(memory)d(algorithm)h(\(L-BF)o(GS\))g(for)f(unconstrained)i (problems)g(describ)q(ed)g(b)o(y)0 2360 y(Liu)d(and)g(No)q(cedal)g([18)o(])f (and)g(Gilb)q(ert)i(and)e(Lemar)o(\023)-21 b(ec)o(hal)15 b([14)o(].)71 2416 y(In)e(the)g(next)g(three)f(sections)i(w)o(e)e(describ)q(e)j(in)e (detail)h(the)f(limited)h(memory)f(matrices,)g(the)f(computation)0 2473 y(of)j(the)g(Cauc)o(h)o(y)g(p)q(oin)o(t,)g(and)h(the)f(minimization)i (of)e(the)g(quadratic)h(problem)g(on)f(a)g(subspace.)945 2603 y(3)p eop %%Page: 4 5 bop 0 120 a Fh(3.)18 b(Limited)e(Memory)h(BF)n(GS)i(Matrices.)71 201 y Fm(In)f(our)g(algorithm,)g(the)g(limited)i(memory)d(BF)o(GS)g(matrices) h(are)g(represen)o(ted)g(in)h(the)f(compact)f(form)0 258 y(describ)q(ed)i(b)o (y)f(Byrd,)f(No)q(cedal)i(and)e(Sc)o(hnab)q(el)j([6)o(].)26 b(A)o(t)17 b(ev)o(ery)g(iterate)g Fg(x)1309 265 y Fe(k)1348 258 y Fm(the)h(algorithm)f(stores)g(a)g(small)0 314 y(n)o(um)o(b)q(er,)e(sa)o (y)g Fg(m)p Fm(,)f(of)h(correction)h(pairs)f Ff(f)p Fg(s)744 321 y Fe(i)758 314 y Fg(;)8 b(y)801 321 y Fe(i)815 314 y Ff(g)p Fg(;)17 b(i)12 b Fm(=)h Fg(k)e Ff(\000)g Fm(1)p Fg(;)d(:)g(:)g(:)t(;)g(k)j Ff(\000)f Fg(m)p Fm(,)15 b(where)625 416 y Fg(s)646 423 y Fe(k)681 416 y Fm(=)e Fg(x)755 423 y Fe(k)q Fk(+1)831 416 y Ff(\000)e Fg(x)903 423 y Fe(k)924 416 y Fg(;)47 b(y)1006 423 y Fe(k)1041 416 y Fm(=)13 b Fg(g)1111 423 y Fe(k)q Fk(+1)1187 416 y Ff(\000)d Fg(g)1254 423 y Fe(k)1275 416 y Fg(:)0 519 y Fm(These)h(correction)h(pairs)f (con)o(tain)g(information)g(ab)q(out)g(the)g(curv)m(ature)g(of)g(the)g (function,)h(and)f(in)h(conjunction)0 575 y(with)k(the)f(BF)o(GS)g(form)o (ula,)f(de\014ne)i(the)g(limited)h(memory)d(iteration)i(matrix)f Fg(B)1404 582 y Fe(k)1425 575 y Fm(.)20 b(The)c(question)g(is)f(ho)o(w)g(to)0 632 y(b)q(est)g(represen)o(t)h(these)f(matrices)g(without)g(explicitly)j (forming)e(them.)71 688 y(In)g([6])f(it)i(is)g(prop)q(osed)f(to)g(use)g(a)g (compact)g(\(or)f(outer)h(pro)q(duct\))g(form)g(to)f(de\014ne)j(the)e (limited)i(memory)0 744 y(matrix)d Fg(B)182 751 y Fe(k)219 744 y Fm(in)h(terms)e(of)h(the)h Fg(n)10 b Ff(\002)g Fg(m)15 b Fm(correction)h(matrices)432 847 y Fg(Y)458 854 y Fe(k)493 847 y Fm(=)d([)o Fg(y)575 854 y Fe(k)q Fc(\000)p Fe(m)656 847 y Fg(;)8 b(:)g(:)g(:)t(;)g(y)779 854 y Fe(k)q Fc(\000)p Fk(1)845 847 y Fm(])g Fg(;)157 b(S)1064 854 y Fe(k)1098 847 y Fm(=)13 b([)o Fg(s)1179 854 y Fe(k)q Fc(\000)p Fe(m)1259 847 y Fg(;)8 b(:)g(:)g(:)d(;)j(s)1382 854 y Fe(k)q Fc(\000)p Fk(1)1448 847 y Fm(])f Fg(:)339 b Fm(\(3)p Fg(:)p Fm(1\))0 949 y(More)11 b(sp)q(eci\014cally)m(,)k(it)d(is)h(sho)o(wn)f(in)g([6])f(that)h(if)g Fg(\022)i Fm(is)f(a)e(p)q(ositiv)o(e)j(scaling)f(parameter,)e(and)h(if)h(the) f Fg(m)g Fm(correction)0 1005 y(pairs)i Ff(f)p Fg(s)154 1012 y Fe(i)169 1005 y Fg(;)8 b(y)212 1012 y Fe(i)225 1005 y Ff(g)248 986 y Fe(k)q Fc(\000)p Fe(m)248 1019 y(i)p Fk(=)p Fe(k)q Fc(\000)p Fk(1)354 1005 y Fm(,)14 b(satisfy)g Fg(s)543 989 y Fe(T)543 1017 y(i)570 1005 y Fg(y)592 1012 y Fe(i)619 1005 y Fg(>)f Fm(0,)h(then)h(the)f(matrix)g(obtained)h(b)o(y)f(up)q(dating)h Fg(\022)q(I)j(m)p Fm(-times,)c(using)h(the)0 1069 y(BF)o(GS)g(form)o(ula)f (and)i(the)f(pairs)h Ff(f)p Fg(s)626 1076 y Fe(i)640 1069 y Fg(;)8 b(y)683 1076 y Fe(i)696 1069 y Ff(g)719 1050 y Fe(k)q Fc(\000)p Fe(m)719 1083 y(i)p Fk(=)p Fe(k)q Fc(\000)p Fk(1)840 1069 y Fm(can)15 b(b)q(e)h(written)g(as)744 1178 y Fg(B)778 1185 y Fe(k)813 1178 y Fm(=)d Fg(\022)q(I)h Ff(\000)d Fg(W)1006 1185 y Fe(k)1027 1178 y Fg(M)1071 1185 y Fe(k)1092 1178 y Fg(W)1141 1159 y Fe(T)1135 1189 y(k)1820 1178 y Fm(\(3)p Fg(:)p Fm(2\))0 1280 y(where)759 1336 y Fg(W)802 1343 y Fe(k)836 1336 y Fm(=)884 1289 y Fd(h)925 1336 y Fg(Y)951 1343 y Fe(k)1014 1336 y Fg(\022)q(S)1064 1343 y Fe(k)1107 1289 y Fd(i)1134 1336 y Fg(;)673 b Fm(\(3)p Fg(:)p Fm(3\))682 1506 y Fg(M)726 1513 y Fe(k)760 1506 y Fm(=)808 1434 y Fd(")853 1477 y Ff(\000)p Fg(D)926 1484 y Fe(k)1025 1477 y Fg(L)1056 1461 y Fe(T)1056 1491 y(k)874 1534 y Fg(L)905 1541 y Fe(k)989 1534 y Fg(\022)q(S)1042 1517 y Fe(T)1039 1547 y(k)1070 1534 y Fg(S)1098 1541 y Fe(k)1140 1434 y Fd(#)1164 1445 y Fc(\000)p Fk(1)1211 1506 y Fg(;)596 b Fm(\(3)p Fg(:)p Fm(4\))0 1619 y(and)15 b(where)h Fg(L)251 1626 y Fe(k)287 1619 y Fm(and)g Fg(D)414 1626 y Fe(k)450 1619 y Fm(are)f(the)g Fg(m)10 b Ff(\002)g Fg(m)15 b Fm(matrices)461 1750 y(\()p Fg(L)510 1757 y Fe(k)531 1750 y Fm(\))549 1757 y Fe(i;j)601 1750 y Fm(=)649 1678 y Fd(\()703 1722 y Fm(\()p Fg(s)742 1729 y Fe(k)q Fc(\000)p Fe(m)p Fc(\000)p Fk(1+)p Fe(i)907 1722 y Fm(\))925 1705 y Fe(T)952 1722 y Fm(\()p Fg(y)992 1729 y Fe(k)q Fc(\000)p Fe(m)p Fc(\000)p Fk(1+)p Fe(j)1160 1722 y Fm(\))42 b(if)15 b Fg(i)e(>)g(j)703 1778 y Fm(0)494 b(otherwise)p Fg(;)1820 1750 y Fm(\(3)p Fg(:)p Fm(5\))580 1900 y Fg(D)618 1907 y Fe(k)651 1900 y Fm(=)13 b Fg(diag)793 1853 y Fd(h)813 1900 y Fg(s)834 1881 y Fe(T)834 1911 y(k)q Fc(\000)p Fe(m)914 1900 y Fg(y)936 1907 y Fe(k)q Fc(\000)p Fe(m)1017 1900 y Fg(;)8 b(:)g(:)g(:)t(;)g(s)1139 1881 y Fe(T)1139 1911 y(k)q Fc(\000)p Fk(1)1205 1900 y Fg(y)1227 1907 y Fe(k)q Fc(\000)p Fk(1)1294 1853 y Fd(i)1321 1900 y Fg(:)486 b Fm(\(3)p Fg(:)p Fm(6\))0 1991 y(\(W)l(e)19 b(should)h(p)q(oin)o(t)f(out)g (that)f(\(3.2\))f(is)j(a)e(sligh)o(t)i(rearrangemen)o(t)e(of)g(equation)i (\(3.5\))d(in)j([6)o(]\).)30 b(Note)19 b(that)0 2047 y(since)c Fg(M)154 2054 y Fe(k)189 2047 y Fm(is)f(a)f(2)p Fg(m)7 b Ff(\002)g Fm(2)p Fg(m)14 b Fm(matrix,)f(and)h(since)g Fg(m)g Fm(is)g(c)o(hosen)g(to)f (b)q(e)h(a)g(small)g(in)o(teger,)g(the)f(cost)h(of)f(computing)0 2104 y(the)h(in)o(v)o(erse)g(in)h(\(3.4\))d(is)i(negligible.)22 b(It)14 b(is)h(sho)o(wn)e(in)i([6)o(])e(that)h(b)o(y)f(using)i(the)f(compact) f(represen)o(tation)h(\(3.2\))0 2160 y(v)m(arious)h(computations)f(in)o(v)o (olving)i Fg(B)665 2167 y Fe(k)701 2160 y Fm(b)q(ecome)f(inexp)q(ensiv)o(e.) 22 b(In)15 b(particular,)g(the)g(pro)q(duct)g(of)f Fg(B)1734 2167 y Fe(k)1770 2160 y Fm(times)h(a)0 2217 y(v)o(ector,)f(whic)o(h)i(o)q (ccurs)g(often)f(in)h(the)f(algorithm)g(of)g(this)h(pap)q(er,)f(can)g(b)q(e)h (p)q(erformed)f(e\016cien)o(tly)l(.)71 2273 y(There)k(is)g(a)f(similar)i (represen)o(tation)f(of)g(the)f(in)o(v)o(erse)i(limited)g(memory)f(BF)o(GS)f (matrix)g Fg(H)1708 2280 y Fe(k)1748 2273 y Fm(that)g(ap-)0 2330 y(pro)o(ximates)d(the)g(in)o(v)o(erse)h(of)e(the)i(Hessian)g(matrix:)731 2448 y Fg(H)769 2455 y Fe(k)803 2448 y Ff(\021)856 2418 y Fm(1)p 856 2438 23 2 v 856 2479 a Fg(\022)884 2448 y(I)d Fm(+)976 2437 y(\026)963 2448 y Fg(W)1006 2455 y Fe(k)1044 2437 y Fm(\026)1027 2448 y Fg(M)1071 2455 y Fe(k)1106 2437 y Fm(\026)1093 2448 y Fg(W)1142 2430 y Fe(T)1136 2460 y(k)1169 2448 y Fg(;)638 b Fm(\(3)p Fg(:)p Fm(7\))945 2603 y(4)p eop %%Page: 5 6 bop 0 120 a Fm(where)774 165 y(\026)761 177 y Fg(W)804 184 y Fe(k)838 177 y Ff(\021)886 129 y Fd(h)931 159 y Fk(1)p 931 166 18 2 v 931 192 a Fe(\022)954 177 y Fg(Y)980 184 y Fe(k)1043 177 y Fg(S)1071 184 y Fe(k)1113 177 y Fg(;)1134 129 y Fd(i)538 366 y Fm(\026)521 378 y Fg(M)565 385 y Fe(k)599 378 y Ff(\021)647 281 y Fd(2)647 354 y(6)647 381 y(4)747 321 y Fm(0)264 b Ff(\000)p Fg(R)1104 302 y Fc(\000)p Fk(1)1104 335 y Fe(k)696 436 y Ff(\000)p Fg(R)766 417 y Fc(\000)p Fe(T)766 450 y(k)862 436 y Fg(R)897 417 y Fc(\000)p Fe(T)897 450 y(k)952 436 y Fm(\()p Fg(D)1008 443 y Fe(k)1039 436 y Fm(+)1089 418 y Fk(1)p 1089 425 V 1089 452 a Fe(\022)1112 436 y Fg(Y)1149 419 y Fe(T)1138 449 y(k)1176 436 y Fg(Y)1202 443 y Fe(k)1224 436 y Fm(\))p Fg(R)1277 417 y Fc(\000)p Fk(1)1277 450 y Fe(k)1344 281 y Fd(3)1344 354 y(7)1344 381 y(5)1379 378 y Fg(;)0 535 y Fm(and)459 613 y(\()p Fg(R)512 620 y Fe(k)532 613 y Fm(\))550 620 y Fe(i;j)603 613 y Fm(=)651 541 y Fd(\()705 585 y Fm(\()p Fg(s)744 592 y Fe(k)q Fc(\000)p Fe(m)p Fc(\000)p Fk(1+)p Fe(i)908 585 y Fm(\))926 568 y Fe(T)954 585 y Fm(\()p Fg(y)994 592 y Fe(k)q Fc(\000)p Fe(m)p Fc(\000)p Fk(1+)p Fe(j)1162 585 y Fm(\))41 b(if)16 b Fg(i)c Ff(\024)h Fg(j)705 641 y Fm(0)493 b(otherwise.)1820 613 y(\(3)p Fg(:)p Fm(8\))0 729 y(\(W)l(e)15 b(note)g(that)f(\(3.7\))g(is)i(a)f(sligh)o(t)g (rearrangemen)o(t)g(of)f(equation)i(\(3.1\))d(in)j([6]\).)71 785 y(Represen)o(tations)21 b(of)e(limited)k(memory)c(matrices)i(analogous)f (to)f(these)i(also)f(exist)h(for)f(other)g(quasi-)0 842 y(Newton)15 b(up)q(date)g(form)o(ulae,)f(suc)o(h)h(as)g(SR1)g(or)f(DFP)h(\(see)f([6]\),)g (and)h(in)g(principle)j(could)e(b)q(e)f(used)h(for)e Fg(B)1839 849 y Fe(k)1876 842 y Fm(in)0 898 y(our)e(algorithm.)19 b(W)l(e)13 b(consider)g(here)g(only)g(BF)o(GS)f(since)h(w)o(e)f(ha)o(v)o(e)g (considerable)i(computational)f(exp)q(erience)0 955 y(in)j(the)f (unconstrained)i(case)e(indicating)i(the)e(limited)i(memory)e(BF)o(GS)g(p)q (erforms)f(w)o(ell.)71 1011 y(Since)k(the)f(b)q(ounds)g(on)g(the)g(problem)g (ma)o(y)f(prev)o(en)o(t)h(the)f(line)j(searc)o(h)d(from)g(satisfying)h (\(2.6\))e(\(see)i Ff(x)p Fm(6\),)0 1068 y(w)o(e)g(cannot)g(guaran)o(tee)g (that)g(the)g(condition)i Fg(s)835 1051 y Fe(T)835 1081 y(k)863 1068 y Fg(y)885 1075 y Fe(k)923 1068 y Fg(>)d Fm(0)h(alw)o(a)o(ys)g(holds)h (\(cf.)26 b(Dennis)19 b(and)e(Sc)o(hnab)q(el)i([12)o(]\).)0 1124 y(Therefore)d(to)f(main)o(tain)h(the)f(p)q(ositiv)o(e)i(de\014niteness)g (of)f(the)f(limited)j(memory)d(BF)o(GS)g(matrix,)g(w)o(e)h(discard)0 1181 y(a)f(correction)g(pair)h Ff(f)p Fg(s)388 1188 y Fe(k)409 1181 y Fg(;)8 b(y)452 1188 y Fe(k)473 1181 y Ff(g)15 b Fm(if)h(the)f(curv)m (ature)g(condition)799 1283 y Fg(s)820 1264 y Fe(T)820 1294 y(k)848 1283 y Fg(y)870 1290 y Fe(k)904 1283 y Fg(>)e Fm(eps)d Ff(k)p Fg(y)r Ff(k)1095 1264 y Fk(2)1820 1283 y Fm(\(3)p Fg(:)p Fm(9\))0 1385 y(is)20 b(not)g(satis\014ed)g(for)g(a)f(small)i(p)q(ositiv)o(e) g(constan)o(t)e(eps.)34 b(If)20 b(this)g(happ)q(ens)h(w)o(e)f(do)g(not)f (delete)i(the)f(oldest)0 1441 y(correction)11 b(pair,)g(as)f(is)h(normally)g (done)g(in)g(limited)i(memory)d(up)q(dating.)19 b(This)11 b(means)f(that)g (the)h Fg(m)f Fm(directions)0 1498 y(in)16 b Fg(S)81 1505 y Fe(k)117 1498 y Fm(and)g Fg(Y)232 1505 y Fe(k)269 1498 y Fm(ma)o(y)e (actually)i(include)i(some)d(with)g(indices)i(less)f(than)f Fg(k)c Ff(\000)g Fg(m)p Fm(.)0 1654 y Fh(4.)37 b(The)18 b(generalized)f(Cauc) n(h)n(y)j(p)r(oin)n(t.)71 1735 y Fm(The)e(ob)s(jectiv)o(e)h(of)f(the)h(pro)q (cedure)h(describ)q(ed)g(in)g(this)f(section)g(is)g(to)f(\014nd)i(the)e (\014rst)h(lo)q(cal)g(minimizer)0 1792 y(of)d(the)g(quadratic)g(mo)q(del)h (along)g(the)f(piece-wise)i(linear)g(path)e(obtained)g(b)o(y)g(pro)s(jecting) g(p)q(oin)o(ts)h(along)f(the)0 1848 y(steep)q(est)f(descen)o(t)g(direction,)g Fg(x)560 1855 y Fe(k)590 1848 y Ff(\000)10 b Fg(tg)673 1855 y Fe(k)694 1848 y Fm(,)k(on)o(to)g(the)g(feasible)i(region.)k(W)l(e)15 b(de\014ne)g Fg(x)1448 1832 y Fk(0)1481 1848 y Fm(=)e Fg(x)1555 1855 y Fe(k)1591 1848 y Fm(and,)h(throughout)0 1905 y(this)21 b(section,)h(drop)e(the)g(index)i Fg(k)f Fm(of)f(the)h(outer)f(iteration,)h (so)f(that)g Fg(g)r(;)8 b(x)19 b Fm(and)h Fg(B)j Fm(stand)e(for)e Fg(g)1731 1912 y Fe(k)1752 1905 y Fg(;)8 b(x)1799 1912 y Fe(k)1840 1905 y Fm(and)0 1961 y Fg(B)34 1968 y Fe(k)56 1961 y Fm(.)31 b(W)l(e)19 b(use)h(subscripts)f(to)g(denote)g(the)g(comp)q(onen)o(ts)g(of)g (a)g(v)o(ector;)g(for)g(example)h Fg(g)1557 1968 y Fe(i)1589 1961 y Fm(denotes)g(the)f Fg(i)p Fm(-th)0 2017 y(comp)q(onen)o(t)e(of)f Fg(g)r Fm(.)23 b(Sup)q(erscripts)18 b(will)g(b)q(e)f(used)g(to)f(represen)o (t)h(iterates)f(during)h(the)g(piece-wise)i(searc)o(h)d(for)0 2074 y(the)f(Cauc)o(h)o(y)g(p)q(oin)o(t.)71 2130 y(T)l(o)f(de\014ne)j(the)e (breakp)q(oin)o(ts)h(in)g(eac)o(h)f(co)q(ordinate)h(direction)g(w)o(e)f (compute)631 2288 y Fg(t)647 2295 y Fe(i)674 2288 y Fm(=)722 2189 y Fd(8)722 2226 y(>)722 2239 y(<)722 2314 y(>)722 2326 y(:)780 2231 y Fm(\()p Fg(x)824 2215 y Fk(0)824 2243 y Fe(i)853 2231 y Ff(\000)c Fg(u)925 2238 y Fe(i)939 2231 y Fm(\))p Fg(=g)1002 2238 y Fe(i)1057 2231 y Fm(if)k Fg(g)1120 2238 y Fe(i)1147 2231 y Fg(<)e Fm(0)780 2288 y(\()p Fg(x)824 2271 y Fk(0)824 2300 y Fe(i)853 2288 y Ff(\000)e Fg(l)913 2295 y Fe(i)926 2288 y Fm(\))p Fg(=g)989 2295 y Fe(i)1057 2288 y Fm(if)k Fg(g)1120 2295 y Fe(i)1147 2288 y Fg(>)e Fm(0)780 2344 y Ff(1)232 b Fm(otherwise,)1820 2288 y(\(4)p Fg(:)p Fm(1\))945 2603 y(5)p eop %%Page: 6 7 bop 0 120 a Fm(and)12 b(sort)f Ff(f)p Fg(t)212 127 y Fe(i)226 120 y Fg(;)d(i)k Fm(=)h(1)p Fg(;)8 b(:)g(:)g(:)t(;)g(n)p Ff(g)j Fm(in)i(increasing)g(order)f(to)f(obtain)h(the)g(ordered)h(set)e Ff(f)p Fg(t)1416 104 y Fe(j)1447 120 y Fm(:)i Fg(t)1489 104 y Fe(j)1520 120 y Ff(\024)g Fg(t)1584 104 y Fe(j)r Fk(+1)1648 120 y Fg(;)8 b(j)14 b Fm(=)f(1)p Fg(;)8 b(:::;)g(n)p Ff(g)p Fm(.)0 177 y(This)15 b(is)f(done)h(b)o(y)f(means)g(of)f(a)h(heapsort)f (algorithm)h([1].)19 b(W)l(e)14 b(then)g(searc)o(h)g(along)g Fg(P)6 b Fm(\()p Fg(x)1527 160 y Fk(0)1555 177 y Ff(\000)i Fg(tg)r(;)g(l)q(;)g(u)p Fm(\),)j(a)j(piece-)0 233 y(wise)i(linear)g(path)f (whic)o(h)h(can)g(b)q(e)f(expressed)h(as)636 365 y Fg(x)662 372 y Fe(i)676 365 y Fm(\()p Fg(t)p Fm(\))d(=)788 293 y Fd(\()843 337 y Fg(x)869 320 y Fk(0)869 349 y Fe(i)898 337 y Ff(\000)e Fg(tg)982 344 y Fe(i)1052 337 y Fm(if)k Fg(t)e Ff(\024)g Fg(t)1186 344 y Fe(i)843 393 y Fg(x)869 377 y Fk(0)869 406 y Fe(i)898 393 y Ff(\000)e Fg(t)960 400 y Fe(i)974 393 y Fg(g)996 400 y Fe(i)1052 393 y Fm(otherwise.)0 504 y(Supp)q(ose)16 b(that)f(w)o(e)g(are)g (examining)h(the)f(in)o(terv)m(al)i([)p Fg(t)910 488 y Fe(j)r Fc(\000)p Fk(1)973 504 y Fg(;)8 b(t)1010 488 y Fe(j)1028 504 y Fm(].)19 b(Let)d(us)f(de\014ne)h(the)g(\()p Fg(j)c Ff(\000)e Fm(1\)-th)15 b(breakp)q(oin)o(t)g(as)811 607 y Fg(x)837 588 y Fe(j)r Fc(\000)p Fk(1)913 607 y Fm(=)e Fg(x)p Fm(\()p Fg(t)1021 588 y Fe(j)r Fc(\000)p Fk(1)1085 607 y Fm(\))0 709 y(so)i(that)f(on)h([)p Fg(t)246 692 y Fe(j)r Fc(\000)p Fk(1)310 709 y Fg(;)8 b(t)347 692 y Fe(j)365 709 y Fm(])738 765 y Fg(x)p Fm(\()p Fg(t)p Fm(\))k(=)h Fg(x)902 746 y Fe(j)r Fc(\000)p Fk(1)976 765 y Fm(+)d(\001)p Fg(td)1099 746 y Fe(j)r Fc(\000)p Fk(1)1163 765 y Fg(;)0 849 y Fm(where)823 905 y(\001)p Fg(t)j Fm(=)g Fg(t)e Ff(\000)f Fg(t)1026 886 y Fe(j)r Fc(\000)p Fk(1)0 988 y Fm(and)617 1066 y Fg(d)641 1044 y Fe(j)r Fc(\000)p Fk(1)641 1079 y Fe(i)717 1066 y Fm(=)765 994 y Fd(\()819 1038 y Ff(\000)p Fg(g)876 1045 y Fe(i)932 1038 y Fm(if)16 b Fg(t)990 1021 y Fe(j)r Fc(\000)p Fk(1)1066 1038 y Fg(<)d(t)1130 1045 y Fe(i)819 1094 y Fm(0)90 b(otherwise)p Fg(:)1820 1066 y Fm(\(4)p Fg(:)p Fm(2\))71 1187 y(Using)15 b(this)h(notation)f(w)o(e)g(write)g(the)g(quadratic)h(\(2.1\),)d (on)i(the)g(line)i(segmen)o(t)e([)p Fg(x)p Fm(\()p Fg(t)1531 1170 y Fe(j)r Fc(\000)p Fk(1)1594 1187 y Fm(\))p Fg(;)8 b(x)p Fm(\()p Fg(t)1693 1170 y Fe(j)1710 1187 y Fm(\)],)14 b(as)222 1289 y Fg(m)p Fm(\()p Fg(x)p Fm(\))41 b(=)h Fg(f)15 b Fm(+)10 b Fg(g)548 1270 y Fe(T)575 1289 y Fm(\()p Fg(x)g Ff(\000)g Fg(x)700 1270 y Fk(0)720 1289 y Fm(\))g(+)798 1271 y Fk(1)p 798 1278 18 2 v 798 1305 a(2)821 1289 y Fm(\()p Fg(x)g Ff(\000)g Fg(x)946 1270 y Fk(0)966 1289 y Fm(\))984 1270 y Fe(T)1011 1289 y Fg(B)r Fm(\()p Fg(x)h Ff(\000)f Fg(x)1173 1270 y Fk(0)1193 1289 y Fm(\))365 1363 y(=)42 b Fg(f)15 b Fm(+)10 b Fg(g)548 1344 y Fe(T)575 1363 y Fm(\()p Fg(z)616 1344 y Fe(j)r Fc(\000)p Fk(1)689 1363 y Fm(+)h(\001)p Fg(td)813 1344 y Fe(j)r Fc(\000)p Fk(1)876 1363 y Fm(\))f(+)954 1345 y Fk(1)p 954 1352 V 954 1378 a(2)977 1363 y Fm(\()p Fg(z)1018 1344 y Fe(j)r Fc(\000)p Fk(1)1091 1363 y Fm(+)h(\001)p Fg(td)1215 1344 y Fe(j)r Fc(\000)p Fk(1)1278 1363 y Fm(\))1296 1344 y Fe(T)1323 1363 y Fg(B)r Fm(\()p Fg(z)1400 1344 y Fe(j)r Fc(\000)p Fk(1)1474 1363 y Fm(+)g(\001)p Fg(td)1598 1344 y Fe(j)r Fc(\000)p Fk(1)1661 1363 y Fm(\))p Fg(;)0 1465 y Fm(where)782 1521 y Fg(z)805 1502 y Fe(j)r Fc(\000)p Fk(1)881 1521 y Fm(=)i Fg(x)955 1502 y Fe(j)r Fc(\000)p Fk(1)1028 1521 y Ff(\000)e Fg(x)1100 1502 y Fk(0)1119 1521 y Fg(:)688 b Fm(\(4)p Fg(:)p Fm(3\))0 1605 y(Therefore)15 b(on)g(the)g(line)i(segmen)o(t)e([)p Fg(x)p Fm(\()p Fg(t)682 1588 y Fe(j)r Fc(\000)p Fk(1)745 1605 y Fm(\))p Fg(;)8 b(x)p Fm(\()p Fg(t)844 1588 y Fe(j)861 1605 y Fm(\)],)14 b Fg(m)p Fm(\()p Fg(x)p Fm(\))h(can)g(b)q(e)h(written)f(as)g(a)g(quadratic)g(in)h (\001)p Fg(t)p Fm(,)265 1716 y(^)-31 b Fg(m)p Fm(\(\001)p Fg(t)p Fm(\))41 b(=)h(\()p Fg(f)15 b Fm(+)10 b Fg(g)629 1697 y Fe(T)656 1716 y Fg(z)679 1697 y Fe(j)r Fc(\000)p Fk(1)752 1716 y Fm(+)803 1698 y Fk(1)p 803 1705 V 803 1732 a(2)825 1716 y Fg(z)848 1697 y Fe(j)r Fc(\000)p Fk(1)912 1688 y Fe(T)939 1716 y Fg(B)r(z)998 1697 y Fe(j)r Fc(\000)p Fk(1)1063 1716 y Fm(\))f(+)i(\()p Fg(g)1178 1697 y Fe(T)1204 1716 y Fg(d)1228 1697 y Fe(j)r Fc(\000)p Fk(1)1302 1716 y Fm(+)f Fg(d)1371 1697 y Fe(j)r Fc(\000)p Fk(1)1434 1688 y Fe(T)1462 1716 y Fg(B)r(z)1521 1697 y Fe(j)r Fc(\000)p Fk(1)1585 1716 y Fm(\)\001)p Fg(t)505 1799 y Fm(+)545 1781 y Fk(1)p 545 1788 V 545 1814 a(2)568 1799 y Fm(\()p Fg(d)610 1780 y Fe(j)r Fc(\000)p Fk(1)672 1771 y Fe(T)700 1799 y Fg(B)r(d)760 1780 y Fe(j)r Fc(\000)p Fk(1)824 1799 y Fm(\)\001)p Fg(t)896 1780 y Fk(2)428 1868 y Ff(\021)42 b Fg(f)527 1875 y Fe(j)r Fc(\000)p Fk(1)600 1868 y Fm(+)11 b Fg(f)673 1849 y Fc(0)668 1879 y Fe(j)r Fc(\000)p Fk(1)731 1868 y Fm(\001)p Fg(t)g Fm(+)846 1850 y Fk(1)p 846 1857 V 846 1883 a(2)869 1868 y Fg(f)896 1849 y Fc(00)891 1879 y Fe(j)r Fc(\000)p Fk(1)954 1868 y Fm(\001)p Fg(t)1008 1849 y Fk(2)1028 1868 y Fg(;)0 1970 y Fm(where)k(the)h(parameters)e(of)h (this)g(one-dimensional)j(quadratic)d(are)603 2076 y Fg(f)625 2083 y Fe(j)r Fc(\000)p Fk(1)702 2076 y Fm(=)d Fg(f)k Fm(+)10 b Fg(g)856 2060 y Fe(T)883 2076 y Fg(z)906 2060 y Fe(j)r Fc(\000)p Fk(1)980 2076 y Fm(+)1030 2058 y Fk(1)p 1030 2065 V 1030 2092 a(2)1053 2076 y Fg(z)1076 2060 y Fe(j)r Fc(\000)p Fk(1)1139 2051 y Fe(T)1167 2076 y Fg(B)r(z)1226 2060 y Fe(j)r Fc(\000)p Fk(1)1290 2076 y Fg(;)658 2157 y(f)685 2140 y Fc(0)680 2169 y Fe(j)r Fc(\000)p Fk(1)756 2157 y Fm(=)j Fg(g)828 2140 y Fe(T)855 2157 y Fg(d)879 2140 y Fe(j)r Fc(\000)p Fk(1)952 2157 y Fm(+)e Fg(d)1022 2140 y Fe(j)r Fc(\000)p Fk(1)1085 2131 y Fe(T)1112 2157 y Fg(B)r(z)1171 2140 y Fe(j)r Fc(\000)p Fk(1)1235 2157 y Fg(;)572 b Fm(\(4.4\))758 2240 y Fg(f)785 2223 y Fc(00)780 2252 y Fe(j)r Fc(\000)p Fk(1)856 2240 y Fm(=)13 b Fg(d)928 2223 y Fe(j)r Fc(\000)p Fk(1)991 2214 y Fe(T)1019 2240 y Fg(B)r(d)1079 2223 y Fe(j)r Fc(\000)p Fk(1)1143 2240 y Fg(:)664 b Fm(\(4.5\))0 2342 y(Di\013eren)o(tiating)27 b(^)-31 b Fg(m)o Fm(\(\001)p Fg(t)p Fm(\))18 b(and)h(equating)f(to)g(zero,)h(w)o(e)e(obtain)i(\001)p Fg(t)1173 2324 y Fc(\003)1211 2342 y Fm(=)f Ff(\000)p Fg(f)1326 2325 y Fc(0)1321 2354 y Fe(j)r Fc(\000)p Fk(1)1385 2342 y Fg(=f)1435 2325 y Fc(00)1430 2354 y Fe(j)r Fc(\000)p Fk(1)1493 2342 y Fm(.)29 b(Since)20 b Fg(B)h Fm(is)d(p)q(ositiv)o(e)0 2403 y(de\014nite,)e (this)e(de\014nes)i(a)e(minimizer)i(pro)o(vided)f Fg(t)860 2386 y Fe(j)r Fc(\000)p Fk(1)932 2403 y Fm(+)9 b(\001)p Fg(t)1030 2385 y Fc(\003)1065 2403 y Fm(lies)15 b(on)f([)p Fg(t)1233 2386 y Fe(j)r Fc(\000)p Fk(1)1297 2403 y Fg(;)8 b(t)1334 2386 y Fe(j)1352 2403 y Fm(\).)19 b(Otherwise)c(the)g(generalized)0 2459 y(Cauc)o(h)o(y)g(p)q(oin)o(t)h(lies)g(at)f Fg(x)p Fm(\()p Fg(t)476 2443 y Fe(j)r Fc(\000)p Fk(1)539 2459 y Fm(\))g(if)h Fg(f)641 2443 y Fc(0)636 2472 y Fe(j)r Fc(\000)p Fk(1)712 2459 y Ff(\025)d Fm(0,)h(and)i(b)q(ey)o(ond)f(or)g(at)g Fg(x)p Fm(\()p Fg(t)1228 2443 y Fe(j)1246 2459 y Fm(\))g(if)h Fg(f)1348 2443 y Fc(0)1343 2472 y Fe(j)r Fc(\000)p Fk(1)1419 2459 y Fg(<)d Fm(0.)945 2603 y(6)p eop %%Page: 7 8 bop 71 120 a Fm(If)16 b(the)h(generalized)h(Cauc)o(h)o(y)e(p)q(oin)o(t)h(has) f(not)h(b)q(een)g(found)g(after)f(exploring)h(the)g(in)o(terv)m(al)g([)p Fg(t)1701 104 y Fe(j)r Fc(\000)p Fk(1)1765 120 y Fg(;)8 b(t)1802 104 y Fe(j)1820 120 y Fm(],)16 b(w)o(e)0 177 y(set)467 233 y Fg(x)493 214 y Fe(j)524 233 y Fm(=)d Fg(x)598 214 y Fe(j)r Fc(\000)p Fk(1)671 233 y Fm(+)e(\001)p Fg(t)771 214 y Fe(j)r Fc(\000)p Fk(1)834 233 y Fg(d)858 214 y Fe(j)r Fc(\000)p Fk(1)921 233 y Fg(;)144 b Fm(\001)p Fg(t)1132 214 y Fe(j)r Fc(\000)p Fk(1)1208 233 y Fm(=)13 b Fg(t)1272 214 y Fe(j)1301 233 y Ff(\000)d Fg(t)1362 214 y Fe(j)r Fc(\000)p Fk(1)1426 233 y Fg(;)381 b Fm(\(4)p Fg(:)p Fm(6\))0 315 y(and)14 b(up)q(date)h(the)g(directional)h (deriv)m(ativ)o(es)f Fg(f)793 299 y Fc(0)788 327 y Fe(j)821 315 y Fm(and)f Fg(f)935 299 y Fc(00)930 327 y Fe(j)971 315 y Fm(as)g(the)g(searc)o(h)g(mo)o(v)o(es)g(to)f(the)i(next)f(in)o(terv)m(al.) 21 b(Let)14 b(us)0 376 y(assume)i(for)g(the)h(momen)o(t)e(that)h(only)h(one)g (v)m(ariable)g(b)q(ecomes)g(activ)o(e)g(at)f Fg(t)1330 360 y Fe(j)1348 376 y Fm(,)h(and)f(let)h(us)g(denote)f(its)h(index)0 433 y(b)o(y)e Fg(b)p Fm(.)k(Then)d Fg(t)250 440 y Fe(b)280 433 y Fm(=)d Fg(t)344 416 y Fe(j)363 433 y Fm(,)i(and)g(w)o(e)g(zero)g(out)g (the)g(corresp)q(onding)h(comp)q(onen)o(t)g(of)f(the)g(searc)o(h)g (direction,)785 533 y Fg(d)809 514 y Fe(j)840 533 y Fm(=)e Fg(d)912 514 y Fe(j)r Fc(\000)p Fk(1)985 533 y Fm(+)e Fg(g)1053 540 y Fe(b)1069 533 y Fg(e)1090 540 y Fe(b)1108 533 y Fg(;)699 b Fm(\(4)p Fg(:)p Fm(7\))0 632 y(where)15 b Fg(e)152 639 y Fe(b)185 632 y Fm(is)h(the)f Fg(b)p Fm(-th)g(unit)h(v)o(ector.)j(F)l(rom)14 b(the)i(de\014nitions)g(\(4.3\))e(and)h(\(4.6\))f(w)o(e)h(ha)o(v)o(e)726 732 y Fg(z)749 713 y Fe(j)780 732 y Fm(=)e Fg(z)851 713 y Fe(j)r Fc(\000)p Fk(1)925 732 y Fm(+)d(\001)p Fg(t)1024 713 y Fe(j)r Fc(\000)p Fk(1)1088 732 y Fg(d)1112 713 y Fe(j)r Fc(\000)p Fk(1)1175 732 y Fg(:)632 b Fm(\(4)p Fg(:)p Fm(8\))0 832 y(Therefore)15 b(using)h(\(4.4\),)d(\(4.5\),)g(\(4.7\))h(and)h(\(4.8\))f(w)o(e)h(obtain)296 941 y Fg(f)323 922 y Fc(0)318 952 y Fe(j)378 941 y Fm(=)42 b Fg(g)479 922 y Fe(T)506 941 y Fg(d)530 922 y Fe(j)558 941 y Fm(+)10 b Fg(d)627 922 y Fe(j)645 913 y(T)673 941 y Fg(B)r(z)732 922 y Fe(j)378 1026 y Fm(=)42 b Fg(g)479 1007 y Fe(T)506 1026 y Fg(d)530 1007 y Fe(j)r Fc(\000)p Fk(1)603 1026 y Fm(+)10 b Fg(g)672 1007 y Fk(2)670 1037 y Fe(b)701 1026 y Fm(+)h Fg(d)771 1007 y Fe(j)r Fc(\000)p Fk(1)834 998 y Fe(T)861 1026 y Fg(B)r(z)920 1007 y Fe(j)r Fc(\000)p Fk(1)995 1026 y Fm(+)f(\001)p Fg(t)1094 1007 y Fe(j)r Fc(\000)p Fk(1)1158 1026 y Fg(d)1182 1007 y Fe(j)r Fc(\000)p Fk(1)1245 998 y Fe(T)1272 1026 y Fg(B)r(d)1332 1007 y Fe(j)r Fc(\000)p Fk(1)1406 1026 y Fm(+)h Fg(g)1474 1033 y Fe(b)1491 1026 y Fg(e)1512 1007 y Fe(T)1512 1037 y(b)1539 1026 y Fg(B)r(z)1598 1007 y Fe(j)378 1095 y Fm(=)42 b Fg(f)482 1076 y Fc(0)477 1106 y Fe(j)r Fc(\000)p Fk(1)551 1095 y Fm(+)10 b(\001)p Fg(t)650 1076 y Fe(j)r Fc(\000)p Fk(1)714 1095 y Fg(f)741 1076 y Fc(00)736 1106 y Fe(j)r Fc(\000)p Fk(1)809 1095 y Fm(+)h Fg(g)879 1076 y Fk(2)877 1106 y Fe(b)908 1095 y Fm(+)f Fg(g)975 1102 y Fe(b)992 1095 y Fg(e)1013 1076 y Fe(T)1013 1106 y(b)1041 1095 y Fg(B)r(z)1100 1076 y Fe(j)1820 1095 y Fm(\(4.9\))0 1194 y(and)490 1294 y Fg(f)517 1275 y Fc(00)512 1305 y Fe(j)580 1294 y Fm(=)41 b Fg(d)680 1275 y Fe(j)698 1267 y(T)726 1294 y Fg(B)r(d)786 1275 y Fe(j)580 1379 y Fm(=)g Fg(d)680 1360 y Fe(j)r Fc(\000)p Fk(1)743 1351 y Fe(T)771 1379 y Fg(B)r(d)831 1360 y Fe(j)r Fc(\000)p Fk(1)905 1379 y Fm(+)10 b(2)p Fg(g)995 1386 y Fe(b)1012 1379 y Fg(e)1033 1360 y Fe(T)1033 1390 y(b)1060 1379 y Fg(B)r(d)1120 1360 y Fe(j)r Fc(\000)p Fk(1)1194 1379 y Fm(+)h Fg(g)1264 1360 y Fk(2)1262 1390 y Fe(b)1283 1379 y Fg(e)1304 1386 y Fe(b)1321 1360 y(T)1349 1379 y Fg(B)r(e)1406 1386 y Fe(b)580 1448 y Fm(=)41 b Fg(f)683 1429 y Fc(00)678 1459 y Fe(j)r Fc(\000)p Fk(1)752 1448 y Fm(+)11 b(2)p Fg(g)843 1455 y Fe(b)859 1448 y Fg(e)880 1429 y Fe(T)880 1459 y(b)908 1448 y Fg(B)r(d)968 1429 y Fe(j)r Fc(\000)p Fk(1)1042 1448 y Fm(+)f Fg(g)1111 1429 y Fk(2)1109 1459 y Fe(b)1130 1448 y Fg(e)1151 1455 y Fe(b)1168 1429 y(T)1196 1448 y Fg(B)r(e)1253 1455 y Fe(b)1271 1448 y Fg(:)513 b Fm(\(4.10\))0 1547 y(The)15 b(only)h(exp)q(ensiv)o(e)h(computations)e(in)h(\(4)p Fg(:)p Fm(9\))e(and)h(\(4)p Fg(:)p Fm(10\))e(are)647 1647 y Fg(e)668 1654 y Fe(b)686 1628 y(T)713 1647 y Fg(B)r(z)772 1628 y Fe(j)791 1647 y Fg(;)53 b(e)878 1654 y Fe(b)895 1628 y(T)923 1647 y Fg(B)r(d)983 1628 y Fe(j)r Fc(\000)p Fk(1)1047 1647 y Fg(;)f(e)1133 1654 y Fe(b)1151 1628 y(T)1178 1647 y Fg(B)r(e)1235 1654 y Fe(b)1253 1647 y Fg(;)0 1747 y Fm(whic)o(h)14 b(can)g(require)h Fg(O)q Fm(\()p Fg(n)p Fm(\))e(op)q(erations)g(since)i Fg(B)h Fm(is)f(a)e(dense)h(limited)i(memory)d(matrix.)19 b(Therefore)13 b(it)h(w)o(ould)0 1803 y(app)q(ear)f(that)f(the)h(the)g(computation)g(of)f (the)h(generalized)i(Cauc)o(h)o(y)e(p)q(oin)o(t)g(could)h(require)f Fg(O)q Fm(\()p Fg(n)1644 1787 y Fk(2)1664 1803 y Fm(\))g(op)q(erations,)0 1860 y(since)h(in)f(the)g(w)o(orst)f(case)g Fg(n)h Fm(segmen)o(ts)g(of)f(the) h(piece-wise)i(linear)e(path)g(can)g(b)q(e)g(examined.)20 b(This)14 b(cost)e(w)o(ould)0 1916 y(b)q(e)17 b(prohibitiv)o(e)h(for)e(large)h (problems.)24 b(Ho)o(w)o(ev)o(er,)16 b(using)h(the)g(limited)h(memory)e(BF)o (GS)g(form)o(ula)g(\(3.2\))f(and)0 1973 y(the)g(de\014nition)i(\(4.2\),)d (the)h(up)q(dating)h(form)o(ulae)f(\(4.9\)-\(4.10\))d(b)q(ecome)434 2072 y Fg(f)461 2054 y Fc(0)456 2084 y Fe(j)487 2072 y Fm(=)h Fg(f)562 2054 y Fc(0)557 2084 y Fe(j)r Fc(\000)p Fk(1)631 2072 y Fm(+)d(\001)p Fg(t)730 2054 y Fe(j)r Fc(\000)p Fk(1)794 2072 y Fg(f)821 2054 y Fc(00)816 2084 y Fe(j)r Fc(\000)p Fk(1)890 2072 y Fm(+)g Fg(g)959 2054 y Fk(2)957 2084 y Fe(b)988 2072 y Fm(+)h Fg(\022)q(g)1078 2079 y Fe(b)1095 2072 y Fg(z)1118 2050 y Fe(j)1116 2086 y(b)1146 2072 y Ff(\000)g Fg(g)1214 2079 y Fe(b)1231 2072 y Fg(w)1265 2054 y Fe(T)1264 2084 y(b)1292 2072 y Fg(M)5 b(W)1390 2054 y Fe(T)1418 2072 y Fg(z)1441 2054 y Fe(j)1459 2072 y Fg(;)325 b Fm(\(4)p Fg(:)p Fm(11\))392 2179 y Fg(f)419 2160 y Fc(00)414 2190 y Fe(j)453 2179 y Fm(=)13 b Fg(f)528 2160 y Fc(00)523 2190 y Fe(j)r Fc(\000)p Fk(1)596 2179 y Ff(\000)e Fm(2)p Fg(\022)q(g)711 2160 y Fk(2)709 2190 y Fe(b)740 2179 y Ff(\000)g Fm(2)p Fg(g)831 2186 y Fe(b)847 2179 y Fg(w)881 2160 y Fe(T)880 2190 y(b)909 2179 y Fg(M)5 b(W)1007 2160 y Fe(T)1035 2179 y Fg(d)1059 2160 y Fe(j)r Fc(\000)p Fk(1)1132 2179 y Fm(+)10 b Fg(\022)q(g)1223 2160 y Fk(2)1221 2190 y Fe(b)1253 2179 y Ff(\000)g Fg(g)1322 2160 y Fk(2)1320 2190 y Fe(b)1341 2179 y Fg(w)1375 2160 y Fe(T)1374 2190 y(b)1403 2179 y Fg(M)5 b(w)1485 2186 y Fe(b)1501 2179 y Fg(;)283 b Fm(\(4)p Fg(:)p Fm(12\))0 2266 y(where)13 b Fg(w)163 2249 y Fe(T)162 2279 y(b)204 2266 y Fm(stands)g(for)f(the)h Fg(b)p Fm(-th)g(ro)o(w)f(of)h (the)g(matrix)g Fg(W)6 b Fm(.)19 b(The)14 b(only)f Fg(O)q Fm(\()p Fg(n)p Fm(\))g(op)q(erations)g(remaining)h(in)g(\(4.11\))0 2322 y(and)k(\(4.12\))d(are)j Fg(W)352 2306 y Fe(T)379 2322 y Fg(z)402 2306 y Fe(j)438 2322 y Fm(and)g Fg(W)578 2306 y Fe(T)606 2322 y Fg(d)630 2306 y Fe(j)r Fc(\000)p Fk(1)693 2322 y Fm(.)27 b(W)l(e)18 b(note,)f(ho)o(w)o(ev)o(er,)g(from)g(\(4.7\))f(and)i (\(4.8\))e(that)h Fg(z)1667 2306 y Fe(j)1702 2322 y Fm(and)h Fg(d)1817 2306 y Fe(j)1853 2322 y Fm(are)0 2379 y(up)q(dated)13 b(at)f(ev)o(ery)g(iteration)h(b)o(y)f(a)h(simple)h(computation.)k(Therefore) 13 b(if)g(w)o(e)f(main)o(tain)h(the)f(t)o(w)o(o)f(2)p Fg(m)p Fm(-v)o(ectors)498 2479 y Fg(p)521 2460 y Fe(j)552 2479 y Ff(\021)i Fg(W)649 2460 y Fe(T)677 2479 y Fg(d)701 2460 y Fe(j)732 2479 y Fm(=)f Fg(W)828 2460 y Fe(T)856 2479 y Fm(\()p Fg(d)898 2460 y Fe(j)r Fc(\000)p Fk(1)971 2479 y Fm(+)e Fg(g)1038 2486 y Fe(b)1055 2479 y Fg(e)1076 2486 y Fe(b)1094 2479 y Fm(\))i(=)h Fg(p)1195 2460 y Fe(j)r Fc(\000)p Fk(1)1268 2479 y Fm(+)e Fg(g)1336 2486 y Fe(b)1353 2479 y Fg(w)1386 2486 y Fe(b)1402 2479 y Fg(;)945 2603 y Fm(7)p eop %%Page: 8 9 bop 381 120 a Fg(c)401 101 y Fe(j)431 120 y Ff(\021)13 b Fg(W)528 101 y Fe(T)556 120 y Fg(z)579 101 y Fe(j)610 120 y Fm(=)g Fg(W)707 101 y Fe(T)735 120 y Fm(\()p Fg(z)776 101 y Fe(j)r Fc(\000)p Fk(1)849 120 y Fm(+)e(\001)p Fg(t)949 101 y Fe(j)r Fc(\000)p Fk(1)1012 120 y Fg(d)1036 101 y Fe(j)r Fc(\000)p Fk(1)1099 120 y Fm(\))i(=)g Fg(c)1198 101 y Fe(j)r Fc(\000)p Fk(1)1271 120 y Fm(+)d(\001)p Fg(t)1370 101 y Fe(j)r Fc(\000)p Fk(1)1434 120 y Fg(p)1457 101 y Fe(j)r Fc(\000)p Fk(1)1520 120 y Fg(;)0 191 y Fm(then)16 b(up)q(dating)g Fg(f)324 175 y Fc(0)319 203 y Fe(j)352 191 y Fm(and)g Fg(f)468 175 y Fc(00)463 203 y Fe(j)504 191 y Fm(using)g(the)g(expressions)478 279 y Fg(f)505 260 y Fc(0)500 290 y Fe(j)531 279 y Fm(=)d Fg(f)606 260 y Fc(0)601 290 y Fe(j)r Fc(\000)p Fk(1)675 279 y Fm(+)d(\001)p Fg(t)774 260 y Fe(j)r Fc(\000)p Fk(1)838 279 y Fg(f)865 260 y Fc(00)860 290 y Fe(j)r Fc(\000)p Fk(1)933 279 y Fm(+)h Fg(g)1003 260 y Fk(2)1001 290 y Fe(b)1032 279 y Fm(+)f Fg(\022)q(g)1121 286 y Fe(b)1139 279 y Fg(z)1162 257 y Fe(j)1160 293 y(b)1190 279 y Ff(\000)h Fg(g)1258 286 y Fe(b)1275 279 y Fg(w)1309 260 y Fe(T)1308 290 y(b)1336 279 y Fg(M)5 b(c)1405 260 y Fe(j)1423 279 y Fg(;)506 363 y(f)533 344 y Fc(00)528 375 y Fe(j)567 363 y Fm(=)13 b Fg(f)642 344 y Fc(00)637 375 y Fe(j)r Fc(\000)p Fk(1)711 363 y Ff(\000)d Fg(\022)q(g)802 344 y Fk(2)800 375 y Fe(b)832 363 y Ff(\000)h Fm(2)p Fg(g)923 370 y Fe(b)939 363 y Fg(w)973 344 y Fe(T)972 375 y(b)1001 363 y Fg(M)5 b(p)1073 344 y Fe(j)r Fc(\000)p Fk(1)1146 363 y Ff(\000)10 b Fg(g)1215 344 y Fk(2)1213 375 y Fe(b)1234 363 y Fg(w)1268 344 y Fe(T)1267 375 y(b)1296 363 y Fg(M)5 b(w)1378 370 y Fe(b)1394 363 y Fg(;)0 434 y Fm(will)17 b(require)e(only)h Fg(O)q Fm(\()p Fg(m)432 418 y Fk(2)451 434 y Fm(\))f(op)q(erations.)k(If)d(more)e(than)h(one)g(v)m (ariable)h(b)q(ecomes)g(activ)o(e)f(at)f Fg(t)1623 418 y Fe(j)1656 434 y Fm({)h(an)g(at)o(ypical)0 491 y(situation)21 b({)f(w)o(e)g(rep)q(eat)h (the)f(up)q(dating)i(pro)q(cess)e(just)g(describ)q(ed,)k(b)q(efore)c (examining)i(the)f(new)f(in)o(terv)m(al)0 547 y([)p Fg(t)29 531 y Fe(j)47 547 y Fg(;)8 b(t)84 531 y Fe(j)r Fk(+1)147 547 y Fm(].)20 b(W)l(e)15 b(ha)o(v)o(e)g(th)o(us)g(b)q(een)h(able)g(to)f(ac)o (hiev)o(e)h(a)e(signi\014can)o(t)j(reduction)f(in)g(the)f(cost)g(of)g (computing)g(the)0 604 y(generalized)i(Cauc)o(h)o(y)e(p)q(oin)o(t.)0 677 y Fn(Remark.)k Fl(The)c(examination)g(of)h(the)g(\014rst)f(se)n(gment)f (of)i(the)g(pr)n(oje)n(cte)n(d)f(ste)n(ep)n(est)f(desc)n(ent)g(p)n(ath,)i (during)g(the)0 733 y(c)n(omputation)j(of)f(the)h(gener)n(alize)n(d)e(Cauchy) h(p)n(oint,)h(r)n(e)n(quir)n(es)e Fg(O)q Fm(\()p Fg(n)p Fm(\))h Fl(op)n(er)n(ations.)26 b(However)18 b(al)r(l)g(subse)n(quent)0 790 y(se)n(gments)f(r)n(e)n(quir)n(e)h(only)h Fg(O)q Fm(\()p Fg(m)541 773 y Fk(2)560 790 y Fm(\))f Fl(op)n(er)n(ations,)h(wher)n(e)g Fg(m)g Fl(is)f(the)h(numb)n(er)f(of)h(c)n(orr)n(e)n(ction)f(ve)n(ctors)h (stor)n(e)n(d)f(in)0 846 y(the)f(limite)n(d)e(memory)i(matrix.)71 919 y Fm(Since)h Fg(m)f Fm(is)g(usually)i(small,)f(sa)o(y)e(less)h(than)g (10,)g(the)g(cost)f(of)h(examining)h(all)g(segmen)o(ts)e(after)h(the)g (\014rst)0 976 y(one)g(is)f(negligible.)27 b(The)16 b(follo)o(wing)h (algorithm)g(describ)q(es)h(in)f(more)f(detail)h(ho)o(w)f(to)g(ac)o(hiev)o(e) h(these)f(sa)o(vings)0 1032 y(in)k(computation.)33 b(Note)19 b(that)g(it)g(is)h(not)f(necessary)h(to)f(k)o(eep)h(trac)o(k)e(of)h(the)h Fg(n)p Fm(-v)o(ector)f Fg(z)1593 1016 y Fe(j)1631 1032 y Fm(since)h(only)g (the)0 1089 y(comp)q(onen)o(t)e Fg(z)258 1067 y Fe(j)256 1103 y(b)294 1089 y Fm(corresp)q(onding)h(to)e(the)h(b)q(ound)g(that)f(has)h(b)q (ecome)h(activ)o(e)e(is)i(needed)g(to)e(up)q(date)h Fg(f)1792 1096 y Fe(j)1811 1070 y Fc(0)1840 1089 y Fm(and)0 1145 y Fg(f)22 1152 y Fe(j)41 1127 y Fc(00)62 1145 y Fm(.)0 1252 y Fn(Algorithm)g(CP:)e (Computation)j(of)e(the)h(generalized)h(Cauc)o(h)o(y)d(p)q(oin)o(t.)0 1308 y Fm(Giv)o(en)g Fg(x;)8 b(l)q(;)g(u;)g(g)q Fm(,)k(and)j Fg(B)h Fm(=)d Fg(\022)q(I)h Ff(\000)c Fg(W)c(M)f(W)744 1292 y Fe(T)68 1928 y Ff(\017)23 b Fm(Initialize)381 1377 y(F)l(or)49 b Fg(i)13 b Fm(=)g(1)p Fg(;)8 b(:)g(:)g(:)t(;)g(n)15 b Fm(compute)497 1549 y Fg(t)513 1556 y Fe(i)541 1549 y Fm(:=)601 1451 y Fd(8)601 1488 y(>)601 1501 y(<)601 1575 y(>)601 1588 y(:)659 1493 y Fm(\()p Fg(x)703 1500 y Fe(i)727 1493 y Ff(\000)10 b Fg(u)798 1500 y Fe(i)812 1493 y Fm(\))p Fg(=g)875 1500 y Fe(i)930 1493 y Fm(if)16 b Fg(g)994 1500 y Fe(i)1020 1493 y Fg(<)d Fm(0)659 1549 y(\()p Fg(x)703 1556 y Fe(i)727 1549 y Ff(\000)d Fg(l)786 1556 y Fe(i)800 1549 y Fm(\))p Fg(=g)863 1556 y Fe(i)930 1549 y Fm(if)16 b Fg(g)994 1556 y Fe(i)1020 1549 y Fg(>)d Fm(0)659 1606 y Ff(1)226 b Fm(otherwise)1217 1549 y(\()p Fg(n)15 b Fm(op)q(erations\)) 497 1755 y Fg(d)521 1762 y Fe(i)548 1755 y Fm(:=)608 1683 y Fd(\()663 1727 y Fm(0)89 b(if)16 b Fg(t)833 1734 y Fe(i)860 1727 y Fm(=)d(0)663 1783 y Ff(\000)p Fg(g)720 1790 y Fe(i)775 1783 y Fm(otherwise)410 1959 y Ff(F)54 b Fm(:=)13 b Ff(f)p Fg(i)f Fm(:)g Fg(t)650 1966 y Fe(i)677 1959 y Fg(>)h Fm(0)i Ff(g)425 2015 y Fg(p)49 b Fm(:=)13 b Fg(W)607 1999 y Fe(T)635 2015 y Fg(d)74 b Fm(\(2)p Fg(mn)14 b Fm(op)q(erations)i(\))428 2072 y Fg(c)49 b Fm(:=)13 b(0)409 2128 y Fg(f)436 2111 y Fc(0)497 2128 y Fm(:=)g Fg(g)582 2111 y Fe(T)609 2128 y Fg(d)f Fm(=)h Ff(\000)p Fg(d)752 2111 y Fe(T)780 2128 y Fg(d)74 b Fm(\()p Fg(n)15 b Fm(op)q(erations\))399 2184 y Fg(f)426 2168 y Fc(00)497 2184 y Fm(:)20 b(=)13 b Fg(\022)q(d)624 2168 y Fe(T)652 2184 y Fg(d)d Ff(\000)g Fg(d)755 2168 y Fe(T)782 2184 y Fg(W)c(M)f(W)929 2168 y Fe(T)958 2184 y Fg(d)12 b Fm(=)h Ff(\000)p Fg(\022)q(f)1126 2168 y Fc(0)1149 2184 y Ff(\000)d Fg(p)1217 2168 y Fe(T)1244 2184 y Fg(M)5 b(p)75 b Fm(\()p Fg(O)q Fm(\()p Fg(m)1503 2168 y Fk(2)1522 2184 y Fm(\))14 b(op)q(erations\))327 2247 y(\001)p Fg(t)381 2254 y Fe(min)497 2247 y Fm(:=)f Ff(\000)603 2227 y Fe(f)624 2215 y Fb(0)p 598 2237 41 2 v 598 2263 a Fe(f)619 2253 y Fb(00)383 2309 y Fg(t)399 2316 y Fe(old)497 2309 y Fm(:=)g(0)431 2365 y Fg(t)50 b Fm(:=)13 b(min)q Ff(f)p Fg(t)673 2372 y Fe(i)700 2365 y Fm(:)22 b Fg(i)12 b Ff(2)h(F)5 b(g)45 b Fm(\(using)15 b(the)h(heapsort)e(algorithm\))428 2422 y Fg(b)49 b Fm(:=)13 b Fg(i)20 b Fm(suc)o(h)h(that)e Fg(t)821 2429 y Fe(i)848 2422 y Fm(=)13 b Fg(t)61 b Fm(\(remo)o(v)o(e)14 b Fg(b)h Fm(from)f Ff(F)5 b Fm(\).)393 2478 y(\001)p Fg(t)50 b Fm(:=)13 b Fg(t)e Ff(\000)f Fm(0)945 2603 y(8)p eop %%Page: 9 10 bop 68 120 a Ff(\017)23 b Fm(Examination)15 b(of)g(subsequen)o(t)h(segmen)o (ts)114 177 y(While)g(\001)p Fg(t)301 184 y Fe(min)380 177 y Ff(\025)d Fm(\001)p Fg(t)i Fm(do)385 292 y Fg(x)411 270 y Fe(cp)411 306 y(b)487 292 y Fm(:=)577 220 y Fd(\()631 263 y Fg(u)657 270 y Fe(b)736 263 y Fm(if)24 b Fg(d)810 270 y Fe(b)840 263 y Fg(>)13 b Fm(0)631 320 y Fg(l)645 327 y Fe(b)736 320 y Fm(if)24 b Fg(d)810 327 y Fe(b)840 320 y Fg(<)13 b Fm(0)408 396 y Fg(z)429 403 y Fe(b)487 396 y Fm(:=)42 b Fg(x)603 374 y Fe(cp)603 410 y(b)648 396 y Ff(\000)11 b Fg(x)720 403 y Fe(b)426 465 y Fg(c)41 b Fm(:=)h Fg(c)10 b Fm(+)g(\001)p Fg(tp)75 b Fm(\()p Fg(O)q Fm(\()p Fg(m)p Fm(\))14 b(op)q(erations\))407 533 y Fg(f)434 515 y Fc(0)487 533 y Fm(:=)42 b Fg(f)604 515 y Fc(0)626 533 y Fm(+)10 b(\001)p Fg(t)q(f)753 515 y Fc(00)784 533 y Fm(+)g Fg(g)853 515 y Fk(2)851 545 y Fe(b)883 533 y Fm(+)g Fg(\022)q(g)972 540 y Fe(b)990 533 y Fg(z)1011 540 y Fe(b)1038 533 y Ff(\000)g Fg(g)1105 540 y Fe(b)1122 533 y Fg(w)1156 515 y Fe(T)1155 545 y(b)1184 533 y Fg(M)5 b(c)58 b Fm(\()p Fg(O)q Fm(\()p Fg(m)1423 517 y Fk(2)1442 533 y Fm(\))15 b(op)q(erations\))398 602 y Fg(f)425 584 y Fc(00)487 602 y Fm(:=)42 b Fg(f)604 584 y Fc(00)635 602 y Ff(\000)11 b Fg(\022)q(g)727 584 y Fk(2)725 614 y Fe(b)757 602 y Ff(\000)f Fm(2)p Fg(g)847 609 y Fe(b)864 602 y Fg(w)898 584 y Fe(T)897 614 y(b)925 602 y Fg(M)5 b(p)10 b Ff(\000)g Fg(g)1076 584 y Fk(2)1074 614 y Fe(b)1095 602 y Fg(w)1129 584 y Fe(T)1128 614 y(b)1157 602 y Fg(M)5 b(w)1239 609 y Fe(b)1314 602 y Fm(\()p Fg(O)q Fm(\()p Fg(m)1426 586 y Fk(2)1445 602 y Fm(\))15 b(op)q(erations\))423 671 y Fg(p)41 b Fm(:=)h Fg(p)10 b Fm(+)g Fg(g)677 678 y Fe(b)694 671 y Fg(w)727 678 y Fe(b)803 671 y Fm(\()p Fg(O)q Fm(\()p Fg(m)p Fm(\))k(op)q(erations\)) 405 740 y Fg(d)429 747 y Fe(b)487 740 y Fm(:=)42 b(0)325 830 y(\001)p Fg(t)379 837 y Fe(min)487 830 y Fm(:=)g Ff(\000)622 799 y Fg(f)649 783 y Fc(0)p 617 819 49 2 v 617 861 a Fg(f)644 848 y Fc(00)381 915 y Fg(t)397 922 y Fe(old)487 915 y Fm(:=)g Fg(t)430 983 y(t)f Fm(:=)h(min)q Ff(f)p Fg(t)692 990 y Fe(i)719 983 y Fm(:)22 b Fg(i)12 b Ff(2)h(F)5 b(g)60 b Fm(\(using)15 b(the)h(heapsort)e(algorithm\))426 1052 y Fg(b)41 b Fm(:=)h Fg(i)20 b Fm(suc)o(h)g(that)g Fg(t)840 1059 y Fe(i)867 1052 y Fm(=)13 b Fg(t)46 b Fm(\(Remo)o(v)o(e)15 b Fg(b)f Fm(from)h Ff(F)5 b Fm(\))392 1121 y(\001)p Fg(t)41 b Fm(:=)h Fg(t)10 b Ff(\000)h Fg(t)665 1128 y Fe(old)114 1208 y Fm(end)k(while)68 1297 y Ff(\017)23 b Fm(\001)p Fg(t)168 1304 y Fe(min)247 1297 y Fm(:=)13 b Fg(max)p Ff(f)p Fm(\001)p Fg(t)475 1304 y Fe(min)542 1297 y Fg(;)8 b Fm(0)p Ff(g)68 1386 y(\017)23 b Fg(t)130 1393 y Fe(old)191 1386 y Fm(:=)12 b Fg(t)267 1393 y Fe(old)326 1386 y Fm(+)f(\001)p Fg(t)426 1393 y Fe(min)68 1475 y Ff(\017)23 b Fg(x)140 1453 y Fe(cp)140 1488 y(i)187 1475 y Fm(:=)13 b Fg(x)274 1482 y Fe(i)298 1475 y Fm(+)e Fg(t)360 1482 y Fe(old)408 1475 y Fg(d)432 1482 y Fe(i)446 1475 y Fg(;)17 b Ff(8)11 b Fg(i)19 b Fm(suc)o(h)11 b(that)19 b Fg(t)764 1482 y Fe(i)792 1475 y Ff(\025)13 b Fg(t)68 1563 y Ff(\017)23 b Fm(F)l(or)14 b(all)i Fg(i)d Ff(2)g(F)19 b Fm(with)c Fg(t)501 1570 y Fe(i)529 1563 y Fm(=)e Fg(t)p Fm(,)i(remo)o(v)o(e)f Fg(i)h Fm(from)f Ff(F)5 b Fm(.)68 1652 y Ff(\017)23 b Fg(c)12 b Fm(:=)h Fg(c)c Fm(+)i(\001)p Fg(t)336 1659 y Fe(min)403 1652 y Fg(p)0 1753 y Fm(The)k(last)h(step)f(of)g(this)g(algorithm)g(up)q(dates)h(the)f(2)p Fg(m)p Fm(-v)o(ector)g Fg(c)f Fm(so)h(that)g(up)q(on)h(termination)781 1840 y Fg(c)c Fm(=)h Fg(W)910 1822 y Fe(T)938 1840 y Fm(\()p Fg(x)982 1822 y Fe(c)1009 1840 y Ff(\000)e Fg(x)1081 1847 y Fe(k)1102 1840 y Fm(\))p Fg(:)664 b Fm(\(4)p Fg(:)p Fm(13\))0 1928 y(This)17 b(v)o(ector)f(will)i(b)q(e)f(used)g(to)f(initialize)k(the)c (subspace)h(minimization)i(when)e(the)g(primal)g(direct)g(metho)q(d)0 1984 y(or)e(the)g(conjugate)g(gradien)o(t)g(metho)q(d)g(are)g(used,)h(as)e (will)j(b)q(e)f(discussed)h(in)f(the)f(next)g(section.)71 2040 y(Our)j(op)q(eration)f(coun)o(ts)h(only)g(tak)o(e)f(in)o(to)g(accoun)o(t)g(m) o(ultiplications)j(and)e(divisions.)29 b(Note)17 b(that)g(there)0 2097 y(are)c(no)g Fg(O)q Fm(\()p Fg(n)p Fm(\))g(computations)g(inside)j(the)d (lo)q(op.)20 b(If)14 b Fg(n)920 2104 y Fe(int)981 2097 y Fm(denotes)g(the)f (total)g(n)o(um)o(b)q(er)h(of)f(segmen)o(ts)g(explored,)0 2153 y(then)g(the)f(total)g(cost)g(of)f(Algorithm)i(CP)f(is)h(\(2)p Fg(m)t Fm(+)t(2\))p Fg(n)t Fm(+)t Fg(O)q Fm(\()p Fg(m)1085 2137 y Fk(2)1105 2153 y Fm(\))t Ff(\002)t Fg(n)1193 2160 y Fe(int)1255 2153 y Fm(op)q(erations)f(plus)i Fg(n)8 b Fm(log)g Fg(n)k Fm(op)q(erations)0 2210 y(whic)o(h)k(is)g(the)f(appro)o(ximate)g(cost) f(of)h(the)h(heapsort)e(algorithm)h([1].)0 2346 y Fh(5.)j(Metho)r(ds)g(for)h (subspace)g(minimi)o(zation.)71 2422 y Fm(Once)11 b(the)f(Cauc)o(h)o(y)g(p)q (oin)o(t)g Fg(x)552 2406 y Fe(c)580 2422 y Fm(has)g(b)q(een)h(found,)g(w)o(e) f(pro)q(ceed)h(to)f(appro)o(ximately)g(minimize)i(the)f(quadratic)0 2479 y(mo)q(del)17 b Fg(m)176 2486 y Fe(k)213 2479 y Fm(o)o(v)o(er)e(the)g (space)h(of)g(free)g(v)m(ariables,)h(and)e(imp)q(ose)i(the)f(b)q(ounds)h(on)e (the)h(problem.)23 b(W)l(e)15 b(consider)945 2603 y(9)p eop %%Page: 10 11 bop 0 120 a Fm(three)12 b(approac)o(hes)g(to)f(minimize)k(the)d(mo)q(del,)h (a)f(direct)h(primal)f(metho)q(d)h(based)f(on)g(the)g(Sherman-Morrison-)0 177 y(W)l(o)q(o)q(dbury)i(form)o(ula,)g(a)f(primal)i(iterativ)o(e)f(metho)q (d)h(using)f(the)g(conjugate)g(gradien)o(t)g(metho)q(d,)g(and)g(a)g(direct)0 233 y(dual)j(metho)q(d)g(using)g(Lagrange)f(m)o(ultipiers.)25 b(Whic)o(h)17 b(of)f(these)g(is)h(most)f(appropriate)g(seems)h(problem)g(de-) 0 289 y(p)q(enden)o(t,)h(and)e(w)o(e)h(ha)o(v)o(e)f(exp)q(erimen)o(ted)i(n)o (umerically)h(with)e(all)g(three.)24 b(In)18 b(all)f(these)g(approac)o(hes)f (w)o(e)h(\014rst)0 346 y(w)o(ork)e(on)h(minimizing)j Fg(m)452 353 y Fe(k)489 346 y Fm(ignoring)e(the)f(b)q(ounds,)g(and)h(at)e(an)h (appropriate)g(p)q(oin)o(t)h(truncate)e(the)h(mo)o(v)o(e)g(so)0 402 y(as)f(to)f(satisfy)h(the)h(b)q(ound)g(constrain)o(ts.)71 459 y(The)c(follo)o(wing)h(notation)f(will)i(b)q(e)f(used)f(throughout)g (this)g(section.)20 b(The)12 b(in)o(teger)h Fg(t)f Fm(denotes)h(the)f(n)o(um) o(b)q(er)0 515 y(of)j(free)g(v)m(ariables)h(at)e(the)h(Cauc)o(h)o(y)g(p)q (oin)o(t)g Fg(x)767 499 y Fe(c)785 515 y Fm(;)f(in)i(other)f(w)o(ords)f (there)h(are)f Fg(n)c Ff(\000)g Fg(t)16 b Fm(v)m(ariables)g(at)e(b)q(ound)i (at)e Fg(x)1883 499 y Fe(c)1901 515 y Fm(.)0 572 y(As)j(in)h(the)f(previous)g (section)h Ff(F)j Fm(denotes)c(the)g(set)f(of)h(indices)i(corresp)q(onding)f (to)e(the)h(free)g(v)m(ariables,)h(and)0 628 y(w)o(e)g(note)g(that)f(this)i (set)e(is)i(de\014ned)g(up)q(on)g(completion)g(of)f(the)g(Cauc)o(h)o(y)g(p)q (oin)o(t)h(computation.)28 b(W)l(e)18 b(de\014ne)0 685 y Fg(Z)31 692 y Fe(k)69 685 y Fm(to)e(b)q(e)h(the)g Fg(n)12 b Ff(\002)f Fg(t)17 b Fm(matrix)g(whose)f(columns)i(are)e(unit)i(v)o(ectors)d(\(i.e.)25 b(columns)17 b(of)g(the)g(iden)o(tit)o(y)g(matrix\))0 741 y(that)d(span)g (the)g(subspace)h(of)f(the)h(free)f(v)m(ariables)i(at)d Fg(x)952 725 y Fe(c)970 741 y Fm(.)19 b(Similarly)e Fg(A)1229 748 y Fe(k)1264 741 y Fm(denotes)e(the)f Fg(n)9 b Ff(\002)f Fm(\()p Fg(n)h Ff(\000)f Fg(t)p Fm(\))15 b(matrix)f(of)0 798 y(activ)o(e)i(constrain) o(t)g(gradien)o(ts)g(at)g Fg(x)627 781 y Fe(c)644 798 y Fm(,)g(whic)o(h)h (consists)f(of)g Fg(n)11 b Ff(\000)g Fg(t)17 b Fm(unit)f(v)o(ectors.)22 b(Note)16 b(that)g Fg(A)1658 781 y Fe(T)1658 811 y(k)1685 798 y Fg(Z)1716 805 y Fe(k)1752 798 y Fm(=)e(0)i(and)0 854 y(that)765 910 y Fg(A)799 917 y Fe(k)820 910 y Fg(A)854 892 y Fe(T)854 922 y(k)892 910 y Fm(+)11 b Fg(Z)969 917 y Fe(k)990 910 y Fg(Z)1024 892 y Fe(T)1021 922 y(k)1064 910 y Fm(=)i Fg(I)t(:)671 b Fm(\(5)p Fg(:)p Fm(1\))0 1094 y Fn(5.1.)17 b(A)g(Direct)h(Primal)f(Metho)q(d.)71 1175 y Fm(In)d(a)g(primal)h(approac)o(h,)f(w)o(e)g(\014x)h(the)f Fg(n)8 b Ff(\000)h Fg(t)14 b Fm(v)m(ariables)i(at)e(b)q(ound)h(at)e(the)i (generalized)h(Cauc)o(h)o(y)e(p)q(oin)o(t)g Fg(x)1883 1158 y Fe(c)1901 1175 y Fm(,)0 1231 y(and)f(solv)o(e)g(the)g(quadratic)g(problem)g (\(2.3\))e(o)o(v)o(er)h(the)h(subspace)h(of)e(the)h(remaining)h Fg(t)f Fm(free)g(v)m(ariables,)h(starting)0 1288 y(from)f Fg(x)132 1271 y Fe(c)164 1288 y Fm(and)h(imp)q(osing)h(the)f(free)g(v)m(ariable)i(b)q (ounds)e(\(2.4\).)k(Th)o(us)c(w)o(e)g(consider)h(only)f(the)h(p)q(oin)o(ts)f Fg(x)f Ff(2)f(<)1839 1271 y Fe(n)1877 1288 y Fm(of)0 1344 y(the)j(form)816 1401 y Fg(x)d Fm(=)h Fg(x)928 1382 y Fe(c)956 1401 y Fm(+)d Fg(Z)1032 1408 y Fe(k)1062 1389 y Fm(^)1054 1401 y Fg(d)o(;)730 b Fm(\(5)p Fg(:)p Fm(2\))0 1484 y(where)139 1472 y(^)131 1484 y Fg(d)14 b Fm(is)i(a)e(v)o(ector)g(of)h(dimension)h Fg(t)p Fm(.)k(Using)c(this)f(notation,)f(for)g(p)q(oin)o(ts)h(of)g(the)g(form)f (\(5.2\))f(w)o(e)h(can)h(write)0 1541 y(the)g(quadratic)h(\(2.1\))d(as)132 1662 y Fg(m)172 1669 y Fe(k)194 1662 y Fm(\()p Fg(x)p Fm(\))41 b(=)g Fg(f)395 1669 y Fe(k)427 1662 y Fm(+)11 b Fg(g)497 1643 y Fe(T)495 1673 y(k)523 1662 y Fm(\()p Fg(x)f Ff(\000)h Fg(x)649 1643 y Fe(c)676 1662 y Fm(+)f Fg(x)747 1643 y Fe(c)775 1662 y Ff(\000)g Fg(x)846 1669 y Fe(k)868 1662 y Fm(\))f(+)946 1631 y(1)p 946 1651 23 2 v 946 1693 a(2)974 1662 y(\()p Fg(x)g Ff(\000)i Fg(x)1099 1643 y Fe(c)1126 1662 y Fm(+)g Fg(x)1198 1643 y Fe(c)1225 1662 y Ff(\000)g Fg(x)1297 1669 y Fe(k)1318 1662 y Fm(\))1336 1643 y Fe(T)1363 1662 y Fg(B)1397 1669 y Fe(k)1419 1662 y Fm(\()p Fg(x)f Ff(\000)g Fg(x)1544 1643 y Fe(c)1572 1662 y Fm(+)g Fg(x)1643 1643 y Fe(c)1670 1662 y Ff(\000)h Fg(x)1742 1669 y Fe(k)1763 1662 y Fm(\))297 1770 y(=)41 b(\()p Fg(g)413 1777 y Fe(k)444 1770 y Fm(+)11 b Fg(B)524 1777 y Fe(k)545 1770 y Fm(\()p Fg(x)589 1751 y Fe(c)617 1770 y Ff(\000)f Fg(x)688 1777 y Fe(k)709 1770 y Fm(\)\))745 1751 y Fe(T)772 1770 y Fm(\()p Fg(x)g Ff(\000)g Fg(x)897 1751 y Fe(c)915 1770 y Fm(\))g(+)993 1739 y(1)p 993 1759 V 993 1801 a(2)1021 1770 y(\()p Fg(x)f Ff(\000)i Fg(x)1146 1751 y Fe(c)1163 1770 y Fm(\))1181 1751 y Fe(T)1208 1770 y Fg(B)1242 1777 y Fe(k)1264 1770 y Fm(\()p Fg(x)f Ff(\000)g Fg(x)1389 1751 y Fe(c)1407 1770 y Fm(\))g(+)g Fg(\015)297 1877 y Ff(\021)381 1865 y Fm(^)373 1877 y Fg(d)397 1859 y Fe(T)427 1877 y Fm(^)-25 b Fg(r)447 1859 y Fe(c)474 1877 y Fm(+)524 1847 y(1)p 524 1867 V 524 1909 a(2)560 1865 y(^)552 1877 y Fg(d)576 1859 y Fe(T)620 1866 y Fm(^)603 1877 y Fg(B)637 1884 y Fe(k)667 1865 y Fm(^)659 1877 y Fg(d)10 b Fm(+)g Fg(\015)s(;)1042 b Fm(\(5.3\))0 1990 y(where)15 b Fg(\015)j Fm(is)d(a)g(constan)o(t,)824 2035 y(^)813 2046 y Fg(B)847 2053 y Fe(k)882 2046 y Fm(=)e Fg(Z)964 2028 y Fe(T)961 2058 y(k)992 2046 y Fg(B)1026 2053 y Fe(k)1048 2046 y Fg(Z)1079 2053 y Fe(k)0 2130 y Fm(is)j(the)f(reduced)h (Hessian)g(of)f Fg(m)551 2137 y Fe(k)572 2130 y Fm(,)g(and)692 2232 y(^)-25 b Fg(r)712 2213 y Fe(c)742 2232 y Fm(=)13 b Fg(Z)824 2213 y Fe(T)821 2243 y(k)852 2232 y Fm(\()p Fg(g)892 2239 y Fe(k)923 2232 y Fm(+)d Fg(B)1002 2239 y Fe(k)1024 2232 y Fm(\()p Fg(x)1068 2213 y Fe(c)1095 2232 y Ff(\000)g Fg(x)1166 2239 y Fe(k)1188 2232 y Fm(\)\))0 2334 y(is)15 b(the)f(reduced)h(gradien)o(t)f(of) g Fg(m)558 2341 y Fe(k)594 2334 y Fm(at)f Fg(x)674 2318 y Fe(c)691 2334 y Fm(.)20 b(Using)15 b(\(3.2\))d(and)j(\(4.13\),)d(w)o(e)i(can)g (express)g(this)h(reduced)g(gradien)o(t)0 2391 y(as)596 2447 y(^)-25 b Fg(r)616 2428 y Fe(c)646 2447 y Fm(=)13 b Fg(Z)728 2428 y Fe(T)725 2458 y(k)756 2447 y Fm(\()p Fg(g)796 2454 y Fe(k)827 2447 y Fm(+)d Fg(\022)q Fm(\()p Fg(x)938 2428 y Fe(c)966 2447 y Ff(\000)g Fg(x)1037 2454 y Fe(k)1059 2447 y Fm(\))f Ff(\000)i Fg(W)1175 2454 y Fe(k)1196 2447 y Fg(M)1240 2454 y Fe(k)1262 2447 y Fg(c)p Fm(\))p Fg(;)507 b Fm(\(5)p Fg(:)p Fm(4\))934 2603 y(10)p eop %%Page: 11 12 bop 0 120 a Fm(whic)o(h,)16 b(giv)o(en)f(that)g(the)g(v)o(ector)f Fg(c)h Fm(w)o(as)f(sa)o(v)o(ed)h(from)g(the)g(Cauc)o(h)o(y)g(p)q(oin)o(t)g (computation,)g(costs)g(\(2)p Fg(m)9 b Fm(+)h(1\))p Fg(t)g Fm(+)0 177 y Fg(O)q Fm(\()p Fg(m)94 160 y Fk(2)113 177 y Fm(\))15 b(extra)g(op)q(erations.)20 b(Then)15 b(the)h(subspace)g(problem)f(\(2.3\))f (can)h(b)q(e)h(form)o(ulated)f(as)584 275 y(min)125 b(^)-32 b Fg(m)815 282 y Fe(k)836 275 y Fm(\()862 263 y(^)854 275 y Fg(d)p Fm(\))12 b Ff(\021)964 263 y Fm(^)956 275 y Fg(d)980 258 y Fe(T)1009 275 y Fm(^)-25 b Fg(r)1029 258 y Fe(c)1056 275 y Fm(+)1107 257 y Fk(1)p 1107 264 18 2 v 1107 290 a(2)1138 263 y Fm(^)1129 275 y Fg(d)1153 258 y Fe(T)1197 263 y Fm(^)1181 275 y Fg(B)1215 282 y Fe(k)1245 263 y Fm(^)1236 275 y Fg(d)10 b Fm(+)h Fg(\015)480 b Fm(\(5.5\))486 350 y(sub)s(ject)20 b(to)52 b Fg(l)756 357 y Fe(i)779 350 y Ff(\000)11 b Fg(x)851 334 y Fe(c)851 362 y(i)881 350 y Ff(\024)937 338 y Fm(^)929 350 y Fg(d)953 357 y Fe(i)979 350 y Ff(\024)i Fg(u)1053 357 y Fe(i)1077 350 y Ff(\000)e Fg(x)1149 334 y Fe(c)1149 362 y(i)1257 350 y Fg(i)h Ff(2)h(F)5 b Fg(;)441 b Fm(\(5.6\))0 448 y(where)19 b(the)g(subscript)h Fg(i)e Fm(denotes)h(the)g Fg(i)p Fm(-th)g(comp)q(onen)o (t)g(of)f(a)h(v)o(ector.)30 b(The)19 b(minimization)i(\(5.5\))d(can)h(b)q(e)0 505 y(solv)o(ed)f(either)g(b)o(y)f(a)g(direct)h(metho)q(d,)f(as)g(w)o(e)g (discuss)h(here,)g(or)f(b)o(y)g(an)g(iterativ)o(e)g(metho)q(d)h(as)f (discussed)i(in)0 561 y(the)c(next)h(subsection,)f(and)h(the)f(constrain)o (ts)g(\(5.6\))e(can)j(b)q(e)g(imp)q(osed)g(b)o(y)f(bac)o(ktrac)o(king.)71 618 y(Since)h(the)e(reduced)i(limited)g(memory)e(matrix)922 606 y(^)911 618 y Fg(B)945 625 y Fe(k)981 618 y Fm(is)h(a)f(small-rank)i (correction)e(of)h(a)f(diagonal)h(matrix,)0 674 y(w)o(e)f(can)h(formally)g (compute)g(its)g(in)o(v)o(erse)g(b)o(y)f(means)h(of)f(the)h (Sherman-Morrison-W)l(o)q(o)q(dbury)g(form)o(ula,)f(and)0 731 y(obtain)h(the)h(unconstrained)g(solution)g(of)f(the)g(subspace)h(problem)g (\(5.5\),)826 817 y(^)818 829 y Fg(d)842 810 y Fe(u)877 829 y Fm(=)d Ff(\000)971 817 y Fm(^)960 829 y Fg(B)996 809 y Fc(\000)p Fk(1)994 843 y Fe(k)1046 829 y Fm(^)-25 b Fg(r)1066 810 y Fe(c)1083 829 y Fg(:)724 b Fm(\(5)p Fg(:)p Fm(7\))0 927 y(W)l(e)15 b(can)h(then)f(bac)o (ktrac)o(k)f(to)o(w)o(ards)g(the)h(feasible)i(region,)e(if)h(necessary)l(,)f (to)f(obtain)859 1013 y(^)851 1025 y Fg(d)875 1006 y Fc(\003)907 1025 y Fm(=)f Fg(\013)984 1006 y Fc(\003)1012 1013 y Fm(^)1004 1025 y Fg(d)1028 1006 y Fe(u)1050 1025 y Fg(;)0 1123 y Fm(where)i(the)h(p)q (ositiv)o(e)g(scalar)f Fg(\013)536 1106 y Fc(\003)571 1123 y Fm(is)h(de\014ned)g(b)o(y)392 1221 y Fg(\013)421 1202 y Fc(\003)454 1221 y Fm(=)d(max)o Ff(f)p Fg(\013)g Fm(:)33 b Fg(\013)12 b Ff(\024)h Fm(1)p Fg(;)28 b(l)864 1228 y Fe(i)887 1221 y Ff(\000)11 b Fg(x)959 1202 y Fe(c)959 1232 y(i)989 1221 y Ff(\024)i Fg(\013)1074 1209 y Fm(^)1066 1221 y Fg(d)1090 1202 y Fe(u)1090 1232 y(i)1125 1221 y Ff(\024)g Fg(u)1199 1228 y Fe(i)1223 1221 y Ff(\000)d Fg(x)1294 1202 y Fe(c)1294 1232 y(i)1312 1221 y Fg(;)52 b(i)13 b Ff(2)g(F)5 b(g)p Fg(:)297 b Fm(\(5)p Fg(:)p Fm(8\))0 1318 y(Therefore)15 b(the)g(appro)o(ximate)g(solution)k(\026)-26 b Fg(x)15 b Fm(of)g(the)g(subproblem)i(\(2.3\)-\(2.4\))12 b(is)j(giv)o(en)h (b)o(y)641 1449 y(\026)-26 b Fg(x)664 1456 y Fe(i)691 1449 y Fm(=)739 1377 y Fd(\()793 1419 y Fg(x)819 1402 y Fe(c)819 1431 y(i)1079 1419 y Fm(if)16 b Fg(i)h(=)-28 b Ff(2)13 b(F)793 1479 y Fg(x)819 1463 y Fe(c)819 1491 y(i)847 1479 y Fm(+)d(\()p Fg(Z)941 1486 y Fe(k)970 1467 y Fm(^)962 1479 y Fg(d)986 1463 y Fc(\003)1006 1479 y Fm(\))1024 1486 y Fe(i)1079 1479 y Fm(if)16 b Fg(i)c Ff(2)h(F)5 b Fm(.)1820 1449 y(\(5)p Fg(:)p Fm(9\))71 1580 y(It)17 b(only)i(remains)f(to)f(consider)h(ho)o(w)f(to)g(p)q(erform)h (the)g(computation)f(in)i(\(5.7\).)25 b(Since)20 b Fg(B)1657 1587 y Fe(k)1696 1580 y Fm(is)e(giv)o(en)g(b)o(y)0 1636 y(\(3.2\))c(and)h Fg(Z)231 1620 y Fe(T)228 1649 y(k)259 1636 y Fg(Z)290 1643 y Fe(k)324 1636 y Fm(=)e Fg(I)t Fm(,)h(the)h(reduced)i(matrix)829 1625 y(^)819 1636 y Fg(B)g Fm(is)f(giv)o(en)g(b)o(y)691 1723 y(^)680 1734 y Fg(B)f Fm(=)e Fg(\022)q(I)i Ff(\000)10 b Fm(\()p Fg(Z)931 1715 y Fe(T)958 1734 y Fg(W)c Fm(\)\()p Fg(M)f(W)1141 1715 y Fe(T)1169 1734 y Fg(Z)s Fm(\))p Fg(;)0 1832 y Fm(where)14 b(w)o(e)f(ha)o(v)o(e)g(dropp)q(ed)h(the)g(subscripts)g(for)e(simplicit)o(y)l (.)22 b(Applying)15 b(the)f(Sherman-Morrison-W)l(o)q(o)q(dbury)0 1889 y(form)o(ula)h(\(see)g(for)g(example)h([22)o(]\),)e(w)o(e)h(obtain)428 2003 y(^)417 2014 y Fg(B)453 1995 y Fc(\000)p Fk(1)514 2014 y Fm(=)567 1983 y(1)p 567 2004 23 2 v 567 2045 a Fg(\022)595 2014 y(I)e Fm(+)679 1983 y(1)p 679 2004 V 679 2045 a Fg(\022)706 2014 y(Z)740 1995 y Fe(T)768 2014 y Fg(W)6 b Fm(\()p Fg(I)14 b Ff(\000)919 1983 y Fm(1)p 919 2004 V 919 2045 a Fg(\022)947 2014 y(M)5 b(W)1045 1995 y Fe(T)1073 2014 y Fg(Z)s(Z)1141 1995 y Fe(T)1169 2014 y Fg(W)h Fm(\))1236 1967 y Fc(\000)p Fk(1)1283 2014 y Fg(M)f(W)1381 1995 y Fe(T)1409 2014 y Fg(Z)1448 1983 y Fm(1)p 1448 2004 V 1448 2045 a Fg(\022)1476 2014 y(;)308 b Fm(\(5)p Fg(:)p Fm(10\))0 2134 y(so)15 b(that)f(the)i(unconstrained)g (subspace)g(Newton)f(direction)1081 2122 y(^)1073 2134 y Fg(d)1097 2118 y Fe(u)1135 2134 y Fm(is)g(giv)o(en)h(b)o(y)427 2245 y(^)419 2257 y Fg(d)443 2238 y Fe(u)478 2257 y Fm(=)531 2227 y(1)p 531 2247 V 531 2288 a Fg(\022)561 2257 y Fm(^)-25 b Fg(r)581 2238 y Fe(c)608 2257 y Fm(+)668 2227 y(1)p 658 2247 43 2 v 658 2288 a Fg(\022)680 2275 y Fk(2)706 2257 y Fg(Z)740 2238 y Fe(T)768 2257 y Fg(W)6 b Fm(\()p Fg(I)13 b Ff(\000)919 2227 y Fm(1)p 919 2247 23 2 v 919 2288 a Fg(\022)946 2257 y(M)5 b(W)1044 2238 y Fe(T)1072 2257 y Fg(Z)s(Z)1140 2238 y Fe(T)1168 2257 y Fg(W)h Fm(\))1235 2210 y Fc(\000)p Fk(1)1282 2257 y Fg(M)f(W)1380 2238 y Fe(T)1408 2257 y Fg(Z)g Fm(^)-25 b Fg(r)1464 2238 y Fe(c)1481 2257 y Fg(:)303 b Fm(\(5)p Fg(:)p Fm(11\))0 2366 y(Giv)o(en)18 b(a)f(set)g(of)g(free)g(v)m(ariables)i(at)e Fg(x)665 2349 y Fe(c)699 2366 y Fm(that)g(determines)h(the)g(matrix)f Fg(Z)s Fm(,)g(and)h(a)f(limited)i(memory)e(BF)o(GS)0 2422 y(matrix)j Fg(B)j Fm(de\014ned)e(in)g(terms)f(of)g Fg(\022)q(;)8 b(W)26 b Fm(and)20 b Fg(M)5 b Fm(,)22 b(the)e(follo)o(wing)h(pro)q(cedure)g (implemen)o(ts)g(the)g(approac)o(h)0 2479 y(just)d(describ)q(ed.)31 b(Note)18 b(that)g(since)h(the)f(columns)h(of)f Fg(Z)k Fm(are)c(unit)h(v)o (ectors,)f(the)g(op)q(eration)h Fg(Z)s(v)r Fm(,)f(amoun)o(ts)934 2603 y(11)p eop %%Page: 12 13 bop 0 120 a Fm(to)16 b(selecting)j(appropriate)e(elemen)o(ts)g(from)g Fg(v)r Fm(.)24 b(Here)18 b(and)f(throughout)f(the)h(pap)q(er)h(our)e(op)q (eration)i(coun)o(ts)0 177 y(include)e(only)e(m)o(ultiplications)i(and)e (divisions.)21 b(Recall)16 b(that)d Fg(t)h Fm(denotes)g(the)f(n)o(um)o(b)q (er)h(of)g(free)f(v)m(ariables)i(and)0 233 y(that)f Fg(m)i Fm(is)f(the)h(n)o(um)o(b)q(er)f(of)g(corrections)g(stored)g(in)h(the)f (limited)i(memory)e(matrix.)71 338 y Fn(Direct)j(Primal)f(Metho)q(d.)56 430 y Fm(1.)22 b(Compute)15 b Fg(Z)5 b Fm(^)-25 b Fg(r)367 413 y Fe(c)399 430 y Fm(b)o(y)15 b(\(5.4\))73 b(\(\(2)p Fg(m)9 b Fm(+)h(1\))p Fg(t)g Fm(+)h Fg(O)q Fm(\()p Fg(m)990 413 y Fk(2)1009 430 y Fm(\))k(op)q(erations\))56 523 y(2.)22 b Fg(v)14 b Fm(:=)f Fg(W)260 507 y Fe(T)287 523 y Fg(Z)5 b Fm(^)-25 b Fg(r)343 507 y Fe(c)435 523 y Fm(\(2)p Fg(mt)15 b Fm(op)q(erations\))56 616 y(3.)22 b Fg(v)14 b Fm(:=)f Fg(M)5 b(v)75 b Fm(\()p Fg(O)q Fm(\()p Fg(m)469 600 y Fk(2)488 616 y Fm(\))15 b(op)q(erations\))56 709 y(4.)22 b(F)l(orm)14 b Fg(N)j Ff(\021)c Fm(\()p Fg(I)g Ff(\000)437 691 y Fk(1)p 437 698 18 2 v 437 725 a Fe(\022)460 709 y Fg(M)5 b(W)558 693 y Fe(T)585 709 y Fg(Z)s(Z)653 693 y Fe(T)681 709 y Fg(W)h Fm(\))168 802 y Ff(\017)23 b Fg(N)17 b Fm(:=)333 784 y Fk(1)p 333 791 V 333 818 a Fe(\022)356 802 y Fg(W)405 786 y Fe(T)433 802 y Fg(Z)s(Z)501 786 y Fe(T)529 802 y Fg(W)80 b Fm(\(2)p Fg(m)733 786 y Fk(2)752 802 y Fg(t)11 b Fm(+)f Fg(mt)16 b Fm(op)q(erations\))168 875 y Ff(\017)23 b Fg(N)17 b Fm(:=)12 b Fg(I)i Ff(\000)c Fg(M)5 b(N)79 b Fm(\()p Fg(O)q Fm(\()p Fg(m)684 858 y Fk(3)703 875 y Fm(\))15 b(op)q(erations\))56 968 y(5.)22 b Fg(v)14 b Fm(:=)f Fg(N)253 951 y Fc(\000)p Fk(1)299 968 y Fg(v)76 b Fm(\()p Fg(O)q Fm(\()p Fg(m)509 951 y Fk(3)528 968 y Fm(\))15 b(op)q(erations\))56 1061 y(6.)122 1049 y(^)114 1061 y Fg(d)138 1044 y Fe(u)172 1061 y Fm(:=)238 1043 y Fk(1)p 238 1050 V 238 1077 a Fe(\022)263 1061 y Fm(^)-25 b Fg(r)283 1044 y Fe(c)310 1061 y Fm(+)369 1043 y Fk(1)p 360 1050 35 2 v 360 1078 a Fe(\022)377 1068 y Fj(2)400 1061 y Fg(Z)434 1044 y Fe(T)462 1061 y Fg(W)6 b(v)76 b Fm(\(2)p Fg(mt)10 b Fm(+)g Fg(t)16 b Fm(op)q(erations\))56 1154 y(7.)22 b(Find)16 b Fg(\013)251 1138 y Fc(\003)286 1154 y Fm(satisfying)f(\(5.8\))73 b(\()p Fg(t)15 b Fm(op)q(erations\))56 1247 y(8.)22 b(Compute)17 b(\026)-25 b Fg(x)337 1254 y Fe(i)366 1247 y Fm(as)15 b(in)h(\(5.9\))72 b(\()p Fg(t)16 b Fm(op)q(erations\))0 1339 y(The)21 b(total)e(cost)h(of)g (this)h(subspace)g(minimization)h(step)e(based)h(on)f(the)h (Sherman-Morrison-W)l(o)q(o)q(dbury)0 1396 y(form)o(ula)15 b(is)699 1452 y(2)p Fg(m)762 1433 y Fk(2)782 1452 y Fg(t)10 b Fm(+)h(6)p Fg(mt)f Fm(+)g(4)p Fg(t)h Fm(+)f Fg(O)q Fm(\()p Fg(m)1177 1433 y Fk(3)1196 1452 y Fm(\))583 b(\(5)p Fg(:)p Fm(12\))0 1534 y(op)q(erations.)38 b(This)22 b(is)g(quite)f(acceptable)i (when)e Fg(t)h Fm(is)g(small,)h(i.e.)38 b(when)22 b(there)f(are)g(few)g(free) g(v)m(ariables.)0 1591 y(Ho)o(w)o(ev)o(er)14 b(in)i(man)o(y)e(problems)h(the) g(opp)q(osite)h(is)f(true:)20 b(few)14 b(constrain)o(ts)h(are)f(activ)o(e)h (and)g Fg(t)g Fm(is)h(large.)j(In)d(this)0 1647 y(case)i(the)f(cost)g(of)g (the)h(direct)g(primal)g(metho)q(d)g(can)g(b)q(e)g(quite)g(large,)g(but)f (the)h(follo)o(wing)g(mec)o(hanism)g(can)0 1704 y(pro)o(vide)e(signi\014can)o (t)g(sa)o(vings.)71 1760 y(Note)e(that)h(when)h Fg(t)f Fm(is)h(large,)f(it)g (is)h(the)f(computation)g(of)g(the)g(matrix)252 1890 y Fg(W)301 1872 y Fe(T)329 1890 y Fg(Z)s(Z)397 1872 y Fe(T)425 1890 y Fg(W)k Fm(=)535 1818 y Fd(")588 1862 y Fg(Y)625 1846 y Fe(T)580 1919 y Fg(\022)q(S)633 1902 y Fe(T)682 1818 y Fd(#)713 1890 y Fg(Z)s(Z)781 1872 y Fe(T)817 1843 y Fd(h)857 1890 y Fg(Y)52 b(\022)q(S)1009 1843 y Fd(i)1041 1890 y Fm(=)1089 1818 y Fd(")1143 1862 y Fg(Y)1179 1846 y Fe(T)1207 1862 y Fg(Z)s(Z)1275 1846 y Fe(T)1303 1862 y Fg(Y)67 b(\022)q(Y)1455 1846 y Fe(T)1483 1862 y Fg(Z)s(Z)1551 1846 y Fe(T)1579 1862 y Fg(S)1134 1919 y(\022)q(S)1187 1902 y Fe(T)1215 1919 y Fg(Z)s(Z)1283 1902 y Fe(T)1311 1919 y Fg(Y)52 b(\022)1411 1902 y Fk(2)1431 1919 y Fg(S)1462 1902 y Fe(T)1489 1919 y Fg(Z)s(Z)1557 1902 y Fe(T)1585 1919 y Fg(S)1637 1818 y Fd(#)0 2027 y Fm(in)19 b(step)f(4,)h(whic)o(h)g (requires)g(2)p Fg(m)580 2010 y Fk(2)599 2027 y Fg(t)g Fm(op)q(erations,)f (that)g(driv)o(es)h(up)f(the)h(cost.)28 b(F)l(ortunately)18 b(w)o(e)g(can)g(reduce)0 2083 y(the)g(cost)g(when)h(only)g(a)e(few)i(v)m (ariables)g(en)o(ter)f(or)g(lea)o(v)o(e)g(the)h(activ)o(e)f(set)g(from)f(one) i(iteration)f(to)g(the)g(next)0 2140 y(b)o(y)g(sa)o(ving)g(the)g(matrices)g Fg(Y)511 2123 y Fe(T)538 2140 y Fg(Z)s(Z)606 2123 y Fe(T)634 2140 y Fg(Y)s(;)i(S)727 2123 y Fe(T)754 2140 y Fg(Z)s(Z)822 2123 y Fe(T)850 2140 y Fg(Y)28 b Fm(and)18 b Fg(S)1026 2123 y Fe(T)1053 2140 y Fg(Z)s(Z)1121 2123 y Fe(T)1150 2140 y Fg(S)s Fm(.)28 b(These)18 b(matrices)g(can)g(b)q(e)h(up)q(dated)g(to)0 2196 y(accoun)o(t)c(for)f(the)i(parts)e(of)h(the)g(inner)h(pro)q(ducts)g (corresp)q(onding)g(to)e(v)m(ariables)j(that)d(ha)o(v)o(e)h(c)o(hanged)h (status,)0 2253 y(and)f(to)g(add)g(ro)o(ws)f(and)i(columns)g(corresp)q (onding)g(to)f(the)g(new)g(step.)20 b(In)c(our)f(computational)g(exp)q (erimen)o(ts)0 2309 y(in)j Ff(x)p Fm(6)e(w)o(e)h(ha)o(v)o(e)f(implemen)o(ted) j(this)e(device.)26 b(In)17 b(addition,)h(when)f Fg(t)g Fm(is)h(m)o(uc)o(h)e (larger)h(then)g Fg(n)12 b Ff(\000)f Fg(t)p Fm(,)17 b(it)g(seems)0 2366 y(more)d(e\016cien)o(t)h(to)e(use)i(the)f(relationship)i Fg(Y)773 2349 y Fe(T)800 2366 y Fg(Z)s(Z)868 2349 y Fe(T)896 2366 y Fg(Y)23 b Fm(=)13 b Fg(Y)1030 2349 y Fe(T)1057 2366 y Fg(Y)19 b Ff(\000)8 b Fg(Y)1182 2349 y Fe(T)1210 2366 y Fg(AA)1278 2349 y Fe(T)1305 2366 y Fg(Y)j Fm(,)j(whic)o(h)h(follo)o(ws)f(from)g (\(5.1\),)e(to)0 2422 y(compute)k Fg(Y)221 2406 y Fe(T)249 2422 y Fg(Z)s(Z)317 2406 y Fe(T)345 2422 y Fg(Y)10 b Fm(.)22 b(Similar)17 b(relationships)g(can)f(b)q(e)g(used)h(for)e(the)g(matrices)h Fg(S)1452 2406 y Fe(T)1479 2422 y Fg(Z)s(Z)1547 2406 y Fe(T)1575 2422 y Fg(Y)26 b Fm(and)16 b Fg(S)1747 2406 y Fe(T)1774 2422 y Fg(Z)s(Z)1842 2406 y Fe(T)1870 2422 y Fg(S)s Fm(.)0 2479 y(An)f(expression)i(using)f(these)f(relationships)i(is)e(describ)q(ed)j(at)c (the)h(end)h(of)f Ff(x)p Fm(5.3.)934 2603 y(12)p eop %%Page: 13 14 bop 0 120 a Fn(5.2.)17 b(A)g(primal)h(conjugate)g(gradien)o(t)g(metho)q(d.)71 201 y Fm(Another)d(approac)o(h)g(for)g(appro)o(ximately)h(solving)h(the)e (subspace)i(problem)f(\(5.5\))e(is)i(to)f(apply)h(the)g(con-)0 258 y(jugate)f(gradien)o(t)g(metho)q(d)g(to)g(the)g(p)q(ositiv)o(e)h (de\014nite)h(linear)f(system)839 349 y(^)828 360 y Fg(B)862 367 y Fe(k)892 348 y Fm(^)884 360 y Fg(d)908 341 y Fe(u)943 360 y Fm(=)d Ff(\000)r Fm(^)-25 b Fg(r)1048 341 y Fe(c)1065 360 y Fg(;)719 b Fm(\(5)p Fg(:)p Fm(13\))0 462 y(and)20 b(stop)e(the)i (iteration)f(when)h(a)f(b)q(oundary)h(is)g(encoun)o(tered)g(or)f(when)g(the)h (residual)g(is)g(small)g(enough.)0 519 y(Note)13 b(that)g(the)h(accuracy)f (of)g(the)h(solution)g(con)o(trols)g(the)f(rate)g(of)g(con)o(v)o(ergence)h (of)f(the)h(algorithm,)f(once)h(the)0 575 y(correct)g(activ)o(e)h(set)f(is)h (iden)o(ti\014ed,)h(and)f(should)g(therefore)f(b)q(e)h(c)o(hosen)g(with)g (care.)k(W)l(e)c(follo)o(w)g(Conn,)f(Gould)0 632 y(and)h(T)l(oin)o(t)h([8)o (])f(and)g(stop)g(the)g(conjugate)g(gradien)o(t)g(iteration)h(when)f(the)h (residual)i(^)-25 b Fg(r)16 b Fm(of)f(\(5.13\))e(satis\014es)686 747 y Ff(k)r Fm(^)-25 b Fg(r)q Ff(k)12 b Fg(<)h Fm(min)q(\(0)p Fg(:)p Fm(1)p Fg(;)988 696 y Fd(q)p 1027 696 85 2 v 1027 747 a Ff(k)r Fm(^)-25 b Fg(r)1072 734 y Fe(c)1089 747 y Ff(k)p Fm(\))p Ff(k)r Fm(^)g Fg(r)1175 728 y Fe(c)1192 747 y Ff(k)p Fg(:)0 849 y Fm(W)l(e)13 b(also)g(stop)f(the)h(iteration)g(at)f(a)h(b)q(ound) h(when)f(a)g(conjugate)f(gradien)o(t)h(step)g(is)g(ab)q(out)g(to)f(violate)h (a)g(b)q(ound,)0 906 y(th)o(us)g(guaran)o(teeing)f(that)g(\(5.6\))g(is)h (satis\014ed.)20 b(The)13 b(conjugate)f(gradien)o(t)h(metho)q(d)g(is)g (appropriate)g(here)g(since)0 962 y(almost)i(all)h(of)f(the)g(eigen)o(v)m (alues)i(of)638 951 y(^)627 962 y Fg(B)661 969 y Fe(k)698 962 y Fm(are)e(iden)o(tical.)71 1019 y(W)l(e)f(no)o(w)f(describ)q(e)j(the)e (conjugate)g(gradien)o(t)g(metho)q(d)h(and)f(giv)o(e)g(its)h(op)q(eration)f (coun)o(ts.)19 b(Note)14 b(that)f(the)0 1075 y(e\013ectiv)o(e)g(n)o(um)o(b)q (er)h(of)f(v)m(ariables)h(is)g Fg(t)p Fm(,)f(the)h(n)o(um)o(b)q(er)f(of)g (free)g(v)m(ariables.)21 b(Giv)o(en)13 b Fg(B)1403 1082 y Fe(k)1425 1075 y Fm(,)g(the)h(follo)o(wing)f(pro)q(cedure)0 1132 y(computes)i(an)g (appro)o(ximate)g(solution)h(of)f(\(5.5\).)71 1238 y Fn(The)i(conjugate)i (gradien)o(t)f(metho)q(d.)56 1344 y Fm(1.)24 b(^)-25 b Fg(r)13 b Fm(:=)i(^)-25 b Fg(r)231 1328 y Fe(c)263 1344 y Fm(computed)16 b(b)o(y)f(\(5.4\))72 b(\(\(2)p Fg(m)9 b Fm(+)i(1\))p Fg(t)f Fm(+)g Fg(O)q Fm(\()p Fg(m)1063 1328 y Fk(2)1082 1344 y Fm(\))15 b(op)q(erations\))56 1438 y(2.)22 b Fg(p)12 b Fm(:=)h Ff(\000)r Fm(^)-25 b Fg(r)q(;)315 1426 y Fm(^)307 1438 y Fg(d)13 b Fm(:=)f(0,)j(and)g Fg(\032)567 1445 y Fk(2)599 1438 y Fm(:=)g(^)-26 b Fg(r)681 1421 y Fe(T)711 1438 y Fm(^)h Fg(r)75 b Fm(\()p Fg(t)15 b Fm(op)q(erations\)) 56 1532 y(3.)22 b(Stop)15 b(if)g Ff(k)r Fm(^)-25 b Fg(r)q Ff(k)12 b Fg(<)h Fm(min)q(\(0)p Fg(:)p Fm(1)p Fg(;)563 1495 y Fd(p)p 602 1495 V 602 1532 a Ff(k)r Fm(^)-25 b Fg(r)647 1519 y Fe(c)664 1532 y Ff(k)p Fm(\))p Ff(k)r Fm(^)g Fg(r)750 1515 y Fe(c)767 1532 y Ff(k)56 1626 y Fm(4.)22 b Fg(\013)143 1633 y Fk(1)175 1626 y Fm(:=)13 b(max)o Ff(f)p Fg(\013)g Fm(:)410 1614 y(^)410 1626 y Fg(l)g Ff(\024)j Fm(^)-26 b Fg(x)511 1609 y Fe(c)539 1626 y Fm(+)592 1614 y(^)584 1626 y Fg(d)10 b Fm(+)g Fg(\013p)j Ff(\024)j Fm(^)-26 b Fg(u)p Ff(g)74 b Fm(\()p Fg(t)15 b Fm(op)q(erations\))56 1719 y(5.)22 b Fg(q)14 b Fm(:=)220 1708 y(^)209 1719 y Fg(B)243 1726 y Fe(k)265 1719 y Fg(p)74 b Fm(\(4)p Fg(mt)15 b Fm(op)q(erations\))56 1813 y(6.)22 b Fg(\013)143 1820 y Fk(2)175 1813 y Fm(:=)13 b Fg(\032)260 1820 y Fk(2)279 1813 y Fg(=p)325 1797 y Fe(T)352 1813 y Fg(q)76 b Fm(\()p Fg(t)15 b Fm(op)q(erations\))56 1907 y(7.)22 b(If)15 b Fg(\013)188 1914 y Fk(2)221 1907 y Fg(>)d(\013)297 1914 y Fk(1)332 1907 y Fm(set)411 1895 y(^)403 1907 y Fg(d)h Fm(:=)508 1895 y(^)500 1907 y Fg(d)d Fm(+)h Fg(\013)609 1914 y Fk(1)628 1907 y Fg(p)k Fm(and)h(stop;)114 1982 y(otherwise)f(compute:)168 2076 y Ff(\017)222 2064 y Fm(^)214 2076 y Fg(d)d Fm(:=)319 2064 y(^)311 2076 y Fg(d)d Fm(+)i Fg(\013)419 2083 y Fk(2)439 2076 y Fg(p)74 b Fm(\()p Fg(t)15 b Fm(op)q(erations\))168 2149 y Ff(\017)25 b Fm(^)-25 b Fg(r)13 b Fm(:=)i(^)-25 b Fg(r)11 b Fm(+)f Fg(\013)415 2156 y Fk(2)435 2149 y Fg(q)76 b Fm(\()p Fg(t)15 b Fm(op)q(erations\))168 2222 y Ff(\017)23 b Fg(\032)238 2229 y Fk(1)269 2222 y Fm(:=)13 b Fg(\032)354 2229 y Fk(2)373 2222 y Fm(;)28 b Fg(\032)438 2229 y Fk(2)470 2222 y Fm(=)15 b(^)-25 b Fg(r)540 2206 y Fe(T)569 2222 y Fm(^)g Fg(r)q Fm(;)27 b Fg(\014)15 b Fm(:=)d Fg(\032)754 2229 y Fk(2)774 2222 y Fg(=\032)821 2229 y Fk(1)914 2222 y Fm(\()p Fg(t)j Fm(op)q(erations\))168 2295 y Ff(\017)23 b Fg(p)12 b Fm(:=)h Ff(\000)r Fm(^)-25 b Fg(r)11 b Fm(+)f Fg(\014)r(p)75 b Fm(\()p Fg(t)15 b Fm(op)q(erations\))168 2368 y Ff(\017)23 b Fm(go)14 b(to)h(3)934 2603 y(13)p eop %%Page: 14 15 bop 0 120 a Fm(The)19 b(matrix-v)o(ector)f(m)o(ultiplication)j(of)d(step)h(5) g(should)g(b)q(e)h(p)q(erformed)f(as)f(describ)q(ed)j(in)e([6].)30 b(The)19 b(total)0 177 y(op)q(eration)c(coun)o(t)g(of)g(this)h(conjugate)f (gradien)o(t)g(pro)q(cedure)h(is)g(appro)o(ximately)558 279 y(\(2)p Fg(m)9 b Fm(+)i(2\))p Fg(t)f Fm(+)g(\(4)p Fg(m)f Fm(+)i(6\))p Fg(t)f Ff(\002)g Fg(citer)h Fm(+)f Fg(O)q Fm(\()p Fg(m)1298 260 y Fk(2)1318 279 y Fm(\))p Fg(;)448 b Fm(\(5)p Fg(:)p Fm(14\))0 381 y(where)19 b Fg(citer)g Fm(is)h(the)f(n)o(um)o(b)q(er)g(of)f(conjugate)g (gradien)o(t)h(iterations.)31 b(Comparing)18 b(this)i(to)e(the)h(cost)f(of)g (the)0 437 y(primal)d(direct)g(metho)q(d)g(\(5.12\))e(it)h(seems)h(that,)e (for)h Fg(t)f(>>)h(m)p Fm(,)g(the)g(direct)h(metho)q(d)g(is)g(more)f (e\016cien)o(t)h(unless)0 494 y Fg(citer)20 b Ff(\024)g Fg(m=)p Fm(2.)32 b(Note)18 b(that)h(the)g(costs)g(of)g(b)q(oth)h(metho)q(ds)f (increase)h(as)f(the)g(n)o(um)o(b)q(er)h(of)f(free)g(v)m(ariables)i Fg(t)0 550 y Fm(b)q(ecomes)f(larger.)34 b(Since)21 b(the)f(limited)i(memory)d (matrix)h Fg(B)1078 557 y Fe(k)1119 550 y Fm(is)h(a)e(rank)h(2)p Fg(m)f Fm(correction)h(of)g(the)g(iden)o(tit)o(y)0 607 y(matrix,)13 b(the)g(termination)g(prop)q(erties)h(of)e(the)h(conjugate)g(gradien)o(t)g (metho)q(d)g(guaran)o(tee)f(that)h(the)g(subspace)0 663 y(problem)j(will)h(b) q(e)f(solv)o(ed)f(in)h(at)f(most)f(2)p Fg(m)h Fm(conjugate)g(gradien)o(t)g (iterations.)71 720 y(W)l(e)d(should)g(p)q(oin)o(t)h(out)e(that)h(the)g (conjugate)f(gradien)o(t)h(iteration)g(could)h(stop)e(at)h(a)f(b)q(oundary)i (ev)o(en)f(when)0 776 y(the)k(unconstrained)h(solution)g(of)f(the)g(subspace) h(problem)g(is)f(inside)i(the)f(b)q(o)o(x.)22 b(Consider)17 b(for)e(example)i(the)0 832 y(case)g(when)g(the)g(unconstrained)h(solution)f (lies)h(near)f(a)f(corner)h(and)g(the)f(starting)g(p)q(oin)o(t)i(of)e(the)h (conjugate)0 889 y(gradien)o(t)i(iteration)h(lies)g(near)g(another)f(corner)g (along)g(the)g(same)g(edge)h(of)f(the)g(b)q(o)o(x.)32 b(Then)20 b(the)g(iterates)0 945 y(could)f(so)q(on)f(fall)h(outside)g(of)e(the)h (feasible)i(region.)29 b(This)18 b(example)h(also)f(illustrates)i(the)e (di\016culties)i(that)0 1002 y(the)15 b(conjugate)g(gradien)o(t)g(approac)o (h)g(can)g(ha)o(v)o(e)g(on)g(nearly)h(degenerate)f(problems)h([11)o(].)0 1158 y Fn(5.3.)34 b(A)17 b(dual)i(metho)q(d)e(for)h(subspace)f(minimization.) 71 1239 y Fm(Since)i(it)f(often)f(happ)q(ens)h(that)f(the)h(n)o(um)o(b)q(er)g (of)f(activ)o(e)h(b)q(ounds)g(is)g(small)h(relativ)o(e)f(to)f(the)h(size)g (of)f(the)0 1296 y(problem)f(it)g(should)g(b)q(e)g(e\016cien)o(t)h(to)d (handle)j(these)f(b)q(ounds)g(explicitly)i(with)e(Lagrange)f(m)o(ultipliers.) 23 b(Suc)o(h)0 1352 y(an)15 b(approac)o(h)g(is)h(often)f(referred)g(to)f(as)h (a)g(dual)h(or)f(a)g(range)g(space)g(metho)q(d)g(\(see)h([15)o(]\).)71 1409 y(W)l(e)f(will)i(write)844 1465 y Fg(x)12 b Ff(\021)h Fg(x)956 1472 y Fe(k)988 1465 y Fm(+)d Fg(d;)0 1548 y Fm(and)15 b(restrict)g Fg(x)271 1555 y Fe(k)303 1548 y Fm(+)10 b Fg(d)15 b Fm(to)g(lie)h(on)f(the)h(subspace)g(of)e(free)i(v)m(ariables)g(at)f Fg(x)1244 1532 y Fe(c)1276 1548 y Fm(b)o(y)g(imp)q(osing)i(the)e(condition) 733 1651 y Fg(A)767 1632 y Fe(T)767 1662 y(k)795 1651 y Fg(d)41 b Fm(=)h Fg(A)971 1632 y Fe(T)971 1662 y(k)999 1651 y Fm(\()p Fg(x)1043 1632 y Fe(c)1070 1651 y Ff(\000)10 b Fg(x)1141 1658 y Fe(k)1162 1651 y Fm(\))860 1720 y Ff(\021)42 b Fg(b)957 1727 y Fe(k)978 1720 y Fg(:)0 1822 y Fm(\(Recall)22 b(that)e Fg(A)298 1829 y Fe(k)340 1822 y Fm(is)h(the)g(matrix)g(of)f(constrain)o(t)g(gradien)o (ts.\))36 b(Using)21 b(this)g(notation)g(w)o(e)f(form)o(ulate)g(the)0 1878 y(subspace)c(problem)g(as)772 1980 y(min)73 b Fg(g)942 1987 y Fe(k)963 1964 y(T)990 1980 y Fg(d)10 b Fm(+)1074 1962 y Fk(1)p 1074 1969 18 2 v 1074 1996 a(2)1097 1980 y Fg(d)1121 1964 y Fe(T)1148 1980 y Fg(B)1182 1987 y Fe(k)1204 1980 y Fg(d)569 b Fm(\(5.15\))644 2049 y(sub)s(ject)20 b(to)132 b Fg(A)1014 2033 y Fe(T)1014 2062 y(k)1042 2049 y Fg(d)12 b Fm(=)h Fg(b)1146 2056 y Fe(k)1797 2049 y Fm(\(5.16\))923 2118 y Fg(l)g Ff(\024)g Fg(x)1024 2125 y Fe(k)1056 2118 y Fm(+)d Fg(d)i Ff(\024)h Fg(u:)573 b Fm(\(5.17\))0 2220 y(W)l(e)17 b(\014rst)g(solv)o(e)g(this)h(problem)g (without)f(the)g(b)q(ound)h(constrain)o(t)f(\(5.17\).)24 b(The)17 b(optimalit)o(y)h(conditions)g(for)0 2277 y(\(5.15\)-\(5.16\))12 b(are)686 2379 y Fg(g)708 2386 y Fe(k)739 2379 y Fm(+)e Fg(B)818 2386 y Fe(k)840 2379 y Fg(d)864 2360 y Fc(\003)894 2379 y Fm(+)g Fg(A)973 2386 y Fe(k)995 2379 y Fg(\025)1022 2360 y Fc(\003)1054 2379 y Fm(=)j(0)p Fg(;)659 b Fm(\(5.18\))925 2448 y Fg(A)959 2429 y Fe(T)959 2459 y(k)987 2448 y Fg(d)1011 2429 y Fc(\003)1043 2448 y Fm(=)13 b Fg(b)1111 2455 y Fe(k)1132 2448 y Fg(:)652 b Fm(\(5.19\))934 2603 y(14)p eop %%Page: 15 16 bop 0 120 a Fm(Pre-m)o(ultiplying)18 b(\(5.18\))13 b(b)o(y)i Fg(A)555 104 y Fe(T)555 133 y(k)583 120 y Fg(H)621 127 y Fe(k)642 120 y Fm(,)f(where)i Fg(H)839 127 y Fe(k)875 120 y Fm(is)g(the)f(in)o(v)o (erse)h(of)f Fg(B)1236 127 y Fe(k)1257 120 y Fm(,)g(w)o(e)g(obtain)608 222 y Fg(A)642 203 y Fe(T)642 233 y(k)669 222 y Fg(H)707 229 y Fe(k)728 222 y Fg(g)750 229 y Fe(k)781 222 y Fm(+)c Fg(A)861 203 y Fe(T)861 233 y(k)889 222 y Fg(d)913 203 y Fc(\003)942 222 y Fm(+)f Fg(A)1021 203 y Fe(T)1021 233 y(k)1049 222 y Fg(H)1087 229 y Fe(k)1108 222 y Fg(A)1142 229 y Fe(k)1164 222 y Fg(\025)1191 203 y Fc(\003)1222 222 y Fm(=)j(0)p Fg(;)0 324 y Fm(and)i(using)h(\(5.19\))e (w)o(e)h(obtain)643 381 y(\()p Fg(A)695 362 y Fe(T)695 392 y(k)723 381 y Fg(H)761 388 y Fe(k)782 381 y Fg(A)816 388 y Fe(k)837 381 y Fm(\))p Fg(\025)882 362 y Fc(\003)914 381 y Fm(=)e Ff(\000)p Fg(A)1031 362 y Fe(T)1031 392 y(k)1059 381 y Fg(H)1097 388 y Fe(k)1118 381 y Fg(g)1140 388 y Fe(k)1171 381 y Ff(\000)d Fg(b)1236 388 y Fe(k)1257 381 y Fg(:)527 b Fm(\(5)p Fg(:)p Fm(20\))0 464 y(Since)22 b(the)e(columns)i(of)d Fg(A)480 471 y Fe(k)522 464 y Fm(are)h(unit)h(v)o(ectors)f(and)h Fg(A)992 471 y Fe(k)1033 464 y Fm(has)g(full)g(column)h(rank,)f(w)o(e)f(see)g Fg(A)1677 448 y Fe(T)1677 477 y(k)1705 464 y Fg(H)1743 471 y Fe(k)1764 464 y Fg(A)1798 471 y Fe(k)1840 464 y Fm(is)h(a)0 521 y(principal)d(submatrix)d(of)g Fg(H)496 528 y Fe(k)517 521 y Fm(.)20 b(Th)o(us)15 b(\(5.20\))e(determines)j Fg(\025)1053 504 y Fc(\003)1072 521 y Fm(,)f(and)h(hence)g Fg(d)1339 504 y Fc(\003)1373 521 y Fm(is)g(giv)o(en)g(b)o(y)753 623 y Fg(B)787 630 y Fe(k)809 623 y Fg(d)833 604 y Fc(\003)865 623 y Fm(=)c Ff(\000)p Fg(A)981 630 y Fe(k)1003 623 y Fg(\025)1030 604 y Fc(\003)1060 623 y Ff(\000)e Fg(g)1127 630 y Fe(k)1148 623 y Fg(:)636 b Fm(\(5)p Fg(:)p Fm(21\))0 725 y(\(In)16 b(the)g(sp)q(ecial)h (case)f(where)g(there)f(are)h(no)f(activ)o(e)h(constrain)o(ts,)f(w)o(e)g (simply)i(obtain)f Fg(B)1553 732 y Fe(k)1575 725 y Fg(d)1599 708 y Fc(\003)1631 725 y Fm(=)e Ff(\000)p Fg(g)1737 732 y Fe(k)1758 725 y Fg(:)p Fm(\))h(If)h(the)0 781 y(v)o(ector)c Fg(x)160 788 y Fe(k)185 781 y Fm(+)t Fg(d)248 765 y Fc(\003)280 781 y Fm(violates)h(the)f(b)q(ounds)h(\(5.17\),)e(w)o(e)h(bac)o(ktrac)o(k)f(to)g (the)i(feasible)g(region)g(along)f(the)g(line)i(joining)0 838 y(this)i(infeasible)h(p)q(oin)o(t)f(and)f(the)g(generalized)i(Cauc)o(h)o(y)e (p)q(oin)o(t)h Fg(x)1115 821 y Fe(c)1132 838 y Fm(.)71 894 y(The)g(linear)i(system)d(\(5.20\))g(can)h(b)q(e)h(solv)o(ed)g(b)o(y)f(the)g (Sherman-Morrison-W)l(o)q(o)q(dbury)h(form)o(ula.)23 b(Using)0 951 y(the)15 b(in)o(v)o(erse)h(limited)h(memory)e(BF)o(GS)f(matrix)h (\(3.7\),)e(and)j(recalling)h(the)e(iden)o(tit)o(y)h Fg(A)1515 934 y Fe(T)1515 964 y(k)1542 951 y Fg(A)1576 958 y Fe(k)1611 951 y Fm(=)d Fg(I)t Fm(,)h(w)o(e)h(obtain)605 1074 y Fg(A)639 1055 y Fe(T)639 1085 y(k)667 1074 y Fg(H)705 1081 y Fe(k)726 1074 y Fg(A)760 1081 y Fe(k)794 1074 y Fm(=)847 1043 y(1)p 847 1063 23 2 v 847 1105 a Fg(\022)875 1074 y(I)e Fm(+)e(\()p Fg(A)1006 1055 y Fe(T)1046 1062 y Fm(\026)1033 1074 y Fg(W)6 b Fm(\)\()1135 1062 y(\026)1118 1074 y Fg(M)1180 1062 y Fm(\026)1167 1074 y Fg(W)1216 1055 y Fe(T)1244 1074 y Fg(A)p Fm(\))p Fg(:)0 1189 y Fm(\(W)l(e)18 b(ha)o(v)o(e)g(again)g(omitted)g(the)h(subscripts)g(of)f Fg(M)r(;)8 b(W)24 b Fm(and)18 b Fg(\022)i Fm(for)e(simplicit)o(y)l(.\))31 b(Applying)20 b(the)e(Sherman-)0 1245 y(Morrison-W)l(o)q(o)q(dbury)d(form)o (ula)g(w)o(e)g(obtain)333 1354 y(\()p Fg(A)385 1335 y Fe(T)385 1365 y(k)412 1354 y Fg(H)450 1361 y Fe(k)471 1354 y Fg(A)505 1361 y Fe(k)527 1354 y Fm(\))545 1335 y Fc(\000)p Fk(1)604 1354 y Fm(=)e Fg(\022)q(I)i Ff(\000)10 b Fg(\022)q(A)810 1335 y Fe(T)810 1365 y(k)852 1342 y Fm(\026)838 1354 y Fg(W)d Fm(\()p Fg(I)13 b Fm(+)d Fg(\022)1024 1342 y Fm(\026)1006 1354 y Fg(M)1069 1342 y Fm(\026)1056 1354 y Fg(W)1105 1335 y Fe(T)1133 1354 y Fg(A)1167 1361 y Fe(k)1188 1354 y Fg(A)1222 1335 y Fe(T)1222 1365 y(k)1263 1342 y Fm(\026)1250 1354 y Fg(W)c Fm(\))1317 1325 y Fc(\000)p Fk(1)1381 1342 y Fm(\026)1364 1354 y Fg(M)1426 1342 y Fm(\026)1413 1354 y Fg(W)1462 1335 y Fe(T)1490 1354 y Fg(A)1524 1361 y Fe(k)1545 1354 y Fg(\022)q(:)217 b Fm(\(5)p Fg(:)p Fm(22\))71 1456 y(Giv)o(en)18 b Fg(g)227 1463 y Fe(k)248 1456 y Fm(,)h(a)e(set)h(of)g(activ)o(e)g(v)m(ariables)i(at)d Fg(x)859 1439 y Fe(c)895 1456 y Fm(that)g(determines)i(the)f(matrix)g(of)g (constrain)o(t)f(gradien)o(ts)0 1512 y Fg(A)34 1519 y Fe(k)55 1512 y Fm(,)i(and)f(an)h(in)o(v)o(erse)f(limited)i(memory)e(BF)o(GS)g(matrix) g Fg(H)1063 1519 y Fe(k)1084 1512 y Fm(,)g(the)h(follo)o(wing)g(pro)q(cedure) g(implemen)o(ts)g(the)0 1569 y(dual)f(approac)o(h)g(just)f(describ)q(ed.)29 b(Let)17 b(us)h(recall)h(that)e Fg(t)g Fm(denotes)h(the)g(n)o(um)o(b)q(er)g (of)f(free)g(v)m(ariables)i(and)f(let)0 1625 y(us)k(de\014ne)g Fg(t)218 1632 y Fe(a)262 1625 y Fm(=)i Fg(n)14 b Ff(\000)h Fg(t)p Fm(,)23 b(so)e(that)g Fg(t)647 1632 y Fe(a)689 1625 y Fm(denotes)h(the)f(n)o(um)o(b)q(er)h(of)f(activ)o(e)h(constrain)o(ts)e(at)h Fg(x)1637 1609 y Fe(c)1655 1625 y Fm(.)38 b(As)21 b(b)q(efore,)0 1682 y(the)c(op)q(eration)f(coun)o(ts)h(giv)o(en)g(b)q(elo)o(w)g(include)h (only)g(m)o(ultiplications)g(and)f(divisions,)i(and)d Fg(m)h Fm(denotes)f(the)0 1738 y(n)o(um)o(b)q(er)g(of)e(corrections)i(stored)e(in)i (the)g(limited)h(memory)d(matrix.)0 1844 y Fn(Dual)k(metho)q(d)0 1901 y Fm(If)d Fg(t)61 1908 y Fe(a)95 1901 y Fm(=)e(0,)i(compute)g Fg(d)402 1884 y Fc(\003)434 1901 y Fm(=)e Ff(\000)p Fg(H)555 1908 y Fe(k)577 1901 y Fg(g)599 1908 y Fe(k)632 1901 y Fm(=)g Ff(\000)720 1883 y Fk(1)p 720 1890 18 2 v 720 1916 a Fe(\022)743 1901 y Fg(g)765 1908 y Fe(k)796 1901 y Ff(\000)855 1889 y Fm(\026)842 1901 y Fg(W)908 1889 y Fm(\026)891 1901 y Fg(M)953 1889 y Fm(\026)940 1901 y Fg(W)989 1884 y Fe(T)1017 1901 y Fg(g)1039 1908 y Fe(k)1075 1901 y Fm(as)i(follo)o(ws)68 1995 y Ff(\017)23 b Fg(w)13 b Fm(:=)234 1983 y(\026)221 1995 y Fg(W)270 1978 y Fe(T)297 1995 y Fg(g)319 2002 y Fe(k)415 1995 y Fm(\(0)h(op)q(erations\))68 2088 y Ff(\017)23 b Fg(w)13 b Fm(:=)238 2077 y(\026)221 2088 y Fg(M)5 b(w)75 b Fm(\()p Fg(O)q Fm(\()p Fg(m)490 2072 y Fk(2)509 2088 y Fm(\))14 b(op)q(erations\))68 2182 y Ff(\017)23 b Fg(d)138 2166 y Fc(\003)170 2182 y Fm(:=)12 b Ff(\000)270 2164 y Fk(1)p 270 2171 V 270 2198 a Fe(\022)293 2182 y Fg(g)315 2189 y Fe(k)346 2182 y Ff(\000)405 2171 y Fm(\026)392 2182 y Fg(W)6 b(w)75 b Fm(\(\(2)p Fg(m)9 b Fm(+)i(1\))p Fg(n)j Fm(op)q(erations\))0 2276 y(If)h Fg(t)61 2283 y Fe(a)95 2276 y Fg(>)e Fm(0,)i(compute)56 2370 y(1.)22 b Fg(u)12 b Fm(=)h Ff(\000)p Fg(A)269 2353 y Fe(T)269 2383 y(k)297 2370 y Fg(H)335 2377 y Fe(k)356 2370 y Fg(g)378 2377 y Fe(k)409 2370 y Ff(\000)e Fg(b)h Fm(=)h Ff(\000)575 2352 y Fk(1)p 575 2359 V 575 2386 a Fe(\022)598 2370 y Fg(A)632 2353 y Fe(T)632 2383 y(k)660 2370 y Fg(g)682 2377 y Fe(k)713 2370 y Ff(\000)d Fg(A)792 2353 y Fe(T)833 2358 y Fm(\026)820 2370 y Fg(W)886 2358 y Fm(\026)869 2370 y Fg(M)931 2358 y Fm(\026)918 2370 y Fg(W)967 2353 y Fe(T)995 2370 y Fg(g)1017 2377 y Fe(k)1048 2370 y Ff(\000)g Fg(b)168 2464 y Ff(\017)23 b Fg(b)12 b Fm(:=)g Fg(A)340 2447 y Fe(T)340 2477 y(k)368 2464 y Fm(\()p Fg(x)412 2447 y Fe(c)439 2464 y Ff(\000)f Fg(x)511 2471 y Fe(k)532 2464 y Fm(\))74 b(\(0)14 b(op)q(erations\))934 2603 y(15)p eop %%Page: 16 17 bop 168 120 a Ff(\017)23 b Fg(v)14 b Fm(:=)324 109 y(\026)311 120 y Fg(W)360 104 y Fe(T)387 120 y Fg(g)409 127 y Fe(k)505 120 y Fm(\(0)g(op)q(erations\))168 192 y Ff(\017)23 b Fg(v)14 b Fm(:=)327 181 y(\026)311 192 y Fg(M)5 b(w)75 b Fm(\()p Fg(O)q Fm(\()p Fg(m)580 176 y Fk(2)598 192 y Fm(\))15 b(op)q(erations\))168 264 y Ff(\017)23 b Fg(u)12 b Fm(:=)h Fg(A)347 248 y Fe(T)347 278 y(k)388 253 y Fm(\026)374 264 y Fg(W)7 b(v)76 b Fm(\(2)p Fg(mt)619 271 y Fe(a)654 264 y Fm(op)q(erations\))168 336 y Ff(\017)23 b Fg(u)12 b Fm(:=)h Ff(\000)353 319 y Fk(1)p 353 326 18 2 v 353 352 a Fe(\022)376 336 y Fg(A)410 320 y Fe(T)410 350 y(k)437 336 y Fg(g)459 343 y Fe(k)491 336 y Ff(\000)d Fg(u)g Ff(\000)h Fg(b)73 b Fm(\()p Fg(t)745 343 y Fe(a)781 336 y Fm(op)q(erations\)) 56 440 y(2.)22 b Fg(\025)141 424 y Fc(\003)172 440 y Fm(=)13 b Ff(\000)p Fm(\()p Fg(A)307 424 y Fe(T)307 453 y(k)335 440 y Fg(H)373 447 y Fe(k)394 440 y Fg(A)428 447 y Fe(k)450 440 y Fm(\))468 424 y Fc(\000)p Fk(1)514 440 y Fg(u)g Fm(=)g Ff(\000)p Fg(\022)q(u)e Fm(+)g Fg(\022)763 424 y Fk(2)783 440 y Fg(A)817 424 y Fe(T)817 453 y(k)858 429 y Fm(\026)845 440 y Fg(W)6 b Fm(\()p Fg(I)13 b Fm(+)e Fg(\022)1030 429 y Fm(\026)1013 440 y Fg(M)1076 429 y Fm(\026)1062 440 y Fg(W)1111 424 y Fe(T)1139 440 y Fg(A)1173 447 y Fe(k)1194 440 y Fg(A)1228 424 y Fe(T)1228 453 y(k)1269 429 y Fm(\026)1256 440 y Fg(W)6 b Fm(\))1323 414 y Fc(\000)p Fk(1)1387 429 y Fm(\026)1370 440 y Fg(M)1432 429 y Fm(\026)1419 440 y Fg(W)1468 424 y Fe(T)1496 440 y Fg(A)1530 447 y Fe(k)1551 440 y Fg(u)168 533 y Ff(\017)23 b Fg(w)13 b Fm(:=)334 522 y(\026)321 533 y Fg(W)370 517 y Fe(T)397 533 y Fg(A)431 540 y Fe(k)453 533 y Fg(u)74 b Fm(\(2)p Fg(mt)650 540 y Fe(a)686 533 y Fm(op)q(erations\))168 605 y Ff(\017)23 b Fm(F)l(orm)346 594 y(\026)333 605 y Fg(N)17 b Fm(=)c(\()p Fg(I)g Fm(+)e Fg(\022)571 594 y Fm(\026)554 605 y Fg(M)617 594 y Fm(\026)604 605 y Fg(W)653 589 y Fe(T)680 605 y Fg(A)714 612 y Fe(k)736 605 y Fg(A)770 589 y Fe(T)770 618 y(k)811 594 y Fm(\026)797 605 y Fg(W)c Fm(\))250 677 y Fn({)312 666 y Fm(\026)299 677 y Fg(N)17 b Fm(:=)12 b Fg(\022)q(W)484 661 y Fe(T)513 677 y Fg(A)547 684 y Fe(k)568 677 y Fg(A)602 661 y Fe(T)602 691 y(k)630 677 y Fg(W)80 b Fm(\(\(2)p Fg(m)852 661 y Fk(2)881 677 y Fm(+)10 b Fg(m)p Fm(\))p Fg(t)1000 684 y Fe(a)1036 677 y Fm(op)q(erations\))250 742 y Fn({)312 730 y Fm(\026)299 742 y Fg(N)17 b Fm(:=)12 b Fg(I)i Fm(+)509 730 y(\026)492 742 y Fg(M)555 730 y Fm(\026)542 742 y Fg(N)78 b Fm(\()p Fg(O)q Fm(\()p Fg(m)769 725 y Fk(3)788 742 y Fm(\))15 b(op)q(erations\))168 814 y Ff(\017)23 b Fg(w)13 b Fm(:=)334 802 y(\026)321 814 y Fg(N)363 797 y Fc(\000)p Fk(1)426 802 y Fm(\026)409 814 y Fg(M)5 b(w)75 b Fm(\()p Fg(O)q Fm(\()p Fg(m)678 797 y Fk(3)697 814 y Fm(\))15 b(op)q(erations\))168 886 y Ff(\017)23 b Fg(\025)241 869 y Fc(\003)272 886 y Fm(:=)13 b Fg(\022)355 869 y Fk(2)375 886 y Fg(A)409 869 y Fe(T)409 899 y(k)450 874 y Fm(\026)437 886 y Fg(W)6 b(w)75 b Fm(\(2)p Fg(mt)691 893 y Fe(a)727 886 y Fm(op)q(erations\))168 958 y Ff(\017)23 b Fg(\025)241 941 y Fc(\003)272 958 y Fm(:=)13 b Ff(\000)p Fg(\022)q(u)e Fm(+)g Fg(\025)500 941 y Fc(\003)593 958 y Fm(\()p Fg(t)627 965 y Fe(a)663 958 y Fm(op)q(erations\))56 1051 y(3.)22 b Fg(d)138 1034 y Fc(\003)170 1051 y Fm(=)13 b Ff(\000)p Fg(H)291 1058 y Fe(k)312 1051 y Fm(\()p Fg(A)364 1058 y Fe(k)385 1051 y Fg(\025)412 1034 y Fc(\003)442 1051 y Fm(+)d Fg(g)509 1058 y Fe(k)530 1051 y Fm(\))i(=)h Ff(\000)648 1033 y Fk(1)p 648 1040 V 648 1067 a Fe(\022)671 1051 y Fm(\()p Fg(A)723 1058 y Fe(k)744 1051 y Fg(\025)771 1034 y Fc(\003)801 1051 y Fm(+)d Fg(g)868 1058 y Fe(k)889 1051 y Fm(\))g Ff(\000)976 1039 y Fm(\026)962 1051 y Fg(W)d Fm(\()1046 1039 y(\026)1030 1051 y Fg(M)1092 1039 y Fm(\026)1078 1051 y Fg(W)1127 1034 y Fe(T)1155 1051 y Fg(A)1189 1058 y Fe(k)1211 1051 y Fg(\025)1238 1034 y Fc(\003)1267 1051 y Fm(+)j Fg(v)r Fm(\))168 1144 y Ff(\017)23 b Fg(w)13 b Fm(:=)334 1132 y(\026)321 1144 y Fg(W)370 1127 y Fe(T)397 1144 y Fg(A)431 1151 y Fe(k)453 1144 y Fg(\025)480 1127 y Fc(\003)573 1144 y Fm(\(2)p Fg(mt)670 1151 y Fe(a)706 1144 y Fm(op)q(erations\))168 1216 y Ff(\017)23 b Fg(w)13 b Fm(:=)338 1204 y(\026)321 1216 y Fg(M)5 b(w)11 b Fm(+)f Fg(v)76 b Fm(\(2)p Fg(m)14 b Fm(op)q(erations\))168 1288 y Ff(\017)23 b Fg(d)238 1271 y Fc(\003)270 1288 y Fm(:=)12 b Ff(\000)370 1270 y Fk(1)p 370 1277 V 370 1304 a Fe(\022)393 1288 y Fm(\()p Fg(A)445 1295 y Fe(k)466 1288 y Fg(\025)493 1271 y Fc(\003)523 1288 y Fm(+)e Fg(g)590 1295 y Fe(k)611 1288 y Fm(\))g(+)698 1276 y(\026)684 1288 y Fg(W)d(w)75 b Fm(\(\(2)p Fg(m)9 b Fm(+)h(1\))p Fg(n)15 b Fm(op)q(erations\))0 1381 y(Bac)o(ktrac)o(k)f (if)i(necessary:)68 1472 y Ff(\017)23 b Fm(Compute)15 b Fg(\013)340 1456 y Fc(\003)372 1472 y Fm(=max)g Ff(f)p Fg(\013)e Fm(:)22 b Fg(l)621 1479 y Fe(i)648 1472 y Ff(\024)13 b Fg(x)722 1479 y Fe(c)749 1472 y Fm(+)d Fg(\013)p Fm(\()p Fg(x)867 1479 y Fe(k)899 1472 y Fm(+)g Fg(d)968 1456 y Fc(\003)998 1472 y Ff(\000)g Fg(x)1069 1456 y Fe(c)1086 1472 y Fm(\))j Ff(\024)g Fg(u)1191 1479 y Fe(i)1205 1472 y Fg(;)8 b(i)j Ff(2)i(F)5 b(g)73 b Fm(\()p Fg(t)16 b Fm(op)q(erations\))68 1565 y Ff(\017)23 b Fm(Set)18 b(\026)-26 b Fg(x)13 b Fm(=)g Fg(x)305 1572 y Fe(c)332 1565 y Fm(+)d Fg(\013)406 1549 y Fc(\003)426 1565 y Fm(\()p Fg(x)470 1572 y Fe(k)501 1565 y Fm(+)h Fg(d)571 1549 y Fc(\003)600 1565 y Ff(\000)g Fg(x)672 1549 y Fe(c)689 1565 y Fm(\).)79 b(\()p Fg(t)15 b Fm(op)q(erations\))71 1705 y(Since)k(the)g(v)o(ectors)f Fg(S)463 1688 y Fe(T)490 1705 y Fg(g)512 1712 y Fe(k)551 1705 y Fm(and)h Fg(Y)679 1688 y Fe(T)707 1705 y Fg(g)729 1712 y Fe(k)768 1705 y Fm(ha)o(v)o(e)f(b)q(een)h(computed)g(while)h(up)q(dating)f Fg(H)1555 1712 y Fe(k)1595 1705 y Fm([6)o(],)g(they)f(can)h(b)q(e)0 1761 y(sa)o(v)o(ed)c(so)g(that)f(the)h(pro)q(duct)539 1750 y(\026)526 1761 y Fg(W)575 1745 y Fe(T)603 1761 y Fg(g)625 1768 y Fe(k)661 1761 y Fm(requires)h(no)f(further)g(computation.)71 1818 y(The)k(total)g(n)o(um)o(b)q(er)h(of)f(op)q(erations)h(of)f(this)h(pro)q (cedure,)h(when)f(no)f(b)q(ounds)i(are)e(activ)o(e)g(\()p Fg(t)1713 1825 y Fe(a)1754 1818 y Fm(=)h(0\),)g(is)0 1874 y(\(2)p Fg(m)9 b Fm(+)i(1\))p Fg(n)f Fm(+)g Fg(O)q Fm(\()p Fg(m)353 1857 y Fk(2)372 1874 y Fm(\).)20 b(If)15 b Fg(t)484 1881 y Fe(a)521 1874 y Fm(b)q(ounds)h(are)f(activ)o(e,)596 1973 y(\(2)p Fg(m)10 b Fm(+)g(3\))p Fg(n)g Fm(+)h(9)p Fg(mt)935 1980 y Fe(a)965 1973 y Fm(+)g(2)p Fg(m)1074 1955 y Fk(2)1093 1973 y Fg(t)1109 1980 y Fe(a)1141 1973 y Fm(+)f Fg(O)q Fm(\()p Fg(m)1280 1955 y Fk(3)1299 1973 y Fm(\))0 2073 y(op)q(erations)15 b(are)f(required)h(to)f (compute)g(the)h(unconstrained)g(subspace)g(solution.)21 b(A)14 b(comparison)h(with)f(the)0 2129 y(cost)j(of)f(the)h(primal)h(computation)f (implemen)o(ted)i(as)d(describ)q(ed)j(ab)q(o)o(v)o(e,)e(giv)o(en)h(in)f (\(5.12\),)f(indicates)i(that)0 2186 y(the)d(dual)h(metho)q(d)g(w)o(ould)g(b) q(e)f(less)h(exp)q(ensiv)o(e)h(when)f(the)f(n)o(um)o(b)q(er)h(of)f(b)q(ound)h (v)m(ariables)h(is)e(m)o(uc)o(h)h(less)f(than)0 2242 y(the)g(n)o(um)o(b)q(er) h(of)f(free)g(v)m(ariables.)71 2298 y(Ho)o(w)o(ev)o(er,)23 b(this)g(comparison)f(do)q(es)h(not)g(tak)o(e)f(in)o(to)g(accoun)o(t)h(the)f (devices)i(for)e(sa)o(ving)h(costs)f(in)h(the)0 2355 y(computation)c(of)g (inner)i(pro)q(ducts)f(discussed)h(at)d(the)i(end)g(of)f Ff(x)p Fm(5.1.)32 b(In)20 b(fact,)g(the)f(primal)h(and)g(dual)g(ap-)0 2411 y(proac)o(hes)e(can)h(b)q(e)g(brough)o(t)f(closer)h(b)o(y)f(noting)g (that)g(the)h(matrix)f(\()p Fg(I)13 b Ff(\000)1292 2393 y Fk(1)p 1292 2400 V 1292 2427 a Fe(\022)1314 2411 y Fg(M)5 b(W)1412 2395 y Fe(T)1440 2411 y Fg(Z)s(Z)1508 2395 y Fe(T)1536 2411 y Fg(W)h Fm(\))1603 2385 y Fc(\000)p Fk(1)1650 2411 y Fg(M)24 b Fm(app)q(earing)0 2479 y(in)17 b(\(5.10\))d(can)i(b)q(e)h(sho)o(wn)e(to)g (b)q(e)i(iden)o(tical)h(to)d(the)h(matrix)g(\()p Fg(I)d Fm(+)e Fg(\022)1194 2467 y Fm(\026)1177 2479 y Fg(M)1239 2467 y Fm(\026)1226 2479 y Fg(W)1275 2462 y Fe(T)1303 2479 y Fg(A)1337 2486 y Fe(k)1358 2479 y Fg(A)1392 2462 y Fe(T)1392 2492 y(k)1433 2467 y Fm(\026)1420 2479 y Fg(W)6 b Fm(\))1487 2453 y Fc(\000)p Fk(1)1551 2467 y Fm(\026)1534 2479 y Fg(M)21 b Fm(in)c(\(5.22\).)j(If)c(the)934 2603 y(16)p eop %%Page: 17 18 bop 0 120 a Fm(second)17 b(expression)h(is)f(used)g(in)g(the)g(primal)g (approac)o(h,)f(the)h(cost)f(for)g(the)h(primal)g(computation)g(b)q(ecomes)0 177 y(2)p Fg(m)63 160 y Fk(2)82 177 y Fg(t)98 184 y Fe(a)123 177 y Fm(+)t(6)p Fg(mt)t Fm(+)t(4)p Fg(t)t Fm(+)t Fg(O)q Fm(\()p Fg(m)460 160 y Fk(3)480 177 y Fm(\),)12 b(making)g(it)h(more)e(comp)q(etitiv) o(e)i(with)g(the)f(dual)h(approac)o(h.)18 b(W)l(e)13 b(ha)o(v)o(e)e(used)i (this)0 233 y(expression)i(for)f(the)g(primal)i(metho)q(d)e(in)h(our)g (computational)f(exp)q(erimen)o(ts.)21 b(Also,)14 b(in)h(the)g(dual)g (approac)o(h,)0 289 y(as)e(with)i(the)e(primal,)i(w)o(e)f(sa)o(v)o(e)f(and)h (reuse)g(the)g(inner)h(pro)q(ducts)f(in)1197 278 y(\026)1183 289 y Fg(W)1232 273 y Fe(T)1260 289 y Fg(A)1294 296 y Fe(k)1315 289 y Fg(A)1349 273 y Fe(T)1349 303 y(k)1390 278 y Fm(\026)1377 289 y Fg(W)20 b Fm(that)13 b(are)h(relev)m(an)o(t)g(for)f(the)0 346 y(next)i(iteration.)0 502 y Fh(6.)37 b(Numerical)16 b(Exp)r(erime)o(n)n (ts)71 583 y Fm(W)l(e)k(ha)o(v)o(e)h(tested)f(our)h(limited)h(memory)f (algorithm)f(using)i(the)f(three)f(options)h(for)f(subspace)i(mini-)0 640 y(mization)e(\(the)g(direct)g(primal,)h(primal)g(conjugate)e(gradien)o(t) g(and)h(dual)h(metho)q(ds\),)f(and)g(compared)f(the)0 696 y(results)c(with)f (those)g(obtained)h(with)g(the)g(subroutine)g(SBMIN)g(of)f(LANCELOT)h([10)o (].)k(Both)14 b(our)g(co)q(de)h(and)0 753 y(LANCELOT)h(w)o(ere)f(terminated)g (when)632 855 y Ff(k)p Fg(P)6 b Fm(\()p Fg(x)734 862 y Fe(k)765 855 y Ff(\000)11 b Fg(g)833 862 y Fe(k)854 855 y Fg(;)d(l)q(;)g(u)p Fm(\))g Ff(\000)i Fg(x)1034 862 y Fe(k)1055 855 y Ff(k)1078 862 y Fc(1)1128 855 y Fg(<)j Fm(10)1222 836 y Fc(\000)p Fk(5)1269 855 y Fg(:)538 b Fm(\(6)p Fg(:)p Fm(1\))0 957 y(\(Note)14 b(from)g(\(2.2\))e (that)i Fg(P)6 b Fm(\()p Fg(x)518 964 y Fe(k)548 957 y Ff(\000)j Fg(g)614 964 y Fe(k)635 957 y Fg(;)f(l)q(;)g(u)p Fm(\))e Ff(\000)j Fg(x)812 964 y Fe(k)848 957 y Fm(is)15 b(the)g(pro)s(jected)f(gradien)o(t.\)) 19 b(The)c(algorithm)f(w)o(e)g(tested)h(is)0 1013 y(giv)o(en)h(as)e(follo)o (ws.)0 1120 y Fn(L-BF)o(GS-B)19 b(Algorithm)0 1176 y Fm(Cho)q(ose)i(a)f (starting)g(p)q(oin)o(t)i Fg(x)533 1183 y Fk(0)552 1176 y Fm(,)g(and)f(a)f (in)o(teger)h Fg(m)g Fm(that)f(determines)i(the)f(n)o(um)o(b)q(er)g(of)f (limited)j(memory)0 1233 y(corrections)15 b(stored.)20 b(De\014ne)c(the)f (initial)i(limited)g(memory)e(matrix)g(to)f(b)q(e)i(the)f(iden)o(tit)o(y)h (and)g(set)f Fg(k)e Fm(:=)g(0.)56 1326 y(1.)22 b(If)15 b(the)g(con)o(v)o (ergence)h(test)e(\(6.1\))g(is)i(satis\014ed)f(stop.)56 1420 y(2.)22 b(Compute)15 b(the)g(Cauc)o(h)o(y)g(p)q(oin)o(t)g(b)o(y)h(Algorithm)f (CP)l(.)56 1514 y(3.)22 b(Compute)15 b(a)h(searc)o(h)f(direction)j Fg(d)703 1521 y Fe(k)739 1514 y Fm(b)o(y)e(either)h(the)f(direct)g(primal)h (metho)q(d,)f(the)g(conjugate)f(gradien)o(t)114 1570 y(metho)q(d)g(or)g(the)g (dual)h(metho)q(d.)56 1664 y(4.)22 b(P)o(erform)g(a)h(line)i(searc)o(h)e (along)g Fg(d)736 1671 y Fe(k)757 1664 y Fm(,)i(sub)s(ject)e(to)g(the)g(b)q (ounds)h(on)g(the)f(problem,)j(to)c(compute)i(a)114 1721 y(steplength)17 b Fg(\025)362 1728 y Fe(k)383 1721 y Fm(,)f(and)h(set)f Fg(x)600 1728 y Fe(k)q Fk(+1)681 1721 y Fm(=)f Fg(x)757 1728 y Fe(k)789 1721 y Fm(+)d Fg(\025)863 1728 y Fe(k)884 1721 y Fg(d)908 1728 y Fe(k)929 1721 y Fm(.)23 b(The)17 b(line)h(searc)o(h)e(starts)f(with)i(the)g (unit)g(steplength,)114 1777 y(satis\014es)e(\(2.5\))f(with)h Fg(\013)e Fm(=)g(10)629 1761 y Fc(\000)p Fk(4)676 1777 y Fm(,)h(and)i (attempts)e(to)h(satisfy)f(\(2.6\))g(with)i Fg(\014)e Fm(=)f(0)p Fg(:)p Fm(9.)56 1871 y(5.)22 b(Compute)15 b Ff(r)p Fg(f)5 b Fm(\()p Fg(x)420 1878 y Fe(k)q Fk(+1)486 1871 y Fm(\).)56 1965 y(6.)22 b(If)c Fg(y)184 1972 y Fe(k)224 1965 y Fm(satis\014es)g(\(3.9\))f (with)h(eps=)g(2)p Fg(:)p Fm(2)12 b Ff(\002)g Fm(10)892 1948 y Fc(\000)p Fk(16)956 1965 y Fm(,)19 b(add)f Fg(s)1100 1972 y Fe(k)1140 1965 y Fm(and)g Fg(y)1253 1972 y Fe(k)1293 1965 y Fm(to)f Fg(S)1379 1972 y Fe(k)1419 1965 y Fm(and)h Fg(Y)1536 1972 y Fe(k)1558 1965 y Fm(.)28 b(If)19 b(more)e(than)h Fg(m)114 2021 y Fm(up)q(dates)d(are)g(stored,)g(delete)h(the)f(oldest)h(column)g(from) e Fg(S)1145 2028 y Fe(k)1182 2021 y Fm(and)h Fg(Y)1296 2028 y Fe(k)1318 2021 y Fm(.)56 2115 y(7.)22 b(Up)q(date)15 b Fg(S)306 2099 y Fe(T)303 2128 y(k)333 2115 y Fg(S)361 2122 y Fe(k)382 2115 y Fm(,)g Fg(Y)447 2099 y Fe(T)436 2128 y(k)474 2115 y Fg(Y)500 2122 y Fe(k)522 2115 y Fm(,)g Fg(L)581 2122 y Fe(k)617 2115 y Fm(and)h Fg(R)741 2122 y Fe(k)761 2115 y Fm(,)f(and)h(set)f Fg(\022)f Fm(=)f Fg(y)1056 2099 y Fe(T)1054 2128 y(k)1083 2115 y Fg(y)1105 2122 y Fe(k)1127 2115 y Fg(=y)1174 2099 y Fe(T)1172 2128 y(k)1201 2115 y Fg(s)1222 2122 y Fe(k)1244 2115 y Fg(:)56 2209 y Fm(8.)22 b(Set)15 b Fg(k)f Fm(:=)e Fg(k)f Fm(+)g(1)k(and)g(go)g(to)f (1.)71 2303 y(The)i(line)j(searc)o(h)d(w)o(as)g(p)q(erformed)h(b)o(y)g(means) f(of)g(the)h(routine)g(of)f(Mor)o(\023)-21 b(e)15 b(and)i(Th)o(uen)o(te)g ([19)o(])f(whic)o(h)i(at-)0 2359 y(tempts)12 b(to)g(enforce)g(the)h(W)l(olfe) f(conditions)i(\(2.5\))d(and)h(\(2.6\))f(b)o(y)h(a)g(sequence)i(of)e(p)q (olynomial)i(in)o(terp)q(olations.)0 2416 y(Since)f(steplengths)g(greater)e (than)g(one)h(ma)o(y)f(b)q(e)i(tried,)g(w)o(e)e(prev)o(en)o(t)h(the)g (routine)g(from)f(generating)h(infeasible)0 2472 y(p)q(oin)o(ts)i(b)o(y)f (de\014ning)i(the)f(maxim)o(um)f(steplength)h(for)f(the)h(routine)g(as)f(the) g(step)h(to)f(the)g(closest)h(b)q(ound)g(along)934 2603 y(17)p eop %%Page: 18 19 bop 0 120 a Fm(the)19 b(curren)o(t)g(searc)o(h)g(direction.)33 b(This)20 b(approac)o(h)e(implies)k(that,)d(if)g(the)g(ob)s(jectiv)o(e)g (function)h(is)g(b)q(ounded)0 177 y(b)q(elo)o(w,)e(the)f(line)h(searc)o(h)f (will)i(generate)e(a)f(p)q(oin)o(t)i Fg(x)909 184 y Fe(k)q Fk(+1)992 177 y Fm(that)f(satis\014es)g(\(2.5\))e(and)i(either)h(satis\014es) f(\(2.6\))f(or)0 233 y(hits)g(a)g(b)q(ound)g(that)g(w)o(as)e(not)i(activ)o(e) g(at)f Fg(x)751 240 y Fe(k)772 233 y Fm(.)22 b(W)l(e)16 b(ha)o(v)o(e)f (observ)o(ed)h(that)f(this)h(line)i(searc)o(h)d(p)q(erforms)h(w)o(ell)h(in)0 289 y(practice.)j(The)12 b(\014rst)g(implemen)o(tation)i(of)e(our)g (algorithm)h(used)g(a)f(bac)o(ktrac)o(king)g(line)i(searc)o(h,)e(but)h(w)o(e) f(found)0 346 y(that)j(this)h(approac)o(h)g(has)f(a)h(dra)o(wbac)o(k.)k(On)d (sev)o(eral)f(problems)g(the)g(bac)o(ktrac)o(king)f(line)j(searc)o(h)d (generated)0 402 y(steplengths)f(for)g(whic)o(h)g(the)g(up)q(dating)h (condition)g(\(3.9\))d(did)j(not)e(hold,)h(and)g(the)g(BF)o(GS)f(up)q(date)i (had)f(to)f(b)q(e)0 459 y(skipp)q(ed.)23 b(This)16 b(resulted)h(in)f(v)o(ery) f(p)q(o)q(or)h(p)q(erformance)g(in)g(some)g(problems.)21 b(In)c(con)o(trast,) d(when)i(using)g(the)0 515 y(new)g(line)h(searc)o(h,)f(the)g(up)q(date)g(is)g (v)o(ery)g(rarely)f(skipp)q(ed)j(in)f(our)e(tests,)g(and)h(the)g(p)q (erformance)g(of)f(the)h(co)q(de)0 572 y(w)o(as)g(mark)o(edly)i(b)q(etter.)26 b(T)l(o)17 b(explore)h(this)f(further)g(w)o(e)g(compared)h(the)f(early)g(v)o (ersion)h(of)f(our)g(co)q(de,)h(using)0 628 y(the)f(bac)o(ktrac)o(king)f (line)i(searc)o(h,)f(with)g(the)f(L-BF)o(GS)h(co)q(de)g(for)f(unconstrained)i (optimization)g([18)o(])e(\(whic)o(h)0 685 y(uses)g(the)f(routine)h(of)f(Mor) o(\023)-21 b(e)14 b(and)h(Th)o(uen)o(te\))g(on)h Fl(unc)n(onstr)n(aine)n(d)d Fm(problems,)j(and)f(found)h(that)f(L-BF)o(GS)g(w)o(as)0 741 y(sup)q(erior)g(in)g(a)f(signi\014can)o(t)i(n)o(um)o(b)q(er)e(of)g(problems.) 21 b(Our)14 b(results)h(suggest)f(that)f(bac)o(ktrac)o(king)h(line)i(searc)o (hes)0 798 y(can)g(signi\014can)o(tly)i(degrade)f(the)f(p)q(erformance)h(of)f (BF)o(GS,)f(and)i(that)e(satisfying)i(the)f(W)l(olfe)h(conditions)h(as)0 854 y(often)d(as)g(p)q(ossible)i(is)e(imp)q(ortan)o(t)g(in)h(practice.)71 910 y(Our)h(co)q(de)h(is)g(written)g(in)g(double)h(precision)g(F)o(OR)l(TRAN) f(77.)26 b(F)l(or)17 b(more)g(details)i(on)e(ho)o(w)g(to)g(up)q(date)0 967 y(the)e(limited)h(memory)e(matrices)h(in)g(step)g(7)f(see)h([6)o(].)20 b(When)15 b(testing)f(the)h(routine)g(SBMIN)g(of)f(LANCELOT)0 1023 y([10)o(])f(w)o(e)f(tried)i(three)f(options:)19 b(BF)o(GS,)12 b(SR1)i(and)f(exact)g(Hessians.)19 b(In)14 b(all)g(these)f(cases)g(w)o(e)g (used)h(the)f(default)0 1080 y(settings)i(of)g(LANCELOT.)71 1136 y(The)e(test)g(problems)h(w)o(ere)f(selected)h(from)f(the)g(CUTE)g (collection)i([4])e(\(whic)o(h)g(con)o(tains)h(some)f(problems)0 1193 y(from)g(the)h(MINP)l(A)o(CK-2)g(collection)h([2]\).)j(All)e(b)q(ound)e (constrained)h(problems)f(in)h(CUTE)e(w)o(ere)h(tested,)f(but)0 1249 y(some)d(problems)i(w)o(ere)e(discarded)i(for)f(one)f(of)h(the)g(follo)o (wing)g(reasons:)17 b(\(a\))10 b(the)h(n)o(um)o(b)q(er)g(of)g(v)m(ariables)h (w)o(as)e(less)0 1306 y(than)k(4;)f(\(b\))h(the)g(v)m(arious)g(algorithms)g (con)o(v)o(erged)g(to)f(a)g(di\013eren)o(t)h(solution)h(p)q(oin)o(t;)g(\(c\)) e(to)q(o)g(man)o(y)h(instances)0 1362 y(of)20 b(essen)o(tially)i(the)f(same)f (problem)h(are)f(giv)o(en)h(in)h(CUTE;)d(in)j(this)f(case)f(w)o(e)g(selected) i(a)e(represen)o(tativ)o(e)0 1419 y(sample.)71 1475 y(The)g(results)g(of)g (our)f(n)o(umerical)j(tests)d(are)h(giv)o(en)g(in)h(T)l(ables)f(1,)h(2)f(and) g(3.)34 b(All)21 b(computations)f(w)o(ere)0 1531 y(p)q(erformed)d(on)g(a)f (Sun)i(SP)l(AR)o(Cstation)f(2)g(with)g(a)f(40-MHz)h(CPU)g(and)g(32-MB)f (memory)l(.)24 b(W)l(e)17 b(record)g(the)0 1588 y(n)o(um)o(b)q(er)h(of)f(v)m (ariables)i Fg(n)f Fm(the)g(n)o(um)o(b)q(er)g(of)f(function)i(ev)m(aluations) g(and)f(the)g(total)f(run)h(time.)28 b(The)17 b(limited)0 1644 y(memory)e(co)q(de)h(alw)o(a)o(ys)e(computes)i(the)f(function)i(and)e (gradien)o(t)h(together,)e(so)h(that)g(nfg)g(denotes)g(the)h(total)0 1701 y(n)o(um)o(b)q(er)h(of)e(function)i(and)g(gradien)o(t)f(ev)m(aluations.) 24 b(Ho)o(w)o(ev)o(er)15 b(for)h(LANCELOT)h(the)f(n)o(um)o(b)q(er)h(of)f (function)0 1757 y(ev)m(aluations)d(is)g(not)e(necessarily)j(the)e(same)g(as) f(the)i(n)o(um)o(b)q(er)f(of)g(gradien)o(t)g(ev)m(aluations,)h(and)f(in)h (the)f(tables)h(w)o(e)0 1814 y(only)i(record)f(the)g(n)o(um)o(b)q(er)h(\(nf)t (\))e(of)h(function)h(ev)m(aluations.)21 b(The)14 b(notation)g(F1)g (indicates)i(that)d(the)i(solution)0 1870 y(w)o(as)h(not)h(obtained)h(after)f (999)f(function)j(ev)m(aluations,)f(and)g(F2)e(indicates)j(that)e(the)g(run)h (w)o(as)e(terminated)0 1927 y(b)q(ecause)g(the)e(next)h(iterate)f(generated)h (from)f(the)h(line)h(searc)o(h)e(w)o(as)g(to)q(o)g(close)h(to)f(the)h(curren) o(t)f(iterate)h(to)f(b)q(e)0 1983 y(recognized)k(as)d(distinct.)25 b(This)17 b(o)q(ccurred)g(only)g(in)g(some)f(cases)g(when)h(the)g(limited)h (memory)e(metho)q(d)g(w)o(as)0 2040 y(quite)i(near)e(the)h(solution)h(but)f (w)o(as)f(unable)i(to)f(meet)f(the)h(stopping)h(criterion.)26 b(In)17 b(T)l(able)h(1)e(nact)h(denotes)0 2096 y(the)d(n)o(um)o(b)q(er)h(of)e (activ)o(e)i(b)q(ounds)g(at)e(the)h(solution;)h(if)g(this)f(n)o(um)o(b)q(er)h (is)f(zero)g(it)h(do)q(es)f(not)g(mean)g(that)f(b)q(ounds)0 2152 y(w)o(ere)i(not)g(encoun)o(tered)h(during)g(the)f(solution)h(pro)q (cess.)934 2603 y(18)p eop %%Page: 19 20 bop 8 80 1898 2 v 7 137 2 57 v 670 137 V 712 120 a Fm(L-BF)o(GS-B)p 964 137 V 93 w(L-BF)o(GS-B)p 1281 137 V 83 w(LANCELOT)p 1598 137 V 55 w(LANCELOT)p 1904 137 V 7 193 V 33 176 a(Problem)257 b(n)78 b(nact)p 670 193 V 124 w(m=5)p 964 193 V 198 w(m=17)p 1281 193 V 196 w(BF)o(GS)p 1598 193 V 210 w(SR1)p 1904 193 V 8 195 1898 2 v 7 251 2 57 v 670 251 V 702 234 a(nfg)88 b(time)p 964 251 V 56 w(nfg)110 b(time)p 1281 251 V 80 w(nf)g(time)p 1598 251 V 57 w(nf)123 b(time)p 1904 251 V 8 253 1898 2 v 8 263 V 7 319 2 57 v 33 302 a(ALLINIT)238 b(4)140 b(1)p 670 319 V 72 w(19)95 b(0.13)p 964 319 V 72 w(19)117 b(0.16)p 1281 319 V 71 w(18)h(0.19)p 1598 319 V 48 w(12)131 b(0.14)p 1904 319 V 8 321 1898 2 v 7 377 2 57 v 33 361 a(BDEXP)218 b(100)140 b(0)p 670 377 V 72 w(15)95 b(0.28)p 964 377 V 72 w(16)117 b(0.40)p 1281 377 V 71 w(18)h(1.31)p 1598 377 V 48 w(24)131 b(1.59)p 1904 377 V 8 379 1898 2 v 7 436 2 57 v 33 419 a(BIGGS5)259 b(6)140 b(1)p 670 436 V 50 w(117)94 b(1.08)p 964 436 V 72 w(69)117 b(1.80)p 1281 436 V 71 w(47)h(0.70)p 1598 436 V 48 w(41)131 b(0.58)p 1904 436 V 8 437 1898 2 v 7 494 2 57 v 33 477 a(BQPGASIM)153 b(50)140 b(7)p 670 494 V 72 w(25)95 b(0.36)p 964 494 V 72 w(23)117 b(0.54)p 1281 494 V 49 w(322)g(9.07)p 1598 494 V 71 w(8)131 b(0.59)p 1904 494 V 8 495 1898 2 v 7 552 2 57 v 33 535 a(BQPGA)o(USS)108 b(2003)116 b(27)p 670 552 V 66 w(F1)49 b(349.75)p 964 552 V 64 w(F1)72 b(631.31)p 1281 552 V 63 w(F1)50 b(1722.40)p 1598 552 V 47 w(20)62 b(2149.33)p 1904 552 V 8 553 1898 2 v 7 610 2 57 v 33 593 a(HA)l(TFLD)o(A)204 b(4)140 b(0)p 670 610 V 72 w(39)95 b(0.21)p 964 610 V 72 w(36)117 b(0.32)p 1281 610 V 71 w(39)h(0.32)p 1598 610 V 48 w(36)131 b(0.31)p 1904 610 V 8 612 1898 2 v 7 668 2 57 v 33 651 a(HA)l(TFLDB)205 b(4)140 b(1)p 670 668 V 72 w(34)95 b(0.16)p 964 668 V 72 w(32)117 b(0.26)p 1281 668 V 71 w(31)h(0.25)p 1598 668 V 48 w(33)131 b(0.31)p 1904 668 V 8 670 1898 2 v 7 726 2 57 v 33 709 a(HA)l(TFLDC)181 b(25)140 b(0)p 670 726 V 72 w(23)95 b(0.26)p 964 726 V 72 w(23)117 b(0.44)p 1281 726 V 94 w(5)h(0.14)p 1598 726 V 71 w(5)131 b(0.11)p 1904 726 V 8 728 1898 2 v 7 784 2 57 v 33 767 a(HS110)276 b(50)117 b(50)p 670 784 V 95 w(2)95 b(0.03)p 964 784 V 94 w(2)118 b(0.06)p 1281 784 V 94 w(2)g(0.17)p 1598 784 V 71 w(2)131 b(0.13)p 1904 784 V 8 786 1898 2 v 7 842 2 57 v 33 825 a(HS25)322 b(3)140 b(1)p 670 842 V 95 w(2)95 b(0.06)p 964 842 V 94 w(2)118 b(0.05)p 1281 842 V 94 w(1)g(0.06)p 1598 842 V 71 w(1)131 b(0.05)p 1904 842 V 8 844 1898 2 v 7 901 2 57 v 33 884 a(HS45)322 b(5)140 b(5)p 670 901 V 72 w(11)95 b(0.04)p 964 901 V 72 w(11)117 b(0.02)p 1281 901 V 94 w(3)h(0.05)p 1598 901 V 71 w(3)131 b(0.04)p 1904 901 V 8 902 1898 2 v 7 959 2 57 v 33 942 a(JNLBRNGA)82 b(15625)71 b(5657)p 670 959 V 49 w(313)49 b(843.59)p 964 959 V 48 w(280)g(1222.33)p 1281 959 V 70 w(24)h(1478.94)p 1598 959 V 47 w(24)62 b(1474.09)p 1904 959 V 8 960 1898 2 v 7 1017 2 57 v 33 1000 a(JNLBRNGB)129 b(100)117 b(68)p 670 1017 V 72 w(61)95 b(1.23)p 964 1017 V 72 w(55)117 b(2.33)p 1281 1017 V 94 w(6)h(0.45)p 1598 1017 V 71 w(6)131 b(0.45)p 1904 1017 V 8 1018 1898 2 v 7 1075 2 57 v 33 1058 a(LINVERSE)172 b(19)140 b(6)p 670 1075 V 72 w(63)95 b(0.71)p 964 1075 V 49 w(107)117 b(3.88)p 1281 1075 V 64 w(F1)95 b(19.72)p 1598 1075 V 48 w(18)131 b(0.51)p 1904 1075 V 8 1077 1898 2 v 7 1133 2 57 v 33 1116 a(MAXLIKA)204 b(8)140 b(1)p 670 1133 V 66 w(F1)71 b(59.09)p 964 1133 V 49 w(157)94 b(12.31)p 1281 1133 V 64 w(F1)72 b(208.60)p 1598 1133 V 48 w(98)108 b(24.65)p 1904 1133 V 8 1135 1898 2 v 7 1191 2 57 v 33 1174 a(MCCORMCK)118 b(10)140 b(0)p 670 1191 V 72 w(11)95 b(0.10)p 964 1191 V 72 w(11)117 b(0.13)p 1281 1191 V 94 w(7)h(0.13)p 1598 1191 V 71 w(7)131 b(0.10)p 1904 1191 V 8 1193 1898 2 v 7 1249 2 57 v 33 1232 a(NONSCOMP)k(25)140 b(0)p 670 1249 V 72 w(31)95 b(0.34)p 964 1249 V 72 w(30)117 b(0.72)p 1281 1249 V 94 w(6)h(0.15)p 1598 1249 V 71 w(9)131 b(0.19)p 1904 1249 V 8 1251 1898 2 v 7 1307 2 57 v 33 1290 a(OBSTCLAE)108 b(5625)71 b(2724)p 670 1307 V 49 w(282)49 b(277.71)p 964 1307 V 48 w(308)72 b(495.42)p 1281 1307 V 93 w(7)50 b(1765.61)p 1598 1307 V 70 w(7)62 b(1763.42)p 1904 1307 V 8 1309 1898 2 v 7 1365 2 57 v 33 1348 a(OBSTCLAL)133 b(100)117 b(63)p 670 1365 V 72 w(15)95 b(0.30)p 964 1365 V 72 w(14)117 b(0.32)p 1281 1365 V 94 w(8)h(0.61)p 1598 1365 V 71 w(8)131 b(0.50)p 1904 1365 V 8 1367 1898 2 v 7 1424 2 57 v 33 1407 a(OBSTCLBL)k(100)117 b(84)p 670 1424 V 72 w(11)95 b(0.24)p 964 1424 V 72 w(11)117 b(0.25)p 1281 1424 V 94 w(5)h(0.44)p 1598 1424 V 71 w(5)131 b(0.45)p 1904 1424 V 8 1425 1898 2 v 7 1482 2 57 v 33 1465 a(OBSTCLBM)76 b(15625)71 b(4309)p 670 1482 V 49 w(133)49 b(383.61)p 964 1482 V 48 w(138)72 b(641.45)p 1281 1482 V 93 w(7)50 b(1275.77)p 1598 1482 V 70 w(7)62 b(1275.72)p 1904 1482 V 8 1483 1898 2 v 7 1540 2 57 v 33 1523 a(OBSTCLBU)129 b(100)117 b(84)p 670 1540 V 72 w(12)95 b(0.25)p 964 1540 V 72 w(12)117 b(0.26)p 1281 1540 V 94 w(4)h(0.39)p 1598 1540 V 71 w(4)131 b(0.43)p 1904 1540 V 8 1541 1898 2 v 7 1598 2 57 v 33 1581 a(P)l(ALMER1)209 b(4)140 b(0)p 670 1598 V 72 w(34)95 b(0.25)p 964 1598 V 72 w(24)117 b(0.20)p 1281 1598 V 71 w(37)h(0.61)p 1598 1598 V 48 w(32)131 b(0.52)p 1904 1598 V 8 1600 1898 2 v 7 1656 2 57 v 33 1639 a(P)l(ALMER2)209 b(4)140 b(0)p 670 1656 V 66 w(F1)94 b(2.14)p 964 1656 V 72 w(30)117 b(0.33)p 1281 1656 V 71 w(88)h(1.23)p 1598 1656 V 48 w(27)131 b(0.36)p 1904 1656 V 8 1658 1898 2 v 7 1714 2 57 v 33 1697 a(P)l(ALMER3)209 b(4)140 b(1)p 670 1714 V 66 w(F2)94 b(0.59)p 964 1714 V 65 w(F2)117 b(0.19)p 1281 1714 V 71 w(71)h(0.92)p 1598 1714 V 48 w(49)131 b(0.61)p 1904 1714 V 8 1716 1898 2 v 7 1772 2 57 v 33 1755 a(P)l(ALMER4)209 b(4)140 b(0)p 670 1772 V 72 w(25)95 b(0.15)p 964 1772 V 72 w(25)117 b(0.16)p 1281 1772 V 64 w(F1)95 b(12.13)p 1598 1772 V 48 w(44)131 b(0.57)p 1904 1772 V 8 1774 1898 2 v 7 1830 2 57 v 33 1813 a(PR)o(OBPENL)c(500)140 b(0)p 670 1830 V 95 w(3)95 b(0.37)p 964 1830 V 94 w(3)118 b(0.38)p 1281 1830 V 94 w(4)g(2.51)p 1598 1830 V 71 w(3)131 b(1.98)p 1904 1830 V 8 1832 1898 2 v 7 1888 2 57 v 33 1872 a(PSPDOC)237 b(4)140 b(0)p 670 1888 V 72 w(11)95 b(0.07)p 964 1888 V 72 w(11)117 b(0.10)p 1281 1888 V 94 w(3)h(0.06)p 1598 1888 V 71 w(3)131 b(0.06)p 1904 1888 V 8 1890 1898 2 v 7 1947 2 57 v 33 1930 a(S368)333 b(8)140 b(2)p 670 1947 V 72 w(11)95 b(0.12)p 964 1947 V 72 w(12)117 b(0.16)p 1281 1947 V 64 w(F1)95 b(24.58)p 1598 1947 V 48 w(33)131 b(0.77)p 1904 1947 V 8 1948 1898 2 v 7 2005 2 57 v 33 1988 a(TORSION1)147 b(100)117 b(68)p 670 2005 V 72 w(12)95 b(0.22)p 964 2005 V 72 w(10)117 b(0.16)p 1281 2005 V 94 w(7)h(0.59)p 1598 2005 V 71 w(7)131 b(0.53)p 1904 2005 V 8 2006 1898 2 v 7 2063 2 57 v 33 2046 a(TORSION2)147 b(100)117 b(68)p 670 2063 V 72 w(11)95 b(0.21)p 964 2063 V 72 w(10)117 b(0.19)p 1281 2063 V 94 w(6)h(0.37)p 1598 2063 V 71 w(6)131 b(0.37)p 1904 2063 V 8 2064 1898 2 v 7 2121 2 57 v 33 2104 a(TORSION3)147 b(100)117 b(88)p 670 2121 V 95 w(5)95 b(0.10)p 964 2121 V 94 w(5)118 b(0.11)p 1281 2121 V 94 w(5)g(0.38)p 1598 2121 V 71 w(5)131 b(0.27)p 1904 2121 V 8 2123 1898 2 v 7 2179 2 57 v 33 2162 a(TORSION4)147 b(100)117 b(88)p 670 2179 V 95 w(7)95 b(0.15)p 964 2179 V 94 w(7)118 b(0.15)p 1281 2179 V 94 w(4)g(0.37)p 1598 2179 V 71 w(4)131 b(0.37)p 1904 2179 V 8 2181 1898 2 v 7 2237 2 57 v 33 2220 a(TORSION6)102 b(14884)48 b(12316)p 670 2237 V 49 w(301)h(835.80)p 964 2237 V 48 w(400)g(1582.75)p 1281 2237 V 70 w(10)72 b(171.64)p 1598 2237 V 48 w(10)85 b(168.67)p 1904 2237 V 8 2239 1898 2 v 51 2324 a(T)l(able)16 b(1.)k(T)l(est)15 b(results)g(of)g(new)g(limited)i (memory)e(metho)q(d)h(\(L-BF)o(GS-B\))e(using)i(primal)g(metho)q(d)g(for)185 2381 y(subspace)g(minimization,)h(and)e(results)h(of)e(LANCELOT's)i(BF)o(GS)f (and)g(SR1)h(options.)934 2603 y(19)p eop %%Page: 20 21 bop 96 80 1721 2 v 95 137 2 57 v 595 137 V 636 120 a Fm(L-BF)o(GS-B)p 889 137 V 82 w(L-BF)o(GS-B)p 1183 137 V 94 w(L-BF)o(GS-B)p 1500 137 V 105 w(L-BF)o(GS-B)p 1816 137 V 95 193 V 121 176 a(Problem)257 b(n)p 595 193 V 125 w(m=3)p 889 193 V 198 w(m=5)p 1183 193 V 198 w(m=17)p 1500 193 V 197 w(m=29)p 1816 193 V 96 195 1721 2 v 95 251 2 57 v 595 251 V 627 234 a(nfg)87 b(time)p 889 251 V 57 w(nfg)g(time)p 1183 251 V 57 w(nfg)110 b(time)p 1500 251 V 57 w(nfg)g(time)p 1816 251 V 96 253 1721 2 v 96 263 V 95 319 2 57 v 121 302 a(ALLINIT)238 b(4)p 595 319 V 72 w(22)95 b(0.13)p 889 319 V 71 w(19)g(0.13)p 1183 319 V 71 w(19)118 b(0.16)p 1500 319 V 71 w(19)f(0.17)p 1816 319 V 96 321 1721 2 v 95 377 2 57 v 121 361 a(BDEXP)218 b(100)p 595 377 V 72 w(15)95 b(0.29)p 889 377 V 71 w(15)g(0.28)p 1183 377 V 71 w(16)118 b(0.40)p 1500 377 V 71 w(16)f(0.44)p 1816 377 V 96 379 1721 2 v 95 436 2 57 v 121 419 a(BIGGS5)259 b(6)p 595 436 V 49 w(127)95 b(0.74)p 889 436 V 48 w(117)g(1.08)p 1183 436 V 71 w(69)118 b(1.80)p 1500 436 V 71 w(71)f(3.20)p 1816 436 V 96 437 1721 2 v 95 494 2 57 v 121 477 a(BQPGASIM)153 b(50)p 595 494 V 72 w(28)95 b(0.34)p 889 494 V 71 w(25)g(0.36)p 1183 494 V 71 w(23)118 b(0.54)p 1500 494 V 71 w(23)f(0.56)p 1816 494 V 96 495 1721 2 v 95 552 2 57 v 121 535 a(BQPGA)o(USS)108 b(2003)p 595 552 V 64 w(F1)49 b(352.09)p 889 552 V 64 w(F1)g(349.75)p 1183 552 V 64 w(F1)72 b(631.31)p 1500 552 V 64 w(F1)g(932.04)p 1816 552 V 96 553 1721 2 v 95 610 2 57 v 121 593 a(HA)l(TFLD)o(A)204 b(4)p 595 610 V 72 w(47)95 b(0.17)p 889 610 V 71 w(39)g(0.21)p 1183 610 V 71 w(36)118 b(0.32)p 1500 610 V 71 w(37)f(0.50)p 1816 610 V 96 612 1721 2 v 95 668 2 57 v 121 651 a(HA)l(TFLDB)205 b(4)p 595 668 V 72 w(37)95 b(0.15)p 889 668 V 71 w(34)g(0.16)p 1183 668 V 71 w(32)118 b(0.26)p 1500 668 V 71 w(32)f(0.27)p 1816 668 V 96 670 1721 2 v 95 726 2 57 v 121 709 a(HA)l(TFLDC)181 b(25)p 595 726 V 72 w(25)95 b(0.23)p 889 726 V 71 w(23)g(0.26)p 1183 726 V 71 w(23)118 b(0.44)p 1500 726 V 71 w(23)f(0.50)p 1816 726 V 96 728 1721 2 v 95 784 2 57 v 121 767 a(HS110)276 b(50)p 595 784 V 95 w(2)95 b(0.05)p 889 784 V 94 w(2)g(0.03)p 1183 784 V 94 w(2)118 b(0.06)p 1500 784 V 94 w(2)f(0.04)p 1816 784 V 96 786 1721 2 v 95 842 2 57 v 121 825 a(HS25)322 b(3)p 595 842 V 95 w(2)95 b(0.07)p 889 842 V 94 w(2)g(0.06)p 1183 842 V 94 w(2)118 b(0.05)p 1500 842 V 94 w(2)f(0.07)p 1816 842 V 96 844 1721 2 v 95 901 2 57 v 121 884 a(HS45)322 b(5)p 595 901 V 72 w(11)95 b(0.04)p 889 901 V 71 w(11)g(0.04)p 1183 901 V 71 w(11)118 b(0.02)p 1500 901 V 71 w(11)f(0.04)p 1816 901 V 96 902 1721 2 v 95 959 2 57 v 121 942 a(JNLBRNGA)82 b(15625)p 595 959 V 48 w(382)49 b(891.11)p 889 959 V 48 w(313)g(843.59)p 1183 959 V 48 w(280)g(1222.33)p 1500 959 V 48 w(322)g(1916.31)p 1816 959 V 96 960 1721 2 v 95 1017 2 57 v 121 1000 a(JNLBRNGB)129 b(100)p 595 1017 V 72 w(84)95 b(1.32)p 889 1017 V 71 w(61)g(1.23)p 1183 1017 V 71 w(55)118 b(2.33)p 1500 1017 V 71 w(56)f(3.55)p 1816 1017 V 96 1018 1721 2 v 95 1075 2 57 v 121 1058 a(LINVERSE)172 b(19)p 595 1075 V 72 w(96)95 b(0.91)p 889 1075 V 71 w(63)g(0.71)p 1183 1075 V 48 w(107)118 b(3.88)p 1500 1075 V 71 w(99)f(4.68)p 1816 1075 V 96 1077 1721 2 v 95 1133 2 57 v 121 1116 a(MAXLIKA)204 b(8)p 595 1133 V 65 w(F1)72 b(57.76)p 889 1133 V 64 w(F1)g(59.09)p 1183 1133 V 48 w(157)95 b(12.31)p 1500 1133 V 48 w(118)g(13.32)p 1816 1133 V 96 1135 1721 2 v 95 1191 2 57 v 121 1174 a(MCCORMCK)118 b(10)p 595 1191 V 72 w(12)95 b(0.10)p 889 1191 V 71 w(11)g(0.10)p 1183 1191 V 71 w(11)118 b(0.13)p 1500 1191 V 71 w(11)f(0.11)p 1816 1191 V 96 1193 1721 2 v 95 1249 2 57 v 121 1232 a(NONSCOMP)135 b(25)p 595 1249 V 72 w(32)95 b(0.29)p 889 1249 V 71 w(31)g(0.34)p 1183 1249 V 71 w(30)118 b(0.72)p 1500 1249 V 71 w(30)f(0.88)p 1816 1249 V 96 1251 1721 2 v 95 1307 2 57 v 121 1290 a(OBSTCLAE)108 b(5625)p 595 1307 V 48 w(266)49 b(253.63)p 889 1307 V 48 w(282)g(277.71)p 1183 1307 V 48 w(308)72 b(495.42)p 1500 1307 V 48 w(284)g(641.66)p 1816 1307 V 96 1309 1721 2 v 95 1365 2 57 v 121 1348 a(OBSTCLAL)133 b(100)p 595 1365 V 72 w(16)95 b(0.32)p 889 1365 V 71 w(15)g(0.30)p 1183 1365 V 71 w(14)118 b(0.32)p 1500 1365 V 71 w(14)f(0.24)p 1816 1365 V 96 1367 1721 2 v 95 1424 2 57 v 121 1407 a(OBSTCLBL)135 b(100)p 595 1424 V 72 w(12)95 b(0.22)p 889 1424 V 71 w(11)g(0.24)p 1183 1424 V 71 w(11)118 b(0.25)p 1500 1424 V 71 w(11)f(0.22)p 1816 1424 V 96 1425 1721 2 v 95 1482 2 57 v 121 1465 a(OBSTCLBM)76 b(15625)p 595 1482 V 48 w(140)49 b(349.89)p 889 1482 V 48 w(133)g(383.61)p 1183 1482 V 48 w(138)72 b(641.45)p 1500 1482 V 48 w(153)g(948.07)p 1816 1482 V 96 1483 1721 2 v 95 1540 2 57 v 121 1523 a(OBSTCLBU)129 b(100)p 595 1540 V 72 w(11)95 b(0.22)p 889 1540 V 71 w(12)g(0.25)p 1183 1540 V 71 w(12)118 b(0.26)p 1500 1540 V 71 w(12)f(0.25)p 1816 1540 V 96 1541 1721 2 v 95 1598 2 57 v 121 1581 a(P)l(ALMER1)209 b(4)p 595 1598 V 72 w(34)95 b(0.19)p 889 1598 V 71 w(34)g(0.25)p 1183 1598 V 71 w(24)118 b(0.20)p 1500 1598 V 71 w(24)f(0.20)p 1816 1598 V 96 1600 1721 2 v 95 1656 2 57 v 121 1639 a(P)l(ALMER2)209 b(4)p 595 1656 V 72 w(37)95 b(0.21)p 889 1656 V 64 w(F1)g(2.14)p 1183 1656 V 71 w(30)118 b(0.33)p 1500 1656 V 71 w(30)f(0.31)p 1816 1656 V 96 1658 1721 2 v 95 1714 2 57 v 121 1697 a(P)l(ALMER3)209 b(4)p 595 1714 V 72 w(93)95 b(0.50)p 889 1714 V 64 w(F2)g(0.59)p 1183 1714 V 64 w(F2)118 b(0.19)p 1500 1714 V 64 w(F2)f(0.20)p 1816 1714 V 96 1716 1721 2 v 95 1772 2 57 v 121 1755 a(P)l(ALMER4)209 b(4)p 595 1772 V 65 w(F2)95 b(0.63)p 889 1772 V 71 w(25)g(0.15)p 1183 1772 V 71 w(25)118 b(0.16)p 1500 1772 V 71 w(25)f(0.16)p 1816 1772 V 96 1774 1721 2 v 95 1830 2 57 v 121 1813 a(PR)o(OBPENL)127 b(500)p 595 1830 V 95 w(3)95 b(0.36)p 889 1830 V 94 w(3)g(0.37)p 1183 1830 V 94 w(3)118 b(0.38)p 1500 1830 V 94 w(3)f(0.36)p 1816 1830 V 96 1832 1721 2 v 95 1888 2 57 v 121 1872 a(PSPDOC)237 b(4)p 595 1888 V 72 w(15)95 b(0.10)p 889 1888 V 71 w(11)g(0.07)p 1183 1888 V 71 w(11)118 b(0.10)p 1500 1888 V 71 w(11)f(0.09)p 1816 1888 V 96 1890 1721 2 v 95 1947 2 57 v 121 1930 a(S368)333 b(8)p 595 1947 V 72 w(12)95 b(0.16)p 889 1947 V 71 w(11)g(0.12)p 1183 1947 V 71 w(12)118 b(0.16)p 1500 1947 V 71 w(12)f(0.15)p 1816 1947 V 96 1948 1721 2 v 95 2005 2 57 v 121 1988 a(TORSION1)147 b(100)p 595 2005 V 72 w(12)95 b(0.17)p 889 2005 V 71 w(12)g(0.22)p 1183 2005 V 71 w(10)118 b(0.16)p 1500 2005 V 71 w(10)f(0.17)p 1816 2005 V 96 2006 1721 2 v 95 2063 2 57 v 121 2046 a(TORSION2)147 b(100)p 595 2063 V 72 w(14)95 b(0.23)p 889 2063 V 71 w(11)g(0.21)p 1183 2063 V 71 w(10)118 b(0.19)p 1500 2063 V 71 w(10)f(0.19)p 1816 2063 V 96 2064 1721 2 v 95 2121 2 57 v 121 2104 a(TORSION3)147 b(100)p 595 2121 V 95 w(5)95 b(0.10)p 889 2121 V 94 w(5)g(0.10)p 1183 2121 V 94 w(5)118 b(0.11)p 1500 2121 V 94 w(5)f(0.11)p 1816 2121 V 96 2123 1721 2 v 95 2179 2 57 v 121 2162 a(TORSION4)147 b(100)p 595 2179 V 95 w(8)95 b(0.17)p 889 2179 V 94 w(7)g(0.15)p 1183 2179 V 94 w(7)118 b(0.15)p 1500 2179 V 94 w(7)f(0.16)p 1816 2179 V 96 2181 1721 2 v 95 2237 2 57 v 121 2220 a(TORSION6)102 b(14884)p 595 2237 V 48 w(298)49 b(646.21)p 889 2237 V 48 w(301)g(835.80)p 1183 2237 V 48 w(400)g(1582.75)p 1500 2237 V 48 w(385)g(2442.59)p 1816 2237 V 96 2239 1721 2 v 84 2322 a(T)l(able)16 b(2.)k(Results)c(of)f(new) g(limited)i(memory)e(metho)q(d,)g(using)h(the)f(primal)h(metho)q(d)g(for)e (subspace)345 2378 y(minimization,)j(for)d(v)m(arious)i(v)m(alues)g(of)f(the) g(memory)g(parameter)f Fg(m)p Fm(.)934 2603 y(20)p eop %%Page: 21 22 bop 101 80 1711 2 v 100 137 2 57 v 600 137 V 641 120 a Fm(L-BF)o(GS-B)p 894 137 V 82 w(L-BF)o(GS-B)p 1188 137 V 94 w(L-BF)o(GS-B)p 1505 137 V 77 w(LANCELOT)p 1811 137 V 100 193 V 126 176 a(Problem)257 b(n)p 600 193 V 108 w(primal)p 894 193 V 188 w(dual)p 1188 193 V 242 w(cg)p 1505 193 V 214 w(Hessian)p 1811 193 V 101 195 1711 2 v 100 251 2 57 v 600 251 V 632 234 a(nfg)87 b(time)p 894 251 V 57 w(nfg)g(time)p 1188 251 V 57 w(nfg)110 b(time)p 1505 251 V 57 w(nf)123 b(time)p 1811 251 V 101 253 1711 2 v 101 263 V 100 319 2 57 v 126 302 a(ALLINIT)238 b(4)p 600 319 V 72 w(19)95 b(0.13)p 894 319 V 71 w(19)g(0.13)p 1188 319 V 71 w(19)118 b(0.13)p 1505 319 V 71 w(7)130 b(0.09)p 1811 319 V 101 321 1711 2 v 100 377 2 57 v 126 361 a(BDEXP)218 b(100)p 600 377 V 72 w(15)95 b(0.28)p 894 377 V 71 w(15)g(0.28)p 1188 377 V 71 w(15)118 b(0.34)p 1505 377 V 48 w(11)130 b(0.80)p 1811 377 V 101 379 1711 2 v 100 436 2 57 v 126 419 a(BIGGS5)259 b(6)p 600 436 V 49 w(117)95 b(1.08)p 894 436 V 48 w(130)g(0.97)p 1188 436 V 49 w(101)117 b(0.99)p 1505 436 V 48 w(19)130 b(0.29)p 1811 436 V 101 437 1711 2 v 100 494 2 57 v 126 477 a(BQPGASIM)153 b(50)p 600 494 V 72 w(25)95 b(0.36)p 894 494 V 71 w(25)g(0.34)p 1188 494 V 71 w(26)118 b(0.45)p 1505 494 V 71 w(4)130 b(0.40)p 1811 494 V 101 495 1711 2 v 100 552 2 57 v 126 535 a(BQPGA)o(USS)108 b(2003)p 600 552 V 64 w(F1)49 b(349.75)p 894 552 V 64 w(F1)g(293.81)p 1188 552 V 64 w(F1)72 b(471.29)p 1505 552 V 71 w(9)62 b(1908.49)p 1811 552 V 101 553 1711 2 v 100 610 2 57 v 126 593 a(HA)l(TFLD)o(A)204 b(4)p 600 610 V 72 w(39)95 b(0.21)p 894 610 V 71 w(39)g(0.18)p 1188 610 V 71 w(43)118 b(0.22)p 1505 610 V 48 w(24)130 b(0.22)p 1811 610 V 101 612 1711 2 v 100 668 2 57 v 126 651 a(HA)l(TFLDB)205 b(4)p 600 668 V 72 w(34)95 b(0.16)p 894 668 V 71 w(34)g(0.15)p 1188 668 V 71 w(35)118 b(0.15)p 1505 668 V 48 w(21)130 b(0.20)p 1811 668 V 101 670 1711 2 v 100 726 2 57 v 126 709 a(HA)l(TFLDC)181 b(25)p 600 726 V 72 w(23)95 b(0.26)p 894 726 V 71 w(23)g(0.20)p 1188 726 V 71 w(24)118 b(0.29)p 1505 726 V 71 w(5)130 b(0.14)p 1811 726 V 101 728 1711 2 v 100 784 2 57 v 126 767 a(HS110)276 b(50)p 600 784 V 95 w(2)95 b(0.03)p 894 784 V 94 w(2)g(0.05)p 1188 784 V 94 w(2)118 b(0.05)p 1505 784 V 71 w(2)130 b(0.17)p 1811 784 V 101 786 1711 2 v 100 842 2 57 v 126 825 a(HS25)322 b(3)p 600 842 V 95 w(2)95 b(0.06)p 894 842 V 94 w(2)g(0.07)p 1188 842 V 94 w(2)118 b(0.07)p 1505 842 V 71 w(1)130 b(0.06)p 1811 842 V 101 844 1711 2 v 100 901 2 57 v 126 884 a(HS45)322 b(5)p 600 901 V 72 w(11)95 b(0.04)p 894 901 V 71 w(11)g(0.04)p 1188 901 V 71 w(11)118 b(0.02)p 1505 901 V 71 w(3)130 b(0.04)p 1811 901 V 101 902 1711 2 v 100 959 2 57 v 126 942 a(JNLBRNGA)82 b(15625)p 600 959 V 48 w(313)49 b(843.59)p 894 959 V 48 w(347)g(988.31)p 1188 959 V 49 w(552)g(1629.03)p 1505 959 V 47 w(22)62 b(1775.51)p 1811 959 V 101 960 1711 2 v 100 1017 2 57 v 126 1000 a(JNLBRNGB)129 b(100)p 600 1017 V 72 w(61)95 b(1.23)p 894 1017 V 71 w(61)g(1.16)p 1188 1017 V 71 w(62)118 b(1.35)p 1505 1017 V 71 w(5)130 b(0.40)p 1811 1017 V 101 1018 1711 2 v 100 1075 2 57 v 126 1058 a(LINVERSE)172 b(19)p 600 1075 V 72 w(63)95 b(0.71)p 894 1075 V 71 w(63)g(0.64)p 1188 1075 V 49 w(192)117 b(1.79)p 1505 1075 V 48 w(19)130 b(0.50)p 1811 1075 V 101 1077 1711 2 v 100 1133 2 57 v 126 1116 a(MAXLIKA)204 b(8)p 600 1133 V 65 w(F1)72 b(59.09)p 894 1133 V 64 w(F1)g(59.90)p 1188 1133 V 49 w(801)94 b(46.88)p 1505 1133 V 71 w(9)130 b(2.09)p 1811 1133 V 101 1135 1711 2 v 100 1191 2 57 v 126 1174 a(MCCORMCK)118 b(10)p 600 1191 V 72 w(11)95 b(0.10)p 894 1191 V 71 w(11)g(0.08)p 1188 1191 V 71 w(11)118 b(0.07)p 1505 1191 V 71 w(5)130 b(0.08)p 1811 1191 V 101 1193 1711 2 v 100 1249 2 57 v 126 1232 a(NONSCOMP)135 b(25)p 600 1249 V 72 w(31)95 b(0.34)p 894 1249 V 71 w(31)g(0.30)p 1188 1249 V 71 w(34)118 b(0.37)p 1505 1249 V 71 w(9)130 b(0.20)p 1811 1249 V 101 1251 1711 2 v 100 1307 2 57 v 126 1290 a(OBSTCLAE)108 b(5625)p 600 1307 V 48 w(282)49 b(277.71)p 894 1307 V 48 w(292)g(292.78)p 1188 1307 V 49 w(301)71 b(302.50)p 1505 1307 V 71 w(6)62 b(1696.45)p 1811 1307 V 101 1309 1711 2 v 100 1365 2 57 v 126 1348 a(OBSTCLAL)133 b(100)p 600 1365 V 72 w(15)95 b(0.30)p 894 1365 V 71 w(15)g(0.30)p 1188 1365 V 71 w(15)118 b(0.35)p 1505 1365 V 71 w(5)130 b(0.50)p 1811 1365 V 101 1367 1711 2 v 100 1424 2 57 v 126 1407 a(OBSTCLBL)135 b(100)p 600 1424 V 72 w(11)95 b(0.24)p 894 1424 V 71 w(11)g(0.27)p 1188 1424 V 71 w(11)118 b(0.21)p 1505 1424 V 71 w(3)130 b(0.36)p 1811 1424 V 101 1425 1711 2 v 100 1482 2 57 v 126 1465 a(OBSTCLBM)76 b(15625)p 600 1482 V 48 w(133)49 b(383.61)p 894 1482 V 48 w(151)g(431.99)p 1188 1482 V 49 w(230)71 b(650.09)p 1505 1482 V 71 w(6)62 b(2300.28)p 1811 1482 V 101 1483 1711 2 v 100 1540 2 57 v 126 1523 a(OBSTCLBU)129 b(100)p 600 1540 V 72 w(12)95 b(0.25)p 894 1540 V 71 w(12)g(0.32)p 1188 1540 V 71 w(11)118 b(0.22)p 1505 1540 V 71 w(2)130 b(0.31)p 1811 1540 V 101 1541 1711 2 v 100 1598 2 57 v 126 1581 a(P)l(ALMER1)209 b(4)p 600 1598 V 72 w(34)95 b(0.25)p 894 1598 V 71 w(34)g(0.23)p 1188 1598 V 71 w(32)118 b(0.21)p 1505 1598 V 48 w(30)130 b(0.44)p 1811 1598 V 101 1600 1711 2 v 100 1656 2 57 v 126 1639 a(P)l(ALMER2)209 b(4)p 600 1656 V 65 w(F1)95 b(2.14)p 894 1656 V 64 w(F1)g(1.77)p 1188 1656 V 71 w(31)118 b(0.19)p 1505 1656 V 48 w(21)130 b(0.28)p 1811 1656 V 101 1658 1711 2 v 100 1714 2 57 v 126 1697 a(P)l(ALMER3)209 b(4)p 600 1714 V 65 w(F2)95 b(0.59)p 894 1714 V 64 w(F2)g(0.66)p 1188 1714 V 71 w(34)118 b(0.25)p 1505 1714 V 48 w(38)130 b(0.48)p 1811 1714 V 101 1716 1711 2 v 100 1772 2 57 v 126 1755 a(P)l(ALMER4)209 b(4)p 600 1772 V 72 w(25)95 b(0.15)p 894 1772 V 71 w(25)g(0.14)p 1188 1772 V 71 w(30)118 b(0.19)p 1505 1772 V 48 w(41)130 b(0.55)p 1811 1772 V 101 1774 1711 2 v 100 1830 2 57 v 126 1813 a(PR)o(OBPENL)d(500)p 600 1830 V 95 w(3)95 b(0.37)p 894 1830 V 94 w(3)g(0.39)p 1188 1830 V 94 w(3)118 b(0.39)p 1505 1830 V 71 w(2)130 b(1.59)p 1811 1830 V 101 1832 1711 2 v 100 1888 2 57 v 126 1872 a(PSPDOC)237 b(4)p 600 1888 V 72 w(11)95 b(0.07)p 894 1888 V 71 w(11)g(0.09)p 1188 1888 V 71 w(11)118 b(0.08)p 1505 1888 V 71 w(7)130 b(0.10)p 1811 1888 V 101 1890 1711 2 v 100 1947 2 57 v 126 1930 a(S368)333 b(8)p 600 1947 V 72 w(11)95 b(0.12)p 894 1947 V 71 w(11)g(0.11)p 1188 1947 V 71 w(12)118 b(0.16)p 1505 1947 V 71 w(5)130 b(0.13)p 1811 1947 V 101 1948 1711 2 v 100 2005 2 57 v 126 1988 a(TORSION1)147 b(100)p 600 2005 V 72 w(12)95 b(0.22)p 894 2005 V 71 w(12)g(0.21)p 1188 2005 V 71 w(12)118 b(0.20)p 1505 2005 V 71 w(4)130 b(0.31)p 1811 2005 V 101 2006 1711 2 v 100 2063 2 57 v 126 2046 a(TORSION2)147 b(100)p 600 2063 V 72 w(11)95 b(0.21)p 894 2063 V 71 w(11)g(0.26)p 1188 2063 V 71 w(11)118 b(0.21)p 1505 2063 V 71 w(4)130 b(0.35)p 1811 2063 V 101 2064 1711 2 v 100 2121 2 57 v 126 2104 a(TORSION3)147 b(100)p 600 2121 V 95 w(5)95 b(0.10)p 894 2121 V 94 w(5)g(0.11)p 1188 2121 V 94 w(5)118 b(0.11)p 1505 2121 V 71 w(2)130 b(0.29)p 1811 2121 V 101 2123 1711 2 v 100 2179 2 57 v 126 2162 a(TORSION4)147 b(100)p 600 2179 V 95 w(7)95 b(0.15)p 894 2179 V 94 w(7)g(0.19)p 1188 2179 V 94 w(7)118 b(0.15)p 1505 2179 V 71 w(3)130 b(0.31)p 1811 2179 V 101 2181 1711 2 v 100 2237 2 57 v 126 2220 a(TORSION6)102 b(14884)p 600 2237 V 48 w(301)49 b(835.80)p 894 2237 V 48 w(318)g(991.12)p 1188 2237 V 49 w(364)71 b(841.01)p 1505 2237 V 71 w(9)85 b(150.15)p 1811 2237 V 101 2239 1711 2 v 0 2322 a(T)l(able)16 b(3.)j(Results)d(of)e(new) h(limited)i(memory)d(metho)q(d)h(using)g(three)g(metho)q(ds)g(for)f(subspace) h(minimization)141 2378 y(\(primal,)g(dual)h(and)g(cg\),)e(for)g Fg(m)f Fm(=)g(5,)i(and)g(results)g(of)g(LANCELOT)h(using)g(exact)f(Hessian.) 934 2603 y(21)p eop %%Page: 22 23 bop 71 120 a Fm(The)15 b(di\013erences)h(in)g(the)f(n)o(um)o(b)q(er)h(of)e (function)i(ev)m(aluations)g(required)g(b)o(y)f(the)h(direct)f(primal)h(and)g (dual)0 177 y(metho)q(ds)22 b(are)f(due)i(to)e(rounding)h(errors,)g(and)g (are)g(relativ)o(ely)g(small.)41 b(Their)22 b(computing)g(time)g(is)g(also)0 233 y(quite)16 b(similar)h(due)f(to)f(use)h(of)f(the)h(form)f(describ)q(ed)i (at)e(the)h(end)g(of)f Ff(x)p Fm(5.3.)20 b(Our)c(computational)g(exp)q (erience)0 289 y(suggests)c(to)h(us)g(that)f(the)i(conjugate)e(gradien)o(t)h (metho)q(d)h(for)e(subspace)i(minimization)h(is)f(the)f(least)g(e\013ectiv)o (e)0 346 y(approac)o(h;)20 b(it)f(tends)h(to)e(tak)o(e)g(more)h(time)g(and)g (function)h(ev)m(aluations.)32 b(Ev)o(en)19 b(though)g(T)l(able)h(3)e(app)q (ears)0 402 y(to)e(indicate)i(that)e(the)g(cg)h(option)f(results)h(in)h(few)o (er)e(failures,)h(tests)f(with)h(di\013eren)o(t)g(v)m(alues)g(of)f Fg(m)h Fm(resulted,)0 459 y(o)o(v)o(erall,)c(in)f(three)h(more)e(failures)j (for)d(the)h(cg)g(metho)q(d)h(than)f(for)f(the)h(primal)h(metho)q(d.)20 b(The)12 b(limited)i(memory)0 515 y(metho)q(d)h(is)g(sometimes)f(unable)i(to) e(lo)q(cate)h(the)g(solution)g(accurately)l(,)g(and)g(this)g(can)f(result)h (in)h(an)e(excessiv)o(e)0 572 y(n)o(um)o(b)q(er)19 b(of)f(function)h(ev)m (aluations)g(\(F1\))e(or)h(failure)h(to)f(mak)o(e)g(further)g(progress)g (\(F2\).)28 b(The)18 b(reasons)g(for)0 628 y(this)e(are)f(not)f(clear)i(to)f (us,)g(but)g(are)g(b)q(eing)h(in)o(v)o(estigated.)71 685 y(The)h(tests)g (describ)q(ed)j(here)e(are)f(not)h(in)o(tended)h(to)e(establish)i(the)e(sup)q (eriorit)o(y)i(of)e(LANCELOT)i(or)e(of)0 741 y(the)c(new)f(limited)i(memory)e (algorithm,)h(since)g(these)g(metho)q(ds)g(are)f(designed)i(for)d(solving)j (di\013eren)o(t)e(t)o(yp)q(es)h(of)0 798 y(problems.)20 b(LANCELOT)15 b(is)g(tailored)f(for)g(sparse)f(or)h(partially)h(separable)g(problems)f (whereas)g(the)g(limited)0 854 y(memory)d(metho)q(d)h(is)g(w)o(ell)h(suited)f (for)f(unstructured)i(or)e(dense)h(problems.)19 b(W)l(e)12 b(use)g(LANCELOT)h(simply)g(as)0 910 y(a)e(b)q(enc)o(hmark,)i(and)e(for)g (this)h(reason)f(ran)g(it)h(only)g(with)g(its)g(default)g(settings)f(and)h (did)g(not)g(exp)q(erimen)o(ts)g(with)0 967 y(its)k(v)m(arious)g(options)g (to)f(\014nd)h(the)g(one)g(that)f(w)o(ould)h(giv)o(e)g(the)g(b)q(est)g (results)g(on)g(these)g(problems.)22 b(Ho)o(w)o(ev)o(er,)0 1023 y(a)15 b(few)g(observ)m(ations)g(on)h(the)f(t)o(w)o(o)f(metho)q(ds)h (can)g(b)q(e)h(made.)71 1080 y(The)11 b(results)g(in)h(T)l(able)g(1)f (indicate)h(that)f(the)g(limited)i(memory)d(metho)q(d)i(usually)g(requires)g (more)f(function)0 1136 y(ev)m(aluations)19 b(than)e(the)h(SR1)g(option)g(of) g(Lancelot.)28 b(\(The)17 b(BF)o(GS)h(option)g(of)f(Lancelot)h(is)h(clearly)f (inferior)0 1193 y(to)f(the)h(SR1)g(option\).)28 b(Ho)o(w)o(ev)o(er)17 b(in)h(terms)g(of)f(computing)h(time)g(the)g(limited)i(memory)d(metho)q(d)h (is)h(often)0 1249 y(the)d(most)g(e\016cien)o(t;)h(this)f(can)h(b)q(e)g(seen) f(b)o(y)g(examining)i(the)e(problems)h(with)g(at)e(least)h(50)g(or)g(100)f(v) m(ariables)0 1306 y(\(whic)o(h)h(are)e(the)h(only)h(ones)f(for)f(whic)o(h)i (times)g(are)e(meaningful\).)21 b(T)l(o)15 b(our)g(surprise)g(w)o(e)g(observ) o(e)g(in)h(T)l(able)g(3)0 1362 y(that)e(ev)o(en)i(when)f(using)h(exact)f (Hessians,)g(LANCELOT)h(requires)g(more)f(time)g(than)g(the)g(limited)i (memory)0 1419 y(metho)q(d)e(on)f(man)o(y)h(of)f(the)h(large)f(problems.)21 b(This)15 b(is)g(in)g(spite)h(of)e(the)h(fact)f(that)g(the)g(ob)s(jectiv)o(e) h(function)h(in)0 1475 y(all)i(these)g(problems)g(has)f(a)g(signi\014can)o(t) i(degree)e(of)g(the)h(kind)g(of)f(partial)h(separabilit)o(y)h(that)d (LANCELOT)0 1531 y(is)j(designed)h(to)d(exploit.)31 b(On)18 b(the)h(other)f(hand,)h(LANCELOT)g(using)g(exact)f(Hessians)h(requires)g(a)f (m)o(uc)o(h)0 1588 y(smaller)e(n)o(um)o(b)q(er)f(of)g(function)h(ev)m (aluations)h(than)e(the)g(limited)i(memory)e(metho)q(d.)71 1644 y(T)l(aking)g(ev)o(erything)h(together,)e(the)h(new)h(algorithm)f (\(L-BF)o(GS-B\))g(has)g(most)g(of)f(the)i(e\016ciency)g(of)f(the)0 1701 y(unconstrained)g(limited)h(memory)e(algorithm)g(\(L-BF)o(GS\))f([18)o (])h(together)f(with)h(the)h(capabilit)o(y)g(of)f(handling)0 1757 y(b)q(ounds,)19 b(at)f(the)g(cost)f(of)h(a)g(signi\014can)o(tly)h(more)f (complex)h(co)q(de.)29 b(Lik)o(e)18 b(the)h(unconstrained)g(metho)q(d,)f(the) 0 1814 y(b)q(ound)d(limited)g(memory)e(algorithm's)h(main)g(adv)m(an)o(tages) f(are)g(its)h(lo)o(w)g(computational)g(cost)f(p)q(er)h(iteration,)0 1870 y(its)i(mo)q(dest)f(storage)f(requiremen)o(ts,)h(and)h(its)g(abilit)o(y) g(to)f(solv)o(e)g(problems)h(in)h(whic)o(h)f(the)f(Hessian)h(matrices)0 1927 y(are)i(large,)h(unstructured,)h(dense,)f(and)g(una)o(v)m(ailable.)32 b(It)18 b(is)h(less)g(lik)o(ely)i(to)c(b)q(e)j(comp)q(etitiv)o(e,)g(in)f (terms)f(of)0 1983 y(function)h(ev)m(aluations,)g(when)g(an)f(exact)g (Hessian)g(is)h(a)o(v)m(ailable,)h(or)e(when)g(signi\014can)o(t)h(adv)m(an)o (tage)f(can)g(b)q(e)0 2040 y(tak)o(en)c(of)f(structure.)20 b(Ho)o(w)o(ev)o(er)13 b(this)i(study)f(app)q(ears)g(to)g(indicate)h(that,)f (in)h(terms)e(of)h(computing)h(time,)f(the)0 2096 y(new)e(limited)i(memory)d (co)q(de)h(is)g(v)o(ery)g(comp)q(etitiv)o(e)h(with)f(all)g(v)o(ersions)g(of)f (LANCELOT)i(on)f(man)o(y)f(problems.)71 2152 y(A)g(co)q(de)g(implemen)o(ting) i(the)e(new)g(algorithm)g(is)h(describ)q(ed)h(in)f([23)o(],)f(and)g(can)g(b)q (e)h(obtained)g(b)o(y)f(con)o(tacting)0 2209 y(the)k(authors)g(at)f(no)q (cedal@eecs.n)o(wu.edu.)934 2603 y(22)p eop %%Page: 23 24 bop 0 120 a Fh(7.)18 b(*)71 198 y Fm(References)23 284 y([1])j(A.)c(V.)f (Aho,)h(J.)g(E.)g(Hop)q(croft)f(and)h(J.)g(D.)f(Ullman,)i Fl(The)f(Design)g (and)h(A)o(nalysis)e(of)i(Computer)h(A)o(lgo-)93 341 y(rithms.)d Fm(Reading,)g(Mass:)j(Addison-W)l(esley)e(Pub.)e(Co.,)f(1974.)23 431 y([2])21 b(B.)13 b(M.)f(Av)o(eric)o(k)h(and)g(J.)g(J.)f(Mor)o(\023)-21 b(e,)11 b(\\User)i(guide)h(for)e(the)g(MINP)l(A)o(CK-2)h(test)g(problem)g (collection,")h(Ar-)93 488 y(gonne)d(National)f(Lab)q(oratory)l(,)h (Mathematics)e(and)i(Computer)f(Science)i(Division)g(Rep)q(ort)e(ANL/MCS-)93 544 y(TM-157,)k(\(Argonne,)h(IL,)g(1991\).)23 635 y([3])21 b(D.P)l(.)g(Bertsek)m(as,)h(\\Pro)s(jected)f(Newton)g(metho)q(ds)h(for)e (optimization)j(problems)f(with)f(simple)i(con-)93 691 y(strain)o(ts",)14 b Fl(SIAM)h(J.)h(Contr)n(ol)g(and)g(Optimization)f Fm(20)g(\(1982\):)j (221-246.)23 782 y([4])j(I.)h(Bongartz,)g(A.R.)f(Conn,)i(N.I.M.)e(Gould,)i (Ph.L.)e(T)l(oin)o(t)51 b(\(1993\).)d(\\CUTE:)21 b(constrained)h(and)93 838 y(unconstrained)d(testing)e(en)o(vironmen)o(t",)g(Researc)o(h)h(Rep)q (ort,)g(IBM)f(T.J.)g(W)l(atson)f(Researc)o(h)i(Cen)o(ter,)93 895 y(Y)l(orkto)o(wn,)c(USA.)23 985 y([5])21 b(J.)16 b(V.)f(Burk)o(e,)g(and)g (J.)h(J.)f(Mor)o(\023)-21 b(e,)13 b(\\On)j(the)f(iden)o(ti\014cation)i(of)e (activ)o(e)h(constrain)o(ts",)e Fl(SIAM)h(J.)h(Numer.)93 1042 y(A)o(nal.)f Fm(V)l(ol.)g(25,)f(No)h(5)g(\(1988\):)j(1197-1211.)23 1133 y([6])j(R.)16 b(H.)f(Byrd,)g(J.)g(No)q(cedal)i(and)e(R.)h(B.)f(Sc)o (hnab)q(el,)h(\\Represen)o(tation)g(of)f(quasi-Newton)h(matrices)f(and)93 1189 y(their)h(use)f(in)h(limited)h(memory)d(metho)q(ds",)h Fl(Mathematic)n(al)h(Pr)n(o)n(gr)n(amming)e Fm(63,)g(4,)g(1994,)g(pp.)h (129-156)23 1280 y([7])21 b(P)l(.)c(H.)f(Calamai,)g(and)h(J.)g(J.)f(Mor)o (\023)-21 b(e,)15 b(\\Pro)s(jected)h(gradien)o(t)g(metho)q(ds)h(for)f (linearly)i(constrained)f(prob-)93 1336 y(lems")f Fl(Mathematic)n(al)g(Pr)n (o)n(gr)n(amming)f Fm(39)f(\(1987\):)k(93-116)23 1427 y([8])j(A.)g(R.)f (Conn,)i(N.)e(I.)h(M.)f(Gould,)i(and)f(PH.)f(L.)h(T)l(oin)o(t,)g(\\T)l (esting)g(a)f(class)h(of)g(metho)q(ds)f(for)h(solving)93 1483 y(minimization)c(problems)d(with)h(simple)g(b)q(ounds)g(on)f(the)g(v)m (ariables",)h Fl(Mathematics)g(of)h(Computation.)93 1540 y Fm(V)l(ol.)g(50,)e(No)h(182)f(\(1988\):)k(399-430.)23 1630 y([9])j(A.)13 b(R.)h(Conn,)f(N.)g(I.)g(M.)f(Gould,)i(and)f(PH.)g(L.)g(T)l (oin)o(t,)h(\\Global)f(con)o(v)o(ergence)h(of)e(a)h(class)h(of)f(trust)f (region)93 1687 y(algorithms)19 b(for)g(optimization)h(with)f(simple)i(b)q (ounds",)f Fl(SIAM)f(J.)g(Numer.)i(A)o(nal.)p Fm(,)d(v)o(ol.)h(25)f (\(1988\):)93 1743 y(433-460.)0 1834 y([10])j(A.R.)e(Conn,)h(N.I.M.)e(Gould,) i(Ph.L.)e(T)l(oin)o(t)46 b(\(1992\).)d(\\)p Fa(LANCELOT)p Fm(:)19 b(a)f(F)o(OR)l(TRAN)i(pac)o(k)m(age)f(for)93 1890 y(large-scale)d(nonlinear)h (optimization)f(\(Release)g(A\)",)f(Num)o(b)q(er)g(17)g(in)h(Springer)g (Series)g(in)g(Computa-)93 1947 y(tional)g(Mathematics,)f(Springer-V)l (erlag,)h(New)f(Y)l(ork.)0 2037 y([11])21 b(A.R.)15 b(Conn)h(and)f(J.)g(Mor)o (\023)-21 b(e,)13 b(Priv)m(ate)j(comm)o(unication.)0 2128 y([12])21 b(J.)e(E.)e(Dennis,)j(Jr.)e(and)g(R.)g(B.)g(Sc)o(hnab)q(el,)j Fl(Numeric)n(al)e(Metho)n(ds)f(for)i(Unc)n(onstr)n(aine)n(d)d(Optimization)93 2184 y(and)g(Nonline)n(ar)e(Equations)p Fm(,)f(Englew)o(o)q(o)q(d)i (Cli\013s,)f(N.J.:)20 b(Pren)o(tice-Hall,)c(1983.)0 2275 y([13])21 b Fl(Harwel)r(l)d(Subr)n(outine)f(Libr)n(ary,)g(R)n(ele)n(ase)g(10)41 b Fm(\(1990\).)d(Adv)m(anced)18 b(Computing)f(Departmen)o(t,)e(AEA)93 2332 y(Industrial)i(T)l(ec)o(hnology)l(,)f(Harw)o(ell)f(Lab)q(oratory)l(,)g (Oxfordshire,)g(United)h(Kingdom.)0 2422 y([14])21 b(J.C.)16 b(Gilb)q(ert)g(and)g(C.)g(Lemar)o(\023)-21 b(ec)o(hal,)15 b(\\Some)g(n)o (umerical)i(exp)q(erimen)o(ts)g(with)f(v)m(ariable)h(storage)e(quasi-)93 2479 y(Newton)g(algorithms,")g Fl(Mathematic)n(al)h(Pr)n(o)n(gr)n(amming)f Fm(45)f(\(1989\))g(407{436.)934 2603 y(23)p eop %%Page: 24 25 bop 0 120 a Fm([15])21 b(P)l(.)16 b(E.)f(Gill,)j(W)d(Murra)o(y)g(and)h(M.)f (H.)h(W)l(righ)o(t,)f Fl(Pr)n(actic)n(al)i(Optimization)p Fm(,)e(London:)22 b(Academic)17 b(Press,)93 177 y(1981.)0 270 y([16])k(A.)13 b(A.)f(Goldstein,)h(\\Con)o(v)o(ex)f(programming)g(in)h(Hilb)q(ert)h(space",) f Fl(Bul)r(l.)g(A)o(mer.)h(Math.)g(So)n(c.)e Fm(70)f(\(1964\):)93 327 y(709-710.)0 421 y([17])21 b(E.)12 b(S.)f(Levitin)j(and)e(B.)f(T.)h(P)o (oly)o(ak,)f(\\Constrained)h(minimization)i(problems",)e Fl(USSR)g(Comput.)i (Math.)93 477 y(and)j(Math.)f(Phys.)f Fm(6)g(\(1966\):)j(1-50.)0 571 y([18])j(D.)11 b(C.)g(Liu)h(and)f(J.)h(No)q(cedal,)g(\\On)g(the)f (limited)i(memory)e(BF)o(GS)f(metho)q(d)i(for)e(large)i(scale)f(optimization) 93 627 y(metho)q(ds",)k Fl(Mathematic)n(al)h(Pr)n(o)n(gr)n(amming)f Fm(45)g(\(1989\):)j(503-528.)0 721 y([19])j(J.)c(J.)g(Mor)o(\023)-21 b(e)15 b(and)i(D.J.)e(Th)o(uen)o(te)i(\(1990\),)e(\\On)i(line)h(searc)o(h)f (algorithms)f(with)h(guaran)o(teed)g(su\016cien)o(t)93 778 y(decrease",)e(Mathematics)f(and)h(Computer)f(Science)j(Division)f(Preprin)o (t)f(MCS-P153-0590,)d(Argonne)93 834 y(National)k(Lab)q(oratory)e(\(Argonne,) h(IL\).)0 928 y([20])21 b(J.)13 b(J.)f(Mor)o(\023)-21 b(e)11 b(and)h(G.)g(T)l(oraldo,)g(\\Algorithms)g(for)g(b)q(ound)h(constrained)g (quadratic)g(programming)e(prob-)93 984 y(lems",)16 b Fl(Numer.)g(Math.)g Fm(55)e(\(1989\):)k(377-400.)0 1078 y([21])j(J.)16 b(No)q(cedal,)h(\\Up)q (dating)g(quasi-Newton)f(matrices)g(with)h(limited)h(storage",)c Fl(Mathematics)j(of)g(Com-)93 1135 y(putation)f Fm(35)f(\(1980\):)j(773-782.) 0 1228 y([22])j(J.M.)e(Ortega)h(and)g(W.C.)e(Rhein)o(b)q(oldt,)23 b(\\Iterativ)o(e)d(Solution)h(of)e(Nonlinear)i(Equations)f(in)h(Sev)o(eral)93 1285 y(V)l(ariables",)16 b(Academic)g(Press,)f(\(1970\))0 1379 y([23])21 b(C.)d(Zh)o(u,)g(R.H.)g(Byrd,)h(P)l(.)e(Lu)i(and)f(J.)g(No)q (cedal,)i(\\LBF)o(GS-B:)d(F)l(ortran)g(subroutines)i(for)e(large-scale)93 1435 y(b)q(ound)h(constrained)g(optimization",)f(Rep)q(ort)g(NAM-11,)g(EECS)g (Departmen)o(t,)f(North)o(w)o(estern)f(Uni-)93 1492 y(v)o(ersit)o(y)l(,)g (1994.)934 2603 y(24)p eop %%Trailer end userdict /end-hook known{end-hook}if %%EOF SHAR_EOF fi # end of overwriting check if test -f 'compact.ps' then echo shar: will not over-write existing file "'compact.ps'" else cat << \SHAR_EOF > 'compact.ps' %!PS-Adobe-2.0 %%Creator: dvips 5.47 Copyright 1986-91 Radical Eye Software %%Title: paper.dvi %%Pages: 31 1 %%BoundingBox: 0 0 612 792 %%EndComments %%BeginProcSet: tex.pro /TeXDict 200 dict def TeXDict begin /N /def load def /B{bind def}N /S /exch load def /X{S N}B /TR /translate load N /isls false N /vsize 10 N /@rigin{ isls{[0 1 -1 0 0 0]concat}if 72 Resolution div 72 VResolution div neg scale Resolution VResolution vsize neg mul TR matrix currentmatrix dup dup 4 get round 4 exch put dup dup 5 get round 5 exch put setmatrix}N /@letter{/vsize 10 N}B /@landscape{/isls true N /vsize -1 N}B /@a4{/vsize 10.6929133858 N}B /@a3{ /vsize 15.5531 N}B /@ledger{/vsize 16 N}B /@legal{/vsize 13 N}B /@manualfeed{ statusdict /manualfeed true put}B /@copies{/#copies X}B /FMat[1 0 0 -1 0 0]N /FBB[0 0 0 0]N /nn 0 N /IE 0 N /ctr 0 N /df-tail{/nn 8 dict N nn begin /FontType 3 N /FontMatrix fntrx N /FontBBox FBB N string /base X array /BitMaps X /BuildChar{CharBuilder}N /Encoding IE N end dup{/foo setfont}2 array copy cvx N load 0 nn put /ctr 0 N[}B /df{/sf 1 N /fntrx FMat N df-tail} B /dfs{div /sf X /fntrx[sf 0 0 sf neg 0 0]N df-tail}B /E{pop nn dup definefont setfont}B /ch-width{ch-data dup length 5 sub get}B /ch-height{ch-data dup length 4 sub get}B /ch-xoff{128 ch-data dup length 3 sub get sub}B /ch-yoff{ ch-data dup length 2 sub get 127 sub}B /ch-dx{ch-data dup length 1 sub get}B /ch-image{ch-data dup type /stringtype ne{ctr get /ctr ctr 1 add N}if}B /id 0 N /rw 0 N /rc 0 N /gp 0 N /cp 0 N /G 0 N /sf 0 N /CharBuilder{save 3 1 roll S dup /base get 2 index get S /BitMaps get S get /ch-data X pop /ctr 0 N ch-dx 0 ch-xoff ch-yoff ch-height sub ch-xoff ch-width add ch-yoff setcachedevice ch-width ch-height true[1 0 0 -1 -.1 ch-xoff sub ch-yoff .1 add]{ch-image} imagemask restore}B /D{/cc X dup type /stringtype ne{]}if nn /base get cc ctr put nn /BitMaps get S ctr S sf 1 ne{dup dup length 1 sub dup 2 index S get sf div put}if put /ctr ctr 1 add N}B /I{cc 1 add D}B /bop{userdict /bop-hook known{bop-hook}if /SI save N @rigin 0 0 moveto}N /eop{clear SI restore showpage userdict /eop-hook known{eop-hook}if}N /@start{userdict /start-hook known{start-hook}if /VResolution X /Resolution X 1000 div /DVImag X /IE 256 array N 0 1 255{IE S 1 string dup 0 3 index put cvn put}for}N /p /show load N /RMat[1 0 0 -1 0 0]N /BDot 260 string N /rulex 0 N /ruley 0 N /v{/ruley X /rulex X V}B /V statusdict begin /product where{pop product dup length 7 ge{0 7 getinterval(Display)eq}{pop false}ifelse}{false}ifelse end{{gsave TR -.1 -.1 TR 1 1 scale rulex ruley false RMat{BDot}imagemask grestore}}{{gsave TR -.1 -.1 TR rulex ruley scale 1 1 false RMat{BDot}imagemask grestore}}ifelse B /a{ moveto}B /delta 0 N /tail{dup /delta X 0 rmoveto}B /M{S p delta add tail}B /b{ S p tail}B /c{-4 M}B /d{-3 M}B /e{-2 M}B /f{-1 M}B /g{0 M}B /h{1 M}B /i{2 M}B /j{3 M}B /k{4 M}B /w{0 rmoveto}B /l{p -4 w}B /m{p -3 w}B /n{p -2 w}B /o{p -1 w }B /q{p 1 w}B /r{p 2 w}B /s{p 3 w}B /t{p 4 w}B /x{0 S rmoveto}B /y{3 2 roll p a}B /bos{/SS save N}B /eos{clear SS restore}B end %%EndProcSet TeXDict begin 1000 300 300 @start /Fa 1 51 df<7FFFFF80FFFFFF80C0000180C0000180 C0000180C0000180C0000180C0000180C0000180C0000180C0000180C0000180C0000180C00001 80C0000180C0000180C0000180C0000180C0000180C0000180C0000180C0000180C0000180FFFF FF807FFFFF8019197C9B22>50 D E /Fb 1 108 df<3C000C000C001800180018001870319032 30340038007F00618061906190C1A0C0C00C117E9010>107 D E /Fc 1 1 df0 D E /Fd 14 106 df<0006000C001800300070006000C001 C0018003800300070006000E000C001C001C001800380038003800300070007000700070007000 7000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E00070 0070007000700070007000300038003800380018001C001C000C000E0006000700030003800180 01C000C00060007000300018000C00060F4A788119>16 DI34 DI<00001C00003C0000F80001E0 0003C0000780000F00000E00001E00003C00003C00003C00007800007800007800007800007800 007800007800007800007800007800007800007800007800007800007800007800007800007800 007800007800007800007800007800007800007800007800007800007800007800007800007800 00780000780000780000780000780000780000780000F00000F00000F00001E00001E00003C000 0380000700000E00001C0000780000E00000E000007800001C00000E000007000003800003C000 01E00001E00000F00000F00000F000007800007800007800007800007800007800007800007800 007800007800007800007800007800007800007800007800007800007800007800007800007800 007800007800007800007800007800007800007800007800007800007800007800007800007800 007800007800007800007800003C00003C00003C00001E00000E00000F000007800003C00001E0 0000F800003C00001C167C7B8121>40 D50 DII<001C001C001C001C001C001C001C001C001C001C001C001C 001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C00 1C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C 001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C00 1CFFFCFFFCFFFC0E4A80811C>III88 D104 DI E /Fe 14 108 df0 D<70F8F8F87005057C8D0D>I< 400004C0000C6000183000301800600C00C006018003030001860000CC00007800003000003000 00780000CC000186000303000601800C00C0180060300030600018C0000C40000416187A9623> I17 D<000000C0000003C000000F0000003C000000F0000003C00000070000 001C00000078000001E00000078000001E00000078000000E0000000780000001E000000078000 0001E0000000780000001C0000000700000003C0000000F00000003C0000000F00000003C00000 00C0000000000000000000000000000000000000000000000000000000007FFFFF80FFFFFFC01A 247C9C23>20 DI<0000000400000000020000000002000000000100000000008000000000400000000020FFFF FFFFFCFFFFFFFFFC00000000200000000040000000008000000001000000000200000000020000 0000040026107D922D>33 D<003FF800FFF803C0000700000C0000180000300000300000600000 600000C00000C00000C00000FFFFF8FFFFF8C00000C00000C00000600000600000300000300000 1800000C000007000003C00000FFF8003FF8151C7C981E>50 D<00000C00000C00001800001800 00300000300000600000600000C00000C000018000018000018000030000030000060000060000 0C00000C0000180000180000300000300000600000600000C00000C00001800001800003000003 00000600000600000600000C00000C0000180000180000300000300000600000600000C0000040 0000162C7AA000>54 D<400001C0000360000660000660000630000C30000C30000C1800181800 181800180FFFF00FFFF00C00300600600600600600600300C00300C001818001818001818000C3 0000C30000C300006600006600006600003C00003C00003C000018000018001821809F19>56 D<00001E000F00FF0011810F0020C60F8060E40780407807C0407003E0407801E0207801F03078 00F0187800F80C780078047C003C067C0038027C0060027C0180027FFE00427C3C00467C3C0044 7C3C00447C3C0038783C0000783C0000783C0000783C0000783C0000783C0040781E00E0703C00 20703C0020601E0810E01E3008C00FC0078003001E227EA021>60 D<000F0038006000E001C001 C001C001C001C001C001C001C001C001C001C001C001C001C001C0038007001E00F8001E000700 038001C001C001C001C001C001C001C001C001C001C001C001C001C001C001C000E00060003800 0F102D7DA117>102 DI<4020C030C030C030C030 C030C030C030C030C030C030C030C030C030C030C030C030C030C030C030C030C030C030C030C0 30C030C030C030C030C030C030C030C030C030C030C030C030C030C030C030C030C030C030C030 C03040200C2E7BA117>107 D E /Ff 10 111 df<00F0031806080C0C0C0C180C180C180C3018 30183030302068C0678060006000C000C000C00080000E147E8D12>26 D<60F0F0701010202040 40040A7D830A>59 D<0008001800300030003000600060006000C000C000C00180018001800300 03000600060006000C000C000C00180018001800300030003000600060006000C000C0000D217E 9812>61 D<0000C00000C00001C00001C00003C00005C00005E00008E00008E00010E00020E000 20E00040E000C0E00080E001FFF0010070020070040070040070080070180070FE03FE17177F96 1A>65 D<1FFFFE381C0E201C04601C04401C044038048038040038000038000070000070000070 0000700000E00000E00000E00000E00001C00001C00001C00001C00003C0007FFC0017177F9615 >84 D<0300038003000000000000000000000000001C002400460046008C000C00180018001800 31003100320032001C0009177F960C>105 D<00180038001000000000000000000000000001C0 022004300430086000600060006000C000C000C000C001800180018001806300E300C60078000D 1D80960E>I<1F0006000600060006000C000C000C000C00181C1866188E190C32003C003F0031 8060C060C460C460C8C0C8C0700F177E9612>I<383C1E0044C6630047028100460301008E0703 000C0603000C0603000C060600180C0600180C0620180C0C20180C0C4030180440301807801B0E 7F8D1F>109 D<383C0044C6004702004602008E06000C06000C06000C0C00180C00180C401818 40181880300880300F00120E7F8D15>I E /Fg 55 122 df<007C0001C3000701810E01C11E00 C11C00E23C00E27800E27800E47800E4F000E8F000F0F000F0F000E0F000E07000E07003E03004 6118383207C01C18147E931D>11 D<0000F8000306000406000803001003002003004007004007 00800700800601000E01000C0107F80104700207D802001C02001C02001E04001E04001E04001E 04001E08003C08003C08003C0800781800701400F01400E01201C0218700207C00200000200000 40000040000040000040000080000080000080000018297F9F1A>I<03E0040FF0081FF8083FF8 10301C10400C10C004208004200002400002400002800002800002800003000003000003000002 00000200000200000600000600000600000C00000C00000C00000C000018000018000018000010 00161E7F9318>I<001E0000610000C08001808001800003800003000003800003800003C00001 F00000F800007C0001FC00070E000E0E001E06001C06003C0600780600780600780600F00400F0 0400F00C00F00800F008007018007010003020001840000F800011207E9F14>I<001E00006300 00C38001C1800381800301C00701C00F01C00E01C01E03C01C03C03C03C03C03C03C03C0780780 7FFF807FFF80780780F00F00F00F00F00F00F00E00F01E00E01C00E03C00E03800E03000607000 60E00070C0003180001E000012207E9F15>18 D<07000001C00000E00000E00000F00000700000 7000007800003800003800003C00001C00001C00001E00000E00000E00000F00000700000F0000 378000638000C38001C3C00381C00701C00E01E01C00E03800E07000F0F00070E00070C0003815 207D9F1B>21 D<000F800038C000606000C07001C0700380780380780700780700780700780E00 F00E00F00E00F00E01E01C01C01C01C01E03801E0700390C0038F0003800003800007000007000 00700000700000E00000E00000E00000C00000151E7F9318>26 D<007FFE01FFFE07FFFE0F0780 1E03801C01C03801C07001C07001C07001C0E00380E00380E00380E00700E00700E00E00600C00 3018001860000F800017147E931A>I<70F8F8F87005057C840D>58 D<70F8FCFC740404040808 10102040060E7C840D>I<000100030003000600060006000C000C000C00180018001800300030 003000600060006000C000C000C00180018001800300030003000600060006000C000C000C0018 0018001800300030003000600060006000C000C000C000102D7DA117>61 DI<0000020000 00060000000E0000000E0000001E0000001F0000002F0000002F0000004F0000008F0000008F00 00010F0000010F0000020F0000040F0000040F0000080F80000807800010078000200780002007 80007FFF800040078000800780018007800100078002000780020007C0040003C00C0003C01E00 07C0FF807FFC1E207E9F22>65 D<00FFFFE0000F0078000F003C000F001C000F001E001E001E00 1E001E001E001E001E001E003C003C003C003C003C0078003C00F0007803C0007FFF80007803C0 007801E000F000F000F000F000F000F000F0007001E000F001E000F001E000F001E000E003C001 E003C003C003C0038003C00F0007801E00FFFFF0001F1F7E9E22>I<0000FE0200078186001C00 4C0038003C0060003C00C0001C01C0001803800018070000180F0000181E0000101E0000103C00 00003C00000078000000780000007800000078000000F0000000F0000000F0000000F0000000F0 0000807000008070000080700001003800010038000200180004000C001800060020000381C000 00FE00001F217E9F21>I<00FFFFE000000F007800000F001C00000F000E00000F000700001E00 0700001E000380001E000380001E000380003C000380003C000380003C000380003C0003800078 00078000780007800078000780007800078000F0000F0000F0000F0000F0000E0000F0001E0001 E0001C0001E0003C0001E000380001E000700003C000E00003C001C00003C003800003C0070000 07803C0000FFFFF00000211F7E9E26>I<00FFFFFF000F000E000F0006000F0002000F0002001E 0002001E0002001E0002001E0002003C0004003C0400003C0400003C0400007808000078180000 7FF8000078180000F0100000F0100000F0100000F0100001E0000001E0000001E0000001E00000 03C0000003C0000003C0000003C0000007C00000FFFE0000201F7E9E1D>70 D<00FFF9FFF0000F801F00000F001E00000F001E00000F001E00001E003C00001E003C00001E00 3C00001E003C00003C007800003C007800003C007800003C007800007800F000007FFFF0000078 00F000007800F00000F001E00000F001E00000F001E00000F001E00001E003C00001E003C00001 E003C00001E003C00003C007800003C007800003C007800003C007800007C00F8000FFF8FFF800 241F7E9E26>72 D<00FFFC000F80000F00000F00000F00001E00001E00001E00001E00003C0000 3C00003C00003C0000780000780000780000780000F00000F00000F00000F00001E00001E00001 E00001E00003C00003C00003C00003C00007C000FFFC00161F7F9E14>I<001FFF0000F80000F0 0000F00000F00001E00001E00001E00001E00003C00003C00003C00003C0000780000780000780 000780000F00000F00000F00000F00001E00001E00301E00781E00F83C00F83C00F07800807000 40E00021C0001F000018207D9E19>I<00FFFC00000F8000000F0000000F0000000F0000001E00 00001E0000001E0000001E0000003C0000003C0000003C0000003C000000780000007800000078 00000078000000F0000000F0000000F0000000F0004001E0008001E0008001E0018001E0010003 C0030003C0030003C0060003C00E0007803C00FFFFFC001A1F7E9E1F>76 D<00FF00001FF0000F00003F00000B80003E00000B80005E00000B80005E0000138000BC000013 80013C00001380013C00001380023C000023800278000023800478000023800878000021C00878 000041C010F0000041C020F0000041C020F0000041C040F0000081C041E0000081C081E0000081 C101E0000081C101E0000100E203C0000100E203C0000100E403C0000100E803C0000200E80780 000200F00780000200F00780000600E00780000F00C00F8000FFE0C1FFF8002C1F7E9E2C>I<00 FF803FF0000F800780000F800200000BC00200000BC002000013C004000011E004000011E00400 0011E004000020F008000020F008000020F808000020780800004078100000403C100000403C10 0000403C100000801E200000801E200000801E200000800F200001000F400001000F4000010007 C000010007C00002000780000200038000020003800006000380000F00010000FFE0010000241F 7E9E25>I<0001FC0000070700001C01C0003000E000E0006001C0007003800070078000380700 00380E0000381E0000381C0000383C0000383C00003878000078780000787800007878000078F0 0000F0F00000F0F00000E0F00001E0F00001C0F00003C0700003807000070078000F0038001E00 38003C001C0070000E00E0000783800001FC00001D217E9F23>I<0001FC0000070700001C01C0 003000E000E000E001C000700380007007800078070000380F0000381E0000381E0000383C0000 383C00007878000078780000787800007878000078F00000F0F00000F0F00000E0F00001E0F000 01C0F00003C070000380701C070070600F0038811E0038813C001C8170000E81E0000783808001 FD0080000101800001010000038300000386000003FE000003FC000001F8000000F0001D297E9F 24>81 D<00FFFF80000F01E0000F0070000F0038000F003C001E003C001E003C001E003C001E00 3C003C0078003C0078003C00F0003C01E00078038000780F00007FF80000781C0000F00E0000F0 0F0000F0070000F0078001E00F0001E00F0001E00F0001E00F0003C01E0003C01E0203C01E0203 C01E0407C00E04FFFC0718000003E01F207E9E23>I<0007E0800018118000300B000060070000 C0070001C0030001800200038002000380020003800200038000000380000003C0000003F80000 03FF800001FFC00000FFE000003FF0000003F0000000F000000070000000700000007000200070 0020007000200060006000E0006000C0006001C00070018000E8030000C60E000081F800001921 7D9F1C>I<7FFC1FF807C003C00780010007800100078001000F0002000F0002000F0002000F00 02001E0004001E0004001E0004001E0004003C0008003C0008003C0008003C0008007800100078 0010007800100078001000F0002000F0002000F0002000F0004000F00040007000800070010000 30020000380400000C18000007E000001D207C9E1F>85 DII89 D<007FFFF800FC00F000E001E000C003C001 8007800100078003000F0002001E0002003C00040078000000F8000000F0000001E0000003C000 00078000000F0000000F0000001E0000003C00000078008000F0008001F0010001E0010003C003 00078002000F0006001E0004003E000C003C003C007800F800FFFFF8001D1F7D9E1F>I<00F180 0389C00707800E03801C03803C0380380700780700780700780700F00E00F00E00F00E00F00E10 F01C20F01C20703C20705C40308C400F078014147E9318>97 D<07803F8007000700070007000E 000E000E000E001C001C001CF01D0C3A0E3C0E380F380F700F700F700F700FE01EE01EE01EE01C E03CE038607060E031C01F0010207E9F14>I<007C01C207010E0F1E0F1C0E3C04780078007800 F000F000F000F000F00070017002300418380FC010147E9314>I<0000780003F8000070000070 0000700000700000E00000E00000E00000E00001C00001C000F1C00389C00707800E03801C0380 3C0380380700780700780700780700F00E00F00E00F00E00F00E10F01C20F01C20703C20705C40 308C400F078015207E9F18>I<007C01C207010E011C013C013802780C7BF07C00F000F000F000 F0007000700170023004183807C010147E9315>I<00007C0000CE00019E00039E00030C000700 000700000700000700000E00000E00000E0000FFF0000E00000E00001C00001C00001C00001C00 001C0000380000380000380000380000380000700000700000700000700000700000E00000E000 00E00000E00000C00001C000318000798000F300006200003C000017297E9F16>I<001E300071 3800E0F001C0700380700780700700E00F00E00F00E00F00E01E01C01E01C01E01C01E01C01E03 801E03800E07800E0B8006170001E700000700000700000E00000E00300E00781C00F038006070 003FC000151D809316>I<01E0000FE00001C00001C00001C00001C00003800003800003800003 8000070000070000071F000761800E80C00F00C00E00E00E00E01C01C01C01C01C01C01C01C038 0380380380380380380704700708700E08700E10700610E006206003C016207E9F1A>I<00E001 E001E000C000000000000000000000000000000E00130023804380438043808700070007000E00 0E001C001C001C20384038403840388019000E000B1F7E9E10>I<0000C00001E00001E00001C0 000000000000000000000000000000000000000000001E00006300004380008380010380010380 020700000700000700000700000E00000E00000E00000E00001C00001C00001C00001C00003800 00380000380000380000700000700030700078E000F1C0006380003E00001328819E13>I<01E0 000FE00001C00001C00001C00001C0000380000380000380000380000700000700000701E00706 100E08700E10F00E20F00E40601C80001D00001E00001FC000387000383800383800381C207038 40703840703840701880E01880600F0014207E9F18>I<1E07C07C00231861860023A032030043 C03403004380380380438038038087007007000700700700070070070007007007000E00E00E00 0E00E00E000E00E00E000E00E01C101C01C01C201C01C038201C01C038401C01C0184038038018 801801800F0024147E9328>109 D<1E07802318C023A06043C0704380704380708700E00700E0 0700E00700E00E01C00E01C00E01C00E03821C03841C07041C07081C03083803101801E017147E 931B>I<03C1E004621804741C08781C08701E08701E10E01E00E01E00E01E00E01E01C03C01C0 3C01C03C01C0380380780380700380E003C1C0072380071E000700000700000E00000E00000E00 000E00001C00001C0000FFC000171D819317>112 D<00F0400388C00705800E03801C03803C03 80380700780700780700780700F00E00F00E00F00E00F00E00F01C00F01C00703C00705C0030B8 000F380000380000380000700000700000700000700000E00000E0000FFE00121D7E9314>I<1E 1E0023210023C38043C7804387804383008700000700000700000700000E00000E00000E00000E 00001C00001C00001C00001C000038000018000011147E9315>I<007C01820301060306070606 0E00078007F803FC01FE001F00077007F006F006E004400820301FC010147E9315>I<00C000E0 01C001C001C001C003800380FFF8038007000700070007000E000E000E000E001C001C001C001C 10382038203820384018800F000D1C7F9B10>I<0F00601180702180E021C0E041C0E04380E083 81C00701C00701C00701C00E03800E03800E03800E03840E07080C07080C07080E0F1006131003 E1E016147E931A>I<0F01801183C02183E021C1E041C0E0438060838040070040070040070040 0E00800E00800E00800E01000E01000C02000E04000E040006180001E00013147E9316>I<0F00 6060118070F02180E0F821C0E07841C0E0384380E0188381C0100701C0100701C0100701C0100E 0380200E0380200E0380200E0380400E0380400E0380800E078080060781000709860001F07800 1D147E9321>I<03C1C00C62201034701038F02038F020386040700000700000700000700000E0 0000E00000E00000E02061C040F1C040F1C080E2C080446300383C0014147E931A>I<0F006011 80702180E021C0E041C0E04380E08381C00701C00701C00701C00E03800E03800E03800E03800E 07000C07000C07000E0F00061E0003EE00000E00000E00001C0078180078380070700060600021 C0001F0000141D7E9316>I E /Fh 40 122 df<387CFEFEFE7C3807077C8610>46 D<00180000780001F800FFF800FFF80001F80001F80001F80001F80001F80001F80001F80001F8 0001F80001F80001F80001F80001F80001F80001F80001F80001F80001F80001F80001F80001F8 0001F80001F80001F80001F8007FFFE07FFFE013207C9F1C>49 D<03FC000FFF003C1FC07007E0 7C07F0FE03F0FE03F8FE03F8FE01F87C01F83803F80003F80003F00003F00007E00007C0000F80 001F00003E0000380000700000E01801C0180380180700180E00380FFFF01FFFF03FFFF07FFFF0 FFFFF0FFFFF015207D9F1C>I<00FE0007FFC00F07E01E03F03F03F03F81F83F81F83F81F81F03 F81F03F00003F00003E00007C0001F8001FE0001FF000007C00001F00001F80000FC0000FC3C00 FE7E00FEFF00FEFF00FEFF00FEFF00FC7E01FC7801F81E07F00FFFC001FE0017207E9F1C>I<00 00E00001E00003E00003E00007E0000FE0001FE0001FE00037E00077E000E7E001C7E00187E003 07E00707E00E07E00C07E01807E03807E07007E0E007E0FFFFFEFFFFFE0007E00007E00007E000 07E00007E00007E00007E000FFFE00FFFE17207E9F1C>I<1000201E01E01FFFC01FFF801FFF00 1FFE001FF8001BC00018000018000018000018000019FC001FFF001E0FC01807E01803E00003F0 0003F00003F80003F83803F87C03F8FE03F8FE03F8FC03F0FC03F07007E03007C01C1F800FFF00 03F80015207D9F1C>I<001F8000FFE003F07007C0F00F01F81F01F83E01F83E01F87E00F07C00 007C0000FC0800FC7FC0FCFFE0FD80F0FF00F8FE007CFE007CFC007EFC007EFC007EFC007E7C00 7E7C007E7C007E3C007C3E007C1E00F80F00F00783E003FFC000FF0017207E9F1C>I<60000078 00007FFFFE7FFFFE7FFFFC7FFFF87FFFF87FFFF0E00060E000C0C00180C00300C0030000060000 0C00001C0000180000380000780000780000F00000F00000F00001F00001F00001F00003F00003 F00003F00003F00003F00003F00003F00001E00017227DA11C>I<000070000000007000000000 F800000000F800000000F800000001FC00000001FC00000003FE00000003FE00000003FE000000 06FF000000067F0000000E7F8000000C3F8000000C3F800000183FC00000181FC00000381FE000 00300FE00000300FE00000600FF000006007F00000E007F80000FFFFF80000FFFFF800018001FC 00018001FC00038001FE00030000FE00030000FE000600007F000600007F00FFE00FFFF8FFE00F FFF825227EA12A>65 DI<0003FE0080001FFF 818000FF01E38001F8003F8003E0001F8007C0000F800F800007801F800007803F000003803F00 0003807F000001807E000001807E00000180FE00000000FE00000000FE00000000FE00000000FE 00000000FE00000000FE00000000FE000000007E000000007E000001807F000001803F00000180 3F000003801F800003000F8000030007C000060003F0000C0001F800380000FF00F000001FFFC0 000003FE000021227DA128>I69 DI<0003FE0040001FFFC0C0007F 00F1C001F8003FC003F0000FC007C00007C00FC00003C01F800003C03F000001C03F000001C07F 000000C07E000000C07E000000C0FE00000000FE00000000FE00000000FE00000000FE00000000 FE00000000FE00000000FE000FFFFC7E000FFFFC7F00001FC07F00001FC03F00001FC03F00001F C01F80001FC00FC0001FC007E0001FC003F0001FC001FC003FC0007F80E7C0001FFFC3C00003FF 00C026227DA12C>I 73 D76 DII82 D<01FC0407FF8C1F03FC3C007C7C003C78001C78001CF8000CF8000CFC 000CFC0000FF0000FFE0007FFF007FFFC03FFFF01FFFF80FFFFC03FFFE003FFE0003FF00007F00 003F00003FC0001FC0001FC0001FE0001EE0001EF0003CFC003CFF00F8C7FFE080FF8018227DA1 1F>I85 D<07FC001FFF803F07C03F03E03F01 E03F01F01E01F00001F00001F0003FF003FDF01FC1F03F01F07E01F0FC01F0FC01F0FC01F0FC01 F07E02F07E0CF81FF87F07E03F18167E951B>97 D<00FF8007FFE00F83F01F03F03E03F07E03F0 7C01E07C0000FC0000FC0000FC0000FC0000FC0000FC00007C00007E00007E00003E00301F0060 0FC0E007FF8000FE0014167E9519>99 D<0001FE000001FE0000003E0000003E0000003E000000 3E0000003E0000003E0000003E0000003E0000003E0000003E0000003E0001FC3E0007FFBE000F 81FE001F007E003E003E007E003E007C003E00FC003E00FC003E00FC003E00FC003E00FC003E00 FC003E00FC003E00FC003E007C003E007C003E003E007E001E00FE000F83BE0007FF3FC001FC3F C01A237EA21F>I<00FE0007FF800F87C01E01E03E01F07C00F07C00F8FC00F8FC00F8FFFFF8FF FFF8FC0000FC0000FC00007C00007C00007E00003E00181F00300FC07003FFC000FF0015167E95 1A>I<003F8000FFC001E3E003C7E007C7E00F87E00F83C00F80000F80000F80000F80000F8000 0F8000FFFC00FFFC000F80000F80000F80000F80000F80000F80000F80000F80000F80000F8000 0F80000F80000F80000F80000F80000F80000F80000F80007FF8007FF80013237FA211>I104 D<1C003E007F007F007F 003E001C000000000000000000000000000000FF00FF001F001F001F001F001F001F001F001F00 1F001F001F001F001F001F001F001F001F001F00FFE0FFE00B247EA310>I108 DI I<00FE0007FFC00F83E01E00F03E00F87C007C7C007C7C007CFC007EFC007EFC007EFC007EFC00 7EFC007EFC007E7C007C7C007C3E00F81F01F00F83E007FFC000FE0017167E951C>II<00FE030007FF87000FC1C7001F006F003F003F007E003F007E 001F007C001F00FC001F00FC001F00FC001F00FC001F00FC001F00FC001F00FC001F007E001F00 7E001F003E003F001F007F000FC1DF0007FF9F0001FC1F0000001F0000001F0000001F0000001F 0000001F0000001F0000001F0000001F000000FFE00000FFE01B207E951E>II<0FF3003FFF00781F00600700 E00300E00300F00300FC00007FE0007FF8003FFE000FFF0001FF00000F80C00780C00380E00380 E00380F00700FC0E00EFFC00C7F00011167E9516>I<0180000180000180000180000380000380 000780000780000F80003F8000FFFF00FFFF000F80000F80000F80000F80000F80000F80000F80 000F80000F80000F80000F80000F81800F81800F81800F81800F81800F830007C30003FE0000F8 0011207F9F16>III121 D E /Fi 52 122 df<00FC000182000703 000607000E02000E00000E00000E00000E00000E0000FFFF000E07000E07000E07000E07000E07 000E07000E07000E07000E07000E07000E07000E07000E07000E07007F0FE0131A809915>12 D<007E1F8001C170400703C060060380E00E0380400E0380000E0380000E0380000E0380000E03 8000FFFFFFE00E0380E00E0380E00E0380E00E0380E00E0380E00E0380E00E0380E00E0380E00E 0380E00E0380E00E0380E00E0380E00E0380E00E0380E07F8FE3FC1E1A809920>14 D<60F0F07010101020204080040B7D830B>44 DI<60F0F06004047D83 0B>I<078018603030303060186018E01CE01CE01CE01CE01CE01CE01CE01CE01CE01CE01CE01C 6018601870383030186007800E187E9713>48 D<03000700FF0007000700070007000700070007 000700070007000700070007000700070007000700070007000700FFF00C187D9713>I<0F8010 6020304038803CC01CE01C401C003C003800380070006000C00180010002000404080410043008 3FF87FF8FFF80E187E9713>I<0F8010E02070607870382038007800700070006000C00F8000E0 00700038003C003CE03CE03CC03C4038407030E00F800E187E9713>I<00300030007000F000F0 01700370027004700C7008701070307020704070C070FFFF00700070007000700070007007FF10 187F9713>I<30183FF03FE03FC02000200020002000200027C03860203000380018001C001C40 1CE01CE01C80184038403030E00F800E187E9713>I<01E006100C1818383038300070006000E0 00E7C0E860F030F018E018E01CE01CE01C601C601C701830183030186007C00E187E9713>I<40 007FFE7FFC7FFC4008801080108020004000400080018001800100030003000300030007000700 070007000700070002000F197E9813>I<078018603030201860186018601870103C303E600F80 07C019F030F86038401CC00CC00CC00CC00C6008201018600FC00E187E9713>I<078018603030 70306018E018E018E01CE01CE01C601C603C303C185C0F9C001C00180018003870307060604021 801F000E187E9713>I<000C0000000C0000000C0000001E0000001E0000003F00000027000000 2700000043800000438000004380000081C0000081C0000081C0000100E0000100E00001FFE000 020070000200700006007800040038000400380008001C0008001C001C001E00FF00FFC01A1A7F 991D>65 DI<003F0201C0C603002E0E001E1C000E1C0006380006780002700002 700002F00000F00000F00000F00000F00000F000007000027000027800023800041C00041C0008 0E000803003001C0C0003F00171A7E991C>III I<003F020001C0C60003002E000E001E001C000E001C0006003800060078000200700002007000 0200F0000000F0000000F0000000F0000000F0000000F001FFC070000E0070000E0078000E0038 000E001C000E001C000E000E000E000300160001C06600003F82001A1A7E991E>I73 D76 D78 D<007F000001C1C000070070000E0038001C001C003C001E0038000E0078000F0070000700F000 0780F0000780F0000780F0000780F0000780F0000780F0000780F000078078000F0078000F0038 000E003C001E001C001C000E0038000700700001C1C000007F0000191A7E991E>I82 D<0FC21836200E6006C006 C002C002C002E00070007E003FE01FF807FC003E000E00070003800380038003C002C006E004D8 1887E0101A7E9915>I<7FFFFF00701C0700401C0100401C0100C01C0180801C0080801C008080 1C0080001C0000001C0000001C0000001C0000001C0000001C0000001C0000001C0000001C0000 001C0000001C0000001C0000001C0000001C0000001C0000001C0000001C000003FFE000191A7F 991C>II<3F8070C070E020700070007007F01C7030707070E070E071E071E0F1 71FB1E3C10107E8F13>97 DI<07F80C1C381C30087000E000E000E000E000E000 E0007000300438080C1807E00E107F8F11>I<007E00000E00000E00000E00000E00000E00000E 00000E00000E00000E0003CE000C3E00380E00300E00700E00E00E00E00E00E00E00E00E00E00E 00E00E00600E00700E00381E001C2E0007CFC0121A7F9915>I<07C01C3030187018600CE00CFF FCE000E000E000E0006000300438080C1807E00E107F8F11>I<01F0031807380E100E000E000E 000E000E000E00FFC00E000E000E000E000E000E000E000E000E000E000E000E000E000E007FE0 0D1A80990C>I<0FCE187330307038703870387038303018602FC02000600070003FF03FFC1FFE 600FC003C003C003C0036006381C07E010187F8F13>II<18003C003C00180000 0000000000000000000000FC001C001C001C001C001C001C001C001C001C001C001C001C001C00 1C00FF80091A80990A>I108 DII<07E01C38300C700E6006E007E007E007E007E007E0076006700E381C1C 3807E010107F8F13>II114 D<1F2060E04020C020C020F0007F003FC01FE000F080708030C030C020F0 408F800C107F8F0F>I<0400040004000C000C001C003C00FFC01C001C001C001C001C001C001C 001C001C201C201C201C201C200E4003800B177F960F>I IIIII E /Fj 2 51 df<0C003C00CC000C00 0C000C000C000C000C000C000C000C000C000C000C00FF8009107E8F0F>49 D<1F00618040C08060C0600060006000C00180030006000C00102020207FC0FFC00B107F8F0F> I E /Fk 10 62 df<0102040C1818303070606060E0E0E0E0E0E0E0E0E0E06060607030301818 0C04020108227D980E>40 D<8040203018180C0C0E060606070707070707070707070606060E0C 0C18183020408008227E980E>I<00300000300000300000300000300000300000300000300000 3000003000003000FFFFFCFFFFFC00300000300000300000300000300000300000300000300000 300000300000300016187E931B>43 D<07C018303018701C600C600CE00EE00EE00EE00EE00EE0 0EE00EE00EE00E600C600C701C30181C7007C00F157F9412>48 D<03000700FF00070007000700 070007000700070007000700070007000700070007000700070007007FF00C157E9412>I<0F80 30E040708030C038E0384038003800700070006000C00180030006000C08080810183FF07FF0FF F00D157E9412>I<0FE030306018701C701C001C00180038006007E000300018000C000E000EE0 0EE00EC00C401830300FE00F157F9412>I<20303FE03FC0240020002000200020002F8030E020 700030003800384038E038E0388030406020C01F000D157E9412>53 D<01F00608080C181C301C 70006000E000E3E0EC30F018F00CE00EE00EE00E600E600E300C3018183007C00F157F9412>I< FFFFFCFFFFFC000000000000000000000000000000000000FFFFFCFFFFFC160A7E8C1B>61 D E /Fl 61 123 df<00003FE00000E01000018038000380780003007800070030000700000007 000000070000000E0000000E0000000E000000FFFFE0000E00E0001C01C0001C01C0001C01C000 1C01C0001C03800038038000380380003803800038070000380700007007000070071000700E20 00700E2000700E2000E00E2000E0064000E0038000E0000000C0000001C0000001C00000318000 0079800000F3000000620000003C0000001D29829F1A>12 D<0E1F3F3F1D0102020404081020C0 080E779F0E>39 D<000100020004000800100020006000C0018001800300070006000E000C001C 0018003800380030007000700060006000E000E000C000C000C000C000C000C000C000C000C000 C000C000C000C0004000600060002000100010000800102E79A113>I<00100000080000040000 060000020000030000030000030000010000018000018000018000018000018000018000018000 0380000380000380000300000300000300000700000700000600000600000E00000C00000C0000 1C0000180000380000300000700000600000E00000C0000180000100000300000600000C000018 0000300000600000800000112E80A113>I<1C3C3C3C3C040408081020204080060E7D840E>44 D<7FF0FFE07FE00C037D8A10>I<70F8F8F0E005057B840E>I<000200020006000E003C00DC031C 001C0038003800380038007000700070007000E000E000E000E001C001C001C001C00380038003 8003800780FFF80F1E7B9D17>49 D<001F000061800080E00100E0020070022070042070041070 0820F00820F00820F00840E00881E00703C0000380000700000C00001800006000008000030000 0400000800401000401000802001807E030047FF0041FE0080FC00807800141F7C9D17>I<001F 800060E00080700100300200380420380420380410380420700460700380600000E00001C00003 0000FE00001C00000600000700000780000780000780300780780780780780F00F00800F00401E 00401C0040380020E0001F8000151F7C9D17>I<0000600000E00000E00000E00001C00001C000 01C0000380000380000300000700000700000600000E00000C0000180000180000300000300000 630000C700008700010700030700060E00040E00080E003F8E00607C00801FC0001C00001C0000 380000380000380000380000700000700000600013277E9D17>I<00C06000FFC001FF8001FE00 010000010000020000020000020000020000040000047800058C000606000C0700080700000780 000780000780000780000F00700F00F00F00F00E00E01E00801C0080380080300040600061C000 1F0000131F7B9D17>I<0007C0001C200030200060E000C1E00181E00380C00700000F00000E00 001E00001E78001D84003E06003E07003C07007C0780780780780780780780700F00700F00F00F 00F00E00F01E00701C00601C0070380030700010C0000F8000131F7B9D17>I<08E0100BF01017 F8201FF8603E19C0380E80200080600100400300800300000600000E00000C00001C00001C0000 380000380000700000700000F00000F00001E00001E00001E00003C00003C00003C00007C00007 8000078000030000141F799D17>I<001F000061800080C0010060030060060060060060060060 0E00C00F00800F818007C30007E40003F80001F80003FC00047E00183F00300F00200700600700 C00300C00300C00300800600800600C00C00C008004030003060001F8000131F7B9D17>I<070F 1F1F0E0000000000000000000070F8F8F0E008147B930E>58 D<00000200000006000000060000 000E0000001E0000001E0000003F0000002F0000004F0000004F0000008F0000010F0000010F00 00020F0000020F0000040F00000C0F0000080F0000100F0000100F0000200F80003FFF80004007 8000C007800080078001000780010007800200078002000780060007801E000F80FF807FF81D20 7E9F22>65 D<01FFFFC0001E00F0001E0078001E0038001E003C003C003C003C003C003C003C00 3C003C0078007800780078007800F0007801E000F0078000FFFE0000F00F8000F003C001E001C0 01E001E001E001E001E001E003C001E003C001E003C001E003C001C0078003C00780078007800F 0007801E000F007800FFFFE0001E1F7D9E20>I<0000FE0200078186001C004C0038003C006000 3C00C0001C01C0001803800018070000180F0000181E0000101E0000103C0000003C0000007800 0000780000007800000078000000F0000000F0000000F0000000F0000000F00000807000008070 000080700001003800010038000200180004000C001800060020000381C00000FE00001F217A9F 21>I<01FFFFFE001E001C001E000C001E0004001E0004003C0004003C0004003C0004003C0004 0078080800780800007808000078180000F0300000FFF00000F0300000F0300001E0200001E020 0001E0200001E0001003C0002003C0002003C0004003C000400780008007800180078001000780 07000F001F00FFFFFE001F1F7D9E1F>69 D<01FFFFFC001E0038001E0018001E0008001E000800 3C0008003C0008003C0008003C00080078001000780800007808000078080000F0100000F03000 00FFF00000F0300001E0200001E0200001E0200001E0200003C0000003C0000003C0000003C000 00078000000780000007800000078000000F800000FFF800001E1F7D9E1E>I<0000FC04000703 0C001C00980030007800E0007801C000380380003003800030070000300E0000301E0000201E00 00203C0000003C00000078000000780000007800000078000000F0000000F000FFF0F0000780F0 000780F0000F0070000F0070000F0070000F0070001E0038001E0018003E001C002E000E00CC00 0383040000FC00001E217A9F23>I<01FFF3FFE0001F003E00001E003C00001E003C00001E003C 00003C007800003C007800003C007800003C007800007800F000007800F000007800F000007800 F00000F001E00000FFFFE00000F001E00000F001E00001E003C00001E003C00001E003C00001E0 03C00003C007800003C007800003C007800003C007800007800F000007800F000007800F000007 800F00000F801F0000FFF1FFE000231F7D9E22>I<01FFF0001F00001E00001E00001E00003C00 003C00003C00003C0000780000780000780000780000F00000F00000F00000F00001E00001E000 01E00001E00003C00003C00003C00003C0000780000780000780000780000F8000FFF800141F7D 9E12>I<001FFF0000F80000F00000F00000F00001E00001E00001E00001E00003C00003C00003 C00003C0000780000780000780000780000F00000F00000F00000F00001E00001E00301E00781E 00F83C00F83C00F0780080700040E00021C0001F000018207D9E18>I<01FFF03FE0001F000F80 001E000E00001E000800001E001000003C002000003C004000003C010000003C02000000780400 0000780800000078100000007830000000F0F0000000F1F8000000F278000000F478000001E83C 000001F03C000001E03C000001E01E000003C01E000003C01E000003C00F000003C00F00000780 0F00000780078000078007800007800780000F8007C000FFF03FF800231F7D9E23>I<01FFF800 001F0000001E0000001E0000001E0000003C0000003C0000003C0000003C000000780000007800 00007800000078000000F0000000F0000000F0000000F0000001E0000001E0000001E0000001E0 008003C0010003C0010003C0030003C00200078006000780060007800C0007801C000F007800FF FFF800191F7D9E1D>I<01FE00007FC0001E0000FC00001E0000F8000017000178000017000178 0000270002F00000270004F00000270004F00000270008F00000470009E00000470011E0000047 0021E00000470021E00000870043C00000838043C00000838083C00000838083C0000103810780 000103820780000103820780000103840780000203840F00000203880F00000203900F00000203 900F00000401E01E00000401E01E00000401C01E00000C01801E00001C01803E0000FF8103FFC0 002A1F7D9E29>I<01FF007FE0001F000F00001F00040000178004000017800400002780080000 23C008000023C008000023C008000041E010000041E010000041F010000040F010000080F02000 00807820000080782000008078200001003C400001003C400001003C400001001E400002001E80 0002001E800002000F800002000F800004000F0000040007000004000700000C000700001C0002 0000FF80020000231F7D9E22>I<0001FC0000070700001C01C0003000E000E0006001C0007003 80007007800038070000380E0000381E0000381C0000383C0000383C0000387800007878000078 7800007878000078F00000F0F00000F0F00000E0F00001E0F00001C0F00003C070000380700007 0078000F0038001E0038003C001C0070000E00E0000783800001FC00001D217A9F23>I<01FFFF 80001E00E0001E0070001E0038001E003C003C003C003C003C003C003C003C003C007800780078 0078007800F0007800E000F003C000F00F0000FFFC0000F0000001E0000001E0000001E0000001 E0000003C0000003C0000003C0000003C00000078000000780000007800000078000000F800000 FFF000001E1F7D9E1F>I<01FFFF00001E03C0001E00E0001E0070001E0078003C0078003C0078 003C0078003C0078007800F0007800F0007801E0007801C000F0070000F01E0000FFF00000F038 0001E01C0001E01E0001E00E0001E00F0003C01E0003C01E0003C01E0003C01E0007803C000780 3C0807803C0807803C100F801C10FFF00C20000007C01D207D9E21>82 D<0007E040001C18C000 3005800060038000C0038001C00180018001000380010003800100038001000380000003C00000 03C0000003F8000001FF800001FFE000007FF000001FF0000001F8000000780000007800000038 000000380020003800200038002000300060007000600060006000E0007000C000E8038000C606 000081F800001A217D9F1A>I<0FFFFFF01E0780E0180780201007802020078020200F0020600F 0020400F0020400F0020801E0040001E0000001E0000001E0000003C0000003C0000003C000000 3C00000078000000780000007800000078000000F0000000F0000000F0000000F0000001E00000 01E0000001E0000001E0000003E00000FFFF00001C1F789E21>I86 D<00F1800389C00707800E03801C03803C0380380700780700780700780700F00E 00F00E00F00E00F00E20F01C40F01C40703C40705C40308C800F070013147C9317>97 D<07803F8007000700070007000E000E000E000E001C001C001CF01D0C3A0E3C0E380F380F700F 700F700F700FE01EE01EE01EE01CE03CE038607060E031C01F0010207B9F15>I<007E0001C100 0300800E07801E07801C07003C0200780000780000780000F00000F00000F00000F00000F00000 70010070020030040018380007C00011147C9315>I<0000780003F80000700000700000700000 700000E00000E00000E00000E00001C00001C000F1C00389C00707800E03801C03803C03803807 00780700780700780700F00E00F00E00F00E00F00E20F01C40F01C40703C40705C40308C800F07 0015207C9F17>I<007C01C207010E011C013C013802780C7BF07C00F000F000F000F000700070 0170023804183807C010147C9315>I<00007800019C00033C00033C000718000700000700000E 00000E00000E00000E00000E0001FFE0001C00001C00001C00001C000038000038000038000038 0000380000700000700000700000700000700000700000E00000E00000E00000E00000C00001C0 0001C0000180003180007B0000F300006600003C00001629829F0E>I<003C6000E27001C1E003 80E00700E00F00E00E01C01E01C01E01C01E01C03C03803C03803C03803C03803C07003C07001C 0F001C17000C2E0003CE00000E00000E00001C00001C00301C00783800F0700060E0003F800014 1D7E9315>I<01E0000FE00001C00001C00001C00001C000038000038000038000038000070000 070000071E000763000E81800F01C00E01C00E01C01C03801C03801C03801C0380380700380700 380700380E10700E20700C20701C20700C40E00CC060070014207D9F17>I<00C001E001E001C0 00000000000000000000000000000E003300230043804300470087000E000E000E001C001C001C 003840388030807080310033001C000B1F7C9E0E>I<0001800003C00003C00003800000000000 00000000000000000000000000000000003C00004600008700008700010700010700020E00000E 00000E00000E00001C00001C00001C00001C000038000038000038000038000070000070000070 0000700000E00000E00030E00079C000F180006300003C00001228829E0E>I<01E0000FE00001 C00001C00001C00001C0000380000380000380000380000700000700000703C00704200E08E00E 11E00E21E00E40C01C80001D00001E00001FC00038E00038700038700038384070708070708070 7080703100E03100601E0013207D9F15>I<03C01FC0038003800380038007000700070007000E 000E000E000E001C001C001C001C0038003800380038007000700070007100E200E200E200E200 640038000A207C9F0C>I<1C0F80F0002630C318004740640C004780680E004700700E00470070 0E008E00E01C000E00E01C000E00E01C000E00E01C001C01C038001C01C038001C01C038001C01 C0708038038071003803806100380380E10038038062007007006600300300380021147C9325> I<1C0F802630C04740604780604700704700708E00E00E00E00E00E00E00E01C01C01C01C01C01 C01C03843803883803083807083803107003303001C016147C931A>I<007C0001C3000301800E 01C01E01C01C01E03C01E07801E07801E07801E0F003C0F003C0F003C0F00780F00700700F0070 0E0030180018700007C00013147C9317>I<01C1E002621804741C04781C04701E04701E08E01E 00E01E00E01E00E01E01C03C01C03C01C03C01C0380380780380700380E003C1C0072380071E00 0700000700000E00000E00000E00000E00001C00001C0000FFC000171D809317>I<00F0400388 C00705800E03801C03803C0380380700780700780700780700F00E00F00E00F00E00F00E00F01C 00F01C00703C00705C0030B8000F380000380000380000700000700000700000700000E00000E0 000FFE00121D7C9315>I<1C1E002661004783804787804707804703008E00000E00000E00000E 00001C00001C00001C00001C000038000038000038000038000070000030000011147C9313>I< 00FC030206010C030C070C060C000F800FF007F803FC003E000E700EF00CF00CE008401020601F 8010147D9313>I<018001C0038003800380038007000700FFF007000E000E000E000E001C001C 001C001C003800380038003820704070407080708031001E000C1C7C9B0F>I<0E00C03300E023 01C04381C04301C04701C08703800E03800E03800E03801C07001C07001C07001C07101C0E2018 0E20180E201C1E200C264007C38014147C9318>I<0E03803307802307C04383C04301C04700C0 8700800E00800E00800E00801C01001C01001C01001C02001C02001C04001C04001C08000E3000 03C00012147C9315>I<0E00C1C03300E3C02301C3E04381C1E04301C0E04701C060870380400E 0380400E0380400E0380401C0700801C0700801C0700801C0701001C0701001C0602001C0F0200 0C0F04000E13080003E1F0001B147C931E>I<0383800CC4401068E01071E02071E02070C040E0 0000E00000E00000E00001C00001C00001C00001C040638080F38080F38100E5810084C6007878 0013147D9315>I<0E00C03300E02301C04381C04301C04701C08703800E03800E03800E03801C 07001C07001C07001C07001C0E00180E00180E001C1E000C3C0007DC00001C00001C00003800F0 3800F07000E06000C0C0004380003E0000131D7C9316>I<01C04003E08007F1800C1F00080200 0004000008000010000020000040000080000100000200000401000802001002003E0C0063FC00 41F80080E00012147D9313>I E /Fm 84 127 df<001F83E000F06E3001C078780380F8780300 F03007007000070070000700700007007000070070000700700007007000FFFFFF800700700007 007000070070000700700007007000070070000700700007007000070070000700700007007000 070070000700700007007000070070000700700007007000070070007FE3FF001D20809F1B>11 D<003F0000E0C001C0C00381E00701E00701E0070000070000070000070000070000070000FFFF E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700 E00700E00700E00700E00700E00700E07FC3FE1720809F19>I<003FE000E0E001C1E00381E007 00E00700E00700E00700E00700E00700E00700E00700E0FFFFE00700E00700E00700E00700E007 00E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E00700E007 00E07FE7FE1720809F19>I<001F81F80000F04F040001C07C06000380F80F000300F00F000700 F00F00070070000007007000000700700000070070000007007000000700700000FFFFFFFF0007 007007000700700700070070070007007007000700700700070070070007007007000700700700 070070070007007007000700700700070070070007007007000700700700070070070007007007 00070070070007007007007FE3FE3FF02420809F26>I<07070F1C383060C00808779F17>19 D22 D<7038F87CFC7EFC7E743A040204020402080408041008100820104020 0F0E7E9F17>34 D<70F8FCFC74040404080810102040060E7C9F0D>39 D<002000400080010002 0006000C000C00180018003000300030007000600060006000E000E000E000E000E000E000E000 E000E000E000E000E0006000600060007000300030003000180018000C000C0006000200010000 80004000200B2E7DA112>I<800040002000100008000C00060006000300030001800180018001 C000C000C000C000E000E000E000E000E000E000E000E000E000E000E000E000C000C000C001C0 01800180018003000300060006000C00080010002000400080000B2E7DA112>I<000600000006 000000060000000600000006000000060000000600000006000000060000000600000006000000 060000000600000006000000060000FFFFFFF0FFFFFFF000060000000600000006000000060000 000600000006000000060000000600000006000000060000000600000006000000060000000600 00000600001C207D9A23>43 D<70F8FCFC74040404080810102040060E7C840D>II<70F8F8F87005057C840D>I<000100030003000600060006000C000C000C00180018 001800300030003000600060006000C000C000C00180018001800300030003000600060006000C 000C000C00180018001800300030003000600060006000C000C000C000102D7DA117>I<03F000 0E1C001C0E00180600380700700380700380700380700380F003C0F003C0F003C0F003C0F003C0 F003C0F003C0F003C0F003C0F003C0F003C0F003C0F003C0700380700380700380780780380700 1806001C0E000E1C0003F000121F7E9D17>I<018003800F80F380038003800380038003800380 03800380038003800380038003800380038003800380038003800380038003800380038007C0FF FE0F1E7C9D17>I<03F0000C1C00100E00200700400780800780F007C0F803C0F803C0F803C020 07C00007C0000780000780000F00000E00001C0000380000700000600000C00001800003000006 00400C00401800401000803FFF807FFF80FFFF80121E7E9D17>I<03F0000C1C00100E00200F00 780F80780780780780380F80000F80000F00000F00000E00001C0000380003F000003C00000E00 000F000007800007800007C02007C0F807C0F807C0F807C0F00780400780400F00200E001C3C00 03F000121F7E9D17>I<000600000600000E00000E00001E00002E00002E00004E00008E00008E 00010E00020E00020E00040E00080E00080E00100E00200E00200E00400E00C00E00FFFFF0000E 00000E00000E00000E00000E00000E00000E0000FFE0141E7F9D17>I<1803001FFE001FFC001F F8001FE00010000010000010000010000010000010000011F000161C00180E0010070010078000 03800003800003C00003C00003C07003C0F003C0F003C0E00380400380400700200600100E000C 380003E000121F7E9D17>I<007C000182000701000E03800C07801C0780380300380000780000 700000700000F1F000F21C00F40600F80700F80380F80380F003C0F003C0F003C0F003C0F003C0 7003C07003C07003803803803807001807000C0E00061C0001F000121F7E9D17>I<4000007FFF C07FFF807FFF804001008002008002008004000008000008000010000020000020000040000040 0000C00000C00001C0000180000380000380000380000380000780000780000780000780000780 00078000078000030000121F7D9D17>I<03F0000C0C0010060030030020018060018060018060 01807001807803003E03003F06001FC8000FF00003F80007FC000C7E00103F00300F8060038040 01C0C001C0C000C0C000C0C000C0C000806001802001001002000C0C0003F000121F7E9D17>I< 03F0000E18001C0C00380600380700700700700380F00380F00380F003C0F003C0F003C0F003C0 F003C07007C07007C03807C0180BC00E13C003E3C0000380000380000380000700300700780600 780E00700C002018001070000FC000121F7E9D17>I<70F8F8F8700000000000000000000070F8 F8F87005147C930D>I<70F8F8F8700000000000000000000070F0F8F878080808101010202040 051D7C930D>I<7FFFFFE0FFFFFFF0000000000000000000000000000000000000000000000000 0000000000000000FFFFFFF07FFFFFE01C0C7D9023>61 D<000100000003800000038000000380 000007C0000007C0000007C0000009E0000009E0000009E0000010F0000010F0000010F0000020 7800002078000020780000403C0000403C0000403C0000801E0000801E0000FFFE0001000F0001 000F0001000F00020007800200078002000780040003C00E0003C01F0007E0FFC03FFE1F207F9F 22>65 DI<000FC040007030C001C009C0038005C0070003C00E 0001C01E0000C01C0000C03C0000C07C0000407C00004078000040F8000000F8000000F8000000 F8000000F8000000F8000000F8000000F8000000F8000000780000007C0000407C0000403C0000 401C0000401E0000800E000080070001000380020001C0040000703800000FC0001A217D9F21> IIII<000FE0200078186000E004E0038002E0070001E00F0000E01E0000601E0000603C0000 603C0000207C00002078000020F8000000F8000000F8000000F8000000F8000000F8000000F800 0000F8007FFCF80003E0780001E07C0001E03C0001E03C0001E01E0001E01E0001E00F0001E007 0001E0038002E000E0046000781820000FE0001E217D9F24>I II<0FFFC0007C00003C 00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C00003C 00003C00003C00003C00003C00003C00003C00003C00203C00F83C00F83C00F83C00F038004078 0040700030E0000F800012207E9E17>IIIII<001F800000 F0F00001C0380007801E000F000F000E0007001E0007803C0003C03C0003C07C0003E0780001E0 780001E0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0F80001 F0780001E07C0003E07C0003E03C0003C03C0003C01E0007800E0007000F000F0007801E0001C0 380000F0F000001F80001C217D9F23>II<001F800000F0F000 01C0380007801E000F000F000E0007001E0007803C0003C03C0003C07C0003E07C0003E0780001 E0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F0F80001F07800 01E0780001E07C0003E03C0003C03C0F03C01E1087800E2047000F204F0007A03E0001E0380000 F0F010001FB01000003010000038300000387000003FF000001FE000001FE000000FC000000780 1C297D9F23>II<07E0800C19801007803003806001 80600180E00180E00080E00080E00080F00000F000007800007F00003FF0001FFC000FFE0003FF 00001F800007800003C00003C00001C08001C08001C08001C08001C0C00180C00380E00300F006 00CE0C0081F80012217D9F19>I<7FFFFFE0780F01E0600F0060400F0020400F0020C00F003080 0F0010800F0010800F0010800F0010000F0000000F0000000F0000000F0000000F0000000F0000 000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F00 00000F0000000F0000000F0000001F800007FFFE001C1F7E9E21>II87 D<7FF83FF80FE00FC007C0070003C0020001E0040001F00C0000F008 0000781000007C1000003C2000003E4000001E4000000F8000000F8000000780000003C0000007 E0000005E0000009F0000018F8000010780000207C0000603C0000401E0000801F0001800F0001 000780020007C0070003C01F8007E0FFE01FFE1F1F7F9E22>I I91 D<080410082010201040204020804080408040B85CFC7EFC7E 7C3E381C0F0E7B9F17>II<0C001E0033006180C0C080400A067A 9E17>I<1FE000303000781800781C00300E00000E00000E00000E0000FE00078E001E0E00380E 00780E00F00E10F00E10F00E10F01E10781E103867200F83C014147E9317>97 D<0E0000FE00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E00000E3E 000EC3800F01C00F00E00E00E00E00700E00700E00780E00780E00780E00780E00780E00780E00 700E00700E00E00F00E00D01C00CC300083E0015207F9F19>I<03F80E0C1C1E381E380C700070 00F000F000F000F000F000F00070007000380138011C020E0C03F010147E9314>I<000380003F 8000038000038000038000038000038000038000038000038000038000038003E380061B801C07 80380380380380700380700380F00380F00380F00380F00380F00380F003807003807003803803 803807801C07800E1B8003E3F815207E9F19>I<03F0000E1C001C0E0038070038070070070070 0380F00380F00380FFFF80F00000F00000F000007000007000003800801800800C010007060001 F80011147F9314>I<007C00C6018F038F07060700070007000700070007000700FFF007000700 07000700070007000700070007000700070007000700070007000700070007007FF01020809F0E >I<0000E003E3300E3C301C1C30380E00780F00780F00780F00780F00780F00380E001C1C001E 380033E0002000002000003000003000003FFE001FFF800FFFC03001E0600070C00030C00030C0 0030C000306000603000C01C038003FC00141F7F9417>I<0E0000FE00000E00000E00000E0000 0E00000E00000E00000E00000E00000E00000E00000E3E000E43000E81800F01C00F01C00E01C0 0E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0 FFE7FC16207F9F19>I<1C001E003E001E001C000000000000000000000000000E007E000E000E 000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E00FFC00A1F809E0C> I<00E001F001F001F000E0000000000000000000000000007007F000F000700070007000700070 00700070007000700070007000700070007000700070007000700070007000706070F060F0C061 803F000C28829E0E>I<0E0000FE00000E00000E00000E00000E00000E00000E00000E00000E00 000E00000E00000E0FF00E03C00E03000E02000E04000E08000E10000E30000E70000EF8000F38 000E1C000E1E000E0E000E07000E07800E03800E03C00E03E0FFCFF815207F9F18>I<0E00FE00 0E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E000E 000E000E000E000E000E000E000E000E000E00FFE00B20809F0C>I<0E1F01F000FE618618000E 81C81C000F00F00E000F00F00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E00 0E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E000E00E00E 000E00E00E00FFE7FE7FE023147F9326>I<0E3E00FE43000E81800F01C00F01C00E01C00E01C0 0E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C0FFE7FC 16147F9319>I<01F800070E001C03803801C03801C07000E07000E0F000F0F000F0F000F0F000 F0F000F0F000F07000E07000E03801C03801C01C0380070E0001F80014147F9317>I<0E3E00FE C3800F01C00F00E00E00E00E00F00E00700E00780E00780E00780E00780E00780E00780E00700E 00F00E00E00F01E00F01C00EC3000E3E000E00000E00000E00000E00000E00000E00000E00000E 0000FFE000151D7F9319>I<03E0800619801C05803C0780380380780380700380F00380F00380 F00380F00380F00380F003807003807803803803803807801C0B800E138003E380000380000380 000380000380000380000380000380000380003FF8151D7E9318>I<0E78FE8C0F1E0F1E0F0C0E 000E000E000E000E000E000E000E000E000E000E000E000E000E00FFE00F147F9312>I<1F9030 704030C010C010C010E00078007F803FE00FF00070803880188018C018C018E030D0608F800D14 7E9312>I<020002000200060006000E000E003E00FFF80E000E000E000E000E000E000E000E00 0E000E000E000E080E080E080E080E080610031001E00D1C7F9B12>I<0E01C0FE1FC00E01C00E 01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E01C00E 03C00603C0030DC001F1FC16147F9319>III<7FC3FC0F01E00701C007018003810001C20000E40000EC 00007800003800003C00007C00004E000087000107000303800201C00601E01E01E0FF07FE1714 809318>II<3FFF380E200E201C40384078407000E001E0 01C00380078007010E011E011C0338027006700EFFFE10147F9314>II<1C 043F1863F080E00E047C9D17>126 D E /Fn 46 120 df45 D<387CFEFEFE7C3807077C860F>I<00E00001E0000FE000FFE000F3E00003E00003E00003E000 03E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E00003E000 03E00003E00003E00003E00003E00003E000FFFF80FFFF80111D7C9C1A>49 D<07F0001FFE00383F007C1F80FE0FC0FE0FC0FE0FE0FE07E07C07E03807E0000FE0000FC0000F C0001F80001F00003E0000780000F00000E00001C0000380600700600E00601C00E01FFFC03FFF C07FFFC0FFFFC0FFFFC0131D7D9C1A>I<01FC0007FF000E0F801E0FC03F07E03F07E03F07E03F 07E01E0FC0000FC0000F80001F0001FC0001FC00000F800007C00003E00003F00003F83803F87C 03F8FE03F8FE03F8FE03F0FC03F07807E03C0FC01FFF8003FC00151D7E9C1A>I<0001C00003C0 0007C00007C0000FC0001FC0003BC00073C00063C000C3C00183C00383C00703C00E03C00C03C0 1803C03803C07003C0E003C0FFFFFEFFFFFE0007C00007C00007C00007C00007C00007C000FFFE 00FFFE171D7F9C1A>I<3803803FFF803FFF003FFE003FFC003FF0003F80003000003000003000 0030000033F80037FE003C1F00380F801007C00007C00007E00007E07807E0FC07E0FC07E0FC07 E0FC07C0780FC0600F80381F001FFC0007F000131D7D9C1A>I<003F0001FFC007E0E00F81E01F 03F01E03F03E03F07C03F07C01E07C0000FC1000FCFF00FDFFC0FD03E0FE01F0FE01F0FC01F8FC 01F8FC01F8FC01F87C01F87C01F87C01F83C01F03E01F01E03E00F07C007FF8001FE00151D7E9C 1A>I<0000E000000000E000000001F000000001F000000001F000000003F800000003F8000000 06FC00000006FC0000000EFE0000000C7E0000000C7E000000183F000000183F000000303F8000 00301F800000701FC00000600FC00000600FC00000C007E00000FFFFE00001FFFFF000018003F0 00018003F000030001F800030001F800060001FC00060000FC000E0000FE00FFE00FFFE0FFE00F FFE0231F7E9E28>65 D<0007FC02003FFF0E00FE03DE03F000FE07E0003E0FC0001E1F80001E3F 00000E3F00000E7F0000067E0000067E000006FE000000FE000000FE000000FE000000FE000000 FE000000FE0000007E0000007E0000067F0000063F0000063F00000C1F80000C0FC0001807E000 3803F0007000FE01C0003FFF800007FC001F1F7D9E26>67 DIII72 DI76 DII<001FF80000FFFF0001F81F8007E007E00FC003F01F8001F81F0000F83F0000FC7F00 00FE7E00007E7E00007EFE00007FFE00007FFE00007FFE00007FFE00007FFE00007FFE00007FFE 00007FFE00007F7E00007E7F0000FE7F0000FE3F0000FC3F8001FC1F8001F80FC003F007E007E0 01F81F8000FFFF00001FF800201F7D9E27>II<001FF80000FF FF0001F81F8007E007E00FC003F01F8001F81F8001F83F0000FC7F0000FE7F0000FE7E00007EFE 00007FFE00007FFE00007FFE00007FFE00007FFE00007FFE00007FFE00007FFE00007F7E00007E 7E00007E7F0000FE3F0000FC3F87C1FC1F8FE1F80FD833F007F83FE001F81F8000FFFF00001FFE 0300001E0300001F0700000FFF00000FFF000007FE000007FE000003FC000001F8000000F02028 7D9E27>II<03FC080FFF381E03F83800F8700078700038F00038F00018F00018F8 0000FC00007FC0007FFE003FFF801FFFE00FFFF007FFF000FFF80007F80000FC00007C00003CC0 003CC0003CC0003CE00038E00078F80070FE01E0E7FFC081FF00161F7D9E1D>I<7FFFFFFC7FFF FFFC7C07E07C7007E01C6007E00C6007E00CE007E00EC007E006C007E006C007E006C007E00600 07E0000007E0000007E0000007E0000007E0000007E0000007E0000007E0000007E0000007E000 0007E0000007E0000007E0000007E0000007E0000007E0000007E00003FFFFC003FFFFC01F1E7E 9D24>II87 D89 D<07FC001FFF003F0F803F07C03F03E03F03E00C03E00003E0007FE007FBE01F 03E03C03E07C03E0F803E0F803E0F803E0FC05E07E0DE03FF8FE0FE07E17147F9319>97 D<01FE0007FF801F0FC03E0FC03E0FC07C0FC07C0300FC0000FC0000FC0000FC0000FC0000FC00 007C00007E00003E00603F00C01F81C007FF0001FC0013147E9317>99 D<0007F80007F80000F8 0000F80000F80000F80000F80000F80000F80000F80000F80000F801F8F80FFEF81F83F83E01F8 7E00F87C00F87C00F8FC00F8FC00F8FC00F8FC00F8FC00F8FC00F87C00F87C00F87E00F83E01F8 1F07F80FFEFF03F8FF18207E9F1D>I<01FE0007FF800F83C01E01E03E00F07C00F07C00F8FC00 F8FFFFF8FFFFF8FC0000FC0000FC00007C00007C00003E00181E00180F807007FFE000FF801514 7F9318>I<001F8000FFC001F3E003E7E003C7E007C7E007C3C007C00007C00007C00007C00007 C000FFFC00FFFC0007C00007C00007C00007C00007C00007C00007C00007C00007C00007C00007 C00007C00007C00007C00007C00007C0003FFC003FFC0013207F9F10>I<01FC3C07FFFE0F079E 1E03DE3E03E03E03E03E03E03E03E03E03E01E03C00F07800FFF0009FC001800001800001C0000 1FFF800FFFF007FFF81FFFFC3C007C70003EF0001EF0001EF0001E78003C78003C3F01F80FFFE0 01FF00171E7F931A>II<1C003E00 7F007F007F003E001C00000000000000000000000000FF00FF001F001F001F001F001F001F001F 001F001F001F001F001F001F001F001F001F00FFE0FFE00B217EA00E>I108 DII<01FF0007FFC01F83F03E00F83E00F87C007C7C007CFC007EFC007EFC007EFC007EFC00 7EFC007E7C007C7C007C3E00F83E00F81F83F007FFC001FF0017147F931A>II114 D<0FE63FFE701E600EE006E006F800FFC07FF83FFC1FFE03FE001FC007C007E007F006F81EFFFC C7F010147E9315>I<01800180018003800380038007800F803F80FFFCFFFC0F800F800F800F80 0F800F800F800F800F800F800F860F860F860F860F8607CC03F801F00F1D7F9C14>IIII E /Fo 47 123 df<003F07E00001C09C 18000380F018000701F03C000E01E03C000E00E018000E00E000000E00E000000E00E000000E00 E000000E00E00000FFFFFFFC000E00E01C000E00E01C000E00E01C000E00E01C000E00E01C000E 00E01C000E00E01C000E00E01C000E00E01C000E00E01C000E00E01C000E00E01C000E00E01C00 0E00E01C000E00E01C000E00E01C007FC7FCFF80211D809C23>14 D<60F0F86808080810102040 80050C7C9C0C>39 D<60F0F0701010101020204080040C7C830C>44 D I<60F0F06004047C830C>I66 D<001F808000E061800180198007 0007800E0003801C0003801C00018038000180780000807800008070000080F0000000F0000000 F0000000F0000000F0000000F0000000F0000000F0000000700000807800008078000080380000 801C0001001C0001000E000200070004000180080000E03000001FC000191E7E9C1E>IIII<001F808000E06180018019800700 07800E0003801C0003801C00018038000180780000807800008070000080F0000000F0000000F0 000000F0000000F0000000F0000000F000FFF0F0000F8070000780780007807800078038000780 1C0007801C0007800E00078007000B800180118000E06080001F80001C1E7E9C21>III78 D<003F800000E0E000038038000700 1C000E000E001C0007003C00078038000380780003C0780003C0700001C0F00001E0F00001E0F0 0001E0F00001E0F00001E0F00001E0F00001E0F00001E0700001C0780003C0780003C038000380 3C0007801C0007000E000E0007001C000380380000E0E000003F80001B1E7E9C20>I82 D<07E0801C1980300580700380600180E00180E00080E00080E00080F00000F800007C0000 7FC0003FF8001FFE0007FF0000FF80000F800007C00003C00001C08001C08001C08001C0C00180 C00180E00300D00200CC0C0083F800121E7E9C17>I<7FFFFFC0700F01C0600F00C0400F004040 0F0040C00F0020800F0020800F0020800F0020000F0000000F0000000F0000000F0000000F0000 000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F0000000F00 00000F0000000F0000001F800003FFFC001B1C7F9B1E>IIII89 D<1FC000307000783800781C00301C00001C00001C0001FC000F1C00381C00701C00601C00E01C 40E01C40E01C40603C40304E801F870012127E9115>97 D I<07E00C301878307870306000E000E000E000E000E000E00060007004300418080C3007C00E12 7E9112>I<003F0000070000070000070000070000070000070000070000070000070000070003 E7000C1700180F00300700700700600700E00700E00700E00700E00700E00700E0070060070070 0700300700180F000C370007C7E0131D7E9C17>I<03E00C301818300C700E6006E006FFFEE000 E000E000E00060007002300218040C1803E00F127F9112>I<00F8018C071E061E0E0C0E000E00 0E000E000E000E00FFE00E000E000E000E000E000E000E000E000E000E000E000E000E000E000E 000E007FE00F1D809C0D>I<00038003C4C00C38C01C3880181800381C00381C00381C00381C00 1818001C38000C300013C0001000003000001800001FF8001FFF001FFF803003806001C0C000C0 C000C0C000C06001803003001C0E0007F800121C7F9215>II<18003C003C0018000000000000000000000000000000FC001C001C001C001C001C001C001C 001C001C001C001C001C001C001C001C001C00FF80091D7F9C0C>I<00C001E001E000C0000000 00000000000000000000000FE000E000E000E000E000E000E000E000E000E000E000E000E000E0 00E000E000E000E000E000E000E060E0F0C0F1C061803E000B25839C0D>IIIII<03F000 0E1C00180600300300700380600180E001C0E001C0E001C0E001C0E001C0E001C0600180700380 3003001806000E1C0003F00012127F9115>II<03C1000C3300180B00300F0070 0700700700E00700E00700E00700E00700E00700E00700600700700700300F00180F000C370007 C700000700000700000700000700000700000700000700003FE0131A7E9116>II<1F90 30704030C010C010E010F8007F803FE00FF000F880388018C018C018E010D0608FC00D127F9110 >I<04000400040004000C000C001C003C00FFE01C001C001C001C001C001C001C001C001C001C 101C101C101C101C100C100E2003C00C1A7F9910>IIII121 D<7FFC70386038407040F040E041C003C0038007000F040E041C043C0C380870087038FFF80E12 7F9112>I E end %%EndProlog %%BeginSetup %%Feature: *Resolution 300 TeXDict begin %%EndSetup %%Page: 1 1 bop 551 120 a Fo(NOR)m(THWESTERN)14 b(UNIVERSITY)535 170 y(Departmen)o(t)f (of)g(Electrical)h(Engineering)670 220 y(and)g(Computer)f(Science)196 789 y Fn(REPRESENT)l(A)l(TIONS)18 b(OF)f(QUASI-NEWTON)i(MA)l(TRICES)220 846 y(AND)e(THEIR)h(USE)g(IN)f(LIMITED)h(MEMOR)l(Y)f(METHODS)850 1058 y Fm(b)o(y)285 1162 y Fl(R)o(ichar)n(d)f(H.)g(Byr)n(d,)626 1145 y Fk(1)662 1162 y Fl(Jor)n(ge)g(No)n(c)n(e)n(dal)940 1145 y Fk(2)975 1162 y Fl(and)g(R)n(ob)n(ert)g(B.)g(Schnab)n(el)1445 1145 y Fk(1)611 1573 y Fm(T)l(ec)o(hnical)h(Rep)q(ort)f(NAM-03)514 1724 y(June)g(1992;)e(revised)i(Jan)o(uary)f(21,)f(1996)p 0 2162 700 2 v 52 2189 a Fj(1)81 2205 y Fi(Computer)e(Science)h(Departmen)o(t,) g(Univ)o(ersit)o(y)h(of)d(Colorado,)i(Campus)g(Bo)o(x)f(430,)g(Boulder,)h (Colorado)g(80309.)0 2250 y(These)j(authors)h(w)o(ere)f(supp)q(orted)i(b)o(y) e(the)h(Air)f(F)m(orce)g(O\016ce)g(of)g(Scien)o(ti\014c)i(Researc)o(h)f (under)g(Gran)o(t)g(AF)o(OSR-90-)0 2296 y(0109,)i(the)f(Arm)o(y)g(Researc)o (h)g(O\016ce)g(under)g(Gran)o(t)g(D)o(AAL03-91-0151)h(and)f(the)g(National)i (Science)f(F)m(oundation)0 2342 y(under)14 b(Gran)o(ts)f(CCR-8920519)h(and)g (CCR-9101795)52 2371 y Fj(2)82 2387 y Fi(Departmen)o(t)g(of)f(Electrical)i (Engineering)h(and)e(Computer)f(Science,)i(North)o(w)o(estern)e(Univ)o(ersit) o(y)m(,)h(Ev)n(anston)h(Il)0 2433 y(60208.)i(This)10 b(author)h(w)o(as)e (supp)q(orted)j(b)o(y)e(the)g(U.S.)f(Departmen)o(t)i(of)e(Energy)m(,)i(under) g(Gran)o(t)f(DE-F)o(G02-87ER25047-)0 2479 y(A001,)j(and)h(b)o(y)f(National)i (Science)f(F)m(oundation)h(Gran)o(ts)f(CCR-9101359)f(and)h(ASC-9213149)g(.) 863 2603 y Fm(1)p eop %%Page: 1 2 bop 196 120 a Fn(REPRESENT)l(A)l(TIONS)18 b(OF)f(QUASI-NEWTON)i(MA)l(TRICES) 220 177 y(AND)e(THEIR)h(USE)g(IN)f(LIMITED)h(MEMOR)l(Y)f(METHODS)850 389 y Fm(b)o(y)319 493 y Fl(R)o(ichar)n(d)f(H.)g(Byr)n(d,)h(Jor)n(ge)f(No)n (c)n(e)n(dal)f(and)h(R)n(ob)n(ert)g(B.)h(Schnab)n(el)746 963 y Fm(ABSTRA)o(CT)114 1110 y Fo(W)m(e)f(deriv)o(e)h(compact)g(represen)o (tations)h(of)f(BF)o(GS)f(and)h(symmetric)e(rank-one)i(matrices)g(for)114 1159 y(optimization.)d(These)d(represen)o(tations)g(allo)o(w)d(us)i(to)g (e\016cien)o(tly)f(implemen)o(t)e(limited)h(memory)114 1209 y(metho)q(ds)13 b(for)h(large)g(constrained)h(optimization)c(problems.)18 b(In)c(particular,)f(w)o(e)h(discuss)i(ho)o(w)114 1259 y(to)c(compute)h(pro)r (jections)g(of)f(limited)f(memory)f(matrices)i(on)o(to)h(subspaces.)19 b(W)m(e)13 b(also)f(presen)o(t)114 1309 y(a)h(compact)f(represen)o(tation)j (of)e(the)h(matrices)f(generated)h(b)o(y)g(Bro)o(yden's)f(up)q(date)h(for)f (solving)114 1359 y(systems)h(of)f(nonlinear)g(equations.)0 1504 y Fl(Key)18 b(wor)n(ds:)24 b Fm(Quasi-Newton)19 b(metho)q(d,)e (constrained)h(optimization,)g(limited)h(memory)e(metho)q(d,)0 1560 y(large-scale)f(optimization.)0 1636 y Fl(A)o(bbr)n(eviate)n(d)f(title:) 20 b Fm(Represen)o(tation)c(of)f(quasi-Newton)h(matrices.)0 1812 y Fh(1.)i(In)n(tro)r(duction.)71 1894 y Fm(Limited)24 b(memory)e(quasi-Newton)h(metho)q(ds)f(are)h(kno)o(wn)f(to)g(b)q(e)h (e\013ectiv)o(e)g(tec)o(hniques)h(for)0 1950 y(solving)17 b(certain)g (classes)g(of)f(large-scale)h(unconstrained)g(optimization)h(problems)f (\(Buc)o(kley)g(and)0 2006 y(Le)i(Nir)g(\(1983\),)e(Liu)j(and)f(No)q(cedal)g (\(1989\),)e(Gilb)q(ert)j(and)e(Lemar)o(\023)-21 b(ec)o(hal)19 b(\(1989\)\))d(.)30 b(They)18 b(mak)o(e)0 2063 y(simple)13 b(appro)o(ximations)f(of)f(Hessian)i(matrices,)f(whic)o(h)h(are)e(often)h(go) q(o)q(d)g(enough)g(to)f(pro)o(vide)h(a)g(fast)0 2119 y(rate)i(of)h(linear)h (con)o(v)o(ergence,)e(and)i(require)f(minimal)i(storage.)h(F)l(or)c(these)i (reasons)e(it)h(is)g(desirable)0 2176 y(to)d(use)h(limited)h(memory)e(appro)o (ximations)g(also)h(for)f(solving)h(problems)g(that)f(include)j(constrain)o (ts.)0 2232 y(Ho)o(w)o(ev)o(er,)e(most)g(algorithms)h(for)f(constrained)i (optimization)g(require)g(the)f(pro)s(jection)g(of)f(Hessian)0 2289 y(appro)o(ximations)20 b(on)o(to)g(the)h(subspace)g(of)f(activ)o(e)h (constrain)o(ts)f(and)h(other)f(matrix)g(calculations)0 2345 y(that)14 b(can)h(b)q(e)g(exp)q(ensiv)o(e)i(when)e(the)g(n)o(um)o(b)q(er)g (of)f(v)m(ariables)i(is)f(large.)20 b(This)15 b(is)g(true)g(ev)o(en)g(if)g (limited)0 2402 y(memory)e(appro)o(ximations)g(are)g(used,)h(unless)h(sp)q (ecial)g(care)e(is)h(tak)o(en)f(in)h(their)g(represen)o(tation)g(and)0 2458 y(manipulation.)863 2603 y(1)p eop %%Page: 2 3 bop 71 120 a Fm(In)12 b(this)g(pap)q(er)g(w)o(e)f(deriv)o(e)i(new)f(represen) o(tations)f(of)g(limited)j(memory)d(quasi-Newton)h(matrices)0 177 y(and)20 b(sho)o(w)f(ho)o(w)h(to)f(use)h(them)g(e\016cien)o(tly)h(in)g (the)f(kind)h(of)e(matrix)g(computations)h(required)h(in)0 233 y(constrained)16 b(optimization)h(metho)q(ds.)22 b(W)l(e)16 b(presen)o(t)f(new)h(expressions)h(for)e(b)q(oth)h(the)g(BF)o(GS)f(and)0 289 y(symmetric)g(rank-one)g(form)o(ulae)g(for)f(optimization,)i(and)f(also)g (deriv)o(e)h(a)e(compact)h(expression)h(for)0 346 y(Bro)o(yden's)e(metho)q(d) g(for)f(solving)i(systems)e(of)g(nonlinear)j(equations.)k(W)l(e)14 b(b)q(eliev)o(e)h(that)f(these)g(new)0 402 y(compact)g(represen)o(tations)h (of)f(quasi-Newton)h(matrices)g(are)f(of)g(in)o(terest)h(in)g(their)h(o)o(wn) e(righ)o(t,)g(but)0 459 y(in)i(this)g(pap)q(er)f(w)o(e)g(fo)q(cus)h(on)f (their)g(use)h(in)g(limited)h(memory)e(metho)q(ds.)71 515 y(T)l(o)e(motiv)m (ate)g(the)g(new)h(matrix)e(represen)o(tations)i(w)o(e)f(b)q(egin)h(b)o(y)f (describing)i(the)f(limited)h(mem-)0 572 y(ory)c(BF)o(GS)g(metho)q(d)h(for)f (unconstrained)i(optimization.)19 b(It)12 b(is)g(a)f(v)m(ariation)h(of)g(the) f(standard)h(BF)o(GS)0 628 y(metho)q(d,)j(whic)o(h)h(is)g(giv)o(en)f(b)o(y) 520 730 y Fg(x)546 737 y Ff(k)q Fk(+1)625 730 y Fm(=)e Fg(x)699 737 y Ff(k)730 730 y Fe(\000)e Fg(\025)803 737 y Ff(k)823 730 y Fg(H)861 737 y Ff(k)882 730 y Fg(g)904 737 y Ff(k)976 730 y Fg(k)j Fm(=)f(0)p Fg(;)8 b Fm(1)p Fg(;)g Fm(2)p Fg(;)f(:::)423 b Fm(\(1)p Fg(:)p Fm(1\))0 832 y(where)16 b Fg(\025)159 839 y Ff(k)195 832 y Fm(is)g(a)f(steplength,)h Fg(g)534 839 y Ff(k)571 832 y Fm(is)g(the)g(gradien)o(t)f(of)g(the)h(ob)s(jectiv)o(e)f(function)i Fg(f)h Fm(:)13 b Fn(R)1482 816 y Ff(n)1519 832 y Fe(!)g Fn(R)j Fm(at)f Fg(x)1714 839 y Ff(k)1735 832 y Fm(,)0 889 y(and)f(where)h(the)f(in)o (v)o(erse)g(Hessian)h(appro)o(ximation)f Fg(H)949 896 y Ff(k)984 889 y Fm(is)h(up)q(dated)f(at)g(ev)o(ery)g(iteration)g(b)o(y)g(means)0 945 y(of)h(the)g(form)o(ula)600 1002 y Fg(H)638 1009 y Ff(k)q Fk(+1)717 1002 y Fm(=)e Fg(V)801 983 y Ff(T)792 1013 y(k)829 1002 y Fg(H)867 1009 y Ff(k)888 1002 y Fg(V)915 1009 y Ff(k)946 1002 y Fm(+)d Fg(\032)1015 1009 y Ff(k)1036 1002 y Fg(s)1057 1009 y Ff(k)1079 1002 y Fg(s)1100 983 y Ff(T)1100 1013 y(k)1128 1002 y Fg(;)514 b Fm(\(1)p Fg(:)p Fm(2\))0 1085 y(where)523 1141 y Fg(\032)547 1148 y Ff(k)581 1141 y Fm(=)13 b(1)p Fg(=y)699 1123 y Ff(T)697 1153 y(k)726 1141 y Fg(s)747 1148 y Ff(k)768 1141 y Fg(;)99 b(V)907 1148 y Ff(k)940 1141 y Fm(=)13 b Fg(I)h Fe(\000)c Fg(\032)1091 1148 y Ff(k)1112 1141 y Fg(y)1134 1148 y Ff(k)1156 1141 y Fg(s)1177 1123 y Ff(T)1177 1153 y(k)1204 1141 y Fg(;)438 b Fm(\(1)p Fg(:)p Fm(3\))0 1225 y(and)528 1281 y Fg(s)549 1288 y Ff(k)583 1281 y Fm(=)13 b Fg(x)657 1288 y Ff(k)q Fk(+1)733 1281 y Fe(\000)e Fg(x)805 1288 y Ff(k)826 1281 y Fg(;)78 b(y)939 1288 y Ff(k)973 1281 y Fm(=)13 b Fg(g)1043 1288 y Ff(k)q Fk(+1)1119 1281 y Fe(\000)e Fg(g)1187 1288 y Ff(k)1208 1281 y Fg(:)0 1365 y Fm(\(see)19 b(e.g.)30 b(Fletc)o(her)19 b(\(1987\)\).)29 b(W)l(e)19 b(sa)o(y)f(that)g(the)h(matrix)g Fg(H)1101 1372 y Ff(k)q Fk(+1)1185 1365 y Fm(is)h(obtained)f(b)o(y)g(up)q (dating)h Fg(H)1727 1372 y Ff(k)0 1421 y Fm(using)c(the)f(pair)h Fe(f)p Fg(s)335 1428 y Ff(k)356 1421 y Fg(;)8 b(y)399 1428 y Ff(k)420 1421 y Fe(g)p Fm(.)71 1478 y(The)17 b(limited)j(memory)d(BF)o(GS)g (metho)q(d)g(is)h(an)g(adaptation)f(of)g(the)g(BF)o(GS)g(metho)q(d)h(to)f (large)0 1534 y(problems.)22 b(The)16 b(implemen)o(tation)h(describ)q(ed)h(b) o(y)e(Liu)h(and)f(No)q(cedal)h(\(1989\))d(is)i(almost)f(iden)o(tical)0 1590 y(to)20 b(that)f(of)h(the)h(standard)f(BF)o(GS)g(metho)q(d)g({)g(the)h (only)g(di\013erence)g(is)g(in)g(the)g(matrix)f(up)q(date.)0 1647 y(Instead)c(of)g(storing)f(the)h(matrices)g Fg(H)666 1654 y Ff(k)687 1647 y Fm(,)f(one)h(stores)f(a)h(certain)g(n)o(um)o(b)q(er,)g(sa)o (y)f Fg(m)p Fm(,)g(of)h(pairs)g Fe(f)p Fg(s)1655 1654 y Ff(i)1669 1647 y Fg(;)8 b(y)1712 1654 y Ff(i)1725 1647 y Fe(g)0 1703 y Fm(that)15 b(de\014ne)i(them)f(implicitl)q(y)l(.)25 b(The)16 b(pro)q(duct)h Fg(H)872 1710 y Ff(k)893 1703 y Fg(g)915 1710 y Ff(k)952 1703 y Fm(is)f(obtained)h(b)o(y)f(p)q(erforming)g(a)g(sequence)h (of)0 1760 y(inner)d(pro)q(ducts)g(in)o(v)o(olving)g Fg(g)515 1767 y Ff(k)550 1760 y Fm(and)f(the)g Fg(m)g Fm(most)g(recen)o(t)g(v)o(ector) f(pairs)i Fe(f)p Fg(s)1296 1767 y Ff(i)1310 1760 y Fg(;)8 b(y)1353 1767 y Ff(i)1367 1760 y Fe(g)p Fm(.)19 b(After)12 b(computing)0 1816 y(the)18 b(new)h(iterate,)g(the)f(oldest)h(pair)g(is)f(deleted)i(from)e (the)g(set)g Fe(f)p Fg(s)1170 1823 y Ff(i)1185 1816 y Fg(;)8 b(y)1228 1823 y Ff(i)1241 1816 y Fe(g)p Fm(,)19 b(and)f(is)h(replaced)h(b)o (y)e(the)0 1873 y(new)o(est)h(one.)31 b(The)19 b(algorithm)g(therefore)g(alw) o(a)o(ys)f(k)o(eeps)h(the)g Fg(m)g Fm(most)f(recen)o(t)h(pairs)g Fe(f)p Fg(s)1595 1880 y Ff(i)1609 1873 y Fg(;)8 b(y)1652 1880 y Ff(i)1666 1873 y Fe(g)19 b Fm(to)0 1929 y(de\014ne)f(the)e(iteration)h (matrix.)23 b(This)17 b(approac)o(h)f(is)h(suitable)h(for)d(large)i(problems) g(b)q(ecause)g(it)g(has)0 1986 y(b)q(een)d(observ)o(ed)g(in)g(practice)g (that)e(small)i(v)m(alues)h(of)e Fg(m)g Fm(\(sa)o(y)f Fg(m)h Fe(2)g Fm([3)p Fg(;)8 b Fm(7]\))i(giv)o(e)k(satisfactory)e(results.)71 2042 y(Let)h(us)h(describ)q(e)h(the)f(up)q(dating)g(pro)q(cess)g(in)g(more)f (detail.)21 b(Supp)q(ose)14 b(that)f(the)h(curren)o(t)f(iterate)0 2099 y(is)19 b Fg(x)75 2106 y Ff(k)115 2099 y Fm(and)g(that)f(w)o(e)g(ha)o(v) o(e)h(stored)f(the)h Fg(m)f Fm(pairs)h Fe(f)p Fg(s)927 2106 y Ff(i)941 2099 y Fg(;)8 b(y)984 2106 y Ff(i)998 2099 y Fe(g)p Fg(;)19 b(i)f Fm(=)h Fg(k)14 b Fe(\000)e Fg(m;)c(:::;)g(k)j Fe(\000)i Fm(1.)30 b(W)l(e)19 b(c)o(ho)q(ose)f(a)0 2163 y(\\basic)d(matrix")g Fg(H)349 2139 y Fk(\(0\))345 2177 y Ff(k)410 2163 y Fm(\(usually)h(a)f (diagonal)g(matrix\))g(and)g(up)q(date)g(it)h Fg(m)f Fm(times)g(using)g(the)h (BF)o(GS)0 2219 y(form)o(ula)f(and)g(the)g Fg(m)g Fm(pairs)h Fe(f)p Fg(s)544 2226 y Ff(i)558 2219 y Fg(;)8 b(y)601 2226 y Ff(i)614 2219 y Fe(g)p Fg(;)17 b(i)12 b Fm(=)h Fg(k)e Fe(\000)g Fg(m;)d(:::;)g(k)g Fe(\000)i Fm(1.)20 b(F)l(rom)14 b(\(1.2\))f(w)o(e)i(see)h (that)e Fg(H)1582 2226 y Ff(k)1618 2219 y Fm(can)h(b)q(e)0 2276 y(written)g(as)194 2378 y Fg(H)232 2385 y Ff(k)295 2378 y Fm(=)372 2331 y Fd(\020)397 2378 y Fg(V)433 2359 y Ff(T)424 2389 y(k)q Fc(\000)p Fk(1)497 2378 y Fe(\001)8 b(\001)g(\001)e Fg(V)594 2359 y Ff(T)585 2389 y(k)q Fc(\000)p Ff(m)664 2331 y Fd(\021)697 2378 y Fg(H)739 2354 y Fk(\(0\))735 2392 y Ff(k)793 2378 y Fm(\()o Fg(V)837 2385 y Ff(k)q Fc(\000)p Ff(m)925 2378 y Fe(\001)i(\001)g(\001)d Fg(V)1012 2385 y Ff(k)q Fc(\000)p Fk(1)1078 2378 y Fm(\))295 2469 y(+)42 b Fg(\032)396 2476 y Ff(k)q Fc(\000)p Ff(m)483 2422 y Fd(\020)508 2469 y Fg(V)544 2450 y Ff(T)535 2480 y(k)q Fc(\000)p Fk(1)608 2469 y Fe(\001)8 b(\001)g(\001)e Fg(V)705 2450 y Ff(T)696 2480 y(k)q Fc(\000)p Ff(m)p Fk(+1)820 2422 y Fd(\021)853 2469 y Fg(s)874 2476 y Ff(k)q Fc(\000)p Ff(m)954 2469 y Fg(s)975 2450 y Ff(T)975 2480 y(k)q Fc(\000)p Ff(m)1063 2469 y Fm(\()p Fg(V)1108 2476 y Ff(k)q Fc(\000)p Ff(m)p Fk(+1)1240 2469 y Fe(\001)i(\001)g(\001)d Fg(V)1327 2476 y Ff(k)q Fc(\000)p Fk(1)1393 2469 y Fm(\))863 2603 y(2)p eop %%Page: 3 4 bop 295 127 a Fm(+)42 b Fg(\032)396 134 y Ff(k)q Fc(\000)p Ff(m)p Fk(+1)528 80 y Fd(\020)553 127 y Fg(V)589 109 y Ff(T)580 139 y(k)q Fc(\000)p Fk(1)653 127 y Fe(\001)8 b(\001)g(\001)e Fg(V)750 109 y Ff(T)741 139 y(k)q Fc(\000)p Ff(m)p Fk(+2)865 80 y Fd(\021)898 127 y Fg(s)919 134 y Ff(k)q Fc(\000)p Ff(m)p Fk(+1)1044 127 y Fg(s)1065 109 y Ff(T)1065 139 y(k)q Fc(\000)p Ff(m)p Fk(+1)1198 127 y Fm(\()p Fg(V)1243 134 y Ff(k)q Fc(\000)p Ff(m)p Fk(+2)1375 127 y Fe(\001)i(\001)g(\001)e Fg(V)1463 134 y Ff(k)q Fc(\000)p Fk(1)1528 127 y Fm(\))295 233 y(+)372 200 y(.)372 216 y(.)372 233 y(.)295 302 y(+)42 b Fg(\032)396 309 y Ff(k)q Fc(\000)p Fk(1)462 302 y Fg(s)483 309 y Ff(k)q Fc(\000)p Fk(1)549 302 y Fg(s)570 283 y Ff(T)570 313 y(k)q Fc(\000)p Fk(1)637 302 y Fg(:)1005 b Fm(\(1.4\))71 404 y(There)11 b(is)g(a)g(recursiv)o (e)h(form)o(ula)f(\(No)q(cedal)g(\(1980\)\))e(that)i(tak)o(es)f(adv)m(an)o (tage)g(of)h(the)g(symmetry)f(of)0 460 y(this)15 b(expression)g(to)f(compute) h(the)f(pro)q(duct)h Fg(H)830 467 y Ff(k)851 460 y Fg(g)873 467 y Ff(k)909 460 y Fm(e\016cien)o(tly)l(.)21 b(As)15 b(a)f(result,)g(the)h (computation)f(of)0 517 y(the)e(searc)o(h)g(direction)i(in)f(the)f(limited)i (memory)d(BF)o(GS)h(metho)q(d)g(for)g(unconstrained)h(optimization)0 573 y(can)i(b)q(e)h(p)q(erformed)g(v)o(ery)f(economically)l(.)71 630 y(It)j(turns)g(out,)g(ho)o(w)o(ev)o(er,)g(that)f(in)i(t)o(w)o(o)e(resp)q (ects)i(this)f(recursiv)o(e)h(form)o(ula)f(is)h(m)o(uc)o(h)f(less)h(eco-)0 686 y(nomical)e(for)e(some)h(of)f(the)h(calculations)i(required)f(when)f (constrain)o(ts)g(are)f(presen)o(t.)22 b(First,)15 b(when)0 743 y(the)j(constrain)o(ts)g(are)g(sparse)g(the)h(recursion)g(do)q(es)f(not)g (tak)o(e)g(go)q(o)q(d)g(adv)m(an)o(tage)g(of)g(this)h(sparsit)o(y)l(.)0 799 y(F)l(or)13 b(example,)i(if)f Fg(e)331 806 y Ff(i)359 799 y Fm(is)h(a)e(unit)i(v)o(ector,)e(the)h(computation)g(of)f Fg(H)1112 806 y Ff(k)1133 799 y Fg(e)1154 806 y Ff(i)1183 799 y Fm(is)h(almost)f(as)h(exp)q(ensiv)o(e)h(as)f(the)0 855 y(computation)g(of)f Fg(H)352 862 y Ff(k)373 855 y Fg(g)395 862 y Ff(k)416 855 y Fm(.)20 b(Second,)14 b(man)o(y)f(algorithms)h(for)f(constrained)i (optimization)f(require)h(the)0 912 y(direct)k(Hessian)h(appro)o(ximation,)f Fg(B)656 919 y Ff(k)696 912 y Fm(=)f Fg(H)791 893 y Fc(\000)p Fk(1)787 926 y Ff(k)856 912 y Fm(instead)i(of)e(the)g(in)o(v)o(erse)h(BF)o (GS)f(appro)o(ximation,)0 968 y Fg(H)38 975 y Ff(k)59 968 y Fm(.)i(Ho)o(w)o(ev)o(er,)13 b(there)h(app)q(ears)g(to)g(b)q(e)h(no)f (analogous)f(recursion)i(for)f(the)g(Hessian)h(appro)o(ximation)0 1025 y Fg(B)34 1032 y Ff(k)70 1025 y Fm(and,)g(as)f(p)q(oin)o(ted)i(out)e(in) i(Section)f(4.2,)f(a)g(straigh)o(tforw)o(ard)f(implemen)o(tation)i(turns)g (out)f(to)g(b)q(e)0 1081 y(quite)i(costly)l(.)71 1138 y(After)h(deriving)i (our)f(new)g(quasi-Newton)g(represen)o(tations)g(in)g(Section)h(2,)f(w)o(e)f (sho)o(w)h(in)g(Sec-)0 1194 y(tion)h(3)f(ho)o(w)g(they)g(can)h(b)q(e)g(used)g (in)g(limited)i(memory)c(metho)q(ds)i(in)g(a)f(w)o(a)o(y)g(that)f(is)i (e\016cien)o(t)h(for)0 1251 y(unconstrained)j(optimization,)h(and)e(gets)f (around)h(b)q(oth)g(of)f(these)h(di\016culties)i(in)e(constrained)0 1307 y(optimization)16 b(calculations.)0 1463 y Fl(Notation.)30 b Fm(The)19 b(n)o(um)o(b)q(er)g(of)f(v)m(ariables)i(in)f(the)g(optimization)g (problem)g(is)g Fg(n)p Fm(,)g(and)g(the)g(n)o(um)o(b)q(er)0 1520 y(of)h(correction)h(pairs)g(used)g(in)h(the)f(limited)h(memory)e(metho)q (ds)h(is)g Fg(m)p Fm(.)36 b(The)21 b(Hessian)h(appro)o(xi-)0 1576 y(mation)c(is)g(denoted)g(b)o(y)g Fg(B)479 1583 y Ff(k)501 1576 y Fm(,)g(and)g(the)g(in)o(v)o(erse)g(Hessian)h(appro)o(ximation)e(is)i Fg(H)1418 1583 y Ff(k)1439 1576 y Fm(.)28 b(The)18 b Fg(i)p Fm(-th)f(unit)0 1633 y(v)o(ector)d(is)g(written)h(as)f Fg(e)415 1640 y Ff(i)429 1633 y Fm(.)19 b(A)c(diagonal)g(matrix)f(with)g(diagonal)h (elemen)o(ts)g Fg(\022)1327 1640 y Fk(1)1347 1633 y Fg(;)8 b(:::;)g(\022)1449 1640 y Ff(n)1484 1633 y Fm(is)15 b(denoted)g(b)o(y)0 1689 y(diag)q([)p Fg(\022)118 1696 y Fk(1)138 1689 y Fg(;)8 b(:::;)g(\022)240 1696 y Ff(n)261 1689 y Fm(].)0 1845 y Fh(2.)37 b(Compact)18 b(Represen)n(tations)f(of)i(BF)n(GS)h(Matrices)71 1926 y Fm(W)l(e)d(will)h(no)o(w)e(describ)q(e)j(new)e(represen)o(tations)g (of)f(the)h(in)o(v)o(erse)g(and)g(direct)h(BF)o(GS)e(matrices,)0 1983 y(and)k(sho)o(w)g(ho)o(w)g(to)f(compute)h(sev)o(eral)h(t)o(yp)q(es)f(of) g(matrix-v)o(ector)f(pro)q(ducts)h(e\016cien)o(tly)l(.)37 b(In)21 b(this)0 2039 y(section)15 b(w)o(e)f(will)h(consider)h(the)e(up)q(dating)h (pro)q(cess)f(in)h(a)f(general)h(setting,)f(and)g(will)i(not)e(restrict)g(it) 0 2096 y(to)h(the)g(case)g(of)g(limited)i(memory)d(metho)q(ds.)71 2152 y(Let)h(us)g(de\014ne)i(the)e Fg(n)10 b Fe(\002)h Fg(k)16 b Fm(matrices)f Fg(S)753 2159 y Ff(k)790 2152 y Fm(and)g Fg(Y)904 2159 y Ff(k)941 2152 y Fm(b)o(y)410 2254 y Fg(S)438 2261 y Ff(k)472 2254 y Fm(=)e([)o Fg(s)553 2261 y Fk(0)573 2254 y Fg(;)8 b(:)g(:)g(:)d(;)j(s)696 2261 y Ff(k)q Fc(\000)p Fk(1)762 2254 y Fm(])f Fg(;)158 b(Y)979 2261 y Ff(k)1013 2254 y Fm(=)13 b([)p Fg(y)1096 2261 y Fk(0)1116 2254 y Fg(;)8 b(:)g(:)g(:)t(;)g(y)1239 2261 y Ff(k)q Fc(\000)p Fk(1)1305 2254 y Fm(])f Fg(:)317 b Fm(\(2)p Fg(:)p Fm(1\))0 2357 y(W)l(e)13 b(\014rst)f(pro)o(v)o(e)g(a)g (preliminary)i(lemma)f(on)f(pro)q(ducts)h(of)f(pro)s(jection)g(matrices)h (that)f(will)i(b)q(e)f(useful)0 2413 y(in)j(subsequen)o(t)g(analysis)g(and)f (is)h(also)f(in)o(teresting)h(in)g(its)f(o)o(wn)g(righ)o(t.)863 2603 y(3)p eop %%Page: 4 5 bop 0 120 a Fn(Lemma)17 b(2.1)23 b Fl(The)16 b(pr)n(o)n(duct)h(of)f(a)h(set)e (of)i Fg(k)g Fl(pr)n(oje)n(ction)f(matric)n(es)g(of)g(the)h(form)g(\(1.3\))f (satis\014es)597 203 y Fg(V)624 210 y Fk(0)650 203 y Fe(\001)8 b(\001)g(\001)e Fg(V)738 210 y Ff(k)q Fc(\000)p Fk(1)816 203 y Fm(=)13 b Fg(I)h Fe(\000)d Fg(Y)970 210 y Ff(k)991 203 y Fg(R)1026 184 y Fc(\000)p Fk(1)1026 217 y Ff(k)1073 203 y Fg(S)1104 184 y Ff(T)1101 214 y(k)1131 203 y Fg(;)511 b Fm(\(2)p Fg(:)p Fm(2\))0 286 y Fl(wher)n(e)16 b Fg(R)163 293 y Ff(k)200 286 y Fl(is)g(the)h Fg(k)11 b Fe(\002)f Fg(k)18 b Fl(matrix)531 397 y Fm(\()p Fg(R)584 404 y Ff(k)604 397 y Fm(\))622 404 y Ff(i;j)674 397 y Fm(=)722 325 y Fd(\()777 369 y Fg(s)798 352 y Ff(T)798 381 y(i)p Fc(\000)p Fk(1)857 369 y Fg(y)879 376 y Ff(j)r Fc(\000)p Fk(1)984 369 y Fm(if)c Fg(i)f Fe(\024)g Fg(j)777 425 y Fm(0)184 b(otherwise)1205 397 y Fg(:)437 b Fm(\(2)p Fg(:)p Fm(3\))0 531 y Fn(Pro)q(of.)20 b Fm(Pro)q(ceeding)c(b)o(y)e(induction) j(w)o(e)e(note)g(that)f(\(2.2\))f(holds)j(for)e Fg(k)g Fm(=)f(1,)h(b)q (ecause)i(in)g(this)f(case)0 587 y(the)g(righ)o(t)g(hand)h(side)g(of)f (\(2.2\))e(is)679 663 y Fg(I)h Fe(\000)c Fg(y)780 670 y Fk(0)839 632 y Fm(1)p 805 652 91 2 v 805 696 a Fg(s)826 680 y Ff(T)826 708 y Fk(0)854 696 y Fg(y)876 703 y Fk(0)901 663 y Fg(s)922 644 y Ff(T)922 674 y Fk(0)962 663 y Fm(=)j Fg(V)1037 670 y Fk(0)1056 663 y Fg(:)586 b Fm(\(2)p Fg(:)p Fm(4\))0 764 y(No)o(w)15 b(w)o(e)h(assume)g(that)f(\(2.2\))g(holds)i(for)e(some)h Fg(k)q Fm(,)g(and)g(consider)h Fg(k)12 b Fm(+)f(1.)22 b(If)16 b(w)o(e)g(write)g(the) g(matrix)0 820 y Fg(R)35 827 y Ff(k)q Fk(+1)116 820 y Fm(as)635 934 y Fg(R)670 941 y Ff(k)q Fk(+1)749 934 y Fm(=)797 837 y Fd(2)797 910 y(6)797 937 y(4)845 875 y Fg(R)880 882 y Ff(k)943 875 y Fg(S)974 858 y Ff(T)971 888 y(k)1001 875 y Fg(y)1023 882 y Ff(k)862 988 y Fm(0)985 970 y Fk(1)p 975 977 38 2 v 975 1003 a Ff(\032)993 1009 y Fb(k)1065 837 y Fd(3)1065 910 y(7)1065 937 y(5)1100 934 y Fg(;)0 1063 y Fm(w)o(e)f(see)g(that)541 1165 y Fg(R)576 1146 y Fc(\000)p Fk(1)576 1179 y Ff(k)q Fk(+1)655 1165 y Fm(=)703 1069 y Fd(2)703 1142 y(6)703 1168 y(4)751 1109 y Fg(R)786 1090 y Fc(\000)p Fk(1)786 1123 y Ff(k)875 1109 y Fe(\000)p Fg(\032)934 1116 y Ff(k)955 1109 y Fg(R)990 1090 y Fc(\000)p Fk(1)990 1123 y Ff(k)1037 1109 y Fg(S)1068 1093 y Ff(T)1065 1123 y(k)1095 1109 y Fg(y)1117 1116 y Ff(k)781 1222 y Fm(0)180 b Fg(\032)1008 1229 y Ff(k)1159 1069 y Fd(3)1159 1142 y(7)1159 1168 y(5)1194 1165 y Fg(:)448 b Fm(\(2)p Fg(:)p Fm(5\))0 1294 y(This)16 b(implies)h(that)165 1432 y Fg(I)c Fe(\000)e Fg(Y)270 1439 y Ff(k)q Fk(+1)337 1432 y Fg(R)372 1413 y Fc(\000)p Fk(1)372 1446 y Ff(k)q Fk(+1)438 1432 y Fg(S)469 1413 y Ff(T)466 1443 y(k)q Fk(+1)573 1432 y Fm(=)42 b Fg(I)14 b Fe(\000)729 1385 y Fd(h)770 1432 y Fg(Y)796 1439 y Ff(k)838 1432 y Fg(y)860 1439 y Ff(k)902 1385 y Fd(i)929 1335 y(2)929 1408 y(6)929 1435 y(4)978 1376 y Fg(R)1013 1357 y Fc(\000)p Fk(1)1013 1390 y Ff(k)1101 1376 y Fe(\000)p Fg(\032)1160 1383 y Ff(k)1181 1376 y Fg(R)1216 1357 y Fc(\000)p Fk(1)1216 1390 y Ff(k)1263 1376 y Fg(S)1294 1360 y Ff(T)1291 1389 y(k)1321 1376 y Fg(y)1343 1383 y Ff(k)1007 1489 y Fm(0)181 b Fg(\032)1235 1496 y Ff(k)1386 1335 y Fd(3)1386 1408 y(7)1386 1435 y(5)1421 1335 y(2)1421 1408 y(6)1421 1435 y(4)1469 1376 y Fg(S)1500 1359 y Ff(T)1497 1389 y(k)1474 1488 y Fg(s)1495 1472 y Ff(T)1495 1502 y(k)1548 1335 y Fd(3)1548 1408 y(7)1548 1435 y(5)573 1566 y Fm(=)42 b Fg(I)14 b Fe(\000)c Fg(Y)755 1573 y Ff(k)777 1566 y Fg(R)812 1547 y Fc(\000)p Fk(1)812 1580 y Ff(k)859 1566 y Fg(S)890 1547 y Ff(T)887 1577 y(k)927 1566 y Fm(+)h Fg(\032)997 1573 y Ff(k)1017 1566 y Fg(Y)1043 1573 y Ff(k)1065 1566 y Fg(R)1100 1547 y Fc(\000)p Fk(1)1100 1580 y Ff(k)1147 1566 y Fg(S)1178 1547 y Ff(T)1175 1577 y(k)1205 1566 y Fg(y)1227 1573 y Ff(k)1249 1566 y Fg(s)1270 1547 y Ff(T)1270 1577 y(k)1308 1566 y Fe(\000)f Fg(\032)1377 1573 y Ff(k)1398 1566 y Fg(y)1420 1573 y Ff(k)1442 1566 y Fg(s)1463 1547 y Ff(T)1463 1577 y(k)573 1635 y Fm(=)42 b(\()p Fg(I)13 b Fe(\000)e Fg(Y)773 1642 y Ff(k)795 1635 y Fg(R)830 1616 y Fc(\000)p Fk(1)830 1649 y Ff(k)877 1635 y Fg(S)908 1616 y Ff(T)905 1646 y(k)935 1635 y Fm(\)\()p Fg(I)i Fe(\000)d Fg(\032)1073 1642 y Ff(k)1094 1635 y Fg(y)1116 1642 y Ff(k)1138 1635 y Fg(s)1159 1616 y Ff(T)1159 1646 y(k)1186 1635 y Fm(\))p Fg(:)0 1718 y Fm(Using)16 b(this)f(with)h(the)f(inductiv)o(e)i(h)o(yp)q (othesis)f(of)f(\(2.2\))f(w)o(e)g(ha)o(v)o(e)h(that)457 1800 y Fg(V)484 1807 y Fk(0)511 1800 y Fe(\001)8 b(\001)g(\001)d Fg(V)598 1807 y Ff(k)660 1800 y Fm(=)42 b(\()p Fg(I)14 b Fe(\000)c Fg(Y)860 1807 y Ff(k)882 1800 y Fg(R)917 1781 y Fc(\000)p Fk(1)917 1814 y Ff(k)964 1800 y Fg(S)995 1782 y Ff(T)992 1812 y(k)1022 1800 y Fm(\)\()p Fg(I)j Fe(\000)d Fg(\032)1160 1807 y Ff(k)1181 1800 y Fg(y)1203 1807 y Ff(k)1225 1800 y Fg(s)1246 1782 y Ff(T)1246 1812 y(k)1274 1800 y Fm(\))660 1869 y(=)42 b(\()p Fg(I)14 b Fe(\000)c Fg(Y)860 1876 y Ff(k)q Fk(+1)927 1869 y Fg(R)962 1850 y Fc(\000)p Fk(1)962 1883 y Ff(k)q Fk(+1)1028 1869 y Fg(S)1059 1851 y Ff(T)1056 1881 y(k)q Fk(+1)1122 1869 y Fm(\))p Fg(;)0 1952 y Fm(whic)o(h)16 b(establishes)h(the)e(pro)q(duct)g(relation)h(\(2.2\))e (for)g(all)j Fg(k)q Fm(.)663 b Fa(2)71 2027 y Fm(It)18 b(should)h(b)q(e)g(p)q (oin)o(ted)g(out)e(that)h(this)g(lemma)h(holds)g(for)e(the)h(pro)q(duct)h(of) e(an)o(y)h(sequence)h(of)0 2083 y(pro)s(jections)e(on)o(to)g(spaces)h(of)f (dimension)i Fg(n)12 b Fe(\000)g Fm(1)17 b(and)h(is)g(a)f(useful)i(but)f (little-kno)o(wn)h(result.)27 b(Es-)0 2140 y(sen)o(tially)18 b(the)e(same)g(result)h(is)f(also)h(men)o(tioned)g(b)o(y)f(W)l(alk)o(er)g (\(1988\))f(in)i(the)f(con)o(text)g(of)g(pro)q(ducts)0 2196 y(of)e(Householder)i(transformations.)j(The)c(lemma)g(can)g(b)q(e)g (generalized)i(to)d(pro)s(jections)h(on)o(to)e(sub-)0 2253 y(spaces)h(of)g(arbitrary)g(and)g(di\013eren)o(t)h(dimensions,)h(in)f(whic)o (h)g(case)f(the)g(matrix)g Fg(R)1428 2260 y Ff(k)1463 2253 y Fm(b)q(ecomes)h(blo)q(c)o(k)0 2309 y(upp)q(er)h(triangular.)71 2366 y(The)11 b(follo)o(wing)i(theorem)e(giv)o(es)g(a)h(compact)f(represen)o (tation)g(of)g(the)h(matrix)f Fg(H)1440 2373 y Ff(k)1473 2366 y Fm(obtained)h(after)0 2422 y Fg(k)i Fm(BF)o(GS)e(up)q(dates.)19 b(W)l(e)13 b(will)h(later)e(see)h(that)f(this)h(represen)o(tation)g(is)g (often)f(more)g(con)o(v)o(enien)o(t)h(than)0 2479 y(\(1.4\).)863 2603 y(4)p eop %%Page: 5 6 bop 0 120 a Fn(Theorem)17 b(2.2)22 b Fl(L)n(et)d Fg(H)431 127 y Fk(0)470 120 y Fl(b)n(e)h(symmetric)f(and)h(p)n(ositive)f(de\014nite)g(and) h(assume)f(that)i(the)e Fg(k)i Fl(p)n(airs)0 177 y Fe(f)p Fg(s)44 184 y Ff(i)58 177 y Fg(;)8 b(y)101 184 y Ff(i)115 177 y Fe(g)138 157 y Ff(k)q Fc(\000)p Fk(1)138 189 y Ff(i)p Fk(=0)218 177 y Fl(satisfy)14 b Fg(s)379 160 y Ff(T)379 189 y(i)407 177 y Fg(y)429 184 y Ff(i)456 177 y Fg(>)f Fm(0)p Fl(.)19 b(L)n(et)14 b Fg(H)675 184 y Ff(k)710 177 y Fl(b)n(e)g(obtaine)n(d)g(by)g(up)n(dating)g Fg(H)1218 184 y Fk(0)1258 177 y Fg(k)h Fl(times)f(using)g(the)g(inverse)0 233 y(BF)o(GS)h(formula)j(\(1.2\))d(and)i(the)f(p)n(airs)g Fe(f)p Fg(s)740 240 y Ff(i)754 233 y Fg(;)8 b(y)797 240 y Ff(i)811 233 y Fe(g)834 214 y Ff(k)q Fc(\000)p Fk(1)834 246 y Ff(i)p Fk(=0)900 233 y Fl(.)21 b(Then)65 380 y Fg(H)103 387 y Ff(k)136 380 y Fm(=)13 b Fg(H)222 387 y Fk(0)252 380 y Fm(+)297 333 y Fd(h)338 380 y Fg(S)366 387 y Ff(k)429 380 y Fg(H)467 387 y Fk(0)486 380 y Fg(Y)512 387 y Ff(k)555 333 y Fd(i)582 284 y(2)582 357 y(6)582 383 y(4)630 325 y Fg(R)665 305 y Fc(\000)p Ff(T)665 339 y(k)720 325 y Fm(\()p Fg(D)776 332 y Ff(k)807 325 y Fm(+)d Fg(Y)889 308 y Ff(T)878 338 y(k)916 325 y Fg(H)954 332 y Fk(0)974 325 y Fg(Y)1000 332 y Ff(k)1021 325 y Fm(\))p Fg(R)1074 305 y Fc(\000)p Fk(1)1074 339 y Ff(k)1163 325 y Fe(\000)p Fg(R)1233 305 y Fc(\000)p Ff(T)1233 339 y(k)817 438 y Fe(\000)p Fg(R)887 419 y Fc(\000)p Fk(1)887 453 y Ff(k)1214 438 y Fm(0)1309 284 y Fd(3)1309 357 y(7)1309 383 y(5)1344 284 y(2)1344 357 y(6)1344 383 y(4)1424 324 y Fg(S)1455 307 y Ff(T)1452 337 y(k)1392 437 y Fg(Y)1429 420 y Ff(T)1418 450 y(k)1456 437 y Fg(H)1494 444 y Fk(0)1534 284 y Fd(3)1534 357 y(7)1534 383 y(5)1570 380 y Fg(;)72 b Fm(\(2)p Fg(:)p Fm(6\))0 526 y Fl(wher)n(e)16 b Fg(R)163 533 y Ff(k)200 526 y Fl(is)g(as)g(given)g(in)g(\(2.3\))f(and)i Fg(D)722 533 y Ff(k)759 526 y Fl(is)f(the)g Fg(k)11 b Fe(\002)g Fg(k)17 b Fl(diagonal)f(matrix)555 622 y Fg(D)593 629 y Ff(k)626 622 y Fm(=)d(diag)766 575 y Fd(h)785 622 y Fg(s)806 604 y Ff(T)806 634 y Fk(0)834 622 y Fg(y)856 629 y Fk(0)876 622 y Fg(;)8 b(:)g(:)g(:)d(;)j (s)999 604 y Ff(T)999 634 y(k)q Fc(\000)p Fk(1)1065 622 y Fg(y)1087 629 y Ff(k)q Fc(\000)p Fk(1)1154 575 y Fd(i)1181 622 y Fg(:)461 b Fm(\(2)p Fg(:)p Fm(7\))0 741 y Fn(Pro)q(of.)20 b Fm(W)l(e)15 b(write)g(the)h(BF)o(GS)e(form)o(ula)h(\(1.2\))f(as)635 829 y Fg(H)673 836 y Ff(k)707 829 y Fm(=)f Fg(M)799 836 y Ff(k)830 829 y Fm(+)e Fg(N)913 836 y Ff(k)934 829 y Fg(;)57 b(k)14 b Fe(\025)f Fm(1)542 b(\(2)p Fg(:)p Fm(8\))0 917 y(where)15 b Fg(M)175 924 y Ff(k)212 917 y Fm(and)g Fg(N)337 924 y Ff(k)373 917 y Fm(are)g(de\014ned)i(recursiv)o(ely)f(b)o(y)653 964 y Fd(\()708 1007 y Fg(M)752 1014 y Fk(0)784 1007 y Fm(=)d Fg(H)870 1014 y Fk(0)708 1064 y Fg(M)752 1071 y Ff(k)q Fk(+1)831 1064 y Fm(=)g Fg(V)915 1047 y Ff(T)906 1077 y(k)943 1064 y Fg(M)987 1071 y Ff(k)1008 1064 y Fg(V)1035 1071 y Ff(k)1056 1064 y Fg(;)1655 1036 y Fm(\(2)p Fg(:)p Fm(9\))0 1154 y(and)565 1160 y Fd(\()619 1203 y Fg(N)656 1210 y Fk(1)688 1203 y Fm(=)g Fg(\032)760 1210 y Fk(0)779 1203 y Fg(s)800 1210 y Fk(0)820 1203 y Fg(s)841 1187 y Ff(T)841 1215 y Fk(0)619 1260 y Fg(N)656 1267 y Ff(k)q Fk(+1)735 1260 y Fm(=)g Fg(V)819 1243 y Ff(T)810 1273 y(k)847 1260 y Fg(N)884 1267 y Ff(k)905 1260 y Fg(V)932 1267 y Ff(k)963 1260 y Fm(+)d Fg(\032)1032 1267 y Ff(k)1053 1260 y Fg(s)1074 1267 y Ff(k)1096 1260 y Fg(s)1117 1243 y Ff(T)1117 1273 y(k)1145 1260 y Fg(:)1632 1232 y Fm(\(2)p Fg(:)p Fm(10\))71 1341 y(First)k(note,)h (from)g(the)g(de\014nition)i(of)e Fg(M)779 1348 y Ff(k)815 1341 y Fm(and)h(\(2.2\),)d(that)437 1437 y Fg(M)481 1444 y Ff(k)544 1437 y Fm(=)621 1390 y Fd(\020)646 1437 y Fg(V)682 1418 y Ff(T)673 1448 y(k)q Fc(\000)p Fk(1)746 1437 y Fe(\001)8 b(\001)g(\001)e Fg(V)844 1418 y Ff(T)834 1448 y Fk(0)871 1390 y Fd(\021)903 1437 y Fg(H)941 1444 y Fk(0)968 1437 y Fm(\()p Fg(V)1013 1444 y Fk(0)1040 1437 y Fe(\001)i(\001)g(\001)e Fg(V)1128 1444 y Ff(k)q Fc(\000)p Fk(1)1194 1437 y Fm(\))544 1521 y(=)42 b(\()p Fg(I)13 b Fe(\000)e Fg(S)746 1528 y Ff(k)767 1521 y Fg(R)802 1502 y Fc(\000)p Ff(T)802 1535 y(k)857 1521 y Fg(Y)893 1503 y Ff(T)883 1533 y(k)921 1521 y Fm(\))p Fg(H)977 1528 y Fk(0)996 1521 y Fm(\()p Fg(I)i Fe(\000)e Fg(Y)1119 1528 y Ff(k)1140 1521 y Fg(R)1175 1502 y Fc(\000)p Fk(1)1175 1535 y Ff(k)1223 1521 y Fg(S)1254 1503 y Ff(T)1251 1533 y(k)1281 1521 y Fm(\))p Fg(:)320 b Fm(\(2.11\))71 1610 y(Next,)14 b(w)o(e)h(will)i(sho)o(w)e(b)o(y)g (induction)i(that)639 1698 y Fg(N)676 1705 y Ff(k)710 1698 y Fm(=)c Fg(S)786 1705 y Ff(k)807 1698 y Fg(R)842 1679 y Fc(\000)p Ff(T)842 1712 y(k)897 1698 y Fg(D)935 1705 y Ff(k)956 1698 y Fg(R)991 1679 y Fc(\000)p Fk(1)991 1712 y Ff(k)1038 1698 y Fg(S)1069 1679 y Ff(T)1066 1709 y(k)1096 1698 y Fg(:)523 b Fm(\(2)p Fg(:)p Fm(12\))0 1786 y(This)13 b(is)g(true)g(for)f Fg(k)h Fm(=)g(1,)g(for)f(in)h(this)g(case)g(the)f(righ)o(t)h(hand)g(side)g (of)f(\(2.12\))f(is)i Fg(\032)1363 1793 y Fk(0)1382 1786 y Fg(s)1403 1793 y Fk(0)1423 1786 y Fg(s)1444 1770 y Ff(T)1444 1798 y Fk(0)1472 1786 y Fm(,)g(whic)o(h)g(equals)0 1843 y Fg(N)37 1850 y Fk(1)56 1843 y Fm(.)20 b(No)o(w)15 b(let)g(us)h(assume)f(that)f (\(2.12\))f(is)j(true)f(for)g Fg(k)q Fm(.)20 b(Then,)15 b(b)o(y)g(the)g (de\014nition)i(\(2.10\))d(of)h Fg(N)r(;)465 1931 y(N)502 1938 y Ff(k)q Fk(+1)581 1931 y Fm(=)e Fg(V)665 1912 y Ff(T)656 1942 y(k)693 1931 y Fg(S)721 1938 y Ff(k)742 1931 y Fg(R)777 1912 y Fc(\000)p Ff(T)777 1945 y(k)832 1931 y Fg(D)870 1938 y Ff(k)891 1931 y Fg(R)926 1912 y Fc(\000)p Fk(1)926 1945 y Ff(k)973 1931 y Fg(S)1004 1912 y Ff(T)1001 1942 y(k)1031 1931 y Fg(V)1058 1938 y Ff(k)1089 1931 y Fm(+)d Fg(\032)1158 1938 y Ff(k)1179 1931 y Fg(s)1200 1938 y Ff(k)1222 1931 y Fg(s)1243 1912 y Ff(T)1243 1942 y(k)1270 1931 y Fg(:)349 b Fm(\(2)p Fg(:)p Fm(13\))0 2019 y(T)l(o)15 b(simplify)i(this)f(expression,)f(w)o(e)g(note)g(from)g(\(1.3\))e (and)j(\(2.5\))d(that)402 2108 y Fg(R)437 2088 y Fc(\000)p Fk(1)437 2122 y Ff(k)484 2108 y Fg(S)515 2089 y Ff(T)512 2119 y(k)542 2108 y Fg(V)569 2115 y Ff(k)631 2108 y Fm(=)42 b Fg(R)743 2088 y Fc(\000)p Fk(1)743 2122 y Ff(k)790 2108 y Fg(S)821 2089 y Ff(T)818 2119 y(k)848 2108 y Fm(\()p Fg(I)13 b Fe(\000)e Fg(\032)969 2115 y Ff(k)990 2108 y Fg(y)1012 2115 y Ff(k)1033 2108 y Fg(s)1054 2089 y Ff(T)1054 2119 y(k)1082 2108 y Fm(\))631 2237 y(=)708 2190 y Fd(h)748 2237 y Fg(R)783 2218 y Fc(\000)p Fk(1)783 2251 y Ff(k)872 2237 y Fe(\000)p Fg(\032)931 2244 y Ff(k)952 2237 y Fg(R)987 2218 y Fc(\000)p Fk(1)987 2251 y Ff(k)1034 2237 y Fg(S)1065 2221 y Ff(T)1062 2250 y(k)1092 2237 y Fg(y)1114 2244 y Ff(k)1157 2190 y Fd(i)1184 2140 y(2)1184 2213 y(6)1184 2239 y(4)1232 2180 y Fg(S)1263 2164 y Ff(T)1260 2193 y(k)1237 2293 y Fg(s)1258 2277 y Ff(T)1258 2306 y(k)1311 2140 y Fd(3)1311 2213 y(7)1311 2239 y(5)631 2378 y Fm(=)708 2331 y Fd(h)748 2378 y Fg(R)783 2359 y Fc(\000)p Fk(1)783 2392 y Ff(k)872 2378 y Fe(\000)p Fg(\032)931 2385 y Ff(k)952 2378 y Fg(R)987 2359 y Fc(\000)p Fk(1)987 2392 y Ff(k)1034 2378 y Fg(S)1065 2362 y Ff(T)1062 2392 y(k)1092 2378 y Fg(y)1114 2385 y Ff(k)1157 2331 y Fd(i)1184 2378 y Fg(S)1215 2359 y Ff(T)1212 2389 y(k)q Fk(+1)631 2469 y Fm(=)708 2422 y Fd(h)748 2469 y Fg(I)46 b Fm(0)857 2422 y Fd(i)884 2469 y Fg(R)919 2450 y Fc(\000)p Fk(1)919 2483 y Ff(k)q Fk(+1)985 2469 y Fg(S)1016 2450 y Ff(T)1013 2480 y(k)q Fk(+1)1079 2469 y Fg(:)540 b Fm(\(2.14\))863 2603 y(5)p eop %%Page: 6 7 bop 0 120 a Fm(Also,)15 b(using)h(\(2.5\))e(w)o(e)h(can)g(write)g Fg(s)630 127 y Ff(k)667 120 y Fm(as)647 223 y Fg(s)668 230 y Ff(k)703 223 y Fm(=)e Fg(S)779 230 y Ff(k)q Fk(+1)845 223 y Fg(R)880 204 y Fc(\000)p Ff(T)880 237 y(k)q Fk(+1)946 223 y Fg(e)967 230 y Ff(k)q Fk(+1)1049 192 y Fm(1)p 1038 213 45 2 v 1038 254 a Fg(\032)1062 261 y Ff(k)1088 223 y Fg(:)531 b Fm(\(2)p Fg(:)p Fm(15\))0 329 y(Substituting)16 b(this)g(and)f(\(2.14\))f (in)i(\(2.13\),)d(w)o(e)i(ha)o(v)o(e)0 511 y Fg(N)37 518 y Ff(k)q Fk(+1)144 511 y Fm(=)42 b Fg(S)249 518 y Ff(k)q Fk(+1)316 511 y Fg(R)351 492 y Fc(\000)p Ff(T)351 525 y(k)q Fk(+1)424 439 y Fd(")469 482 y Fg(I)469 539 y Fm(0)513 439 y Fd(#)545 511 y Fg(D)583 518 y Ff(k)612 464 y Fd(h)652 511 y Fg(I)j Fm(0)761 464 y Fd(i)788 511 y Fg(R)823 492 y Fc(\000)p Fk(1)823 525 y Ff(k)q Fk(+1)889 511 y Fg(S)920 492 y Ff(T)917 522 y(k)q Fk(+1)993 511 y Fm(+)11 b Fg(S)1067 518 y Ff(k)q Fk(+1)1133 511 y Fg(R)1168 492 y Fc(\000)p Ff(T)1168 525 y(k)q Fk(+1)1241 376 y Fd(2)1241 449 y(6)1241 474 y(6)1241 499 y(6)1241 524 y(6)1241 551 y(4)1290 412 y Fm(0)1357 462 y(.)1374 475 y(.)1392 487 y(.)1449 548 y(0)1528 586 y Fk(1)p 1518 593 38 2 v 1518 620 a Ff(\032)1536 626 y Fb(k)1581 376 y Fd(3)1581 449 y(7)1581 474 y(7)1581 499 y(7)1581 524 y(7)1581 551 y(5)1616 511 y Fg(R)1651 492 y Fc(\000)p Fk(1)1651 525 y Ff(k)q Fk(+1)1717 511 y Fg(S)1748 492 y Ff(T)1745 522 y(k)q Fk(+1)144 742 y Fm(=)42 b Fg(S)249 749 y Ff(k)q Fk(+1)316 742 y Fg(R)351 723 y Fc(\000)p Ff(T)351 756 y(k)q Fk(+1)424 645 y Fd(2)424 718 y(6)424 744 y(4)473 683 y Fg(D)511 690 y Ff(k)585 683 y Fm(0)491 795 y(0)588 778 y Fk(1)p 578 785 V 578 811 a Ff(\032)596 817 y Fb(k)641 645 y Fd(3)641 718 y(7)641 744 y(5)676 742 y Fg(R)711 723 y Fc(\000)p Fk(1)711 756 y Ff(k)q Fk(+1)777 742 y Fg(S)808 723 y Ff(T)805 753 y(k)q Fk(+1)144 903 y Fm(=)g Fg(S)249 910 y Ff(k)q Fk(+1)316 903 y Fg(R)351 884 y Fc(\000)p Ff(T)351 917 y(k)q Fk(+1)417 903 y Fg(D)455 910 y Ff(k)q Fk(+1)521 903 y Fg(R)556 884 y Fc(\000)p Fk(1)556 917 y Ff(k)q Fk(+1)622 903 y Fg(S)653 884 y Ff(T)650 914 y(k)q Fk(+1)716 903 y Fg(:)0 987 y Fm(This)16 b(pro)o(v)o(es)e(\(2.12\))g(for)g Fg(k)d Fm(+)g(1.)71 1044 y(Finally)16 b(b)o(y)f(expanding)i(the)e(expression)185 1183 y Fg(H)223 1190 y Fk(0)253 1183 y Fm(+)298 1136 y Fd(h)339 1183 y Fg(S)367 1190 y Ff(k)429 1183 y Fg(H)467 1190 y Fk(0)487 1183 y Fg(Y)513 1190 y Ff(k)555 1136 y Fd(i)582 1086 y(2)582 1159 y(6)582 1185 y(4)631 1127 y Fg(R)666 1108 y Fc(\000)p Ff(T)666 1141 y(k)721 1127 y Fm(\()p Fg(D)777 1134 y Ff(k)807 1127 y Fm(+)c Fg(Y)889 1110 y Ff(T)879 1140 y(k)917 1127 y Fg(H)955 1134 y Fk(0)974 1127 y Fg(Y)1000 1134 y Ff(k)1022 1127 y Fm(\))p Fg(R)1075 1108 y Fc(\000)p Fk(1)1075 1141 y Ff(k)1163 1127 y Fe(\000)p Fg(R)1233 1108 y Fc(\000)p Ff(T)1233 1141 y(k)818 1241 y Fe(\000)p Fg(R)888 1222 y Fc(\000)p Fk(1)888 1255 y Ff(k)1215 1241 y Fm(0)1309 1086 y Fd(3)1309 1159 y(7)1309 1185 y(5)1345 1086 y(2)1345 1159 y(6)1345 1185 y(4)1425 1126 y Fg(S)1456 1110 y Ff(T)1453 1139 y(k)1393 1239 y Fg(Y)1429 1223 y Ff(T)1419 1252 y(k)1457 1239 y Fg(H)1495 1246 y Fk(0)1535 1086 y Fd(3)1535 1159 y(7)1535 1185 y(5)0 1324 y Fm(w)o(e)k(see)g(that)g(it)g (is)h(equal)g(to)f Fg(M)550 1331 y Ff(k)581 1324 y Fm(+)c Fg(N)664 1331 y Ff(k)684 1324 y Fm(,)k(where)h Fg(M)888 1331 y Ff(k)924 1324 y Fm(and)g Fg(N)1050 1331 y Ff(k)1086 1324 y Fm(are)e(giv)o(en)i(b)o(y)f (\(2.11\))f(and)h(\(2.12\).)1714 1381 y Fa(2)0 1456 y Fm(Note)g(that)f(the)i (conditions)g Fg(s)525 1440 y Ff(T)525 1468 y(i)553 1456 y Fg(y)575 1463 y Ff(i)602 1456 y Fg(>)d Fm(0)25 b Fg(i)12 b Fm(=)h(0)p Fg(;)8 b(:::;)g(k)g Fe(\000)j Fm(1)k(ensure)h(that)e Fg(R)1269 1463 y Ff(k)1305 1456 y Fm(is)i(nonsingular,)g(so)f(that)0 1512 y(\(2.6\))h(is)h(w)o(ell)i(de\014ned.)27 b(Indeed)19 b(it)e(is)h(w)o (ell)g(kno)o(wn)f(\(Fletc)o(her)h(\(1987\)\))d(that)h(the)i(BF)o(GS)e(form)o (ula)0 1569 y(preserv)o(es)f(p)q(ositiv)o(e)h(de\014niteness)h(if)f Fg(s)669 1552 y Ff(T)669 1581 y(i)697 1569 y Fg(y)719 1576 y Ff(i)746 1569 y Fg(>)d Fm(0)i(for)f(all)i Fg(i)p Fm(.)71 1625 y(Theorem)h(2.2)f(giv)o(es)i(us)g(a)f(matrix)g(represen)o(tation)g(of)g (the)h(in)o(v)o(erse)g(Hessian)g(appro)o(ximation)0 1682 y Fg(H)38 1689 y Ff(k)59 1682 y Fm(.)24 b(W)l(e)17 b(no)o(w)f(presen)o(t)g(an)h (analogous)f(expression)h(for)f(the)h(direct)g(Hessian)g(appro)o(ximation)g Fg(B)1714 1689 y Ff(k)1735 1682 y Fm(.)0 1738 y(The)e(direct)h(BF)o(GS)f(up)q (date)h(form)o(ula,)e(i.e.)21 b(the)15 b(in)o(v)o(erse)g(of)g(\(1.2\))f(is)h (giv)o(en)h(b)o(y)544 1851 y Fg(B)578 1858 y Ff(k)q Fk(+1)658 1851 y Fm(=)d Fg(B)740 1858 y Ff(k)772 1851 y Fe(\000)822 1821 y Fg(B)856 1828 y Ff(k)878 1821 y Fg(s)899 1828 y Ff(k)921 1821 y Fg(s)942 1804 y Ff(T)942 1834 y(k)970 1821 y Fg(B)1004 1828 y Ff(k)p 822 1841 204 2 v 850 1884 a Fg(s)871 1868 y Ff(T)871 1898 y(k)899 1884 y Fg(B)933 1891 y Ff(k)955 1884 y Fg(s)976 1891 y Ff(k)1040 1851 y Fm(+)1091 1821 y Fg(y)1113 1828 y Ff(k)1135 1821 y Fg(y)1159 1804 y Ff(T)1157 1834 y(k)p 1091 1841 96 2 v 1091 1884 a Fg(y)1115 1868 y Ff(T)1113 1898 y(k)1143 1884 y Fg(s)1164 1891 y Ff(k)1191 1851 y Fg(:)428 b Fm(\(2)p Fg(:)p Fm(16\))0 1961 y Fn(Theorem)17 b(2.3)22 b Fl(L)n(et)e Fg(B)428 1968 y Fk(0)468 1961 y Fl(b)n(e)f(symmetric)h(and)g(p)n(ositive)g(de\014nite) f(and)h(assume)f(that)i(the)f Fg(k)h Fl(p)n(airs)0 2018 y Fe(f)p Fg(s)44 2025 y Ff(i)58 2018 y Fg(;)8 b(y)101 2025 y Ff(i)115 2018 y Fe(g)138 1999 y Ff(k)q Fc(\000)p Fk(1)138 2031 y Ff(i)p Fk(=0)220 2018 y Fl(satisfy)16 b Fg(s)383 2001 y Ff(T)383 2030 y(i)411 2018 y Fg(y)433 2025 y Ff(i)461 2018 y Fg(>)d Fm(0)p Fl(.)22 b(L)n(et)16 b Fg(B)681 2025 y Ff(k)719 2018 y Fl(b)n(e)g(obtaine)n(d) g(by)h(up)n(dating)g Fg(B)1233 2025 y Fk(0)1280 2018 y Fg(k)g Fl(times)g(using)f(the)g(dir)n(e)n(ct)0 2074 y(BF)o(GS)f(formula)j(\(2.16\))e (and)g(the)g(p)n(airs)g Fe(f)p Fg(s)763 2081 y Ff(i)778 2074 y Fg(;)8 b(y)821 2081 y Ff(i)834 2074 y Fe(g)857 2055 y Ff(k)q Fc(\000)p Fk(1)857 2087 y Ff(i)p Fk(=0)923 2074 y Fl(.)21 b(Then)285 2201 y Fg(B)319 2208 y Ff(k)353 2201 y Fm(=)13 b Fg(B)435 2208 y Fk(0)465 2201 y Fe(\000)511 2154 y Fd(h)551 2201 y Fg(B)585 2208 y Fk(0)605 2201 y Fg(S)633 2208 y Ff(k)696 2201 y Fg(Y)722 2208 y Ff(k)764 2154 y Fd(i)792 2129 y(")837 2173 y Fg(S)868 2156 y Ff(T)865 2186 y(k)895 2173 y Fg(B)929 2180 y Fk(0)949 2173 y Fg(S)977 2180 y Ff(k)1061 2173 y Fg(L)1092 2180 y Ff(k)888 2229 y Fg(L)919 2212 y Ff(T)919 2242 y(k)1040 2229 y Fe(\000)p Fg(D)1113 2236 y Ff(k)1155 2129 y Fd(#)1179 2140 y Fc(\000)p Fk(1)1234 2129 y Fd(")1279 2173 y Fg(S)1310 2156 y Ff(T)1307 2186 y(k)1337 2173 y Fg(B)1371 2180 y Fk(0)1303 2229 y Fg(Y)1339 2212 y Ff(T)1329 2242 y(k)1411 2129 y Fd(#)1443 2201 y Fg(;)176 b Fm(\(2)p Fg(:)p Fm(17\))0 2315 y Fl(wher)n(e)16 b Fg(L)159 2322 y Ff(k)197 2315 y Fl(is)g(the)g Fg(k)11 b Fe(\002)g Fg(k)17 b Fl(matrix)533 2428 y Fm(\()p Fg(L)582 2435 y Ff(k)603 2428 y Fm(\))621 2435 y Ff(i;j)674 2428 y Fm(=)722 2356 y Fd(\()776 2399 y Fg(s)797 2383 y Ff(T)797 2411 y(i)p Fc(\000)p Fk(1)856 2399 y Fg(y)878 2406 y Ff(j)r Fc(\000)p Fk(1)983 2399 y Fl(if)g Fg(i)12 b(>)h(j)776 2456 y Fm(0)184 b Fl(otherwise)1202 2428 y Fg(:)417 b Fm(\(2)p Fg(:)p Fm(18\))863 2603 y(6)p eop %%Page: 7 8 bop 0 120 a Fn(Pro)q(of.)20 b Fm(Let)15 b(us)g(write)h(\(2.6\))d(as)663 177 y Fg(H)701 184 y Ff(k)734 177 y Fm(=)g Fg(H)820 184 y Fk(0)850 177 y Fm(+)d Fg(U)926 184 y Ff(k)948 177 y Fg(C)981 184 y Ff(k)1002 177 y Fg(U)1038 158 y Ff(T)1033 188 y(k)1065 177 y Fg(;)554 b Fm(\(2)p Fg(:)p Fm(19\))0 260 y(where)669 316 y Fg(U)700 323 y Ff(k)734 316 y Fm(=)782 269 y Fd(h)822 316 y Fg(S)850 323 y Ff(k)913 316 y Fg(H)951 323 y Fk(0)971 316 y Fg(Y)997 323 y Ff(k)1039 269 y Fd(i)1066 316 y Fg(;)0 405 y Fm(and)433 483 y Fg(C)466 490 y Ff(k)499 483 y Fm(=)547 411 y Fd(")592 455 y Fg(R)627 436 y Fc(\000)p Ff(T)627 469 y(k)682 455 y Fm(\()p Fg(D)738 462 y Ff(k)769 455 y Fm(+)10 b Fg(Y)851 439 y Ff(T)840 468 y(k)878 455 y Fg(H)916 462 y Fk(0)936 455 y Fg(Y)962 462 y Ff(k)984 455 y Fm(\))p Fg(R)1037 436 y Fc(\000)p Fk(1)1037 469 y Ff(k)1125 455 y Fe(\000)p Fg(R)1195 436 y Fc(\000)p Ff(T)1195 469 y(k)779 513 y Fe(\000)p Fg(R)849 494 y Fc(\000)p Fk(1)849 527 y Ff(k)1176 513 y Fm(0)1271 411 y Fd(#)1302 483 y Fg(:)0 597 y Fm(By)15 b(direct)h(m)o(ultiplication)i(w)o(e)d(can)g(v)o(erify)h(that) e(the)h(in)o(v)o(erse)h(of)f Fg(C)1181 604 y Ff(k)1217 597 y Fm(is)500 729 y Fg(C)536 710 y Fc(\000)p Fk(1)533 743 y Ff(k)596 729 y Fm(=)644 657 y Fd(")726 701 y Fm(0)211 b Fe(\000)p Fg(R)1030 708 y Ff(k)689 757 y Fe(\000)p Fg(R)759 740 y Ff(T)759 770 y(k)828 757 y Fe(\000)p Fm(\()p Fg(D)919 764 y Ff(k)950 757 y Fm(+)11 b Fg(Y)1032 740 y Ff(T)1022 770 y(k)1060 757 y Fg(H)1098 764 y Fk(0)1117 757 y Fg(Y)1143 764 y Ff(k)1165 757 y Fm(\))1203 657 y Fd(#)1235 729 y Fg(:)384 b Fm(\(2)p Fg(:)p Fm(20\))0 864 y(Applying)18 b(the)e(Sherman-Morrison-W)l(o)q(o)q(dbury)h(form)o(ula)e (\(Ortega)g(and)h(Rhein)o(b)q(oldt)j(\(1970\)\))14 b(to)0 920 y(\(2.19\))f(w)o(e)i(obtain)400 1022 y Fg(B)434 1029 y Ff(k)498 1022 y Fm(=)41 b Fg(B)608 1029 y Fk(0)639 1022 y Fe(\000)10 b Fg(B)718 1029 y Fk(0)738 1022 y Fg(U)769 1029 y Ff(k)791 1022 y Fm(\()p Fg(I)j Fm(+)e Fg(C)921 1029 y Ff(k)941 1022 y Fg(U)977 1003 y Ff(T)972 1033 y(k)1005 1022 y Fg(B)1039 1029 y Fk(0)1059 1022 y Fg(U)1090 1029 y Ff(k)1111 1022 y Fm(\))1129 1003 y Fc(\000)p Fk(1)1176 1022 y Fg(C)1209 1029 y Ff(k)1230 1022 y Fg(U)1266 1003 y Ff(T)1261 1033 y(k)1294 1022 y Fg(B)1328 1029 y Fk(0)498 1091 y Fm(=)41 b Fg(B)608 1098 y Fk(0)639 1091 y Fe(\000)10 b Fg(B)718 1098 y Fk(0)738 1091 y Fg(U)769 1098 y Ff(k)791 1091 y Fm(\()p Fg(C)845 1072 y Fc(\000)p Fk(1)842 1105 y Ff(k)901 1091 y Fm(+)h Fg(U)983 1072 y Ff(T)978 1102 y(k)1010 1091 y Fg(B)1044 1098 y Fk(0)1065 1091 y Fg(U)1096 1098 y Ff(k)1117 1091 y Fm(\))1135 1072 y Fc(\000)p Fk(1)1182 1091 y Fg(U)1218 1072 y Ff(T)1213 1102 y(k)1245 1091 y Fg(B)1279 1098 y Fk(0)1299 1091 y Fg(:)320 b Fm(\(2.21\))0 1193 y(No)o(w)447 1317 y Fg(U)483 1298 y Ff(T)478 1328 y(k)511 1317 y Fg(B)545 1324 y Fk(0)565 1317 y Fg(U)596 1324 y Ff(k)659 1317 y Fm(=)736 1245 y Fd(")812 1288 y Fg(S)843 1272 y Ff(T)840 1302 y(k)781 1345 y Fg(Y)817 1328 y Ff(T)807 1358 y(k)845 1345 y Fg(H)883 1352 y Fk(0)923 1245 y Fd(#)955 1317 y Fg(B)989 1324 y Fk(0)1017 1270 y Fd(h)1057 1317 y Fg(S)1085 1324 y Ff(k)1148 1317 y Fg(H)1186 1324 y Fk(0)1205 1317 y Fg(Y)1231 1324 y Ff(k)1274 1270 y Fd(i)659 1458 y Fm(=)736 1386 y Fd(")781 1429 y Fg(S)812 1413 y Ff(T)809 1443 y(k)839 1429 y Fg(B)873 1436 y Fk(0)893 1429 y Fg(S)921 1436 y Ff(k)1015 1429 y Fg(S)1046 1413 y Ff(T)1043 1443 y(k)1073 1429 y Fg(Y)1099 1436 y Ff(k)805 1486 y Fg(Y)841 1469 y Ff(T)831 1499 y(k)869 1486 y Fg(S)897 1493 y Ff(k)984 1486 y Fg(Y)1020 1469 y Ff(T)1010 1499 y(k)1048 1486 y Fg(H)1086 1493 y Fk(0)1105 1486 y Fg(Y)1131 1493 y Ff(k)1174 1386 y Fd(#)1205 1458 y Fg(:)0 1593 y Fm(Therefore)15 b(using)h(\(2.20\))389 1727 y Fg(C)425 1708 y Fc(\000)p Fk(1)422 1741 y Ff(k)482 1727 y Fm(+)11 b Fg(U)564 1709 y Ff(T)559 1739 y(k)591 1727 y Fg(B)625 1734 y Fk(0)646 1727 y Fg(U)677 1734 y Ff(k)711 1727 y Fm(=)759 1655 y Fd(")838 1699 y Fg(S)869 1683 y Ff(T)866 1712 y(k)896 1699 y Fg(B)930 1706 y Fk(0)951 1699 y Fg(S)979 1706 y Ff(k)1076 1699 y Fg(S)1107 1683 y Ff(T)1104 1712 y(k)1134 1699 y Fg(Y)1160 1706 y Ff(k)1192 1699 y Fe(\000)g Fg(R)1273 1706 y Ff(k)804 1756 y Fg(Y)840 1739 y Ff(T)830 1769 y(k)868 1756 y Fg(S)896 1763 y Ff(k)927 1756 y Fe(\000)f Fg(R)1007 1739 y Ff(T)1007 1769 y(k)1138 1756 y Fe(\000)p Fg(D)1211 1763 y Ff(k)1314 1655 y Fd(#)1346 1727 y Fg(:)0 1862 y Fm(Note)15 b(that)f(the)i(matrix)e Fg(L)465 1869 y Ff(k)502 1862 y Fm(de\014ned)i(b)o(y)g(\(2.18\))d(can)i(b)q (e)h(written)f(as)699 1964 y Fg(L)730 1971 y Ff(k)764 1964 y Fm(=)e Fg(S)843 1946 y Ff(T)840 1976 y(k)870 1964 y Fg(Y)896 1971 y Ff(k)928 1964 y Fe(\000)d Fg(R)1008 1971 y Ff(k)1029 1964 y Fg(;)590 b Fm(\(2)p Fg(:)p Fm(22\))0 2066 y(so)15 b(that)486 2144 y Fg(C)522 2125 y Fc(\000)p Fk(1)519 2158 y Ff(k)579 2144 y Fm(+)10 b Fg(U)660 2125 y Ff(T)655 2155 y(k)688 2144 y Fg(B)722 2151 y Fk(0)742 2144 y Fg(U)773 2151 y Ff(k)807 2144 y Fm(=)855 2072 y Fd(")900 2116 y Fg(S)931 2099 y Ff(T)928 2129 y(k)958 2116 y Fg(B)992 2123 y Fk(0)1012 2116 y Fg(S)1040 2123 y Ff(k)1124 2116 y Fg(L)1155 2123 y Ff(k)951 2172 y Fg(L)982 2156 y Ff(T)982 2186 y(k)1103 2172 y Fe(\000)p Fg(D)1176 2179 y Ff(k)1218 2072 y Fd(#)1250 2144 y Fg(:)369 b Fm(\(2)p Fg(:)p Fm(23\))0 2260 y(Substituting)16 b(this)g(in)o(to)f(\(2.21\))e(w)o(e)i(obtain)h(\(2.17\).) 1714 2317 y Fa(2)0 2398 y Fm(In)h(the)f(next)g(sections)g(w)o(e)g(will)i(sho) o(w)d(that)g(the)i(new)f(form)o(ulae)g(\(2.17\))e(and)i(\(2.6\),)e(whic)o(h)j (at)e(\014rst)0 2455 y(app)q(ear)g(rather)g(cum)o(b)q(ersome,)g(are)f (actually)i(v)o(ery)f(con)o(v)o(enien)o(t)h(for)e(some)h(calculations)h (arising)g(in)863 2603 y(7)p eop %%Page: 8 9 bop 0 120 a Fm(constrained)16 b(optimization.)23 b(Before)16 b(doing)h(so)e(w)o(e)h(mak)o(e)f(a)h(remark)f(concerning)i(the)f(implemen-)0 177 y(tation)f(of)g(\(2.17\).)71 233 y(The)g(middle)i(matrix)e(in)h (\(2.17\),)680 290 y Fd(")725 333 y Fg(S)756 317 y Ff(T)753 347 y(k)783 333 y Fg(B)817 340 y Fk(0)838 333 y Fg(S)866 340 y Ff(k)949 333 y Fg(L)980 340 y Ff(k)777 390 y Fg(L)808 373 y Ff(T)808 403 y(k)928 390 y Fe(\000)p Fg(D)1001 397 y Ff(k)1043 290 y Fd(#)1632 362 y Fm(\(2)p Fg(:)p Fm(24\))0 488 y(is)23 b(inde\014nite.)44 b(Ho)o(w)o(ev)o(er)21 b(w)o(e)h(no)o(w)g(sho)o(w)g(that)f (its)i(in)o(v)o(ersion)g(can)g(b)q(e)g(carried)g(out)f(using)h(the)0 544 y(Cholesky)d(factorization)f(of)g(a)f(related)i(matrix.)32 b(First)18 b(w)o(e)h(re-order)h(the)f(blo)q(c)o(ks)h(of)e(\(2.24\))g(and)0 601 y(note)d(that)164 649 y Fd(")209 693 y Fe(\000)p Fg(D)282 700 y Ff(k)397 693 y Fg(L)428 677 y Ff(T)428 706 y(k)230 749 y Fg(L)261 756 y Ff(k)345 749 y Fg(S)376 733 y Ff(T)373 763 y(k)403 749 y Fg(B)437 756 y Fk(0)457 749 y Fg(S)485 756 y Ff(k)527 649 y Fd(#)564 721 y Fm(=)612 649 y Fd(")715 693 y Fg(D)754 669 y Fk(1)p Ff(=)p Fk(2)753 707 y Ff(k)920 693 y Fm(0)657 759 y Fe(\000)p Fg(L)723 766 y Ff(k)745 759 y Fg(D)784 735 y Fc(\000)p Fk(1)p Ff(=)p Fk(2)783 773 y Ff(k)908 759 y Fg(J)933 766 y Ff(k)975 649 y Fd(#)8 b(")1052 698 y Fe(\000)p Fg(D)1126 674 y Fk(1)p Ff(=)p Fk(2)1125 712 y Ff(k)1222 698 y Fg(D)1261 674 y Fc(\000)p Fk(1)p Ff(=)p Fk(2)1260 712 y Ff(k)1344 698 y Fg(L)1375 681 y Ff(T)1375 711 y(k)1105 754 y Fm(0)156 b Fg(J)1313 738 y Ff(T)1309 767 y(k)1423 649 y Fd(#)1455 721 y Fg(;)164 b Fm(\(2)p Fg(:)p Fm(25\))0 851 y(where)15 b Fg(J)156 858 y Ff(k)193 851 y Fm(is)h(the)f(lo)o(w)o(er)g(triangular)g(matrix)g(that)g (satis\014es)579 947 y Fg(J)604 954 y Ff(k)625 947 y Fg(J)654 928 y Ff(T)650 958 y(k)695 947 y Fm(=)e Fg(S)774 928 y Ff(T)771 958 y(k)801 947 y Fg(B)835 954 y Fk(0)855 947 y Fg(S)883 954 y Ff(k)914 947 y Fm(+)e Fg(L)991 954 y Ff(k)1012 947 y Fg(D)1051 928 y Fc(\000)p Fk(1)1050 961 y Ff(k)1098 947 y Fg(L)1129 928 y Ff(T)1129 958 y(k)1157 947 y Fg(:)462 b Fm(\(2)p Fg(:)p Fm(26\))0 1043 y(The)15 b(follo)o(wing)h(result)g(sho)o(ws)f(that)f Fg(J)666 1050 y Ff(k)703 1043 y Fm(exists)h(and)h(is)f(nonsingular.)0 1142 y Fn(Theorem)i(2.4)22 b Fl(If)d Fg(B)396 1149 y Fk(0)435 1142 y Fl(is)g(p)n(ositive)f(de\014nite)h(and)g Fg(s)930 1125 y Ff(T)930 1154 y(i)958 1142 y Fg(y)980 1149 y Ff(i)1012 1142 y Fg(>)f Fm(0)p Fg(;)8 b(i)16 b Fm(=)i(0)p Fg(;)8 b(:::;)g(k)i Fe(\000)j Fm(1)p Fl(,)19 b(then)g(the)g(matrix)0 1198 y Fg(S)31 1182 y Ff(T)28 1211 y(k)58 1198 y Fg(B)92 1205 y Fk(0)112 1198 y Fg(S)140 1205 y Ff(k)172 1198 y Fm(+)10 b Fg(L)248 1205 y Ff(k)269 1198 y Fg(D)308 1179 y Fc(\000)p Fk(1)307 1212 y Ff(k)355 1198 y Fg(L)386 1182 y Ff(T)386 1211 y(k)430 1198 y Fl(is)16 b(p)n(ositive)g(de\014nite.)0 1320 y Fn(Pro)q(of.)j Fm(F)l(rom)12 b(the)i(de\014nition)g(\(2.7\))e(w)o(e)h(see)g(that)g Fg(D)932 1327 y Ff(k)966 1320 y Fm(is)g(p)q(ositiv)o(e)h(de\014nite)h(and)e(hence)h Fg(S)1576 1304 y Ff(T)1573 1333 y(k)1603 1320 y Fg(B)1637 1327 y Fk(0)1658 1320 y Fg(S)1686 1327 y Ff(k)1713 1320 y Fm(+)0 1376 y Fg(L)31 1383 y Ff(k)52 1376 y Fg(D)91 1357 y Fc(\000)p Fk(1)90 1391 y Ff(k)138 1376 y Fg(L)169 1360 y Ff(T)169 1390 y(k)209 1376 y Fm(is)f(p)q(ositiv)o(e)g(semi-de\014nite.)21 b(Supp)q(ose)14 b(that)d Fg(u)996 1360 y Ff(T)1024 1376 y Fm(\()p Fg(S)1073 1360 y Ff(T)1070 1390 y(k)1099 1376 y Fg(B)1133 1383 y Fk(0)1154 1376 y Fg(S)1182 1383 y Ff(k)1207 1376 y Fm(+)t Fg(L)1277 1383 y Ff(k)1299 1376 y Fg(D)1338 1357 y Fc(\000)p Fk(1)1337 1391 y Ff(k)1385 1376 y Fg(L)1416 1360 y Ff(T)1416 1390 y(k)1443 1376 y Fm(\))p Fg(u)i Fm(=)g(0)f(for)f(some)0 1433 y(v)o(ector)k Fg(u)p Fm(.)20 b(Then)15 b Fg(L)345 1416 y Ff(T)345 1446 y(k)373 1433 y Fg(u)e Fm(=)g(0)i(and)g Fg(S)614 1440 y Ff(k)635 1433 y Fg(u)e Fm(=)g(0,)i(whic)o(h)h(in)g(turn)f(implies)i (that)e Fg(Y)1346 1416 y Ff(T)1336 1446 y(k)1374 1433 y Fg(S)1402 1440 y Ff(k)1423 1433 y Fg(u)e Fm(=)g(0.)20 b(Recalling)0 1489 y(\(2.22\))13 b(w)o(e)i(ha)o(v)o(e)f Fg(Y)339 1473 y Ff(T)328 1503 y(k)367 1489 y Fg(S)395 1496 y Ff(k)428 1489 y Fm(=)f Fg(L)507 1473 y Ff(T)507 1503 y(k)545 1489 y Fm(+)d Fg(R)625 1473 y Ff(T)625 1503 y(k)652 1489 y Fm(,)15 b(so)f(that)h Fg(R)869 1473 y Ff(T)869 1503 y(k)896 1489 y Fg(u)e Fm(=)g(0.)19 b(Since)e Fg(R)1192 1473 y Ff(T)1192 1503 y(k)1234 1489 y Fm(is)e(triangular)h(with)f (p)q(ositiv)o(e)0 1546 y(diagonal,)g(w)o(e)g(conclude)i(that)e Fg(u)d Fm(=)h(0.)1714 1602 y Fa(2)0 1682 y Fm(Therefore,)f(only)h(the)g (Cholesky)g(factorization)f(of)g(the)g Fg(k)6 b Fe(\002)t Fg(k)14 b Fm(symmetric)e(p)q(ositiv)o(e)i(de\014nite)f(matrix)0 1738 y Fg(S)31 1722 y Ff(T)28 1751 y(k)58 1738 y Fg(B)92 1745 y Fk(0)112 1738 y Fg(S)140 1745 y Ff(k)173 1738 y Fm(+)e Fg(L)250 1745 y Ff(k)272 1738 y Fg(D)311 1719 y Fc(\000)p Fk(1)310 1752 y Ff(k)358 1738 y Fg(L)389 1722 y Ff(T)389 1751 y(k)433 1738 y Fm(needs)18 b(to)e(b)q(e)i(computed,)f(to)g(implemen)o(t)h(\(2.17\).)24 b(This)17 b(is)h(preferable)g(to)0 1794 y(factorizing)g(the)f(inde\014nite)i (2)p Fg(k)12 b Fe(\002)g Fm(2)p Fg(k)18 b Fm(matrix)f(\(2.24\).)24 b(W)l(e)17 b(will)i(discuss)f(the)f(implemen)o(tation)h(of)0 1851 y(\(2.17\))13 b(in)j(more)f(detail)h(in)g(section)g(3.2,)e(in)i(the)f (con)o(text)g(of)g(limited)i(memory)d(metho)q(ds.)0 1999 y Fh(3.)k(Application)g(to)h(the)f(Limited)e(Memory)g(Metho)r(d.)71 2078 y Fm(Since)g(w)o(e)f(kno)o(w)g(that)g Fg(k)h Fm(BF)o(GS)f(up)q(dates)h (can)g(b)q(e)g(written)f(in)h(the)g(compact)f(forms)f(\(2.6\))g(and)0 2134 y(\(2.17\),)k(it)h(is)g(easy)g(to)f(describ)q(e)j(a)d(limited)j(memory)d (implemen)o(tation.)33 b(W)l(e)19 b(k)o(eep)g(the)g Fg(m)g Fm(most)0 2191 y(recen)o(t)c(correction)g(pairs)g Fe(f)p Fg(s)502 2198 y Ff(i)516 2191 y Fg(;)8 b(y)559 2198 y Ff(i)573 2191 y Fe(g)14 b Fm(to)g(implicitly)k(de\014ne)e(the)f(iteration)g(matrix.)k(This) d(set)e(of)h(pairs)0 2247 y(is)g(refreshed)g(at)e(ev)o(ery)h(iteration)h(b)o (y)f(remo)o(ving)g(the)h(oldest)g(pair)f(and)h(adding)g(a)f(newly)h (generated)0 2304 y(pair.)26 b(W)l(e)17 b(assume)g(that)f Fg(m)h Fm(is)g(constan)o(t,)g(but)g(it)g(is)h(not)e(di\016cult)j(to)d(adapt)h(all)h (the)f(form)o(ulae)f(of)0 2360 y(this)g(section)f(to)g(the)g(case)g(when)h Fg(m)f Fm(c)o(hanges)g(at)g(ev)o(ery)g(iteration.)71 2417 y(Supp)q(ose)c (that)f(at)f(the)i(curren)o(t)f(iterate)g Fg(x)780 2424 y Ff(k)812 2417 y Fm(w)o(e)g(wish)h(to)f(construct)g(the)g(in)o(v)o(erse)h(limited)i (memory)0 2479 y(BF)o(GS)f(matrix)h Fg(H)318 2486 y Ff(k)339 2479 y Fm(.)19 b(W)l(e)12 b(do)h(so)f(b)o(y)h(implicitly)j(up)q(dating)e Fg(H)1055 2455 y Fk(\(0\))1051 2493 y Ff(k)1101 2479 y Fm(,)f(the)g(basic)g (matrix,)g Fg(m)f Fm(times)h(using)863 2603 y(8)p eop %%Page: 9 10 bop 0 120 a Fm(the)17 b(2)p Fg(m)g Fm(v)o(ectors)f Fe(f)p Fg(s)360 127 y Ff(k)q Fc(\000)p Ff(m)440 120 y Fg(;)8 b(:)g(:)g(:)d(;)j(s)563 127 y Ff(k)q Fc(\000)p Fk(1)629 120 y Fe(g)17 b Fm(and)g Fe(f)p Fg(y)804 127 y Ff(k)q Fc(\000)p Ff(m)884 120 y Fg(;)8 b(:)g(:)g(:)d(;)j(y) 1008 127 y Ff(k)q Fc(\000)p Fk(1)1074 120 y Fe(g)p Fm(,)17 b(whic)o(h)h(ha)o(v)o(e)f(b)q(een)h(sa)o(v)o(ed.)25 b(Let)18 b(us)0 184 y(assume)13 b(that)f Fg(H)293 161 y Fk(\(0\))289 199 y Ff(k)352 184 y Fm(=)h Fg(\015)424 191 y Ff(k)445 184 y Fg(I)t Fm(,)g(for)f(some)h(p)q(ositiv)o(e)h(scalar)f Fg(\015)991 191 y Ff(k)1012 184 y Fm(.)19 b(F)l(rom)12 b(\(2.6\))g(w)o(e)g(see)i(that)e (the)h(resulting)0 241 y(matrix)i(is)78 389 y Fg(H)116 396 y Ff(k)150 389 y Fm(=)e Fg(\015)222 396 y Ff(k)243 389 y Fg(I)g Fm(+)322 342 y Fd(h)362 389 y Fg(S)390 396 y Ff(k)453 389 y Fg(\015)477 396 y Ff(k)498 389 y Fg(Y)524 396 y Ff(k)566 342 y Fd(i)593 292 y(2)593 365 y(6)593 392 y(4)642 333 y Fg(R)677 314 y Fc(\000)p Ff(T)677 347 y(k)732 333 y Fm(\()p Fg(D)788 340 y Ff(k)818 333 y Fm(+)e Fg(\015)888 340 y Ff(k)909 333 y Fg(Y)945 317 y Ff(T)935 347 y(k)973 333 y Fg(Y)999 340 y Ff(k)1021 333 y Fm(\))p Fg(R)1074 314 y Fc(\000)p Fk(1)1074 347 y Ff(k)1162 333 y Fe(\000)p Fg(R)1232 314 y Fc(\000)p Ff(T)1232 347 y(k)822 447 y Fe(\000)p Fg(R)892 428 y Fc(\000)p Fk(1)892 461 y Ff(k)1213 447 y Fm(0)1308 292 y Fd(3)1308 365 y(7)1308 392 y(5)1343 292 y(2)1343 365 y(6)1343 392 y(4)1417 333 y Fg(S)1448 316 y Ff(T)1445 346 y(k)1391 446 y Fg(\015)1415 453 y Ff(k)1436 446 y Fg(Y)1473 429 y Ff(T)1462 459 y(k)1521 292 y Fd(3)1521 365 y(7)1521 392 y(5)1556 389 y Fg(;)86 b Fm(\(3)p Fg(:)p Fm(1\))0 546 y(where)15 b(no)o(w)346 603 y Fg(S)374 610 y Ff(k)408 603 y Fm(=)e([)o Fg(s)489 610 y Ff(k)q Fc(\000)p Ff(m)570 603 y Fg(;)8 b(:)g(:)g(:)t(;)g(s)692 610 y Ff(k)q Fc(\000)p Fk(1)758 603 y Fm(])g Fg(;)157 b(Y)975 610 y Ff(k)1009 603 y Fm(=)13 b([)p Fg(y)1092 610 y Ff(k)q Fc(\000)p Ff(m)1172 603 y Fg(;)8 b(:)g(:)g(:)d(;)j(y)1296 610 y Ff(k)q Fc(\000)p Fk(1)1362 603 y Fm(])f Fg(;)260 b Fm(\(3)p Fg(:)p Fm(2\))0 686 y(and)15 b(where)h Fg(R)255 693 y Ff(k)291 686 y Fm(and)f Fg(D)417 693 y Ff(k)453 686 y Fm(are)g(the)g Fg(m)10 b Fe(\002)h Fg(m)k Fm(matrices)372 817 y(\()p Fg(R)425 824 y Ff(k)446 817 y Fm(\))464 824 y Ff(i;j)516 817 y Fm(=)564 745 y Fd(\()618 789 y Fm(\()p Fg(s)657 796 y Ff(k)q Fc(\000)p Ff(m)p Fc(\000)p Fk(1+)p Ff(i)822 789 y Fm(\))840 772 y Ff(T)867 789 y Fm(\()p Fg(y)907 796 y Ff(k)q Fc(\000)p Ff(m)p Fc(\000)p Fk(1+)p Ff(j)1076 789 y Fm(\))41 b(if)16 b Fg(i)c Fe(\024)h Fg(j)618 845 y Fm(0)494 b(otherwise)1356 817 y Fg(;)286 b Fm(\(3)p Fg(:)p Fm(3\))0 950 y(and)497 1006 y Fg(D)535 1013 y Ff(k)569 1006 y Fm(=)12 b Fg(diag)711 959 y Fd(h)730 1006 y Fg(s)751 987 y Ff(T)751 1017 y(k)q Fc(\000)p Ff(m)832 1006 y Fg(y)854 1013 y Ff(k)q Fc(\000)p Ff(m)934 1006 y Fg(;)c(:)g(:)g(:)d(;)j(s)1057 987 y Ff(T)1057 1017 y(k)q Fc(\000)p Fk(1)1123 1006 y Fg(y)1145 1013 y Ff(k)q Fc(\000)p Fk(1)1211 959 y Fd(i)1238 1006 y Fg(:)404 b Fm(\(3)p Fg(:)p Fm(4\))0 1095 y(After)17 b(the)g(new)g(iterate)f Fg(x)467 1102 y Ff(k)q Fk(+1)551 1095 y Fm(is)h(generated,)g(w)o(e)g(obtain)g Fg(S)1060 1102 y Ff(k)q Fk(+1)1143 1095 y Fm(b)o(y)g(deleting)h Fg(s)1402 1102 y Ff(k)q Fc(\000)p Ff(m)1500 1095 y Fm(from)e Fg(S)1637 1102 y Ff(k)1675 1095 y Fm(and)0 1151 y(adding)g(the)f(new)h (displacemen)o(t)g Fg(s)613 1158 y Ff(k)635 1151 y Fm(.)k(The)15 b(matrix)g Fg(Y)935 1158 y Ff(k)q Fk(+1)1017 1151 y Fm(is)h(up)q(dated)g(in)g (the)f(same)g(fashion.)71 1208 y(This)d(describ)q(es)i(the)f(general)f(step)g (when)h Fg(k)h(>)f(m)p Fm(.)19 b(F)l(or)11 b(the)i(\014rst)e(few)h (iterations,)h(when)g Fg(k)h Fe(\024)e Fg(m)p Fm(,)0 1269 y(w)o(e)k(need)h (only)f(replace)h Fg(m)f Fm(b)o(y)g Fg(k)h Fm(in)g(the)f(form)o(ulae)g(ab)q (o)o(v)o(e.)22 b(W)l(e)16 b(ha)o(v)o(e)f(assumed)h(that)g Fg(H)1570 1245 y Fk(\(0\))1566 1283 y Ff(k)1630 1269 y Fm(=)f Fg(\015)1704 1276 y Ff(k)1724 1269 y Fg(I)0 1326 y Fm(b)q(ecause)j(this)g(c)o(hoice)g(is)g (common)f(in)h(practice)g(\(see)g(Gilb)q(ert)g(and)g(Lemar)o(\023)-21 b(ec)o(hal)17 b(\(1989\))e(and)j(Liu)0 1382 y(and)13 b(No)q(cedal)h (\(1989\)\).)j(Other)d(form)o(ulae)f(for)f(the)h(initial)i(matrix)e(could)h (also)f(b)q(e)h(used,)f(but)h(w)o(ould)0 1439 y(probably)i(result)f(in)h(a)f (more)g(exp)q(ensiv)o(e)i(computation.)71 1495 y(A)e(limited)i(memory)e (matrix)h(based)f(on)h(the)f(direct)i(BF)o(GS)e(form)o(ula)g(is)h(also)f (easily)i(obtained.)0 1557 y(Let)22 b(the)f(basic)h(matrix)f(b)q(e)h(of)f (the)g(form)f Fg(B)805 1533 y Fk(\(0\))803 1571 y Ff(k)876 1557 y Fm(=)j Fg(\033)960 1564 y Ff(k)981 1557 y Fg(I)t Fm(,)f(for)f(some)g (p)q(ositiv)o(e)h(scalar)f Fg(\033)1571 1564 y Ff(k)1593 1557 y Fm(.)38 b(F)l(rom)0 1624 y(\(2.17\))18 b(w)o(e)i(see)g(that)f(if)h(w)o(e)g (up)q(date)g Fg(B)700 1600 y Fk(\(0\))698 1638 y Ff(k)768 1624 y Fg(m)g Fm(times)g(using)g(the)g(v)o(ectors)f Fe(f)p Fg(s)1363 1631 y Ff(k)q Fc(\000)p Ff(m)1444 1624 y Fg(;)8 b(:)g(:)g(:)t(;)g(s)1566 1631 y Ff(k)q Fc(\000)p Fk(1)1632 1624 y Fe(g)20 b Fm(and)0 1681 y Fe(f)p Fg(y)45 1688 y Ff(k)q Fc(\000)p Ff(m)125 1681 y Fg(;)8 b(:)g(:)g(:)d(;)j(y)249 1688 y Ff(k)q Fc(\000)p Fk(1)315 1681 y Fe(g)p Fm(,)14 b(w)o(e)h(obtain)287 1824 y Fg(B)321 1831 y Ff(k)355 1824 y Fm(=)e Fg(\033)429 1831 y Ff(k)450 1824 y Fg(I)h Fe(\000)529 1777 y Fd(h)570 1824 y Fg(\033)596 1831 y Ff(k)617 1824 y Fg(S)645 1831 y Ff(k)708 1824 y Fg(Y)734 1831 y Ff(k)776 1777 y Fd(i)803 1752 y(")848 1795 y Fg(\033)874 1802 y Ff(k)896 1795 y Fg(S)927 1779 y Ff(T)924 1809 y(k)954 1795 y Fg(S)982 1802 y Ff(k)1065 1795 y Fg(L)1096 1802 y Ff(k)896 1852 y Fg(L)927 1835 y Ff(T)927 1865 y(k)1044 1852 y Fe(\000)p Fg(D)1117 1859 y Ff(k)1159 1752 y Fd(#)1184 1763 y Fc(\000)p Fk(1)1238 1752 y Fd(")1283 1795 y Fg(\033)1309 1802 y Ff(k)1331 1795 y Fg(S)1362 1779 y Ff(T)1359 1809 y(k)1304 1852 y Fg(Y)1341 1835 y Ff(T)1330 1865 y(k)1409 1752 y Fd(#)1441 1824 y Fg(;)201 b Fm(\(3)p Fg(:)p Fm(5\))0 1958 y(where)15 b Fg(S)159 1965 y Ff(k)181 1958 y Fg(;)8 b(Y)228 1965 y Ff(k)249 1958 y Fg(;)g(D)308 1965 y Ff(k)343 1958 y Fm(are)15 b(giv)o(en)g(b)o(y)h(\(3.2\))d(and)j (\(3.4\),)d(and)i(where)h Fg(L)1170 1965 y Ff(k)1206 1958 y Fm(is)g(de\014ned)g(b)o(y)427 2093 y(\()p Fg(L)476 2100 y Ff(k)497 2093 y Fm(\))515 2100 y Ff(i;j)567 2093 y Fm(=)615 2021 y Fd(\()670 2065 y Fg(s)691 2048 y Ff(T)691 2078 y(k)q Fc(\000)p Ff(m)p Fc(\000)p Fk(1+)p Ff(i)855 2065 y Fg(y)877 2072 y Ff(k)q Fc(\000)p Ff(m)p Fc(\000)p Fk(1+)p Ff(j)1088 2065 y Fm(if)g Fg(i)c(>)h(j)670 2121 y Fm(0)395 b(otherwise)1308 2093 y Fg(:)334 b Fm(\(3)p Fg(:)p Fm(6\))71 2226 y(W)l(e)14 b(no)o(w)h(describ)q(e)h(pro)q(cedures)g (for)e(p)q(erforming)h(computations)g(with)g(these)g(compact)g(repre-)0 2282 y(sen)o(tations)g(of)g(limited)i(memory)d(BF)o(GS)h(matrices.)0 2438 y Fn(3.1.)i(Computations)h(in)o(v)o(olving)g Fg(H)700 2445 y Ff(k)721 2438 y Fn(.)863 2603 y Fm(9)p eop %%Page: 10 11 bop 71 120 a Fm(W)l(e)18 b(consider)h(sev)o(eral)f(pro)q(ducts)h(in)o(v)o (olving)h(the)e(in)o(v)o(erse)h(limited)h(memory)d(matrix)h Fg(H)1633 127 y Ff(k)1654 120 y Fm(.)29 b(T)l(o)0 177 y(sa)o(v)o(e)16 b(computations)h(w)o(e)f(will)j(store,)d(in)i(addition)f(to)g(the)g(t)o(w)o (o)e Fg(n)c Fe(\002)h Fg(m)17 b Fm(matrices)f Fg(S)1479 184 y Ff(k)1517 177 y Fm(and)h Fg(Y)1633 184 y Ff(k)1655 177 y Fm(,)g(the)0 233 y Fg(m)11 b Fe(\002)h Fg(m)17 b Fm(matrices)g Fg(Y)376 217 y Ff(T)365 246 y(k)403 233 y Fg(Y)429 240 y Ff(k)451 233 y Fm(,)g Fg(R)516 240 y Ff(k)537 233 y Fm(,)g(and)g Fg(D)695 240 y Ff(k)716 233 y Fm(.)26 b(Since)19 b(in)e(practice)h Fg(m)f Fm(is)h(v)o(ery)f(small,)h(sa)o(y)e Fg(m)g Fe(\024)g Fm(7,)h(the)0 289 y(storage)f(space)i(required)h(b)o(y)f(these)g(three)g(auxiliary)h (matrices)e(is)h(negligible.)31 b(In)18 b(the)g(op)q(eration)0 346 y(coun)o(ts)f(giv)o(en)h(b)q(elo)o(w)h(w)o(e)e(concen)o(trate)g(on)h(m)o (ultiplications)i(since)e(the)g(arithmetic)g(consists)g(pri-)0 402 y(marily)i(of)e(inner)i(pro)q(ducts,)g(so)f(that)f(the)h(n)o(um)o(b)q(er) g(of)g(additions)h(is)f(similar)i(to)d(the)h(n)o(um)o(b)q(er)g(of)0 459 y(m)o(ultiplications.)30 b(W)l(e)18 b(note)g(that)f(for)g(the)h(rest)f (of)h(this)g(section)g Fg(S)1193 466 y Ff(k)1215 459 y Fg(;)8 b(Y)1262 466 y Ff(k)1283 459 y Fg(;)g(R)1339 466 y Ff(k)1359 459 y Fg(;)g(D)1418 466 y Ff(k)1438 459 y Fg(;)g(L)1490 466 y Ff(k)1528 459 y Fm(are)18 b(de\014ned)0 515 y(b)o(y)d(\(3.2\)-\(3.4\))d (and)k(\(3.6\).)0 597 y Fn(Computation)j(of)e Fg(H)416 604 y Ff(k)437 597 y Fg(g)459 604 y Ff(k)480 597 y Fn(.)0 653 y Fm(This)e(pro)q(duct)f(de\014nes)h(the)f(searc)o(h)g(direction)h(in)g(a)e (limited)j(memory)d(metho)q(d)i(for)e(unconstrained)0 710 y(optimization.)22 b(Since)17 b(some)e(of)h(the)f(calculations)i(in)o(v)o(olv)o(ed)g(in)f(the)g (pro)q(duct)g Fg(H)1416 717 y Ff(k)1437 710 y Fg(g)1459 717 y Ff(k)1496 710 y Fm(o)q(ccur)g(also)f(in)0 766 y(the)g(up)q(date)h(of)f Fg(H)321 773 y Ff(k)342 766 y Fm(,)g(it)g(is)h(e\016cien)o(t)g(to)e(consider) j(b)q(oth)e(op)q(erations)g(together.)71 822 y(A)o(t)d(the)g Fg(k)q Fm(-th)h(iteration)g(of)f(the)g(limited)j(memory)d(algorithm)g(for)g (unconstrained)i(optimization)0 879 y(w)o(e)20 b(m)o(ust)h(up)q(date)g(our)f (represen)o(tation)h(of)f Fg(H)833 886 y Ff(k)q Fc(\000)p Fk(1)920 879 y Fm(to)g(get)h Fg(H)1101 886 y Ff(k)1122 879 y Fm(,)g(compute)g(the)g (searc)o(h)g(direction)0 935 y Fe(\000)p Fg(H)73 942 y Ff(k)94 935 y Fg(g)116 942 y Ff(k)156 935 y Fm(and)d(p)q(erform)g(a)g(line)i(searc)o (h.)29 b(T)l(o)18 b(up)q(date)h Fg(H)981 942 y Ff(k)q Fc(\000)p Fk(1)1065 935 y Fm(w)o(e)f(delete)i(a)e(column)h(from)e(and)i(add)0 992 y(a)d(new)g(column)h(to)f(eac)o(h)g(of)g(the)g(matrices)h Fg(S)797 999 y Ff(k)q Fc(\000)p Fk(1)879 992 y Fm(and)f Fg(Y)994 999 y Ff(k)q Fc(\000)p Fk(1)1061 992 y Fm(,)g(and)h(mak)o(e)e(corresp)q (onding)i(up)q(dates)0 1048 y(to)f Fg(R)92 1055 y Ff(k)q Fc(\000)p Fk(1)158 1048 y Fg(;)23 b(Y)231 1032 y Ff(T)220 1061 y(k)q Fc(\000)p Fk(1)287 1048 y Fg(Y)313 1055 y Ff(k)q Fc(\000)p Fk(1)397 1048 y Fm(and)17 b Fg(D)525 1055 y Ff(k)q Fc(\000)p Fk(1)591 1048 y Fm(.)25 b(W)l(e)18 b(will)g(sho)o(w)f(that)f(these)h(up)q (dates)h(can)f(b)q(e)h(done)f(in)h Fg(O)q Fm(\()p Fg(m)1711 1032 y Fk(2)1730 1048 y Fm(\))0 1105 y(op)q(erations)c(b)o(y)g(storing)g(a)f (small)i(amoun)o(t)e(of)g(additional)j(information.)j(F)l(or)14 b(example,)g(from)g(\(3.3\))0 1161 y(w)o(e)h(see)h(that)f(the)g(new)h (triangular)g(matrix)f Fg(R)808 1168 y Ff(k)844 1161 y Fm(is)h(formed)g(from) e Fg(R)1186 1168 y Ff(k)q Fc(\000)p Fk(1)1268 1161 y Fm(b)o(y)h(deleting)i (the)f(\014rst)f(ro)o(w)0 1218 y(and)g(column,)h(adding)g(a)f(new)g(column)h (on)g(the)f(righ)o(t,)g(whic)o(h)h(is)f(giv)o(en)h(b)o(y)620 1320 y Fg(S)651 1301 y Ff(T)648 1331 y(k)678 1320 y Fg(y)700 1327 y Ff(k)q Fc(\000)p Fk(1)780 1320 y Fm(=)d Fg(S)859 1301 y Ff(T)856 1331 y(k)886 1320 y Fm(\()p Fg(g)926 1327 y Ff(k)956 1320 y Fe(\000)e Fg(g)1024 1327 y Ff(k)q Fc(\000)p Fk(1)1090 1320 y Fm(\))p Fg(;)534 b Fm(\(3)p Fg(:)p Fm(7\))0 1422 y(and)12 b(adding)g(a)f(new)h(ro)o(w)f(on)h(the)f(b)q(ottom,)g(whic)o(h)i(is)f(zero)f (in)i(its)f(\014rst)f Fg(m)s Fe(\000)s Fm(1)g(comp)q(onen)o(ts.)19 b(It)12 b(w)o(ould)0 1478 y(app)q(ear)17 b(that)f(this)g(requires)i Fg(mn)e Fm(m)o(ultiplications.)27 b(Ho)o(w)o(ev)o(er,)15 b(note)i(from)e (\(3.1\))g(that)h(the)h(v)o(ector)0 1535 y Fg(S)31 1518 y Ff(T)28 1548 y(k)58 1535 y Fg(g)80 1542 y Ff(k)117 1535 y Fm(and)f(the)g(\014rst)f Fg(m)c Fe(\000)g Fm(1)k(comp)q(onen)o(ts)h(of)g Fg(S)849 1518 y Ff(T)846 1548 y(k)876 1535 y Fg(g)898 1542 y Ff(k)q Fc(\000)p Fk(1)980 1535 y Fm(ha)o(v)o(e)f(to)h(b)q(e)g(calculated)i(in)e(the)g(pro)q (cess)g(of)0 1591 y(computing)d Fg(H)260 1598 y Ff(k)281 1591 y Fg(g)303 1598 y Ff(k)337 1591 y Fm(and)g Fg(H)461 1598 y Ff(k)q Fc(\000)p Fk(1)527 1591 y Fg(g)549 1598 y Ff(k)q Fc(\000)p Fk(1)616 1591 y Fm(.)19 b(Th)o(us)12 b(w)o(e)h(ma)o(y)f(sa)o(v)o(e)g(the)h (\014rst)f Fg(m)5 b Fe(\000)g Fm(1)14 b(comp)q(onen)o(ts)f(of)f Fg(S)1633 1575 y Ff(T)1630 1604 y(k)1660 1591 y Fg(g)1682 1598 y Ff(k)q Fc(\000)p Fk(1)0 1648 y Fm(from)i(the)g(previous)h(iteration,)f(and) h(w)o(e)f(need)h(only)g(compute)f Fg(s)1122 1631 y Ff(T)1122 1661 y(k)q Fc(\000)p Fk(1)1189 1648 y Fg(g)1211 1655 y Ff(k)q Fc(\000)p Fk(1)1277 1648 y Fm(,)g(whic)o(h)h(can)f(b)q(e)h(obtained)0 1704 y(with)k Fg(O)q Fm(\()p Fg(m)201 1688 y Fk(2)220 1704 y Fm(\))g(w)o(ork,)g(as)f(w)o(e)h(will)h(sho)o(w)f(b)q(elo)o(w.)31 b(Th)o(us)19 b(to)f(compute)h Fg(S)1276 1688 y Ff(T)1273 1717 y(k)1303 1704 y Fg(y)1325 1711 y Ff(k)q Fc(\000)p Fk(1)1411 1704 y Fm(b)o(y)g(the)g(di\013erence)0 1761 y(\(3.7\))14 b(will)i(require)g (only)g Fg(O)q Fm(\()p Fg(m)541 1744 y Fk(2)560 1761 y Fm(\))f(op)q (erations.)20 b(The)15 b(matrix)g Fg(Y)1110 1744 y Ff(T)1099 1774 y(k)1137 1761 y Fg(Y)1163 1768 y Ff(k)1200 1761 y Fm(can)g(b)q(e)h(up)q (dated)g(in)g(a)f(similar)0 1817 y(w)o(a)o(y)f(sa)o(ving)h(another)g Fg(mn)g Fm(m)o(ultiplications.)71 1874 y(An)g(up)q(dating)h(pro)q(cess)g (that)e(implemen)o(ts)j(these)e(sa)o(vings)h(in)g(computation)f(is)h(as)e (follo)o(ws.)21 b(A)o(t)0 1930 y Fg(x)26 1937 y Ff(k)47 1930 y Fm(,)15 b(the)g(follo)o(wing)h(data)f(has)g(b)q(een)h(sa)o(v)o(ed)f(from)f (the)i(previous)g(iteration:)780 2032 y Fg(g)804 2013 y Ff(T)802 2043 y(k)q Fc(\000)p Fk(1)868 2032 y Fg(g)890 2039 y Ff(k)q Fc(\000)p Fk(1)956 2032 y Fg(;)258 2134 y(s)279 2115 y Ff(T)279 2145 y(i)307 2134 y Fg(g)329 2141 y Ff(k)q Fc(\000)p Fk(1)545 2134 y Fg(i)c Fm(=)h Fg(k)e Fe(\000)g Fg(m)f Fe(\000)g Fm(1)p Fg(;)e(:)g(:)g(:)d(;)j(k)i Fe(\000)g Fm(2)p Fg(;)157 b Fm(\(i)p Fg(:)p Fm(e)p Fg(:)20 b(S)1321 2115 y Ff(T)1318 2145 y(k)q Fc(\000)p Fk(1)1385 2134 y Fg(g)1407 2141 y Ff(k)q Fc(\000)p Fk(1)1473 2134 y Fm(\))0 2218 y(and)261 2274 y Fg(y)285 2255 y Ff(T)283 2285 y(i)312 2274 y Fg(g)334 2281 y Ff(k)q Fc(\000)p Fk(1)550 2274 y Fg(i)13 b Fm(=)g Fg(k)e Fe(\000)f Fg(m)g Fe(\000)h Fm(1)p Fg(;)d(:)g(:)g(:)t(;)g(k)j Fe(\000)f Fm(2)150 b(\(i)p Fg(:)p Fm(e)p Fg(:)20 b(Y)1312 2255 y Ff(T)1302 2285 y(k)q Fc(\000)p Fk(1)1369 2274 y Fg(g)1391 2281 y Ff(k)q Fc(\000)p Fk(1)1457 2274 y Fm(\))p Fg(:)0 2358 y Fm(No)o(w)15 b(w)o(e)f(compute)i(the)f (quan)o(tities)h(corresp)q(onding)g(to)f(the)g(presen)o(t)g(iteration.)20 b(W)l(e)c(b)q(egin)g(with)545 2460 y Fg(s)566 2441 y Ff(T)566 2471 y(k)q Fc(\000)p Fk(1)633 2460 y Fg(g)655 2467 y Ff(k)q Fc(\000)p Fk(1)734 2460 y Fm(=)d Fe(\000)p Fg(\025)844 2467 y Ff(k)q Fc(\000)p Fk(1)910 2460 y Fg(g)934 2441 y Ff(T)932 2471 y(k)q Fc(\000)p Fk(1)998 2460 y Fg(H)1036 2467 y Ff(k)q Fc(\000)p Fk(1)1102 2460 y Fg(g)1124 2467 y Ff(k)q Fc(\000)p Fk(1)1190 2460 y Fg(;)851 2603 y Fm(10)p eop %%Page: 11 12 bop 0 120 a Fm(whic)o(h)16 b(b)o(y)f(\(3.1\))f(is)h(equal)h(to)25 279 y Fe(\000)r Fg(\025)89 286 y Ff(k)q Fc(\000)p Fk(1)156 279 y Fg(\015)180 286 y Ff(k)q Fc(\000)p Fk(1)245 279 y Fg(g)269 261 y Ff(T)267 291 y(k)q Fc(\000)p Fk(1)334 279 y Fg(g)356 286 y Ff(k)q Fc(\000)p Fk(1)424 279 y Fe(\000)r Fg(\025)488 286 y Ff(k)q Fc(\000)p Fk(1)555 279 y Fg(w)589 261 y Ff(T)588 291 y(k)623 183 y Fd(2)623 256 y(6)623 282 y(4)672 223 y Fg(R)707 204 y Fc(\000)p Ff(T)707 237 y(k)q Fc(\000)p Fk(1)773 223 y Fm(\()p Fg(D)829 230 y Ff(k)q Fc(\000)p Fk(1)905 223 y Fm(+)10 b Fg(\015)974 230 y Ff(k)q Fc(\000)p Fk(1)1040 223 y Fg(Y)1076 206 y Ff(T)1066 236 y(k)q Fc(\000)p Fk(1)1133 223 y Fg(Y)1159 230 y Ff(k)q Fc(\000)p Fk(1)1226 223 y Fm(\))p Fg(R)1279 204 y Fc(\000)p Fk(1)1279 237 y Ff(k)q Fc(\000)p Fk(1)1386 223 y Fe(\000)p Fg(R)1456 204 y Fc(\000)p Ff(T)1456 237 y(k)q Fc(\000)p Fk(1)940 337 y Fe(\000)p Fg(R)1010 318 y Fc(\000)p Fk(1)1010 352 y Ff(k)q Fc(\000)p Fk(1)1442 337 y Fm(0)1543 183 y Fd(3)1543 256 y(7)1543 282 y(5)1578 279 y Fg(w)1611 286 y Ff(k)1655 279 y Fm(\(3)p Fg(:)p Fm(8\))0 436 y(where)623 539 y Fg(w)656 546 y Ff(k)689 539 y Fm(=)737 442 y Fd(2)737 515 y(6)737 542 y(4)830 482 y Fg(S)861 466 y Ff(T)858 496 y(k)q Fc(\000)p Fk(1)924 482 y Fg(g)946 489 y Ff(k)q Fc(\000)p Fk(1)786 595 y Fg(\015)810 602 y Ff(k)q Fc(\000)p Fk(1)876 595 y Fg(Y)912 579 y Ff(T)902 609 y(k)q Fc(\000)p Fk(1)968 595 y Fg(g)990 602 y Ff(k)q Fc(\000)p Fk(1)1077 442 y Fd(3)1077 515 y(7)1077 542 y(5)1113 539 y Fg(:)0 685 y Fm(This)i(requires)g(only)g Fg(O)q Fm(\()p Fg(m)457 668 y Fk(2)476 685 y Fm(\))f(op)q(erations)h(since)g Fg(g)854 668 y Ff(T)852 698 y(k)q Fc(\000)p Fk(1)918 685 y Fg(g)940 692 y Ff(k)q Fc(\000)p Fk(1)1007 685 y Fm(,)f Fg(S)1062 668 y Ff(T)1059 698 y(k)q Fc(\000)p Fk(1)1126 685 y Fg(g)1148 692 y Ff(k)q Fc(\000)p Fk(1)1225 685 y Fm(and)g Fg(Y)1346 668 y Ff(T)1335 698 y(k)q Fc(\000)p Fk(1)1402 685 y Fg(g)1424 692 y Ff(k)q Fc(\000)p Fk(1)1502 685 y Fm(ha)o(v)o(e)g(already)0 741 y(b)q(een)16 b(sa)o(v)o(ed)f(from)g(the)g(previous)h(iteration.)71 798 y(Next)f(w)o(e)g (compute)g(the)g(inner)i(pro)q(ducts)821 899 y Fg(g)845 881 y Ff(T)843 911 y(k)872 899 y Fg(g)894 906 y Ff(k)915 899 y Fg(;)360 1001 y(s)381 982 y Ff(T)381 1012 y(i)409 1001 y Fg(g)431 1008 y Ff(k)602 1001 y Fg(i)12 b Fm(=)h Fg(k)e Fe(\000)g Fg(m;)d(:)g(:)g(:)d (;)j(k)i Fe(\000)g Fm(1)p Fg(;)157 b Fm(\(i)p Fg(:)p Fm(e)p Fg(:)20 b(S)1300 982 y Ff(T)1297 1012 y(k)1327 1001 y Fg(g)1349 1008 y Ff(k)1370 1001 y Fm(\))0 1084 y(and)349 1141 y Fg(y)373 1122 y Ff(T)371 1152 y(i)401 1141 y Fg(g)423 1148 y Ff(k)594 1141 y Fg(i)12 b Fm(=)h Fg(k)e Fe(\000)g Fg(m;)d(:)g(:)g(:)d(;)j(k)i Fe(\000)g Fm(1)p Fg(;)157 b Fm(\(i)p Fg(:)p Fm(e)p Fg(:)20 b(Y)1298 1122 y Ff(T)1287 1152 y(k)1325 1141 y Fg(g)1347 1148 y Ff(k)1368 1141 y Fm(\))p Fg(:)0 1224 y Fm(With)e(this)h(information,)f(the) g(new)h(comp)q(onen)o(ts)f(of)f Fg(R)994 1231 y Ff(k)1015 1224 y Fg(;)8 b(Y)1072 1208 y Ff(T)1062 1237 y(k)1100 1224 y Fg(Y)1126 1231 y Ff(k)1165 1224 y Fm(and)18 b Fg(D)1294 1231 y Ff(k)1315 1224 y Fm(,)h(can)f(b)q(e)h(computed)f(in)0 1281 y Fg(O)q Fm(\()p Fg(m)p Fm(\))c(w)o(ork)h(b)o(y)g(the)g(form)o(ulae)336 1382 y Fg(s)357 1364 y Ff(T)357 1394 y(i)384 1382 y Fg(y)406 1389 y Ff(k)q Fc(\000)p Fk(1)515 1382 y Fm(=)41 b Fg(s)612 1364 y Ff(T)612 1394 y(i)640 1382 y Fg(g)662 1389 y Ff(k)693 1382 y Fe(\000)11 b Fg(s)760 1364 y Ff(T)760 1394 y(i)788 1382 y Fg(g)810 1389 y Ff(k)q Fc(\000)p Fk(1)1026 1382 y Fg(i)h Fm(=)h Fg(k)e Fe(\000)g Fg(m;)d(:)g(:)g(:)t(;)g(k)j Fe(\000)f Fm(1)p Fg(;)215 b Fm(\(3.9\))333 1451 y Fg(y)357 1432 y Ff(T)355 1462 y(i)384 1451 y Fg(y)406 1458 y Ff(k)q Fc(\000)p Fk(1)515 1451 y Fm(=)41 b Fg(y)615 1432 y Ff(T)613 1462 y(i)643 1451 y Fg(g)665 1458 y Ff(k)696 1451 y Fe(\000)10 b Fg(y)765 1432 y Ff(T)763 1462 y(i)793 1451 y Fg(g)815 1458 y Ff(k)q Fc(\000)p Fk(1)1031 1451 y Fg(i)i Fm(=)h Fg(k)e Fe(\000)g Fg(m;)d(:)g(:)g(:)d(;)j(k)i Fe(\000)g Fm(2)p Fg(;)187 b Fm(\(3.10\))296 1520 y Fg(y)320 1501 y Ff(T)318 1531 y(k)q Fc(\000)p Fk(1)384 1520 y Fg(y)406 1527 y Ff(k)q Fc(\000)p Fk(1)515 1520 y Fm(=)41 b Fe(\000)p Fg(g)650 1501 y Ff(T)648 1531 y(k)678 1520 y Fg(g)700 1527 y Ff(k)731 1520 y Fm(+)10 b(2\()p Fg(g)839 1527 y Ff(k)870 1520 y Fe(\000)g Fg(g)937 1527 y Ff(k)q Fc(\000)p Fk(1)1003 1520 y Fm(\))1021 1501 y Ff(T)1048 1520 y Fg(g)1070 1527 y Ff(k)1102 1520 y Fm(+)g Fg(g)1171 1501 y Ff(T)1169 1531 y(k)q Fc(\000)p Fk(1)1235 1520 y Fg(g)1257 1527 y Ff(k)q Fc(\000)p Fk(1)1323 1520 y Fg(:)296 b Fm(\(3.11\))71 1622 y(W)l(e)13 b(no)o(w)f(giv)o(e)i(a)e(complete)i(description)h(of)d(the)i(pro)q(cedure)g (for)e(up)q(dating)i Fg(H)1418 1629 y Ff(k)1452 1622 y Fm(and)f(computing)0 1678 y Fg(H)38 1685 y Ff(k)59 1678 y Fg(g)81 1685 y Ff(k)102 1678 y Fm(.)0 1784 y Fn(Algorithm)18 b(3.1)d Fm(\(Step)g(Computation)g(for)f (Unconstrained)i(Minimization\))71 1841 y(Let)h Fg(x)180 1848 y Ff(k)218 1841 y Fm(b)q(e)h(the)f(curren)o(t)g(iterate.)25 b(Giv)o(en)18 b Fg(s)844 1848 y Ff(k)q Fc(\000)p Fk(1)911 1841 y Fg(;)8 b(y)954 1848 y Ff(k)q Fc(\000)p Fk(1)1020 1841 y Fm(,)17 b Fg(g)1072 1848 y Ff(k)1093 1841 y Fm(,)g(the)g(matrices)g Fg(S)1415 1848 y Ff(k)q Fc(\000)p Fk(1)1481 1841 y Fm(,)h Fg(Y)1538 1848 y Ff(k)q Fc(\000)p Fk(1)1604 1841 y Fm(,)f Fg(R)1669 1848 y Ff(k)q Fc(\000)p Fk(1)1735 1841 y Fm(,)0 1897 y Fg(Y)36 1881 y Ff(T)26 1910 y(k)q Fc(\000)p Fk(1)93 1897 y Fg(Y)119 1904 y Ff(k)q Fc(\000)p Fk(1)186 1897 y Fm(,)d Fg(D)251 1904 y Ff(k)q Fc(\000)p Fk(1)317 1897 y Fm(,)h(the)h(v)o(ectors)e Fg(S)609 1881 y Ff(T)606 1910 y(k)q Fc(\000)p Fk(1)672 1897 y Fg(g)694 1904 y Ff(k)q Fc(\000)p Fk(1)760 1897 y Fm(,)h Fg(Y)825 1881 y Ff(T)814 1910 y(k)q Fc(\000)p Fk(1)881 1897 y Fg(g)903 1904 y Ff(k)q Fc(\000)p Fk(1)984 1897 y Fm(and)h(the)f(scalar)g Fg(g)1304 1881 y Ff(T)1302 1910 y(k)q Fc(\000)p Fk(1)1368 1897 y Fg(g)1390 1904 y Ff(k)q Fc(\000)p Fk(1)1456 1897 y Fm(:)56 1991 y(1.)22 b(Up)q(date)15 b Fg(S)303 1998 y Ff(k)324 1991 y Fg(;)8 b(Y)371 1998 y Ff(k)56 2084 y Fm(2.)22 b(Compute)15 b Fg(g)335 2068 y Ff(T)333 2098 y(k)361 2084 y Fg(g)383 2091 y Ff(k)404 2084 y Fg(;)8 b(S)456 2068 y Ff(T)453 2098 y(k)483 2084 y Fg(g)505 2091 y Ff(k)526 2084 y Fg(;)14 b Fm(and)i Fg(Y)678 2068 y Ff(T)668 2098 y(k)706 2084 y Fg(g)728 2091 y Ff(k)56 2178 y Fm(3.)22 b(Compute)15 b Fg(s)332 2162 y Ff(T)332 2191 y(k)q Fc(\000)p Fk(1)398 2178 y Fg(g)420 2185 y Ff(k)q Fc(\000)p Fk(1)502 2178 y Fm(b)o(y)g(\(3.8\))56 2272 y(4.)22 b(Up)q(date)15 b Fg(R)310 2279 y Ff(k)331 2272 y Fg(;)8 b(Y)388 2255 y Ff(T)378 2285 y(k)415 2272 y Fg(Y)441 2279 y Ff(k)478 2272 y Fm(and)16 b Fg(D)605 2279 y Ff(k)641 2272 y Fm(with)f(the)h(aid)f(of)g (\(3.9\)-\(3.11\).)56 2365 y(5.)22 b(Compute)15 b Fg(\015)335 2372 y Ff(k)355 2365 y Fm(;)g(for)g(example)656 2467 y Fg(\015)680 2474 y Ff(k)713 2467 y Fm(=)e Fg(y)783 2474 y Ff(k)q Fc(\000)p Fk(1)850 2448 y Ff(T)878 2467 y Fg(s)899 2474 y Ff(k)q Fc(\000)p Fk(1)965 2467 y Fg(=y)1010 2474 y Ff(k)q Fc(\000)p Fk(1)1077 2448 y Ff(T)1104 2467 y Fg(y)1126 2474 y Ff(k)q Fc(\000)p Fk(1)1193 2467 y Fg(:)426 b Fm(\(3)p Fg(:)p Fm(12\))851 2603 y(11)p eop %%Page: 12 13 bop 56 120 a Fm(6.)22 b(Compute)370 246 y Fg(p)13 b Fm(=)454 174 y Fd(")499 219 y Fg(R)534 199 y Fc(\000)p Ff(T)534 233 y(k)589 219 y Fm(\()p Fg(D)645 226 y Ff(k)675 219 y Fm(+)e Fg(\015)745 226 y Ff(k)766 219 y Fg(Y)802 202 y Ff(T)792 232 y(k)830 219 y Fg(Y)856 226 y Ff(k)878 219 y Fm(\))p Fg(R)931 199 y Fc(\000)p Fk(1)931 233 y Ff(k)977 219 y Fm(\()p Fg(S)1026 202 y Ff(T)1023 232 y(k)1053 219 y Fg(g)1075 226 y Ff(k)1096 219 y Fm(\))f Fe(\000)g Fg(\015)1193 226 y Ff(k)1214 219 y Fg(R)1249 199 y Fc(\000)p Ff(T)1249 233 y(k)1304 219 y Fm(\()p Fg(Y)1358 202 y Ff(T)1348 232 y(k)1386 219 y Fg(g)1408 226 y Ff(k)1429 219 y Fm(\))839 276 y Fe(\000)p Fg(R)909 257 y Fc(\000)p Fk(1)909 290 y Ff(k)957 276 y Fm(\()p Fg(S)1006 260 y Ff(T)1003 289 y(k)1032 276 y Fg(g)1054 283 y Ff(k)1075 276 y Fm(\))p Fg(:)1467 174 y Fd(#)56 389 y Fm(7.)22 b(Compute)624 451 y Fg(H)662 458 y Ff(k)683 451 y Fg(g)705 458 y Ff(k)739 451 y Fm(=)13 b Fg(\015)811 458 y Ff(k)832 451 y Fg(g)854 458 y Ff(k)885 451 y Fm(+)930 404 y Fd(h)971 451 y Fg(S)999 458 y Ff(k)1061 451 y Fg(\015)1085 458 y Ff(k)1106 451 y Fg(Y)1132 458 y Ff(k)1175 404 y Fd(i)1202 451 y Fg(p:)71 557 y Fm(In)g(this)g(pro)q (cedure,)h(step)e(2)h(requires)g(\(2)p Fg(m)5 b Fm(+)g(1\))p Fg(n)12 b Fm(m)o(ultiplications;)k(step)d(7)f(requires)i(\(2)p Fg(m)5 b Fm(+)g(1\))p Fg(n)0 613 y Fm(m)o(ultiplications;)16 b(step)e(5)g(dep)q(ends)i(on)e(the)g(form)o(ula)f(used)i(for)e Fg(\015)1127 620 y Ff(k)1162 613 y Fm(\(the)h(c)o(hoice)h(\(3.12\))d(is)j (free)f(since)0 670 y(b)q(oth)d(inner)h(pro)q(ducts)g(ha)o(v)o(e)e(b)q(een)i (stored\);)g(all)f(other)g(steps)g(cost)g(at)f(most)g Fg(O)q Fm(\()p Fg(m)1393 653 y Fk(2)1412 670 y Fm(\))h(m)o(ultiplications,)0 726 y(for)g(a)g(total)g(of)h(\(4)p Fg(m)s Fm(+)s(2\))p Fg(n)s Fm(+)s Fg(O)q Fm(\()p Fg(m)579 710 y Fk(2)598 726 y Fm(\))f(m)o (ultiplications.)21 b(Note,)12 b(ho)o(w)o(ev)o(er,)f(that)g(when)h(this)g (pro)q(cedure)0 783 y(is)i(part)f(of)h(an)f(algorithm)h(using)h(a)e(line)j (searc)o(h)d(pro)q(cedure,)i(the)f(scalar)f Fg(s)1282 766 y Ff(T)1282 796 y(k)q Fc(\000)p Fk(1)1349 783 y Fg(g)1371 790 y Ff(k)q Fc(\000)p Fk(1)1451 783 y Fm(is)h(also)g(required)0 839 y(for)e(the)g(line)i(searc)o(h,)e(whereas)g Fg(g)566 823 y Ff(T)564 852 y(k)593 839 y Fg(g)615 846 y Ff(k)648 839 y Fm(is)h(lik)o(ely)h(to)d(b)q(e)i(needed)h(to)e(c)o(hec)o(k)g(the)g(stopping)h (conditions)g(of)0 896 y(the)i(algorithm.)20 b(Therefore)14 b(the)h(amoun)o(t)g(of)f(extra)g(w)o(ork)g(required)i(to)e(up)q(date)i Fg(H)1455 903 y Ff(k)1491 896 y Fm(and)f(compute)0 952 y(the)i(step)g (direction)h(is)g(4)p Fg(mn)11 b Fm(+)g Fg(O)q Fm(\()p Fg(m)658 936 y Fk(2)678 952 y Fm(\))16 b(in)i(that)e(case.)25 b(Of)18 b(course)f(for)f(large)h(problems)g(the)g(term)0 1009 y(4)p Fg(mn)e Fm(predominates.)71 1065 y(As)c(will)j(b)q(e)e(seen)g(in)h(Section)f (4.1)f(this)h(is)g(the)g(same)g(amoun)o(t)e(of)i(w)o(ork)e(p)q(er)j (iteration)f(as)f(required)0 1122 y(b)o(y)17 b(the)h(t)o(w)o(o-lo)q(op)e (recursion)i(describ)q(ed)h(b)o(y)f(No)q(cedal)g(\(1980\),)e(and)h(as)g(far)g (as)f(w)o(e)h(kno)o(w)g(there)g(is)0 1178 y(no)h(more)g(e\016cien)o(t)g(w)o (a)o(y)f(to)h(implemen)o(t)h(the)f(unconstrained)h(limited)h(memory)e(BF)o (GS)f(metho)q(d.)0 1234 y(Th)o(us)12 b(the)g(t)o(w)o(o)f(approac)o(hes)h(are) f(equally)j(e\016cien)o(t)e(for)g(unconstrained)h(problems,)g(but,)f(as)g(p)q (oin)o(ted)0 1291 y(out)18 b(in)h(Section)h(4.1,)e(the)g(compact)g(matrix)g (represen)o(tations)h(deriv)o(ed)g(in)g(this)g(pap)q(er)g(are)f(more)0 1347 y(economical)c(when)g(computing)f(certain)h(quan)o(tities)g(arising)f (in)h(sparse)f(constrained)h(optimization)0 1404 y(calculations.)0 1483 y Fn(The)k(pro)q(duct)g Fg(H)342 1490 y Ff(k)363 1483 y Fg(v)r Fn(.)0 1540 y Fm(Let)i(us)h(consider)g(the)f(computation)g(of)g(the) g(pro)q(duct)g Fg(H)1038 1547 y Ff(k)1059 1540 y Fg(v)r Fm(,)h(where)f Fg(v)i Fm(is)f(an)f(arbitrary)f(v)o(ector.)0 1596 y(F)l(rom)14 b(\(3.1\))g(w)o(e)h(see)g(that)g(this)h(pro)q(duct)f(is)h(giv)o(en)g(b)o(y)47 1750 y Fg(H)85 1757 y Ff(k)106 1750 y Fg(v)e Fm(=)f Fg(\015)214 1757 y Ff(k)235 1750 y Fg(v)f Fm(+)314 1702 y Fd(h)355 1750 y Fg(S)383 1757 y Ff(k)445 1750 y Fg(\015)469 1757 y Ff(k)490 1750 y Fg(Y)516 1757 y Ff(k)559 1702 y Fd(i)586 1653 y(2)586 1726 y(6)586 1752 y(4)634 1694 y Fg(R)669 1675 y Fc(\000)p Ff(T)669 1708 y(k)724 1694 y Fm(\()p Fg(D)780 1701 y Ff(k)811 1694 y Fm(+)e Fg(\015)880 1701 y Ff(k)901 1694 y Fg(Y)938 1677 y Ff(T)927 1707 y(k)965 1694 y Fg(Y)991 1701 y Ff(k)1013 1694 y Fm(\))p Fg(R)1066 1675 y Fc(\000)p Fk(1)1066 1708 y Ff(k)1154 1694 y Fe(\000)p Fg(R)1224 1675 y Fc(\000)p Ff(T)1224 1708 y(k)815 1808 y Fe(\000)p Fg(R)885 1788 y Fc(\000)p Fk(1)885 1822 y Ff(k)1205 1808 y Fm(0)1300 1653 y Fd(3)1300 1726 y(7)1300 1752 y(5)1335 1653 y(2)1335 1726 y(6)1335 1752 y(4)1409 1693 y Fg(S)1440 1677 y Ff(T)1437 1706 y(k)1467 1693 y Fg(v)1384 1806 y(\015)1408 1813 y Ff(k)1429 1806 y Fg(Y)1465 1789 y Ff(T)1455 1819 y(k)1493 1806 y Fg(v)1537 1653 y Fd(3)1537 1726 y(7)1537 1752 y(5)1572 1750 y Fg(:)47 b Fm(\(3)p Fg(:)p Fm(13\))0 1908 y(T)l(o)10 b(carry)g(out)g(the)g(computation)g(w)o(e)g(\014rst)g(compute)h (the)f(pro)q(ducts)h Fg(S)1203 1892 y Ff(T)1200 1921 y(k)1230 1908 y Fg(v)h Fm(and)e Fg(Y)1384 1892 y Ff(T)1373 1921 y(k)1411 1908 y Fg(v)r Fm(,)h(whic)o(h)g(together)0 1965 y(require)j(2)p Fg(mn)e Fm(m)o(ultiplications.)22 b(T)l(o)12 b(m)o(ultiply)j(the)d(resulting) i(2)p Fg(m)f Fm(v)o(ector)f(b)o(y)g(the)h(middle)i(2)p Fg(m)5 b Fe(\002)g Fm(2)p Fg(m)0 2021 y Fm(matrix)18 b(in)o(v)o(olv)o(es)h(3)f (solutions)h(of)g(triangular)f(systems)g(and)h(one)f(m)o(ultiplication)j(b)o (y)e(an)f Fg(m)12 b Fe(\002)h Fg(m)0 2078 y Fm(matrix.)31 b(Finally)l(,)21 b(it)e(tak)o(es)g(2)p Fg(mn)f Fm(m)o(ultiplications)k(to)c(m)o(ultiply)j([)p Fg(S)1223 2085 y Ff(k)1262 2078 y Fg(\015)1286 2085 y Ff(k)1307 2078 y Fg(Y)1333 2085 y Ff(k)1355 2078 y Fm(])e(with)g(the)g(resulting)0 2134 y(2)p Fg(m)d Fm(v)o(ector.)21 b(Th)o(us,)16 b(if)h(w)o(e)e(include)k (the)d(pro)q(duct)g Fg(\015)906 2141 y Ff(k)927 2134 y Fg(v)i Fm(and)e(ignore)g Fg(O)q Fm(\()p Fg(m)p Fm(\))f(op)q(erations,)h(the)g(whole) 0 2190 y(computation)f(requires)h(\(4)p Fg(m)10 b Fm(+)g(1\))p Fg(n)g Fm(+)700 2173 y Fk(5)p 700 2180 18 2 v 700 2206 a(2)723 2190 y Fg(m)763 2174 y Fk(2)798 2190 y Fm(m)o(ultiplications.)0 2270 y Fn(Pro)q(ducts)18 b(of)f(the)h(form)e Fg(v)523 2253 y Ff(T)551 2270 y Fg(H)589 2277 y Ff(k)610 2270 y Fg(v)j Fn(and)f Fg(u)778 2253 y Ff(T)805 2270 y Fg(H)843 2277 y Ff(k)864 2270 y Fg(v)r Fn(.)0 2326 y Fm(Consider)f(the)f(w)o(eigh)o(ted)h(scalar)f(pro)q (duct)h Fg(v)787 2310 y Ff(T)814 2326 y Fg(H)852 2333 y Ff(k)873 2326 y Fg(v)h Fm(where)f Fg(v)h Fm(is)f(an)f(arbitrary)g(v)o(ector,)f(and)i (where)0 2383 y(w)o(e)e(assume)g(that)f(the)i(v)o(ector)e Fg(H)575 2390 y Ff(k)596 2383 y Fg(v)j Fm(is)f(not)f(needed.)21 b(Using)16 b(\(3.1\))d(w)o(e)i(ha)o(v)o(e)66 2479 y Fg(v)90 2460 y Ff(T)117 2479 y Fg(H)155 2486 y Ff(k)176 2479 y Fg(v)f Fm(=)f Fg(\015)284 2486 y Ff(k)305 2479 y Fg(v)329 2460 y Ff(T)356 2479 y Fg(v)f Fm(+)f(\()p Fg(R)489 2459 y Fc(\000)p Fk(1)489 2493 y Ff(k)535 2479 y Fg(S)566 2460 y Ff(T)563 2490 y(k)593 2479 y Fg(v)r Fm(\))635 2460 y Ff(T)662 2479 y Fm(\()p Fg(D)718 2486 y Ff(k)749 2479 y Fm(+)f Fg(\015)818 2486 y Ff(k)839 2479 y Fg(Y)876 2460 y Ff(T)865 2490 y(k)903 2479 y Fg(Y)929 2486 y Ff(k)951 2479 y Fm(\)\()p Fg(R)1022 2459 y Fc(\000)p Fk(1)1022 2493 y Ff(k)1068 2479 y Fg(S)1099 2460 y Ff(T)1096 2490 y(k)1126 2479 y Fg(v)r Fm(\))g Fe(\000)g Fm(2)p Fg(\015)1270 2486 y Ff(k)1291 2479 y Fg(v)1315 2460 y Ff(T)1342 2479 y Fg(Y)1368 2486 y Ff(k)1390 2479 y Fg(R)1425 2459 y Fc(\000)p Fk(1)1425 2493 y Ff(k)1472 2479 y Fg(S)1503 2460 y Ff(T)1500 2490 y(k)1530 2479 y Fg(v)r(:)65 b Fm(\(3)p Fg(:)p Fm(14\))851 2603 y(12)p eop %%Page: 13 14 bop 0 120 a Fm(W)l(e)18 b(\014rst)g(compute)g Fg(S)396 104 y Ff(T)393 133 y(k)423 120 y Fg(v)i Fm(and)f Fg(Y)593 104 y Ff(T)583 133 y(k)621 120 y Fg(v)r Fm(,)f(whic)o(h)h(requires)g(2)p Fg(mn)f Fm(m)o(ultiplications.)31 b(Next)18 b(w)o(e)g(solv)o(e)g(a)0 177 y(triangular)f(system)f(to)g(get)g Fg(R)535 157 y Fc(\000)p Fk(1)535 191 y Ff(k)582 177 y Fg(S)613 160 y Ff(T)610 190 y(k)640 177 y Fg(v)r Fm(,)g(whic)o(h)i(w)o(e)e(sa)o(v)o(e,)g(m)o(ultiply)i(b)o(y)e (the)h(matrix)f Fg(D)1519 184 y Ff(k)1551 177 y Fm(+)c Fg(\015)s(Y)1660 160 y Ff(T)1651 190 y(k)1688 177 y Fg(Y)1714 184 y Ff(k)1735 177 y Fm(,)0 233 y(compute)17 b Fg(v)210 217 y Ff(T)237 233 y Fg(v)i Fm(and)e(do)g(some)f(order)h Fg(m)g Fm(inner)h(pro)q(ducts.)25 b(Th)o(us)17 b(the)g(total)f(cost)h(of)f(this)i(compu-)0 289 y(tation)g(is)g(\(2)p Fg(m)11 b Fm(+)h(1\))p Fg(n)g Fm(+)456 272 y Fk(3)p 456 279 18 2 v 456 305 a(2)479 289 y Fg(m)519 273 y Fk(2)550 289 y Fm(+)h Fg(O)q Fm(\()p Fg(m)p Fm(\):)24 b(roughly)18 b(half)g(of)g(what)f(the)h(cost)g(w)o(ould)g(b)q(e)g(if)h(w)o(e) e(\014rst)0 346 y(computed)f Fg(H)248 353 y Ff(k)269 346 y Fg(v)h Fm(and)e(then)g Fg(v)523 329 y Ff(T)551 346 y Fg(H)589 353 y Ff(k)610 346 y Fg(v)r Fm(.)71 402 y(If)g(w)o(e)f(wish)h(to)f(compute)h (the)g(pro)q(duct)g Fg(u)798 386 y Ff(T)826 402 y Fg(H)864 409 y Ff(k)885 402 y Fg(v)h Fm(for)e(t)o(w)o(o)g(arbitrary)g(v)o(ectors)g Fg(u)g Fm(and)h Fg(v)i Fm(the)d(cost)0 459 y(is)i(more,)e(since)108 561 y Fg(u)134 542 y Ff(T)161 561 y Fg(H)199 568 y Ff(k)221 561 y Fg(v)43 b Fm(=)f Fg(\015)387 568 y Ff(k)407 561 y Fg(u)433 542 y Ff(T)461 561 y Fg(v)12 b Fm(+)e(\()p Fg(R)593 542 y Fc(\000)p Fk(1)593 575 y Ff(k)640 561 y Fg(S)671 542 y Ff(T)668 572 y(k)698 561 y Fg(u)p Fm(\))742 542 y Ff(T)769 561 y Fm(\()p Fg(D)825 568 y Ff(k)856 561 y Fm(+)g Fg(\015)925 568 y Ff(k)946 561 y Fg(Y)983 542 y Ff(T)972 572 y(k)1010 561 y Fg(Y)1036 568 y Ff(k)1058 561 y Fm(\)\()p Fg(R)1129 542 y Fc(\000)p Fk(1)1129 575 y Ff(k)1175 561 y Fg(S)1206 542 y Ff(T)1203 572 y(k)1233 561 y Fg(v)r Fm(\))g Fe(\000)g Fg(\015)1354 568 y Ff(k)1375 561 y Fg(u)1401 542 y Ff(T)1429 561 y Fg(Y)1455 568 y Ff(k)1476 561 y Fg(R)1511 542 y Fc(\000)p Fk(1)1511 575 y Ff(k)1558 561 y Fg(S)1589 542 y Ff(T)1586 572 y(k)1616 561 y Fg(v)363 630 y Fe(\000)p Fg(\015)422 637 y Ff(k)443 630 y Fg(u)469 611 y Ff(T)496 630 y Fg(S)524 637 y Ff(k)546 630 y Fg(R)581 611 y Fc(\000)p Ff(T)581 644 y(k)635 630 y Fg(Y)672 611 y Ff(T)661 641 y(k)699 630 y Fg(v)0 732 y Fm(can)k(b)q(e)g(seen)h(to)e(require)h(\(4)p Fg(m)7 b Fm(+)g(1\))p Fg(n)g Fm(+)g(2)p Fg(m)755 715 y Fk(2)782 732 y Fm(+)g Fg(O)q Fm(\()p Fg(m)p Fm(\))14 b(m)o(ultiplications.)22 b(This)14 b(is)g(only)g(sligh)o(tly)h(less)0 788 y(exp)q(ensiv)o(e)j(than)e (computing)i Fg(H)578 795 y Ff(k)599 788 y Fg(v)g Fm(and)e(then)h(taking)g (the)f(inner)i(pro)q(duct)f(of)f(the)h(result)g(with)f Fg(u)p Fm(,)0 845 y(whic)o(h)g(w)o(ould)g(cost)e(\(4)p Fg(m)c Fm(+)g(2\))p Fg(n)g Fm(+)g Fg(O)q Fm(\()p Fg(m)709 828 y Fk(2)729 845 y Fm(\))k(m)o(ultiplications.)0 926 y Fn(The)k(Pro)q(duct)g Fg(A)345 910 y Ff(T)372 926 y Fg(H)410 933 y Ff(k)431 926 y Fg(A)p Fn(.)0 983 y Fm(A)g(related)g(computation)f(is)h(the)g(problem)h(of)e(computing)h (the)g(matrix)f Fg(A)1328 966 y Ff(T)1356 983 y Fg(H)1394 990 y Ff(k)1415 983 y Fg(A)g Fm(where)h Fg(A)g Fm(is)g(an)0 1039 y Fg(n)t Fe(\002)t Fg(t)c Fm(matrix)e(with)g Fg(t)h Fe(\024)g Fg(n)p Fm(.)20 b(This)13 b(computation)f(o)q(ccurs)g(when)h(solving)g(the)g (constrained)g(nonlinear)0 1096 y(optimization)j(problem,)720 1152 y(minimize)43 b Fg(f)5 b Fm(\()p Fg(x)p Fm(\))603 b(\(3)p Fg(:)p Fm(15\))670 1236 y(sub)s(ject)20 b(to)40 b Fg(c)p Fm(\()p Fg(x)p Fm(\))11 b(=)i(0)554 b(\(3)p Fg(:)p Fm(16\))0 1319 y(with)17 b Fg(n)h Fm(v)m(ariables)g(and)f Fg(t)h Fm(constrain)o(ts.)25 b(This)17 b(problem)h(is)g(frequen)o(tly)f(solv)o(ed)h(b)o(y)f(the)g(sequen)o (tial)0 1375 y(quadratic)12 b(programming)f(metho)q(d,)i(whic)o(h)f(at)g(ev)o (ery)g(iteration)g(solv)o(es)g(a)f(subproblem)i(of)f(the)g(form)620 1478 y(minimize)43 b Fg(g)864 1459 y Ff(T)862 1489 y(k)891 1478 y Fg(d)9 b Fm(+)975 1460 y Fk(1)p 975 1467 V 975 1493 a(2)997 1478 y Fg(d)1021 1459 y Ff(T)1049 1478 y Fg(B)1083 1485 y Ff(k)1104 1478 y Fg(d)504 b Fm(\(3)p Fg(:)p Fm(17\))631 1584 y(sub)s(ject)20 b(to)40 b Fg(A)909 1566 y Ff(T)909 1596 y(k)936 1584 y Fg(d)13 b Fm(=)g Fe(\000)p Fg(c)1076 1591 y Ff(k)1097 1584 y Fg(;)522 b Fm(\(3)p Fg(:)p Fm(18\))0 1668 y(where)18 b Fg(A)168 1675 y Ff(k)207 1668 y Fm(is)g(the)g(matrix)f(of)h (constrain)o(t)f(gradien)o(ts)g(at)g(the)h(curren)o(t)g(iterate)f Fg(x)1427 1675 y Ff(k)1449 1668 y Fm(,)g Fg(c)1499 1675 y Ff(k)1538 1668 y Fm(is)h(a)g(v)o(ector)0 1724 y(of)d(length)i Fg(t)p Fm(,)f(and)f Fg(B)359 1731 y Ff(k)395 1724 y Fm(=)f Fg(H)486 1705 y Fc(\000)p Fk(1)482 1738 y Ff(k)548 1724 y Fm(is)i(an)g(appro)o (ximation)g(to)f(the)g(Hessian)i(of)e(the)h(Lagrangian)g(of)f(the)0 1781 y(problem.)21 b(If)15 b Fg(A)274 1788 y Ff(k)311 1781 y Fm(has)g(full)h(rank,)f(the)g(solution)h(to)f(\(3.17\)-\(3.18)o(\))d(can)k (b)q(e)f(expressed)h(as)677 1883 y Fg(d)c Fm(=)h Fe(\000)p Fg(H)834 1890 y Ff(k)855 1883 y Fm(\()p Fg(g)895 1890 y Ff(k)926 1883 y Fm(+)e Fg(A)1006 1890 y Ff(k)1027 1883 y Fg(\025)p Fm(\))560 b(\(3)p Fg(:)p Fm(19\))0 1985 y(where)15 b(the)h(Lagrange)e(m)o(ultiplier)k Fg(\025)d Fm(satis\014es)571 2087 y(\()p Fg(A)623 2068 y Ff(T)623 2098 y(k)650 2087 y Fg(H)688 2094 y Ff(k)709 2087 y Fg(A)743 2094 y Ff(k)764 2087 y Fm(\))p Fg(\025)d Fm(=)h Fe(\000)p Fg(A)938 2068 y Ff(T)938 2098 y(k)966 2087 y Fg(H)1004 2094 y Ff(k)1025 2087 y Fg(g)1047 2094 y Ff(k)1078 2087 y Fm(+)e Fg(c)1144 2094 y Ff(k)1165 2087 y Fg(:)454 b Fm(\(3)p Fg(:)p Fm(20\))0 2189 y(Let)19 b(us)h(supp)q(ose)g(that)e Fg(H)463 2196 y Ff(k)504 2189 y Fm(is)h(a)g(limited)i(memory)e(matrix)g(represen)o(ted)h(in)g(the)f (compact)g(form)0 2246 y(\(3.1\).)j(Then)17 b(the)f(matrix)g Fg(A)512 2229 y Ff(T)512 2259 y(k)540 2246 y Fg(H)578 2253 y Ff(k)599 2246 y Fg(A)633 2253 y Ff(k)670 2246 y Fm(ma)o(y)g(b)q(e)h (e\016cien)o(tly)h(computed)e(b)o(y)h(\014rst)f(computing)h Fg(S)1666 2229 y Ff(T)1663 2259 y(k)1693 2246 y Fg(A)1727 2253 y Ff(k)0 2302 y Fm(and)h Fg(Y)127 2286 y Ff(T)117 2315 y(k)155 2302 y Fg(A)189 2309 y Ff(k)210 2302 y Fm(,)g(whic)o(h)h(require)f(2)p Fg(mnt)g Fm(m)o(ultiplications,)j(then)d Fg(R)1120 2283 y Fc(\000)p Fk(1)1120 2316 y Ff(k)1167 2302 y Fg(S)1198 2286 y Ff(T)1195 2315 y(k)1225 2302 y Fg(A)1259 2309 y Ff(k)1280 2302 y Fm(,)g(requiring)1512 2284 y Fk(1)p 1512 2291 V 1512 2318 a(2)1534 2302 y Fg(m)1574 2286 y Fk(2)1594 2302 y Fg(t)g Fm(m)o(ulti-)0 2359 y(plications,)e(and)g (then)f(computing)85 2461 y Fg(\015)109 2468 y Ff(k)130 2461 y Fg(A)164 2442 y Ff(T)164 2472 y(k)192 2461 y Fg(A)226 2468 y Ff(k)257 2461 y Fm(+)c(\()p Fg(R)356 2442 y Fc(\000)p Fk(1)356 2475 y Ff(k)402 2461 y Fg(S)433 2442 y Ff(T)430 2472 y(k)460 2461 y Fg(A)494 2468 y Ff(k)516 2461 y Fm(\))534 2442 y Ff(T)561 2461 y Fm(\()p Fg(D)617 2468 y Ff(k)648 2461 y Fm(+)f Fg(\015)717 2468 y Ff(k)738 2461 y Fg(Y)775 2442 y Ff(T)764 2472 y(k)802 2461 y Fg(Y)828 2468 y Ff(k)850 2461 y Fm(\)\()p Fg(R)921 2442 y Fc(\000)p Fk(1)921 2475 y Ff(k)967 2461 y Fg(S)998 2442 y Ff(T)995 2472 y(k)1025 2461 y Fg(A)1059 2468 y Ff(k)1081 2461 y Fm(\))f Fe(\000)i Fm(2)p Fg(\015)1201 2468 y Ff(k)1222 2461 y Fg(A)1256 2442 y Ff(T)1256 2472 y(k)1283 2461 y Fg(Y)1309 2468 y Ff(k)1331 2461 y Fg(R)1366 2442 y Fc(\000)p Fk(1)1366 2475 y Ff(k)1413 2461 y Fg(S)1444 2442 y Ff(T)1441 2472 y(k)1471 2461 y Fg(A)1505 2468 y Ff(k)1526 2461 y Fg(;)93 b Fm(\(3)p Fg(:)p Fm(21\))851 2603 y(13)p eop %%Page: 14 15 bop 0 120 a Fm(whic)o(h)13 b(requires)g Fg(m)335 104 y Fk(2)355 120 y Fg(t)t Fm(+)421 102 y Fk(3)p 421 109 18 2 v 421 136 a(2)443 120 y Fg(mt)499 104 y Fk(2)524 120 y Fm(+)569 102 y Fk(1)p 569 109 V 569 136 a(2)591 120 y Fm(\()p Fg(t)625 104 y Fk(2)649 120 y Fm(+)t Fg(t)p Fm(\))p Fg(n)t Fm(+)t Fg(O)q Fm(\(\(max)r Fe(f)p Fg(m;)8 b(t)p Fe(g)p Fm(\))1092 104 y Fk(2)1110 120 y Fm(\))k(m)o(ultiplications.)22 b(Ignoring)12 b(lo)o(w)o(er)0 177 y(order)j(terms,)f(this)i(is)g(a)e(total)h(of)566 295 y(\(2)p Fg(m)9 b Fm(+)707 277 y Fk(1)p 707 284 V 707 311 a(2)729 295 y Fg(t)i Fm(+)806 277 y Fk(1)p 806 284 V 806 311 a(2)829 295 y Fm(\))p Fg(tn)f Fm(+)951 265 y(3)p 951 285 23 2 v 951 326 a(2)979 295 y(\()p Fg(m)f Fm(+)i Fg(t)p Fm(\))p Fg(mt)0 408 y Fm(m)o(ultiplications.)24 b(As)15 b(long)h(as)f Fg(m)h Fm(and)g Fg(t)g Fm(are)f(fairly)h(small)h(this)f(is)g(not)f(extremely)i(exp)q(ensiv)o (e)g(and)0 464 y(is)j(m)o(uc)o(h)g(less)g(than)f(the)h(cost)f(of)g(computing) h(the)f(matrix)g Fg(H)1112 471 y Ff(k)1134 464 y Fg(A)1168 471 y Ff(k)1208 464 y Fm(\014rst,)h(and)g(then)g(m)o(ultiplying)0 521 y(b)o(y)e Fg(A)100 504 y Ff(T)100 534 y(k)128 521 y Fm(.)29 b(T)l(o)18 b(solv)o(e)h(\(3.20\))d(requires)j(the)g(Cholesky)g(factorization) f(of)g Fg(A)1299 504 y Ff(T)1299 534 y(k)1326 521 y Fg(H)1364 528 y Ff(k)1385 521 y Fg(A)1419 528 y Ff(k)1459 521 y Fm(whic)o(h)h(requires) 5 559 y Fk(1)p 5 566 18 2 v 5 593 a(6)28 577 y Fg(t)44 561 y Fk(3)79 577 y Fm(m)o(ultiplications.)j(The)15 b(other)f(matrix)h(v)o(ector) f(pro)q(ducts)h(required)h(in)g(\(3.19\))d(and)i(\(3.20\))d(cost)0 634 y(ab)q(out)20 b(\(2)p Fg(t)13 b Fm(+)h(4)p Fg(m)p Fm(\))p Fg(n)p Fm(,)21 b(if)f(certain)h(quan)o(tities)g(computed)f(in)h(other)f (parts)f(of)h(the)g(pro)q(cedure)h(are)0 690 y(sa)o(v)o(ed)15 b(and)g(reused)h(appropriately)l(.)0 771 y Fn(Op)q(erations)j(with)f Fg(H)426 778 y Ff(k)464 771 y Fn(and)g(sparse)f(constrain)o(ts.)0 828 y Fm(W)l(e)i(no)o(w)g(consider)g(computations)g(similar)h(to)f(those)f (in)i(the)f(previous)h(section)g(but)f(where)g(the)0 884 y(v)o(ectors)f(and)h (matrices)g(m)o(ultiplying)i Fg(H)720 891 y Ff(k)760 884 y Fm(are)e(sparse)f(\(but)h(recall)h(that)e Fg(H)1350 891 y Ff(k)1390 884 y Fm(is)i(dense\).)31 b(This)19 b(is)0 941 y(an)h(imp)q(ortan)o(t)f(case) g(b)q(ecause,)j(ev)o(en)e(though)f Fg(g)862 948 y Ff(k)883 941 y Fg(;)8 b(S)932 948 y Ff(k)953 941 y Fm(,)20 b(and)g Fg(Y)1105 948 y Ff(k)1146 941 y Fm(are)g(not)f(lik)o(ely)i(to)e(b)q(e)i(sparse,)f(it)0 997 y(is)f(v)o(ery)f(common)g(to)f(ha)o(v)o(e)h(constrained)h(optimization)g (problems)g(where)g(the)f(gradien)o(ts)g(of)g(the)0 1054 y(constrain)o(ts,)e (and)g(th)o(us)g(the)h(matrix)f Fg(A)g Fm(in)h(\(3.18\))e(are)h(sparse.)23 b(A)16 b(sp)q(ecial)j(case)d(in)h(whic)o(h)g(w)o(e)f(are)0 1110 y(v)o(ery)k(in)o(terested)g(is)g(the)g(case)g(of)f(a)h(minimization)i (sub)s(ject)e(to)f(b)q(ound)i(constrain)o(ts,)f(where)g(the)0 1167 y(matrices)f(dealt)h(with)g(are)f(actually)i(submatrices)f(of)f(the)g (iden)o(tit)o(y)l(.)34 b(Signi\014can)o(t)21 b(reductions)f(in)0 1223 y(computational)c(cost)e(result)i(in)g(suc)o(h)f(problems)h(if)g (e\016cien)o(t)g(sparse)f(storage)f(is)h(used.)71 1280 y(The)j(pro)q(duct)h Fg(H)379 1287 y Ff(k)400 1280 y Fg(e)421 1287 y Ff(i)453 1280 y Fm(requires)g(2)p Fg(mn)12 b Fm(+)h Fg(O)q Fm(\()p Fg(m)871 1263 y Fk(2)890 1280 y Fm(\))18 b(m)o(ultiplications.)31 b(This)19 b(is)g(easy)f(to)f(see)i(from)0 1336 y(\(3.13\),)11 b(since)i Fg(S)281 1320 y Ff(T)278 1349 y(k)308 1336 y Fg(e)329 1343 y Ff(i)356 1336 y Fm(and)g Fg(Y)478 1320 y Ff(T)468 1349 y(k)506 1336 y Fg(e)527 1343 y Ff(i)554 1336 y Fm(require)g(only)g Fg(O)q Fm(\()p Fg(m)p Fm(\))f(indexing)i(op)q(erations.)19 b(F)l(or)12 b(the)h(same)f(reason,)0 1392 y(w)o(e)j(see)g(from)g(\(3.14\))e (that)i Fg(e)499 1376 y Ff(T)499 1405 y(i)526 1392 y Fg(H)564 1399 y Ff(k)585 1392 y Fg(e)606 1399 y Ff(i)636 1392 y Fm(can)g(b)q(e)h (computed)g(with)f Fg(O)q Fm(\()p Fg(m)1188 1376 y Fk(2)1207 1392 y Fm(\))g(m)o(ultiplications.)71 1449 y(Consider)j(no)o(w)f Fg(A)394 1432 y Ff(T)422 1449 y Fg(H)460 1456 y Ff(k)481 1449 y Fg(A)h Fm(in)h(the)f(case)g(where)g Fg(A)g Fm(is)g(an)g Fg(n)12 b Fe(\002)g Fg(t)19 b Fm(sparse)e(matrix)h(with)g Fg(n)1613 1456 y Ff(A)1660 1449 y Fm(non-)0 1505 y(zeros.)28 b(W)l(e)18 b(p)q(erform)g(this)g(computation)g(b)o(y)g(\(3.21\).)27 b(The)18 b(pro)q(ducts)g Fg(S)1297 1489 y Ff(T)1294 1519 y(k)1324 1505 y Fg(A)g Fm(and)h Fg(Y)1504 1489 y Ff(T)1494 1519 y(k)1532 1505 y Fg(A)f Fm(together)0 1562 y(require)i(2)p Fg(mn)247 1569 y Ff(A)294 1562 y Fm(m)o(ultiplications.)33 b(The)20 b(bac)o(k-solv)o(e) f Fg(R)986 1543 y Fc(\000)p Fk(1)986 1576 y Ff(k)1033 1562 y Fg(S)1064 1545 y Ff(T)1061 1575 y(k)1091 1562 y Fg(A)g Fm(requires)1323 1544 y Fk(1)p 1323 1551 V 1323 1577 a(2)1346 1562 y Fg(mt)1402 1545 y Fk(2)1441 1562 y Fm(m)o(ultiplications,)0 1618 y(and)j(the)g(rest)g (of)f(the)h(op)q(erations)g(require)h(2)p Fg(mt)886 1602 y Fk(2)921 1618 y Fm(+)14 b Fg(m)1010 1602 y Fk(2)1030 1618 y Fg(t)h Fm(+)g Fg(O)q Fm(\(\()p Fg(max)p Fe(f)p Fg(m;)8 b(t)p Fe(g)p Fm(\))1414 1602 y Fk(2)1454 1618 y Fm(m)o(ultiplications)0 1675 y(plus)19 b(the)g(op)q(erations)g(of)f Fg(A)495 1658 y Ff(T)522 1675 y Fg(A)h Fm(whic)o(h)g(cost)f(at)g(most)f Fg(tn)1021 1682 y Ff(A)1069 1675 y Fm(m)o(ultiplications.)32 b(Th)o(us)18 b(the)h(total)e(is)0 1731 y Fg(O)q Fm(\()p Fg(max)p Fe(f)p Fg(m;)8 b(t)p Fe(g)p Fm(\))p Fg(n)312 1738 y Ff(A)352 1731 y Fm(+)13 b(\(2)p Fg(t)f Fm(+)522 1713 y Fk(3)p 522 1720 V 522 1747 a(2)545 1731 y Fg(m)p Fm(\))p Fg(mt)h Fm(+)g Fg(O)q Fm(\(\()p Fg(max)p Fe(f)p Fg(m;)8 b(t)p Fe(g)p Fm(\))1023 1715 y Fk(2)1040 1731 y Fm(\))19 b(.)32 b(Th)o(us)19 b(w)o(e)g(see)g(that,)g (while)i(in)f(the)0 1788 y(previous)c(section)g(the)g(computational)g (e\013ort)e(in)i(most)f(tasks)g(w)o(as)f(roughly)i(prop)q(ortional)g(to)f (the)0 1844 y(n)o(um)o(b)q(er)f(of)g(v)m(ariables)h Fg(n)p Fm(,)f(in)g(the)g(sparse)g(case)g(it)g(is)g(prop)q(ortional)g(to)g(the)g(n)o (um)o(b)q(er)g(of)f(non-zeros)h(in)0 1901 y(the)h(sparse)g(arra)o(y)f(under)i (consideration.)0 2057 y Fn(3.2.)h(Op)q(erations)h(with)h Fg(B)521 2064 y Ff(k)71 2138 y Fm(W)l(e)h(no)o(w)f(consider)i(the)f(direct)h(Hessian)g (appro)o(ximation)f Fg(B)1165 2145 y Ff(k)1187 2138 y Fm(.)34 b(T)l(o)20 b(tak)o(e)f(adv)m(an)o(tage)g(of)h(the)0 2194 y(decomp)q(osition)c (\(2.25\),)d(w)o(e)i(rewrite)h(\(3.5\))d(as)39 2341 y Fg(B)73 2348 y Ff(k)108 2341 y Fm(=)g Fg(\033)182 2348 y Ff(k)203 2341 y Fg(I)h Fe(\000)282 2294 y Fd(h)323 2341 y Fg(Y)349 2348 y Ff(k)412 2341 y Fg(\033)438 2348 y Ff(k)459 2341 y Fg(S)487 2348 y Ff(k)529 2294 y Fd(i)556 2269 y(")601 2317 y Fe(\000)p Fg(D)675 2293 y Fk(1)p Ff(=)p Fk(2)674 2331 y Ff(k)772 2317 y Fg(D)811 2293 y Fc(\000)p Fk(1)p Ff(=)p Fk(2)810 2331 y Ff(k)893 2317 y Fg(L)924 2301 y Ff(T)924 2331 y(k)654 2374 y Fm(0)156 b Fg(J)862 2357 y Ff(T)858 2387 y(k)973 2269 y Fd(#)997 2281 y Fc(\000)p Fk(1)1044 2269 y Fd(")1146 2313 y Fg(D)1185 2289 y Fk(1)p Ff(=)p Fk(2)1184 2327 y Ff(k)1351 2313 y Fm(0)1089 2379 y Fe(\000)p Fg(L)1155 2386 y Ff(k)1177 2379 y Fg(D)1216 2355 y Fc(\000)p Fk(1)p Ff(=)p Fk(2)1215 2393 y Ff(k)1339 2379 y Fg(J)1364 2386 y Ff(k)1407 2269 y Fd(#)1431 2277 y Fc(\000)p Fk(1)1486 2269 y Fd(")1551 2313 y Fg(Y)1588 2296 y Ff(T)1577 2326 y(k)1531 2369 y Fg(\033)1557 2376 y Ff(k)1578 2369 y Fg(S)1609 2353 y Ff(T)1606 2382 y(k)1657 2269 y Fd(#)1688 2341 y Fg(;)1632 2434 y Fm(\(3)p Fg(:)p Fm(22\))851 2603 y(14)p eop %%Page: 15 16 bop 0 120 a Fm(where)14 b Fg(J)155 127 y Ff(k)191 120 y Fm(is)g(de\014ned)i (b)o(y)d(\(2.26\).)18 b(W)l(e)c(use)g(this)h(expression,)g(b)q(oth)f(in)h (the)f(sparse)f(and)i(dense)f(case,)0 177 y(to)h(compute)g(sev)o(eral)g(pro)q (ducts)h(in)o(v)o(olving)g Fg(B)806 184 y Ff(k)828 177 y Fm(.)0 257 y Fn(Up)q(date)j(of)e Fg(B)280 264 y Ff(k)320 257 y Fn(and)h(the)f(pro)q (duct)i Fg(B)743 264 y Ff(k)764 257 y Fg(v)r Fn(.)0 314 y Fm(This)24 b(computation)f(is)h(required)g(when)f(applying)i(limited)g(memory)e(metho)q (ds)g(to)f(solv)o(e)i(con-)0 370 y(strained)17 b(optimization)h(problem.)25 b(It)17 b(o)q(ccurs,)h(for)e(example,)h(in)h(the)f(algorithm)g(for)f (nonlinearly)0 426 y(constrained)21 b(problems)g(dev)o(elop)q(ed)h(b)o(y)e (Mahidhara)g(and)g(Lasdon)h(\(1990\),)e(and)i(in)g(the)f(primal)0 483 y(limited)i(memory)d(algorithm)h(for)f(b)q(ound)i(constrained)g (optimization)g(describ)q(ed)h(b)o(y)d(Byrd,)i(Lu)0 539 y(and)15 b(No)q(cedal)i(\(1993\).)71 596 y(The)j(follo)o(wing)g(pro)q(cedure,)h(whic)o (h)g(is)f(based)g(on)g(the)g(represen)o(tation)f(\(3.22\),)g(describ)q(es)i (in)0 652 y(detail)15 b(the)g Fg(k)q Fm(-th)f(step)h(of)f(an)g(iteration)h (that)e(\014rst)h(up)q(dates)h Fg(B)1087 659 y Ff(k)1123 652 y Fm(and)g(then)g(computes)f(the)h(pro)q(duct)0 709 y Fg(B)34 716 y Ff(k)56 709 y Fg(v)i Fm(for)d(an)h(arbitrary)g(v)o(ector)f Fg(v)r Fm(.)0 813 y Fn(Algorithm)k(3.2)71 870 y Fm(Let)24 b Fg(x)187 877 y Ff(k)232 870 y Fm(b)q(e)h(the)f(curren)o(t)g(iterate,)i(and)e (assume)g(that)f(the)h(matrices)g Fg(S)1401 877 y Ff(k)q Fc(\000)p Fk(1)1468 870 y Fm(,)h Fg(Y)1532 877 y Ff(k)q Fc(\000)p Fk(1)1599 870 y Fm(,)h Fg(L)1669 877 y Ff(k)q Fc(\000)p Fk(1)1735 870 y Fm(,)0 926 y Fg(S)31 910 y Ff(T)28 939 y(k)q Fc(\000)p Fk(1)94 926 y Fg(S)122 933 y Ff(k)q Fc(\000)p Fk(1)189 926 y Fm(,)14 b(and)g Fg(D)341 933 y Ff(k)q Fc(\000)p Fk(1)421 926 y Fm(ha)o(v)o(e)g(b)q (een)h(stored.)k(The)c(v)o(ectors)e Fg(s)1052 933 y Ff(k)q Fc(\000)p Fk(1)1119 926 y Fg(;)8 b(y)1162 933 y Ff(k)q Fc(\000)p Fk(1)1242 926 y Fm(ha)o(v)o(e)14 b(just)g(b)q(een)h(computed,)0 983 y(and)g(the)h(v)o(ector)e Fg(v)j Fm(is)f(giv)o(en.)56 1086 y(1.)22 b(Obtain)16 b Fg(S)296 1093 y Ff(k)317 1086 y Fg(;)8 b(Y)364 1093 y Ff(k)385 1086 y Fm(,)15 b(b)o(y)g(up)q(dating)h Fg(S)697 1093 y Ff(k)q Fc(\000)p Fk(1)778 1086 y Fm(and)g Fg(Y)893 1093 y Ff(k)q Fc(\000)p Fk(1)960 1086 y Fm(.)56 1179 y(2.)22 b(Compute)15 b Fg(L)342 1186 y Ff(k)363 1179 y Fg(;)8 b(S)415 1162 y Ff(T)412 1192 y(k)441 1179 y Fg(S)469 1186 y Ff(k)505 1179 y Fm(and)16 b Fg(D)632 1186 y Ff(k)653 1179 y Fm(.)56 1272 y(3.)22 b(Compute)15 b Fg(\033)337 1279 y Ff(k)358 1272 y Fm(;)g(for)f(example)656 1371 y Fg(\033)682 1378 y Ff(k)716 1371 y Fm(=)f Fg(y)786 1378 y Ff(k)q Fc(\000)p Fk(1)852 1352 y Ff(T)880 1371 y Fg(s)901 1378 y Ff(k)q Fc(\000)p Fk(1)968 1371 y Fg(=s)1012 1378 y Ff(k)q Fc(\000)p Fk(1)1078 1352 y Ff(T)1106 1371 y Fg(s)1127 1378 y Ff(k)q Fc(\000)p Fk(1)1193 1371 y Fg(:)426 b Fm(\(3)p Fg(:)p Fm(23\))56 1489 y(4.)22 b(Compute)15 b(the)g(Cholesky)h(factorization)f(of)f Fg(\033)925 1496 y Ff(k)947 1489 y Fg(S)978 1472 y Ff(T)975 1502 y(k)1005 1489 y Fg(S)1033 1496 y Ff(k)1064 1489 y Fm(+)c Fg(L)1140 1496 y Ff(k)1162 1489 y Fg(D)1201 1470 y Fc(\000)p Fk(1)1200 1503 y Ff(k)1248 1489 y Fg(L)1279 1472 y Ff(T)1279 1502 y(k)1321 1489 y Fm(to)15 b(obtain)g Fg(J)1543 1496 y Ff(k)1565 1489 y Fg(J)1594 1472 y Ff(T)1590 1502 y(k)1622 1489 y Fm(.)56 1582 y(5.)22 b(Compute)770 1668 y Fg(p)12 b Fm(=)853 1597 y Fd(")919 1640 y Fg(Y)955 1624 y Ff(T)945 1653 y(k)983 1640 y Fg(v)898 1697 y(\033)924 1704 y Ff(k)945 1697 y Fg(S)976 1680 y Ff(T)973 1710 y(k)1003 1697 y Fg(v)1048 1597 y Fd(#)1080 1668 y Fg(:)56 1799 y Fm(6.)22 b(P)o(erform)14 b(a)h(forw)o(ard)f(and)h(then)g(a)g(bac)o(kw) o(ard)g(solv)o(e)g(to)g(obtain)404 1932 y Fg(p)e Fm(:=)500 1860 y Fd(")545 1908 y Fe(\000)p Fg(D)619 1884 y Fk(1)p Ff(=)p Fk(2)618 1922 y Ff(k)716 1908 y Fg(D)755 1884 y Fc(\000)p Fk(1)p Ff(=)p Fk(2)754 1922 y Ff(k)837 1908 y Fg(L)868 1892 y Ff(T)868 1921 y(k)598 1964 y Fm(0)156 b Fg(J)806 1948 y Ff(T)802 1978 y(k)917 1860 y Fd(#)941 1871 y Fc(\000)p Fk(1)988 1860 y Fd(")1090 1903 y Fg(D)1129 1879 y Fk(1)p Ff(=)p Fk(2)1128 1917 y Ff(k)1295 1903 y Fm(0)1033 1969 y Fe(\000)p Fg(L)1099 1976 y Ff(k)1121 1969 y Fg(D)1160 1945 y Fc(\000)p Fk(1)p Ff(=)p Fk(2)1159 1983 y Ff(k)1283 1969 y Fg(J)1308 1976 y Ff(k)1351 1860 y Fd(#)1375 1868 y Fc(\000)p Fk(1)1422 1932 y Fg(p:)56 2083 y Fm(7.)22 b(Compute)638 2144 y Fg(B)672 2151 y Ff(k)694 2144 y Fg(v)14 b Fm(=)f Fg(\033)804 2151 y Ff(k)826 2144 y Fg(v)f Fe(\000)905 2097 y Fd(h)945 2144 y Fg(Y)971 2151 y Ff(k)1035 2144 y Fg(\033)1061 2151 y Ff(k)1082 2144 y Fg(S)1113 2128 y Ff(T)1110 2157 y(k)1161 2097 y Fd(i)1188 2144 y Fg(p:)71 2253 y Fm(The)18 b(\014rst)g(step)g(of)f (this)i(pro)q(cedure,)g(in)g(whic)o(h)g(the)f(oldest)g(columns)h(of)f(the)g (matrices)g Fg(S)1669 2260 y Ff(k)q Fc(\000)p Fk(1)1735 2253 y Fm(,)0 2309 y Fg(Y)26 2316 y Ff(k)q Fc(\000)p Fk(1)109 2309 y Fm(are)e(replaced)i(b)o(y)e(the)h(v)o(ectors)f Fg(s)688 2316 y Ff(k)q Fc(\000)p Fk(1)754 2309 y Fm(,)h(and)f Fg(y)895 2316 y Ff(k)q Fc(\000)p Fk(1)962 2309 y Fm(,)g(do)q(es)h(not)f(require)h(an)o(y)f (arithmetic.)25 b(Step)0 2366 y(2)18 b(requires)h(2)p Fg(m)f Fm(inner)i(pro)q(ducts)e(to)g(form)g(the)g(new)h(columns)g(of)f(matrices)g Fg(L)1406 2373 y Ff(k)1428 2366 y Fm(,)g Fg(S)1490 2349 y Ff(T)1487 2379 y(k)1517 2366 y Fg(S)1545 2373 y Ff(k)1585 2366 y Fm(and)g Fg(D)1714 2373 y Ff(k)1735 2366 y Fm(,)0 2422 y(whic)o(h)d(cost)f(2)p Fg(mn)h Fm(m)o(ultiplications.)22 b(The)15 b(c)o(hoice)g(of)f Fg(\033)958 2429 y Ff(k)994 2422 y Fm(in)h(step)g(3)f(costs)g(only)h(one)g(m) o(ultiplication)0 2479 y(since)24 b(b)q(oth)e Fg(y)255 2486 y Ff(k)q Fc(\000)p Fk(1)322 2462 y Ff(T)350 2479 y Fg(s)371 2486 y Ff(k)q Fc(\000)p Fk(1)460 2479 y Fm(and)h Fg(s)577 2486 y Ff(k)q Fc(\000)p Fk(1)643 2462 y Ff(T)671 2479 y Fg(s)692 2486 y Ff(k)q Fc(\000)p Fk(1)781 2479 y Fm(ha)o(v)o(e)f(b)q(een)i(calculated) g(in)g(step)e(2.)42 b(In)23 b(step)g(4)f(the)851 2603 y(15)p eop %%Page: 16 17 bop 0 120 a Fm(Cholesky)17 b(factorization)g(of)f(the)h(p)q(ositiv)o(e)h (de\014nite)g(matrix)f Fg(\033)1104 127 y Ff(k)1125 120 y Fg(S)1156 104 y Ff(T)1153 133 y(k)1183 120 y Fg(S)1211 127 y Ff(k)1244 120 y Fm(+)11 b Fg(L)1321 127 y Ff(k)1343 120 y Fg(D)1382 101 y Fc(\000)p Fk(1)1381 134 y Ff(k)1429 120 y Fg(L)1460 104 y Ff(T)1460 133 y(k)1504 120 y Fm(costs)16 b Fg(O)q Fm(\()p Fg(m)1711 104 y Fk(3)1730 120 y Fm(\))0 177 y(m)o(ultiplications.)22 b(Step)15 b(5)e(costs)h(2)p Fg(mn)g Fm(m)o(ultiplications.)22 b(The)15 b(forw)o(ard)e(and)h(the)g(bac)o(kw)o(ard)g(solv)o(es)0 233 y(of)c(2)p Fg(m)q Fe(\002)q Fm(2)p Fg(m)h Fm(triangular)g(systems)f(in)h (step)g(6)f(cost)h Fg(O)q Fm(\()p Fg(m)950 217 y Fk(2)969 233 y Fm(\))f(m)o(ultiplications.)21 b(Step)11 b(7)f(costs)g(\(2)p Fg(m)q Fm(+)q(1\))p Fg(n)0 289 y Fm(m)o(ultiplications.)21 b(In)12 b(summary)l(,)g(this)g(pro)q(cedure)g(costs)f(2)p Fg(mn)s Fm(+)s Fg(O)q Fm(\()p Fg(m)1209 273 y Fk(3)1228 289 y Fm(\))g(m)o (ultiplications)j(from)d(step)0 346 y(1)i(to)g(step)g(4,)h(where)f(the)h (matrix)f Fg(B)620 353 y Ff(k)655 346 y Fm(is)h(de\014ned;)h(and)f(costs)f (\(4)p Fg(m)7 b Fm(+)g(1\))p Fg(n)g Fm(+)g Fg(O)q Fm(\()p Fg(m)1405 329 y Fk(2)1422 346 y Fm(\))14 b(m)o(ultiplications)0 402 y(from)g(step)i(5)f (to)f(step)h(7,)g(where)g(the)h(pro)q(duct)f Fg(B)858 409 y Ff(k)880 402 y Fg(v)i Fm(is)e(calculated.)0 484 y Fn(The)j(w)o(eigh)o(ted)f (scalar)h(pro)q(duct)g Fg(v)695 467 y Ff(T)723 484 y Fg(B)757 491 y Ff(k)778 484 y Fg(v)r Fn(.)0 540 y Fm(This)i(pro)q(duct)g(o)q(ccurs,)h (for)e(example,)i(in)g(the)e(conjugate)h(gradien)o(t)f(inner-iteration)i(as)f (w)o(ell)g(as)0 597 y(in)g(the)g(Cauc)o(h)o(y)f(p)q(oin)o(t)h(computation)f (of)g(the)g(primal)i(algorithm)e(describ)q(ed)i(b)o(y)f(Byrd,)g(Lu)g(and)0 653 y(No)q(cedal)c(\(1993\).)i(Using)e(\(3.22\))d(w)o(e)i(ha)o(v)o(e)53 800 y Fg(v)77 781 y Ff(T)104 800 y Fg(B)138 807 y Ff(k)160 800 y Fg(v)f Fm(=)f Fg(\033)270 807 y Ff(k)291 800 y Fg(v)315 781 y Ff(T)342 800 y Fg(v)f Fe(\000)e Fg(v)445 781 y Ff(T)473 800 y Fg(W)522 781 y Ff(T)516 811 y(k)549 728 y Fd(")594 776 y Fe(\000)p Fg(D)668 752 y Fk(1)p Ff(=)p Fk(2)667 790 y Ff(k)765 776 y Fg(D)804 752 y Fc(\000)p Fk(1)p Ff(=)p Fk(2)803 790 y Ff(k)887 776 y Fg(L)918 760 y Ff(T)918 789 y(k)648 833 y Fm(0)156 b Fg(J)856 816 y Ff(T)852 846 y(k)966 728 y Fd(#)990 739 y Fc(\000)p Fk(1)1037 728 y Fd(")1140 771 y Fg(D)1179 747 y Fk(1)p Ff(=)p Fk(2)1178 785 y Ff(k)1344 771 y Fm(0)1082 837 y Fe(\000)p Fg(L)1148 844 y Ff(k)1170 837 y Fg(D)1209 813 y Fc(\000)p Fk(1)p Ff(=)p Fk(2)1208 851 y Ff(k)1333 837 y Fg(J)1358 844 y Ff(k)1400 728 y Fd(#)1424 736 y Fc(\000)p Fk(1)1471 800 y Fg(W)1514 807 y Ff(k)1535 800 y Fg(v)r(;)60 b Fm(\(3)p Fg(:)p Fm(24\))0 936 y(where)704 1013 y Fg(W)747 1020 y Ff(k)781 1013 y Fm(=)829 941 y Fd(")894 985 y Fg(Y)931 969 y Ff(T)920 998 y(k)874 1041 y Fg(\033)900 1048 y Ff(k)921 1041 y Fg(S)952 1025 y Ff(T)949 1055 y(k)1000 941 y Fd(#)1032 1013 y Fg(:)0 1135 y Fm(W)l(e)20 b(\014rst)f(compute)g(and)h(store)f(the)h(matrix)f(v)o(ector)g(pro)q(ducts)g Fg(Y)1183 1118 y Ff(T)1172 1148 y(k)1211 1135 y Fg(v)r(;)8 b(\033)1282 1142 y Ff(k)1302 1135 y Fg(S)1333 1118 y Ff(T)1330 1148 y(k)1360 1135 y Fg(v)r Fm(,)20 b(whic)o(h)g(determine)0 1191 y Fg(W)43 1198 y Ff(k)64 1191 y Fg(v)r Fm(,)f(and)g(whic)o(h)h(require)f (2)p Fg(mn)g Fm(m)o(ultiplications.)33 b(Then)19 b(w)o(e)g(solv)o(e)g(t)o(w)o (o)e(2)p Fg(m)c Fe(\002)f Fm(2)p Fg(m)19 b Fm(triangular)0 1247 y(systems,)g(and)g(compute)h(the)f(scalar)g(pro)q(duct)g(of)g(t)o(w)o(o) e(2)p Fg(m)p Fm(-v)o(ectors;)j(all)g(of)f(these)g(cost)f(at)h(most)0 1304 y Fg(O)q Fm(\()p Fg(m)94 1287 y Fk(2)113 1304 y Fm(\))d(m)o (ultiplications.)25 b(The)16 b(last)g(part)g(is)g(to)g(compute)g Fg(\033)1072 1311 y Ff(k)1093 1304 y Fg(v)1117 1287 y Ff(T)1144 1304 y Fg(v)r Fm(,)g(and)g(subtract)f(the)i(previously)0 1360 y(computed)22 b(scalar)g(from)f(it.)39 b(The)22 b(total)f(cost)g(of)h(this)g (computation)f(is)i(\(2)p Fg(m)13 b Fm(+)i(1\))p Fg(n)f Fm(+)h Fg(O)q Fm(\()p Fg(m)1711 1344 y Fk(2)1730 1360 y Fm(\))0 1417 y(m)o(ultiplications.)28 b(Of)17 b(course)g(in)h(the)f(case)g Fg(v)g Fm(=)f Fg(g)885 1424 y Ff(k)906 1417 y Fm(,)h(whic)o(h)h(is)f(often)g (required,)h(using)g(previously)0 1473 y(computed)k(quan)o(tities)h(form)e (the)h(computation)f(of)h Fg(H)1000 1480 y Ff(k)1043 1473 y Fm(w)o(ould)g(allo)o(w)g(this)g(to)f(b)q(e)i(reduced)g(to)0 1530 y Fg(O)q Fm(\()p Fg(m)94 1513 y Fk(2)113 1530 y Fm(\).)0 1611 y Fn(Sparse)17 b(computations)i(with)f Fg(B)645 1618 y Ff(k)0 1668 y Fm(Calculations)i(in)o(v)o(olving)g(the)f(pro)q(duct)g(of)f Fg(B)807 1675 y Ff(k)848 1668 y Fm(and)g(sparse)h(v)o(ectors)f(in)o(v)o(olv)o (e)h(sa)o(vings)g(similar)h(to)0 1724 y(those)14 b(in)o(v)o(olving)h Fg(H)349 1731 y Ff(k)370 1724 y Fm(;)f(for)g(example,)h(computing)g Fg(B)914 1731 y Ff(k)935 1724 y Fg(e)956 1731 y Ff(i)985 1724 y Fm(requires)g(2)p Fg(mn)8 b Fm(+)g Fg(O)q Fm(\()p Fg(m)1390 1707 y Fk(3)1409 1724 y Fm(\))14 b(m)o(ultiplications.)0 1780 y(A)k(sp)q(ecial)h(but)f(imp)q(ortan)o(t)f(sparse)g(case)h(concerns)g (minimization)h(problems)g(sub)s(ject)e(to)g(b)q(ound)0 1837 y(constrain)o(ts,)k(in)h(whic)o(h)g(the)e(constrain)o(t)h(gradien)o(ts)f(are) h(submatrices)g(of)f(the)h(iden)o(tit)o(y)h(matrix.)0 1893 y(Minimizing)16 b(o)o(v)o(er)d(a)g(subspace)h(in)h(that)e(case)g(in)o(v)o (olv)o(es)h(computations)g(with)g(the)g(reduced)g(Hessian)0 1950 y(appro)o(ximation)317 1938 y(^)301 1950 y Fg(B)335 1957 y Ff(k)369 1950 y Fm(=)f Fg(Z)451 1933 y Ff(T)479 1950 y Fg(B)513 1957 y Ff(k)535 1950 y Fg(Z)s Fm(,)g(where)g Fg(Z)j Fm(is)d(an)g Fg(n)5 b Fe(\002)948 1941 y Fm(^)947 1950 y Fg(t)14 b Fm(matrix)f(whose)f (columns)i(are)f(unit)g(v)o(ectors.)0 2006 y(Th)o(us)i(the)g(subspace)h (problem)g(is)g(of)f(size)744 1998 y(^)744 2006 y Fg(t)p Fm(.)71 2063 y(T)l(o)f(express)312 2051 y(^)296 2063 y Fg(B)330 2070 y Ff(k)367 2063 y Fm(w)o(e)h(use)g(\(3.22\))f(to)g(obtain)31 2198 y(^)14 2209 y Fg(B)48 2216 y Ff(k)112 2209 y Fm(=)41 b Fg(\033)214 2216 y Ff(k)241 2198 y Fm(^)236 2209 y Fg(I)13 b Fe(\000)315 2162 y Fd(h)362 2200 y Fm(^)355 2211 y Fg(Y)381 2218 y Ff(k)444 2211 y Fg(\033)470 2218 y Ff(k)499 2200 y Fm(^)492 2211 y Fg(S)520 2218 y Ff(k)562 2162 y Fd(i)589 2137 y(")634 2186 y Fe(\000)p Fg(D)708 2162 y Fk(1)p Ff(=)p Fk(2)707 2200 y Ff(k)805 2186 y Fg(D)844 2162 y Fc(\000)p Fk(1)p Ff(=)p Fk(2)843 2200 y Ff(k)926 2186 y Fg(L)957 2169 y Ff(T)957 2199 y(k)687 2242 y Fm(0)156 b Fg(J)895 2226 y Ff(T)891 2255 y(k)1005 2137 y Fd(#)1029 2149 y Fc(\000)p Fk(1)1077 2137 y Fd(")1179 2181 y Fg(D)1218 2157 y Fk(1)p Ff(=)p Fk(2)1217 2195 y Ff(k)1384 2181 y Fm(0)1122 2247 y Fe(\000)p Fg(L)1188 2254 y Ff(k)1209 2247 y Fg(D)1248 2223 y Fc(\000)p Fk(1)p Ff(=)p Fk(2)1247 2261 y Ff(k)1372 2247 y Fg(J)1397 2254 y Ff(k)1439 2137 y Fd(#)1464 2145 y Fc(\000)p Fk(1)1518 2137 y Fd(")1591 2170 y Fm(^)1584 2181 y Fg(Y)1620 2165 y Ff(T)1610 2194 y(k)1563 2241 y Fg(\033)1589 2248 y Ff(k)1618 2230 y Fm(^)1611 2241 y Fg(S)1642 2225 y Ff(T)1639 2254 y(k)1689 2137 y Fd(#)1721 2209 y Fg(;)0 2357 y Fm(where)135 2345 y(^)130 2357 y Fg(I)16 b Fm(=)d Fg(Z)248 2340 y Ff(T)275 2357 y Fg(Z)k Fm(is)d(the)f(iden)o(tit)o(y)h(matrix)f(of)g(size)890 2348 y(^)889 2357 y Fg(t)q Fm(,)g(and)1025 2345 y(^)1018 2357 y Fg(Y)1044 2364 y Ff(k)1079 2357 y Fm(=)g Fg(Z)1161 2340 y Ff(T)1188 2357 y Fg(Y)1214 2364 y Ff(k)1249 2357 y Fm(and)1343 2345 y(^)1336 2357 y Fg(S)1364 2364 y Ff(k)1398 2357 y Fm(=)g Fg(Z)1480 2340 y Ff(T)1507 2357 y Fg(S)1535 2364 y Ff(k)1570 2357 y Fm(are)1644 2348 y(^)1644 2357 y Fg(t)7 b Fe(\002)f Fg(m)0 2413 y Fm(submatrices)17 b(of)f Fg(Y)331 2420 y Ff(k)369 2413 y Fm(and)g Fg(S)486 2420 y Ff(k)507 2413 y Fm(.)24 b(The)16 b(pro)q(cedure)h(of)f(m)o(ultiplying)j(the)d(reduced)i(Hessian)1580 2402 y(^)1564 2413 y Fg(B)1598 2420 y Ff(k)1636 2413 y Fm(b)o(y)e(an)0 2470 y(arbitrary)191 2461 y(^)190 2470 y Fg(t)q Fm(-v)o(ector)d(^)-25 b Fg(v)14 b Fm(is)f(similar)g(to)e(steps)i(5)e(to)h(7)g(of)f(Algorithm)i(3.2) e(and)h(costs)g(\(4)p Fg(m)t Fm(+)t(1\))1560 2461 y(^)1559 2470 y Fg(t)s Fm(+)t Fg(O)q Fm(\()p Fg(m)1711 2453 y Fk(2)1730 2470 y Fm(\))851 2603 y(16)p eop %%Page: 17 18 bop 0 122 a Fm(m)o(ultiplications.)26 b(Similarly)l(,)18 b(the)f(w)o(eigh)o (ted)f(scalar)g(pro)q(duct)j(^)-25 b Fg(v)1130 105 y Ff(T)1174 110 y Fm(^)1157 122 y Fg(B)1191 129 y Ff(k)1215 122 y Fm(^)g Fg(v)18 b Fm(costs)e(\(2)p Fg(m)10 b Fm(+)h(1\))1545 113 y(^)1544 122 y Fg(t)g Fm(+)g Fg(O)q Fm(\()p Fg(m)1711 105 y Fk(2)1730 122 y Fm(\))0 178 y(m)o(ultiplications.)71 235 y(In)21 b(this)f(case)h(w)o(e) f(see)g(signi\014can)o(t)i(reductions)f(in)g(computational)g(cost,)f (resulting)i(in)f(w)o(ork)0 291 y(prop)q(ortional)16 b(to)318 283 y(^)317 291 y Fg(t)g Fm(rather)e(than)h(to)g Fg(n)p Fm(.)0 446 y Fh(4.)j(Alternativ)n(e)f(F)-5 b(orm)n(ulae.)71 527 y Fm(F)l(or)11 b(the)i(sak)o(e)f(of)g(completeness)i(w)o(e)e(no)o(w)g(review)h (t)o(w)o(o)f(other)g(kno)o(wn)g(approac)o(hes)g(for)g(handling)0 583 y(limited)21 b(memory)e(matrices.)33 b(The)20 b(\014rst)f(approac)o(h)g (exploits)i(the)f(symmetry)e(and)i(structure)f(of)0 639 y(\(1.4\),)i(giving)h (rise)g(to)f(an)g(e\016cien)o(t)h(t)o(w)o(o-lo)q(op)f(recursion)h(for)f (computing)h(pro)q(ducts)g(using)g(the)0 696 y(in)o(v)o(erse)15 b(Hessian)g(appro)o(ximation.)k(The)c(second)g(approac)o(h)e(is)i(for)f(the)g (direct)h(BF)o(GS)f(up)q(date)h(and)0 752 y(consists)g(of)g(a)g(straigh)o (tforw)o(ard)e(sequence)k(of)d(m)o(ultiplications.)0 907 y Fn(4.1.)j(The)h(Tw)o(o-Lo)q(op)g(Recursion)71 988 y Fm(The)d(follo)o(wing)i (recursiv)o(e)f(form)o(ula)f(computes)h(the)g(step)f(direction)i Fg(H)1320 995 y Ff(k)1341 988 y Fg(g)1363 995 y Ff(k)1400 988 y Fm(for)e(unconstrained)0 1044 y(minimization.)25 b(It)16 b(is)h(giv)o(en)g(in)g(No)q(cedal)g(\(1980\))e(and)h(is)h(based)g(on)f(the)g (recursion)h(dev)o(elop)q(ed)h(b)o(y)0 1101 y(Matthies)d(and)g(Strang)f (\(1979\))f(for)h(the)h(standard)g(BF)o(GS)f(metho)q(d.)20 b(As)15 b(b)q(efore,)g Fg(H)1476 1108 y Ff(k)1511 1101 y Fm(represen)o(ts)g (a)0 1157 y(limited)i(memory)d(BF)o(GS)h(appro)o(ximation)g(of)g(the)g(in)o (v)o(erse)g(Hessian.)21 b(It)15 b(is)h(obtained)g(b)o(y)f(applying)0 1219 y Fg(m)j Fm(up)q(dates)h(to)e(a)h(basic)h(matrix)f Fg(H)641 1195 y Fk(\(0\))637 1233 y Ff(k)706 1219 y Fm(using)h(the)f Fg(m)g Fm(most)g(recen)o(t)g(correction)g(pairs,)h(whic)o(h)g(w)o(e)0 1276 y(lab)q(el)e(for)d(simplicit)o(y)k(\()p Fg(s)426 1283 y Fk(0)445 1276 y Fg(;)8 b(y)488 1283 y Fk(0)508 1276 y Fm(\))p Fg(;)g(:::;)g Fm(\()o Fg(s)645 1283 y Ff(m)p Fc(\000)p Fk(1)721 1276 y Fg(;)g(y)764 1283 y Ff(m)p Fc(\000)p Fk(1)842 1276 y Fm(\).)56 1381 y(1.)22 b Fg(q)14 b Fm(=)f Fg(g)218 1388 y Ff(k)56 1474 y Fm(2.)22 b(F)l(or)14 b Fg(i)e Fm(=)h Fg(m)d Fe(\000)h Fm(1)p Fg(;)d(:)g(:)g(:)t(;)g Fm(0)114 1489 y Fd(")159 1532 y Fg(\013)188 1539 y Ff(i)250 1532 y Fm(=)48 b Fg(\032)357 1539 y Ff(i)370 1532 y Fg(s)391 1516 y Ff(T)391 1544 y(i)419 1532 y Fg(q)92 b Fm(\(store)14 b Fg(\013)689 1539 y Ff(i)704 1532 y Fm(\))180 1589 y Fg(q)43 b Fm(:=)f Fg(q)12 b Fe(\000)e Fg(\013)439 1596 y Ff(i)453 1589 y Fg(y)475 1596 y Ff(i)56 1702 y Fm(3.)22 b Fg(r)13 b Fm(=)g Fg(H)238 1678 y Fk(\(0\))234 1716 y Ff(k)285 1702 y Fg(q)56 1795 y Fm(4.)22 b(F)l(or)14 b Fg(i)e Fm(=)h(0)p Fg(;)8 b Fm(1)p Fg(;)g(:)g(:)g(:)t(;)g(m)h Fe(\000)i Fm(1)114 1810 y Fd(")159 1853 y Fg(\014)50 b Fm(=)e Fg(\032)342 1860 y Ff(i)355 1853 y Fg(y)379 1837 y Ff(T)377 1865 y(i)407 1853 y Fg(r)165 1910 y(r)42 b Fm(:=)g Fg(r)11 b Fm(+)f Fg(s)416 1917 y Ff(i)430 1910 y Fm(\()p Fg(\013)477 1917 y Ff(i)501 1910 y Fe(\000)h Fg(\014)573 1917 y Ff(i)587 1910 y Fm(\))56 2005 y(5.)22 b Fg(H)152 2012 y Ff(k)173 2005 y Fg(g)195 2012 y Ff(k)228 2005 y Fm(=)13 b Fg(r)71 2115 y Fm(Excluding)g(step)f(3,)g(this)g(algorithm)g(requires)h(4)p Fg(mn)e Fm(m)o(ultiplications;)k(if)d Fg(H)1380 2091 y Fk(\(0\))1376 2129 y Ff(k)1439 2115 y Fm(is)g(diagonal)h(then)0 2171 y Fg(n)22 b Fm(additional)i(m)o(ultiplications)h(are)d(needed.)42 b(When)22 b(used)h(for)f(unconstrained)h(minimization)0 2228 y(the)e(computation)g(and) g(storage)f(cost)g(is)i(th)o(us)f(essen)o(tially)h(the)f(same)g(as)f(using)i (form)o(ula)f(\(2.6\))0 2292 y(implemen)o(ted)j(as)d(describ)q(ed)j(in)f (Section)g(3.1,)g(as)e(long)i(as)e Fg(H)1139 2268 y Fk(\(0\))1135 2306 y Ff(k)1208 2292 y Fm(is)i(a)e(scalar)h(m)o(ultiple)i(of)e(the)0 2349 y(iden)o(tit)o(y)l(.)j(Ho)o(w)o(ev)o(er,)16 b(the)h(t)o(w)o(o)e(lo)q(op) i(recursion)h(has)e(the)h(adv)m(an)o(tage)f(that)g(the)h(m)o(ultiplication)i (b)o(y)0 2411 y(the)c(basic)g(matrix)f Fg(H)380 2387 y Fk(\(0\))376 2425 y Ff(k)441 2411 y Fm(is)h(isolated)h(form)e(the)g(rest)g(of)g(the)h (computations.)20 b(As)14 b(a)g(result)i(the)e(t)o(w)o(o-)0 2478 y(lo)q(op)i(recursion)g(will)i(b)q(e)e(less)g(exp)q(ensiv)o(e)h(than)f (\(2.6\))d(in)k(the)e(case)h(when)g Fg(H)1338 2454 y Fk(\(0\))1334 2492 y Ff(k)1400 2478 y Fm(di\013ers)g(from)f Fg(H)1686 2454 y Fk(\(0\))1682 2492 y Ff(k)q Fc(\000)p Fk(1)851 2603 y Fm(17)p eop %%Page: 18 19 bop 0 127 a Fm(b)o(y)19 b(more)g(than)h(a)f(simple)i(scalar)e(m)o (ultiplication,)k(since)e(the)e(en)o(tire)h(matrix)f Fg(Y)1446 111 y Ff(T)1435 141 y(k)1473 127 y Fg(H)1515 103 y Fk(\()p Ff(k)q Fk(\))1511 140 y(0)1563 127 y Fg(Y)1589 134 y Ff(k)1631 127 y Fm(w)o(ould)0 184 y(then)d(ha)o(v)o(e)e(to)h(b)q(e)h(up)q(dated.)71 240 y(Ho)o(w)o(ev)o(er,)d(the)i(t)o(w)o(o-lo)q(op)f(recursion)h(cannot)g(b)q (e)g(e\016cien)o(tly)h(adapted)f(for)f(sparse)h(pro)s(jections.)0 297 y(Let)k(us)g(consider)g(for)f(example)i(the)f(pro)q(duct)g Fg(H)876 304 y Ff(k)897 297 y Fg(e)918 304 y Ff(i)932 297 y Fm(,)g(whic)o(h)h(can)f(b)q(e)g(obtained)g(b)o(y)g(replacing)h Fg(g)1727 304 y Ff(k)0 353 y Fm(with)d Fg(e)126 360 y Ff(i)156 353 y Fm(in)h(the)e(t)o(w)o(o-lo)q(op)g(recursion.)24 b(Since)18 b(the)f(v)o(ectors)e Fg(s)1069 360 y Ff(i)1100 353 y Fm(and)i Fg(y)1212 360 y Ff(i)1242 353 y Fm(are)f(in)i(general)f(not)f(sparse,)0 410 y(w)o(e)e(see)h(that)f(only)i(the)e(computation)h(of)f Fg(\013)759 417 y Ff(m)p Fc(\000)p Fk(1)852 410 y Fm(in)i(step)f(2)f(results) h(in)h(sa)o(vings.)j(Th)o(us)c(steps)f(2)h(and)0 466 y(4)g(require)h(\(4)p Fg(m)9 b Fe(\000)i Fm(1\))p Fg(n)k Fm(m)o(ultiplications)i({)e(almost)g(the)g (same)g(as)g(in)h(the)f(dense)h(case.)71 523 y(W)l(e)g(should)i(also)f(men)o (tion)g(that)f(while)i(the)f(compact)f(form)g(\(2.6\))f(has)i(an)f(analog)h (\(2.17\))e(for)0 579 y(the)e(direct)i(up)q(date,)e(w)o(e)h(kno)o(w)e(of)h (no)h(pro)q(cedure)g(analogous)f(to)g(the)g(t)o(w)o(o)f(lo)q(op)i(recursion)g (that)f(can)0 641 y(compute)i(the)h(direct)g(up)q(date)f(from)g Fg(B)688 617 y Fk(\(0\))686 655 y Ff(k)736 641 y Fg(;)8 b(S)785 648 y Ff(k)805 641 y Fm(,)15 b(and)g Fg(Y)947 648 y Ff(k)984 641 y Fm(in)h Fg(O)q Fm(\()p Fg(mn)p Fm(\))f(op)q(erations.)71 697 y(Mathematically)l(,)f(the)g(relation)h(of)f(the)g(t)o(w)o(o-lo)q(op)f (recursion)i(to)f(\(2.6\))e(can)i(b)q(e)h(seen)g(if)f(w)o(e)g(note)0 754 y(that)g(\(2.6\))g(can)h(b)q(e)h(expressed)257 864 y Fg(H)295 871 y Ff(k)329 864 y Fm(=)d(\()p Fg(I)g Fe(\000)d Fg(S)501 871 y Ff(k)523 864 y Fg(R)558 845 y Fc(\000)p Ff(T)558 878 y(k)612 864 y Fg(Y)649 845 y Ff(T)676 864 y Fm(\))p Fg(H)736 840 y Fk(\(0\))732 878 y Ff(k)783 864 y Fm(\()p Fg(I)j Fe(\000)e Fg(Y)906 871 y Ff(k)927 864 y Fg(R)962 845 y Fc(\000)p Fk(1)962 878 y Ff(k)1009 864 y Fg(S)1040 845 y Ff(T)1037 875 y(k)1067 864 y Fm(\))f(+)h Fg(S)1169 871 y Ff(k)1190 864 y Fg(R)1225 845 y Fc(\000)p Ff(T)1225 878 y(k)1280 864 y Fg(D)1318 871 y Ff(k)1338 864 y Fg(R)1373 845 y Fc(\000)p Fk(1)1373 878 y Ff(k)1421 864 y Fg(S)1452 845 y Ff(T)1449 875 y(k)1479 864 y Fg(:)0 966 y Fm(The)k(v)o(ector)e(made)h(up)h(of)f(the)g(co)q(e\016cien)o (ts)h Fg(\013)799 973 y Ff(i)828 966 y Fm(can)f(then)h(b)q(e)g(seen)g(to)e(b) q(e)i Fg(R)1322 947 y Fc(\000)p Fk(1)1322 980 y Ff(k)1369 966 y Fg(S)1400 950 y Ff(T)1397 979 y(k)1427 966 y Fg(g)1449 973 y Ff(k)1470 966 y Fm(,)f(and)h(the)f(\014nal)0 1022 y(v)m(alue)g(of)e(the)h (v)o(ector)f Fg(q)i Fm(is)f(\()p Fg(I)c Fe(\000)c Fg(Y)564 1029 y Ff(k)586 1022 y Fg(R)621 1003 y Fc(\000)p Fk(1)621 1037 y Ff(k)668 1022 y Fg(S)699 1006 y Ff(T)696 1036 y(k)726 1022 y Fm(\))p Fg(g)766 1029 y Ff(k)787 1022 y Fm(.)18 b(Note)13 b(that)f(in)h(the)g(t)o(w)o(o-lo)q(op)f(pro)q(cedure)i(ev)o(erything)0 1079 y(is)h(computed)g(afresh)f(at)f(eac)o(h)i(iteration,)f(th)o(us)g(making) h(it)f(easier)h(to)f(c)o(hange)g(parameters)g(suc)o(h)g(as)0 1141 y Fg(H)42 1117 y Fk(\(0\))38 1155 y Ff(k)89 1141 y Fm(,)g(while)j (implemen)o(ting)g(\(2.6\))c(in)o(v)o(olv)o(es)j(sa)o(ving)f(and)g(up)q (dating)h(more)e(computed)i(quan)o(tities,)0 1197 y(th)o(us)f(making)g (information)h(suc)o(h)f(as)g(sparse)g(pro)s(jections)g(of)g Fg(H)j Fm(more)d(immediately)i(accessible.)71 1254 y(A)g(close)h(examination) g(of)f(the)h(t)o(w)o(o-lo)q(op)e(recursion)j(rev)o(eals)e(that)g(it)g(is)h (similar)h(in)f(structure)0 1310 y(to)g(computations)g(of)g(gradien)o(ts)h(b) o(y)f(means)h(of)f(the)h(adjoin)o(t)f(metho)q(d)h(\(or)f(the)g(rev)o(erse)h (mo)q(de)f(of)0 1367 y(automatic)g(di\013eren)o(tiation)g(\(Griew)o(ank)g (\(1989\)\).)26 b(In)19 b(fact)f(Gilb)q(ert)h(and)f(No)q(cedal)h(\(1991\))e (sho)o(w)0 1423 y(that)c(there)i(is)f(a)g(precise)i(relationship)f(b)q(et)o (w)o(een)g(these)f(t)o(w)o(o)f(algorithms:)19 b(the)c(t)o(w)o(o-lo)q(op)e (recursion)0 1480 y(can)h(b)q(e)g(obtained)h(b)o(y)f(applying)h(the)f(adjoin) o(t)f(metho)q(d)h(to)f(compute)h(the)g(gradien)o(t)g(of)f(the)h(function)0 1536 y Fg(h)p Fm(\()p Fg(g)r Fm(\))f(=)153 1518 y Fk(1)p 153 1525 18 2 v 153 1552 a(2)176 1536 y Fg(g)200 1519 y Ff(T)227 1536 y Fg(H)265 1543 y Ff(k)286 1536 y Fg(g)k Fm(with)f(resp)q(ect)h(to)e (its)h(argumen)o(t)f Fg(g)r Fm(,)g(where)i Fg(H)1133 1543 y Ff(k)1170 1536 y Fm(is)f(the)g(limited)i(memory)e(BF)o(GS)0 1592 y(matrix.)i(The)12 b(scalars)g Fg(\013)427 1599 y Ff(i)441 1592 y Fm(,)g(whic)o(h)g(are)g(sa)o(v)o(ed)f(during)h(the)g(\014rst)f(lo)q (op,)i(corresp)q(ond)f(to)f(the)g(quan)o(tities)0 1649 y(referred)k(to)g(as)g (the)g(adjoin)o(t)g(v)m(ariables)i(in)f(the)f(optimal)g(con)o(trol)g (literature.)0 1805 y Fn(4.2.)i(A)g(Straigh)o(tforw)o(ard)g(Approac)o(h.)71 1886 y Fm(The)e(direct)h(BF)o(GS)f(form)o(ula)f(\(2.16\))g(can)h(b)q(e)h (written)f(as)611 1988 y Fg(B)645 1995 y Ff(k)q Fk(+1)725 1988 y Fm(=)e Fg(B)807 1995 y Ff(k)839 1988 y Fe(\000)d Fg(a)908 1995 y Ff(k)929 1988 y Fg(a)953 1970 y Ff(T)953 2000 y(k)991 1988 y Fm(+)g Fg(b)1056 1995 y Ff(k)1077 1988 y Fg(b)1097 1970 y Ff(T)1097 2000 y(k)1124 1988 y Fg(;)518 b Fm(\(4)p Fg(:)p Fm(1\))0 2091 y(where)483 2157 y Fg(a)507 2164 y Ff(k)541 2157 y Fm(=)652 2126 y Fg(B)686 2133 y Ff(k)708 2126 y Fg(s)729 2133 y Ff(k)p 594 2146 217 2 v 594 2199 a Fm(\()p Fg(s)633 2183 y Ff(T)633 2213 y(k)660 2199 y Fg(B)694 2206 y Ff(k)716 2199 y Fg(s)737 2183 y Ff(T)737 2213 y(k)765 2199 y Fm(\))788 2167 y Fj(1)p 787 2173 16 2 v 787 2194 a(2)815 2157 y Fg(;)157 b(b)1005 2164 y Ff(k)1038 2157 y Fm(=)1148 2126 y Fg(y)1170 2133 y Ff(k)p 1091 2146 157 2 v 1091 2199 a Fm(\()p Fg(y)1133 2183 y Ff(T)1131 2213 y(k)1160 2199 y Fg(s)1181 2206 y Ff(k)1203 2199 y Fm(\))1226 2167 y Fj(1)p 1226 2173 16 2 v 1226 2194 a(2)1253 2157 y Fg(:)0 2275 y Fm(A)12 b(straigh)o(tforw)o(ard)f(implemen)o (tation)i(of)f(the)h(limited)h(memory)e(metho)q(d)h(consists)f(of)g(sa)o (ving)h(these)0 2332 y(in)o(termediate)i(v)o(ectors)e Fg(a)440 2339 y Ff(i)469 2332 y Fm(and)h Fg(b)576 2339 y Ff(i)604 2332 y Fm(to)f(de\014ne)j(the)e(iteration)g(matrix.)19 b(It)c(has)f(b)q(een)h (used)g(b)o(y)f(sev)o(eral)0 2388 y(authors)h(including)i(Mahidhara)f(and)f (Lasdon)h(\(1990\).)851 2603 y(18)p eop %%Page: 19 20 bop 71 127 a Fm(In)13 b(a)g(t)o(ypical)h(iteration)g Fg(k)q Fm(,)f(the)h(matrix)f Fg(B)799 134 y Ff(k)834 127 y Fm(is)g(obtained)h(b)o(y) f(up)q(dating)i(a)e(starting)f(matrix)h Fg(B)1700 103 y Fk(\(0\))1698 142 y Ff(k)0 184 y Fg(m)i Fm(times)h(using)f(the)h Fg(m)f Fm(most)f(recen)o (t)h(pairs,)h(whic)o(h)g(w)o(e)e(denote)i(for)f(simplicit)o(y)l(,)620 279 y(\()p Fg(s)659 286 y Fk(0)679 279 y Fg(;)8 b(y)722 286 y Fk(0)741 279 y Fm(\))p Fg(;)g(:)g(:)g(:)t(;)g Fm(\()p Fg(s)899 286 y Ff(m)p Fc(\000)p Fk(1)977 279 y Fg(;)g(y)1020 286 y Ff(m)p Fc(\000)p Fk(1)1098 279 y Fm(\))p Fg(:)0 375 y Fm(F)l(rom)14 b(\(4.1\))g(w)o(e)h(see)g(that)g Fg(B)501 382 y Ff(k)538 375 y Fm(can)g(b)q(e)h(written)f(as)572 508 y Fg(B)606 515 y Ff(k)641 508 y Fm(=)e Fg(B)725 484 y Fk(\(0\))723 522 y Ff(k)783 508 y Fm(+)828 455 y Ff(m)p Fc(\000)p Fk(1)836 468 y Fd(X)838 559 y Ff(i)p Fk(=0)904 508 y Fm([)p Fg(b)937 515 y Ff(i)951 508 y Fg(b)971 489 y Ff(T)971 519 y(i)1008 508 y Fe(\000)d Fg(a)1077 515 y Ff(i)1091 508 y Fg(a)1115 489 y Ff(T)1115 519 y(i)1143 508 y Fm(])p Fg(;)486 b Fm(\(4)p Fg(:)p Fm(2\))0 638 y(where)15 b(the)h(v)o(ectors)e Fg(a)388 645 y Ff(i)402 638 y Fg(;)8 b(b)443 645 y Ff(i)471 638 y Fm(can)16 b(b)q(e)f(computed)h(b)o(y)f(means)g(of)g(the) g(follo)o(wing)h(form)o(ula:)0 740 y(F)l(or)f Fg(k)e Fm(=)g(0)p Fg(;)8 b Fm(1)p Fg(;)g(:)g(:)g(:)t(;)g(m)h Fe(\000)h Fm(1)56 828 y(1.)769 884 y Fg(b)789 891 y Ff(k)822 884 y Fm(=)j Fg(y)892 891 y Ff(k)914 884 y Fg(=)p Fm(\()p Fg(y)979 866 y Ff(T)977 896 y(k)1005 884 y Fg(s)1026 891 y Ff(k)1048 884 y Fm(\))1071 852 y Fj(1)p 1071 858 16 2 v 1071 879 a(2)1655 884 y Fm(\(4)p Fg(:)p Fm(3\))56 982 y(2.)540 1066 y Fg(a)564 1073 y Ff(k)598 1066 y Fm(=)g Fg(B)682 1042 y Fk(\(0\))680 1080 y Ff(k)730 1066 y Fg(s)751 1073 y Ff(k)783 1066 y Fm(+)828 1014 y Ff(k)q Fc(\000)p Fk(1)831 1026 y Fd(X)832 1117 y Ff(i)p Fk(=0)900 1019 y Fd(h)920 1066 y Fm(\()p Fg(b)958 1048 y Ff(T)958 1078 y(i)985 1066 y Fg(s)1006 1073 y Ff(k)1027 1066 y Fm(\))p Fg(b)1065 1073 y Ff(i)1089 1066 y Fe(\000)d Fm(\()p Fg(a)1176 1048 y Ff(T)1176 1078 y(i)1203 1066 y Fg(s)1224 1073 y Ff(k)1246 1066 y Fm(\))p Fg(a)1288 1073 y Ff(i)1302 1019 y Fd(i)1655 1066 y Fm(\(4)p Fg(:)p Fm(4\))56 1196 y(3.)753 1252 y Fg(a)777 1259 y Ff(k)811 1252 y Fm(:=)i Fg(a)895 1259 y Ff(k)917 1252 y Fg(=)p Fm(\()p Fg(s)979 1234 y Ff(T)979 1264 y(k)1006 1252 y Fg(a)1030 1259 y Ff(k)1051 1252 y Fm(\))1074 1220 y Fj(1)p 1074 1226 V 1074 1247 a(2)1096 1252 y Fg(:)546 b Fm(\(4)p Fg(:)p Fm(5\))71 1373 y(A)o(t)14 b(the)i(next)f(iteration)h(w)o(e)f(rep)q(eat)h(this)g(pro)q (cess,)f(except)h(that)f(the)g(pair)h(\()p Fg(s)1423 1380 y Fk(0)1443 1373 y Fg(;)8 b(y)1486 1380 y Fk(0)1505 1373 y Fm(\))15 b(is)h(replaced)0 1429 y(b)o(y)e(the)h(new)f(pair)h(\()p Fg(s)364 1436 y Ff(m)397 1429 y Fg(;)8 b(y)440 1436 y Ff(m)473 1429 y Fm(\).)19 b(The)c(v)o(ectors)e Fg(a)793 1436 y Ff(i)822 1429 y Fm(need)i(to)e(b)q(e)i(recomputed)g(form)f(scratc)o(h)f(since)j(they)0 1486 y(all)h(dep)q(end)h(on)e(the)g(deleted)h(pair)g(\()p Fg(s)657 1493 y Fk(0)676 1486 y Fg(;)8 b(y)719 1493 y Fk(0)739 1486 y Fm(\).)22 b(On)16 b(the)g(other)g(hand,)g(the)h(v)o(ectors)e Fg(b)1449 1493 y Ff(i)1479 1486 y Fm(and)h(the)g(inner)0 1542 y(pro)q(ducts)f Fg(b)208 1526 y Ff(T)208 1554 y(i)235 1542 y Fg(s)256 1549 y Ff(k)292 1542 y Fm(can)f(b)q(e)h(sa)o(v)o(ed)f(from)f(the)i (previous)g(iteration,)f(and)h(only)f(the)h(new)f(v)m(alues)i Fg(b)1628 1549 y Ff(m)1675 1542 y Fm(and)0 1607 y Fg(b)20 1591 y Ff(T)20 1619 y(i)47 1607 y Fg(s)68 1614 y Ff(m)119 1607 y Fm(need)j(to)e(b)q(e)i(computed.)28 b(T)l(aking)18 b(this)g(in)o(to)g(accoun) o(t,)g(and)g(assuming)g(that)f Fg(B)1538 1583 y Fk(\(0\))1536 1621 y Ff(k)1603 1607 y Fm(=)g Fg(I)k Fm(w)o(e)0 1664 y(\014nd)16 b(that)e(appro)o(ximately)630 1720 y(3)p Fg(=)p Fm(2)c Fg(m)749 1702 y Fk(2)768 1720 y Fg(n)16 b Fm(m)o(ultiplications)r Fg(;)0 1800 y Fm(are)f(needed)h(to)f(determine)h(the)f(limited)j(memory)c(matrix.)71 1857 y(T)l(o)h(compute)h Fg(B)357 1864 y Ff(m)391 1857 y Fg(v)r Fm(,)f(for)h(some)f(v)o(ector)h Fg(v)f Fe(2)f Fg(R)882 1840 y Ff(n)906 1857 y Fm(,)h(using)i(\(4.2\))d(requires)j(4)p Fg(mn)f Fm(m)o(ultiplications.)0 1913 y(This)e(approac)o(h)f(is)h(therefore)f(less)i (e\016cien)o(t)f(than)f(that)g(based)h(on)f(the)h(compact)f(matrix)g (represen-)0 1970 y(tation)h(describ)q(ed)j(in)e(section)g(3.2.)k(Indeed,)d (whereas)f(the)f(pro)q(duct)h Fg(B)1241 1977 y Ff(k)1263 1970 y Fg(v)h Fm(costs)e(the)h(same)f(in)i(b)q(oth)0 2026 y(cases,)d(up)q(dating)h (the)f(represen)o(tation)g(of)g(the)g(limited)h(memory)f(matrix)f(using)i (the)f(compact)g(form)0 2083 y(requires)18 b(only)f(2)p Fg(mn)g Fm(m)o(ultiplications,)j(compared)d(to)f(3)p Fg(=)p Fm(2)10 b Fg(m)1091 2066 y Fk(2)1111 2083 y Fg(n)17 b Fm(m)o(ultiplications)j(needed) e(b)o(y)f(the)0 2139 y(approac)o(h)e(describ)q(ed)i(in)f(this)g(section.)0 2287 y Fh(5.)i(Compact)g(Represen)n(tation)g(of)g(SR1)h(Matrices.)71 2366 y Fm(In)13 b(this)h(section)f(w)o(e)g(dev)o(elop)h(compact)f(represen)o (tations)g(of)g(matrices)g(generated)g(b)o(y)g(the)g(sym-)0 2422 y(metric)h(rank-one)f(\(SR1\))h(form)o(ula.)19 b(These)14 b(represen)o(tations)f(are)g(similar)i(to)e(the)g(ones)h(deriv)o(ed)h(for)0 2479 y(the)g(BF)o(GS)g(form)o(ula,)f(but)i(under)g(some)f(conditions)h (require)g(less)g(storage.)851 2603 y(19)p eop %%Page: 20 21 bop 71 120 a Fm(The)15 b(SR1)h(up)q(date)f(form)o(ula)g(is)h(giv)o(en)g(b)o (y)476 239 y Fg(B)510 246 y Ff(k)q Fk(+1)589 239 y Fm(=)d Fg(B)671 246 y Ff(k)703 239 y Fm(+)754 208 y(\()p Fg(y)794 215 y Ff(k)825 208 y Fe(\000)d Fg(B)904 215 y Ff(k)926 208 y Fg(s)947 215 y Ff(k)969 208 y Fm(\)\()p Fg(y)1027 215 y Ff(k)1058 208 y Fe(\000)g Fg(B)1137 215 y Ff(k)1159 208 y Fg(s)1180 215 y Ff(k)1202 208 y Fm(\))1220 191 y Ff(T)p 754 228 494 2 v 849 270 a Fm(\()p Fg(y)889 277 y Ff(k)920 270 y Fe(\000)h Fg(B)1000 277 y Ff(k)1021 270 y Fg(s)1042 277 y Ff(k)1064 270 y Fm(\))1082 257 y Ff(T)1109 270 y Fg(s)1130 277 y Ff(k)1252 239 y Fm(;)390 b(\(5)p Fg(:)p Fm(1\))0 355 y(see)19 b(for)g(example)h(Fletc)o(her)f(\(1987\).)30 b(Note)19 b(that)f(this)h(up)q(date)h(is)g(w)o(ell)g(de\014ned)g(only)g(if)f (the)g(de-)0 411 y(nominator)13 b(\()p Fg(B)270 418 y Ff(k)292 411 y Fg(s)313 418 y Ff(k)341 411 y Fe(\000)7 b Fg(y)405 418 y Ff(k)426 411 y Fm(\))444 395 y Ff(T)471 411 y Fg(s)492 418 y Ff(k)527 411 y Fm(is)14 b(nonzero.)20 b(In)14 b(recen)o(t)f(implemen)o (tations)i(of)e(the)g(SR1)h(metho)q(d,)g(the)0 468 y(up)q(date)k(is)g(simply) g(skipp)q(ed)h(if)f(this)g(denominator)f(is)h(v)o(ery)f(small)h(relativ)o(e)g (to)f Fe(k)p Fg(s)1458 475 y Ff(k)1479 468 y Fe(kk)p Fg(B)1559 475 y Ff(k)1581 468 y Fg(s)1602 475 y Ff(k)1635 468 y Fe(\000)12 b Fg(y)1704 475 y Ff(k)1725 468 y Fe(k)0 524 y Fm(\(Conn,)20 b(Gould)f(and)h(T)l(oin)o(t)f(\(1988\),)f(Khalfan,)j(Byrd)f(and)f(Sc)o(hnab)q (el)i(\(1993\)\).)30 b(Since)21 b(the)f(SR1)0 581 y(up)q(date)c(do)q(es)f (not)f(ha)o(v)o(e)h(the)g(prop)q(ert)o(y)g(of)f(hereditary)i(p)q(ositiv)o(e)g (de\014niteness,)g(there)f(is)g(no)g(reason)0 637 y(to)i(enforce)g(the)h (curv)m(ature)g(condition)g Fg(s)724 621 y Ff(T)724 650 y(k)752 637 y Fg(y)774 644 y Ff(k)812 637 y Fg(>)f Fm(0)g(as)g(with)h(BF)o(GS)f(up)q (dating,)h(and)g(w)o(e)f(will)i(th)o(us)0 694 y(consider)14 b(a)g(sequence)g(of)f(up)q(dates)h(to)f(an)g(arbitrary)g(matrix)g Fg(B)1104 701 y Fk(0)1138 694 y Fm(sub)s(ject)g(only)h(to)f(the)h(assumption) 0 750 y(that)g(the)i(up)q(date)g(is)f(w)o(ell)h(de\014ned.)0 845 y Fn(Theorem)h(5.1)22 b Fl(L)n(et)e(the)g(symmetric)h(matrix)f Fg(B)882 852 y Fk(0)923 845 y Fl(b)n(e)g(up)n(date)n(d)g Fg(k)i Fl(times)e(by)g(me)n(ans)f(of)i(the)f(SR1)0 901 y(formula)g(\(5.1\))g(using)f (the)g(p)n(airs)h Fe(f)p Fg(s)651 908 y Ff(i)665 901 y Fg(;)8 b(y)708 908 y Ff(i)722 901 y Fe(g)745 882 y Ff(k)q Fc(\000)p Fk(1)745 914 y Ff(i)p Fk(=0)811 901 y Fl(,)20 b(and)g(assume)f(that)i(e)n (ach)e(up)n(date)i(is)e(wel)r(l)g(de\014ne)n(d,)0 958 y(i.e.)i Fg(s)105 941 y Ff(T)105 970 y(j)132 958 y Fm(\()p Fg(B)184 965 y Ff(j)203 958 y Fg(s)224 965 y Ff(j)253 958 y Fe(\000)10 b Fg(y)320 965 y Ff(j)339 958 y Fm(\))i Fe(6)p Fm(=)h(0;)8 b Fg(j)14 b Fm(=)f(0)p Fg(;)8 b(:::;)g(k)g Fe(\000)i Fm(1)p Fl(.)21 b(Then)15 b(the)i(r)n(esulting)e(matrix)i Fg(B)1339 965 y Ff(k)1377 958 y Fl(is)f(given)f(by)205 1057 y Fg(B)239 1064 y Ff(k)273 1057 y Fm(=)e Fg(B)355 1064 y Fk(0)386 1057 y Fm(+)d(\()p Fg(Y)475 1064 y Ff(k)507 1057 y Fe(\000)g Fg(B)586 1064 y Fk(0)606 1057 y Fg(S)634 1064 y Ff(k)655 1057 y Fm(\)\()p Fg(D)729 1064 y Ff(k)760 1057 y Fm(+)g Fg(L)836 1064 y Ff(k)868 1057 y Fm(+)g Fg(L)944 1039 y Ff(T)944 1069 y(k)982 1057 y Fe(\000)g Fg(S)1058 1039 y Ff(T)1055 1069 y(k)1085 1057 y Fg(B)1119 1064 y Fk(0)1139 1057 y Fg(S)1167 1064 y Ff(k)1189 1057 y Fm(\))1207 1039 y Fc(\000)p Fk(1)1253 1057 y Fm(\()p Fg(Y)1297 1064 y Ff(k)1329 1057 y Fe(\000)g Fg(B)1408 1064 y Fk(0)1429 1057 y Fg(S)1457 1064 y Ff(k)1478 1057 y Fm(\))1496 1039 y Ff(T)1523 1057 y Fg(;)119 b Fm(\(5)p Fg(:)p Fm(2\))0 1149 y Fl(wher)n(e)16 b Fg(S)156 1156 y Ff(k)177 1149 y Fg(;)8 b(Y)224 1156 y Ff(k)245 1149 y Fg(;)g(D)304 1156 y Ff(k)324 1149 y Fl(,)15 b(and)h Fg(L)472 1156 y Ff(k)508 1149 y Fl(ar)n(e)g(as)f(de\014ne)n(d)g(in)g (\(2.1\),)g(\(2.7\))g(and)g(\(2.18\),)g(and)h(the)g(matrix)g Fg(M)1679 1156 y Ff(k)1713 1149 y Fe(\021)0 1206 y Fm(\()p Fg(D)56 1213 y Ff(k)87 1206 y Fm(+)10 b Fg(L)163 1213 y Ff(k)195 1206 y Fm(+)g Fg(L)271 1189 y Ff(T)271 1219 y(k)309 1206 y Fe(\000)g Fg(S)385 1189 y Ff(T)382 1219 y(k)412 1206 y Fg(B)446 1213 y Fk(0)466 1206 y Fg(S)494 1213 y Ff(k)515 1206 y Fm(\))16 b Fl(is)g(nonsingular.)0 1322 y Fn(Pro)q(of.)k Fm(W)l(e)15 b(pro)q(ceed)h(b)o(y)f(induction.)22 b(When)15 b Fg(k)f Fm(=)f(1)i(the)g (righ)o(t)g(hand)h(side)g(of)f(\(5.2\))e(is)360 1433 y Fg(B)394 1440 y Fk(0)425 1433 y Fm(+)d(\()p Fg(y)510 1440 y Fk(0)540 1433 y Fe(\000)g Fg(B)619 1440 y Fk(0)639 1433 y Fg(s)660 1440 y Fk(0)680 1433 y Fm(\))840 1402 y(1)p 703 1423 297 2 v 703 1464 a(\()p Fg(y)743 1471 y Fk(0)773 1464 y Fe(\000)g Fg(B)852 1471 y Fk(0)873 1464 y Fg(s)894 1471 y Fk(0)914 1464 y Fm(\))932 1451 y Ff(T)959 1464 y Fg(s)980 1471 y Fk(0)1005 1433 y Fm(\()p Fg(y)1045 1440 y Fk(0)1075 1433 y Fe(\000)g Fg(B)1154 1440 y Fk(0)1174 1433 y Fg(s)1195 1440 y Fk(0)1215 1433 y Fm(\))1233 1414 y Ff(T)1273 1433 y Fm(=)j Fg(B)1355 1440 y Fk(1)1375 1433 y Fg(:)0 1550 y Fm(Let)i(us)h(no)o(w)e(assume)i(that)e(\(5.2\))g(holds)i(for) e(some)h Fg(k)q Fm(.)20 b(De\014ne)545 1642 y Fg(Q)581 1649 y Ff(k)615 1642 y Fm(=)13 b([)o Fg(q)695 1649 y Fk(0)715 1642 y Fg(;)8 b(:)g(:)g(:)d(;)j(q)837 1649 y Ff(k)q Fc(\000)p Fk(1)903 1642 y Fm(])k(=)h Fg(Y)1002 1649 y Ff(k)1034 1642 y Fe(\000)e Fg(B)1114 1649 y Fk(0)1134 1642 y Fg(S)1162 1649 y Ff(k)1183 1642 y Fg(;)459 b Fm(\(5)p Fg(:)p Fm(3\))0 1734 y(and)556 1790 y Fg(M)600 1797 y Ff(k)634 1790 y Fm(=)13 b Fg(D)720 1797 y Ff(k)751 1790 y Fm(+)d Fg(L)827 1797 y Ff(k)859 1790 y Fm(+)g Fg(L)935 1771 y Ff(T)935 1801 y(k)973 1790 y Fe(\000)g Fg(S)1049 1771 y Ff(T)1046 1801 y(k)1076 1790 y Fg(B)1110 1797 y Fk(0)1130 1790 y Fg(S)1158 1797 y Ff(k)1180 1790 y Fg(:)462 b Fm(\(5)p Fg(:)p Fm(4\))0 1868 y(Therefore)646 1925 y Fg(B)680 1932 y Ff(k)715 1925 y Fm(=)13 b Fg(B)797 1932 y Fk(0)827 1925 y Fm(+)d Fg(Q)908 1932 y Ff(k)930 1925 y Fg(M)979 1906 y Fc(\000)p Fk(1)974 1939 y Ff(k)1026 1925 y Fg(Q)1062 1906 y Ff(T)1062 1936 y(k)1089 1925 y Fg(:)0 2003 y Fm(Applying)17 b(the)e(SR1)h(up)q(date)g(\(5.1\))d(to)i Fg(B)723 2010 y Ff(k)760 2003 y Fm(w)o(e)g(ha)o(v)o(e)8 2126 y Fg(B)42 2133 y Ff(k)q Fk(+1)150 2126 y Fm(=)42 b Fg(B)261 2133 y Fk(0)292 2126 y Fm(+)10 b Fg(Q)373 2133 y Ff(k)394 2126 y Fg(M)443 2107 y Fc(\000)p Fk(1)438 2140 y Ff(k)491 2126 y Fg(Q)527 2107 y Ff(T)527 2137 y(k)564 2126 y Fm(+)615 2095 y(\()p Fg(y)655 2102 y Ff(k)686 2095 y Fe(\000)g Fg(B)765 2102 y Fk(0)786 2095 y Fg(s)807 2102 y Ff(k)838 2095 y Fe(\000)h Fg(Q)920 2102 y Ff(k)941 2095 y Fg(M)990 2076 y Fc(\000)p Fk(1)985 2109 y Ff(k)1037 2095 y Fg(Q)1073 2078 y Ff(T)1073 2108 y(k)1101 2095 y Fg(s)1122 2102 y Ff(k)1143 2095 y Fm(\)\()p Fg(y)1201 2102 y Ff(k)1232 2095 y Fe(\000)g Fg(B)1312 2102 y Fk(0)1332 2095 y Fg(s)1353 2102 y Ff(k)1385 2095 y Fe(\000)f Fg(Q)1466 2102 y Ff(k)1488 2095 y Fg(M)1537 2076 y Fc(\000)p Fk(1)1532 2109 y Ff(k)1584 2095 y Fg(Q)1620 2078 y Ff(T)1620 2108 y(k)1647 2095 y Fg(s)1668 2102 y Ff(k)1690 2095 y Fm(\))1708 2078 y Ff(T)p 615 2116 1121 2 v 842 2161 a Fm(\()p Fg(y)882 2168 y Ff(k)913 2161 y Fe(\000)h Fg(B)993 2168 y Fk(0)1013 2161 y Fg(s)1034 2168 y Ff(k)1056 2161 y Fm(\))1074 2148 y Ff(T)1101 2161 y Fg(s)1122 2168 y Ff(k)1154 2161 y Fe(\000)f Fg(s)1220 2145 y Ff(T)1220 2175 y(k)1248 2161 y Fg(Q)1284 2168 y Ff(k)1305 2161 y Fg(M)1354 2142 y Fc(\000)p Fk(1)1349 2175 y Ff(k)1401 2161 y Fg(Q)1437 2145 y Ff(T)1437 2175 y(k)1465 2161 y Fg(s)1486 2168 y Ff(k)150 2264 y Fm(=)42 b Fg(B)261 2271 y Fk(0)292 2264 y Fm(+)10 b Fg(Q)373 2271 y Ff(k)394 2264 y Fg(M)443 2244 y Fc(\000)p Fk(1)438 2278 y Ff(k)491 2264 y Fg(Q)527 2245 y Ff(T)527 2275 y(k)564 2264 y Fm(+)615 2232 y(\()p Fg(q)653 2239 y Ff(k)684 2232 y Fe(\000)g Fg(Q)765 2239 y Ff(k)787 2232 y Fg(M)836 2213 y Fc(\000)p Fk(1)831 2246 y Ff(k)883 2232 y Fg(w)916 2239 y Ff(k)937 2232 y Fm(\)\()p Fg(q)993 2239 y Ff(k)1024 2232 y Fe(\000)g Fg(Q)1105 2239 y Ff(k)1127 2232 y Fg(M)1176 2213 y Fc(\000)p Fk(1)1171 2246 y Ff(k)1223 2232 y Fg(w)1256 2239 y Ff(k)1277 2232 y Fm(\))1295 2216 y Ff(T)p 615 2253 708 2 v 789 2298 a Fg(q)811 2283 y Ff(T)809 2313 y(k)838 2298 y Fg(s)859 2305 y Ff(k)891 2298 y Fe(\000)g Fg(w)970 2283 y Ff(T)969 2313 y(k)998 2298 y Fg(M)1047 2279 y Fc(\000)p Fk(1)1042 2313 y Ff(k)1094 2298 y Fg(w)1127 2305 y Ff(k)150 2378 y Fm(=)42 b Fg(B)261 2385 y Fk(0)292 2378 y Fm(+)337 2331 y Fd(h)357 2378 y Fg(q)377 2385 y Ff(k)398 2378 y Fg(q)420 2359 y Ff(T)418 2389 y(k)458 2378 y Fe(\000)10 b Fg(q)523 2385 y Ff(k)545 2378 y Fm(\()p Fg(w)597 2359 y Ff(T)596 2389 y(k)624 2378 y Fg(M)673 2359 y Fc(\000)p Fk(1)668 2392 y Ff(k)720 2378 y Fm(\))p Fg(Q)774 2359 y Ff(T)774 2389 y(k)811 2378 y Fe(\000)h Fg(Q)893 2385 y Ff(k)914 2378 y Fm(\()p Fg(M)981 2359 y Fc(\000)p Fk(1)976 2392 y Ff(k)1028 2378 y Fg(w)1061 2385 y Ff(k)1082 2378 y Fm(\))p Fg(q)1122 2359 y Ff(T)1120 2389 y(k)232 2469 y Fm(+)p Fg(Q)303 2476 y Ff(k)325 2469 y Fm(\()p Fg(\016)363 2476 y Ff(k)384 2469 y Fg(M)433 2450 y Fc(\000)p Fk(1)428 2483 y Ff(k)490 2469 y Fm(+)g Fg(M)585 2450 y Fc(\000)p Fk(1)580 2483 y Ff(k)632 2469 y Fg(w)665 2476 y Ff(k)686 2469 y Fg(w)720 2450 y Ff(T)719 2480 y(k)747 2469 y Fg(M)796 2450 y Fc(\000)p Fk(1)791 2483 y Ff(k)843 2469 y Fm(\))p Fg(Q)897 2450 y Ff(T)897 2480 y(k)925 2422 y Fd(i)952 2469 y Fg(=\016)995 2476 y Ff(k)1016 2469 y Fg(;)851 2603 y Fm(20)p eop %%Page: 21 22 bop 0 120 a Fm(where)15 b(w)o(e)g(ha)o(v)o(e)g(de\014ned)754 177 y Fg(w)787 184 y Ff(k)820 177 y Fm(=)e Fg(Q)904 158 y Ff(T)904 188 y(k)932 177 y Fg(s)953 184 y Ff(k)974 177 y Fg(;)668 b Fm(\(5)p Fg(:)p Fm(5\))0 260 y(and)15 b(where)h(the)f(denominator)615 361 y Fg(\016)635 368 y Ff(k)698 361 y Fe(\021)41 b Fg(q)796 342 y Ff(T)794 372 y(k)824 361 y Fg(s)845 368 y Ff(k)877 361 y Fe(\000)10 b Fg(w)956 342 y Ff(T)955 372 y(k)983 361 y Fg(M)1032 342 y Fc(\000)p Fk(1)1027 375 y Ff(k)1080 361 y Fg(w)1113 368 y Ff(k)1655 361 y Fm(\(5.6\))698 430 y(=)41 b(\()p Fg(y)814 437 y Ff(k)846 430 y Fe(\000)10 b Fg(B)925 437 y Ff(k)947 430 y Fg(s)968 437 y Ff(k)990 430 y Fm(\))1008 411 y Ff(T)1035 430 y Fg(s)1056 437 y Ff(k)0 531 y Fm(is)16 b(non-zero)f(b)o(y)g(assumption.) 20 b(W)l(e)c(ma)o(y)e(express)i(this)f(as)125 687 y Fg(B)159 694 y Ff(k)q Fk(+1)267 687 y Fm(=)42 b Fg(B)378 694 y Fk(0)408 687 y Fm(+)468 657 y(1)p 459 677 42 2 v 459 719 a Fg(\016)479 726 y Ff(k)513 687 y Fm([)o Fg(Q)561 694 y Ff(k)593 687 y Fg(q)613 694 y Ff(k)634 687 y Fm(])655 615 y Fd(")700 659 y Fg(M)749 640 y Fc(\000)p Fk(1)744 673 y Ff(k)796 659 y Fm(\()p Fg(\016)834 666 y Ff(k)855 659 y Fg(I)14 b Fm(+)c Fg(w)967 666 y Ff(k)988 659 y Fg(w)1022 643 y Ff(T)1021 672 y(k)1049 659 y Fg(M)1098 640 y Fc(\000)p Fk(1)1093 673 y Ff(k)1146 659 y Fm(\))41 b Fe(\000)p Fg(M)1289 640 y Fc(\000)p Fk(1)1284 673 y Ff(k)1336 659 y Fg(w)1369 666 y Ff(k)835 717 y Fe(\000)p Fg(w)904 700 y Ff(T)903 730 y(k)932 717 y Fg(M)981 697 y Fc(\000)p Fk(1)976 731 y Ff(k)1286 717 y Fm(1)1411 615 y Fd(#)1443 591 y(2)1443 664 y(6)1443 690 y(4)1491 631 y Fg(Q)1527 614 y Ff(T)1527 644 y(k)1498 744 y Fg(q)1520 727 y Ff(T)1518 757 y(k)1575 591 y Fd(3)1575 664 y(7)1575 690 y(5)1611 687 y Fg(:)31 b Fm(\(5.7\))0 846 y(No)o(w,)13 b(from)g(de\014nitions)j(\(5.3\),)c(\(5.4\))h(and)h(\(5.5\)) e(w)o(e)i(see)g(that)g(the)g(new)g(matrix)g Fg(M)1458 853 y Ff(k)q Fk(+1)1538 846 y Fm(is)h(giv)o(en)f(b)o(y)634 980 y Fg(M)678 987 y Ff(k)q Fk(+1)757 980 y Fm(=)805 908 y Fd(")850 952 y Fg(M)894 959 y Ff(k)976 952 y Fg(w)1009 959 y Ff(k)852 1008 y Fg(w)886 992 y Ff(T)885 1022 y(k)957 1008 y Fg(q)979 992 y Ff(T)977 1022 y(k)1006 1008 y Fg(s)1027 1015 y Ff(k)1070 908 y Fd(#)1102 980 y Fg(;)0 1114 y Fm(and)h(b)o(y)h(direct)f(m)o (ultiplication,)j(using)e(\(5.3\),)d(\(5.5\))g(and)j(\(5.6\),)d(w)o(e)i(see)g (that)261 1176 y Fd(")306 1220 y Fg(M)350 1227 y Ff(k)432 1220 y Fg(w)465 1227 y Ff(k)308 1276 y Fg(w)342 1260 y Ff(T)341 1290 y(k)413 1276 y Fg(q)435 1260 y Ff(T)433 1290 y(k)463 1276 y Fg(s)484 1283 y Ff(k)526 1176 y Fd(#)8 b(")603 1220 y Fg(M)652 1201 y Fc(\000)p Fk(1)647 1234 y Ff(k)699 1220 y Fm(\()p Fg(\016)737 1227 y Ff(k)758 1220 y Fg(I)14 b Fm(+)c Fg(w)870 1227 y Ff(k)891 1220 y Fg(w)925 1203 y Ff(T)924 1233 y(k)953 1220 y Fg(M)1002 1201 y Fc(\000)p Fk(1)997 1234 y Ff(k)1049 1220 y Fm(\))41 b Fe(\000)p Fg(M)1192 1201 y Fc(\000)p Fk(1)1187 1234 y Ff(k)1240 1220 y Fg(w)1273 1227 y Ff(k)738 1277 y Fe(\000)p Fg(w)807 1261 y Ff(T)806 1291 y(k)835 1277 y Fg(M)884 1258 y Fc(\000)p Fk(1)879 1291 y Ff(k)1189 1277 y Fm(1)1314 1176 y Fd(#)1360 1218 y Fm(1)p 1351 1238 V 1351 1279 a Fg(\016)1371 1286 y Ff(k)1410 1248 y Fm(=)13 b Fg(I)t(:)160 b Fm(\(5)p Fg(:)p Fm(8\))0 1389 y(Therefore)18 b Fg(M)253 1396 y Ff(k)q Fk(+1)338 1389 y Fm(is)g(in)o(v)o (ertible,)i(with)f Fg(M)759 1370 y Fc(\000)p Fk(1)754 1403 y Ff(k)q Fk(+1)838 1389 y Fm(giv)o(en)g(b)o(y)f(the)g(second)h(matrix)f(in)g (\(5.8\),)f(but)i(this)0 1445 y(is)e(the)f(matrix)f(app)q(earing)i(in)g (\(5.7\).)k(Th)o(us,)16 b(w)o(e)f(see)i(that)e(\(5.7\))f(is)j(equiv)m(alen)o (t)h(to)d(equation)i(\(5.2\))0 1502 y(with)f Fg(k)g Fm(replaced)g(b)o(y)f Fg(k)d Fm(+)e(1,)k(whic)o(h)j(observ)m(ation)e(establishes)i(the)e(result.) 1699 1558 y Fa(2)71 1639 y Fm(Since)20 b(the)g(SR1)f(metho)q(d)h(is)g(self)g (dual,)g(the)g(in)o(v)o(erse)f(form)o(ula)g(can)h(b)q(e)g(obtained)g(simply)g (b)o(y)0 1696 y(replacing)c Fg(B)r(;)8 b(s;)g(y)16 b Fm(b)o(y)f Fg(H)q(;)8 b(y)r(;)g(s)14 b Fm(resp)q(ectiv)o(ely)i(\(see)f(Dennis)h(and)f (Sc)o(hnab)q(el)h(\(1983\)\).)i(Alternativ)o(ely)l(,)0 1752 y(if)12 b Fg(B)72 1759 y Ff(k)105 1752 y Fm(is)g(in)o(v)o(ertible,)h (application)g(of)e(the)g(Sherman-Morrison-W)l(o)q(o)q(dbury)h(form)o(ula)f (to)g(\(5.2\))f(sho)o(ws)0 1809 y(the)15 b(in)o(v)o(erse)h(of)f Fg(B)315 1816 y Ff(k)352 1809 y Fm(is)g(giv)o(en)h(b)o(y)194 1910 y Fg(H)232 1917 y Ff(k)266 1910 y Fm(=)d Fg(H)352 1917 y Fk(0)382 1910 y Fm(+)d(\()p Fg(S)473 1917 y Ff(k)504 1910 y Fe(\000)h Fg(H)588 1917 y Fk(0)607 1910 y Fg(Y)633 1917 y Ff(k)655 1910 y Fm(\)\()p Fg(R)726 1917 y Ff(k)756 1910 y Fm(+)g Fg(R)837 1891 y Ff(T)837 1921 y(k)874 1910 y Fe(\000)g Fg(D)958 1917 y Ff(k)989 1910 y Fe(\000)f Fg(Y)1071 1891 y Ff(T)1060 1921 y(k)1098 1910 y Fg(H)1136 1917 y Fk(0)1156 1910 y Fg(Y)1182 1917 y Ff(k)1203 1910 y Fm(\))1221 1891 y Fc(\000)p Fk(1)1268 1910 y Fm(\()p Fg(S)1314 1917 y Ff(k)1345 1910 y Fe(\000)h Fg(H)1429 1917 y Fk(0)1448 1910 y Fg(Y)1474 1917 y Ff(k)1496 1910 y Fm(\))1514 1891 y Ff(T)1541 1910 y Fg(:)101 b Fm(\(5)p Fg(:)p Fm(9\))0 2011 y(Ho)o(w)o(ev)o(er,)22 b(in)h(the)g(con)o(text)e(of)h (unconstrained)h(optimization,)h(since)f(the)g(SR1)f(up)q(date)h(is)f(not)0 2068 y(alw)o(a)o(ys)15 b(p)q(ositiv)o(e)i(de\014nite)h(this)e(form)o(ula)g (is)g(not)g(as)g(lik)o(ely)h(to)f(b)q(e)g(useful)i(in)e(step)g(computation)g (as)0 2124 y(is)g(the)f(in)o(v)o(erse)h(BF)o(GS)e(up)q(date.)71 2180 y(It)h(should)h(b)q(e)g(clear)g(ho)o(w)e(to)h(dev)o(elop)h(limited)h (memory)e(SR1)h(metho)q(ds.)k(In)15 b(\(5.2\))f(w)o(e)h(replace)0 2245 y Fg(B)34 2252 y Fk(0)68 2245 y Fm(with)g(the)f(basic)h(matrix)f(at)g (the)g Fg(k)q Fm(-th)h(iteration,)f(whic)o(h)h(w)o(e)f(denoted)h(earlier)g(b) o(y)f Fg(B)1536 2221 y Fk(\(0\))1534 2259 y Ff(k)1584 2245 y Fm(,)g(and)h Fg(S)1727 2252 y Ff(k)0 2301 y Fm(and)f Fg(Y)113 2308 y Ff(k)148 2301 y Fm(should)h(no)o(w)f(con)o(tain)g(the)f Fg(m)h Fm(most)f(recen)o(t)h(corrections,)g(as)f(in)i(\(3.2\).)j(Sa)o(vings)c (in)g(storage)0 2366 y(can)f(b)q(e)h(ac)o(hiev)o(ed)g(if)g Fg(B)398 2342 y Fk(\(0\))396 2380 y Ff(k)459 2366 y Fm(is)g(k)o(ept)f (\014xed)h(for)e(all)i Fg(k)q Fm(,)g(for)e(in)i(this)g(case)f(the)g(only)h Fg(n)p Fm(-v)o(ectors)f(one)g(needs)0 2422 y(to)18 b(store)g(are)g(the)h Fg(m)f Fm(columns)i(of)e Fg(Q)665 2429 y Ff(k)686 2422 y Fm(.)30 b(This)20 b(w)o(ould)e(result)i(also)e(in)i(some)e(sa)o(vings)g(in)i(the)e (cost)0 2479 y(of)e(up)q(dating)i(the)f(matrix)g Fg(M)522 2486 y Ff(k)543 2479 y Fm(,)g(dep)q(ending)j(on)c(the)h(step)g(computation)g (strategy)f(used.)25 b(On)18 b(the)851 2603 y(21)p eop %%Page: 22 23 bop 0 127 a Fm(other)13 b(hand,)h(if)g Fg(B)318 103 y Fk(\(0\))316 142 y Ff(k)380 127 y Fm(is)g(a)f(scalar)h(m)o(ultiple)h(of)e(the)h(iden)o (tit)o(y)g(and,)g(as)f(is)h(often)f(the)h(case,)f(one)h(w)o(an)o(ts)0 184 y(to)j(c)o(hange)g(the)g(scalar)h(at)e(eac)o(h)i(iteration,)f(then)h(b)q (oth)g Fg(S)1027 191 y Ff(k)1065 184 y Fm(and)g Fg(Y)1182 191 y Ff(k)1220 184 y Fm(m)o(ust)f(b)q(e)h(stored)f(separately)l(,)0 240 y(and)f(the)f(storage)g(and)h(up)q(dating)g(costs)f(of)g(the)h(limited)i (memory)d(SR1)h(and)f(BF)o(GS)h(metho)q(ds)f(are)0 297 y(similar.)71 353 y(W)l(e)10 b(will)j(not)d(giv)o(e)h(detailed)h(algorithms)f(for)f (computing)h(pro)q(ducts)g(in)o(v)o(olving)h(limited)h(memory)0 410 y(SR1)f(matrices)f(b)q(ecause)i(the)e(ideas)h(are)f(v)o(ery)g(similar)i (to)e(those)g(describ)q(ed)i(in)f(the)g(previous)g(section.)0 466 y(One)h(p)q(oin)o(t,)g(ho)o(w)o(ev)o(er,)f(that)g(is)h(w)o(orth)f (discussing)i(is)f(ho)o(w)f(to)g(compute)g(the)h(denominator)g(in)g(\(5.1\),) 0 523 y(at)20 b(eac)o(h)h(stage)e(of)i(the)f(limited)j(memory)d(up)q(dating,) i(to)e(determine)i(if)f(the)g(up)q(date)g(should)g(b)q(e)0 579 y(skipp)q(ed.)h(The)15 b(condition)518 669 y Fg(s)539 650 y Ff(T)539 680 y(j)566 669 y Fm(\()p Fg(B)618 676 y Ff(j)637 669 y Fg(s)658 676 y Ff(j)686 669 y Fe(\000)c Fg(y)754 676 y Ff(j)772 669 y Fm(\))i Fe(6)p Fm(=)g(0)p Fg(;)37 b(j)15 b Fm(=)e(0)p Fg(;)8 b(:::;)g(k)g Fe(\000)j Fm(1)p Fg(;)408 b Fm(\(5)p Fg(:)p Fm(10\))0 759 y(can)18 b(b)q(e)h(exp)q(ensiv)o(e)h(to)d (test.)28 b(Note)18 b(ho)o(w)o(ev)o(er)f(that)h(\(5.10\))e(is)i(equiv)m(alen) o(t)i(to)e(the)g(nonsingularit)o(y)0 816 y(of)g(the)h(principal)j(minors)d (of)f Fg(M)583 823 y Ff(k)605 816 y Fm(.)31 b(Th)o(us,)19 b(when)g(using)h (the)f(form)f(\(5.2\))f(in)j(a)f(limited)i(memory)0 872 y(metho)q(d,)15 b(the)h(condition)g(\(5.10\))e(could)i(b)q(e)g(tested)f(when)h(computing)g(a) f(triangular)g(factorization)0 929 y(of)e Fg(M)94 936 y Ff(k)129 929 y Fm(without)g(piv)o(oting,)h(with)g(the)g(test)f(for)g(a)g(zero)g(on)g (the)h(diagonal)g(of)f(the)h(factor)e(b)q(eing)j(made)0 985 y(relativ)o(e)d(to)f(the)h(magnitude)g(of)f Fg(Q)590 992 y Ff(k)623 985 y Fm(and)h Fg(S)736 992 y Ff(k)757 985 y Fm(.)19 b(Skipping)14 b(an)d(up)q(date)h(w)o(ould)g(corresp)q(ond)g(to)f(deleting)0 1042 y(the)k(corresp)q(onding)h(ro)o(w)f(and)g(column)h(of)f Fg(M)801 1049 y Ff(k)822 1042 y Fm(.)0 1182 y Fh(6.)j(Represen)n(tation)g(of) h(Bro)n(yden)f(Matrices)g(for)g(Nonlinear)g(Equations.)71 1259 y Fm(A)c(widely)i(used)g(secan)o(t)e(appro)o(ximation)g(to)g(the)h(Jacobian)g (matrix)g(of)f(a)g(system)g(of)h(nonlinear)0 1315 y(equations,)615 1372 y Fg(F)6 b Fm(\()p Fg(x)p Fm(\))13 b(=)g(0)p Fg(;)47 b(F)19 b Fm(:)12 b Fe(<)962 1353 y Ff(n)998 1372 y Fe(!)h(<)1089 1353 y Ff(n)1113 1372 y Fg(;)529 b Fm(\(6)p Fg(:)p Fm(1\))0 1449 y(is)16 b(the)f(Bro)o(yden)g(up)q(date)h(\(Bro)o(yden)f(\(1965\)\),)586 1568 y Fg(A)620 1575 y Ff(k)q Fk(+1)699 1568 y Fm(=)e Fg(A)781 1575 y Ff(k)813 1568 y Fm(+)863 1538 y(\()p Fg(y)903 1545 y Ff(k)934 1538 y Fe(\000)e Fg(A)1014 1545 y Ff(k)1035 1538 y Fg(s)1056 1545 y Ff(k)1078 1538 y Fm(\))p Fg(s)1117 1521 y Ff(T)1117 1551 y(k)p 863 1558 282 2 v 958 1601 a Fg(s)979 1585 y Ff(T)979 1615 y(k)1007 1601 y Fg(s)1028 1608 y Ff(k)1149 1568 y Fg(:)493 b Fm(\(6)p Fg(:)p Fm(2\))0 1687 y(Here)12 b Fg(s)125 1694 y Ff(k)160 1687 y Fm(=)h Fg(x)234 1694 y Ff(k)q Fk(+1)304 1687 y Fe(\000)t Fg(x)369 1694 y Ff(k)391 1687 y Fm(,)j Fg(y)442 1694 y Ff(k)476 1687 y Fm(=)d Fg(F)6 b Fm(\()p Fg(x)603 1694 y Ff(k)q Fk(+1)670 1687 y Fm(\))t Fe(\000)t Fg(F)g Fm(\()p Fg(x)810 1694 y Ff(k)832 1687 y Fm(\),)12 b(and)g Fg(A)994 1694 y Ff(k)1027 1687 y Fm(is)h(the)f(appro)o(ximation)g(to)g(the)g(Jacobian) 0 1743 y(of)f Fg(F)6 b Fm(.)19 b(In)11 b(this)h(section)g(w)o(e)e(describ)q (e)j(compact)e(expressions)h(of)e(Bro)o(yden)i(matrices)f(that)f(are)h (similar)0 1800 y(to)k(those)f(giv)o(en)i(for)f(BF)o(GS)f(and)i(SR1.)k(As)15 b(b)q(efore,)g(w)o(e)g(de\014ne)406 1890 y Fg(S)434 1897 y Ff(k)468 1890 y Fm(=)e([)p Fg(s)550 1897 y Fk(0)570 1890 y Fg(;)8 b(:)g(:)g(:)t(;)g(s)692 1897 y Ff(k)q Fc(\000)p Fk(1)758 1890 y Fm(])g Fg(;)157 b(Y)975 1897 y Ff(k)1009 1890 y Fm(=)13 b([)p Fg(y)1092 1897 y Fk(0)1112 1890 y Fg(;)8 b(:)g(:)g(:)d(;)j(y)1236 1897 y Ff(k)q Fc(\000)p Fk(1)1301 1890 y Fm(])g Fg(;)320 b Fm(\(6)p Fg(:)p Fm(3\))0 1980 y(and)15 b(w)o(e)g(assume)g(that)g(the)g(v)o (ectors)g Fg(s)665 1987 y Ff(i)694 1980 y Fm(are)g(non-zero.)0 2072 y Fn(Theorem)i(6.1)22 b Fl(L)n(et)d Fg(A)427 2079 y Fk(0)465 2072 y Fl(b)n(e)g(a)g(nonsingular)e(starting)i(matrix,)h(and)f(let)f Fg(A)1341 2079 y Ff(k)1381 2072 y Fl(b)n(e)h(obtaine)n(d)f(by)h(up-)0 2129 y(dating)d Fg(A)172 2136 y Fk(0)218 2129 y Fg(k)i Fl(times)e(using)f(Br) n(oyden)-5 b('s)16 b(formula)h(\(6.2\))f(and)g(the)h(p)n(airs)f Fe(f)p Fg(s)1312 2136 y Ff(i)1326 2129 y Fg(;)8 b(y)1369 2136 y Ff(i)1383 2129 y Fe(g)1406 2110 y Ff(k)q Fc(\000)p Fk(1)1406 2142 y Ff(i)p Fk(=0)1472 2129 y Fl(.)21 b(Then)557 2219 y Fg(A)591 2226 y Ff(k)625 2219 y Fm(=)13 b Fg(A)707 2226 y Fk(0)737 2219 y Fm(+)d(\()p Fg(Y)826 2226 y Ff(k)858 2219 y Fe(\000)g Fg(A)937 2226 y Fk(0)957 2219 y Fg(S)985 2226 y Ff(k)1006 2219 y Fm(\))p Fg(N)1066 2200 y Fc(\000)p Fk(1)1061 2233 y Ff(k)1113 2219 y Fg(S)1144 2200 y Ff(T)1141 2230 y(k)1171 2219 y Fg(;)471 b Fm(\(6)p Fg(:)p Fm(4\))0 2309 y Fl(wher)n(e)16 b Fg(N)165 2316 y Ff(k)202 2309 y Fl(is)g(the)h Fg(k)11 b Fe(\002)f Fg(k)18 b Fl(matrix)531 2428 y Fm(\()p Fg(N)586 2435 y Ff(k)607 2428 y Fm(\))625 2435 y Ff(i;j)677 2428 y Fm(=)725 2356 y Fd(\()779 2399 y Fg(s)800 2383 y Ff(T)800 2411 y(i)p Fc(\000)p Fk(1)860 2399 y Fg(s)881 2406 y Ff(j)r Fc(\000)p Fk(1)986 2399 y Fl(if)e Fg(i)c Fe(\024)h Fg(j)779 2456 y Fm(0)184 b Fl(otherwise)1204 2428 y Fg(:)438 b Fm(\(6)p Fg(:)p Fm(5\))851 2603 y(22)p eop %%Page: 23 24 bop 0 120 a Fn(Pro)q(of.)20 b Fm(It)15 b(is)h(easy)f(to)f(sho)o(w)h(\(using)g (induction\))i(that)e Fg(A)1014 127 y Ff(k)1050 120 y Fm(can)h(b)q(e)f (written)h(as)723 222 y Fg(A)757 229 y Ff(k)791 222 y Fm(=)d Fg(B)873 229 y Ff(k)905 222 y Fm(+)e Fg(C)984 229 y Ff(k)1005 222 y Fg(;)637 b Fm(\(6)p Fg(:)p Fm(6\))0 324 y(where)15 b Fg(B)165 331 y Ff(k)202 324 y Fm(and)h Fg(C)324 331 y Ff(k)360 324 y Fm(are)f(de\014ned)h(recursiv)o(ely)h(b)o(y)507 385 y Fd(\()561 428 y Fg(B)595 435 y Fk(0)628 428 y Fm(=)c Fg(A)710 435 y Fk(0)561 485 y Fg(B)595 492 y Ff(k)q Fk(+1)674 485 y Fm(=)g Fg(B)756 492 y Ff(k)778 485 y Fm(\()p Fg(I)g Fe(\000)e Fg(\032)899 492 y Ff(k)920 485 y Fg(s)941 492 y Ff(k)962 485 y Fg(s)983 468 y Ff(T)983 498 y(k)1011 485 y Fm(\))40 b Fe(8)p Fg(k)14 b Fe(\025)f Fm(0)p Fg(;)1655 457 y Fm(\(6)p Fg(:)p Fm(7\))0 589 y(and)412 595 y Fd(\()466 638 y Fg(C)499 645 y Fk(0)531 638 y Fm(=)g(0)466 695 y Fg(C)499 702 y Ff(k)q Fk(+1)578 695 y Fm(=)g Fg(C)659 702 y Ff(k)680 695 y Fm(\()p Fg(I)g Fe(\000)e Fg(\032)801 702 y Ff(k)821 695 y Fg(s)842 702 y Ff(k)864 695 y Fg(s)885 678 y Ff(T)885 708 y(k)913 695 y Fm(\))f(+)g Fg(\032)1010 702 y Ff(k)1031 695 y Fg(y)1053 702 y Ff(k)1075 695 y Fg(s)1096 678 y Ff(T)1096 708 y(k)1164 695 y Fe(8)p Fg(k)k Fe(\025)f Fm(0)p Fg(;)1655 667 y Fm(\(6)p Fg(:)p Fm(8\))0 780 y(and)i(where)747 837 y Fg(\032)771 844 y Ff(k)804 837 y Fm(=)e(1)p Fg(=s)919 818 y Ff(T)919 848 y(k)946 837 y Fg(s)967 844 y Ff(k)989 837 y Fg(:)0 920 y Fm(Considering)f(\014rst)f Fg(B)370 927 y Ff(k)403 920 y Fm(w)o(e)g(note)g(that)f(it)i(can)f(b)q(e)h(expressed)g(as)f(the)g(pro) q(duct)g(of)g Fg(B)1410 927 y Fk(0)1441 920 y Fm(with)h(a)f(sequence)0 977 y(of)k(pro)s(jection)g(matrices,)435 1079 y Fg(B)469 1086 y Ff(k)503 1079 y Fm(=)e Fg(B)585 1086 y Fk(0)605 1079 y Fm(\()p Fg(I)h Fe(\000)c Fg(\032)726 1086 y Fk(0)745 1079 y Fg(s)766 1086 y Fk(0)786 1079 y Fg(s)807 1060 y Ff(T)807 1090 y Fk(0)835 1079 y Fm(\))e Fe(\001)g(\001)g(\001)d Fm(\()p Fg(I)13 b Fe(\000)e Fg(\032)1042 1086 y Ff(k)q Fc(\000)p Fk(1)1108 1079 y Fg(s)1129 1086 y Ff(k)q Fc(\000)p Fk(1)1195 1079 y Fg(s)1216 1060 y Ff(T)1216 1090 y(k)q Fc(\000)p Fk(1)1283 1079 y Fm(\))p Fg(:)341 b Fm(\(6)p Fg(:)p Fm(9\))0 1181 y(No)o(w)10 b(w)o(e)g(apply)h(Lemma)f(2.1,)g(with)g Fg(y)15 b Fm(:=)d Fg(s)f Fm(in)g(the)f(de\014nition)i(\(1.3\),)e(to)f(this)i (pro)q(duct)f(of)g(pro)s(jections)0 1237 y(to)15 b(yield)h(the)g(relation)626 1294 y Fg(B)660 1301 y Ff(k)695 1294 y Fm(=)d Fg(A)777 1301 y Fk(0)807 1294 y Fe(\000)d Fg(A)886 1301 y Fk(0)906 1294 y Fg(S)934 1301 y Ff(k)955 1294 y Fg(N)997 1275 y Fc(\000)p Fk(1)992 1308 y Ff(k)1044 1294 y Fg(S)1075 1275 y Ff(T)1072 1305 y(k)1102 1294 y Fg(;)517 b Fm(\(6)p Fg(:)p Fm(10\))0 1377 y(for)15 b(all)h Fg(k)d Fe(\025)g Fm(1.)71 1434 y(Next)i(w)o(e)g(sho)o(w)f(b)o(y)h(induction)i (that)e Fg(C)758 1441 y Ff(k)794 1434 y Fm(has)g(the)g(compact)g(represen)o (tation)713 1536 y Fg(C)746 1543 y Ff(k)780 1536 y Fm(=)e Fg(Y)854 1543 y Ff(k)875 1536 y Fg(N)917 1517 y Fc(\000)p Fk(1)912 1550 y Ff(k)964 1536 y Fg(S)995 1517 y Ff(T)992 1547 y(k)1022 1536 y Fg(:)597 b Fm(\(6)p Fg(:)p Fm(11\))0 1638 y(By)17 b(the)g(de\014nition)i (\(6.8\),)d(w)o(e)h(ha)o(v)o(e)f(that)h Fg(C)790 1645 y Fk(1)825 1638 y Fm(=)f Fg(y)898 1645 y Fk(0)918 1638 y Fg(\032)942 1645 y Fk(0)961 1638 y Fg(s)982 1621 y Ff(T)982 1649 y Fk(0)1010 1638 y Fm(,)h(whic)o(h)h(agrees)f(with)g(\(6.11\))e(for)i Fg(k)g Fm(=)f(1.)0 1694 y(Assume)f(no)o(w)g(that)g(\(6.11\))e(holds)j(for)e Fg(k)q Fm(.)20 b(Then)c(b)o(y)f(\(6.8\),)13 1796 y Fg(C)46 1803 y Ff(k)q Fk(+1)153 1796 y Fm(=)42 b Fg(Y)256 1803 y Ff(k)278 1796 y Fg(N)320 1777 y Fc(\000)p Fk(1)315 1811 y Ff(k)366 1796 y Fg(S)397 1778 y Ff(T)394 1808 y(k)425 1796 y Fm(\()p Fg(I)13 b Fe(\000)d Fg(\032)545 1803 y Ff(k)566 1796 y Fg(s)587 1803 y Ff(k)609 1796 y Fg(s)630 1778 y Ff(T)630 1808 y(k)658 1796 y Fm(\))f(+)i Fg(\032)755 1803 y Ff(k)776 1796 y Fg(y)798 1803 y Ff(k)819 1796 y Fg(s)840 1778 y Ff(T)840 1808 y(k)153 1865 y Fm(=)42 b Fg(Y)256 1872 y Ff(k)278 1865 y Fg(N)320 1846 y Fc(\000)p Fk(1)315 1879 y Ff(k)366 1865 y Fg(S)397 1847 y Ff(T)394 1877 y(k)435 1865 y Fe(\000)10 b Fg(\032)504 1872 y Ff(k)525 1865 y Fg(Y)551 1872 y Ff(k)573 1865 y Fg(N)615 1846 y Fc(\000)p Fk(1)610 1879 y Ff(k)661 1865 y Fg(S)692 1847 y Ff(T)689 1877 y(k)719 1865 y Fg(s)740 1872 y Ff(k)762 1865 y Fg(s)783 1847 y Ff(T)783 1877 y(k)821 1865 y Fm(+)g Fg(\032)890 1872 y Ff(k)911 1865 y Fg(y)933 1872 y Ff(k)955 1865 y Fg(s)976 1847 y Ff(T)976 1877 y(k)153 1970 y Fm(=)230 1923 y Fd(h)270 1970 y Fg(Y)296 1977 y Ff(k)360 1970 y Fg(y)382 1977 y Ff(k)424 1923 y Fd(i)451 1898 y(")496 1942 y Fg(N)538 1923 y Fc(\000)p Fk(1)533 1956 y Ff(k)626 1942 y Fe(\000)p Fg(\032)685 1949 y Ff(k)707 1942 y Fg(N)749 1923 y Fc(\000)p Fk(1)744 1956 y Ff(k)795 1942 y Fg(S)826 1925 y Ff(T)823 1955 y(k)853 1942 y Fg(s)874 1949 y Ff(k)529 1998 y Fm(0)198 b(0)917 1898 y Fd(#)7 b(")993 1941 y Fg(S)1024 1925 y Ff(T)1021 1955 y(k)998 1998 y Fg(s)1019 1981 y Ff(T)1019 2011 y(k)1072 1898 y Fd(#)1106 1970 y Fm(+)1152 1923 y Fd(h)1192 1970 y Fg(Y)1218 1977 y Ff(k)1282 1970 y Fg(y)1304 1977 y Ff(k)1346 1923 y Fd(i)1373 1898 y(")1418 1941 y Fm(0)52 b(0)1418 1998 y(0)41 b Fg(\032)1506 2005 y Ff(k)1548 1898 y Fd(#)8 b(")1625 1941 y Fg(S)1656 1925 y Ff(T)1653 1955 y(k)1629 1998 y Fg(s)1650 1981 y Ff(T)1650 2011 y(k)1703 1898 y Fd(#)153 2111 y Fm(=)42 b Fg(Y)256 2118 y Ff(k)q Fk(+1)331 2039 y Fd(")375 2083 y Fg(N)417 2064 y Fc(\000)p Fk(1)412 2097 y Ff(k)506 2083 y Fe(\000)p Fg(\032)565 2090 y Ff(k)586 2083 y Fg(N)628 2064 y Fc(\000)p Fk(1)623 2097 y Ff(k)674 2083 y Fg(S)705 2067 y Ff(T)702 2096 y(k)732 2083 y Fg(s)753 2090 y Ff(k)408 2139 y Fm(0)187 b Fg(\032)642 2146 y Ff(k)796 2039 y Fd(#)828 2111 y Fg(S)856 2118 y Ff(k)q Fk(+1)922 2111 y Fg(:)697 b Fm(\(6.12\))0 2243 y(Note,)14 b(ho)o(w)o(ev)o(er,)g(that)432 2303 y Fd(")477 2348 y Fg(N)519 2328 y Fc(\000)p Fk(1)514 2362 y Ff(k)607 2348 y Fe(\000)p Fg(\032)666 2355 y Ff(k)687 2348 y Fg(N)729 2328 y Fc(\000)p Fk(1)724 2362 y Ff(k)776 2348 y Fg(S)807 2331 y Ff(T)804 2361 y(k)834 2348 y Fg(s)855 2355 y Ff(k)510 2404 y Fm(0)187 b Fg(\032)744 2411 y Ff(k)897 2303 y Fd(#)8 b(")974 2344 y Fg(N)1011 2351 y Ff(k)1074 2344 y Fg(S)1105 2328 y Ff(T)1102 2358 y(k)1132 2344 y Fg(s)1153 2351 y Ff(k)992 2401 y Fm(0)1115 2383 y Fk(1)p 1105 2390 38 2 v 1105 2417 a Ff(\032)1123 2423 y Fb(k)1195 2303 y Fd(#)1232 2375 y Fm(=)13 b Fg(I)t(;)315 b Fm(\(6)p Fg(:)p Fm(13\))851 2603 y(23)p eop %%Page: 24 25 bop 0 120 a Fm(whic)o(h)23 b(implies)h(that)d(the)h(second)h(matrix)e(on)h (the)g(righ)o(t)g(hand)h(side)g(of)e(\(6.12\))f(is)j Fg(N)1578 101 y Fc(\000)p Fk(1)1573 134 y Ff(k)q Fk(+1)1639 120 y Fm(.)40 b(By)0 177 y(induction)24 b(this)e(establishes)h(\(6.11\).)38 b(Finally)l(,)25 b(substituting)d(\(6.10\))e(and)i(\(6.11\))e(in)j(\(6.6\),)e (w)o(e)0 233 y(obtain)15 b(\(6.4\).)1699 289 y Fa(2)71 371 y Fm(W)l(e)d(no)o(w)f(deriv)o(e)i(a)f(compact)g(represen)o(tation)g(of)f(the) i(in)o(v)o(erse)f(Bro)o(yden)g(up)q(date)h(whic)o(h)g(is)f(giv)o(en)0 427 y(b)o(y)526 512 y Fg(A)560 493 y Fc(\000)p Fk(1)560 526 y Ff(k)q Fk(+1)639 512 y Fm(=)h Fg(A)721 493 y Fc(\000)p Fk(1)721 526 y Ff(k)778 512 y Fm(+)829 481 y(\()p Fg(s)868 488 y Ff(k)899 481 y Fe(\000)e Fg(A)979 462 y Fc(\000)p Fk(1)979 495 y Ff(k)1026 481 y Fg(y)1048 488 y Ff(k)1069 481 y Fm(\))p Fg(s)1108 464 y Ff(T)1108 494 y(k)1136 481 y Fg(A)1170 462 y Fc(\000)p Fk(1)1170 495 y Ff(k)p 829 501 389 2 v 936 547 a Fg(s)957 531 y Ff(T)957 561 y(k)985 547 y Fg(A)1019 528 y Fc(\000)p Fk(1)1019 561 y Ff(k)1066 547 y Fg(y)1088 554 y Ff(k)1632 512 y Fm(\(6)p Fg(:)p Fm(14\))0 626 y(\(see)k(for)g(example)h(Dennis)g(and)f(Sc)o(hnab)q(el)i (\(1983\)\).)0 720 y Fn(Theorem)g(6.2)22 b Fl(L)n(et)f Fg(A)429 701 y Fc(\000)p Fk(1)429 732 y(0)497 720 y Fl(b)n(e)g(a)h(nonsingular)e (starting)g(matrix,)k(and)d(let)g Fg(A)1391 701 y Fc(\000)p Fk(1)1391 734 y Ff(k)1459 720 y Fl(b)n(e)g(obtaine)n(d)g(by)0 777 y(up)n(dating)14 b Fg(A)215 757 y Fc(\000)p Fk(1)215 789 y(0)281 777 y Fg(k)h Fl(times)e(using)g(the)h(inverse)f(Br)n(oyden)h(formula) g(\(6.14\))g(and)g(the)g(p)n(airs)f Fe(f)p Fg(s)1574 784 y Ff(i)1588 777 y Fg(;)8 b(y)1631 784 y Ff(i)1645 777 y Fe(g)1668 757 y Ff(k)q Fc(\000)p Fk(1)1668 789 y Ff(i)p Fk(=0)1734 777 y Fl(.)0 833 y(Then)325 889 y Fg(A)359 870 y Fc(\000)p Fk(1)359 903 y Ff(k)419 889 y Fm(=)13 b Fg(A)501 870 y Fc(\000)p Fk(1)501 902 y(0)558 889 y Fe(\000)e Fm(\()p Fg(A)656 870 y Fc(\000)p Fk(1)656 902 y(0)703 889 y Fg(Y)729 896 y Ff(k)761 889 y Fe(\000)f Fg(S)834 896 y Ff(k)855 889 y Fm(\)\()p Fg(M)935 896 y Ff(k)966 889 y Fm(+)h Fg(S)1043 871 y Ff(T)1040 901 y(k)1070 889 y Fg(A)1104 870 y Fc(\000)p Fk(1)1104 902 y(0)1151 889 y Fg(Y)1177 896 y Ff(k)1199 889 y Fm(\))1217 871 y Fc(\000)p Fk(1)1263 889 y Fg(S)1294 871 y Ff(T)1291 901 y(k)1321 889 y Fg(A)1355 870 y Fc(\000)p Fk(1)1355 902 y(0)1403 889 y Fg(;)216 b Fm(\(6)p Fg(:)p Fm(15\))0 973 y Fl(wher)n(e)16 b Fg(S)156 980 y Ff(k)194 973 y Fl(and)g Fg(Y)308 980 y Ff(k)346 973 y Fl(ar)n(e)g(given)g(by)g (\(6.3\))g(and)g Fg(M)847 980 y Ff(k)885 973 y Fl(is)g(the)g Fg(k)11 b Fe(\002)g Fg(k)17 b Fl(matrix)510 1108 y Fm(\()p Fg(M)572 1115 y Ff(k)593 1108 y Fm(\))611 1115 y Ff(i;j)663 1108 y Fm(=)711 1036 y Fd(\()765 1079 y Fe(\000)p Fg(s)821 1063 y Ff(T)821 1091 y(i)p Fc(\000)p Fk(1)881 1079 y Fg(s)902 1086 y Ff(j)r Fc(\000)p Fk(1)1007 1079 y Fl(if)f Fg(i)d(>)g(j)765 1136 y Fm(0)219 b Fl(otherwise)1226 1108 y Fg(:)393 b Fm(\(6)p Fg(:)p Fm(16\))0 1265 y Fn(Pro)q(of.)20 b Fm(Let)495 1321 y Fg(U)e Fm(=)13 b Fg(Y)618 1328 y Ff(k)650 1321 y Fe(\000)d Fg(A)729 1328 y Fk(0)749 1321 y Fg(S)777 1328 y Ff(k)798 1321 y Fg(;)158 b(V)1005 1303 y Ff(T)1045 1321 y Fm(=)13 b Fg(N)1135 1302 y Fc(\000)p Fk(1)1130 1335 y Ff(k)1182 1321 y Fg(S)1213 1303 y Ff(T)1210 1333 y(k)1240 1321 y Fg(;)0 1405 y Fm(so)i(that)f(\(6.4\))g (b)q(ecomes)705 1461 y Fg(A)739 1468 y Ff(k)773 1461 y Fm(=)f Fg(A)855 1468 y Fk(0)885 1461 y Fm(+)d Fg(U)5 b(V)1003 1442 y Ff(T)1030 1461 y Fg(:)0 1545 y Fm(Applying)17 b(the)e(Sherman-Morrison-W)l (o)q(o)q(dbury)h(form)o(ula)f(w)o(e)g(obtain)115 1647 y Fg(A)149 1628 y Fc(\000)p Fk(1)149 1661 y Ff(k)238 1647 y Fm(=)41 b Fg(A)348 1628 y Fc(\000)p Fk(1)348 1659 y(0)406 1647 y Fe(\000)10 b Fg(A)485 1628 y Fc(\000)p Fk(1)485 1659 y(0)533 1647 y Fg(U)5 b Fm(\()p Fg(I)13 b Fm(+)d Fg(V)702 1628 y Ff(T)729 1647 y Fg(A)763 1628 y Fc(\000)p Fk(1)763 1659 y(0)811 1647 y Fg(U)5 b Fm(\))865 1628 y Fc(\000)p Fk(1)911 1647 y Fg(V)948 1628 y Ff(T)976 1647 y Fg(A)1010 1628 y Fc(\000)p Fk(1)1010 1659 y(0)238 1716 y Fm(=)41 b Fg(A)348 1697 y Fc(\000)p Fk(1)348 1728 y(0)406 1716 y Fe(\000)10 b Fg(A)485 1697 y Fc(\000)p Fk(1)485 1728 y(0)533 1716 y Fm(\()p Fg(Y)577 1723 y Ff(k)608 1716 y Fe(\000)g Fg(A)687 1723 y Fk(0)707 1716 y Fg(S)735 1723 y Ff(k)756 1716 y Fm(\)\()p Fg(I)j Fm(+)e Fg(N)913 1697 y Fc(\000)p Fk(1)908 1730 y Ff(k)960 1716 y Fg(S)991 1697 y Ff(T)988 1727 y(k)1018 1716 y Fg(A)1052 1697 y Fc(\000)p Fk(1)1052 1728 y(0)1099 1716 y Fm(\()p Fg(Y)1143 1723 y Ff(k)1174 1716 y Fe(\000)g Fg(A)1254 1723 y Fk(0)1274 1716 y Fg(S)1302 1723 y Ff(k)1323 1716 y Fm(\)\))1359 1697 y Fc(\000)p Fk(1)1405 1716 y Fg(N)1447 1697 y Fc(\000)p Fk(1)1442 1730 y Ff(k)1494 1716 y Fg(S)1525 1697 y Ff(T)1522 1727 y(k)1552 1716 y Fg(A)1586 1697 y Fc(\000)p Fk(1)1586 1728 y(0)238 1785 y Fm(=)41 b Fg(A)348 1765 y Fc(\000)p Fk(1)348 1797 y(0)406 1785 y Fe(\000)10 b Fm(\()p Fg(A)503 1765 y Fc(\000)p Fk(1)503 1797 y(0)550 1785 y Fg(Y)576 1792 y Ff(k)608 1785 y Fe(\000)g Fg(S)681 1792 y Ff(k)703 1785 y Fm(\)\()p Fg(N)776 1792 y Ff(k)806 1785 y Fm(+)g Fg(S)882 1766 y Ff(T)879 1796 y(k)909 1785 y Fg(A)943 1765 y Fc(\000)p Fk(1)943 1797 y(0)991 1785 y Fg(Y)1017 1792 y Ff(k)1049 1785 y Fe(\000)g Fg(S)1125 1766 y Ff(T)1122 1796 y(k)1152 1785 y Fg(S)1180 1792 y Ff(k)1201 1785 y Fm(\))1219 1766 y Fc(\000)p Fk(1)1266 1785 y Fg(S)1297 1766 y Ff(T)1294 1796 y(k)1324 1785 y Fg(A)1358 1765 y Fc(\000)p Fk(1)1358 1797 y(0)1405 1785 y Fg(:)0 1887 y Fm(By)15 b(\(6.5\))f(and)h(\(6.16\))f(w)o(e)h(ha)o(v)o(e)f(that)h Fg(N)707 1894 y Ff(k)738 1887 y Fe(\000)10 b Fg(S)814 1870 y Ff(T)811 1900 y(k)841 1887 y Fg(S)869 1894 y Ff(k)903 1887 y Fm(=)j Fg(M)995 1894 y Ff(k)1016 1887 y Fm(,)i(whic)o(h)h(giv)o(es)g (\(6.15\).)1714 1943 y Fa(2)0 2025 y Fm(Note)k(that)f(since)i(w)o(e)f(ha)o(v) o(e)g(assumed)g(that)f(all)i(the)f(up)q(dates)h(giv)o(en)f(b)o(y)g(\(6.14\))e (exist,)k(w)o(e)d(ha)o(v)o(e)0 2081 y(implicitly)14 b(assumed)c(the)h (nonsingularit)o(y)g(of)f Fg(A)824 2088 y Ff(k)846 2081 y Fm(.)18 b(This)11 b(nonsingularit)o(y)h(along)e(with)h(the)g(Sherman-)0 2137 y(Morrison)k(form)o(ula)f(ensures)i(that)f(\()p Fg(M)681 2144 y Ff(k)712 2137 y Fm(+)10 b Fg(S)788 2121 y Ff(T)785 2151 y(k)815 2137 y Fg(A)849 2118 y Fc(\000)p Fk(1)849 2150 y(0)897 2137 y Fg(Y)923 2144 y Ff(k)944 2137 y Fm(\))15 b(is)h(nonsingular.)71 2194 y(These)h(represen)o(tations)g(of)g(Bro)o(yden)g(matrices)h(ha)o(v)o(e)f (b)q(een)h(used)g(b)o(y)f(Biegler,)i(No)q(cedal)f(and)0 2250 y(Sc)o(hmid)d(\(1993\))d(to)h(appro)o(ximate)g(a)h(p)q(ortion)g(of)f(the)h (Hessian)g(of)f(the)h(Lagrangian)g(in)g(a)g(successiv)o(e)0 2307 y(quadratic)h(programming)g(metho)q(d)g(for)g(constrained)h (optimization.)851 2603 y(24)p eop %%Page: 25 26 bop 0 120 a Fh(7.)37 b(Relation)17 b(to)i(Multiple)e(Secan)n(t)i(Up)r(dates.) 71 201 y Fm(There)13 b(is)h(a)e(close)i(algebraic)g(corresp)q(ondence,)h(and) e(in)h(certain)g(sp)q(ecial)g(cases)g(an)f(equiv)m(alence,)0 258 y(b)q(et)o(w)o(een)j(the)h(represen)o(tations)f(of)g(a)f(sequence)j(of)d (quasi-Newton)i(up)q(dates)g(that)e(ha)o(v)o(e)h(b)q(een)h(dis-)0 314 y(cussed)i(in)g(this)g(pap)q(er,)g(and)g(m)o(ultiple)h(secan)o(t)e(up)q (dates)h(that)e(ha)o(v)o(e)h(previously)i(b)q(een)f(discussed)0 371 y(b)o(y)d(sev)o(eral)g(authors)f(including)k(Barnes)c(\(1965\),)f(Ga)o(y) h(and)h(Sc)o(hnab)q(el)i(\(1978\),)c(Sc)o(hnab)q(el)j(\(1983\),)0 427 y(and)c(Khalfan)g(\(1989\).)18 b(In)13 b(this)g(section)h(w)o(e)e (brie\015y)i(discuss)g(this)f(corresp)q(ondence,)h(for)e(the)h(BF)o(GS,)0 484 y(SR1,)j(and)f(Bro)o(yden)h(up)q(dates.)21 b(W)l(e)16 b(also)f(mak)o(e)g (a)g(few)h(commen)o(ts)f(ab)q(out)g(the)g(tradeo\013s)g(b)q(et)o(w)o(een)0 540 y(using)f(these)g(t)o(w)o(o)e(t)o(yp)q(es)i(of)f(up)q(dates.)20 b(In)14 b(additional)h(to)d(the)i(notation)f(of)g(the)h(preceding)h (sections,)0 597 y(w)o(e)f(use)g(the)g(notation)f(that)507 585 y(\026)497 597 y Fg(R)532 604 y Ff(k)567 597 y Fm(is)h(the)g Fg(k)9 b Fe(\002)e Fg(k)15 b Fm(matrix)f(that)f(is)h(the)g(strict)g(upp)q(er) h(triangle)f(of)g Fg(S)1661 580 y Ff(T)1658 610 y(k)1688 597 y Fg(Y)1714 604 y Ff(k)1735 597 y Fm(,)0 653 y(i.e.)88 642 y(\026)78 653 y Fg(R)113 660 y Ff(k)149 653 y Fm(=)i Fg(R)235 660 y Ff(k)266 653 y Fe(\000)10 b Fg(D)349 660 y Ff(k)385 653 y Fm(where)16 b Fg(R)552 660 y Ff(k)588 653 y Fm(and)f Fg(D)714 660 y Ff(k)750 653 y Fm(are)g(de\014ned)i(b)o(y)e(\(2.3\))e(and)j(\(2.7\).)i (Th)o(us)652 755 y Fg(S)683 736 y Ff(T)680 766 y(k)710 755 y Fg(Y)736 762 y Ff(k)770 755 y Fm(=)13 b Fg(L)849 762 y Ff(k)880 755 y Fm(+)e Fg(D)964 762 y Ff(k)995 755 y Fm(+)1050 744 y(\026)1040 755 y Fg(R)1075 762 y Ff(k)1655 755 y Fm(\(7)p Fg(:)p Fm(1\))0 857 y(where)k Fg(L)162 864 y Ff(k)199 857 y Fm(is)h(de\014ned)g(in)g (\(2.18\).)71 914 y(Multiple)22 b(secan)o(t)f(up)q(dates)g(are)f(up)q(dates)i (that)e(enforce)h(the)g(last)f Fg(k)i Fm(secan)o(t)f(equations,)h(i.e.)0 970 y(in)e(the)g(notation)f(of)g(Section)i(1)e Fg(B)622 977 y Ff(k)644 970 y Fg(S)672 977 y Ff(k)713 970 y Fm(=)h Fg(Y)794 977 y Ff(k)835 970 y Fm(or)g Fg(H)934 977 y Ff(k)955 970 y Fg(Y)981 977 y Ff(k)1022 970 y Fm(=)h Fg(S)1106 977 y Ff(k)1127 970 y Fm(.)33 b(While)21 b(the)e(pap)q(ers)h(men)o(tioned)0 1027 y(ab)q(o)o(v)o(e)15 b(generally)i(consider)f(using)g(m)o(ultiple)h (secan)o(t)f(up)q(date)g(to)e(up)q(date)i Fg(B)1331 1034 y Ff(k)1369 1027 y Fm(to)e Fg(B)1458 1034 y Ff(k)q Fk(+1)1525 1027 y Fm(,)h(analogous)0 1083 y(up)q(dates)f(to)e(those)h(considered)i(in)f (this)f(pap)q(er)h(w)o(ould)g(arise)f(from)g(using)h(m)o(ultiple)h(secan)o(t) d(up)q(dates)0 1140 y(to)g(up)q(date)h Fg(B)237 1147 y Fk(0)269 1140 y Fm(to)f Fg(B)356 1147 y Ff(k)390 1140 y Fm(or)g Fg(H)481 1147 y Fk(0)513 1140 y Fm(to)f Fg(H)603 1147 y Ff(k)624 1140 y Fm(.)19 b(This)13 b(is)g(the)g(con)o(text)e(in)j(whic)o(h)f(w)o(e)f (consider)h(m)o(ultiple)h(secan)o(t)0 1196 y(up)q(dates)i(in)g(this)f (section.)71 1253 y(In)g(this)g(con)o(text,)f(the)g(m)o(ultiple)j(secan)o(t)d (v)o(ersion)h(of)f(the)h(direct)g(BF)o(GS)f(up)q(date)h(applied)i(to)d Fg(B)1728 1260 y Fk(0)0 1309 y Fm(is)i(giv)o(en)f(b)o(y)350 1372 y Fg(B)384 1379 y Ff(k)418 1372 y Fm(=)e Fg(B)500 1379 y Fk(0)530 1372 y Fm(+)e Fg(Y)602 1379 y Ff(k)624 1372 y Fm(\()p Fg(Y)678 1353 y Ff(T)668 1383 y(k)705 1372 y Fg(S)733 1379 y Ff(k)754 1372 y Fm(\))772 1343 y Fc(\000)p Fk(1)819 1372 y Fg(Y)856 1353 y Ff(T)845 1383 y(k)893 1372 y Fe(\000)g Fg(B)973 1379 y Fk(0)993 1372 y Fg(S)1021 1379 y Ff(k)1042 1372 y Fm(\()p Fg(S)1091 1353 y Ff(T)1088 1383 y(k)1118 1372 y Fg(B)1152 1379 y Fk(0)1172 1372 y Fg(S)1200 1379 y Ff(k)1221 1372 y Fm(\))1239 1343 y Fc(\000)p Fk(1)1286 1372 y Fg(S)1317 1353 y Ff(T)1314 1383 y(k)1344 1372 y Fg(B)1378 1379 y Fk(0)1655 1372 y Fm(\(7)p Fg(:)p Fm(2\))0 1455 y(or)k(using)h(a)f(represen)o(tation)g(analogous)g(to)f (\(2.17\),)257 1598 y Fg(B)291 1605 y Ff(k)326 1598 y Fm(=)f Fg(B)408 1605 y Fk(0)438 1598 y Fe(\000)484 1551 y Fd(h)524 1598 y Fg(B)558 1605 y Fk(0)578 1598 y Fg(S)606 1605 y Ff(k)669 1598 y Fg(Y)695 1605 y Ff(k)737 1551 y Fd(i)765 1526 y(")809 1570 y Fg(S)840 1553 y Ff(T)837 1583 y(k)867 1570 y Fg(B)901 1577 y Fk(0)922 1570 y Fg(S)950 1577 y Ff(k)1075 1570 y Fm(0)879 1626 y(0)110 b Fe(\000)p Fg(Y)1084 1610 y Ff(T)1073 1640 y(k)1112 1626 y Fg(S)1140 1633 y Ff(k)1182 1526 y Fd(#)1206 1538 y Fc(\000)p Fk(1)1261 1526 y Fd(")1306 1570 y Fg(S)1337 1553 y Ff(T)1334 1583 y(k)1364 1570 y Fg(B)1398 1577 y Fk(0)1330 1626 y Fg(Y)1366 1610 y Ff(T)1356 1640 y(k)1439 1526 y Fd(#)1470 1598 y Fg(;)172 b Fm(\(7)p Fg(:)p Fm(3\))0 1733 y(\(assuming)18 b Fg(k)g Fe(\024)g Fg(n)p Fm(\).)28 b(The)18 b(matrix)f Fg(B)679 1740 y Ff(k)719 1733 y Fm(giv)o(en)h(b)o(y)g(\(7.2\))f(alw)o(a)o(ys)g(ob)q(eys)h(the)g Fg(k)h Fm(secan)o(t)f(equations)0 1789 y Fg(B)34 1796 y Ff(k)56 1789 y Fg(S)84 1796 y Ff(k)123 1789 y Fm(=)g Fg(Y)202 1796 y Ff(k)224 1789 y Fm(.)28 b(Sc)o(hnab)q(el)20 b(\(1983\))d(sho)o(ws)h(that,)f (assuming)i Fg(B)1088 1796 y Fk(0)1126 1789 y Fm(is)g(symmetric)g(and)f(p)q (ositiv)o(e)h(de\014-)0 1846 y(nite,)f Fg(B)140 1853 y Ff(k)179 1846 y Fm(is)g(symmetric)g(if)g(and)f(only)h(if)g Fg(Y)768 1829 y Ff(T)757 1859 y(k)795 1846 y Fg(S)823 1853 y Ff(k)862 1846 y Fm(is)g(symmetric,)g(and)f(in)h(addition)h Fg(B)1508 1853 y Ff(k)1547 1846 y Fm(is)f(p)q(ositiv)o(e)0 1902 y(de\014nite)d(if)g (and)f(only)g(if)h Fg(Y)463 1886 y Ff(T)453 1916 y(k)491 1902 y Fg(S)519 1909 y Ff(k)554 1902 y Fm(is)f(p)q(ositiv)o(e)h(de\014nite.)21 b(These)14 b(conditions)h(are)f(satis\014ed)g(if)h Fg(f)5 b Fm(\()p Fg(x)p Fm(\))13 b(is)h(a)0 1959 y(p)q(ositiv)o(e)h(de\014nite)h (quadratic,)e(but)g(not)g(in)h(general)g(otherwise.)k(Sc)o(hnab)q(el)d (\(1983\))d(discusses)i(w)o(a)o(ys)0 2015 y(to)i(p)q(erturb)h Fg(Y)252 2022 y Ff(k)291 2015 y Fm(to)355 2004 y(~)348 2015 y Fg(Y)374 2022 y Ff(k)413 2015 y Fm(so)f(that)579 2004 y(~)572 2015 y Fg(Y)609 1999 y Ff(T)598 2028 y(k)636 2015 y Fg(S)664 2022 y Ff(k)703 2015 y Fm(is)h(symmetric)f(and)h(p)q(ositiv)o(e)g (de\014nite,)h(at)e(the)g(cost)g(of)g(no)0 2072 y(longer)d(exactly)g (satisfying)h(the)e(original)i(secan)o(t)f(equations)g(other)g(than)f(the)h (most)f(recen)o(t.)20 b(These)0 2128 y(p)q(erturbations)d(ha)o(v)o(e)f(some)g (relation)h(to)f(the)g(comparisons)h(of)f(this)g(section,)h(and)g(w)o(e)f (will)i(return)0 2185 y(to)d(them)g(shortly)l(.)71 2241 y(By)f(comparing)g (the)g(m)o(ultiple)i(secan)o(t)e(up)q(date)h(\(7.3\))e(and)h(the)g(represen)o (tation)g(for)g Fg(k)h Fm(consecu-)0 2298 y(tiv)o(e,)h(standard)f(BF)o(GS)h (up)q(dates)g(\(2.17\),)e(it)i(is)h(clear)f(that)f(these)i(t)o(w)o(o)d(form)o (ulas)i(are)f(v)o(ery)h(similar)0 2354 y(algebraically)l(.)23 b(It)15 b(is)h(also)g(immediate)g(that)f(if)h Fg(Y)866 2338 y Ff(T)855 2367 y(k)893 2354 y Fg(S)921 2361 y Ff(k)955 2354 y Fm(=)e Fg(D)1042 2361 y Ff(k)1063 2354 y Fm(,)h(the)h(m)o(ultiple)h(BF)o (GS)e(up)q(date)h(to)f Fg(B)1728 2361 y Fk(0)0 2410 y Fm(is)e(equiv)m(alen)o (t)i(to)d(p)q(erforming)i Fg(k)g Fm(standard)e(BF)o(GS)h(up)q(dates.)19 b(This)14 b(condition,)g(whic)o(h)g(means)f(that)0 2467 y Fg(s)21 2450 y Ff(T)21 2479 y(i)49 2467 y Fg(y)71 2474 y Ff(j)102 2467 y Fm(=)g(0)f(for)h(all)g Fg(i)g Fe(6)p Fm(=)g Fg(j)s Fm(,)f(is)h(satis\014ed) g(if)h Fg(f)5 b Fm(\()p Fg(x)p Fm(\))12 b(is)h(quadratic)g(and)g(the)g(step)g (directions)h(are)e(m)o(utually)851 2603 y(25)p eop %%Page: 26 27 bop 0 120 a Fm(conjugate,)18 b(but)h(not)f(in)h(general)f(otherwise.)30 b(In)19 b(general,)g(the)f(t)o(w)o(o)f(form)o(ulas)h(\(2.17\))e(and)j (\(7.3\))0 177 y(result)d(in)g(di\013eren)o(t)f(matrices)g Fg(B)575 184 y Ff(k)597 177 y Fm(.)71 233 y(Iden)o(tical)g(commen)o(ts)f(are) g(true)g(regarding)g(the)h(BF)o(GS)e(up)q(date)i(to)f(the)g(in)o(v)o(erse)g (Hessian.)21 b(The)0 289 y(in)o(v)o(erse)16 b(form)e(of)h(the)g(m)o(ultiple)i (BF)o(GS)e(up)q(date)h(\(7.3\))d(is)34 449 y Fg(H)72 456 y Ff(k)106 449 y Fm(=)g Fg(H)192 456 y Fk(0)222 449 y Fm(+)267 402 y Fd(h)308 449 y Fg(S)336 456 y Ff(k)398 449 y Fg(H)436 456 y Fk(0)456 449 y Fg(Y)482 456 y Ff(k)524 402 y Fd(i)552 352 y(2)552 425 y(6)552 452 y(4)600 393 y Fg(W)649 374 y Fc(\000)p Ff(T)643 407 y(k)714 393 y Fm(+)e Fg(W)809 374 y Fc(\000)p Fk(1)803 407 y Ff(k)856 393 y Fm(\()p Fg(Y)910 376 y Ff(T)900 406 y(k)938 393 y Fg(H)976 400 y Fk(0)995 393 y Fg(Y)1021 400 y Ff(k)1043 393 y Fm(\))p Fg(W)1110 374 y Fc(\000)p Ff(T)1104 407 y(k)1206 393 y Fe(\000)p Fg(W)1290 374 y Fc(\000)p Fk(1)1284 407 y Ff(k)813 508 y Fe(\000)p Fg(W)897 489 y Fc(\000)p Ff(T)891 522 y(k)1261 508 y Fm(0)1359 352 y Fd(3)1359 425 y(7)1359 452 y(5)1394 352 y(2)1394 425 y(6)1394 452 y(4)1474 393 y Fg(S)1505 376 y Ff(T)1502 406 y(k)1443 506 y Fg(Y)1479 489 y Ff(T)1469 519 y(k)1507 506 y Fg(H)1545 513 y Fk(0)1585 352 y Fd(3)1585 425 y(7)1585 452 y(5)1655 449 y Fm(\(7)p Fg(:)p Fm(4\))0 614 y(where)19 b Fg(W)178 621 y Ff(k)217 614 y Fm(=)g Fg(Y)307 597 y Ff(T)297 627 y(k)335 614 y Fg(S)363 621 y Ff(k)384 614 y Fm(.)29 b(Again,)19 b(assuming)g Fg(H)814 621 y Fk(0)852 614 y Fm(is)g(p)q(ositiv)o(e)g(de\014nite,)h(this)f(matrix)f(is)h(symmetric)0 670 y(and)e(p)q(ositiv)o(e)g(de\014nite)h(if)e(and)h(only)g(if)g Fg(Y)734 654 y Ff(T)724 684 y(k)762 670 y Fg(S)790 677 y Ff(k)827 670 y Fm(is)g(symmetric)g(and)f(p)q(ositiv)o(e)h(de\014nite.)25 b(Again,)17 b(the)0 727 y(algebraic)i(forms)e(for)h(\(7.4\))e(and)j(\(2.6\))d (are)i(v)o(ery)g(similar,)i(and)e(b)o(y)g(comparing)g(these)h(equations)0 783 y(and)c(recalling)h(de\014nitions)h(\(2.3\))c(and)i(\(2.7\),)e(it)i(is)g (immediate)h(that)e(the)g(up)q(dates)i(are)e(iden)o(tical)j(if)0 840 y Fg(Y)36 823 y Ff(T)26 853 y(k)64 840 y Fg(S)92 847 y Ff(k)126 840 y Fm(=)c Fg(D)212 847 y Ff(k)233 840 y Fm(,)i(and)g(in)h (general)g(are)f(di\013eren)o(t)g(otherwise.)71 896 y(F)l(rom)d(these)i (comparisons,)f(one)h(can)f(see)h(that)f(in)h(the)f(con)o(text)g(of)g (limited)i(memory)e(metho)q(ds,)0 953 y(the)21 b(m)o(ultiple)i(BF)o(GS)e(up)q (dates)h(\(7.3\))d(or)i(\(7.4\))f(w)o(ould)h(o\013er)g(similar)h(algebraic)g (e\016ciencies)i(to)0 1009 y(the)18 b(represen)o(tations)h(\(2.17\))d(or)i (\(2.6\))f(for)h(a)g(sequence)h(of)f(standard)g(BF)o(GS)g(up)q(dates,)i(that) d(are)0 1066 y(discussed)j(in)g(this)f(pap)q(er.)31 b(The)19 b(m)o(ultiple)h(BF)o(GS)e(up)q(dates)i(ha)o(v)o(e)e(the)h(disadv)m(an)o (tage,)g(ho)o(w)o(ev)o(er,)0 1122 y(that)g Fg(B)137 1129 y Ff(k)178 1122 y Fm(or)g Fg(H)276 1129 y Ff(k)317 1122 y Fm(is)h(not)f(in)h (general)g(symmetric)g(and)g(p)q(ositiv)o(e)g(de\014nite)h(ev)o(en)f(if)g (the)g(condition)0 1179 y Fg(s)21 1162 y Ff(T)21 1191 y(i)49 1179 y Fg(y)71 1186 y Ff(i)103 1179 y Fg(>)e Fm(0,)g Fg(i)f Fm(=)h(0)p Fg(;)8 b(:::;)g(k)i Fe(\000)i Fm(1,)19 b(that)e(guaran)o(tees)g (that)h(the)g(matrix)g(pro)q(duced)h(b)o(y)f Fg(k)h Fm(consecutiv)o(e,)0 1235 y(standard)i(BF)o(GS)f(up)q(dates)i(is)f(symmetric)g(and)g(p)q(ositiv)o (e)h(de\014nite,)i(is)e(satis\014ed.)37 b(Instead,)23 b(the)0 1291 y(m)o(ultiple)d(secan)o(t)f(up)q(dates)g(require)g(the)g(m)o(uc)o(h)f (stronger)g(condition)i(that)e Fg(Y)1381 1275 y Ff(T)1371 1305 y(k)1409 1291 y Fg(S)1437 1298 y Ff(k)1477 1291 y Fm(b)q(e)h(symmetric)0 1348 y(and)g(p)q(ositiv)o(e)h(de\014nite,)i(and)d(there)g(do)q(es)h(not)e (app)q(ear)i(to)e(b)q(e)i(a)f(practical)h(w)o(a)o(y)e(to)g(enforce)i(this)0 1404 y(condition)14 b(computationally)l(.)20 b(Sc)o(hnab)q(el)13 b(\(1983\))e(has)h(instead)h(considered)g(w)o(a)o(ys)f(to)f(p)q(erturb)i Fg(Y)1674 1411 y Ff(k)1708 1404 y Fm(to)7 1449 y(~)0 1461 y Fg(Y)26 1468 y Ff(k)61 1461 y Fm(so)f(that)217 1449 y(~)210 1461 y Fg(Y)247 1444 y Ff(T)236 1474 y(k)274 1461 y Fg(S)302 1468 y Ff(k)337 1461 y Fm(is)h(symmetric)g(and)g(p)q(ositiv)o(e)h (de\014nite,)h(and)e(the)g(most)f(recen)o(t)h(secan)o(t)g(condition)0 1517 y(\(i.e.)35 b(the)21 b(last)f(column)h(of)f Fg(Y)533 1524 y Ff(k)555 1517 y Fm(\))g(is)g(unc)o(hanged.)36 b(In)21 b(addition,)i(if)e (the)f(columns)h(of)f Fg(S)1560 1524 y Ff(k)1601 1517 y Fm(are)g(not)0 1574 y(strongly)g(linear)h(indep)q(enden)o(t,)j(the)c(up)q(dates)h(\(7.3\))e (or)h(\(7.4\))e(ma)o(y)i(b)q(e)h(n)o(umerical)g(unstable)g(so)0 1630 y(some)13 b(secan)o(t)h(pairs)g(m)o(ust)f(b)q(e)h(dropp)q(ed)h(from)e Fg(S)841 1637 y Ff(k)875 1630 y Fm(and)h Fg(Y)988 1637 y Ff(k)1010 1630 y Fm(.)19 b(Due)14 b(to)f(the)h(additional)h(computations)0 1687 y(required)k(b)o(y)f(these)f(p)q(erturbations,)i(and)f(the)g(lac)o(k)g (of)f(symmetry)g(and)h(p)q(ositiv)o(e)h(de\014niteness)g(in)0 1743 y(the)13 b(unp)q(erturb)q(ed)i(m)o(ultiple)f(secan)o(t)f(BF)o(GS)f(up)q (date,)i(it)f(do)q(es)g(not)g(seem)g(adv)m(an)o(tageous)f(to)h(use)g(the)0 1800 y(m)o(ultiple)j(secan)o(t)f(BF)o(GS)f(up)q(date)h(rather)f(than)h Fg(k)h Fm(consecutiv)o(e,)f(standard)f(BF)o(GS)g(up)q(dates)h(in)h(the)0 1856 y(con)o(text)j(of)f(limited)j(memory)e(metho)q(ds.)32 b(An)20 b(in)o(teresting)f(related)h(question)g(is)g(whether)f(there)0 1912 y(is)e(a)g(natural)f(p)q(erturbation)h(of)g Fg(Y)595 1919 y Ff(k)633 1912 y Fm(that)f(causes)h(the)g(m)o(ultiple)h(secan)o(t)f(up)q (date)g(to)f(b)q(e)h(equiv)m(alen)o(t)0 1969 y(to)i(\(2.17\);)g(this)h(do)q (es)g(not)g(seem)f(to)g(b)q(e)h(the)g(case,)g(but)g(as)f(men)o(tioned)i(b)q (elo)o(w)f(the)f(situation)h(is)0 2025 y(di\013eren)o(t)15 b(for)g(the)g(SR1)h(up)q(date.)71 2082 y(No)o(w)j(w)o(e)h(turn)g(to)g(the)g (SR1)h(up)q(date.)36 b(The)20 b(m)o(ultiple)i(secan)o(t)e(SR1)h(up)q(date,)h (whic)o(h)f(to)e(our)0 2138 y(kno)o(wledge)d(w)o(as)e(\014rst)h(discussed)i (in)f(Sc)o(hnab)q(el)h(\(1983\),)c(if)i(applied)i(to)e Fg(B)1288 2145 y Fk(0)1323 2138 y Fm(is)h(giv)o(en)g(b)o(y)298 2240 y Fg(B)332 2247 y Ff(k)367 2240 y Fm(=)d Fg(B)449 2247 y Fk(0)479 2240 y Fm(+)d(\()p Fg(Y)568 2247 y Ff(k)600 2240 y Fe(\000)g Fg(B)679 2247 y Fk(0)700 2240 y Fg(S)728 2247 y Ff(k)749 2240 y Fm(\)\(\()p Fg(Y)829 2247 y Ff(k)860 2240 y Fe(\000)g Fg(B)939 2247 y Fk(0)959 2240 y Fg(S)987 2247 y Ff(k)1008 2240 y Fm(\))1026 2222 y Ff(T)1054 2240 y Fg(S)1082 2247 y Ff(k)1103 2240 y Fm(\))1121 2222 y Fc(\000)p Fk(1)1168 2240 y Fm(\()p Fg(Y)1212 2247 y Ff(k)1243 2240 y Fe(\000)h Fg(B)1323 2247 y Fk(0)1343 2240 y Fg(S)1371 2247 y Ff(k)1392 2240 y Fm(\))1410 2222 y Ff(T)1437 2240 y Fg(:)205 b Fm(\(7)p Fg(:)p Fm(5\))0 2343 y(The)13 b(matrix)g Fg(B)271 2350 y Ff(k)306 2343 y Fm(giv)o(en)g(b)o(y)g(\(7.5\))f(alw)o(a)o(ys) g(ob)q(eys)i(the)f Fg(k)h Fm(secan)o(t)e(equations)i Fg(B)1343 2350 y Ff(k)1365 2343 y Fg(S)1393 2350 y Ff(k)1427 2343 y Fm(=)f Fg(Y)1501 2350 y Ff(k)1522 2343 y Fm(.)19 b(Assuming)0 2399 y Fg(B)34 2406 y Fk(0)68 2399 y Fm(is)c(symmetric,)f Fg(B)380 2406 y Ff(k)415 2399 y Fm(is)h(symmetric)f(if)g(and)g(only)h(if)f Fg(Y)983 2382 y Ff(T)972 2412 y(k)1010 2399 y Fg(S)1038 2406 y Ff(k)1074 2399 y Fm(is)g(symmetric,)g(whic)o(h)h(is)f(true)g(if)h Fg(f)5 b Fm(\()p Fg(x)p Fm(\))0 2455 y(is)17 b(quadratic)g(but)g(not)f (necessarily)i(otherwise.)24 b(Lik)o(e)18 b(the)e(standard)g(SR1)h(up)q (date,)h Fg(B)1542 2462 y Ff(k)1580 2455 y Fm(giv)o(en)f(b)o(y)851 2603 y(26)p eop %%Page: 27 28 bop 0 120 a Fm(\(7.5\))13 b(is)i(not)f(necessarily)i(p)q(ositiv)o(e)g (de\014nite)g(ev)o(en)f(if)g(the)f(necessary)h(conditions)h(for)e(the)g (standard)0 177 y(BF)o(GS)h(or)f(m)o(ultiple)j(BF)o(GS)e(up)q(date)h(to)e(b)q (e)i(p)q(ositiv)o(e)g(de\014nite)h(are)e(met.)71 233 y(Comparing)21 b(the)g(m)o(ultiple)i(SR1)e(up)q(date)h(\(7.5\))e(to)g(the)h(form)o(ula)g (\(5.2\))f(for)g Fg(k)i Fm(consecutiv)o(e,)0 289 y(standard)c(SR1)i(up)q (dates,)g(it)f(is)g(clear)h(that)e(the)h(only)g(di\013erence)h(b)q(et)o(w)o (een)g(these)f(t)o(w)o(o)e(form)o(ulae)0 346 y(is)h(that)f(\(7.5\))e(con)o (tains)j(the)f(term)g Fg(Y)668 329 y Ff(T)657 359 y(k)695 346 y Fg(S)723 353 y Ff(k)762 346 y Fm(as)g(part)f(of)h(the)h(middle,)h(in)o(v)o (erse)f(expression,)g(instead)0 402 y(of)f(the)g(symmetric)g(term)g Fg(D)506 409 y Ff(k)538 402 y Fm(+)12 b Fg(L)616 409 y Ff(k)648 402 y Fm(+)g Fg(L)726 386 y Ff(T)726 416 y(k)770 402 y Fm(in)18 b(\(5.2\).)24 b(Recalling)c(that)c Fg(Y)1293 386 y Ff(T)1283 416 y(k)1321 402 y Fg(S)1349 409 y Ff(k)1387 402 y Fm(=)1449 391 y(\026)1440 402 y Fg(R)1475 386 y Ff(T)1475 416 y(k)1513 402 y Fm(+)c Fg(D)1598 409 y Ff(k)1630 402 y Fm(+)g Fg(L)1708 386 y Ff(T)1708 416 y(k)1735 402 y Fm(,)0 459 y(it)17 b(is)h(immediate)f (that)g(\(7.5\))e(and)i(\(5.2\))e(are)i(iden)o(tical)h(if)1045 447 y(\026)1035 459 y Fg(R)1070 466 y Ff(k)1107 459 y Fm(=)g Fg(L)1191 466 y Ff(k)1212 459 y Fm(,)f(i.e.)25 b(if)17 b Fg(s)1389 442 y Ff(T)1389 471 y(i)1417 459 y Fg(y)1439 466 y Ff(j)1475 459 y Fm(=)g Fg(s)1548 442 y Ff(T)1548 471 y(j)1576 459 y Fg(y)1598 466 y Ff(i)1629 459 y Fm(for)f(all)0 515 y(0)d Fe(\024)h Fg(i;)8 b(j)14 b Fe(\024)g Fg(k)d Fe(\000)g Fm(1.)21 b(This)16 b(condition)h(is)f (true)f(for)g Fg(f)5 b Fm(\()p Fg(x)p Fm(\))15 b(quadratic,)h(and)f(in)i (this)f(case)f(the)h(m)o(ultiple)0 572 y(SR1)21 b(up)q(date)h(is)f(the)g (same)g(as)g Fg(k)h Fm(consecutiv)o(e,)h(standard)d(SR1)i(up)q(dates.)37 b(This)22 b(should)g(come)0 628 y(as)d(no)g(surprise,)i(b)q(ecause)f(the)g (quadratic)f(termination)h(result)g(for)e(the)i(standard)f(SR1)g(up)q(date)0 685 y(also)h(implies)i(that)d(the)h(up)q(date)h(preserv)o(es)e(all)i(past)f (secan)o(t)f(equations,)i(as)f(do)q(es)g(the)g(m)o(ultiple)0 741 y(secan)o(t)15 b(form)f(of)g(the)h(SR1.)20 b(Note)15 b(that)f(the)h (condition)h(for)e(the)h(equiv)m(alence)j(of)c(the)h(m)o(ultiple)i(SR1)0 798 y(to)f Fg(k)i Fm(consecutiv)o(e,)f(standard)f(SR1)h(up)q(dates)g(is)g (far)f(milder)i(than)f(the)f(assumption)h(of)f(conjugate)0 854 y(directions)d(required)f(for)f(the)h(equiv)m(alence)i(of)d Fg(k)i Fm(standard)e(BF)o(GS)g(up)q(dates)h(to)f(the)g(m)o(ultiple)j(BF)o(GS) 0 910 y(in)i(the)f(quadratic)h(case.)71 967 y(F)l(or)d(non-quadratic)i Fg(f)5 b Fm(\()p Fg(x)p Fm(\),)14 b(ho)o(w)o(ev)o(er,)f(the)i(standard)f(and) h(m)o(ultiple)h(SR1)f(up)q(dates)g(will)h(gener-)0 1023 y(ally)i(b)q(e)f (di\013eren)o(t.)26 b(Again,)17 b(the)g(algebraic)h(costs)e(asso)q(ciated)h (with)h(using)f(the)g(up)q(dates)h(are)e(v)o(ery)0 1080 y(similar,)e(while)f (the)g(m)o(ultiple)h(SR1)e(has)g(the)h(disadv)m(an)o(tage)f(that)g(it)g(do)q (es)h(not,)f(in)h(general,)g(preserv)o(e)0 1136 y(symmetry)l(,)i(while)i(a)e (sequence)i(of)e(standard)h(SR1)g(up)q(dates)g(do)q(es.)21 b(Also,)16 b(it)g(is)g(easier)g(to)f(monitor)0 1193 y(stabilit)o(y)j(of)f (the)g(standard)g(SR1,)h(since)g(this)g(only)f(in)o(v)o(olv)o(es)h (considering)h(eac)o(h)e(individual)j(term)0 1249 y(\()p Fg(y)40 1256 y Ff(j)68 1249 y Fe(\000)11 b Fg(B)148 1256 y Ff(j)167 1249 y Fg(s)188 1256 y Ff(j)206 1249 y Fm(\))224 1228 y Ff(T)251 1249 y Fg(s)272 1256 y Ff(j)309 1249 y Fm(rather)17 b(than)g(the)h(matrix)g (\()p Fg(Y)832 1256 y Ff(k)863 1249 y Fe(\000)11 b Fg(B)943 1256 y Fk(0)963 1249 y Fg(S)991 1256 y Ff(k)1012 1249 y Fm(\))1030 1228 y Ff(T)1057 1249 y Fg(S)1085 1256 y Ff(k)1106 1249 y Fm(.)28 b(F)l(or)17 b(this)h(reason,)g(a)f(sequence)i(of)0 1306 y(standard)c(SR1)g (up)q(dates)g(w)o(ould)h(seem)f(preferable)h(to)e(the)h(m)o(ultiple)i(SR1)e (up)q(date)h(in)g(the)f(con)o(text)0 1362 y(of)i(limited)j(memory)d(metho)q (ds.)28 b(It)18 b(is)h(in)o(teresting)f(to)f(note)h(that)f(if)h Fg(Y)1258 1369 y Ff(k)1298 1362 y Fm(is)g(p)q(erturb)q(ed)h(to)e(the)1707 1351 y(~)1700 1362 y Fg(Y)1726 1369 y Ff(k)0 1419 y Fm(that)c(one)g(obtains)h (b)o(y)f(m)o(ultiplying)j Fg(B)672 1426 y Ff(k)707 1419 y Fm(giv)o(en)e(in)h (\(5.2\))d(b)o(y)h Fg(S)1072 1426 y Ff(k)1093 1419 y Fm(,)h(then)f(the)h(m)o (ultiple)h(secan)o(t)e(up)q(date)0 1475 y(b)q(ecomes)21 b(iden)o(tical)h(to)e (\(5.2\).)34 b(The)20 b(same)g(relationship)i(is)f(not)f(true)g(for)g(the)g (m)o(ultiple)j(BF)o(GS)0 1531 y(up)q(date.)71 1588 y(Finally)18 b(w)o(e)f(consider)h(the)g(Bro)o(yden)f(up)q(date)h(for)e(nonlinear)j (equations.)26 b(A)17 b(m)o(ultiple)i(secan)o(t)0 1644 y(v)o(ersion)i(of)e (Bro)o(yden's)h(up)q(date)h(has)f(b)q(een)i(considered)g(b)o(y)e(sev)o(eral)g (authors)g(including)j(Barnes)0 1701 y(\(1965\),)16 b(Ga)o(y)g(and)h(Sc)o (hnab)q(el)i(\(1978\),)d(and)h(Sc)o(hnab)q(el)i(\(1983\).)24 b(In)18 b(a)f(limited)i(con)o(text)e(using)h(the)0 1757 y(notation)d(of)g (Section)h(6,)e(it)i(is)f(giv)o(en)h(b)o(y)517 1859 y Fg(A)551 1866 y Ff(k)585 1859 y Fm(=)d Fg(A)667 1866 y Fk(0)697 1859 y Fm(+)d(\()p Fg(Y)786 1866 y Ff(k)818 1859 y Fe(\000)g Fg(A)897 1866 y Fk(0)917 1859 y Fg(S)945 1866 y Ff(k)966 1859 y Fm(\)\()p Fg(S)1033 1841 y Ff(T)1030 1871 y(k)1059 1859 y Fg(S)1087 1866 y Ff(k)1109 1859 y Fm(\))1127 1841 y Fc(\000)p Fk(1)1173 1859 y Fg(S)1204 1841 y Ff(T)1201 1871 y(k)1655 1859 y Fm(\(7)p Fg(:)p Fm(6\))0 1962 y(This)17 b(up)q(date)h(is)f(w)o(ell)h(de\014ned)g(as)f (long)g(as)g Fg(S)803 1969 y Ff(k)841 1962 y Fm(has)f(full)j(column)e(rank,)g (and)g(ob)q(eys)g(the)g Fg(k)h Fm(secan)o(t)0 2018 y(equations)e Fg(A)238 2025 y Ff(k)259 2018 y Fg(S)287 2025 y Ff(k)321 2018 y Fm(=)d Fg(Y)395 2025 y Ff(k)416 2018 y Fm(.)71 2074 y(Comparing)k(\(7.6\))e (to)i(the)g(form)o(ula)g(\(6.4\))f(for)g Fg(k)i Fm(consecutiv)o(e,)h (standard)e(Bro)o(yden)g(up)q(dates,)0 2131 y(one)i(sees)g(that)f(the)h(only) h(di\013erence)g(is)f(in)h(the)f(matrix)f(in)i(the)f(middle)h(of)f(the)g (form)o(ula)f(that)g(is)0 2187 y(in)o(v)o(erted.)h(In)13 b(the)e(m)o(ultiple) j(secan)o(t)d(up)q(date)i(it)f(is)g Fg(S)894 2171 y Ff(T)891 2201 y(k)921 2187 y Fg(S)949 2194 y Ff(k)970 2187 y Fm(,)g(while)h(in)g (\(6.4\))d(it)i(is)g(the)g(upp)q(er)h(triangular)0 2244 y(p)q(ortion)j(of)f (this)h(matrix,)f(including)j(the)e(main)g(diagonal.)22 b(Therefore,)15 b(the)g(t)o(w)o(o)g(up)q(dates)h(are)f(the)0 2300 y(same)h(if)g(the)g (directions)h(in)g Fg(S)526 2307 y Ff(k)563 2300 y Fm(are)f(orthogonal.)21 b(The)16 b(preference)h(b)q(et)o(w)o(een)g(these)f(t)o(w)o(o)e(form)o(ulas)0 2357 y(do)q(es)d(not)f(app)q(ear)h(to)f(b)q(e)i(clear)f(cut.)18 b(The)11 b(form)o(ula)f(\(6.4\))f(has)i(the)g(adv)m(an)o(tage)f(that)g(it)h (is)g(w)o(ell)h(de\014ned)0 2413 y(for)i(an)o(y)f Fg(S)181 2420 y Ff(k)203 2413 y Fm(,)g(while)j(\(7.6\))d(is)h(only)h(w)o(ell)g (de\014ned)h(n)o(umerically)g(if)e(the)g Fg(k)i Fm(step)e(directions)h(that)e (mak)o(e)0 2470 y(up)f Fg(S)90 2477 y Ff(k)123 2470 y Fm(are)g(su\016cien)o (tly)h(linearly)h(indep)q(enden)o(t.)21 b(\(If)12 b(they)f(are)h(not,)g(only) g(some)g(subset)g(of)f(them)h(can)851 2603 y(27)p eop %%Page: 28 29 bop 0 120 a Fm(b)q(e)19 b(utilized)h(in)e(a)g(n)o(umerical)h(implemen)o (tation)g(of)f(the)g(m)o(ultiple)h(Bro)o(yden)f(metho)q(d;)h(this)g(is)f(the) 0 177 y(approac)o(h)f(that)g(has)g(b)q(een)i(tak)o(en)e(in)h(implemen)o (tations)h(of)e(this)g(up)q(date.\))27 b(On)18 b(the)g(other)f(hand,)0 233 y(\(7.6\))i(alw)o(a)o(ys)i(enforces)g(the)g Fg(k)h Fm(prior)g(secan)o(t)e (equations)i(while)g(\(6.4\))e(generally)i(only)g(enforces)0 289 y(the)g(most)g(recen)o(t)g(equation.)41 b(Th)o(us)22 b(it)g(w)o(ould)h (probably)f(b)q(e)h(w)o(orth)o(while)g(considering)g(either)0 346 y(metho)q(d)13 b(\(or)g(their)h(in)o(v)o(erse)f(form)o(ulations\))g(in)h (a)f(limited)i(memory)e(metho)q(d)g(for)g(solving)h(nonlinear)0 402 y(equations.)42 b(Note)22 b(that)g(the)g(k)o(ey)h(di\013erence)g(b)q(et)o (w)o(een)g(this)g(comparison)f(and)h(the)g(preceding)0 459 y(comparisons)18 b(of)f(the)h(BF)o(GS)f(and)h(SR1)h(based)f(form)o(ulae)f(is) i(that)e(symmetry)l(,)g(whic)o(h)i(in)f(general)0 515 y(is)e(inconsisten)o(t) h(with)f(satisfying)f(m)o(ultiple)j(secan)o(t)d(equations,)h(is)g(not)f(a)g (factor)g(in)h(the)g(nonlinear)0 572 y(equations)g(case)f(but)g(is)h(a)f (factor)f(for)g(up)q(dates)i(for)f(optimization)h(problems.)0 728 y Fl(A)n(cknow)r(le)n(dgement.)35 b Fm(W)l(e)21 b(w)o(ould)g(lik)o(e)h (to)e(thank)g(P)o(eih)o(uang)i(Lu)f(for)f(considerable)i(help)g(in)g(the)0 784 y(preparation)15 b(of)g(this)h(pap)q(er.)851 2603 y(28)p eop %%Page: 29 30 bop 0 120 a Fn(REFERENCES)0 212 y Fm(J.)16 b(Barnes,)g(\\An)g(algorithm)g (for)g(solving)h(nonlinear)g(equations)g(based)f(on)g(the)h(secan)o(t)e (metho)q(d,")0 259 y Fl(Computer)i(Journal)e Fm(8)g(\(1965\))e(66{67.)0 342 y(L.)19 b(Biegler,)h(J.)f(No)q(cedal)h(and)f(C.)f(Sc)o(hmid,)j(\\Reduced) f(Hessian)f(metho)q(ds)g(for)f(large)h(scale)h(con-)0 387 y(strained)k (optimization,")i(T)l(ec)o(hnical)g(Rep)q(ort,)g(Departmen)o(t)d(of)g (Electrical)i(Engineering)h(and)0 434 y(Computer)15 b(Science,)h(North)o(w)o (estern)f(Univ)o(ersit)o(y)g(\(Ev)m(anston,)g(IL,)g(1993\).)0 516 y(C.G.)d(Bro)o(yden,)h(\\A)g(class)h(of)f(metho)q(ds)g(for)g(solving)h (nonlinear)h(sim)o(ultaneous)f(equations,")f Fl(Math-)0 563 y(ematics)j(of)h(Computation)e Fm(19)g(\(1965\))e(577{593.)0 646 y(A.)i(Buc)o(kley)h(and)g(A.)f(LeNir,)h(\\QN-lik)o(e)h(v)m(ariable)f (storage)e(conjugate)h(gradien)o(ts",)g Fl(Mathematic)n(al)0 693 y(Pr)n(o)n(gr)n(amming)f Fm(27,)h(\(1983\))e(103-119.)0 775 y(R.H.)j(Byrd,)h(P)l(.)f(Lu)h(and)g(J.)f(No)q(cedal,)h(\\A)g(limited)h (memory)e(algorithm)g(for)g(b)q(ound)h(constrained)0 820 y(optimization,")j (T)l(ec)o(hnical)h(Rep)q(ort,)f(Departmen)o(t)e(of)h(Electrical)i (Engineering)f(and)f(Computer)0 867 y(Science,)e(North)o(w)o(estern)d(Univ)o (ersit)o(y)i(\(Ev)m(anston,)e(IL,)i(1993\).)0 949 y(A.R.)e(Conn,)g(N.I.M.)g (Gould,)g(and)h(Ph.L.)f(T)l(oin)o(t,)g(\\T)l(esting)g(a)g(class)h(of)f(metho) q(ds)g(for)g(solving)h(min-)0 994 y(imization)i(problems)g(with)f(simple)h(b) q(ounds)g(on)f(the)g(v)m(ariables,")g Fl(Mathematics)h(of)g(Computation)0 1041 y Fm(50/182)d(\(1988\))f(399{430.)0 1124 y(J.E.)k(Dennis)h(Jr.)27 b(and)18 b(R.B.)f(Sc)o(hnab)q(el,)j Fl(Numeric)n(al)e(metho)n(ds)g(for)h(unc) n(onstr)n(aine)n(d)e(optimization)0 1171 y(and)f(nonline)n(ar)f(e)n(quations) g Fm(\(Pren)o(tice-Hall,)h(1983\).)0 1256 y(R.)h(Fletc)o(her,)f Fl(Pr)n(actic)n(al)h(Metho)n(ds)g(of)h(Optimization)e Fm(\(second)h (edition\))g(\(John)g(Wiley)h(and)e(Sons,)0 1305 y(Chic)o(hester,)f(1987\).)0 1388 y(D.M.)20 b(Ga)o(y)g(and)i(R.B.)f(Sc)o(hnab)q(el,)j(\\Solving)f(systems) e(of)g(nonlinear)h(equations)g(b)o(y)f(Bro)o(yden's)0 1432 y(metho)q(d)14 b(with)g(pro)s(jected)g(up)q(dates,")g(in:)20 b(O.)13 b(L.)h(Mangasarian,)f(R.)h(R.)g(Mey)o(er,)f(and)h(S.)g(M.)f(Robin-)0 1479 y(son,)i(eds.,)f Fl(Nonline)n(ar)h(Pr)n(o)n(gr)n(amming)g Fm(3)g(\(Academic)h(Press,)e(New)i(Y)l(ork,)e(1978\))g(245{281.)0 1562 y(J.C.)21 b(Gilb)q(ert)i(and)f(C.)f(Lemar)o(\023)-21 b(ec)o(hal,)23 b(\\Some)e(n)o(umerical)i(exp)q(erimen)o(ts)g(with)f(v)m(ariable)h(storage)0 1609 y(quasi-Newton)16 b(algorithms,")e Fl(Mathematic)n(al)i(Pr)n(o)n(gr)n (amming)f Fm(45)g(\(1989\))e(407{436.)0 1692 y(J.C.)21 b(Gilb)q(ert)i(and)e (J.)h(No)q(cedal,)i(\\The)d(limited)j(memory)d(step)g(computation)h(and)g (automatic)0 1736 y(di\013eren)o(tiation,")e(T)l(ec)o(hnical)g(Rep)q(ort)g (NAM)e(02,)h(Departmen)o(t)f(of)g(Electrical)i(Engineering)h(and)0 1783 y(Computer)16 b(Science,)j(North)o(w)o(estern)c(Univ)o(ersit)o(y)j(\(Ev) m(anston,)e(IL,)h(1991\),)e(to)h(app)q(ear)h(in)g Fl(Applie)n(d)0 1830 y(Math)g(L)n(etters)p Fm(.)0 1899 y(A.)c(Griew)o(ank,)g(\\On)h (automatic)f(di\013eren)o(tiation,")h(in:)19 b(M.)13 b(Iri)h(and)g(K.)f(T)l (anab)q(e,)h(eds.,)f Fl(Mathemat-)0 1946 y(ic)n(al)j(Pr)n(o)n(gr)n(amming)e Fm(\(Klu)o(w)o(er)h(Academic)i(Publishers,)f(T)l(oky)o(o,)e(1989\))g(83{107.) 0 2029 y(H.)19 b(F)l(a)o(y)o(ez)g(Khalfan,)i(\\T)l(opics)f(in)g(quasi-Newton) g(metho)q(ds)f(for)g(unconstrained)i(optimization,")0 2076 y(Ph.D.)14 b(thesis,)i(Departmen)o(t)e(of)h(Mathematics,)f(Univ)o(ersit)o(y)i (of)f(Colorado)f(\(Boulder,)i(CO,)f(1989\).)0 2158 y(H.)e(F)l(a)o(y)o(ez)f (Khalfan,)i(R.H.)e(Byrd,)i(and)f(R.B.)f(Sc)o(hnab)q(el,)j(\\A)e(theoretical)h (and)f(exp)q(erimen)o(tal)h(study)0 2205 y(of)h(the)g(symmetric)g(rank)g(one) h(up)q(date,")f Fl(SIAM)g(Journal)h(on)g(Optimization)f Fm(3)g(\(1993\))e (1{24.)0 2288 y(D.C.)k(Liu)i(and)f(J.)g(No)q(cedal,)i(\\On)e(the)g(limited)i (memory)e(BF)o(GS)f(metho)q(d)h(for)g(large)g(scale)h(opti-)0 2335 y(mization,")c Fl(Mathematic)n(al)i(Pr)n(o)n(gr)n(amming)d Fm(45)h(\(1989\))e(503{528.)0 2417 y(D.Q.)d(Mahidhara)g(and)h(L.)f(Lasdon,)h (\\An)g(SQP)f(algorithm)h(for)e(large)i(sparse)f(nonlinear)i(programs,")0 2462 y(T)l(ec)o(hnical)21 b(rep)q(ort,)f(MSIS)g(Departmen)o(t,)f(Sc)o(ho)q (ol)h(of)f(Business)h(Administration,)h(Univ)o(ersit)o(y)f(of)851 2603 y(29)p eop %%Page: 30 31 bop 0 120 a Fm(T)l(exas)15 b(\(Austin,)g(TX,)g(1990\).)0 203 y(H.)g(Matthies)g(and)g(G.)g(Strang,)f(\\The)h(Solution)h(of)f(nonlinear)i (\014nite)f(elemen)o(t)g(equations,")f Fl(Inter-)0 250 y(national)h(Journal)g (of)g(Numeric)n(al)h(Metho)n(ds)f(in)f(Engine)n(ering)e Fm(14)i(\(1979\))e (1613{1626.)0 332 y(J.)22 b(No)q(cedal,)i(\\Up)q(dating)e(quasi-Newton)g (matrices)g(with)g(limited)i(storage,")d Fl(Mathematics)i(of)0 379 y(Computation)16 b Fm(35)e(\(1980\))g(773{782.)0 462 y(J.M.)h(Ortega)g (and)g(W.C.)f(Rhein)o(b)q(oldt,)k Fl(Iter)n(ative)e(Solution)g(of)h(Nonline)n (ar)e(Equations)h(in)g(Sever)n(al)0 509 y(V)m(ariables)e Fm(\(Academic)i (Press,)f(1970\).)0 591 y(R.B.)f(Sc)o(hnab)q(el,)h(\\Quasi-Newton)g(metho)q (ds)f(using)h(m)o(ultiple)h(secan)o(t)d(equations,")h(T)l(ec)o(hnical)i(Re-)0 638 y(p)q(ort)e(CU-CS-247-83,)f(Departmen)o(t)h(of)g(Computer)g(Science,)i (Univ)o(ersit)o(y)f(of)f(Colorado)g(\(Boulder,)0 688 y(CO,)h(1983\).)0 771 y(H.F.)j(W)l(alk)o(er,)i(\\Implemen)o(tation)h(of)e(the)g(GMRES)g(metho)q (d)h(using)g(Householder)g(transforma-)0 818 y(tions",)15 b Fl(SIAM)g(Journal)h(on)g(Scienti\014c)f(and)h(Statistic)n(al)f(Computing)g Fm(9/1)g(\(1988\))e(152{163.)851 2603 y(30)p eop %%Trailer end userdict /end-hook known{end-hook}if %%EOF SHAR_EOF fi # end of overwriting check cd .. if test ! -d 'Drivers' then mkdir 'Drivers' fi cd 'Drivers' if test ! -d 'Dp' then mkdir 'Dp' fi cd 'Dp' if test -f 'driver1.f' then echo shar: will not over-write existing file "'driver1.f'" else cat << \SHAR_EOF > 'driver1.f' c DRIVER 1 c -------------------------------------------------------------- c SIMPLE DRIVER FOR L-BFGS-B (version 2.3) c -------------------------------------------------------------- c c L-BFGS-B is a code for solving large nonlinear optimization c problems with simple bounds on the variables. c c The code can also be used for unconstrained problems and is c as efficient for these problems as the earlier limited memory c code L-BFGS. c c This is the simplest driver in the package. It uses all the c default settings of the code. c c c References: c c [1] R. H. Byrd, P. Lu, J. Nocedal and C. Zhu, ``A limited c memory algorithm for bound constrained optimization'', c SIAM J. Scientific Computing 16 (1995), no. 5, pp. 1190--1208. c c [2] C. Zhu, R.H. Byrd, P. Lu, J. Nocedal, ``L-BFGS-B: FORTRAN c Subroutines for Large Scale Bound Constrained Optimization'' c Tech. Report, NAM-11, EECS Department, Northwestern University, c 1994. c c (Postscript files of these papers are available via anonymous c ftp to ece.nwu.edu in the directory pub/lbfgs/lbfgs_bcm.) c c * * * c c NEOS, November 1994. (Latest revision April 1997.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c NOTE: The user should adapt the subroutine 'timer' if 'etime' is c not available on the system. An example for system c AIX Version 3.2 is available at the end of this driver. c c ************** program driver c This simple driver demonstrates how to call the L-BFGS-B code to c solve a sample problem (the extended Rosenbrock function c subject to bounds on the variables). The dimension n of this c problem is variable. integer nmax, mmax, lenwa parameter (nmax = 1024, mmax = 17) parameter (lenwa = 2*mmax*nmax + 4*nmax + + 11*mmax*mmax + 8*mmax) c nmax is the dimension of the largest problem to be solved. c mmax is the maximum number of limited memory corrections. c lenwa is the corresponding real workspace required. c Declare the variables needed by the code. c A description of all these variables is given at the end of c the driver. character*60 task, csave logical lsave(4) integer n, m, iprint, + nbd(nmax), iwa(3*nmax), isave(44) double precision f, factr, pgtol, + x(nmax), l(nmax), u(nmax), g(nmax), dsave(29), + wa(lenwa) c Declare a few additional variables for this sample problem. double precision t1, t2 integer i c We wish to have output at every iteration. iprint = 1 c We specify the tolerances in the stopping criteria. factr = 1.0d+7 pgtol = 1.0d-5 c We specify the dimension n of the sample problem and the number c m of limited memory corrections stored. (n and m should not c exceed the limits nmax and mmax respectively.) n = 25 m = 5 c We now provide nbd which defines the bounds on the variables: c l specifies the lower bounds, c u specifies the upper bounds. c First set bounds on the odd-numbered variables. do 10 i = 1, n, 2 nbd(i) = 2 l(i) = 1.0d0 u(i) = 1.0d2 10 continue c Next set bounds on the even-numbered variables. do 12 i = 2, n, 2 nbd(i) = 2 l(i) =-1.0d2 u(i) = 1.0d2 12 continue c We now define the starting point. do 14 i = 1, n x(i) = 3.0d0 14 continue c We now write the heading of the output. write (6,16) 16 format(/ 5x, 'Solving sample problem.' + / 5x, ' (f = 0.0 at the optimal solution.)' /) c We start the iteration by initializing task. task = 'START' c ------- The beginning of the loop ---------- 111 continue c This is the call to the L-BFGS-B code. call setulb(n,m,x,l,u,nbd,f,g,factr,pgtol,wa,iwa,task,iprint, + csave,lsave,isave,dsave) if (task(1:2) .eq. 'FG') then c The minimization routine has returned to request the c function f and gradient g values at the current x. c Compute function value f for the sample problem. f = .25d0*(x(1) - 1.d0)**2 do 20 i = 2, n f = f + (x(i) - x(i-1)**2)**2 20 continue f = 4.d0*f c Compute gradient g for the sample problem. t1 = x(2) - x(1)**2 g(1) = 2.d0*(x(1) - 1.d0) - 1.6d1*x(1)*t1 do 22 i = 2, n-1 t2 = t1 t1 = x(i+1) - x(i)**2 g(i) = 8.d0*t2 - 1.6d1*x(i)*t1 22 continue g(n) = 8.d0*t1 c Go back to the minimization routine. goto 111 elseif (task(1:5) .eq. 'NEW_X') then c The minimization routine has returned with a new iterate, c and we have opted to continue the iteration. goto 111 else c We terminate execution when task is neither FG nor NEW_X. c We print the information contained in the string task c if the default output is not used and the execution is c not stopped intentionally by the user. if (iprint .le. -1 .and. task(1:4) .ne. 'STOP') write(6,*) task endif c ---------- The end of the loop ------------- stop end c======================= The end of driver1 ============================ c -------------------------------------------------------------- c DESCRIPTION OF THE VARIABLES IN L-BFGS-B c -------------------------------------------------------------- c c n is an INTEGER variable that must be set by the user to the c number of variables. It is not altered by the routine. c c m is an INTEGER variable that must be set by the user to the c number of corrections used in the limited memory matrix. c It is not altered by the routine. Values of m < 3 are c not recommended, and large values of m can result in excessive c computing time. The range 3 <= m <= 20 is recommended. c c x is a DOUBLE PRECISION array of length n. On initial entry c it must be set by the user to the values of the initial c estimate of the solution vector. Upon successful exit, it c contains the values of the variables at the best point c found (usually an approximate solution). c c l is a DOUBLE PRECISION array of length n that must be set by c the user to the values of the lower bounds on the variables. If c the i-th variable has no lower bound, l(i) need not be defined. c c u is a DOUBLE PRECISION array of length n that must be set by c the user to the values of the upper bounds on the variables. If c the i-th variable has no upper bound, u(i) need not be defined. c c nbd is an INTEGER array of dimension n that must be set by the c user to the type of bounds imposed on the variables: c nbd(i)=0 if x(i) is unbounded, c 1 if x(i) has only a lower bound, c 2 if x(i) has both lower and upper bounds, c 3 if x(i) has only an upper bound. c c f is a DOUBLE PRECISION variable. If the routine setulb returns c with task(1:2)= 'FG', then f must be set by the user to c contain the value of the function at the point x. c c g is a DOUBLE PRECISION array of length n. If the routine setulb c returns with taskb(1:2)= 'FG', then g must be set by the user to c contain the components of the gradient at the point x. c c factr is a DOUBLE PRECISION variable that must be set by the user. c It is a tolerance in the termination test for the algorithm. c The iteration will stop when c c (f^k - f^{k+1})/max{|f^k|,|f^{k+1}|,1} <= factr*epsmch c c where epsmch is the machine precision which is automatically c generated by the code. Typical values for factr on a computer c with 15 digits of accuracy in double precision are: c factr=1.d+12 for low accuracy; c 1.d+7 for moderate accuracy; c 1.d+1 for extremely high accuracy. c The user can suppress this termination test by setting factr=0. c c pgtol is a double precision variable. c On entry pgtol >= 0 is specified by the user. The iteration c will stop when c c max{|proj g_i | i = 1, ..., n} <= pgtol c c where pg_i is the ith component of the projected gradient. c The user can suppress this termination test by setting pgtol=0. c c wa is a DOUBLE PRECISION array of length c (2mmax + 4)nmax + 11mmax^2 + 8mmax used as workspace. c This array must not be altered by the user. c c iwa is an INTEGER array of length 3nmax used as c workspace. This array must not be altered by the user. c c task is a CHARACTER string of length 60. c On first entry, it must be set to 'START'. c On a return with task(1:2)='FG', the user must evaluate the c function f and gradient g at the returned value of x. c On a return with task(1:5)='NEW_X', an iteration of the c algorithm has concluded, and f and g contain f(x) and g(x) c respectively. The user can decide whether to continue or stop c the iteration. c When c task(1:4)='CONV', the termination test in L-BFGS-B has been c satisfied; c task(1:4)='ABNO', the routine has terminated abnormally c without being able to satisfy the termination conditions, c x contains the best approximation found, c f and g contain f(x) and g(x) respectively; c task(1:5)='ERROR', the routine has detected an error in the c input parameters; c On exit with task = 'CONV', 'ABNO' or 'ERROR', the variable task c contains additional information that the user can print. c This array should not be altered unless the user wants to c stop the run for some reason. See driver2 or driver3 c for a detailed explanation on how to stop the run c by assigning task(1:4)='STOP' in the driver. c c iprint is an INTEGER variable that must be set by the user. c It controls the frequency and type of output generated: c iprint<0 no output is generated; c iprint=0 print only one line at the last iteration; c 0100 print details of every iteration including x and g; c When iprint > 0, the file iterate.dat will be created to c summarize the iteration. c c csave is a CHARACTER working array of length 60. c c lsave is a LOGICAL working array of dimension 4. c On exit with task = 'NEW_X', the following information is c available: c lsave(1) = .true. the initial x did not satisfy the bounds; c lsave(2) = .true. the problem contains bounds; c lsave(3) = .true. each variable has upper and lower bounds. c c isave is an INTEGER working array of dimension 44. c On exit with task = 'NEW_X', it contains information that c the user may want to access: c isave(30) = the current iteration number; c isave(34) = the total number of function and gradient c evaluations; c isave(36) = the number of function value or gradient c evaluations in the current iteration; c isave(38) = the number of free variables in the current c iteration; c isave(39) = the number of active constraints at the current c iteration; c c See the subroutine setulb.f for a description of other c information contained in isave. c c dsave is a DOUBLE PRECISION working array of dimension 29. c On exit with task = 'NEW_X', it contains information that c the user may want to access: c dsave(2) = the value of f at the previous iteration; c dsave(5) = the machine precision epsmch generated by the code; c dsave(13) = the infinity norm of the projected gradient; c c See the subroutine setulb.f for a description of other c information contained in dsave. c c -------------------------------------------------------------- c END OF THE DESCRIPTION OF THE VARIABLES IN L-BFGS-B c -------------------------------------------------------------- c c << An example of subroutine 'timer' for AIX Version 3.2 >> c c subroutine timer(ttime) c double precision ttime c integer itemp, integer mclock c c itemp = mclock() c ttime = dble(itemp)*1.0d-2 c return c end c----------------------------------------------------------------------- SHAR_EOF fi # end of overwriting check if test -f 'driver2.f' then echo shar: will not over-write existing file "'driver2.f'" else cat << \SHAR_EOF > 'driver2.f' c DRIVER 2 c -------------------------------------------------------------- c CUSTOMIZED DRIVER FOR L-BFGS-B (version 2.3) c -------------------------------------------------------------- c c L-BFGS-B is a code for solving large nonlinear optimization c problems with simple bounds on the variables. c c The code can also be used for unconstrained problems and is c as efficient for these problems as the earlier limited memory c code L-BFGS. c c This driver illustrates how to control the termination of the c run and how to design customized output. c c References: c c [1] R. H. Byrd, P. Lu, J. Nocedal and C. Zhu, ``A limited c memory algorithm for bound constrained optimization'', c SIAM J. Scientific Computing 16 (1995), no. 5, pp. 1190--1208. c c [2] C. Zhu, R.H. Byrd, P. Lu, J. Nocedal, ``L-BFGS-B: FORTRAN c Subroutines for Large Scale Bound Constrained Optimization'' c Tech. Report, NAM-11, EECS Department, Northwestern University, c 1994. c c (Postscript files of these papers are available via anonymous c ftp to ece.nwu.edu in the directory pub/lbfgs/lbfgs_bcm.) c c * * * c c NEOS, November 1994. (Latest revision April 1997.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************** program driver c This driver shows how to replace the default stopping test c by other termination criteria. It also illustrates how to c print the values of several parameters during the course of c the iteration. The sample problem used here is the same as in c DRIVER1 (the extended Rosenbrock function with bounds on the c variables). integer nmax, mmax, lenwa parameter (nmax = 1024, mmax = 17) parameter (lenwa = 2*mmax*nmax + 4*nmax + + 11*mmax*mmax + 8*mmax) c nmax is the dimension of the largest problem to be solved. c mmax is the maximum number of limited memory corrections. c lenwa is the corresponding real workspace required. c Declare the variables needed by the code. c A description of all these variables is given at the end of c driver1. character*60 task, csave logical lsave(4) integer n, m, iprint, + nbd(nmax), iwa(3*nmax), isave(44) double precision f, factr, pgtol, + x(nmax), l(nmax), u(nmax), g(nmax), dsave(29), + wa(lenwa) c Declare a few additional variables for the sample problem. double precision t1, t2 integer i c We suppress the default output. iprint = -1 c We suppress both code-supplied stopping tests because the c user is providing his own stopping criteria. factr = 0.0d0 pgtol = 0.0d0 c We specify the dimension n of the sample problem and the number c m of limited memory corrections stored. (n and m should not c exceed the limits nmax and mmax respectively.) n = 25 m = 5 c We now specify nbd which defines the bounds on the variables: c l specifies the lower bounds, c u specifies the upper bounds. c First set bounds on the odd numbered variables. do 10 i = 1, n, 2 nbd(i) = 2 l(i) = 1.0d0 u(i) = 1.0d2 10 continue c Next set bounds on the even numbered variables. do 12 i = 2, n, 2 nbd(i) = 2 l(i) =-1.0d2 u(i) = 1.0d2 12 continue c We now define the starting point. do 14 i = 1, n x(i) = 3.0d0 14 continue c We now write the heading of the output. write (6,16) 16 format(/ 5x, 'Solving sample problem.' + / 5x, ' (f = 0.0 at the optimal solution.)' /) c We start the iteration by initializing task. task = 'START' c ------- The beginning of the loop ---------- 111 continue c This is the call to the L-BFGS-B code. call setulb(n,m,x,l,u,nbd,f,g,factr,pgtol,wa,iwa,task,iprint, + csave,lsave,isave,dsave) if (task(1:2) .eq. 'FG') then c The minimization routine has returned to request the c function f and gradient g values at the current x. c Compute function value f for the sample problem. f = .25d0*(x(1) - 1.d0)**2 do 20 i = 2, n f = f + (x(i) - x(i-1)**2)**2 20 continue f = 4.d0*f c Compute gradient g for the sample problem. t1 = x(2) - x(1)**2 g(1) = 2.d0*(x(1) - 1.d0) - 1.6d1*x(1)*t1 do 22 i = 2, n-1 t2 = t1 t1 = x(i+1) - x(i)**2 g(i) = 8.d0*t2 - 1.6d1*x(i)*t1 22 continue g(n) = 8.d0*t1 c Go back to the minimization routine. goto 111 elseif (task(1:5) .eq. 'NEW_X') then c c The minimization routine has returned with a new iterate. c At this point have the opportunity of stopping the iteration c or observing the values of certain parameters c c First are two examples of stopping tests. c Note: task(1:4) must be assigned the value 'STOP' to terminate c the iteration and ensure that the final results are c printed in the default format. The rest of the character c string task may be used to store other information. c 1) Terminate if the total number of f and g evaluations c exceeds 99. if (isave(34) .ge. 99) + task='STOP: TOTAL NO. of f AND g EVALUATIONS EXCEEDS LIMIT' c 2) Terminate if |proj g|/(1 + |f|) < 1.0d-10, where c "proj g" denoted the projected gradient if (dsave(13) .le. 1.d-10*(1.0d0 + abs(f))) + task='STOP: THE PROJECTED GRADIENT IS SUFFICIENTLY SMALL' c We now wish to print the following information at each c iteration: c c 1) the current iteration number, isave(30), c 2) the total number of f and g evaluations, isave(34), c 3) the value of the objective function f, c 4) the norm of the projected gradient, dsve(13) c c See the comments at the end of driver1 for a description c of the variables isave and dsave. write (6,'(2(a,i5,4x),a,1p,d12.5,4x,a,1p,d12.5)') 'Iterate' + ,isave(30),'nfg =',isave(34),'f =',f,'|proj g| =',dsave(13) c If the run is to be terminated, we print also the information c contained in task as well as the final value of x. if (task(1:4) .eq. 'STOP') then write (6,*) task write (6,*) 'Final X=' write (6,'((1x,1p, 6(1x,d11.4)))') (x(i), i = 1, n) endif c Go back to the minimization routine. goto 111 else c We terminate execution when task is neither FG nor NEW_X. c We print the information contained in the string task c if the default output is not used and the execution is c not stopped intentionally by the user. if (iprint .le. -1 .and. task(1:4) .ne. 'STOP') write(6,*) task endif c ---------- the end of the loop ------------- stop end c======================= The end of driver2 ============================ SHAR_EOF fi # end of overwriting check if test -f 'driver3.f' then echo shar: will not over-write existing file "'driver3.f'" else cat << \SHAR_EOF > 'driver3.f' c DRIVER 3 c -------------------------------------------------------------- c TIME-CONTROLLED DRIVER FOR L-BFGS-B (version 2.3) c -------------------------------------------------------------- c c L-BFGS-B is a code for solving large nonlinear optimization c problems with simple bounds on the variables. c c The code can also be used for unconstrained problems and is c as efficient for these problems as the earlier limited memory c code L-BFGS. c c This driver shows how to terminate a run after some prescribed c CPU time has elapsed, and how to print the desired information c before exiting. c c References: c c [1] R. H. Byrd, P. Lu, J. Nocedal and C. Zhu, ``A limited c memory algorithm for bound constrained optimization'', c SIAM J. Scientific Computing 16 (1995), no. 5, pp. 1190--1208. c c [2] C. Zhu, R.H. Byrd, P. Lu, J. Nocedal, ``L-BFGS-B: FORTRAN c Subroutines for Large Scale Bound Constrained Optimization'' c Tech. Report, NAM-11, EECS Department, Northwestern University, c 1994. c c (Postscript files of these papers are available via anonymous c ftp to ece.nwu.edu in the directory pub/lbfgs/lbfgs_bcm.) c c * * * c c NEOS, November 1994. (Latest revision April 1997.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************** program driver c This time-controlled driver shows that it is possible to terminate c a run by elapsed CPU time, and yet be able to print all desired c information. This driver also illustrates the use of two c stopping criteria that may be used in conjunction with a limit c on execution time. The sample problem used here is the same as in c driver1 and driver2 (the extended Rosenbrock function with bounds c on the variables). integer nmax, mmax, lenwa parameter (nmax = 1024, mmax = 17) parameter (lenwa = 2*mmax*nmax + 4*nmax + + 11*mmax*mmax + 8*mmax) c nmax is the dimension of the largest problem to be solved. c mmax is the maximum number of limited memory corrections. c lenwa is the corresponding real workspace required. c Declare the variables needed by the code. c A description of all these variables is given at the end of c driver1. character*60 task, csave logical lsave(4) integer n, m, iprint, + nbd(nmax), iwa(3*nmax), isave(44) double precision f, factr, pgtol, + x(nmax), l(nmax), u(nmax), g(nmax), dsave(29), + wa(lenwa) c Declare a few additional variables for the sample problem c and for keeping track of time. double precision t1, t2, time1, time2, tlimit integer i, j c We specify a limite on the CPU time (in seconds). tlimit = 0.2 c We suppress the default output. (The user could also elect to c use the default output by choosing iprint >= 0.) iprint = -1 c We suppress the code-supplied stopping tests because we will c provide our own termination conditions factr = 0.0d0 pgtol = 0.0d0 c We specify the dimension n of the sample problem and the number c m of limited memory corrections stored. (n and m should not c exceed the limits nmax and mmax respectively.) n = 1000 m = 10 c We now specify nbd which defines the bounds on the variables: c l specifies the lower bounds, c u specifies the upper bounds. c First set bounds on the odd-numbered variables. do 10 i = 1, n, 2 nbd(i) = 2 l(i) = 1.0d0 u(i) = 1.0d2 10 continue c Next set bounds on the even-numbered variables. do 12 i = 2, n, 2 nbd(i) = 2 l(i) =-1.0d2 u(i) = 1.0d2 12 continue c We now define the starting point. do 14 i = 1, n x(i) = 3.0d0 14 continue c We now write the heading of the output. write (6,16) 16 format(/ 5x, 'Solving sample problem.' + / 5x, ' (f = 0.0 at the optimal solution.)' /) c We start the iteration by initializing task. task = 'START' c ------- The beginning of the loop ---------- c We begin counting the CPU time. call timer(time1) 111 continue c This is the call to the L-BFGS-B code. call setulb(n,m,x,l,u,nbd,f,g,factr,pgtol,wa,iwa,task,iprint, + csave,lsave,isave,dsave) if (task(1:2) .eq. 'FG') then c The minimization routine has returned to request the c function f and gradient g values at the current x. c Before evaluating f and g we check the CPU time spent. call timer(time2) if (time2 - time1 .gt. tlimit) then task='STOP: CPU EXCEEDING THE TIME LIMIT.' c Note: Assigning task(1:4)='STOP' will terminate the run; c setting task(7:9)='CPU' will restore the information at c the latest iterate generated by the code so that it can c be correctly printed in the default format. c In this driver we have chosen to disable the c printing options of the code (we set iprint=-1); c instead we are using customized output: we print the c latest value of x, the corresponding function value f and c the norm of the projected gradient |proj g|. c We print out the information contained in task. write (6,*) task c We print the latest iterate contained in wa(j+1:j+n), where j = 3*n + 2*m*n + 11*m**2 write (6,*) 'Latest iterate X =' write (6,'((1x,1p, 6(1x,d11.4)))') (wa(i), i = j + 1, j + n) c We print the function value f and the norm of the projected c gradient |proj g| at the last iterate; they are stored in c dsave(2) and dsave(13) respectively. write (6,'(a,1p,d12.5,4x,a,1p,d12.5)') + 'At latest iterate f =',dsave(2),'|proj g| =',dsave(13) else c The time limit has not been reached and we compute c the function value f for the sample problem. f = .25d0*(x(1) - 1.d0)**2 do 20 i = 2, n f = f + (x(i) - x(i-1)**2)**2 20 continue f = 4.d0*f c Compute gradient g for the sample problem. t1 = x(2) - x(1)**2 g(1) = 2.d0*(x(1) - 1.d0) - 1.6d1*x(1)*t1 do 22 i = 2, n-1 t2 = t1 t1 = x(i+1) - x(i)**2 g(i) = 8.d0*t2 - 1.6d1*x(i)*t1 22 continue g(n) = 8.d0*t1 endif c Go back to the minimization routine. goto 111 elseif (task(1:5) .eq. 'NEW_X') then c The minimization routine has returned with a new iterate. c The time limit has not been reached, and we test whether c the following two stopping tests are satisfied: c 1) Terminate if the total number of f and g evaluations c exceeds 900. if (isave(34) .ge. 900) + task='STOP: TOTAL NO. of f AND g EVALUATIONS EXCEEDS LIMIT' c 2) Terminate if |proj g|/(1 + |f|) < 1.0d-10. if (dsave(13) .le. 1.d-10*(1.0d0 + abs(f))) + task='STOP: THE PROJECTED GRADIENT IS SUFFICIENTLY SMALL' c We wish to print the following information at each iteration: c 1) the current iteration number, isave(30), c 2) the total number of f and g evaluations, isave(34), c 3) the value of the objective function f, c 4) the norm of the projected gradient, dsve(13) c c See the comments at the end of driver1 for a description c of the variables isave and dsave. write (6,'(2(a,i5,4x),a,1p,d12.5,4x,a,1p,d12.5)') 'Iterate' + ,isave(30),'nfg =',isave(34),'f =',f,'|proj g| =',dsave(13) c If the run is to be terminated, we print also the information c contained in task as well as the final value of x. if (task(1:4) .eq. 'STOP') then write (6,*) task write (6,*) 'Final X=' write (6,'((1x,1p, 6(1x,d11.4)))') (x(i), i = 1, n) endif c Go back to the minimization routine. goto 111 else c We terminate execution when task is neither FG nor NEW_X. c We print the information contained in the string task c if the default output is not used and the execution is c not stopped intentionally by the user. if (iprint .le. -1 .and. task(1:4) .ne. 'STOP') write(6,*) task endif c ---------- the end of the loop ------------- stop end c======================= The end of driver3 ============================ SHAR_EOF fi # end of overwriting check if test -f 'RES1' then echo shar: will not over-write existing file "'RES1'" else cat << \SHAR_EOF > 'RES1' Solving sample problem. (f = 0.0 at the optimal solution.) RUNNING THE L-BFGS-B CODE * * * Machine precision = 2.220D-16 N = 25 M = 5 At X0 0 variables are exactly at the bounds At iterate 0 f= 3.46000D+03 |proj g|= 1.03000D+02 At iterate 1 f= 2.39769D+03 |proj g|= 6.50700D+01 At iterate 2 f= 1.43806D+02 |proj g|= 3.64039D+01 At iterate 3 f= 7.28161D+01 |proj g|= 2.29042D+01 At iterate 4 f= 1.60308D+01 |proj g|= 6.95409D+00 At iterate 5 f= 5.18725D+00 |proj g|= 9.05481D+00 At iterate 6 f= 2.04175D+00 |proj g|= 1.87286D+01 At iterate 7 f= 3.58121D-01 |proj g|= 1.72813D+00 At iterate 8 f= 1.70653D-01 |proj g|= 6.63850D-01 At iterate 9 f= 4.57704D-02 |proj g|= 3.03952D-01 At iterate 10 f= 1.09873D-02 |proj g|= 7.33718D-01 At iterate 11 f= 3.26619D-03 |proj g|= 1.64859D-01 At iterate 12 f= 1.79439D-03 |proj g|= 6.03892D-02 At iterate 13 f= 3.51940D-04 |proj g|= 6.48089D-02 At iterate 14 f= 6.87570D-05 |proj g|= 2.43944D-02 At iterate 15 f= 1.73289D-05 |proj g|= 1.06253D-02 At iterate 16 f= 1.12164D-05 |proj g|= 8.43505D-03 At iterate 17 f= 6.93160D-06 |proj g|= 4.63340D-03 At iterate 18 f= 6.62821D-07 |proj g|= 2.50289D-03 At iterate 19 f= 1.56981D-07 |proj g|= 2.26481D-03 At iterate 20 f= 2.66344D-08 |proj g|= 5.64255D-04 At iterate 21 f= 9.55714D-09 |proj g|= 1.98780D-04 At iterate 22 f= 2.15224D-09 |proj g|= 1.34043D-04 At iterate 23 f= 7.79867D-10 |proj g|= 1.43370D-04 * * * Tit = total number of iterations Tnf = total number of function evaluations Tnint = total number of segments explored during Cauchy searches Skip = number of BFGS updates skipped Nact = number of active bounds at final generalized Cauchy point Projg = norm of the final projected gradient F = final function value * * * N Tit Tnf Tnint Skip Nact Projg F 25 23 27 48 0 0 1.434D-04 7.799D-10 F = 7.79866771512390D-010 CONVERGENCE: REL_REDUCTION_OF_F <= FACTR*EPSMCH Cauchy time 1.000E-01 seconds. Subspace minimization time 3.000E-01 seconds. Line search time 4.000E-02 seconds. Total User time 5.500E-01 seconds. SHAR_EOF fi # end of overwriting check if test -f 'RES2' then echo shar: will not over-write existing file "'RES2'" else cat << \SHAR_EOF > 'RES2' Solving sample problem. (f = 0.0 at the optimal solution.) Iterate 1 nfg = 5 f = 2.39769D+03 |proj g| = 6.50700D+01 Iterate 2 nfg = 6 f = 1.43806D+02 |proj g| = 3.64039D+01 Iterate 3 nfg = 7 f = 7.28161D+01 |proj g| = 2.29042D+01 Iterate 4 nfg = 8 f = 1.60308D+01 |proj g| = 6.95409D+00 Iterate 5 nfg = 9 f = 5.18725D+00 |proj g| = 9.05481D+00 Iterate 6 nfg = 10 f = 2.04175D+00 |proj g| = 1.87286D+01 Iterate 7 nfg = 11 f = 3.58121D-01 |proj g| = 1.72813D+00 Iterate 8 nfg = 12 f = 1.70653D-01 |proj g| = 6.63850D-01 Iterate 9 nfg = 13 f = 4.57704D-02 |proj g| = 3.03952D-01 Iterate 10 nfg = 14 f = 1.09873D-02 |proj g| = 7.33718D-01 Iterate 11 nfg = 15 f = 3.26619D-03 |proj g| = 1.64859D-01 Iterate 12 nfg = 16 f = 1.79439D-03 |proj g| = 6.03892D-02 Iterate 13 nfg = 17 f = 3.51940D-04 |proj g| = 6.48089D-02 Iterate 14 nfg = 18 f = 6.87570D-05 |proj g| = 2.43944D-02 Iterate 15 nfg = 19 f = 1.73289D-05 |proj g| = 1.06253D-02 Iterate 16 nfg = 20 f = 1.12164D-05 |proj g| = 8.43505D-03 Iterate 17 nfg = 21 f = 6.93160D-06 |proj g| = 4.63340D-03 Iterate 18 nfg = 22 f = 6.62821D-07 |proj g| = 2.50289D-03 Iterate 19 nfg = 23 f = 1.56981D-07 |proj g| = 2.26481D-03 Iterate 20 nfg = 24 f = 2.66344D-08 |proj g| = 5.64255D-04 Iterate 21 nfg = 25 f = 9.55714D-09 |proj g| = 1.98780D-04 Iterate 22 nfg = 26 f = 2.15224D-09 |proj g| = 1.34043D-04 Iterate 23 nfg = 27 f = 7.79867D-10 |proj g| = 1.43370D-04 Iterate 24 nfg = 28 f = 1.28217D-10 |proj g| = 7.70946D-05 Iterate 25 nfg = 29 f = 3.76554D-11 |proj g| = 1.94693D-05 Iterate 26 nfg = 30 f = 1.66351D-11 |proj g| = 1.11018D-05 Iterate 27 nfg = 31 f = 4.30981D-12 |proj g| = 5.32394D-06 Iterate 28 nfg = 32 f = 1.02566D-12 |proj g| = 4.10333D-06 Iterate 29 nfg = 33 f = 4.69837D-13 |proj g| = 2.91914D-06 Iterate 30 nfg = 34 f = 1.61806D-13 |proj g| = 9.80593D-07 Iterate 31 nfg = 35 f = 5.64321D-14 |proj g| = 6.15108D-07 Iterate 32 nfg = 36 f = 1.45588D-14 |proj g| = 9.20422D-07 Iterate 33 nfg = 38 f = 8.73787D-15 |proj g| = 2.38756D-07 Iterate 34 nfg = 39 f = 7.16113D-15 |proj g| = 8.33112D-08 Iterate 35 nfg = 40 f = 6.52164D-15 |proj g| = 7.55013D-08 Iterate 36 nfg = 41 f = 6.07160D-15 |proj g| = 5.23093D-08 Iterate 37 nfg = 42 f = 5.83834D-15 |proj g| = 4.05614D-08 Iterate 38 nfg = 44 f = 5.81427D-15 |proj g| = 8.37439D-09 Iterate 39 nfg = 45 f = 5.80983D-15 |proj g| = 2.72259D-09 Iterate 40 nfg = 46 f = 5.80907D-15 |proj g| = 1.48130D-09 Iterate 41 nfg = 47 f = 5.80890D-15 |proj g| = 4.56930D-09 Iterate 42 nfg = 48 f = 5.80876D-15 |proj g| = 1.08595D-09 Iterate 43 nfg = 49 f = 5.80874D-15 |proj g| = 5.41803D-10 Iterate 44 nfg = 50 f = 5.80871D-15 |proj g| = 3.14749D-10 Iterate 45 nfg = 51 f = 5.80870D-15 |proj g| = 2.82095D-10 Iterate 46 nfg = 52 f = 5.80870D-15 |proj g| = 4.33279D-10 Iterate 47 nfg = 53 f = 5.80870D-15 |proj g| = 5.88143D-11 STOP: THE PROJECTED GRADIENT IS SUFFICIENTLY SMALL Final X= 1.0000D+00 1.0000D+00 1.0000D+00 1.0000D+00 1.0000D+00 1.0000D+00 1.0000D+00 1.0000D+00 1.0000D+00 1.0000D+00 1.0001D+00 1.0002D+00 1.0003D+00 1.0007D+00 1.0013D+00 1.0026D+00 1.0052D+00 1.0105D+00 1.0210D+00 1.0425D+00 1.0867D+00 1.1810D+00 1.3948D+00 1.9454D+00 3.7844D+00 SHAR_EOF fi # end of overwriting check if test -f 'RES3' then echo shar: will not over-write existing file "'RES3'" else cat << \SHAR_EOF > 'RES3' Solving sample problem. (f = 0.0 at the optimal solution.) STOP: CPU EXCEEDING THE TIME LIMIT. Latest iterate X = 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 3.0000D+00 At latest iterate f = 1.43860D+05 |proj g| = 1.03000D+02 SHAR_EOF fi # end of overwriting check if test -f 'iterate.dat' then echo shar: will not over-write existing file "'iterate.dat'" else cat << \SHAR_EOF > 'iterate.dat' RUNNING THE L-BFGS-B CODE it = iteration number nf = number of function evaluations nint = number of segments explored during the Cauchy search nact = number of active bounds at the generalized Cauchy point sub = manner in which the subspace minimization terminated: con = converged, bnd = a bound was reached itls = number of iterations performed in the line search stepl = step length used tstep = norm of the displacement (total step) projg = norm of the projected gradient f = function value * * * Machine precision = 2.220D-16 N = 25 M = 5 it nf nint nact sub itls stepl tstep projg f 0 1 - - - - - - 1.030D+02 3.460D+03 1 5 25 24 --- 3 1.2D-02 4.2D+00 6.507D+01 2.398D+03 2 6 1 0 con 0 1.0D+00 4.2D+00 3.640D+01 1.438D+02 3 7 1 0 con 0 1.0D+00 6.8D-01 2.290D+01 7.282D+01 4 8 1 0 con 0 1.0D+00 1.1D+00 6.954D+00 1.603D+01 5 9 1 0 con 0 1.0D+00 6.5D-01 9.055D+00 5.187D+00 6 10 1 0 bnd 0 1.0D+00 5.9D-01 1.873D+01 2.042D+00 7 11 1 0 bnd 0 1.0D+00 1.4D-01 1.728D+00 3.581D-01 8 12 1 0 bnd 0 1.0D+00 6.6D-02 6.639D-01 1.707D-01 9 13 1 0 bnd 0 1.0D+00 9.3D-02 3.040D-01 4.577D-02 10 14 1 0 bnd 0 1.0D+00 6.1D-02 7.337D-01 1.099D-02 11 15 1 0 bnd 0 1.0D+00 1.7D-02 1.649D-01 3.266D-03 12 16 1 0 bnd 0 1.0D+00 6.9D-03 6.039D-02 1.794D-03 13 17 1 0 bnd 0 1.0D+00 1.2D-02 6.481D-02 3.519D-04 14 18 2 2 bnd 0 1.0D+00 5.8D-03 2.439D-02 6.876D-05 15 19 1 0 bnd 0 1.0D+00 1.7D-03 1.063D-02 1.733D-05 16 20 1 0 bnd 0 1.0D+00 3.6D-04 8.435D-03 1.122D-05 17 21 1 0 bnd 0 1.0D+00 3.3D-04 4.633D-03 6.932D-06 18 22 1 0 bnd 0 1.0D+00 9.1D-04 2.503D-03 6.628D-07 19 23 1 0 bnd 0 1.0D+00 2.0D-04 2.265D-03 1.570D-07 20 24 1 0 con 0 1.0D+00 1.2D-04 5.643D-04 2.663D-08 21 25 1 0 con 0 1.0D+00 3.2D-05 1.988D-04 9.557D-09 22 26 1 0 con 0 1.0D+00 3.3D-05 1.340D-04 2.152D-09 23 27 1 0 con 0 1.0D+00 1.6D-05 1.434D-04 7.799D-10 CONVERGENCE: REL_REDUCTION_OF_F <= FACTR*EPSMCH Total User time 5.500E-01 seconds. SHAR_EOF fi # end of overwriting check cd .. cd .. if test ! -d 'Info' then mkdir 'Info' fi cd 'Info' if test -f 'depend' then echo shar: will not over-write existing file "'depend'" else cat << \SHAR_EOF > 'depend' blas1/dnrm2.f blas1/daxpy.f blas1/dcopy.f blas1/ddot.f linpack/dpofa.f blas1/dscal.f linpack/dtrsl.f SHAR_EOF fi # end of overwriting check cd .. if test ! -d 'Src' then mkdir 'Src' fi cd 'Src' if test ! -d 'Dp' then mkdir 'Dp' fi cd 'Dp' if test -f 'timer.f' then echo shar: will not over-write existing file "'timer.f'" else cat << \SHAR_EOF > 'timer.f' subroutine timer(ttime) double precision ttime c ********* c c Subroutine timer c c This subroutine is used to determine user time. In a typical c application, the user time for a code segment requires calls c to subroutine timer to determine the initial and final time. c c The subroutine statement is c c subroutine timer(ttime) c c where c c ttime is an output variable which specifies the user time. c c Argonne National Laboratory and University of Minnesota. c MINPACK-2 Project. c c Modified October 1990 by Brett M. Averick. c c ********** real temp real tarray(2) real etime c The first element of the array tarray specifies user time temp = etime(tarray) ttime = dble(tarray(1)) return end c====================== The end of timer =============================== SHAR_EOF fi # end of overwriting check if test -f 'src.f' then echo shar: will not over-write existing file "'src.f'" else cat << \SHAR_EOF > 'src.f' c================ L-BFGS-B (version 2.3) ========================== subroutine setulb(n, m, x, l, u, nbd, f, g, factr, pgtol, wa, iwa, + task, iprint, csave, lsave, isave, dsave) character*60 task, csave logical lsave(4) integer n, m, iprint, + nbd(n), iwa(3*n), isave(44) double precision f, factr, pgtol, x(n), l(n), u(n), g(n), + wa(2*m*n+4*n+11*m*m+8*m), dsave(29) c ************ c c Subroutine setulb c c This subroutine partitions the working arrays wa and iwa, and c then uses the limited memory BFGS method to solve the bound c constrained optimization problem by calling mainlb. c (The direct method will be used in the subspace minimization.) c c n is an integer variable. c On entry n is the dimension of the problem. c On exit n is unchanged. c c m is an integer variable. c On entry m is the maximum number of variable metric corrections c used to define the limited memory matrix. c On exit m is unchanged. c c x is a double precision array of dimension n. c On entry x is an approximation to the solution. c On exit x is the current approximation. c c l is a double precision array of dimension n. c On entry l is the lower bound on x. c On exit l is unchanged. c c u is a double precision array of dimension n. c On entry u is the upper bound on x. c On exit u is unchanged. c c nbd is an integer array of dimension n. c On entry nbd represents the type of bounds imposed on the c variables, and must be specified as follows: c nbd(i)=0 if x(i) is unbounded, c 1 if x(i) has only a lower bound, c 2 if x(i) has both lower and upper bounds, and c 3 if x(i) has only an upper bound. c On exit nbd is unchanged. c c f is a double precision variable. c On first entry f is unspecified. c On final exit f is the value of the function at x. c c g is a double precision array of dimension n. c On first entry g is unspecified. c On final exit g is the value of the gradient at x. c c factr is a double precision variable. c On entry factr >= 0 is specified by the user. The iteration c will stop when c c (f^k - f^{k+1})/max{|f^k|,|f^{k+1}|,1} <= factr*epsmch c c where epsmch is the machine precision, which is automatically c generated by the code. Typical values for factr: 1.d+12 for c low accuracy; 1.d+7 for moderate accuracy; 1.d+1 for extremely c high accuracy. c On exit factr is unchanged. c c pgtol is a double precision variable. c On entry pgtol >= 0 is specified by the user. The iteration c will stop when c c max{|proj g_i | i = 1, ..., n} <= pgtol c c where pg_i is the ith component of the projected gradient. c On exit pgtol is unchanged. c c wa is a double precision working array of length c (2mmax + 4)nmax + 11mmax^2 + 8mmax. c c iwa is an integer working array of length 3nmax. c c task is a working string of characters of length 60 indicating c the current job when entering and quitting this subroutine. c c iprint is an integer variable that must be set by the user. c It controls the frequency and type of output generated: c iprint<0 no output is generated; c iprint=0 print only one line at the last iteration; c 0100 print details of every iteration including x and g; c When iprint > 0, the file iterate.dat will be created to c summarize the iteration. c c csave is a working string of characters of length 60. c c lsave is a logical working array of dimension 4. c On exit with 'task' = NEW_X, the following information is c available: c If lsave(1) = .true. then the initial X has been replaced by c its projection in the feasible set; c If lsave(2) = .true. then the problem is constrained; c If lsave(3) = .true. then each variable has upper and lower c bounds; c c isave is an integer working array of dimension 44. c On exit with 'task' = NEW_X, the following information is c available: c isave(22) = the total number of intervals explored in the c search of Cauchy points; c isave(26) = the total number of skipped BFGS updates before c the current iteration; c isave(30) = the number of current iteration; c isave(31) = the total number of BFGS updates prior the current c iteration; c isave(33) = the number of intervals explored in the search of c Cauchy point in the current iteration; c isave(34) = the total number of function and gradient c evaluations; c isave(36) = the number of function value or gradient c evaluations in the current iteration; c if isave(37) = 0 then the subspace argmin is within the box; c if isave(37) = 1 then the subspace argmin is beyond the box; c isave(38) = the number of free variables in the current c iteration; c isave(39) = the number of active constraints in the current c iteration; c n + 1 - isave(40) = the number of variables leaving the set of c active constraints in the current iteration; c isave(41) = the number of variables entering the set of active c constraints in the current iteration. c c dsave is a double precision working array of dimension 29. c On exit with 'task' = NEW_X, the following information is c available: c dsave(1) = current 'theta' in the BFGS matrix; c dsave(2) = f(x) in the previous iteration; c dsave(3) = factr*epsmch; c dsave(4) = 2-norm of the line search direction vector; c dsave(5) = the machine precision epsmch generated by the code; c dsave(7) = the accumulated time spent on searching for c Cauchy points; c dsave(8) = the accumulated time spent on c subspace minimization; c dsave(9) = the accumulated time spent on line search; c dsave(11) = the slope of the line search function at c the current point of line search; c dsave(12) = the maximum relative step length imposed in c line search; c dsave(13) = the infinity norm of the projected gradient; c dsave(14) = the relative step length in the line search; c dsave(15) = the slope of the line search function at c the starting point of the line search; c dsave(16) = the square of the 2-norm of the line search c direction vector. c c Subprograms called: c c L-BFGS-B Library ... mainlb. c c c References: c c [1] R. H. Byrd, P. Lu, J. Nocedal and C. Zhu, ``A limited c memory algorithm for bound constrained optimization'', c SIAM J. Scientific Computing 16 (1995), no. 5, pp. 1190--1208. c c [2] C. Zhu, R.H. Byrd, P. Lu, J. Nocedal, ``L-BFGS-B: a c limited memory FORTRAN code for solving bound constrained c optimization problems'', Tech. Report, NAM-11, EECS Department, c Northwestern University, 1994. c c (Postscript files of these papers are available via anonymous c ftp to ece.nwu.edu in the directory pub/lbfgs/lbfgs_bcm.) c c * * * c c NEOS, November 1994. (Latest revision April 1997.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************ integer l1,l2,l3,lws,lr,lz,lt,ld,lwa,lwy,lsy,lss,lwt,lwn,lsnd if (task .eq. 'START') then isave(1) = m*n isave(2) = m**2 isave(3) = 4*m**2 isave(4) = 1 isave(5) = isave(4) + isave(1) isave(6) = isave(5) + isave(1) isave(7) = isave(6) + isave(2) isave(8) = isave(7) + isave(2) isave(9) = isave(8) isave(10) = isave(9) + isave(2) isave(11) = isave(10) + isave(3) isave(12) = isave(11) + isave(3) isave(13) = isave(12) + n isave(14) = isave(13) + n isave(15) = isave(14) + n isave(16) = isave(15) + n endif l1 = isave(1) l2 = isave(2) l3 = isave(3) lws = isave(4) lwy = isave(5) lsy = isave(6) lss = isave(7) lwt = isave(9) lwn = isave(10) lsnd = isave(11) lz = isave(12) lr = isave(13) ld = isave(14) lt = isave(15) lwa = isave(16) call mainlb(n,m,x,l,u,nbd,f,g,factr,pgtol, + wa(lws),wa(lwy),wa(lsy),wa(lss),wa(lwt), + wa(lwn),wa(lsnd),wa(lz),wa(lr),wa(ld),wa(lt), + wa(lwa),iwa(1),iwa(n+1),iwa(2*n+1),task,iprint, + csave,lsave,isave(22),dsave) return end c======================= The end of setulb ============================= subroutine mainlb(n, m, x, l, u, nbd, f, g, factr, pgtol, ws, wy, + sy, ss, wt, wn, snd, z, r, d, t, wa, + index, iwhere, indx2, task, iprint, + csave, lsave, isave, dsave) character*60 task, csave logical lsave(4) integer n, m, iprint, nbd(n), index(n), + iwhere(n), indx2(n), isave(23) double precision f, factr, pgtol, + x(n), l(n), u(n), g(n), z(n), r(n), d(n), t(n), + wa(8*m), ws(n, m), wy(n, m), sy(m, m), ss(m, m), + wt(m, m), wn(2*m, 2*m), snd(2*m, 2*m), dsave(29) c ************ c c Subroutine mainlb c c This subroutine solves bound constrained optimization problems by c using the compact formula of the limited memory BFGS updates. c c n is an integer variable. c On entry n is the number of variables. c On exit n is unchanged. c c m is an integer variable. c On entry m is the maximum number of variable metric c corrections allowed in the limited memory matrix. c On exit m is unchanged. c c x is a double precision array of dimension n. c On entry x is an approximation to the solution. c On exit x is the current approximation. c c l is a double precision array of dimension n. c On entry l is the lower bound of x. c On exit l is unchanged. c c u is a double precision array of dimension n. c On entry u is the upper bound of x. c On exit u is unchanged. c c nbd is an integer array of dimension n. c On entry nbd represents the type of bounds imposed on the c variables, and must be specified as follows: c nbd(i)=0 if x(i) is unbounded, c 1 if x(i) has only a lower bound, c 2 if x(i) has both lower and upper bounds, c 3 if x(i) has only an upper bound. c On exit nbd is unchanged. c c f is a double precision variable. c On first entry f is unspecified. c On final exit f is the value of the function at x. c c g is a double precision array of dimension n. c On first entry g is unspecified. c On final exit g is the value of the gradient at x. c c factr is a double precision variable. c On entry factr >= 0 is specified by the user. The iteration c will stop when c c (f^k - f^{k+1})/max{|f^k|,|f^{k+1}|,1} <= factr*epsmch c c where epsmch is the machine precision, which is automatically c generated by the code. c On exit factr is unchanged. c c pgtol is a double precision variable. c On entry pgtol >= 0 is specified by the user. The iteration c will stop when c c max{|proj g_i | i = 1, ..., n} <= pgtol c c where pg_i is the ith component of the projected gradient. c On exit pgtol is unchanged. c c ws, wy, sy, and wt are double precision working arrays used to c store the following information defining the limited memory c BFGS matrix: c ws, of dimension n x m, stores S, the matrix of s-vectors; c wy, of dimension n x m, stores Y, the matrix of y-vectors; c sy, of dimension m x m, stores S'Y; c ss, of dimension m x m, stores S'S; c wt, of dimension m x m, stores the Cholesky factorization c of (theta*S'S+LD^(-1)L'); see eq. c (2.26) in [3]. c c wn is a double precision working array of dimension 2m x 2m c used to store the LEL^T factorization of the indefinite matrix c K = [-D -Y'ZZ'Y/theta L_a'-R_z' ] c [L_a -R_z theta*S'AA'S ] c c where E = [-I 0] c [ 0 I] c c snd is a double precision working array of dimension 2m x 2m c used to store the lower triangular part of c N = [Y' ZZ'Y L_a'+R_z'] c [L_a +R_z S'AA'S ] c c z(n),r(n),d(n),t(n),wa(8*m) are double precision working arrays. c z is used at different times to store the Cauchy point and c the Newton point. c c c index is an integer working array of dimension n. c In subroutine freev, index is used to store the free and fixed c variables at the Generalized Cauchy Point (GCP). c c iwhere is an integer working array of dimension n used to record c the status of the vector x for GCP computation. c iwhere(i)=0 or -3 if x(i) is free and has bounds, c 1 if x(i) is fixed at l(i), and l(i) .ne. u(i) c 2 if x(i) is fixed at u(i), and u(i) .ne. l(i) c 3 if x(i) is always fixed, i.e., u(i)=x(i)=l(i) c -1 if x(i) is always free, i.e., no bounds on it. c c indx2 is an integer working array of dimension n. c Within subroutine cauchy, indx2 corresponds to the array iorder. c In subroutine freev, a list of variables entering and leaving c the free set is stored in indx2, and it is passed on to c subroutine formk with this information. c c task is a working string of characters of length 60 indicating c the current job when entering and leaving this subroutine. c c iprint is an INTEGER variable that must be set by the user. c It controls the frequency and type of output generated: c iprint<0 no output is generated; c iprint=0 print only one line at the last iteration; c 0100 print details of every iteration including x and g; c When iprint > 0, the file iterate.dat will be created to c summarize the iteration. c c csave is a working string of characters of length 60. c c lsave is a logical working array of dimension 4. c c isave is an integer working array of dimension 23. c c dsave is a double precision working array of dimension 29. c c c Subprograms called c c L-BFGS-B Library ... cauchy, subsm, lnsrlb, formk, c c errclb, prn1lb, prn2lb, prn3lb, active, projgr, c c freev, cmprlb, matupd, formt. c c Minpack2 Library ... timer, dpmeps. c c Linpack Library ... dcopy, ddot. c c c References: c c [1] R. H. Byrd, P. Lu, J. Nocedal and C. Zhu, ``A limited c memory algorithm for bound constrained optimization'', c SIAM J. Scientific Computing 16 (1995), no. 5, pp. 1190--1208. c c [2] C. Zhu, R.H. Byrd, P. Lu, J. Nocedal, ``L-BFGS-B: FORTRAN c Subroutines for Large Scale Bound Constrained Optimization'' c Tech. Report, NAM-11, EECS Department, Northwestern University, c 1994. c c [3] R. Byrd, J. Nocedal and R. Schnabel "Representations of c Quasi-Newton Matrices and their use in Limited Memory Methods'', c Mathematical Programming 63 (1994), no. 4, pp. 129-156. c c (Postscript files of these papers are available via anonymous c ftp to ece.nwu.edu in the directory pub/lbfgs/lbfgs_bcm.) c c * * * c c NEOS, November 1994. (Latest revision April 1997.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************ logical prjctd,cnstnd,boxed,updatd,wrk character*3 word integer i,k,nintol,itfile,iback,nskip, + head,col,iter,itail,iupdat, + nint,nfgv,info,ifun, + iword,nfree,nact,ileave,nenter double precision theta,fold,ddot,dr,rr,tol,dpmeps, + xstep,sbgnrm,ddum,dnorm,dtd,epsmch, + cpu1,cpu2,cachyt,sbtime,lnscht,time1,time2, + gd,gdold,stp,stpmx,time double precision one,zero parameter (one=1.0d0,zero=0.0d0) if (task .eq. 'START') then call timer(time1) c Generate the current machine precision. epsmch = dpmeps() fold = 0.0d0 dnorm = 0.0d0 cpu1 = 0.0d0 gd = 0.0d0 sbgnrm = 0.0d0 stp = 0.0d0 stpmx = 0.0d0 gdold = 0.0d0 dtd = 0.0d0 c Initialize counters and scalars when task='START'. c for the limited memory BFGS matrices: col = 0 head = 1 theta = one iupdat = 0 updatd = .false. iback = 0 itail = 0 ifun = 0 iword = 0 nact = 0 ileave = 0 nenter = 0 c for operation counts: iter = 0 nfgv = 0 nint = 0 nintol = 0 nskip = 0 nfree = n c for stopping tolerance: tol = factr*epsmch c for measuring running time: cachyt = 0 sbtime = 0 lnscht = 0 c 'word' records the status of subspace solutions. word = '---' c 'info' records the termination information. info = 0 itfile = 0 if (iprint .ge. 1) then c open a summary file 'iterate.dat' open (8, file = 'iterate.dat', status = 'unknown') itfile = 8 endif c Check the input arguments for errors. call errclb(n,m,factr,l,u,nbd,task,info,k) if (task(1:5) .eq. 'ERROR') then call prn3lb(n,x,f,task,iprint,info,itfile, + iter,nfgv,nintol,nskip,nact,sbgnrm, + zero,nint,word,iback,stp,xstep,k, + cachyt,sbtime,lnscht) return endif call prn1lb(n,m,l,u,x,iprint,itfile,epsmch) c Initialize iwhere & project x onto the feasible set. call active(n,l,u,nbd,x,iwhere,iprint,prjctd,cnstnd,boxed) c The end of the initialization. else c restore local variables. prjctd = lsave(1) cnstnd = lsave(2) boxed = lsave(3) updatd = lsave(4) nintol = isave(1) itfile = isave(3) iback = isave(4) nskip = isave(5) head = isave(6) col = isave(7) itail = isave(8) iter = isave(9) iupdat = isave(10) nint = isave(12) nfgv = isave(13) info = isave(14) ifun = isave(15) iword = isave(16) nfree = isave(17) nact = isave(18) ileave = isave(19) nenter = isave(20) theta = dsave(1) fold = dsave(2) tol = dsave(3) dnorm = dsave(4) epsmch = dsave(5) cpu1 = dsave(6) cachyt = dsave(7) sbtime = dsave(8) lnscht = dsave(9) time1 = dsave(10) gd = dsave(11) stpmx = dsave(12) sbgnrm = dsave(13) stp = dsave(14) gdold = dsave(15) dtd = dsave(16) c After returning from the driver go to the point where execution c is to resume. if (task(1:5) .eq. 'FG_LN') goto 666 if (task(1:5) .eq. 'NEW_X') goto 777 if (task(1:5) .eq. 'FG_ST') goto 111 if (task(1:4) .eq. 'STOP') then if (task(7:9) .eq. 'CPU') then c restore the previous iterate. call dcopy(n,t,1,x,1) call dcopy(n,r,1,g,1) f = fold endif goto 999 endif endif c Compute f0 and g0. task = 'FG_START' c return to the driver to calculate f and g; reenter at 111. goto 1000 111 continue nfgv = 1 c Compute the infinity norm of the (-) projected gradient. call projgr(n,l,u,nbd,x,g,sbgnrm) if (iprint .ge. 1) then write (6,1002) iter,f,sbgnrm write (itfile,1003) iter,nfgv,sbgnrm,f endif if (sbgnrm .le. pgtol) then c terminate the algorithm. task = 'CONVERGENCE: NORM OF PROJECTED GRADIENT <= PGTOL' goto 999 endif c ----------------- the beginning of the loop -------------------------- 222 continue if (iprint .ge. 99) write (6,1001) iter + 1 iword = -1 c if (.not. cnstnd .and. col .gt. 0) then c skip the search for GCP. call dcopy(n,x,1,z,1) wrk = updatd nint = 0 goto 333 endif cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc c c Compute the Generalized Cauchy Point (GCP). c cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc call timer(cpu1) call cauchy(n,x,l,u,nbd,g,indx2,iwhere,t,d,z, + m,wy,ws,sy,wt,theta,col,head, + wa(1),wa(2*m+1),wa(4*m+1),wa(6*m+1),nint, + iprint,sbgnrm,info,epsmch) if (info .ne. 0) then c singular triangular system detected; refresh the lbfgs memory. if(iprint .ge. 1) write (6, 1005) info = 0 col = 0 head = 1 theta = one iupdat = 0 updatd = .false. call timer(cpu2) cachyt = cachyt + cpu2 - cpu1 goto 222 endif call timer(cpu2) cachyt = cachyt + cpu2 - cpu1 nintol = nintol + nint c Count the entering and leaving variables for iter > 0; c find the index set of free and active variables at the GCP. call freev(n,nfree,index,nenter,ileave,indx2, + iwhere,wrk,updatd,cnstnd,iprint,iter) nact = n - nfree 333 continue c If there are no free variables or B=theta*I, then c skip the subspace minimization. if (nfree .eq. 0 .or. col .eq. 0) goto 555 cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc c c Subspace minimization. c cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc call timer(cpu1) c Form the LEL^T factorization of the indefinite c matrix K = [-D -Y'ZZ'Y/theta L_a'-R_z' ] c [L_a -R_z theta*S'AA'S ] c where E = [-I 0] c [ 0 I] if (wrk) call formk(n,nfree,index,nenter,ileave,indx2,iupdat, + updatd,wn,snd,m,ws,wy,sy,theta,col,head,info) if (info .ne. 0) then c nonpositive definiteness in Cholesky factorization; c refresh the lbfgs memory and restart the iteration. if(iprint .ge. 1) write (6, 1006) info = 0 col = 0 head = 1 theta = one iupdat = 0 updatd = .false. call timer(cpu2) sbtime = sbtime + cpu2 - cpu1 goto 222 endif c compute r=-Z'B(xcp-xk)-Z'g (using wa(2m+1)=W'(xcp-x) c from 'cauchy'). call cmprlb(n,m,x,g,ws,wy,sy,wt,z,r,wa,index, + theta,col,head,nfree,cnstnd,info) if (info .ne. 0) goto 444 c call the direct method. call subsm(n,m,nfree,index,l,u,nbd,z,r,ws,wy,theta, + col,head,iword,wa,wn,iprint,info) 444 continue if (info .ne. 0) then c singular triangular system detected; c refresh the lbfgs memory and restart the iteration. if(iprint .ge. 1) write (6, 1005) info = 0 col = 0 head = 1 theta = one iupdat = 0 updatd = .false. call timer(cpu2) sbtime = sbtime + cpu2 - cpu1 goto 222 endif call timer(cpu2) sbtime = sbtime + cpu2 - cpu1 555 continue cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc c c Line search and optimality tests. c cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc c Generate the search direction d:=z-x. do 40 i = 1, n d(i) = z(i) - x(i) 40 continue call timer(cpu1) 666 continue call lnsrlb(n,l,u,nbd,x,f,fold,gd,gdold,g,d,r,t,z,stp,dnorm, + dtd,xstep,stpmx,iter,ifun,iback,nfgv,info,task, + boxed,cnstnd,csave,isave(22),dsave(17)) if (info .ne. 0 .or. iback .ge. 20) then c restore the previous iterate. call dcopy(n,t,1,x,1) call dcopy(n,r,1,g,1) f = fold if (col .eq. 0) then c abnormal termination. if (info .eq. 0) then info = -9 c restore the actual number of f and g evaluations etc. nfgv = nfgv - 1 ifun = ifun - 1 iback = iback - 1 endif task = 'ABNORMAL_TERMINATION_IN_LNSRCH' iter = iter + 1 goto 999 else c refresh the lbfgs memory and restart the iteration. if(iprint .ge. 1) write (6, 1008) if (info .eq. 0) nfgv = nfgv - 1 info = 0 col = 0 head = 1 theta = one iupdat = 0 updatd = .false. task = 'RESTART_FROM_LNSRCH' call timer(cpu2) lnscht = lnscht + cpu2 - cpu1 goto 222 endif else if (task(1:5) .eq. 'FG_LN') then c return to the driver for calculating f and g; reenter at 666. goto 1000 else c calculate and print out the quantities related to the new X. call timer(cpu2) lnscht = lnscht + cpu2 - cpu1 iter = iter + 1 c Compute the infinity norm of the projected (-)gradient. call projgr(n,l,u,nbd,x,g,sbgnrm) c Print iteration information. call prn2lb(n,x,f,g,iprint,itfile,iter,nfgv,nact, + sbgnrm,nint,word,iword,iback,stp,xstep) goto 1000 endif 777 continue c Test for termination. if (sbgnrm .le. pgtol) then c terminate the algorithm. task = 'CONVERGENCE: NORM OF PROJECTED GRADIENT <= PGTOL' goto 999 endif ddum = max(abs(fold), abs(f), one) if ((fold - f) .le. tol*ddum) then c terminate the algorithm. task = 'CONVERGENCE: REL_REDUCTION_OF_F <= FACTR*EPSMCH' if (iback .ge. 10) info = -5 c i.e., to issue a warning if iback>10 in the line search. goto 999 endif c Compute d=newx-oldx, r=newg-oldg, rr=y'y and dr=y's. do 42 i = 1, n r(i) = g(i) - r(i) 42 continue rr = ddot(n,r,1,r,1) if (stp .eq. one) then dr = gd - gdold ddum = -gdold else dr = (gd - gdold)*stp call dscal(n,stp,d,1) ddum = -gdold*stp endif if (dr .le. epsmch*ddum) then c skip the L-BFGS update. nskip = nskip + 1 updatd = .false. if (iprint .ge. 1) write (6,1004) dr, ddum goto 888 endif cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc c c Update the L-BFGS matrix. c cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc updatd = .true. iupdat = iupdat + 1 c Update matrices WS and WY and form the middle matrix in B. call matupd(n,m,ws,wy,sy,ss,d,r,itail, + iupdat,col,head,theta,rr,dr,stp,dtd) c Form the upper half of the pds T = theta*SS + L*D^(-1)*L'; c Store T in the upper triangular of the array wt; c Cholesky factorize T to J*J' with c J' stored in the upper triangular of wt. call formt(m,wt,sy,ss,col,theta,info) if (info .ne. 0) then c nonpositive definiteness in Cholesky factorization; c refresh the lbfgs memory and restart the iteration. if(iprint .ge. 1) write (6, 1007) info = 0 col = 0 head = 1 theta = one iupdat = 0 updatd = .false. goto 222 endif c Now the inverse of the middle matrix in B is c [ D^(1/2) O ] [ -D^(1/2) D^(-1/2)*L' ] c [ -L*D^(-1/2) J ] [ 0 J' ] 888 continue c -------------------- the end of the loop ----------------------------- goto 222 999 continue call timer(time2) time = time2 - time1 call prn3lb(n,x,f,task,iprint,info,itfile, + iter,nfgv,nintol,nskip,nact,sbgnrm, + time,nint,word,iback,stp,xstep,k, + cachyt,sbtime,lnscht) 1000 continue c Save local variables. lsave(1) = prjctd lsave(2) = cnstnd lsave(3) = boxed lsave(4) = updatd isave(1) = nintol isave(3) = itfile isave(4) = iback isave(5) = nskip isave(6) = head isave(7) = col isave(8) = itail isave(9) = iter isave(10) = iupdat isave(12) = nint isave(13) = nfgv isave(14) = info isave(15) = ifun isave(16) = iword isave(17) = nfree isave(18) = nact isave(19) = ileave isave(20) = nenter dsave(1) = theta dsave(2) = fold dsave(3) = tol dsave(4) = dnorm dsave(5) = epsmch dsave(6) = cpu1 dsave(7) = cachyt dsave(8) = sbtime dsave(9) = lnscht dsave(10) = time1 dsave(11) = gd dsave(12) = stpmx dsave(13) = sbgnrm dsave(14) = stp dsave(15) = gdold dsave(16) = dtd 1001 format (//,'ITERATION ',i5) 1002 format + (/,'At iterate',i5,4x,'f= ',1p,d12.5,4x,'|proj g|= ',1p,d12.5) 1003 format (2(1x,i4),5x,'-',5x,'-',3x,'-',5x,'-',5x,'-',8x,'-',3x, + 1p,2(1x,d10.3)) 1004 format (' ys=',1p,e10.3,' -gs=',1p,e10.3,' BFGS update SKIPPED') 1005 format (/, +' Singular triangular system detected;',/, +' refresh the lbfgs memory and restart the iteration.') 1006 format (/, +' Nonpositive definiteness in Cholesky factorization in formk;',/, +' refresh the lbfgs memory and restart the iteration.') 1007 format (/, +' Nonpositive definiteness in Cholesky factorization in formt;',/, +' refresh the lbfgs memory and restart the iteration.') 1008 format (/, +' Bad direction in the line search;',/, +' refresh the lbfgs memory and restart the iteration.') return end c======================= The end of mainlb ============================= subroutine active(n, l, u, nbd, x, iwhere, iprint, + prjctd, cnstnd, boxed) logical prjctd, cnstnd, boxed integer n, iprint, nbd(n), iwhere(n) double precision x(n), l(n), u(n) c ************ c c Subroutine active c c This subroutine initializes iwhere and projects the initial x to c the feasible set if necessary. c c iwhere is an integer array of dimension n. c On entry iwhere is unspecified. c On exit iwhere(i)=-1 if x(i) has no bounds c 3 if l(i)=u(i) c 0 otherwise. c In cauchy, iwhere is given finer gradations. c c c * * * c c NEOS, November 1994. (Latest revision June 1996.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************ integer nbdd,i double precision zero parameter (zero=0.0d0) c Initialize nbdd, prjctd, cnstnd and boxed. nbdd = 0 prjctd = .false. cnstnd = .false. boxed = .true. c Project the initial x to the easible set if necessary. do 10 i = 1, n if (nbd(i) .gt. 0) then if (nbd(i) .le. 2 .and. x(i) .le. l(i)) then if (x(i) .lt. l(i)) then prjctd = .true. x(i) = l(i) endif nbdd = nbdd + 1 else if (nbd(i) .ge. 2 .and. x(i) .ge. u(i)) then if (x(i) .gt. u(i)) then prjctd = .true. x(i) = u(i) endif nbdd = nbdd + 1 endif endif 10 continue c Initialize iwhere and assign values to cnstnd and boxed. do 20 i = 1, n if (nbd(i) .ne. 2) boxed = .false. if (nbd(i) .eq. 0) then c this variable is always free iwhere(i) = -1 c otherwise set x(i)=mid(x(i), u(i), l(i)). else cnstnd = .true. if (nbd(i) .eq. 2 .and. u(i) - l(i) .le. zero) then c this variable is always fixed iwhere(i) = 3 else iwhere(i) = 0 endif endif 20 continue if (iprint .ge. 0) then if (prjctd) write (6,*) + 'The initial X is infeasible. Restart with its projection.' if (.not. cnstnd) + write (6,*) 'This problem is unconstrained.' endif if (iprint .gt. 0) write (6,1001) nbdd 1001 format (/,'At X0 ',i9,' variables are exactly at the bounds') return end c======================= The end of active ============================= subroutine bmv(m, sy, wt, col, v, p, info) integer m, col, info double precision sy(m, m), wt(m, m), v(2*col), p(2*col) c ************ c c Subroutine bmv c c This subroutine computes the product of the 2m x 2m middle matrix c in the compact L-BFGS formula of B and a 2m vector v; c it returns the product in p. c c m is an integer variable. c On entry m is the maximum number of variable metric corrections c used to define the limited memory matrix. c On exit m is unchanged. c c sy is a double precision array of dimension m x m. c On entry sy specifies the matrix S'Y. c On exit sy is unchanged. c c wt is a double precision array of dimension m x m. c On entry wt specifies the upper triangular matrix J' which is c the Cholesky factor of (thetaS'S+LD^(-1)L'). c On exit wt is unchanged. c c col is an integer variable. c On entry col specifies the number of s-vectors (or y-vectors) c stored in the compact L-BFGS formula. c On exit col is unchanged. c c v is a double precision array of dimension 2col. c On entry v specifies vector v. c On exit v is unchanged. c c p is a double precision array of dimension 2col. c On entry p is unspecified. c On exit p is the product Mv. c c info is an integer variable. c On entry info is unspecified. c On exit info = 0 for normal return, c = nonzero for abnormal return when the system c to be solved by dtrsl is singular. c c Subprograms called: c c Linpack ... dtrsl. c c c * * * c c NEOS, November 1994. (Latest revision June 1996.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************ integer i,k,i2 double precision sum if (col .eq. 0) return c PART I: solve [ D^(1/2) O ] [ p1 ] = [ v1 ] c [ -L*D^(-1/2) J ] [ p2 ] [ v2 ]. c solve Jp2=v2+LD^(-1)v1. p(col + 1) = v(col + 1) do 20 i = 2, col i2 = col + i sum = 0.0d0 do 10 k = 1, i - 1 sum = sum + sy(i,k)*v(k)/sy(k,k) 10 continue p(i2) = v(i2) + sum 20 continue c Solve the triangular system call dtrsl(wt,m,col,p(col+1),11,info) if (info .ne. 0) return c solve D^(1/2)p1=v1. do 30 i = 1, col p(i) = v(i)/sqrt(sy(i,i)) 30 continue c PART II: solve [ -D^(1/2) D^(-1/2)*L' ] [ p1 ] = [ p1 ] c [ 0 J' ] [ p2 ] [ p2 ]. c solve J^Tp2=p2. call dtrsl(wt,m,col,p(col+1),01,info) if (info .ne. 0) return c compute p1=-D^(-1/2)(p1-D^(-1/2)L'p2) c =-D^(-1/2)p1+D^(-1)L'p2. do 40 i = 1, col p(i) = -p(i)/sqrt(sy(i,i)) 40 continue do 60 i = 1, col sum = 0.d0 do 50 k = i + 1, col sum = sum + sy(k,i)*p(col+k)/sy(i,i) 50 continue p(i) = p(i) + sum 60 continue return end c======================== The end of bmv =============================== subroutine cauchy(n, x, l, u, nbd, g, iorder, iwhere, t, d, xcp, + m, wy, ws, sy, wt, theta, col, head, p, c, wbp, + v, nint, iprint, sbgnrm, info, epsmch) integer n, m, head, col, nint, iprint, info, + nbd(n), iorder(n), iwhere(n) double precision theta, epsmch, + x(n), l(n), u(n), g(n), t(n), d(n), xcp(n), + wy(n, col), ws(n, col), sy(m, m), + wt(m, m), p(2*m), c(2*m), wbp(2*m), v(2*m) c ************ c c Subroutine cauchy c c For given x, l, u, g (with sbgnrm > 0), and a limited memory c BFGS matrix B defined in terms of matrices WY, WS, WT, and c scalars head, col, and theta, this subroutine computes the c generalized Cauchy point (GCP), defined as the first local c minimizer of the quadratic c c Q(x + s) = g's + 1/2 s'Bs c c along the projected gradient direction P(x-tg,l,u). c The routine returns the GCP in xcp. c c n is an integer variable. c On entry n is the dimension of the problem. c On exit n is unchanged. c c x is a double precision array of dimension n. c On entry x is the starting point for the GCP computation. c On exit x is unchanged. c c l is a double precision array of dimension n. c On entry l is the lower bound of x. c On exit l is unchanged. c c u is a double precision array of dimension n. c On entry u is the upper bound of x. c On exit u is unchanged. c c nbd is an integer array of dimension n. c On entry nbd represents the type of bounds imposed on the c variables, and must be specified as follows: c nbd(i)=0 if x(i) is unbounded, c 1 if x(i) has only a lower bound, c 2 if x(i) has both lower and upper bounds, and c 3 if x(i) has only an upper bound. c On exit nbd is unchanged. c c g is a double precision array of dimension n. c On entry g is the gradient of f(x). g must be a nonzero vector. c On exit g is unchanged. c c iorder is an integer working array of dimension n. c iorder will be used to store the breakpoints in the piecewise c linear path and free variables encountered. On exit, c iorder(1),...,iorder(nleft) are indices of breakpoints c which have not been encountered; c iorder(nleft+1),...,iorder(nbreak) are indices of c encountered breakpoints; and c iorder(nfree),...,iorder(n) are indices of variables which c have no bound constraits along the search direction. c c iwhere is an integer array of dimension n. c On entry iwhere indicates only the permanently fixed (iwhere=3) c or free (iwhere= -1) components of x. c On exit iwhere records the status of the current x variables. c iwhere(i)=-3 if x(i) is free and has bounds, but is not moved c 0 if x(i) is free and has bounds, and is moved c 1 if x(i) is fixed at l(i), and l(i) .ne. u(i) c 2 if x(i) is fixed at u(i), and u(i) .ne. l(i) c 3 if x(i) is always fixed, i.e., u(i)=x(i)=l(i) c -1 if x(i) is always free, i.e., it has no bounds. c c t is a double precision working array of dimension n. c t will be used to store the break points. c c d is a double precision array of dimension n used to store c the Cauchy direction P(x-tg)-x. c c xcp is a double precision array of dimension n used to return the c GCP on exit. c c m is an integer variable. c On entry m is the maximum number of variable metric corrections c used to define the limited memory matrix. c On exit m is unchanged. c c ws, wy, sy, and wt are double precision arrays. c On entry they store information that defines the c limited memory BFGS matrix: c ws(n,m) stores S, a set of s-vectors; c wy(n,m) stores Y, a set of y-vectors; c sy(m,m) stores S'Y; c wt(m,m) stores the c Cholesky factorization of (theta*S'S+LD^(-1)L'). c On exit these arrays are unchanged. c c theta is a double precision variable. c On entry theta is the scaling factor specifying B_0 = theta I. c On exit theta is unchanged. c c col is an integer variable. c On entry col is the actual number of variable metric c corrections stored so far. c On exit col is unchanged. c c head is an integer variable. c On entry head is the location of the first s-vector c (or y-vector) in S (or Y). c On exit col is unchanged. c c p is a double precision working array of dimension 2m. c p will be used to store the vector p = W^(T)d. c c c is a double precision working array of dimension 2m. c c will be used to store the vector c = W^(T)(xcp-x). c c wbp is a double precision working array of dimension 2m. c wbp will be used to store the row of W corresponding c to a breakpoint. c c v is a double precision working array of dimension 2m. c c nint is an integer variable. c On exit nint records the number of quadratic segments explored c in searching for the GCP. c c iprint is an INTEGER variable that must be set by the user. c It controls the frequency and type of output generated: c iprint<0 no output is generated; c iprint=0 print only one line at the last iteration; c 0100 print details of every iteration including x and g; c When iprint > 0, the file iterate.dat will be created to c summarize the iteration. c c sbgnrm is a double precision variable. c On entry sbgnrm is the norm of the projected gradient at x. c On exit sbgnrm is unchanged. c c info is an integer variable. c On entry info is 0. c On exit info = 0 for normal return, c = nonzero for abnormal return when the the system c used in routine bmv is singular. c c Subprograms called: c c L-BFGS-B Library ... hpsolb, bmv. c c Linpack ... dscal dcopy, daxpy. c c c References: c c [1] R. H. Byrd, P. Lu, J. Nocedal and C. Zhu, ``A limited c memory algorithm for bound constrained optimization'', c SIAM J. Scientific Computing 16 (1995), no. 5, pp. 1190--1208. c c [2] C. Zhu, R.H. Byrd, P. Lu, J. Nocedal, ``L-BFGS-B: FORTRAN c Subroutines for Large Scale Bound Constrained Optimization'' c Tech. Report, NAM-11, EECS Department, Northwestern University, c 1994. c c (Postscript files of these papers are available via anonymous c ftp to ece.nwu.edu in the directory pub/lbfgs/lbfgs_bcm.) c c * * * c c NEOS, November 1994. (Latest revision April 1997.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************ logical xlower,xupper,bnded integer i,j,col2,nfree,nbreak,pointr, + ibp,nleft,ibkmin,iter double precision f1,f2,dt,dtm,tsum,dibp,zibp,dibp2,bkmin, + tu,tl,wmc,wmp,wmw,ddot,tj,tj0,neggi,sbgnrm, + f2_org double precision one,zero parameter (one=1.0d0,zero=0.0d0) c Check the status of the variables, reset iwhere(i) if necessary; c compute the Cauchy direction d and the breakpoints t; initialize c the derivative f1 and the vector p = W'd (for theta = 1). if (sbgnrm .le. zero) then if (iprint .ge. 0) write (6,*) 'Subgnorm = 0. GCP = X.' call dcopy(n,x,1,xcp,1) return endif bnded = .true. nfree = n + 1 nbreak = 0 ibkmin = 0 bkmin = zero col2 = 2*col f1 = zero if (iprint .ge. 99) write (6,3010) c We set p to zero and build it up as we determine d. do 20 i = 1, col2 p(i) = zero 20 continue c In the following loop we determine for each variable its bound c status and its breakpoint, and update p accordingly. c Smallest breakpoint is identified. do 50 i = 1, n neggi = -g(i) if (iwhere(i) .ne. 3 .and. iwhere(i) .ne. -1) then c if x(i) is not a constant and has bounds, c compute the difference between x(i) and its bounds. if (nbd(i) .le. 2) tl = x(i) - l(i) if (nbd(i) .ge. 2) tu = u(i) - x(i) c If a variable is close enough to a bound c we treat it as at bound. xlower = nbd(i) .le. 2 .and. tl .le. zero xupper = nbd(i) .ge. 2 .and. tu .le. zero c reset iwhere(i). iwhere(i) = 0 if (xlower) then if (neggi .le. zero) iwhere(i) = 1 else if (xupper) then if (neggi .ge. zero) iwhere(i) = 2 else if (abs(neggi) .le. zero) iwhere(i) = -3 endif endif pointr = head if (iwhere(i) .ne. 0 .and. iwhere(i) .ne. -1) then d(i) = zero else d(i) = neggi f1 = f1 - neggi*neggi c calculate p := p - W'e_i* (g_i). do 40 j = 1, col p(j) = p(j) + wy(i,pointr)* neggi p(col + j) = p(col + j) + ws(i,pointr)*neggi pointr = mod(pointr,m) + 1 40 continue if (nbd(i) .le. 2 .and. nbd(i) .ne. 0 + .and. neggi .lt. zero) then c x(i) + d(i) is bounded; compute t(i). nbreak = nbreak + 1 iorder(nbreak) = i t(nbreak) = tl/(-neggi) if (nbreak .eq. 1 .or. t(nbreak) .lt. bkmin) then bkmin = t(nbreak) ibkmin = nbreak endif else if (nbd(i) .ge. 2 .and. neggi .gt. zero) then c x(i) + d(i) is bounded; compute t(i). nbreak = nbreak + 1 iorder(nbreak) = i t(nbreak) = tu/neggi if (nbreak .eq. 1 .or. t(nbreak) .lt. bkmin) then bkmin = t(nbreak) ibkmin = nbreak endif else c x(i) + d(i) is not bounded. nfree = nfree - 1 iorder(nfree) = i if (abs(neggi) .gt. zero) bnded = .false. endif endif 50 continue c The indices of the nonzero components of d are now stored c in iorder(1),...,iorder(nbreak) and iorder(nfree),...,iorder(n). c The smallest of the nbreak breakpoints is in t(ibkmin)=bkmin. if (theta .ne. one) then c complete the initialization of p for theta not= one. call dscal(col,theta,p(col+1),1) endif c Initialize GCP xcp = x. call dcopy(n,x,1,xcp,1) if (nbreak .eq. 0 .and. nfree .eq. n + 1) then c is a zero vector, return with the initial xcp as GCP. if (iprint .gt. 100) write (6,1010) (xcp(i), i = 1, n) return endif c Initialize c = W'(xcp - x) = 0. do 60 j = 1, col2 c(j) = zero 60 continue c Initialize derivative f2. f2 = -theta*f1 f2_org = f2 if (col .gt. 0) then call bmv(m,sy,wt,col,p,v,info) if (info .ne. 0) return f2 = f2 - ddot(col2,v,1,p,1) endif dtm = -f1/f2 tsum = zero nint = 1 if (iprint .ge. 99) + write (6,*) 'There are ',nbreak,' breakpoints ' c If there are no breakpoints, locate the GCP and return. if (nbreak .eq. 0) goto 888 nleft = nbreak iter = 1 tj = zero c------------------- the beginning of the loop ------------------------- 777 continue c Find the next smallest breakpoint; c compute dt = t(nleft) - t(nleft + 1). tj0 = tj if (iter .eq. 1) then c Since we already have the smallest breakpoint we need not do c heapsort yet. Often only one breakpoint is used and the c cost of heapsort is avoided. tj = bkmin ibp = iorder(ibkmin) else if (iter .eq. 2) then c Replace the already used smallest breakpoint with the c breakpoint numbered nbreak > nlast, before heapsort call. if (ibkmin .ne. nbreak) then t(ibkmin) = t(nbreak) iorder(ibkmin) = iorder(nbreak) endif c Update heap structure of breakpoints c (if iter=2, initialize heap). endif call hpsolb(nleft,t,iorder,iter-2) tj = t(nleft) ibp = iorder(nleft) endif dt = tj - tj0 if (dt .ne. zero .and. iprint .ge. 100) then write (6,4011) nint,f1,f2 write (6,5010) dt write (6,6010) dtm endif c If a minimizer is within this interval, c locate the GCP and return. if (dtm .lt. dt) goto 888 c Otherwise fix one variable and c reset the corresponding component of d to zero. tsum = tsum + dt nleft = nleft - 1 iter = iter + 1 dibp = d(ibp) d(ibp) = zero if (dibp .gt. zero) then zibp = u(ibp) - x(ibp) xcp(ibp) = u(ibp) iwhere(ibp) = 2 else zibp = l(ibp) - x(ibp) xcp(ibp) = l(ibp) iwhere(ibp) = 1 endif if (iprint .ge. 100) write (6,*) 'Variable ',ibp,' is fixed.' if (nleft .eq. 0 .and. nbreak .eq. n) then c all n variables are fixed, c return with xcp as GCP. dtm = dt goto 999 endif c Update the derivative information. nint = nint + 1 dibp2 = dibp**2 c Update f1 and f2. c temporarily set f1 and f2 for col=0. f1 = f1 + dt*f2 + dibp2 - theta*dibp*zibp f2 = f2 - theta*dibp2 if (col .gt. 0) then c update c = c + dt*p. call daxpy(col2,dt,p,1,c,1) c choose wbp, c the row of W corresponding to the breakpoint encountered. pointr = head do 70 j = 1,col wbp(j) = wy(ibp,pointr) wbp(col + j) = theta*ws(ibp,pointr) pointr = mod(pointr,m) + 1 70 continue c compute (wbp)Mc, (wbp)Mp, and (wbp)M(wbp)'. call bmv(m,sy,wt,col,wbp,v,info) if (info .ne. 0) return wmc = ddot(col2,c,1,v,1) wmp = ddot(col2,p,1,v,1) wmw = ddot(col2,wbp,1,v,1) c update p = p - dibp*wbp. call daxpy(col2,-dibp,wbp,1,p,1) c complete updating f1 and f2 while col > 0. f1 = f1 + dibp*wmc f2 = f2 + 2.0d0*dibp*wmp - dibp2*wmw endif f2 = max(epsmch*f2_org,f2) if (nleft .gt. 0) then dtm = -f1/f2 goto 777 c to repeat the loop for unsearched intervals. else if(bnded) then f1 = zero f2 = zero dtm = zero else dtm = -f1/f2 endif c------------------- the end of the loop ------------------------------- 888 continue if (iprint .ge. 99) then write (6,*) write (6,*) 'GCP found in this segment' write (6,4010) nint,f1,f2 write (6,6010) dtm endif if (dtm .le. zero) dtm = zero tsum = tsum + dtm c Move free variables (i.e., the ones w/o breakpoints) and c the variables whose breakpoints haven't been reached. call daxpy(n,tsum,d,1,xcp,1) 999 continue c Update c = c + dtm*p = W'(x^c - x) c which will be used in computing r = Z'(B(x^c - x) + g). if (col .gt. 0) call daxpy(col2,dtm,p,1,c,1) if (iprint .gt. 100) write (6,1010) (xcp(i),i = 1,n) if (iprint .ge. 99) write (6,2010) 1010 format ('Cauchy X = ',/,(4x,1p,6(1x,d11.4))) 2010 format (/,'---------------- exit CAUCHY----------------------',/) 3010 format (/,'---------------- CAUCHY entered-------------------') 4010 format ('Piece ',i3,' --f1, f2 at start point ',1p,2(1x,d11.4)) 4011 format (/,'Piece ',i3,' --f1, f2 at start point ', + 1p,2(1x,d11.4)) 5010 format ('Distance to the next break point = ',1p,d11.4) 6010 format ('Distance to the stationary point = ',1p,d11.4) return end c====================== The end of cauchy ============================== subroutine cmprlb(n, m, x, g, ws, wy, sy, wt, z, r, wa, index, + theta, col, head, nfree, cnstnd, info) logical cnstnd integer n, m, col, head, nfree, info, index(n) double precision theta, + x(n), g(n), z(n), r(n), wa(4*m), + ws(n, m), wy(n, m), sy(m, m), wt(m, m) c ************ c c Subroutine cmprlb c c This subroutine computes r=-Z'B(xcp-xk)-Z'g by using c wa(2m+1)=W'(xcp-x) from subroutine cauchy. c c Subprograms called: c c L-BFGS-B Library ... bmv. c c c * * * c c NEOS, November 1994. (Latest revision June 1996.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************ integer i,j,k,pointr double precision a1,a2 if (.not. cnstnd .and. col .gt. 0) then do 26 i = 1, n r(i) = -g(i) 26 continue else do 30 i = 1, nfree k = index(i) r(i) = -theta*(z(k) - x(k)) - g(k) 30 continue call bmv(m,sy,wt,col,wa(2*m+1),wa(1),info) if (info .ne. 0) then info = -8 return endif pointr = head do 34 j = 1, col a1 = wa(j) a2 = theta*wa(col + j) do 32 i = 1, nfree k = index(i) r(i) = r(i) + wy(k,pointr)*a1 + ws(k,pointr)*a2 32 continue pointr = mod(pointr,m) + 1 34 continue endif return end c======================= The end of cmprlb ============================= subroutine errclb(n, m, factr, l, u, nbd, task, info, k) character*60 task integer n, m, info, k, nbd(n) double precision factr, l(n), u(n) c ************ c c Subroutine errclb c c This subroutine checks the validity of the input data. c c c * * * c c NEOS, November 1994. (Latest revision April 1997.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************ integer i double precision zero parameter (zero=0.0d0) c Check the input arguments for errors. if (n .le. 0) task = 'ERROR: N .LE. 0' if (m .le. 0) task = 'ERROR: M .LE. 0' if (factr .lt. zero) task = 'ERROR: FACTR .LT. 0' c Check the validity of the arrays nbd(i), u(i), and l(i). do 10 i = 1, n if (nbd(i) .lt. 0 .or. nbd(i) .gt. 3) then c return task = 'ERROR: INVALID NBD' info = -6 k = i endif if (nbd(i) .eq. 2) then if (l(i) .gt. u(i)) then c return task = 'ERROR: NO FEASIBLE SOLUTION' info = -7 k = i endif endif 10 continue return end c======================= The end of errclb ============================= subroutine formk(n, nsub, ind, nenter, ileave, indx2, iupdat, + updatd, wn, wn1, m, ws, wy, sy, theta, col, + head, info) integer n, nsub, m, col, head, nenter, ileave, iupdat, + info, ind(n), indx2(n) double precision theta, wn(2*m, 2*m), wn1(2*m, 2*m), + ws(n, m), wy(n, m), sy(m, m) logical updatd c ************ c c Subroutine formk c c This subroutine forms the LEL^T factorization of the indefinite c c matrix K = [-D -Y'ZZ'Y/theta L_a'-R_z' ] c [L_a -R_z theta*S'AA'S ] c where E = [-I 0] c [ 0 I] c The matrix K can be shown to be equal to the matrix M^[-1]N c occurring in section 5.1 of [1], as well as to the matrix c Mbar^[-1] Nbar in section 5.3. c c n is an integer variable. c On entry n is the dimension of the problem. c On exit n is unchanged. c c nsub is an integer variable c On entry nsub is the number of subspace variables in free set. c On exit nsub is not changed. c c ind is an integer array of dimension nsub. c On entry ind specifies the indices of subspace variables. c On exit ind is unchanged. c c nenter is an integer variable. c On entry nenter is the number of variables entering the c free set. c On exit nenter is unchanged. c c ileave is an integer variable. c On entry indx2(ileave),...,indx2(n) are the variables leaving c the free set. c On exit ileave is unchanged. c c indx2 is an integer array of dimension n. c On entry indx2(1),...,indx2(nenter) are the variables entering c the free set, while indx2(ileave),...,indx2(n) are the c variables leaving the free set. c On exit indx2 is unchanged. c c iupdat is an integer variable. c On entry iupdat is the total number of BFGS updates made so far. c On exit iupdat is unchanged. c c updatd is a logical variable. c On entry 'updatd' is true if the L-BFGS matrix is updatd. c On exit 'updatd' is unchanged. c c wn is a double precision array of dimension 2m x 2m. c On entry wn is unspecified. c On exit the upper triangle of wn stores the LEL^T factorization c of the 2*col x 2*col indefinite matrix c [-D -Y'ZZ'Y/theta L_a'-R_z' ] c [L_a -R_z theta*S'AA'S ] c c wn1 is a double precision array of dimension 2m x 2m. c On entry wn1 stores the lower triangular part of c [Y' ZZ'Y L_a'+R_z'] c [L_a+R_z S'AA'S ] c in the previous iteration. c On exit wn1 stores the corresponding updated matrices. c The purpose of wn1 is just to store these inner products c so they can be easily updated and inserted into wn. c c m is an integer variable. c On entry m is the maximum number of variable metric corrections c used to define the limited memory matrix. c On exit m is unchanged. c c ws, wy, sy, and wtyy are double precision arrays; c theta is a double precision variable; c col is an integer variable; c head is an integer variable. c On entry they store the information defining the c limited memory BFGS matrix: c ws(n,m) stores S, a set of s-vectors; c wy(n,m) stores Y, a set of y-vectors; c sy(m,m) stores S'Y; c wtyy(m,m) stores the Cholesky factorization c of (theta*S'S+LD^(-1)L') c theta is the scaling factor specifying B_0 = theta I; c col is the number of variable metric corrections stored; c head is the location of the 1st s- (or y-) vector in S (or Y). c On exit they are unchanged. c c info is an integer variable. c On entry info is unspecified. c On exit info = 0 for normal return; c = -1 when the 1st Cholesky factorization failed; c = -2 when the 2st Cholesky factorization failed. c c Subprograms called: c c Linpack ... dcopy, dpofa, dtrsl. c c c References: c [1] R. H. Byrd, P. Lu, J. Nocedal and C. Zhu, ``A limited c memory algorithm for bound constrained optimization'', c SIAM J. Scientific Computing 16 (1995), no. 5, pp. 1190--1208. c c [2] C. Zhu, R.H. Byrd, P. Lu, J. Nocedal, ``L-BFGS-B: a c limited memory FORTRAN code for solving bound constrained c optimization problems'', Tech. Report, NAM-11, EECS Department, c Northwestern University, 1994. c c (Postscript files of these papers are available via anonymous c ftp to ece.nwu.edu in the directory pub/lbfgs/lbfgs_bcm.) c c * * * c c NEOS, November 1994. (Latest revision April 1997.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************ integer m2,ipntr,jpntr,iy,is,jy,js,is1,js1,k1,i,k, + col2,pbegin,pend,dbegin,dend,upcl double precision ddot,temp1,temp2,temp3,temp4 double precision zero parameter (zero=0.0d0) c Form the lower triangular part of c WN1 = [Y' ZZ'Y L_a'+R_z'] c [L_a+R_z S'AA'S ] c where L_a is the strictly lower triangular part of S'AA'Y c R_z is the upper triangular part of S'ZZ'Y. if (updatd) then if (iupdat .gt. m) then c shift old part of WN1. do 10 jy = 1, m - 1 js = m + jy call dcopy(m-jy,wn1(jy+1,jy+1),1,wn1(jy,jy),1) call dcopy(m-jy,wn1(js+1,js+1),1,wn1(js,js),1) call dcopy(m-1,wn1(m+2,jy+1),1,wn1(m+1,jy),1) 10 continue endif c put new rows in blocks (1,1), (2,1) and (2,2). pbegin = 1 pend = nsub dbegin = nsub + 1 dend = n iy = col is = m + col ipntr = head + col - 1 if (ipntr .gt. m) ipntr = ipntr - m jpntr = head do 20 jy = 1, col js = m + jy temp1 = zero temp2 = zero temp3 = zero c compute element jy of row 'col' of Y'ZZ'Y do 15 k = pbegin, pend k1 = ind(k) temp1 = temp1 + wy(k1,ipntr)*wy(k1,jpntr) 15 continue c compute elements jy of row 'col' of L_a and S'AA'S do 16 k = dbegin, dend k1 = ind(k) temp2 = temp2 + ws(k1,ipntr)*ws(k1,jpntr) temp3 = temp3 + ws(k1,ipntr)*wy(k1,jpntr) 16 continue wn1(iy,jy) = temp1 wn1(is,js) = temp2 wn1(is,jy) = temp3 jpntr = mod(jpntr,m) + 1 20 continue c put new column in block (2,1). jy = col jpntr = head + col - 1 if (jpntr .gt. m) jpntr = jpntr - m ipntr = head do 30 i = 1, col is = m + i temp3 = zero c compute element i of column 'col' of R_z do 25 k = pbegin, pend k1 = ind(k) temp3 = temp3 + ws(k1,ipntr)*wy(k1,jpntr) 25 continue ipntr = mod(ipntr,m) + 1 wn1(is,jy) = temp3 30 continue upcl = col - 1 else upcl = col endif c modify the old parts in blocks (1,1) and (2,2) due to changes c in the set of free variables. ipntr = head do 45 iy = 1, upcl is = m + iy jpntr = head do 40 jy = 1, iy js = m + jy temp1 = zero temp2 = zero temp3 = zero temp4 = zero do 35 k = 1, nenter k1 = indx2(k) temp1 = temp1 + wy(k1,ipntr)*wy(k1,jpntr) temp2 = temp2 + ws(k1,ipntr)*ws(k1,jpntr) 35 continue do 36 k = ileave, n k1 = indx2(k) temp3 = temp3 + wy(k1,ipntr)*wy(k1,jpntr) temp4 = temp4 + ws(k1,ipntr)*ws(k1,jpntr) 36 continue wn1(iy,jy) = wn1(iy,jy) + temp1 - temp3 wn1(is,js) = wn1(is,js) - temp2 + temp4 jpntr = mod(jpntr,m) + 1 40 continue ipntr = mod(ipntr,m) + 1 45 continue c modify the old parts in block (2,1). ipntr = head do 60 is = m + 1, m + upcl jpntr = head do 55 jy = 1, upcl temp1 = zero temp3 = zero do 50 k = 1, nenter k1 = indx2(k) temp1 = temp1 + ws(k1,ipntr)*wy(k1,jpntr) 50 continue do 51 k = ileave, n k1 = indx2(k) temp3 = temp3 + ws(k1,ipntr)*wy(k1,jpntr) 51 continue if (is .le. jy + m) then wn1(is,jy) = wn1(is,jy) + temp1 - temp3 else wn1(is,jy) = wn1(is,jy) - temp1 + temp3 endif jpntr = mod(jpntr,m) + 1 55 continue ipntr = mod(ipntr,m) + 1 60 continue c Form the upper triangle of WN = [D+Y' ZZ'Y/theta -L_a'+R_z' ] c [-L_a +R_z S'AA'S*theta] m2 = 2*m do 70 iy = 1, col is = col + iy is1 = m + iy do 65 jy = 1, iy js = col + jy js1 = m + jy wn(jy,iy) = wn1(iy,jy)/theta wn(js,is) = wn1(is1,js1)*theta 65 continue do 66 jy = 1, iy - 1 wn(jy,is) = -wn1(is1,jy) 66 continue do 67 jy = iy, col wn(jy,is) = wn1(is1,jy) 67 continue wn(iy,iy) = wn(iy,iy) + sy(iy,iy) 70 continue c Form the upper triangle of c WN= [ LL' L^-1(-L_a'+R_z')] c [(-L_a +R_z)L'^-1 S'AA'S*theta ] c first Cholesky factor (1,1) block of wn to get LL' c with L' stored in the upper triangle of wn. call dpofa(wn,m2,col,info) if (info .ne. 0) then info = -1 return endif c then form L^-1(-L_a'+R_z') in the (1,2) block. col2 = 2*col do 71 js = col+1 ,col2 call dtrsl(wn,m2,col,wn(1,js),11,info) 71 continue c Form S'AA'S*theta + (L^-1(-L_a'+R_z'))'L^-1(-L_a'+R_z') in the c upper triangle of (2,2) block of wn. do 72 is = col+1, col2 do 74 js = is, col2 wn(is,js) = wn(is,js) + ddot(col,wn(1,is),1,wn(1,js),1) 74 continue 72 continue c Cholesky factorization of (2,2) block of wn. call dpofa(wn(col+1,col+1),m2,col,info) if (info .ne. 0) then info = -2 return endif return end c======================= The end of formk ============================== subroutine formt(m, wt, sy, ss, col, theta, info) integer m, col, info double precision theta, wt(m, m), sy(m, m), ss(m, m) c ************ c c Subroutine formt c c This subroutine forms the upper half of the pos. def. and symm. c T = theta*SS + L*D^(-1)*L', stores T in the upper triangle c of the array wt, and performs the Cholesky factorization of T c to produce J*J', with J' stored in the upper triangle of wt. c c Subprograms called: c c Linpack ... dpofa. c c c * * * c c NEOS, November 1994. (Latest revision June 1996.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************ integer i,j,k,k1 double precision ddum double precision zero parameter (zero=0.0d0) c Form the upper half of T = theta*SS + L*D^(-1)*L', c store T in the upper triangle of the array wt. do 52 j = 1, col wt(1,j) = theta*ss(1,j) 52 continue do 55 i = 2, col do 54 j = i, col k1 = min(i,j) - 1 ddum = zero do 53 k = 1, k1 ddum = ddum + sy(i,k)*sy(j,k)/sy(k,k) 53 continue wt(i,j) = ddum + theta*ss(i,j) 54 continue 55 continue c Cholesky factorize T to J*J' with c J' stored in the upper triangle of wt. call dpofa(wt,m,col,info) if (info .ne. 0) then info = -3 endif return end c======================= The end of formt ============================== subroutine freev(n, nfree, index, nenter, ileave, indx2, + iwhere, wrk, updatd, cnstnd, iprint, iter) integer n, nfree, nenter, ileave, iprint, iter, + index(n), indx2(n), iwhere(n) logical wrk, updatd, cnstnd c ************ c c Subroutine freev c c This subroutine counts the entering and leaving variables when c iter > 0, and finds the index set of free and active variables c at the GCP. c c cnstnd is a logical variable indicating whether bounds are present c c index is an integer array of dimension n c for i=1,...,nfree, index(i) are the indices of free variables c for i=nfree+1,...,n, index(i) are the indices of bound variables c On entry after the first iteration, index gives c the free variables at the previous iteration. c On exit it gives the free variables based on the determination c in cauchy using the array iwhere. c c indx2 is an integer array of dimension n c On entry indx2 is unspecified. c On exit with iter>0, indx2 indicates which variables c have changed status since the previous iteration. c For i= 1,...,nenter, indx2(i) have changed from bound to free. c For i= ileave+1,...,n, indx2(i) have changed from free to bound. c c c * * * c c NEOS, November 1994. (Latest revision June 1996.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************ integer iact,i,k nenter = 0 ileave = n + 1 if (iter .gt. 0 .and. cnstnd) then c count the entering and leaving variables. do 20 i = 1, nfree k = index(i) if (iwhere(k) .gt. 0) then ileave = ileave - 1 indx2(ileave) = k if (iprint .ge. 100) write (6,*) + 'Variable ',k,' leaves the set of free variables' endif 20 continue do 22 i = 1 + nfree, n k = index(i) if (iwhere(k) .le. 0) then nenter = nenter + 1 indx2(nenter) = k if (iprint .ge. 100) write (6,*) + 'Variable ',k,' enters the set of free variables' endif 22 continue if (iprint .ge. 99) write (6,*) + n+1-ileave,' variables leave; ',nenter,' variables enter' endif wrk = (ileave .lt. n+1) .or. (nenter .gt. 0) .or. updatd c Find the index set of free and active variables at the GCP. nfree = 0 iact = n + 1 do 24 i = 1, n if (iwhere(i) .le. 0) then nfree = nfree + 1 index(nfree) = i else iact = iact - 1 index(iact) = i endif 24 continue if (iprint .ge. 99) write (6,*) + nfree,' variables are free at GCP ',iter + 1 return end c======================= The end of freev ============================== subroutine hpsolb(n, t, iorder, iheap) integer iheap, n, iorder(n) double precision t(n) c ************ c c Subroutine hpsolb c c This subroutine sorts out the least element of t, and puts the c remaining elements of t in a heap. c c n is an integer variable. c On entry n is the dimension of the arrays t and iorder. c On exit n is unchanged. c c t is a double precision array of dimension n. c On entry t stores the elements to be sorted, c On exit t(n) stores the least elements of t, and t(1) to t(n-1) c stores the remaining elements in the form of a heap. c c iorder is an integer array of dimension n. c On entry iorder(i) is the index of t(i). c On exit iorder(i) is still the index of t(i), but iorder may be c permuted in accordance with t. c c iheap is an integer variable specifying the task. c On entry iheap should be set as follows: c iheap .eq. 0 if t(1) to t(n) is not in the form of a heap, c iheap .ne. 0 if otherwise. c On exit iheap is unchanged. c c c References: c Algorithm 232 of CACM (J. W. J. Williams): HEAPSORT. c c * * * c c NEOS, November 1994. (Latest revision June 1996.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c ************ integer i,j,k,indxin,indxou double precision ddum,out if (iheap .eq. 0) then c Rearrange the elements t(1) to t(n) to form a heap. do 20 k = 2, n ddum = t(k) indxin = iorder(k) c Add ddum to the heap. i = k 10 continue if (i.gt.1) then j = i/2 if (ddum .lt. t(j)) then t(i) = t(j) iorder(i) = iorder(j) i = j goto 10 endif endif t(i) = ddum iorder(i) = indxin 20 continue endif c Assign to 'out' the value of t(1), the least member of the heap, c and rearrange the remaining members to form a heap as c elements 1 to n-1 of t. if (n .gt. 1) then i = 1 out = t(1) indxou = iorder(1) ddum = t(n) indxin = iorder(n) c Restore the heap 30 continue j = i+i if (j .le. n-1) then if (t(j+1) .lt. t(j)) j = j+1 if (t(j) .lt. ddum ) then t(i) = t(j) iorder(i) = iorder(j) i = j goto 30 endif endif t(i) = ddum iorder(i) = indxin c Put the least member in t(n). t(n) = out iorder(n) = indxou endif return end c====================== The end of hpsolb ============================== subroutine lnsrlb(n, l, u, nbd, x, f, fold, gd, gdold, g, d, r, t, + z, stp, dnorm, dtd, xstep, stpmx, iter, ifun, + iback, nfgv, info, task, boxed, cnstnd, csave, + isave, dsave) character*60 task, csave logical boxed, cnstnd integer n, iter, ifun, iback, nfgv, info, + nbd(n), isave(2) double precision f, fold, gd, gdold, stp, dnorm, dtd, xstep, + stpmx, x(n), l(n), u(n), g(n), d(n), r(n), t(n), + z(n), dsave(13) c ********** c c Subroutine lnsrlb c c This subroutine calls subroutine dcsrch from the Minpack2 library c to perform the line search. Subroutine dscrch is safeguarded so c that all trial points lie within the feasible region. c c Subprograms called: c c Minpack2 Library ... dcsrch. c c Linpack ... dtrsl, ddot. c c c * * * c c NEOS, November 1994. (Latest revision June 1996.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ********** integer i double precision ddot,a1,a2 double precision one,zero,big parameter (one=1.0d0,zero=0.0d0,big=1.0d+10) double precision ftol,gtol,xtol parameter (ftol=1.0d-3,gtol=0.9d0,xtol=0.1d0) if (task(1:5) .eq. 'FG_LN') goto 556 dtd = ddot(n,d,1,d,1) dnorm = sqrt(dtd) c Determine the maximum step length. stpmx = big if (cnstnd) then if (iter .eq. 0) then stpmx = one else do 43 i = 1, n a1 = d(i) if (nbd(i) .ne. 0) then if (a1 .lt. zero .and. nbd(i) .le. 2) then a2 = l(i) - x(i) if (a2 .ge. zero) then stpmx = zero else if (a1*stpmx .lt. a2) then stpmx = a2/a1 endif else if (a1 .gt. zero .and. nbd(i) .ge. 2) then a2 = u(i) - x(i) if (a2 .le. zero) then stpmx = zero else if (a1*stpmx .gt. a2) then stpmx = a2/a1 endif endif endif 43 continue endif endif if (iter .eq. 0 .and. .not. boxed) then stp = min(one/dnorm, stpmx) else stp = one endif call dcopy(n,x,1,t,1) call dcopy(n,g,1,r,1) fold = f ifun = 0 iback = 0 csave = 'START' 556 continue gd = ddot(n,g,1,d,1) if (ifun .eq. 0) then gdold=gd if (gd .ge. zero) then c the directional derivative >=0. c Line search is impossible. info = -4 return endif endif call dcsrch(f,gd,stp,ftol,gtol,xtol,zero,stpmx,csave,isave,dsave) xstep = stp*dnorm if (csave(1:4) .ne. 'CONV' .and. csave(1:4) .ne. 'WARN') then task = 'FG_LNSRCH' ifun = ifun + 1 nfgv = nfgv + 1 iback = ifun - 1 if (stp .eq. one) then call dcopy(n,z,1,x,1) else do 41 i = 1, n x(i) = stp*d(i) + t(i) 41 continue endif else task = 'NEW_X' endif return end c======================= The end of lnsrlb ============================= subroutine matupd(n, m, ws, wy, sy, ss, d, r, itail, + iupdat, col, head, theta, rr, dr, stp, dtd) integer n, m, itail, iupdat, col, head double precision theta, rr, dr, stp, dtd, d(n), r(n), + ws(n, m), wy(n, m), sy(m, m), ss(m, m) c ************ c c Subroutine matupd c c This subroutine updates matrices WS and WY, and forms the c middle matrix in B. c c Subprograms called: c c Linpack ... dcopy, ddot. c c c * * * c c NEOS, November 1994. (Latest revision June 1996.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************ integer j,pointr double precision ddot double precision one parameter (one=1.0d0) c Set pointers for matrices WS and WY. if (iupdat .le. m) then col = iupdat itail = mod(head+iupdat-2,m) + 1 else itail = mod(itail,m) + 1 head = mod(head,m) + 1 endif c Update matrices WS and WY. call dcopy(n,d,1,ws(1,itail),1) call dcopy(n,r,1,wy(1,itail),1) c Set theta=yy/ys. theta = rr/dr c Form the middle matrix in B. c update the upper triangle of SS, c and the lower triangle of SY: if (iupdat .gt. m) then c move old information do 50 j = 1, col - 1 call dcopy(j,ss(2,j+1),1,ss(1,j),1) call dcopy(col-j,sy(j+1,j+1),1,sy(j,j),1) 50 continue endif c add new information: the last row of SY c and the last column of SS: pointr = head do 51 j = 1, col - 1 sy(col,j) = ddot(n,d,1,wy(1,pointr),1) ss(j,col) = ddot(n,ws(1,pointr),1,d,1) pointr = mod(pointr,m) + 1 51 continue if (stp .eq. one) then ss(col,col) = dtd else ss(col,col) = stp*stp*dtd endif sy(col,col) = dr return end c======================= The end of matupd ============================= subroutine prn1lb(n, m, l, u, x, iprint, itfile, epsmch) integer n, m, iprint, itfile double precision epsmch, x(n), l(n), u(n) c ************ c c Subroutine prn1lb c c This subroutine prints the input data, initial point, upper and c lower bounds of each variable, machine precision, as well as c the headings of the output. c c c * * * c c NEOS, November 1994. (Latest revision June 1996.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************ integer i if (iprint .ge. 0) then write (6,7001) epsmch write (6,*) 'N = ',n,' M = ',m if (iprint .ge. 1) then write (itfile,2001) epsmch write (itfile,*)'N = ',n,' M = ',m write (itfile,9001) if (iprint .gt. 100) then write (6,1004) 'L =',(l(i),i = 1,n) write (6,1004) 'X0 =',(x(i),i = 1,n) write (6,1004) 'U =',(u(i),i = 1,n) endif endif endif 1004 format (/,a4, 1p, 6(1x,d11.4),/,(4x,1p,6(1x,d11.4))) 2001 format ('RUNNING THE L-BFGS-B CODE',/,/, + 'it = iteration number',/, + 'nf = number of function evaluations',/, + 'nint = number of segments explored during the Cauchy search',/, + 'nact = number of active bounds at the generalized Cauchy point' + ,/, + 'sub = manner in which the subspace minimization terminated:' + ,/,' con = converged, bnd = a bound was reached',/, + 'itls = number of iterations performed in the line search',/, + 'stepl = step length used',/, + 'tstep = norm of the displacement (total step)',/, + 'projg = norm of the projected gradient',/, + 'f = function value',/,/, + ' * * *',/,/, + 'Machine precision =',1p,d10.3) 7001 format ('RUNNING THE L-BFGS-B CODE',/,/, + ' * * *',/,/, + 'Machine precision =',1p,d10.3) 9001 format (/,3x,'it',3x,'nf',2x,'nint',2x,'nact',2x,'sub',2x,'itls', + 2x,'stepl',4x,'tstep',5x,'projg',8x,'f') return end c======================= The end of prn1lb ============================= subroutine prn2lb(n, x, f, g, iprint, itfile, iter, nfgv, nact, + sbgnrm, nint, word, iword, iback, stp, xstep) character*3 word integer n, iprint, itfile, iter, nfgv, nact, nint, + iword, iback double precision f, sbgnrm, stp, xstep, x(n), g(n) c ************ c c Subroutine prn2lb c c This subroutine prints out new information after a successful c line search. c c c * * * c c NEOS, November 1994. (Latest revision June 1996.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************ integer i,imod c 'word' records the status of subspace solutions. if (iword .eq. 0) then c the subspace minimization converged. word = 'con' else if (iword .eq. 1) then c the subspace minimization stopped at a bound. word = 'bnd' else if (iword .eq. 5) then c the truncated Newton step has been used. word = 'TNT' else word = '---' endif if (iprint .ge. 99) then write (6,*) 'LINE SEARCH',iback,' times; norm of step = ',xstep write (6,2001) iter,f,sbgnrm if (iprint .gt. 100) then write (6,1004) 'X =',(x(i), i = 1, n) write (6,1004) 'G =',(g(i), i = 1, n) endif else if (iprint .gt. 0) then imod = mod(iter,iprint) if (imod .eq. 0) write (6,2001) iter,f,sbgnrm endif if (iprint .ge. 1) write (itfile,3001) + iter,nfgv,nint,nact,word,iback,stp,xstep,sbgnrm,f 1004 format (/,a4, 1p, 6(1x,d11.4),/,(4x,1p,6(1x,d11.4))) 2001 format + (/,'At iterate',i5,4x,'f= ',1p,d12.5,4x,'|proj g|= ',1p,d12.5) 3001 format(2(1x,i4),2(1x,i5),2x,a3,1x,i4,1p,2(2x,d7.1),1p,2(1x,d10.3)) return end c======================= The end of prn2lb ============================= subroutine prn3lb(n, x, f, task, iprint, info, itfile, + iter, nfgv, nintol, nskip, nact, sbgnrm, + time, nint, word, iback, stp, xstep, k, + cachyt, sbtime, lnscht) character*60 task character*3 word integer n, iprint, info, itfile, iter, nfgv, nintol, + nskip, nact, nint, iback, k double precision f, sbgnrm, time, stp, xstep, cachyt, sbtime, + lnscht, x(n) c ************ c c Subroutine prn3lb c c This subroutine prints out information when either a built-in c convergence test is satisfied or when an error message is c generated. c c c * * * c c NEOS, November 1994. (Latest revision April 1997.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************ integer i if (task(1:5) .eq. 'ERROR') goto 999 if (iprint .ge. 0) then write (6,3003) write (6,3004) write(6,3005) n,iter,nfgv,nintol,nskip,nact,sbgnrm,f if (iprint .ge. 100) then write (6,1004) 'X =',(x(i),i = 1,n) endif if (iprint .ge. 1) write (6,*) ' F =',f endif 999 continue if (iprint .ge. 0) then write (6,3009) task if (info .ne. 0) then if (info .eq. -1) write (6,9011) if (info .eq. -2) write (6,9012) if (info .eq. -3) write (6,9013) if (info .eq. -4) write (6,9014) if (info .eq. -5) write (6,9015) if (info .eq. -6) write (6,*)' Input nbd(',k,') is invalid.' if (info .eq. -7) + write (6,*)' l(',k,') > u(',k,'). No feasible solution.' if (info .eq. -8) write (6,9018) if (info .eq. -9) write (6,9019) endif if (iprint .ge. 1) write (6,3007) cachyt,sbtime,lnscht write (6,3008) time if (iprint .ge. 1) then if (info .eq. -4 .or. info .eq. -9) then write (itfile,3002) + iter,nfgv,nint,nact,word,iback,stp,xstep endif write (itfile,3009) task if (info .ne. 0) then if (info .eq. -1) write (itfile,9011) if (info .eq. -2) write (itfile,9012) if (info .eq. -3) write (itfile,9013) if (info .eq. -4) write (itfile,9014) if (info .eq. -5) write (itfile,9015) if (info .eq. -8) write (itfile,9018) if (info .eq. -9) write (itfile,9019) endif write (itfile,3008) time endif endif 1004 format (/,a4, 1p, 6(1x,d11.4),/,(4x,1p,6(1x,d11.4))) 3002 format(2(1x,i4),2(1x,i5),2x,a3,1x,i4,1p,2(2x,d7.1),6x,'-',10x,'-') 3003 format (/, + ' * * *',/,/, + 'Tit = total number of iterations',/, + 'Tnf = total number of function evaluations',/, + 'Tnint = total number of segments explored during', + ' Cauchy searches',/, + 'Skip = number of BFGS updates skipped',/, + 'Nact = number of active bounds at final generalized', + ' Cauchy point',/, + 'Projg = norm of the final projected gradient',/, + 'F = final function value',/,/, + ' * * *') 3004 format (/,3x,'N',3x,'Tit',2x,'Tnf',2x,'Tnint',2x, + 'Skip',2x,'Nact',5x,'Projg',8x,'F') 3005 format (i5,2(1x,i4),(1x,i6),(2x,i4),(1x,i5),1p,2(2x,d10.3)) 3007 format (/,' Cauchy time',1p,e10.3,' seconds.',/ + ' Subspace minimization time',1p,e10.3,' seconds.',/ + ' Line search time',1p,e10.3,' seconds.') 3008 format (/,' Total User time',1p,e10.3,' seconds.',/) 3009 format (/,a60) 9011 format (/, +' Matrix in 1st Cholesky factorization in formk is not Pos. Def.') 9012 format (/, +' Matrix in 2st Cholesky factorization in formk is not Pos. Def.') 9013 format (/, +' Matrix in the Cholesky factorization in formt is not Pos. Def.') 9014 format (/, +' Derivative >= 0, backtracking line search impossible.',/, +' Previous x, f and g restored.',/, +' Possible causes: 1 error in function or gradient evaluation;',/, +' 2 rounding errors dominate computation.') 9015 format (/, +' Warning: more than 10 function and gradient',/, +' evaluations in the last line search. Termination',/, +' may possibly be caused by a bad search direction.') 9018 format (/,' The triangular system is singular.') 9019 format (/, +' Line search cannot locate an adequate point after 20 function',/ +,' and gradient evaluations. Previous x, f and g restored.',/, +' Possible causes: 1 error in function or gradient evaluation;',/, +' 2 rounding error dominate computation.') return end c======================= The end of prn3lb ============================= subroutine projgr(n, l, u, nbd, x, g, sbgnrm) integer n, nbd(n) double precision sbgnrm, x(n), l(n), u(n), g(n) c ************ c c Subroutine projgr c c This subroutine computes the infinity norm of the projected c gradient. c c c * * * c c NEOS, November 1994. (Latest revision April 1997.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************ integer i double precision gi double precision zero parameter (zero=0.0d0) sbgnrm = zero do 15 i = 1, n gi = g(i) if (nbd(i) .ne. 0) then if (gi .lt. zero) then if (nbd(i) .ge. 2) gi = max((x(i)-u(i)),gi) else if (nbd(i) .le. 2) gi = min((x(i)-l(i)),gi) endif endif sbgnrm = max(sbgnrm,abs(gi)) 15 continue return end c======================= The end of projgr ============================= subroutine subsm(n, m, nsub, ind, l, u, nbd, x, d, ws, wy, theta, + col, head, iword, wv, wn, iprint, info) integer n, m, nsub, col, head, iword, iprint, info, + ind(nsub), nbd(n) double precision theta, + l(n), u(n), x(n), d(n), + ws(n, m), wy(n, m), + wv(2*m), wn(2*m, 2*m) c ************ c c Subroutine subsm c c Given xcp, l, u, r, an index set that specifies c the active set at xcp, and an l-BFGS matrix B c (in terms of WY, WS, SY, WT, head, col, and theta), c this subroutine computes an approximate solution c of the subspace problem c c (P) min Q(x) = r'(x-xcp) + 1/2 (x-xcp)' B (x-xcp) c c subject to l<=x<=u c x_i=xcp_i for all i in A(xcp) c c along the subspace unconstrained Newton direction c c d = -(Z'BZ)^(-1) r. c c The formula for the Newton direction, given the L-BFGS matrix c and the Sherman-Morrison formula, is c c d = (1/theta)r + (1/theta*2) Z'WK^(-1)W'Z r. c c where c K = [-D -Y'ZZ'Y/theta L_a'-R_z' ] c [L_a -R_z theta*S'AA'S ] c c Note that this procedure for computing d differs c from that described in [1]. One can show that the matrix K is c equal to the matrix M^[-1]N in that paper. c c n is an integer variable. c On entry n is the dimension of the problem. c On exit n is unchanged. c c m is an integer variable. c On entry m is the maximum number of variable metric corrections c used to define the limited memory matrix. c On exit m is unchanged. c c nsub is an integer variable. c On entry nsub is the number of free variables. c On exit nsub is unchanged. c c ind is an integer array of dimension nsub. c On entry ind specifies the coordinate indices of free variables. c On exit ind is unchanged. c c l is a double precision array of dimension n. c On entry l is the lower bound of x. c On exit l is unchanged. c c u is a double precision array of dimension n. c On entry u is the upper bound of x. c On exit u is unchanged. c c nbd is a integer array of dimension n. c On entry nbd represents the type of bounds imposed on the c variables, and must be specified as follows: c nbd(i)=0 if x(i) is unbounded, c 1 if x(i) has only a lower bound, c 2 if x(i) has both lower and upper bounds, and c 3 if x(i) has only an upper bound. c On exit nbd is unchanged. c c x is a double precision array of dimension n. c On entry x specifies the Cauchy point xcp. c On exit x(i) is the minimizer of Q over the subspace of c free variables. c c d is a double precision array of dimension n. c On entry d is the reduced gradient of Q at xcp. c On exit d is the Newton direction of Q. c c ws and wy are double precision arrays; c theta is a double precision variable; c col is an integer variable; c head is an integer variable. c On entry they store the information defining the c limited memory BFGS matrix: c ws(n,m) stores S, a set of s-vectors; c wy(n,m) stores Y, a set of y-vectors; c theta is the scaling factor specifying B_0 = theta I; c col is the number of variable metric corrections stored; c head is the location of the 1st s- (or y-) vector in S (or Y). c On exit they are unchanged. c c iword is an integer variable. c On entry iword is unspecified. c On exit iword specifies the status of the subspace solution. c iword = 0 if the solution is in the box, c 1 if some bound is encountered. c c wv is a double precision working array of dimension 2m. c c wn is a double precision array of dimension 2m x 2m. c On entry the upper triangle of wn stores the LEL^T factorization c of the indefinite matrix c c K = [-D -Y'ZZ'Y/theta L_a'-R_z' ] c [L_a -R_z theta*S'AA'S ] c where E = [-I 0] c [ 0 I] c On exit wn is unchanged. c c iprint is an INTEGER variable that must be set by the user. c It controls the frequency and type of output generated: c iprint<0 no output is generated; c iprint=0 print only one line at the last iteration; c 0100 print details of every iteration including x and g; c When iprint > 0, the file iterate.dat will be created to c summarize the iteration. c c info is an integer variable. c On entry info is unspecified. c On exit info = 0 for normal return, c = nonzero for abnormal return c when the matrix K is ill-conditioned. c c Subprograms called: c c Linpack dtrsl. c c c References: c c [1] R. H. Byrd, P. Lu, J. Nocedal and C. Zhu, ``A limited c memory algorithm for bound constrained optimization'', c SIAM J. Scientific Computing 16 (1995), no. 5, pp. 1190--1208. c c c c * * * c c NEOS, November 1994. (Latest revision June 1996.) c Optimization Technology Center. c Argonne National Laboratory and Northwestern University. c Written by c Ciyou Zhu c in collaboration with R.H. Byrd, P. Lu-Chen and J. Nocedal. c c c ************ integer pointr,m2,col2,ibd,jy,js,i,j,k double precision alpha,dk,temp1,temp2 double precision one,zero parameter (one=1.0d0,zero=0.0d0) if (nsub .le. 0) return if (iprint .ge. 99) write (6,1001) c Compute wv = W'Zd. pointr = head do 20 i = 1, col temp1 = zero temp2 = zero do 10 j = 1, nsub k = ind(j) temp1 = temp1 + wy(k,pointr)*d(j) temp2 = temp2 + ws(k,pointr)*d(j) 10 continue wv(i) = temp1 wv(col + i) = theta*temp2 pointr = mod(pointr,m) + 1 20 continue c Compute wv:=K^(-1)wv. m2 = 2*m col2 = 2*col call dtrsl(wn,m2,col2,wv,11,info) if (info .ne. 0) return do 25 i = 1, col wv(i) = -wv(i) 25 continue call dtrsl(wn,m2,col2,wv,01,info) if (info .ne. 0) return c Compute d = (1/theta)d + (1/theta**2)Z'W wv. pointr = head do 40 jy = 1, col js = col + jy do 30 i = 1, nsub k = ind(i) d(i) = d(i) + wy(k,pointr)*wv(jy)/theta + + ws(k,pointr)*wv(js) 30 continue pointr = mod(pointr,m) + 1 40 continue do 50 i = 1, nsub d(i) = d(i)/theta 50 continue c Backtrack to the feasible region. alpha = one temp1 = alpha do 60 i = 1, nsub k = ind(i) dk = d(i) if (nbd(k) .ne. 0) then if (dk .lt. zero .and. nbd(k) .le. 2) then temp2 = l(k) - x(k) if (temp2 .ge. zero) then temp1 = zero else if (dk*alpha .lt. temp2) then temp1 = temp2/dk endif else if (dk .gt. zero .and. nbd(k) .ge. 2) then temp2 = u(k) - x(k) if (temp2 .le. zero) then temp1 = zero else if (dk*alpha .gt. temp2) then temp1 = temp2/dk endif endif if (temp1 .lt. alpha) then alpha = temp1 ibd = i endif endif 60 continue if (alpha .lt. one) then dk = d(ibd) k = ind(ibd) if (dk .gt. zero) then x(k) = u(k) d(ibd) = zero else if (dk .lt. zero) then x(k) = l(k) d(ibd) = zero endif endif do 70 i = 1, nsub k = ind(i) x(k) = x(k) + alpha*d(i) 70 continue if (iprint .ge. 99) then if (alpha .lt. one) then write (6,1002) alpha else write (6,*) 'SM solution inside the box' end if if (iprint .gt.100) write (6,1003) (x(i),i=1,n) endif if (alpha .lt. one) then iword = 1 else iword = 0 endif if (iprint .ge. 99) write (6,1004) 1001 format (/,'----------------SUBSM entered-----------------',/) 1002 format ( 'ALPHA = ',f7.5,' backtrack to the BOX') 1003 format ('Subspace solution X = ',/,(4x,1p,6(1x,d11.4))) 1004 format (/,'----------------exit SUBSM --------------------',/) return end c====================== The end of subsm =============================== subroutine dcsrch(f,g,stp,ftol,gtol,xtol,stpmin,stpmax, + task,isave,dsave) character*(*) task integer isave(2) double precision f,g,stp,ftol,gtol,xtol,stpmin,stpmax double precision dsave(13) c ********** c c Subroutine dcsrch c c This subroutine finds a step that satisfies a sufficient c decrease condition and a curvature condition. c c Each call of the subroutine updates an interval with c endpoints stx and sty. The interval is initially chosen c so that it contains a minimizer of the modified function c c psi(stp) = f(stp) - f(0) - ftol*stp*f'(0). c c If psi(stp) <= 0 and f'(stp) >= 0 for some step, then the c interval is chosen so that it contains a minimizer of f. c c The algorithm is designed to find a step that satisfies c the sufficient decrease condition c c f(stp) <= f(0) + ftol*stp*f'(0), c c and the curvature condition c c abs(f'(stp)) <= gtol*abs(f'(0)). c c If ftol is less than gtol and if, for example, the function c is bounded below, then there is always a step which satisfies c both conditions. c c If no step can be found that satisfies both conditions, then c the algorithm stops with a warning. In this case stp only c satisfies the sufficient decrease condition. c c A typical invocation of dcsrch has the following outline: c c task = 'START' c 10 continue c call dcsrch( ... ) c if (task .eq. 'FG') then c Evaluate the function and the gradient at stp c goto 10 c end if c c NOTE: The user must no alter work arrays between calls. c c The subroutine statement is c c subroutine dcsrch(f,g,stp,ftol,gtol,xtol,stpmin,stpmax, c task,isave,dsave) c where c c f is a double precision variable. c On initial entry f is the value of the function at 0. c On subsequent entries f is the value of the c function at stp. c On exit f is the value of the function at stp. c c g is a double precision variable. c On initial entry g is the derivative of the function at 0. c On subsequent entries g is the derivative of the c function at stp. c On exit g is the derivative of the function at stp. c c stp is a double precision variable. c On entry stp is the current estimate of a satisfactory c step. On initial entry, a positive initial estimate c must be provided. c On exit stp is the current estimate of a satisfactory step c if task = 'FG'. If task = 'CONV' then stp satisfies c the sufficient decrease and curvature condition. c c ftol is a double precision variable. c On entry ftol specifies a nonnegative tolerance for the c sufficient decrease condition. c On exit ftol is unchanged. c c gtol is a double precision variable. c On entry gtol specifies a nonnegative tolerance for the c curvature condition. c On exit gtol is unchanged. c c xtol is a double precision variable. c On entry xtol specifies a nonnegative relative tolerance c for an acceptable step. The subroutine exits with a c warning if the relative difference between sty and stx c is less than xtol. c On exit xtol is unchanged. c c stpmin is a double precision variable. c On entry stpmin is a nonnegative lower bound for the step. c On exit stpmin is unchanged. c c stpmax is a double precision variable. c On entry stpmax is a nonnegative upper bound for the step. c On exit stpmax is unchanged. c c task is a character variable of length at least 60. c On initial entry task must be set to 'START'. c On exit task indicates the required action: c c If task(1:2) = 'FG' then evaluate the function and c derivative at stp and call dcsrch again. c c If task(1:4) = 'CONV' then the search is successful. c c If task(1:4) = 'WARN' then the subroutine is not able c to satisfy the convergence conditions. The exit value of c stp contains the best point found during the search. c c If task(1:5) = 'ERROR' then there is an error in the c input arguments. c c On exit with convergence, a warning or an error, the c variable task contains additional information. c c isave is an integer work array of dimension 2. c c dsave is a double precision work array of dimension 13. c c Subprograms called c c MINPACK-2 ... dcstep c c MINPACK-1 Project. June 1983. c Argonne National Laboratory. c Jorge J. More' and David J. Thuente. c c MINPACK-2 Project. October 1993. c Argonne National Laboratory and University of Minnesota. c Brett M. Averick, Richard G. Carter, and Jorge J. More'. c c ********** double precision zero,p5,p66 parameter(zero=0.0d0,p5=0.5d0,p66=0.66d0) double precision xtrapl,xtrapu parameter(xtrapl=1.1d0,xtrapu=4.0d0) logical brackt integer stage double precision finit,ftest,fm,fx,fxm,fy,fym,ginit,gtest, + gm,gx,gxm,gy,gym,stx,sty,stmin,stmax,width,width1 c Initialization block. if (task(1:5) .eq. 'START') then c Check the input arguments for errors. if (stp .lt. stpmin) task = 'ERROR: STP .LT. STPMIN' if (stp .gt. stpmax) task = 'ERROR: STP .GT. STPMAX' if (g .ge. zero) task = 'ERROR: INITIAL G .GE. ZERO' if (ftol .lt. zero) task = 'ERROR: FTOL .LT. ZERO' if (gtol .lt. zero) task = 'ERROR: GTOL .LT. ZERO' if (xtol .lt. zero) task = 'ERROR: XTOL .LT. ZERO' if (stpmin .lt. zero) task = 'ERROR: STPMIN .LT. ZERO' if (stpmax .lt. stpmin) task = 'ERROR: STPMAX .LT. STPMIN' c Exit if there are errors on input. if (task(1:5) .eq. 'ERROR') return c Initialize local variables. brackt = .false. stage = 1 finit = f ginit = g gtest = ftol*ginit width = stpmax - stpmin width1 = width/p5 c The variables stx, fx, gx contain the values of the step, c function, and derivative at the best step. c The variables sty, fy, gy contain the value of the step, c function, and derivative at sty. c The variables stp, f, g contain the values of the step, c function, and derivative at stp. stx = zero fx = finit gx = ginit sty = zero fy = finit gy = ginit stmin = zero stmax = stp + xtrapu*stp task = 'FG' goto 1000 else c Restore local variables. if (isave(1) .eq. 1) then brackt = .true. else brackt = .false. endif stage = isave(2) ginit = dsave(1) gtest = dsave(2) gx = dsave(3) gy = dsave(4) finit = dsave(5) fx = dsave(6) fy = dsave(7) stx = dsave(8) sty = dsave(9) stmin = dsave(10) stmax = dsave(11) width = dsave(12) width1 = dsave(13) endif c If psi(stp) <= 0 and f'(stp) >= 0 for some step, then the c algorithm enters the second stage. ftest = finit + stp*gtest if (stage .eq. 1 .and. f .le. ftest .and. g .ge. zero) + stage = 2 c Test for warnings. if (brackt .and. (stp .le. stmin .or. stp .ge. stmax)) + task = 'WARNING: ROUNDING ERRORS PREVENT PROGRESS' if (brackt .and. stmax - stmin .le. xtol*stmax) + task = 'WARNING: XTOL TEST SATISFIED' if (stp .eq. stpmax .and. f .le. ftest .and. g .le. gtest) + task = 'WARNING: STP = STPMAX' if (stp .eq. stpmin .and. (f .gt. ftest .or. g .ge. gtest)) + task = 'WARNING: STP = STPMIN' c Test for convergence. if (f .le. ftest .and. abs(g) .le. gtol*(-ginit)) + task = 'CONVERGENCE' c Test for termination. if (task(1:4) .eq. 'WARN' .or. task(1:4) .eq. 'CONV') goto 1000 c A modified function is used to predict the step during the c first stage if a lower function value has been obtained but c the decrease is not sufficient. if (stage .eq. 1 .and. f .le. fx .and. f .gt. ftest) then c Define the modified function and derivative values. fm = f - stp*gtest fxm = fx - stx*gtest fym = fy - sty*gtest gm = g - gtest gxm = gx - gtest gym = gy - gtest c Call dcstep to update stx, sty, and to compute the new step. call dcstep(stx,fxm,gxm,sty,fym,gym,stp,fm,gm, + brackt,stmin,stmax) c Reset the function and derivative values for f. fx = fxm + stx*gtest fy = fym + sty*gtest gx = gxm + gtest gy = gym + gtest else c Call dcstep to update stx, sty, and to compute the new step. call dcstep(stx,fx,gx,sty,fy,gy,stp,f,g, + brackt,stmin,stmax) endif c Decide if a bisection step is needed. if (brackt) then if (abs(sty-stx) .ge. p66*width1) stp = stx + p5*(sty - stx) width1 = width width = abs(sty-stx) endif c Set the minimum and maximum steps allowed for stp. if (brackt) then stmin = min(stx,sty) stmax = max(stx,sty) else stmin = stp + xtrapl*(stp - stx) stmax = stp + xtrapu*(stp - stx) endif c Force the step to be within the bounds stpmax and stpmin. stp = max(stp,stpmin) stp = min(stp,stpmax) c If further progress is not possible, let stp be the best c point obtained during the search. if (brackt .and. (stp .le. stmin .or. stp .ge. stmax) + .or. (brackt .and. stmax-stmin .le. xtol*stmax)) stp = stx c Obtain another function and derivative. task = 'FG' 1000 continue c Save local variables. if (brackt) then isave(1) = 1 else isave(1) = 0 endif isave(2) = stage dsave(1) = ginit dsave(2) = gtest dsave(3) = gx dsave(4) = gy dsave(5) = finit dsave(6) = fx dsave(7) = fy dsave(8) = stx dsave(9) = sty dsave(10) = stmin dsave(11) = stmax dsave(12) = width dsave(13) = width1 end c====================== The end of dcsrch ============================== subroutine dcstep(stx,fx,dx,sty,fy,dy,stp,fp,dp,brackt, + stpmin,stpmax) logical brackt double precision stx,fx,dx,sty,fy,dy,stp,fp,dp,stpmin,stpmax c ********** c c Subroutine dcstep c c This subroutine computes a safeguarded step for a search c procedure and updates an interval that contains a step that c satisfies a sufficient decrease and a curvature condition. c c The parameter stx contains the step with the least function c value. If brackt is set to .true. then a minimizer has c been bracketed in an interval with endpoints stx and sty. c The parameter stp contains the current step. c The subroutine assumes that if brackt is set to .true. then c c min(stx,sty) < stp < max(stx,sty), c c and that the derivative at stx is negative in the direction c of the step. c c The subroutine statement is c c subroutine dcstep(stx,fx,dx,sty,fy,dy,stp,fp,dp,brackt, c stpmin,stpmax) c c where c c stx is a double precision variable. c On entry stx is the best step obtained so far and is an c endpoint of the interval that contains the minimizer. c On exit stx is the updated best step. c c fx is a double precision variable. c On entry fx is the function at stx. c On exit fx is the function at stx. c c dx is a double precision variable. c On entry dx is the derivative of the function at c stx. The derivative must be negative in the direction of c the step, that is, dx and stp - stx must have opposite c signs. c On exit dx is the derivative of the function at stx. c c sty is a double precision variable. c On entry sty is the second endpoint of the interval that c contains the minimizer. c On exit sty is the updated endpoint of the interval that c contains the minimizer. c c fy is a double precision variable. c On entry fy is the function at sty. c On exit fy is the function at sty. c c dy is a double precision variable. c On entry dy is the derivative of the function at sty. c On exit dy is the derivative of the function at the exit sty. c c stp is a double precision variable. c On entry stp is the current step. If brackt is set to .true. c then on input stp must be between stx and sty. c On exit stp is a new trial step. c c fp is a double precision variable. c On entry fp is the function at stp c On exit fp is unchanged. c c dp is a double precision variable. c On entry dp is the the derivative of the function at stp. c On exit dp is unchanged. c c brackt is an logical variable. c On entry brackt specifies if a minimizer has been bracketed. c Initially brackt must be set to .false. c On exit brackt specifies if a minimizer has been bracketed. c When a minimizer is bracketed brackt is set to .true. c c stpmin is a double precision variable. c On entry stpmin is a lower bound for the step. c On exit stpmin is unchanged. c c stpmax is a double precision variable. c On entry stpmax is an upper bound for the step. c On exit stpmax is unchanged. c c MINPACK-1 Project. June 1983 c Argonne National Laboratory. c Jorge J. More' and David J. Thuente. c c MINPACK-2 Project. October 1993. c Argonne National Laboratory and University of Minnesota. c Brett M. Averick and Jorge J. More'. c c ********** double precision zero,p66,two,three parameter(zero=0.0d0,p66=0.66d0,two=2.0d0,three=3.0d0) double precision gamma,p,q,r,s,sgnd,stpc,stpf,stpq,theta sgnd = dp*(dx/abs(dx)) c First case: A higher function value. The minimum is bracketed. c If the cubic step is closer to stx than the quadratic step, the c cubic step is taken, otherwise the average of the cubic and c quadratic steps is taken. if (fp .gt. fx) then theta = three*(fx - fp)/(stp - stx) + dx + dp s = max(abs(theta),abs(dx),abs(dp)) gamma = s*sqrt((theta/s)**2 - (dx/s)*(dp/s)) if (stp .lt. stx) gamma = -gamma p = (gamma - dx) + theta q = ((gamma - dx) + gamma) + dp r = p/q stpc = stx + r*(stp - stx) stpq = stx + ((dx/((fx - fp)/(stp - stx) + dx))/two)* + (stp - stx) if (abs(stpc-stx) .lt. abs(stpq-stx)) then stpf = stpc else stpf = stpc + (stpq - stpc)/two endif brackt = .true. c Second case: A lower function value and derivatives of opposite c sign. The minimum is bracketed. If the cubic step is farther from c stp than the secant step, the cubic step is taken, otherwise the c secant step is taken. else if (sgnd .lt. zero) then theta = three*(fx - fp)/(stp - stx) + dx + dp s = max(abs(theta),abs(dx),abs(dp)) gamma = s*sqrt((theta/s)**2 - (dx/s)*(dp/s)) if (stp .gt. stx) gamma = -gamma p = (gamma - dp) + theta q = ((gamma - dp) + gamma) + dx r = p/q stpc = stp + r*(stx - stp) stpq = stp + (dp/(dp - dx))*(stx - stp) if (abs(stpc-stp) .gt. abs(stpq-stp)) then stpf = stpc else stpf = stpq endif brackt = .true. c Third case: A lower function value, derivatives of the same sign, c and the magnitude of the derivative decreases. else if (abs(dp) .lt. abs(dx)) then c The cubic step is computed only if the cubic tends to infinity c in the direction of the step or if the minimum of the cubic c is beyond stp. Otherwise the cubic step is defined to be the c secant step. theta = three*(fx - fp)/(stp - stx) + dx + dp s = max(abs(theta),abs(dx),abs(dp)) c The case gamma = 0 only arises if the cubic does not tend c to infinity in the direction of the step. gamma = s*sqrt(max(zero,(theta/s)**2-(dx/s)*(dp/s))) if (stp .gt. stx) gamma = -gamma p = (gamma - dp) + theta q = (gamma + (dx - dp)) + gamma r = p/q if (r .lt. zero .and. gamma .ne. zero) then stpc = stp + r*(stx - stp) else if (stp .gt. stx) then stpc = stpmax else stpc = stpmin endif stpq = stp + (dp/(dp - dx))*(stx - stp) if (brackt) then c A minimizer has been bracketed. If the cubic step is c closer to stp than the secant step, the cubic step is c taken, otherwise the secant step is taken. if (abs(stpc-stp) .lt. abs(stpq-stp)) then stpf = stpc else stpf = stpq endif if (stp .gt. stx) then stpf = min(stp+p66*(sty-stp),stpf) else stpf = max(stp+p66*(sty-stp),stpf) endif else c A minimizer has not been bracketed. If the cubic step is c farther from stp than the secant step, the cubic step is c taken, otherwise the secant step is taken. if (abs(stpc-stp) .gt. abs(stpq-stp)) then stpf = stpc else stpf = stpq endif stpf = min(stpmax,stpf) stpf = max(stpmin,stpf) endif c Fourth case: A lower function value, derivatives of the c same sign, and the magnitude of the derivative does not c decrease. If the minimum is not bracketed, the step is either c stpmin or stpmax, otherwise the cubic step is taken. else if (brackt) then theta = three*(fp - fy)/(sty - stp) + dy + dp s = max(abs(theta),abs(dy),abs(dp)) gamma = s*sqrt((theta/s)**2 - (dy/s)*(dp/s)) if (stp .gt. sty) gamma = -gamma p = (gamma - dp) + theta q = ((gamma - dp) + gamma) + dy r = p/q stpc = stp + r*(sty - stp) stpf = stpc else if (stp .gt. stx) then stpf = stpmax else stpf = stpmin endif endif c Update the interval which contains a minimizer. if (fp .gt. fx) then sty = stp fy = fp dy = dp else if (sgnd .lt. zero) then sty = stx fy = fx dy = dx endif stx = stp fx = fp dx = dp endif c Compute the new step. stp = stpf end c====================== The end of dcstep ============================== SHAR_EOF fi # end of overwriting check if test -f 'dpmeps.f' then echo shar: will not over-write existing file "'dpmeps.f'" else cat << \SHAR_EOF > 'dpmeps.f' double precision function dpmeps() c ********** c c Subroutine dpeps c c This subroutine computes the machine precision parameter c dpmeps as the smallest floating point number such that c 1 + dpmeps differs from 1. c c This subroutine is based on the subroutine machar described in c c W. J. Cody, c MACHAR: A subroutine to dynamically determine machine parameters, c ACM Trans. Math. Soft., 14, 1988, pages 303-311. c c The subroutine statement is: c c subroutine dpeps(dpmeps) c c where c c dpmeps is a double precision variable. c On entry dpmeps need not be specified. c On exit dpmeps is the machine precision. c c MINPACK-2 Project. February 1991. c Argonne National Laboratory and University of Minnesota. c Brett M. Averick. c c ******* integer i,ibeta,irnd,it,itemp,negep double precision a,b,beta,betain,betah,temp,tempa,temp1, + zero,one,two data zero,one,two /0.0d0,1.0d0,2.0d0/ c determine ibeta, beta ala malcolm. a = one b = one 10 continue a = a + a temp = a + one temp1 = temp - a if (temp1 - one .eq. zero) go to 10 20 continue b = b + b temp = a + b itemp = int(temp - a) if (itemp .eq. 0) go to 20 ibeta = itemp beta = dble(ibeta) c determine it, irnd. it = 0 b = one 30 continue it = it + 1 b = b * beta temp = b + one temp1 = temp - b if (temp1 - one .eq. zero) go to 30 irnd = 0 betah = beta/two temp = a + betah if (temp - a .ne. zero) irnd = 1 tempa = a + beta temp = tempa + betah if ((irnd .eq. 0) .and. (temp - tempa .ne. zero)) irnd = 2 c determine dpmeps. negep = it + 3 betain = one/beta a = one do 40 i = 1, negep a = a*betain 40 continue 50 continue temp = one + a if (temp - one .ne. zero) go to 60 a = a*beta go to 50 60 continue dpmeps = a if ((ibeta .eq. 2) .or. (irnd .eq. 0)) go to 70 a = (a*(one + a))/two temp = one + a if (temp - one .ne. zero) dpmeps = a 70 return end c====================== The end of dpmeps ============================== SHAR_EOF fi # end of overwriting check if test -f 'blas1.f' then echo shar: will not over-write existing file "'blas1.f'" else cat << \SHAR_EOF > 'blas1.f' double precision function dnrm2(n,x,incx) integer n,incx double precision x(n) c ********** c c Function dnrm2 c c Given a vector x of length n, this function calculates the c Euclidean norm of x with stride incx. c c The function statement is c c double precision function dnrm2(n,x,incx) c c where c c n is a positive integer input variable. c c x is an input array of length n. c c incx is a positive integer variable that specifies the c stride of the vector. c c Subprograms called c c FORTRAN-supplied ... abs, max, sqrt c c MINPACK-2 Project. February 1991. c Argonne National Laboratory. c Brett M. Averick. c c ********** integer i double precision scale dnrm2 = 0.0d0 scale = 0.0d0 do 10 i = 1, n, incx scale = max(scale, abs(x(i))) 10 continue if (scale .eq. 0.0d0) return do 20 i = 1, n, incx dnrm2 = dnrm2 + (x(i)/scale)**2 20 continue dnrm2 = scale*sqrt(dnrm2) return end subroutine daxpy(n,da,dx,incx,dy,incy) c c constant times a vector plus a vector. c uses unrolled loops for increments equal to one. c jack dongarra, linpack, 3/11/78. c double precision dx(*),dy(*),da integer i,incx,incy,ix,iy,m,mp1,n c if(n.le.0)return if (da .eq. 0.0d0) return if(incx.eq.1.and.incy.eq.1)go to 20 c c code for unequal increments or equal increments c not equal to 1 c ix = 1 iy = 1 if(incx.lt.0)ix = (-n+1)*incx + 1 if(incy.lt.0)iy = (-n+1)*incy + 1 do 10 i = 1,n dy(iy) = dy(iy) + da*dx(ix) ix = ix + incx iy = iy + incy 10 continue return c c code for both increments equal to 1 c c c clean-up loop c 20 m = mod(n,4) if( m .eq. 0 ) go to 40 do 30 i = 1,m dy(i) = dy(i) + da*dx(i) 30 continue if( n .lt. 4 ) return 40 mp1 = m + 1 do 50 i = mp1,n,4 dy(i) = dy(i) + da*dx(i) dy(i + 1) = dy(i + 1) + da*dx(i + 1) dy(i + 2) = dy(i + 2) + da*dx(i + 2) dy(i + 3) = dy(i + 3) + da*dx(i + 3) 50 continue return end subroutine dcopy(n,dx,incx,dy,incy) c c copies a vector, x, to a vector, y. c uses unrolled loops for increments equal to one. c jack dongarra, linpack, 3/11/78. c double precision dx(*),dy(*) integer i,incx,incy,ix,iy,m,mp1,n c if(n.le.0)return if(incx.eq.1.and.incy.eq.1)go to 20 c c code for unequal increments or equal increments c not equal to 1 c ix = 1 iy = 1 if(incx.lt.0)ix = (-n+1)*incx + 1 if(incy.lt.0)iy = (-n+1)*incy + 1 do 10 i = 1,n dy(iy) = dx(ix) ix = ix + incx iy = iy + incy 10 continue return c c code for both increments equal to 1 c c c clean-up loop c 20 m = mod(n,7) if( m .eq. 0 ) go to 40 do 30 i = 1,m dy(i) = dx(i) 30 continue if( n .lt. 7 ) return 40 mp1 = m + 1 do 50 i = mp1,n,7 dy(i) = dx(i) dy(i + 1) = dx(i + 1) dy(i + 2) = dx(i + 2) dy(i + 3) = dx(i + 3) dy(i + 4) = dx(i + 4) dy(i + 5) = dx(i + 5) dy(i + 6) = dx(i + 6) 50 continue return end double precision function ddot(n,dx,incx,dy,incy) c c forms the dot product of two vectors. c uses unrolled loops for increments equal to one. c jack dongarra, linpack, 3/11/78. c double precision dx(*),dy(*),dtemp integer i,incx,incy,ix,iy,m,mp1,n c ddot = 0.0d0 dtemp = 0.0d0 if(n.le.0)return if(incx.eq.1.and.incy.eq.1)go to 20 c c code for unequal increments or equal increments c not equal to 1 c ix = 1 iy = 1 if(incx.lt.0)ix = (-n+1)*incx + 1 if(incy.lt.0)iy = (-n+1)*incy + 1 do 10 i = 1,n dtemp = dtemp + dx(ix)*dy(iy) ix = ix + incx iy = iy + incy 10 continue ddot = dtemp return c c code for both increments equal to 1 c c c clean-up loop c 20 m = mod(n,5) if( m .eq. 0 ) go to 40 do 30 i = 1,m dtemp = dtemp + dx(i)*dy(i) 30 continue if( n .lt. 5 ) go to 60 40 mp1 = m + 1 do 50 i = mp1,n,5 dtemp = dtemp + dx(i)*dy(i) + dx(i + 1)*dy(i + 1) + * dx(i + 2)*dy(i + 2) + dx(i + 3)*dy(i + 3) + dx(i + 4)*dy(i + 4) 50 continue 60 ddot = dtemp return end subroutine dscal(n,da,dx,incx) c c scales a vector by a constant. c uses unrolled loops for increment equal to one. c jack dongarra, linpack, 3/11/78. c modified 3/93 to return if incx .le. 0. c double precision da,dx(*) integer i,incx,m,mp1,n,nincx c if( n.le.0 .or. incx.le.0 )return if(incx.eq.1)go to 20 c c code for increment not equal to 1 c nincx = n*incx do 10 i = 1,nincx,incx dx(i) = da*dx(i) 10 continue return c c code for increment equal to 1 c c c clean-up loop c 20 m = mod(n,5) if( m .eq. 0 ) go to 40 do 30 i = 1,m dx(i) = da*dx(i) 30 continue if( n .lt. 5 ) return 40 mp1 = m + 1 do 50 i = mp1,n,5 dx(i) = da*dx(i) dx(i + 1) = da*dx(i + 1) dx(i + 2) = da*dx(i + 2) dx(i + 3) = da*dx(i + 3) dx(i + 4) = da*dx(i + 4) 50 continue return end SHAR_EOF fi # end of overwriting check if test -f 'linpack.f' then echo shar: will not over-write existing file "'linpack.f'" else cat << \SHAR_EOF > 'linpack.f' subroutine dpofa(a,lda,n,info) integer lda,n,info double precision a(lda,*) c c dpofa factors a double precision symmetric positive definite c matrix. c c dpofa is usually called by dpoco, but it can be called c directly with a saving in time if rcond is not needed. c (time for dpoco) = (1 + 18/n)*(time for dpofa) . c c on entry c c a double precision(lda, n) c the symmetric matrix to be factored. only the c diagonal and upper triangle are used. c c lda integer c the leading dimension of the array a . c c n integer c the order of the matrix a . c c on return c c a an upper triangular matrix r so that a = trans(r)*r c where trans(r) is the transpose. c the strict lower triangle is unaltered. c if info .ne. 0 , the factorization is not complete. c c info integer c = 0 for normal return. c = k signals an error condition. the leading minor c of order k is not positive definite. c c linpack. this version dated 08/14/78 . c cleve moler, university of new mexico, argonne national lab. c c subroutines and functions c c blas ddot c fortran sqrt c c internal variables c double precision ddot,t double precision s integer j,jm1,k c begin block with ...exits to 40 c c do 30 j = 1, n info = j s = 0.0d0 jm1 = j - 1 if (jm1 .lt. 1) go to 20 do 10 k = 1, jm1 t = a(k,j) - ddot(k-1,a(1,k),1,a(1,j),1) t = t/a(k,k) a(k,j) = t s = s + t*t 10 continue 20 continue s = a(j,j) - s c ......exit if (s .le. 0.0d0) go to 40 a(j,j) = sqrt(s) 30 continue info = 0 40 continue return end subroutine dtrsl(t,ldt,n,b,job,info) integer ldt,n,job,info double precision t(ldt,*),b(*) c c c dtrsl solves systems of the form c c t * x = b c or c trans(t) * x = b c c where t is a triangular matrix of order n. here trans(t) c denotes the transpose of the matrix t. c c on entry c c t double precision(ldt,n) c t contains the matrix of the system. the zero c elements of the matrix are not referenced, and c the corresponding elements of the array can be c used to store other information. c c ldt integer c ldt is the leading dimension of the array t. c c n integer c n is the order of the system. c c b double precision(n). c b contains the right hand side of the system. c c job integer c job specifies what kind of system is to be solved. c if job is c c 00 solve t*x=b, t lower triangular, c 01 solve t*x=b, t upper triangular, c 10 solve trans(t)*x=b, t lower triangular, c 11 solve trans(t)*x=b, t upper triangular. c c on return c c b b contains the solution, if info .eq. 0. c otherwise b is unaltered. c c info integer c info contains zero if the system is nonsingular. c otherwise info contains the index of c the first zero diagonal element of t. c c linpack. this version dated 08/14/78 . c g. w. stewart, university of maryland, argonne national lab. c c subroutines and functions c c blas daxpy,ddot c fortran mod c c internal variables c double precision ddot,temp integer case,j,jj c c begin block permitting ...exits to 150 c c check for zero diagonal elements. c do 10 info = 1, n c ......exit if (t(info,info) .eq. 0.0d0) go to 150 10 continue info = 0 c c determine the task and go to it. c case = 1 if (mod(job,10) .ne. 0) case = 2 if (mod(job,100)/10 .ne. 0) case = case + 2 go to (20,50,80,110), case c c solve t*x=b for t lower triangular c 20 continue b(1) = b(1)/t(1,1) if (n .lt. 2) go to 40 do 30 j = 2, n temp = -b(j-1) call daxpy(n-j+1,temp,t(j,j-1),1,b(j),1) b(j) = b(j)/t(j,j) 30 continue 40 continue go to 140 c c solve t*x=b for t upper triangular. c 50 continue b(n) = b(n)/t(n,n) if (n .lt. 2) go to 70 do 60 jj = 2, n j = n - jj + 1 temp = -b(j+1) call daxpy(j,temp,t(1,j+1),1,b(1),1) b(j) = b(j)/t(j,j) 60 continue 70 continue go to 140 c c solve trans(t)*x=b for t lower triangular. c 80 continue b(n) = b(n)/t(n,n) if (n .lt. 2) go to 100 do 90 jj = 2, n j = n - jj + 1 b(j) = b(j) - ddot(jj-1,t(j+1,j),1,b(j+1),1) b(j) = b(j)/t(j,j) 90 continue 100 continue go to 140 c c solve trans(t)*x=b for t upper triangular. c 110 continue b(1) = b(1)/t(1,1) if (n .lt. 2) go to 130 do 120 j = 2, n b(j) = b(j) - ddot(j-1,t(1,j),1,b(1),1) b(j) = b(j)/t(j,j) 120 continue 130 continue 140 continue 150 continue return end SHAR_EOF fi # end of overwriting check cd .. cd .. # End of shell archive exit 0