1 07/16/86  fortran_large_arrays.gi
  2 
  3 Fortran has been extended to allow large and very large arrays in
  4 automatic, static and COMMON storage.  These arrays are stored in
  5 separate segments outside the stack and linkage area.
  6 
  7 
  8 Large arrays:
  9 
 10 Large arrays (LAs) should be used when the size of the arrays in a
 11 storage class (automatic or static) would exceed the normal limits of
 12 that storage class.  For automatic storage, the normal aggregate limit
 13 is about 62000 words.  For static storage, the normal aggregate limit
 14 is about 128K words.  The use of large arrays extends this storage
 15 limit to an arbitrarily large amount.  Every individual array must
 16 occupy less than one segment (262144 words) and is contained in a
 17 single segment.  There is some additional overhead to allocate and
 18 initialize the extended storage.  There is no restriction on the type
 19 or use of large arrays.
 20 
 21 Large arrays are available through the -large_array (-la) (automatic
 22 and static), -la_auto (automatic only), or -la_static (static only)
 23 control arguments or the la, la_auto, or la_static arguments in a
 24 %global statement.
 25 
 26 
 27 Very large arrays:
 28 
 29 Very large arrays (VLAs) are those which are longer than 255K words.
 30 This includes COMMON blocks which are longer than 255K words.  The
 31 limit now is 2**24 words for each array or COMMON block.  In addition
 32 to the overhead for allocation and initialization, there is addressing
 33 overhead for VLAs and the array parameters of programs that use VLAs.
 34 
 35 Very large arrays are available through the -very_large_array (-vla)
 36 (COMMON, automatic, and static), -very_large_common (-vlc) (COMMON
 37 only), -vla_auto (automatic only), or -vla_static (static only)
 38 control arguments or the vla, vlc, vla_auto, or vla_static arguments
 39 in a %global statement.
 40 
 41 There are some restrictions on the use of VLAs.
 42 - They may not contain characters.
 43 - Binary I/O is limited to a segment or less in length per record.
 44 - Permanent very large COMMON blocks are not supported.
 45 
 46 Programs that expect VLA parameters but otherwise do not use LAs or
 47 VLAs should use the -vla_parm control argument or the vla_parm %global
 48 argument.  In this case, the only extra overhead is for addressing the
 49 (possibly) VLA parameters.
 50 
 51 LA and VLA features affect the entire compilation unit.
 52 
 53 Array initialization information is now stored in a condensed format,
 54 so that an object segment can contain the initialization for several
 55 VLAs.
 56 
 57 
 58 Location of Storage:
 59 
 60 Use of the LA/VLA features requires large amounts of quota.  By
 61 default, the extended storage segments are placed in the process
 62 directory.  If there is not enough space there, the user can use the
 63 value_set command to define "fsm_dir_dir_path".  The extended storage
 64 segments are placed in pool directories within the process directory
 65 or "fsm_dir_dir_path".  There is potentially a separate directory for
 66 each ring.  For example, the pool directory for ring 4 has the name
 67 fsm_dir_4.  Access on individual segments is set to "rw" for the owner
 68 and to "null" for everyone else.
 69 
 70 The command list_fortran_storage (lfs) lists the extended storage
 71 segments assigned to the user's process.  They are grouped according
 72 to the storage they are extensions of.  For example, all the segments
 73 for a program's static LAs and VLAs are grouped together, identified
 74 by a pointer to the beginning of the program's active linkage section
 75 (the copy that is being used, not the copy in the object segment).
 76 
 77 
 78 COMMON:
 79 
 80 There are some changes concerning COMMON.  VLA COMMON (VLC) is
 81 allocated when a program is entered, rather than when the block is
 82 first referenced.  When a program that uses VLC is bound with a
 83 non-VLA program that uses the same COMMON block, the binder will make
 84 all references be to the VLC block.  If the non-VLA program is
 85 executed first, the block will be allocated as a VLC but the program
 86 will reference only the first segment.
 87 
 88 All the commands that deal with COMMON have been modified to work with
 89 VLC.  This includes set_fortran_common, the external variable
 90 commands, and print_link_info.  The run command has been modified to
 91 work correctly with LAs/VLAs/VLC.
 92 
 93 
 94 Overhead in allocating and freeing storage:
 95 
 96 There is substantial overhead in allocating and freeing the storage
 97 associated with LA's and VLA's as compared to normal arrays.  For
 98 static and common storage, the allocation overhead cost is paid once
 99 per process (unless a program is terminated and then reinvoked).  Thus
100 it is not very expensive or inefficient to use static or common LA's
101 and VLA's.  However, the allocation overhead for automatic LA's and
102 VLA's occurs every time the compilation unit which declares them is
103 entered and the freeing overhead occurs every time on exit.  If a
104 program is compiled in one piece, then the allocation and freeing
105 overhead for automatic LA's and VLA's occurs only once per run of the
106 program.  However, if the program is made of separately compiled
107 pieces, then a high price might be paid for allocation and
108 deallocation.  Therefore, if a program is known to to contain only
109 certain storage classes of LA's and VLA's, they can be compiled using
110 the appropriate storage specific LA and VLA arguments (la_auto,
111 la_static, vla_auto, vla_static, very_large_common) to reduce the
112 overhead required.  Only the vla_parm argument does not cause any
113 entry/exit overhead.
114 
115 
116 Static storage:
117 
118 The normal limit on the amount of static storage in an unbound segment
119 is 128K words.  However, the binder will not bind a set of objects if
120 the total amount of internal static storage used by those objects
121 exceeds 16383 words.  Using the la_static argument causes the static
122 arrays to go into fortran's managed storage and hence are addressed
123 differently.  The effect is that programs may then have arbitrarily
124 large amounts of static storage and can be bound.
125 
126 
127 Probe:
128 
129 Probe has been modified to work with VLAs.  In addition, probe can
130 print the values of the base pointers to LA and VLA segments.  These
131 pointers have names which are printed in the storage class section of
132 program listings' symbol maps and which can be obtained through the
133 probe value request.
134 
135 Base pointers to LA storage are of the form:
136      la$auto$xxxxxx or la$static$xxxxxx
137 where xxxxxx is the octal offset from the beginning of the stack or
138 static section of the pointer to the first segment.
139 
140 Base pointers to VLA segments are of the form:
141    vla$auto$name  or vla$static$name  or vla$common$name
142 where "name" is the name of the array or COMMON block.  Blank COMMON
143 has the name of vla$common$_.
144 
145 
146 Conditions raised:
147 
148 Two new conditions can be raised by Fortran runtime.
149 fortran_storage_manager_error is raised if an error occurs during the
150 assignment or creation of external segments.  fortran_storage_error is
151 raised if an error occurs in the initialization of external storage or
152 in the creation of a COMMON block.  The accompanying messages describe
153 the cause of the error and possible correction.  These conditions are
154 usually restartable if the cause of the error is corrected.
155 
156 
157 Implications for subroutine libraries:
158 
159 The use of subroutine libraries poses some efficiency problems with
160 the use of VLA's.  These problems must be addressed by the library
161 managers of individual sites.
162 
163   1. VLA addressing  is a superset of  normal hardware addressing
164      of  the  Multics  processor.   This  permits  all subroutine
165      libraries to be compiled with VLA addressing for parameters,
166      through either "-vla_parm" or "%global vla_parm;".
167 
168   2. It  is  possible  that  certain programs  will  be noticably
169      impacted by the  use of VLA addressing in  very tight loops,
170      or in complex operations, such as might occur in things like
171      matrix multiplies or inversions.  In  such cases it might be
172      advisable  to  copy  VLA  parameters to  a  local  area, and
173      re-copy the result at the end of computation.
174 
175   3. It  may be  advisable to take  internal routines  of a large
176      libary set, such as IMSL, which are known to NEVER be passed
177      VLA's  and  to  separately  re-compile  them  so  that their
178      internal interfaces will only expect normal arrays.
179 
180   4. A  final  possibility  would  be two  distinct  and separate
181      libraries,  each  compiled  for  maximum  speed  in  the two
182      calling  types; VLA  calls and  normal calls.   This poses a
183      management problem in making segments known, dynamic linking
184      and storage management, and  management of user problems and
185      education.