1 09/17/87  Multics Simplified I/O Redirection (pipes)
  2 
  3 
  4 The pipe facility of the command processor provides the ability to
  5 redirect stream I/O.  The delimiter string semi-colon, vertical bar
  6 (";|") instructs the command processor to redirect the I/O attachments
  7 of the standard switches user_input and user_output.  The pipe
  8 component delimiter is interpreted similarly to the command line
  9 delimiter ";" with pipe streams limited to one command line.  The
 10 command processor pipe facility is designed to provide simplified data
 11 flow control.
 12 
 13 
 14 Notes on basic usage:
 15 
 16 The simplest form of a pipe is,
 17 
 18 cmd1 ;| cmd2
 19 
 20 where cmd1 is the name of a command which outputs to user_output and
 21 cmd2 is a command that gets data from user_input and may output data to
 22 user_output.
 23 
 24 
 25 The command line,
 26 
 27 pwd ;| sm User_id
 28 
 29 results in the output of the command print_wd(pwd) being routed as
 30 input to the command send_message (sm).  Some commands will not
 31 execute correctly without an end of information token.  To use the pipe
 32 facility with these commands it is necessary to add the value at the
 33 end of the input.
 34 
 35 
 36 The command line,
 37 
 38 pwd ;| input; ioa_ "." ;| input ;| sdm User_id -sj "the pathname"
 39 
 40 routes the output of the print_wd (pwd) command to the file "input".
 41 Then ioa_ is used to add a period to the file and finally the file is
 42 routed as input to the send_mail (sdm) command.
 43 
 44 
 45 Notes on commands and files usage:
 46 
 47 In addition to the ability to route data from one command to another,
 48 the pipe facility is available for more complex functions.
 49 
 50 Data can be routed from a command to a file.
 51 
 52 The command line,
 53 
 54 list ;| list_command_output
 55 
 56 is expanded to,
 57 
 58 list ;| vfile_ [wd]>list_command_output -extend
 59 
 60 The expanded command line instructs the command processor to attach the
 61 user_output switch to the file list_command_output in the current
 62 working directory.  The output of the list command is then placed into
 63 the file.
 64 
 65 
 66 Notes on commands and I/O module usage:
 67 
 68 It is possible to specify an I/O module in the description of an output
 69 data store.
 70 
 71 The command line,
 72 
 73 list ;| tape_mult_ M9999 -write -den 6250
 74 
 75 will route the output of the list command to the magnetic tape M9999
 76 through the Multics standard format tape I/O module tape_mult_.
 77 
 78 
 79 Any Multics I/O module may be specified.  For a description of the
 80 Multics I/O modules see the Multics manual "Multics Subroutines and I/O
 81 Modules" (AG93) and Section 4 of "Multics Programmer's Reference
 82 Manual" (AG91).
 83 
 84 
 85 Data can be routed from a file to a command.
 86 
 87 The command line,
 88 
 89 input_file ;| sm GWMay
 90 
 91 expands to,
 92 
 93 vfile_ [wd]>input_file -extend ;| sm GWMay
 94 
 95 The expanded command line instructs the command processor to route the
 96 data contained in the file input_file as input to the command
 97 send_message (sm).
 98 
 99 Any Multics I/O module may be given to specify the source of stored
100 data.
101 
102 
103 Notes on interfile usage:
104 
105 Data can be routed from one file to another.
106 
107 vfile_ [wd]>input_file -extend ;| tape_mult_ m9999 -write -den 6250
108 
109 results in the file input_file being copied to the tape m9999.
110 
111 The command line,
112 
113 file1 ;| file2
114 
115 is expanded to,
116 
117 vfile_ [wd]>file1 -extend ;| vfile_ [wd]>file2 -extend
118 
119 The expanded command line instructs the command processor to copy the
120 contents of the file file1 to the file file2.  The pipe facility will
121 allow the same file name to be given for input and output.
122 
123 file ;| file
124 
125 will result in the contents of the file being appended to the end of
126 itself.
127 
128 
129 Because most data storing done on the Multics system is handled with
130 magnetic disk, the defaulting of file names to the I/O module vfile_ is
131 provided for ease of use.  If desired, control arguments for vfile_ may
132 be specified with the file name.  Pathnames may also be given.
133 
134 The command line,
135 
136 >udd>Multics>file1 ;| file2 -truncate
137 
138 expands to,
139 
140 vfile_ udd>Multics>file1 -extend ;| vfile_ [wd]>file2
141                                        -extend -truncate
142 
143 
144 The pipe facility may be used for more complex functions.
145 
146 The various components of a pipe may be chained together in order
147 to create a stream of data that accomplishes many tasks.
148 
149 The command line,
150 
151 ls ;| list_file ;| sm GWMay
152 
153 expands to,
154 
155 ls ;| vfile_ [wd]>list_file -extend ;| sm GWMay
156 
157 The expanded  command line results in  several functions.  First,
158 the  output  of  the  list  (ls)  command  is  placed in the file
159 list_file.   Then the file  list_file is routed  as input to  the
160 command send_message (sm).
161 
162 There  is no  limit on  the number  of pipe  delimiters used in a
163 command line.
164 
165 
166 Notes on default pipe output
167 
168 The  pipe facility  provides for   the default  output of  a pipe
169 stream.
170 
171 The command line,
172 
173 ls ;|
174 
175 is expanded to,
176 
177 ls ;| vfile_ [wd]>pipeout -extend
178 
179 The output of the list command  is placed into the file "pipeout"
180 in the current working directory.
181 
182 
183 Notes on iteration with pipes:
184 
185 The  pipe facility  can be   combined with  iteration.  The  pipe
186 facility limits iteration to a single delimited pipe component.
187 
188 The command line,
189 
190 (pwd ls who) ;| output
191 
192 The  effect of  the command  line is  to route  the output of all
193 three commands to the file "output".
194 
195 The command line,
196 
197 input ;| (cmd1 cmd2 file1) ;| output
198 
199 instructs the command processor to route the data from "input" to
200 the commands "cmd1" and "cmd2" and the file "file1" where "input"
201 is either  a command or file.   The output of all  three are then
202 routed to "output" which may be a command or file.
203 
204 
205 A possible  use of this command  line would be when  the commands
206 cmd1 and cmd2 use input from  "input" but do not output any data.
207 In such  a case, the file  "file" would contain the  same data as
208 was routed out  of "input".  This makes it possible  to route the
209 data through the pipe to "output".
210 
211 
212 The command line,
213 
214 a (;|b-c -d;|e) ;| f
215 
216 is not an acceptable use of iteration and pipe facility.
217 
218 
219 Notes on active strings with pipes:
220 
221 The pipe facility can be used in active strings.
222 
223 The command line,
224 
225 ([segs **]);| file
226 
227 will copy the contents of each file and output of each command in
228 the current working directory to the file "file".
229 
230 
231 The command line,
232 
233 pl1 ([ls >udd>pl1_dir>ab*.pl1 -no_header -primary -name;|])
234 
235 instructs  the command  processor to   return the  output of  the
236 command  ls as the  active return string.   The output of  a pipe
237 stream is returned only when the active string is terminated with
238 the pipe delimiter just before the ending right bracket "]".
239 
240 
241 By  default, new  line characters   are removed  from the  active
242 function return string.  When it  is desirable to retain new line
243 characters, an additional vertical bar is appended.
244 
245 The command line,
246 
247 value_set command_results ||[string [ls -all -sort;||]
248 
249 will return the output of the ls command with new line characters
250 unaltered.
251 
252 
253 The pipe facility, iteration and active string can be combined in
254 a single command line.
255 
256 The command line,
257 
258 sm ([who ;| match /Multics/;|]) [pwd;|]
259 
260 routes the  output of the who  command to a filter  "match".  The
261 names  output by  match are   returned to  the send_message  (sm)
262 command in  the active function return string  as the destination
263 of  the message.   The output  of the  print_wd (pwd)  command is
264 returned as the message to be sent.
265 
266 
267 Notes on restrictions on active strings:
268 
269 It is not  possible to intermix pipes and  normal active function
270 invocations within the same active function bracket pair.
271 
272 The command line,
273 
274 string [time; pwd;|]
275 
276 attempts to call  the time active function then  execute the pipe
277 pwd;| and is in error.  The correct usage is,
278 
279 string [time][pwd;|]
280 
281 Active functions can be used within pipes as long as they are not
282 within the same active string level as the pipe.  For example:
283 
284 string [ls [hd]>*.[date];|]
285 
286 
287 Notes on alternate I/O types
288 
289 The pipe  facility can be used  with I/O types other  than stream
290 I/O.   In  order  to  access  sequential  data  type  files,  the
291 record_stream_ I/O module must be  used as the intermediary.  The
292 command line,
293 
294 ls ;| record_stream_ -length 80
295   -target "tape_nstd_ m9999 -block 80 -write -den 1600"
296 
297 will output the stream data of the list command to the sequential
298 data store m9999 controlled by the tape_nstd_ I/O module.
299 
300 
301 Notes on command/file conflicts in pipes:
302 
303 The simplicity of the pipe  facility syntax may lead to conflicts
304 between file and command names.  In order to supply the simplest
305 possible  syntax, the  ability  to  explicitly identify  files and
306 commands can be overridden.  When using the pipe facility, a pipe
307 string may begin with one of the following:
308 
309 1) a Multics command name.
310 
311 2) a Multics I/O module name.
312 
313 3) a Multics file name that can be followed by
314    vfile_ control arguments.
315 
316 
317 The Multics pipe facility uses the current search list to automatically
318 determine the type of entry that starts a pipe string.  A segment is
319 determined to be a command when it contains a linkage section and an
320 entrypoint matching the name given in the command line.
321 
322 
323 A Multics I/O module is a special type of object segment that does not
324 contain an entrypoint that matches the name given on the command line.
325 All I/O modules do however contain an entrypoint which is a composite
326 of the name given with an attach identifier.
327 
328 
329 To facilitate the fewest number of key strokes, lines which start with
330 a name that is not a command or an I/O module are considered filenames
331 by default.
332 
333 
334 Files have no particular identifying features other than that they do
335 not meet the requirements of an object segment.
336 
337 Because of the defaulting of names to be files, there may be occasions
338 when a file name is given when a command was intended and the other way
339 around.  When this happens, the pipe command line may perform
340 differently than was intended.
341 
342 
343 Notes on locating a conflict
344 
345 When a pipe command line does not perform as expected, the where(wh)
346 command may help determine the cause.
347 
348 With the command line,
349 
350 ls ;| list
351 
352 the user may have intended the output of the "ls" command to be placed
353 into a file named "list".  The actual execution of this line will
354 result in the ls command being executed twice.  With the second
355 execution displaying to the terminal.
356 
357 
358 By typing,
359 
360 where ls
361 
362 and
363 
364 where list
365 
366 the user will easily see where the list entry is being taken from.
367 
368 
369 To determine if the entry is a file or command, the print_link_info
370 (pli) command can be used.  Using the example from above with the pli
371 command results in the command line:
372 
373 pli [wh list] -he
374 
375 If the list entry is an object segment, the information displayed by
376 the command line will indicate so.  If the entry is not an object, an
377 inconsistency in the segment is diagnosed.
378 
379 
380 Notes on resolving command/file conflicts:
381 
382 The conflicts between files and commands can be resolved by explicitly
383 stating the I/O module in the attachment and by using complete
384 pathnames.
385 
386 If the user really does want to override the list command and create a
387 file named "list" then the command line:
388 
389 ls ;| [wd]>list
390 
391 should be used.
392 
393 
394 If by chance there is a command in the current working directory named
395 "list", the I/O module name should be included in the attachment.  The
396 command line:
397 
398 ls ;| vfile_ [wd]>list
399 
400 will replace the command with a file named "list.
401 
402 
403 Whenever it is necessary to override commands with filenames or when a
404 specific location of a file is important, complete pathnames should be
405 given.  For example, when using the pipe facility in exec_coms or
406 abbreviations, it is a good idea to use complete pathnames to make sure
407 that the proper files and commands are referenced.