| ActiveTcl User Guide |

# Running other programs from Tcl - exec, open

So far the lessons have dealt with programming within the Tcl interpreter. However, Tcl is also useful as a scripting language to tie other packages or programs together. To accomplish this function, Tcl has two ways to start another program:

- `open` ...... run a new program with I/O connected to a file descriptor
- `exec` ...... run a new program as a subprocess

The `open` call is the same call that is used to open a file. If the first character in the file name argument is a "pipe" symbol (`|`), then `open` will treat the rest of the argument as a program name, and will run that program with the standard input or output connected to a file descriptor. This "pipe" connection can be used to read the output from that other program or to write fresh input data to it or both.

If the "pipe" is opened for both reading and writing you must be aware that the pipes are buffered. The output from a `puts` command will be saved in an I/O buffer until the buffer is full, or until you execute a `flush` command to force it to be transmitted to the other program. The output of this other program will not be available to a `read` or `gets` until its output buffer is filled up or flushed explicitly.

(Note: as this is internal to this other program, there is no way that your Tcl script can influence that. The other program simply must cooperate. Well, that is not entirely true: the `expect` extension actually works around this limitation by exploiting deep system features.)

The `exec` call is similar to invoking a program (or a set of programs piped together) from the prompt in an interactive shell or a DOS-box or in a UNIX/Linux shell script. It supports several styles of output redirection, or it can return the output of the other program(s) as the return value of the `exec` call.

`open` `|``progName` `?access?`
  Returns a file descriptor for the pipe. The `progName` argument must start with the pipe symbol. If `progName` is enclosed in quotes or braces, it can include arguments to the subprocess.
`exec` `?switches?` `arg1` `?arg2?` ... `?argN?`
  `exec` treats its arguments as the names and arguments for a set of programs to run. If the first `args` start with a `"-"`, then they are treated as `switches` to the `exec` command, instead of being invoked as subprocesses or subprocess options.

  `switches` are:

  `-keepnewline`
    Retains a trailing newline in the pipeline's output. Normally a trailing newline will be deleted.
  `--`
    Marks the end of the switches. The next string will be treated as `arg1`, even if it starts with a "`-`"
  `arg1` ... `argN` can be one of:
  - the name of a program to execute
  - a command line argument for the subprocess
  - an I/O redirection instruction.
  - an instruction to put the new program in the background:

        exec myprog &

    will start the program `myprog` in the background, and return immediately. There is no connection between that program and the Tcl script, both can run on independently.
    The & must be the last argument - you can use all other types of arguments in front of it.

    [NOTE: add information on how to wait for the program to finish?]

  There are many I/O redirection commands. The main subset of these commands is:
  `|`
    Pipes the standard output of the command preceding the pipe symbol into the standard input of the command following the pipe symbol.
  `< fileName`
    The first program in the pipe will read input from `fileName`.
  `<@ fileID`
    The first program in the pipe will read input from the Tcl descriptor `fileID`. `fileID` is the value returned from an `open` `...` `"r"` command.
  `<< value`
    The first program in the pipe will read `value` as its input.
  `> fileName`
    The output of the last program in the pipe will be sent to `fileName`. Any previous contents of `fileName` will be lost.
  `>> fileName`
    The output of the last program in the pipe will be appended to `fileName`.
  `2> fileName`
    The standard error from all the programs in the pipe will be sent to `fileName`. Any previous contents of `fileName` will be lost.
  `2>> fileName`
    The standard error from all the programs in the pipe will be appended to `fileName`.
  `>@ fileID`
    The output from the last program in the pipe will be written to `fileID`. `fileID` is the value returned from an `open` `...` `"w"` command.

If you are familiar with shell programming, there are a few differences to be aware of when you are writing Tcl scripts that use the `exec` and `open` calls.

- You don't need the quotes that you would put around arguments to escape them from the shell expanding them. In the example, the argument to the `sed` command is not put in quotes. If it were put in quotes, the quotes would be passed to `sed`, instead of being stripped off (as the shell does), and `sed` would report an error.
- If you use the `open` `|cmd` `"r+"` construct, you must follow each puts with a flush to force Tcl to send the command from its buffer to the program. The output from the program itself may be buffered in its output buffer.

  You can sometimes force the output from the external program to flush by sending an `exit` command to the process.

  You can also use the `fconfigure` command to make a connection (channel) unbuffered.

  As already mentioned, `expect` extension to Tcl provides a much better interface to other programs, which in particular handles the buffering problem.
  [NOTE: add good reference to expect]

- If one of the commands in an `open` `|cmd` fails the `open` does not return an error. However, attempting to read input from the file descriptor with `gets` `$file` will return an empty string. Using the `gets` `$file` `input` construct will return a character count of -1.
- Tcl does not expand file names like the UNIX/Linux shells do. So:

      exec ls *.tcl

  will fail - there is most probably no file with the literal name "*.tcl".

  If you need such an expansion, you should use the `glob` command:

      eval exec ls [glob *.tcl]

  or, from Tcl 8.5 onwards:

      exec ls {expand}[glob *.tcl]

  where the `{expand}` prefix is used to force the list to become individual arguments.
- If one of the commands in an `exec` call fails to execute, the `exec` will return an error, and the error output will include the last line describing the error.

- The `exec` treats any output to standard error to be an indication that the external program failed. This is simply a conservative assumption: many programs behave that way and they are sloppy in setting return codes.

  Some programs however write to standard error without intending this as an indication of an error. You can guard against this from upsetting your script by using the `catch` command:

      if { [catch { exec ls *.tcl } msg] } {
         puts "Something seems to have gone wrong but we will ignore it"
      }

  To inspect the return code from a program and the possible reason for failure, you can use the global `errorInfo` variable:

      if { [catch { exec ls *.tcl } msg] } {
         puts "Something seems to have gone wrong:"
         puts "Information about it: $::errorInfo"
      }

  ## Example

      #
      # Write a Tcl script to get a platform-independent program:
      #
      # Create a unique (mostly) file name for a Tcl program
      set TMPDIR "/tmp"
      if { [info exists ::env(TMP)] } {
          set TMPDIR $::env(TMP)
      }
      set tempFileName "$TMPDIR/invert_[pid].tcl"

      # Open the output file, and
      # write the program to it

      set outfl [open $tempFileName w]

      puts $outfl {
          set len [gets stdin line]
          if {$len < 5} {exit -1}

          for {set i [expr {$len-1}]} {$i >= 0} {incr i -1} {
              append l2 [string range $line $i $i]
          }
          puts $l2
          exit 0
      }

      # Flush and close the file
      flush $outfl
      close $outfl

      #
      # Run the new Tcl script:
      #
      # Open a pipe to the program (for both reading and writing: r+)
      #
      set io [open "|[info nameofexecutable] $tempFileName" r+]

      #
      # send a string to the new program
      #     *MUST FLUSH*
      puts $io "This will come back backwards."
      flush $io

      # Get the reply, and display it.
      set len [gets $io line]

      puts  "To reverse: 'This will come back backwards.'"
      puts "Reversed is: $line"
      puts "The line is $len characters long"

      # Run the program with input defined in an exec call

      set invert [exec [info nameofexecutable] $tempFileName << \
             "ABLE WAS I ERE I SAW ELBA"]

      # display the results
      puts "The inversion of 'ABLE WAS I ERE I SAW ELBA' is \n $invert"

      # Clean up
      file delete $tempFileName