Yax

Yax stands for Yet Another eXpect. The intention is to let you script interactions with other programs: the name could just as easily refer to the conversations between your script and other programs. Yax integrates well with Ruby and Rake or Rant. Yax was born out of frustration with other solutions in this space. Yax has only been tested under OS X.

Status

Yax is currently pre-release software, which has not yet undergone internal reviews. Several developers use Yax to build the following software components under OS X:

SoQt
Fox 1.2
FXScintilla
wxWindows
GLUI
ARToolkit
Coin
libpng (with zlib)
jpeg
libtiff
libwmf
GD
GraphicsMagic
Swig
Ruby OpenGL (James Adam’s tweak of Yoshiyuki Kusano’s code)
rubyARToolkit
rubyCoin
qtRuby
FXRuby
rMagick

Installation

Yax can be downloaded in several forms:

A RubyGem, containing only Yax. Due to lack of resources, the RubyGem is untested.
A development snapshot containing Yax and nearly everything it depends on, in a .tgz or .zip archive. The snapshot contains a complete Amber development environment. Before using the development snapshot, you need to:
1. Setup Amber (included in the snapshot): see ReadMe_Amber.txt
2. Install libraries required by Yax:
```
    cd $DevRoot/Projects/yax
    rake install_lib
```

Use

Start out by creating a new Session, either by Yax::Session.spawn (if the default attributes are satisfactory) or by Yax::Session.new, followed by attribute adjustments, and then spawn sent to the Session instance. In either case, you then send a series of Yax::Session#command messages. If these commands include a time limit, they will block until completion. If a command does not include a time limit, you can follow it up with one or more expect / respond pairs. After the last expect / respond, you can either finish the command, or just issue another command (the previous one will finish automatically). Once you are done with the session, you can wait for it to finish, or explicitly ask it to die.

An example interaction is shown below.

Extra Features

Time limits can be applied to operations that involve a response from the spawned process. These limits can be expressed as numbers of seconds, or as Strings in the form dd:hh:mm:ss. In either case the script should contain actual measurements (which can come from an execution time log if @time_log_path is specified). Adjustments for alternate operating conditions can be made through the @timeout_multiplier and @timeout_offset attributes.

Yax::Session attempts to intelligently handle sudo. If @password_prompt is set and @password is not, the first command will result in a request for the user’s password for the password. Then every expect will include @password_prompt This feature accommodates systems that authenticate only if some predetermined time has elapsed since the last successful authentication. If you use this feature, you should be careful with the transcripts, which are likely to contain passwords.

Yax::Session includes convenience methods for:

File operations (cd, rm, cp, mv, ln, mkdir, chmod and cmp)
Downloading archives
Unpacking archives
Creating archives
Using concise Symbols to represent long cumbersome urls and directory paths
File content modification (useful for on-the-fly makefile or source-code modifications)
For environmental variable specification

The Yax module also provides convenient access to most features of a default instance of Yax::Session.

You can log the session to a transcript (which can be anything that responds to <<). Each line is prefixed by a character that explains what the line means: see the @prefixes hash. The default prefixes are:

 [s>]   Spawn a new shell
 [c>]   A command
 [x>]   One or more expected patterns or Strings
 [i>]   A response to an expected pattern or String
 [o>]   Output
 [m>]   Output that matches an expected pattern or String
 [e>]   Error output (some programs output innocuous information here
 [n>]   A note
 [d>]   Entering a directory
 [f>]   A file operation

Example

        require 'nist/yax'
        include Yax

        # Prepare the default Yax::Session
        def setup
            $YAX=Yax::Session.new
            yax.url_hash = {
                :fox => 'http://www.fox-toolkit.org/ftp/fox-1.2.18.tar.gz',
                :swig => 'http://easynews.dl.sourceforge.net/sourceforge/swig/swig-1.3.25.tar.gz',
                :fxRuby => 'http://rubyforge.org/frs/download.php/4003/FXRuby-1.2.6.tar.gz',
                :arToolKit => 'http://easynews.dl.sourceforge.net/sourceforge/artoolkit/ARToolKit-2.70.1.tgz'
                }
            yax.dir_overide = {
                'ARToolKit-2.70.1' => 'ARToolKit'
                }
            yax.spawn
        end

        # Build the ARToolkit. This uses the alternative (non-XCode)
        # instructions from README.txt, because they are better suited
        # to automation.Defaults should be satisfactory for most
        # relatively recent macs.
        # GL_NV_texture_rectangle is supported on most NVidia graphics cards
        # and on ATi Radeon and better graphics cards
        def build_arToolKit(fastTextureMapping=true, textureRectangle=true)
                banner 'Build ARToolKit'
                untgz :arToolKit
                cd :arToolKit
                cmd './Configure'
                # "Dose your Mac have fast texture mapping hardware? (y or n)"
                expect "Enter :",                     '0:0:0:1'
                snd( fastTextureMapping ? 'y' : 'n' )
                # "Build gsub libraries with texture rectangle support? (y or n)"
                expect "Enter :",                     '0:0:0:1'
                snd( textureRectangle ? 'y' : 'n' )
                finish                                '0:0:15:0'
                cmd make,                             '0:1:0:0'
        end

        setup
        build_arToolKit

Caveats

If @password_prompt is non-nil (which lets you script commands that require an administrative password) you will be prompted for a password. Currently the password is echoed to the screen. If you save the transcript of the session you should remove your password.

Yax has only been tested spawning bash on OS X (that’s why it was developed). I would like to hear from anyone who has used it on another platform, or to script some other process.

To use Yax::Session#curl, you must have curl installed. Get it from curl.haxx.se

Implementation Notes

The stdout returned by PTY.spawn seems to behave oddly, at least under OS X. The output from a given command may be delayed, and show up in a subsequent command. To eliminate this problem I attempted to use popen3 instead. The popen3 stdout has its own problems:

IO.select raises an exception when fed this stdout This prevents the standard Ruby implementation of expect (in expect.rb) from working.
When I redefine expect without IO.select,
- getc misses some characters.
- gets sometimes hangs.

To work around the problem with the PTY.spawn stdout, _flush() was added, along with a few other tweaks controlled by pty_workaround. On platforms other than OS X you might want to set pty_workaround to false.

The idea for redirecting error output to a file came from Ara T. Howard’s Session (www.codeforpeople.com/lib/ruby/session/).

Potential improvements

Machine processable history in addition to transcript (for example, being able to use result of ‘ls’ later in script) transcript is useful for debugging interactions, but not for capturing results. (but see next item).
Expect can currently take multiple patterns, with different responses (and subsequent expectations) for each one. This allows for rather complicated scripting, since the script is not be limited to linear interactions. It would be good to have some simple way of structuring these interactions. One way might be with SimpleMachine. Command output (and errors) would be events. Expectations would be guards. The history would continue to be linear, but it should additionally be mapped onto the machine structure so that we can ask "What was the output from state Z" without interpreting the linear history all over again.
Could show error output in realtime (instead of after the expect). I think I prefer to have all the errors at the end though.
Allow operations without time limits? Currently passing a nil maxSeconds to :command means "don’t block, we are going to expect/respond". Need to pass in some thing else (zero? a Symbol?) to tell it "block until finished". This would allow us to get rid of cd_max_seconds, which would be more consistent with the other file operations. On the other hand, in practice everything has a time limit.
Password should not be echoed
Replace dump with something that escapes non-printables, but does not put quotes around strings (in irb ‘foo’.dump returns \"foo\"

Author: A. Griesser

Official contribution of the National Institute of Standards and Technology; not subject to copyright in the United States. Certain commercial equipment, instruments, or materials are identified in this paper to foster understanding. Such identification does not imply recommendation or endorsement by the National Institute of Standards and Technology, nor does it imply that the materials or equipment identified are necessarily the best available for the purpose.

ReadMe.txt

Yax