Tuesday, June 06, 2006

Ruby Kernel system, exec and %x

The Ruby Core Library documentation is very similar for Kernel.system, Kernel.exec and %x[..]. Recently I needed to kick off a system process, so I spent some time working with all 3 options.

Kernel.exec does exactly what the documentation states:
Replaces the current process by running the given external command. If exec is given a single argument, that argument is taken as a line that is subject to shell expansion before being executed. If multiple arguments are given, the second and subsequent arguments are passed as parameters to command with no shell expansion. If the first argument is a two-element array, the first element is the command to be executed, and the second argument is used as the argv[0] value, which may show up in process listings. In MSDOS environments, the command is executed in a subshell; otherwise, one of the exec(2) system calls is used, so the running command may inherit some of the environment of the original program (including open file descriptors).

exec "echo *" # echoes list of files in current directory
# never get here

exec "echo", "*" # echoes an asterisk
# never get here
An important thing to notice is that it replaces the current process. To immediately see what is meant by 'replaces', try running exec from irb:
focus:~/work/eaa jay$ irb
irb(main):001:0> exec 'svn st'
focus:~/work/eaa jay$
A more realistic example could be, assume you wanted to execute an external command from rake and you tried to use exec. Your external command would be executed, but rake would never finish. Not exactly desired results.

Kernel.system behaves very similarly, but does not replace the current process. The documentation states the following:
Executes cmd in a subshell, returning true if the command was found and ran successfully, false otherwise. An error status is available in $?. The arguments are processed in the same way as for Kernel::exec.

system("echo *")
system("echo", "*")

produces:

config.h main.rb
*
The same irb test shows us that system will return true or false, and $? can be used to get a Process::Status instance.
irb(main):014:0> s = system 'uptime'
10:56 up 3 days, 23:10, 2 users, load averages: 0.17 0.17 0.14
=> true
irb(main):015:0> s.class
=> TrueClass
irb(main):016:0> $?.class
=> Process::Status
Process::Status does contain the exitstatus Fixnum, but it does not appear to capture any errors that may occur.
irb(main):019:0> s = system "ruby -e 'invalid ruby $%^&'"
-e:1: parse error, unexpected $undefined., expecting $
invalid ruby $%^&
^
=> false
Though I do like knowing my external application failed, it is nice to know why it failed. If you can redirect the output to a file this isn't a problem; however, you have 2 options if the execution output is simply dumped to stdout. The first option is to use system and redirect output to a file
irb(main):003:0> system 'uptime > results.log'
=> true
irb(main):004:0> exit
focus:~/work/eaa jay$ more results.log
13:09 up 4 days, 1:23, 2 users, load averages: 0.21 0.24 0.19
Another option is to use %x[..] and save the results. This option is more often used when the results need to be captured and used to determine further execution flow. To capture the results of a %x[..] you need to assign a variable to the return value.
irb(main):001:0> result = %x[uptime]
=> "13:16 up 4 days, 1:30, 2 users, load averages: 0.39 0.29 0.23\n"
irb(main):002:0> p result
"13:16 up 4 days, 1:30, 2 users, load averages: 0.39 0.29 0.23\n"
Using system allows you to deal with the output when false is returned. However, if you are always going to open the file and process the results you are likely better off using %x[..] in the first place.

21 comments:

  1. Anonymous9:44 AM

    Good article. Just what I needed!

    ReplyDelete
  2. Anonymous1:05 AM

    Thanks for the post. If you need STDOUT and STDERR too, you can always pipe all output using &> instead:

    Kernel.system("cmd &> output.txt")

    and read it back in to your hearts content.

    ReplyDelete
  3. Anonymous12:10 AM

    Thanks for the tutorial!
    Helped me here!
    hugs from brazil

    ReplyDelete
  4. Anonymous5:42 PM

    Great Info!

    But is it possible with %x() to pass arguments to the command to prevent shell expension? (like exec())

    ReplyDelete
  5. Anonymous8:30 PM

    I probably need more info to try to address your problem; however, as a starting point I noticed that putting quotes around the * causes the behavior to change.

    irb(main):008:0> %x[echo *]
    => "README Rakefile app components config db doc lib log public script test tmp vendor\n"
    irb(main):009:0> %x[echo '*']
    => "*\n"

    ReplyDelete
  6. Anonymous8:54 PM

    yeah that seems to work!

    Example: %x[echo ; ls -la] # this execute ls -la
    %[echo '; ls -la'] # this doesnt

    where ";ls -la" would be a dynamic input and you dont want a user to execute arbitrary commands.

    Thanks

    I also just found this which seems to accomplish the same (command_runner.rb):
    http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/117715

    ReplyDelete
  7. hi,

    (at least in linux) there's another way of capturing the program output - without printing it to the console as well while the command is executing; this can also be found in ruby's documentation of the Class:IO...

    first, run the desired command, redirecting its output ('2>&1'): ls_cmd= IO.popen("ls 2>&1")

    then, call the returned object's readlines function to get the output generated: puts ls_cmd.readlines

    all output will be returned in an array, there's no output while running the command...

    ReplyDelete
  8. Anonymous8:07 PM

    Hello,

    yo pass arguments you can do:
    what = 'man'
    result = %x[whereis #{what}]

    ReplyDelete
  9. Anonymous4:51 PM

    Exactly what I needed. I didn't know about %x[], and that works perfect. Thanks!

    ReplyDelete
  10. how can i restore mysql dump file using ruby on rails

    ReplyDelete
  11. there's also the backtick operator:
    `uptime`

    ReplyDelete
  12. Anonymous12:16 PM

    Thanks for the tip! I knew about the system method and the backtics but couldn't find out how to process the output of a system call until I found out about %x[..]

    Kudos,

    Mauvis

    ReplyDelete
  13. There is a slight problem when using a command in system to redirect the output of any command using pipe, >, or >>. You will never get a failure of the original command as it seems system gets the redirect command return value passed back.

    Example, running system("make | tee output.txt") where "make" fails to build at some point should return false when called in system, but will give you a true due to the correct execution of the redirected output through a pipe.

    I still have not found a solution yet...

    ReplyDelete
  14. Anonymous3:07 PM

    erics i suggest u do each command individually so u can check whether its fails or not

    ReplyDelete
  15. Why is this so that exec behaves differently with and without arguments ?

    As the documentation says:

    If exec is given a single argument, that argument is taken as a line that is subject to shell expansion before being executed.

    -- and --

    If multiple arguments are given, the second and subsequent arguments are passed as parameters to command with no shell expansion.

    This way the command exec('echo', '*') will always fail yet it means the same as exec('echo *')

    Can anybody explain this ?

    ReplyDelete
  16. Very well explained!! I was looking for it. Thanks.

    ReplyDelete
  17. Rishav, "echo *" and running "echo" with the "*" argument can be, conceptually, very very different.

    Compare this:

    exec("ls ; touch /tmp/attack")

    to this:

    exec("ls", "; "touch /tmp/attack")

    The latter is much safer.

    I found this blog post looking for a way to do %x{} with parameterized expressions. Guess it can't be done.

    ReplyDelete
  18. #!/usr/bin/ruby

    require 'yaml'
    yml = YAML::load(File.open('userinfo.yml'))

    yml.each_pair {|key, value|

    #system ("echo #{value}")
    value.each do |item|
    puts item

    system("sendEmail -f test@mail.com -t item -m hello -s smtps.mail.com")
    end
    }


    The question is HOW , can i use the variable "item" inside system()

    ReplyDelete
  19. It's just ruby, so you should be able to do "#{item}" instead of "item".

    ReplyDelete
  20. Guilherme1:43 PM

    Good article.

    Could you help me please... I wanna know the IPTables using remote connections.
    For example at the same network we have one host and others virtual machines. So the host will connection remotely on this vm's and run iptables.
    However this must be a ruby implementation..
    I've tried Net-SSH, however is so I can't run any example provided there.

    Thanks. I'll really appreciate that ;)
    Giulherme

    ReplyDelete
  21. Thanks for this article! I really need this

    ReplyDelete

Note: Only a member of this blog may post a comment.