viernes, 29 de abril de 2011

Released Shikashi v0.4.0 (and dependencies)

Shikashi is an sandbox for ruby that handles all ruby method calls executed in the interpreter to allow or deny these calls depending on the receiver object, the method name, the source file from where the call was originated

For more info about the project, visit the project page

You can install the gem by doing


gem install shikashi


New Enhancements

removed evalmimic dependency

evalmimic is a gem to emulate the behavior of the binding argument of the eval method (the default binding), to allow the API to do this:

a = 5
Sandbox.run("a+1", Privileges.allow_method(:+)) # return 6


The implementation of evalmimic is somewhat complex because it relies in a C extension and even implements some low level hacks using unsupported features of MRI (e.g. evalmimic will not compile and install for ruby 1.9)

So it was decided to remove the evalmimic dependency from shikashi and evalhook, and remove the feature shown in the above example. The only difference now is that you must add the binding as parameter if you decide to execute the sandbox in that way.

a = 5
Sandbox.run("a+1", Privileges.allow_method(:+), binding) # return 6


And if you do not specify the binding, the default behavior is use the global binding nested in the sandbox namespace


Sugar Syntax

As seen in previous code examples, it's no longer necessary to instanciate lot of objects in order to execute code in the sandbox, only Sandbox.run and Privileges now use method chaining syntax. Example:

require "shikashi"

Sandbox.run('print "hello world\n"', Privileges.allow_method(:print))

$a = 1
Sandbox.run('print $a, "\n"',
Privileges.allow_method(:print).allow_global_read(:$a)
)



Control over read access of constants and global variables

Now, you must grant read privileges over global variables and constants in order to allow the read access to them. By default, trying to access to global variables and constants will result on SecurityError exceptions. Constants defined inside the base namespace of the sandbox are allowed by default (e.g. classes defined in the same code)

# this will work
include Shikashi
Sandbox.run("
class X
def foo
end
end
X.new.foo
", Privileges.allow_method(:new))

$a = 4
Sandbox.run("$a", Privileges.allow_global_read(:$a)) # 4

A = 4
Sandbox.run("A", Privileges.allow_const_read("A") # 4

Sandbox.run("$a") # raise SecurityError

Sandbox.run("A") # raise SecurityError




Interception of method calls using super on evalhook


Now, call to super methods are intercepted by evalhook and rejected by shikashi when appropiate

include Shikashi
#=begin
Sandbox.run("
class X
def system(*args)
super # raise SecurityError
end
end
X.new.system('ls -l')
", Privileges.allow_method(:new))


Refactor to use Ruby2Ruby on partialruby

Partialruby is the gem that emulates ruby using ruby to allow the changes to AST needed by evalhook for interceptions. Up to version 0.1.0, partialruby implements the emulation with abstract tree processing from scratch. Now, at released version 0.2.0, partialruby relies on more mature and stable gem Ruby2Ruby which converts AST to executable ruby source code

Links


martes, 19 de abril de 2011

Released ImageRuby - a flexible ruby gem for image processing

ImageRuby is a flexible gem for image processing, it's designed to be easy to install and use. The core of ImageRuby is written in pure ruby and the installation is as simple as execute "gem install imageruby". The API of the library take advantage of sugar syntax constructions specific of ruby language such as method chaining and blocks. e.g, the next code loads an image from "input.bmp", crop the rectangle from 0,0 to 49,49 (50x50 pixels), replace the orange color to red color, replace the black color to orange color and finally saves the image as "output.bmp"

require "imageruby"

ImageRuby::Image.from_file("input.bmp")[0..49,0..49].
color_replace(Color.orange,Color.red).
color_replace(Color.black,Color.orange).
save("output.bmp", :bmp)

Current Status

The ImageRuby had his first release (version 0.1.0) the last week with imageruby-c (extension to override a few methods with C methods) and imageruby-bmp (support for save and load images on BMP format),

You can install the gem by doing

gem install imageruby


The current version o imageruby has no dependencies, and it's installation should be as simple as that, optionally, you can install imageruby-c to improve the performance of draw and color_replace methods and imageruby-bmp to have support for bmp images.
The extensions take effect automagically by installing the gem, there is no need to do something specific in the ruby script (e.g. when install imageruby-c the methods will be optimized)

Goals of the project

  • Become synonymous of image processing in the ruby world, displacing other options in the "market" but re-using the existent stuff (giving credit, obviously)
  • Highlight deficiencies in the ruby language when used for heavy processing (e.g. draw method implemented in ruby is very slow)
  • Spike workarounds for the Ruby language issues
  • Spike "soft" dependency model or plugin gems to avoid the problems of "static" dependencies

Competitors

There is a range of gems for image processing on ruby, most of these are based on C libraries and implies the need for a compiler in order to install them. Some have "low level" bugs such memory leaks but most of them have many possibilities through their API

  • RMagick: Ruby wrapper of the archi-famous ImageMagick library. Has reported problems with low level memory handling and is not recommended for services environment (such as a Rails application) but is good for scripting
  • ImageScience: More stable alternative to RMagick, written on top of FreeImage and RubyInline (for more optimal code in C), an issue when using this library is the RubyInline dependency which implies the need for GCC installed on the system, something that is simple on Linux but is so tricky in other environments such Windows and Heroku
  • Camellia: A C/C++ library with a Ruby interface apparently directed to photo processing, there is no gem and the installation should be using the classic command sequence ./configure; make; make install.
  • Devil: Wrapper of C library of the same name. It has a wide range of operations over an image including blur and equalize. (see documentation)
  • Ruby-Processing: Well documented library mainly oriented to interactive media including video features. Has nice pencil functions
Future

New image formats

One of the next big steps will be develop decoder and encoders for common image formats, the encoders and decoders can be released as separated gems (like imageruby-bmp) without need for change imageruby core. In fact, anyone can make their own encoder or decoder.

Competitors become allies

Re-use the existing development around image processing in the ruby world, divided in two big groups: Interfaces and Implementations

Reusing interfaces implies the creation of "mocks" or "wrappers" of ImageRuby providing the same API of other library (e.g. RMagick or ImageScience) to reduce the learning curve of ImageRuby API and provide a method to replace that library with ImageScience transparently for code developed on top of that (e.g. a rails application using ImageMagick and then the ImageMagick gem is replaced by ImageRuby, app will not need to be modified)

Reusing implementation is about build features on top of other image libraries, but without setting a "hard" o "static" dependency with the library, but by creating a optional extension for ImageRuby (something like a port between libraries). Example: imageruby-devil.

Could be installed so:
gem install imageruby-devil

And used like
# is not necessary to make a explicit require of imageruby-devil
require "imageruby"

image = ImageRuby::Image.from_file("input.bmp")

image.color_replace!(Color.black, Color.purple) # use a method of ImageRuby
image.devil.blur(1) # use a method of Devil library on the image

image.save("output.bmp", :bmp)

Feedback for Ruby improvements

One of the goals of ImageRuby is to Highlight deficiencies in the ruby language , methods such draw or color_replace process hundred of thousands of pixels per image which in the ruby language implies still much more unnecessary rubymorphic calls just to access each bit of pixel data of the processed images. This produces a very very slow result, the current workaround for this issue is override the "heavy" methods with the imageruby-c extension. This was achieved as a optional extension to avoid the unnecessary hard dependency of have a compiler in the system, ImageRuby without imageruby-c installed HAS the draw and color_replace method, but with the imageruby-c gem installed, is much much more fast (near to 100x more faster)

Links

ImageRuby rdoc reference: http://tario.github.com/imageruby/doc/
ImageRuby GitHub site: https://github.com/tario/imageruby

lunes, 4 de abril de 2011

Evalhook v0.3.0 released and shikashi works on Heroku

Evalhook is a vital dependency of shikashi gem, provide the feature of execute managed ruby code in the way of allow "hooks" of common events like method calls, system calls, assignment of global variables and constants, etc...

Fixed Issues

With this release, problems of compatibility of actual version of shikashi (v0.3.1 at the moment of writing this post) were solved. The most important change was refactor of the hooking to make it more portable, less prone to error and implemented in pure ruby (the C extension was removed from the gem). this implies that now shikashi works on heroku and possibly in other environments

Update/Install

To update current installation of evalhook gem, run in terminal:

gem install evalhook

NOTE: To update the gem on heroku you must change Gemfile and Gemfile.lock files, then push this files to your application git repository, setting the content of Gemfile to something like that:

source :rubygems

gem 'shikashi'
gem 'evalhook', '0.3.0'
gem 'partialruby', '0.1.0'


Fundamentals

The most important concept behind evalhook is the AST hook technique which consist in change the Abstract Syntax Tree before it is interpreted and executed (view this other post for more info)

In previous versions of evalhook, the method to implement the AST hook was change the internal structures (representations of AST) on the version 1.8 of Matz Ruby Interpreter. This implementation breaks the encapsulation of MRI and depends on the specific implementation of that ruby interpreter making a very poor portability and being prone to error (e.g. the ruby interpreter crashes in heroku)

The reimplementation of the evalhook core is based on partialruby, a "para-interpreter" which executes ruby code by emulating it using ruby. Partialruby also exports the hooking services needed by evalhook thus avoiding the need for "hacks" to implement that service in a interpreter that does not have that. This concept is much more portable and theoretically should work with any interpreter of ruby 1.8 code.

Links

Shikashi Project description: http://tario-project.blogspot.com/2011/03/shikashi-flexible-sandbox-for-ruby.html
AST Hook fundamentals: http://tario-project.blogspot.com/2011/03/how-tree-hooks-works-and-what-are.html
Evalhook on Github https://github.com/tario/evalhook
Shikashi on Github: https://github.com/tario/shikashi
Heroku; http://heroku.com