To add a custom transition, you have to register a transition function at the Transitions class.

Here is a sample:

Transitions.register("shakyLinear",
    function(ratio:Number):Number
    {
       if (ratio == 0.0 || ratio == 1.0) return ratio;
       else return ratio + Math.sin(getTimer() / 100) * 0.1 - 0.05;
    });

Now you can use that transition just like any other when you create a tween:

var tween:Tween = new Tween(object, 2.0, "shakyLinear");

Here are a few more transition that you can add to your Starling toolbox:

Transitions.register("easeInCubic",
    function(ratio:Number):Number
    {
        return ratio * ratio * ratio * ratio;
    });	
 
Transitions.register("easeInQuadratic",
    function(ratio:Number):Number
    {
        return ratio * ratio * ratio * ratio  * ratio;
    });			
 
Transitions.register("easeOutCubic",
    function(ratio:Number):Number
    {
        var invRatio:Number = ratio - 1.0;
        return invRatio * invRatio * invRatio * invRatio  * invRatio * invRatio  * invRatio + 1;
    });