Video Scrubbing

A common feature of most video players is the ability to ‘scrub’ video or to ‘scrub’ the time line of a video. When we say ‘scrub’ we basically mean being able to change the position of the video by clicking on a line or icon and dragging it along a line that represents the video and the video changes in real time. That leads us into how we make the time line visually and then making it so we can ‘scrub’ it. The first thing we do is then is to create our Xaml. Check out our Xaml:

<Rectangle x:Name="TBStatusBarBg" Opacity="0.5" Width="327.167" Height="2" Fill="#FF8C8989" RadiusX="0" RadiusY="0" Canvas.Left="15" Canvas.Top="7.667" /><Rectangle x:Name="TBStatusBarDownloadProg" Opacity="0.5" Width="0" Height="2" Fill="#FF6C6B6B" RadiusX="0" RadiusY="0" Canvas.Left="15.333" Canvas.Top="8" /><Rectangle x:Name="TBStatusBarPlayProgress" Opacity="0.5" Width="0" Height="2" Fill="#FF353434" RadiusX="0" RadiusY="0" Canvas.Left="15" Canvas.Top="8.333" /><Rectangle x:Name="ScrubIcon" Opacity="1" Width="3" Height="8" Fill="#999999" RadiusX="5" RadiusY="5" Canvas.Left="15" Canvas.Top="6" Cursor="Hand" /><Rectangle x:Name="ScrubMask" Width="327.167" Opacity="0" Height="10" Fill="#FFF0E8E8" Canvas.Left="15" Canvas.Top="5" Cursor="Hand" />

You can make your time line any way you want but we will use 5 simple rectangles. So using our time line we will show how much the current video is downloaded and the current position in the play back. So with that our first rectangle is the background. This is the core visual representation of the video length. This doesn’t actually do anything and is static regardless of the video source but contrast of the progress and position indicator are relative to this on a percentage basis.

The next rectangle is used to show how much the video is downloaded. This rectangle is animated by an event off of the media element. The third rectangle is our position indicator that will show where the video is in playback and will in fact be the key thing that changes when we actually ‘scrub’ the video. Following that we have a rectangle that is used to as our indicators icon. It is rendered perpendicular to the others and is generally what a user will click on to scrub the time line. The last rectangle is actually an invisible mask we place over the whole sets that is actually what we bind mouse events to so that things are not so hard to click on.

The next two Xaml components are what we need to make the time line work. A media element and some story boards for animation. There are two story boards. One is a timer and the other is the position play back animation to smooth out the position change. Together this smoothes out the changes in position when scrubbing and during playback.

<Canvas.Resources> <Storyboard x:Name="Timer" Storyboard.TargetName="MediaPlayer" > <DoubleAnimation x:Name="TimerWidth" Storyboard.TargetProperty="Width" Duration="0:0:.1" /> </Storyboard> <Storyboard x:Name="PositionPlayBack" Storyboard.TargetName="TBStatusBarPlayProgress" > <DoubleAnimation x:Name="PositionPlayBackWidth" Storyboard.TargetProperty="Width" To="0" Duration="0:0:.1" /> <DoubleAnimation x:Name="PositionIconLeft" Storyboard.TargetName="ScrubIcon" Storyboard.TargetProperty="(Canvas.Left)" To="0" Duration="0:0:.1" /> </Storyboard> </Canvas.Resources> <MediaElement x:Name="MediaElement" Canvas.Left="0" Canvas.Top="0" Width="361" Height="280" ></MediaElement>


Now the hard part is actually in to wire things up. All the eventing we are doing programmatically but you could do it in Xaml as well. In our application object on load we create references to all the Xaml bits we will need to muck up. Then we bind events to the media element, to our animations and to the ‘scrub’ mask and then we set up the media element and we are complete in the onload. That key thing we want to keep in mind here is that we are starting up the timer animation. We could have this in a canvas trigger but doing it in code like this ensures that the time doesn’t start until everything is initialized and it does us some good.

this.TBStatusBarBg = rootElement.findName("TBStatusBarBg");
this.PositionPlayBackWidth = rootElement.findName("PositionPlayBackWidth");
this.PositionPlayBack = rootElement.findName("PositionPlayBack");
this.PositionIconLeft = rootElement.findName("PositionIconLeft");
this.Timer = rootElement.findName("Timer");
this.TBStatusBarPlayProgress = rootElement.findName("TBStatusBarPlayProgress");
this.ScrubIcon = rootElement.findName("ScrubIcon");
this.ScrubMask = rootElement.findName("ScrubMask");

this.MediaElement.addEventListener("DownloadProgressChanged", Silverlight.createDelegate(this, this.MediaElement_DownloadProgressChanged));
this.MediaElement.addEventListener("MediaOpened", Silverlight.createDelegate(this, this.MediaElement_MediaOpened));
this.Timer.addEventListener("Completed", Silverlight.createDelegate(this, this.Timer_Completed));
this.ScrubMask.addEventListener("MouseLeftButtonDown", Silverlight.createDelegate(this, this.ScrubMask_MouseLeftButtonDown));
this.ScrubMask.addEventListener("MouseMove", Silverlight.createDelegate(this, this.ScrubMask_MouseMove));
this.ScrubMask.addEventListener("MouseLeftButtonUp", Silverlight.createDelegate(this, this.ScrubMask_MouseLeftButtonUp));

this.MediaElement.SetValue( "AutoPlay", false );
this.MediaElement.SetValue( "Volume", this.DefaultVolume );
this.MediaElement.Source = this.Source;
this.Timer.begin();

The Timer then is only .1 seconds and then fires the completed event that we bound to it. In this event we check that the MediaOpened flag has been set and that that mouse is captured or not and starts the timer again. The media opened event that is bound to the media element sets the media opened flag when the video can start playing. Now if the media opened flag is set and the mouse is not captured then it calculates the width based on the width of the background of the time line vs the length of the video taken from the natural direction based on seconds and then the current position. Then we pause the current position animation and set the two values, one for the position bar and the other for the ‘icon’ or fourth rectangle we use as our scrub icon. After we set the critical values we then start the animation again so the change is smooth.

if( this.MediaOpened && !this.ScrubIconMouseCapture )
{
var Width = ( this.TBStatusBarBg.Width / this.MediaElement.NaturalDuration.Seconds ) * this.MediaElement.position.seconds;

this.PositionPlayBack.pause();

this.PositionPlayBackWidth.To = Width;
this.PositionIconLeft.To = Width + 15;

this.PositionPlayBack.begin();
}
this.Timer.begin();

So now we are ready to setup our scrub the time line. But first lets go over the download progress changed event. We bound this to show on our time line how much of the video is downloaded. So this event then sets the width of the download progress bar on our time line and then once it is downloaded it starts the media element playing if our property auto play we created in our class is set to true.

sender.findName("TBStatusBarDownloadProg").SetValue("Width", this.MediaElement.GetValue("DownloadProgress") * this.TBStatusBarBg.Width );

if( this.MediaElement.GetValue("DownloadProgress") == 1 )
{
if( this.AutoPlay )
{
this.Play();
}
}

A scrub operation then starts when a user clicks on our mask rectangle. This fires off the event Mouse Left Button Down. Here we first set the mouse capture flag, we call the pause method we created on the application class. We could just ‘pause’ the video our selves but since we might want to call this method in a number of places and in our application we want to change the play icon too we just wrap it in a method called pause which we call here. Once the flag is set and pause is executed we then actually ‘capture’ the mouse. Remembver on this method we loose the capture without being able to track it if the user moves off the Silverlight control. If we do nothing else if the user clicks some where on the time line to scrub then position will stay some where else until the move the mouse. So to make sure this weird effect doesn’t happy we also fire our mouse move event so that the position immediately goes to the correct position on the time line and the video play back.

ScrubMask_MouseLeftButtonDown: function(sender, eventArgs)
{
this.ScrubIconMouseCapture = true;
this.Pause();
sender.captureMouse();
this.ScrubMask_MouseMove(sender, eventArgs);
},

ScrubMask_MouseMove: function(sender, eventArgs)
{
if( this.ScrubIconMouseCapture )
{
var x = eventArgs.getPosition(null).x - 40

if( x > this.TBStatusBarBg.Width )
{
x = this.TBStatusBarBg.Width + 15;
}
else if( x < x =" 15;" to =" x" to =" x;" newtime =" x"> this.MediaElement.NaturalDuration.Seconds )
{
NewTime = this.MediaElement.NaturalDuration.Seconds;
}

try
{
this.MediaElement.position = "00:" + this.formatTime(NewTime);
}
catch(Exception)
{

}
}
},

The mouse move event as the user holds down the mouse button then actually mucks up our time line and video and is actually ‘scrubbing’ the video. First this method fires any time the mouse moves over the mask so as to not mess with the user too much we want to make sure we have the mouse capture before doing anything. If we do we then grab the x value of the mouse and change it by our offset of the control. Next we need to make sure the value is not to big or to small, for example if the user moves the mouse way past either end of the time line we would effectively be past the end or before the beginning so we reset our ‘x’ value that we will use accordingly using an if else if statement based on the bounds of the time line. Since the position change animation is current paused during the capture event because we called pause on mouse down we just set the two double animation values. Then we restart the animation. Also on regular mouse moves we just set the values and call the begin method. After this we then create a new time value conversion for what the new play back position will be. This is based on our x value divided by the line width divided by natural duration of the video. Then if the new time is greater then the natural duration we set it to the end of the media. Otherwise we go ahead and set the new position and use some error handling to make sure this operation doesn’t blow up.
When the user is done with scrubbing the video with our mouse move event and they lift the mouse button up we call our mouse up button to let the video continue playing. In this event we set our mouse capture flag back to false, call the play method which we wrapped so we could change icons etc like our pause we implemented and then we can release our mouse capture.

ScrubMask_MouseLeftButtonUp: function(sender, eventArgs)
{
this.ScrubIconMouseCapture = false;
this.Play();
sender.releaseMouseCapture();
},

You can implement this a number of ways but this is how I hacked it together and it works well. Now that you can scrub the video time line and your dangerous to yourself and others