Showing posts with label Deep Zoom. Show all posts
Showing posts with label Deep Zoom. Show all posts

Thursday, August 21, 2008

Whats Next for DeepZoom

So I'll go out on a limb and without much research I can totally see this:

http://www.informationweek.com/news/personal_tech/cameras/showArticle.jhtml?articleID=210102274

mixed into some deepzoom. maybe in Silverlight 2.5? or 3.0?

Thursday, April 3, 2008

Actually Using Multi Scale Images in Silverlight - DeepZoom and Mouse Wheel Support with only Silverlight Code...

Using Silver Dragon is pretty straight forward. We start with adding a multiScaleImage to our Xaml and then pointing it at our collection 'bin' file. Generally we would take the output from DeepZoom and then input it into our web project, that we are using to build out our Silverlight application. Then we point the source to the info.bin file. In the case of the collection we built in the last sections it will look something like this:

<MultiScaleImage x:Name="SilverdragonImage" Source="/collection/info.bin" Height="400" Width="400"/>

With just this, when we run our project the MultiScaleImage will animate the zoom of our collection up to the point of being constrained by either the height or width whichever occurs first. Using our collection we created in DeepZoom we then using a default empty solution with only the MultiScaleImage added we will get this figure on the screen.

At this point we can bind events to our MultiScaleImage. The mouse events are typical but we might want to use some of the additional events such as ImageOpenSucceeded, where we can fix up the UI as needed relevant to the image. Once everything is wired up our Xaml.

<MultiScaleImage x:Name="MyFirstMultiScaleImage" Height="400" Width="400"
MouseLeftButtonDown="MyFirstMultiScaleImage_MouseLeftButtonDown" MouseLeftButtonUp="MyFirstMultiScaleImage_MouseLeftButtonUp" MouseMove="MyFirstMultiScaleImage_MouseMove" MouseLeave="MyFirstMultiScaleImage_MouseLeave"
ImageOpenFailed="MyFirstMultiScaleImage_ImageOpenFailed"
ImageOpenSucceeded="MyFirstMultiScaleImage_ImageOpenSucceeded" />

This shows the key events on the MultiScaleImage element we might want to wire up but for zooming in and out we might want to consider using the scroll events, for the mouse wheel off of the window object to give the control rich functionality for users. This will require a bit of hacking in our object so we don't have to use JavaScript in our page. Then we will actually build our class such that it will all be Silverlight code, which by design doesn't support the mouse wheel events. This primarily has to do with the fact, that different browsers expose it differently where we can overcome this in using client side ECMA code, that will be generated in our Silverlight application. Let us start with the click related functionality. For some of the functionality we are going to do, we will need to include another 'using' reference at the top of our application source or C# user control page. The using needs to look like this:

using System.Windows.Browser;

This gives us access to the DOM bridge we will use. To build out all functionality we will use we will need to add some private members and flags for state tacking and references. The initial values we need include the following listing:

public string Source = "SilverDragon/source images/OutputSdi/collection/info.bin";
private bool _IsMouseDown = false;
private bool _IsDragging = false;
private Point _LastImagePoint = new Point(0, 0);
private bool _ErrorState = false;

The first one is used as our Source for MultiScaleImage, the next two are obvious flags for conditions in our application and a point object to show us what the last 'point' was, and the last one is if the class is in an error state and broken. This all need to be members of our user control, and you can tell that is the case by the use of the key word 'private' in front of the declaration. Next we need to make sure the constructor has what it needs to set up the application into the state we will need: The constructor should have the following code listing:

if (Source.IndexOf("http") < 0)
{
string URL = System.Windows.Browser.HtmlPage.Document.DocumentUri.AbsoluteUri;
Source = URL.Substring(0, URL.LastIndexOf('/')) + "/" + Source;
}

MyFirstMultiScaleImage.Source = new Uri(Source);
HtmlPage.RegisterScriptableObject("MySilverlightObject", this);

Here we will make sure the source has a hard URL and if not we will complete it assuming the relative path is relative to our execution location on the server via http. With this we can now look at the actual binding events we use.

void MyFirstMultiScaleImage_MouseLeftButtonDown(object sender, MouseButtonEventArgs e)
{
if (!_ErrorState)
{ MyFirstMultiScaleImage.CaptureMouse();
_IsMouseDown = true; _IsDragging = false;
_LastImagePoint = MyFirstMultiScaleImage.ElementToLogicalPoint(e.GetPosition(MyFirstMultiScaleImage));
}
}

Here we check to see if the application is in its error state and if not we can perform the event. First we capture the mouse in case we are going to do any panning and we test our event flags for mouse down and dragging. Then we get the last point and save it if we are panning and/or zooming in or out. Next let's do our Mouse up with this listing:

void MyFirstMultiScaleImage_MouseLeftButtonUp(object sender, MouseButtonEventArgs e){
if(_IsMouseDown && !_ErrorState)
{
if(!_IsDragging)
{
Point MyPoint = MyFirstMultiScaleImage.ElementToLogicalPoint(e.GetPosition(MyFirstMultiScaleImage));
double ThisFactor = (Keyboard.Modifiers == ModifierKeys.Alt) ? 0.5 : 2;
MyFirstMultiScaleImage.ZoomAboutLogicalPoint(ThisFactor, MyPoint.X, MyPoint.Y);
}
MyFirstMultiScaleImage.ReleaseMouseCapture();
}
_IsMouseDown = false;
}

In this case we can see we checked that the mouse down flag is set and the error state is not true. If those are correct we also check to see if we have been 'dragging,' and if so we grab the current zoom point, calculate a factor based on the alt key, then perform a zoom operation, release the mouse capture and set the mouse down flag to false. This method then deals with either what was the mouse down event or if there was the drag operation or scroll operation. Next we will actually do our mouse move event with this listing.

void MyFirstMultiScaleImage_MouseMove(object sender, MouseEventArgs e)
{
if (_IsMouseDown && !_ErrorState)
{
_IsDragging = true;

Point p = MyFirstMultiScaleImage.ElementToLogicalPoint(e.GetPosition(MyFirstMultiScaleImage));

Point delta = new Point(p.X - _LastImagePoint.X, p.Y - _LastImagePoint.Y);

MyFirstMultiScaleImage.ViewportOrigin = new Point(MyFirstMultiScaleImage.ViewportOrigin.X - delta.X, MyFirstMultiScaleImage.ViewportOrigin.Y - delta.Y);

}
}

Here we first test for the mouse down and error state flags. If they are correct we start our drag operation by setting the value to true. Next we then grab the current point and calculate the delta point, from the last point and then set the viewport origin on our MultScaleImage. Nice and simple right? Now lets add a few methods to help deal with error conditions. First the ImageOpenFailed event that fires if something goes wrong with the source using this listing:

private void MyFirstMultiScaleImage_ImageOpenFailed(object sender, ExceptionRoutedEventArgs e){ _ErrorState = true; }

All it needs to do is set the error state flag if there is a source failure, which will basically set the application into an error state. We could add more code here to do things like switch sources, or something else if we wanted to. Next on the off chance that we are in an error state and the image open succeeds then this method will set the application back into a good state with this listing of the Image Open Succeeded event:

private void MyFirstMultiScaleImage_ImageOpenSucceeded(object sender, RoutedEventArgs e){ _ErrorState = false; }

This sets our application back into a happy state where we can perform deep zoom operations, but next we need to deal with the mouse leave event so we don't leave our MultiScaleImage in a weird state. We do this by using the mouse leave event like this listing:

private void MyFirstMultiScaleImage_MouseLeave(object sender, MouseEventArgs e)
{
_IsDragging = false;
_IsMouseDown = false;
MyFirstMultiScaleImage.ReleaseMouseCapture();
}

Here we reset both; the dragging and mouse down flags and release the mouse capture if it is captured, so we can move on. This basically completes the functionality that Silverlight 2 supports, but we can build mouse wheel support with the following section

HACK - Mouse Wheel Events in SL and with the MultiScaleImage

The 'Mouse Wheel Events' are not really supported. This is in large part due to disparity between how the different browsers support getting this event into a plug in. That however being the case also makes it worth our while to look at the fact that the mouse wheel support inside the context of the scripting engine for browsers is fairly well built out such that we can build a client side solution that feeds into our Silverlight application. However this approach feels like to much of a hack, and we can build out this same hack and make it all in our Silverlight. This still however is a hack, but it really is a lot more pleasant when the Silverlight class can wire all this on its won without us having to build out anything special in our web page (html, asp or whatever). Remember we already added the using statement so we could use the DOM bridge? Now let us start by adding a couple more properties at the top of our class with this listing.

private static string _MOUSEWHEELHANDLER_JS = "function OnMouseWheel() { ($get('Xaml1')).content.MySilverlightObject.MouseWheel(window.event.clientX, window.event.clientY, window.event.wheelDelta); } ";

private static string _MOUSEWHEELEVENT_JS = "window.onmousewheel = document.onmousewheel = OnMouseWheel; if(window.addEventListener) { window.addEventListener('DOMMouseScroll', OnMouseWheel, false); }";

Gasp, I know many of you will note that these two values contain ECMA script. Since Silverlight supports what we need and scripting does, but we don't want to be writing everything in the client, this approach allows all to be built out in our Silverlight code and it makes the class more easily reused. Next we need to add a little private method that does our black magic here with this listing:

private void InsertECMAScript(string JavaScript)
{
HtmlElement Script = HtmlPage.Document.CreateElement("script");
Script.SetAttribute("type", "text/javascript");
Script.SetProperty("text", JavaScript);
HtmlPage.Document.DocumentElement.AppendChild(script);
}

I know this really might give some people heart burn, but it really makes our DeepZoom class more portable. In this case anything we pass into this element will get tacked onto the end of our web page HTML DOM and since it is script up front it gets executed. We use this function in our object constructor like this:

InsertECMAScript(_MOUSEWHEELHANDLER_JS);
InsertECMAScript(_MOUSEWHEELEVENT_JS);

Again, this is really giving some people heart burn, but I think it is cool and much less hassle. One thing to note though is we still didn't add our Mouse Wheel event in our code. Todo this we can add this event handler as in this listing:

[ScriptableMember]
public void MouseWheel(double x, double y, int delta)
{
if (!_ErrorState && !_IsDragging)
{
double ZoomMultiple = (delta < 0) ? 1 / 1.33 : 1.33;

Point p = MyFirstMultiScaleImage.ElementToLogicalPoint(new Point(x, y));

MyFirstMultiScaleImage.ZoomAboutLogicalPoint(ZoomMultiple, p.X, p.Y);
}
}

When we compile it and run it, we now get the click zoom in and out behavior. We can use the alt key when we zoom out on click, and the mouse wheel event is tied into our MultiScaleZoom and it is a reusable class.

Thursday, March 13, 2008

Using Expression DeepZoom

The key tool for building out MultiScaleImages we will use in Silverlight is Expression DeepZoom. DeepZoom is a Simple tool like Encoder for example that allows you to import images and produce the multi-layer image collections and tile structure used by DeepZoom (MultiScaleImages) in Silverlight to produce the coolness we are looking for.


DeepZoom (talk about double entendre) then creates a DeepZoom Project lets you import all the high resolution images you want to use. Then you can compose your collection of images and then it will export and build all of your multilayer image collections, tiles and other bits as configured. With that then we can embed these into Silverlight work with multilayered images.
We learn how to use the application by opening Expression DeepZoom and getting familiar with the user interface.


Currently the interface is pretty simple. We have our splash screen that pops up. That shows existing projects or lets you open new ones etc. We also can see the tabs which are basically the 3 steps to building a Multi layered image collection and getting it ready for Silverlight. From the top file menu we can open the project dialog.


Once we give our project a name and select the type (Seadragon Project) we then can start adding images to our multi-layered image collection. On the import tab in the UI we can see we have a view area and then a scrolllist of our images. At first this will be blank and at the bottom we have a button called 'Add Image.' If you click the 'Add Image' button you can use the standard windows API file dialog to find a cool high resolution image that you want to multi layer. If you add several you will see them on the design surface.


Now that we have added some images we can start arranging them in our collection. Start by click on the tab '2. Compose' at the top of the interface. Then you can drag the images from the right on to our design surface.


Let us take a look at the interface. At the top of the UI we can see a tool bar at the top and there are 5 tools which by default will have the first one selected. The first three icons represent the cursor state on the compose view of the application. The default is for dragging our icons onto the surface. Selecting second icon you can drag the design surface.


You can see how the map in the lower left of the composition surface has moved showing what the pan functionality looks like and this map shows you where on the surface that you are. The third icon on the compose tool bar is for zooming. This allows you to drill into any given location on the composition surface.


Looking closely we can see the drop down menus for the last two icons on the composition tool bar. You can see the second icon from the right is alignment items (i.e. left, right, top, bottom etc). if you select one then all your images get laid out in the collection accordingly. The menu for the last button that distributes images horizontally or vertically.


Once we get all our images on our composition surface we are ready to export our collection. There are a number of settings but for DeepZoom in Silverlight we are good with just having a location and exporting. We do this by clicking the export button.


DeepZoom then builds our Silver Dragon (SDI) file and builds out all over tiled images. If we are just going to use it for Silverlight and are using one image we can just grab its corresponding structure including the .bin, xml and the tile jpg's. Now we are ready to actually do Seadragon, er I mean SilverDragon, er I mean DeepZoom(MultiScaleImages).

Wednesday, March 5, 2008

Silverlight MIX Announcement(s)

In the keynote today Scott G annouced SL 2. is publicly available. very cool. my favorite part whas how much they showed our Entertainment Tonight work in Silverlight. among my favorite features annouced include (I will skip all the new bits Scott put out on his blog last week):

* support for Silverlight on MAC, Windows, Linux and Mobile devices
* improved preformance especially dynamic adaptive streaming and bit rate throttling, including IIS features to support to provide bit rate throttling.
* Silverlight Add templates in visual studio
* Burning Silverlight right into video using ecoder 2
* Multi-language including Silverlight in Python
* Rich WPF UI framework (layout, databinding, skinning and styling system)
* Robust networking (post, SOAP, WCF, sockets etc.)
* LINQ and Databinding
* Local Store and cache between browser sessions
* Controls include: data grids, list box, sliders, radios, buttons calender and date picker controls, checkboxs etc.
* all controls are being shipped with open source license.
* test framework for silveright.
* control templating stucture including the abilty to change animations, and restructure the visual tree
* full intellisense and design experience in VS
* DeepZoom.... formerly Silverdragon formerly SeaDragon taking smut to the next level allowing arbitrary zooming to any resolution smoothly.
* Silverlight Sharepoint bits to make it easier to put silverlight web parts in sharepoint

SeaDragon (DeepZoom) really is cool with the 2billion pixel size images... :)