UI Virtualization
Today’s post is motivated by a scenario that’s common in business applications: displaying and interacting with a large data set. We’ll quickly run into performance problems if we use the naïve approach of loading the entire data set into memory and creating UI elements for each data item. Fortunately, there are some things we can do to make sure our applications perform well, even with extremely large data sets.
The first approach is called “UI virtualization.” A control that supports UI virtualization is smart enough to create only the UI elements needed to display the data items that are actually visible on screen. For example, suppose we have a scrolling ListBox that is data bound to 1,000,000 items but only 100 are visible at any time. Without UI virtualization, the ListBox would create a million ListBoxItems - a slow process - and include them in the UI, even though only a hundred of them are visible. With UI virtualization, on the other hand, the ListBox would only create 100 ListBoxItems (or a few more, to improve scrolling performance).
The second approach, called “data virtualization,” goes one step further. A control that uses data virtualization doesn’t load all the data items into memory. Instead, it only loads the ones it needs to display. In our ListBox example above, a solution using data virtualization would only keep about 100 data items in memory at any given time.
In this post, I will talk about the current level of support for UI virtualization in Silverlight and WPF. In my next post, I will discuss data virtualization.
UI virtualization in Silverlight
Silverlight 3 just shipped! I am very pleased to say that with this release, Silverlight’s ListBox now supports UI virtualization! This feature was not part of the beta release - it’s brand new in the final release. I have to admit it’s my favorite new feature of Silverlight 3.
It was possible to work around the lack of UI virtualization before (in fact, I wrote a virtualized ListBox in Silverlight a year ago), but it wasn’t straightforward. I’m very glad that this feature is now part of Silverlight 3!
If you’re familiar with the UI virtualization provided by WPF’s controls, you’re probably curious about the level of support for virtualization in Silverlight. Just like WPF, Silverlight supports container recycling, but there is no support for deferred scrolling or for UI virtualization with hierarchical data. I will expand on these concepts below while discussing UI virtualization in WPF.
UI virtualization in WPF
WPF has supported UI virtualization for a long time. The ListBox and ListView controls use VirtualizingStackPanel as their default panel, and VSP knows to create UI containers (ListBoxItems or ListViewItems) when new items are about to be shown in the UI, and to discard those containers when items are scrolled out of view.
If you’re using another ItemsControl (such as ComboBox) that doesn’t use VirtualizingStackPanel by default, you can change the panel used by the control in a very simple way:
<ComboBox ItemsSource="{Binding}">
<ComboBox.ItemsPanel>
<ItemsPanelTemplate>
<VirtualizingStackPanel />
</ItemsPanelTemplate>
</ComboBox.ItemsPanel>
</ComboBox>
The UI virtualization support in .NET 3.5 was already pretty solid, but the WPF team decided to further improve UI virtualization in .NET 3.5 SP1. With that release, the following new features were introduced:
Container recycling
.NET 3.5 SP1 supports the reuse of UI containers already in memory. For example, imagine that when a ListBox is loaded, 30 ListBoxItems are created to display the visible data. When the user scrolls the ListBox, instead of discarding ListBoxItems that scroll out of view and creating new ones for the data items that scroll into view, WPF reuses the existing ListBoxItems. This results in significant performance improvements compared to previous versions because it decreases the time spent initializing ListBoxItems. And since garbage collection is not instantaneous, it also reduces the number of ListBoxItems in memory at one time.
You can enable container recycling by setting the attached property “VirtualizingStackPanel.VirtualizationMode” to “Recycling” on your control:
<ListBox VirtualizingStackPanel.VirtualizationMode="Recycling" … />
To maintain backwards compatibility with the behavior of earlier versions, container recycling is disabled by default (the default VirtualizationMode is “Standard”). As a rule of thumb, I suggest setting this property every time you create a control that requires scrolling to display data items.
Silverlight 3 also supports container recycling, but it’s enabled by default for ListBox, so there is no need to set the “VirtualizationMode” to “Recycling” explicitly. This is a slight incompatibility between the two frameworks. Because I frequently switch back and forth between Silverlight and WPF, I’d rather be explicit about it every time so that I don’t forget it when I need it.
Deferred scrolling
“Deferred scrolling” is a feature that allows the user to drag the scroll bar thumb around without changing the displayed items until the scroll bar thumb is released. This improves the application’s perceived responsiveness to scrolling when the items are displayed using complex templates, though of course, the user can’t see the items they’re scrolling through.
With .NET 3.5 SP1, it is possible to enable deferred scrolling by setting an attached property on the control:
<ListBox ScrollViewer.IsDeferredScrollingEnabled="True" … />
Again, for backwards compatibility reasons, this feature is disabled by default. Deferred scrolling is not supported in Silverlight 3.
Hierarchical data
In .NET 3.5 SP1, the WPF team extended UI virtualization to TreeView by adding support for hierarchical data to VirtualizingStackPanel. As a consequence, the container recycling and deferred scrolling features discussed above also apply to hierarchical data. UI virtualization is disabled by default in TreeView - here’s how you enable it:
<TreeView VirtualizingStackPanel.IsVirtualizing="True" … />
This property is useful not just for TreeView, but for any control that uses VirtualizingStackPanel and that doesn’t set IsVirtualizing to true (ItemsControl, for example). ListBox already sets IsVirtualizing to True by default, so there is no need to set it explicitly.
Silverlight 3 doesn’t support UI virtualization for hierarchical data. It also doesn’t allow you to set the “IsVirtualizing” property. If your Silverlight control scrolls and uses a VirtualizingStackPanel to display non-hierarchical data, UI virtualization is enabled automatically.
Limitations
.NET 3.5 SP1 fixed many previous limitations on UI virtualization, but a couple still remain:
- ScrollViewer currently allows two scrolling modes: smooth pixel-by-pixel scrolling (CanContentScroll = false) or discrete item-by-item scrolling (CanContentScroll = true). Currently WPF supports UI virtualization only when scrolling by item. Pixel-based scrolling is also called “physical scrolling” and item-based scrolling is also called “logical scrolling”.
- When using data binding’s “Grouping” feature, there is no support for UI virtualization.
These are really the same limitation. If you look at the default style for ListBox, ListView and ComboBox, you will find the following trigger:
<Trigger Property="IsGrouping" Value="true">
<Setter Property="ScrollViewer.CanContentScroll" Value="false"/>
</Trigger>
The implementation of Grouping assumes that each group is a separate item in the ItemsControl that contains it. Since each group can (and typically does) have many sub-items, scrolling by item would result in a really bad user experience - scrolling down a bit would cause a big jump to the top of the next group. That’s why the team decided to switch to pixel based scrolling when displaying grouped data. The unfortunate consequence is that no UI virtualization is supported when grouping.
I’m often asked if there is a way to work around this limitation. Well, anything is possible, but there is no *easy* workaround. You would have to re-implement significant portions of the current virtualization logic to combine pixel-based scrolling with UI virtualization. You would also have to solve some interesting problems that come with it. For example, how do you calculate the size of the thumb when the item containers have different heights? (Remember that you don’t know the height of the virtualized containers - you only know the height of the containers that are currently displayed.) You could assume an average based on the heights you do know, or you could keep a list with the item heights as items are brought into memory (which would increase accuracy of the thumb size as the user interacts with the control). You could also decide that pixel-based scrolling only works with items that are of the same height - this would simplify the solution. So, yes, you could come up with a solution to work around this limitation, but it’s not trivial.
And this brings me to another change introduced in Silverlight 3. Silverlight 2′s ListBox used to support only pixel-based scrolling, but with the introduction of UI virtualization in Silverlight 3, the default scrolling mode for ListBox is now item-based. Unlike WPF, Silverlight’s ScrollViewer doesn’t have a “CanContentScroll” property. In Silverlight 3, if your ListBox uses VSP it will scroll by item and have virtualization enabled, and if you change it to use StackPanel instead, it will scroll by pixel and have virtualization disabled.
Joe Gannon
This info is great and I would agree this is really a big enhancement for version 3.
July 11, 2009 at 6:39 pm
Amit
It is looks like we should wait for silverlight 4 or 5 to create a serious application there are many diferences and limitations in sl-3.
July 11, 2009 at 10:44 pm
Guillem
Thank you for this summary of virtualization.
I am wondering how sorting can be applied on virtualized datasets, you have not mentioned it. Moreover, does it make any sense to do virtualization and sorting on a frequently updated dataset automatically?
The scenario I am talking about is:
- you have a large set of items which you do not want to display for performance reasons
- each of these items can potentially be updated at any point in time
- when enabling sorting on the item’s properties, the sort would need to be reevaluated based on new updated values in the dataset
I know this goes a bit beyond your post, but I am interested in your opinion.
July 12, 2009 at 12:00 am
Bea
Hi Guillem,
When your in-memory collection is sorted, the control sees that as a collection “reset”. The control doesn’t care whether the reset is caused by sorting, filtering, fetching large amounts of new data, or whatever else. When the control is informed of a reset, it figures out how to display the new collection (essentially, the same process it goes through when the data is loaded initially). It figures out once again what data items are visible and should have containers created to wrap them, and which ones can be virtualized.
So, in summary, I expect this scenario to “just work”. Let me know if it doesn’t.
Bea
July 12, 2009 at 2:10 pm
siva
SL maturing well…
July 12, 2009 at 12:46 pm
Wessty
With the release of Silverlight 3, I have been running to catch up on the updates and changes that have been added throughout the framework. This article has aided me a lot in seeing how SL has matured since version two. Great post.
July 13, 2009 at 7:00 am
Mark
Is virtualization supported for ComboBox when using non-hierarchical data? Simply setting the panel to VSP doesn’t appear to work - every item gets created when the drop down occurs. Changing it to ListBox (i.e. just the tag and nothing else) then only creates the visible items + 1.
July 13, 2009 at 9:57 am
Bea
Hi Mark,
Yes, ComboBox should virtualize its items when setting its Panel to VSP. Here’s the XAML:
<ComboBox ItemsSource=”{Binding}” DisplayMemberPath=”Name” Name=”cb” >
<ComboBox.ItemsPanel>
<ItemsPanelTemplate>
<VirtualizingStackPanel />
</ItemsPanelTemplate>
</ComboBox.ItemsPanel>
</ComboBox>
This should virtualize UI both in WPF and Silverlight. Silverlight has one other slight limitation, when compared to WPF: the whole list of items is enumerated at load time. As a consequence, a list that is really really large could take a little while to load, but once it’s loaded, it should scroll very fast. WPF’s internal code only enumerates through the visible data items, so load time in WPF should be really quick, independently from the size of the data source.
What technology are you using - WPF or Silverlight? If you’re using Silverlight, are you sure you’re using Silverlight 3? How are you counting the number of items created?
Bea
July 13, 2009 at 10:07 pm
Mark
It’s SL3. I’m setting the panel template as you show, to determine when it’s getting the items, I’ve got a DataTemplate which binds to a property and when the property getter executes I’m outputing a debug line. When I virtualize a ListBox I see only the visible items (~+1) hit the getter. When I change it to a ComboBox all items are hit when the drop down occurs - even though I have the drop down size to show only 2-3.
Here’s an example showing ComboBox next to ListBox — perhaps it truly is virtualizing, it just isn’t what I expected..
http://files.me.com/mark.c.smith/dnifp7
July 14, 2009 at 11:55 am
Bea
Hi Mark,
Ah, yes, you’re hitting the Silverlight “limitation” that I mention on my first reply to you. I will talk more about this on my next post, in the context of data virtualization. Silverlight does indeed access all the data items, while WPF only accesses the ones that are displayed. This is a limitation of the current implementation of Silverlight, and makes it very hard to implement custom data virtualization solutions. However, that is independent of how many UI elements get created to wrap that data (UI virtualization). Even though all data items are accessed, Silverlight should only create containers for the data items that need to be displayed.
The issue you found is of course less than ideal for Silverlight. It will result in a delay at load time that is proportional to the number of items in the collection.
Does my explanation make sense?
Bea
July 14, 2009 at 12:37 pm
Mark
Yes, thanks for the clarification - I look forward to your post!
July 14, 2009 at 2:26 pm
Bea
Hi Mark,
I tried your scenario (ComboBox using VPS bound to a large data set) and it seems in fact that it’s not doing UI virtualization. This works well in WPF, but not in Silverlight.
Setting a breakpoint in the getter doesn’t confirm this (since Silverlight gets all items at load time as I explain in today’s post), so I used an attached property on ComboBoxItem to check how many wrappers are being created:
<Style x:Key=”itemStyle” TargetType=”ComboBoxItem”>
<Setter Property=”local:MainPage.MyProperty” Value=”1″ />
</Style>
Then, when I register for the attached property, I add a handler that increments a static int every time the property is changed. As you mention, I see as many ComboBoxItems being created as items in my collection. This seems like a bug in Silverlight.
Someone else commented on this issue in my blog and shared a workaround. I am about to approve it, so you can look for it further down in the list of comments for this post. Hopefully you can join forces in working around this problem.
Thanks,
Bea
July 26, 2009 at 4:16 pm
ivl
I’ve got the same problem as Mark.
ListBox works very well with the VSP whereas ComboBox doesn’t in Silverlight 3. I wonder whether one can make the ComboBox use a ListBox for it’s drop down popup…
July 15, 2009 at 6:14 pm
ivl
Ok so I got a little bit further. I created a control which inherits from ComboBox and supports Virtualization by using a ListBox for the drop down. It seems that when you set the MaxHeight property of ListBox to a value, Virtualization automatically works. The only thing left to do is to get keyboard navigation working with this ComboBox so that it precicely emulates the default ComboBox. Any comments on the following code are welcome.
Here’s the code:
public class VirtualizingComboBox : ComboBox
{
private ComboBoxListBox _listBox;
public override void OnApplyTemplate()
{
base.OnApplyTemplate();
ScrollViewer scrollViewer = GetTemplateChild(“ScrollViewer”) as ScrollViewer;
if (scrollViewer != null)
{
_listBox = new ComboBoxListBox(this)
{
ItemTemplate = ItemTemplate,
ItemContainerStyle = ItemContainerStyle,
ItemsSource = ItemsSource,
MaxHeight = 120, // TODO: Bind this to MaxDropDownHeight? what if MaxDropDownHeight changes?
BorderThickness = new Thickness(0),
Padding = new Thickness(0),
Margin = new Thickness(0),
SelectionMode = SelectionMode.Extended
};
Binding binding = new Binding(“SelectedIndex”)
{
Source = this,
Mode = BindingMode.TwoWay
};
_listBox.SetBinding(ListBox.SelectedIndexProperty, binding);
scrollViewer.Content = _listBox;
}
}
protected override void OnDropDownOpened(EventArgs e)
{
if (_listBox != null)
_listBox.Focus();
base.OnDropDownOpened(e);
}
protected override void OnKeyDown(KeyEventArgs e)
{
if (!IsDropDownOpen)
base.OnKeyDown(e);
}
private class ComboBoxListBox : ListBox
{
private int _focusIndex;
private VirtualizingComboBox _comboBox;
public ComboBoxListBox(VirtualizingComboBox comboBox)
: base()
{
_comboBox = comboBox;
_focusIndex = comboBox.SelectedIndex;
}
protected override void OnKeyDown(KeyEventArgs e)
{
e.Handled = true;
// TODO: The following does not work.
// Need to make keyboard navigation work, including
// Space, Enter, PageUp, PageDown, etc.
int num = -1;
switch (e.Key)
{
case Key.Up:
num = _focusIndex - 1;
break;
case Key.Down:
num = _focusIndex + 1;
break;
default:
e.Handled = false;
break;
}
if (num != -1)
{
num = Math.Min(num, base.Items.Count - 1);
FocusItem(num);
_focusIndex = num;
}
}
protected override DependencyObject GetContainerForItemOverride()
{
ListBoxItem item = (ListBoxItem)base.GetContainerForItemOverride();
item.MouseLeftButtonUp += new MouseButtonEventHandler(Item_MouseLeftButtonUp);
return item;
}
private void Item_MouseLeftButtonUp(object sender, MouseButtonEventArgs e)
{
if (!e.Handled)
{
if (_comboBox != null)
{
e.Handled = true;
_comboBox.IsDropDownOpen = false;
}
}
}
protected override void OnGotFocus(RoutedEventArgs e)
{
_focusIndex = SelectedIndex;
if (_focusIndex >= 0)
{
FocusItem(_focusIndex);
}
//base.OnGotFocus(e);
}
private void FocusItem(int index)
{
ListBoxItem item = (ListBoxItem)ItemContainerGenerator.ContainerFromIndex(index);
if (item != null)
{
item.Focus();
}
}
}
}
July 16, 2009 at 3:04 pm
ivl
Sorry just a minor correction to the previous code snippet.
No need for the line which reads:
SelectionMode = SelectionMode.Extended
I was just testing out the different selection modes of the ListBox to see if that can help with getting the keyboard navigation working.
July 16, 2009 at 3:10 pm
Bea
Hi Ivan,
You and Mark are both hitting a Silverlight bug.
I like your approach to solving this problem. Were you able to get keyboard navigation working? That could be pretty tricky…
If you’ve gotten further on this, I would appreciate that you would post the final solution - either here or in your blog if you have one. Your solution may save others quite a bit of time.
Thanks,
Bea
July 26, 2009 at 4:26 pm
ivl
Hi Bea,
Sorry for the super late reply, but I was overseas.
Unfortunately I still haven’t gotten the keyboard navigation working properly. I spent a few hours after my posting trying, but a lot of things in the ComboBox and Selector classes are marked as internal to the System.Windows assembly, making it very difficult to implement.
The keyboard navigation works fine when the drop down of my VirtualizingComboBox is closed, but doesn’t work properly when the drop down is open. Due to time constraints I left the VirtualizingComboBox as is.
I am interested in Thomas’ solution further down below (thanks Thomas) and hope that he posts some code. I might look at the AutoCompleteBox code myself in Reflector when I revisit this problem at a later stage.
Thanks Bea.
September 7, 2009 at 4:55 am
Jose Fajardo
Hi,
Firstly great article..
I completely understand the need to Virtualize, however being forced to use the item based scrolling over the pixel based scrolling is very very bad from an user experience perspective.
Visually it looks so much nicer when scrolling by pixel.
Anychance you could share some insights onto why it needed to be item based scrolling for virtualized lists?
July 17, 2009 at 6:03 pm
Bea
Hi Jose,
When using item-based scrolling, it’s easy to calculate the size of the scrollbar thumb correctly. In this case, the size of the thumb depends on the total number of items and number of items visible, and we have that information.
When using pixel-based scrolling, the size of the thumb depends on the numer of pixels visible and total number of pixels. If we wanted to calculate the number of pixels used to display the whole list, we would have to render all UI item wrappers so that we could measure them, which goes against what we’re trying to achieve with UI virtualization. Remember that items can be of different heights, so we can not assume the height of items that are not rendered. It’s a much harder problem to solve.
Does this make sense?
Bea
July 26, 2009 at 2:37 pm
Jose Fajardo
Thanks Bea, thats pretty much what I expected.
Now the question i ask is this, if we didn’t care about setting the size of the thumb and actually wanted to hide the scrollbar and only expose a touch like way of navigating the list (like the iPhone scroll lists). Could we somehow tweak the virtualized logic to work for pixel based movement and just ignore setting the thumb heights?
Basically is the only reason for using item based scrolling for virtualized lists because of the thumb and setting it to the correct height?
July 27, 2009 at 5:11 pm
Bea
Sure, anything is possible
You would need to re-implement significant portions of the virtualization code, though, such as VirtualizingStackPanel and ItemContainerGenerator and potentially more. It would be possible, but not straightforward, so it really depends how much time and motivation you have to solve this problem. If you do decide to take on the challenge, please tell me about it and I’ll advertise it on my blog.
Bea
July 27, 2009 at 5:28 pm
abcd
Thanks, it helped me a lot!
July 21, 2009 at 4:29 am
Simon Hecht
Hey,
I had the problem that virtualisation helped me but scrolling went slow because my listbox’s items are too big to load fast enough to scroll fluently. Recycling worked but just for a few seconds until garbadge collector came. So I did a little trick: Each time an item loads I add a reference to the control in which the listbox exist which causes that garbadge collector has to go home with empty hands and scrolling spikes just for the first time.
July 28, 2009 at 6:12 am
Simon Hecht
Doesn’t seem to work. I have some strange behaviour now. When I add a column in the item’s listview the listbox starts creating new items.
July 28, 2009 at 6:38 am
Bea
Hi Simon,
In your scenario, I think the best choice is to simplify the styling of each ListBox item. You may need to profile your app.
Bea
November 20, 2009 at 10:30 am
Thomas
Hi Mark, Ivan and Bea,
Another workaround for the ComboBox issue is to use a AutoCompleteBox with a template so it looks like a combobox.
This should work since the AutoCompleteBox internally uses a ListBox.
I will post some code later.
Thomas
August 28, 2009 at 1:02 pm
Bea
Great idea. Thanks for posting!
Bea
August 28, 2009 at 1:07 pm
majkez
Dear Bea,
is it possible to use virtualization in Silverlight 3 on Canvas?
Thanks in advance!
Michael
December 30, 2009 at 4:04 am
Bea
Hi Michael,
Canvas does not have support for UI virtualization, in Silverlight or WPF. VirtualizingStackPanel is the one panel that does.
Thanks,
Bea
January 21, 2010 at 11:38 am