Large lists and virtualization

[ataft]

I'm running into performance issues with lists larger than ~4,000 items. Are there any plans for supporting very large lists? Either through virtualization, like http://bl.ocks.org/jasondavies/3689677. Or forcing a search to narrow down the result so there aren't too many items. I suppose this would also require bootstrap-select to be based off of JSON rather than a element, but this might also be a great addition for performance improvement.

[Nedlinin]

This is also an issue for me. Would love to see it resolved.

[caseyjhol]

I do have plans to implement this in an upcoming update.

[stewkuhlo]

+1

[caseyjhol]

I just added this feature in the virtual-scrolling branch. It's still a WIP, and there are a few features missing, some I'm aware of (the startsWith feature for example), and some I'm sure I'm not. Try it out - I'm looking for feedback.

[ataft]

Wow, this is so great, thanks! I've checked it out on some large lists, and the performance is excellent!

Here are some items from my initial look at it:

1. As I scroll, the listbox width shrinks and expands to fit the virtual content
2. Items are not showing in my list, but they show up when I search. This is probably because I'm loading values when the user opens the list ('show.bs.dropdown') and then calling selectpicker('refresh') when they're loaded.
3. Performance is excellent up to 100-200K items, which is more than enough. The select list with all the options is still in the DOM, which I suppose is a requirement for bootstrap-select to work, but if the list could be based off a JSON object rather than a select then there'd be no limits.

Excited for it to be released. Thanks again!

[caseyjhol]

Can you link me to an example (JSFiddle or Plunker) of your setup so I can troubleshoot and see exactly what you're referring to? I can't recreate issue 1 - what web browser are you using?

[longlivetmj]

I was struggling with ajax-bootstrap-select due to performance issues. With the given modifications it works as charm. But minor issues.

1. Undefined option is shown while searching
2. Already cached data is not shown
   Checking the code to resolve the same.

Thanks a million

[ataft]

I'm using Chrome 56.0.2924.87, but the same issue occurs in Internet Explorer 11 too. In IE, the handle on the scrollbar stays throughout scrolling at least, but in Chrome it jumps around like crazy.

Here's a JSFiddle: http://jsfiddle.net/andytaft/54m1ybzn/1/

I had to copy the raw source into the fiddle as I wasn't sure if/how to link to the github source (JSFiddle wants a CDN)...

[caseyjhol]

Thanks - able to recreate it now. Working on a fix. Just updated it with some more performance improvements (opening the menu with a lot of options was previously very slow).

[caseyjhol]

The issue with the width has been fixed. Working on the disappearing option issue.

[andrews05]

I tried this out, but it doesn't seem to work with optgroups yet?

[caseyjhol]

Give it another shot now - there was a bug in my code preventing it from working. Looks like there are still some issues using the arrow keys to navigate the menu with optgroups.

[andrews05]

Yup, that fixed it!
New issue is when I search I get duplicate results. Every match it finds is added again at the end of the list.

[caseyjhol]

Noticed that too...working on a fix.

[caseyjhol]

Search should work properly again - still working on getting the arrow keys to navigate optgroup menus properly.

[andrews05]

Thanks! Something I think could be really helpful in managing such large menus would be an ability to show just the checked items, so you can see at a glance which ones are checked.

[andrews05]

Next issue: the checkmark doesn't get toggled when you select an item while searching.

[caseyjhol]

Made some more changes, and arrow key navigation is better, but still not close to being ready. Let me know if any of your issues persist or if you notice anything else.

[andrews05]

Checkmark works correctly now, thanks.
Initialisation speed is definitely improved over previous version but can still be slow. Do you think there are any further optimisations that could be made here?

[caseyjhol]

Made another update - keyboard navigation should work correctly now. I still need to clean up some of the code/variables, and I do have plans for more optimizations. How many options are in the menu you're testing? What kind of performance are you seeing currently?

[andrews05]

I've got ~6000 options and initialisation time is measuring ~1500ms. This is only slightly faster than the previous version but the time for the intial render (harder to measure) is noticeably quicker.

Dividers are causing problems with the scroll position. Possibly because they have margin rather than height?

[caseyjhol]

Is that using the latest version I just released tonight? The divider margin issue should be fixed. What browser are you using? In the latest version of Chrome, 5000 options takes between 200-300ms for me. 50000 options is around 2.5 seconds. I've still got ideas to improve performance too.

[andrews05]

I'm using the Safari Technology Preview. Upon testing the standard release of Safari, it works fine! Must be an issue with the current tech preview, sorry about that.
I still have an issue with dividers in the latest release though, tested in Chrome also.
Searching with optgroups seems to be causing errors now, but not always - not sure of exact circumstance.

[caseyjhol]

Can you link me to an example of your use case? Having trouble recreating that in my test environment.

[ataft]

This is great! Initial testing looks good in my app, but I'll continue to bang away on it. I've put together a fiddle on the most recent version with 40K values in case others want to test: http://jsfiddle.net/54m1ybzn/12/

One very minor thing, you'll see that the "selected-text-format" only goes up to 100 Items Selected when you select all the items...

[Fellach]

Hi @caseyjhol,

I've found the & is encoded to entity & in the option's name. Is it intentional?

[caseyjhol]

@Fellach That wasn't intentional (a remnant from the previous method of using $.html() to get option content). It should work now.

[Fellach]

Confirm, works like a charm. Thank you

[caseyjhol]

Available in v1.13.0-alpha.

[andrews05]

Works pretty good so far, great job! Only thing I noticed was the search box isn't focused automatically when you open the menu.

[caseyjhol]

@andrews05 Hrmmm, it's focused in all of my testing. What browser/OS are you using?

[andrews05]

Tried it again and it works. Sorry, must have had something wrong the first time.

[ataft]

So far the alpha release is looking good. I've run into an issue when doing a refresh on an open list. Check out this fiddle: http://jsfiddle.net/54m1ybzn/14/. Open the list, do not scroll, and select an item. You'll notice that the checkmark does not appear until you start scrolling. This occurs in both IE 11 and Chrome 59.0.

I'm having a couple issues specific to IE 11 as well, but I'm not able to reproduce them in a fiddle, only in my app. Here are the problems:

1. I get an "Unable to get property 'liHeight' of undefined or null reference" error (Line 613: li.height = this.sizeInfo.liHeight;). It only occurs when I have two selectpickers on one page, one with values selected, the other with no values selected. Then close the one with nothing selected, and then re-open, then the error occurs. It was difficult to reproduce, but I can do it every time now that I know the exact sequence of events.
2. In IE and only in my app (not the fiddle), I have to click an item twice for it to be selected (display above and show the checkmark). Scrolling and then selecting an item causes it to jump to the top of the list, not selecting the item because I'd have to click it twice in order for this to happen.

If you're not able to identify the IE-only items above, then I'll work towards getting a working fiddle that can show them.

[caseyjhol]

@ataft I was able to reproduce your first issue, and I know what the problem is (setSize is only called when calling refresh while the menu is open, otherwise just liHeight is called, as setSize will be called anyway next time the menu is open). However, if calling refresh on show.bs.select, the open class hasn't been added yet (not until shown), which is causing that particular issue. I'll see if there's a better way to check, or if I can get away with just always calling setSize.

I wasn't able to recreate either of your IE11 issues.

[ataft]

@caseyjhol Thanks for looking into it. I've tested and believe that I can use shown.bs.select instead of show.bs.select, so no need for you to do anything. It worked OK in prior versions, so you may want to address the issue just in case, but I don't need you to. Thanks!

Also, I figured out what my IE issues were. On show.bs.select, I was setting the placeholder attribute on the search box to "Search...", as shown below:

In IE, the first click on an item was taking the focus from the search box, then the second click was selecting the item. Since the search is always focused (was it always this way?), I don't need the placeholder there anyways. I had it there originally because my app users are, let's just say not very web-saavy, and wouldn't necessarily know that that neat little box at the top was for searching.

Thus, I have no open issues with the virtualization. It's really a great feature, thanks so much!!!

[caseyjhol]

I'll check into it - I'd like to keep behavior as close as possible. You should be able to set the placeholder using liveSearchPlaceholder, instead of setting it manually on show.

$.fn.selectpicker.Constructor.DEFAULTS.liveSearchPlaceholder= 'Search...';

[ataft]

Aha, it's the placeholder itself that's causing the issues in IE. Check out the following fiddle using the liveSearchPlaceholder code (using IE of course):
http://jsfiddle.net/54m1ybzn/18/

Also, the "Select All" function on a medium-sized list (~2000 values) is instant in Chrome, but takes a few seconds in IE. Is there anything to be done here? It's crashed my IE browser a couple times too. I despise IE like everyone else, but unfortunately it's the preferred browser at many enterprise-level organizations (probably for legacy apps)...

[caseyjhol]

Hrmmm...that might be related to #851. Although, I noticed that Internet Explorer implements placeholder differently than other browsers - once the input is focused, the placeholder is hidden (rather than remaining visible until there is text in the input). The input is always focused to allow users to start typing right away, so the placeholder is never visible.

I'll look into the performance issues with Select All in IE.

[ataft]

I found one more IE issue, but it's impossible for me to reproduce in a jsfiddle for some reason. When I scroll all the way to the bottom, the items all disappear, as shown in the attached "before" and "after" images. In IE, I noticed that the .dropdown-menu.inner "margin-top" is being set to 0 when I reach the bottom, but that in Chrome that doesn't happen. It's been very difficult to troubleshoot and it's somewhat sporadic, but I'm hoping maybe it's an easy fix for you. Seems like it's in the code around here:
https://github.com/silviomoreto/bootstrap-select/blob/virtual-scrolling/dist/js/bootstrap-select.js#L724

[caseyjhol]

@ataft Please check out v1.13.0-beta and let me know if the problem persists. https://github.com/snapappointments/bootstrap-select/releases/tag/v1.13.0-beta

[ataft]

Looks great, thanks again for your quick response and awesome library!

[tomasfakta]

@caseyjhol wanted to say thanks here, ran into performance issues on IE11 , this fixed it

[caseyjhol]

Released in v1.13.0!

[HoudaMessaoudi]

thank you for your amazing work! i'm a bit confused about how to get the selected options(multiple mode) once the submit button is clicked.is there a specific way or a function to do that ?

[dliebner]

There seems to be an issue as of 1.13.9. If the dropdown goes into virtualization mode (from having many options), it adds a margin-bottom. If you update the dropdown to contain fewer options, thus smaller than the virtualization threshold, the margin-bottom stays, so you just get this huge gap at the bottom.

Easy fix at line 1265:
} else { menuInner.firstChild.style.marginTop = ''; menuInner.firstChild.style.marginBottom = ''; }

[caseyjhol]

@dliebner That issue was fixed in v1.13.10.

[cesarhfborges]

Not Worked for me,
any other possibilities?

[McKinleyWC]

Not Worked for me,
any other possibilities?

Late reply, but if you still need it:
in bootstrap-select.js, find the line with the if statement if (isVirtual === true) and add the else to that statement. The line number may be different if you've modified the file.