[Bug] Mega repro project for CollectionView issues here

[Tommigun1980]

Description

This is a repro project for the numerous CollectionView issues and regressions I have reported as of recently. I decided to make a separate ticket for this so I can refer to it and its instructions in one place. I spent a lot of time working on this and isolating the issues so I would really appreciate if these issues were fixed in a swift manner (as these are holding my app back). The CollectionView has received a lot of solid updates in the latest preview builds but unfortunately also a few regressions, so I hope this could be added to your test suite once the issues have been resolved. Thank you very much!

The repro project can be downloaded here: https://drive.google.com/file/d/1J6tuLyoOwjSIlKBMMT9zJaSpQf_iTcQX/view?usp=sharing

The repro project is very close to a real-life chat application (albeit an ugly one):

And contains instructions on how to reproduce the issues:

Breakdown

This repro project showcases 7 different bugs. I'll walk through them one at a time, with instructions, screenshots and videos of every issue.

Please note that I only tested this on iOS - I leave it to you to run the below tests on Android also.

Bug 1

Update: Fix should land in #13678 (have not verified this yet).

Xamarin.Forms 5.0.0-pre5 introduced a regression where a CollectionView's contents show up as empty if the IsVisible flag is toggled to True just as contents load. When the repro project is loaded it erroneously shows the following EmptyView in pre5 and later:

The above screen will not show once this issue is fixed (and won't show up if downgrading to 5.0.0-pre4).

At this point you can do two things:

1. Press the 'Send' button, at which point it crashes with NSInternalInconsistencyException:

2. Press 'Reload' which refreshes the CollectionView, which seems to be a workaround for now. Do this to proceed to bugs 2 to 7.

Video of bug 1 is here: https://drive.google.com/file/d/1VgrGcEZejeOVAMpDl9px28Hfaq8Z1jII/view?usp=sharing

This issue is reported at #13126 (comment), which also contains a more limited repro project (but I'd recommend using this repro project instead).

Bug 2

Update: Will also probably get fixed by #13678 (have not verified this yet).

This is a follow-up issue of Bug 1, and was also introduced in 5.0.0-pre5 (this is probably the same issue as Bug 1 but a different manifestation of it).

Once the CollectionView is refreshed by pressing 'Reload', it briefly shows the EmptyView. This lasts only for a fraction of a second, but is very visible as it looks like things are jumping around. This happens while there is data in the CollectionView - for some reason it still quickly displays the EmptyView - so this seems to be a timing issue of sorts.

Video of bug 2 is here (the issue is briefly visible at 10 seconds into the video, just after the spinner vanishes but before the CollectionView's contents are shown): https://drive.google.com/file/d/1ZZ24fXUEhkeneObknoS8o60uiymsGoh6/view?usp=sharing.

Bug 3

Update: Seems to have been fixed by #12919.

The CollectionView jumps around wildly and looks very glitchy when elements are deleted from it. To reproduce, pull left on any of your messages (the ones starting with 'You:') to reveal a SwipeView, and press the delete button. Please note that this is not a 100% repro, the glitch is different depending on your scroll position and the length of the message deleted.

Video of bug 3 is here (it seems to try to scroll to the last element no matter what element is deleted, but often fails to do so either scrolling too little or too much, and the elements themselves also seem to have a glitchy animation): https://drive.google.com/file/d/1vZk60gz7CfDAuHUO69YD9hzJcOCCxGxV/view?usp=sharing

Bug 4

Update: Partially fixed by #12919, but an issue still remains: #13231 (comment).

The CollectionView jumps around wildly and looks very glitchy when elements are added to it. To reproduce, write some text into the entry field and press 'Send'. Please note that this is not a 100% repro, the glitch is different depending on your scroll position and the length of the message added.

Video of bug 4 is here: https://drive.google.com/file/d/1Tp3pMX4havcZAsGTdIK9K-LQi9EsW1Go/view?usp=sharing

Bug 5

Update: Seems to have been fixed by #12919.

Scrolling is often extremely jerky/glitchy after having added a new element. To reproduce, press 'Send' once (and scroll to the end of the list if it doesn't by itself), and then slowly scroll upwards - make small dragging motions where you release the finger - it looks like something is pulling the view down while you scroll up. This is not a 100% repro and may require a few tries.

Video of bug 5 is here (the 'twitch' at 26 seconds into the video is especially visible): https://drive.google.com/file/d/1qfS7wRDVO5AfnoXyhXUH6M1HbW_8t4z4/view?usp=sharing

Bug 6

Update: Not fixed yet, separately tracked in #13547 and #13675.

Even though 'KeepLastItemInView' is used, the CollectionView shows the first elements when data initially becomes available for it. Reproduced by seeing the following when the CollectionView loads (instead of the last message, which would be correct):

Bug 7

Update: Have not seen this anymore at all, probably fixed by #12919.

This may have been fixed in 5.0.0-pre6 as I could no longer reproduce it, but it was hard to reproduce after the latest commits in #12919 to begin with, so adding it here to keep an eye out for it. As explained in #12919 (comment), some elements in a CollectionView were cut off in XF build 32817.

The second to last message here shows this error (the speech bubble contains two lines, but only the first is drawn - this was not reproducible with a ListView at the time):

... but it may have been fixed in 5.0.0-pre6. This repro app can be used for validating that this doesn't regress.

Basic Information

• Version with issue: 5.0.0-pre6
• Last known good version: For bugs 1 and 2: 5.0.0-pre4. No working version for the rest of the bugs.
• Platform Target Frameworks:
  • iOS: 14.2
• NuGet Packages: XF 5.0.0-pre6, Xamarin.FFImageLoading.Forms 2.4.11.982
• Affected Devices: iOS (please also test all of these on Android, my findings were for iOS this time)

Environment

Show/Hide Visual Studio info

=== Visual Studio Community 2019 for Mac (Preview) ===

Version 8.9 Preview (8.9 build 964)
Installation UUID: f4c2e8d8-481a-4311-847b-759fa99c5218
	GTK+ 2.24.23 (Raleigh theme)
	Xamarin.Mac 6.18.0.23 (d16-6 / 088c73638)

	Package version: 612000113

=== Mono Framework MDK ===

Runtime:
	Mono 6.12.0.113 (2020-02/4fdfb5b1fd5) (64-bit)
	Package version: 612000113

=== Roslyn (Language Service) ===

3.9.0-2.20573.18+bc20ce579f9b17c9153f3f54a0690d23c482d899

=== NuGet ===

Version: 5.8.0.6860

=== .NET Core SDK ===

SDK: /usr/local/share/dotnet/sdk/5.0.101/Sdks
SDK Versions:
	5.0.101
	5.0.100
	3.1.404
	3.1.403
MSBuild SDKs: /Applications/Visual Studio.app/Contents/Resources/lib/monodevelop/bin/MSBuild/Current/bin/Sdks

=== .NET Core Runtime ===

Runtime: /usr/local/share/dotnet/dotnet
Runtime Versions:
	5.0.1
	5.0.0
	3.1.10
	3.1.9

=== .NET Core 3.1 SDK ===

SDK: 3.1.404

=== Xamarin.Profiler ===

Version: 1.6.15.68
Location: /Applications/Xamarin Profiler.app/Contents/MacOS/Xamarin Profiler

=== Updater ===

Version: 11

=== Xamarin.Android ===

Version: 11.1.99.218 (Visual Studio Community)
Commit: xamarin-android/master/9bc0003
Android SDK: /Users/tommikiviniemi/Library/Developer/Xamarin/android-sdk-macosx
	Supported Android versions:
		None installed

SDK Tools Version: 26.1.1
SDK Platform Tools Version: 30.0.4
SDK Build Tools Version: 30.0.2

Build Information: 
Mono: ac59637
Java.Interop: xamarin/java.interop/master@99897b2
ProGuard: Guardsquare/proguard/v7.0.0@84e8560
SQLite: xamarin/sqlite/3.32.1@1a3276b
Xamarin.Android Tools: xamarin/xamarin-android-tools/main@ab650f5

=== Microsoft OpenJDK for Mobile ===

Java SDK: /Users/tommikiviniemi/Library/Developer/Xamarin/jdk/microsoft_dist_openjdk_1.8.0.25
1.8.0-25
Android Designer EPL code available here:
https://github.com/xamarin/AndroidDesigner.EPL

=== Android SDK Manager ===

Version: 16.9.0.19
Hash: 9771026
Branch: remotes/origin/HEAD
Build date: 2020-11-21 02:03:08 UTC

=== Android Device Manager ===

Version: 16.9.0.13
Hash: bbbb387
Branch: remotes/origin/main
Build date: 2020-11-21 02:03:28 UTC

=== Apple Developer Tools ===

Xcode 12.2 (17535)
Build 12B45b

=== Xamarin.Mac ===

Xamarin.Mac not installed. Can't find /Library/Frameworks/Xamarin.Mac.framework/Versions/Current/Version.

=== Xamarin.iOS ===

Version: 14.9.0.27 (Visual Studio Community)
Hash: f4c9327fa
Branch: main
Build date: 2020-11-19 10:57:31-0500

=== Xamarin Designer ===

Version: 16.9.0.105
Hash: f693266a2
Branch: remotes/origin/main
Build date: 2020-12-02 05:56:40 UTC

=== Build Information ===

Release ID: 809000964
Git revision: 0ab5ced8f8f39c1837b5879f51401c6fd1ae41ef
Build date: 2020-12-15 08:05:14-05
Build branch: release-8.9
Xamarin extensions: 0ab5ced8f8f39c1837b5879f51401c6fd1ae41ef

=== Operating System ===

Mac OS X 10.16.0
Darwin 20.2.0 Darwin Kernel Version 20.2.0
    Wed Dec  2 20:39:59 PST 2020
    root:xnu-7195.60.75~1/RELEASE_X86_64 x86_64

Reproduction Link

The repro project can be downloaded here: https://drive.google.com/file/d/1J6tuLyoOwjSIlKBMMT9zJaSpQf_iTcQX/view?usp=sharing

[Tommigun1980]

Ping @hartez.

[developer9969]

@Tommigun1980 what you did is excellent as is the only way we can get collection view fixed. I have noticed some weird behavior in some of the apps when using stable 4.8.0.1821 at times the collection view does not load the bound ObservableCollection or ObservableRangeCollection , it seems that needs loading twice!!
I am trying to build a repo but cannot replicate in simple repo, downgrading to 4.8.0.1560 and collectionview works.

Also I have noticed that when displaying a form with a collection view it kind of weirdly bounce /or jerky no smooth. I will try to build a repo that shows the issue . I know it does not help without a repo , I thought I would mention it.

I also had lots of issue with footer and visibility in the end I gave up .

[Tommigun1980]

@Tommigun1980 what you did is excellent as is the only way we can get collection view fixed. I have noticed some weird behavior in some of the apps when using stable 4.8.0.1821 at times the collection view does not load the bound ObservableCollection or ObservableRangeCollection , it seems that needs loading twice!!
I am trying to build a repo but cannot replicate in simple repo, downgrading to 4.8.0.1560 and collectionview works.

Also I have noticed that when displaying a form with a collection view it kind of weirdly bounce /or jerky no smooth. I will try to build a repo that shows the issue . I know it does not help without a repo , I thought I would mention it.

I also had lots of issue with footer and visibility in the end I gave up .

Thanks. It sounds to me like the issues you are describing are bugs 1 (CollectionView regression where it needs to load twice) and 5 (jerkiness when scrolling) in this repro project.

Header and footer issues are reported at #13212, maybe those were the issues you were seeing?

All of these have repros now so hopefully Xamarin comes through and fixes them fast.

[Tommigun1980]

Hi. Is there any hope of getting these fixed?

Thanks a lot!

[Tommigun1980]

Hi @hartez and thanks for assigning this.

These are all fundamental concepts in the CollectionView and I think should be prioritized highly, and the repro project can be used for validating these easily (I can see they were just added to the backlog).

I'll summarise the bugs in this report.

1. The CollectionView regressed to showing no items in 5.0.0-pre5, and when manipulated in that state it crashes the app.
2. Adding items to, and removing items from, the CollectionView is very glitchy (can not be used in production).
3. Scrolling the CollectionView is glitchy (can not be used in production).
4. Problems with scrolling to bottom -- I am doing my darnest to try to make a chat feature, where adding new messages will scroll to reveal them if the scroll view is already at the bottom. I don't think it is possible to achieve without fixing the scrolling behaviours, or implementing [Enhancement] Custom Scrolling for CollectionView #13198.

If these issues can not be fixed I will continue to use a hacked ListView, but the hacks I am using are timing dependent and stop working when there's lots of contents in the view.

Please advice on how to proceed. I have committed to this technology and am at a point where I need clear answers on if I can implement a chat with Xamarin.Forms at all.

Thank you so much!

[FrankSzendzielarz]

I seem to have jittery scrolling in both CollectionView and ListView, with improved compatibility in CollectionView for SwipeViews in the 'cells'. Removing images and simplifying element layout does reduce the amount of stutter. What can I do to go about debugging this stuff on my own? I'd like to know how to performance analyse the pauses that cause the scroll stutter.

[Tommigun1980]

@jsuarezruiz @hartez I'd really appreciate some correspondence on these four CollectionView bugs I have opened:
#13592, #13547, #13548, #13231.

I spent a ton of time isolating the issues and making repro projects for them, and I need some kind of resolution as I have a hard time finish up my work here before they are resolved. So I'd really appreciate some comments on time estimates, or any kind of correspondence regarding them.
Thank you.

[Eltee-Taiwo]

This is getting really frustrating...

[hartez]

Bug 1 on this list appears to be the same thing as #13551. We've got a PR for that, it's working its way through review and testing right now, and is currently slated for 5.0.0 SR3.

[hartez]

Bug 4 on this list appears to no longer be a problem in 5.0.0.1931. I believe it was addressed by #12919.

[hartez]

Bug 3 also appears to no longer be a problem in 5.0.0.1931. I think that one was also fixed by #12919.

[Eltee-Taiwo]

Thanks for the feedback. I'm mostly here for #13551 so I'll wait for that 👍

[hartez]

I've created a new issue #13675 for bug 6 so we have an easier place to track it.

[hartez]

I'm no longer seeing bug 5 after updating to 5.0.0.1931. I believe this was also a symptom of the problems addressed in #12919.

[hartez]

I also can't seem to reproduce bug 2 after updating.

[Tommigun1980]

Hi @hartez and thank you a million times for looking into these!

I updated this project to 5.0.0.1931 and unfortunately I can still see bug 4 (the "adding elements visual glitch") happening sometimes. The updates to the CollectionView have been extremely solid and most of the issues have indeed been fixed (I updated the issue list at the top with fix status per bug)!

I have only been able to reproduce the aformentioned issue under the following conditions:

1. 'KeepLastItemInView' is used, AND
2. The scroll bar is not scrolled to the bottom

It does not seem to be reproducible anymore when using other modes than 'KeepLastItemInView', or if the message is added while the view is scrolled to the bottom.

Here is a video of bug 4 with 5.0.0.1931 (it happens roughly 12 seconds into the video):
https://drive.google.com/file/d/1OPYMD21MgI_zqMx5q7zEwtDv5J4uL4ja/view?usp=sharing

The elements themselves don't glitch out anymore, but the view scrolls far past the newly added element. If the view is scrolled while in this state, it will 'snap' to correct itself.
Thank you.

[bakerhillpins]

Suggestion for @Tommigun1980. In the future if you might please consider uploading examples and imagery to GitHub directly rather than using Google Drive. While I can understand how Drive can be useful many Enterprise systems now block Drive for security reasons. Images/videos/projects (as zip files) can all be directly uploaded via the bar at the bottom of the comment editor.

I'm investing in Xamarin and would like understand issues that you've linked here but I can't because all of your information is blocked from access by my employer's systems.

Thanks!