Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CIS-2084, CIS-2085] New Message List Diffing Implementation #2226

Merged
merged 49 commits into from
Aug 24, 2022

Conversation

nuno-vieira
Copy link
Member

@nuno-vieira nuno-vieira commented Aug 9, 2022

🔗 Issue Links

CIS-2084
CIS-2085
#2094

Bonus:
#2184
CIS-1965

🎯 Goal

The goal of this PR is to fix the crashes in the message list related to data inconsistencies in performBatchUpdates and also to fix the message list jumps when receiving multiple messages and the user is scrolled up.

📝 Summary

  • Adds DifferenceKit dependency to StreamChatUI.
  • Removes the_messageListDiffingEnabled flag, since now we only have 1 implementation of the message list and it is not experimental anymore
  • Instead of relying on ListChange updates from StreamChat, now the StreamChatUI applies changes from the diffing calculation of the old messages snapshot, and the newest message snapshot
  • Skips bottom insertions at the bottom when the user is scrolled up. With the diffing logic in place, now it is easier to skip bottom insertion updates when the user is scrolled up. This avoids the message list jumps because the insertions at the bottom don't actually happen until the user scrolls to the bottom again. This not only avoids the message list jumps but also makes the message list more performant and efficient.
  • Improves the performance of the message list when multiple messages are being inserted at the same time and there are already a lot of messages loaded. (Most of this improvement was by skipping channel list updates while the channel list is not visible. After profiling the application, the main thread was mostly blocked by the channel list updates, and the root cause is being fixed here: [POC] Move Model mappings to Background #2168. Either way, it doesn't make sense for the channel list to impact the message list while the user is on the message list. The [POC] Move Model mappings to Background #2168 PR will also improve even further the message list performance especially when it is being overloaded. Right now the DTO mappings on the main thread are the only optimisation left in the Message List. Even though we have right now a good 56/60FPS average when the message list is being overloaded.
  • Removes SentryKit from StreamChat. For some reason. SentryKit was added as a dependency of StreamChat, instead of the DemoApp

Performance Improvements:

All tests were performed in an iPhone XR iOS 15.5 and for an interval of 10s.

Loading previous messages continuously
image

Scrolling while inserting multiple new messages at the bottom through stress test script
image

🎨 Showcase

Loading Previous Messages No jumps
RPReplay_Final1660064107.MP4
RPReplay_Final1660064236.MP4

🧪 Manual Testing Notes

  • Mob Regression Testing

☑️ Contributor Checklist

  • I have signed the Stream CLA (required)
  • Changelog is updated with client-facing changes
  • New code is covered by unit tests
  • This change follows zero ⚠️ policy (required)
  • Comparison screenshots added for visual changes
  • Affected documentation updated (docusaurus, tutorial, CMS)

🎁 Meme

@nuno-vieira nuno-vieira added 🐞 Bug An issue or PR related to a bug 🎨 SDK: StreamChatUI Tasks related to the StreamChatUI SDK labels Aug 9, 2022
@nuno-vieira nuno-vieira requested a review from a team as a code owner August 9, 2022 14:49
@nuno-vieira nuno-vieira changed the title [CIS-2084, CIS-2085] Refactor Message List with DifferenceKit [CIS-2084, CIS-2085] New Message List Diffing Implementation Aug 9, 2022
@emerge-tools
Copy link

emerge-tools bot commented Aug 9, 2022

📏 Size Analysis

Total install size 9.6 MB | This change: ⬆️ +45.7 kB (+0.476%)

Image of diff

🗂 See size breakdown
Item Install size
➖ StreamChat.framework/Strings.CFStrings ⬇️ 46.5 kB
➕ ChatSample/Strings.CFStrings ⬆️ 46.4 kB
ChatSample/External Methods ⬆️ 39.7 kB
StreamChat.framework/DYLD.String Table ⬇️ 29.7 kB
StreamChat.framework/Strings.Unmapped ⬇️ 22.3 kB
➖ StreamChat.framework/SentryCrashReportConverter ⬇️ 21.9 kB
StreamChat.framework/Code Signature ⬇️ 21.2 kB
ChatSample/Strings.Unmapped ⬆️ 21.1 kB
➖ StreamChat.framework/SentryOptions ⬇️ 20.7 kB
➕ StreamChatUI.framework/StreamChatUI.ChatMessage.isContentEqual(to) ⬆️ 20.4 kB
ChatSample/Code Signature ⬆️ 20.2 kB
➖ StreamChat.framework/SentryTracer ⬇️ 17.4 kB
➖ StreamChat.framework/SentryClient ⬇️ 17.3 kB
StreamChat.framework/DYLD.Exports ⬇️ 16.3 kB
StreamChat.framework/External Methods ⬇️ 16.2 kB
➖ StreamChat.framework/SentryScope ⬇️ 16.2 kB
➖ StreamChat.framework/SentryFileManager ⬇️ 15.9 kB
ChatSample/DYLD.Exports ⬆️ 14.7 kB
➖ StreamChat.framework/SentryHub ⬇️ 12.8 kB
➖ StreamChat.framework/SentryCrash ⬇️ 12.6 kB

🔎 See the full size analysis (c5b16c5) merging into develop (86daace)

Provide a baseSha and pullRequestNumber with your upload to ensure diffs are correct. Read more in the docs

@GetStream GetStream deleted a comment from Stream-SDK-Bot Aug 9, 2022
@Stream-SDK-Bot
Copy link
Collaborator

1 Warning
⚠️ Big PR

Generated by 🚫 Danger

Copy link
Contributor

@martinmitrevski martinmitrevski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work @nuno-vieira 👏 . It seems much more stable now, and the performance is better.
I've noticed few issues that we need to fix before merging:

  • bug with pagination broken (see comments)
  • reactions don't work when in "skip inserts" mode (also in comments)
    Additionally, we would need some tests before we can merge it.

Copy link
Contributor

@polqf polqf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks really good, I am just concerned with certain pieces of custom logic. We don't want this evolution to end up being like the previous one where it was full of patches and custom logic.

Also, we should be embedding DifferenceKit as part of StreamChatUI

@nuno-vieira nuno-vieira force-pushed the fix/CIS-2085-Message-list-with-difference-kit branch from 4bdf4f1 to 2e3e59a Compare August 10, 2022 15:09
…nserted while skipping messages

This caused some data concurrency issues because a lot of data updates are triggered. In order to avoid this reloadData() is much more safe.
It was using the data controller directly, instead of the data source messages
@nuno-vieira nuno-vieira force-pushed the fix/CIS-2085-Message-list-with-difference-kit branch from ceb4fae to eba3eae Compare August 17, 2022 13:57
@GetStream GetStream deleted a comment from sonarcloud bot Aug 23, 2022
@sonarcloud
Copy link

sonarcloud bot commented Aug 24, 2022

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 4 Code Smells

100.0% 100.0% Coverage
0.0% 0.0% Duplication

@nuno-vieira nuno-vieira merged commit af219c3 into develop Aug 24, 2022
@nuno-vieira nuno-vieira deleted the fix/CIS-2085-Message-list-with-difference-kit branch August 24, 2022 11:16
@nuno-vieira nuno-vieira added the ⚡ Performance An Issue or PR related to performance improvements label Aug 24, 2022
@polqf polqf mentioned this pull request Sep 1, 2022
polqf pushed a commit that referenced this pull request Sep 1, 2022
* Add cell heights cache so that new contentSize calculation is more precise

* Remove `_messageListDiffingEnabled` flag and diffable data source implementation

* WIP DifferenceKit

* Move DiffKit logic to Message List only

* Improve performance by using ChatMessage directly instead of mapping always to DiffChatMessage

* Improve performance of the message list by skipping channel list updates

* Disable animations when loading new messages

Since the this was actually not causing the messages to jump

* Fix not loading previous messages when there are skipped messages

* Add `ListChange` helpers

* WIP Skipping Messages

* Move the skipping logic to the message list view

* Improve the performance of Message Content Equality

* Fix Giphy not updating the UI when moved from original position

* Add code docs to cell height cache + channel list skip rendering

* Add comments explaining CATransaction in DiffKit reload

* Update CHANGELOG.md

* Update documentation of our dependencies

* When skipping messages, only skip insertions, not all updates

* Improve the readability of skipping messages

* Always store the original messages from the data source to avoid data inconsistencies

* Provide comments for the new introduced methods

* Add Sentry dependency to DemoApp and remove it from StreamChat

* Properly add DifferenceKit dependency

* Ignore third party dependencies when formatting code with SwiftFormat

* Add required declarations changes to DiffKit dependency

Also adds a function to make the removePublicDeclarations.sh more readable

* Undo changes on SwiftyMarkdown done by our SwiftFormat

* Add missing required declaration changes to DiffKit Source files

* Move DiffKit logic to the correct files

* Add missing message id comparison in ChatMessage Content Equality

* Fix crash when message from same user but with difference device is inserted while skipping messages

This caused some data concurrency issues because a lot of data updates are triggered. In order to avoid this reloadData() is much more safe.

* Remove willUpdateMessages events since it is actually not needed

* Fix message cell layout options not being correctly calculated

It was using the data controller directly, instead of the data source messages

* Make sure to use the messages from the data source in ChatThreadVC

* When a message is hard deleted, make sure to reload the previous cell

* Fix thread root message not being updated

* Fix and Add missing coverage to ChannelListVC

* Fix E2E Tests + First message from participant not appearing

* Fix updating replyCount of the parent message in E2E Tests

* Fix Xcode 12 compilation

* Add E2E Test to coverage giphy message move scroll to bottom case

* Fix false positives when asserting messages in E2E Tests

* Add E2E Test case for when hard deleting last message from group

* Add unit test coverage to when should reload skipped messages

* Improve ListChange test coverage
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🐞 Bug An issue or PR related to a bug ⚡ Performance An Issue or PR related to performance improvements 🎨 SDK: StreamChatUI Tasks related to the StreamChatUI SDK
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants