[Chat-ng]: Appflowy Editor editor integration #2332

gtalha07 · 2024-10-29T17:26:05Z

The first set iterative change is to integrate Appflowy Editor integration (with built-in set of mention components block) as a chat-ng input component which will support markdown formatting. This'll also fix the bugs we've seen with input format which we have to accustom with various handlers for ensuring that proper markdown is passed and that won't be needed since the editor would emit correct format and ensure the renderer catches it correctly and display.

As this section of the codebase is pretty complex and time-consuming to follow, the input component isn't entirely migrated. As it accounts numerous issues being appearing when trying to do so. So, having it started from the ground is the best course to take but that also means it's limited in its functionality compared to current state of CustomInput.

This only contains functional wise:

Simple send message state .i.e including trailing rendering of showing either attachment/send button depending on editorState current state.
Emoji handlers with editorState.
Attachment handlers.
Chat Compose draft state. (Not yet with reply/edit builder)

But also improves UI of the input:

Dynamic height adjustment of input field.
Proper markdown support for input.
Mentions built-in extension for Appflowy. See html_editor directory for new mentions block support.

Desktop Demo with Mentions UI:

Screen.Recording.2024-11-10.at.9.43.30.PM.mov

Mobile UI:

ScreenRecording_11-10-2024.21-39-27_1.mp4

codecov · 2024-10-29T17:28:03Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 26.91%. Comparing base (4d77a69) to head (cd28e67).
Report is 41 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2332      +/-   ##
==========================================
- Coverage   27.05%   26.91%   -0.15%     
==========================================
  Files         596      615      +19     
  Lines       41433    42146     +713     
==========================================
+ Hits        11211    11342     +131     
- Misses      30222    30804     +582

Flag	Coverage Δ
integration-test	`37.05% <ø> (+0.02%)`	⬆️
unittest	`18.70% <ø> (-0.03%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

gnunicorn · 2024-11-05T14:42:13Z

@gtalha07 the video doesn't load here. secondly, I tried to start reviewing but my focus is a bit scattered this week and I couldn't get through... so you might want to ping someone else to do it if you want this in earlier...

kumarpalsinh25 · 2024-11-06T06:39:44Z

@gtalha07 the video doesn't load here. secondly, I tried to start reviewing but my focus is a bit scattered this week and I couldn't get through... so you might want to ping someone else to do it if you want this in earlier...

Let me do a review although I don't understand much details about Applyflowy Editor and Mentioned related work.

kumarpalsinh25

I will basic review of this PR. However I don't much understand mentioned and editor related work.

Also can I have video reference for Room/User mention related flow?

app/lib/features/chat_ng/widgets/chat_input.dart

app/lib/features/chat/widgets/custom_input.dart

app/lib/common/widgets/html_editor/html_editor.dart

app/lib/common/widgets/html_editor/components/user_mention_block.dart

app/lib/common/widgets/html_editor/components/room_mention_block.dart

kumarpalsinh25

It is quite difficulty to judge mentioned related code review without it's visual reference. More over seeing issues in basic code standards during review.

And I see that after first round of PR review, there are additional thing gets added to this PR with below commit which makes PR review process quite difficulty and need to review whole things again carefully to track entire changes.
3c5a6af

I am not feeling confident to approve this PR, I would like to have once round of review from @gnunicorn whenever he gets some time.

app/lib/common/widgets/html_editor/components/mention_block.dart

app/lib/common/widgets/html_editor/components/mention_content.dart

app/lib/common/widgets/html_editor/components/mention_handler.dart

kumarpalsinh25 · 2024-11-08T06:13:47Z

app/lib/common/widgets/html_editor/components/mention_handler.dart

+
+    final node = widget.editorState.getNodeAtPath(selection.end.path);
+    if (node == null) return;
+
+    final text = node.delta?.toPlainText() ?? '';
+    if (text.length < startOffset) return;
+
+    setState(() {
+      _search = text.substring(startOffset);
+    });
+  }
+
+  void _selectItem(String id, String displayName) {
+    final selection = widget.editorState.selection;
+    if (selection == null) return;
+
+    final transaction = widget.editorState.transaction;
+    final node = widget.editorState.getNodeAtPath(selection.end.path);
+    if (node == null) return;
+
+    // Delete the search text and trigger character
+    transaction.deleteText(
+      node,
+      startOffset - 1,
+      selection.end.offset - startOffset + 1,
+    );
+
+    // Insert the mention
+    transaction.insertText(
+      node,
+      startOffset - 1,
+      displayName.isNotEmpty ? displayName : id,
+      attributes: {
+        MentionBlockKeys.mention: {
+          MentionBlockKeys.type: widget.mentionType.name,
+          if (widget.mentionType == MentionType.user)
+            MentionBlockKeys.userId: id
+          else
+            MentionBlockKeys.roomId: id,
+        },
+      },
+    );
+
+    widget.editorState.apply(transaction);
+    widget.onDismiss();
+  }
+
+  void _scrollToSelected() {
+    final itemPosition = _selectedIndex * kItemHeight;
+    if (itemPosition < _scrollController.offset) {
+      _scrollController.animateTo(
+        itemPosition,
+        duration: const Duration(milliseconds: 200),
+        curve: Curves.easeOut,
+      );
+    } else if (itemPosition + kItemHeight >
+        _scrollController.offset +
+            _scrollController.position.viewportDimension) {
+      _scrollController.animateTo(
+        itemPosition +
+            kItemHeight -
+            _scrollController.position.viewportDimension,
+        duration: const Duration(milliseconds: 200),
+        curve: Curves.easeOut,
+      );
+    }
+  }
+
+  void _deleteCharacterAtSelection() {
+    final selection = widget.editorState.selection;
+    if (selection == null || !selection.isCollapsed) {
+      return;
+    }
+
+    final node = widget.editorState.getNodeAtPath(selection.end.path);
+    final delta = node?.delta;
+    if (node == null || delta == null) {
+      return;
+    }
+
+    _search = delta.toPlainText().substring(
+          startOffset,
+          startOffset - 1 + _search.length,
+        );
+  }
+
+  bool _canDeleteLastCharacter() {
+    final selection = widget.editorState.selection;
+    if (selection == null || !selection.isCollapsed) {
+      return false;
+    }
+
+    final delta = widget.editorState.getNodeAtPath(selection.start.path)?.delta;
+    if (delta == null) {
+      return false;
+    }
+
+    return delta.isNotEmpty;
+  }
+}


Ignoring code review of this part as I am not quite understanding logic here.

app/lib/common/widgets/html_editor/components/mention_menu.dart

gtalha07 · 2024-11-11T10:20:01Z

app/lib/common/widgets/html_editor/components/mention_block.dart

+import 'package:flutter/material.dart';
+import 'package:flutter_riverpod/flutter_riverpod.dart';
+
+class MentionBlock extends ConsumerWidget {


The UI component of the inline mentions in editor (of how it should be displayed when triggered)

gtalha07 · 2024-11-11T10:20:36Z

app/lib/common/widgets/html_editor/components/mention_item.dart

+import 'package:acter_avatar/acter_avatar.dart';
+import 'package:flutter/material.dart';
+
+class MentionItem extends StatelessWidget {


Mention list UI item component which gets displayed as part of mention menu list.

gtalha07 · 2024-11-11T10:23:20Z

app/lib/common/widgets/html_editor/components/mention_list.dart

+    }
+  }
+
+  void _selectItem(String id, String displayName) {


handler when user selects mention item from list and processes how the editor should execute mention block inline

gtalha07 · 2024-11-11T10:24:37Z

app/lib/common/widgets/html_editor/components/mention_list.dart

+    widget.onDismiss();
+  }
+
+  void _scrollToSelected() {


useful for navigating mention list through keyboard navigation arrows (up, down) to scroll accordingly using custom scroll controller

gtalha07 · 2024-11-11T10:26:21Z

app/lib/common/widgets/html_editor/components/mention_list.dart

+    }
+  }
+
+  bool _canDeleteLastCharacter() {


checks whether editor state selection is at mention block.

gtalha07 · 2024-11-11T10:27:33Z

app/lib/common/widgets/html_editor/html_editor.dart

+    );
+  }
+
+  InlineSpan customizeAttributeDecorator(


For customizing the styling of custom component block, here we can extend it beyond mentions (for future reference).

gnunicorn

I have not done a full review and I don't pretend that I have understood this whole, but I'd like to see two major suggestions applied before going forward, as it will make it easier to understand what is even meant to be going on:

using typed content for the Attributes and everywhere really, rather than a bunch of keyed-strings of keys for a Map... that isn't necessary and makes it harder for the reader and the computer to confirm this is correct
remove the isDesktop-checks. They are almost always wrong and they are for sure here ... if we have a hardware keyboard or a mouse isn't platform specific and pretending it is makes the code more complex than just dealing with that being the case...

gnunicorn · 2024-11-05T14:34:43Z

.changes/2332-chat-ng-editor.md

@@ -0,0 +1 @@
+- Chat-NG now supports Appflowy editor with proper markdown support. Some/Other features might be disabled or limited for now.


Suggested change

- Chat-NG now supports Appflowy editor with proper markdown support. Some/Other features might be disabled or limited for now.

- [Labs] Chat-NG now offers a fresh new WYSIWYG editor. More features upcoming.

app/lib/features/chat_ng/widgets/chat_messages.dart

gnunicorn · 2024-11-11T12:59:36Z

app/lib/common/widgets/html_editor/components/mention_block.dart

+  final Map<String, dynamic> mention;
+  final String userRoomId;
+  final Node node;
+  final int index;


don't we usually keep the list above the constructor?

Hmm, yes. I can change it as I didn't really seem to thought much about it before.

gnunicorn · 2024-11-11T12:59:44Z

app/lib/common/widgets/html_editor/components/mention_block.dart

+  final int index;
+  @override
+  Widget build(BuildContext context, WidgetRef ref) {


Suggested change

final int index;

@override

Widget build(BuildContext context, WidgetRef ref) {

final int index;

@override

Widget build(BuildContext context, WidgetRef ref) {

gnunicorn · 2024-11-11T13:02:47Z

app/lib/common/widgets/html_editor/services/mention_shortcuts.dart

+) {
+  return [
+    CharacterShortcutEvent(
+      character: '@',


let's use a global static for these!?!

gnunicorn · 2024-11-11T13:09:17Z

app/lib/common/widgets/html_editor/components/mention_item.dart

+    final isDesktop = desktopPlatforms.contains(Theme.of(context).platform);
+
+    return Container(
+      height: 60,
+      // selection color is only for desktop with keyboard navigation
+      color: (isSelected && isDesktop)
+          ? Theme.of(context).colorScheme.primary
+          : null,


isDesktop is an incorrect check here: pads can also have keyboards and laptops might also have touch devices. what I think you want to detect for here is a hardware keyboard, no?

but then also, how would isSelected even come about if not through a hardware-keyboard? I think this check is pointless and should be removed and the rest left to the outer part.

Suggested change

final isDesktop = desktopPlatforms.contains(Theme.of(context).platform);

return Container(

height: 60,

// selection color is only for desktop with keyboard navigation

color: (isSelected && isDesktop)

? Theme.of(context).colorScheme.primary

: null,

return Container(

height: 60,

color: isSelected ? Theme.of(context).colorScheme.primary : null,

I see, good point. I'll change the conditional 👍🏻 !

gnunicorn · 2024-11-11T13:19:44Z

app/lib/common/widgets/html_editor/components/mention_list.dart

+      attributes: {
+        MentionBlockKeys.mention: {
+          MentionBlockKeys.type: widget.mentionType.name,
+          if (widget.mentionType == MentionType.user)
+            MentionBlockKeys.userId: id
+          else
+            MentionBlockKeys.roomId: id,
+          MentionBlockKeys.displayName: displayName,
+        },
+      },


so ... attributes Attributes = Map<String, dynamic>, right? so, why not have the "mention"-string-key point to a typed object. all those MentionBlockKeys-with strings make it really hard to follow and easy to mess up. I'd rather have a typed:

Suggested change

attributes: {

MentionBlockKeys.mention: {

MentionBlockKeys.type: widget.mentionType.name,

if (widget.mentionType == MentionType.user)

MentionBlockKeys.userId: id

else

MentionBlockKeys.roomId: id,

MentionBlockKeys.displayName: displayName,

},

},

attributes: {

MENTIONS_KEY: widget.mentionType == MentionType.user ?

MentionAttributes.user(userId: id, displayName: displayName)

MentionAttributes.room(roomId: id, displayName: displayName)

},

For a type MentionAttributes, along the lines of (pseudocode):

class MentionAttributes { final String? userId; final String? roomId; final String? displayName; final MentionType type; const MentionAttributes({this.userId, this.roomId, required this.type, this.displayName}); factory MentionAtrributes.user(required this.userId, {this.displayName}) { type = MentionType.user; } }

gnunicorn · 2024-11-11T13:21:43Z

app/lib/common/widgets/html_editor/components/mention_list.dart

+    if (isDesktop) {
+      // we only want to listen keyboard events on desktop
+      return Focus(
+        focusNode: _focusNode,
+        onKeyEvent: (node, event) => _handleKeyEvent(node, event, suggestions),
+        child: menuWidget,
+      );
+    }


We can disregard this flaky desktop check (iPads and Mobiles can have hardware keyboards attached to it) by directly interacting with HardwareKeyboardDetector. Which we should be doing here rather than doing some flaky check and focus grabbing ...

gnunicorn · 2024-11-11T13:22:39Z

app/lib/common/widgets/html_editor/components/mention_list.dart

+
+  @override
+  Widget build(BuildContext context) {
+    final isDesktop = desktopPlatforms.contains(Theme.of(context).platform);


reminder: this is almost always bad. checking for the specific feature you want is probably the better idea. In this case: check for/directly use the actual hardware keyboard..

gnunicorn · 2024-11-11T13:24:36Z

app/lib/common/widgets/html_editor/components/mention_block.dart

+    required WidgetRef ref,
+    required AvatarOptions avatarOptions,
+  }) {
+    final desktopPlatforms = [


mouse can also be plugged/connected to phones. don't do a check like that.

set up code for appflowy editor mentions plugin

69f7e3d

gtalha07 added 7 commits October 30, 2024 13:38

implement mentions fetching logic based on query

191a8f8

start from the basic UI setup

7868d77

add send message functionality back

01f7315

fix editor focus when sending message

a16dd7f

implement emoji picker logic

6fe1e5a

fix ordering state

7bc47fa

restore attachment button functionality

f0ab803

gtalha07 marked this pull request as ready for review November 4, 2024 19:57

gtalha07 changed the title ~~Appflowy Editor mentions support~~ [Chat-ng]: Appflowy Editor editor integration Nov 4, 2024

gtalha07 requested review from gnunicorn, kumarpalsinh25 and emilvincentz November 4, 2024 20:13

gtalha07 force-pushed the chat-ng-input branch from b3fd900 to f0ab803 Compare November 4, 2024 20:25

gtalha07 and others added 5 commits November 4, 2024 20:28

fix merge conflicts

9fcf9d8

fix lint errors

71fcd4f

Merge branch 'main' into chat-ng-input

16c9256

add changelogs

5679cab

make editor size more responsive on desktop

0ef9aa3

fix and update compose draft state with editorState

0be9f23

kumarpalsinh25 reviewed Nov 6, 2024

View reviewed changes

gtalha07 added 2 commits November 7, 2024 12:14

Feedback review: clean up and organise chat editor

3c5a6af

Feedback review: clean up and organise mention block

def4f25

gtalha07 requested review from kumarpalsinh25 and bitfriend November 7, 2024 13:12

kumarpalsinh25 requested changes Nov 8, 2024

View reviewed changes

gtalha07 marked this pull request as draft November 8, 2024 13:45

gtalha07 added 5 commits November 8, 2024 19:17

fix and organise mentions structure

29f4ab3

fix mention selection logic

224e62b

add L10n strings

01d17e4

make inline mentions prettier

4e172c0

keep keyboard events only listenable for desktop version

cd28e67

gtalha07 marked this pull request as ready for review November 10, 2024 21:52

gtalha07 requested review from kumarpalsinh25 and removed request for bitfriend, emilvincentz and kumarpalsinh25 November 10, 2024 21:53

gtalha07 commented Nov 11, 2024

View reviewed changes

gnunicorn reviewed Nov 11, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Chat-ng]: Appflowy Editor editor integration #2332

[Chat-ng]: Appflowy Editor editor integration #2332

gtalha07 commented Oct 29, 2024 •

edited

Loading

codecov bot commented Oct 29, 2024 •

edited

Loading

gnunicorn commented Nov 5, 2024

kumarpalsinh25 commented Nov 6, 2024

kumarpalsinh25 left a comment

kumarpalsinh25 left a comment •

edited

Loading

kumarpalsinh25 Nov 8, 2024

gtalha07 Nov 11, 2024

gtalha07 Nov 11, 2024

gtalha07 Nov 11, 2024

gtalha07 Nov 11, 2024

gtalha07 Nov 11, 2024

gtalha07 Nov 11, 2024

gnunicorn left a comment

gnunicorn Nov 5, 2024

gnunicorn Nov 11, 2024

gtalha07 Nov 11, 2024

gnunicorn Nov 11, 2024

gnunicorn Nov 11, 2024

gnunicorn Nov 11, 2024

gtalha07 Nov 11, 2024

gnunicorn Nov 11, 2024

gnunicorn Nov 11, 2024

gnunicorn Nov 11, 2024

gnunicorn Nov 11, 2024

		@@ -0,0 +1 @@
		- Chat-NG now supports Appflowy editor with proper markdown support. Some/Other features might be disabled or limited for now.

	- Chat-NG now supports Appflowy editor with proper markdown support. Some/Other features might be disabled or limited for now.
	- [Labs] Chat-NG now offers a fresh new WYSIWYG editor. More features upcoming.

[Chat-ng]: Appflowy Editor editor integration #2332

Are you sure you want to change the base?

[Chat-ng]: Appflowy Editor editor integration #2332

Conversation

gtalha07 commented Oct 29, 2024 • edited Loading

Desktop Demo with Mentions UI:

Mobile UI:

codecov bot commented Oct 29, 2024 • edited Loading

Codecov Report

gnunicorn commented Nov 5, 2024

kumarpalsinh25 commented Nov 6, 2024

kumarpalsinh25 left a comment

Choose a reason for hiding this comment

kumarpalsinh25 left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gnunicorn left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gtalha07 commented Oct 29, 2024 •

edited

Loading

codecov bot commented Oct 29, 2024 •

edited

Loading

kumarpalsinh25 left a comment •

edited

Loading