A few random ideas and suggestions possibly. #181
GrimblyGorn
started this conversation in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I previously had started a Feature Request for more nodes over here, and rather than extend that this seems a better place for new additions. So, with those amended to this now here's a few more nodes that would be handy to have.
This LLul extension, which may require more than one node and possibly some .js gui stuff, idk. That's why I haven't tried it yet tbh.
This Latent Couple extension could be handy too.
These Multidiffusion & TiledVAE extensions would be very handy. I only have 4gb vram to work with so dissecting things into larger images or outpainting to larger images is very appealing in general. Also, there's this Multidiffusion and this MixedDiffusion, which could both be as useful too.
Not really sure if this makes sense but I found this the other day and it seems interesting enough to play with if that were an option. Also, they mention the HF pipelines and their usage. It's not clear if those also mean it must run remotely though. If they could simply be used as a plugin method for extending this with such options that could be handy to have. I'd prefer they run locally though or not at all, so those pipelines may not be possible. But, the Attend and Excite thing does still look interesting to implement.
The context menu for nodes includes "Pin" and various "Modes" for them, but those seem to change nothing. Could be handy if those were more fully utilized and visually responsive. I would assume setting the Mode to Never meant it was Disabled. If not, that would be handy to have. Also then, it could perhaps be made to visually represent this new Disabled status and close off the downstream nodes it's connected to. This is similar to Rhino/Grasshopper and in general many useful ideas from that could be applied here for further gui improvements.
I'm not sure on the details specifically, but various grid nodes would be handy I think. Things for crafting various xyz grids with various controls over the outputs and the Legend/Text Adornments included in the image. There's a few examples of such things for Automatic's gui. Could be handy here too.
Possibly more useful would be various grid based latent manipulators. Creating grids of latent images to stitch back into other grids. Also, nodes for crafting various grids from a collection of smaller images. So, I guess nodes that output a list of images from a grid and nodes that take a list of images to make a grid. This may basically be the Multidiffsion/TiledVAE thing above too though, so possibly this point is already made :)
Some generic noise generator nodes (Gaussian, Perlin, etc). These could be handy later for Controlnet type images or for some interesting Latent blending images. Like the Double Exposure models try for but with multiple noise layers blended together instead.
Also, some less generic noise nodes like this Wave Function Collapse which seems like it could makes some very useful results in a similar fashion to those mentioned above. That guy has some other interesting libraries that could be used for other nodes too.
Some type of embedding selector could be handy. Not really sure how that could work in this gui or if a node would make sense for it, but I have many hundreds of embeds and it would be very useful to readily view them and select them, or copy/paste them somehow. Likely, with the custom formatting for use in ComfyUI already handled as well. That could help improve experimenting with those more fluidly than is currently an option.
Some visual means of browsing all these Loras, embeds, and models. Not really sure how that could be implemented in this gui though. I found this the other day. If something like that could be implemented into this gui it could prove quite helpful. Maybe with some pulldown drawer type box to display all the images and data in? Then we could readily see what we were selecting and what we had to choose from, and possibly click embed keywords or similar right there to apply them or at least copy them for use.
Some method for extensions to add a repo to a list contained in a .json to allow random repos to be linked, to pull in new nodes from. Maybe it could be a .sh file with a wget list of all the repos officially added or something. It doesn't have to start off complicated to be useful from the start. Having a plugin directory for modded nodes and node packs you could add, or anything else for that matter would prove useful for everyone I think. Likely whatever they are doing for the Automatic webui could be ported over to this, at least for this extension portion. It could possibly be some startup option in the gui with a list of checkboxes for all the plugins/node packs and their descriptions. Then you select the ones you want, it runs the DL process for them, then restarts or something.
Another idea that keeps coming to mind for me lately is an alternate way to display the nodes I have available. Something like the Ribbon bar in Grasshopper3D would be nice possibly. The current access to nodes is fine for using them, but it's not as useful for seeing what's possible with them. Having them more visually available to contemplate while doing other things is helpful for crafting new things you may not have thought of otherwise. Also, having a lot more nodes will quickly leave more forgotten and neglected than if they were more present and visible to remind you of them. In Grasshopper I have more than 10,000 nodes available. This is how I can claim such things. Even with their increased visibility it is too many now, and many are forgotten and rarely used. But, there are many that are used much more than they ever would have been thanks to the visibility, and it has led to me making many interesting things having them laying around to remind me they can offer such things. If the plugin system above is made this should be worked out after too. It will prove useful.
Optional descriptions for the node at the bottom would be nice. Some secondary bubble attached to the node with the description for what it is, why that matters, how it's used, etc. Just something to remind people when they're completely stuck and brain-dead as to what they're even looking at maybe :)
Also, along the lines of greater visibility and access to nodes once more are added, something like this would be quite handy possibly. Since it has no pics and seemingly they are hard to find around the web now, I took some of my own to showcase how it works.
This 1st one shows the main options when you hit Alt+Mousewheel Click. These would be the base menu items for ComfyUI, the categories.
This 2nd one shows one of those being selected, so for ComfyUI this would be a submenu and it's nested extras.
Something like this could be very handy alongside many new nodes, and even alongside the proposed UI upgrades above.
This may not be an option, but it seems like if ComfyUI is based off litegraph.js there should be some way to make use of all the nodes it natively starts with over here. There's a bunch of nodes in those script files that would likely be very useful general nodes for use in ComfyUI. Just a thought.
There's at least one attempt already that I've run across which attempts to arrange the nodes in some fashion. If possible I think something like this would be very handy. Sadly they seem not share any code for that to start from, but perhaps someone has an idea where to start on that already or something similar already started. In Grasshopper it's extremely useful for making very neat and organized node graphs. That proves more useful the larger they get too.
In cases where the node has been Collapsed, there are connections made, and there is only one output connector. It will show the connector in the collapsed form. However, it will not let you make use of it and connect it to more things in this collapsed form. Instead you must un-collapse it and then make use of it. In cases where there are many connectors this is a useful requirement to determine which is being selected. For those with only one it should be useable in the collapsed form still since there is no question of which is selected. In cases where there are no connections made and the node is Collapsed, it will not show the connector. This makes sense too if the display is showing connections instead of connectors, but this modified variant I've proposed would be more useful. Having it show the connectors on those with only one helps them remain usable in any form they take on. This adds to usability by default I would think. This seems to only affect the outputs possibly as the inputs on collapsed nodes seem to allow making connection to still. For the Clip Text encode at least. This may be a bug then I guess?
There seems to be no way of telling what values I'm passing around in certain cases. In most cases it's very clear from having entered the value. But, there's a lot of instances where it's not as clear and, there's currently no good way to display these values. Something that allowed a universal type input, then displayed that value, before passing it to an output would be helpful. A passthrough display node for value checking, watching, or bug fixing possibly. In Grasshopper, you can simply hover over the input/output connectors and it'll pop open a little display tooltips with the values that connector is receiving or passing. That could be very handy here too. This somewhat fails when there's too much data for the little popup window to display. In those cases, a node to display the data would be helpful too.
Building from the one above, some logging nodes. Basically, a universal input data logger node similar to the one mentioned above for checking data being passed through, but this one would keep a log file of them going instead. Something like a running list accumulator/generator node I guess? This could also be handy if dynamically displaying the data isn't as easy as with Grasshopper. Then the data could at least be logged for review elsewhere and/or later. I think these data nodes will prove more useful over time if implemented higher up. Such, that they can have an overview of the data types created and used by all the various custom nodes people are making. Though I'm not certain how viable that is. It would be useful too for a running list of wildcard/dynamic prompts generated and used.
With the above mentioned data/logging nodes made, it could make sense to have a few nodes for chaining said data/logs directly into a .json structure to output a .json file. Seems like that could prove useful for various things. Possibly more so, if those files were able to be rolled back into this through other nodes. Like a workflow in a workflow, or some type of worklist in a workflow? Idk, seems handy either way with the others above to support/justify its usage.
Not all nodes require a percentage bar like the KSamplers, but some can take a very long time to run still. It could be useful to have in the top-left corner, a little green checked line circling around the node open/close button. This could also vary its colors if desired to indicate various status changes. Possibly, red means an error occurred and the console should be checked or something? Having an little in-app console could be really useful in some cases too. This little circling indicator is simply a means to indicate the node is still thinking. They are outlined currently which is nice, but with no animations for that, it could appear stuck at times. Alternately, I have seen a rather nice rainbow glow outline that was animated. No idea where though :)
Given the above two points. Profiling methods per node seem an option then too. Could be useful for extreme repeat grind workflows that need to be optimized later. Also, for any A/B variant testing some profiling data would be useful to compare them with.
As pointed out here animated wires can be handy. Alternate wire colors, and wire type/layouts are also handy at times. These can go a long way to informing you of what's going on and where. Along with helping make neat looking workflows from the various layout styles. Various node adornments around the perimeter can have the same effect and be very helpful at times. Knime uses optional connectors on the edges to pass around various meta data and use it as needed along the way. There’s an entire category of nodes in Knime for manipulating and playing with this meta data. These types of enhancements will prove useful for any node based system. Though I will also say that any such “extras” like animations, that are technically unnecessary for correct operation, should be switchable someplace. There should always be toggles for such things so you can avoid it becoming bloated unavoidably, and unusable for those with low specs.
Loops, timers, and step sequencer type things. All of those could be useful too :)
Add a new context menu item “Clone All” that clones the selected nodes and all the wiring connecting them. This would be very useful when selecting large blocks of nodes to replicate 3-4 times. Having to wire such things manually becomes increasingly more difficult on each block with more wires in the way and distance from the source nodes. This could all be mitigated from that new menu item though.
Also for Context Menu items there are currently 2 items in the menu for every node, Inputs and Outputs. These seem to never be available for interaction. I propose they be utilized as a sub-menu with all available connections as menu items, upon highlighting a connection/menu item the wire can highlight in the gui thus making this an informative tool as well as a disconnect method. So, these sub-menu items should allow viewing the connections, and removing them easily as well. Also, yes there should be multiple inputs for individual connectors on nodes that could accommodate such things. This could be by way of Ctrl+Drag or Alt+Drag etc. Basically you can decide when to add extras to one or disconnect the others with your new one in the current fashion. The you'd have item type connections, and list type connections, and would only be lacking in tree type connections to match Grasshopper. In GH you hold the hotkey then drag the wire and it'll change the cursor to include a little plus symbol alongside their other wiring indicator icon it normally shows. Then you can readily see you're amending that wire to whatever is already connected there. Then the node could run them in sequence or whatever as desired.
Also for wiring, a combo hotkey+right-click could be handy for removal of any wire beneath the cursor at the time. Something like that could also be extended to a drag motion so you can swipe across a huge row of wires at once to remove them all. These subtle extras go a long way over time towards helping save time and sanity. In this case with each wire already having a disconnect point on the wire possibly you could have an option to double click the node, which highlights all the wires And brings them to the foreground above all other nodes and wires. Then you could more easily disconnect them through the current on-wire deletion points. Also this would be helpful on its own for tracing each node in complex graphs more quickly than is currently an option once density crowds them all in too much.
A set of generic Rest nodes, should prove very handy very quickly I would think. The standard Get, Post, etc. could go a long way to extending options for things not yet considered. Some obvious less generic ones could be specific to openAI thus allowing a nodepack with all the nodes required for a chatGPT session right in the ComfyUI gui. This could very quickly lead to interesting results.
An alternate hotkey driven method for importing images. Specifically, something allowing to drag an image into the gui, then with the hotkey down upon dropping the image, it imports only the positive/negative prompts. So, instead of pulling the whole workflow it's just getting some core elements to build from and dropping those into your current workflow instead of overwriting it. Maybe it could include other bits too like seeds or possibly even a little list of inclusions we could edit someplace to our liking.
Similar to the above, buried in 28, an alternate method of wiring could be useful. For example, set it such that using Alt + dropping the wire on a connector does like normal. But, while still holding the Alt key your cursor then acts as the wire from the same node the wire you dropped came from. So, you could Alt+ drop, then go click 5-6 more connections and wire them too, really quickly. This could have a little alternate cursor icon to indicate what's happening and that your in this "mode". It should save loads of time when spinning up large workflows from scratch or making huge chained alterations.
A testing function to auto-spawn nodes for quicker testing. So, maybe I have nodes in "Grim/tests" or "Grim/latent" this could allow for pointing to either of those and all nodes in them are spawned and chained as required. No idea how that could be implemented offhand but it does sound really handy for saving time on such tests. Possibly by way of a launch command/flag maybe (--test Grim/latent) or something like that. May not be worth the effort required to implement though.
That's all I have for now. If I have more later I can just amend them here now.
Beta Was this translation helpful? Give feedback.
All reactions