Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add SixelImage #436

Open
wants to merge 4 commits into
base: v2-exp
Choose a base branch
from
Open

Conversation

CannibalVox
Copy link

@CannibalVox CannibalVox commented Nov 14, 2024

This PR adds support for sixels to lipgloss. Sixels are a protocol for writing images to the terminal by writing a large blob of ANSI-escaped data. They function by encoding columns of 6 pixels into a single character (in much the same way base64 encodes data 6 bits at a time). Sixel images are paletted, with a palette established at the beginning of the image blob and pixels identifying palette entires by index while writing the pixel data.

Sixels are written one 6-pixel-tall band at a time, one color at a time. For each band, a single color's pixels are written, then a carriage return is written to bring the "cursor" back to the beginning of a band where a new color is selected and pixels written. This continues until the entire band has been drawn, at which time a line break is written to begin the next band.

Supporting sixels requires overcoming a few challenges:

  1. Sixels traditionally supported only 256 colors. Modern terminals that have added sixel support may not have this restriction, but there's no real way of knowing, and anyway, too many colors massively inflate the size of a sixel image. As a result, a 256 color quantization process needs to be undergone on the image to produce the palette. sixel_palette.go has a Median Cut implementation for sixels.

  2. Sixels do not support alpha, so how should it be handled? When writing the sixel image, we use the param p2=1 which leaves pixels that are not written as part of the sixel's pixel data the color that is already in the terminal window. For fully-transparent pixels we simply do not write the pixels as part of the pixel data. But what about semi-transparent pixels? In Style.RenderSixelImage, if the style has a background color, I use the alpha channel in the palette to mix the background color with the pixel color to produce a semi-transparent effect. Be aware that sixel's color space is only 0-100 rather than the traditional 0-255, so it is sometimes possible for the human eye to distinguish between the background color in the terminal and the background color as drawn with sixels. Generally it looks good as long as the two colors aren't side by side.
    image
    If there is no background color in the style, the pixels are drawn fully-opaque at the color they appear in the image.

  3. Because colors must be drawn one at a time, we generate the pixel data in two phases: first, we traverse the image and write filled bits to a BitSet. Then, we use the BitSet to generate the pixel data string and store it for later use. Style.RenderSixelImage is responsible for generating the palette, image size, and pixel data into a coherent ANSI blob.

I chose to not tackle these challenges (though I can if they need to be solved before merge):

  • There are not multiple render options. For instance, it's not unusual for sixel images to support dithering (increases the size of the ANSI blob due to defeating RLE, but improves how it looks). I believe it's also possible to take advantage of overdraw to increase the use of RLE and make pixel data smaller, but at the cost of longer creation times.
  • sixelColor is right now just an arbitrary struct that does not implement color.Color. If this were fixed, it would be possible for sixelPalette to be turned into a general-purpose Median Cut utility.
  • SixelImages do not know or attempt to know how large they are in terminal characters. Additionally, on Windows, because Windows Terminal has chosen to size sixels such that characters are 10px by 20px, images are distorted based on what your current font's aspect ratio is. Hypothetically, the sixel code in lipgloss could figure out how big everything in the terminal is and make sixels compatible with other Width/Height measuring methods in Style, and/or provide size correction on windows. My current thought is that size correction could be done to the image itself before passing it into lipgloss, though.

Copy link
Member

@aymanbagabas aymanbagabas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is amazing! Though, I would move the ANSI escape sequence part to x/ansi in a separate PR 🙂

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants