Skip to content

Commit

Permalink
Added experimental line reconstruction shaders
Browse files Browse the repository at this point in the history
  • Loading branch information
bloc97 committed Jul 26, 2021
1 parent 4e2cd02 commit 15ef379
Show file tree
Hide file tree
Showing 4 changed files with 739 additions and 0 deletions.
94 changes: 94 additions & 0 deletions glsl/Line-Reconstruction/Anime4K_DeRing.glsl
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
//!DESC Anime4K-v4.0-De-Ring-Compute-Statistics
//!HOOK LUMA
//!BIND HOOKED
//!SAVE GAUSS
//!COMPONENTS 2

#define KERNELSIZE 5 //Kernel size, must be an positive odd integer.
#define KERNELHALFSIZE 2 //Half of the kernel size without remainder. Must be equal to trunc(KERNELSIZE/2).

#define L_tex HOOKED_tex

float comp_max_x(vec2 pos) {

float g = 0;

for (int i=0; i<KERNELSIZE; i++) {
float di = float(i - KERNELHALFSIZE);
float df = HOOKED_pt.x * di;

g = max(g, (L_tex(pos + vec2(df, 0)).x));
}

return g;
}
float comp_min_x(vec2 pos) {

float g = 0;

for (int i=0; i<KERNELSIZE; i++) {
float di = float(i - KERNELHALFSIZE);
float df = HOOKED_pt.x * di;

g = min(g, (L_tex(pos + vec2(df, 0)).x));
}

return g;
}

vec4 hook() {
return vec4(comp_max_x(HOOKED_pos), comp_min_x(HOOKED_pos), 0, 0);
}

//!DESC Anime4K-v4.0-De-Ring-Compute-Statistics
//!HOOK LUMA
//!BIND HOOKED
//!BIND GAUSS
//!SAVE GAUSS
//!COMPONENTS 2

#define KERNELSIZE 5 //Kernel size, must be an positive odd integer.
#define KERNELHALFSIZE 2 //Half of the kernel size without remainder. Must be equal to trunc(KERNELSIZE/2).

#define L_tex GAUSS_tex

float comp_max_y(vec2 pos) {

float g = 0;

for (int i=0; i<KERNELSIZE; i++) {
float di = float(i - KERNELHALFSIZE);
float df = HOOKED_pt.y * di;

g = max(g, (L_tex(pos + vec2(0, df)).x));
}

return g;
}
float comp_min_y(vec2 pos) {

float g = 0;

for (int i=0; i<KERNELSIZE; i++) {
float di = float(i - KERNELHALFSIZE);
float df = HOOKED_pt.y * di;

g = min(g, (L_tex(pos + vec2(0, df)).y));
}

return g;
}
vec4 hook() {
return vec4(comp_max_y(HOOKED_pos), comp_min_y(HOOKED_pos), 0, 0);
}

//!DESC Anime4K-v4.0-De-Ring
//!HOOK NATIVE
//!BIND HOOKED
//!BIND GAUSS

vec4 hook() {
float luma_clamp = min(HOOKED_tex(HOOKED_pos).x, (GAUSS_tex(HOOKED_pos).x));
luma_clamp = max(luma_clamp, (GAUSS_tex(HOOKED_pos).y));
return vec4(luma_clamp, HOOKED_tex(HOOKED_pos).yzw);
}
Loading

2 comments on commit 15ef379

@Artins90
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks amazing, eyes in particular are extremely close to waifu2x which is a huge accomplishment.
Line thinning seems a bit too strong on already thin lines, the thinned lines are roughly half the thickness of the original ones.
Thicker lines don't seem to be affected.

@bloc97
Copy link
Owner Author

@bloc97 bloc97 commented on 15ef379 Jul 26, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the feedback. The "line thinning" effect is actually not intended but was learned by the algorithm. Nowhere in the training we specified for it to thin lines. We suspect it learned this as most lines in anime are actually either very thick (20px+ wide for solid shapes like eyebrows, mouth etc.) or very thin (1-2 px), and when we correct the distribution shift, the image will naturally contain very thin lines and very thick lines, with nothing in between. 4K anime does not exist and the algorithm simply learned to reproduce 1080p anime in 4K format... I'm not sure if this can even be fixed, as we cannot get ground truth for 4K anime images.

Edit: To put it simply, 2px line in 1080p is twice the size as a 2px line in 4K. The line reconstruction algorithm preserves thickness with respect to pixel size, and not "screen" thickness. If you view the anime on a 4K monitor 2x larger and compare with the original on a 1080p monitor, line thickness would be the same.

Please sign in to comment.