showvalues display multi-line string incorrectly

[islent]

using ProgressMeter

function testplot(n = 10)
    p = Progress(n)
    for iter in 1:n
        sleep(0.2)
        s = "line 1\nline 2\nline 3"
        next!(p; showvalues = [("lines", s)])
    end
end

Output:

julia> testplot()
Progress:  20%|█████████████                                                    |  ETA: 0:00:02
  lines:  line 1
Progress:  30%|███████████████████▌                                             |  ETA: 0:00:01
  lines:  line 1
Progress:  40%|██████████████████████████                                       |  ETA: 0:00:01
  lines:  line 1
Progress:  50%|████████████████████████████████▌                                |  ETA: 0:00:01
  lines:  line 1
Progress:  60%|███████████████████████████████████████                          |  ETA: 0:00:01
  lines:  line 1
Progress:  70%|█████████████████████████████████████████████▌                   |  ETA: 0:00:01
  lines:  line 1
Progress:  80%|████████████████████████████████████████████████████             |  ETA: 0:00:00
  lines:  line 1
Progress:  90%|██████████████████████████████████████████████████████████▌      |  ETA: 0:00:00
  lines:  line 1
Progress: 100%|█████████████████████████████████████████████████████████████████| Time: 0:00:02
  lines:  line 1
line 2
line 3

Need to detect the number of lines contained in string?

[lrnv]

I second this behavior I am struggling with the same thing right now, because I want to use UnicodePlots.jl to display a graph there, and it keeps getting the number of lines wrong. I have the following repl :

Progress:  77%|██████████████████████████████▁        |  ETA: 0:00:02 ( 1.08 ms/it)
  Plot:                         ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀Running Average loss⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ 
                       ⡤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⢤ 
Progress:  80%|███████████████████████████████        |  ETA: 0:00:02 ( 1.08 ms/it)
  Plot:                         ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀Running Average loss⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ 
                       ⡤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⢤ 
Progress:  82%|███████████████████████████████▇       |  ETA: 0:00:02 ( 1.08 ms/it)
  Plot:                         ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀Running Average loss⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ 
                       ⡤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⢤ 
Progress:  86%|█████████████████████████████████▃     |  ETA: 0:00:02 ( 1.07 ms/it)
  Plot:                         ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀Running Average loss⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ 
                       ⡤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⢤ 
Progress:  89%|██████████████████████████████████▆    |  ETA: 0:00:01 ( 1.06 ms/it)
  Plot:                         ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀Running Average loss⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ 
                       ⡤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⢤ 
Progress:  92%|███████████████████████████████████▇   |  ETA: 0:00:01 ( 1.06 ms/it)
  Plot:                         ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀Running Average loss⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ 
                       ⡤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⢤ 
Progress:  95%|████████████████████████████████████▇  |  ETA: 0:00:01 ( 1.05 ms/it)
  Plot:                         ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀Running Average loss⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ 
                       ⡤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⢤ 
Progress:  97%|██████████████████████████████████████ |  ETA: 0:00:00 ( 1.05 ms/it)
  Plot:                         ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀Running Average loss⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ 
                       ⡤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⢤ 
Progress: 100%|███████████████████████████████████████|  ETA: 0:00:00 ( 1.05 ms/it)
  Plot:                         ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀Running Average loss⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ 
                       ⡤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⢤ 
Progress: 100%|███████████████████████████████████████| Time: 0:00:10 ( 1.05 ms/it)
  Plot:                         ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀Running Average loss⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ 
                       ⡤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⠤⢤ 
             10¹⸱⁶⁰⁷⁹¹ ⡇⢧⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢸ 
                       ⡇⠀⢧⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢸ 
                       ⡇⠀⠈⢧⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢸ 
   Avg loss            ⡇⠀⠀⠘⣆⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢸ 
                       ⡇⠀⠀⠀⠘⡆⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢸ 
                       ⡇⠀⠀⠀⠀⠙⣆⣀⢀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢸ 
                       ⡇⠀⠀⠀⠀⠀⠀⠉⠉⠉⠉⠙⠙⠛⠓⠶⠶⠤⢤⣤⣀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢸ 
            10⁻³⸱³⁸⁷¹⁶ ⡇⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠉⠉⠙⠛⠒⠒⠶⠤⠤⢤⣤⣤⣀⣠⣤⣀⣀⣀⣀⣠⣄⢸ 
                       ⠓⠒⠒⠒⠒⠒⠒⠒⠒⠒⠒⠒⠒⠒⠒⠒⠒⠒⠒⠒⠒⠒⠒⠒⠒⠒⠒⠒⠒⠒⠒⠒⠒⠒⠒⠒⠒⠒⠒⠒⠒⠚ 
                       ⠀0⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀10000⠀ 
                       ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀iter⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ 

julia> 

Moreover, the color does not pass through. Is there a way to let the colored output stay as it way instead of making the whole plot blue ?

[islent]

Yeah. I found the problem while trying to use UnicodePlots (which output figure as strings with "\n"). It would be a fascinating feature if we can plot under progress bars!

[lrnv]

The problem is definitely in the printvalues function:

ProgressMeter.jl/src/ProgressMeter.jl

Lines 584 to 607 in 5826a75

	# Internal method to print additional values below progress bar
	function printvalues!(p::AbstractProgress, showvalues; color = :normal, truncate = false)
	length(showvalues) == 0 && return
	maxwidth = maximum(Int[length(string(name)) for (name, _) in showvalues])
	p.numprintedvalues = 0
	for (name, value) in showvalues
	msg = "\n " * rpad(string(name) * ": ", maxwidth+2+1) * string(value)
	max_len = (displaysize(p.output)::Tuple{Int,Int})[2]
	# I don't understand why the minus 1 is necessary here, but empircally
	# it is needed.
	msg_lines = ceil(Int, (length(msg)-1) / max_len)
	if truncate && msg_lines >= 2
	# For multibyte characters, need to index with nextind.
	printover(p.output, msg[1:nextind(msg, 1, max_len-1)] * "…", color)
	p.numprintedvalues += 1
	else
	printover(p.output, msg, color)
	p.numprintedvalues += msg_lines
	end
	end
	p
	end

This function turns every value passed in showvalues into a string before plotting it, to be able to count the number of lines it'll take.

It should instead pass objects in the dictionary to the printing functions, which knows better how they should be printed (including colors in UnicodePlots for example). However, the computation of the number of lines should be completely reworked, and I must admit i do not have the knowledge to do it myself.

---

Ok I got something. The follwoing scheme should work to rpovide the right number of lines :

D = Dict(:First_thing => "lol",:second_thing => big(π),:plot =>UnicodePlots.scatterplot(1:100,log.(1:100)))

n_lines = length(D)
for (key,value) in D
    n_lines += sum([1 for i in eachmatch(r"\n",string((key,value)))])
end

# and showing : 
for (key,value) in D
    display((key,value))
end
# actually takes the right number of lines. 

Then, the current behavior is that printover only goes back to the start of the line and then prints the message s = string(value) as an abstractstring, which uses several lines as it should.

It should, for each (name,value) pair in showvalues, move back a certain number of lines coresponding to the previous calculations before passing the value itself (and not string(value)) to printstyled.

Hopefully, this should work correctly.

I'll implement it and try a PR when i have some time.

[adrhill]

It looks like UnicodePlots now works well with ProgressMeter:
https://user-images.githubusercontent.com/20258504/210437952-1073f1d6-2170-42b9-a9ca-63de052235e8.gif

using ProgressMeter
using UnicodePlots

n = 20
xs = Float64[]
p = Progress(n)
for iter = 1:n
    append!(xs, rand())
    sleep(0.5)
    plot = lineplot(xs)
    str = "\n" * string(plot; color=true) # use ANSI color codes and prepend newline
    ProgressMeter.next!(p; showvalues = [(:UnicodePlot, str)])
end

However I couldn't get the "line 1\nline 2\nline 3" example above to clear printed lines correctly either.

Moreover, the color does not pass through. Is there a way to let the colored output stay as it way instead of making the whole plot blue ?

This is fixed in the code above by using UnicodePlots' string(plot; color=true).

[islent]

I have an idea: replace \n with spaces while matching the size of terminal width.

A better example:

function replace_EOL_with_space(s)
    width = displaysize(stdout)[2]
    lines = split(s, "\n")
    filled = lines .* " ".^(width .- sizeof.(lines))
    return join(filled)
end

function testplot(n = 10)
    p = Progress(n)
    for iter in 1:n
        sleep(0.2)
        s = "line 1\nline 2\nline 3"
        next!(p; showvalues = [("lines", replace_EOL_with_space(s))])
    end
end

Maybe this could be integrated in ProgressMeter.next!. I'll open a PR for it.