Performance maintenance release #389
Comments
|
Could you give an idea what a stress test should look like ? I may come up with something. |
A stress test is code that will intentionally abuse the good nature and features of Shoes. Here is a few examples of stress tests:
The current cache is on-disk where, for example, files or data are temporarily saved on disk. The problem with that is in the fact that creating, opening, accessing and closing files are costly operations. A single SQLite database ( |
|
@backorder good. Some methods from the skillwheel can be useful as they comply with those requirements. We can adjust them to a state we need. Here is anexample. def set (img, options={}, &block )
img.hover { @hovers.show text: options[:text], header: options[:header] , size: 9, text2: options[:text2], width: options[:width], height: options[:height]; img.scale 1.25, 1.25 }
img.leave { @hovers.hide; img.scale 0.8, 0.8 }
img.click { @hovers.hide; block.call if block_given? }
end
set ( image "pics/misc/s_damage.png", left: 80, top: 11, width: 50 ), text: pane_text[4], width: 500, height: 40
This happens when Shoes is drawing. Unless you had anything else in mind cycling through a drawing pattern will be good enough.
How can this be achieved ? Threading is not something that works flawlessly in Shoes. |
Many things keep Shoes busy. We just need sure way(s) to reproduce the problem.
Threading is a problem, RE: Challenge: Ruby uses GIL, GTK is not thread-safe. Maybe have many arts being animated in one single path-animation does clearly demonstrate how slow animation is. Now if one would want to write a game in Shoes, let's say a Super Mario World clone, that wouldn't go too well according to |
|
Here are multiple animations at once. Shoes.app do
def anim i
animate(24) do |frame|
@counter[i].replace "FRAME #{i+1} #{frame*(i+1)}"
end
end
@counter = []
@counter << para("STARTING")
@counter << para("SECOND", left: 0, top: 30)
@counter << para("THIRD", left: 0, top: 60)
i = 0
3.times do
anim i
i+=1
end
end@backorder regarding threads and drawings (probably animations too). Some time ago I found a way to cheat Shoes. Not sure how to explain it though. My second app has 3 tabs. One of the tabs is called "online store" when you click on "Update package list" the app makes an http query downloads the new stuff (if any) and shows them to the user. While the transaction is going "Loading" animation is shown to the user. Usually when the transaction is undergoing the app is frozen until its completion but in this case the it is threaded so the user can have control over other app functions. Here is the code sample responsible for it: button("Update package list", left: 30, top: 10, width: 360, height: 20) do
check_dl == 0? nil : (messages 0; next)
@pack_contain.clear { spinner left: 113, top: 90, start: true, tooltip: "Waiting for something?" }
Thread.new do
repo_data = get_url @server_url
start do
repo_data.nil? ? ( messages 3 ) : ( File.open('NCF_repository/package_list.txt', "w") { |f| f.write repo_data } )
main_pack_block_online
end
end
end |
Interesting approach. Also, this is not a stress test considering that Shoes do display without any efforts. There is something interesting though. I increased the FPS to one million and it's clear that it's peaking way before. Displaying the actual FPS (Frame Per Second) would tell us what the actual peak is. NOTE: Shoes internal might be able to provide an FPS counter enabled in debugging mode.
Threads created in Ruby suffer from GIL (Global Interpreter Lock). It means they are in fact executed one at a time. No matter what you do it won't really work. Ruby team plan to fix this on Ruby 3.0. Might take a while. haha
You are correct. Downloading data usually freeze Shoes UI. It is normally not noticeable for small chunk of data but our tests on larger data (ISO file) demonstrated that it will completely blank the UI until download is finished. Also the thread doesn't do anything useful here (GIL again). |
|
Additional reference: https://en.wikipedia.org/wiki/Stress_testing_(software) |
|
Inspired from your code. The more counters, the lower the actual FPS. The more the FPS, the less it seems to make a difference? Sounds like there is frame dropping. You can profile and enjoy the results. You should obtain the FPS you initially set up when you lower NUMBER_OF_COUNTERS = 150
FPS = 24
Shoes.app do
@counter = []
@text = []
NUMBER_OF_COUNTERS.times do |n|
@counter << Time.now
@text << para
animate(FPS) do |frames|
if (0 == (frames % FPS))
@text[n].text = "FPS #{ FPS / ((frames >= FPS) ? (Time.now - @counter[n]) : 1.0)}\n"
@counter[n] = Time.now
end
end
end
end |
|
Hmm. Simple things have a crazy amount of NUMBER_OF_BUTTONS = 2000
Shoes.app do
NUMBER_OF_BUTTONS.times do |n|
button "[#{n}]"
end
end
An empty Shoes app looks like this: |
|
Awesome :). this is about 9 to_s per cycle. but why? :/ |
|
We will know when 3.3.9 comes. For now we should slowly write stress tests and collect data. It might be as simple as refreshing the window even when it doesn't need it. Or as complicated as adding widgets cause underlying hidden elements to refresh. Scrolling may also refresh everything even when not visible. Hours of pleasure guaranteed. |
Please be careful claiming this as a truth. It's not safe in certain situations (gthreads and its nuanced) - does shoes use those special situations - I don't think we do. Shoes one gtk thread (mainloop) Explain how the samples/simple/download.rb works when calling lib/shoes/download.rb if threading is unsafe. Ruby Threads however do have the GIL locking issue. If you really care about performance, then using newer Rubies is what you want. |
Please - bench mark the simple sdbm key-value store used for shoes external image caches versus sqlite3, both inserts and fetches. Most images are cached in memory and never get loaded from external cache (and only once). Optimization requires knowledge, not speculation.
Or you could do that. |
My mistake. I remember now that we simply need to tell GLib when we enter/leave the GTK thread.
Ruby performance is not the core of the problem and the difference in performance when upgrading from x.x.x to x.x.z is generally small. GIL is however a big problem. Threaded applications are considerably faster in jRuby. Wouldn't agree that most of the bottleneck is in Shoes? It wasn't built with performance in mind.
This issue is exactly where we build the said knowledge but it all start from hypothesis. Based on your feedback, a representative benchmark sdbm/sqlite would be the very thing to do. |
You are missing the important thing: Are Shoes 4 applications faster than Shoes 3.3.x? - that's your benchmark - not ruby vs jruby - we run Shoes.. You probably know that java swt uses gtk3 and cocoa just like Shoes 3 so they use cairo and pango too. You would also know they have to translate drawing from Shoes4 -> swt-> cairo/pango and they are fighting Benchmark properly with context that matters to you. |
|
Is there a way to simulate hover without actually hovering with the mouse? It will be good for automation scripts. |
|
I am not saying Shoes 4 is faster (or better) than Shoes 3. I am not suggesting to move to jRuby. I am saying threading in Ruby is useless. Any GUI application needs a way to effectively balance visual components and its tasks.
I am not expecting you to agree with everything I say. You say sdbm is fine? Alright, we can profile and benchmark the thing and see how it turns out. Maybe it is fine! Does anyone have ever extensively profile and benchmark Shoes3? If nobody has, then nobody knows exactly and anything we say is speculative. So I did setup this issue as a conversation starter and to investigate in order to get all the answers that we need. |
|
I did build and profile Shoes 3 with the -gprof flag once, Might still be an option in the linux rakefiles. For the script I ran, most of the cpu time was inside Ruby and not Shoes/cairo/pango/gtk3 (you need a ruby with debugging info to do that). I also know that graphics performance is hardware constrained - by both the cpu and the gpu that gtk was built to support on that platform - not to mention disk speed if loading things.
Perhaps you could look at samples/simple/download.rb and lib/shoes.download and see it threading is useless. It may not be all you want but it is working. Finding bottlenecks in performance is multi-dimensional - it's not easy to do properly and it's not easy to fix in code unless done properly. Wholesale code changes because you think something is slow and have a better idea is not a proper evaluation remember typhoeus? |
|
Threading works for me though. My app does not freeze during the http requests when it is threaded. |
@dredknight when a task performed by a thread is ever so small, you won't notice that it actually had to finish before moving to the next thread. You can test by yourself but nothing runs in parallel in Ruby.
We should definitively make it happen again on all platforms.
I get your point about time spent in Ruby versus Shoes & friends. Though let's not forget some of the Ruby classes and methods are defined in Shoes/C. Those we can improve on. For example, one thing is clear is that there are too many draw calls. It's impossible to need that many calls. If we could somehow reduce the calls by, say, 20%, it might make a whole lot of difference to Shoes users.
What would be your take on Cairo/Pango vs Skia? Skia is developed by Google and used on major web browsers and widely used applications. The C API is fairly similar to Cairo but is also more wholesome and actively developed.
Shoes directly using CURL in an independent C thread might have been a better solution but more work. To be fair, the tests with Typhoeus on Ruby alone were extremely promising. In the Shoes ecosystem? Not so much but still an improvement over the previous method (including dealing with https, right?) and advanced users get Typhoeus gem included with Shoes. It should be noted that the difference between tests on Ruby alone versus Shoes should also tell us there is room for improvement in Shoes.
Absolutely true. You might have your own suspicions about what the bottlenecks are. How about you share with us and @dredknight and I work on some stress tests for those? Listen, reading my initial post again clearly shows that things like caching was worded in a way that does not imply anything more than an investigation, e.g. "review, may consider". Maybe you misunderstood the purpose of this issue but this is really an investigative process. We get the tools we need, write the tests we need and investigate the bottlenecks before anything else happen. Hopefully it sounds reasonable to you. |
|
@backorder I believe it is not threaded but for some reason there is a visible difference with and without the thread. The code above without the thread.new end row simply does not execute the rotation animation (or at least this is what it look like because the user does not see the spinner). I am currently finishing a few scripts for the app. After that I will have even more time to dedicate on building performance tools. this is something I am very keen to learn and get better at! |
Improve on one or two of the lines of rb_call_something. That's not the problem.
You should look at time in method , not counts. Windows/X/Cocoa compress multiple draws - have done so almost forever. Fascinating topic but nothing Shoes can or should touch.
C++ ? Not me, I've suffered enough.
No problem unless you are asking me to do the coding so you can explore. Low priority for me. |
Excellent!
This is only true for system calls but not for Ruby calls (such as draw).
Haha! It is written in C++ but it has a C API.
Right. That's why it's suggested for a future release. It's not surprising that it is low priority for you because you also spend most of your time on Shoes/C. We can accept the performances of Ruby because Shoes never meant to be the fastest around the corner but there are few things that need to be addressed, such as UI responsiveness, slow display and animate. |
|
The threshold on my machine seems to be 117, 118 images. Profiled for 5 seconds. IMAGES = [
"shoes-icon.png",
"shoes-icon-blue.png",
"shoes-icon-federales.png",
"shoes-icon-red.png"
]
NUMBER_OF_IMAGES = 117
Shoes.app do
@images = []
@interpolator = (tmp = (0..50).collect { |n| -n }) + tmp.reverse
NUMBER_OF_IMAGES.times do
@images << image("#{DIR}/static/#{IMAGES.sample}")
end
@counter = 1
animate(@fps = 60) do |frame|
@images.each { |img| img.rotate(@interpolator.first) }
if ((frame / @fps) == @counter)
@counter += 1
@interpolator.push @interpolator.shift
end
end
end
|
|
Posting some more code testing the limits of Shoes. Brownian motion. 190 animated ovals is fine, 192 is not. Wondering why path-animation sample is performing so poorly. NUMBER_OF_SHAPES = 192
Shoes.app do
@shapes = []
NUMBER_OF_SHAPES.times do
fill rgb(rand(255), rand(255), rand(255))
@shapes << oval(rand(self.width), rand(self.height), rand(100))
end
animate(60) do
@shapes.each do |shape|
mx = rand > 0.5 ? +1 : -1
my = rand > 0.5 ? +1 : -1
shape.move shape.left + mx, shape.top + my
end
end
end |
|
@dredknight I created a branch performance on Shoes repo to avoid polluting this thread with too much stuff. Don't be shy to add stuff in Tests/performance. More about Shoes and branches: https://github.com/shoes/shoes3/wiki/Git,-Github-and-Shoes @ccoupe branch related instructions are working well. Good job. |
- Fairly decent performances - Adding Shoes art (cairo-based) performances decrease significantly
|
Slot manipulations are surprisingly fast on widgets, texts and images. Significant performance decrease when introducing Shoes arts (cairo-based). |
|
Be aware that Shoes decides how much time to give to Ruby vs Gtk event handling and it differs on Linux vs Windows. see shoes_app_g_poll() in shoes/native/gtk.c for Linux/BSD and shoes_native_loop() (same file) which happens to be the adjustment I made to dredknight cpu hogging bug. As @backorder remembers the Shoes 2 and 3.1 code took 100% of a core doing nothing and no one knew why so I moved Shoes Windows to use Gtk3. The Linux polling is _why's with some comments from me. Mysterious place.
My head hurts figuring out the statistics for each or the rand()s combinations in that script . You might need to run that for a minute or two. Also I believe animate has a small memory leak that I never found. There is also ruby gc at play. |
I am open to alternatives and suggestions. Right now I am trying to figure out the tipping points of Shoes. Things that are abusive enough to cause disruption in Shoes but not enough to have it show a blank window. Later on we can come up with a clean set of stress tests. Would you prefer caching random values in a YAML file then load up in an array? It's difficult to abuse Shoes without randomness but this approach could work. I could otherwise set a number of iterations for each operation. It is also possible to create animate-less code but it will show a different aspect, namely startup and setting UI. Sometimes Shoes takes some time to show up. That could be interesting. All the cool things in Shoes go through animate. Eventually we need to fix this. What a coincidence that is issue #1 ! Running slot_manipulation for 5 minutes
|
|
The profiler is missing a bit of features. It would be good to have time the profiler ran. Perhaps even the ability to set a timer to auto-stop the profiler. Also, we should consider a GUI-less profiler (similar to packing app) where we we could write a stress tests main app or something. |
|
It's just Ruby and most of that is just display/reporting - hack away. There is a |
|
The terminal view is much more useful for decision making. Be aware, Windows/Ruby doesn't provide cpu-time so you can't make decsions based on that - just clock time for windows. |
|
Thanks for the feedback. |
|
Hello everyone, I am not sure how radio button is made but it generates absurd amount of draws. Shoes.app do
def set_checkers stuff, place
stuff.each_with_index do |x, i|
place.append do
flow do
radio :item;
end
end
end
end
ITEMS = Array.new( 100, 1 )
desktop = stack left: 20, top: 40, width: 200;
set_checkers ITEMS, desktop
end
While experimenting on how to make things worse for shoes I kind of broke it #391 |
|
So far it seems that |
|
Btw how does shoes draws things exactly? If it draws them dot by dot it kind of makes sense to call the function that many times. |
There are some parts of Shoes that I do not fully understand. @ccoupe knows best. One thing is for sure: we are pushing Shoes to the limits in this thread and we are going to uncover a lot of defects. :) |
|
holy moly I found something!!! Shoes.app do
ITEMS = Array.new( 1000, 1 )
ITEMS.each_with_index do |x, i|
flow;
end
end
|
|
I'm getting a little over 3000 draw on an empty Shoes.app ran for one minute. It's probably not all that bad to be called so many times but it doesn't hurt to investigate and make sure. |
|
3000 draw in 60 seconds means 50 FPS, which is fair (could be a bit faster) and would explain the number of calls on an empty Shoes.app. |
Each flow has an internal canvas (drawing surface) and it has background to possibly draw. Counts need context - how fast are those draws (in cpu time and clock time) and the knowledge of what your test script is really testing. |
|
Can you tell me where I can look for what makes the drawing in Shoes? I want to poke inside. May be some ideas will materialize. |
Let's talk about an hypothetical 3.3.9 release focusing on performances.
image[]? Are Image[] and Image[]= undocumented? #285)References
https://github.com/shoes/shoes3/wiki/Profiling
https://en.wikipedia.org/wiki/Skia_Graphics_Engine
https://en.wikipedia.org/wiki/Stress_testing_(software)
https://developer.gnome.org/gdk3/stable/gdk3-Threads.html
The text was updated successfully, but these errors were encountered: