Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RAH v4 #737

Closed
wants to merge 10 commits into from
Closed

RAH v4 #737

wants to merge 10 commits into from

Conversation

Ebag333
Copy link
Contributor

@Ebag333 Ebag333 commented Sep 15, 2016

Carry over from v2. See the following for history:
#680
#688
#689

Merging from pyfa-org/master to the RAH v2 branch failed spectacularly, so creating a new clean branch.

@Ebag333
Copy link
Contributor Author

Ebag333 commented Sep 15, 2016

Paging @blitzmann and @MrNukealizer for code review and comments/thoughts.

Maybe this can get snuck in to the next build?

Copy link
Contributor

@MrNukealizer MrNukealizer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is definitely improving, but reducing the resistanceShiftAmount between cycles does not give accurate results. It can get fairly close, but it will never be correct. To give an accurate result when the RAH loops through a series of profiles, it needs to give the average of all the profiles in the loop.

# If damage pattern is even across the board, we "reset" back to original resist values.
logger.debug("Setting adaptivearmorhardener resists to uniform profile.")
# Do nothing, because the RAH gets recalculated, so just don't change it
runLoop = 0
Copy link
Contributor

@MrNukealizer MrNukealizer Sep 16, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why use the default profile for uniform damage instead of actually seeing how the RAH changes?

Copy link
Contributor Author

@Ebag333 Ebag333 Sep 16, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why use the default profile for uniform damage instead of actually seeing how the RAH changes?

We could, but the primary issue is that there is no easy way to distinguish between the default damage profile and a damage profile that has uniform damage.

I want to be able to get back to the default, so that people who don't use the damage profiles get the behavior they have come to expect. It also lets them see the difference in how the damage reduction looks when using RAH with defaults (old version) and the new reactive version.

Not to mention people compare Pyfa and EFT all the time, so this lets you keep the EFT like behavior (and better shows just how awesome Pyfa is).

If there was a clear indication which damage profile you had selected (and not just what the percentages are), then I'd be more inclined to do exactly this.

Until that happens though, it's very difficult to see at a glance which damage profile you have selected.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand why someone would want to see it at 15/15/15/15. Maybe it's just the way I use pyfa, but I can't think of any situation where you'd be looking at your ship's defenses and not want the reactive hardener to react.

# Most likely the RAH is cycling between two different profiles and is in an infinite loop.
# Reduce the amount of resists we shift each loop, to try and stabilize on a more average profile.
if resistanceShiftAmount > .01:
resistanceShiftAmount = resistanceShiftAmount - .01
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This causes inaccurate results. It can get fairly close to the actual results, but it will almost never be correct.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This causes inaccurate results. It can get fairly close to the actual results, but it will almost never be correct.

Anything we do will be inaccurate. Why? Because the resists are changing in a loop, and it's impossible to show a fluctuating value in a static field.

You went with the route of averaging the resists. There's a few problems with this.

One, it's not particularly clean. You end up with values like .8933473434. It's harder for an end user to grep 89.3% vs 89.5%.

Two, because the UI rounds it off at a 3 digits, you can end up with rounding issues, where the displayed resists don't add up to 100% (or adds up to more than 100%).I

Three, you have to store and loop through all the resists you've done previously. This is very expensive execution wise, as you end up with loops within loops. With this method we only care about the outer loop.

Unfortunately, we can't show that one tick you'll get hit for 533 damage, and the next tick will get hit for 574. If we reduce the swings (by reducing the transfer amount) we end up with something very close to the average resists without introducing complicated numbers.

It's a compromise, but any solution chosen will be one. This just happens to be the fastest, cleanest, and fairly close to accurate (that has been demonstrated so far).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One, it's not particularly clean. You end up with values like .8933473434. It's harder for an end user to grep 89.3% vs 89.5%.

Can you give an example of that? Out of all my tests (many try to be as complicated as possible), only one gives results ending in 0.75%/0.25%. I haven't seen any values more complicated than that, and the vast majority are multiples of 1% or 0.5%.

Two, because the UI rounds it off at a 3 digits, you can end up with rounding issues, where the displayed resists don't add up to 100% (or adds up to more than 100%).

I haven't seen any examples of that either. It's rare to get two decimal places, and I'm not sure if there's any situation that would give 3, let alone more.

Unfortunately, we can't show that one tick you'll get hit for 533 damage, and the next tick will get hit for 574. If we reduce the swings (by reducing the transfer amount) we end up with something very close to the average resists without introducing complicated numbers.

Or we could use the actual averages, which aren't complicated. That also takes into account the fact that a 6% swing can be enough to keep changing while 5% could get stuck one a value that's not near the average.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ultimately it comes down to this. Using your code for averages significantly slows down the cycling. There is 3x speed difference, which is not insignificant.

Basically the difference ends up being spending a lot of processing time to record and average out the last few results....or do this:
http://www.lightandmatter.com/html_books/lm/ch18/figs/sc-strongly-damped.png

The end result is very similar. If one is three times as fast, why not use it?

Copy link
Contributor

@MrNukealizer MrNukealizer Sep 16, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In theory that should work something like this: http://imgur.com/le3bwsg
If it worked that way it would be a great solution. The problem is that most of the time when the RAH enters a loop it's bouncing around in such a way that it can't converge on the average point. Reducing the shift amount between cycles tends to result in the RAH getting stuck at a point completely unrelated to the average of the loop cycles, or even any of the cycles themselves. I imagine it as normally skipping over a point of resistance, but bouncing off when the shift amount is reduced, kind of like this: http://imgur.com/6VxrL3Z

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/r/theydidthemath is all I can think of when reading these comments.

@MrNukealizer are you actually plugging these things into an application and graphing them? That is awesome, and helps visualize what the hell is going on.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, I don't graph it. That was just my attempt to illustrate the effect I've noticed by looking at the numbers.


adaptiveResists_tuple = [0.0] * 12
for damagePatternType in damagePattern_tuple:
attr = "armor%sDamageResonance" % damagePatternType[0].capitalize()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why the call to capitalize() when it's already capitalized?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why the call to capitalize() when it's already capitalized?

To normalize data.

CCP is really bad about this, and it doesn't help that you'll find examples of this scattered around.

For example, you'll find EM, Em, and em, but for this particular case we have to have Em or it won't read properly.

It's just a good coding practice.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The thing is you're not normalizing CCP data, you're normalizing hard coded strings that are defined in the same file.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, that's a fair point. Mostly it was just a carry over from what I've seen done elsewhere.

I don't think that normalizing the string is going to cause significant delays however, so maybe a bit picky?

Copy link
Contributor

@MrNukealizer MrNukealizer Sep 16, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair enough. I'm not very familiar with Python so I was just wondering if that had a purpose and what it was.

damagePattern.explosiveAmount,
damagePattern.explosiveAmount*fit.ship.getModifiedItemAttr('armorExplosiveDamageResonance'),
module.getModifiedItemAttr('armorExplosiveDamageResonance'),
module.getModifiedItemAttr('armorExplosiveDamageResonance')])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not a big deal, but initially sorting the damage types Em, Thermal, Kinetic, Explosive causes incorrect sorting of equal values later. The game seems to sort Em, Explosive, Kinetic, Thermal.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does one go about getting the right section of code for these comments?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not a big deal, but initially sorting the damage types Em, Thermal, Kinetic, Explosive causes incorrect sorting of equal values later. The game seems to sort Em, Explosive, Kinetic, Thermal.

Populating the initial tuple doesn't really matter, because the second thing we do every loop is to sort the tuple by modified damage first, name second. So if we end up in a scenario where two resists have identical damage done (post resists), it'll sort it properly no matter what loop it's on.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Populating the initial tuple doesn't really matter, because the second thing we do every loop is to sort the tuple by modified damage first, name second. So if we end up in a scenario where two resists have identical damage done (post resists), it'll sort it properly no matter what loop it's on.

That's where you're wrong. It will properly sort types that have different amounts of damage, but it won't touch the order of types that have the same amounts of damage. That doesn't matter for the amount of damage taken, thus why it's not a big deal, but it can cause the RAH numbers to appear significantly different than what happens in game. I prefer to be accurate if possible, and changing the order you add the data to the tuple is a simple way to make it more accurate.

Copy link
Contributor Author

@Ebag333 Ebag333 Sep 16, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's where you're wrong. It will properly sort types that have different amounts of damage, but it won't touch the order of types that have the same amounts of damage. That doesn't matter for the amount of damage taken, thus why it's not a big deal, but it can cause the RAH numbers to appear significantly different than what happens in game. I prefer to be accurate if possible, and changing the order you add the data to the tuple is a simple way to make it more accurate.

No, it sorts it correctly.

Here's the Gnosis with only a RAH, damage pattern is 100 | 101 | 101 | 101

Let's dump the resists in order (left being 0, right being 3):

2016-09-15 22:15:31,423 eos.effects.adaptivearmorhardener DEBUG    Presort (low -> high): Em | Thermal | Kinetic | Explosive

You can see the first time we dump resist names, the order matches how we populate the tuple. We haven't sorted it yet (since it's the very first loop), so we don't actually know what's the lowest resist.

We then run through our logic (including the sort, which happens as one of the first things):

2016-09-15 22:15:31,424 eos.effects.adaptivearmorhardener DEBUG    Adaptive Resists, Ship Resists, Modified Damage (EM|The|Kin|Exp) : 0.910000 | 0.790000 | 0.790000 | 0.910000 || 0.614250 | 0.533250 | 0.533250 | 0.614250 || 67.500000 | 68.175000 | 68.175000 | 68.175000

Now next loop lets dump the resist names again to see how it sorted them:

2016-09-15 22:15:31,424 eos.effects.adaptivearmorhardener DEBUG    Presort (low -> high): Em | Explosive | Kinetic | Thermal

Hey, look at that. EM is lowest (as it should be, since it took the least damage). The other three take equal modified damage, and it sorted them alphabetically as expected.

So. All in all, working properly.

@Ebag333
Copy link
Contributor Author

Ebag333 commented Sep 16, 2016

How does one go about getting the right section of code for these comments?

When you are browsing the file, you can comment on a line.

adaptiveResists_tuple = [0.0] * 12
# Apply module resists to the ship (for reals this time and not just pretend)
for damagePatternType in damagePattern_tuple:
attr = "armor%sDamageResonance" % damagePatternType[0].capitalize()
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above.

@MrNukealizer
Copy link
Contributor

I did a little benchmark to compare the performance of V4 and V3. I removed all logging and added calls to time.clock() before doing anything, between initialization and the loop, after the loop, (for V3) after averaging RAH cycles, and after everything is done.

The test was when first loading pyfa with a fit that has three boosters with different RAH setups, and its own RAH that ends up in a 4 cycle loop. All of the boosters get the "uniform profile" with V4 because they have the uniform damage profile. For V3 the first booster's RAH changes to 30/0/0/30, the second stops at 42/0/0/18, and the third booster does a 3 cycle loop.

V3:

2016-09-15 18:29:53,700 root                     INFO     Starting pyfa
2016-09-15 18:29:59,956 eos.effects.adaptivearmorhardener DEBUG    Initialization: 0.000070, Loop: 0.000053, Averaging: 0.000003, Post-loop: 0.000034, Total: 0.000160
2016-09-15 18:29:59,994 eos.effects.adaptivearmorhardener DEBUG    Initialization: 0.000038, Loop: 0.000027, Averaging: 0.000002, Post-loop: 0.000030, Total: 0.000096
2016-09-15 18:30:00,029 eos.effects.adaptivearmorhardener DEBUG    Initialization: 0.000037, Loop: 0.000027, Averaging: 0.000002, Post-loop: 0.000029, Total: 0.000095
2016-09-15 18:30:00,105 eos.effects.adaptivearmorhardener DEBUG    Initialization: 0.015852, Loop: 0.000039, Averaging: 0.000004, Post-loop: 0.000034, Total: 0.015929
2016-09-15 18:30:00,112 service.fit              DEBUG    ==========recalc==========
2016-09-15 18:30:00,144 eos.effects.adaptivearmorhardener DEBUG    Initialization: 0.001952, Loop: 0.000038, Averaging: 0.000004, Post-loop: 0.000027, Total: 0.002020
2016-09-15 18:30:00,721 service.fit              DEBUG    ==========recalc==========
2016-09-15 18:30:00,752 eos.effects.adaptivearmorhardener DEBUG    Initialization: 0.000136, Loop: 0.000090, Averaging: 0.000007, Post-loop: 0.000045, Total: 0.000278

2016-09-15 18:30:07,973 root                     INFO     Starting pyfa
2016-09-15 18:30:12,766 eos.effects.adaptivearmorhardener DEBUG    Initialization: 0.000042, Loop: 0.000028, Averaging: 0.000002, Post-loop: 0.000030, Total: 0.000102
2016-09-15 18:30:12,802 eos.effects.adaptivearmorhardener DEBUG    Initialization: 0.000038, Loop: 0.000027, Averaging: 0.000002, Post-loop: 0.000028, Total: 0.000094
2016-09-15 18:30:12,838 eos.effects.adaptivearmorhardener DEBUG    Initialization: 0.000058, Loop: 0.000038, Averaging: 0.000003, Post-loop: 0.000045, Total: 0.000144
2016-09-15 18:30:12,900 eos.effects.adaptivearmorhardener DEBUG    Initialization: 0.017492, Loop: 0.000039, Averaging: 0.000004, Post-loop: 0.000036, Total: 0.017571
2016-09-15 18:30:12,907 service.fit              DEBUG    ==========recalc==========
2016-09-15 18:30:12,944 eos.effects.adaptivearmorhardener DEBUG    Initialization: 0.003162, Loop: 0.000062, Averaging: 0.000006, Post-loop: 0.000051, Total: 0.003282
2016-09-15 18:30:13,448 service.fit              DEBUG    ==========recalc==========
2016-09-15 18:30:13,474 eos.effects.adaptivearmorhardener DEBUG    Initialization: 0.000089, Loop: 0.000040, Averaging: 0.000004, Post-loop: 0.000028, Total: 0.000161

2016-09-15 18:30:21,312 root                     INFO     Starting pyfa
2016-09-15 18:30:26,279 eos.effects.adaptivearmorhardener DEBUG    Initialization: 0.000043, Loop: 0.000027, Averaging: 0.000002, Post-loop: 0.000031, Total: 0.000102
2016-09-15 18:30:26,312 eos.effects.adaptivearmorhardener DEBUG    Initialization: 0.000050, Loop: 0.000057, Averaging: 0.000003, Post-loop: 0.000057, Total: 0.000167
2016-09-15 18:30:26,346 eos.effects.adaptivearmorhardener DEBUG    Initialization: 0.000038, Loop: 0.000027, Averaging: 0.000002, Post-loop: 0.000030, Total: 0.000097
2016-09-15 18:30:26,404 eos.effects.adaptivearmorhardener DEBUG    Initialization: 0.012980, Loop: 0.000038, Averaging: 0.000004, Post-loop: 0.000031, Total: 0.013053
2016-09-15 18:30:26,414 service.fit              DEBUG    ==========recalc==========
2016-09-15 18:30:26,444 eos.effects.adaptivearmorhardener DEBUG    Initialization: 0.002146, Loop: 0.000038, Averaging: 0.000004, Post-loop: 0.000029, Total: 0.002216
2016-09-15 18:30:26,997 service.fit              DEBUG    ==========recalc==========
2016-09-15 18:30:27,026 eos.effects.adaptivearmorhardener DEBUG    Initialization: 0.000113, Loop: 0.000049, Averaging: 0.000004, Post-loop: 0.000036, Total: 0.000202

V4:

2016-09-15 18:31:36,575 root                     INFO     Starting pyfa
2016-09-15 18:31:41,615 eos.effects.adaptivearmorhardener DEBUG    Initialization: 0.000067, Loop: 0.000040, Post-loop: 0.015862, Total: 0.015970
2016-09-15 18:31:41,654 eos.effects.adaptivearmorhardener DEBUG    Initialization: 0.000202, Loop: 0.000055, Post-loop: 0.002229, Total: 0.002487
2016-09-15 18:31:41,691 eos.effects.adaptivearmorhardener DEBUG    Initialization: 0.000065, Loop: 0.000040, Post-loop: 0.000093, Total: 0.000199
2016-09-15 18:31:41,733 eos.effects.adaptivearmorhardener DEBUG    Initialization: 0.000115, Loop: 0.000456, Post-loop: 0.000089, Total: 0.000660
2016-09-15 18:31:41,742 service.fit              DEBUG    ==========recalc==========
2016-09-15 18:31:41,776 eos.effects.adaptivearmorhardener DEBUG    Initialization: 0.000174, Loop: 0.000424, Post-loop: 0.000087, Total: 0.000686
2016-09-15 18:31:42,345 service.fit              DEBUG    ==========recalc==========
2016-09-15 18:31:42,375 eos.effects.adaptivearmorhardener DEBUG    Initialization: 0.000159, Loop: 0.000547, Post-loop: 0.000091, Total: 0.000797

2016-09-15 18:31:48,134 root                     INFO     Starting pyfa
2016-09-15 18:31:53,154 eos.effects.adaptivearmorhardener DEBUG    Initialization: 0.000066, Loop: 0.000040, Post-loop: 0.014494, Total: 0.014600
2016-09-15 18:31:53,193 eos.effects.adaptivearmorhardener DEBUG    Initialization: 0.000062, Loop: 0.000039, Post-loop: 0.001721, Total: 0.001822
2016-09-15 18:31:53,234 eos.effects.adaptivearmorhardener DEBUG    Initialization: 0.000096, Loop: 0.000059, Post-loop: 0.000109, Total: 0.000264
2016-09-15 18:31:53,283 eos.effects.adaptivearmorhardener DEBUG    Initialization: 0.000202, Loop: 0.000774, Post-loop: 0.000122, Total: 0.001098
2016-09-15 18:31:53,290 service.fit              DEBUG    ==========recalc==========
2016-09-15 18:31:53,319 eos.effects.adaptivearmorhardener DEBUG    Initialization: 0.000195, Loop: 0.000426, Post-loop: 0.000087, Total: 0.000708
2016-09-15 18:31:53,842 service.fit              DEBUG    ==========recalc==========
2016-09-15 18:31:53,872 eos.effects.adaptivearmorhardener DEBUG    Initialization: 0.000114, Loop: 0.000685, Post-loop: 0.000096, Total: 0.000895

2016-09-15 18:31:58,559 root                     INFO     Starting pyfa
2016-09-15 18:32:03,558 eos.effects.adaptivearmorhardener DEBUG    Initialization: 0.000080, Loop: 0.000039, Post-loop: 0.014809, Total: 0.014929
2016-09-15 18:32:03,595 eos.effects.adaptivearmorhardener DEBUG    Initialization: 0.000066, Loop: 0.000041, Post-loop: 0.002852, Total: 0.002958
2016-09-15 18:32:03,628 eos.effects.adaptivearmorhardener DEBUG    Initialization: 0.000062, Loop: 0.000039, Post-loop: 0.000091, Total: 0.000192
2016-09-15 18:32:03,670 eos.effects.adaptivearmorhardener DEBUG    Initialization: 0.000168, Loop: 0.000466, Post-loop: 0.000172, Total: 0.000806
2016-09-15 18:32:03,680 service.fit              DEBUG    ==========recalc==========
2016-09-15 18:32:03,710 eos.effects.adaptivearmorhardener DEBUG    Initialization: 0.000113, Loop: 0.000487, Post-loop: 0.000087, Total: 0.000688
2016-09-15 18:32:04,213 service.fit              DEBUG    ==========recalc==========
2016-09-15 18:32:04,243 eos.effects.adaptivearmorhardener DEBUG    Initialization: 0.000150, Loop: 0.000428, Post-loop: 0.000087, Total: 0.000665

It seems like there's something going on in the background that severely slows down access to ship/module attributes for a moment. It seems to add about 15ms to the post-loop part of the first booster for V4 or the initialization part of the main fit for V3, and about 2ms to the same parts of the next calculation for both.

Besides that spike, it seems like V3 and V4 have fairly similar performance. V3 seems to run a bit faster on my machine though, especially for situations where the RAH loops.

Since you seem to have much worse performance when you test things, perhaps you could do something similar? Just import time and add calls to time.clock() before and after the pre-loop, loop, and post-loop parts, and log the differences. That should help find what's taking so long for you.

@Ebag333
Copy link
Contributor Author

Ebag333 commented Sep 16, 2016

The test was when first loading pyfa with a fit that has three boosters with different RAH setups

I know that boosters were a serious issue when we first cracked this, but they're going to be gone here in a month or so....so.....screw 'em. :)

The booster code significantly slows down the execution of Pyfa (just because we're doing so many recalcs), so I would say just go ahead and do testing without it.

Besides that spike, it seems like V3 and V4 have fairly similar performance. V3 seems to run a bit faster on my machine though, especially for situations where the RAH loops.

It gets complicated because of the surrounding code, but as mentioned in the #688 thread, while the total execution time is similar v4 runs through many more loops in similar execution time as v3.

v4 looped 18 times in 26 ms or 1.4 loops per ms.
v3 looped 6 times in 15 ms, or .4 loops per ms.

This is just because of the extra complexity and additional loops that v3 has, if you removed those I'm sure that would largely go away.

Since you seem to have much worse performance when you test things, perhaps you could do something similar? Just import time and add calls to time.clock() before and after the pre-loop, loop, and post-loop parts, and log the differences. That should help find what's taking so long for you.

I've actually started a new branch to go through the code and look for pain points where code execution slows down. I already found 5 lines of code that (by itself!) adds 4 seconds to the time Pyfa takes to launch.

Some psuedo-code to hopefully clear up some confusion. with how this works (Also, because psuedo code is fun!).

Create Tuple

While Loop
   Update Tuple
   Sort Tuple

   If Uniform Damage Profile
      #Do nothing to allow the RAH to go back to defaults

   If Single Damage Profile
      Set single damage pattern on RAH

   If Multiple Damage Profile
      If No Resists to Vamp
            Break out of loop
      If Looped Too Many Times
            Break out of loop
      Else
            Vamp resists from 2 lowest damage resists
            Apply resists to 2 highest resists

Apply the finalized RAH resists to the ship

@MrNukealizer
Copy link
Contributor

MrNukealizer commented Sep 16, 2016

It gets complicated because of the surrounding code, but as mentioned in the #688 thread, while the total execution time is similar v4 runs through many more loops in similar execution time as v3.

v4 looped 18 times in 26 ms or 1.4 loops per ms.
v3 looped 6 times in 15 ms, or .4 loops per ms.

18/26 is 0.69, not 1.4. That's also another example of how you keep getting astoundingly slow execution; my tests show v4 doing 100 loops in 0.7 ms.

Each loop in v3 takes longer the more loops have been done before it, so it's hard to compare loops per ms. It generally does take longer per loop, but it generally doesn't need to do as many loops as v4, resulting in similar or less total time. If the RAH reaches a profile where it stops changing, v3 will do the same number of loops as v4 (except when v4 reduces the shift amount prematurely and stops short) but a little slower. That generally takes only 4-6 loops, and the v3 code won't be much slower. If the RAH keeps cycling in a loop, v3 will do the absolute minimum number of loops (I've never seen more than 11, though theoretically up to 14), whereas v4 can keep going for 20-100 loops.

I don't deny that looping through past results each loop is inefficient, but it's the most efficient way I know of to determine when the RAH enters a loop, and the worst-case performance should still be better than the current v4 code's worst-case performance. It also stays accurate in the process.

The pseudocode is a good idea, so here's how my method works:

Create Resistance Tuple

While Loop
   If Looped Too Many Times
      Break out of loop

   Create Temporary Tuple
   Sort Temporary Tuple

   If Single Damage Profile
      Set single damage pattern on RAH

   If Dual Damage Profile
      Set dual damage pattern on RAH

   If 3+ Damage Profile
      Vamp resists from 2 lowest damage resists
      Apply resists to 2 highest resists

   Update Resistance Tuple From Temporary Tuple
   If Resistance Tuple is in Cycle History
      Mark the section of cycle history that came followed this profile
      Break out of loop
   Else
      Add resistance tuple to cycle history

Set finalized RAH resists to average of the marked section of cycle history
Apply the finalized RAH resists to the ship

It should work the same except for storing past results and checking of the current cycle matches any of them, then averaging the results since the cycle that matched.

@Ebag333
Copy link
Contributor Author

Ebag333 commented Sep 16, 2016

18/26 is 0.69, not 1.4.

And this is why I let computers do math for me...

The problem is that with multiple tables, multiple loops, it gets progressively worse. With v4, it's a single loop (the subloops aren't actually looping multiple times, it's just for flow control).

Anyway, we're going around in circles now. Overall the v4 code is quite a bit faster, and we could speed it up even further by reducing the number of loops allowed. I don't see any easy ways to speed up the v3 code.

@blitzmann you want to chime in here?

@MrNukealizer
Copy link
Contributor

MrNukealizer commented Sep 16, 2016

Overall the v4 code is quite a bit faster

Do you have any benchmarks that show that? In my tests the v3 RAH effect as a whole consistently runs in the same or less time compared to the v4 effect, as reflected in the timings a few comments above and the very example you chose to demonstrate loop speed. Sure, the loops themselves take longer, but less loops and more efficient pre- and post-loop code always seems to make up for that.

I agree that v4 could be made faster, but as far as I know v3 is as fast as possible while being accurate.

@blitzmann
Copy link
Collaborator

@MrNukealizer @Ebag333 Can you guys post a few very specific test fittings / scenarios so that I can start testing this PR and v3? Also, someone write a bulleted summary of the current issues that may or may not need to be addressed that cannot be agreed upon. Thanks!

@Ebag333
Copy link
Contributor Author

Ebag333 commented Sep 16, 2016

To summarize the current discussion.

In game, the resist profile will switch between 2+ profiles depending on incoming damage. There's 3 different ways of picking the resist profile:

  1. Averaging the last X number of profiles.
  2. Selecting one of the profiles from the last X number of profiles.
  3. Stabilizing on profile very close to the average, by reducing the transfer amount each loop.

The theory is that solutions 1 and 2 will require caching and loops within loops, while solution 3 can be run as part of the normal loop.

There's some other debate about how to handle uniform damage profiles, but since we should keep the existing/historic/EFT behavior (and if you REALLY want to see how it reacts to a uniform damage profile, you can create one that's 9999 | 1000 | 1000 | 1000 as a work around), I don't see that changing.

@MrNukealizer
Copy link
Contributor

MrNukealizer commented Sep 17, 2016

I'll post some tests later. Here are the current issues as I see them:

  1. When the RAH enters an infinite loop v4 attempts to converge on the average value by reducing the change each cycle, while v3 gets the average of the loop values by storing past results and checking them each loop. I find v4 to be inaccurate and @Ebag333 finds v3 to be too slow.
  2. There is the issue of uniform damage. I don't understand why someone would want to see the default 15/15/15/15 instead of how the RAH would handle omni damage.
  3. I think @Ebag333 and I agree on this one. Neither version currently models how the RAH reacts to stacking penalties with a damage control. I think the consensus at the moment is that it's too much trouble to implement for the difference it makes.

@MrNukealizer
Copy link
Contributor

MrNukealizer commented Sep 17, 2016

Ok, here are a few tests. v3 and v4 seem to give identical results for any damage profile with 1-2 types and the performance shouldn't vary much across fits, so I haven't included any such situations here. Just set the damage profile on any random fit to one or two types.

[Gnosis, 99/40/40/0]

Reactive Armor Hardener

Damage profile 99/40/40/0 (purely theoretical). Let's start out strong with the worst case scenario for both algorithms. For this test v3 hits the cycle limit (it would need roughly 65 cycles depending on floating point accuracy) and gives an inaccurate result. v4 gives the most inaccurate result I've seen, actually changing the ship's armor EHP by 6.9% compared to v3.
This test would never apply to reality, but it does a good job of comparing the weaknesses of different algorithms.

Other damage profiles cause similar situations but aren't quite bad enough for v3 to hit the cycle limit, such as 90/40/40/0, 80/30/30/0, and 50/20/20/0.

[Megathron Navy Issue, 0/3/2/3]

Reactive Armor Hardener
[Empty Low slot]
Armor Thermal Hardener II
Armor Thermal Hardener II
Armor Kinetic Hardener II
Armor Explosive Hardener II
Prototype Armor Explosive Hardener I
Prototype Armor Explosive Hardener I

Large Anti-Thermal Pump II
Large Anti-EM Pump I
Large Anti-EM Pump I

Damage profile 0/3/2/3 (Depleted Uranium). This one is interesting because very tiny changes can cause different results. The RAH cycles in game are 9/9/21/21, 3/3/27/27, 0/0/30/30, then a loop of 0/3/33/24, 0/0/34.5/25.5, 0/3/37.5/19.5, 0/0/39/21.

[Paladin, 1/2/2/1]

Reactive Armor Hardener
Core X-Type Armor Kinetic Hardener
Core X-Type Armor Thermal Hardener

Damage profile 1/2/2/1 (lowsec sentry guns). This is an example of the RAH entering a loop without ever reducing a resistance to 0. It goes to 21/9/9/21 then bounces between 15/3/15/27 and 9/9/9/33.

[Gnosis, 3/2/2/0]

Reactive Armor Hardener

Damage profile 3/2/2/0 (done with smartbombs). This test partially shows how the algorithm sorts damage types that are equal. Similar tests can be done with other configurations. In game if Thermal and Kinetic are equal, it will always prefer to take resistances from Kinetic and give them to Thermal. The cycles go 21/21/9/9, 27/15/15/3, 31.5/19.5/9/0, then loop through 34.5/13.5/12/0, 37.5/7.5/15/0, and 40.5/10.5/9/0. Interestingly for the loop the average Kinetic resistance ends up higher than the average Thermal resistance despite the tendency to prefer Thermal when the two are equal.
For practical purposes it makes no difference if it prefers Kinetic or Thermal because they're equal, but one way matches the ingame performance and the other doesn't.

@Ebag333
Copy link
Contributor Author

Ebag333 commented Sep 17, 2016

Damage profile 99/40/40/0 (purely theoretical). Let's start out strong with the worst case scenario for both algorithms. For this test v3 hits the cycle limit (it would need roughly 65 cycles depending on floating point accuracy) and gives an inaccurate result. v4 gives the most inaccurate result I've seen, actually changing the ship's armor EHP by 6.9% compared to v3.

This is a weird one, where EM gets boosted almost every cycle, but then occasionally the damage changes enough to throw a little bit of resists back at therm/kin for one cycle.

Basically no approach we take will be accurate unless we let it cycle for a long time. It took 14 cycles before it got reasonably close to what it roughly stabilizes at (about 4/5ths of the time, then there's that one wild card of 6% where it steals from EM).

This is also a good example of what I was talking about earlier, where because of rounding and uneven numbers the resists get messed up. After 100 loops the resists show as 59.4% EM and .572% Therm, which adds up to 59.972%. The longer we let this run, the more resists we drop and it starts to get further away from being accurate.

After 100 cycles (of doing what it would in game, always transferring 6%), the RAH ended up at:
0.405716 | 0.994284 | 1.000000 | 1.000000 with a total armor EHP of 11055

v4 ended up at:
0.590000 | 0.905000 | 0.905000 | 1.000000 with a total armor EHP of 10136

Which is relatively close to being an accurate representation, given how devilish this particular one is and we don't let it cycle for very long.

So with this particular scenario in mind, the different approaches end up as such.

  • Averaging the resists likely ends up with really weird numbers (this depends on how long we let it run). It's also likely to be relatively inaccurate, as 5 out of 6 will likely be small changes, with 1-2 big jumps when EM takes a hit (just depends on how many you average).
  • Letting it run a long time then cutting it off will end up with resists missing (and weird numbers)
  • Caching the last few resists and looking for a repeat will never happen, as the profiles are not cycling.
  • Reducing the resist transfer amount risks not transferring enough before it gets cut off.

This becomes a really fine line, because after just 11 cycles (at 6%) the resists went to hell and we started getting oddball numbers. Just 2 cycles later, we started dropping resists.

I shifted the number of times v4 is allowed to run, it's really dangerous at the full 6% because just a couple cycles one way or the other and the numbers get mad. By reducing the resist transfer down to 1%, we can transfer resists much more safely and be less likely to get bad numbers (though it's still possible if we go long enough).

I added a check to make sure we don't cycle less than 7 times before reducing resists, and bumped the number of times we transfer resists after the reduction up to 10 (from 5). This slightly increases our chances of hitting weird resist numbers, but should better handle particular scenarios like this one.

So, going back to the original behavior:

After 100 cycles (of doing what it would in game, always transferring 6%), the RAH ended up at:
0.405716 | 0.994284 | 1.000000 | 1.000000 with a total armor EHP of 11055

v4 (with modifications) ended up at:
0.475000 | 0.955000 | 0.970000 | 1.000000 with a total armor EHP of 10690

Is it perfectly accurate? No. Is it fairly reasonably close to being accurate? I think so.

I've submitted the changes to the RAH cycling. It'll take slightly longer to run now, but the biggest change is that it'll always run a minimum of 7 times (unless we run out of resists to steal).

@MrNukealizer
Copy link
Contributor

MrNukealizer commented Sep 17, 2016

Now that I think of it, I probably should've mentioned more about the ideal results from that 99/40/40/0 test. As the RAH cycles it converges on 57.656/1.172/1.172/0.

On the subject of weird numbers, one of the other examples I gave (I think 5/2/2/0) shows as something like "60%/0%/5.273e-08%/0%" in v3. Those other similar situations are interesting because they converge on the end point fast enough that before hitting the limit there are two cycles within v3's tolerance for equality, so it detects a loop and gives a fairly good result. I'm going to play around with the cycle limit and equality tolerance to see if it can handle weird situations like these.

@Ebag333
Copy link
Contributor Author

Ebag333 commented Sep 18, 2016

As the RAH cycles it converges on 57.656/1.172/1.172/0

That adds up to 61.096%.

Double check my math though. :)

@Ebag333
Copy link
Contributor Author

Ebag333 commented Sep 18, 2016

Okay, so I went through these and basically simulated what the RAH does if we let it run long enough (setting performance aside).

For each of these, I let it run 100 times, but basically truncate the log when it stops changing.

[Gnosis, 99/40/40/0]

Reactive Armor Hardener
Adaptive Resists, Ship Resists, Modified Damage (EM|The|Kin|Exp) : 0.790000 | 0.790000 | 0.910000 | 0.910000 || 0.533250 | 0.533250 | 0.614250 | 0.614250 || 66.825000 | 27.000000 | 27.000000 | 0.000000
Adaptive Resists, Ship Resists, Modified Damage (EM|The|Kin|Exp) : 0.730000 | 0.850000 | 0.850000 | 0.970000 || 0.492750 | 0.573750 | 0.573750 | 0.654750 || 52.791750 | 21.330000 | 24.570000 | 0.000000
Adaptive Resists, Ship Resists, Modified Damage (EM|The|Kin|Exp) : 0.685000 | 0.805000 | 0.910000 | 1.000000 || 0.462375 | 0.543375 | 0.614250 | 0.675000 || 48.782250 | 22.950000 | 22.950000 | 0.000000
Adaptive Resists, Ship Resists, Modified Damage (EM|The|Kin|Exp) : 0.655000 | 0.865000 | 0.880000 | 1.000000 || 0.442125 | 0.583875 | 0.594000 | 0.675000 || 45.775125 | 21.735000 | 24.570000 | 0.000000
Adaptive Resists, Ship Resists, Modified Damage (EM|The|Kin|Exp) : 0.625000 | 0.925000 | 0.850000 | 1.000000 || 0.421875 | 0.624375 | 0.573750 | 0.675000 || 43.770375 | 23.355000 | 23.760000 | 0.000000
Adaptive Resists, Ship Resists, Modified Damage (EM|The|Kin|Exp) : 0.595000 | 0.895000 | 0.910000 | 1.000000 || 0.401625 | 0.604125 | 0.614250 | 0.675000 || 41.765625 | 24.975000 | 22.950000 | 0.000000
Adaptive Resists, Ship Resists, Modified Damage (EM|The|Kin|Exp) : 0.565000 | 0.955000 | 0.880000 | 1.000000 || 0.381375 | 0.644625 | 0.594000 | 0.675000 || 39.760875 | 24.165000 | 24.570000 | 0.000000
Adaptive Resists, Ship Resists, Modified Damage (EM|The|Kin|Exp) : 0.535000 | 0.925000 | 0.940000 | 1.000000 || 0.361125 | 0.624375 | 0.634500 | 0.675000 || 37.756125 | 25.785000 | 23.760000 | 0.000000
Adaptive Resists, Ship Resists, Modified Damage (EM|The|Kin|Exp) : 0.505000 | 0.985000 | 0.910000 | 1.000000 || 0.340875 | 0.664875 | 0.614250 | 0.675000 || 35.751375 | 24.975000 | 25.380000 | 0.000000
Adaptive Resists, Ship Resists, Modified Damage (EM|The|Kin|Exp) : 0.475000 | 0.955000 | 0.970000 | 1.000000 || 0.320625 | 0.644625 | 0.654750 | 0.675000 || 33.746625 | 26.595000 | 24.570000 | 0.000000
Adaptive Resists, Ship Resists, Modified Damage (EM|The|Kin|Exp) : 0.452500 | 1.000000 | 0.947500 | 1.000000 || 0.305437 | 0.675000 | 0.639563 | 0.675000 || 31.741875 | 25.785000 | 26.190000 | 0.000000
Adaptive Resists, Ship Resists, Modified Damage (EM|The|Kin|Exp) : 0.426250 | 0.973750 | 1.000000 | 1.000000 || 0.287719 | 0.657281 | 0.675000 | 0.675000 || 30.238312 | 27.000000 | 25.582500 | 0.000000
Adaptive Resists, Ship Resists, Modified Damage (EM|The|Kin|Exp) : 0.413125 | 1.000000 | 0.986875 | 1.000000 || 0.278859 | 0.675000 | 0.666141 | 0.675000 || 28.484156 | 26.291250 | 27.000000 | 0.000000
Adaptive Resists, Ship Resists, Modified Damage (EM|The|Kin|Exp) : 0.406562 | 0.993437 | 1.000000 | 1.000000 || 0.274430 | 0.670570 | 0.675000 | 0.675000 || 27.607078 | 27.000000 | 26.645625 | 0.000000
Adaptive Resists, Ship Resists, Modified Damage (EM|The|Kin|Exp) : 0.403281 | 1.000000 | 0.996719 | 1.000000 || 0.272215 | 0.675000 | 0.672785 | 0.675000 || 27.168539 | 26.822812 | 27.000000 | 0.000000

So we talked about this one before, but basically after loop 11 it starts going off the rails. If we let it run long enough, it eventually gets to essentially 60 | 0 | 0 | 0 (though that would take a VERY long time, and might not ever get there properly because of rounding errors).

So, the various approaches we've discussed.

  1. Caching a picking a profile doesn't really work, because it never repeats.
  2. Averaging is problematic. If we loop long enough, then when you average it's basically an average of the 4 (or however many you do) of the same value (so there's no value). If you don't loop long enough, then you're just basically reverting back a few lines.
  3. If you loop long enough you'll get an essentially accurate value, but you can end up with difficult to read values (and rounding errors).
  4. Reducing the amount transferred ends up with similar numbers as option 3 (depneding on how long you let it run), it's easier to not go too far and run into weird values (because you're transferring less), but if you go to far you'll have the same problems as 3.
[Megathron Navy Issue, 0/3/2/3]

Reactive Armor Hardener
[Empty Low slot]
Armor Thermal Hardener II
Armor Thermal Hardener II
Armor Kinetic Hardener II
Armor Explosive Hardener II
Prototype Armor Explosive Hardener I
Prototype Armor Explosive Hardener I

Large Anti-Thermal Pump II
Large Anti-EM Pump I
Large Anti-EM Pump I
Adaptive Resists, Ship Resists, Modified Damage (EM|The|Kin|Exp) : 0.910000 | 0.910000 | 0.790000 | 0.790000 || 0.235456 | 0.111192 | 0.231075 | 0.129300 || 0.000000 | 0.366568 | 0.585000 | 0.491012
Adaptive Resists, Ship Resists, Modified Damage (EM|The|Kin|Exp) : 0.970000 | 0.970000 | 0.730000 | 0.730000 || 0.250980 | 0.118524 | 0.213525 | 0.119479 || 0.000000 | 0.333577 | 0.462150 | 0.387899
Adaptive Resists, Ship Resists, Modified Damage (EM|The|Kin|Exp) : 1.000000 | 1.000000 | 0.700000 | 0.700000 || 0.258742 | 0.122189 | 0.204750 | 0.114569 || 0.000000 | 0.355571 | 0.427050 | 0.358438
Adaptive Resists, Ship Resists, Modified Damage (EM|The|Kin|Exp) : 1.000000 | 0.970000 | 0.670000 | 0.760000 || 0.258742 | 0.118524 | 0.195975 | 0.124390 || 0.000000 | 0.366568 | 0.409500 | 0.343708
Adaptive Resists, Ship Resists, Modified Damage (EM|The|Kin|Exp) : 1.000000 | 1.000000 | 0.655000 | 0.745000 || 0.258742 | 0.122189 | 0.191587 | 0.121935 || 0.000000 | 0.355571 | 0.391950 | 0.373169
Adaptive Resists, Ship Resists, Modified Damage (EM|The|Kin|Exp) : 1.000000 | 0.970000 | 0.625000 | 0.805000 || 0.258742 | 0.118524 | 0.182812 | 0.131755 || 0.000000 | 0.366568 | 0.383175 | 0.365804
Adaptive Resists, Ship Resists, Modified Damage (EM|The|Kin|Exp) : 1.000000 | 1.000000 | 0.610000 | 0.790000 || 0.258742 | 0.122189 | 0.178425 | 0.129300 || 0.000000 | 0.355571 | 0.365625 | 0.395264

This one basically repeats the last 5 lines repeatedly.

So, the various approaches we've discussed.

  1. Caching a picking a profile works because the profiles repeat (and very cleanly), but it's going to be completely arbitrary.
  2. Averaging works...as long as you pick exactly 5 (and let it run long enough but it only takes a few cycles so that's not an issue).
  3. If you loop long enough you'll just be stuck in a loop, and where ever you stop will be as arbitry as 1.
  4. Reducing the amount transferred ends up with similar numbers as averaging, and you don't run into the issue of selecting too few or too many to average. The trick is to not loop too few times.
[Paladin, 1/2/2/1]

Reactive Armor Hardener
Core X-Type Armor Kinetic Hardener
Core X-Type Armor Thermal Hardener
Adaptive Resists, Ship Resists, Modified Damage (EM|The|Kin|Exp) : 0.790000 | 0.910000 | 0.910000 | 0.790000 || 0.395000 | 0.212940 | 0.214987 | 0.474000 || 0.500000 | 0.468000 | 0.472500 | 0.600000
Adaptive Resists, Ship Resists, Modified Damage (EM|The|Kin|Exp) : 0.850000 | 0.970000 | 0.850000 | 0.730000 || 0.425000 | 0.226980 | 0.200812 | 0.438000 || 0.395000 | 0.425880 | 0.429975 | 0.474000
Adaptive Resists, Ship Resists, Modified Damage (EM|The|Kin|Exp) : 0.910000 | 0.910000 | 0.910000 | 0.670000 || 0.455000 | 0.212940 | 0.214987 | 0.402000 || 0.425000 | 0.453960 | 0.401625 | 0.438000

This one loops the last two.

  1. Same as above.
  2. Essentially the same as above, except you need to average your count in multiples of two (as opposed to multiples of 5 for the last example).
  3. Same as above.
  4. Same as above.
[Gnosis, 3/2/2/0]

Reactive Armor Hardener
Adaptive Resists, Ship Resists, Modified Damage (EM|The|Kin|Exp) : 0.790000 | 0.790000 | 0.910000 | 0.910000 || 0.533250 | 0.533250 | 0.614250 | 0.614250 || 2.025000 | 1.350000 | 1.350000 | 0.000000
Adaptive Resists, Ship Resists, Modified Damage (EM|The|Kin|Exp) : 0.730000 | 0.850000 | 0.850000 | 0.970000 || 0.492750 | 0.573750 | 0.573750 | 0.654750 || 1.599750 | 1.066500 | 1.228500 | 0.000000
Adaptive Resists, Ship Resists, Modified Damage (EM|The|Kin|Exp) : 0.685000 | 0.805000 | 0.910000 | 1.000000 || 0.462375 | 0.543375 | 0.614250 | 0.675000 || 1.478250 | 1.147500 | 1.147500 | 0.000000
Adaptive Resists, Ship Resists, Modified Damage (EM|The|Kin|Exp) : 0.655000 | 0.865000 | 0.880000 | 1.000000 || 0.442125 | 0.583875 | 0.594000 | 0.675000 || 1.387125 | 1.086750 | 1.228500 | 0.000000
Adaptive Resists, Ship Resists, Modified Damage (EM|The|Kin|Exp) : 0.625000 | 0.925000 | 0.850000 | 1.000000 || 0.421875 | 0.624375 | 0.573750 | 0.675000 || 1.326375 | 1.167750 | 1.188000 | 0.000000
Adaptive Resists, Ship Resists, Modified Damage (EM|The|Kin|Exp) : 0.595000 | 0.895000 | 0.910000 | 1.000000 || 0.401625 | 0.604125 | 0.614250 | 0.675000 || 1.265625 | 1.248750 | 1.147500 | 0.000000

This one loops the last three.

  1. Same as above.
  2. Essentially the same as above, except you need to average your count in multiples of three.
  3. Same as above.
  4. Same as above.

So, to go over the four options we have...

  1. Caching and picking a previously used profile - Doesn't really work very well. It's expensive code wise, complex, and since some of these cycles have very large swings isn't any more likely to pick the correct profile. Plus resist configs that never loop break it.
  2. Averaging X number of previous results. We have to cache it, which is a bit expensive (though not as expensive as above). The bigger problem is that we can run into profiles that would require us to average by a specific number (2, 3, and 5 in the examples above).
  3. Basicaly all the problems of 1, but at least isn't as expensive.
  4. If we can figure out a way to make it run long enough (without running to long until we get weird values), then this would give values very similar to Added the character import enhancement (reads native EVE CCP XML), skill exporter, cleanups #2, without needing to worry about the number of resists to average.

I have an idea that might address this, going to go see if I can play around with it and make it work...

@MrNukealizer
Copy link
Contributor

As the RAH cycles it converges on 57.656/1.172/1.172/0

That adds up to 61.096%.

Double check my math though. :)

According to my calculator that adds up to 60%. I'm not sure where you got the extra 1.096% from...

@Ebag333
Copy link
Contributor Author

Ebag333 commented Sep 18, 2016

Added a new check to bail if our bottom two resists get too small (less than the transfer amount). Now for that first example that gets so nasty we end up with:

Adaptive Resists, Ship Resists, Modified Damage (EM|The|Kin|Exp) : 0.410000 | 0.995000 | 0.995000 | 1.000000 || 0.276750 | 0.671625 | 0.671625 | 0.675000 || 27.398250 | 26.865000 | 26.865000 | 0.000000

Which is basically the perfect result!

For the Megathron Navy (example 2), averaging the cycle of 5 we get:

1 | 0.988 | 0.652 | 0.76

And with v4 we get:

1.000000 | 1.000000 | 0.640000 | 0.760000

Our error is just over 1%.

For [Paladin, 1/2/2/1] (example 3), average of the cycle of 2 we get:

0.88 | 0.94 | 0.88 | 0.7

And with v4 we get:

0.870000 | 0.910000 | 0.910000 | 0.710000

Our error is 3% on this one.

Finally for [Gnosis, 3/2/2/0] (example 4), average of the cycle of 3 we get:

0.625 | 0.895 | 0.88 | 1

And with v4 we get:

 0.605000 | 0.890000 | 0.905000 | 1.000000

2% error on this one.

I'm rather happy with the results of the latest tweak to v4. I'm still a bit worried about the performance overall, but I think it's at a reasonable point where it is now. We can also investigate lowering the maximum number of cycles (currently set to 100, which is rather long). 50 is probably a more reasonable number.

@Ebag333
Copy link
Contributor Author

Ebag333 commented Sep 18, 2016

According to my calculator that adds up to 60%. I'm not sure where you got the extra 1.096% from...

Doh. What happens when you're doing it on your mobile and put the decimal in the wrong spot....

@MrNukealizer
Copy link
Contributor

MrNukealizer commented Sep 18, 2016

I made a couple tweaks to v3 to improve accuracy when hitting the cycle limit as well as rounding the output to 3 decimals one decimal. That seems to have made a huge difference in the result of the 99/40/40/0 test, and eliminated that weird 7.82e-08% kinetic resistance in the 8/3/3/0 test.

After those changes I combined v3 and v4 so they run together and report their results and run times. Here's what that showed for the test cases I listed above as well as a few real fits I had open.

Damage pattern: 99/40/40/0, Armor resistances: 32.500000%/32.500000%/32.500000%/32.500000%
v3: 57.500000%/1.300000%/1.200000%/0.000000%, Cycles: 50; Start: 0.000040s, Loop: 0.000319s, End: 0.000022s, Total: 0.000381s
v4: 59.000000%/0.500000%/0.500000%/0.000000%, Cycles: 34; Start: 0.000052s, Loop: 0.002026s, End: 0.000077s, Total: 0.002155s

Damage pattern: 9/4/4/0, Armor resistances: 32.500000%/32.500000%/32.500000%/32.500000%
v3: 54.000000%/3.000000%/3.000000%/0.000000%, Cycles: 33; Start: 0.000039s, Loop: 0.000231s, End: 0.000040s, Total: 0.000310s
v4: 56.000000%/2.000000%/2.000000%/0.000000%, Cycles: 100; Start: 0.000052s, Loop: 0.005854s, End: 0.000118s, Total: 0.006024s

Damage pattern: 8/3/3/0, Armor resistances: 32.500000%/32.500000%/32.500000%/32.500000%
v3: 60.000000%/0.000000%/0.000000%/0.000000%, Cycles: 28; Start: 0.000040s, Loop: 0.000148s, End: 0.000021s, Total: 0.000209s
v4: 59.000000%/0.500000%/0.500000%/0.000000%, Cycles: 34; Start: 0.000052s, Loop: 0.002017s, End: 0.000075s, Total: 0.002144s

Damage pattern: 3/2/2/0, Armor resistances: 32.500000%/32.500000%/32.500000%/32.500000%
v3: 37.500000%/10.500000%/12.000000%/0.000000%, Cycles: 7; Start: 0.000053s, Loop: 0.000041s, End: 0.000021s, Total: 0.000115s
v4: 39.500000%/11.000000%/9.500000%/0.000000%, Cycles: 100; Start: 0.000063s, Loop: 0.005918s, End: 0.000076s, Total: 0.006057s

Damage pattern: 0/3/2/3, Armor resistances: 74.125760%/87.781056%/70.750000%/83.632948%
v3: 0.000000%/1.500000%/36.000000%/22.500000%, Cycles: 8; Start: 0.000033s, Loop: 0.000067s, End: 0.000022s, Total: 0.000121s
v4: 0.000000%/0.000000%/36.000000%/24.000000%, Cycles: 10; Start: 0.000043s, Loop: 0.000559s, End: 0.000091s, Total: 0.000694s

Damage pattern: 1/2/2/1, Armor resistances: 50.000000%/76.600000%/76.375000%/40.000000%
v3: 12.000000%/6.000000%/12.000000%/30.000000%, Cycles: 4; Start: 0.000036s, Loop: 0.000029s, End: 0.000020s, Total: 0.000085s
v4: 13.000000%/9.000000%/9.000000%/29.000000%, Cycles: 100; Start: 0.000052s, Loop: 0.006360s, End: 0.000083s, Total: 0.006494s


Damage pattern: 25/25/25/25, Armor resistances: 32.500000%/32.500000%/32.500000%/32.500000%
v3: 12.000000%/18.000000%/18.000000%/12.000000%, Cycles: 3; Start: 0.000052s, Loop: 0.000048s, End: 0.000094s, Total: 0.000194s
v4: 15.000000%/15.000000%/15.000000%/15.000000%, Cycles: 1; Start: 0.000083s, Loop: 0.000061s, End: 0.000098s, Total: 0.000242s

Damage pattern: 0/3/2/3, Armor resistances: 32.500000%/32.500000%/32.500000%/32.500000%
v3: 0.000000%/30.000000%/0.000000%/30.000000%, Cycles: 4; Start: 0.000062s, Loop: 0.000029s, End: 0.000022s, Total: 0.000113s
v4: 0.000000%/30.000000%/0.000000%/30.000000%, Cycles: 4; Start: 0.000093s, Loop: 0.000279s, End: 0.000075s, Total: 0.000447s

Damage pattern: 1/2/2/1, Armor resistances: 32.500000%/32.500000%/32.500000%/32.500000%
v3: 0.000000%/30.000000%/30.000000%/0.000000%, Cycles: 4; Start: 0.000040s, Loop: 0.000029s, End: 0.000021s, Total: 0.000090s
v4: 0.000000%/30.000000%/30.000000%/0.000000%, Cycles: 4; Start: 0.000052s, Loop: 0.000252s, End: 0.000074s, Total: 0.000379s

Damage pattern: 0/0/0/100, Armor resistances: 77.652142%/83.705440%/83.239106%/91.060857%
v3: 0.000000%/0.000000%/0.000000%/60.000000%, Cycles: 2; Start: 0.000030s, Loop: 0.000020s, End: 0.000021s, Total: 0.000071s
v4: 0.000000%/0.000000%/0.000000%/60.000000%, Cycles: 1; Start: 0.000042s, Loop: 0.000072s, End: 0.000087s, Total: 0.000200s

Damage pattern: 1/2/2/1, Armor resistances: 77.652142%/83.705440%/83.239106%/91.060857%
v3: 0.000000%/30.000000%/30.000000%/0.000000%, Cycles: 4; Start: 0.000029s, Loop: 0.000026s, End: 0.000020s, Total: 0.000074s
v4: 0.000000%/30.000000%/30.000000%/0.000000%, Cycles: 4; Start: 0.000041s, Loop: 0.000214s, End: 0.000085s, Total: 0.000339s

Damage pattern: 21/23/11/45, Armor resistances: 64.662500%/54.061250%/73.496875%/85.865000%
v3: 17.200000%/39.800000%/0.000000%/3.000000%, Cycles: 9; Start: 0.000051s, Loop: 0.000050s, End: 0.000021s, Total: 0.000122s
v4: 16.250000%/41.750000%/0.000000%/2.000000%, Cycles: 100; Start: 0.000082s, Loop: 0.005416s, End: 0.000073s, Total: 0.005572s

Damage pattern: 14/8/0/0, Armor resistances: 72.065177%/72.646013%/79.048883%/88.826071%
v3: 30.000000%/30.000000%/0.000000%/0.000000%, Cycles: 2; Start: 0.000031s, Loop: 0.000018s, End: 0.000021s, Total: 0.000071s
v4: 30.000000%/30.000000%/0.000000%/0.000000%, Cycles: 4; Start: 0.000043s, Loop: 0.000223s, End: 0.000077s, Total: 0.000344s

I'm not sure why, but it seems like the loop in v4 is much slower than it used to be. I might have messed up the timing part, but I don't think so. Maybe applying changes to the module stats and reloading them is just slower than my previous benchmarks indicated.

Also, what happened to checking for a 2-type damage profile and going to 30/30 on the first cycle?

@Ebag333
Copy link
Contributor Author

Ebag333 commented Sep 18, 2016

I'm not sure why, but it seems like the loop in v4 is much slower than it used to be. I might have messed up the timing part, but I don't think so. Maybe applying changes to the module stats and reloading them is just slower than my previous benchmarks indicated.

Running them together isn't likely going to work for showing the times properly. We MIGHT be able to do something with threading, but since Pythons scheduler isn't atomic we couldn't get accurate results anyway for sub 1 second times.

I'm planning on setting up some profiling and running numbers (basically forcing both to loop many times), I might be able to do that today.

V4 IS slower as it cycles to the maximum number of runs more often than it used to. This was so that we could properly calculate it when we run into weird scenarios like that Gnosis, 99/40/40/0 one. Since it takes a good number of loops to hit that one, I had to up the max number of runs, and simply set it to a very large number. As mentioned earlier, that number can (and should) be trimmed down.

Also, what happened to checking for a 2-type damage profile and going to 30/30 on the first cycle?

We can still do this, but I was more concerned with the complex 3/4 damage type profiles. Plus, it only takes 3 cycles to get there, so I'm not really sure that it's worth the extra complexity to save a handful of milliseconds. Once I do some profiling, I'll have a better idea of what that time savings might be.

@MrNukealizer
Copy link
Contributor

Running them together isn't likely going to work for showing the times properly.

It should work. It basically runs the pre-loop part of v3, pre-loop v4, v3's loop, v4's loop, post-loop v4, then post-loop v3, marking the start and end times of each section. I tried reordering the sections and running each version alone, and that made no difference to most of the results. The only exception is that the second time applying resists the ship (yes, it's unusable that way but it was for timing purposes) takes a tiny bit longer because of calculating stacking penalties.

@Ebag333
Copy link
Contributor Author

Ebag333 commented Sep 19, 2016

Well one problem is that v3 uses force. This is no bueno as I spent a week figuring out, we can ONLY use it when those values will never be touched again. Basically it's fairly dangerous to use.

It also means that anything run after v3 won't work correctly.

You can use module.clear() to reset the RAH. It takes off all modifiers, so we can't use it for real, but it can work for testing.

@MrNukealizer
Copy link
Contributor

Well one problem is that v3 uses force.

Good point. I copied that from some old version and didn't think much about it, but that does seem like a problem.

@Ebag333
Copy link
Contributor Author

Ebag333 commented Sep 19, 2016

Just a wee bit.

You can find all the module modifiers in eos.effectHandlerHelpers and if you want to get really deep down the rabbit hole, take a look at modifiedAttributeDict.py. There's a number of helper (so called magic classes) in there that you won't find referenced anywhere else. Many of them aren't directly called, so trying to figure that out was fun.

The initial goal of the rewrite was to update the ship in essentially real time (each loop) to see if we could get it to recalculate for stacking penalties and whatnot. It seems to, but the catch is that the stacking penalty ends up applying multiple times and it all goes to hell. I tried every method we have, none of them work.

For the RAH, you probably want to use increaseItemAttr to modify it (and then of course multiplyItemAttr to apply it to the ship).

That is one interesting side effect of v4, each loop shows up in the affected by tab.

Since two damage types will always result in a 50/50 split, this speeds
up processing a bit.
Unused stuff, formatting, PEP8 standards (most of them), etc
@Ebag333
Copy link
Contributor Author

Ebag333 commented Oct 18, 2016

Most of the changes are just formatting and layout.

I did add a condition for 2 damage type profiles, to skip looping through those. (Makes sense since most ammo is 2 damage types.)

@blitzmann blitzmann closed this Dec 1, 2016
@Ebag333
Copy link
Contributor Author

Ebag333 commented Dec 1, 2016

@blitzmann it's okay, I'm planning on rewritting it from the ground up in Gnosis anyway.

:D

@blitzmann
Copy link
Collaborator

@blitzmann it's okay, I'm planning on rewritting it from the ground up in Gnosis anyway.

FYI, this comes off as extremely petty. Circumventing the work of another contributor because your's did not make the cut (smh)

@Ebag333
Copy link
Contributor Author

Ebag333 commented Dec 1, 2016

Apologies to @MrNukealizer. His PR is good and fits Pyfa, and his work fully deserves to get merged. I did not intend it that way at all, but I should have seen that it would be read that way. Entirely my bad.

To they and explain what I mean by that bad joke.....It does need to be written for new Eos/Pyfa, and it has been my intent to rewrite it from scratch since Pyfa-NG became a thing. But that won't ever go into current Pyfa, so @MrNukealizer's work won't be circumvented.

@blitzmann
Copy link
Collaborator

I didn't think it was intended, just came off that way as you noted. As long as we're all on the same page.

pyfa-ng will indeed need it, and I think having it bundled with your simulation collection would be a good idea. :)

@Ebag333 Ebag333 deleted the RAH-v4 branch March 24, 2017 16:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants