[X] My Assistant

This is a thread for in-depth system’s discussion and is math heavy, much too dry for the regular Ask the Experts. This is a place to learn and discuss the mathematical workings of HV. I may in future type up how I created some spreadsheets for HV. Feel free to ask any related questions (point out glaring flaws) or even post any mathematical/hyper min-maxing thoughts and ideas.

Potential topics:
When to reforge your gear if you want a specific combination.
What is the mathematical perfect stat distribution (since we have all the details, there should be a correct answer).
How I made up the potency potential (including a model on how the 1H proc works), forge materials calculator, when to train which drop training, and see if you can spot any errors.

On drop data, how much is enough?
This relates to the post I made on a recalculation of the drop rates.

Frederiksc make a point that “maybe the numbers I got are just due to chance, because 0.1% (wiki) is similar to 0.2% (my new estimate)”. There is a statistical test which tests this. To do this we set up the Null and alternate hypothesis:

H0 (this is what is previously thought) – the base artifact drop rate is 0.1%
H1 (this is what we want to test) – the base artefact drop rate is more than 0.1%

There also has to be a few assumptions, in mathematics you need to justify them (I can’t just assume that I am correct, that’s cheating, and also known as circular reasoning).

Notation. Normal distribution refers to “Gaussian” distribution.

Assumption 1. The number of monsters in each grindfest is 8321. This number is gotten by estimating a few rounds (about 24), then extrapolating the whole set. There is ~5.125 monsters for the first 24 rounds (starting from ~2-~8), then there are ~8.4 monsters thereafter. With this model the variance is about 2.4/round = 2411 in 1000 rounds. This means we are 95% sure that the number of monsters is between (8224, 8417). If someone want to count more, this number could be more accurate.
Assumption 2. The drop rate of monsters is 27%. The number I calculated from data is identical to the wiki. Thus I think it is accurate to say it is 27% (note this implies the average PL bonus is ~14%).
Assumption 3. The base equipment drop rate is 2.5% (as found in the same post). This is different from wiki, but as you will see later, won’t change the value much.
Assumption 4. Quartermaster and Archaeologist work exactly as stated (ie. Up to double chance). This has specks of evidence from 10bro, and there isn’t significant evidence that it is wrong.

Theory

How do you test something like H0 vs H1. To do this let’s imagine a simple scenario: We flip a coin 100 times, it was heads 100 times. We have a suspicion that the coin is rigged, in this case:

H0 – the chance of heads is 50%
H1 – the chance of heads is ≠50%

It should be noted that there is a small chance that flipping a coin 100 times will get you 100 heads, but this doesn’t happen often (about 1 in 1.26x10^30 times, cue a story about everyone in the world flipping 100 coins every second). It should be really obvious that this is a rigged coin, so let’s downscale the numbers:
We flipped a coin 20 times, and got 16 heads... now it’s not so obvious.

What is done is we calculate the chance of getting such a result, or more extreme, ie. What are the chances of getting something like 16 or higher (ie. 16, 17 … 20 heads, or even the other side 4, 3 … 0 heads, since a low number would be suspect). It turns out that this chance is 1.18%. So if we had a confidence interval of 95%, we are 95% sure that the coin is rigged (in this case we are 98.8% sure the coin is rigged). Google binomial distribution if you want to know how the number is calculated.

Application

Back to the artifact drop rate. I recorded 77 artifact drops in 9 grindfest runs ie. 74889 monsters killed. With an average of 8.56 drops per run. I have archaeologist level 8.

We can calculate how many we expect to get under H0 (artifact rate is 0.1%): 74889*.27*.95*0.001*1.8 = 34.6. But the data found 70. (note on assumption 3. If we assume the wiki to be correct and the equipment base drop rate is 4%, not 2.5%, we would get 74889*.27*.92*0.001*1.8 = 33.5, a small difference). This is a binomial distribution so we can calculate the standard deviation (5.88).

Since there is a large amount of data, we can assume a normal distribution (in fact, for binomial, with a continuity correction, it becomes nearly exactly equal to the normal distribution).

When you test a sample vs a normal distribution, you do a z-test:

In statistical hypothesis testing (what we are doing), people have conveniently done the math on how certain distributions are (ie. If we want to check whether two samples are different, or to check whether ratios are different, or if this bunch of data is different from that bunch of data etc.). A z-test, using the test statistic Z, is calculated such that Z is exactly distributed as a standard normal distribution. So you can calculate Z, then compare it to a standard normal distribution (ie. between -1.96,1.96 is a 95% confidence interval).

Z = (M-µ)/σ
M = sample mean, µ = calculated mean, σ = calculated standard deviation.
Z = (77-35.1)/5.88 = 7.13. (note. We use 35.1=34.6+0.5, This allows the use of the normal distribution to estimate a binomial. Google “continuity correction”)
If this number (Z = 7.13) was less than 1.96, we would say that it was random chance. But because the number is so high, we are sure with a 95% confidence interval that the artifact drop rate is not 0.1%. In fact we are 99.9999999972% sure that it is not due to random chance.

I have assumed a random probably uneven mathematical background, if there is anything that you didn't understand (jargon, method, or theory) feel free to ask.

agreed. math discussion should separated from ask the expert. when I open ask the expert and see math discussion expert there.

nope la nope. math not my thing. move to reading WTS/WTB.

I like the assumptions and with the constructions chosen, it is hard to say no to these numbers. But since we do a unpersonal discussion on a scientific topic, let me be the counterpart trying to argue: "you are just one lucky guy and the wiki is correct".

So, how could I construct you are lucky on artefacts? First, I pick up your numbers. You killed 74889 monsters, resulting in 20594 drops on first roll. Since I want to say you are lucky I must grant wiki numbers, and not a shortend 27% drop chance with 20220 drops. So 27.5% drop chance and expected 20594 drops.
Then we have the equip roll. You used 0.95 for a assumed 2.5% chance and 0.92 for a assumed 4% chance, so I assume you have Quartermaster maxed. So again I have to take wiki numbers and reduce all estimated drops by 8% resulting in 18946 drops. How to set up a nice dice for me.. ?
I take a 10.000 sided Laplace dice (yes, its a virtual dice) with 9.982 black fields and 18 white fields. I roll that dice 18946 times and observe 77 white field rolls. Is this dice rigged?

The theoretical chance to roll a white field is 18/10000, a black one 1 - (18/10000).

H0 : p <= 18/10000, alpha = 5%
n = 18946, p = 18/10000, µ = n * p = 18946 * 18 / 10000 ~ 34.1028
σ = (n * p * ( 1 - p )) ^ 0.5 = (18946 * ... ~ 5.8345 (also > 3, Laplace condition)
checking on alpha upper border
µ + 1.64 * σ = 34.1028 + 1.64 * 5.8345 ~ 43.67 ~ 44
checking on alpha lower border
µ - 1.64 * σ ~ 24,53 ~ 25
so we get a distribution for alpha
{0...24}{25...44}{45...18946}
P(45<=x<=18946) = 0.5 * ( 1-P(25<=x<=44) )
-> r = ... ok, ok, not more worth the time, right?

H0 is therefore to be dismissed, but wait... the probability of still being true with H0? My last straw is the 5% chance, that you get a lot of artefacts less in your next runs or the wiki is wrong...
Anyhow, more n, pls! Think I will start counting artefacts, too, just to make sure you are not that lucky as you seem to be...^^

Chapeau! Excellent work.

This post has been edited by Frederiksc: Oct 5 2015, 19:47

I love min maxing (IMG:[invalid] style_emoticons/default/biggrin.gif)
What is the mathematical perfect stat distribution (since we have all the details, there should be a correct answer).
for 1h heavy just my guess:
0 int
agi=0.5end
str dex end wis are even

Also how do you record how many times you get drops? I have drop tracker but it only record shards token artifact and equips, it will record credit and crystal but only in a final sum amount.

Tally-Ho! This threat is for maths, so your post about stat weighting fits perfectly.

I love min maxing (IMG:[invalid] style_emoticons/default/biggrin.gif)

Who doesn't? ^^ Playing something and getting into it means you have to evolve a strategy. This is followed by fun or disgust. When you have fun, you stick to your stategy and try to elaborate it further.

What is the mathematical perfect stat distribution (since we have all the details, there should be a correct answer).

We do not have all needed details. We lack of the formulas distributing XP on PABs in the character screen and XP needed to level up. These are expected to be calculated in formulas like
XP_Level = (Level + a) ^ b ^ (1+level/c)
and
XP_PAB = (PAB + x) ^ y ^ (1+PAB/z)

For XP_Level we have an additional clue:
Sum n : 0 to 500 of XP_Level index n equals 9,999,999,999,999

for 1h heavy just my guess:
0 int
agi=0.5end
str dex end wis are even

This is called a stat weighting. Well, also a reasoned one. Just keep in mind that at the current state of HV 1h shield has no regeneration in the "infinite" SS, so WIS could be weighted lower alas being also useful for other things (e.g. magical migitaion). If we assume that your parry (governed by DEX) and block (governed only by prof and equip) are your main defences, evade (governed by AGI) is shortened up by burden, you could also lower the weighting for AGI, which you indeed did. So DEX is your most favored PAB.
So I just add a bit on DEX - if you do not mind - and then state a
HARD Weighting: STR 2, DEX 3, AGI 1, END 2 INT 0, WIS 2
We need to soften this due to those pesky a^b^c formulas and your level
estimated SOFT Weighting STR 6, DEX 7, AGI 4, END 6, WIS 6 <- this is just a rough thought, no deep math!
written in total point distribution: STR 6/29, DEX 7/29, AGI 4/29, END 6/29, INT 0, WIS 6/29
6 + 7 + 4 + 6 + 0 + 6 = 29; 29/29 = 1
To deep math it, we need more data, e.g. a fact based calculation for delta END to delta AGI.

Also how do you record how many times you get drops? I have drop tracker but it only record shards token artifact and equips, it will record credit and crystal but only in a final sum amount.

I record no drops at all. I just check after all arenas what I got. I just note what I had before and pay close attention at RE drops, so they do not interfere with arena drops.

This post has been edited by Frederiksc: Oct 6 2015, 15:06

The Empire Wiki strikes back on artefact rolls... or doesn't it?

I just made some calcs on drops I should have. Since I do not IW or GF much but stick to 100% arena and some REs, I just set up this sheet, you can play around with. It features some nice gadgets like observing the drop on artefact chance when training Quartermaster, etc...

It just goes for arena, but I think that's ok. Also I was quite shocked, how much equipment I should get after all arenas and was utterly disappointed recalling last WE's numbers. I think I am at least 100 equipment drops short. Am I not lucky? I tend to drop the wiki, but then again, who would go for "Yeah! Wiki's nice, dude!"


Added the sheet, too.
 Arena_rolls.ods.cut_all_after_ods.txt ( 19.09k ) Number of downloads: 22

This is called a stat weighting. Well, also a reasoned one. Just keep in mind that at the current state of HV 1h shield has no regeneration in the "infinite" SS, so WIS could be weighted lower alas being also useful for other things (e.g. magical migitaion). If we assume that your parry (governed by DEX) and block (governed only by prof and equip) are your main defences, evade (governed by AGI) is shortened up by burden, you could also lower the weighting for AGI, which you indeed did. So DEX is your most favored PAB.
So I just add a bit on DEX - if you do not mind - and then state a
HARD Weighting: STR 2, DEX 3, AGI 1, END 2 INT 0, WIS 2
We need to soften this due to those pesky a^b^c formulas and your level
estimated SOFT Weighting STR 6, DEX 7, AGI 4, END 6, WIS 6 <- this is just a rough thought, no deep math!
written in total point distribution: STR 6/29, DEX 7/29, AGI 4/29, END 6/29, INT 0, WIS 6/29
6 + 7 + 4 + 6 + 0 + 6 = 29; 29/29 = 1
To deep math it, we need more data, e.g. a fact based calculation for delta END to delta AGI.

My original idea was to make my stats as high as possible so add them evenly sounds good. AGI is borderline useless but it has 50% END value of pmi/mmi so I just put it 0.5
WIS will give resist but magic mit/evade has less value than physical so I think it should be lower

My original idea was to make my stats as high as possible so add them evenly sounds good. AGI is borderline useless but it has 50% END value of pmi/mmi so I just put it 0.5
WIS will give resist but magic mit/evade has less value than physical so I think it should be lower

Yeah, sounds reasonable. I tried to adapt by changing the distribution by XP needed to add a point. So when I state END = 2 * WIS, then this means raising END is twice the XP of raising WIS. But that also implies being not so correct stating the following:
END = 2 * WIS
WIS = 2 * INT,
-> 4 * INT = END, but adding XP_ADB results
XP_ADB * 4 * INT != XP_ADB * END, since there can be steps in between raising INT.

So lets try another way: Raise stats beginning from your most favored one. I have chosen END, since all is about END when IW gear. Stat weighting for 2h Mace Shade Arcanist for me (and my fun!):
2 STR, 2 DEX, 2 AGI, 4 END, INT 1, WIS 2
So when raising a stat, I begin with END, then do WIS, STR, DEX, AGI and finally INT (until WIS / 2).

We have a sum of total_exp available on the main screen of e-hentai, but until someone has a correct answer how much XP is freed or needed at any state we just can roughly go for such statements - which are utterly kindergarten like. Counting and adding. Yuck.

So a question like: "How low can I set AGI to have at least 65% MMI / PMI?" -which is a highly interesting one - can just be guessed or answered by experimenting on it.

Uhm... Coffee?
Estimated chance that the thread gets derailed?

Also how do you record how many times you get drops? I have drop tracker but it only record shards token artifact and equips, it will record credit and crystal but only in a final sum amount.

For me, I go to the inventory then select the whole item set, then copy and paste into excel (make sure you paste with matching destination formatting (right click + m) otherwise it'll lag out and be very dirty.

If you notice [www.mediafire.com] my records, I counted towards the left, that is because you can neatly overwrite the item names (so you get a table with rows being the type, and columns being the run number. Then add in any extra data you want (such as equipment/credits).

Uhm... Coffee?
Estimated chance that the thread gets derailed?

100%, that's the fun.

CODE

gf-artifact-equip
1    8    -
2    7    107
3    7    107
4    1    101
5    11    114
6    9    102
7    6    100
8    13    128

some pf gf data, all drop training max except arch 9/10

So you are doing:
drop training calc (but you can't estimate auction price, so good equip chance = 0?)
forge material calc (I think its useless)
reforge/potency calc (this and char calc will be pretty useful)
character calc

I feel like we can add a monster lab something, but the wiki pretty much has all the data.

Man, this post has been up for a week and no post b6 Skillchip? What is wrong with the world. Ill contribute if I get a chance.

CODE

gf-artifact-equip
1    8    -
2    7    107
3    7    107
4    1    101
5    11    114
6    9    102
7    6    100
8    13    128

some pf gf data, all drop training max except arch 9/10

According to your data, you got ~7.75 artifacts and ~108.4 equipment per grindfest. Compare this to what I got (8.55 artifacts and 111.8 eq).

With the same assumptions of 8300 mobs, 27% drop rate:
2.418% vs 2.494% equipment drop rate.
0.2019% vs .2231% artifact drop rate

They aren't statistically significantly different (the variance is much too high). A thing to notice is that your data matches the 'guess' of the artifact rate better (you got ~0.2%, I got ~0.22%). I rounded down in the past because we have the prior knowledge that "things probably make sense and are nice numbers". The data you found matches nicely.

drop training calc (but you can't estimate auction price, so good equip chance = 0?)

My [www.mediafire.com] previous drop rate training calculator used a large amount of user input data (the user has to record how much item drops, artifacts and equipment profit) that they got daily. With the additional input of the drop rates, it is possible to construct a calculator that doesn't require any input (besides current trainings).

What is knows:
drop rates, an estimate of (non salvaged, pfudor) equipment sell prices (503 with 4789 drops), how much you get if you salvage a mag+, an estimate of regular item values

What is still needed is:
what is the drop rate of mag/legendaries
what is the base chance of getting a rare tier gear (for exq,mag,leg)
how much extra do you get from selling a similar rare tier vs base tier
what is the difference in equipment drops for the different difficulties (hard)

This post has been edited by m118w11: Oct 14 2015, 05:04

CODE

Date        Atfc.    Equip    Trophies
2015 10 06    13    114        62
2015 10 07    04    118        64
2015 10 08    05    101        61
2015 10 10    06    119        56
2015 10 11    04    122        60
2015 10 12    04    121        73
2015 10 13    03    124        62
2015 10 14    07    111        61
2015 10 15    08    141        58
2015 10 16    05    117        71
2015 10 17    07    110        59
2015 10 18    05    113        70
2015 10 19    06    110        53
2015 10 20    05    120        72

Total rolls 73020, quite small, but enough for a rough test.
To be more correct on sigma, deleting highest and lowest value of each states:
5.5 Artefacts on average, calculating as 0.09038 instead of 0.16 wiki value. H0 dropped with sigma > 90%
116.583 equips on average, calculating as 0.01915 instead of 0.067549. H0 dropped with sigma > 99.95%!
63 trophies on average, calculating as 0.14583 instead of 0.14652. H0 not dropped.

We can be about sure to say, that Wiki is incorrect on artefact drops.
We can be almost absolutely sure, that Wiki is incorrect on equipment drops.
We can be almost sure, that Wiki is correct on trophies.