Add further mixing variants (all weighted sum mixing based on constant weights)

[heidmic]

We want to add more variants to https://github.com/heidmic/suprb/blob/master/suprb/solution/mixing_model.py

Current mixing is making a model prediction based on all matching classifier's predictions times their inverse errors divided by their experiences (weighted sum).

What we want to test:

1. Add a hyperparameter that governs that only x rules can partake in mixing

• the best x rules (highest fitness)
• x random rules
• x rules randomly chosen using fitness-weighted roulette wheel selection

2. Cap the experience factor to some limit

• rules of experience y times dimensionality are equivalent in their weight to rules of experience > z times dimensionality (assuming identical errors)
• try without accounting for dimensionality
• are rules with experience smaller than z times dimensionality reliable at all? (e.g. z==0.5*d or z==5 assuming no relations to dim)

3. Combine these two
4. Add an option to weigh experience against error with a hyperparameter

[heidmic]

There is one point left open are rules with experience smaller than z times dimensionality reliable at all? (e.g. z==0.5*d or z==5 assuming no relations to dim). This basically asks whether we need a lower bound as well. Maybe something that decreases based on a function (probably nonlinearly). But it might also be detrimental as it strongly discourages specific rules. To implement this in a good way, we would probably need to do something like estimating the "relevant" experience based on some kind of neighbourhood estimation