User Experience Optimization Experiment Simulations

# The `uxo_sim` Package A package for simulations of data matching industry UX optimization experiments, as discussed in: ```<br> @article{van_adelsberg_modeling_2019,<br> title = {Modeling {A}/{B} {Test} {Data} is {Hard}: {Effects} of {Overdispersion}, {RandomWalks}, and {Cointegration}},<br> language = {en},<br> journal = {NeurIPS 2019 Workshop on Robust AI in Financial Services: Data, Fairness, Explainability, Trustworthiness, and Privacy},<br> author = {van Adelsberg, Matthew and Sweeney, Mackenzie},<br> month = dec,<br> year = {2019}<br> }<br> ``` The code for running the simulations is included, along with figures and CSV files for each of three specific simulation runs that are used in a publication currently under review for ICML 2020. ## Packages: 1. `data`: code for running the simulations to produce datasets<br> 2. `viz`: code for visualizing the simulation outputs ## Scripts: 1. `save_datasets`: save CSV for each simulated dataset in the `inventory`<br> 2. `save_figs`: save PNG figure for each simulated dataset in `plots` ## Simulation Datasets: ### `fixed_effects_od_20_21_seed27` Data is simulated from a beta-binomial distribution with overdispersion parameter `gamma=0.01` for each of the two treatments and rates `theta=0.20` and `theta=0.21`. This corresponds to beta distribution parameters `alpha, beta = 19.8, 79.2` and `alpha, beta = 20.79, 78.21`. ### `drift_down_then_up` Data is simulated from a beta-binomial distribution with overdispersion parameter `gamma=0.01` for each of the two treatments. The rates start at `theta=0.20` and `theta=0.21` and then: 1. decrease by 0.005 each day for 20 days<br> 2. increase by 0.005 each day for 30 days<br> 3. stay constant for 10 days The corresponding beta distribution parameters on each day are a function of `theta, gamma` and can be obtained via this function (implemented in `uxo_sims.data.simulations`:<br> ```python<br> def alpha_beta_from_gamma_theta(gamma, theta):<br> virtual_sample_size = 1 / gamma - 1<br> alpha = theta * virtual_sample_size<br> beta = virtual_sample_size - alpha<br> return alpha, beta<br> ``` <br> ### `arm_addition` Data is simulated from a beta-binomial distribution with overdispersion parameter `gamma=0.01` for each of the two treatments. The rates start at `theta=0.10` and `theta=0.11` and increase by 0.005 each day for 40 days. The corresponding beta distribution parameters can be obtained with the same function as indicated in `drift_down_then_up`.