Supplementary data for the vassi Python package

This data repository provides supplementary data for<br>
<br>
"<i>vassi</i> - verifiable, automated scoring of social interactions in animal groups"<br>
<br>
Documentation and example usage of the package are available online at <a href="https://vassi.readthedocs.io/en/latest/">https://vassi.readthedocs.io/en/latest/<a/>. The source code is under version control at <a href="https://vassi.readthedocs.io/en/latest/">https://github.com/pnuehrenberg/vassi/<a/> and also archived here (<i>vassi_source.zip</i>).

<h3>1. <i>social cichlids</i> dataset</h3>

We tested our package on this novel dataset consisting of nine video recordings of groups of cichlid fish (15 <i>Neolamprologus multifasciatus</i> per group). The dataset also contains individual trajectories for each fish (stored in a HDF5 file, can be loaded in Python as numpy arrays; posture data and corresponding time stamps) and behavioral annotations (CSV files, one behavioral event per row). Reencoded video files (compressed using FFMPEG) are available in <i>datasets/social_cichlids/videos</i>.<br>
<br>
All scripts and notebooks from which results were presented in the paper used the same configuration for feature extraction (<i>examples/social_cichlids/features-cichlids.yaml</i>). We provide intermediate results (<i>examples/social_cichlids/results.h5</i> and <i>examples/social_cichlids/k_fold_predictions.h5</i>) for the <i>examples/social_cichlids/results_and_figures-cichlids.ipynb</i> notebook (available in GitHub repository or <i>vassi_source.zip</i>. This notebook reproduces the figures as presented in our paper.<br>
<br>
We also provide the results obtained from hyperparameter optimization using the optuna framework in the same directory (<i>examples/social_cichlids/optimization/*</i>).<br>
<br>
The results from k-fold prediction on the entire dataset (for visualization of networks as presented in the paper) are available in <i>examples/social_cichlids/k_fold_predictions_predictions.csv</i>, which can be loaded as a dataset when complemented with the trajectories file (see <i>vassi_source.zip/examples/social_cichlids/results_and_figures-cichlids.ipynb</i> for details).<br>
<br>
Our paper also presents a comparison between model predictions (behavior counts) and association time as an alternate behavioral proxy for interactions. The raw data files and the corresponding r script are available at <i>examples/social_cichlids/predictions_vs_association</i>.

<h3>2. <i>CALMS21</i> dataset</h3>

In addition, we tested our package on an existing benchmark dataset, the CALMS21 mouse resident-intruder dataset. For convenience, we provide Python scripts to download or convert the original dataset (<i>vassi_source.zip/src/vassi/case_studies/calms21/download.py</i> and <i>vassi_source.zip/src/vassi/case_studies/calms21/convert.py</i>; or available after <i>vassi</i> was installed, see online documentation for more details).<br>
<br>
The original CALMS21 dataset can be downloaded here:<br>
<a href="https://data.caltech.edu/records/s0vdx-0k302">https://data.caltech.edu/records/s0vdx-0k302</a><br>
<br>
<ul>
<li>[Dataset] Jennifer J. Sun, Tomomi Karigo, David J. Anderson, Pietro Perona, Yisong Yue, & Ann Kennedy. (2021). Caltech Mouse Social Interactions (CalMS21) Dataset (1.0) [Data set]. CaltechDATA. <a href="https://doi.org/10.22002/D1.1991">https://doi.org/10.22002/D1.1991</a></li>
<li>[Paper] Sun JJ, Karigo T, Chakraborty D, Mohanty SP, Wild B, Sun Q, Chen C, Anderson DJ, Perona P, Yue Y, Kennedy A. The Multi-Agent Behavior Dataset: Mouse Dyadic Social Interactions. Adv Neural Inf Process Syst. 2021 Dec;2021(DB1):1-15. PMID: 38706835; PMCID: PMC11067713.</li>
</ul>
All scripts and notebooks from which results were presented in the paper used the same configuration for feature extraction (<i>examples/CALMS21/features-mice.yaml</i>). As for the other example dataset, we provide intermediate results (<i>examples/CALMS21/results.h5</i>) for the <i>vassi_source.zip/examples/CALMS21/results_and_figures-mice.ipynb</i> notebook to reproduce the figures presented in our paper.<br>
<br>
We also provide the results obtained from hyperparameter optimization using the optuna framework in the same directory (<i>examples/CALMS21/optimization/*</i>).
<h3>Files</h3>
<pre>
datasets/
└── social_cichlids/
├── videos/
│ ├── GH010423.MP4
│ ├── GH010861.MP4
│ ├── GH013974.MP4
│ ├── GH019910.MP4
│ ├── GH030423.MP4
│ ├── GH030451.MP4
│ ├── GH030861.MP4
│ ├── GH039910.MP4
│ └── GH039931.MP4
├── cichlids_annotations.csv
└── cichlids_trajectories.h5
examples/
└── CALMS21/
├── optimization/
│ ├── optimization-results.yaml
│ ├── optimization-summary.yaml
│ └── optimization-trials.csv
├── features-mice.yaml
└── results.h5
└── social_cichlids/
├── optimization/
│ ├── optimization-results.yaml
│ ├── optimization-summary.yaml
│ └── optimization-trials.csv
├── predictions_vs_association/
│ ├── aggregated_counts-1bl.csv
│ ├── aggregated_counts-3bl.csv
│ ├── aggregated_counts-5bl.csv
│ └── predictions_vs_association.Rmd
├── features-mice.yaml
├── results.h5
└── k_fold_predictions_predictions.csv
└── vassi_source.zip
</pre>