Reproduction Kit for the Paper "The Right Cut: Interaction Data Slicing and its Effects on Keyboard Interaction Metrics"

This is the reproduction kit for the paper "The Right Cut: Interaction Data Slicing and its Effects on Keyboard Interaction Metrics". For understanding this readme, please consult the paper first. The reproduction kit contains: The raw DUX dataset (folders DUX_autoquestdata and DUX_imotionsdata) A version of the AutoQUEST Software to parse the DUX data and calculate interaction metrics using different slicers and slicing parameters (source in folder autoquest-source, compiled binaries in zip file autoquest-distribution-0.2.5-SNAPSHOT-bin-Windows-amd64.zip) Jupyter Notebooks used for analysing the metrics data and to generate the figures and results of the paper (in folder Jupyter_metricanalysis) To execute the reproduction, the following steps must be performed: Run AutoQUEST to calculate metrics data (optional, as the metrics data is already included in this reproduction kit in the Jupyter_metricanalysis) Execute the Jupyter notebooks to analyse the metrics data and to generate the results and figures of the paper (partially optional as the reproduction kit already includes the figures but not the other relevant conclusions of the paper) Run AutoQUEST and Calculate Metrics Data Adapt the file autoquest_script_calculate_metrics.txt and replace any occurrence of <FOLDER> with the absolute folder into which you extracted the reproduction kit, i.e., the folder of this readme Extract the AutoQUEST distribution package autoquest-distribution-0.2.5-SNAPSHOT-bin-Windows-amd64.zip Change into the extracted folder Call java -Xms20480m -Xmx20480m --add-opens=java.base/java.lang=ALL-UNNAMED -jar autoquest-runner-0.2.5-SNAPSHOT-Windows-amd64.jar -ui swt Click on File and Exec Batch File Select the adapted autoquest_script_calculate_metrics.txt file and hit OK AutoQUEST is now parsing the DUX data and calculating metrics using different slicers and slicing parameters. The results of the process are stored in the folder Jupyter_metricanalysis\data. If the metrics data is already located in that folder (as is the case if the reproduction kit is freshly downloaded), AutoQUEST will not overwrite it but fail with a corresponding error message for each of the metrics files. For a better understanding of what AutoQUEST does AutoQUEST has many build-in commands to process data. These commands can be combined into scripts like the autoquest_script_calculate_metrics.txt. In this concrete script, AutoQUEST is instructed to do the following: the two parseDirHTML commands parse the DUX interaction data, i.e., the events caused by the user interactions (once for with and once for without trigger; see paper) the two parseDirIMotions commands do the same but with the emotion data of the DUX dataset as recorded user interactions are sometimes messy, correctKeyInteractionTargets ensures that key interactions are always on the correct text field timeSyncIMotions ensures a time alignment of the recorded interaction data and the user emotions belonging to them annotateEmotions combines the interaction data with the emotion data exportMetrics calculates the metrics for a specific slicer (e.g., TIME) and parameter (e.g., 3000 ms) as a file into the given folder Of the last command, there are diverse combinations of slicer and parameter as well as with and without trigger data. Please refer to the paper for more details. Run Jupyter Notebooks For running the Jupyter Notebooks, copy the folder Jupyter_metricanalysis into a jupyter workspace. It is important to consider package versions. The genuine execution of the notebooks was done using the following package versions: Python (version 3.11.9) Jupyter Lab (version 4.4.4) statsmodels (version 0.14.5) numpy (1.26.2) pandas (2.3.1) seaborn (0.13.2) pyplot (3.10.7) scipy (1.16.2) The Jupyiter notebooks can be executed in any order. They read the data from the data folder (potentially created using AutoQUEST as described before) and produce diverse figures in the figs folder. In addition, they provide diverse output messages to reflect the statements in the paper. The individual notebooks do the following: MetricVsKeyEventCountInSliceAnalysis: This notebook creates the plots mapping the number of key events in a slice to the metric values (stored in folder figs/MetricsVsKeyEventCountInSliceAnalysis). In addition, it analyses how many keyboard metrics have a linear dependency on the number of key events in a slice. VarianceCausedByUserSession_Keyboard: This notebook creates box plots showing the measurements for the metrics per user sessions to identify user specific effects (stored in folder figs/VarianceCausedByUserSession). In addition, it aligns Linear Mixed Models on the data to assess the existence of user specific effects in the measurements. VarianceCausedByKeyInteractionTarget: This notebook creates box plots showing the measurements for the metrics per key interaction target (stored in folder figs/VarianceCausedByTask). In addition, it aligns Linear Mixed Models on the data to assess the existence of user and task specific effects in the measurements.