AudioQuilt: 2D Arrangements of Audio Samples using Metric Learning and Kernelized Sorting
14th International Conference on New Interfaces for Musical Expression (NIME), June 2014
Snare drum navigator. The user is presented with a grid of colored rectangles, each corresponds to a sound sample. Hovering over samples produces sound; similar sounds are placed in proximity. The feature vector consists of MFCC descriptors, log attack time and temporal centroids. We support several coloring schemes, either according to the features used or according to other information we wish to convey. Here we show Isomap coloration (left) and k-means based coloring (right).
Abstract
The modern musician enjoys access to a staggering number of audio samples. Composition software can ship with many gigabytes of data, and there are many more to be found online. However, conventional methods for navigating these libraries are still quite rudimentary, and often involve scrolling through alphabetical lists. We present AudioQuilt, a system for sample exploration that allows audio clips to be sorted according to user taste, and arranged in any desired 2D formation such that similar samples are located near each other. Our method relies on two advances in machine learning. First, metric learning allows the user to shape the audio feature space to match their own preferences. Second, kernelized sorting finds an optimal arrangement for the samples in 2D. We demonstrate our system with three new interfaces for exploring audio samples, and evaluate the technology qualitatively and quantitatively via a pair of user studies.
Links
Citation
Ohad Fried, Zeyu Jin, Reid Oda, and Adam Finkelstein.
"AudioQuilt: 2D Arrangements of Audio Samples using Metric Learning and Kernelized Sorting."
14th International Conference on New Interfaces for Musical Expression (NIME), June 2014.
BibTeX
@inproceedings{Fried:2014:A2A,
author = "Ohad Fried and Zeyu Jin and Reid Oda and Adam Finkelstein",
title = "{AudioQuilt}: {2D} Arrangements of Audio Samples using Metric Learning
and Kernelized Sorting",
booktitle = "14th International Conference on New Interfaces for Musical Expression
(NIME)",
year = "2014",
month = jun
}