State-of-the art voice conversion methods re-synthesize voice from spectral representations such as MFCCs and STRAIGHT, thereby introducing muffled artifacts. We propose a method that circumvents this concern using concatenative synthesis coupled with exemplar-based unit selection. Given parallel speech from source and target speakers as well as a new query from the source, our method stitches together pieces of the target voice. It optimizes for three goals: matching the query, using long consecutive segments, and smooth transitions between the segments. To achieve these goals, we perform unit selection at the frame level and introduce triphone-based preselection that greatly reduces computation and enforces selection of long, contiguous pieces. Our experiments show that the proposed method has better quality than baseline methods, while preserving high individuality.

@article{Jin:2016:CAC,
   author = "Zeyu Jin and Adam Finkelstein and Stephen DiVerdi and Jingwan Lu and
      Gautham J. Mysore",
   title = "{CUTE}: a Concatenative Method for Voice Conversion Using
      Exemplar-based Unit Selection",
   journal = "The 41st IEEE International Conference on Acoustics, Speech and Signal
      Processing (ICASSP)",
   year = "2016",
   month = mar
}