March 16, 2006.
Music 209 : Calendar : Week 9How to use compute spectral representations, and use them to perform signal processing tasks in concatenative synthesis. (David away)
We began with a review of additive synthesis. The review was based on this article in Sound on Sound.
We then introduced the concept of the "common fate" of the parts of a sound model that belong to a particular note. The spectrogram of the Reynolds/McAdams oboe sound shown in class was taken from one of the slides in this talk by Dan Ellis.
We also discussed how the maintenance of phase continuity between the noise and harmonic components of the Derenyi/Dannenberg resynthesized trumpet is an example of using common fate well. This paper describes that system.
We then showed an interesting spectrogram of a sparrow's call. This spectrogram was an example file for the free software package Time-Frequency Toolbox.
We then discussed the time/frequency reassignment algorithm for enhancing spectrograms. The acoustic bass spectrograms shown in the lecture came from Kelley Fitz's webpage. This paper describes the algorithm.
We then discussed combining spectrograms that were processed using filterbanks with different time/frequency tradeoffs. This paper by Hong Leung and Victor Zue describes the approach.
We then discussed a system from Dan Ellis's LabRosa at Columbia, that computed multi-level maps. This paper describes the system.
I then talked about hardware systems I helped build in the 1980s and 1990s. This page summarizes that work.
We ended the talk with a discussion of the "Audio Photoshop" approach to spectral methods (as contrasted with the "Auditory Scene Analysis" approach of conventional resynthesis). We used the phase vocoder as an example. Mark Dolson's classic 1986 tutorial on Phase Vocoding is available here. The examples I played in class (and the software that produced them) are available here. Other good resources include this webpage and Chapter 3 of Jordi Sanhaume's Ph.D thesis, available here.