These project ideas involve drum sounds. Following the project list is a set of notes about concatenative drum synthesis that may be helpful.
Acoustic drum samples are brought to life with a microphone and a drummer. Electronic drum sounds are brought to life with a sound designer experimenting with a synthesis patch.
Is there another way?
A visual analogy may be appropriate here: Photographers create a photo with a camera, artists create a painting with brushes and paint.
However, recent work in computer graphics have focused on using a photo as input, and automatically generating an image that looks like a paint done by a human (or a watercolor, or a pencil sketch ...). A good review of this work (called non-photorealistic rendering) is available here.
A good research project would be to bring this concept from graphics to sound design, by creating algorithms that start with an acoustic drum sample, and produce a drum sample perceived as "electronic" but which is also sonically useful and interesting, and which had the same general properties as the original acoustic sample (attack, decay, etc).
Preserving the match is important, because it lets the acoustic and electronic sounds fuse when used together in the same song, or lets the electronic sound replace the acoustic sound without radically changing the role of the drum in the song.
Ideally, the algorithm would provide a small number of parameters that a user could manipulate in real-time to search the space of electronic drum sounds that can be generated by the sample. Other interface possibilities would provide controls that "broke" with the original sample in some way (say, decreasing the sustain time of the drum), but which also provided the means to modify the original sample in the same way.
The notes that follow these projects describe the history of drum machines and drum phrase loops. Much of this technology tries to humanize the machine on the beat level: while playing a rhythm pattern a few bars long, human drummers display a lot of talent and good taste.
However, drummers have another kind of good taste, that spans the entire song. Drummers decide what patterns to use in each part of the song, what kind of fills to play, what kinds of accents to use, etc.
For the most part, the software described in the notes leaves those decisions up to the person behind the DAW. And if this person is not a drummer, chances are those drum arranging decisions will be not so good. This article in Mix Magazine on looping raises this issue in an interesting way. Of the tools we discussed in this article, Steinberg's Groove Agent (first version reviewed here, an update on new features of its second version here) comes closest to approach this issue, but for the most part it leaves the issue in the hands of the person behind the DAW.
A good project would be to bring arrangement skills shown by good drummers to DAWs. At a minimum, it should recognize that drummers actually listen to the song they are playing to -- and thus, software that helps users choose and place drum patterns and loops, and to choose drum sounds, should be examining an existing song track too.
The project should also recognize that although many drummers are "born" with arranging skills, others learn them from good teachers, and that it should be possible to codify this practice into a tool.
A user-interface inspiration may be the "themes" approach that Apple uses in some of the applications -- for example, iPhoto lets you choose a photo book design "theme" and then customize the process of placing your photos in the theme. Building a drum arrangement out of loops could be managed by selecting and customizing a theme (A:B:B:A patterning, etc).
The project might display and accept arrangement information to the non-drumming producer in an abstract manner. One approach to this visualization is the line graph shown at the bottom of page 2 of this trance mixdown tutorial.
Most DAWs and drum machine plug-ins realize that finding the right loop or drum hit sample for a song in a 10 GB database is a hard problem for humans to solve.
However, most of them focus on meta-data to help the user find the right sound: original tempo, number of beats, a few words to describe the music style associated with the sample, etc. In practice, meta-data often seems best at eliminating happy accidents -- for example, screening out "electronic" samples for a ballad, when in fact the right electronic drum sample might take the song in a great direction.
A good project would be a loop browser that could listen to the loops, and listen to the musical composition under construction, and try to present options that actually relate to the sounds and the music.
Ideally, the browser wouldn't present options as a linear list of samples to click through and listen, but as a graphic construct that showed sonic relations between candidate samples, and which played parts of the samples automatically as one slid a cursor over the graphics.
Much work has been done on navigating the web using graphical constructs (CNET offers this sort of surfing of their site now -- see this link for details). Recent ISMIR conference papers may have also addressed this topic.
[Add David's percussion timbre space prior work here]
Good drummers often retune their drums to match the pitch of the song. This is done by changing the tension on drum head tuning lugs, to affect the enharmonic component frequencies of the Bessel functions produced by striking the head.
This is quite different that changing the pitch of a drum sample using a pitch shifting algorithm.
A good project would be to understand the physical basis of how drum tuning works, and to create algorithms that return tom-toms and bass drums in a realistic way. Extra points for realistically tuning snare drums, or tuning drums within an audio loop without creating artifacts in the cymbal sounds :-). These algorithms should take as input the root key and major/minor status of the song the drums will be used in.
[David's idea -- awaiting his rewrite] Create probability maps of each beat of a drum loop, using these maps to change aspects of the loop's audio, and using real-time controllers to alter the probability maps.
[David's idea -- awaiting his rewrite] When we create algorithms to place two drum samples on the same beat, how do we adjust the onset time and amplitude of the two samples so that they truly fuse? This is an open problem in psychoacoustics.
Since the 1920s, drum kits have been a mainstay of many types of popular music (dance music, pop, rock, jazz, ...). As this lecture is about synthesizing drum kits, students unfamiliar with the basics of drum kits may wish to read this Wikipedia entry, which includes photos and audio samples.
In a broad sense, drums are used for four musical purposes in popular music:
Much of this lecture assumes a certain level of familiarity with how popular music uses drumkits. Analyzing drum arrangements by listening to completed tracks can be difficult, because good drummers are skilled at blending into a performance. To get started, we suggest finding songs that begin with the drums playing a few bars alone, and listening to how the drum pattern "fade into the background" when the other instruments arrive. Here are excerpts of the start of several songs that begin with solo live drummers [mp3s to come].
Recording drum kits in a studio environment is a challenge. Usually, most drums in the kit are close-mic'd, using microphone models that bring out the best in the drum sound. The close-mic signals are usually combined with "overhead" microphones and room microphones to unify the sound -- and thus, the sound of the recording room plays an important role in the final drum sound. This article is a good introduction to capturing a drum kit sound, as is this interview
A full analysis of the physics of percussion instruments is a topic that could fill much of the semester. Thankfully, for the purposes of this course, simple qualitative models of how drums sound are sufficient. Students not familiar with simple drum models may wish to read the articles linked below. These articles appeared in the British music production magazine Sound on Sound:
In a sense, drummers act as human concatenative synthesizers. By choosing when to strike each drum and cymbal, drummers string together short sounds to make a complete performance.
The mechanization of drumming -- by replacing human drummers with a state machine, and acoustic drums with mechanical or electronic elements -- is an old idea. Its original commercial motivation was to let amateur musicians playing home organs to add a rhythm track to their performances.
The first known commercial drum machine is the Model 100 Rhythmate (Chamberland Instruments, Upland CA), released in 1949.
The Rhythmate 100 played an magnetic tape recording of a drum pattern over and over at a constant speed, by looping the tape. The entire drum kit was recorded on the tape loop. The machine had 14 drum loops, which held different rhythm patterns.
About 10 Rhythmate 100s were made. Read more about this machine here. After this product, Chamberland Instruments left the drum loop business, and moved into keyboard instruments -- the tape-loop technology of the company evolved into the Mellotron, a product we will revisit in later lectures.
The small production run of the Rhythmate 100 limited its real-world impact. The Wurlitzer Sideman, introduced in 1959 as a peripheral to organs, is considered the first drum machine that made a real commercial impact. It used mechanical music-box technology to sequence the drum sounds (cuts in a wheel spun by a motor), and electronic circuits to create drum sounds. Some interesting links about this device:
The Wurlitzer Sideman set the formula for drum machines in this era -- fixed patterns and quirky electronic sounds -- but products that followed the Sideman were cheaper and more reliable (mostly by becoming fully electronic and then fully transistorized). The FR1 Rhythm Ace (described here) became the hit product of the genre, and was a featured add-on to Hammond Organs. As this history describes, the ACE company later became Roland, and the drum sounds in the FR1 Rhythm Ace became the foundation for the classic Roland analog drum machine sounds that underlie much of modern dance music. You can download samples of the Rhythm Ace here.
It's easy to see why drum machines would be popular as accessories for home organs. It's less obvious to see why record producers would want to replace a key member of a session band with a machine. However, several factors combined in the late 1970's to start the trend to replace human drummers with machines.
When attempts were made to use drum machines from the Rhythm Ace era for "serious" work, one flaw became immediately obvious: the rhythm patterns were fixed in the factory, and these patterns were not terribly relevant to popular music in the 1970s. A better source of fixed patterns in this era were vinyl records that were cut with studio recordings of a drummer playing a full kit playing song-length drum patterns, complete with fills at regular intervals.
These "Drum Drop" loops sometimes made it onto commercial releases. However, most artists of the era needed more control over the beat patterns than a Drum Drop record could provide. Thus, early adopters in the recording community became interested in a hobbyist product from PAIA Electronics.
The PAIA Programmable Drum Set, released in 1975. As seen in this photo, pads on the front panel of the machine let users program in a pattern. The user interface of the PAIA machine was quite primitive by modern standards, and its analog circuit drum sounds were far removed from real drums. However, its programmability made it unique instrument for a time. Several thousand machines were sold, and the machine was used on early Peter Gabriel albums.
Although the PAIA machine was a step forward, the acceptance of drum machines in the professional music world came with a pair of products from Roger Linn: the LM-1 (released 1979) and the LinnDrum (1982).
As this early company ad shows, the headline feature of the LM-1 was sound production via digitally sampled recordings of acoustic drums. However, as the bullet list in this ad shows, the LM-1 integrated solutions to many problems faced by early drum machine users: programming realistic drum patterns, sequencing patterns into songs, mixing and tuning the drum sounds to make a realistic kit, automatic sample muting (so that starting a closed hi-hat hit automatically cuts off a sounding open hi-hat hit), etc.
For further information on the Linn machines, see this product history web page on Roger's website, and this short interview.
Like the Wurlitzer Sideman in its era, the LM-1 set the formula for the sampled drum machine era. Other companies went to work extending and enhancing the idea in unique ways. Products from this era included the Oberheim DMX, the Sequential Circuits DrumTraks, the E-mu Drumulator, and the RX series by Yamaha.
During the era of sampled drum machines, the Roland Corporation continued to "follow its own drummer" and produce drum machines that used analog circuits for sound production (however, the machines did support programmable patterns). The Roland machines, including the CR78 in 1977, TR-808 in 1980, and the TR-606 in 1981, were not particular commercially successful in their day. The TR-909 (introduced in 1983), a hybrid machine that combined acoustic drum samples and analog drum circuits, also failed to excite the marketplace.
However, time would be kind to the Roland analog machines. Years after the CR-78 was discontinued, the machine was used by Phil Collins in 1981 to get the haunting drum sound used in the beginning of his single "In the Air Tonight". After the release of this song, many CR-78s found their way back into service in the studio.
A similar rebirth befell other Roland machines from the analog era. The drum sounds in the TR-808 and TR-909 became staples of many dance music genres, including house music, throughout the 1990s and up to the present day. A quick listen of the top-selling songs on the iTunes Music Store in December 2005 showed an near-even split between songs with Roland-style synthetic drum sounds and tracks based on acoustic drum sounds. A good description of the TR-808 sounds, and a free library of samples taken from the machine, is available here, a part of this website about the TR-808. See this link for another fan site of early Roland gear.
Synthesizing the drum sounds of Roland drum machines is an art form of its own. A series of articles on this topic appeared in the British music production magazine Sound on Sound:
During mixdown, analog drums may undergo extensive signal processing to create a drum sound appropriate for a music style. A hands-on tutorial for processing analog drums for trance music is available here.
The impact of Moore's Law on the music technology world cannot be overstated. Moore's Law is a prediction of how the manufacturing of the integrated circuits that make up computer chips will improve over time: namely, that the number of transistors that can be practically manufactured on a chip will double every 12-24 months.
By the mid-1990s, by direct consequence of Moore's Law, musicians could buy reasonably priced computers with millions of bytes of main memory, addressed by CPUs that could process hundreds of millions of instructions per second. Concurrent with Moore's Law improvements were improvements of hard disk technology for permanent memory storage, supporting several billion bytes of memory, and the coming of age of compact disks as removable storage (600 million bytes per disk).
This coming of age of the personal computer signalled the start of the decline of hardware drum machines as essential tools for most music producers (along with the decline of many other types of stand-alone music and audio products, such as analog tape machines).
The new studio model placed a general-purpose computer to be the centerpiece of a recording environment, sometimes aided by special-purpose hardware for demanding tasks. As computers became more powerful, the computer gradually replaced more and more special-purpose hardware. This trend continues to the present day, with late-2005 computers sporting GBs of main memory, 100s of GB of hard disk storage, multi-core multi-GHz CPUs, and read/write DVD optical disks.
The application software that runs on the computer is commonly known as a Digital Audio Workstation (DAW). Popular flagship commercial DAWs include Logic Pro and Pro Tools HD. Flagship DAWs usually have lower-priced entry-level DAW products, such as the Logic-derived Garageband or the Pro-Tools HD derived Pro Tools LE (Limited Edition). As one box (the computer) is now the studio, the phrase "in the box" is often used to describe productions done on a DAW.
When using a DAW, the rhythm track construction that was previously done using a drum machine is instead done using the core editing functions of the DAW, sometimes aided by third-party plug-in software specialized from drumming.
Using a DAW running on a modern computer to create drum tracks brings an amazing amount of resources to the rhythm programming problem (CPU, RAM, disk, high-resolution displays, Internet connectivity, etc). Better hardware let software designers take a fresh look at rhythm programming.
This cycle of invention began in the late 1990s and continues to this day. The result: a set of standard drumming features present in most DAWs, augmented by an expanding set of drumming plug-ins for advanced features (a recent round-up article of drumming plug-ins discussed 19 commercial virtual drum machines, and purposely left out whole categories of drum software!).
In the remainder of the lecture, we describe these advancements in an organized way, and show opportunities for research problems to extend the current state of the art.
Hardware drum machines concatenate individual drum hit sounds (sampled or synthesized) to create rhythm patterns.
Some users are in search of a "virtual drum machine". This software emulates the operation of a hardware drum machine within a DAW framework. In practice, the implementation of a virtual drum machine is split between drum software plug-ins and the DAWs that host them. For these programs, part of the job of drum machine emulation is to fix long-standing problems with the drum machine interface.
One problem with hardware drum machines is that it has always been tricky for novices (especially non-drummer novices) to easily program beats that are musically effective replacements for beats of real drummers. This is particularly true for "real-time" programming, that is done by pressing pushbutton switches on the machine front panel as one would hit drums with sticks.
Software drum machines that support real-time programming usually accept drum strike input via "MIDI controllers" -- special purpose user-interface hardware that interfaces to the computer via MIDI. MIDI is the standard communications protocol for musical gestures -- see this tutorial for an introduction. Many types of MIDI controllers are available for drum programming, including controllers that look like drum pads and controllers that look like traditional drum machines front panels.
An alternative to real-time programming is enter a drum score by entering drum hits in an off-line way, much as one would write down a drum score on staff paper. The Roland front-panel interface for this programming method, called step-mode (and described here), was particularly efficient and easy to use. Off-line entry is easy to implement in virtual drum machines -- software can emulate editing of traditional music score notation, or emulate the front panel interface of the classic Roland machines.
Real-time programming and off-line programming have rhythmic "feel" problems that are different, but yet in a way are the same. Even seasoned real-time programmers may find that their best-effort performance isn't quite "right" -- problems range from occasional wrong notes, to rhythms that don't quite feel "right" in some way. Likewise, even accomplished composers find that their notated rhythms may sound stiff or mechanical when played exactly as written by a machine -- the performance lacks the interpretive skill of a good drummer.
From their earliest days, programmable hardware drum machines had post-processing functions to both tighten up sloppy real-time playing (usually called "quantization") and to add life to step-time entry ("swing" to alter beat accents, and "humanization" to introduce the "randomness" of a human player). Note how these features are emphasized in this advertisement for the Linn LM-1, the first sampled drum machine.
The MIDI implementations in DAWs have evolved quantization and humanization functions, in both user-interface and function. The most sophisticated tool of this nature, "Groove Templates", lets a user extract beat patterns from an audio recording, and then use this derived "groove template" to quantize a MIDI loop. This two-part tutorial in Sound on Sound [part 1] [part 2] is a good introduction to use use MIDI editing features to shape the feel of a MIDI drum performance.
Students who have not programmed drum tracks themselves may find the Sound on Sound tutorial a bit abstract. This tutorial in Computer Music magazine [pdf 1] is a very hands-on piece on MIDI programming of Latin percussion (note that the PDF files are meant for European paper sizes, and may need to be scaled down to print correctly on your printer). Several other tutorials of this type may be found by looking at the drumming section of this bibliography.
In addition to rhythm pattern entry and editing/enhancement, virtual drum machines also generating audio for the drum sounds.
The advent of DVD-ROMs and huge hard disks make it possible for drum machines to use many GB of drum samples -- a giant leap forward from the original hardware drum machines whose sample storage was measured in hundreds of KBs. Virtual drum machines (usually plug-ins that are sample-playback systems optimized for drumming) use this storage in the following ways:
Virtual drum machines also put the powerful CPUs of modern machines to good use, by including a complete synthesis engine to process each sample (and often, oscillators to generate electronic drum sounds via synthesis).
Recent examples of plug-ins that are based on virtual drum machine concept are reviewed in-depth here:
Not all composers may be up to the challenge of programming MIDI patterns and choosing drum sounds on their own. In a throwback to the age of fixed-pattern home organ drum machines, a few companies have worked on "Virtual Drummer" software, where users choose from pre-programmed MIDI beats designed for different musical genres. Steinberg's Groove Agent (first version reviewed here, an update on new features of its second version here) is an example product of this type.
However, for the most part, composers that are not interesting at programming at the beat level have left the drum machine metaphor behind, and use the looping approach we describe later in the lecture.
Looking back on the history of drum machines, an idea that occasionally appears is one of recording a drummer playing a rhythm pattern on a complete kit, and using this recording as raw material for a new track.
The advantage of this approach is clear: the drummer sounds real because the drummer is real.
The disadvantage of this approach is also clear: the lack of flexibility. The drummer on the recording played a certain beat, at a certain tempo, with a certain drum sound. One needs to find a pre-recorded track that matches the intent of a new recording project to use this approach successfully.
Drum loops as an alternative to rhythm programming has come of age in the past decade, because advances in hard disk size and computer performance, and the ability to ship 8 GB of sample data on a single DVD-ROM, can solve the flexibility problem.
Drum loop technology (as plug-in products, and as integrated into DAWs) solve the flexibility problem in several different ways.
The first system we discuss is a very recent product, Drum Core (reviewed here). We start with this product because it is so "lo-tech", yet so ingenious in its own way.
Drumcore ships with 10 GB of samples of world class drummers from many genres playing a large collection of rhythm patterns in their specialized style. For example, Alan White (drummer with Yes) represents progressive rock, Sly Dunbar (Bob Marley drummer) represents reggae, and Matt Sorum (Guns and Roses) represents LA hard rock. Each drummer uses his or her personal drum kit, specialized for the style.
The primary format of the patterns are audio mixed to stereo, shipped at several sample rates (44.1 KHz upto 196 KHz) and bit-depths (16 bit and 24-bit). To solve the tempo problem, the drummer systematically plays each pattern at many tempos. With 10 GB of DVD-ROM, and a drummer with patience, one can solve the tempo problem without resorting to technology.
Each loop is also shipped as a MIDI file -- during the studio recording process, close-mic's signals on each drum were processed to extract transients, and manually edited to fix triggering mistakes. A library of single-hit samples for each kit is also supplied. This second format lets a producer drop the MIDI and samples into a virtual drum machine, and then change the samples or the performance as needed.
This solution is hard-disk-space intensive, and has only been practical to implement in the past few years. For example, budget laptops currently ship with 30 GB disks -- installing this program uses one-third of your disk capacity! This solution also limits you to using beats from currently-living drummers who are willing to participate in the process, and who happen to play a beat you want to use during a session.
These limitations pushed technologists to start with a single audio loop, and find ways to change the tempo, drum sounds, and playing styles through software processing. In the sections that follow, we describe different approaches to this technology problem.
When one looks at a drum loop waveform in a graphical audio editor, it is apparent when drums are called "percussion" instruments. Large spikes appear in the waveform at the onset of each hit, which quickly fall towards the noise floor within a few milliseconds after the onset.
Beat-slicing is a method for time-stretching drum loops that takes advantage of this structure. A drum loop is "sliced" into non-overlapping sections that begin at the start of a percussion transient, and ends right before the next percussion transient in the loop. Slicing is easily automated using simple thresholding, or may be performed manually using an editor.
One a loop is converted into slices, each slice can be treated as an independent sample, that may be played back in a virtual drum machine as a normal drum sample.
If the triggering of each sample is carefully timed to match the original timing of the audio loop, the triggered samples will perfectly reproduce the original loop.
However, if the trigger signals are sped up in a uniform way, the audio sounds, to a first approximation, as the original audio loop would if it were played at a faster tempo. Likewise, if the trigger signals are slowed down in a uniform way, the audio sounds like the original audio loop would if it were played at a slower tempo. This tempo manipulation scheme is not without its artifacts. For example, slowing down a loop inserts pure silence into the loop, rather than the background noise of the original sample.
However, workarounds have been developed to minimize these artifacts, and beat-slicing has become a standard way to control loop tempos in the industry. The first popular tool for automatic beat-slicing of audio loops was Recycle, first released in 1995 by Propellorhead Software. REX files, the file format for Recycle, have become an industry standard.
A tutorial showing how to use Recycle for beat slicing is available here. In addition to tempo manipulation, a beat-sliced version of an audio file is also open for the sort of manipulations we discussed in the drum machine part of the lecture (quantization changes, changing the audio sample triggered by a hit, etc). This article discusses how to use Recycle to perform this kind of loop modification. If these tutorials are too abstract, you may wish to begin by reading this hands-on [pdf-file 2] tutorial on REX file manipulation for Latin percussion.
The beat-slicing concept is also used by another popular drum loop program, Spectrasonic's Stylus RMX. Stylus RMX uses its own file format for slices, called Groove Control, that claims to offer improved audio fidelity for large tempo changes. A review of Stylus RMX is available here, and a tutorial for using RMX to do interesting loop modifications is available here.
Beat slicing is a concatenative approach to tempo modification of audio loops: drum hits are located via transient analysis of the loop, and the loop is broken up into drum-like units that can be independently manipulated.
To a certain degree, the method relies on the fact the audio loop contains a collection of percussion sounds. Although beat slicing can be applied to an audio loop containing any sounds and sometimes yield interesting sonic results, the success of processing non-percussive sounds with beat slicing is hard to predict in advance.
An alternative to beat-slicing is to use general-purpose signal processing techniques to change the tempo of the loop. We refer to this method of tempo modification as time-warping. For example, a simple time-warping technique would be to play the loop back at a slower sampling rate to slow down the tempo. For very small tempo changes, this simple approach may work well; for large tempo changes, it introduces obvious pitch-shift artifacts into the timbre of the loop.
However, much more powerful signal-processing techniques have been developed for time-warping, whose sonic artifacts are less objectionable than simple re-sampling. These techniques are worthy of a lecture of their own, and so we will not discuss them further here.
However, we note that these techniques are the method DAWs use for tempo manipulation of loops (including drum loops). Non-trivial techniques were commercially introduced in Acid, a loop-construction tool first released in 1998 (read the review of the latest version of Acid here). Since the release of Acid, other companies have introduced products that advanced the state of the art of time-warping algorithms; notable implementations of good time-warping algorithms are Ableton Live and Apple's Apple Loop technology, which first appeared in Soundtrack.
In some recording situations, there really is no substitute for a live drummer playing on a track. Recently, a few drummers have been pioneering selling custom-player drum tracks as an Internet service.
In this model, a producer contacts the drummer via email, sending along rough mixes of a track in progress whose drum machine would be better discarded for real drums. For a fee, the drummer transfers the rough mix to the DAW in his personal studio, records drums to the track, and sends back a multi-channel audio file to the producer, along with a bill.
For example, LiveDrums offers a service of this nature, charging $99 for 3-week turnaround for a song (prices go up for quicker turnarounds).