| mp4-sa-> sfront reference manual-> real-time |
Sections
|
IntroductionThis chapter describes how to use sfront in stand-alone real-time applications. We begin with basic sound playback, and show how to route audio output to a speaker under different operating systems. We then describe real-time interaction, with a MIDI keyboard and with an audio input source. |
|
Real-time PlaybackSfront can create C programs that send audio output directly to the sound card of a computer. The -aout command-line option is used to specify the hardware driver for audio output. The right panel shows the currently supported hardware arguments to the -aout option, and describes an example program to test real-time playback. If your hardware is not supported, use the -aout std command-line option to create a C program that places audio samples on standard output. When executing the program on the command line, pipe the output to a sound player that can drive the audio hardware. LatencyReal-time programs come in two flavors: interactive and streaming. Interactive programs create audio on the fly, in response to user input. These programs work best if the delay (or latency) from user input to audio output is as short as possible (2 ms is ideal, 10 ms is barely tolerable). However, if the latency is too short, audio output glitches may happen. Short latency is not as important in audio streaming work. It is usually better to use longer latency values, to minimize the chances of audio output glitches. For some audio hardware, 20-40ms of latency is optimal for streaming; other hardware prefers latencies in the 200-400ms range. Sfront chooses a default latency value, based on the nature of the SAOL program (interactive or streaming) and the type of audio hardware. The -latency command-line option overrides the default latency. Try setting the latency to a longer time value to minimize audio output artifacts. Temporal modesSfront programs run under one of three temporal modes: render, playback, or timesync. These modes define a set of real-time behaviors, described on the right panel. In most cases, the default temporal mode for an audio driver works well. Users can override the default behavior by adding the -render, -playback, or -timesync flag to the sfront command line. Real-time audio drivers usually default to playback mode. Programs running in playback mode accurately compute audio output audio, as long as samples are being computed fast enough to keep the audio hardware supplied with data. However, if the audio hardware is in danger of running out of samples, the decoder deviates from MP4-SA compliance, in an effort to maintain the sample stream. Once the danger of sample underrun has past, the decoder returns to accurate computation. A SAOL program may also implement graceful degradation techniques directly, by monitoring the cpuload standard name and pre-emptively reducing CPU usage. In contrast to playback mode, render mode computes an accurate rendition of the MP4-SA performance at all times. The file rendering audio drivers use render mode as the default, and always set cpuload to zero. When render mode is used with a real-time audio driver, cpuload updates to reflect the current CPU usage. The timesync mode forces the program to use at least 1 ms of real time to compute 1 ms of audio, by temporarily halting program execution if audio is computing too fast. Timesync mode may be used to make an audio driver that is not real-time by nature (such as the -aout null dummy driver) to behave in a real-time fashion. Under Linux ...Under Linux, if the C program generated by sfront runs as root, audio glitches and dropouts are significantly reduced. The Linux features used in root mode are described in the sched_setscheduler man page. We have tested sfront root mode carefully, and added special "watchdog" features to detect poorly written SAOL programs. However, any remaining bugs in sfront could have dangerous effects when the C files sfront creates are run as root. In particular, the machine could lock up, requiring a hard reboot, and files may be damaged as a result. Use this feature at your own risk. |
Output Hardware OptionsSee reference section of sfront manual for details on using these options. option Environment ------------------------------ -aout linux Linux. -aout coreaudio Mac OS X -aout pa_win_wmme Microsoft Windows (WMME library). -aout pa_win_ds Microsoft Windows (Directsound). -aout irix SGI Irix. -aout freebsd FreeBSD. -aout hpux HPUX. -aout std Generic -- pipe the output of the C program to a sound player app. -aout null Generic -- ignore the output of the the C program. Examplesfront/examples/rtime/aatest contains a simple audio streaming test. To run this test under Linux, cd into aatest and type "make". To run aatest on other platforms, see the README file in the aatest directory for details. Temporal modes
The real-time behavior
of an sfront program
depends on the temporal
mode it uses. Sfront
sets the default mode to
fit the chosen drivers.
Use one of these options
to override the default:
option description
--------------------------
-render Execute the SAOL
program as
accurately as
possible. Update
the SAOL standard
name cpuload if a
real-time driver
is in use; set
cpuload to 0 if
no real-time
drivers are in use.
-playback Execute the SAOL
program to keep
up with real-time
constraints, even
if accuracy is
sacrificed. Update
the SAOL standard
name cpuload.
-timesync Use the -playback
accuracy semantics.
Also, force the
SAOL program to
take at least 1ms
real-time to compute
1ms audio. Use this
flag with the null
and std audio drivers
to force real-time
behavior.
|
Real-Time InteractionSfront can create C programs that can accept real-time audio input (which is placed on the input_bus) and control input (which provides SASL and MIDI commands to drive the program). The type of interactive input is specified by choosing an audio input and control driver. Audio InputThe -ain command-line option is used to specify the hardware driver for audio input. The right panel shows the currently supported hardware arguments to the -ain option, and describes an example program to test real-time audio input. To reduce the time delay from input to output, use the -latency option to tune system performance. If your hardware is not supported, use the -ain std command-line option to create a C program that reads audio samples from standard input. When executing the C program on the command line, pipe the output of a separate sound recording program into the C program. Control InputThe -cin command-line option is used to specify the driver for control input. The right panel shows the control input drivers present in sfront. Three drivers support real-time input from a MIDI In soundcard jack under Linux: the linmidi driver for OSS, the alsamidi driver for simple MIDI input under ALSA, and the alsaseq driver for the ALSA sequencer. The coremidi driver supports MIDI input under Mac OS X (i.e. from a USB, Firewire, PCI or Cardbus slot). Virtual MIDI sources are not supported (yet). The win32 supports soundcard MIDI In under Microsoft Windows, and should work with both WMME and DirectSound Windows audio drivers (but has not been tested). Use the examples/rtime/linmidi program to test MIDI In performance under Linux. See the README in this example directory for details on using the example under Windows. The ascii driver provides simple real-time MIDI input from an ASCII keyboard. The examples/rtime/ascii example program shows how to use this driver under Linux, Mac OS X, and other Unix variants. The gliss driver is a platform-independent test driver for the control interface, and is used by the examples/gliss example. OS X notesThe coreaudio driver uses the default audio devices specified in System Preferences. If coreaudio is specified for both input and output, it is necessary that default input and output devices in System Preferences refer to the same device. To do this, use the Audio MIDI Setup utility to create an aggregate input/output device, and make the aggregate device the default in System Preferences. Linux/FreeBSD notesWhen used for interactive work, the Linux and FreeBSD audio drivers use the playback temporal mode by default, with a latency of 0.002 seconds. If audio artifacts are a problem, try increasing the latency time by using the -latency option. The Linux and FreeBSD audio driver prints out suggestions for the best sfront flags to use for a particular SAOL program and control driver setup, at the start of execution of the C program generated by sfront. Read these instructions and re-run sfront for best results. The linux audio driver detects if the C program generated by sfront is running as root, and uses special operating system features to reduce audio dropouts (see the man page for sched_setscheduler for details). Two root modes may be available for interactive use:
The Linux audio driver will print out the root modes available for a given control driver and SAOL program. Timesync mode offers superior quality over playback mode, but is only available in certain configurations. We have tested sfront root mode carefully, and added special "watchdog" features to detect poorly written SAOL programs. However, any remaining bugs in sfront could have dangerous effects when the C files sfront creates are run as root. In particular, the machine could lock up, requiring a hard reboot, and files may be damaged as a result. Use sfront root mode at your own risk. File StreamingIf sfront reads an MP4 file using the -bit option, it compiles score information into the sa.c file from the entire file (configuration and streaming sections). For MP4 files with large streaming sections, the generated sa.c may be too large to compile, or may run inefficiently.
The fstr control driver handles large MP4 files
more efficiently. The driver opens the MP4 file during
execution of the sa.c file, and inserts score
information into the running decoder at the correct time.
The -bitc option should be used with this driver,
so that the streaming score information does not get compiled
into the sa.c file:
The examples/claps example shows the fstr control driver
in action. Use the:
command to demonstrate the driver. Note that this technique only helps for MP4 files with large streaming sections. If large amounts of score data are present in the configuration section, this method does not help the data-size problem. Writing New Control DriversSee Part II/1 for information on writing a control driver for sfront. |
Audio Input OptionsSee reference section of sfront manual for details on using these options, including the stability of each driver. option Environment ------------------------------ -ain linux Linux. -ain coreaudio Mac OS X -ain pa_win_wmme Microsoft Windows (WMME library). -ain pa_win_ds Microsoft Windows (Directsound). -ain irix SGI Irix. -ain freebsd FreeBSD. -ain hpux HPUX. -ain std Generic -- pipe the output of the C program to a sound player app. -ain null Generic -- ignore the output of the the C program. Examplesfront/examples/rtime/linain contains a simple reverb system that uses real-time audio input and output. To run this test under Linux, cd into linain and type "make". To run aatest on other platforms, see the README file in the aatest directory for details. Control Input OptionsSee reference section of sfront manual for details on using these options, including the stability of each driver. option function -------------------------- -cin linmidi uses the MIDI port of soundcards under Linux (OSS). Generates MIDI commands. -cin alsamidi uses the MIDI port of soundcards under Linux (ALSA). Generates MIDI commands. -cin alsaseq supports the ALSA sequencer under Linux, for more sophisticated MIDI control than alsamidi. -cin coremidi supports external MIDI input in Mac OS X. -cin freebsdmidi uses the MIDI port of soundcards under FreeBSD (pcm). Generates MIDI commands. -cin ascii maps the ASCII keyboard as a MIDI device. -cin gliss plays piano glissandos, up uses MIDI control, down uses SASL. -cin fstr plays MP4 streaming packets from a file. -cin win32 uses the MIDI port of soundcards under Windows. Generates MIDI commands. |
Plug-in DevelopmentSfront can be used to build AudioUnit plug-ins for use on Mac OS X. AudioUnit plug-ins can be run inside popular commercial applications, such as Ableton Live, Logic, GarageBand, and Final Cut Pro. Sfront can create AudioUnits that act as virtual instruments that are controlled via a MIDI stream (MusicDevice AU type), in-line audio signal processors (Effect AU type), and signal processors that also receive a MIDI control stream (MusicEffect AU type). The right panel shows the sfront command line options related to AudioUnits. This chapter of the MP4-SA book provides details on how to use sfront to create AudioUnits. |
AudioUnit command-line optionsTo create an MusicDevice AU (type aumu): -aout audiounit -cin aucontrolm An Effect AudioUnit (type aufx): -aout audiounit -ain audiounit -cin aucontrol A MusicEffect AudioUnit (type aumf): -aout audiounit -ain audiounit -cin aucontrolm To activate debug mode, use -aout audiounit_debug in lieu of -aout audiounit. -- To set the AU subtype and manufacturer fields: -au_component_subtype Hiss -au_component_manu ucBe To set the manufacturer prefix for the bundle ID string: -au_manu_url edu.berkeley.eecs To set the AU bundle name: -au_filesystem_name hiss Name and manufacturer strings in the user-interface: -au_ui_name "Inline Noise Generator" -au_ui_manu "John Lazzaro, UCB EECS" -- To set the bundle name of a Cocoa UI view: -au_view_bundlename Filter To set the name of the class in the UI view that implements the AUCocoaUIBase protocol: -au_view_baseclass Filter_ViewFactory See this book chapter for more details on AudioUnit development. |
|
Next section: Part I/4: Command Line Options |
|
| mp4-sa-> sfront reference manual-> real-time |