mp4-sa-> the mp4-sa book-> SAOL->buses and execution order

From The MPEG-4 Structured Audio Book by John Lazzaro and John Wawrzynek.

Part II/5: SAOL Buses and Execution Order

Sections

In This Chapter

Statements:

outbus output route send sequence spatialize

Other Language Elements:

inchan inchannels inGroup input input_bus outchan outchannels output_bus srate

Introduction

In this chapter we describe how audio signals flow between instrument instances in a SAOL program.

We begin by showing how an instrument writes audio signals to its output port.

We describe how simple audio output works in SAOL. In this default model, the output ports of all instruments sum onto the system output_bus that becomes the final audio output of the program.

We describe the general audio model of SAOL, where instruments may have audio input as well as output. We describe how to specify the structure of multi-instrument systems, and how to determine the width of audio buses and the order of instrument execution.

We end the chapter with a description of the standard names related to audio buses, and a discussion of the ancillary audio output statements outbus and spatialize.

Output Statement

A SAOL instrument has a single output port, that holds the audio output of the instrument. An instrument generates sound on its output port by using the a-rate output statement.

At the start of an a-pass, the value of the output port is set to zero. An instrument uses output statements to sum values onto the output port. Once the a-pass of the instrument has ended, the output port value maintains its value until the start of the next a-pass.

Like a signal variable, the output port has a width. Unlike a signal variable, the width of the output port is not declared as an integer value.

Instead, we infer the width of the output port. In most cases, we can determine the width of the output port from the properties of the output statements in the instrument.

As shown on the right panel, an output statement takes a list of signal expressions as arguments. The width of the output statement is the sum of the widths of its arguments.

Given this definition of the width of an output statement, we can state several rules about the audio output port of an instrument:

  1. The output port has the width of the widest output statement.
  2. An output statement with width greater than one must have the width of the output port.
  3. An output statement with width greater than one sums data onto the output port channel-by-channel fashion.
  4. A output statement with width equal to one sums its single data value onto each channel of the output port.

The right panel shows examples that follow and break these rules.

The standard name outchan indicates the output port width. To declare an array test whose width is the output port width, use the syntax   test[outchannels].

Syntax

output(a1 [,a2, a3 ...]);

a1, a2, ... are signal expressions

output statement is a-rate.

output width is the sum of the
width of a1, a2, ...

Example [legal]


instr trichannel() {

asig mono, stereo[2], tri[3];

output(mono, stereo); // width 3
output(tri);          // width 3
output(mono);         // width 1

// audio port after a-pass:
// 0: mono + tri[0] + mono 
// 1: stereo[0] + tri[1] + mono
// 2: stereo[1] + tri[2] + mono

}

Example [illegal]


instr rule2_braker() {

asig stereo[2], tri[3];
                
output(tri);    // audio port 
                // takes width 3

output(stereo); // this statement
                // is width 2 
                // --> error
}
Unresolved Output Width

The output width rules work well for most SAOL coding styles. The right panel shows two instruments that are exceptions. In these examples, the rules do not resolve the width of the instruments.

The first example, self_ref, uses arrays of width outchannels in its output statement. This coding method lets programmers create instruments that have an arbitrary output width. As a direct consequence, however, the output width rules fail to resolve a width value.

The second example, no_output, has no output statements, but uses the standard name outchan in its code. The output width of the instrument is needed so that outchan may take on its value, but the rules fail to resolve a value.

In cases like these examples, SAOL programmers may code the output width in an explicit way, via global block code. We describe this binding method in a later section of the chapter.

Well-written programs use this binding method for instruments that do not resolve. If an instrument width is left unresolved, the actual width of the instrument is not normative: two decoders may behave in different ways in response to the same code.

Unresolved Example 1


instr self_ref() {

  asig a[outchannels], i;


  // all statements a-rate

  i = 0;

  while (i < outchan)
   {
     a[i] = arand(1);
   }

  output(a); // width ???
}

Unresolved Example 2


instr no_output() {

  exports ksig osize;

  osize = outchan; // width ???
}

Simple Audio Output

In this section, we describe audio output operation for programs that have no send or route statements. The right panel shows an example program of this type.

Every SAOL program includes an output_bus. The outchannels global parameter sets the width of the output_bus (the default width is one). There is also a storage location for the final audio output of the system, that is the same width as the output_bus.

At the start of each a-pass, all channels of the output_bus are set to zero. The final audio output of the system is also set to zero. As each active instrument instance completes its a-pass, the final value of its output port is added to the output_bus.

If the output port of an instrument has a width greater than one, its width must match the output_bus width, and the output port is added to output_bus on a channel-by-channel basis. If the audio output port of an instrument has width one, this output port value is added to each channel of the output_bus.

After all instances execute, the output_bus is summed onto final audio output of the system. The final audio output is clipped to fall in the range -1.0 to +1.0, and is then usually sent to a file (for storage) or to a D/A converter (for listening).

Note that no information flows from one a-pass to the next on the output_bus, as the output_bus is initialized to zero at the start of each a-pass.

Example


global {
outchannels 2;  // stereo output
}


instr mono () {

asig a;

output(a);
}


instr stereo () {

asig b[2];

output(b);
}


// output_bus at end of a-pass:
//
// 0: a + b[0], clipped to [-1,1]
// 1: a + b[1], clipped to [-1,1]

Effects Instruments

An instrument instance that process audio input is called an effects instrument. Effects instruments are created at the start of program execution, by send statements in the global block.

Effects instrument instances have an audio input port that holds the input values for the instance. The audio input port is specified in the send statement as a set of buses, as we describe in the next section. The audio input port has a width determined by these bus widths.

At the start of an a-pass, the buses that make up the audio input port of an effects instance are set to zero. As the a-pass proceeds, other instruments may sum onto these buses.

When the a-pass of an effects instance begins, the value of its audio input port is copied into the a-rate array standard name input. We describe this process in more detail in the next section.

The standard name inchan holds the width of the audio input port. The array width specifier inchannels also denotes the width of the audio input port; it may be used to declare other arrays in the effects instrument.

An effects instrument typically processes the value held by the input standard name, and writes a modified version to its audio output port.

See the right panel for an example of an effects instrument.

Example


// effects instrument
// scales the audio input port

instr scale(sfactor) 

{
  ivar i;

  // inchannels width of port
  ivar w[inchannels];

  /****************
  /* runs at i-rate
  /****************
  
  i = 0;
  while (i < inchan)
   {
     w[i] = sfactor;	
     i = i + 1;
   }

  /****************
  /* runs at a-rate
  /****************

  // input holds audio input port

  output(w*input);
}

Send Statements

A send statement instantiates an effects instrument. It specifies the parameter initialization for the instrument, and declares the structure of its audio input port.

A send statement may only appear in a global block. It executes at the start of a simulation, after the i-pass of the startup instrument. The send statement argument list has three sections, that are separated by semicolons.

The first section specifies the name of the instrument to be instantiated. The send statement in the example on the right panel instantiates the instrument scale (defined in the right panel of the last section).

The second section is an i-rate expression list, whose values initialize the instrument parameters of the instance. In this example, the parameter sfactor of this instance of scale is initialized to 1.2. The number of instrument parameters must match the number of expressions in the list.

The third section specifies the audio input port for the instrument as a list of one or more buses. Usually these buses are user-defined buses. Like the output_bus, a user-defined bus is initialized to zero at the start of each a-pass, and instruments may sum onto it during its a-pass.

User-defines buses have a width. Bus width may be specified using array declaration syntax, as bus2 in the example shows. Like a global array declaration, the bus width may be a positive integer, or the tokens inchannels or outchannels. These tokens declare buses with the width of the input_bus or output_bus system buses.

User-defined buses may also be declared without a width, as bus1 in the example shows. In this case, the bus width is inferred from the widths of instruments that write to it, as we detail in the next section of this chapter.

The width of the audio input port is the sum of the widths of the user-defined buses in the third section of its send statement. The value of the audio input port is the concatenation of the values on these buses, and is copied into the standard-name array input as detailed in the last section.

All user-defined buses must appear in the third section of at least one send statement.

Example

global {

ivar i1; // set to 4 by startup()

send(scale; 0.3*i1; bus1, bus2[3]);  

}


// if an instr named startup exists,
// it runs at the start of program.
// execution.

instr startup() {

exports ivar i1;

i1 = 4;

}

Route Statements

The default behavior of an instrument is to sum its audio output port to the system output_bus. The route statement changes that default behavior, and redirects the output of an instrument to a user-defined bus. A route statement may only appear in the global block.

The set of route statements that target a user-defined bus set the implicit width for the bus. This width must be compatible with any explicit width declarations that appear in send statements.

A route statements consists of a bus name, followed by a list of instruments. See the right panel for statement syntax. An instrument in the list sums its audio output port onto a section of the targeted bus. The width of the route statement is the sum of the audio output port widths of the listed instruments.

Several route statements may target the same bus. An instrument may be in the instrument list of several route statements, and may appear several times in the same route statement.

If a send statement has declared a bus to have width N, all route statements that target the bus must have width N or width one. This rule also applies to route statements that target the output_bus, whose width is set by the global parameter outchannels.

If a send statement does not provide an explicit width for a bus, these rules describe the width semantics for route statements targeting the bus:

  1. All route statements targeting a bus either must have width one or must have the same non-scalar width.
  2. A user-defined bus has the width of the widest route statement targeting it.

Bus Writing Behavior

If a route statement has width one (and thus has only one instrument in its list), this instrument writes its audio output port value to every channel of the targeted bus.

Otherwise, the first instrument on the list (that has width w1) adds its audio output bus value to the first w1 channels of the targeted bus, the second instrument on the list (that has width w2) adds its audio output bus to the next w2 channels of the targeted bus, etc. In this way, each instrument sums onto its own section of the bus.

Note that a route statement redirects all active instances of an instrument. There is no way in SAOL to route difference instances of the same instrument to different buses.

Unresolved Instrument Widths

A route statements may also act to bind instruments with unresolved widths (as described in a previous section). To bind an unresolved instrument, use a route statement to route it to a bus whose width is known (for example, a bus whose width is declared in a send statement, or the output_bus).

Syntax


route(busname, inst1 [,inst2, ...]);

Example (for next section)


global {

outchannels 2;
route(drybus, left, right);
send(rvb; ; drybus);
route(rvbus, rvb);
send(mix; 0.2, 1 ; rvbus, drybus);

}


instr left()

{
  output(arand(0.2));
}

instr right()

{
  output(arand(0.2));
}

instr rvb() 

{
  output(
        reverb(input[0]+input[1],5));
}

instr mix(rev, dry)

{
  asig out[2];

  out = rev*input[0];
  out[0] = out[0]+ dry*input[1];
  out[1] = out[1]+ dry*input[2];
  output(out);
}

Signal Flow Graphs

A set of send and route statements act together to map out a diagram. The instances created by send statements, along with instruments instanced by SASL lines, create the boxes in this diagram. The route statements draw lines between the boxes, via buses.

The right panel shows the graph implied by the example code shown in the right panel of the previous section. In this example, the non-effects instruments left and right are sent through the effects instrument rvb, and the dry and wet outputs are combined in the mix effects instrument.

The graph of this program suggests the correct execution order for the different instruments during the a-pass.

  1. The left and right instruments should run, and add their outputs onto sections of the drybus.
  2. The rvb should execute, adding its output onto the rvbus.
  3. The mix should run and add its output to the output_bus.

Note that if a different order is used, different sound would be produced. For example, if steps 2 and 3 were reversed, the initial zero value of the rvbus bus would be processed by mix, since mix would run before rvb.

SAOL examines the send and route statements to deduce the best execution order of the instruments in the program. In the next section, we describe the rules SAOL uses for execution order, and how to override them.

Graph of the Example Above

--------     ---------
| left |     | right |
--------     ---------
   |             |
   |             |
   |             |
**************************drybus
   |               |      (w=2)
   |               |
   |               |
-------            |
| rvb |            |
-------            |
   |               |
   |               |
   |               |
***********rvbus   |
       |   (w=1)   |
       |           |
       |           |
     ------------------
     |      mix       |
     ------------------
             |
             |
             |
******************** output_bus
                     (w=2)

Determining Execution Order

The right panel shows the four rules for determining the execution order of instruments in a SAOL program.

The first rule codifies the idea that instruments that produce a signal should run before instruments that use the signal. The second rule handle cases where the first rule breaks down because of loops in graph. The final two rules deal with special instruments.

If the ordering between two instruments is not specified by these rules, the instruments may run in any order.

Applying these rules to the example in the previous section, rule 1 dictates that instrument rvb runs after instruments left and right, but before instrument mix. The execution order of left and right is left unspecified.

The ordering specified by these rules may be overridden by the sequence statement. A sequence statement has a list of instruments as parameters. The order of instruments in the list sets the order of their execution.

The right panel shows the syntax of the sequence statement, and a sample sequence command that sets an explicit ordering for the left and right instruments of the example above.

Sequence statements may only be used in the global block. The set of sequence statements in a global block may not specify an ordering loop.

Execution Order Rules

  1. If instrument 1 is routed to a bus which is sent to an effects instrument 2, instrument 1 executes before instrument 2.
  2. If loops are created by send and route statements, the order of the send statements in the global block defines the order of the instances created by the send statements. A send statement that creates the backward part of a loop does not execute.
  3. Instances of the startup instrument execute first.
  4. If a send statement has the output_bus in its bus list, the instrument instanced by the send statement executes last. This instrument has special semantics, which we describe in a later section.

Sequence Syntax

sequence(inst1, inst2 [,inst3 ...]);

Sequence Example

sequence(left, right);

Applying Execution Order

The execution order of instruments is used in several ways in a SAOL program.

Width Assignment

The execution order is used in the process of determining the width of user-defined buses and of instrument audio input and output ports. See the right panel for details.

Recall that in earlier sections, we described the process of determining the width of audio input and output ports and of user-defined buses. The right panel defines the correct behavior in unusual "edge cases" where looped signal graphs make the width of buses and instruments depend on ordering.

Run-Time Ordering

In the tutorial introduction, we describe how SAOL simulated time is broken up into a-cycles and k-cycles. We show that the a-pass code for an instrument runs during an a-cycle and the i-pass and k-pass code for an instrument runs during a k-cycle.

The execution order defines the order that the i-pass and k-pass code of each instrument in a program runs within a single k-cycle, and defines the order that the a-pass code of each instrument runs within a single a-cycle.

The only exception to this ordering happens in the case of the startup instrument. If an instrument named startup exists in the program, an instance of this instrument is created before the first k-cycle begins, and its i-pass is immediately run.

This exception allows global i-rate variables to be initialized by the startup instrument, so that i-rate expressions in the global block can use these variables during SAOL program initialization. See Part III/3 for the exact timing of the i-pass execution of the startup instrument.

Note that the execution order define an ordering for instruments within a single k-cycle or a-cycle, but not between successive k-cycle or a-cycles.

For example, to execute a program consisting of two instruments for five consecutive a-cycles, it is legal to execute all five a-cycles for the first instrument in the execution order, followed by five a-cycles for the second instrument. This behavior is legal since the order within each a-cycle is preserved.

The SAOL language is designed so that program semantics are not affected by this type of inter-cycle re-ordering. For example, the semantics of import and export (as explained in an earlier chapter) set a deterministic method for sharing global variables that is not affected by inter-pass instrument ordering.

Width Assignment

Instrument width assignment is done in execution order. The following steps are repeated for each instrument as it appears in the execution order.

  1. If the instrument is not a target of a send statement, its audio input port width is set to 1. If an instrument is a target of a send statement, the width of its audio input port is determined by examining the send statement, as described in an earlier section of this chapter.
  2. Using the rules described in an earlier section, we determine the width of the output port for the instrument. If an width of the output port is unresolved, use the method in this earlier section to resolve its width, if possible.
  3. Any route statement whose instruments all have a defined output port width may set the width of its targeted bus, and detect any width mismatch syntax errors, as described in an earlier section.

At the start of the process, a subset of buses have a known width: the input_bus, the output_bus, and all buses whose widths are declared in send statements. As the algorithm processes each instrument, step 3 acts to define the width of the remaining buses.

The results of the algorithm are unspecified if step 1 fires for a send statement before its buses have defined widths. SAOL programmers should explicitly define bus widths, if necessary, to avoid this issue.

System Buses

In this section, we formally define the behavior of the system buses input_bus and output_bus. The input_bus is a system bus that lets SAOL programs process external audio streams, such as WAV files and live microphones. The output_bus holds the final audio output of the SAOL decoder.

Bus Widths

The global parameters inchannels and outchannels may be used to set the width of the input_bus and output_bus, respectively. See the right panel for an example.

If the global parameter inchannels is not set, the input_bus has the width of the external audio stream. If no audio stream is provided, the input_bus has a width of zero. If the global parameter outchannels is not set, the output_bus has a scalar width.

Referencing Widths

In the global block, an array may be declared that has the same width as the input_bus or output_bus, by using the tokens inchannels or outchannels respectively. The width of the systems buses may be used in expressions in the global block, by using the standard names inchan and outchan. See the right panel for examples.

Note that inchannels, outchannels, inchan and outchan serve a different role in instrument code. In an instrument, these tokens and standard names code instrument input and output width, not the width of input_bus and output_bus.

SAOL does not provide direct access to the width of the input_bus or output_bus in instrument code. However, elements of a global block table may be initialized with inchan and outchan; these global variables may then be imported into instrument code. Similar tricks with global block calls to user-defined opcodes (described in Part IV/4) may be used to initialize global scalar variables with the system bus widths.

Sampling Rates

The global parameter srate sets the audio sampling rate of the program. If this parameter is not set, the audio sampling rate is set to the sampling rate of the external audio input. If no audio input is provided, the audio sampling rate is 32,000 cycles per second.

Route and Send Statements

The input_bus may not be the target of a route statement, but may be used in a send statement.

The output_bus may be the target of a route statement. If the route statement has non-scalar width, the width of the route statement must match the width of the output_bus.

An output_bus used in a send statement has special semantics, as detailed below.

Semantics

At the start of an a-pass, the input_bus is initialized with audio samples from the external audio source. The input_bus may be sent to effects instruments using a send statement. Since the input_bus may not be target of a route statement, its value is unaltered throughout the a-pass.

At the start of an a-pass, the output_bus is initialized to zero. All programs have a storage location for the final audio output of the system. At the start of an a-pass, final audio output of the system is initialized to zero.

During the a-pass, instruments that have not been redirected to user-defined buses via the route statement sum onto the output_bus. In addition, route statements may also target the output_bus.

If the output_bus is not used in a send statement, then at the end of the a-pass, the values of the output_bus sums onto the final audio output of the system. In this case, the width of the final audio output is the width of output_bus.

If the output_bus is used in a send statement, then the output of the instance created by that send statement sums onto the final audio output of the system. In this case, the width of the final audio output is the audio output port of the this instance. The output_bus may only be used in a single send statement.

The final audio output is clipped to fall in the range [-1,1] and is then usually sent to a file (for storage) or to a D/A converter (for listening).

Example


global {

// sets system bus width

inchannels  1;
outchannels 2;

// array widths match system buses

ksig inscale[inchannels];
ksig outscale[outchannels];

// using input_bus in a send
// using inchan and outchan

send(rvb; inchan + outchan; input_bus);

// output_bus in a send statement

send(scale; 0.5 ; output_bus);

}

Standard Names

Throughout the chapter, we have referred to standard names that are used in the audio system. The right panel lists these standard names for reference.

The input array may be used only in an instrument, and holds the value of the audio input port. It's companion array, inGroup, codes the bus structure of the send statement that created the instance. See the code examples on the right panel for the semantics of inGroup.

In an instrument, inchan and outchan refer to the width of the audio input port and the audio output port of the instrument. In the global block, inchan refers to the width of the input_bus, and outchan refers the width of output_bus.

Bus Standard Names

ivar inchan;
asig input[inchannels];
ivar inGroup[inchannels];
ivar outchan;

InGroup Examples


// width of bus1: 4
// width of bus2: 2
// width of bus3: 1

send(test1; ; bus1);

// inside test1, standard name
// inGroup is width 4, and has 
// the value [1 1 1 1]

send(test2; ; bus2, bus3);

// inside test2, standard name
// inGroup is width 3, and has 
// the value [1 1 2]

send(test3; ; bus1, bus2, bus3);

// inside test3, standard name
// inGroup is width 7, and has 
// the value [1 1 1 1 2 2 3]

outbus and spatialize

The output statement is the primary way for an instrument to generate audio output. In this section, we describe two other a-rate statements that may generate audio output, the outbus and spatialize statements.

outbus

The outbus statement adds a value directly to a bus. This statement is useful for creating secondary bus structures that are independent from the main signal path flowing through the audio input and output ports of the instruments. These secondary buses may play a role similar to effects and monitor buses on audio mixing consoles.

The first parameter of the outbus statement is the bus name, and the remaining parameters are a list of expressions to add to the bus. The outbus statement has a width, which is the sum of the widths of its expression parameters. The right panel shows the syntax of the statement.

If the outbus statement has a scalar width, the value of the expression is added to each channel of the named bus. Otherwise, the width of the outbus statement must match the width of the named bus, and the array value formed by the concatenation of the expressions in the outbus statement is added to the named bus.

The outbus statement does not create new buses, and does not play a role in determining the width of buses. The statement can only write to buses that already exist: the output_bus, or user-defined buses created in send statements. If only outbus statements write to a user-defined bus, the send statement that creates the bus should declare the bus width explicitly.

An outbus statement may not appear in an instrument that is instantiated by a send statement that uses the output_bus. The outbus statement may not target the special system input_bus.

outbus

outbus(busname, exp1 [,exp2, ...]);

spatialize

The spatialize statement places a monophonic sound in the 3-dimensional space surrounding a listener. The spatialize statement bypasses the SAOL bus system entirely, and adds audio sample data to the final audio output.

The right panel shows the syntax of the spatialize statement. The first parameter to spatialize is a scalar expression of the monophonic sound, and may be a-rate or slower.

The remaining parameters describe the position the monophonic sound should be placed in space. These parameters may be k-rate or slower.

The second parameter codes the azimuthal angle where the sound should be placed, in radians. Zero degrees is directly in front of the listener, and pi/2 is to right of the listener.

The third parameter codes the elevation angle where the sound should be placed, in radians. Zero degrees is in the horizontal plane of the listener, and pi/2 is directly above the listener.

The fourth parameter codes the distance the sound is from the listener, in meters.

All spatialize statements sum onto the final audio output of the system. As usual, the output_bus (or the output of the instance that is sent the output_bus) also sums onto the final audio output of the system.

The final audio output is clipped to [-1,1] at the end of the a-pass, and is then usually sent to a file (for storage) or to a D/A converter (for listening).

The method the spatialize statement uses to place sound is non-normative. The final audio result may sound different on different decoders.

spatialize

spatialize(audio, azimuth, 
           elevation, distance);

Summary

This chapter is the last chapter of Part II. In Part II, we have described most of the SAOL language in a detailed way.

Several statements and standard names that involve the control of instruments have been postponed until Part III/3, since a complete knowledge of SASL (Part III/1) and MIDI (Part III/2) is necessary to understand these structures.

In addition, we postpone a description of user-defined opcodes until Part IV/4. Part IV specializes in opcodes, and includes a complete description of the core opcode library, as well as a few language structures specialized for opcodes.

We also postpone a discussion of templates, a language feature for constructing families of instruments that share common code, until Part V/2.

Next: Part III: Instrument Control
mp4-sa-> the mp4-sa book-> SAOL->buses and execution order

Copyright 1999 John Lazzaro and John Wawrzynek.