CS 312 - Week 9.2 Class
CS 312 Audio Programming Winter 2020

Table of Contents


9.2 Class

In progress. Final web page release sometime Thursday 03/06.

Envelopes

The term envelope in digital audio is generally applied to the overall amplitude shape of a single musical note. Performers on wind and string instruments like the the Oboe and Cello can control the shape of the amplitude envelope. Performers on percussion and plucked or struck strings like the piano, guitar, and snare drum have no control over the shape of the amplitude envelope. It's built into the instrument.

The envelopes of three common instruments are outlined in red.

  • Cello

env_cello.png

  • Oboe

env_oboe.png

  • Guitar

env_guitar.png

The outline of the envelope is converted into amplitude samples from 0-1.0 using the same number of samples as the generated waveform. When the waveform and envelope are multiplied sample by sample the amplitude of the waveform is forced to follow the shape of the envelope.

hw921_plot_adsr

The Attack-Decay-Sustain-Release (ADSR) envelope is the most common audio synthesis envelope and is found in both hardware and software synths. We'll be developing the ADSR envelope using line segments. More sophisticated ADSR's would use natural growth and natural decay exponential curves instead of straight lines. We'll develop a natural decay envelope in hw817.

adsr.png

Setup

cd $HOME312
mkdir hw92

Download this file

hw921_plot_adsr.zip

Unzip the file and move the hw921_plot_adsr folder into the hw92 folder.

You should see these files.

qt92102.png

Open in Qt Creator
Set the build folder to build-debug

Build and run
The GUI appears but only some controls are connected to code.

qt92103.png

GUI controls

The ADSR slider controls in the GUI return values needed for the four segments.
The A, D, and R sliders return time in milliseconds.
The S slider returns the amplitude value, 0-1.0, that's maintained throughout the sustain state.

A
The Attack portion is assumed to begin at sample[0].
The slider value is the time in milliseconds from Attack start to Attack end.
The slider control min/max range is 1-100.
The slider value label needs to display a range of 10-1000 ms.

D
The Decay portion is assumed to begin at sample[attack end + 1].
The slider value is the time in milliseconds from Decay start to Decay end.
The slider control min/max range is 1-100.
The slider value label needs to display a range of 10-1000 ms.

SL
The Sustain Level portion is assumed to begin at sample[decay end + 1].
The slider value is the amplitude value (0-1.0) for the entire Sustain portion.
The slider control min/max range is 0-100.
The slider value label needs to be scaled to a range of 0-1.0.

R
The Release portion is assumed to begin at sample[Sustain end + 1].
The slider value is the time in milliseconds from Release start to Release end.
The slider control min/max range is 1-100.
The slider value label needs to display a range of 10-1000 ms.

Start Note
Sets the MIDI note number of the leftmost "piano key".

Volume
The volume control slider affect the volume of the sine wave generated by the "piano keys".
It is independent of the ADSR values.

Stop Button
Stop the dac stream.

Quit Button Closes the dac stream if it is open. Calls QApplication::quit();

Piano Keys
Each Tool Button widget implements these two slots.

qt92101.png

Every pressed() function looks like this.
You'll need to modify the playNote parameter for each key.

void MainWindow::on_toolButton_Eb_pressed() // Eb is E flat
{
    envAmp = 0;
    isKeyDown = true;
    isKeyUp = false;
    count = 0;
    playNote( MIDIstartNote + 3 );
}

The playNote() function looks like this.

void MainWindow::playNote( const int note )
{
    if ( dac.isStreamRunning() )
        dac.stopStream();
    adsrState = kAtkState;
    isKeyDown = true;
    isNoteOver = false;
    calcADSR_start_end_points();
    freq = midi2frequency( note );
    plotEnvelope();
#if USECALLBACK
    dac.startStream();
#endif
}

Every release() function looks like this.

void MainWindow::on_toolButton_Eb_released()
{
    isKeyUp = true;
    isKeyDown = false;
    stopNote();
}

The stopNote() function looks like this.

void MainWindow::stopNote()
{
#if USECALLBACK
    if ( isNoteOver )
        dac.stopStream();
#endif
}

State Machine

The ADSR envelope can be in one of four states as the sine wave (or plot) is generated.

ADSR states
Attack
Decay
Sustain
Release

Calculating start and end times for each state

  • The Attack, Decay, and Release states are determined by their length in milliseconds translated into number of samples.
  • The Sustain state is an unchanging amplitude value.
  • The Attack state starts at 0, steadily increases to 1.0 over the number of samples in the Attack portion.
  • The Decay state starts at one sample past the Attack end and steadily decreases from 1.0 to the Sustain level over the number of samples in the Decay portion.
  • The Sustain state is indeterminate and starts at one sample past the Decay end and maintains a Sustain amplitude level until a key released message is sent.
  • The Release state starts at one sample past the end of the Sustain portion and steadily decreases from the Sustain level to zero over the number of samples in the Release portion. When the Release portion ends it sets a "note over" flag.

Variables needed

Declared in mainwindow.h.

// In one millisecond there are 44100/1000 samples.
const double kSamplesPerMs = 44.1;
// Implementing a state variable machine with four states
const int kAtkState = 0;
const int kDcyState = 1;
const int kSusState = 2;
const int kRlsState = 3;
// Keep track of current state
extern int adsrState;

// Envelope Attack starts with isKeyDown
extern bool isKeyDown;
// Envelope Release starts with isKeyUp
extern bool isKeyUp;
// end of Release portion sets isNoteOver = true
// sineADSRcallback returns callbackReturnValue with isNoteOver
extern bool isNoteOver;

// initial startup values
// amplitude level for steady state Sustain
extern MY_TYPE lvlS;
// frequency of first piano key button (Middle C)
extern MY_TYPE freq;
// value is calculated for every sample in each ADSR state
extern MY_TYPE envAmp;
// Maximum amplitude of sine wave generated by isKeyDown
extern MY_TYPE sliderVolume;
// count of every sample during all callbacks
extern int count;

// Time in milliseconds for the Attack, Decay, and Release
// ADSR segments based on their slider values
extern int msA;
extern int msD;
extern int msR;

// Calculations where each ADSR segment begins and ends
// in sample indices based on ms time of slider settings
extern int sampStartA;
extern int sampEndA;
extern int sampStartD;
extern int sampEndD;
extern int sampStartS;
extern int sampLenR;
extern int sampStartR;
extern int sampEndR;

// Calculation of the increment needed to take the
// Attack portion from 0-1 over length in samples
// of Attack segment
extern double attackInc;
// Calculation of the increment needed to take the
// Decay portion from 1 down to Sustain level over
// length in samples in Decay segment
extern double decayInc;
// Calculation of the increment needed to take the
// Release portion from Sustain level to 0 over
// length in samples of Release segment after isKeyUp.
extern double releaseInc;

// The number of x axis points in the ADSR plot
const int kEnvLength = 88200; // 2 seconds
// vector to hold the y axis values for the ADSR plot
extern std::vector<MY_TYPE> venv;

You'll need to implement two state machines, one for the ADSR plot and one inside the sine callback function. Both state machines use this structure.

std::vector<MY_TYPE> MainWindow::stuffEnvelopeVector()
{
    for ( uint32_t n = 0; n < last_sample; n++ )
    {
        // Attack phase
        if ( count <= Asamps )
        {
             // do stuff
             // set kDcyState when finished
        }
        // Decay phase
        else if ( adsrState == kDcyState )
        {
             // do stuff
             // set kSusState when finished
        }
        // Sustain phase
        //        else if ( count > sampEndD )
        else if ( adsrState == kSusState )
        {
             // do stuff
             // set kRlsState when finished
        }
        // Release
        else if ( adsrState == kRlsState )
        {
             // do stuff
             // set a "note over" flag when finished
        }
        count++;
    }
    return venv;
}

The Sustain length is known ahead of time for the ADSR plot.
The Sustain length is not known inside the sine callback until the piano key button is released.

Implement the ADSR plot

#define USECALLBACK 0

This #define is near the top of the source file and should be set to zero while you're implementing the ADSR plot. Once the ADSR plot working set it to 1 and begin implementing the state machine inside the sine callback function.

Task 1

Implement calcADSR_start_end_points()

void MainWindow::calcADSR_start_end_points()
{
    /*
        calculate sustain level from slider value
        lvlS = ui->verticalSlider_SL->value() * .01;

        calculate millisecond A D R times from slider value * 10
        msA = ui->verticalSlider_A->value() * 10;

        If you know ms duration there are 44100/1000 samples per millisecond
        calcualate attackInc = 1.0/(msA * kSamplesPerMs)


        calcualate decayInc = (1.0 - lvlS)/(msD * kSamplesPerMs)
        calcualate releasekInc = (lvlS - 0)/(msR * kSamplesPerMs)

        calculate start/end values for A D in samples
        sampStartA = 0;
        sampEndA =  msA * kSamplesPerMs
        sampStartD = end A + 1
        sampEndD = start D + number samples in D
        sampStartS = end D + 1
        NOTE: sampEndS is calculated elsewhere
            for plot it's totalPlotSamples - (A length + D length + R length)
            for sine callback its when key button is released.
        sampLenR = msR * kSamplesPerMs

*/
    // Useful for debugging
    qDebug() << "A start end" << sampStartA << " " << sampEndA;
    qDebug() << "D start end" << sampStartD << " " << sampEndD;
    qDebug() << "S start lvl" << sampStartS << " " << lvlS;
    qDebug() << "R start lvl" << sampLenR;
    // myqDebug() defined at top to display doubles with using scientific notation
    myqDebug() << "attackInc decayInc releaseInc" << attackInc << " " << decayInc << " " << releaseInc;
}

The qDebug() and myqDebug() output will appear in the Application Output panel.

qt92104.png

Task 2

Implement stuffEnvelopeVector()
Note: This is the state machine for the ADSR plot.
The state machine for the sineADSRcallback is similar but requires modifications to the sustain and release segments.

std::vector<MY_TYPE> MainWindow::stuffEnvelopeVector()
{
    calcADSR_start_end_points();

    // Calculate Slength as Slength = kEnvLength - ( sampEndD + sampLenR )

    MY_TYPE samp{0};
    int count{0};

    for ( uint32_t n = 0; n < kEnvLength; n++ )
    {
        // This what I did for attack state
        if ( count <= sampEndA )
        {
            samp += attackInc;
            if ( samp >= 1.0 )
            {
                samp = 1.0;
                adsrState = kDcyState;
            }
            venv.push_back( samp );
        }
        // Decay phase
        else if ( adsrState == kDcyState )
        {
            // decrement samp by decayInc
            // if samp < lvlS
            //    set samp = lvlS
            //    and set adsrState to kSusState
            // venv.push_back( samp );
        }
        // Sustain phase
        else if ( adsrState == kSusState )
        {
            // while Slength > 0
            // Slength--
            //    venv.push_back( samp );
            //    nd set adsrState to kRlsState
        }
        // Release
        else if ( adsrState == kRlsState )
        {
            // decrement samp by releaseInc
            // if samp < 0
            //    set samp = 0
            //    and set adsrState to kDcyState
            // venv.push_back( samp );
        }
        count++;
    }
    return venv;
}
Task 3

Implement plotEnvelope()

void MainWindow::plotEnvelope()
{
    /*
        clear venv
        call stuffEnvelopeVector()
        create QVector<double> x and QVector<double> y using kEnvLength as size
        for loop to set x and y values from 0 < kEnvSize
            x are loop indices
            y are venv values
        customPlot stuff you've used before
            addGraph
            setData
            x axis setRange
            y axis setRange
            replot
        reset adsrState to kAtkState becuase it was changed in stuffEnvelopeVector()
      */
}

At this point the plot should appear.

Task 4

Remember to
#define USECALLBACK 1

Implement the sineADSRCallback function

// Remember to change callback name in open_dac_stream()
int MainWindow::sineADSRCallback( void* outputBuffer, void* /*inputBuffer*/, unsigned int nBufferFrames,
                                  double /*streamTime*/, RtAudioStreamStatus status, void* /*userData*/ )
{
    //    if ( envAmp < 0 )
    //        return 0;

    if ( isNoteOver )
        return callbackReturnValue;

    MY_TYPE* buffer = static_cast<MY_TYPE*>( outputBuffer );
    if ( status )
        std::cout << "Stream underflow detected!" << std::endl;
    static MY_TYPE phz = 0;
    // //phase increment formula
    const MY_TYPE phzinc = k2PIT * freq;
    for ( uint32_t i = 0; i < nBufferFrames; i++ )
    {

        // implement state machine for calculating envAmp for each sample
        // Attack phase
        if ( adsrState == kAtkState )
        {
            // increment envAmp by attackInc
            // if envAmp >= 1.0
            //      set envAmp to 1.0
            //      and set adsrState to kDcyState
        }
        // Decay phase
        else if ( adsrState == kDcyState )
        {
            // decrement envAmp by decay
            // if envAmp < sustain level (lvlS)
            //      set envAmp to lvlS
            //      and set adsrState to kSusState
        }
        // Sustain phase
        else if ( adsrState == kSusState )
        {
            // set envAmp to lvlS
            //      check for isKeyUp
            //      if true set adsrState to kRlsState
        }
        // Release
        else if ( adsrState == kRlsState )
        {
            // set sampStartR to count
            // set sampEndR to sampStartR + sampLenR
            // decrement envAmp by releaseInc
            // if envAmp < 0
            //      set envAmp = 0
            //      and set isNoteOver to true
        }

        *buffer++ = sliderVolume * envAmp * sin( phz );

        phz += phzinc;
        if ( phz >= k2PI )
            phz -= k2PI;
        ++count;
    }
    rta.frameCounter += nBufferFrames;
    if ( rta.checkCount && ( rta.frameCounter >= rta.nFrames ) )
        return callbackReturnValue;

    QApplication::processEvents();

    // Usefull for debuging
    //    myqDebug() << count << " " << adsrState << " " << isKeyUp;
    return 0;
}
Task 5

Hook up the piano key buttons. See sample code earlier on this page.

Build and run

Reducing "pops and clicks" between key presses
Shorten the Release time and/or Attack time and/or Decay time.

Assignment 921

Make it work like mine.

./images/zip/hw921_ADSR.pro.app.zip

Submission

Feel free to email jellinge@carleton.edu on any part of the homework that is unclear to you. Chances are if you have questions other students do too. I will answer those questions by email to everyone in the class. The original sender will remain anonymous. I have tried to provide clues to the homework in the web pages, reading assignments, and labs. Sometimes what seems obvious to me is not obvious to students just learning C++.

Create a folder named hwNN_LastnameFirstname_LastnameFirstname.
Substitute your name and your partner's name for LastnameFirstname_LastnameFirstname.

Remember

  • Boilerplate header at top of every file.
  • Make sure your program compiles before submitting.
  • Programs that compile and produce partially correct output will be graded.
  • You can send notes to me in your homework files by enclosing them in block comments.
  • Programs that do not compile get an automatic F (59%).
  • Submit the homework in only one of the partner's folders.
  • Empty your build or build-debug folders using the command emptyBuildFolder.sh
  • Only include .h, .cpp, Makefiles, CMakeLists.txt
  • Do not include these folders/files
    .git
    .vscode
  • Double check for the above two folders using this command

    ls -a hwNN_LastnameFirstname1_LastnameFirstname2
    # if either .git or .vscode exist in the folder you're submitting remove them with
    # rm -fR .git
    # rm -fR .vscode
    

Hand-in folder contents

/common
├── RtAudio
│   ├── RtAudio.cpp
│   └── RtAudio.h
├── RtMidi
│   ├── RtMidi.cpp
│   └── RtMidi.h
├── hw332_CMidiPacket.cpp
├── hw332_CMidiPacket.h
├── hw411_rand_int.cpp
├── hw411_rand_int.h
├── hw421_CDelayMs.cpp
├── hw421_CDelayMs.h
├── hw422_CAppleMidiSynth.cpp
├── hw422_CAppleMidiSynth.h
├── hw423_CMidiTrack.cpp
├── hw423_CMidiTrack.h
├── hw511_CInstrument.cpp
├── hw511_CInstrument.h
├── libsndfile
│   ├── sndfile.h
│   └── sndfile.hh
└── qcustomplot
    ├── qcustomplot.cpp
    └── qcustomplot.h

/hw92
└── hw921_plot_adsr
    ├── build-debug (empty)
    ├── hw921_plot_adsr.pro
    ├── main.cpp
    ├── mainwindow.cpp
    ├── mainwindow.h
    ├── mainwindow.ui
    ├── rtaudioutils.cpp
    └── rtaudioutils.h

Author: John Ellinger

Created: 2020-03-09 Mon 13:18

Validate