CS 312 - Week 9.2 Class
CS 312 Audio Programming Winter 2020
Table of Contents
9.2 Class
In progress. Final web page release sometime Thursday 03/06.
Envelopes
The term envelope in digital audio is generally applied to the overall amplitude shape of a single musical note. Performers on wind and string instruments like the the Oboe and Cello can control the shape of the amplitude envelope. Performers on percussion and plucked or struck strings like the piano, guitar, and snare drum have no control over the shape of the amplitude envelope. It's built into the instrument.
The envelopes of three common instruments are outlined in red.
- Cello
- Oboe
- Guitar
The outline of the envelope is converted into amplitude samples from 0-1.0 using the same number of samples as the generated waveform. When the waveform and envelope are multiplied sample by sample the amplitude of the waveform is forced to follow the shape of the envelope.
hw921_plot_adsr
The Attack-Decay-Sustain-Release (ADSR) envelope is the most common audio synthesis envelope and is found in both hardware and software synths. We'll be developing the ADSR envelope using line segments. More sophisticated ADSR's would use natural growth and natural decay exponential curves instead of straight lines. We'll develop a natural decay envelope in hw817.
Setup
cd $HOME312 mkdir hw92
Download this file
Unzip the file and move the hw921_plot_adsr folder into the hw92 folder.
You should see these files.
Open in Qt Creator
Set the build folder to build-debug
Build and run
The GUI appears but only some controls are connected to code.
GUI controls
The ADSR slider controls in the GUI return values needed for the four segments.
The A, D, and R sliders return time in milliseconds.
The S slider returns the amplitude value, 0-1.0, that's maintained throughout the sustain state.
A
The Attack portion is assumed to begin at sample[0].
The slider value is the time in milliseconds from Attack start to Attack end.
The slider control min/max range is 1-100.
The slider value label needs to display a range of 10-1000 ms.
D
The Decay portion is assumed to begin at sample[attack end + 1].
The slider value is the time in milliseconds from Decay start to Decay end.
The slider control min/max range is 1-100.
The slider value label needs to display a range of 10-1000 ms.
SL
The Sustain Level portion is assumed to begin at sample[decay end + 1].
The slider value is the amplitude value (0-1.0) for the entire Sustain portion.
The slider control min/max range is 0-100.
The slider value label needs to be scaled to a range of 0-1.0.
R
The Release portion is assumed to begin at sample[Sustain end + 1].
The slider value is the time in milliseconds from Release start to Release end.
The slider control min/max range is 1-100.
The slider value label needs to display a range of 10-1000 ms.
Start Note
Sets the MIDI note number of the leftmost "piano key".
Volume
The volume control slider affect the volume of the sine wave generated by the "piano keys".
It is independent of the ADSR values.
Stop Button
Stop the dac stream.
Quit Button Closes the dac stream if it is open. Calls QApplication::quit();
Piano Keys
Each Tool Button widget implements these two slots.
Every pressed() function looks like this.
You'll need to modify the playNote parameter for each key.
void MainWindow::on_toolButton_Eb_pressed() // Eb is E flat { envAmp = 0; isKeyDown = true; isKeyUp = false; count = 0; playNote( MIDIstartNote + 3 ); }
The playNote() function looks like this.
void MainWindow::playNote( const int note ) { if ( dac.isStreamRunning() ) dac.stopStream(); adsrState = kAtkState; isKeyDown = true; isNoteOver = false; calcADSR_start_end_points(); freq = midi2frequency( note ); plotEnvelope(); #if USECALLBACK dac.startStream(); #endif }
Every release() function looks like this.
void MainWindow::on_toolButton_Eb_released() { isKeyUp = true; isKeyDown = false; stopNote(); }
The stopNote() function looks like this.
void MainWindow::stopNote() { #if USECALLBACK if ( isNoteOver ) dac.stopStream(); #endif }
State Machine
The ADSR envelope can be in one of four states as the sine wave (or plot) is generated.
- ADSR states
- Attack
Decay
Sustain
Release
Calculating start and end times for each state
- The Attack, Decay, and Release states are determined by their length in milliseconds translated into number of samples.
- The Sustain state is an unchanging amplitude value.
- The Attack state starts at 0, steadily increases to 1.0 over the number of samples in the Attack portion.
- The Decay state starts at one sample past the Attack end and steadily decreases from 1.0 to the Sustain level over the number of samples in the Decay portion.
- The Sustain state is indeterminate and starts at one sample past the Decay end and maintains a Sustain amplitude level until a key released message is sent.
- The Release state starts at one sample past the end of the Sustain portion and steadily decreases from the Sustain level to zero over the number of samples in the Release portion. When the Release portion ends it sets a "note over" flag.
Variables needed
Declared in mainwindow.h.
// In one millisecond there are 44100/1000 samples. const double kSamplesPerMs = 44.1; // Implementing a state variable machine with four states const int kAtkState = 0; const int kDcyState = 1; const int kSusState = 2; const int kRlsState = 3; // Keep track of current state extern int adsrState; // Envelope Attack starts with isKeyDown extern bool isKeyDown; // Envelope Release starts with isKeyUp extern bool isKeyUp; // end of Release portion sets isNoteOver = true // sineADSRcallback returns callbackReturnValue with isNoteOver extern bool isNoteOver; // initial startup values // amplitude level for steady state Sustain extern MY_TYPE lvlS; // frequency of first piano key button (Middle C) extern MY_TYPE freq; // value is calculated for every sample in each ADSR state extern MY_TYPE envAmp; // Maximum amplitude of sine wave generated by isKeyDown extern MY_TYPE sliderVolume; // count of every sample during all callbacks extern int count; // Time in milliseconds for the Attack, Decay, and Release // ADSR segments based on their slider values extern int msA; extern int msD; extern int msR; // Calculations where each ADSR segment begins and ends // in sample indices based on ms time of slider settings extern int sampStartA; extern int sampEndA; extern int sampStartD; extern int sampEndD; extern int sampStartS; extern int sampLenR; extern int sampStartR; extern int sampEndR; // Calculation of the increment needed to take the // Attack portion from 0-1 over length in samples // of Attack segment extern double attackInc; // Calculation of the increment needed to take the // Decay portion from 1 down to Sustain level over // length in samples in Decay segment extern double decayInc; // Calculation of the increment needed to take the // Release portion from Sustain level to 0 over // length in samples of Release segment after isKeyUp. extern double releaseInc; // The number of x axis points in the ADSR plot const int kEnvLength = 88200; // 2 seconds // vector to hold the y axis values for the ADSR plot extern std::vector<MY_TYPE> venv;
You'll need to implement two state machines, one for the ADSR plot and one inside the sine callback function. Both state machines use this structure.
std::vector<MY_TYPE> MainWindow::stuffEnvelopeVector() { for ( uint32_t n = 0; n < last_sample; n++ ) { // Attack phase if ( count <= Asamps ) { // do stuff // set kDcyState when finished } // Decay phase else if ( adsrState == kDcyState ) { // do stuff // set kSusState when finished } // Sustain phase // else if ( count > sampEndD ) else if ( adsrState == kSusState ) { // do stuff // set kRlsState when finished } // Release else if ( adsrState == kRlsState ) { // do stuff // set a "note over" flag when finished } count++; } return venv; }
The Sustain length is known ahead of time for the ADSR plot.
The Sustain length is not known inside the sine callback until the piano key button is released.
Implement the ADSR plot
#define USECALLBACK 0
This #define is near the top of the source file and should be set to zero while you're implementing the ADSR plot. Once the ADSR plot working set it to 1 and begin implementing the state machine inside the sine callback function.
Task 1
Implement calcADSR_start_end_points()
void MainWindow::calcADSR_start_end_points() { /* calculate sustain level from slider value lvlS = ui->verticalSlider_SL->value() * .01; calculate millisecond A D R times from slider value * 10 msA = ui->verticalSlider_A->value() * 10; If you know ms duration there are 44100/1000 samples per millisecond calcualate attackInc = 1.0/(msA * kSamplesPerMs) calcualate decayInc = (1.0 - lvlS)/(msD * kSamplesPerMs) calcualate releasekInc = (lvlS - 0)/(msR * kSamplesPerMs) calculate start/end values for A D in samples sampStartA = 0; sampEndA = msA * kSamplesPerMs sampStartD = end A + 1 sampEndD = start D + number samples in D sampStartS = end D + 1 NOTE: sampEndS is calculated elsewhere for plot it's totalPlotSamples - (A length + D length + R length) for sine callback its when key button is released. sampLenR = msR * kSamplesPerMs */ // Useful for debugging qDebug() << "A start end" << sampStartA << " " << sampEndA; qDebug() << "D start end" << sampStartD << " " << sampEndD; qDebug() << "S start lvl" << sampStartS << " " << lvlS; qDebug() << "R start lvl" << sampLenR; // myqDebug() defined at top to display doubles with using scientific notation myqDebug() << "attackInc decayInc releaseInc" << attackInc << " " << decayInc << " " << releaseInc; }
The qDebug() and myqDebug() output will appear in the Application Output panel.
Task 2
Implement stuffEnvelopeVector()
Note: This is the state machine for the ADSR plot.
The state machine for the sineADSRcallback is similar but requires modifications to the sustain and release segments.
std::vector<MY_TYPE> MainWindow::stuffEnvelopeVector() { calcADSR_start_end_points(); // Calculate Slength as Slength = kEnvLength - ( sampEndD + sampLenR ) MY_TYPE samp{0}; int count{0}; for ( uint32_t n = 0; n < kEnvLength; n++ ) { // This what I did for attack state if ( count <= sampEndA ) { samp += attackInc; if ( samp >= 1.0 ) { samp = 1.0; adsrState = kDcyState; } venv.push_back( samp ); } // Decay phase else if ( adsrState == kDcyState ) { // decrement samp by decayInc // if samp < lvlS // set samp = lvlS // and set adsrState to kSusState // venv.push_back( samp ); } // Sustain phase else if ( adsrState == kSusState ) { // while Slength > 0 // Slength-- // venv.push_back( samp ); // nd set adsrState to kRlsState } // Release else if ( adsrState == kRlsState ) { // decrement samp by releaseInc // if samp < 0 // set samp = 0 // and set adsrState to kDcyState // venv.push_back( samp ); } count++; } return venv; }
Task 3
Implement plotEnvelope()
void MainWindow::plotEnvelope() { /* clear venv call stuffEnvelopeVector() create QVector<double> x and QVector<double> y using kEnvLength as size for loop to set x and y values from 0 < kEnvSize x are loop indices y are venv values customPlot stuff you've used before addGraph setData x axis setRange y axis setRange replot reset adsrState to kAtkState becuase it was changed in stuffEnvelopeVector() */ }
At this point the plot should appear.
Task 4
Remember to
#define USECALLBACK 1
Implement the sineADSRCallback function
// Remember to change callback name in open_dac_stream() int MainWindow::sineADSRCallback( void* outputBuffer, void* /*inputBuffer*/, unsigned int nBufferFrames, double /*streamTime*/, RtAudioStreamStatus status, void* /*userData*/ ) { // if ( envAmp < 0 ) // return 0; if ( isNoteOver ) return callbackReturnValue; MY_TYPE* buffer = static_cast<MY_TYPE*>( outputBuffer ); if ( status ) std::cout << "Stream underflow detected!" << std::endl; static MY_TYPE phz = 0; // //phase increment formula const MY_TYPE phzinc = k2PIT * freq; for ( uint32_t i = 0; i < nBufferFrames; i++ ) { // implement state machine for calculating envAmp for each sample // Attack phase if ( adsrState == kAtkState ) { // increment envAmp by attackInc // if envAmp >= 1.0 // set envAmp to 1.0 // and set adsrState to kDcyState } // Decay phase else if ( adsrState == kDcyState ) { // decrement envAmp by decay // if envAmp < sustain level (lvlS) // set envAmp to lvlS // and set adsrState to kSusState } // Sustain phase else if ( adsrState == kSusState ) { // set envAmp to lvlS // check for isKeyUp // if true set adsrState to kRlsState } // Release else if ( adsrState == kRlsState ) { // set sampStartR to count // set sampEndR to sampStartR + sampLenR // decrement envAmp by releaseInc // if envAmp < 0 // set envAmp = 0 // and set isNoteOver to true } *buffer++ = sliderVolume * envAmp * sin( phz ); phz += phzinc; if ( phz >= k2PI ) phz -= k2PI; ++count; } rta.frameCounter += nBufferFrames; if ( rta.checkCount && ( rta.frameCounter >= rta.nFrames ) ) return callbackReturnValue; QApplication::processEvents(); // Usefull for debuging // myqDebug() << count << " " << adsrState << " " << isKeyUp; return 0; }
Task 5
Hook up the piano key buttons. See sample code earlier on this page.
Build and run
Reducing "pops and clicks" between key presses
Shorten the Release time and/or Attack time and/or Decay time.
Assignment 921
Make it work like mine.
Submission
Feel free to email jellinge@carleton.edu on any part of the homework that is unclear to you. Chances are if you have questions other students do too. I will answer those questions by email to everyone in the class. The original sender will remain anonymous. I have tried to provide clues to the homework in the web pages, reading assignments, and labs. Sometimes what seems obvious to me is not obvious to students just learning C++.
Create a folder named hwNN_LastnameFirstname_LastnameFirstname.
Substitute your name and your partner's name for LastnameFirstname_LastnameFirstname.
Remember
- Boilerplate header at top of every file.
- Make sure your program compiles before submitting.
- Programs that compile and produce partially correct output will be graded.
- You can send notes to me in your homework files by enclosing them in block comments.
- Programs that do not compile get an automatic F (59%).
- Submit the homework in only one of the partner's folders.
- Empty your build or build-debug folders using the command emptyBuildFolder.sh
- Only include .h, .cpp, Makefiles, CMakeLists.txt
- Do not include these folders/files
.git
.vscode Double check for the above two folders using this command
ls -a hwNN_LastnameFirstname1_LastnameFirstname2 # if either .git or .vscode exist in the folder you're submitting remove them with # rm -fR .git # rm -fR .vscode
Hand-in folder contents
/common ├── RtAudio │ ├── RtAudio.cpp │ └── RtAudio.h ├── RtMidi │ ├── RtMidi.cpp │ └── RtMidi.h ├── hw332_CMidiPacket.cpp ├── hw332_CMidiPacket.h ├── hw411_rand_int.cpp ├── hw411_rand_int.h ├── hw421_CDelayMs.cpp ├── hw421_CDelayMs.h ├── hw422_CAppleMidiSynth.cpp ├── hw422_CAppleMidiSynth.h ├── hw423_CMidiTrack.cpp ├── hw423_CMidiTrack.h ├── hw511_CInstrument.cpp ├── hw511_CInstrument.h ├── libsndfile │ ├── sndfile.h │ └── sndfile.hh └── qcustomplot ├── qcustomplot.cpp └── qcustomplot.h /hw92 └── hw921_plot_adsr ├── build-debug (empty) ├── hw921_plot_adsr.pro ├── main.cpp ├── mainwindow.cpp ├── mainwindow.h ├── mainwindow.ui ├── rtaudioutils.cpp └── rtaudioutils.h