CS 312 - Week 3.1 Class
CS 312 Audio Programming Winter 2020

Table of Contents

3.1 Class

Setup

  • Mount your course folder using the Go menu in Mac Finder:
    smb://courses.ads.carleton.edu/COURSES/cs312-00-w20
  • Open Mac Terminal
  • Execute:
    setup312 <your_carleton_email_name>
cd $HOME312/cs312
mkdir hw31
cd hw31
mkdir c311_regex_example1
cd c311_regex_example1
touch c311_regex.txt

cd $HOME312/cs312/hw31
mkdir c312_regex_example2
cd c312_regex_example2
touch c312_regex2.txt

cd $HOME312/cs312/hw31
mkdir c313_regex_example3
cd c313_regex_example3
mkdir build
touch c313_helloMidi_regex.cpp
touch c313_helloMidi.txt
touch CMakeLists.txt

Download
hw312_CMidiPacketFriends.zip

Execute in Mac Terminal 

cd $HOME312/cs312/hw31
unzip ~/Downloads/hw312_CMidiPacketFriends.zip -d $HOME312/cs312/hw31/
cd $HOME312/cs312/hw31
rm -fR  __MACOSX

cp hw312_CMidiPacketFriends/hw312_CMidiPacket.h c313_regex_example3/
cp hw312_CMidiPacketFriends/hw312_CMidiPacket.cpp c313_regex_example3/

Regular Expressions (regex)

In theoretical computer science and formal language theory, a regular expression (abbreviated regex or regexp and sometimes called a rational expression) is a sequence ofcharacters that define a search pattern, mainly for use in pattern matching with strings, or string matching, i.e. "find and replace"-like operations. The concept arose in the 1950s, when the American mathematician Stephen Kleene formalized the description of a regular language, and came into common use with the Unix text processing utilities ed, an editor, and grep (global regular expression print), a filter.

Regular expressions are so useful in computing that the various systems to specify regular expressions have evolved to provide both a basic and extended standard for the grammar and syntax; modern regular expressions heavily augment the standard. Regular expression processors are found in several search engines, search and replace dialogs of several word processorsand text editors, and in the command lines of text processing utilities, such as sed and AWK.

Many programming languages provide regular expression capabilities, some built-in, for example Perl, JavaScript, Ruby, AWK, and Tcl, and others via a standard library, for example .NET languages, Java, Python, POSIX C and C++ (since C++11). Most other languages offer regular expressions via a library."
https://en.wikipedia.org/wiki/Regular_expression

Regex expressions and abbreviations

. Find any single character
.* Find zero or more of any character. In effect find to the end of the line
5. Find the number 5 followed by any single character
[0-9] brackets define a range In this case search for a single digit zero to nine
[0-9]+ + means one or more. Find one or more digits
[0-9]* * means zero or more. Find zero or more digits
[0-9]+? ? means don't be greedy. Stop after the first match
^ ^ means the beginning of a line
$ $ means the end of a line
^[0-9]+ Find one or more digits at the beginning of a line
[0-9]+$ Find one or more digits at the end of the line
[^0-9]+ ^ inside a range negates the range. Find anything thats not a number
[0-9]{4} The {4} means exactly 4 digits. Find a four digit number
[0-9]{2,4} Find a number of 2 to 4 digits
[0-9A-Fa-fx]+ Find any decimal or hexadecimal number (0-9 A-F a-f x)
[80|90] Find 80 or 90
[8|9]2 Find 82 or 92
\d Find a decimal digit (0-9)
\s Find any single whitespace character (tabs or spaces)
\w Find any alpha-numeric character A-Z a-z 0-9 and _
\D Not a decimal digit
\S Not a whitespace character
\W Not a alpha-numeric character
\w+ Find all alphanumeric characters
\w+? Find all alphanumeric characters stop at first found
[^\w]+ Find all non alpha-numeric characters like whitespace between words
\W+ Same as above, find all non alpha-numeric characters

c311_regex_example1

Open the c311_regex_example1 folder.

You're given a list of names <firstname> <lastname>.

c311_regex.txt
Copy/paste 

Emeli Barlow
Vernon Chapman
Kyle Thorne
Kelsey Bauer
Primrose Bone
Jago Hoover
Klay Fitzpatrick
Alysia Villegas
Viktor Worthington

You need to format them into CSV format (Comma Separated Values) for importing into a database.
The database requires this order <lastname>, <firstname>.

We'll use regular expressions to get the job done.

Open the the Find/Replace panel and turn on Use Regular Expression.

Shortcuts

Command-F opens Find section only
Command-Option-F opens Find and Replace sections.

vsc31101.png

Use this regular expression for Find. 

(\w+)\s+(\w+)

All matches will be highlighted.

Notes on the Find regex
Parentheses enclose replacement groups named $1, $2, $3…

( begin replacement group $1
\w+ one or more alphanumeric character [a-zA-Z0-9]
) end replacement group $1
\s+ one or more spaces or tabs
( begin replacement group $2
\w+ one or more alphanumeric character [a-zA-Z0-9]
) end replacement group $2

Use this regular expression for Replace.

$2, $1

Click Replace All.

vsc31102.png

Notes on the Replace regex

$1 replacement group $1 from Find regex
, insert comma space
$2 replacement group $2 from Find regex

Close the Find/Replace panel.
Job done.
Note the same regular expression would work on a list of 100,000 names just as easily and almost as quickly.

c312_regex_example2

Setup
Open the c312_regex_example2 folder in vsCode.

We'll parse MIDIDisplay_DLS message strings using regular expressions.

c312_regex2.txt
Copy/paste.
Lines 1-2 use tabs.
Line 3 uses spaces. 

100	c0	11
1001	91	61	101
1002 B2 7 102

Turn on Render Whitespace from the View menu.

Make sure you see the tab arrow separating numbers in lines 1 and 2
Make sure you see the space dot separating numbers in line 3.

vsc31201.png

Open the the Find/Replace panel and turn on Use Regular Expression.

Shortcuts
Command-F opens Find section only
Command-Option-F opens Find and Replace sections.

You can widen the Find/Replace panel by dragging the left edge.

Turn on Use Regular Expression
Shortcut: Command-Option-R

Enter these terms 

^(\w+)\s+(\w+)\s+(\w+)\s*(\w*)$
$1\n$2\n$3\n$4

vsc31203.png

Click the Replace All icon

vsc31202.png

Close the Find/Replace panel.

Regex analysis for Find 

^(\w+)\s+(\w+)\s+(\w+)\s*(\w*)$
^ beginning of line  
(\w+) one or more alphanumeric character [a-zA-Z0-9] Replacement $1
\s+ one or more spaces or tabs  
(\w+) one or more alphanumeric character [a-zA-Z0-9] Replacement $2
\s+ one or more spaces or tabs  
(\w+) one or more alphanumeric character [a-zA-Z0-9] Replacement $3
\s* zero or more spaces or tabs  
(\w*) zero or more alphanumeric character [a-zA-Z0-9] Replacement $4
$ end of line  

Regex analysis for Replace 

$1\n$2\n$3\n$4
$n Replacement group n
\n New line

Alternative regex
Search for everything that is not whitespace.

Choose Undo from the Edit menu to revert back to the original text.
Shortcut: Command-Z

Enter these terms.
Find changes.
Replace stays the same. 

^(\S+)\s+(\S+)\s+(\S+)\s*(\S*)
$1\n$2\n$3\n$4

vsc31204.png

Click the Replace All icon

vsc31205.png

Close the Find/Replace panel.

Regex analysis for Find 

^(\S+)\s+(\S+)\s+(\S+)\s*(\S*)
^ beginning of line  
(\S+) one or more characters that are not whitespace Replacement $1
\s+ one or more spaces or tabs  
(\S+) one or more characters that are not whitespace Replacement $2
\s+ one or more spaces or tabs  
(\S+) one or more characters that are not whitespace Replacement $3
\s* zero or more spaces or tabs  
(\S*) zero or more characters that are not whitespace Replacement $4

c313_helloMidi_regex

IMPORTANT:
If you have not submitted hw212_CMidiPacket you must turn in what you have completed before you leave class today in order to be graded.

I have reworded the About CS312 web page to stipulate that homework from now on is due at the beginning of class one week later (not the end as was stated previously). That applies to hw222_CMidiPacket due at the beginning of class Wednesday.

The hw312_CMidiPacket.h and hw312_CMidiPacket.cpp are my solutions to hw212_CMidiPacket with some new functions added. You're free to compare my solutions with yours and adopt any changes into current hw222_CMidiPacket.

If you've already submitted hw222_ you may resubmit if you want to. Just add "revised" to the folder name.

I will be giving you my solutions to hw222_CMidiPacket as the starting point for class 3.2.

Setup
Open c312_helloMidi_regex folder in vsCode.

The c313_helloMidi.txt file is the hw131_helloMidi.txt file with all comments removed.
We'll read the file line by line and use the regex matches to initialize and print CMidiPackets.

c313_helloMidi.txt 

0	c0	0
0	90	60	110
900	80	60	0
1000	90	55	80
1400	80	55	0
1500	90	55	60
1900	90	55	0
2000	90	56	100
2900	80	56	0
3000	90	55	80
4200	80	55	0
4999	c0	56
5000	90	59	110
5900	80	59	0
6000	90	60	127
8000	80	60	0

c313_helloMidi_regex.cpp
This code is slightly modified from TCPP2 Section 9.4.1, page 116.
Copy/paste 

#include <iostream>
#include <string>
#include <fstream>
#include <regex>

#ifndef HW312_CMIDIPACKET_H_
#include "hw312_CMidiPacket.h"
#endif

using namespace CMP31;

//From TCPP2 9.4.1 page 116
void line_to_midipacket(std::string line)
{
  // use a Raw string
  std::regex pat(R"(^(\S+)\s+(\S+)\s+(\S+)\s*(\S*))");
  std::smatch matches;
  if (std::regex_search(line, matches, pat))
  {
    CMidiPacket mp(matches[0]);
    mp.print();
    //
    std::cout << "# "
              << matches[1] << '\t'
              << matches[2] << '\t'
              << matches[3] << '\t'
              << matches[4] << std::endl;
  }
}

int main()
{
  std::string fname = "../c313_helloMidi.txt";
  std::ifstream ifs;
  ifs.open(fname);
  if (!ifs.is_open())
  {
    std::cout << fname << " could not be opened" << std::endl;
    exit(0);
  }

  std::string line;
  while (getline(ifs, line))
    line_to_midipacket(line);
}

Fix the squiggles
If you open c313_helloMidi_regex.cpp you'll see that vsCode could not find the hw222_CMidiPacket.h file that should be in your common folder.

Fix it.

  • Click the squiggle
  • Click the light bulb
  • Edit the #include path as necessary

CMakeLists.txt
This is our standard CMakeLists.txt template from hw23. 

cmake_minimum_required( VERSION 2.8 )
set( APPNAME "helloMidi" )
project( ${APPNAME} )

#edit next line
#set( HOME "/Volumes/cs312-00-w20/StuWork/<your_email_name>" )
set( HOME "/Users/je" )
# set( COMMON "${HOME}/cs312/common" )

set( CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++17 -Wall " )
set( SOURCE_FILES
  hw312_CMidiPacket.cpp
  c313_helloMidi_regex.cpp
)

# include_directories( ${COMMON} )

add_executable( ${APPNAME} ${SOURCE_FILES} )

Build and run using cmake

Output 

0       c0      0
# 0     c0      0
0       90      60      110
# 0     90      60      110
900     80      60      0
# 900   80      60      0
1000    90      55      80
# 1000  90      55      80
1400    80      55      0
# 1400  80      55      0
1500    90      55      60
# 1500  90      55      60
1900    90      55      0
# 1900  90      55      0
2000    90      56      100
# 2000  90      56      100
2900    80      56      0
# 2900  80      56      0
3000    90      55      80
# 3000  90      55      80
4200    80      55      0
# 4200  80      55      0
4999    c0      56
# 4999  c0      56
5000    90      59      110
# 5000  90      59      110
5900    80      59      0
# 5900  80      59      0
6000    90      60      127
# 6000  90      60      127
8000    80      60      0
# 8000  80      60      0

Note:
Lines beginning with a number were CMidiPackets initialized by string matches[0] and output using mp.print().
Lines beginning with a # are regex replacement group strings output using std::cout

Play it in MIDIDisplay_DLS

Reading

What are friends for?

Question
What is a friend function?
Answer
A friend function is a function declared inside a class declaration and specified with the keyword friend. It is a declaration of a function but not an actual class member. Functions declared as friend inside a class declaration (.h file) are given permission to access all private and public functions and data members of that class. In the class implementation (.cpp file) the friend keyword is dropped and the scope operator :: is not used.
Question
How can I make CMidiPacket read and write to streams using << and >>?
Answer
Overload operator>> for reading
Overload operator<< for writing.
Question
How can I tell if CMidiPacket A == CMidiPacket B?
Answer
Overload operator==. Every data member of A must equal the same data member of B.
Question
How can I use std::sort on CMidiPackets?
Answer
Overload operator<. It's up to you to determine what less than means.

CMidiPacket's Four Friends

Operator overloads for <<, >>, ==, and <

These four functions will be overloaded to do the correct thing on CMidiPacket objects. Deciding what the "correct thing" is is straightforward in three cases but more complicated for operator less.

  • operator<< (put to)
  • operator>> (get from)
  • operator== (equals)
  • operator< (less)

Once those these four operator overload functions have been implemented correctly you can do this. 

using namespace CMP31;

CMidiPacket A;
CMidiPacket B;

// print to std::cout
std::cout << A;
// write to a string stream
std::sstream oss;
oss << A;
// write to a file stream
std::fstream outfile;
outfile << A;

// read from standard input
std::cin >> B;
// read from a string stream
std::sstream iss;
iss >> A;
// read from a file stream
// will probably use getline()

// equality
if ( A == B ) do_something();

// less than
std::sort(A.begin(), A.end());
// or
if ( A < B ) do_something_else();

The CMidiPacket operators will work like this.

std::cout << A
Will print 100 90 60 100 with tabs between numbers.
The status byte will be hex without 0x prefix
std::cout can be replaced with other output streams like <sstream> for writing to strings and <fstream> for writing to files.
std::cin >> A
std::cin can be replaced with other intput streams like <sstream> for reading from strings and <fstream> for reading from files.
A==B
A==B if and only if all data members are equal.
A.timestamp_ == B.timestamp_, A.status_ == B.status and so on for each data member.
A < B
Because CMidiPacket (actually any class) implements operator< the std::sort() function will sort CMidiPackets according to those rules.

Homework 3.1

hw311_regex

You've been hired to convert list of 200,000 names and phone numbers into the CSV (Comma Separated Values) format used by the company database. The format of the file you've been given has three fields: firstname, lastname, phone separated by some unknown whitespace characters. The company database has five fields separated by commas: lastname, firstname, areaCode, exchange, number. The company sales force and marketing department use the area code to group customers by states, the exchange by cities and parts of cities, and the number to get in touch with customers.

You want to transform

Debby Feldman (678)404-1948

into

Feldman Debby 678 404 1948

Setup 

cd $HOME312/cs312/
mkdir hw31
cd hw31
mkdir hw311_regex
cd hw311_regex
touch oldDatabase.txt
touch hw311_answer.txt
code .

oldDatabase.txt
Copy/paste 

Justin	Creighton	(408)274-1391
Sari	Dwyer	(541)713-2241
Luis	Metzler	(660)299-4151
Caridad	Worthy	(225)314-1509
Mana	Mckay	(361)903-5171
Ryan	Pettis	(575)443-3333
Chassidy	Ott	(585)472-1483
Rana	Elkins	(818)623-8830
Harley	Sorenson	(443)782-6537
Rosalina	Hobbs	(402)286-3439
Arlie	Pinto	(430)994-1641
Henry	Samson	(407)699-7668
Verena	Lockett	(732)739-5150
Debby	Feldman	(678)404-1948
Keenan	Vann	(641)921-8478
Angelo	Bounds	(386)397-3184
Taunya	Crutchfield	(671)635-1188
Russell	Zeller	(386)283-7545
Rasheeda	Winfield	(361)216-5271
Elenore	Spaulding	(617)821-7668
Rima	Mcintyre	(763)473-5312
Gabrielle	Smyth	(201)710-6795
Audria	Mcfarland	(479)820-9842
Dusti	Abraham	(469)352-1352
Latrina	Wendt	(513)560-9145

Assignment 3.1.1

Create the vsCode regular expressions for find/replace that produce this transformation.
The company database fields are lastname, firstname, areaCode, exchange, number.

Paste your find and replace regular expressions that produce the following output into hw311_regex.txt.

hw311_answer.txt
Copy/paste 

// boilerplate here
My find regex is here:


My replace regex is here:


My converted output is here:

Output 

Creighton,Justin,408,274,1391
Dwyer,Sari,541,713,2241
...

hw312_CMidiPacketFriends

If you have not submitted hw212_CMidiPacket you must turn in what you have completed before you leave class today in order to be graded.

I have reworded the About CS312 web page to stipulate that homework from now on is due at the beginning of class one week later (not the end as was stated previously). That applies to hw222_CMidiPacket due at the beginning of class Wednesday.

The hw312_CMidiPacket.h and hw312_CMidiPacket.cpp are my solutions to hw212_CMidiPacket with some new functions added. You're free to compare my solutions with yours and adopt any changes into current hw222_CMidiPacket.

If you've already submitted hw222_ you may resubmit if you want to. Just add "revised" to the folder name.

I will be giving you my solutions to hw222_CMidiPacket as the starting point for class 3.2.

Open folder hw312_CMidiPacketFriends in vsCode

You should have these files: 

$HOME312/cs312/hw31
└── hw312_CMidiPacketFriends
    ├── CMakeLists.txt
    ├── build
    ├── hw312_CMidiPacket.cpp
    ├── hw312_CMidiPacket.h
    ├── hw312_main.cpp
    ├── hw312_test_CMidiPacket.cpp
    └── hw312_test_CMidiPacket.h

Build and run 

cd $HOME312/cs312/hw31/hw312_CMidiPacketFriends/build
cmake .. && make && ./cmp31.

The script should execute with no errors and compile and run cmp31.

hw312_CMidiPacket modifications
I've declared four friend functions near the end of the hw312_CMidiPacket.h file. 

// NEW FOR hw312_CMidiPacket
/****************************** FRIENDS ******************************/
// operator<< for output
friend std::ostream &operator<<(std::ostream &os, const CMidiPacket &mp);
// operator<< for input
friend std::istream &operator>>(std::istream &is, CMidiPacket &mp);
// operator== for testing equality
friend bool operator==(const CMidiPacket &a, const CMidiPacket &b);
// operator< for testing less_than for MidiPackets
friend bool operator<(const CMidiPacket &a, const CMidiPacket &b);
/****************************** END FRIENDS ******************************/

Assignment 3.1.2

Implement these friend functions in hw312_CMidiPacket.cpp

  • std::ostream &operator<<(std::ostream &os, const CMidiPacket &mp)
  • std::istream &operator>>(std::istream &is, CMidiPacket &mp)
  • bool operator==(const CMidiPacket &a, const CMidiPacket &b)

We'll leave operator< (less) for next time. It's complicated.

Output before implementing friends 

===> test output operator<<()
# You need to write operator<<
You need to write operator<<
oss.str():
# ---------------------------

===> test input operator >>() from a istringstream
You need to write operator>>
You need to write operator<<
# ---------------------------

===> test_operator_equals
a: You need to write operator<<
b: You need to write operator<<
c: You need to write operator<<
d: You need to write operator<<

a == b You need to write operator==
false
a == c You need to write operator==
false
d == b You need to write operator==
false
# ---------------------------

Output after implementing friends 

===> test output operator<<()
# 100	90	60	100
oss.str(): 100	90	60	100

# ---------------------------

===> test input operator >>() from a istringstream
100	90	60	100
# ---------------------------

===> test_operator_equals
a: 1000	90	60	100
b: 1000	90	60	100
c: 1000	80	60	0
d: 1000	c0	11

a == b true
a == c false
d == b false
# ---------------------------

Submission Format

Feel free to email jellinge@carleton.edu on any part of the homework that is unclear to you. Chances are if you have questions other students do too. I will answer those questions by email to everyone in the class. The original sender will remain anonymous. I have tried to provide clues to the homework in the web pages, reading assignments, and labs. Sometimes what seems obvious to me is not obvious to students just learning C++.

Create a folder named hwNN_LastnameFirstname1_LastnameFirstname2.
Substitute your name and your partner's name for LastnameFirstname1_LastnameFirstname2.

Remember

  • Boilerplate header at top of every file.
  • Make sure your program compiles before submitting.
  • Programs that compile and produce partially correct output will be graded.
  • You can send notes to me in your homework files by enclosing them in block comments.
  • Programs that do not compile get an automatic F (59%).
  • Each partner must submit identical homework folders to the course Hand-in folder.
  • If only one partner submits the homework send me an email explaining why.
  • Empty your build or build-debug folders using the command emptyBuildFolder.sh
  • Only include .h, .cpp, Makefiles, CMakeLists.txt
  • Do not include these folders/files
    .git
    .vscode

  • Double check for the above two folders using this command 

    ls -a hwNN_LastnameFirstname1_LastnameFirstname2
# if either .git or .vscode exist in the folder you're submitting remove them with
# rm -fR .git
# rm -fR .vscode

Contents hw31_LastnameFirstname1_LastnameFirstname2

/hw31_LastnameFirstname1_LastnameFirstname2
├── hw311_regex
│   ├── hw311_answer.txt
│   └── oldDatabase.txt
└── hw312_CMidiPacketFriends
    ├── CMakeLists.txt
    ├── build
    ├── hw312_CMidiPacket.cpp
    ├── hw312_CMidiPacket.h
    ├── hw312_main.cpp
    ├── hw312_test_CMidiPacket.cpp
    └── hw312_test_CMidiPacket.h

Author: John Ellinger

Created: 2020-01-20 Mon 12:26

Validate