Not so long ago we decided to spend a Christmas holiday studying music and its performance. One of us is an amateur mathematician (M) and the other one likes to delve into old psychology textbooks (P), and because we enjoy impressing each other with new facts and insights, we often find ourselves in vehement discussions. Therefore we thought we might have a pleasant and peaceful time by putting our beloved hobby horses aside and embark upon a subject about which neither of us knew much: the timing aspects of music. We became interested in this field because we had noticed, while playing with the computer, our favourite toy, that adding just a bit of random timing noise to a program that played a score in an otherwise metronomically perfect way, made the music much more pleasant to listen to. It seemed as if we could make more sense of it. But we suspected that there was more to timing and expressive performance than adding bits of noise, so we invited a mutual friend who is a retired professional pianist to spend Christmas in our small but well equipped laboratory. Our friend has a great love for the piano and its music, but is completely ignorant of the advances of modern technology. To demonstrate to him our latest sequencer program we asked him to play the theme from the six variations composed by Ludwig van Beethoven on the duet Nel cor più non mi sento, [play] the score of which we had lying around (see Figure 1).
Figure 1. Score of the theme of Nel cor più non mi sento.
Even though he was somewhat disturbed by the touch and harpsichord-like sound of the electronic piano, he was quite fascinated with the possibility of recording and playing back on the same instrument. Enthusiastically we told him that this system was more than just a modern version of the pianola: `You can examine and change every detail you want; for instance, inspect the timing, accurately to the millisecond, add and remove notes, make notes longer or shorter, or louder or softer, and so on and so forth.' Our friend became quite excited and asked : `Could your machine play my performance in a minor key?' We were a bit put off by the simplicity of his demand, but patiently demonstrated the key-change feature. [play] After hearing his performance with the key changed to G minor our friend was not impressed. `O dear, I'm afraid this sounds much too hasty. For example, the "dramatic" e-flat in bar 3 needs more time. Let me play it in minor for you.' [play] When we looked at the timing data of his new performance it indeed showed a different pattern. Upon noticing our disappointed faces our friend remarked `this was not a minor change; it really turns it into another piece. We did not expect your device to know about that, did we?' We kept silent. `But your machine can undoubtedly play the same piece at a faster tempo.' That set us in motion again. We changed the setting of the tempo knob to a tempo one-and-a-half times as high and pushed the play button. [play] The face of our friend again did not show the expression we had hoped for. `I'm awfully sorry, but this is not right! It sounds like a gramophone record played at the wrong speed, but without changing the pitches.' Suspiciously, we wanted some proof for his crude statement and asked him to play it the way he thought it ought to be performed. [play] His version at the higher tempo was indeed different. We had to admit that it sounded more natural than our artificially speeded-up version. What made it sound so much better? We tried to unravel this mystery by examining the timing of the onsets and the offsets of the notes, since these were the variables that could be altered with our electronic keyboard, just like a real harpsichord.
Glossary on Tempo, Metre and Beat.
Our sequencer, a very recent version, had a separate tempo track. In this track, the tempo can be changed from fragment to fragment, even from note to note. With this feature we could put the original score on one track and the timing of the performance, expressed as tempo changes per note, on the tempo track, although it took quite a bit of calculating and editing by hand. After a while we had completely recreated the original performance, but now as a score plus a separate track of expressive timing information. This tempo track looked like the graph in Figure 2a (for clarity we show only the timing of the melody). We could now compare the timing of this performance with the one played at tempo 90 (see Figure 2b). Their form was quite different even by visual inspection, although our ears were, of course, the only valid judges.
Figure 2. Tempo deviations in the performance of the theme at tempo 60 (a) and at tempo 90 (b).
What had happened? The sequencer had speeded everything up by the same amount (which we all agreed sounded awkward), while in the performance the expressive timing appears not to scale up everywhere by the same factor. Our friend adapted his rubato according to the tempo, which he explained to us as: `My timing is very much linked to the musical structure and what I want to communicate of it in an artistic manner to the listener. If I play the piece at another tempo, other structural levels become more important; for instance, at a lower tempo the tactus will shift to a lower level, the subdivisions of the beat will get more "in focus", so to say, and my phrasing will have much more detail.' After some scratching with pen on paper, M found a quite elegant way of representing these changes using simple mathematics. We took the time interval between the onsets of every two succeeding notes and calculated the ratios of these time intervals in the two tempi. If the expressive timing pattern would scale-up linearly, we would find the ratios for all the notes to be around the ratio between the two tempi, and most ratios were indeed around 1.5. There was some variance around that factor, though, and we thought that could be explained by the more elaborate short-span phrasing at the lower tempo. But, even more noticeable was the fact that for some notes the ratio was close to 1. We found that these notes were notated as grace notes in the score. They did not change at all when performed at an another tempo. We also found that not all grace notes behaved like this. For example, the two grace notes that cover an interval of a sixth, in bar 7 and 19, were timed like any other note: they were actually played in a metrical way. Our pianist got really excited about our observations. He pointed at grace notes in the score that were notated in the same way, but that needed a different interpretation, and he started to lecture about the different kinds of ornaments, so popular in the eighteenth century, the difference between acciaccatura and appoggiatura , `ornaments that "crush in" or "lean on" notes', about their possible harmonic or melodic function changing their performance, and so on and so forth. When he noticed that we were getting bored with his lengthy historical observations, he woke us up again with a new, sharp attack on our beautiful sequencer program: `It might be forgivable that your program cannot play the onsets of ornaments correctly, but it also murders the articulation of most notes, especially the staccato ones. And have you heard what the program did to my detailed colouring of the timbre of chords?' Well, in fact, we had not, but we could well understand that the timbral aspect brought about by the chord spread (playing some notes in a chord a tiny bit earlier or later than others) was not kept intact when all timing information is just scaled by a certain factor. And we did not even dare to play the performance again at a lower tempo, afraid that each chord would turn into an arpeggio.
So our sequencer was not so wonderful after all. It could not be used to change something, not even such a minor thing as the key in which the piece was played. Again our pianist explained that a change of key was not a minor thing. The minimal variation that he could think of was the repetition of bars 5-8 at the end of the theme. `The only difference between them is the fact that the second segment is a repetition of the first, and I even expressed that minimal aspect by timing. This problem is exacerbated if the difference between two sections is the overall tempo. Then detailed knowledge about structural levels, articulation, timing of ornamentations and chords, is indispensable.' We had to agree. How dumb of us, after all, to assume that a tempo knob on a commercial sequencer package could be used to adjust the tempo.