GRADUAL CONTROL OF HARMONICITY IN THE CONTEXT OF FREQUENCY MODULATION

Rafael Valle Center For New Music And Audio Technologies University of California Berkeley

ABSTRACT This paper describes the use of neural autocorrelation to achieve gradual control of harmonicity (GCH) in frequency modulation (FM) based vertical pitch structures (VPS). Neural autocorrelation is analyzed to determine its applicability to the gradual control of harmonicity of vertical pitch structures generated by FM. New parameters for compositional use of FM, such as the harmonicity axis (HA), the harmonicity axis limits (HAL) and harmonicity axis deviation (HAD), are introduced. Generalizations are made on the use of the harmonicity axis and harmonicity axis deviation. Finally, a musical example composed using the new parameters is shown from Over The Edge Of Sound, a piece written by the author and commissioned by the Acanthes festival in France. 1. INTRODUCTION

In Frequency Modulation terms, a harmonic sound is obtained when the ratio of the carrier to the modulating frequency is a rational number :

In the case of inharmonic sounds, there is no clear period (T) present.
In Frequency Modulation terms, an inharmonic sound is obtained when the ratio of the carrier to the modulation frequency is an irrational number :

In 1973, John Chowning proposed a new application of the frequency modulation encoding to control audio spectra and a method that provided some control over their harmonicity. Furthermore, sound analysis methods (e.g. divergence of frequential components and virtual fundamental estimation) also enable composers to create harmonic and inharmonic sounds.
Although efcient to compose a harmonicity dichotomy between the generated sounds, these methods do not provide information on the degree of harmonicity or inharmonicity of a given sound, nor do they provide information on the possibility of a middle-term between the two categories 1.
The neural periodicity can be used to complement the previous methods by providing a fast and efcient mechanism based on neurophysiological processing to produce quantitative gradual changes in the periodicity of vertical pitch structures. 2. HARMONIC AND INHARMONIC SOUNDS

Schneider afrms that some inharmonic sounds might carry enough periodicity information to produce pseudo-periods, which will be discussed in short. 3. MODELS FOR THE GENERATION OF HARMONIC AND INHARMONIC SOUNDS

Although the methods analyzed in this paper are mostly used for sound analysis, their process can be easily inverted to aid in the composition of vertical pitch structures. The applicability of three models (1) Divergence of Spectral Components, (2) Virtual Fundamental Estimation and (3) Neural autocorrelation to achieve gradual control of periodicity in FM generated sound is measured here according to acoustics, psychoacoustics psychophysiologic precepts. 3.1. Divergence of Spectral Components This model calculates the inharmonicity of a sound by the weighted divergence of its frequential components from the multiple of the fundamental frequency (Peeters 2004):

Harmonic and Inharmonic sounds are part of the deterministic vibrations, which can be divided in two subgroups, (a) periodic and (b) non-periodic. Periodic vibrations are dened by the function:

Later in 1974, Chowning discovered that using an inharmonicity ratio that is 1 to a power of the golden mean produces inharmonic sounds, but with a "perceptible order when one moved through the frequency space with a constant spectrum" (Roads 1985).
1


It does not take into account two important psychoacoustics properties for the perception of a sound complex, which are the Critical Band and the Virtual Fundamental. According to this model, the analysis of a harmonic series in which an amplitude modulation with modulation frequency of value k (k<15 hz) is added to each frequential component will consider this sound as inharmonic, which is not the case. This happens because the model interprets each amplitude modulated frequential component as two, which produces a mismatch.
In the case of virtual fundamental, the use of divergence of spectral components with an incorrect analysis of the fundamental frequency, e.g. a harmonic spectrum in which some of the rst lower partials are not present, will result in a specious measure of a sounds harmonicity. 3.2. Virtual Fundamental Estimation The missing fundamental effect1 is an important indication that, if a sound has enough periodicity information, a virtual fundamental is detected through higher level processing in the auditory system, in which common periods are detected as coinciding pulses. Although such a sound is perceived with a different timber, it is still perceived as being harmonic.
The Virtual fundamental can be estimated by calculating the greatest common divisor of the frequential components of a sound. The distance between the virtual fundamental of a sound and its lowest present frequential component can be used as a measure of the periodicity. The evolution of periodicity between adjacent partials of the harmonic series is given by the following function:

The Autocorrelation function represents an alternative, according to Schneider, suited to investigate the periodicity of a stationary signal. However, its parameters have to take into account the strategies used by the human auditory system to determine the periodicity of a sound. 4. NEURAL HARMONICITY AND AUTOCORRELATION

Neural harmonicity was the most successful explanatory factor of human percepts and recent psychoacoustical evidence conrm that the perception of consonance-dissonance is governed primarily by the periodicity of a musical interval/chord and not its roughness or beatings (Bidelman 2013). 4.1. Neural autocorrelation The neural autocorrelation model developed by Ebeling2 interprets each pitch as a series of equidistant pulses . Since neural pulses are represented by a density function (t - nT) with a small width directly proportional to , overlapping periodicity areas between terms of a given ratio s ( in frequency modulation s = Fm + Fc/Fc ) are taken into account. A measure value for the degree of overlapping is dened by squaring the autocorrelation function of an interval with vibration ratio s, from which one obtains (s,T). Integrating (s,t) and limiting it to the range of audible frequencies ( 0 to 50 ms ) provides a measure value for the degree of overlap for every vibration ratio s. Finally, the Generalized Coincidence Function (GCF) is dened as (Ebeling 2007):

A plot of the generalized coincidence function for the intervals (s) within an octave, computed on the basis of rectangular impulses of length = 0.8 ms for 100 Hz as the reference frequency of the lowest sound is given in Figure 1.
By this denition and in the context of frequency modulation, the farther away a pair of partials is positioned from its fundamental frequency, the lesser its periodicity is going to be. However, the misleading analysis of the virtual fundamental might lead to erroneous information on the periodicity of a sound. Suppose a sound composed of partials f, f and f. Although these partials were chosen considering f as the fundamental, a correct estimation provides f as the virtual fundamental, which represents the greatest common divisor. Furthermore, in the case of sounds with complex periodicity, such a simple analysis and control, in the context of FM, is not possible, nor precise enough, nor practical.

Figure 1. Generalized coincidence function for the intervals of an octave (calculated on the basis of rectangular pulse trains, = 0.8 ms. Different shades represent nine harmonicity axes.

The missing fundamental phenomenon afrms that the overtones of a sound might suggest its fundamental frequency, although the sound lacks this component itself.

For a throughly explanation of the generalized coincidence function and it's relationship with neural pulses, please see Martin Ebeling's Verschmelzung und neurale Autokorrelation als Grundlage einer Konsonanztheorie
2

5.

NEW PARAMETERS

5.5. Harmonicity Axis Deviation

The harmonicity axis, harmonicity axis limits, and harmonicity axis deviation are new parameters used to control the harmonicity of vertical pitch structures generated with FM. Some considerations are made on traditional FM parameters, such as the Modulation Index and Sidebands, to analyze their interference on the periodicity of an FM sound based on the generalized coincidence function. 5.1. Modulation Index The modulation index 1 will dene the bandwidth of an FM generated signal as well as its sidebands and their amplitude. Some modulation indices might interfere with the expected perception of periodicity, due to different weight of sidebands or even their absence. Further, the described use of FM in this paper is translated to the medium of acoustic instruments and the control of the number of sidebands and their amplitude is regulated by other constraints such as instrumental range, orchestration and other compositional structures. 5.2. Sidebands The frequential components of a frequency modulated generated sound is made of both lower and upper sidebands. The addition of lower sidebands diminishes the distance between the lowest frequential component and its virtual fundamental frequency. This causes a shift in the overall periodicity of a sound, as explained above in the analysis of the virtual fundamental estimation model. In addition, lower sidebands which exceed 0 Hz fold back at their absolute value with an inversion of phase of 180. This produces a signicant modication in the periodicity of the FM generated vertical pitch structure. The compositional constraints mentioned before also apply in this case. For these reasons, only the upper sidebands of the FM generated vertical pitch structures are used. 5.3. Harmonicity Axis The Harmonicity Axis represents the interval s (s = Fm + Fc/ Fc) in which there is an extreme in periodicity K(s), within the range of an octave according to the GCF. They dene nine distinct categories 2 of periodicity and provide composers with a strategy to achieve gradual control over the periodicity of FM generated sounds, which is possible within the same categories, be it within the harmonicity axis limits. 5.4. Harmonicity Axis Limits The harmonicity axis limits are dened as limit values for the interval s, which are within the bandwidth of each of the nine harmonicity axis. In this case, within the limits of a harmonicity axis, the composer has continuous control over the periodicity of FM generated VPS, by deviating from the harmonicity axis.

The deviation represents the distance between the harmonicity axis and another interval s within the harmonicity axis limits. It is inversely proportional to periodicity but not symmetric. 6. ANALYSIS OF NEW PARAMETERS

A discrete analysis of four harmonicity axes is made to measure the periodicity of the generated vertical pitch structures, as well as their estimated fundamental and the ratios between their frequential components. 6.1. FM spectra generated on harmonicity axes Whenever the numerator of the harmonicity axis is added or multiplied by an integer greater than zero, this will create vertical pitch structures whose virtual pitch are the same and that advance, after each operation, in the related harmonic series.3
1.2 1.4 1.6 1.8 Fc*1.6

1.25 1.5 1.75

1.33 1.67

fundamental estimation

Fc

Fc*1.3

Table 1. Division of FM generated pitch structures according to harmonicity axis grouping and their estimated fundamental pitch, calculated by estimating GCD of each vertical pitch structure.

2.0

1.2

Fc

22
... 132 154 176 198 220 242 264 286 308 330 352 374 ...

110

r a t i o
1.20 1.16 1.14 1.12 1.11 1.10 1.09 1.08 1.07 1.07 1.06

1.4

110

r a t i o
1.40

1.6

110

r a t i o

1.8

110

r a t i o

V P S (hz)

132 154 176 198 220 242 264 286 308 330 352 374 ...

154 1.28 198 1.22 242 1.18 286 1.15 330 374 ...
1.13

176

1.60 198

1.80

242

1.37 286

1.44

308

1.27

1.30 374 ... 1.21 374 ...

Table 2. The above table provides the frequential components of vertical pitch structures generated with FM (s = Fm + Fc / Fc) and the ratio between neighboring frequency pairs.

The ratio of the peak deviation to the modulating frequency. Represented by the nine bell curves in Fig. 1 The perception of a virtual fundamental will decrease as the value of the numerator increases.

6.2. Deviation from harmonicity axis A deviation from the harmonicity axis decreases the periodicity of the vertical pitch structure. The decrease is proportional to the deviation, but not symmetric. Once the bandwidth of the harmonicity axis is exceeded, the harmonicity increases and moves towards a new peak and virtual fundamental.
s 1.6666
1.4 1.2857144 1.2222222 1.1818181 1.1538463 1.1333333 1.117647 1.1052631 1.0952381

7.2. Multiple harmonicity axes The following example demonstrates the combination of multiple harmonicity axes to create a vertical pitch structure in which neighboring pairs of pitches, chosen according to their critical band, are given to instruments of similar timbre to enhance the perception of roughness and periodicity. The pitch structure is based on FM generated chords with a periodicity value of 1.6993.
Fc Fm Fc Fm Fc Fm Fc Fm Fc Fm Fc Fm

1.6369
1.3890891 1.2801038 1.2188133 1.1795298 1.1522045 1.1320986 1.1166847 1.1044921 1.0946065

1.6963
1.4115224 1.291545 1.2257335 1.184162 1.1555209 1.1345896 1.1186239 1.1060445 1.0958773

125.6 80!

150.72 96

100.48 64

251.2 160

301.44 192 200.96 128

r a t i o s

Table 3. Ratios of the frequential components generated with FM based on 86% harmonicity axis deviation.

7.

COMPOSING OVER THE EDGE OF SOUND

% of CB 96 13 64 57 80 54 63 49 82 41 54 47 70 40 70 20 70 Ratios 2.75 1.09 1.38 1.24 1.28 1.15 1.15 1.10 1.17 1.08 1.11 1.09 1.15 1.08 1.15 1.04 1.15

In my piece Over The Edge Of Sound, the procedures described above are combined to generate gradual evolutions from periodic to aperiodic sounds and rhythms. Four base low frequency values (8, 10, 12 and 14), and their octaves are used to control the periodicity of vertical pitch structures and their specic frequential regions, which are dened according to compositional constraints related to the critical band, providing thus a control over the roughness and fusion of the pitch structures.

Figures 4 and 5. Main vertical pitch structure in Over The Edge Of sound. Shades of color represent regions and pitch pairs are chosen according to their periodicity, roughness and fusion.

8.

CONCLUSION

The use of neural periodicity in FM provides composers with an efcient and practical method to continuously control the harmonicity of vertical pitch structures. The method described introduces new parameters that allow composers to create pitch structures based on many perceptual factors, including virtual fundamental, roughness, and harmonicity. 9. REFERENCES [1] Bidelman, Gavin. The role of the auditory brainstem in processing musically relevant pitch, Frontiers in Psychology, USA, 2013. [2] Ebeling, M. Verschmelzung und neurale Autokorrelation als Grundlage einer Konsonanztheorie. Frankfurt/M.: Peter Lang, 2007. [3] Peeters, G., A Large Set of Features for Sound Description (similarity and Classication) in the CUIDADO Project. Proceedings of the International Computer Music Conference, Miami, USA, 2004. [4] Plomp, R. and Levelt, W., Tonal Consonance and Critical Bandwidth, J. Acoustic Society of America, 1965 [5] Schneider, A. "Inharmonic Sounds: Implications as to Pitch, Timbre and Consonance", Journal of New Music Research: vol. 29, Swets & Zeitlinger, 2000 [6] Terhardt, E. On the perception of periodic sound uctuations (roughness). Acoustica, 1974.

Figure 2. Global evolution of periodicity in Over The Edge of Sound

7.1. Gradual decrease in harmonicity The following example showcases pitch structures with 86% of deviation from the major sixth harmonicity axis (1.6666) and Fm values equal to 96 and 64 hz respectively. Neighboring pairs are distributed to members of a double string quartet that performs glissandi between the pitch structures.

Fc = 114.4 Fm = 10*23 s = 1.6993 Fc = 91.52 Fm = 8*23 s = 1.6993

Figure 3. Pitch structures with 86% harmonicity axis deviation from 1.6666. Lower partials are omitted.