sdf PhD





Exploring visual representation of sound
in computer music software through
programming and composition



Selected content from
a thesis submitted with a portfolio of works to the University of Huddersfield in partial fulfilment of the requirements for the degree of Doctor of Philosophy

December 2013

Samuel David Freeman

Minor amendments
April–June 2014

3.2 Visyn

3.2.1 Introducing visyn

Visyn began as an exercise in generating visual patterns on the computer screen; it is an ingenuous exploration of 'visual synthesis' founded on the idea of applying techniques known from audio synthesis and musical composition to a graphical domain. The patterns were synthesised as purely abstract visual manifestations of the underlying processes without the role of representing any other thing. At various times this piece has been disregarded as a dead-end, only to be re-remembered and revisited with fresh perspectives at later stages of the project. The final decision to include this work in the portfolio is based on the aesthetic principles of design exhibited in the piece, which with retrospect, are identified as significant to the project as a whole. While the pixels piece ([§3.1]) brought more clear critical engagement with some of the aesthetics involved, visyn – which predates pixels by almost a year – shows that my fondness of matrix cell data manipulation and display had already begun to emerge in my practice.

The final version of the work is visyn_10e.maxpat (illustrated in Figure 3.3 below), and opening that maxpat will suffice for the following discussion of this formative study. […] As with all of my works which are version numbered in this sort of way, only one version should be open in Max at any given time; this is, among other possible reasons, so that the commonly named send, receive, value and jit.matrix objects etc do not crosstalk between versions (which might lead to undesired results). Looking at the creation dates of the files, the oldest in the visyn directory is found to be patternSynth.maxpat; this first draft name that was superseded illustrates something more of the seed of the idea behind this work.

3.2.2 Looking at visyn

At the heart of the visyn system is an engine which outputs five continuously changing signals; these are produced by sinusoidal oscillators whose frequencies keep changing to randomised values in the range of ±1 Hertz. Three of the output signals are labeled R, G, and B and have a value range of 0 to 1; the other two, labeled X and Y, have a bipolar ±1 value range. These bipolar signals are presented as inputs to the signal routing matrix GUI controllers, of which four can be seen, mid-right of the fullscreen maxpat, in Figure 3.3; each of these GUI controlled mapping units is configured to create a continuous path trace on the surface of the data matrix that is displaying on the left of the screen. When two or more traces are active, they all draw with the same set of RGB colour values that are changing over time.

Figure 3.3: visyn_10e

When the patch is loaded, configurations of the 'XY matrix mappings' default to a preset that can be recalled again by clicking the & message on screen; the &' message triggers a similar mapping that has all four, instead of just two, of the available traces active; the clickable r (to randomise) and clear messages apply to all four mapping units. The text labels within the GUI panels of those mapping units can also be clicked to accesses preset configurations locally.

[n3.9]   More examples of which can be seen within the figures of sdfphd_appendix_A_visyn.pdf, found within the visyn folder of the portfolio directories.

Continuous generation of the mapped traces causes a buildup of curves on the display,[n3.9] and it is then more difficult to see the changes as they happen and enjoy new path shapes as they form. A large font clickable message to clear the displayed data matrix is thus provided in the area above the 'XY matrix mappings' section of the GUI; this message is spatially grouped with 'autoclear' controls allowing the data surface to be reset at regular intervals.

At times, the synthesised pattern will yield a striking image that one may wish to capture. It would be a simple matter to add a button, or key command, by which the data matrix could be exported as an image file, but I have explicitly chosen not to do that in visyn. Just as sound is fleeting and transitory, so too the experience of visyn is intended to be rooted in the experiential now.

Above the clear/autoclear section of the visyn patch interface is an audio control section comprising an ezdac~ object (that is green when DSP is on), an open message (to access DSP options), a large font-size number box (labeled 'audio level'; to set the output amplitude, where zero is mute), and a toggle to activate automatic control of that audio level value.

3.2.3 Visual data to control sound parameters

The audio processing part of the system in this study piece was developed experimentally, and although it represents an important step in the progression of ideas being explored by the project, this particular direction was short lived in my work. The method explored for creating continuous sound from the visually displayed data – as well as containing some avoidable programming errors that were only noticed in review of the piece for this analysis – is somewhat impracticable in terms of computational efficiency in its software implementation. Nevertheless the audio accompaniment to the visual synthesis presented does achieve aesthetic goals.

Data analysis of the visually displayed Jitter matrix provides control values, based on RGB levels in specific areas of the display, for many voices of a basic sound synthesis engine. A 'radius system' for designating frequency values to spatial positions on the plane – conceptually related to the spiroid-frequency-space (§4) – was designed for the visyn system. Twelve radii are specified on the surface, each comprising between 9 and 21 sub-matrix sample areas, and each of which corresponding to a voice of the audio synthesis. A frequency value is assigned to each sample area which determines the frequencies of both a sinusoidal oscillator and a resonant filter. For each sample area, the R and the G values from the data analysis control level of sinusoid in the left and the right channels of stereo output, and the B value sets output level (mono/to both channels) for resonant filtered noise. The audible result is a drone that is highly prone to glitches which are oddly welcome as a rhythmic element to the sound.

3.2.4 Ever-changing and always the same

Visyn exhibits the early (within this project) exploration of working with a data matrix cell-to-pixel-display paradigm, visually tracing algorithmic motion through a two-dimensional space changing over time. As with most of my work, the visyn system was developed by incremental steps, with the additive integration of individually simple ideas, each building upon the last.

The visual aspect in this system is not intended as a representation of anything other than itself, but as such it is open to interpretation by the observer. Although the audio is shaped directly by the data that is seen on screen, the synergy is subtle. When the autoclear feature is active the sound swells in waves proportional to the clearing interval; associative description of the filtered noise content and harmonically related sinusoids in the visyn sound may call comparison to such things as, for example, ocean waves and Tibetan singing-bowls for a clichéd but applicable appreciation of the meditative character of the piece. The visual patterns themselves, even without the addition of sound, seem have a captivating way of retaining attention for prolonged periods.

Although there are GUI controls to alter the manifestations of the system, there remains a sameness to the output; the continuous drawing of paths maintains the character of sinusoidal motion that is within a narrow frequency range that limits the speed but allows for reversal of directions. As implied by the inclusion of the & and &' presets, I tend to think of the output visuals produced by the polar mapping types as being those that define the identity of the piece. Whereas deformed Lissajous-like curves and other square filling paths are possible it is the circularly bounded patterns that I most enjoy. Without interacting with the system one may observe and ever-changing and always the same flow of forms with hints of a metaphorical reference to Cymatics, as if the sound were bringing the shapes into being, though it really is the other way around: the sound being derived from the visualised data.

The lasting influence of visyn on my work may be evident in the following sections, and into the later pieces, such as sdfsys, there are aesthetic and technical themes that were seeded in the visyn study.


← 3.1 Pixels

3.3 Sub synth amp map →