6.3 Beta (S D Freeman PhD thesis)

6.3 Beta

6.3.1 sdfsys_beta

Building upon the foundations established in sdf.sys_alpha, the beta version of sdfsys introduces frameworks for greater modularity (at various levels of abstraction), significant enhancements to the gl-ui, and many more module units for loading into the eight spaces of the environment. This section describes sdfsys_beta in terms of the enhancements and additions made after the alpha build of the system.

The _beta folder, under chapter6_sdfsys of the portfolio directories, contains two sub-folders: the sdfsys_b_doc folder contains various documents that support description of sdfsys_beta; and the sdfsys_beta1 folder contains the files that comprise the work itself.

Most of the other works in the portfolio have had their directories 'tidied up' to some extent: for sdf.sys_alpha – as a case in point – only the files needed to run the system, as demonstrated in the video walkthrough of §(6.2.4.d), are included. For sdfsys_beta, however, I have decided to preserve the contents of the folder to the inclusion of everything 'as is'.

The top level patch, to be opened in order to run sdfsys_beta, is sdf.sys_b133.maxpat (shown in Figure 6.10). To enter the fullscreen view of the gl-ui one can click on the icon in the top right corner of the patch; the key-combination [dblmod]+[esc] can also now be used to enter fullscreen, and – as before – the [esc] key is used to exit fullscreen of the gl-ui.

Figure 6.10: sdfsys_b133

6.3.2 The beta design for the gl-ui

To accommodate new ways of working within the system, the layout of the gl-ui was redesigned. The sdfsys_beta design represented in Figure 6.11 was arrived at after process that took several perspectives of the work into consideration. As well as being directed by the technical needs of the new elements to be represented on the gl-ui, the aesthetic matter of aspect ratios was again allowed to influence the design. The five areas of the gl-ui, described below, are named: the main area; the spaces network area; the box area; the system toggle; and the text toggle.

outline
Figure 6.11: sdfsys_beta ui layout

Figure 6.12: sdf_ui at load in sdfsys_b133

The dividing lines that are drawn on the beta gl-ui have been chosen so that that the processes of drawing them, one at a time within the 3:2 window, will only create internal aspect ratios that equate to the intervals of unison, octave or perfect fifth (see Figure 6.13). The alternating horizontal and vertical orientation of the divides also allows for the simple coding of four if-else stages to be used for identifying which of the five gl-ui areas the mouse point is in.

Figure 6.13: sdfsys_b1_ui_ratios

Each of the five gl-ui areas has its own sub-patch within the coding of the system, and these will be described below in turn. With the exception of the clock, which is implemented in its own patcher at the top-level of the maxpat, all of the patching that comprises the system is found within the patcher named 'sdfsys' (Figure 6.14).

Figure 6.14: [p sdfsys]

6.3.3 Colour in sdfsys_beta

Colour is used in three ways within the sdfsys_beta environment:

text: white for information output in the box and system toggle areas of the gl-ui, and green for the text-buffer display, with a red underscore to show the input cursor location;
data-matrices: the 'matrix_view_type' sub-patch of the space structure has implementation for five types of matrix data colouration, see Figure 6.15;
spaces: eight colours for visually identifying different aspects belonging to each of the eight spaces, see §(6.3.5.b).

Figure 6.15: matrix_view_type

6.3.4 _Area_1_Main

outline with label
Figure 6.16: main area

(6.3.4.a) Mousing data

The first inlet of each _Area sub-patch is for mouse input data messages which will only arrive to one of these sub-patches at a time, according to where in the gl-ui the mouse is pointing.^[n6.22] In the main area patch, the mouse input is connected to a poly~ patch that will forward the mouse data messages to any of the eight spaces that currently have their active state value set to 2.

[n6.22] Implementation of the text editor is found in the _Area_5_TextToggle patcher (§6.3.8), but because the editor appears in the main area of the gl-ui, mouse messages gated to _Main are also connected to go to the _Text patch.

(6.3.4.b) Eight spaces

Before programming began for the beta version of sdfsys, a new 'space structure specification' was drafted on paper. This included a new way to work with the spaces network – involving the new spaces network area of the gl-ui; see §6.3.5 – and a better method for linking the parameter attributes of a module patch to their sprites on the gl-ui.

Eight instances of sdf.sys_b.space.structure.maxpat are created and connected in the _Area_1_Main patch. In sdfsys_beta, all eight spaces are the same because the text editor, rather than occupying one of the spaces, is conceptualised as a distinct part of the system (§6.3.8). Messages that target the spaces are received via r sdf.sys.spaces and routed according the integer (0–7) at the start of each message. The send toAllSpaces namespace is also being used. The 'space structure' patches have an outlet from which will come messages targeting the 'box area' of the gl-ui (see §6.3.6).

Whereas, in the alpha build, each module unit would comprise a maxpat and a .js, the beta structure for spaces is based on each instantiated space structure having its own instance of a sdf.sys_b.space.structure.js which handles the creation and control of gl-ui sprites. There is then a protocol by which a module unit maxpat – that will be loaded into a one-voice poly~ object – can interact with its instance of the space structure JavaScript file. This protocol is embodied as an adaptive and re-usable block of patching: a configuration of objects that can be copy-pasted to any new module patch being made, and that requires a number of manual edits to be carefully applied.^[n6.23]

[n6.23] There are notes within the existing module unit maxpat files to aid with creation of new modules; see ~g.null.maxpat which is the default, parameterless, module that is loaded into each space structure when the sdfsys environment starts up.

The flowchart in Figure 6.17 provides a simplified high-level overview of how the space structure specification fits into sdfsys_beta.

Figure 6.17: space_structure_flow

(6.3.4.c) Parameter-sprites

The following table shows the six types of parameter-sprite that can be used by module unit patches for sdfsys_beta. The parameter-sprites of a module unit that is loaded into a space will – when the active state is 2 – appear in the main area of the gl-ui with the colour of space into which that module is loaded.

Type	Glyph	Mapping from position on the plane to parameter-value
Hz		Polar (spiroid)	(r, theta) → (octave-level, pitch-class)	→	(frequency value)
point		Cartesian	(x, y) → (left to right, top to bottom)	→	(0–400, 0–400)
radial		Relative radius	(distance from a reference sprite)	→	(0–400)
xdim		Horizontal slider	(left to right)	→	(0–1)
ydim		Slider	(bottom to top)	→	(0–1)
rotary		Dial (at a fixed location)	(down/left to decrease, up/right to increase)	→	(0–1)

The rotary type of parameter-sprite is unique within this set of sprites because it comprises two parts: the first part is the needle of the dial is shown with the colour of the space to which the sprite belongs, and this is visible whenever that space is in active state 2; the second part is the circumference of a circle that is only shown (and always in grey) when the mouse clicks down on the area of the sprite.

The xdim and ydim types can be moved anywhere within the main area of the gl-ui but only one of the cartesian dimensions is used to set the value of the parameter. The point and radial types use the 0–400 range in order to match the dimensions of the data-matrices, make simplifies certain matters for the visually-linked DSP algorithms that use them.

A radial sprite is created as 'referring to' another sprite. The parameter-value of a radial sprite is set by its distance away from its reference sprite; a radial sprite will thus be moved on screen when its reference sprite is moved (so to remain at the same distance). One radial sprite (let's call it r2) can refer to another radial sprite (r1), but only if the reference sprite of that radial (r1) is a different type of sprite (usually a point type). Better use of recursive programming techniques could probably allow for radials to refer to radials that refer to radials, and so on, but the system in place has been sufficient so far.

6.3.5 _Area_2_SpacesNetwork

outline with label
Figure 6.18: spaces network area

(6.3.5.a) Visual patching

A visual patching paradigm is added to the specification for the eight spaces aspect of sdfsys_beta for connecting matrix and audio data between the spaces. Rather than being part of the fixed the gl-ui layout, the eight frames – and the mini-views of matrix data within the frames – are now more like the sprites of the main area: the frames are smaller now than they were in the alpha build, and they can be moved via click-and-drag to be arranged within the spaces network area of the gl-ui.

In the sdfsys maxpat, mouse point and button messages coming into the spaces network part of sdfsys are processed to create inputs for sdf.sys_b.space.network.js; Figure 6.19 shows the message data flow in both the maxpat and .js elements of this sub-patch.

Figure 6.19: _Area_2_SpacesNetwork_flow

(6.3.5.b) Eight colours for the eight spaces

The set of colours that are used to visually identify each of the eight spaces are defined in sdf.sys_b.space.network.js as an array of RGBA values that is globally accessible to .js functions in different parts of the system:^[n6.24]

[n6.24] The colour comprising green and blue, indexed 4 in this array of arrays, is more commonly called cyan, but the slightly less precise name aqua became habit after the MoE Pixels project (§3.1) during which the latter was chosen for use because I thought it would be more accessible for the workshop participants.

jsG.spacesRGBlist = new Array(9); // each [item] holds [r, g, b, a] value array  
jsG.spacesRGBlist[0] = [1., 0., 0., 0.45];    // red   
jsG.spacesRGBlist[1] = [1., 0.5, 0., 0.45];   // orange 
jsG.spacesRGBlist[2] = [1., 1., 0., 0.45];    // yellow
jsG.spacesRGBlist[3] = [0., 1., 0., 0.45];    // green
jsG.spacesRGBlist[4] = [0., 1., 1., 0.45];    // aqua
jsG.spacesRGBlist[5] = [0., 0.5, 1., 0.45];   // blue
jsG.spacesRGBlist[6] = [0.375, 0., 1., 0.45]; // purple 
jsG.spacesRGBlist[7] = [1., 0., 1., 0.45];    // magenta
jsG.spacesRGBlist[8] = [0.5, 0.5, 0.5, 0.45]; // grey

The frames in the spaces network area start with active state 0, which is represented visually by a thin grey line; states 1 and 2 are shown by thin and think coloured lines using the colours of that .js global array.

(6.3.5.c) Making connections in the spaces network

The spaces network area has two view modes:

mode 0 is for matrix connections;
mode 1 is for audio connections.

In each view mode, the frames can be arranged within the spaces network area, and clicking on the system toggle area (§6.3.7) will switch between the two arrangements of the frames. Connections are made automatically when the frames are drag-dropped, and – referencing systems such as the reacTable (Jord� et al., 2007) and Texture (McLean, 2011, sect. 5.6) – the logic determining network connections is based on the proximity of outputs to inputs on the frames that represent the eight spaces. Outputs are represented by coloured dots on the bottom-right corners of the frames, and inputs by coloured dots on the top-left corners.

Each of the eight spaces has a data-matrix, and so all eight frames always have an output dot when the spaces network area is in view mode 0. If the module will read data from the matrix of another space then it requires a matrix input in the network representation. Matrix inputs, audio inputs, and audio outputs are all optional for the module units. When a module is loaded to sdfsys it will report which of the input and output options are to be used.

I decided to limit the space structure specification to only allow one optional matrix input, and module may have an audio output, or an audio input, or neither, or both.

6.3.6 _Area_3_Box

outline with label
Figure 6.20: box area

In the original prescription for the sdfsys environment, written before commencing to build sdf.sys_alpha, it was stated that text would be used both by the human to instruct the system, and by the system to inform the human. In sdf.sys_alpha this was achieved by letting modules alter the text-buffer, but I was not completely happy with that as a solution; a design in which the human input is being overwritten by the system is at risk of undermining the interactivity. In sdfsys_beta, the box area of the gl-ui is used for displaying textual information.

Information presented in the box area is most often related to the last thing that was clicked on, be that a frame in the spaces network area or a sprite in the main area, but the text display of the box area – comprising eight lines; 23 characters per line – is also targeted by the sdf.sys_b.txt sub-system to report syntax errors and respond to query commands (see §6.3.8).

Whilst musicking with sdfsys_beta there were a number of occasions upon which I wanted access to the box text, such as to be able to copy-paste parameter-values into a script. To facilitate this, I made it so that clicking three times on the box area^[n6.25] will bring up a standard text editor window with a copy of the box text. Although this external access to an element of the self-contained environment is not used often, it is useful to have it available. After the implementation of that feature, I had the idea for a context sensitive copy-paste inspired method – internal to sdfsys – for taking information out of the box area and into the main text-buffer. This an method is used much more often in practice than triple-clicking on the box area.

[n6.25] The count is reset if the mouse point leaves the box area.

The key-command [dblmod]+[B] is used to access the text-buffer, and if the information currently displayed there is one of two recognised formats, then elements of it become available for pasting into the text-buffer with the key-command [dblmod]+[V]. If the box is displaying information about the space of the last frame to be clicked on, then the word 'space' and the appropriate number can be made available for pasting; this can be used as a shortcut to typing commands that target that space, without needing to think in terms of what number it is. If the information in the box area is about a parameter-sprite then the text that can be made available to paste provides the start of a command line for changing the value of that parameter; see §(6.3.8.c) for direction to syntax documentation.

A high-level representation of what is happening in the _Area_3_Box sub-patch of sdfsys is provided by the flowchart in Figure 6.21.

Figure 6.21: _Area_3_Box_flow

6.3.7 _Area_4_SystemToggle

outline with label
Figure 6.22: system toggle

The system toggle area acts as a simple toggle to switch between the two types of view for the spaces network area (§6.3.5). When sdfsys_beta is loaded view mode 0 is selected, clicking on the system toggle changes it to mode 1. The most recent version uses the verbose text elements of 'matrix' and '_audio' in the system toggle area of the gl-ui, but – as may be noticed in videos and photographs – previous beta versions only showed the mode number.

6.3.8 _Area_5_TextToggle

outline with label
Figure 6.23: text toggle

(6.3.8.a) To toggle the text

The fifth area, in the top right corner of the gl-ui, is the text toggle. In terms of its own small square of screen space, it has three ways to respond to being clicked on, but because it represents the text editor of sdfsys_beta it has a lot of patching within it sub-patch.

Three types of click: (1) if the text toggle is clicked when DSP is off, then the DSP Status window will be opened; (2) in general operation, a click will toggle the active state of the the text editor, the key-combination [dblmod]+[T] has also been added for the same purpose; (3) similar to triple-clicking on the box area, clicking four times on the text toggle area will open a windowed copy of the text-buffer so that the text can be taken out of sdfsys.^[n6.26]

[n6.26] It is four clicks, rather than three, to have the active state of the text editor be back as it started; more than two needed to make sure it is intentional.

(6.3.8.b) Editing the text-buffer

Advancing on the alpha build text system, the beta version adds more usability with additional features being implemented as a .js and maxpat hybrid, with the two parts interoperating to affect change on the text-buffer in response to qwerty-based input. For example, it was much quicker in a maxpat than it would have been in JavaScript, to add methods for inserting and deleting lines from the text-buffer; these are actioned via the key-combinations [shift]+[enter] and [shift]+[backspace].

(6.3.8.c) Syntax in sdfsys_beta

The number of commands available in the system has grown significantly. As well as some new system commands, the thisis drawing system has received many additions and improvements. A comprehensive overview of all available commands has been prepared in a .mm file, but it is perhaps best viewed in its .html export version: the sdfsys_b_doc folder has a sub-folder called sdfsys_syntax within which is sdfsys_parse_text3.html. The hierarchical structure of that syntax documentation is summarised here:

Syntax of the text system in sdf.sys_b130
- About this document (sdfsys_parse_text3.html)
  - The lines of text are from the text buffer are delivered as Max (list) messages from [p txt_line_reader], one at time, to [p parse_text].
  - Interpretation of most commands will terminate with triggering the next line of the text buffer to be read.
  - This document shows how the lines of text are interpreted based on keyword routing: the list is routed first by the first symbol in the list.
  - Most keywords are followed by extra arguments, as represented by the hierarchical structuring below.
  - The order of presentation here is matched to that of the keyword routing in sdf.sys_b130.maxpat
  - The keyword commands that are divided into conceptual groups: (commenting), (sdfsys), (thisis), and (other)
  - Examples of a command line are given [in bold]
    - // like this, followed by a description of that command
    - in this example the line is 'commented out'; note the necessary space after the '//'
- [p parse_text]
- (commenting)
  - /*
  - */
  - /..
  - ../
  - //
  - /!
- (sdfsys)
  - help
  - info
  - url
  - pcontrol
  - txt
  - txtfile
  - setup
  - space
  - send
  - draw_to
  - .txt
- (thisis)
  - delay
  - clear
  - lcd
  - clearall
  - thisis
  - unput
  - put
  - draw
  - get
  - set
  - macro
  - spawn
- (other)

6.3.9 Infrastructures and Meta-patches

(6.3.9.a) sdf.sys.rtuis

A framework for what I described as real-time user interface/interaction sequencing (rtuis) has been put in place within sdfsys_beta. It was partly an exercise in forward thinking towards the possibilities of allowing multitouch control of sdfsys, and to make temporal recordings of interaction.^[n6.27] The sequence recording aspect is underdeveloped, but the rtuis sends that go to the various parts of sdfsys have been utilised in other ways: they allow for scripting of elements that were previously only accessible by mouse, such as setting the active state of a space, and moving a frame in the spaces network area. Visible in the top-level patch of the sdfsys_beta, there is a clickable message to open the 'sdf.sys.rtuirec' patch.

[n6.27] As commented on in a rare blog post: http://www.sdfsys.info/blog/2012/03/02/rtuis-and-the-spaces-network/

(6.3.9.b) sdf.sys.network

Also at the top-level patch of sdfsys_beta is a message that can be clicked to open the sdf.sys.network meta-patch that enables Open Sound Control (OSC)^[n6.28] based network communication for sdfsys. It was constructed to be part of the HELOpg SLIME system (Hewitt, Freeman, and Brooks, 2012). One should always launch sdf.sys.network via that message in sdfsys (rather than opening the maxpat in any other way) because there is a setup in sdfsys that uses the onebang object so that the first click on the 'sdf.sys.network open' message will load the meta-patch, whereas all subsequent clicks on the same message will merely bring that maxpat to the front. This functionality is important because the sdf.sys.network meta-patch has many sub-patches that will all overlap onscreen; having ease of access to the main patcher is crucial during its use in performance situations.

[n6.28] cf. (Freed and Schmeder, 2009)

(6.3.9.c) sdf.sys.reboot

The second most important piece of advice that I give when helping people to learn how to make things in MaxMSP – where the first is to do with regular saving of work – is to do with regular rebooting of the work. In theory – and unless programmed not to – a maxpat will load into the same state every time that it is started from scratch. Unexpected behaviour experienced in a patch can often be remedied by closing and reopening. Sometime restarting the MaxMSP environment (or even the operating system of the computer) may be called for, but in general, a patch reboot is all that is needed to reliably reinitialise the work.

For sdfsys, a reboot is the quickest way to reset the spaces network, and when experimenting with different configurations, new modules, and especially when developing something that includes meta-patch scripting of sdfsys, rebooting is a frequent action in the workflow.

Clicking the ; sdf.sys.reboot now message, at the top-level patch of sdfsys_beta, will load a floating window meta-patch that performs three time-delayed actions: (1) after an initial delay period, the sdfsys patch is closed; (2) after another delay period, the sdfsys patch is opened again; (3) finally the reboot patch closes itself. Pressing [esc] will cancel these actions by sending a stop message to the line objects that drive a visual representation of a countdown in time towards each action.

If one is immersed in the sdfsys environment, then one may reboot without cognitively exiting that frame of mind by parsing the following command:

send sdf.sys.reboot now

Note that the word 'now' at the end of that command could actually be anything because it is only used to trigger a bang to open the meta-patch.

(6.3.9.d) Loading other meta-patches

Any maxpat file that is in the MaxMSP search path can be loaded by name from the sdfsys command line with the 'pcontrol' keyword that passes the all of its arguments to MaxMSP object of the same name. The following example opens the meta-patch saved – in the sdfsys_beta1 folder – as controlp.maxpat:

pcontrol load controlp

maxpat
Figure 6.24: controlp after loading seven of the spaces

The controlp meta-patch (Figure 6.24) is based on the human_seeking meta-patch that was created for as an aid to me playing in the sdfsys environment during a HELOpg performance at the Human Seeking event;^[n6.29] quotation after promotional image for that event (Angela Guyton, 2012):

[n6.29] I wrote about the event shortly after; http://www.sdfsys.info/blog/2012/06/06/human-seeking: Other performances at the event included Richard Knight who had controlled feedback loops in and between two mixers without external input, and Takahashi�s Shellfish Concern (Angela Guyton, and Rodrigo Constanzo) who combined live painting with both audio- and visual-domain DSP.

Let's explore SONIC & VISUAL information through IMPROVISATION
The role of TECHNOLOGY is considered, with each artist prioritizing it differently [re] their own practice
& HUMAN CREATIVE POTENTIAL

MACHINE FUNCTIONING
ANIMAL BEING
HUMAN
SEEKING
SUBLIME TRANSMITTING

The sdf.sys.dv module was also created for that performance; it is a module unit that loads into a space but then also opens a meta-patch (the first module to do this). The meta-patch loaded by the sdf.sys.dv module has settings for using digital video (DV) input, and a single parameter-sprite in the sdfsys environment is used to switch between different mappings from the three-plane RGB char input to the one-plane bipolar audio-type float32 data-matrix. By using DV input to the data-matrix of a space in sdfsys, human movement and shape, beyond that afforded by qwerty and mouse, can be used in the soundmaking process.

(6.3.9.e) The sdfsys_beta_xnn_video series

Five screencast recordings with live commentary form a short series exploring sdfsys_beta by introducing the modules set out in the controlp meta-patch. The series begins with reference to walkthrough video of sdf.sys_alpha; see §(6.2.4.d).

Located in the sdfsys_beta_xnn_video sub-folder of the sdfsys_b_doc folder, the videos of this series (totalling approximately 42 minutes duration) are:

sdfsys_beta_x00_thisis.mov (5 minutes 18 seconds);
sdfsys_beta_x01_tildegraphing.mov (10 minute 54 seconds);
sdfsys_beta_x02_dacpond.mov (8 minutes 50 seconds);
sdfsys_beta_x03_adcspiroid.mov (10 minutes 55 seconds);
sdfsys_beta_x04_dv.mov (6 minutes 23 seconds).

← 6.2 Alpha

6.4 Composing with sdfsys →

↑