During the residency in Sussex, I am looking with Chris Kiefer at possible approaches to take using machine learning algorithms.

Together we look into Echo State Networks (other terms: 'recurrent neural networks' and 'reservoir computing') and Conceptors.

The special feature of these algorithms is, is that they can deal with time series data and have memory, in other words: ideal for dealing with realtime sensor data.

After a few explorations we come up with the following scheme to try out:

• the system will have three modes - buttons are used to switch between the modes:

  • learn: the network will entrain itself to the incoming accelerometer data. The network should start oscillating along with the incoming data. Once this has happened, the performer can decide to store the pattern as a conceptor.
  • morph: the network is driven by a mix of the learnt conceptors. The mixing or morphing factors are determined by the stretch sensors.
  • spectral radius: the spectral radius, or the internal weights, of the network are modified by the stretch sensors, this will amplify or diminish the oscillations that happen in the system.

• we use a grid search to find the best networks: that means that we will be training a multitude of networks (e.g. 10) and pick the ones with the lowest/best error rates.

• the output data of the system will be used to change the hue and intensity of the lights projected onto the cocoon.

• the activation vectors will be sonified.