Headtracking for Binaural Ambisonic Playback

Low-Budget headtracker mounted on headphonesHere is a short tutorial about an inexpensive headtracking solution which you can use to improve the binaural listening experience with the ambix Ambisonic tools. The calibration software is currently Windows only, but the Pd patch is naturally cross-platform.

It uses the headtracker described here by Dennis Frie: http://www.rcgroups.com/forums/showthread.php?t=1677559
Parts: arduino nano, GY-85 sensor board.

A Pure Data patch provided by me receives the serial data from the Arduino based headtracker and sends OSC messages to the host of your choice. In this case I will describe the use within the digital audio workstation Reaper.

downloadFor the impatient, here is the Pure Data patch (use with Pd-extended):
pd-patch-headtracker-osc-sender


1)
First you have to get the Arduino and Sensor Board, some soldering and upload the firmware according to the instructions here.

Use the Axis Method for calibrationUse the provided Windows software to calibrate the tracker – for me the Axis Method works best. You might have to repeat the calibration from time to time.

2) Now you can use the Pd Patch I provide to communicate with the tracker. Use Pd-Extended to open it!
Download: pd-patch-headtracker-osc-sender
This patch will translate the serial data from the Arduino board to OSC messages.

reaper_control_surfaces3) Open up the Reaper preferences and add an OSC control surface. Important: Receive on port 8000 and Allow binding messages to REAPER actions and FX learn.

4) Add the ambix_rotator plug-in before ambix_binaural and use the OSC tracking data to control the parameters of ambix_rotator:
Click the Track Envelopes/Automation Button of thReaper Track Envelopes/Automatione Track with ambix_rotator to open up the Automation controls. Now click the Learn… Button next to the parameter Yaw.

Now you have to send a OSC message from the Pd patch to attach the correct message to Yaw, Pitch and Roll. Therefore don’t activate tracking yet (as it sends out all three messages Reaper OSC learn and Pd patchat once and thus does not allow to use the Learn functionality)
Instead click the buttons below OSC_LEARN in the Pd-patch to trigger single OSC messages.

5) You may have to change the serial port in Pd. First click the button list_dev to get a list of you serial devices. Enter the serial port id in the number box open_port.
Now you can activate head tracking and enjoy a stable binaural surround image. Use reset_orientation to define your zero degree direction.

midi2rms – midi volume fader curve sampling

volume-behaviorA very crucial and maybe often underestimated point for working with (volume) faders, especially in live electronics application is their curve/behavior. Which range does it cover? How fast does “it” get “loud”. What is the behavior around the +0 dB mark?

I sampled the midi volume fader behavior from several of the most popular Digital Audio Workstations. The difference is quite obvious as you can see in the graph to the right.

pd-fader-behavior-screenshotI made a Pure Data abstraction as replacement for the frequently used [fadtorms] or [dbtorms]. In connection with the .txt sample files it can be used to scale your midi fader input.
Get the abstraction including sample files and Matlab plots here:

midi2rms_0.11.zip

pd-external: pix_openni 0.10

If you are working with Xbox Kinect Sensor here is another goodie for you – A pure data / Gem external to get Kinect rgb and depth streams as well as skeleton tracking and multiple hand tracking via OpenNI[ref name=”openni”]http://www.openni.org[/ref]

Download the packed Binaries for OSX and Windows pix_openni-0.10.zip .
Latest sourcecode available at github: http://github.com/kronihias/pix_openni

Look at this page if you don’t know how to install a Pd-External.

To control tilt, led and output accelerometer data you can use libfreenect based http://github.com/kronihias/freenect.
To get audio streams you can use libfreenect based http://github.com/kronihias/freenect_audio.
Continue reading

DarwiinRemoteOSC Mountain Lion

darwiinremote-oscI fixed the program DarwiinRemote for Mac OS 10.8 (Mountain Lion) and added Open Sound Control (OSC) support for the most recent version. Download it here:

DarwiinRemoteOSC-2012-12-24-09-47.dmg

DarwiinRemoteOSC allows to connect via Bluetooth to the WiiRemote controller. It also supports Nunchuk, Classic Controller and Balance Board.
All data from the Wii Controller is forwarded via OSC. IP and Port can be changed by command line argument. OSC can be sent to DarwiinRemoteOSC to turn on Force Feedback, LEDs and retrieve Battery status.

Sourcecode available on Github.

Continue reading

Homage to Luciano Berio (1925–2003)

Lithuanian Ensemble NetworkI was invited to perform with the Lithuanian Ensemble Network (LEN/LAT) at the “Jauna muzika” Festival 2013 in Vilnius and played the live electronics for the pieces:

  • Naturale for viola, percussion and recorded voice (1985)
  • Différences for ensemble and tape (1958-1959)
  • Altra voce for mezzo-soprano, alto flute and live electronics (1999)

Continue reading

The Kinect distance sensor as human-machine-interface in audio-visual art projects

excerpt of project thesis

excerpt of project thesis

I finished my project thesis including general information about the Microsoft Kinect sensor device as well as a detailed documentation about it’s application in combination with Pure Data and Gem.

This document might be useful for people working with the Kinect in combination with interactive sound and video installations and performances.

Get the PDF document here!

Live Interfaces

I’ll be presenting my work for the Kinect sensor and Pure Data/Gem at the LIVE INTERFACES: Performance, Art, Music[ref name=”li”]Live Interfaces http://icsrim.org.uk/liveinterfaces/[/ref] conference.

Find the material here:
lipam2012-kronlachner-paper.pdf
lipam2012-kronlachner-poster.pdf

A substantial project report about all Kinect plugins and examples will be available soon.

pd-external: pix_freenect 0.10

(C)2011 Matthias Kronlachner

Version 0.10 of pix_freenect external for Pure Data/Gem now available.
Sourcecode and binary for OSX and Ubuntu 11.10 available at github: https://github.com/kronihias/pix_freenect

Look at this page if you don’t know how to install a Pd-External.

The pix_freenect external can output rgb and depth simultaneously from multiple Kinect sensors.

Features:

  • open specific Kinect sensor by serial number – useful for multiple Kinect installations
  • output registered depthmap -> rgb and depth image are aligned
  • output depthmap directly as 16 bit [mm] values
  • resolution of rgb image can be set to 1280×1024

To get audio streams you can use http://github.com/kronihias/freenect_audio.
Continue reading