Samstag, 17. Dezember 2011


in this hack i take control over a 70EUR rc tank by shortening the remotecontrols buttons with optocouplers. the tank is tracked with REACTIVISION, and a PURE DATA patch generates the control signals that make the tank follow paths from an vector drawing or generative algorythms. i use a spare channel of the remote to control a servo for the pen.

The Tank stands for the evil and lunatic devices created by a devilish industry for the profit of a small group of irresponsible selfish businessmen, generating fortunes in arms industries while the rest of mankind lives in fear or even agony as a result.
Their latest bestsellers are radio controlled killing machines - drones, UAV's, smart bombs, high-tech war machinery that uses a computergame like interface to kill real people.
Being a children's toy, it reflects the procedure how an attitude gets inherited from one generation to another.

I want to break that circle symbolically by imparting the tank with abilities of expression.
These include user reactive behaviors combined with choreographic elements and imagery.

the image below is a computer simulation for a rasterizing algorythm

this is my first project here at the class for digital art in vienna.
it is very good fun here.

Donnerstag, 21. April 2011

ZODIAC - open source GROOVEBOX and FLOW based SEQUENCER for Pd-extended

download it here

create patterns, and link them together.

A - add a point (representing a pattern) to sequence
W - link points with wires (press the circle in the middle of the wire to de/activate the link)
N - edit point data (Pattern Nr + Channel Nr.)
B - bang a point to start chain reaction , NOTE: it needs PLAY to be on.
PAT - (in N) click this to select a pattern from Numpad (patternnr. appears in middle of point)
CHN – (select a channel 1 – 4 from Numpad (point color represents channel)
X – clear all points and wires
L – load a “patch” (not a pd one) by selecting a number from Numpad
S – save all points and wires to txt.file by selecting a number from Numpad
this has 2 functions: first, it represents the notes of a pattern and second it serves as a NUMBERPAD
(1-16), whenever a number needs to be selected.
using the 4x4 Matrix you create the patterns. A pattern consists of 16 NOTES. Click on a square in the
matrix (a NOTE) and it will indicate that its active with a little square in its top left corner.
Patternmaker outputs in channel0, so turn it on.
Every note holds the following 5 values, that you set with the buttons and sliders of :
1. pitch slider (PIT)
2. volume slider (VOL)
3. off-beat slider (OfB): slider up - early, middle – on beat, down – late
4. sample number (s#) select samplenumber from Numpad
5. sample set (SET) select a set of samples via Numpad: as sample number (s#) only allows
16 different wav files used as samples, you can create up to 16 sets of samples, containing
16 files each. As I dont have so many samples, only sets 1 + 2 work.
All these values (except offbeat) show in the corresponding note-square in Patternmaker.
This helps to speed things up as you don't need to set all values for every note separately:
COPY – copy the selected notes values to clipboard
PASTE – select another note and press paste to fill it with the values you just copied.
L – load another pattern via NUMPAD,
NOTE: changes you made to the previous pattern will be discarded unless you press:
S – save a pattern
NOTE: this saves the pattern temporary to an array and by that, changes the specific
pattern in sequencer.
Changes will be discarded when you close pd UNLESS you save using [S] in
MVOL - master volume slider
SPD – metro speed
PLAY – well...
S – saves all notes in all patterns you made to txt.file. It also saves the settings you assigned to the
Effects section, which will be described below.
NOTE: THIS DOESN' T SAVE anything in the sequencer!!! sequencer has its own save/load section.
L – loads another set of patterns and effect settings from txt.file.
we have 5 channels:
Channel0 is used by the Patternmaker.
Channels 1-4 by the Sequencer.
Turn them on to hear something.
NOTE: Channels 1-4 have 4 sample players each. That means, they can play 4 samples at a time.
If you have more than 4 circles of the same color active at a time, samples will get cut or lost.
You can add more players in 0_player if you wish.
this holds 10 sets of 4 values each (set by the sliders below) for effect control.
MAS controls the volume of channel 1 – 4 with the sliders below
ChP controls the Pitch of channel 1 – 4 with the sliders below
ChV is currently not used
FX1 – FX7 holds control values for 7 different effects. You need to wire them in the Player patch and
give the effect the name of the effect control as creation argument; for example [pitchshifter FX4] if
you want to listen that effect to the values sent by slot FX4.
NOTE: It depends on the wiring in Player patch which channel(s) are being affected by an effect.
NOTE: all changes you make to the effect slots get saved, when you press S in GLOBALS.
When loading a file the values are there, but are not sent to the player patch, until you select a slot and
change a value.

Sonntag, 13. März 2011

3d modelling with KINECT + ARDUINO + PD

i did this to check out Kinects precition
and usability for Virtual Reality applications on one side -

and on the other

to create algorythms to SELECT / MOVE / LINK Objects
for multitouch Applications

which is all part of my attempts in developing
inovative human/computer interfaces.

while working with kinect, i realized pretty soon that
its quite annoying to percept something (very essential)like a MOUSECLICK
only from the sceleton Data. I made an attemt that makes
clicks occur by pointing at an object and then straightening the arm,
but this was far too exhaustive after a while.

so i decided to add comfortable clicking posibillities with Datagloves.

i had an arduino and some resistors lying around, went to Lagerhaus ------ARDUINO---GLOVES--HANDSTATES
bought some fancy gloves for 3 euros and soldered this together:
using 1 analog pin per glove i get 4 differnt currents for each fingerstate.
the arduino connects with pd using pduino.
of course it would be much nicer to have a wireless version of this,
it could easily be done using a XBEE arduino that sends data over bluetooth.
and a rumblepack would be cool aswell, to provide haptic feedback.

HOW does it work?
Kinect sends a depth image of the user to ---------skizze1
OPEN NI / NITE which tracks the Users Skeleton and
sends 3D Positions of every joint to
Pure Data over OSC.
the arduino sends the Finger States to PD.

in PD this data is streamed to various patches to serve multiple purposes: --------skizze 2

Essential for 3D modelling is the virtual camera, that allows one to move around
the objects in virtual space.
the Fingerstates 2 + 3 of either hand are reserved for this.
with one finger2 pressed the camera orbits around its target. ------------navigation-----
with one finger3 pressed camera and target moves left/right + up down in screenspace.
also i added head tracking wich moves the camera up/down/left/right/back/front in screenspace.

Finger 1 is reserved for ----------zeitraffer

Here i need to satisfy 2 different requirements.
once i want to Select by pointing at something at the screen for the 2D menue stuff.
for this i made a calibration algorythm that works by pointing
at the top-left and bottom-right corner of the screen. then only handpositions within
this pyramid are used, and mapped to screenspace.
2 cursors are calculated, making a total of 3 with the mousecursor.
selection is done by comparing the unitvectors of camera-->oject and camera-->cursor.

for the modeling i select things by moving my hands in 3d space.
so for this i fix the sceletons hands infront of the camera
to have them moving and rotating with the view.
this works by multiplying the sceletons world coordinates with the camera matrix.

the most demanding part was the multitouch logic, that forbids a certain cursor for other objects, once one is selected. if you select something and move it around, you dont want to drag everything else on the way with it.
so there is many crooks and hacks to avoid unwanted situations in here.

the rest was a piece of cake:
for the modeling i made 2D menupoints that switch between the 4 basic operations needed for mesh-modeling:
[create Verticle] [create QuadFace] [create TriFace] [Move] ----------------create a box and a pyramid

when [create Verticle] is on
a select- and move-able point with a unique ID ins created at the hands 3d position when finger1 is pressed

when [create QuadFace] [create TriFace] is on
a Polygon is created and linked to 3 or 4 Verticals by selecting them one after the other.

finally, i needed to save and load the results of my modeling work. ----------MAX
i did this using the same syntax as the .obj format,
so the model can be interchanged directly with 3dsMAX or other 3D apps.

and of course ------stereo
its double the fun in stereo.

credits go out to
all the people
who made this possible
by developing
these wonderful pieces of
open source software

Pure Data






and all the Kinect Hackers

Donnerstag, 11. November 2010

GEM-Engine 1.0 released!

GEM-Engine 1.0 is a collection of abstractions for PureData Extended.


It employs 3D navigation like you know it from 3D Programms like 3Ds Max or others in GEM, Pd's openGL engine.
It enables you to navigate in 3d space using your mouse, or other input devices. It provides objects to select and move things in space among many other useful little tools for creating your interactive realtime 3d scenes within Pd.

There is also a bunch of MaxScripts, that export coordinate lists of any kind of animated objects from 3DsMax to Pd. By this procedure you can use motion data produced by ParticleFlow, Physics Simulations, Biped-Skeleton Animations or whatever in GEM.
If you use another 3D Programm you will have to write your own export scripts, which shouldnt be hard to do. IF YOU SUCCEED please let me know, so i can include them in the package. I'd also be glad to hear your experiences with it, just drop me a mail

why PD?
  • span the gemwindow over a couple of gfx-card outputs using expanded desktop mode (9 screens @ 1280 x 1024 is the most i did)
  • use multiple inputs for machine user interaction like: Wii Controller (piped via osc from Karl Kenners Glovepie), MIDI devices, sensors with Arduino boards, Live Camera Tracking with openCV or EyesWeb, Wacom tablets, ARToolkit coordinates, SoundAnalysis… (procedures are platform dependend) ....actually pd can connect with whatever runs on electricity.
  • output multichannel audio
  • control mechatronics using Arduino, DMX or MIDI.

All this makes my little Gem engine a good place to start developing interactive multimedia installations or sound reactive VJ sets.

Samstag, 10. November 2007

republik der sehnsucht

a fried and me are currently working on a short called "Republik der Sehnsucht"
in english "REPUBLIC OF DESIRE" about a guy and his girl. Its a modern musical with tunes of my fellow Markus Brandt, and we also did the 3d stuff together. Trailer is on utube
updates soon.

Donnerstag, 6. September 2007

Stretchy Biped Animation

i wrote a maxscript, which assigns a bunch of sliders to Bip01 arms, legs, neck, head and spines local x scale.
it makes it possible to make some freaky cartoon animation within charackter studio.
just name the bip in your scene Bip01, eveluate the script, and then you have a rubber char ; ) har, har!
its right here: