Joe Marshall

Swim Sensing Technology

admin — Fri, 10 May 2013 13:07:10 +0000

So, in my alt.chi paper, I argued that interaction designed for use while swimming was an extremely challenging thing to design and build. So, it seemed right that I should actually attempt to build some swimming technology and design some interactions using it. So, I set out to build a system for 'interactive swimming'. This page describes the current state of the swim sensing technology which I am developing. I currently have not built any applications using it. By that I mean a system which:

Is aware of the movement of the swimmer, their body position, speed through the water etc.
Provides real-time feedback to the user based on the swim position input.

There are a bunch of existing systems in the sports-science literature which use sensors to study swimming, and a few commercial products (eg. Garmin and Swimovate watches, Finis hydrotracker).. These fall into two categories:

Customised pools augmented with customised sensors, networks of custom sensors, sonar detection, video tracking, tow lines etc. to give extremely high quality data for post analysis of the swimming of elite swimmers.
Watches or similar simple accelerometer based objects that count strokes per length and lengths. Essentially, these just mechanise the task of counting, and save the swimmer from doing it.

These are interesting, but I want to do something slightly different. I would like to build systems which provide real time response to the swimmer, and do more than just count strokes or lengths. I also am more interested in lower level swimmers, un-coached swimmers, people swimming in public sessions, people who are not purely performance focused in their desire to swim, which means that systems relying on high levels of instrumentation of pools are likely to be overkill. Clearly there is a trade-off in that I am unlikely to achieve such high data quality, but I think it is worth it for the potential to use the system with a more diverse user base. So, I put together a list of specifications for the technology.

Not augment the pool or require objects off the swimmer - as augmented pools limit use to very high level swimmers in conventional coaching settings.
Use commodity hardware or very cheap custom hardware, ie. low quality sensors rather than extremely expensive industrial or scientific sensors.
Provide real time detection of velocity, acceleration, body angle, strokes, in order to be able to respond to swimming in a more nuanced way.
Real time response to the swimmer

I then spent a few months hacking code, putting mobile phones in waterproof cases, analysing sensor readings, developing algorithms, going swimming, to come up with something which gives the results below. What this essentially shows is me swimming two lengths of my local pool, during a public session. I swum the first length breaststroke, did an open turn and swam back crawl. In the data you can see:

The clear effect of different strokes on velocity and acceleration
Detection of swimming direction
Detection of absolute position in the pool and a quite rough estimation of instantaneous velocity.
Detection of body roll and forwards/backwards angle.

All this data is completely real-time, with no need to wait till the end of a length to estimate velocity over the length. The system uses an unmodified Android mobile phone, which is strapped in a waterproof case to the lower back of the swimmer. We have audio out and headphones in order to provide feedback (and potential for vibro-tactile feedback using the built in vibration motor). Depending on the waterproof case in which the phone is held and the swimming stroke being used, changing the display colour can be used to show a light which could communicate information visually people on the side and other swimmers.

Mobile Interaction Does Not Exist at alt.chi 2013

Joe Marshall — Mon, 29 Apr 2013 12:32:22 +0000

I presented a paper at alt.chi 2013 on my ideas for interaction in motion. Alt.chi is a special venue at the biggest conference in human computer interaction research (ACM SIGCHI Conference on Human Factors in Computing Systems), which solicits papers which challenge the conventional approach of HCI, so the paper essentially takes quite an abrasive approach and suggests that basically no current mobile systems are designed to be used while actually mobile. As an example, I described the challenges of interaction in relation to the development of interacti.ve systems for use while swimming in cold water. It is really a position paper describing how hard it is to do truly mobile interaction, but not offering any solutions as such. The paper (PDF) In the presentation, I demonstrated the various challenges of cold swimming interaction by subjecting myself to them whilst presenting the talk. As I couldn't get a large pool of cold water organised, I used gaffa tape in order to simulate the various ways in which it makes it hard to interact, for example by taping up my hands to simulate the way that getting extremely cold hands forces them into a claw shape. Towards the end of the talk, I taped my mouth up to simulate how it is hard to talk whilst swimming, and a pre-recorded voiceover played for the last bit, during which I pretty much taped my whole body to a table - as you can just about see in the grainy twitter pic below. .

Interaction in Motion

admin — Tue, 20 Nov 2012 13:39:13 +0000

Starting October 2012, I am working on a 3.5 year research project studying 'interaction in motion'. This project is funded by Leverhulme, under their Early Career Fellowship scheme. In this project I am creating and studying interfaces designed to be used by people when they are actively mobile. This is in contrast to most current mobile interfaces which are designed to be actively used only by people who are stopped and looking at a screen. Even systems designed for outdoor and transport use typically require the user to stop in order to do any active interaction with the system. In this project I am studying how technology might work if we are moving in an active manner while interacting, for example when running, cycling, swimming, horse riding, motorbiking or otherwise travelling outdoors without stopping. To study this I'm going to build a bunch of interesting systems to play with, probably primarily focusing on small artistic projects, but I'm also interested in more application focused and stuff too, and am also doing some small focused studies. I'm actively seeking collaborators for this work, if you're interested in any of this, please drop me an email.

Why study this

Movement activities, as well as getting us from one place to another are also in themselves a valuable element of human experience. As well as primarily functional movement, such as commuting to work, many of us also take part in movement activities such as running, walking, swimming for the activity itself; this can be for many reasons, for the fun of the activity, for the excitement of seeing new places, as a physical challenge, in order to encourage physical and mental health etc. Movement and exercise are fundamentally interesting and valuable activities in their own right. So, I argue that we should design interfaces for use during movement for two key reasons:

The intrinsic value of movement as an activity means that there is an argument for doing anything we can to support people in taking part in movement activities.
As I say above (and in slightly more detail) the reality that people do use mobile devices during movement activities means that as interaction designers, we should be designing to support this reality.

Why interaction in motion is hard

admin — Tue, 20 Nov 2012 13:38:55 +0000

Designing systems and user interfaces for active use during movement is hard. If we lose the assumption that a user is always standing still and able to pay full attention to our interface, interface design suddenly has to cope with a far wider range of situations. This page contains a preliminary and non exhaustive list of some challenges created by movement interaction. Movement and attention Numerous studies of use of technology in cars, such as sat nav devices and mobile phones, have shown that our ability to pay attention to driving cars as well as interacting with a device is limited. Focusing on the device can reduce our ability to respond to events occurring on the road, and cause safety issues. This issue of a limited ability to pay attention to multiple activities generalises to other types of movement - for example if we are running hard it is harder to pay attention to interactive systems, and if we are paying attention to an interactive system, it is harder to give our full attention to what we are running over. Physical constraints of movement Movement activities can be physically constraining in ways which limit our ability to interact with devices whilst moving. This can be for several reasons: In some activities, the activity itself demands some kind of physical manipulation of equipment, such as bike handlebars, car steering wheels or ski poles; the way in which we are currently moving may reduce or increase the amount we are constrained to manipulating this equipment at any point. As well as equipment, there are elements of physical form inherent in movement activities that constrain interaction with interactive devices, for example when swimming with a full body stroke, the movement of our entire body is constrained by the stroke, there is an extremely small amount of leeway available for physical interaction with a device. Even in running, an activity where much of the body is relatively unconstrained, the need to look ahead, coupled with the need to move ones arms to run efficiently makes current interactive devices such as mobile phones hard to use whilst running (or even whilst walking fast). Physical, digital and social terrain When we are moving, the terrain we are moving over has a large amount of impact on how much we are able to pay attention to a device, and how physically constrained our movements are. For example in the picture to the left, I was foolish to run up a rocky path whilst using my phone, as I clearly was unable to pay sufficient attention to the phone and the terrain I was moving across. I also really required my hands free to balance whilst running up a steep and slippery path of rocks and mud. In many activities, such as swimming, or even walking in wet weather, the terrain in which the activity is being performed may be actively hostile to electronic devices. Terrain also has a (normally invisible) digital element, network connectivity, access to location services such as GPS, amounts of electrical lighting and even access to electrical power being very much location dependent. Social factors relating to locations may also create a further level of 'terrain'; social factors such as crime rates or our familiarity with an area may have a large amount of influence on how willing we are to openly interact with an expensive device. Designing for movement means designing to take account of the fact that all these factors may change whilst interactions are ongoing. The Visceral nature of movement As discussed, many movement activities are not purely used as ways of getting around, even when this is a secondary purpose, such as on a bike commute. People do them for the enjoyment of doing the activity in itself. A major element of this enjoyment can be the visceral, exciting nature of some movement activities. 'Extreme sports' such as downhill mountain biking and big wave surfing are very much focused on visceral thrills, but arguably many less extreme pastimes involving movement such as going for country walks are motivated by some kind of visceral enjoyment of the activity. Designing systems in particular for extreme activities requires us to build designs which are sensitive to this visceral element of the activity. This challenge very much builds on our work in the Mixed Reality Lab on 'uncomfortable interactions', in that it is typical in many extreme activities to push oneself deliberately beyond what would normally be considered comfortable, and our interactive systems must support this.

Mobile devices do not exist

admin — Tue, 20 Nov 2012 13:36:50 +0000

I would argue that the design of almost all mobile devices is such that they arguably are not meant to be used whilst actually mobile. Instead, they are actually devices which whilst they offer a large amount of portability, are not designed to be actively used while moving.

Stop to interact design

The vast majority of mobile systems are essentially designed around the concept of stop to interact. They are mobile in that they are portable, but in order to actively interact with a system, the user must stop moving, look down at it, interact with the device, and then get going again. Some systems, for example music players, and navigation systems, at least output to the user in between these stops, but they still mostly follow the same stop to interact design.

The reality of mobile device use

Whilst systems are designed for active use only when stopped, in reality, people do use mobile devices while moving all the time. This causes all manner of problems:

Social issues such as 'pavement rage' when people are too busy looking at their phone to pay attention to others they are sharing space with.
Physical issues when people walk without looking, such as tripping over, or walking into the road
Driver distraction by mobile device use, which can cause people to crash their cars.
Software issues when people press the wrong thing, such as sending messages by accident.
Dropped and broken devices.

But what about speech recognition, customised buttons (etc)?

Existing workarounds, such as voice recognition and action specific buttons (eg. headset 'answer call' buttons, music buttons), allow a certain amount of control of systems, but are limited in their application or success so far. Speech recognition in particular appears to offer an awful lot of promise (and has been about to become mainstream since approximately 1995); however, even that is somewhat limited when moving - microphones have to be well placed to catch the sound, whilst not exposed to excessive noise from surroundings, the wind, or whatever equipment is being used by the person in their movement activity.

Poke that Goat

admin — Wed, 07 Nov 2012 15:28:07 +0000

Poke that goat is a 2 to 6 player game for mobile devices. Each player chooses a goat, and then they must try and poke their goat as many times as possible. The first player to poke their goat 20 times wins. Because all the action is on a single screen, this leads to a certain amount of chaos, as each player tries to poke their goat whilst trying to block the other goat pokers. I made it for two reasons: Firstly, I wanted to see how a large number of players playing on one device at the same time actually worked. Secondly, I decided that at two and a half, it was time for Rose to have her first computer game (see also her first synthesiser). It is currently available for Android devices - download it now for free from Google Play.

Uncomfortable Interactions (ACM CHI 2012 Best Paper award)

admin — Sun, 20 May 2012 14:04:16 +0000

Our paper Uncomfortable Interactions (Steve Benford, Chris Greenhalgh, Gabriella Giannachi, Brendan Walker, Joe Marshall, Tom Rodden) won a best paper award at ACM CHI 2012. PDF version. This paper discusses situations in which interface designers may wish to deliberately create discomfort as part of an experience. It focuses on two experiences - the Breathless interactive swing ride, and Ulrike and Eamon Compliant, an interactive performance by Blast Theory.

I seek the nerves in Leonardo Journal

admin — Sun, 27 Nov 2011 16:57:00 +0000

An article in Leonardo Journal about I seek the nerves, aimed at interactive artists and similar types. Download the PDF.

Arduino Octo-Synth

admin — Tue, 15 Nov 2011 15:30:16 +0000

So, my daughter Rose was 18 months old, and I decided it was time for her first synthesizer. Apparently this is not normal, my wife didn't ever have a first synthesizer (mine was a Yamaha PSS-680), but personally I think everyone should have a cheap and nasty electronic musical instrument at some point in their life! I had a bit of a look round, and you can't really buy toddler friendly musical instruments that aren't rubbish. So I am currently trying to build / program one myself, based around an arduino micro-processor module I had hanging around. I had a bit of a requirements/specifications think first and came up with:

It should respond quickly without any obvious delay.
It should be able to play actual tunes and have an octave of in-tune notes.
It should be able to play chords, and/or make horrible noises when you hit lots of notes at once. Kiddy keyboards where it just plays one note at a time are annoying.
All the delicate electronic parts should be inside a box away from prying hands.

I made a quick cardboard and tin-foil prototype of an 8 key capacitative sensing keyboard which worked surprisingly well and is jolly clever - each key only requires one wire, which minimises gubbins in the circuit, and there are no exposed moving parts, which should hopefully reduce the amount of toddler damage. I'm thinking of using some kind of metal, something like a bolt through the box for the final version. As a prototype, I've made a synth in a Hazer Baba box and a bunch of carriage bolts. I've written the synth engine now - features:

8 independent pitch and volume oscillators for nice chord sounds
Touch sensing for chord input (up to 8 notes at once)
Touch sensing for filter modulation and pitch bend
Square, sine, sawtooth, triangle waves
Resonant low pass filter
All the hard bits written in assembler, so oscillators are fast (14 processor cycles per sample), clipping is handled nicely, and the touch sensing code is extremely accurate (it senses the touch capacitance using an unrolled loop with 1 input read per cpu cycle). I think that the oscillator main loop is the minimum possible number of instructions per wave sample (see in the code for why I think this).
There is time to run a little bit of other stuff on the arduino (there would be tons of time if it didn't have 8 oscillators constantly running - 1 or 2 oscillators take far less time.) Without the resonant filter it will happily play 16 independent oscillators at once.

Current version of the source code: octosynth-source. Video of Rose playing with the first boxed version of the Octosynth Video of pitch bend and filter modulation: Video of a first amplifier test version - I've since fixed some electronics bugs that were making the amplifier distort a lot, and it sounds a fair bit better, will do more videos soon. It is very hard to hear on the video as I'm just using a piezo buzzer while I wait for some headphones out plugs / an amplifier chip etc. in the post, but I've previously made a couple of quick test videos of the first (very far from toddler proof) setups.

Breathalising Games: Understanding the Potential of Breath Control in Game Interfaces (ACE 2011)

admin — Sat, 12 Nov 2011 16:51:46 +0000

by Paul Tennent, Duncan Rowland, Joe Marshall, Stefan Rennick Egglestone, Alexander Harrison, Zachary Jaime, Brendan Walker, Steve Benford. Paul Tennent, Duncan Rowland and a few of us put together this paper (PDF) for ACM Advances in Computer Entertainment 2011 (ACE 2011). The paper is based on a selection of breath based games that we have worked on, using the gas mask breathing interface. These games are (read the paper for more details obviously):

Serious Sam Aim Control - A modification to the first person shooter Serious Sam which makes aiming of the gun be perturbed by breathing, meaning that to shoot straight, you have to time breathing with shots, or hold your breath.

Hyperventilation Sports - a twist on the early joystick based computer sports games that required frantic joystick waggling to go fast. In Hyperventilation Sports, you have to breathe as quickly as possible, whilst trying not to overdo it and have to stop. This plays on the physical limitations of breathing, and flirts with the danger of hyperventilation (although games are deliberately short enough to avoid any real danger).

PerPing - a two player breathing controlled tennis game - breathe in to move the bat up, breathe out to move it down, hyperventilate to split the ball in two. Miss the ball and your opponent scores a point. PerPing was run for a large audience at Cheltenham Science Festival (read more about Cheltenham).

Sneak-em-up Players have to sneak past guards without alerting them to their presence. If a player is breathing more heavily, the guards can hear them from further away. This creates a game mechanic where people must strategically hold their breath to pass guards.

Tunnel Run is a game for two players. One player's breathing controls the shape of a tunnel shown on the screen. The second player's breathing controls the flying of a plane going through the tunnel. The first player must try and make the tunnel as hard to fly through as possible, whilst the second player tries to fly for as long as they can. Players take turns creating the tunnel and flying through it, and are scored based on how long they can fly safely for.

Broncomatic at GameCity 2011

admin — Sat, 12 Nov 2011 12:01:52 +0000

(picture by nottinghamgamecity)

The Broncomatic went to the GameCity Festival, where it was run by Stefan Rennick Egglestone with the help of some other people from the lab, as I was busy doing some other work. 116 riders rode it during the day, and there were pretty decent crowds of people watching from the sound of it.

I did some work before this event on making the control panel more sturdy and resilient, and it held up fine, even with quite a large number of riders, which was a big relief.

Broncomatic on Daily Planet (Discovery Channel)

admin — Tue, 01 Nov 2011 11:24:33 +0000

We showed the Broncomatic and related bits and pieces to the Discovery Channel for a segment on Daily Planet, which aired on 1st November 2011. Some great close-ups of the Broncomatic control panel construction. Currently (18th November) the clip is available at the link below: Discovery Channel Broncomatic Clip.

Patio Tile-o-matic

admin — Wed, 13 Jul 2011 14:50:53 +0000

Want to lay a patio with a random layout? If you have a load of slabs of different sizes, and want to lay a patio, working out a layout can be fiddly. You need to make sure that you fit in the slabs, taking account of the border shape of the patio, the size of the slabs, how many slabs you have etc. In addition to that, there are a couple of rules you need to follow, to avoid things that look bad (see the brilliant Paving Expert site for more information). There are a few programs out there that let you generate a random layout. But at least all the ones I've found have two problems: 1) They don't follow the rules of making a nice looking patio, so you often get cross patterns where four slabs meet, or long unbroken lines along slab edges. 2) They don't allow you to set limits on how many of each slab you have, which is a pain, as stone merchants often sell slabs in convenient 'patio packs', with a fixed set of slabs to cover a certain number of square metres. So, to do our patio, I hacked together this software, Patio Tile-o-matic. It works with a set of slabs where all edge lengths are multiples of the width of the smallest slab, or roughly so. For example, my slabs were: 290x290mm 290x600mm 600x600mm 900x600mm which are all roughly multiples of 300mm. You can tell the program how many of each slab you have, by pressing 'M' to open the materials editor. All measurements are in 'grid size', which is a multiple of the smallest slab width - for example with mine, I had 1x1 slabs (290x290) 1x2 slabs (290x600) 2x2 slabs (600x600) 3x2 slabs (900x600) To use it, you first need to measure the shape of your patio. Then, take the smallest slab width, and create a grid of that size aligned to your patio, for example my patio was 400mm long, so I had 13 300mm grid squares in that direction (with a bit of slack left for pointing). In the program, you need to edit the patio shape to be the size and shape of your patio, press E to go into the patio editor (or select it from the menu), and click to make the bits of your grid that are not part of the patio black, and the bits that are, grey. Looking at the image above, you can see my patio has a wider section at the top. The simpler and more square you make this, the more likely you will get a decent layout. If a grid square is half patio and half outside, make it grey, you'll need to do a cut there (you might want to play with the alignment of your grid to the patio shape to reduce the number of cuts) Once you have done your patio shape, press R to generate a slab layout. If you can't generate a layout, check 3 things: 1) Do you have enough slabs to cover the whole space. 2) Try smoothing the edges of the patio to make them less complex - you can always do cuts if you have bits of wall coming into the patio or whatever. 3) Layout happens from the top downwards. There will never be cuts on the top edge - so if you need an edge to be easiest to lay, orient your grid that way round (I put this end next to the house for my patio). It may be easier for it to lay out if the widest edge is at the top. If you like the layout, press 'S' to save it out, this saves it as three files, in the directory that the program lives in - one file is a nice coloured and labelled slab layout diagram (as a png image), the second file is a slab layout in grey, so you can see what it will look like a bit more clearly. The third file is a .slb file, which you can open up again to reload your layout in the software. You can also add 'information boxes', by pressing 'I'. These let you label elements of the patio layout, such as manhole covers, in order that you can see how many slabs you will need to mess with (for example I generated a load of patio layouts, because I wanted to have big slabs over the manhole cover so as to minimise the number of slabs that needed cutting). You can alter the layout constraints by pressing 'C'. Layout constraints stop the layout having long lines of slabs, and also stop it allowing four corners to meet. Bear in mind: 1) The diagrams produced from this do not take account of differing pointing widths - with slabs like I had, where the small ones have 10mm less width to make pointing neater, this is no problem, with more evenly sized slabs, you may find yourself having to fiddle things slightly on the ground. 2) This software presumes everything is a square grid, due to the nature of real stuff, if you have an space enclosed by walls or something, you will probably find yourself doing cuts even if you generate something that theoretically fits perfectly into the space. 3) It is only as good as your measuring and gridding. Make sure you measure accurately (leaving some space for pointing). 4) For some complex designs, or if you don't have enough stone, this just won't work. 5) This is only any good if you want to lay everything to a rectangular grid - if you want lovely curving stuff, you're on your own. 6) The whole thing is a dirty hack, so don't expect it to be that polished. It worked for me, and I have a patio thanks to it, but I can't guarantee much more! On the plus side, for the job I had to do, it is much better than anything else I could find. I'm releasing it here, for free, no guarantees or anything. Windows only I'm afraid. If you use it and like it, please do let me know. If you use it and really love it, you're welcome to paypal a donation to "paypal at joemarshall.org.uk". Download it from here.

Sensory Threads in Leonardo Transactions

admin — Thu, 07 Jul 2011 17:09:48 +0000

We published a 2 page description of Sensory Threads in Leonardo transactions. PDF

PerPing at Cheltenham Science Festival

admin — Mon, 20 Jun 2011 16:40:56 +0000

"Breathe in, and your bat goes up, breath out and your bat goes down. Breath quickly and your bat size increases. Hyperventilate, and the ball splits into two. Miss the ball and your opponent scores a point" PerPing is a further use of the gas mask breathing interface originally developed for the Breathless breathing controlled swing ride. We used the breathing interface to make a selection of games, including PerPing, a two player breathing controlled tennis game somewhat inspired by the classic early video game Pong. We demonstrated PerPing at Cheltenham Science Festival, where several hundred people had a go, and thousands of people watched them play. PerPing and the other breathing games we designed are also the subject of a paper at ACM Advances in Computer Entertainment 2011.

Using Fast Interaction to Create Intense Experiences (CHI 2011)

admin — Tue, 10 May 2011 18:55:25 +0000

I presented this paper (PDF) on my running work at CHI 2011. I did a lot of work studying 'I Seek the Nerves', and used this to suggest some ways of using exercise as part of an interactive system. We suggest that rather than seeing exercise purely as something to encourage people to do with computer systems, we see exercise as something interesting and rich in itself. In particular we argue that doing hard exercise can make people increasingly emotionally involved with the experience which they are taking part in.

The Gas Mask: A Probe for Exploring Fearsome Interactions (alt.chi 2011)

admin — Mon, 09 May 2011 20:57:52 +0000

We published a paper (PDF) on the gas mask work in alt.chi 2011. Steve and Brendan presented the paper entirely whilst wearing gas masks, using gas mask microphones to put their voice into the PA system.

Breath control of amusement rides (CHI 2011)

admin — Mon, 09 May 2011 18:00:44 +0000

I presented a paper that we wrote (PDF) about the Broncomatic breath controlled Bucking Bronco ride at CHI 2011. For more information on the Broncomatic, see the Rides theme.

QuickGo for Android

admin — Tue, 12 Apr 2011 21:39:17 +0000

Android phones have lovely built in google maps navigation. However, the default 'car home' application, and the navigation application are a bit fiddly to enter addresses into compared to a dedicated sat nav system. For example, the onscreen keyboard has letters and numbers separately, with a button to switch between them, and it does not take up much of the screen, in order to allow for content. You also have to go through a bunch of steps to get to the point where you enter an address - click navigate, select 'type destination', then enter it using the small onscreen keyboard. QuickGo is basically a full screen keyboard for entering addresses or postcodes. Once you enter the postcode in, pressing the search button launches navigation to that address. Download it from Android Market on your phone. It is free, but if you like it, feel free to paypal me a donation (to paypal at joemarshall.org.uk)

Gas Masks at Saw Alive

admin — Thu, 04 Nov 2010 21:29:57 +0000

Saw Alive is a horror maze at the Thorpe Park theme park in London, England. When people go into the maze, it is very dark. There are live actors who pretend to attack them, and sets themed to look like torture scenes from the Saw films, complete with moving machinery, and a real electric fence. In this event, we took groups of four people, and split them into two pairs. Two people were kitted up with breathing sensing gas masks, microphones, and also infrared torches with a small infrared camera attached to them. The torch camera meant that people could film what was going on, but could not see any better than a normal participant. We used a small netbook in a bag to transmit audio, video and breathing data back to a base room, where the other pair were watching the breathing traces and video, and listening to the screams of the people in the maze. People who were second through the maze had already heard the screams of their friends in the maze, whilst not really being able to see masses of detail about what was happening to them. This really seemed to pump them up and make the experience seem more scary. We published a paper about the gas masks at CHI 2011.

Breathless in Hackney

admin — Fri, 15 Oct 2010 14:40:32 +0000

Breathless is a performance created by Brendan Walker and the Mixed Reality Lab / Horizon, centered around a large motor controlled swing. The swing is controlled by the breathing of a person wearing a special breathing sensing gas mask (for more pictures of the gas masks see Aerial's website). As the person inhales, a force is added to the swing mechanism in one direction, and as they exhale, in the other direction. This means that to swing higher, the rider must stay calm and breathe in sync with the natural pendulum motion of the swing. If they panic, the swing shudders and generally slows down (breathing directly out of sync stops the swing very quickly). The first performance of Breathless was at Space Studios in Hackney, on October 14th 2010. At this performance, people were first fitted with a gas mask, and put on the swing. However, at this point, they were not controlling the swing - a spotlight was pointing down at a second person, sitting on a chair next to the ride. At the beginning of the ride, the ride was controlled by the breathing of the person on the chair. After a while however, the spotlight changed, to point at the rider of the swing, and their breathing was controlling the ride. All the while, both people are wearing the full face gas masks, which makes the whole experience a bit more claustrophibic and sinister. This is part of our responsive / adaptive ride developments, and is the first ride that we have developed completely from scratch, from the breathing sensor masks, to the motor control, all the software in between (I developed the control software for the ride), and then the performance around it. We published a paper about the gas masks at CHI 2011.

Graffito at Vintage Festival

admin — Sun, 15 Aug 2010 13:22:39 +0000

Graffito is a multi-person drawing application, (written by Nick Bryan-Kinns with Jenn Sheridan/BigDog Interactive), which runs on phones (currently only on iPhones). It allows multiple people to draw on the same canvas at the same time. I've been working with Graffito as part of a team of various people led by Jenn Sheridan (see the Graffito website for more details). We ran Graffito at the Vintage @ Goodwood festival, in their late Eighties themed dance 'tent', which turned out to actually be a roofless enclosure, which led to much joy with weather! It was very neat though - big name DJs, lots of people dancing, and Graffito was bang in the middle of it, on the big (I think it was 30 foot high) LED screen on the dance floor. It was interesting to be embedded right in the event, essentially allowing random people in the crowd to provide the visuals for their dancing. People seemed to find it very exciting to be allowed to control the visuals themselves. People drew lots of different things, including their names, hearts etc. the obligitary penises and swear words (although here the collaborative nature of the system came in, as other people could modify the drawings or words if they felt they weren't appropriate). A particularly interesting point for me was in the first evening we ran it, when the dancefloor was pretty crowded, and the DJ (A Guy Called Gerald) was playing acid house music, and the visuals people generated really seemed to be matching the music in a good way - like the spliff smoking acid house face below.

Deception and Magic in Collaborative Interaction (CHI 2010)

admin — Mon, 12 Apr 2010 17:30:31 +0000

by Joe Marshall, Steve Benford and Tony Pridmore. I presented this paper (PDF) at CHI 2010, which is basically a shorter version of the work on the Cup Game and use of magic. I presented it using a hand drawn presentation that I did in an old notebook, which seemed to work well (and it was jolly quick to draw the diagrams this way). During the presentation, I demonstrated misdirection by creating a situation where I put a large pink soft toy elephant on top of my head, yet no one in the room spotted the elephant, which worked like a charm. Anyone interested in a very much more in depth version of the work in the paper (and more) might be interested in a look at my thesis, which covered this experiment, and also two other performance situations in which deception occurred. There is also a short video on that page. Oh and there is also some more stuff in the magic theme on this site.

Sensory Threads at CHI 2010

admin — Sun, 11 Apr 2010 16:21:30 +0000

We ran Sensory Threads at the CHI 2010 Whole Body Interaction Workshop. This was a slightly cut down version, but we managed to get sensor readings (sound, light, proximity and heart rate) transmitting over an American 3g connection. In this picture, you can see people playing with the noise sensor, by holding it next to a waterfall. The 'mini rumblers', pictured below, back at the base, each vibrate according to one person's sensor reading. The character of their vibrations is different - for example the heart rate rumbler pulses in time with the person's heartbeat, whereas the noise sensor is more of a fuzzy vibration, which gets stronger depending on noise levels. There is a lot of stuff to charge up for this project - here is all the gear charging the night before.

Sensory Threads

admin — Sun, 11 Apr 2010 13:06:41 +0000

Sensory Threads is a wearable sensing project which creates interactive audio soundscapes. How it works is a group of 4 people are each given a wearable sensor box, and a pair of headphones, and sent out into a city (or any other place for that matter). The headphones play back a 'soundscape' - a piece of music which changes based on the sensor readings. Currently, we are using very simple, easy to understand sensors - light, noise, heart rate, and a sonar proximity sensor (which gives a rough idea of how constricted a space someone is in). The interesting thing about Sensory Threads is that the 4 people all have a shared soundscape, and must stay close together (within about 5-10 metres of each other), yet each person has a different sensor, so they may be motivated to move in different ways. For example, the person with the proximity sensor may want to hide in small places to make the sensor go high, whereas the person with the noise sensor may want to go and stand on a busy street corner, where the cars passing will make interesting things happen. Sensory Threads is a collaborative project, various other people are involved in the work, including Proboscis, who are a London based art group and the Centre for Digital Music, Queen Mary University of London, who created the soundscape used in the project.

Broncomatic Experimentalism

admin — Wed, 31 Mar 2010 12:03:04 +0000

We set up the Broncomatic in the Exchange Building at the University of Nottingham Jubilee Campus. The Exchange Building foyer is an area where people come to have lunch, go to cash machines and go to the student union shop. Even towards the end of term, when university is pretty quiet, you can rely on a few passers by. We created a Broncomatic program which very directly translated breathing, with breathing in turning the bronco one direction, breathing out turning it the other, and the speed of turn modulated by the speed at which you breathe in or out. This means you can hold your breath and the bronco will remain pretty still. To be able to control the length of time people were on the ride, we set 3 progressively harder levels. In level 1, the ride spins slowly, which makes it very easy for most people to stay on and get used to their breathing controlling the ride. At level 2, the ride speeds up its spinning, which is a bit more challenging. On level 3, the ride also starts bucking up and down, and the spinning gets really fast - essentially all hell breaks loose at this point. We deliberately kept this first public outing of the Broncomatic low key, with no publicity or promotion, riders were just passers by, who came over and asked us for a ride. Even so, 22 people had a ride on the Bronco, and we filmed 21 post-ride interviews, which has given us a load of interesting things to think about. The little printer stickers with scores really seemed to make people happy - people were talking about the scores, and competing with each other. Some finish line camera pictures from this event are on the broncomatic's flickr page.

Introducing the Broncomatic

admin — Mon, 01 Mar 2010 15:53:56 +0000

As part of the work with the Bucking Bronco, I have been automating the control of it, so that we can make computer controlled adaptive bronco rides.

Rather than delve into the Bronco electronics, I have attached a wooden faceplate with a set of servos to the control panel that is used when manually operating the Bronco. There are several reasons behind this choice:

I wanted the computer's control of the Bronco to be very obvious, so spectators could see how it is controlling the ride as well as the effect on the ride machinery itself.
Aesthetically there is something very attractive about the Heath Robinson nature of the BroncoMatic - a big part of the beauty of the Bronco is the large machinery that runs it - I wanted to keep that aesthetic in the computer control.
No need to mess with high voltage electronics - the Bronco runs at 12 amps and 240v, enough to be quite dangerous to play with.
We keep the Bronco control box in good condition, so it still works for manual control

Some pictures and video of the early development of the Broncomatic are below. Obviously this is very early development - perhaps later versions may involve less gaffa tape, more accurately cut wood and something other than zip-ties, string and lego holding it all together!

First joystick test (quicktime) First test of Broncomatic with speed controls (quicktime)

The Broncomatic Integrator allows the Broncomatic to be connected to the MindMedia Nexus sensors (it is pretty easy to add new sources of sensor data, the Nexus just happened to be to hand when I was doing this test). It makes it very quick to write different mappings between sensor inputs and how the Bronco moves.

Videos of a couple of mappings are shown below (these each took a couple of minutes to write in the Integrator)

Speeds up when you breathe in, slows down when you breathe out Video, complete with heavy breathing (video complete with heavy breathing - quicktime).
Goes one way on breathing in, the other on breathing out (another heavy breathing video - quicktime )

I found a sticker printer hanging around in the lab, and made up a little sticker to give to people when they have ridden the ride. The number relates to the total volume of air that someone has breathed in during the ride. The little graph at the bottom is a trace of the last 20 seconds of their breathing.

Duncan & I built a 'finish line camera', which records video from a single line over time, showing a trace of the bronco movements - giving some quite neat results.

First Experiments

We have now run a public experiment with the Broncomatic, a first description of this experiment is available. We presented this work in a paper at CHI 2011. The Discovery Channel show 'Daily Planet' also did a piece on the Bronco (more details).

For Hire

admin — Thu, 14 Jan 2010 17:10:12 +0000

I’m currently seeking other places to show I seek the nerves under your skin. If you are interested in running it, please get in touch. It is suitable for any venue with access to the outdoors, from city streets to forests, and is quite cheap to run. See the proposal document for technical & safety details.

Thrill Laboratory on the One Show

admin — Thu, 10 Dec 2009 14:23:09 +0000

I was the technical support for this clip - putting Christopher Biggins on a rollercoaster (Mumbo Jumbo at Flamingoland) and recording his physiological responses. This was for the BBC One Show. Everything worked well, so basically I put together a bunch of gear, coded some videos, then got a chance to ride a few rollercoasters while the rest of the shots were being filmed.

The Bronco: A Proof-of-Concept Adaptive Fairground Ride

admin — Thu, 29 Oct 2009 14:39:29 +0000

We published a paper about the Bucking Bronco at ACE 2009. This was just a short paper describing the presentation at Pioneers 09, and the responses from audience members and riders. ACE Bronco paper (PDF)

I Seek the Nerves at Creativity and Cognition

admin — Tue, 20 Oct 2009 11:57:17 +0000

I ran I Seek the Nerves at Berkeley Art Museum, as part of the Creativity and Cognition conference, with the help of my friends Matt & Amelia who live in San Francisco. About 50 people had a go. In case you're interested in knowing more about the practicalities of the performance, you can read the proposal. It was quite an interesting place to run it, there were lots of constraints such as closed gates around the area, people in the way, uphills and downhills, different terrain (grass, concrete, gravel). I did some interviews of participants, and also got a load of logfiles off the equipment, to allow me to explore how people played with the system. This graph shows a participant who really went for it - the jumpy lines show the speed at which the person is running, whereas the straight lines show the speed the system requires them to run (which increases as they get further through the poem). Graph scales are time (in milliseconds) and speed (in a slightly rough numerical measure that doesn't really have a scale). An interesting thing about this participant, can be seen in the green jumpy line in the middle - they clearly went quite fast for a fair bit of time, then (at about 110,000) they noticed the poem fading out and put on a final burst of speed. One participant talked about a strategy they used to do this, by using the higher up parts of the area and the downhill ramps coming off them in order to get the boost they needed for the last section:

"well, I saved myself up for the downhill bit, knowing it was going to get harder and I thought, well the downhill would give me a boost … so I ran round in circles up there."

There were loads of interesting things in the interviews and the logfile data. The only downside of this dataset is that there is no correlation between people interviewed and logfile datasets (the logfile datasets are essentially just one big logfile per set of equipment). This is inevitable when running such a large number of people with only 3 people to put it on, take it off and interview people - in the future I plan to do some runs with one person at a time so that I can link the data together. Finally, it was really cool to see people just going out and doing things with running that they might not have done before - as one person said:

"I found myself running for the sake of running, like, running in, around the building, just, running in all kinds of weird places."

Which is really what it is all about - how cool is that?

Bucking Bronco: Adaptive Ride Experiment No. 1

admin — Wed, 30 Sep 2009 15:55:33 +0000

We presented the Bucking Bronco video at Ubicomp 2009. The paper to accompany the video is here: Bucking Bronco Ubicomp paper (PDF)

Ubikequitous computing: designing interactive experiences for cyclists

admin — Tue, 15 Sep 2009 14:51:37 +0000

This paper, published in Mobile HCI 2009 describes two mobile experiences for people who are cycling, and the unique challenges that this presents. Designing a pervasive experience for people who are cycling is different to designing for walking for many reasons, which we describe in the paper (PDF). The two projects studied were The Sillitoe Trail a mobile experience based in Nottingham, themed around the work of Alan Sillitoe, and Rider Spoke, a mobile game created by art group Blast Theory in collaboration with the Mixed Reality Lab.

Ubikequitous computing: designing interactive experiences for cyclists

Creating Illusion in Computer Aided Performance

admin — Tue, 31 Mar 2009 09:17:57 +0000

This is my PhD thesis. This page links to videos of the three projects studied in my thesis, and a PDF file of the thesis itself. For anyone who hasn't got here via the abstract of my thesis, it is repeated below:

This thesis studies the creation of illusion in computer aided performance. Illusion is created here by using deceptions, and a design framework is presented which suggests several different deception strategies which may be useful. The framework has been developed in an iterative process in tandem with the development of 3 real world performances which were used to explore deception strategies. The first case study presents a system for augmenting juggling performance. The techniques that were developed to control this system demonstrate how deception may become useful even when the core of the performance is not deceptive in any way. This is followed by a magic performance called the Cup Game, which was designed to explicitly test the strategies of deception described in the framework. The final case study is an interactive art installation which presents the illusion of a pet rock that lives in a cage. This demonstrates the usefulness of suspension of disbelief in the creation of illusions. It also demonstrates interesting social effects that are used to strengthen this suspension of disbelief. The idea of creating the impression of a false situation is inspired particularly by previous HCI work on public interaction. This work demonstrated the usefulness of hiding interface use or computer outputs from some people in a situation. The creation of deliberately ambiguous computer interfaces, which allow for a wider variety of interpretations to be made by the user has also been described. The work here goes beyond these techniques to use technology to actively create false impressions. The techniques used in this process are guided by the work of magic performers, and by psychological studies of how magic performance works. As well as artistic performance, it is envisaged that this work may prove applicable to more traditional situations. In addition to the framework itself, the development of the case studies has created several useful algorithms which have wider applications. The case studies are also useful guides for those creating performance systems, or other systems where deceptive techniques may be useful. Download full text

Juggling Tracker Videos I have two videos of the juggling tracker as demonstrations of the way the tracking works, and the range of outputs that can be produced by the tracker. All videos are filmed in one take in real time. The videos are available here. Cup Game Video A video showing the first two tricks of the Cup Game is available as a Windows Media Video file or a Quicktime file. This video may only really make sense if you read the description in the thesis first. Rock Video Rock was featured in New Scientist, the article and accompanying youtube video are available here.

Blue Peter & Thrill Laboratory

admin — Wed, 25 Mar 2009 15:38:35 +0000

I worked as a 'thrill technician' on an experiment shown on Blue Peter (a popular UK children's TV programme), done in collaboration with Thrill Laboratory. We put one of the presenters on 3 roller coasters around the country (Collossus at Thorpe Park, Rage at Adventure Island, Southend, and Kumali at Flamingo Land), and kitted him up with physiological monitoring equipment to measure heart rate, galvanic skin response (how much he was sweating), and facial muscle measurements (frowns and smiles!). Using this equipment, we calculated measures of 'fright' and 'excite', to characterise his response to the different rides. Essentially my work on this project involved running the monitoring technology and making sure everything worked first time (as the response off a rider is less extreme if they do a ride twice in a row). The 'experiments' can be seen on youtube (part 1, part 2).

Bucking Bronco at Pioneers

admin — Wed, 04 Mar 2009 14:20:20 +0000

The Bucking Bronco is a small mechanical ride, controlled by a single operator, commonly seen at fairs and hired for small events. A rider mounts the Bucking Bronco, and then it spins and bucks around, whilst they try to stay on. Once they fall off, the ride has finished. At Pioneers 09, a research showcase for the EPSRC, we took a Bucking Bronco, and a set of riders. Members of the public were allowed to control the ride. However, rather than being able to see the rider and the ride, as a normal operator would, instead the riders were fitted with physiological monitoring technology, and the controller had to use these signals to run the ride, from behind a medical screen (see below). The aim of this was to explore how a ride might adapt using only signals from the physiological monitoring kit. Obviously the people doing this adaptation are rather more sophisticated than a computer algorithm, but this at least provided a proof of concept for adapting rides based on the data. You can see a video of this event on youtube, which was also presented at Ubicomp 2009.

GlowClub: Aerotech Glow Equipment Programmer

admin — Thu, 05 Feb 2009 09:21:01 +0000

New Version 2.6 - 1st November 2011 GlowClub is a graphical programming system for Aerotech glow juggling Equipment. Currently for Windows only. You need to have the club drivers installed already so that windows recognises your clubs. Program colour sequences by dragging and dropping colours onto a timeline. Also supports looping. It lets you program different colour sequences into a whole bunch of clubs (up to 16 currently), and start them off in synchronisation with a soundtrack. Support for starting based on MIDI triggers, for synchronisation with external light shows etc. Each sequence can be sent to 1 or more clubs, so if you want 10 clubs to do one thing, and 6 to do another it'll work. No guarantees that it will work / won't break anything. It isn't anything to do with Aerotech - just something I built, although thanks to Aerotech for the help during development. The help has information on how to use it. Now with an update for the version 2.0 club firmware, high resolution timing & colour ramping. New: Displays audio amplitude graph above timeline, for easier editing. v2.7 June 2012 Bugfixes, hopefully fixes a crash on playback with large numbers of clubs. It's sort of free, although as the splash screen explains, if you find yourself using it for shows, it'd be nice if you'd give me some money, after all you've already spent £1000 on juggling toys. Download it from the link below. glowclubsetup.exe Joe

Interactive Fairground Rides

admin — Tue, 20 Jan 2009 14:08:12 +0000

I have worked as part of a team in the Mixed Reality Laboratory (in collaboration with Brendan Walker's Thrill Laboratory) exploring fairground and theme park rides. We have looked at people's physiological responses to rides, and are exploring how one might adapt rides, to provide a ride experience tailored to the individual rider. To find out more, see the other pages on the rides project.

I seek the nerves at (re)Actor3

admin — Tue, 09 Sep 2008 17:05:01 +0000

I seek the nerves was first run at the (re)Actor3 conference. 30 people had a go, which was quite a good turnout for an experience of running through the dodgy end of Liverpool on a wet, dark evening. It appeared to work really well, with some people getting very into it, and no-one getting hurt or even falling over. I also presented a paper on it during the main conference, the original version of which is available as a word document.

I seek the nerves equipment

admin — Sat, 06 Sep 2008 18:08:34 +0000

I seek the nerves runs on a nokia mobile phone, which is hidden inside a very tasteful flourescent jacket, with some lovely headphones attached to it. To start the experience, you put on the jacket and headphones, and start running, and the audio will start automatically.

I seek the nerves under your skin: an interactive poem controlled by running

admin — Thu, 04 Sep 2008 17:03:43 +0000

I seek the nerves under your skin is a motion based private audio performance which aims to cause accentuated mental states in the listener similar to the heightened mental excitement of live performers. It uses modified mobile phone technology in order to track the motion of a listener. In order to hear the audio, the listener must start running, and constantly accelerate. If they slow, the audio will fade out. This encourages the listener to push themselves beyond their comfort level, and aims to force those who wish to experience the performance into a state of deep concentration, where all they are able to concentrate on is the sounds they are hearing and their movement through the environment. I seek the nerves is a locational work without a fixed location. Whilst the piece itself takes no cues from the environment, the listener is in a public place, and forced to take notice of their surroundings to avoid crashing into them. This was run at the (re)Actor3 conference on Digital Live Art, Liverpool, UK, 3rd September 2008, the Creativity and Cognition in Berkeley, CA, USA, October 2009, and various places in Nottingham and Derbyshire in 2010. The (re)Actor3 proposaldescribing the work in detail is available. To find out more, see the I seek the nerves project.

Cup Game Experiment Results

admin — Mon, 01 Sep 2008 10:39:06 +0000

Results

An interesting result from my point of view is that it is quite easy to trick people, even in a situation where they know that they are likely to be tricked. I applied techniques from magic books in the design of this (particularly from some really interesting stuff from the 1890s about the psychology of magic), and it appears that these magicians really know what they're doing! One person did catch me doing my little bit of sleight of hand, but the other 16 the trick worked on just fine.

When people were asked to trick the computer, most people had no idea at all what to do. People with programming knowledge had a slight advantage here, but even so, it was significantly harder than I was expecting for people to trick the computer when they don't know how it works.

The other thing of note, is that people really do trust the computer. Several people changed from being absolutely certain as to which cup the bead was under, to being convinced they were wrong the moment they saw the computer guessing a different cup. This is despite the evidence of their eyes making them convinced they had the right cup originally.

The Cup Game Experiments

admin — Mon, 18 Aug 2008 21:28:19 +0000

I ran the Cup Game experiments over a couple of weeks in August, using participants gathered by putting up posters around the university.

The 17 participants were mainly students, with a mixture of postgraduates and undergraduates, and a pretty even mix of men and women. Each person got to see an individual magic performance with the Cup Table, set in a small dark booth hidden at the back of the Mixed Reality Lab. All the audience members said that they enjoyed the cup game which was nice.

The results are now available.

Hiding and Revealing Technology in the Cup Table

admin — Sat, 05 Jul 2008 21:26:20 +0000

The Cup Table has two possible states, in which technology is respectively hidden and revealed.

In the hidden state, the top and back of the table is covered with a black cloth. The area below the table at the front is also covered with a black cloth. This means that the technology in the table is invisible to the audience member. It also blocks the cups from my view, making it seem as if the performer cannot possibly see what is happening on the table.

In the revealed state, the audience member can see a laptop computer beneath the table, and a camera above the table, pointing down onto the top of the cups. They can then see that the cups are being tracked by the computer using the camera view.

The hidden state is used for a trick in which the audience do not even know that a computer exists. Afterwards, the computer is revealed, and further tricks are brought in which appear to use the computer openly.

Piloting the Cup Game

admin — Fri, 04 Jul 2008 21:18:32 +0000

The cup game performance was designed in an iterative process by running it on people in the Mixed Reality Lab.

This process was run for 10 people in total, until the performance and the technology had not had any major changes for the last 5 iterations. This also allowed me to get better at tricking people, and to have the script memorised well. I learnt several lessons from the pilot which are summarised below.

People Trust Computers

Initially, I let the computer guess which cup the bead was under and then let the person choose. However, it appeared that even if they thought they knew different, pretty much everyone would trust the computer's guess more than their own.

The Computer Can't Screw Up

If you have hidden the existence of a computer, it really has to work first time, and not crash ever. The first test I ran, had a weird bug that only occurred when the computer was left running for 5 minutes or so - ie. when I was briefing someone on the trick. Fixing it kind of destroyed the illusion. Ooops.

You Can't Screw Up

In some of the earlier tests, I screwed up switching the bead, or shuffled the cups to end in the wrong place. This required some quick thinking to act as if that was what I meant to do, in order to get out of it. In the end, I altered the script so that I had pre-planned get outs for each possible error, after which I somehow managed to perform it fine every time, so didn't have to use them!

They Can't Screw Up

This is really me screwing up in giving instructions, but if the audience member doesn't do what you tell them to, it can really mess up a trick. I altered the game to add a pause before a crucial reveal in one of the tricks, so that in the case of the person doing something wrong, I could reset the trick and get them to do it again.

The Cup Game: Experiments in computer aided deception

admin — Tue, 01 Jul 2008 20:42:11 +0000

The Cup Game is a computer aided magic trick. It is based around a famous trick known as the Shell Game. In the shell game, a small bead is put under one of three cups, shells or walnuts, which are then shuffled, after which the person playing the game has to guess which cup it is under. This is commonly played as a con-game, where people are cheated out of their money in what seems like a fair game, by criminals who use sleight of hand to switch the bead.

This version of the Cup Game is played on a special table which I built out of wood, cloth, and an old desk that was hanging around. It uses technology to create three variants of the trick:

In the first, I am blindfolded, and the audience member then shuffles the cups, after which I am instantly able to detect which cup the bead is under. I claim to be able to do this by sound alone...

In the second trick, I shuffle the cups, and the audience member is shown a computer tracking the cups, to make it extra fair. Then they are allowed to guess, and invariably they get the wrong cup. The computer is allowed to guess, and gets it right.

In the third trick, the audience member is allowed to get their own back on the computer, by trying to fool it.

The Cup Game is an exploration of the ideas in my thesis relating to the creation of illusion by deception, and of using computers within these deceptions.

Read the other posts relating to the magic project for more information.

Pressing the Flesh: Detecting Multiple Touch and Finger Pressure on Arbitrary Surfaces

admin — Thu, 22 May 2008 18:38:46 +0000

Joe Marshall (University of Nottingham, UK), Tony Pridmore (Nottingham University, UK), Mike Pound (University of Nottingham, UK), Steve Benford (University of Nottingham, UK), Boriana Koleva (University of Nottingham, UK) This is the sensing technology behind the Rock, and was published and presented at Pervasive 2008. In the presentation, I actually used a banana as my touch sensitive object, because I wanted to use something I could easily get hold of in Sydney. I think I am probably the first person ever to make a touch sensitive banana. PDF Version

Rock in New Scientist

admin — Fri, 28 Mar 2008 18:36:21 +0000

An article on the finger sensing technology behind the Rock has been published in New Scientist Online. There is also a YouTube video, with some very funny comments.

Rock at Twaddle

admin — Tue, 27 Nov 2007 18:49:16 +0000

Rock went to Twaddle Cabaret in Derby. It worked fantastically during set up, and worked okay on the night, but was a bit overpowered by the sound system / all the people talking.

Eye-balls: Juggling with the virtual

admin — Sat, 15 Sep 2007 17:47:23 +0000

Marshall, J, Benford, S, Pridmore, T in ACM Creativity and Cognition 2007. I presented a demonstration of the juggling tracker to the conference at ACM Creativity and Cognition 2007. This was a demonstration setup, where people came past during the conference. The proposal is available as a PDF.