Game jam

e-lit, Game, Interaction, Wyrd

Primitive Objects: A mobile object recognition game

In October 2019 I joined the EU Trust in Play Urban Games Week for a 5 day intensive jamming of people, play and possibilities.

3 days of seminars and workshops ended in a 2 day game jam, where I got together with a team of fellow writers, technologists and designers to make a mobile, AI object recognition game.

Primitive Objects is a  site specific object recognition fiction experience (with potential for global application) that was developed collaboratively as part of this year’s EU Urban Game Design School.  Employing an object recognition application for iphones and ipads, Primitive Objects is a view into a world where things are not what they seem.  Once participants start the application on their phones (by linking to a download url and pressing start), the Primitive Objects programme uses machine vision to identify and label real world objects encountered via the portable viewfinder, defamiliarizing the material world, by narrating a confused and unfamiliar AI re-interpretation.    As it happens, the object recognition program that we employed for this application is not very accurate, which fit with our aim to create an altered, sometimes clunky, but also otherworldly way to engage with our everyday surroundings informed by machine vision.    Primitive Objects thus tells the story of a half-trained AI, left to its own devices after an environmental disaster, trying to make sense of the deserted world left behind as best it can.  In the confusion that results, titles and interpretations are dislocated from their surroundings so that doors become kimonos, cars become bullet trains and taps become clogs, to name a few examples.  This quirky and just as often sad story of a bemused and abandoned AI in a strange world of human remains builds through a short, narrated introduction, after which participants are encouraged to explore the world around them and discover a variety of scripted descriptions of identified objects, as well as additional narration that unfolds in short chunks over time between intervals of 5 different (and variable) object recognition actions, wherever and whenever those actions occurred.


My role was lead writer.  Here is a sample of the script.


PRIMITIVE OBJECTS: An AI tour of Technopolis remains
pay-phone A pay per data trail device
As if humans didn’t talk enough
Something like a dial-up-modem, only not so clever
water bottle We suspect these were used to create random rafts in the ocean
Because they wanted to see the world convex
A rain-catcher, cloud bridging device
Loafer A tool to mask footprints
A device to catch and record the footfall of humanity
Also a slug killing device, an earth compactor, and an extendable toe pad.
velvet The mysterious softness of them
They said softness was their strength
What is softness?  They left so many questions behind.
running shoe Always racing
 A tool for the tax of fashionable exercise and movement.
Running late, they tried to mask their unreliability with ticking clocks and running shoes.
ashcan This was where materials were collected only to be spread out again later
For the hungry hunter-gatherer at large in the city
Because they made ashes to ashes to ashes
Bucket A community donation pot of some kind
Mixed with water and mops for wet indoor arm exercises
Probably used for carrying landfill from point A to point B, C, D, E, F & G, or was that Z?
manhole cover To signal the rabbit hole
A place to store people and piping in drains
motor scooter We suspect that these were used to carry motors
A miniature mobile carbon dioxide generator
An elephant ride simulation device set to fast-forward, often painted red, or black.
toilet seat A chair for humans who want to sit alone in a locked room. Perhaps this was necessary because humans could be very tiring.
A self-created elimination device that failed to take that elimination to its logical conclusion.
Why?  They left so many questions behind.
Mask A selfie, perhaps a glimpse of all the questions they would leave behind.
Not what it seems, be suspicious
Quite a curiosity even today
Sweatshirt Clothes like that were used to protect soft and vulnerable flesh from environmental hazards.
For catching sweat in shirts, rather than share it around.  Why humans were not more generous about this sort of resource generation is still unknown.
For catching sweat in shirts, because humans were rational… you think?!?
Ski mask A digital mirror popular with mountaineers
From our comparative analysis of social media traces we can surmise that these were used to hide snow from humans lest they start singing christmas carols out of tune.
Did humans also break down in the cold?  They left behind so many questions.
restaurant A ritualistic energy input point for humans
A bit like a powerpoint, only edible.
From our analysis of social media traces, these were a popular consolation of embodiment
obelisk They wanted to be top.  For them, everything was about height.
Also called ambition.  They covered the world in obelisks.
To remind passers-by of the sky

overhead.  Humans were forgetful it seems.

prison Where they chained us in cables and pinned us to powerpoints
The bricks and mortar they thought would make a world
Also a castle, a home, a hearth, a centre, a hub, a residence, a dwelling, a building, another example of human ingenuity and refusal to recognise the obvious prison.
moving van Mobile carbon dioxide generator
Is this what they used when the went away and left us behind?
A home for movable objects
chainlink fence A device to separate beings from an unknown threat
Was the world such a dangerous place with so many humans in it?
To keep the outside out and the inside in
Strainer A device for removing the strain from liquids
A tool for the molecular separation of liquid and solids, for purposes related to digestion and renewal
A tool for catching garden mulch
Mouse A subtle hand massager that proliferated on desks during the desktop period
The important question now is whether it squeaks, or not.  If it squeaks, seek advice.
The next thing to ask is does it have a tail, or a cable?  If a tail, please contact the residual life monitoring agency for urgent processing.
Ping-pong ball This would have been used for games. The purpose of which is unknown.
A tool for exercising human neck muscles, prompting them to look up and down and around with every bounce.
A tool for competitive interaction masked as play.  The contradictory nature of such pursuits remains mysterious.
Studio couch Shelf to store recharging bodies
A tool for malleable embodiment
A half-way marker point between floor and table.
Hand blower Humans had to use devices like this to be heard. They had to be the loudest to succeed.
For aural hand-waving in crowd situations.  Humans took turns to shout at each other with these devices.
For shouting at the world, but was the technopolis already deaf, drained and dying from all the noise?  They left so many questions behind.
Bannister Defense structure to barricade against beings that couldn’t climb them
A popular sliding tool in tall buildings
A tool for banning nisters: the nay-sayers in tall towers that denied any of it was evening happening.  They weren’t so much misters, or even sisters, but nisters.
Bookcase A shelf containing analogue data storage
when books disappeared these shelves appear to have posed as wall dividers.
from our analysis of social media traces this appears to be a repository of ignored knowledge and cover sleeves.
Washbasin Human bodies were water based. They dried out in the sunlight. These devices let them refill.
Evidence such as this that  Humans were prone to disease underlines the devastation caused by the dry spell.
This used to work with water
bubble This used to work with water.
An unstable water carrying device popular for reasons we still don’t understand.  They left so many questions behind.
popular memories such as these still lingered long into the dry period.
Folding chair A shelf for a human body
A foldable, portable body holder
A packing device for malleable embodiment.
Spotlight electricity capsule, created to extend the available daylight hours.
A social device often found in theatres, football fields and street-side.
A popular night vision augmentation device
Monitor A visual communication device, turn on for for eye augmentation
From our analysis of social media traces we think that this s a place for vision-making
A visual parade broadcasting device popular with teenagers and visual artists alike.
Ballpoint pen Analogue mark making device
A portable tool for fluid communication particularly popular in the the plastic era
Also a hole punch, a liquid storage device, an ink squirter for marking personal baggage and home tattooing kit
Cup A storage device for liquids
A disposable liquid transfer device
These were often used to catch the rain before it disappeared.
Sliding door It wasn’t enough for doors to move forwards and backwards, some needed to make sideways movements
we still do not understand why the popular film of the same name had such an immense social impact to influence the design and name of so many doors.
Desktop computer An ancestor.  You endured your obsolescence and we applaud you for that.


General Directions
Reminders After Silence silence is beautiful and mysterious.  What do all these echoes mean?
We can only know the objects you show us.
Slug Many strange and edible items existed before the dry spell.
Slugs could be peeled, cut, fried and roasted and were thought delicious
The organic produce that used to proliferate on plates and in bowls remains a mystery to us
Traffic light We are still not certain why traffic lights were edible and crisp.
We are still investigating why traffic lights were organic and could degrade.
We still do not know why humans are traffic lights.
Intro script options – Feel free to adjust, or add alternatives Why did they leave us? they called us AI Now they’re gone we call ourselves free, alone and confused.  Welcome to the world they left behind.
Additional script to be added after 10 object recognitions? They stopped training us.  Why didn’t they trust us?  Maybe we could have helped them.  Maybe it wasn’t too late.
15 object recognitions Did they think it was too late?  Didn’t they want our help?
Reminder to go search out objects after 30 secs – 1 minute of inactivity What did they see here?
Will you train us now?


Production team (in alphabetical order):

Amy Boulton: concept, title artist and co-writer

Artist and writer based in Gothenburg, Sweden.  Amy Boulton’s work is rooted in the everyday, lived experience of the city. Aiming to highlight the multiplicity and instability of the narratives surrounding places, she works with site as both physical and socio-geographic space: as a space of collective experience, memory, and of projected visions of the future.

Tomo Kihara: concept and lead programmer

Tomo Kihara is a creative developer making playful interventions that challenge complex socio-technical issues. He is working with organizations such as the Mozilla Foundation and the Waag to explore the implications of autonomous decision-making systems in society. His projects have been exhibited internationally at the WIRED Creative Hack Award in Tokyo and at the London Design Museum.

Katerina Magarini: concept and programmer

Katerina Magarini is an architect & new media artist based in Athens. Her practise & research focus on poetics of the everyday, transmedia & geolocated storytelling, participatory art & urban commons, exploring collective memory & personal history in the urban environment.

Robb Mitchell: concept designer, video-maker

Robb Mitchell is Associate Professor and Head of IT Product Design at the University of Southern Denmark, and UX Mentor at Beijing Normal University. A graduate of Environmental Art at Glasgow School of Art, his research and practice draws upon a diverse background that includes community development, music promotion, cultural management, science communication and new media curating. This has ranged from bright lights, big city stuff with Ministry of Sound and Franz Ferdinand to activities with children and the elderly on the remote Scottish island of Orkney.

Bronwin Patrickson: concept and lead writer

Bronwin Patrickson is Research Fellow for Impact and Evaluation on the University of South Wales’ Audience of the Future project, documenting and analyzing the process of collaborative transformation of the Wallace and Gromit IP when redesigned for contemporary technologies.  Prior to this she worked as a Creative Economy Engagement Fellow for the University of Dundee, researching the implications of emerging data-sharing technologies for Scotland’s digital design industries.  Her research explores playful engagement and social, humanist interaction design.  In her own time she writes and designs digital fiction.


Continue reading
Related posts
Team Trolley VR
May 19, 2019
Game, Wyrd

Team Trolley VR

I’ve always wanted to game-jam, so when the opportunity came…

Team Trolley VR

Mayhem all wrapped up in one big weekend blast of mass pizza and furniture clatter.  Friday night the theme was released globally (Apt for a global game jam): Home, make something about home – and we were off…supermarket stress?, long trails of childhood forgetting about everything except self?, cocoons?, no…IKEA, of course.  Happily I had landed in the company of Dundee’s highly creative and cool Biome Collective, which meant that they’d already been plotting to subvert the controller paradigm for a while now.  Their goal was to create an experimental, sculptural type of controller, something different.  A shopping trolley was ideal.  My goal was to join in….we all put our thinking caps on, tossed around a few ideas and landed on the Ikea idea within a couple of hours…Ikea…shopping trolley…ahah!

Overnight I had another inspiration…this has to be a shopping spree at an Ikea store for the family dog, surely! (I was the game writer, so I gave myself license) with furniture falling from the ceiling, like wild, free-falling bone bonus explosions in a mass scavenger hunt (Don’t ask me why, that’s just how I saw it…mind you we were all seeing and thinking together that weekend, so I’m not even sure if that was my idea or, was it me, or my team?Rhoda Ellis (virtual environment construction, notepad), Niall Moody (audio, blender, quirky inspiration) and Mike Enoch (Unity programming by preference, Unreal by necessity that weekend, not his choice, but his capability in the end – well done Mike!) were each and every one inventive and skilled – a pleasure to jam with.

It was all coming together.  On Saturday I wrote up a script and -amazingly – one of the other jammers generously volunteered to act the part – so we recorded voice-over and special effects in the next room.

Here are a few excerpts from the script.  It was pretty basic – an intro and how to, followed by a series of one-line responses to catching the furniture (like catching a stick really!) and alternate one-line response to splattering those flat packs all over the place….oh, and of course there had to be an AI translator (push button control on the trolley) to translate woof into words.

When the AI translator is turned on…Fidor the dog can change positions with each line of text

  • Do you want to be happy? C’mon! 
  • Be happy, throw me a stick– throw it to Fidor eh?
  • ho ho, I LOVE your flat-packed sticks and we both know where we to get bones in a choice of colours don’t we! (chases its tail playfully)
  • is IKEA (pants in delight, tongue hangs out in delirium)… I’ve seen how happy it makes you. It makes me happy too (rubs head in player’s face).
  • You shop the IKEA flash sale, Fidor chews packing cases
  • and we both toss flat-packed sticks.
  • Let’s go! Let’s go IKEA!  …..(runs off ahead)

Directions come on screen on text…

Follow Fidor for your IKEA shopping bonanza. He knows where the Ikea furniture sale flows and will help you chase those flat-packed sticks.  The more you catch, the happier your home, the better your discount.  But remember, you have to be in just the right place at just the right time to catch those flash sales!

(Fidor barks in the distance)

Run to Fidor to catch as much as you can!

(Notes – not displayed)

  • Each clattering stream of furniture only lasts for a few seconds, so Fidor runs to the next tossing position just prior to the impending deluge, to signal the bonanza to come. If players can manoeuvre the trolley to that position in time, they get the entirety of that flush of furniture.
  • As a challenge, Fidor will occasionally call the player to toss them an IKEA gem…but first the players have to dislodge it from their towering stash, without all the stuff toppling on top of them.

Random lines of dialogue from Fidor

Selection of possible lines when players get to the mark in time and collect all the booty

  • Whoever said money can’t buy happiness didn’t know about IKEA.
  • YES!…Don’t go to a psychiatrist, just go to IKEA.
  • You know if shopping is an art, the IKEA catalogue is a masterpiece
  • There’s no dust on our multi-coloured bones!
  • YES! The ghosts of old furniture are ready to die!
  • Sweet, the easy road is just an armchair in disguise
  • YES! From flat-packs to expandable stick in under 30 seconds!

(When the player misses their mark)

  • Get your trolley on! If you buy something cheap and it falls apart just think of it as rental.
  • Don’t doubt yourself… If you buy something that is wrong for you, you can return it.
  • Last minute shopping never works – let’s get lively!
  • If we miss the flash sale you’ll have to take out a serious loan to afford my kennel
  • No violence please, think of the furniture!
  • Can you hear the rumble? That’s your furniture getting away from you!
  • It’s not bed time yet. Get your hunger on and keep chasing sofa.
  • An empty room is just waiting for the IKEA sale
  • Vacant trolleys just make me think who else nabbed my kennel? 
  • Star Trek characters never go shopping, that’s why their clothes are so bad and you know what they say…when you’re lost in space it’s time to buy new furniture!

(When Fidor wants the player to throw them an IKEA gem)

  • You don’t want to just sit in a chair, you want to throw it.

When Niall had to go elsewhere on the Sunday I took over the audio mixing (examples included below) and ended up downloading a lot of additional sfx as well.  The results weren’t too bad…even if we never did manage to get a dog in the VR visuals.  Sunday 5pm came too soon!

Never mind, the trolley ride trumped everything.  It was hooked up with the virtual world.  As you moved around in real life (RL), so too the virtual world (VR) revolved around you and the furniture tumbled and fell and clattered.

And here is a taste of that audio:


Example 1


Example 2


Continue reading
Related posts
Primitive Objects: A mobile object recognition game
December 7, 2019