Breaking Brain: Computer Science of the future

Code breaking all sounds very ‘007’ (or a bit Alan Turing if you’re into your WWII and Manchester history). For many it conjures images of special agents embroiled in top secret espionage; or perhaps a lone revolutionary working by candlelight towards some crucial eureka moment. But, what about breaking the ultimate code, that of the brain? Here I’ll explore some real life advances in neuroscience which may sound like science fiction, but are, in fact, all real…

Communicating with vegetative patients using neuroimaging:

No, this isn’t a work of science fiction; using state of the art technology, outwardly unresponsive patients can now communicate with the outside world (see here for the full article)

Due to developments in neuroscientific research, we can predict what brain activity will look like when people are asked to imagine performing certain actions (such as playing tennis); and, amazingly, it is this knowledge that forms the basis of such communication.

Scientists asked outwardly unresponsive patients yes or no questions whilst scanning their brains using functional magnetic resonance imaging (or fMRI). Patients were instructed to imagine playing tennis if the answer was yes, or to do nothing if the answer was no. Incredibly, it was found that one patient who had been outwardly unresponsive for five months following a road traffic accident, was able to respond in this way. Scientists could be sure that this was a specific response to their question (and not just random brain activity) because of the well-known brain ‘signature’ which follows the imagination of playing tennis. Indeed the patient’s brain responses for this task could not be distinguished from those of a normally functioning person performing this task. Technology such as this could help physicians to make crucial decisions about the care of outwardly unresponsive patients, and could help families find ways to communicate with their loved ones.

A window on the mind:

Some neuroscientific research can ‘train’ computers to respond or learn like a human brain – so called ‘neural network models’. One notable example of this is the work of Nishimoto and colleagues from Berkeley, USA. Nishimoto and colleagues used fMRI to scan the occipital cortex (the visual centre in the brain) of people watching clips of movies. The movie scenes were then categorised mathematically on a great number of features (e.g. the presence of colour, the nature of any movement, the presence of lines etc.). With this data, scientists set about developing a model that could match the mathematical categorisation of video data to real-time brain activity. Essentially, this involved looking for patterns in the mathematical categorisation that matched patterns in brain activity; such that one could say when a person views a picture with property A, then brain activity pattern B is reliably produced. The more movie clips and brain activity that were analysed, the better the model became.

Incredibly, the finished model could take fMRI data from an unknown clip and generate an accurate visual representation of the associated movie. Take a look at the YouTube clip that shows this happening here, and/or read the full scientific article here.

If you’re not already excited by this, just imagine the possibilities with a tool like this… Ever wanted to remember your dream from the night before but it all seems a bit vague and out of reach? It’s theoretically possible that this technology could ‘record’ a dreamer’s visions. And, what about recording your thoughts and even feelings for future playback?

Although this is beginning to descend into ‘science-fiction’; the basic premise of using computer science to model human behaviour could in theory, be applied to any modality from vision to touch – and who knows, maybe one day, feelings and emotions. With the ever growing and impressive repertoire of neuroscientific advances, it seems that today’s musings could be tomorrow’s reality.

Post by: Gemma Barnacle