Mind Reading

published on July 9, 2020

Mind reading?Of course notI love readingLook, mind reading might soundlike pseudoscientific–pardon my language–bullshootBut its scientific counterpart,thought identification,

Is very much a real thingIt's based in neuroimagingand machine learning,and what's really cool isthat experiments in mind readingaren't just about spyingon what someone is thinkingThey're about figuring outwhat thoughts are even made ofI mean, when I thinkof something,

What does that mental pictureactually look like?What resolution is it in?How high fidelityis a memory,and how do they changeover time?Well, in this episode,I'm going to look at howreading people's minds

Can help us answerthese questionsMy journey begins right hereat the University of OregonI'm meeting with Dr Brice Kuhlfrom the Kuhl labHe's a neuroscientistwho uses neuroimagingand machine learning to figureout what people are thinkingwithout them telling him

So tell me whatyou're doing hereWell, I'm in the cognitiveneuroscience program here,and I study human memoryMy lab primarily usesneuroimaging methods,so we do a lot of work usingfunctional magneticresonance imaging,

Or fMRIAnd how do you usefMRI to investigate memories?We're looking at the patternof neural activityWhen you form a memory,there's a certain patternAnd we can recordthat patternand then test whetherthat pattern is reinstated

Or reactivated at a later point,like when you're remembering itDoes that mean we can look atthe patterns of brain activityand deduce what it is that isbeing remembered, or recalled,or even just thought?Yes, and so we call thatdecodingSo it basically takesyour input pattern

As some pattern of activitythat we recordwhile you're rememberingsomethingAnd we make a predictionabout what you're rememberingYou can see how this soundslike mind readingYes It sounds like thatSo, Brice, what are you goingto do to me today?

So, what we're goingto be doing todayis uncharted territoryfor usSo we're going to be trying outa kind of new variantof the experiment on youSo I can't guaranteeany particular resultsBut it representswhere the field is

And wherewe're trying to goToday, you're going toparticipate in an experimentwhere you'll be studying facesSo we're goingto have you study12 pictures of celebritiesPeople I already amfamiliar with

-People that you know, yeah-OkayAnd you're going to tryto remember those picturesThen we're going to have you gointo the MRI scannerTry to bring that pictureto mind as vividly as possibleAnd we're going to be recordingyour brain activityas you try to imaginethese pictures

We're going to tryto build the faceEssentially draw a picture ofwhat you're remembering-A picture?-A pictureAn actual picturethat we can print outand I could, like,hang on my wallIf you wanted

The first step is for me to memorize the 12 specific celebrity photographs Brice will later try to detect me thinking about I sat down to do this graduate student, Max The success of his predictions depend, in part, on my ability to recall these faces

as vividly as possible while inside the fMRIAll right, soI think I have a pretty goodmemory of all of those-Great-I feel the stakes are high With the celebrity faces hopefully memorized, it's time for the next step:

going through the metal detector and into the fMRI, where Brice will record and monitor my brain activity,and then later feed it into hisalgorithm to rebuild the faces This will be the first time he's attempted to reconstruct faces from long-term memory,

which is very difficult, because we're relying on how clearly I can remember the celebrity photos I saw an hour agoI love its eyesLook at thatWouldn't the kid be like,"It's going to eat me"? An fMRI monitors the activity within the brain

by dividing it up into thousands of small cubes called voxels, or volumetric pixels Each of these voxels contains hundreds of thousands of neurons Using fMRI, we are able to detect blood flow within these voxels,

which means that that part of the brain is active If I'm shown several pictures of people with mustaches, my brain will react to the features for each face But there will be a common area of my brain that is engaged throughout That may be the area of mybrain that reacts to mustaches

So later, when I imagine a face, if Brice notices that area is engaged, he can predict that I am thinking about a mustacheSo right now Michael'sin the scanner,and he's seeing words appearon the screen one at a time,

And he's tryingto visualize the face,remember the face in as muchdetail as possibleWhat you can see here arethe images that we're acquiringWe get one of thesebrain volumes every two secondsSo these are refreshing in realtime as we collect the images With part one of the fMRI session over,

it's time for part two, where Brice and his team will learn the language of my brain activity, so they can later decode by brain scansHi, MichaelYou doing okay still?Yup They'll show me hundreds of unique faces,

and record how my brain reacts to certain facial characteristics They will then use this information to reconstruct the celebrity faces I thought about during the first phase of the scanReally, the more faces thatwe can show Michael, the better

So we're going to basically keephim in thereas long as he's comfortable Two hours was the maximum time we could get in the fMRI But I was able to look at over 400 faces, which should be enough to get

some pretty interesting resultsHey, Michael, you did itThat was greatWe're going to comeget you outAll rightYeah, so these just showsome of the picturesthat we were takingwhile you were in there

Some images of your brainNow we are goingto crunch some numbersMax is going to analyzeyour dataWe'll meet upagain tomorrow,where we'll lookat the results,where we try to actuallyreconstruct the face images

From the brain datathat we just collectedAll rightWell, see you tomorrowAll rightThanks a lotMax, thank you as wellI can't waitYou better pullan all-nighterI want this datato be perfect

All right, so I am backat Dr Kuhl's labOvernight, his teamcrunched the data,and I can't wait to see whatthey think they saw me thinkingHow are my results?I think they look goodWe're going to take a lookin just a moment here

All right,I can't wait-So can I just take a seat?-Yeah, have a seatAll right, sofirst of allwhat am I seeing?Oh, okay, well,these are the picturesI actually memorized

-That's right-And this is whatyou've reconstructedfrom my imagination-That's right-Oh, wow OkayOkay, so this is oneof the reconstructionsthat was generatedInteresting

So that's John ChoNot bad Not bad-Can we see the side by side?-YeahI see, you know, similaritiesin the kind of facialexpressions in generalYou know, you could almostsee the hairline matching here

The shape of the faceI also thought was–It kind of hada square shape to it-Yes Yes-So those are the thingsthat came out to meAnd so when I wasvisualizingthis image of John Cho,

The squareness of the face wasthe first, most salient thingI just kept thinking,he was the square guyExcellent, all rightSo that's Megan FoxMm-hmmYou're going to show us the–side by side

The side by side RightYou can see the pictureyou actually saw,and that's the reconstructionwe generatedI'll you thisMegan Fox, I was not ableto have a really clear picturein my mindFor some reason, this imageof her was really hard for me

To bring back into my mindThe sternness in the face wassomething that I did pick up onSo I did sense that there was–It looked feminineAnd you picked upon the sternnessAnd so together,that produces a match Keep in mind that Brice and his team

have read these from my memory But when I remember a face, do I picture every detail simultaneously with photographic accuracy? Or do I just attend to a few at a time? By reading my mind, they may be seeing

how bad my memory is, and how it works-Me! Me!-Okay, so that is yourreconstructionof me thinking aboutthis image of myselfThat's rightWhere'd the beard go?

I don't knowI was hoping you could tell meFor instance, this is a picture of me remembering my own faceIt really doesn't look like me, but the question is: how good am I at picturing myself? I don't think of my own face that often, so the strangeness in the result

may be as much about flaws in my own memory and mental picture of myself as flaws in the technologySo that's Jennifer Lawrence,I believeThat's Jennifer Lawrence?It looks like it's JenniferLawrence's much older uncleNothing here was toomind-blowingly close

But this is something thatyou're just starting out tryingthese sort of long-termmemories What Brice and his team read in my mind might have been more accurate if they'd shown me thousands rather than hundreds of images in the fMRI, because then the algorithm would have learned

the language of my brain more thoroughly But regardless, the quality of my memories would have still been an issue I mean, look what happens when memory is cut out of the equation entirely Brice also read my brain activity

when I was looking at faces in the fMRI not just imagining them And those results were much closer than those reconstructed from my memoryOkay, so, what am I looking atright here?Okay, so what you're seeing here

In the top row,these are images that you sawwhile you were in the scannerBelow that, in this bottom row,these are the reconstructionsthat we draw from the patternsof brain activity we collected-This is from the source image-RightThese are from my brain

– Right- They're pretty closeYeah, overall they werepretty closeSo not perfectThese are– you can see there'ssome variability in theseBut this is consistentwith what we've found before,that the reconstructionsthat we generated,

When you're viewingthe faces,there is some correspondencebetween the actual faceSo this is kind ofa sanity check,that we can actuallyreconstruct the images-when you're viewing them-Right, rightThey're pretty good

Well, Brice, Max,thank you so muchfor letting mebe a part of thisI hope my data's usefulThank youIt's been a lot of funIt's always useful for usto think about these thingsDr Brice Kuhl's memory researchis showing that it's possible

For a computerto read someone's mindTo figure outwhat they're thinkingBut a lot of progressstill needs to be madeI mean, if you want to knowwhat I'm thinking right now,for example,it's still easier to just ask meto tell you

But what if I can'ttell you?Dr Yukiyasu Kamitaniis a researcher,professor and pioneerexploring the frontierbehind the wall of sleepI've come hereto Kyoto Universityto meet with him and to seewhat it's like

To read not what someoneis thinking,but what someoneis dreamingKamitani sensei,I'm Michael-Hi, I'm Yuki-Yuki, nice to meet you For the last ten years, Dr Kamitani has been at the forefront

of machine mind readingThe subject is, you know,ready to go in Similar to Brice Kuhl, his early experiments explored reconstructing images shown to subjects in an fMRI based on their brain activity In Kamitani's case,

the images were black-and-white shapes, and the reconstructions were strikingly accurate Recently, Kamitani has focused on using deep neural networks and machine learning to decipher subjects' brain activity while they view much more complex photographs

What you're seeing is theresult of a deep neural network processing the brain activity of a subject looking at the photograph This could have myriad applications in the future, for example, in criminal investigations and interpersonal communication

This is far from perfectBut I think you still see some,you know, eyes and, you knowWell, yeahAnd colors tooYeah, to some extent, yeahHis most current work, however, is about the subconscious

He's attempting something extremely ambitious: recording our dreamsWould you call yourselfa sleep researcher,or a vision researcher?Maybe a brain decoderA brain decoder

That's a pretty cooljob descriptionCan you show me anything fromwhat you're doing with dreams?Mm-hmm, yeah Dr Kamitani's work on dream decoding begins with a similar process to Dr Kuhl's: showing the test subject thousands of images

while they are in an fMRI in order to learn what the brain looks like when it is thinking of certain things Once the machine-learning algorithm is pretty good at identifying what images the subject is thinking about, the subject is placed in an fMRI

with an EEG cap on their head, and invited to fall asleep When the EEG waves indicate that the person is dreaming, the algorithm predicts which kinds of things the subject is most likely dreaming about Right now, the algorithm looks for 20 categories

Things like buildings, transportation, and characters in a language Researchers then awaken the subject, ask them what they were dreaming about, and see if the algorithm's prediction and the person's recollection match

Here is actual data from one of Kamitani's experiments Below is a word cloud of categories The name of each category get bigger or smaller in real time based on the probability that they are presentin the subject's current dream Now, as you can see,activity is currently strongest

for the category "character," meaning written language At this point the subject was awoken, and this is what they reportedThat's pretty spooky–Right? I mean, you–you spied on their dream

Yeah, in a wayButthe accuracy'snot that great, soWell, the accuracy'snot that great but, you know,my normal accuracy for guessingpeople's dreams is zeroRight While continuing his research

into predicting the content of dreams, Dr Kamitani is embarking on his newest project: actually reconstructing images from our dreamsSo you've broughtsome of the reconstructionsthat your lab has createdMm-hmm

of dreamsRight, they all look like dreamsabout blobsYeahI mean, I want to justtake a step back andappreciate that what we'relooking at on this screenare, in a way, some of the firstphotographs of a dream

Mm-hmm We are looking at the earliest phase of revolutionary research One day, we may able to have images, or even record movies, of our own dreams And Dr Kamitani is the only person in the world

doing this so farHe's a lone explorer journeying into our subconsciousSo this work hasn't evenbeen published yetNo-Thank you for showing it to me-The insights that researcherslike Dr Kuhl and Dr Kamitani

Might be capable of achievingin the futurebecause of mind readingare difficultto fully fathomBut let's slow downfor a second,because we're talkingabout a technologythat can know usbetter than we know ourselves

Should we reallybe doing this?Well, to addressthat question,I'm going to meetwith an expert in ethics,neuroscienceand artificial intelligence:Julia BossmannShe's the director of strategyat Fathom Computing,

A council memberof the World Economic Forum,an alum of Ray Kurzweil'sSingularity University,and a former presidentof the Foresight Institute,a think tank specializingin future technologiesand their impactsJulia, thanks for taking sometime to chat

-Yeah, of course-You are the perfect personfor me to bringthese questions to-Mm-hmm-And they're deep questionsBut I think they'reextremely important,and they're becomingmore and more pressingI think we're living in suchan interesting time right now,

Because we're in this timewhere brains and machinesare actually movingcloser togetherSo when it comesto being ableto look at brain activity,where arethe ethical lines here?How private shouldmy internal thoughts be?

Like with any powerfultechnology,it depends on the handsthat wield itAll these new technologiesare things that can make whoeveruses them more powerfulSo we want to not blamethe technology, but we want to–how is it being used,

And who is using it?So how do we make surethat this technologyis in the right hands?So I think it's very importantto involve peoplewho act on policy and lawto understand what is comingin the future

I am hopeful aboutthe collaborative aspect of itLet's talk aboutthe good things nowI mean, what arethe applications here?Yeah, so if we think aboutthe late Stephen Hawking,for example,if he had a way of richerinterfacing with the world

Or with computers,we can only imaginewhat he could haveshared with usThose with locked-in syndrome,right?They are thereThey know that they are thereBut we just need somethingto look into their brainto see what it isthat they are trying to say,

-or what they're feeling-Right, exactlySo, what do you sayto peoplethat have that kind of fearof technology,of us surrendering our truenatural selves to technology?There is something enticingabout getting to the next levelof what some people might calla human evolution

Or civilization development,and so onIn a way, we are already notliving natural lives, right?Because then most of us woulddie before the age of,I don't know, 30 or 40We would have all kindsof diseasesWe would not wearthis clothing

We wouldn't have eyeglassesor contact lensesWe wouldn't have antibioticsWe are already kind ofvery futuristic cyborgsif we compare ourselvesto the human that was living10,000 years agoand was geneticallyalmost identical

With who we are nowYeah, we really areIn order to understandcognition,right now we basically haveto either just ask peopleto talk aboutwhat they're thinking,or observe their behavior

But reading thoughts directlywould be a lot betterThat is how Dr Kuhlis studying memory,and it's how Dr Kamitaniis studying sleep and dreamsBut even though the technologyhas a long way to go,it's easy to seehow ethical questionscould become an issue

Well, here's the thing:there is no such thingas a totally wild humanWe are co-evolvingwith technologyHumans and technology todayare inseparableNow, it's true that we needto be carefulabout every new thing we do,

But we cannot change the factthat they will happenIt's a story we've lived throughagain and againYou know, we could havesat around foreverdebating whether or nota speed limit should existand who should havethe authority to enforce itBut we didn't

Instead, we went aheadand invented cars,and responsibly figured outthe details as we went alongEthical questionsabout new technologiesdo the most good whenthey facilitate the technology,not when they needlesslyhinder progressSo follow your dreams

And, as soon as you can,show them to meAnd, as always,thanks for watching

Related Videos

hey guys techrax here coming at you guys with another giveaway this is gonna be an iPhone 5s two of these brand new phones not the one in this video guys this i...
hey guy there's a popular game in the app store called flappy bird right now and it's supposedly really annoying believe it or not I haven't played ...
hey guys tech Rex here so right here with me I have an Apple iPhone 5s this is the gold one I also have with me some liquid nitrogen yes this is the real deal a...
hey guys techrax here right here I have the iPhone 5s with me this is the champagne or the gold color I still can't really figure out if it's champagne ...
hey guys tear cracks here so this video really exciting video I have the new Samsung Galaxy s5 for you guys this is the long-awaited cellular devices releases i...
everyone techrax here in this video I've got the latest Samsung Galaxy s5 right here as well as the Apple iPhone 5s and we're going to be doing a simple...
everyone techrax here in this video guys have a really exciting device this is the Samsung Galaxy s5 charcoal black and I'm really liking I after seeing the...
everyone techrax here here with me off the Samsung Galaxy s5 this is a perfectly fine s5 it is cracked from the drop test that I had with also one minor neck as...
hey guys tetrax here so in this video I'm going to try and burn the newly released Samsung Galaxy s5 this is the shimmering white 16 gigabyte model and if y...
everyone techrax here so I got my burn Samsung Galaxy s5 and I wanted to see whether the heartbeat sensor would still work the heart monitor on your galaxy s5 a...
hey guys Tech Rex here so I'm really excited to bring you guys a giveaway for my channel but this time I'm actually teaming up with a buddy of mine your...
hey guys techrax here so in this video I'm going to be hopefully instructing you guys how to make your very own a tech sandwich slash burger slash meal so y...
hey guys techrax here so I've got a galaxy s5 here this is the copper gold hopefully you guys can see pretty well it is sunset so it's getting a little ...
hey guys techrax here so right here with me I have a professional deep fryer in here is already some canola or corn or whatever oil I don't know vegetable o...
hey guys techrax here so just trying to make this video short and quick i'm having recently i got five hundred thousand subscribers and yeah most of you guy...
hey guys texture so in this video I have a drop test on the latest LG g3 device now this is actually the gold-coloured LG g3 this has not been released in the U...
hey guys techrax here so in this video we'll teach you guys how to make your iPhone indestructible this is essentially a case that's been around for yea...
hey guys techrax here soon in this video we had a train run over the iPhone 5s so we actually did this in two different instances initially we had a Space Gray ...
hey guys tech cracks here so in this video we're going to be dropped testing the newly released Amazon fire phone this is exclusive for AT&T and I belie...
hey guys techrax here so right here with me I have the Amazon fire phone this is the one I dropped and you know I thought what better what else do I do with thi...
Be the first to comment “Mind Reading”

There are no comments yet.