How Lifelogging Can Help Those Living with Dementia: An Interview with Cathal Gurrin, Lifelogger and Associate Professor at the School of Computing, Dublin City University

Live-in care provider, Elder, spoke to Cathal Gurrin, Lifelogger and Associate Professor at the School of Computing, DCU about how personal data and lifelogging can be used to aid memory loss. Computer scientist Cathal Gurrin thinks that lifelogging – the process of capturing and recording your everyday experience, through photos, video, sensors and other tools – could prove a valuable way of augmenting our lived world and creating a richer human experience. We talked to him about the potential and the challenges of a ‘search engine of you’.

What is lifelogging?

Lifelogging started in the US about 20 years ago when Gordon Bell, a computer scientist and researcher at Microsoft, built a big database of everything he did, where he went, who he spoke to, the TV he watched and the books he wrote.

At the same time in Cambridge, Lindsay Williams was building a wearable camera at Microsoft Research to help her to be able to find her glasses – a simple thing like that. Bell got involved in that work and both research streams led to what we now know as lifelogging. Microsoft funded eight research groups around the world to build prototype lifelog tools, one of which was here at Dublin City University.

A lifelog basically captures a detailed diary of everything you do, using pictures, videos, locations, activity trackers and many more easy-to-use sensors. Right now there’s no such thing as a ‘lifelogging tool’ on the market so we work with different devices that are commonly available to build a lifelog; a smartphone can build part and fitness trackers and heart rate monitors can record the medical side, for example.

We also do interesting work with small wearable cameras like the Microsoft SenseCam, which can take pictures of what you’re doing every so often. These photos capture the world as you see it, since they are worn on a necklace round your neck. Our software looks at those pictures and infers meaning – are you having your medicine? Are you going to meet your friends? Are you driving to the shops? We then use that information to build what will essentially be a diary that we can use to support people.

“Lifelogging is like having somebody standing beside you writing notes – you can refer back to them for assistance … I don’t see memory support as any different to wearing glasses for eye support”

Essentially, we’re creating a digitised version of you; everything you eat, everywhere you go, every interior, every physical response you have – all your experiences go into your private database and from this we can create different tools such as search engines. Imagine being able to Google yourself, ‘when did I… what did I… why did I?’ to find out answers.

How did you get interested in this area of research?

I’m a computing professor and my PhD was on building search engines. I worked on my internship with the Head of Search in Google and built the world’s first Irish language search engine (that nobody really used!) back in 2002.

When the Microsoft SenseCam first came to our research group, I saw this as something that might enable me to search people’s life experiences. I wanted somebody in the university to wear the camera for a week, but nobody would. So, I wore the camera on a trip to Norway for a fortnight in 2006 and sent pictures back every day mapped to video as my daily diary. Since then I don’t think I’ve had more than 10 days without wearing a camera and I’m probably pushing on for 20m pictures by now.

With these types of diaries, you can then build a search engine to find your content – and I’m the research challenge for my PhD students, who help me answer things like,’ ‘What wine did I drink at Christmas dinner three years ago?’

What is the potential for lifelogging in the ageing and healthcare space?

Lifelogging started off being about supporting people to find things in their daily life, and it was quickly picked up by medical researchers. Healthcare professionals at Addenbrookes Hospital in Cambridge started using these cameras in 2007 to understand the impact of memory impairment on daily life, for example.

The idea is to try to build tools to provide guidance and memory support for people. It’s believed that this may be especially helpful for somebody with early stage Alzheimer’s to help reinforce memory that would otherwise be lost.

We already have all the technology we need to build such a guidance system. For example, a company called Aira in San Diego uses Google Glass to assist blind people. If a blind person is trying to find the ATM, they can use the Google Glass to call a call centre and somebody can see what they see and guide them.

It’s a simple technology and we see lifelogging as being the next stage in this technical progression. Who will you ask when you need help? You will ask your lifelog and it will tell you where, what, why, when, etc. The initial experiments using lifelogs in this way have shown that there can be a big benefit to using it to support human memory,

Are there any ethical grey areas in lifelogging?

I’m a university professor and I’m lucky that we have ethical approval for doing this kind of work. But ethics and privacy are really big things for us, and we have always taken them very seriously – in fact some of the tools we built a couple of years ago to blur faces in lifelogs are already automatically blurring faces in a new start-up company called SeekLayer which helps to protect people’s privacy on CCTV video. But there are grey areas. If you wear a fitness tracker it’s only recording data about you, but once you put on a wearable camera it records data about everybody else, and this is a privacy challenge.

A good example of this was Google Glass which launched five years ago. This was a lifelogging platform – although Google wouldn’t necessarily admit that – and could capture pictures and remind people about things, but it didn’t take off. Part of the reason wasn’t the technology, it was that people didn’t like cameras on other people walking around.

As society gets used to these kinds of devices it will become the norm though; Apple is releasing wearable camera glasses in 2023, Microsoft is building them, and Google is selling Google Glass version 2.

“All your experiences go into a digital database and from this we can create tools such as search engines. Imagine being able to Google yourself to find out answers?”

However, in our work, what we call ‘a camera’ doesn’t actually have to be a camera. We don’t need to store a single picture to build our personal search engines. We can build devices that look just as your eye looks, hypothesise and make decisions, but don’t ever even have to store a single image or picture. We can answer questions like ‘who was it that called to my door this morning?’, or ’where are my glasses?’ and not invade anyone’s sense of personal privacy. Then this technology becomes more socially acceptable.

Basically, we don’t care about the pictures or the videos we get – we care about understanding this data through the application of computer science tools like artificial intelligence. We as humans are still the most important thing, but we can make our life easier by doing this, especially as we get older.

Are you ‘outsourcing consciousness’ in a way by lifelogging?

Can it be a capture of consciousness? Nobody can answer that question; science does not have a detailed enough understanding of what human consciousness is yet. The only thing we know for sure is that it arises as a by-product of information processing in our brains.

Lifelogging is like having somebody standing beside you writing notes – you don’t exist in those notes, you refer back to them for assistance, so it’s not about consciousness or about uploading yourself to a computer.

I don’t see memory support as any different to the fact that I’m wearing glasses for eye support. People think it’s outsourcing a major important human function, but my eyes are important, and I get help with those; my memory’s important so why shouldn’t I get help, especially in terms of cognitive impairment?

Do you see this as a technology that people will use to plan ahead and log themselves before memory impairment starts?

Yes, I think that will happen. Part of the reason I’m interested in doing my own lifelog is that my mother has dementia and I see her forgetting every day. I don’t want to be faced with the situation where I can’t remember my ‘now’. Looking at my genetic makeup I have a high chance of getting Alzheimer’s so I might as well have the backup, and I have decided to spend more than a decade doing this. I am already very lucky to have my life experience digitised since 2006.

But it won’t just be used for those reasons. There will be many other reasons why people will lifelog; for example to assist productivity or learning. Once you’ve seen a positive benefit for you in your job by putting on Google Glasses and running the lifelog app, you’ll do it, and your colleagues will do it, and the whole thing will propagate. Lifelogging will assist you in many different ways  because you have access to all the things you’ve done that have worked. Then it just becomes part of daily life and an enhancement of our abilities as people and as a society.

Do you think it’ll collapse the now and the past in the lived experience?

No, but it will always make the past accessible, just as your memory makes the past accessible; but you’ll have better data and a better memory. Reminiscing about the smell of your grandmother’s cooking or something from your childhood, you’ll be able to recreate much more of that in an immersive and data-rich environment.

It’s like going back and seeing really detailed life experiences. You’ll also be able to go back through the experiences and details of people who are no longer with us. All in all, it’s going to make for a much richer human experience.

This article first appeared on Elder, a solution providing an alternative to residential care home by providing access to first rate live-in care, elderly care and dementia care. You can read more interviews from Elder’s Magazine here.