How are you feeling right now?

“You can spend a long time sifting through this data. There is so much in it,” says Dr. Ramona Schödel. The psychologist at Professor Markus Bühner's Chair of Psychological Methodology and Diagnostics is a mobile sensing expert. Mobile sensing can be used to analyze the digital data that users constantly produce with their smartphones in order to answer psychological questions.

The aim is to understand people better than would ever have been possible with traditional methods of psychology. The pathway to doing so is to describe someone’s personality and behavior based on their digital data – a practice known in the trade as ‘digital phenotyping’. Expert circles see so much potential in this methodology that they speak of a ‘quantum leap’, according to an introductory article in the professional journal Psychologische Rundschau (Psychology Review). For Ramona Schödel and her team in the PhoneStudy project, one of the most pressing research questions right now is therefore: How well can someone’s personality and behavior in the analogue world genuinely be modeled on the basis of digital data? To find out, they developed an app specially for this purpose.

What likes on a social media platform reveal

Several studies in recent years have shown how much personal information can be revealed by digital trails. For example, 260 likes on a social media platform are sufficient to predict someone’s personality. Moreover, the PhoneStudy project itself has already demonstrated that usage data alone – how often which apps are used, for example – is enough to draw conclusions about ‘the big five’, a well-established concept of personality traits. For instance, information about a person’s day/night rhythm as reflected in their smartphone usage allows statements to be made about how conscientious they are.

Yet Schödel still finds herself repeatedly surprised as she analyzes the collected data. Recent studies under the aegis of the project – some of them as yet unpublished – show that there are limits to the validity of statements derived from digital data.

The privacy paradox

Anyone taking part in one of the project’s studies downloads the PhoneStudy app to their smartphone. The app itself and all the studies are subject to very stringent privacy requirements. The relevant Ethics Commission is likewise involved in each case.

For these reasons, detailed privacy information is the first thing shown to participants. Which is the moment when many drop out. While this is frustrating for the scientists, they admit that it precisely matches their expectations: “The things we record are also tracked by tech companies that sell smartphones and apps. But if you take part in one of our studies, you first leaf through a few pages of privacy information. From the dropout rate at this point, we can see that this acts as a deterrent to potential participants. On the other hand, I have no doubt that the same individuals use many popular apps. Researchers refer to the privacy paradox: If enough value is added, digital offerings will still be used even if very sensitive data has to be disclosed in return.”

Another factor is that, in the world of commercial providers, privacy policies can quickly be clicked away by simply checking a box, so people may not be consciously aware of them. On a selective basis, the academic studies that have been published for a number of years based on mobile sensing at least now show the public what information users of digital media disclose about themselves day in, day out without really thinking about it.

Identifying structures in data

Enlarge

The PhoneStudy app was created in collaboration with media IT people and is constantly being developed and improved. Unnoticed in the background, it keeps a record of the digital data generated when someone uses their smartphone. “We think carefully about what data we need to answer a specific question. If we want to track the person’s radius of movement, for example, we also record GPS data. But there are studies where that is irrelevant. Since our app has a modular design, we can activate and deactivate such features at will.”

This pure usage data is combined with what is known as the experience sampling method: Several times a day, short messages containing questions such as “How do you feel right now?” are sent to the person’s smartphone.

In this way, researchers want to discover the pieces of the puzzle that makes up the person’s everyday routine. Possible correlations that are of relevance to personality research thus come to light: For example, what happens in a person’s environment or in their behavior when they are feeling especially bad or especially good? And is it possible to predict their momentary subjective perceptions on the basis of the data?

To begin with, however, the app supplies “raw” data, as Schödel puts it. For example, the database on the server displays nothing but time stamps at which a certain app was used. This information only becomes interesting when it is analyzed more selectively. As a rule, the software codes spend several days raking through the mountains of data.

One app, many studies

The team around the Chair of Psychological Methodology and Diagnostics designs studies primarily to find out what conclusions, if any, can be drawn from mobile sensing. That said, the possibilities for the application are so broad and interest among experts is so great that content issues, too, are being addressed. These issues show, for example, how a knowledge of digital data can benefit people and society at large.

Together with colleagues at the Universities of Münster and Osnabrück, the team looks at how people deal with crises, for example. The CoCo project began in 2021, while the coronavirus pandemic was still raging. Then came the outbreak of the war against Ukraine. In a large-scale mobile sensing study, the researchers are now investigating how all these crises affect people’s wellbeing.

Frank Niklas, Professor for Educational Psychology and Family Studies at LMU, uses the app himself. As part of the ‘Learning4Kids’ project, it helps him study how children interact with tablets. It is thus easier to understand how learning apps work and how children in educationally disadvantaged families can be supported.

In cooperation with the Leibniz Institute for Psychology (ZPID) and the German Institute for Economic Research (DIW Berlin), the app is now also being used for surveys over longer periods under the auspices of the SOEP Innovation Sample.

While her vision is still some way off in the future, Ramona Schödel can imagine using the app to accompany individuals for years with “well-reasoned questions”. In the field of mental health, for example, that is of interest not only from a research perspective. “Other research groups have already produced initial papers on how digital data can play a part in spotting depression. For example, depressive patients have a limited radius of movement, they tend to withdraw into themselves. These are indicators that can readily be recorded from a smartphone and can be used to help develop preventive early-warning systems.”

In the meantime, the PhoneStudy team will continue researching what mobile sensing data reveals – and learning how to read people’s digital trails.

Smartphones certainly appear unbeatable when it comes to documenting behavior. Whereas people seldom recall things exactly – when asked “How any times did you use your smartphone today?”, for example – the app simply keeps an accurate record.

The researchers now also know that there are limits to the validity of conclusions drawn from data tracking, such as when trying to assess a person’s current mood. “People themselves are far better able to provide information about something as subjective as their emotional state than digital data could ever do,” Schödel says. That may sound sobering to the ears of a researcher, but it is reassuring for users: It is good to know that, at least in some respects, we still know more about ourselves than our phones do.

The PhoneStudy Project: More about the project

Project Coping with Corona: How are people coping with crises?