top of page
saasasa.jpg

Researching the persuasiveness of mindfulness applications and their
impact on UX and behaviour change.

Research Goal

To understand how persuasive features are utilised in mindfulness applications and how this may affect user experience and impact on user behaviour change.

Research Questions

The project aimed to answer two research questions:

  1. What varieties of persuasive system features were utilised in current native mindfulness applications? and what techniques are beneficial for evaluating them? 

  2. How do persuasive features effect the user experience and can they impact user behaviour for uptake?

Objectives

  1. Conduct a formal expert evaluation of the 5 most popular mindfulness applications and determine what type of persuasive features are utilised and;

  2. Conduct a 4 week diary study with users to see how persuasive features were experienced and how they impacted the users experience.

  3. To quantitatively measure participants behaviour change over the course of the study and determine if persuasive features impact behaviour change.

Side contributions

  1. A practical application of the Persuasive System Design (PSD) framework and the the 10 Persuasive Design Heuristics in the evaluation of a systems persuasive features

  2. A practical use case on how behaviour change can be quantifiably measured using the 3D RAB model in the context of mindfulness apps.

What I Learned

Number 1: There are two effective tools for evaluating technologies for persuasive features utilised.The Persuasive Systems Design (PSD) model is the most comprehensive tool for evaluating persuasive technologies and the 10 Persuasive Design Heuristics is not as effective, but a more suitable discount evaluation tool for designers.

Number 2: Mindfulness applications can deploy up to 19 different persuasive features, however more persuasive features does not necessarily mean a more effective persuasion.

Number 3: Persuasion is linked with user experience and only 6 or the 19 persuasive features contributed to users having a positive user experience in the user study with Headspace.For example, during the study participants highlighted that reminders could become annoying as it reminded them at times that were not convenient.

Number 4: Randomly thrown together persuasive techniques can make users feel manipulated, not in control and make them abandon the experience, the key to persuading is to keep is simple.

Number 5: There are two effective tools for evaluating technologies for persuasive features utilised.The Persuasive Systems Design (PSD) model is the most comprehensive tool for evaluating persuasive technologies and the 10 Persuasive Design Heuristics is not as effective, but a more suitable discount evaluation tool for designers.

Number 6: Persuasive features are not the key to behaviour change. A user must want to change their behaviour (i.e. to practice mindfulness) in the first place. Without this motivation it is unlikely a good user experience (ability to perform the action) and persuasive feature (triggers) will persuade the user solely on their own.

Number 7: Behaviour change can be document and measured using the 3B-RAB model created by Wiafe. The model can be deployed in practical application to capture and measure participants behavioural change. However, a short 4 week study is unlikely to uncover sufficient results as 4 weeks is not long enough for participants to pick up a new behaviour or have effective change.

Read on for more details of the study, methodology, data analysis and detailed results.

Expert Review

Five

Mindfulness

Applications

Headspace

iMindfulness

Calm

Buddhify

Smiling mind

Two frameworks designed to evaluate persuasive systems

Persuasive Systems Design (PSD) Framework

10 Persuasive Design Heuristics

Created by Oninas-Kukkonen’s in 2007, is an evaluation tool that consists of four system features (Primary Task Support, Dialogue Support, System Credibility Support & System Credibility Support) and features 28 design principles within these features, that can be used to evaluate a system against.

Created by Kientz’s in 2010, these are a set of 10 design heuristics specifically tailored to evaluate persuasive systems with.

Methodology

Step 1

The author of the study first carried out an expert review evaluating all 5 mindfulness applications against the two persuasive frameworks.  A cognitive walkthrough technique was utilised to allow familiarity with the applications and their functionalities, by performing regular tasks in the apps.

Step 2

An independent UX expert checked the authors review. The UX expert  was introduced to each application and was allowed time to explore the application on their own accord as

Then the UX expert would either agree or disagree with the authors evaluation interpretation if the persuasive feature was utilised or not.

PSD Results

Of the 28 design principles within the PSD Framework, the mindfulness apps presented the total amount of persuasive features:

Headspace
19/28

Calm
19/28

Buddhify
17/28

Smiling Mind 17/28

iMindfulness 11/28

Pros and Cons of the method

  • Pro: An effective framework to determine if a application or technology utilises persuasive features.

  • Pro: Provides real world examples for each persuasive feature, which allows the user to understand it quicker.

  • Pro: Works well when used as an expert evaluation review tool.

  • Con: Examples given in the framework are slightly outdated based on website examples.  Additional examples in different technologies like AR, VR or mobile would be beneficial.

  • Con: The framework doesn't provide method on how to use the framework, so if used by multiple researchers it could be applied differently. A standard methodology would benefit.

10 Persuasive Heuristics results

Of the 10 persuasive design heuristics design principles within the PSD Framework, the mindfulness apps presented the total amount of persuasive features:

Calm
10/10

Buddhify
10/10

Headspace
9/10

Smiling Mind 9/10

iMindfulness
6/10

Pros and Cons of the method

  • Pro: A time efficient discount evaluation tool making it suitable for designers.

  • Pro: Easy to read and intuitive descriptions are given for each heuristic making them easy to work with.

  • Con: Each heuristic would benefit from some examples given to put the heuristics in context.

  • Con: The Heuristics were more suited to highlighting any usability issues relating to persuasiveness, rather than a tool to evaluate the persuasive features.

Diary Study

Overview

A qualitative 28-day diary study was carried out to explore the user experience of the persuasive features of Headspace, if they were effective and if they contributed to behaviour change.

The study recruited 11 participants, of which one participant took part in the pilot study.

 

Participants were encouraged to use Headspace in a self-directed way and no reminders were sent. Participants were only contacted when they were required to be interviewed during the second and the fourth week. No reminders were utilised as the study aimed at looking at persuasive features and behaviour change, all of which may have been influenced by sending constant reminders.

 

Data collection was done through both open-ended and structured questions. The design of the questions was based around the 19 persuasive principles identified in the expert review of Headspace using the PSD model, in the first part of this study.

Participants

11
 

Day study

28

28

Month Access to Headspace

1

Methodology

The study was conducted according to the following timeline:

  1.  Recruitment and pre-screening participants 2 weeks prior to study commencing and categorising participants attitude and behaviour based on the 3D-RAD model.

  2. Pilot Study for 10 days with 1 participant

  3. 4 week qualitative Diary Study begins

  4. Interviews at the two weeks halfway point

  5. Interviews at final week

  6. Exit interview and categorising participants new attitudes and behaviour based on the3D-RAD model

  7. Follow-up interview 2 weeks after the study has ended to re-categorise participantsattitudes and behaviour based on the 3D-RAD model

Recruitment

The study recruited 11 participants, of which one participant took part in the pilot study.

 

A convenience sampling method was utilised for recruitment. This type of sampling focuses on a sample that is easy to access and readily available. As a diary study requires a high level of commitment, participants were known to the author of this study.

 

Participants were recruited 14 days before the study began. This aimed to reduce the impact of priming participants, reduce the possibility of potentially persuading participants and influencing participants attitude and behaviour towards mindfulness.

None of the participants recruited had used Headspace before. Two participants, P1 and P3

had practiced mindfulness before the study.

Screenshot 2022-03-26 at 22.08.54.png

Diary Study Design

The study consisted of a 28-day qualitative diary study and participant interviews during the second and final week of the study.

 

Participants were encouraged to use Headspace in a self-directed way and no reminders were sent. Participants were only contacted when they were required to be interviewed during the second and the fourth week. No reminders were utilised as the study

aimed at looking at persuasive features and behaviour change, all of which may have been influenced by sending constant reminders.

Data collection was done through both open-ended and structured questions. The design of the questions was based around the 19 persuasive principles identified in the expert review of Headspace using the PSD model, in the first part of this study. Participants were given the choice of a physical (printed) or virtual (google document) copy of the booklet. Only P2 & the Pilot participant used a printed copy of the diary study. All other participants used the online version of the diary study.

Examples of Diary Study Pages

Screenshot 2022-03-27 at 15.07.31.png
Screenshot 2022-03-27 at 15.07.09.png

Interviews

Semi-structured Interviews were conducted on the second and final week of the Diary Study. As participants were encouraged to use Headspace in a self-direct way, there was a risk that some participants would not send in data regularly. Therefore, interviews were also utilised, as this promotes richness in data through comprehensive understanding of participants’ views and opinions.

 

The interview questions were carefully curated based on the 19 persuasive principles identified in the expert review of Headspace.This allowed in-depth data to be gathered and allowed probing of participants responses.

Data analysis

The diary study and interview data were analysed using thematic analysis. Thematic analysis identifies, examines and interprets patterns within the qualitative data. As the diary study and interview questions had been tailored to the specific persuasive features, each question had an identified theme allowing it to be categorised simply.

Screenshot 2022-03-27 at 17.07.44.png

Diary Study Results

The mere presence of persuasive features might not be enough to make an application or a system more persuasive. It was therefore important to see how participants experience these features and if they contributed to a positive experience or a negative experience.

 

All 10 participants completed the 28-day diary study. However, the number of days for each participant submitted data ranged from only 3 days (P2) to 26 days (P3). Participant P3, completed every page of the diary study as they used Headspace twice a day sometimes.

 

As participants were encouraged to use headspace in a self-instructed way, every entry into the study correlates to using Headspace once. One of the biggest barriers to use was participants finding time in their schedule.

Screenshot 2022-03-27 at 17.10.07.png

Over the course of the study it was found that all participants responded positively to 6/19 persuasive techniques utilised in Headspace. 

Participants had mixed responses (i.e. some had a positive, whilst others negative) to 10/19 persuasive technique utilised in Headspace and  lastly, participants responded negatively to 3/19 persuasive techniques used Headspace.

6/19

Persuasive features contributed to a positive UX.

These were:

  1. Reduction

  2. Tunnelling

  3. Liking

  4. Trustworthiness

  5. Surface Credibility

  6. Rehearsal

10/19

Persuasive features contributed to a neutral UX.

These were:

  1. Tailoring

  2. Personalisation

  3. Self-monitoring

  4. Expertise

  5. Simulation

  6. Praise

  7. Rewards

  8. Reminders

  9. Social Role

  10. Suggestion

3/19

Persuasive features contributed to a negative UX.

These were:

  1. Real-world feel

  2. Social Comparison

  3. Normative Influence.

Measuring Behaviour Change

Measuring behaviour change

Current behaviour

Attitude towards  behaviour

Attitude towards changing behaviour

Quantifying behaviour change

Measuring behaviour change in a quantitative way is something that at the point this study was conducted there was very little studies or research and methods that captured behaviour change and how it can be captured in the context of persuasive features 

 

The framework that was utilised was the 3D-RAB Model, created by Wiafe et al in 2011. The 3D-RAB model aims to evaluate and implement persuasive technology systems from a behaviour change perspective.

 

The model takes into account the relation between the user’s current behaviour (CB), attitude towards the target behaviour (ATTB) and attitude towards maintaining or changing current behaviour (ATCMB).

 

Wiafe highlighted that one use of the model is its ability to monitor a user’s progressive change during a persuasive intervention. As no studies at the time of this study had utilised this model in the context of monitoring a user behaviour change, this study aimed to utilise this model and prove if the 3D-RAB model could prove to be an effective tool.

Behaviour Change

=

Motivation

+

Ability

The 3D-RAB Model

The 3D-RAB Model is based around Cognitive Dissonance and it is defined as: This refers to a situation involving conflicting attitudes, beliefs or behaviours that participants may experience. It is the inconsistency between the user attitude and behaviour.

 

For example, a user may have a positive attitude to mindfulness as they understand it is good for wellbeing, however they don’t actually perform the behaviour. This would be Cognitive Dissonance.

Each participants before and after states for the three areas were captured, these were:

  • Current Behaviour (CB)

  • Attitude Towards Target Behaviour (ATTB) and;

  • Attitude Towards Maintaining/Changing Target Behaviour (ATMCB)

 

Participants CB, ATTB and ATCMB were classified as either positive or negative. Where participants expressed a neutral attitude.

By categorising participants answers to CB, ATTB and ATCMB as positive or negative, this allowed each participants attitude and behaviour to be categorised into 1 of 8 states. Whereby State 1 is participants are performing the target behaviour and State 8 they are not.

 

Additionally, each state provides a value for the participants Cognitive Dissonance, Stability, Expected Natural State Transition Tendency and their Targeted State Towards Persuasion. These are defined as:

Stability: This refers to how stable a user’s attitude and behaviour is towards the target b behaviour. For example, if a user is Stable(+) they are likely to continue the target behaviour. If it is Stable(-) they are highly unlikely to take up the target behaviour. finally, if it is

Unstable(+) they are likely to move closer towards the target behaviour or if it is Unstable(-) they are likely to move away from the target behaviour. As stability is Unstable users may move in either direction.

Expected Natural State Transition Tendency: This is the expected state a user will transition to based on their answers to CB, ATTB and ATCMB.

Targeted State Towards Persuasion: This is the possible behavioural state they would need to transition to towards the target behaviour.

3D-RAB Model and the 8 states

Using The 3D-RAB Model

To begin participants initial behaviour was categorised two weeks before the diary study began. The following questions were asked to determine CB, ATTB and ATMCB. Answers would either be positive or negative. These questions were:

 

  1. CB (current behaviour) - Have you or do you currently practice mindfulness?

  2. ATTB (attitude towards target behaviour…) - Is mindfulness/meditation something you have wanted to learn/do?

  3. ATMCB (attitude towards maintaining changing behaviour) - Would you say you are ready to change your current behaviour to incorporate mindfulness into your routine?

At study end participants final behaviour was categorised at the end of the diary study. The following questions were asked to determine CB, ATTB and ATMCB. These were:

 

  1. CB - Have you been using Headspace?

  2. ATTB - Would you describe your attitude towards meditating as positive, negative or neutral?

  3. ATMCB - Will you continue to use headspace/practice mindfulness?

This was then repeated 4 weeks after to the study to see is any participants continued to use Headspace.

An example of participants 8 behavioural measurements over the course of the study are shown below. participant 8 started off with a positive attitude toward the behaviour change and using Headspace, however their Cognitive Dissonance was strong and despite a positive attitude this did not translate into persuasion and taking up using Headspace.

Screenshot 2022-03-27 at 17.43.59.png

Behaviour Change Outcomes

At the end of the 28 days of using Headspace all participants attitude and behaviour states transitioned towards the target behaviour (State 1) i.e. the long-term adoption of Headspace. However, only 3 participants (P1, P7, P10) ended the study on the target behaviour (State 1).

When participants behaviour was captured 4 weeks after the study had ended. Only Participant P1 remained on state 1 and continued to use Headspace.

The remaining Participants did not continue to use Headspace after the study had ended, resulting in their behaviours moving

towards state 8.

Screenshot 2022-03-27 at 17.44.08.png

Discussion

Answering The Research Questions

Research Question 1: What varieties of persuasive system features were utilised in current native mindfulness applications? and what techniques are beneficial for evaluating them? 

In this study, the PSD model and the 10 PDH were found to be valuable evaluation frameworks to analyse persuasive technology. This research revealed that the PSD model was beneficial for high level detailed expert evaluations and the 10 PDH for quicker discounts evaluations at early designs stages when a quicker discount evaluation method would be most valuable.

 

Both methods would be suitable for defining requirements when designers and researchers begin to plan and design persuasive technology.

The study highlighted that current mindfulness application can employ up to 19 different persuasive features, however they can each have different outcomes when it comes to user experience. 

 

More is sometimes better, however in the context of persuasion and mindfulness applications using more persuasive features does not guarantee that the technology is going to be more persuasive. The study demonstrated that it is important to consider how the persuasive features are utilised and how they can fit into the user goals and routine.

 

Furthermore, the way users experience the feature can have an impact on their attitude towards the technology which can better help support persuasive technology . For example, during the study participants highlighted that reminders could become annoying as it reminded them at times that were not convenient. Additionally, designers should be cautious of rewards and streaks in mindfulness applications, fore example, it was often perceived as patronising and creating forced use (i.e. a user felt bad not keeping their meditation streak up and only used the application to continue their streak).

Research Question 2: How do persuasive features effect the user experience and can they impact user behaviour for uptake?

Research Objective 1: Conduct a formal expert evaluation of the 5 most popular mindfulness applications and determine what type of persuasive features are utilised.

In this study, the PSD model and the 10 PDH were found to be valuable evaluation frameworks to analyse persuasive technology. This research revealed that the PSD model was beneficial for high level detailed expert evaluations and the 10 PDH for quicker discounts evaluations at early designs stages when a quicker discount evaluation method would be most valuable.

 

Both methods would be suitable for defining requirements when designers and researchers begin to plan and design persuasive technology.

The study highlighted that current mindfulness application can employ up to 19 different persuasive features, however they can each have different outcomes when it comes to user experience. 

 

The study also contributed on the side that the PSD model is a more effective tool to evaluate persuasive features in technology compared to the 10 PDH model. However, the 10 PDH offers a quick discount evaluation method suitable for designers where time is important in the process.

Research Objective 2: Conduct a 4 week diary study with users to see how persuasive features were experienced and how they impacted the user experience.

More is sometimes better, however in the context of persuasion and mindfulness applications using more persuasive features does not guarantee that the technology is going to be more persuasive. The study demonstrated that it is important to consider how the persuasive features are utilised and how they can fit into the user goals and routine.

User experience is closely linked to persuasion. By providing a good user experience this can increase the chances of persuasion. Pairing this with appropriate persuasive features can increase the chances of persuasion. Randomly thrown together persuasive techniques can make users feel manipulated, not in control and make them abandon the experience, the key to persuading is to keep is simple.

 

Furthermore, the way users experience the feature can have an impact on their attitude towards the technology which can better help support persuasive technology. For example, during the study participants highlighted that reminders could become annoying as it reminded them at times that were not convenient. Additionally, designers should be cautious of rewards and streaks in mindfulness applications, for example, it was often perceived as patronising and creating forced use (i.e. a user felt bad not keeping their meditation streak up and only used the application to continue their streak).

Research Objective 3: To quantitatively measure participants behaviour change over the course of the study and determine if persuasive features impact behaviour change.

Behaviour change is an interesting area and one that can be difficult to solely influence just by designing a persuasive piece of technology. A user must want to change their behaviour (i.e. to practice mindfulness) in the first place. Without this motivation it is unlikely a good user experience (ability to perform the action) and persuasive feature (triggers) will persuade the user solely on their own.

Persuasive features can aide behaviour change, as they can have a positive or negative impact on the users experience which can in term impact their use or take up of the technology used.

Behaviour Change often takes time and a 4 week diary study to study this was unlikely sufficient to accurately measure and document this. The habit of taking up mindfulness was also likely, too big of a behavioural change for a users to grasps in 4 weeks as the benefits from performing the habit can take several months to full realise.

What the study was able to show is how the 3D-RAB model can be deployed in practical application to  capture participants attitude and track their cognitive dissonance over the study, which if designers had the connivence of running longer more detailed studies, it could provide benefit to documenting the technology and its impact on user behaviour. 

Thank you for reading.

bottom of page