Research Article
Ann Depress Anxiety. 2021; 8(1): 1107.
Bot to the Rescue? Effects of a Fully Automated Conversational Agent on Anxiety and Depression: A Randomized Controlled Trial
Gutu SM, Cosmoiu A, Cojocaru D, Turturescu T, Popoviciu CM and Giosan C*
Department of Psychology, University of Bucharest, Romania
Corresponding author: Cezar Giosan, Department of Psychology, University of Bucharest, Romania
Received: July 31, 2021; Accepted: August 17, 2021; Published: August 24, 2021
Abstract
Web-based conversational agents powered by Artificial Intelligence (AI) and rooted in cognitive-behavioral therapy have been proven efficacious in alleviating the symptoms of anxiety and depression, when compared to passive controls. However, the benefits of a fully automated agent vs. active controls have not yet been examined. Furthermore, the potential impact of such interventions on the transdiagnostic factors underlying anxiety and depression is not known.
To elucidate this, 95 adults were randomized to receive (1) a 2-week intervention with an AI-powered chatbot (Woebot) (n=39) or (2) regular psychoeducational materials (n=54). In completers’ analyses, significant main effects of time were obtained for one of the primary outcomes, anxiety, and for the secondary outcomes, transdiagnostic factors, with both groups showing decreased anxiety and intolerance of uncertainty and increased rumination, selfcompassion, guilt and shame. No group by time interaction effects were found for either of the primary outcomes, depression and anxiety, or for the secondary outcomes. Intent-to-Treat analyses also revealed no significant effects of group on the primary or secondary outcomes. Our findings point to the necessity of further research to better understand the areas where chatbots might bring benefits superior to those obtained through simple and inexpensive strategies.
Keywords: Mobile mental health; mHealth; Depression; Anxiety; Transdiagnostic factors; Health apps
Abbreviations
AI: Artificial Intelligence; ANOVA: Analysis of Variance; CBT: Cognitive-Behavioral Therapy; ITT: Intention-to-Treat; M: Mean; RCT: Randomized Controlled Trial; SD: Standard Deviation
Introduction
Mental disorders, which affect up to 29% of people in their lifetime [1] and come with significant societal and personal costs [2], have increased in their prevalence and severity [3-6]. The onset period for several mental disorders, especially mood and anxiety disorders, is early 20’s [4,7,8], with significantly higher rates of depression found in college students than in the general population [9]. Subclinical levels of depression and anxiety also lead to significant impairment [10,11].
While these realities point to the critical role of interventions in alleviating such symptoms, only 35.5-50.3 % of serious cases in developed countries and 76.3% - 85.4% in less developed countries end up receiving professional care [12]. The 2018 National Survey on Drug Use and Health reported that up to 56.7% of Americans with some form mental illness received no treatment, regardless of the form and severity of mental illness [13]. Tellingly, only 16.4% of students meeting the criteria for a mental illness receive adequate treatment for it [7].
Although the reasons for not receiving psychological care vary across individuals, some of the reported barriers are: treatment not needed, lack of time, preference for self-management, and perceived stigma and embarrassment [14,15].
In addition to these obstacles, the current pandemic context brings forth additional limitations for conventional face-to-face psychological assistance, as social distancing, mask wearing, and surface disinfection are mandatory, pointing to the importance of exploring alternative means of providing psychological care, such as automated CBT interventions recommended by The National Institute for Health and Care Excellence, which can offer information and guidance similar to treatments delivered by standard methods [16].
Owing to the enormous recent increase in computing power, conversational agents (chatbots) powered by Artificial Intelligence (AI) (e.g., Replika, Shim, Woebot, Wysa) have emerged as a potentially useful therapeutic method in the recent years. Chatbots are cheap, easily accessible, and do not suffer from scale-up challenges. However, while the use of therapy bots has increased recently, the technology behind this kinds of interventions is still experimental in nature and the field lacks high-quality evidence derived from randomized controlled studies [17].
Given the fact that many such solutions may be marketed to vulnerable individuals, the necessity of rigorously validating their claims of mental health improvements with their use becomes imperative. Some evidence for the benefits of the use of chatbots in psychiatry is positive, but there are concerns about the lack of higher quality evidence for any type of diagnosis and interventions in mental health research that uses them [18]. A systematic review on these types of interventions found that they can be effective in reducing depression, anxiety, stress, and substance use, but, out of the apps that were reviewed, only two were available for commercial use [19].
There is some evidence that web-based conversational agents rooted in cognitive-behavioral theory can be efficacious in alleviating the symptoms of some mental health conditions, such as anxiety and depression. For instance, a pilot randomized clinical trial on the effectiveness and adherence of an AI-powered smartphone app, delivering strategies used in positive psychology and CBT interventions using a conversational interface, reported no significant changes in the intervention group compared to a waitlist on any of the outcome measures. However, when the analysis included only the participants who adhered to the intervention, there was a significant group-by-time interaction effect on psychological well-being and perceived stress, with small to large effect sizes [20]. Likewise, another RCT on an AI-powered psychological intervention-Tess-showed significant reduction of self-reported symptoms of depression and anxiety in college students, compared to a control group who received informational materials [21].
Another AI-driven conversational agent-Woebot, a fullyautomated CBT-driven chatbot – also showed promise in an RCT which compared it to a passive control group, in that it led to a higher decrease in depression and anxiety, although the control group’s adherence to the intervention was not examined [22].
To date, to our knowledge, the research on the mediators and mechanisms of change in automated, computerized interventions has solely focused on symptom changes through specific therapeutic protocols for mental disorders. However, there is a growing body of evidence on the impact of transdiagnostic factors on mental health [23]. Transdiagnostic factors are vulnerability factors that overlap across several mental disorders [24]. Thus, treatments targeting key transdiagnostic factors (i.e., common vulnerability factors) could have a general impact across multiple disorders and prove efficacious in preventing declines in mental health [24].
For depression and anxiety, which tend to co-occur [25], the following major transdiagnostic factors have been identified: rumination, guilt, shame, intolerance of uncertainty (all associated with negative outcomes [23]), and self-compassion (associated with positive outcomes [26]).
Recent progress on the merits of AI-powered conversational agents notwithstanding, little is currently known about the potential impact of such chatbots on the transdiagnostic factors that underlie anxiety and depression.
To this end, the present study’s objectives were twofold: (1) evaluate the efficacy in reducing anxiety and depression using a CBToriented conversational agent-Woebot-compared to an active control group, who received psychoeducational materials that they needed to show mastery of, and (2) to examine the role of this conversational agent in reducing the severity of the transdiagnostic factors associated with depression and anxiety.
Methods
Recruitment and procedure
Potential participants were recruited through announcements on social media websites such as Facebook and Instagram. The inclusionary criteria were: at least 18 years old; access to a computer/ mobile phone/tablet and the Internet; and the ability to read and write in English (at least B2 level of English in the Common European Framework of Reference). The study was approved by the Ethics Committee of a large university in Europe.
After signing an informed consent, all participants were assigned a personal code and sent an online baseline evaluation. Confirmed participants (i.e., those who completed the baseline evaluation) were randomized to either the experimental (i.e., Woebot) or the control group. After approximately one week (T2), all enrolled participants were contacted to fill out an instrument assessing the transdiagnostic factors, and those in the experimental group were required to send a screenshot of their time spent on Woebot and their check-in diagram to check for treatment adherence. After two weeks (T3), the participants were contacted once again to complete the initial set of scales, and those in the experimental condition also sent screenshots of their check-in diagram and time spent within the app. The primary outcomes (anxiety and depression) were measured at preintervention and post-intervention, whereas rumination, intolerance of uncertainty, shame, guilt, and self-compassion were additionally assessed mid-intervention (after seven days), in order to test for their effects as mediators of treatment outcome. Participants who completed all three sets of evaluations were entered in a raffle for the opportunity to win the equivalent of US $20.
Data collection was done exclusively online; the online instruments were created using Google Forms and QuestionPro.
Participants
An a priori power analysis was conducted with the G*Power [27], as informed by previous trials exploring the efficacy of fully automated conversational agents [22]. For a medium effect size of f = .25 (i.e., approximately equivalent to a partial η2 of .06), at a statistical power of .80 and an alpha of .05, a total number of 38 participants (19 per trial arm) was deemed sufficient. However, to allow for attrition, a higher number of participants was recruited (Figure 1).
Figure 1: CONSORT flow chart.
From the initial sample, 42 participants (20.8%) scored over the cut-off score for severe depression, 53 participants (26.8%) scored over the cut-off score for severe anxiety at pre-treatment.
A final sample of 95 adults (1 male, 94 female), from a non-clinical population, aged from 19 to 43 (Mage = 21.8, SD = 4.86) completed our trial.
Interventions
The experimental group (Woebot): Woebot is a fully automated, AI-powered conversational agent based on CBT principle, designed for non-clinical use. It provides users with daily conversations and mood tracking. Building upon users’ replies to general questions about context and mood, the Woebot Health’s Conversation Management System uses a modular approach to offer relevant psychoeducational materials and brief interventions, such as behavioral activation, mindfulness, cognitive restructuring, relaxation, gratitude journal and other. The app has been extensively described elsewhere [22,28].
Active control condition: In the active control condition, participants received a daily e-mail with a minimal psychoeducational intervention, consisting of mental health information sheets from the Centre for Clinical Interventions [29] and were asked to reply to questions (i.e., short pop-quizzes) from the materials provided. For example, on day 2, after reading the information sheet provided, participants were required to answer the following question: “What are the names of the evidence-based therapies mentioned in the Depression Information Sheet 03?”. They replied via e-mail and their response was logged if they provided the correct answer. The estimated daily time required for this task was 5 minutes.
Measures
The following demographic information was collected: age, gender, and educational level.
Anxiety and depression: To measure symptoms of anxiety and depression, the Depression, Anxiety and Stress Scales-DASS were used [30]. DASS is scored on a four-point scale (0 = never, 3 = very frequently). The Cronbach’s alpha for this study was .88 for depression and .82 for anxiety.
Transdiagnostic factors: Rumination was assessed with the Ruminative Responses Scale-RRS [31], a 22-item instrument using a four-point Likert scale (1 = almost never, 4 = almost always). The Cronbach’s alpha for this study was a = .93.
Self-Compassion was assessed with the Self-Compassion Scale (SCS) [32], a 26-item instrument using a five-point Likert scale (1 = almost never, 5 = almost always). It assesses 6 components of selfcompassion, with the following Cronbach’s alphas: Self-Kindness (a=.89), Self-Judgment (a=.88), Common Humanity (a=.84), Isolation (a=.84), Mindfulness (a=.84) and Over-Identification (a=.81).
Guilt and Shame were assessed using the 16-item Guilt and Shame Proneness Scale (GASP) [33]. GASP uses a 7-point Likert scale (1 = very unlikely, 7 = very likely) and evaluates the following factors: Guilt-Negative-Behavior-Evaluation (a=.70), Guilt-Repair (a=.51), Shame-Negative-Self-Evaluation (a=.73) and Shame-Withdraw (a=.46). Lower reliability is expected in scenario-based measures as each item contains unique variance for the given scenario (e.g., [33- 35]).
Intolerance of uncertainty was measured using the Intolerance of Uncertainty Scale (IUS) [36], a 12-item instrument using a five-point Likert (1 = not at all characteristic of me; 5 = entirely characteristic of me). The Cronbach’s alpha for this study was a = .91.
Usability
The level of engagement in the intervention was assessed as follows: for the experimental condition (Woebot), the total number of interactions (i.e., moods recorded) with the bot over the 2-week period was recorded, as detailed earlier in 2.1. Recruitment and Procedure. For those in the control condition, the number of correct responses to the questions was recorded.
Statistical approach
To ensure that no pre-existing differences between the experimental groups could bias the results of the trial, independent samples t-tests were conducted for demographic variables, as well as the baseline levels of the primary and secondary outcomes were assessed with independent samples t-tests (i.e., for age, depression, anxiety, rumination, intolerance of uncertainty, self-compassion, guilt and shame) and chi-square analyses (i.e., for gender, engagement with an ongoing therapeutic process and the presence of a medical diagnosis, as assessed by a single self-report item). Differences between completers and drop-outs in terms of demographic variables and pre-treatment scores were also assessed using independent samples t-tests and chi-square tests.
To assess the interventions’ efficacy in reducing depression and anxiety - the primary outcomes of interest - 2 x 2 Repeated Measures ANOVAs were conducted on completers only, with time (i.e., preintervention vs. post-intervention) as a within-subject variable and group (i.e., Woebot vs. psychoeducation) as a between-subjects variable. Furthermore, to examine the effects of the Woebot app on the secondary outcomes, 3 x 2 Repeated Measures ANOVAs were conducted on completers only, with time (i.e., pre-intervention vs. mid-intervention vs. post-intervention) as a within-subject variable and group (i.e., Woebot vs. psychoeducation) as a between-subjects variables. Missing data were handled with the multiple imputation procedure [37], with data assumed to be missing at random. Consequently, intent-to-treat analyses were performed using ANCOVA on the post-treatment pooled scores for the primary and secondary outcomes with group as a factor and pre-treatment scores as covariates. Statistical corrections for homogeneity of variance and sphericity were applied where appropriate.
Results
Baseline measures and demographics
Independent samples t-tests showed that there were no significant differences at baseline between the group randomized to the experimental condition (Woebot) and the control condition, as detailed in Table 1.
Table 1: Baseline characteristics.
Importantly, there was no significant difference in the number of participants dropping out of the experimental group vs. the control group (60 vs. 47), χ2 (2) = 3.35, p = .067) although a trend favoring retention in the control group was observed.
Analyses indicated no differences between completers and non-completers on any of the variables (for the primary outcomes, i.e., depression and anxiety, and for the secondary outcomes, i.e., rumination, intolerance of uncertainty, self-compassion), with the exception of gender, with males more likely to drop out (χ2 (1) = 10.59, p = .001.
Primary outcomes
Completers’ analyses revealed no significant interactions between group and time for the study’s primary outcomes (Table 2). Additionally, no main effects of group were found for any of the primary outcomes: (a) depression, F (1, 93) = .61, p = .435; (b) anxiety, F (1, 93) = .02, p = .874. Main effects of time were also nonsignificant in the case of depression, F (1, 93) = .84, p = .360, but were significant for anxiety, F (1, 93) = 4.70, p = .033 partial η2 = .05, revealing a decrease in time for both groups.
Group
Mean
SD
95% CI
F
P
Depression
Control
5.96
5.31
4.58-7.33
0.65
0.423
Experimental
4.88
4.76
3.30-6.45
Anxiety
Control
4.5
4.24
3.34-5.65
0.09
0.767
Experimental
4.24
4.3
2.92-5.56
Rumination
Control
79.31
15.52
75.05-83.57
1.42
0.245
Experimental
82.87
16.09
77.98-87.77
Intolerance of Uncertainty
Control
28.98
8.4
26.44-31.52
0.13
0.977
Experimental
30.2
10.56
27.28-33.10
Self-Compassion
Control
83.27
17.49
78.66-87.89
1.45
0.238
Experimental
85.78
16.49
80.48-91.07
Guilt and Shame
Control
37.65
8.13
35.30-39.99
1.03
0.344
Experimental
40.15
9.34
37.45-42.83
Table 2: Means and SDs at post-intervention and time X group interaction effects for primary and secondary outcomes in completers’ analyses.
Secondary outcomes
No significant group by time interactions were found for the transdiagnostic factors (Table 2). There were no main effects for: (c) rumination, F (1, 93) = .15, p = .696; (d) intolerance of uncertainty, F (1, 93) = .51, p = .473; (e) self-compassion, F (1, 93) = .42, p = .838; (f) guilt and shame, F (1, 93) = .82, p = .366. Main effects of time were significant for (c) rumination, F (1.77, 165), = 3.22, p = .048, partial η2 = .03; (d) intolerance of uncertainty, F (1.68, 156.50) = 10.22, p < .001, partial η2 = .10; (e) self-compassion, F (1.63, 151.56) = 7.41, p = .002, partial η2 = .07; (f) guilt and shame, F (1.60, 149.16) = 6.64, p = .004, partial η2 = .07. These main effects of time revealed an increase in rumination and guilt and shame, but also in self-compassion, as well as a decrease in intolerance of uncertainty across the three time points (see Table 2 for means and standard deviations at post-intervention for the primary and secondary outcomes).
Intent to Treat (ITT)
ITT analyses revealed no significant effects of group on the primary and secondary outcomes (Table 3). Pooled means and standard error of means for the control and experimental groups at post-treatment when controlling for pre-treatment scores are also presented in Table 3.
Group
Mean
SE
95% CI
F
P
Depression
Control
6.23
0.67
4.81-7.65
1.03
0.313
Experimental
5.76
0.84
4.00-7.53
Anxiety
Control
4.86
0.51
3.84-5.88
0.12
0.735
Experimental
4.56
0.54
3.47-5.66
Rumination
Control
78.89
1.59
75.73-82.05
2.17
0.144
Experimental
81.84
1.68
78.50-85.18
Intolerance of Uncertainty
Control
29.45
0.85
27.78-31.13
0.01
0.92
Experimental
29.06
0.85
27.39-30.73
Self-Compassion
Control
83.08
1.67
79.75-86.40
1.97
0.163
Experimental
84.45
1.9
80.60-88.30
Guilt and Shame
Control
39.28
1.08
37.13-41.14
1.63
0.205
Experimental
39.88
1.38
37.04-42.72
Table 3: Means and standard error of means at post-intervention and main effects of group for primary and secondary outcomes in ITT analyses.
Discussion
The current study examined the efficacy of a fully automated, AIpowered conversational agent (Woebot) in alleviating the symptoms of anxiety and depression in a non-clinical sample, compared to an active control who received psychoeducational materials. An additional aim of the study was to examine Woebot’s vs. the psychoeducational intervention’s effect on the transdiagnostic factors involved in the etiology and maintenance of depression and anxiety.
While both conditions showed changes over time, there were no significant differences in the primary outcomes (i.e., anxiety and depression,) between the two groups from pre-intervention to postintervention, suggesting a lack of added efficacy associated with the experimental condition.
As far as the transdiagnostic factors were concerned, there were no significant group-by-time interactions, as well as no significant effects of group, suggesting that the examined transdiagnostic factors are not mediators of treatment outcomes in this case. While there were no significant interactions, the results, however, did show a significant main effect of time on anxiety and transdiagnostic factors, suggesting that the experimental and the control condition were similarly beneficial. Across both groups, increases in rumination, self-compassion, guilt, and shame, as well as decreases in anxiety and intolerance of uncertainty were observed. As the participants in both groups appeared to show improvements in their anxiety levels as well as increases in self-compassion and decreases in intolerance of uncertainty, it is possible that both interventions contributed substantially towards these effects. However, this assertion is not fully supported by the increases in rumination, guilt, and shame. It is possible that, through psychoeducation, both interventions raised the participants’ awareness of their mental symptoms, thus potentially increasing their tendencies to ruminate and become ashamed of them. Moreover, since both the experimental and active control conditions were focused on reflections on oneself, and given that guilt is a self-conscious emotion [38], it is also possible that participants were primed for the intensification of these emotions, especially since the data were collected during the ongoing COVID-19 pandemic, which is associated with mental health consequences as yet not fully elucidated.
Our results also showed that there was a trend towards more dropouts in the experimental group. While non-significant, this difference could be linked to the fact that AI chatbots are still grappling with limitations in terms of content, with users reporting impersonal remarks, repetitions, misunderstandings, and lack of meaningful interactions [21], factors that can be associated negatively with retention.
To our knowledge, this is the first study to examine the efficacy of an AI-powered conversational agent compared to a non-passive control condition; however, no differential efficacy was found. This suggests that the psychoeducational components in both conditions were the only significant drivers of the improvement. This, in turn, suggests that simple methods, such as regular emails sent to people, may be enough to obtain the same therapeutic benefits as technically sophisticated solutions, which, in most cases, incur substantial development and maintenance costs, which are passed to the consumers.
Strengths and Limitations
The present study adds to the body of evidence regarding the efficacy of AI-powered conversational agents on emotional disorders and attempts to shed light on the effects of such an agent on the transdiagnostic factors associated with depression and anxiety.
Despite the intriguing results we obtained, the study also has several important limitations. First, it was not possible to track the time spent in-app engaging with Woebot with high precision, because our participants had different mobile operating systems, which made this impractical or impossible. Therefore, we relied solely on the check-in diagram completed by the participants to assess treatment adherence (at least one check-in daily), which approximated their app usage. Second, we conducted our study on a non-clinical sample, drawn from the general population. While previous studies on conversational agents also used non-clinical samples e.g., [22], future studies should try to examine these relationships on clinical samples, in order to draw more definite conclusions regarding the efficacy of such solutions in reducing symptoms of anxiety and depression. Third, an almost exclusive female sample was examined, which makes the generalization of these findings to males questionable.
Future studies should incorporate these limitations and, also, include other AI-powered conversational agents, to better understand their efficacy in relation to simple interventions such as regular psychoeducational materials delivered to the inboxes.
Comparison with Prior Work
In contrast to results from a previous study [22], using an AI-powered chatbot was not associated with a more significant reduction in self-reported symptoms of anxiety and depression than a control condition. Unlike previous studies, our research included an active control condition, to better evaluate the potential benefits of a conversational agent in reducing symptoms of depression and anxiety.
Conclusion
Our findings support and expand on previous studies e.g., [39], which found that the effects of web-based interventions are smaller when compared to active, as opposed to passive controls. In our study, we found that employing a simple form of active intervention (emails requiring an answer) is comparable in benefits to a fully automated, AI-driven chatbot. Further research is needed to better understand the areas where automated bots might have an edge over simpler, potentially more economical interventions.
References
- Abd-alrazaq AA, Alajlani M, Alalwan AA, Bewick BM, Gardner P, Househ M. An overview of the features of chatbots in mental health: A scoping review. International Journal of Medical Informatics. 2019; 132: 103978.
- Sander L, Rausch L, Baumeister H. Effectiveness of Internet-Based Interventions for the Prevention of Mental Disorders: A Systematic Review and Meta-Analysis. JMIR Mental Health. 2019; 3: e38.
- Duffy ME, Twenge JM, Joiner TE. Trends in Mood and Anxiety Symptoms and Suicide-Related Outcomes Among U.S. Undergraduates, 2007-2018: Evidence From Two National Surveys. Journal of Adolescent Health. 2019; 65: 590-598.
- Hunt J, Eisenberg D. Mental Health Problems and Help-Seeking Behavior Among College Students. Journal of Adolescent Health. 2010; 46: 3-10.
- Lipson SK, Lattie EG, Eisenberg D. Increased Rates of Mental Health Service Utilization by U.S. College Students: 10-Year Population-Level Trends (2007- 2017). Psychiatric Services. 2018; 70: 60-63.
- Zivin K, Eisenberg D, Gollust SE, Golberstein E. Persistence of mental health problems and needs in a college student population. Journal of Affective Disorders. 2009; 117: 180-185.
- Auerbach RP, Alonso J, Axinn WG, Cuijpers P, Ebert DD, Green JG, et al. Mental disorders among college students in the WHO World Mental Health Surveys. Psychological Medicine. 20169; 46: 2955-2970.
- de Girolamo G, Dagani J, Purcell R, Cocchi A, McGorry PD. Age of onset of mental disorders and use of mental health services: Needs, opportunities and obstacles. Epidemiology and Psychiatric Sciences; Verona. 2012; 21: 47-57.
- Ibrahim AK, Kelly SJ, Adams CE, Glazebrook C. A systematic review of studies of depression prevalence in university students. Journal of Psychiatric Research. 2013; 47: 391-400.
- Cuijpers P. Minor depression: Risk profiles, functional disability, health care use and risk of developing major depression. Journal of Affective Disorders. 2004; 79: 71-79.
- Fergusson DM, Horwood LJ, Ridder EM, Beautrais AL. Subthreshold Depression in Adolescence and Mental Health Outcomes in Adulthood. Archives of General Psychiatry. 2005; 62: 66-72.
- Demyttenaere K, Bruffaerts R, Posada-Villa J, Gasquet I, Kovess V, Lepine JP, et al. Prevalence, severity, and unmet need for treatment of mental disorders in the World Health Organization World Mental Health Surveys. JAMA. 2004; 291: 2581-2590.
- McCance-Katz EF. The National Survey on Drug Use and Health: 2018. 2019; 58.
- Clement S, Schauman O, Graham T, Maggioni F, Evans-Lacko S, Bezborodovs N, et al. What is the impact of mental health-related stigma on help-seeking? A systematic review of quantitative and qualitative studies. Psychological Medicine. 2015; 45: 11-27.
- Gulliver A, Griffiths KM, Christensen H. Perceived barriers and facilitators to mental health help-seeking in young people: A systematic review. BMC Psychiatry. 2010; 10: 113.
- Mayor S. NICE advocates computerised CBT. BMJ : British Medical Journal. 2016; 332: 504.
- Bendig E, Erb B, Schulze-Thuesing L, Baumeister H. The Next Generation: Chatbots in Clinical Psychology and Psychotherapy to Foster Mental Health. A Scoping Review. Verhaltenstherapie. 2019; 1-13.
- Christensen H, Griffiths KM, Farrer L. Adherence in Internet Interventions for Anxiety and Depression: Systematic Review. Journal of Medical Internet Research. 2009; 11: e13.
- Donker T, Petrie K, Proudfoot J, Clarke J, Birch M-R, Christensen H. Smartphones for Smarter Delivery of Mental Health Programs: A Systematic Review. Journal of Medical Internet Research. 2013; 15: e247.
- Ly KH, Ly A-M, Andersson G. A fully automated conversational agent for promoting mental well-being: A pilot RCT using mixed methods. Internet Interventions. 2017; 10: 39-46.
- Fulmer R, Joerin A, Gentile B, Lakerink L, Rauws M. Using Psychological Artificial Intelligence (Tess) to Relieve Symptoms of Depression and Anxiety: Randomized Controlled Trial. JMIR Mental Health. 2018; 5: e64.
- Fitzpatrick KK, Darcy A, Vierhile M. Delivering Cognitive Behavior Therapy to Young Adults with Symptoms of Depression and Anxiety Using a Fully Automated Conversational Agent (Woebot): A Randomized Controlled Trial. JMIR Mental Health. 2019; 4: e19.
- Rodriguez-Seijas C, Eaton NR, Krueger RF. How Transdiagnostic Factors of Personality and Psychopathology Can Inform Clinical Assessment and Intervention. Journal of Personality Assessment. 2015; 97: 425-435.
- Krueger RF, Eaton NR. Transdiagnostic factors of mental disorders. World Psychiatry. 2015; 14: 27-29.
- Hirschfeld RMA. The Comorbidity of Major Depression and Anxiety Disorders: Recognition and Management in Primary Care. Primary Care Companion to the Journal of Clinical Psychiatry. 2001; 3: 244-254.
- Benda J, Korinek D, Kadlecik P, Loskotova M, Vyhnanek A, Zitkova T. Self-compassion and shame-proneness in four different mental disorders: Comparison with healthy controls. Ceskoslovenska Psychologie. 2018; 62: 529.
- Faul F, Erdfelder E, Lang A-G, Buchner A. G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods. 2007; 39: 175-191.
- Mental Health Chatbot|Woebot. 2020.
- Centre for Clinical Interventions. Depression Self-Help Resources. Information Sheets & Workbooks. 2018.
- Lovibond PF, Lovibond SH. The structure of negative emotional states: Comparison of the Depression Anxiety Stress Scales (DASS) with the Beck Depression and Anxiety Inventories. Behaviour Research and Therapy. 1995; 33: 335-343.
- Treynor W, Gonzalez R, Nolen-Hoeksema S. Rumination Reconsidered: A Psychometric Analysis. Cognitive Therapy and Research. 2003; 27: 247-259.
- NEFF KD. The Development and Validation of a Scale to Measure Self- Compassion. Self and Identity. 2003; 2: 223-250.
- Cohen TR, Wolf ST, Panter AT, Insko CA. Introducing the GASP scale: A new measure of guilt and shame proneness. Journal of Personality and Social Psychology. 2011; 100: 947-966.
- John OP, Benet-Martínez V. Measurement: Reliability, construct validation, and scale construction. In Handbook of research methods in social and personality psychology, 2nd ed. Cambridge University Press. 2014; 473-503.
- Schmitt N. Uses and abuses of coefficient alpha. Psychological Assessment. 1996; 8: 350-353.
- Carleton RN, Norton MAPJ, Asmundson GJG. Fearing the unknown: A short version of the Intolerance of Uncertainty Scale. Journal of Anxiety Disorders. 2007; 21: 105-117.
- Little RJA, Rubin DB. Statistical Analysis with Missing Data. John Wiley & Sons. 2019.
- Lewis M. Self-conscious emotions: Embarrassment, pride, shame, and guilt. In Handbook of emotions, 3rd ed. The Guilford Press. 2008; 742-756.
- Firth J, Torous J, Carney R, Newb, J, Cosco TD, Christensen H, et al. Digital Technologies in the Treatment of Anxiety: Recent Innovations and Future Directions. Current Psychiatry Reports. 2018; 20: 44.
Download PDF