Assessing mental health apps marketplaces with objective metrics from 29,190 data points from 278 apps.
Lagan S; Department of Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA.
D'Mello R; Department of Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA.
Vaidyam A; Department of Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA.
Bilden R; Department of Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA.
Torous J; Department of Psychiatry, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA.
Acta psychiatrica Scandinavica [Acta Psychiatr Scand] 2021 Aug; Vol. 144 (2), pp. 201-210. Date of Electronic Publication: 2021 Apr 29.
Typ publikacji :
Journal Article; Research Support, Non-U.S. Gov't; Review
Imprint Name(s) :
Publication: Malden, MA : Wiley-Blackwell
Original Publication: Copenhagen : Munksgaard International Publishers
MeSH Terms :
Benchmarking ; Humans
Holmes EA, O'Connor RC, Perry VH, et al. Multidisciplinary research priorities for the COVID-19 pandemic: a call for action for mental health science. Lancet Psychiatry. 2020;7(6):547-560. https://doi.org/10.1016/S2215-0366(20)30168-1.
Czeisler MÉ, Lane RI, Petrosky E, et al. Mental health, substance use, and suicidal ideation during the COVID-19 pandemic-United States, June 24-30, 2020. MMWR. Morbidity Mortality Weekly Report. 2020;69(32):1049-1057. https://doi.org/10.15585/mmwr.mm6932a1.
Approaching 1,000 Mental Health Startups in 2020. https://whatif.vc/blog/approaching-1000-mental-health-startups-in2020. Accessed September 29, 2020.
Don M, Trina H, Estee N, et al. How kaiser permanente created a mental health and wellness digital ecosystem. NEJM Catalyst. 2021;2(1): https://doi.org/10.1056/CAT.20.0295.
Melcher J, Torous J. Smartphone apps for college mental health: a concern for privacy and quality of current offerings. Psychiatr Serv. 2020;71(11):1114-1119. https://doi.org/10.1176/appi.ps.202000098.
Martinengo L, Van Galen L, Lum E, Kowalski M, Subramaniam M, Car J. Suicide prevention and depression apps’ suicide risk assessment and management: a systematic assessment of adherence to clinical guidelines. BMC Med. 2019;17(1):231. https://doi.org/10.1186/s12916-019-1461-z.
Huckvale K, Torous J, Larsen ME. Assessment of the data sharing and privacy practices of smartphone apps for depression and smoking cessation. JAMA Netw Open. 2019;2(4):e192542. https://doi.org/10.1001/jamanetworkopen.2019.2542.
Understanding Mobile Apps. Federal Trade Commission Consumer Information. Published February 21, 2017. https://www.consumer.ftc.gov/articles/0018-understanding-mobile-apps. Accessed March 22, 2021.
Alqahtani F, Orji R. Insights from user reviews to improve mental health apps. Health Informatics J. 2020;26(3):2042-2066. https://doi.org/10.1177/1460458219896492.
Larsen ME, Nicholas J, Christensen H. Quantifying app store dynamics: longitudinal tracking of mental health apps. JMIR MHealth UHealth. 2016;4(3):e96. https://doi.org/10.2196/mhealth.6020.
Carlo AD, Hosseini Ghomi R, Renn BN, Areán PA. By the numbers: ratings and utilization of behavioral health mobile applications. Npj Digit Med. 2019;2(1):54. https://doi.org/10.1038/s41746-019-0129-6.
Soklaridis S, Lin E, Lalani Y, Rodak T, Sockalingam S. Mental health interventions and supports during COVID- 19 and other medical pandemics: A rapid systematic review of the evidence. Gen Hosp Psychiatry. 2020;66:133-146. https://doi.org/10.1016/j.genhosppsych.2020.08.007.
Marshall JM, Dunstan DA, Bartik W. Apps with maps-anxiety and depression mobile apps with evidence-based frameworks: systematic search of major app stores. JMIR Ment Health. 2020;7(6):e16525. https://doi.org/10.2196/16525.
Bergin A, Davies EB. Technology Matters: Mental health apps - separating the wheat from the chaff. Child Adolesc Ment Health. 2020;25(1):51-53. https://doi.org/10.1111/camh.12363.
Boudreaux ED, Waring ME, Hayes RB, Sadasivam RS, Mullen S, Pagoto S. Evaluating and selecting mobile health apps: strategies for healthcare providers and healthcare organizations. Transl Behav Med. 2014;4(4):363-371. https://doi.org/10.1007/s13142-014-0293-9.
Levine DM, Co Z, Newmark LP, et al. Design and testing of a mobile health application rating tool. Npj Digit Med. 2020;3(1):1-7. https://doi.org/10.1038/s41746-020-0268-9.
Leigh S, Ouyang J, Mimnagh C. Effective? Engaging? Secure? Applying the ORCHA-24 framework to evaluate apps for chronic insomnia disorder. Evid Based Ment Health. 2017;20(4):e20. https://doi.org/10.1136/eb-2017-102751.
Lagan S, Aquino P, Emerson MR, Fortuna K, Walker R, Torous J. Actionable health app evaluation: translating expert frameworks into objective metrics. Npj Digit Med. 2020;3(1):1-8. https://doi.org/10.1038/s41746-020-00312-4.
Lagan S, Camacho E, Torous J. Is there a clinically relevant, publicly accessible app for that? Exploring the clinical relevance and availability of mobile apps for schizophrenia and psychosis. Schizophrenia Res. 2020;https://doi.org/10.1016/j.schres.2020.11.007.
Lagan S, Ramakrishnan A, Lamont E, Ramakrishnan A, Frye M, Torous J. Digital health developments and drawbacks: a review and analysis of top-returned apps for bipolar disorder. Int J Bipolar Disord. 2020;8(1):39. https://doi.org/10.1186/s40345-020-00202-4.
Muñoz AO, Camacho E, Torous J. Marketplace and literature review of spanish language mental health apps. Front Digit Health. 2021;3: https://doi.org/10.3389/fdgth.2021.615366.
Henson P, David G, Albright K, Torous J. Deriving a practical framework for the evaluation of health apps. Lancet Digit Health. 2019;1(2):e52-e54. https://doi.org/10.1016/S2589-7500(19)30013-5.
Fortuna KL, DiMilia PR, Lohman MC, et al. Feasibility, acceptability, and preliminary effectiveness of a peer-delivered and technology supported self-management intervention for older adults with serious mental illness. Psychiatr Q. 2018;89(2):293-305. https://doi.org/10.1007/s11126-017-9534-7.
Naslund JA, Aschbrenner KA, Marsch LA, Bartels SJ. The future of mental health care: peer-to-peer support and social media. Epidemiol Psychiatr Sci. 2016;25(2):113-122. https://doi.org/10.1017/S2045796015001067.
Larsen M, Nicholas J, Han J, et al. Here’s something I prepared earlier: a review of the time to publication of cross-sectional reviews of smartphone health apps. BMJ Open. 2020;10(12):e039817. https://doi.org/10.1136/bmjopen-2020-039817.
BIDMC - Division of Digital Psychiatry. https://apps.digitalpsych.org/. Accessed December 9, 2020.
Baumel A, Faber K, Mathur N, Kane JM, Muench F. Enlight: a comprehensive quality and therapeutic potential evaluation tool for mobile and web-based eHealth interventions. J Med Internet Res. 2017;19(3):e82. https://doi.org/10.2196/jmir.7270.
Wasil AR, Gillespie S, Shingleton R, Wilks CR, Weisz JR. Examining the reach of smartphone apps for depression and anxiety. Am J Psychiatry. 2020;177(5):464-465. https://doi.org/10.1176/appi.ajp.2019.19090905.
Wasil AR, Gillespie S, Schell T, Lorenzo-Luaces L, DeRubeis RJ. Estimating the real-world usage of mobile apps for mental health: development and application of two novel metrics. World Psychiatry. 2021;20(1):137-138. https://doi.org/10.1002/wps.20827.
Wasil AR, Venturo-Conerly KE, Shingleton RM, Weisz JR. A review of popular smartphone apps for depression and anxiety: Assessing the inclusion of evidence-based content. Behav Res Ther. 2019;123:103498. https://doi.org/10.1016/j.brat.2019.103498.
Torous J, Vaidyam A. Multiple uses of app instead of using multiple apps - a case for rethinking the digital health technology toolbox. Epidemiol Psychiatr Sci. 2020;29: https://doi.org/10.1017/S2045796020000013.
Wasil AR, Gillespie S, Patel R, et al. Reassessing evidence-based content in popular smartphone apps for depression and anxiety: Developing and applying user-adjusted analyses. J Consult Clin Psychol. 2020;88(11):983-993. https://doi.org/10.1037/ccp0000604.
Gordon WJ, Patel V, Thornhill W, Bates DW, Landman A. Characteristics of patients using patient-facing application programming interface technology at a US Health Care System. JAMA Netw Open. 2020;3(10):e2022408. https://doi.org/10.1001/jamanetworkopen.2020.22408.
App Privacy Details - App Store. Apple Developer. https://developer.apple.com/app-store/app-privacy-details/. Accessed December 9, 2020.
Fowler G. How to read an iPhone app privacy label - The Washington Post. Published January 29, 2021. https://www.washingtonpost.com/technology/2021/01/29/how-to-read-iphone-privacy-labels/. Accessed March 27, 2021.
Yellowlees P. The need to develop specialized formularies of apps and web-based tools. Acad Psychiatry. 2018;42(5):677-678. https://doi.org/10.1007/s40596-018-0971-x.
Gordon WJ, Landman A, Zhang H, Bates DW. Beyond validation: getting health apps into clinical practice. Npj Digit Med. 2020;3(1):1-6. https://doi.org/10.1038/s41746-019-0212-z.
Conway JR, Lex A, Gehlenborg N. UpSetR: an R package for the visualization of intersecting sets and their properties. Bioinformatics. 2017;33(18):2938-2940. https://doi.org/10.1093/bioinformatics/btx364.
Contributed Indexing :
Keywords: apps*; digital health*; internet therapy*
Entry Date(s) :
Date Created: 20210409 Date Completed: 20210910 Latest Revision: 20210910
Update Code :
Objective: Utilizing a standard framework that may help clinicians and patients to identify relevant mental health apps, we sought to gain a comprehensive picture of the space by searching for, downloading, and reviewing 278 mental health apps from both the iOS and Android stores.
Methods: 278 mental health apps from the Apple iOS store and Google Play store were downloaded and reviewed in a standardized manner by trained app raters using a validated framework. Apps were evaluated with this framework comprising 105 questions and covering app origin and accessibility, privacy and security, inputs and outputs, clinical foundation, features and engagement style, and interoperability.
Results: Our results confirm that app stars and downloads-even for the most popular apps by these metrics-did not correlate with more clinically relevant metrics related to privacy/security, effectiveness, and engagement. Most mental health apps offer similar functionality, with 16.5% offering both mood tracking and journaling and 7% offering psychoeducation, deep breathing, mindfulness, journaling, and mood tracking. Only 36.4% of apps were updated with a 100-day window, and 7.5% of apps had not been updated in four years.
Conclusion: Current app marketplace metrics commonly used to evaluate apps do not offer an accurate representation of individual apps or a comprehensive overview of the entire space. The majority of apps overlap in terms of features offered, with many domains and other features not well represented. Selecting an appropriate app continues to require personal matching given no clear trends or guidance offered by quantitative metrics alone.
(© 2021 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd.)