{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,14]],"date-time":"2026-10-14T08:55:16Z","timestamp":1791968116035,"version":"4.3.4"},"update-to":[{"DOI":"10.3310\/nihropenres.13791.1","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2025,7,16]],"date-time":"2025-07-16T00:00:00Z","timestamp":1752624000000}}],"reference-count":26,"publisher":"National Institute for Health and Care Research","license":[{"start":{"date-parts":[[2025,7,16]],"date-time":"2025-07-16T00:00:00Z","timestamp":1752624000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"The Health Foundation","award":["NotKnown"],"award-info":[{"award-number":["NotKnown"]}]},{"name":"NIHR Applied Research Centre West Midlands","award":["NIHR200165"],"award-info":[{"award-number":["NIHR200165"]}]}],"content-domain":{"domain":["openresearch.nihr.ac.uk"],"crossmark-restriction":false},"short-container-title":["NIHR Open Res"],"abstract":"<ns3:p>Background It is easy to overlook risk factors that require specific healthcare actions. This is particularly true in maternity care, which deals with a natural process where risk might be distinguished from normality at many points in the care pathway. In this paper, we describe the effects of clinical decision support, first in the form of paper checklists and then in the form of an electronic checklist to screen for risks of (1) venous thromboembolism (VTE), (2) intrauterine growth restriction (IUGR), (3) high body mass index (BMI), and (4) gestational diabetes mellitus (GDM). Here, we report a retrospective observational study on the effects of screening algorithms introduced first on paper and then on the computer. Methods We screened sequential maternity records at three time points: baseline, following the introduction of a paper checklist, and following the introduction of the electronic system. First, we examined (at each time-point) the proportion of pregnancies appropriately screened at each time point. Second, we examined the proportion of correct actions taken following a positive screening result. The study was conducted at a District General Hospital in Scotland between 2011 and 2015, which covered the introduction of the above system to screen patients and suggest appropriate management for positive cases. Results We found that the introduction of a paper checklist was associated with an increased proportion of pregnancies appropriately screened and correct actions taken contingent on positive screening. These trends continued after the introduction of the electronic system. For example, when investigating those at risk of venous thrombosis and embolism (VTE), 0\/200 risk assessments were made correctly in 2011 with no formal prompts. This increased to 69\/200 (0.345) in 2014 when using paper checklists, and this trend continued in 2015 when we found 192\/200 (0.965) risk assessments were made correctly for antenatal and intrapartum VTE using the electronic checklists. In 2011, we observed that the number of actions contingent on positive screening for VTE completed correctly was 0\/1. Increasing to 17\/24 (0.708) in 2014 when using paper checklists but plateauing in 2015 when 20\/33 (0.667) contingent actions were completed correctly using electronic system prompts. Conclusions Compliance with maternity guideline recommendations for VTE, high BMI, high risk of fetal growth restriction, and GDM improved over time with the introduction of paper and electronic prompts. Tweetable abstract Introduction of electronic maternity screening was associated with improved adherence to established guidelines compared to no screening or paper-based screening for VTE, BMI and GDM. Data Sharing Statement Nil additional unpublished data from the study are available.<\/ns3:p>","DOI":"10.3310\/nihropenres.13791.2","type":"journal-article","created":{"date-parts":[[2025,7,16]],"date-time":"2025-07-16T12:55:06Z","timestamp":1752670506000},"page":"37","update-policy":"https:\/\/doi.org\/10.3310\/nihropenres.crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["The Safe Assessment Form to Evaluate Risks (\u2018SAFER\u2019) chart \u2013 a clinical practice evaluation study following introduction of electronic risk identification in pregnancies in Scotland"],"prefix":"10.3310","volume":"5","author":[{"given":"Alex","family":"Viner","sequence":"first","affiliation":[{"name":"Glasgow Scotland Queen Elizabeth University Hospital"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Oscar","family":"Deeks","sequence":"additional","affiliation":[{"name":"School of Health Sciences Birmingham England University of Birmingham"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3572-7886","authenticated-orcid":false,"given":"Pamela","family":"Nayyar","sequence":"additional","affiliation":[{"name":"School of Health Sciences Birmingham England University of Birmingham"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jennifer","family":"Allison","sequence":"additional","affiliation":[{"name":"Kirkcaldy Scotland Victoria Hospital"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sarah","family":"Murray","sequence":"additional","affiliation":[{"name":"Centre for Reproductive Health Edinburgh Scotland The University of Edinburgh"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Katherine","family":"Ainslie","sequence":"additional","affiliation":[{"name":"Melrose Scotland NHS Borders"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Neil","family":"Cockburn","sequence":"additional","affiliation":[{"name":"School of Health Sciences Birmingham England University of Birmingham"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0634-984X","authenticated-orcid":false,"given":"Richard","family":"Lilford","sequence":"additional","affiliation":[{"name":"School of Health Sciences Birmingham England University of Birmingham"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Brian","family":"Magowan","sequence":"additional","affiliation":[{"name":"Melrose Scotland NHS Borders"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"6221","published-online":{"date-parts":[[2025,7,16]]},"reference":[{"key":"ref-1","doi-asserted-by":"publisher","first-page":"1351-5","DOI":"10.1056\/NEJM197612092952405","article-title":"Protocol-based computer reminders, the quality of care and the non-perfectability of man.","volume":"295","author":"C McDonald","year":"1976","journal-title":"N Engl J Med."},{"key":"ref-2","doi-asserted-by":"publisher","DOI":"10.1016\/j.imu.2023.101183","article-title":"The effect of digital antimicrobial stewardship programmes on antimicrobial usage, length of stay, mortality and cost.","volume":"37","author":"N Trotter","year":"2023","journal-title":"Inform Med Unlocked."},{"key":"ref-3","doi-asserted-by":"publisher","first-page":"1968-1978","DOI":"10.1016\/j.socscimed.2012.01.041","article-title":"Decision tools to support women\u2019s decision making in pregnancy and birth: a systematic review and meta-analysis.","volume":"74","author":"M Dugas","year":"2012","journal-title":"Soc Sci Med."},{"key":"ref-4","doi-asserted-by":"publisher","first-page":"1181-4","DOI":"10.1136\/bmj.305.6863.1181","article-title":"Effect of using protocols on medical care: randomised trial of three methods of taking an antenatal history.","volume":"305","author":"R Lilford","year":"1992","journal-title":"BMJ."},{"key":"ref-5","article-title":"Reducing the risk of venous thromboembolism during pregnancy and the puerperium","year":"2015"},{"key":"ref-6","article-title":"Saving Lives, Improving Mothers\u2019 Care. Lessons learned to inform maternity care from the UK and Ireland confidential enquiries into maternal deaths and morbidity 2013\u201315","author":"M Knight","year":"2017"},{"key":"ref-7","doi-asserted-by":"publisher","DOI":"10.1136\/bmjopen-2013-003942","article-title":"Association between stillbirth rates in England and uptake of accreditation training in customized fetal growth assessment.","volume":"3","author":"J Gardosi","year":"2013","journal-title":"BMJ Open."},{"key":"ref-8","article-title":"Small-for-gestational-age fetus, investigation and management","year":"2015"},{"key":"ref-9","article-title":"Management of women with obesity in pregnancy","year":"2010"},{"key":"ref-10","article-title":"Management of diabetes: a national clinical guideline","year":"2010"},{"key":"ref-11","article-title":"Diagnosis and treatment of gestational diabetes","year":"2011"},{"key":"ref-12","doi-asserted-by":"publisher","first-page":"2477-2486","DOI":"10.1056\/NEJMoa042973","article-title":"Effect of treatment of gestational diabetes mellitus on pregnancy outcomes.","volume":"352","author":"C Crowther","year":"2005","journal-title":"N Engl J Med."},{"key":"ref-13","doi-asserted-by":"publisher","first-page":"1339-1348","DOI":"10.1056\/NEJMoa0902430","article-title":"A multicentre, randomized trial of treatment for mild gestational diabetes.","volume":"361","author":"M Landon","year":"2009","journal-title":"N Engl J Med."},{"key":"ref-14","doi-asserted-by":"publisher","first-page":"163-169","DOI":"10.1136\/qshc.2007.023648","article-title":"An epistemology of patient safety research: a framework for study design and interpretation. Part 2. study design.","volume":"17","author":"C Brown","year":"2008","journal-title":"Qual Saf Health Care."},{"key":"ref-15","doi-asserted-by":"publisher","first-page":"303-10","DOI":"10.1136\/bmjqs-2015-004372","article-title":"Secular trends and evaluation of complex interventions: the rising tide phenomenon.","volume":"25","author":"Y Chen","year":"2015","journal-title":"BMJ Qual Saf."},{"key":"ref-16","doi-asserted-by":"publisher","first-page":"c4413","DOI":"10.1136\/bmj.c4413","article-title":"Evaluating policy and service interventions: a framework to guide selection and interpretation of study end points.","volume":"341","author":"R Lilford","year":"2010","journal-title":"BMJ."},{"key":"ref-17","doi-asserted-by":"publisher","DOI":"10.1016\/j.eclinm.2024.102822","article-title":"Clinical decision support systems for maternity care: a systematic review and meta-analysis.","volume":"76","author":"N Cockburn","year":"2024","journal-title":"EClinicalMedicine."},{"key":"ref-18","doi-asserted-by":"publisher","first-page":"25-36","DOI":"10.4338\/ACI-2012-10-RA-0044","article-title":"An e-health decision support system for improving compliance of health workers to the maternity care protocols in South Africa.","volume":"4","author":"V Horner","year":"2013","journal-title":"Appl Clin Inform."},{"key":"ref-19","doi-asserted-by":"publisher","first-page":"e126-e36","DOI":"10.1016\/S2589-7500(21)00269-7","article-title":"A digital health registry with clinical decision support for improving quality of antenatal care in Palestine (eRegQual): a pragmatic, cluster-randomised, controlled, superiority trial.","volume":"4","author":"M Venkateswaran","year":"2022","journal-title":"Lancet Digit Health."},{"key":"ref-20","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0123940","article-title":"Assessment of the quality of antenatal care services provided by health workers using a mobile phone decision support application in Northern Nigeria: a pre\/post-intervention study.","volume":"10","author":"M McNabb","year":"2015","journal-title":"PLoS One."},{"key":"ref-21","doi-asserted-by":"publisher","DOI":"10.1136\/bmjopen-2023-074770","article-title":"Use of a digital job-aid in improving antenatal clinical protocols and quality of care in rural primary-level health facilities in Burkina Faso: a quasi-experimental evaluation.","volume":"13","author":"A Ma\u00efga","year":"2023","journal-title":"BMJ Open."},{"key":"ref-22","doi-asserted-by":"publisher","first-page":"536-543","DOI":"10.1177\/1357633X16653540","article-title":"Usability and feasibility of a mobile health system to provide comprehensive Antenatal Care in low-income countries: PANDA mHealth pilot study in Madagascar.","volume":"23","author":"A Benski","year":"2017","journal-title":"J Telemed Telecare."},{"key":"ref-23","doi-asserted-by":"publisher","DOI":"10.1186\/s12913-017-2457-z","article-title":"Cost-effectiveness of an electronic clinical decision support system for improving quality of antenatal and childbirth care in rural Tanzania: an intervention study.","volume":"17","author":"H Saronga","year":"2017","journal-title":"BMC Health Serv Res."},{"key":"ref-24","doi-asserted-by":"publisher","DOI":"10.1136\/bmjquality.u212405.w5122","article-title":"Venous thromboembolism capture on electronic systems in obstetrics patients at St Thomas\u2019 Hospital.","volume":"5","author":"A Ahmad","year":"2016","journal-title":"BMJ Qual Improv Rep."},{"key":"ref-25","first-page":"1-3","article-title":"The information paradox at the heart of non-directive counselling.","volume":"6","author":"R Lilford","year":"2024","journal-title":"NIHR ARC West Midlands & NIHR Midlands PSRC News Blog."},{"key":"ref-26","doi-asserted-by":"publisher","first-page":"702-4","DOI":"10.1001\/2013.jamainternmed.61","article-title":"Information overload and missed test results in electronic health record-based settings.","volume":"173","author":"H Singh","year":"2013","journal-title":"JAMA Intern Med."}],"container-title":["NIHR Open Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/openresearch.nihr.ac.uk\/articles\/5-37\/v2\/xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/openresearch.nihr.ac.uk\/articles\/5-37\/v2\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/openresearch.nihr.ac.uk\/articles\/5-37\/v2\/iparadigms","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,14]],"date-time":"2026-10-14T08:39:15Z","timestamp":1791967155000},"score":1,"resource":{"primary":{"URL":"https:\/\/openresearch.nihr.ac.uk\/articles\/5-37\/v2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7,16]]},"references-count":26,"URL":"https:\/\/doi.org\/10.3310\/nihropenres.13791.2","relation":{"has-version":[{"id-type":"doi","id":"10.3310\/nihropenres.13791.1","asserted-by":"subject"}],"has-review":[{"id-type":"doi","id":"10.3310\/nihropenres.15256.r37070","asserted-by":"subject"},{"id-type":"doi","id":"10.3310\/nihropenres.15256.r36973","asserted-by":"subject"},{"id-type":"doi","id":"10.3310\/nihropenres.15256.r36973","asserted-by":"object"},{"id-type":"doi","id":"10.3310\/nihropenres.15256.r37070","asserted-by":"object"}]},"ISSN":["2633-4402"],"issn-type":[{"value":"2633-4402","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,7,16]]},"assertion":[{"value":"Indexed","URL":"https:\/\/openresearch.nihr.ac.uk\/articles\/5-37\/v2#article-reports","order":0,"name":"referee-status","label":"Referee status","group":{"name":"current-referee-status","label":"Current Referee Status"}},{"value":"10.3310\/nihropenres.14980.r35401, Steven Ondersma, Charles Stewart Mott Department of Public Health, Michigan State University, Flint, Michigan, 24 May 2025, version 1, 2 approved with reservations","URL":"https:\/\/openresearch.nihr.ac.uk\/articles\/5-37\/v1#referee-response-35401","order":0,"name":"referee-response-35401","label":"Referee Report","group":{"name":"article-reports","label":"Article Reports"}},{"value":"<b>Richard Lilford<\/b>; \n<i>Posted: 03 Jul 2025<\/i>; We thank the reviewer for their attentive reading of our paper and for insightful suggestions.  We may have missed other literature. Our search would have included the quality literature covering antenatal care. It was done independently of a recent literature review cited in our article (Cockburn et al. 2024) and our search included all articles on prompts\/action suggestions cited in this paper. Sampling at the end of calendar years versus continuous observation. Doing so would have introduced (additional) sources of bias. Had we done so, we would have needed an adjustment period for the clinicians to get used to each new prompt system as it was introduced. More observations would have been required. Role of reviewer judgement. This relates to the &quot; observations\u201d subheading under methods. As stated, we retrieved the case notes. The case notes included the paper checklists that we introduced in the earlier epoch and the electronic print outs from the later epoch. We now make that clearer. We then go on to specify that the checklists and electronic printouts were scrutinised to determine whether the data was collected. Then the case notes themselves were examined to see whether the contingent actions were followed. We have rewritten this part to make this clearer. Thank you. We then add \u201cThus the data collected were based on clear boxes on the paper or electronic checklist, while the contingent actions were extracted from the case notes.\u201d &nbsp;Then in the \u2018limitations\u2019 section of the discussion we add \u201cThe elicitation of data collected by clinical staff was based on clear check boxes. However, the contingent actions had to be ascertained from case notes. While this exercise is not complex, we cannot exclude the possibility that some of the actions were overlooked (leading to underestimation of effects) and we did not have resources to duplicate observations. However, the finding of ceiling effects for some observations suggests that few, if any, of the contingent actions were missed\u201d. Thank you for this observation. Case notes selection. Again, thank you. We have clarified that they were consecutive. Selection of prompts. We now add \u201cThe clinical service decided to implement four sets of standards. They were selected because they all had national endorsement (see below). We therefore examined these standards.\u201d Paper checklists were \u201c\n<b>replaced<\/b>\u201d by electronic checklists. Thank you - we should have been clearer. High rate of adherence explained by increased use of required fields: evidence- based prompts on EHRs. We had made it clear that the systems described here were forerunners of electronic notes - first paragraph of \u2018relevance\u2019 section in the discussion. We had also discussed a more general trend under \u2018causal inference.\u2019 We have added sample size to the abstract. We have rewritten the \u2018Results\u2019 section to include the statistical findings. We agree that the tweetable abstract was too positive and have modified it.","URL":"https:\/\/openresearch.nihr.ac.uk\/articles\/5-37\/v2#referee-comment-3457","order":1,"name":"referee-comment-3457","label":"Referee Comment","group":{"name":"article-reports","label":"Article Reports"}},{"value":"10.3310\/nihropenres.14980.r35641, Mxolisi  W Ngwenya, University of Limpopo, Mankweng, Limpopo, 09 Jun 2025, version 1, 2 approved with reservations","URL":"https:\/\/openresearch.nihr.ac.uk\/articles\/5-37\/v1#referee-response-35641","order":2,"name":"referee-response-35641","label":"Referee Report","group":{"name":"article-reports","label":"Article Reports"}},{"value":"<b>Richard Lilford<\/b>; \n<i>Posted: 03 Jul 2025<\/i>; We thank the reviewer for their thorough consideration of our paper.  \n<u>Abstract<\/u> We have highlighted the methodology in the abstract. To remain within the word limit, we have mentioned the research the gap and the relationship between taking action and outcome in the Introduction.  \n<u>Introduction<\/u>  Further detail on decision tools and relation to outcomes added. Potential advantages of digital over paper checklists and prompts added.   \n<u>Methods<\/u>  Design added. We explicate the design by adding that the consecutive sampling was sequential from on a pre-defined date. We add that the checklist was implemented \u2018verbatim\u2019 form the published checklist with no adaptation, to the contingent action recommended by the Boolean (if\/then) algorithm (as laid out in Figure 2). We did not include hypertension because this would have required a more complex algorithm on which there is less consensus. We have not altered the text because hypertension is one of many potential triggers for a prompt. We have not produced exclusion criteria since there was only one already listed as our inclusion criteria. We had stated that we sampled at the end of each calendar year and now add \u201cNovember and December\u201d. We now state \u201cbooking clinic\u201d \u2013 which also answers the next question.   \n<u>Results<\/u>  Since the cases were consecutive, they represent the users of the service. Our focus was clinical behaviour in terms of adherence to checklist and guidelines for contingent action.   \n<u>Discussion and Conclusion<\/u> We now specify the relationship between observed change in process (clinical behaviour) and outcome.","URL":"https:\/\/openresearch.nihr.ac.uk\/articles\/5-37\/v2#referee-comment-3458","order":3,"name":"referee-comment-3458","label":"Referee Comment","group":{"name":"article-reports","label":"Article Reports"}},{"value":"10.3310\/nihropenres.15256.r36973, Dimitri Abrahamsson, University of California San Diego School of Medicine, San Francisco, California, 23 Sep 2025, version 2, indexed","URL":"https:\/\/openresearch.nihr.ac.uk\/articles\/5-37\/v2#referee-response-36973","order":4,"name":"referee-response-36973","label":"Referee Report","group":{"name":"article-reports","label":"Article Reports"}},{"value":"<b>Richard Lilford<\/b>; \n<i>Posted: 03 Nov 2025<\/i>; Thank you very much for reading our paper and providing comments with which we largely agree.  Causal inference weak. Agree and stated in the text. It is up to the reader to reflect on whether the magnitude of change would have occurred under a no-intervention counterfactual \u2013 we provide some data to show that such a temporal trend is plausible. Consecutive or random sampling. &nbsp;It was consecutive \u2013 amendment already made in response to another reviewer. Demographics.&nbsp; Our sample was simply pregnant women as they presented with the stated exclusion. Sample size \u2013 power calculation.&nbsp; On the point that a power calculation is needed we must disagree. We disagree for the same reason that power calculations are otiose in systematic reviews \u2013 the sample was pre-set and we calculate the confidence intervals based on the observed data. Consider expanding discussion to include potential barriers to implementing \u2018contingent actions\u2019. We presume the reviewer refers to clinicians responding to prompts. We had included the issue of information overload and human factors that attest how clinicians respond to prompts.","URL":"https:\/\/openresearch.nihr.ac.uk\/articles\/5-37\/v2#referee-comment-3497","order":5,"name":"referee-comment-3497","label":"Referee Comment","group":{"name":"article-reports","label":"Article Reports"}},{"value":"10.3310\/nihropenres.15256.r37070, Abi Merriel, University of Liverpool, Liverpool, 16 Oct 2025, version 2, indexed","URL":"https:\/\/openresearch.nihr.ac.uk\/articles\/5-37\/v2#referee-response-37070","order":6,"name":"referee-response-37070","label":"Referee Report","group":{"name":"article-reports","label":"Article Reports"}},{"value":"<b>Richard Lilford<\/b>; \n<i>Posted: 03 Nov 2025<\/i>; Thank you very much for your careful reading our contribution.  Why only 100 cases. This decision was based on two factors. First, making most efficient use of limited resources and second, the size of effect that would be clinically meaningful. Given the low baseline adherence, the investigators were not looking for small magnitude change \u2013 indeed we ended up with precise observations of large magnitude. Why was the project done. &nbsp;To examine the effect of the prompts on clinical behaviour. Prescription of \u2018antibiotics\u2019 for high BMI.&nbsp; Thank you for spotting the error \u2013 we have corrected this to \u2018vitamins\u2019. Descriptive stats in wrong place. There is no right or wrong answer here as descriptive statistics often (?usually ) appear in the results. Please may we avoid re-ordering the paper? Explanation of the GVIF value. We explain that the Generalised VIF is a measurement of collinearity \u2013 however we now spell out the VIF acronym. Thank you.","URL":"https:\/\/openresearch.nihr.ac.uk\/articles\/5-37\/v2#referee-comment-3498","order":7,"name":"referee-comment-3498","label":"Referee Comment","group":{"name":"article-reports","label":"Article Reports"}},{"value":"This project is funded by the National Institute for Health and Care Research (NIHR) under its [NIHR Applied Research Collaboration West Midlands (ARCWM) and Midlands Patient Safety Research Collaboration (PSRC) (Grant Reference Number NIHR200165)]. The views expressed are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.","order":8,"name":"grant-information","label":"Grant Information"},{"value":"This is an open access article distributed under the terms of the Creative Commons Attribution Licence, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.","order":0,"name":"copyright-info","label":"Copyright"}]}}