• Assessing the quality of teleconsultations in a store-and-forward telemedicine network

      Wootton, Richard; Liu, Joanne; Bonnardot, Laurent (Frontiers Media, 2014-07-01)
      Store and forward telemedicine in resource-limited settings is becoming a relatively mature activity. However, there are few published reports about quality measurement in telemedicine, except in image-based specialties, and they mainly relate to high- and middle-income countries. In 2010, Medecins Sans Frontieres (MSF) began to use a store-and-forward telemedicine network to assist its field staff in obtaining specialist advice. To date, more than 1000 cases have been managed with the support of telemedicine, from a total of 40 different countries. We propose a method for assessing the overall quality of the teleconsultations provided in a store-and-forward telemedicine network. The assessment is performed at regular intervals by a panel of observers, who-independently-respond to a questionnaire relating to a randomly-chosen past case. The answers to the questionnaire allow two different dimensions of quality to be assessed: the quality of the process itself and the outcome, defined as the value of the response to three of the four parties concerned, i.e. the patient, the referring doctor and the organisation. It is not practicable to estimate the value to society by this technique. The feasibility of the method was demonstrated by using it in the MSF telemedicine network, where process-quality scores, and user-value scores, appeared to be stable over a nine-month trial period. This was confirmed by plotting the cusum of a portmanteau statistic (the sum of the four scores) over the study period. The proposed quality assessment method appears feasible in practice, and will form one element of a quality assurance programme for MSF's telemedicine network in future. The method is a generally applicable one, which can be used in many forms of medical interaction.
    • Assessing the Quality of Teleconsultations in a Store-And-Forward Telemedicine Network - Long-Term Monitoring Taking into Account Differences between Cases

      Wootton, Richard; Liu, Joanne; Bonnardot, Laurent (Frontiers Media, 2014-10-28)
      We have previously proposed a method for assessing the quality of individual teleconsultation cases; this paper proposes an additional step to allow the long-term monitoring of quality. The basic scenario is a teleconsultation system (aka an e-referral system or a tele-expertise system) where the referrer posts a question about a clinical case, the question is relayed to an appropriate expert, and the chosen expert provides an answer. The people running this system want assurances that it is stable, i.e., they want routine quality assurance information about the "output" from the "process." This requires two things. It needs a method of assessing the quality of individual patient consultations. And it needs a method for taking into account differences between patients, so that these quality assessments can be compared longitudinally. Building on the previously proposed methodology, the present paper proposes two techniques for measuring the difficulty posed by a particular teleconsultation. The first is an indirect method, similar to a willingness to pay economic estimation. The second is a direct method. Using these two methods with real data from a telemedicine network showed that the first method was feasible, but did not produce useful results in a pilot trial. The second method, while more laborious, was also feasible and did produce useful results. Thus, when output quality is measured, an allowance can be made for the characteristics of the case submitted. This means that fluctuations in output quality can be attributed to variations in the process (network) or to variations in the raw materials (queries submitted to the network). Long-term quality assurance should assist those providing telemedicine services in low-resource settings to ensure that the services are operated effectively and efficiently, despite the constraints and complexities of the environment.
    • Quality Assurance of Teleconsultations in a Store-and-Forward Telemedicine Network - Obtaining Patient Follow-up Data and User Feedback

      Wootton, Richard; Liu, Joanne; Bonnardot, Laurent (Frontiers Media, 2014-11-07)
      User surveys in telemedicine networks confirm that follow-up data are essential, both for the specialists who provide advice and for those running the system. We have examined the feasibility of a method for obtaining follow-up data automatically in a store-and-forward network. We distinguish between follow-up, which is information about the progress of a patient and is based on outcomes, and user feedback, which is more general information about the telemedicine system itself, including user satisfaction and the benefits resulting from the use of telemedicine. In the present study, we were able to obtain both kinds of information using a single questionnaire. During a 9-month pilot trial in the Médecins Sans Frontières telemedicine network, an email request for information was sent automatically by the telemedicine system to each referrer exactly 21 days after the initial submission of the case. A total of 201 requests for information were issued by the system and these elicited 41 responses from referrers (a response rate of 20%). The responses were largely positive. For example, 95% of referrers found the advice helpful, 90% said that it clarified their diagnosis, 94% said that it assisted with management of the patient, and 95% said that the telemedicine response was of educational benefit to them. Analysis of the characteristics of the referrers who did not respond, and their cases, did not suggest anything different about them in comparison with referrers who did respond. We were not able to identify obvious factors associated with a failure to respond. Obtaining data by automatic request is feasible. It provides useful information for specialists and for those running the network. Since obtaining follow-up data is essential to best practice, one proposal to improve the response rate is to simplify the automatic requests so that only patient follow-up information is asked for, and to restrict user feedback requests to the cases being assessed each month by the quality assurance panel.
    • Teledermatology in Low-Resource Settings: The MSF Experience with a Multilingual Tele-Expertise Platform

      Delaigue, Sophie; Morand, Jean-Jacques; Olson, David; Wootton, Richard; Bonnardot, Laurent (Frontiers Media, 2014-11-14)
      In 2010, Médecins Sans Frontières (MSF) launched a tele-expertise system to improve the access to specialized clinical support for its field health workers. Among medical specialties, dermatology is the second most commonly requested type of tele-expertise. The aim of the present study was to review all MSF teledermatology cases in the first 4 years of operation. Our hypothesis was that the review would enable the identification of key areas for improvement in the current MSF teledermatology system.
    • Telemedicine in low-resource settings

      Wootton, Richard; Bonnardot, Laurent (Frontiers Media, 2015-01-21)