Critical view on methods and limitations
A weak point of the survey is the very small number of responses, which could not be significantly increased even with three reminders per email. Its validity is thus reduced, but tendencies can nevertheless be observed. The small number of responses gives rise to a certain selectivity: the possibility is increased that interested persons (or ones who are particularly annoyed) respond. Since the application is currently a voluntary additional offer which is to be tested for possible uses, precisely these interested and critical persons were seen to be an interesting target group for this survey.
Two major problems which led to a large number of students not downloading the app despite being interested and having actively requested and received personal access data, was identified in an additional survey: the access to the app through an individual log-in on a platform, and the unfamiliar download route of the web-app, which differs greatly from the usual app-store process [9
These barriers could be overcome through the development of a native app and free access without login, but in this case there would be financial (programming costs) and legal (copyright obligations) difficulties to resolve.
This survey is not a conclusive evaluation study, for which a larger number of responses would have been required, but it represents a feasibility study, a pilot survey in which emphasis is also placed on the qualitative description, namely the ways in which the app can be used by students, and a collection of ideas - which scenarios students themselves could imagine for its use in teaching. The evaluation has been therefore been made to a great extent qualitatively, in which case smaller numbers of responses may provide suggestions.
Target group and users
This application, in which the short versions of the DEGAM Guidelines were processed on a one-to-one basis for the small smartphone screen, is intended for the use of undergraduate medical students and doctors during and after completing specialist training, who are concerned with patient-centered care in the context of primary care (medical school, theory, practice).
Previous use of a smartphone in work / university environments was quoted as ranging from “not at all” to “very often”, whereby there were considerably more regular users than smartphone-newcomers. This corresponds with the study by Schulz et al. 2012 [7
], which describes a high degree of smartphone usage among students of dentistry and medicine and the increasing wish of 89% of respondents for more e-learning offers. This high degree of smartphone usage and the demand of medical students for more apps for their learning process have also been shown internationally in the study by Payne et al. [8
] and Mosa et al., that in particular young doctors and students made use of PDAs (Personal Digital Assistants), which are now increasingly being replaced by smartphones [4
In the responses of the students, previous use of guidelines tended towards “to a lesser extent” (expressed as a simplified school grade – 1 being best and 6 the lowest grade – this was 3.6) and knowledge of the DEGAM Guidelines was said to be even poorer (“grade” 4.4) whereby a certain level of interest was said to exist (“grade” 2.9) – of course this had to be the case, otherwise the students would not have registered as testers and downloaded the app onto their smartphone.
Installation and feedback on the app
Download and installation of this (unfamiliar) web-app worked well and without any great problems among these in this smartphone experienced group, and clarity of arrangement and design were judged positively. The grades 1.6 and 2.0 take into account evaluations made of the first version of the app in April 2011, as a result of which (“more colour please”) improvements were made in colour and design for the 2nd and 3rd versions.
Use of the application
Situations in which students used the app were primarily “in waiting periods”, “while using public transport” etc., and “before/during/after lectures”. This corresponds with the theoretical considerations of Lemos et al., who propagate the idea of using waiting periods for learning, and the possibility of incorporating environmental information (situated learning) into their concept “How can m-learning (mobile learning) improve the quality of the teaching of medicine?”
]. Mosa et al. also describe this usage of apps by medical students for learning and revision during spare moments in medical school, when no textbook is available. [4
The application was judged to be helpful with regard to use in spare moments / waiting periods and in connection with lectures (“grades” 2.0 and 2.1). Deployment in these situations is also one of the reasons why all students attested the application a clear advantage over the printed and pdf-versions of the guidelines (“grade” 1.6).
„I’ve always got the guidelines with me and save a lot of paper. Perfect for quick reference.“
„Always with me, and otherwise I’d have to carry the print version around“
With regard to patient contact there were few and very varied responses, which can be explained by the lack of patient contact of the majority of responding students (only two had contact with patients: one in a clinical attachment and one in his final year rotation in General Practice).
This background is reflected again in the question of how the application was used: namely “general interest, learning about the guidelines and for reference reasons”, “to learn and expand my own knowledge”, “for a lecture (homework, preparation, follow-up)” and “consultation regarding a specific (theoretical) question”. It was also sometimes “consulted with regard to a specific patient-based question”, whereby the virtual patients of the “Online Course on General Practice Guidelines” were included. Helpfulness and usefulness of the app when used in the former ways were “graded” between 1.6 and 1.9, in specific patient-related questions the assessments ranged from 1 to 6 with few responses (“grade” 2.4).
(Two persons, both on courses not involving patient contact – gave an answer to each evaluation question, although they were explicitly asked to only provide a response to those questions which actually applied to them: both frequently evaluated items with a “6”; for instance “use during house visits” and “use before/during/after patient contact”. Obviously these negative evaluations really mean: “does not apply”. However, since from a purely scientific point of view the comments of all participants in the study have to be taken seriously, these negative assessments were not removed from the evaluation and were therefore incorporated into the “average grades”.)
Influence of the application on interest in, and knowledge of the, guidelines
After interest in, and knowledge of, the guidelines had been judged as being poor before use of the application (see above), an improvement was registered in both categories after using the application. This is reflected in free text comments:
“I think it’s a good thing that the guidelines are available as an app. It’s a very dry subject, so the combination with the smartphone is a perfect way to arouse interest”.
All in all the application had given the users a better understanding of the guidelines (“grade” 1.7) and it was considered to be “helpful to their requirements”. Since the guidelines are relevant to examinations, this positive assessment of the app by students is a strong argument for the integration of the application into the training.
Suggestions for use in medical training
The students named the practical sections of the training as being particularly suitable for use of the application:
“The application is very suitable for the Online Guidelines Course. I could also imagine it being useful in clinical attachment and during the final year”.
“Tip as a source of reference and for preparation for the clinical attachment in General Practice”
But it is also suitable for university courses:
“Used the app to prepare for lectures and courses …… good way of reminding oneself of the most important facts before a lecture.”
But they also saw the limitations in usage:
“I think a “compulsory” use in lectures or seminars would be problematical, because by no means all students have a smartphone, and compulsory use in team work would not be productive.”
In view of the alterations made in the Medical Licensure Act of 11.5.2012 with the introduction of a compulsory elective period in General Practice / Primary Care , the application also lends itself to use:
“The app would certainly come in very handy during an elective in General Practice.”
Positive comments on the application
The app itself was well received as an innovative application:
“I was pretty much taken with the DEGAM Guideline app. A move in the right direction at last, away from all those fusty old systems.”
“I‘ve always got the Guidelines with me and save a whole load of paper. Ideal for quick reference.”
And students even saw a benefit in using the application on PCs, although on the larger screen the pdf-versions of the short versions of the guidelines would have been easily legible, a benefit which extended to include tablets and iPads:
“For this [working on patient cases on a PC] using the application was very helpful, because it was not necessary to plough through all the individual guidelines. Everything was clearly laid out and well-arranged. The most important points are accentuated, which makes it quick and easy to find the right solutions.”
“The optical presentation of the information is very pleasing and one doesn’t get swamped with information as is the case with the “jam-packed” short versions of the guidelines.”
There was much praise for the design and the clarity of presentation:
“The optical presentation is very pleasing and information is easy to find and presented very clearly.”
(Even if this opinion was occasionally contradicted:
“I find the app in its present form relatively free of benefits, because it is totally confusing and badly structured”.)
Criticism and suggestions regarding the application
In the first versions “more colour” was demanded, which has already been implemented. The text-heaviness however, which was often a reason for complaint, stems from the original (the short versions of the guidelines were adopted word for word, the only illustrations being flow-charts). The interactive algorithms which were requested are also dependent on the originals and could successfully be incorporated into the most recent guideline “chest pain”.
Until now it has been mainly General Practitioners, who regularly use the guidelines, who have requested the addition of the long versions of the guidelines, although students have also expressed the same wish:
„Apart from that it would be nice if patient information, long versions and other additional pieces of information, which are so far only available via a link from the web-app, could be saved for offline use“.
The wish for the addition of a search function was also expressed several times:
“A search function would be good to help find things faster”
Simplified access for students and reliable offline use could be achieved by placing native apps in app-stores.