Evaluation

The evaluation task was a work realized by the MISS and XRpartner. HON has developed the online interactive interface (below in Fig. 1) for the evaluation. The goal of this evaluation was to assess if WRAPIN really met to the user needs and particularly to know if it can help patients and individuals to judge information available on the web. The analysis of the results made clear the important impact of WRAPIN to meeting needs expressed by great number of users. There were three types of questionnaires:Technical, Satisfaction and Ergonomic. The evaluation was online and compared WRAPIN with an other Web Search Engine. There were four types of queries: Free and pre-formatted questions and free and pre-formatted URL for more than 30 medical disciplines.

Figure 1: Sample of the user evaluation

Here is the conclusion of this study:

WRAPIN addresses a real need. In an environment where patients and individuals are seeking more and more reliable and trustworthy medical and health care information, WRAPIN has the capacity to reply with synthetic information, independently from advertising sources.
The use of WRAPIN has, on the whole, been perceived as informative, reliable and trustworthy. Yet in a number of cases, the replies given by the system were irrelevant, but it was also true for the other engine. The rate of irrelevance/impertinence should be scientifically reckoned, and finally reduced. It seems the upfront lexical analysis of the queries, as well as the ergonomic of the reformulation leave room for improvement.
In the view of the testers, WRAPIN is over-performing other engines when the replies are pertinent. By providing a synthetic and reliable reply to a query, or an assessment of a document further to the submission of an URL, WRAPIN goes beyond the other engines and brings a valuable tool to users seeking medical and health care information.
It has been felt helpful to have a human that has reviewed web sites for appropriateness and correctness.