Virtual expert systems and decision accuracy of non-experts in technology consulting

Abstract

This objectivist, experimental study investigated the influence of virtual expert systems (VES) on the decision accuracy of non-expert consultants within a technology consulting contact centre environment. Because of the overwhelming availability of conceptual information, non-expert consultants experience challenges in making accurate decisions, and would benefit from augmented technologies, such as VES. VES hold the ability to capture and scale large volumes of decision variables for consideration by human experts when making decisions. A total of 40 participants were randomly selected from contact centres in the Eastern Cape and Gauteng provinces of South Africa for this study. Human logic was captured and scaled into a technology fault finding virtual expert and administered as an experiment to group participants. The experimental and control group participants were randomly assigned to the respective groups of 20 participants each. The control group was exposed to the paper-based, fault-finding manual. The pre-test and post-test data were collected based on four decision accuracy measures, namely individual performance, average call handling time, first call resolution and customer service. The Clarify performance system of the participating technology consulting company was used as data collection tool to record the findings used for Chapter 5. Statistical data analyses were performed using ANOVA and two-tailed significance tests to test the relationship between VES and decision accuracy in the pre-test and post-test phases of the study. The study found that the participant scores on the decision accuracy measures were only statistically significant on the measure of first call resolution measure (significance score of a p value <.05). On the other (three) measures, the scores obtained from experimental group participants showed more improvement than that of the control group participants. Consequently, the hypothesis that the use of VES enhance decision accuracy amongst non-expert technology consultants was accepted and the alternative hypothesis rejected. Some limitations pertaining to the resultant Hawthorne effect (the effect when some employees work harder and perform better when they are participants in an experiment) was noted amongst participants. This effect resulted from the use of team leaders in monitoring performance during the experiment and the involvement of the technology consulting company in determining the performance norms of the identified measures. Another limitation of the study related to the size of the sample where only two provinces were included. The limitation may affect the generalisation of results to other future settings when such a study is repeated. It was recommended that future studies in this field should make provision for a larger population, inclusive of other provinces to avoid these limitations.