Development, test-retest reliability and validity of the Pharmacy Value-Added Services Questionnaire (PVASQ)

Main Article Content

Christine L. Tan
Mohamed A. Hassali
Fahad Saleem
Asrul A. Shafie
Hisham Aljadhey
Vincent B. Gan


Pharmaceutical Services, Health Knowledge, Attitudes, Practice, Validation Studies as Topic, Questionnaires, Malaysia


Objective: (i) To develop the Pharmacy Value-Added Services Questionnaire (PVASQ) using emerging themes generated from interviews. (ii) To establish reliability and validity of questionnaire instrument.

Methods: Using an extended Theory of Planned Behavior as the theoretical model, face-to-face interviews generated salient beliefs of pharmacy value-added services. The PVASQ was constructed initially in English incorporating important themes and later translated into the Malay language with forward and backward translation. Intention (INT) to adopt pharmacy value-added services is predicted by attitudes (ATT), subjective norms (SN), perceived behavioral control (PBC), knowledge and expectations. Using a 7-point Likert-type scale and a dichotomous scale, test-retest reliability (N=25) was assessed by administrating the questionnaire instrument twice at an interval of one week apart. Internal consistency was measured by Cronbach’s alpha and construct validity between two administrations was assessed using the kappa statistic and the intraclass correlation coefficient (ICC). Confirmatory Factor Analysis, CFA (N=410) was conducted to assess construct validity of the PVASQ.

Results: The kappa coefficients indicate a moderate to almost perfect strength of agreement between test and retest. The ICC for all scales tested for intra-rater (test-retest) reliability was good. The overall Cronbach’ s alpha (N=25) is 0.912 and 0.908 for the two time points. The result of CFA (N=410) showed most items loaded strongly and correctly into corresponding factors. Only one item was eliminated.

Conclusions: This study is the first to develop and establish the reliability and validity of the Pharmacy Value-Added Services Questionnaire instrument using the Theory of Planned Behavior as the theoretical model. The translated Malay language version of PVASQ is reliable and valid to predict Malaysian patients’ intention to adopt pharmacy value-added services to collect partial medicine supply.

Abstract 3167 | PDF Downloads 1716 online appendix Downloads 1079 Appendix VASQ Questionairre Downloads 275


1. Azmi M, Akmal SA, Chua G, eds. A national survey on the use of medicines (NSUM) by Malaysian consumers. Selangor: Quality Use of Medicines, Pharmaceutical Services Division, Ministry of Health Malaysia; 2013.

2. Ministry of Health Malaysia Malaysia. (2011). Annual Report Ministry of Health Malaysia. Malaysia: Ministry of Health Malaysia. Available at: (accessed July 7, 2015).

3. Lin YF, Lin YM, Sheng LH, Chien HY, Chang TJ, Zheng CM, Lu HP. First drive-thorugh pharmacy services in Taiwan. J Chin Med Assoc. 2013;76(1):37-41. doi: 10.1016/j.jcma.2012.10.001

4. McMillan SS1, Sav A, Kelly F, King MA, Whitty JA, Wheeler AJ. Is the pharmacy profession innovative enough?: meeting the needs of Australian residents with chronic conditions and their carers using the nominal group technique. BMC Health Serv Res. 2014;14:476. doi: 10.1186/1472-6963-14-476

5. Whitty JA, Kendall E, Sav A, Kelly F, McMillan SS, King MA, Wheeler AJ. Preferences for the delivery of community pharmacy services to help manage chronic conditions. Res Social Adm Pharm. 2015;11(2):197-215. doi: 10.1016/j.sapharm.2014.06.007

6. Valluri S, Seoane-Vazquez E, Rodriguez-Monguio R, Szeinbach SL. Drug utilization and cost in a Medicaid population: A simulation study of community vs. mail order pharmacy. BMC Health Serv Res. 2007;7:122.

7. Kirking DM, Ascione FJ, Richards JW. Choices in prescription-drug benefit programs: mail versus community pharmacy services. Milbank Q. 1990;68(1):29-51.

8. Johnsrud M, Lawson KA, Shepherd MD. Comparison of mail-order with community pharmacy in plan sponsor cost and member cost in two large pharmacy benefit plans. J Manag Care Pharm. 2007;13(2):122-134.

9. Carroll NV, Brusilovsky I, York B, Oscar R. Comparison of costs of community and mail service pharmacy. J Am Pharm Assoc (2003). 2005;45(3):336-343.

10. Zhang L, Zakharyan A, Stockl KM, Harada AS, Curtis BS, Solow BK. Mail-order pharmacy use and medication adherence among Medicare Part D beneficiaries with diabetes. J Med Econ. 2011;14(5):562-567. doi: 10.3111/13696998.2011.598200

11. Schmittdiel JA, Karter AJ, Dyer W, Parker M, Uratsu C, Chan J, Duru OK.The comparative effectiveness of mail order pharmacy use vs local pharmacy use on LDL-C control in new statin users. J Gen Intern Med. 2011;26(12):1396-1402. doi: 10.1007/s11606-011-1805-7

12. Duru OK, Schmittdiel JA, Dyer WT, Parker MM, Uratsu CS, Chan J, Karter AJ. Mail-order pharmacy use and adherence to diabetes-related medications. Am J Manag Care. 2010;16(1):33-40.

13. Devine S, Vlahiotis A, Sundar H. A comparison of diabetes medication adherence and healthcare costs in patients using mail order pharmacy and retail pharmacy. J Med Econ. 2010;13(2):203-211. doi: 10.3111/13696991003741801

14. Motheral BR, Heinle SM. Predictors of satisfaction of health plan members with prescription drug benefits. Am J Health Syst Pharm. 2004;61(10):1007-1014.

15. Johnson JA, Coons SJ, Hays RD, Sabers D, Jones P, Langley PC. A comparison of satisfaction with mail versus traditional pharmacy services. J Manag Care Pharm. 1997;3(3):327-337.

16. Ajzen I, Fishbein M. Understanding Attitudes and Predicting Social Behaviour. Englewood Cliffs, NJ: Prentice-Hall; 1980.

17. Fishbein M, Ajzen I. Belief, Attitude, Intention, and Behaviour: An Intoduction to Theory and Research. MA: Addison-Wesley; 1975.

18. Ajzen I. The Theory of Planned Behavior. Organiz Behav Human Decis Proc. 1991;50:179-211.

19. Ajzen I. From Intentions to Actions: A Theory of Planned Behaviour. In: Kuhl J, Beckmann J, eds. Action-control: From cognition to behavior. Heidelberg: Springer; 1985.

20. Ajzen I. The theory of planned behaviour: Reactions and reflections. Psychol Health. 2011;26(9):1113-1127. doi: 10.1080/08870446.2011.613995

21. McEachan RRC, Conner M, Taylor NJ, Jane LR. Prospective prediction of health-related behaviours with the Theory of Planned Behaviour: a meta-analysis. Health Psychol Rev. Sep 2011;5(2):97-144. doi: 10.1080/17437199.2010.521684

22. Sentosa I, Nik Mat NK. Examining a theory of planned behaviour (TPB) and technology acceptance model (TAM) in internetpurchasing using structural equation modeling. J Arts Sci Commerce. 2012;3(2):62-77.

23. Carmack CC, Lewis-Moss RK. Examining the theory of planned behavior applied to condom use: the effect-indicator vs. causal-indicator models. J Prim Prev. 2009;30(6):659-676. doi: 10.1007/s10935-009-0199-3

24. Courneya KS, Plotnikoff RC, Hotz SB, Birkett NJ. Social support and the theory of planned behaviour in the exercise domain. Am J Health Behav. 2000;24(4):300-308. doi: 10.5993/AJHB.24.4.6

25. Freeney Y, O'Connell M. The predictors of the intention to leave school early among a representative sample of Irish second-level students. Br Educ Res J. 2012;38(4):557-574. doi: 10.1080/01411926.2011.563838

26. Yoon C. Theory of planned behavior and ethics theory in digital piracy: an integrated model. J Business Ethics. 2011;100(3):405-417.

27. Montano D, Kasprzyk D. Theory of Reasoned Action, Theory of Planned Behavior, and The Integrated Behavioral Model. In: Glanz K, Rimer BK, K V, eds. Health Behavior And Health Education. 4th ed. San Francisco: Jossey-Bass; 2008.

28. Francis JJ, Eccles MP, Johnston M, Walker A, Grimshaw J, Foy R, Kaner EFS, Smith L, Bonetti D. Constructing questionnaires based on the Theory of Planned Behaviour: A manual for health services researchers. United Kingdom 2004. ISBN: 0-9540161-5-7

29. Sekaran U, Bougie R. Research methods for business: a skill-building approach. 6th ed: West Sussex: Wiley; 2013. ISBN: 978-1-119-94225-2.

30. Nichols DP. Choosing an intraclass correlation coefficient. SPSS Keywords. 1998.

31. Field A. Discovering Statistics Using SPSS. 2nd (and sex, drugs and rock 'n' roll) ed. London: Sage; 2005.

32. Statistics Solutions. Confirmatory Factor Analysis. 2013. Available at: (accessed March 9, 2015).

33. Kim JO, Mueller CW. Factor analysis: statistical methods and practical issues. Newburry Park: Sage Publications; 1978.

34. Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika. 1951;16:297-334.

35. Brown JD. Likert items and scales of measurement? SHIKEN: JALT Testing & Evaluation SIG Newsletter. March 2011;1:10-14.

36. Carifio J, Perla RJ. Ten common misunderstandings, misconceptions, persistent myths and urban legends about Likert scales and Likert response formats and their antidotes. J Soc Sci. 2007;3(3):106-116.

37. Pallant J. SPSS Survival Manual: A step by step guide to data analysis using SPSS. 4th ed. Australia: Allen & Unwin; 2011.

38. Kline P. The handbook of psychological testing. 2nd ed. London: Routledge; 1999.

39. Cortina JM. What is coefficient alpha? An examination of theory and applications. J Appl Psychol. 1993;78:98-104.

40. Torres-Harding SR, Siers B, Olson BD. Development and psychometric evaluation of the Social Justice Scale (SJS). Am J Comm Psychol. 2012;50:77-88.

41. Kor K, Mullan BA. Sleep hygiene behaviours: an application of the theory of planned behaviour and the investigation of perceived autonomy support, past behaviour and response inhibition. Psychol Health. 2011;26(9):1208-1224. doi: 10.1080/08870446.2010.551210

42. Blackman NJM, Koval JJ. Estimating rater agreement in 2x2 tables: Correction for chance and intraclass correlation. Appl Psychol Meas. 1993;17(3):211-223.

43. Cohen J. A coefficient of agreement for nominal scales. Educ Psychol Meas. 1960;20:37-46.

44. Fleiss JL, Cohen J. The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educ Psychol Meas. 1973;33:613-619.

45. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159-174.

46. Portney LG, Watkins MP. Foundations of clinical research: applications to practice. 2nd ed. Upper Saddle River, NJ: Prentice Hall Health; 2000.

47. Wilcox R. Modern statistics for the social behavioural sciences: a practical introduction. Los Angeles: Taylor & Francis Group; 2012.

48. Jakobsson U, Westergren A. Statistical methods for assessing agreement for ordinal data. Scand J Caring Sci. 2005;19(4):427-431.

49. Sim J, Wright CC. The Kappa statistic in reliability studies: use, interpretation, and sample size requirements. Physical Therapy. 2005;85:257-268.

50. Spitzer RL, Cohen J, Fleiss JL, Endicott J. Quantification of agreement in psychiatric diagnosis. Arch Gen Psychiatry. 1967;17(1):83-87.

51. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;1(8476):307-310.

52. Altman DG. Practical statistics for medical research. London: Chapman & Hall; 1991.

53. Streiner DL, Norman GR. Health measurement scales. A practical guide to their development and use. 2nd ed: Oxford University Press, Oxford; 1995.

54. Gorsuch RL. Factor analysis. 2nd ed. Hillsdale, NJ: Erlbaum; 1983.

55. Nunnally JC. Psychometric theory. 2nd ed. New York: McGraw-Hill; 1978.

56. Everitt BS. Multivariate analysis: the need for data, and other problems. Br J Psychiatry. 1975;126:237-240.

57. Comrey AL, Lee HB. A first course in factor analysis. Hillsdale, NJ: Erlbaum; 1992.

58. Kaiser HF. A second-generation little Jiffy. Psychometrika. 1970;35:401-415.

59. Hutcheson G, Sofroniou N. The multivariate social scientist. London: Sage; 1999.

60. Norris M, Lecavalier L. Evaluating the use of exploratory factor analysis in development disability psychological research. J Autism Dev Disord. 2010;40(1):8-20. doi: 10.1007/s10803-009-0816-2

61. Stevens JP. Applied multivariate statistics for the social sciences. 2nd ed. Hillsdale, NJ: Erlbaum; 1992.

62. Marsh HW, Hau KT. Confirmatory factor analysis: strategies for small sample size. Statistical strategies for small sample research. Thousand Oaks, CA: Sage Publications; 1999.

63. Fabrigar LR, Wegener DT, MacCallum RC, Strahan EJ. Evaluating the use of exploratory factor analysis in psychological research. Psychol Methods. 1999;4(3):272-299.

64. Duarte-Silva D, Figueiras A, Herdeiro MT, Teixeira Rodrigues A, Silva Branco F, Polonia J, Figueiredo IV. PERSYVE- Design and validation of a questionanaire about adverse effects of antihypertensive drugs. Pharm Pract (Granada). 2014;12(2):396.

Most read articles by the same author(s)

1 2 > >>