1. This article was downloaded by: [ University of Florida] , [ Holly Donohoe]
On: 04 January 2012, At: 08: 26
Publisher: Routledge
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered
office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK
Journal of Ecotourism
Publication details, including instructions for authors and
subscription information:
http:/ / www.tandfonline.com/ loi/ reco20
A Delphi toolkit for ecotourism
research
Holly M. Donohoe
a
a
Department of Tourism, Recreation and Sport Management,
University of Florida, Gainesville, FL, USA
Available online: 15 Jan 2011
To cite this article: Holly M. Donohoe (2011): A Delphi toolkit for ecotourism research, Journal of
Ecotourism, 10:1, 1-20
To link to this article: http:/ / dx.doi.org/ 10.1080/ 14724040903418897
PLEASE SCROLL DOWN FOR ARTICLE
Full terms and conditions of use: http: / / www.tandfonline.com/ page/ terms-and-
conditions
This article may be used for research, teaching, and private study purposes. Any
substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,
systematic supply, or distribution in any form to anyone is expressly forbidden.
The publisher does not give any warranty express or implied or make any representation
that the contents will be complete or accurate or up to date. The accuracy of any
instructions, formulae, and drug doses should be independently verified with primary
sources. The publisher shall not be liable for any loss, actions, claims, proceedings,
demand, or costs or damages whatsoever or howsoever caused arising directly or
indirectly in connection with or arising out of the use of this material.
2. A Delphi toolkit for ecotourism research
Holly M. Donohoeā
Department of Tourism, Recreation and Sport Management, University of Florida, Gainesville,
FL, USA
(Received 5 September 2008; final version received 13 October 2009)
The Delphi is considered a legitimate and valuable research technique in a range of
research ļ¬elds. Its value is based on its utility for forecasting, issue identiļ¬cation or
prioritisation, and concept or framework development. However, systematic guidelines
for designing and executing a Delphi are not widely available. Concomitantly, a
plethora of methodological interpretations are proving a source of confusion for the
Delphi architect. This paper explores the utility of a proposed generic Delphi toolkit
(GDT) for designing and implementing a Delphi study. An ecotourism research
example is introduced as a basis for testing the GDT and for a critical review of its
beneļ¬ts and challenges for ecotourism research.
Keywords: ecotourism; Delphi technique; research design
Introduction
It is now approaching 60 years since the ļ¬rst Delphi was conducted and more than 30 since it
was ļ¬rst applied by tourism researchers. During this time, the Delphi has broadened its ļ¬elds
and frequency of application. Despite the methodās growing popularity, the Delphi literature
hosts two compelling concerns that serve as the foundation for this paper. The ļ¬rst concern is
related to methodological reļ¬nement. Methodological critique and evaluation are necessary
for identifying opportunities and constraints and for encouraging methodological reļ¬nement
and best practice (Alberts, 2007; Brown, 2007). In its place, the Delphi literature is dominated
by one-off case studies that lack sufļ¬cient detail about their design and execution to support
methodological evaluation (Powell, 2003). Mullen (2003) and Rowe and Wright (1999)
argued that our knowledge about the structure and potential of the Delphi is poor as a
result. This trend is inexorably linked to a second concern related to Delphi architecture.
The literature has come to suggest that there is no reference standard for Delphi architects
and there has been difļ¬culty in drafting a āgenericā or āuniversalā description of the
method and its procedures (Linstone & Turoff, 2002). As a consequence, there exists real
potential for confusion and ad hoc Delphi design and this raises serious questions about
the methodās validity and reliability. Given that guidelines are essential for Delphi design
and administration, that a lack of guidelines is contributing to confusion and that confusion
is impeding methodological review, the development and evaluation of a āgenericā description
of the method is required for methodological reļ¬nement.
ISSN 1472-4049 print/ISSN 1747-7638 online
# 2011 Taylor & Francis
DOI: 10.1080/14724040903418897
http://www.informaworld.com
ā
Email: hdonohoe@hhp.uļ¬.edu
Journal of Ecotourism
Vol. 10, No. 1, March 2011, 1ā20
Downloaded
by
[University
of
Florida],
[Holly
Donohoe]
at
08:26
04
January
2012
3. Day and Bobeva (2005) proposed a generic Delphi toolkit (GDT) for providing guidance
for Delphi research. If the GDT is substantiated as a well-proven and robust instrument, the
authors claim that the Delphi could be applied in a wider range of disciplines and methodologi-
cal confusion could be reduced. The validity of the GDT has yet to be tested; therefore, the
remainder of this paper addresses the call for Delphi research concerned with methodological
testing and reļ¬nement by testing the GDT. In doing so, this paper offers several contributions
towards addressing the aforementioned concerns. First, a critical review of the Delphi tech-
nique is provided and the GDT is introduced. Second, a case study example ā a research
agenda concerned with cultural sensitivity and ecotourism ā is presented in order to test the
utility of the GDT for guiding Delphi architecture and administration. By testing the validity
and appropriateness of GDT, this paper seeks to consolidate understanding of the Delphi
technique and to offer insight into its application based on a ecotourism research case.
The Delphi technique
The Delphi is a technique used to systematically combine expert knowledge and opinion to
arrive at an informed group consensus (MoeĢller & Shafer, 1994). The Delphi technique was
originally developed as a tool for soliciting opinion from a group of experts in order to
inform a forecasting process and it is now accepted as a valid technique for a variety of
research purposes (Landeta, 2006). Accordingly, the Delphi technique has been applied
in a breadth of ļ¬elds that include: environmental management (Gokhale, 2001; Wright,
2006), business marketing and management (Chevron, 1998; Hayes, 2007), recreation
(Austin, Lee, & Getz, 2008; Vaugeois et al., 2005), and health (de Meyrick, 2003). In
the tourism case, the Delphi has been applied for the purposes of long-range travel and
tourism planning (MoeĢller & Shafer, 1994), to predict change in the tourism industry
(Lloyd, La Lopa, & Braunlich, 2000; Yeong, Keng, & Leng, 1989), to develop ecotourism
and sustainable tourism management and evaluation frameworks (Miller, 2001; Spenceley,
2005), and to develop and/or reļ¬ne ecotourism deļ¬nitions (Garrod, 2003).
The Delphi technique is a structured communication process that facilitates (but does
not force) consensus (Linstone & Turoff, 1975). It is founded on the use of techniques
that aim to develop from a group of experts, an agreed view or shared interpretation on a
topic or issue where there is contradiction, controversy, or conļ¬ict (Day & Bobeva,
2005). Individual expert contributions are collected through iterative survey rounds,
group opinion is assessed, and opportunity is provided for individuals to revise their con-
tributions (controlled feedback) (Linstone & Turoff, 1975). Evaluation of the Delphi tech-
nique reveals a set of key Delphi attributes and beneļ¬ts (Donohoe & Needham, 2009):
(1) Legitimacy: An established research technique.
(2) Suitability: Well suited to complex problems where the contributions of experts
would contribute to advancing understanding and knowledge about the problem.
(3) Proximity: āVirtual laboratoryā where physical meetings are not required.
(4) Reflexivity: By design, participants think through concepts and questions ā so that
quality and objectivity of the data collected and the credibility of the studyās ļ¬nd-
ings may be enhanced.
(5) Flexibility: Methodological adaptation facilitates a comprehensive understanding
of the research problem.
(6) Repetition: Multiple iterations move the group towards an informed judgement
about a complex problem.
(7) Anonymity: Reduces the risk(s) for group dynamics to inļ¬uence outcomes.
2 H.M. Donohoe
Downloaded
by
[University
of
Florida],
[Holly
Donohoe]
at
08:26
04
January
2012
4. Delphi proponents advocate anonymity as a means for facilitating rational and reļ¬exive
individual contribution and for reducing the inļ¬uence of dominant participants, the power
of persuasion, and the bandwagon effect that is associated with other group methods
(Duboff, 2007; Linstone & Turoff, 2002; Rowe & Wright, 1999). Sunstein (2006)
argued that anonymity is what distinguishes the Delphi from other group techniques and
on this basis, Wolfers and Zitzwitz (2004) and Rowe and Wright (1999) have found the
Delphi to be more accurate than other group techniques. However, anonymity does not
rule out the possibility of researchers giving undue credence to the opinion of select
experts, the tendency of the individuals to be inļ¬uenced by the statistical measure of
group judgement, or the comments of panelists reported in summary reports (Wright,
2006). Other potential problems associated with the Delphi have been comprehensively
reviewed elsewhere. Representative examples include critiques and discussion related to
expert selection (Alberts, 2007; Phillimore & Goodson, 2004), questionnaire ambiguity
(Sackman, 1975; Seely, Iglarsh, & Edgell, 1980), false consensus (Linstone & Turoff,
1975), panel stability, the substantive quality of panelists contributions (Alberts, 2007;
Mullen, 2003; Wright, 2006), and the validity of the results (Landeta, 2006; Powell,
2003). Antecedent research suggests that the Delphi technique, like other methods, is not
free from ļ¬aws, bias, and other potential pitfalls or dangers (Briedenhann & Butts, 2006;
Murray, 1979). It is recommended that Delphi administrators recognise and address
potential constraints and the importance of professional judgement in control of the
exercise (Delbecq, Van de Ven, & Gustafson, 1975; Powell, 2003). Furthermore, care
must be afforded the ļ¬nal justiļ¬cation for the technique on the basis of its utility, its
suitability for the research problem, its potential advantages, and the measures available
for mitigating potential problems. Day and Bobeva (2005) contended that the GDT could
provide a basis for such justiļ¬cation and the guidance for doing so. As reminder, this
has yet to be substantiated in practice.
The GDT
Day and Bobevaās (2005) GDT is a three-stage model for designing and implementing a
Delphi study. The model is informed by a comprehensive analysis of the Delphi literature.
Commonalities between Delphi techniques used by other researchers are identiļ¬ed and syn-
thesised into a set of generic Delphi guidelines. In this regard, the GDT is a reference stan-
dard for the design, adaptation, and implementation of a Delphi study. The ļ¬rst stage,
āExplorationā, demands rigorous preparation where care is afforded the basis for consensus
(purpose statement), the establishment of participant selection criteria, and the creation of
an expert panel; the design of a data collection and analysis instrument; the identiļ¬cation of
an initial set of issues to be tested through Delphi implementation and/or pilot testing. Day
and Bobeva (2005) presented a set of generic design criteria and common options to help
guide Delphi architects through this preparatory phase (Table 1). For the purposes of this
paper, the term āExplorationā has been changed to āPreparationā to simplify the language
and make clear the stage objectives.
The second stage, referred to as āDistillationā, typically involves three rounds of surveys
that are circulated to a predetermined expert panel. The literature indicates that a scoping
round (Round 1), that is, a preliminary or general survey round, is commonly used
during the preparation phase as a means to circulate and solicit reactions to an introduction
package (initial problem and position statement) and/or pilot test surveys (Green, Hunter, &
Moore, 1990; Hurd & McLean, 2004; Weber & Ladkin, 2003). This scoping round is
optional, though it is recommended as it may help to identify ambiguities, it allows
Journal of Ecotourism 3
Downloaded
by
[University
of
Florida],
[Holly
Donohoe]
at
08:26
04
January
2012
5. some degree of early validity testing and methodological reļ¬nement, improves the feasi-
bility of administration, and allows the researcher to narrow the studyās focus (Garrod &
Fyall, 2005; Powell, 2003). Following the optional scoping round, the Delphi moves
into the āDistillationā stage where a survey is developed on the basis of the purpose state-
ment and the outcomes of the preparatory phase. The survey is circulated to the expert panel
and responses are collected (Round 2). The results are synthesised into a summary report
that is circulated to participants with the subsequent round. The report may not only
include simple statistical analysis (frequency tabulation, median, etc.), but it may also
include representative comments or criticisms from participants (anonymous) and a
consensus status update. The preparation of the report allows the Delphi administrators
the opportunity to analyse interim results and to monitor progress (or lack thereof)
towards consensus while the distribution of the report allows participants the opportunity
to reļ¬ect on their responses in light of the groupās judgement (Gordon, 1994; Rowe &
Wright, 1999). Subsequent rounds are essentially the same as Round 2, with surveys
(developed on the basis of the previous round) and summary reports provided to partici-
pants. The purpose of the iterative rounds, and particularly the feedback element that
comprise the āDistillationā stage of the Delphi, is to move the group towards a convergence
of opinion (Day & Bobeva, 2005).
In the ļ¬nal round, a synthesis and/or evolving consensus statement is distributed and a
ļ¬nal judgement is requested from participants (Gordon, 1994). It is important to note that
additional rounds may be necessary if the Delphi has not yet reached a critical point for
study termination. To complete this assessment, there are two recommended approaches:
extensive statistical testing and simple statistical reporting. Examples of extensive statistical
testing for consensus include Okoli and Pawlowskiās (2004) use of Kendallās W coefļ¬cient
of concordance and Schmidtās (1997) consensus ratio. Others prefer simple statistics for
measuring the degree of convergence (Day & Bobeva, 2005). For example, the use of
the mean/median to measure the control tendency and the standard deviation as a determi-
nant of convergence is common in the tourism literature (Garrod & Fyall, 2005; Green
et al., 1990; Kaynak & Macauley, 1984; Miller, 2001). Holey, Feeley, Dixon, and Whittaker
(2007, p. 52) concur, adding that the evolution of consensus can also be ascertained by
descriptive analysis of group trends such as an increase in agreement percentages, conver-
gence of range with standard deviations, and as a decrease in the number of comments
made. When sufļ¬cient convergence has been achieved, the Delphi rounds cease and the
group judgement is used to inform the ļ¬nal report and the initial research problem
(Stage Three). āConvergenceā is commonly used in the literature to describe Delphi activi-
ties and objectives during this stage; thus, it replaces āDistillationā in the remainder of this
paper.
Table 1. Generic design tool: Delphi criteria and options for the preparation phase.
Generic criteria Design options
1. Purpose of the study Building, exploration, testing, evaluation
2. Participants Homogeneous or heterogeneous groups
3. Anonymity of panel Full or partial
4. Number of rounds Between 2 and 10
5. Concurrency of rounds Sequential set of rounds
6. Mode of operation Face-to-face or remote access
7. Communication media Paper-and-pen, Internet-based, telephone, fax, email
Adapted after Day and Bobeva (2005).
4 H.M. Donohoe
Downloaded
by
[University
of
Florida],
[Holly
Donohoe]
at
08:26
04
January
2012
6. The third stage of the Delphi, named the āUtilisationā stage, includes the development
and dissemination of the ļ¬nal Delphi report. The results from the āConvergenceā phase are
integrated into a ļ¬nal consensus judgement and a report is developed on this basis. As an
option for Delphi administrators, a draft may be circulated to Delphi participants for one last
review. Although a group judgement has been achieved at this point, circulation of the draft
provides participants with a ļ¬nal opportunity to reļ¬ect on their responses and comment on
the Delphi outcome. Commentary is then integrated into the ļ¬nal report. Consideration
must also be given to the impacts of the consensus judgement on the initial research
problem during this stage of the Delphi. That is, the ļ¬nal report, and particularly the con-
sensus statement, is to be applied to the original research purpose so that the central research
question may be addressed. Dissemination is a distinctive feature of this stage and dissemi-
nation activities may be classiļ¬ed as both āshort termā and ālong termā in nature (Day &
Bobeva, 2005). In the short term, the ļ¬nal report is distributed to Delphi participants and
others who may have assisted in the Delphi exercise and/or have an interest in its
outcome (e.g. supporting organisations, research assistants, etc.). Long-term knowledge
dissemination activities include related publications that address both the substance and
the structure of the study. Although, āUtilisationā aptly describes some of the activities at
this stage, it does not describe the essential Delphi product. Therefore, āConsensusā is
used for the remainder of this paper to describe the third stage of a Delphi.
Delphi design and implementation: an ecotourism example
Using the GDT, the following sections critically assess the utility of the model for Delphi
architecture and administration. Each of the generic stages is explored through a Delphi
study concerned with ecotourism and cultural sensitivity. The decisions involved in its
design and the critical issues revealed by its implementation serve as focus. The ambition
is to bring the GDT to life in order to test its utility for guiding Delphi architecture generally,
and ecotourism research speciļ¬cally.
Purpose of the study
The purpose of the research is to develop a cultural sensitivity deļ¬nition and to test its
validity among a group of international ecotourism experts (Table 2). The Delphi technique
is applied in order to identify and describe the deļ¬nitional elements as well as the
opportunities and barriers to culturally sensitive ecotourism. The results are to inform an
examination of the extent to which progress towards making ecotourism sensitive to
Table 2. Delphi response rates and convergence measures.
Round one Round two Round three
Response ratea
60% 84% 77%
Convergence measures Mean scoreb
4.2 4.26 4.34
Standard deviation 0.97 0.87 0.63
a
The research was completed between January 2008 and July 2008. As the research moved into the summer months
and the busy ecotourism season, it was expected that participation would decline. Therefore, a larger initial group
was sought in order to mitigate a potentially high attrition rate. This proved to be a useful strategy.
b
Deļ¬nitional appropriateness scores measured on a ļ¬ve-point Likert-type scale where: 5 Ā¼ very appropriate, 4 Ā¼
appropriate, 3 Ā¼ somewhat appropriate, 2 Ā¼ not very appropriate, 1 Ā¼ not appropriate.
Journal of Ecotourism 5
Downloaded
by
[University
of
Florida],
[Holly
Donohoe]
at
08:26
04
January
2012
7. cultural differences (and similarities) in a global community is to proceed. In completing
these ļ¬nal steps, the research is to offer a practical product ā a deļ¬nition ā that contributes
to ecotourism discourse related to deļ¬nitions, cultural sensitivity, and management
guidelines.
The reason for this research focus is directly linked to a critical development in the eco-
tourism research and the recent identiļ¬cation of an ecotourism problem: the lack of cultural
sensitivity in ecotourism theory and praxis. Since 2000, a growing number of ecotourism
researchers are suggesting that a set of values ā embedded in āWesternā understandings
of human/environment relationship ā are being superimposed on āotherā or ānon-
Westernā ecotourism destinations and their value systems (Backman & Morais, 2001;
Braden & Prudnikova, 2008; Carrier & Macleod, 2005; Cater, 2006; de la Barre, 2005;
Jamal, Borges, & Stronza, 2006; Stark, 2002; Weaver & Lawton, 2007). A lack of sensi-
tivity to the cultural context of ecotourism presents perils related to goal achievement.
The literature suggests that the real danger exists for model transference failure when a
single āecotourism mouldā is used; there exists potential for beneļ¬ts to be replaced with
insecurity, resentment, conļ¬ict, ecological degradation, and economic loss (Vivanco,
2002). Cater (2006, p. 36) asserted that āif we uncritically accept Western-constructed
ecotourism as the be-all-to-end-all, we do so at our, and othersā perilā. In response to
these concerns, ecotourism researchers and practitioners are calling for increased cultural
sensitivity, increased reļ¬exivity in research and praxis, and a re-thinking of the so-called
āuniversalā ecotourism knowledge. Sofeild (2007, p. 158) concurred: āwhere different
world views and different aesthetics are involved, a greater degree of acceptance and
understanding for difference is imperativeā.
For ecotourism to truly exemplify sustainable development and the core ecotourism
tenets, the āmouldā must be sensitive to culture (Cater, 2006; de la Barre, 2005; Honey,
1999; Humberstone, 2004; McCool & Moisey, 2001; Stark, 2002; Vivanco, 2002). In
the ideal case, cultural sensitivity is a key ecotourism attribute as culture has the potential
to affect ecotourism opportunities, experiences, and management (Jamal, Borges, &
Stronza, 2006). Perceptions, attitudes, and values may also have the potential to affect
opportunities, experiences, and management, and this cultural dimension has the capacity
to affect ecotourismās social relevancy (Nyiri, 2006; Sofeild, 2007). Therefore, the attribute
denotes the sensitivity level of ecotourism to cultural differences and similarities and it
ultimately dictates the adaptation investments (e.g. policy development, training
modules, regulations, programmes, research agendas) required of ecotourism stakeholders
and leaders to make more acceptable the sharing of core deļ¬nitions, planning and manage-
ment tools and expertise, and international standards in the cultural mosaic of our global
human landscape. To achieve this culturally sensitive ideal, multiple (and overlooked)
ecotourism realities need to be identiļ¬ed and the inļ¬uences, potential conļ¬icts, and out-
comes of such phenomena must be understood. The latter requires an epistemology and
methodology that highlights and prioritises cultural sensitivity and that is counter-
hegemonic by design (Denzin & Lincoln, 2005; Kincheloe & McLaren, 2005; Saukko,
2005). This includes deļ¬nitions, standards, research models, visions, and strategies that
are sensitive to the diversity of cultural values and identities that exist in the ecotourism
landscape (de la Barre, 2005). Given the contemporary ecotourism context and the
rapid development of ecotourism markets in emergent destinations ā such as Bhutan,
Cambodia, China, Thailand, and Vietnam, a response to this call ā as Soļ¬eld suggests,
is imperative.
This research problem and purpose statement is the foundation for all other design
decisions during the āPreparationā stage of the Delphi.
6 H.M. Donohoe
Downloaded
by
[University
of
Florida],
[Holly
Donohoe]
at
08:26
04
January
2012
8. Methodology
The Delphi technique was selected over traditional group methods such as the group inter-
view or the focus group, as a valid method for eliciting knowledge and opinion from a
group of international ecotourism experts. The method was judged to be a best-ļ¬t on the
basis of its suitability for tourism research (Donohoe & Needham, 2009; Garrod & Fyall,
2005; Green et al., 1990) and its demonstrated utility for informing a deļ¬nition develop-
ment process (Garrod, 2003). Delphi attributes (described earlier in the text) may also be
considered advantages that are congruent with the research purpose and problem. The struc-
ture, utility, and purpose of the Delphi technique are congruent with the work of de la Barre
(2005) and Stark (2002) who suggest a need for a āconstructed spaceā for consensus build-
ing. Given that ecotourism has historically been advanced by non-inclusive knowledge
systems and that this process ļ¬nds us now in a potentially calamitous predicament, it
makes the most sense to apply a ācounter-hegemonicā methodology that supports Haber-
masās (1990) principle of āuniversalisabilityā and the achievement of inclusive consensus.
In this way, the product of the Delphi study is to reļ¬ect multiple interests, values, and exper-
tise while eliminating the risk of furthering Caterās (2006) āhegemonic sphereā (an exclusive
or Western-derived model). Instead, the Delphi method offers a constructed space for con-
sensus building. It facilitates reļ¬exivity and it allows for alternative viewpoints and/or
voices of the āotherā to be heard.
It is important to acknowledge that the Delphi exercise is to produce a deļ¬nition that is
positioned in both time and space. This is an important consideration because the conditions
that are framing the exercise, including the global popularity of ecotourism and the recent
discourse related to cultural sensitivity, are likely to affect the outcome. The experts who are
to comprise the Delphi panel are also representative of a diversity of truths located in time
and space, and their opinions and contributions are likely to reļ¬ect this fact. Therefore, it is
recognised that the outcome of the research is contextual and subjective.
Implementing the Delphi
The preparation stage
The literature establishes that Delphi study objective achievement is intrinsically linked to a
carefully designed and executed Delphi plan (Day & Bobeva, 2005; Miller, 2001; Philli-
more & Goodson, 2004). Based on the GDT, a three-stage Delphi implementation plan
was developed to guide the study (Figure 1).
In the preparation stage, the expert panel design is identiļ¬ed as the most important
design decision (Needham & de LoeĢ, 1990). Therefore, a Delphi study requires that the
declaration of selection criteria, the selection of experts, and the management of an
expert panel (Green et al., 1990; Linstone & Turoff, 2002) be carefully managed. A
Delphi study does not rely on a statistically representative sample of participants to
achieve valid results (Okoli & Pawlowski, 2004; Powell, 2003). Guidance suggests that
design decisions related to Delphi panel size, characteristics, and composition must ābe
governed by the purpose of the investigationā, the scope of the problem, and the resources
available (Cantrill, Sibbald, & Buetow, 1996, p. 69). The expertise represented on the panel
must therefore be congruent with the research topic and issues.
For this study, a panel of international ecotourism experts was desired so that the Delphi
captures a diversity of experience, knowledge, skills, and cultural perspectives. Guided by
the GDT, a seven-phase purposive sampling strategy was used for identifying, recruiting
and selecting participants. It is detailed here because in application, a number of Delphi
Journal of Ecotourism 7
Downloaded
by
[University
of
Florida],
[Holly
Donohoe]
at
08:26
04
January
2012
9. design challenges were revealed and the decisions made to mitigate such challenges may
help to shed light on the techniqueās strengths and weaknesses.
Phase 1 ā panel size: The GDT suggests that there is variation in the requisite size for a
Delphi panel, but limited guidance is provided for making panel size decisions (panel size
that will best serve the research agenda). Thus, the researcher consulted the Delphi literature,
and particularly the tourism literature, for guidance in this regard. Linstone (1978) asserted
that accuracy deteriorates with smaller panel sizes and improves with larger numbers. For
Delphi studies that are composed of a mixed group of experts (heterogeneous group), Brie-
denhann and Butts (2006) recommend a larger group. The tourism literature contains
examples of panels in the range of 7ā100 participants (Choi & Sirakaya, 2006; Green
et al., 1990; Kaynak & Marandu, 2006; Kuo & Chiu, 2006; Lloyd, La Lopa, & Braunlich,
2000; Miller, 2001; OāConnor & Frew, 2004; Tsaur, Lin, & Lin, 2006; Weber & Ladkin,
2003; Yeong, Keng, & Leng, 1989). The challenge in constructing a heterogeneous panel
of international ecotourism experts is in ensuring that the panel includes a diversity of cultural
backgrounds, perspectives, and experience. Given the global context of the research and its
cultural theme, this balance is imperative for producing a relevant and inclusive consensus.
Thus a goal of 75ā125 participants was judged to be congruent both with the aims of the
study and Delphi research standards. Also, the goal was purposefully broad to facilitate
Figure 1. Three-stage Delphi approach.
8 H.M. Donohoe
Downloaded
by
[University
of
Florida],
[Holly
Donohoe]
at
08:26
04
January
2012
10. inclusion and to mitigate attrition between rounds (see Phase 5). A minimum panel size of 20
participants was also determined on the basis of the research objectives as well as recommen-
dations found in the literature (Archer, 1980; Linstone & Turoff, 2002). The minimum estab-
lishes a critical level under which the Delphi panel can no longer be considered stable and the
Delphi can no longer be considered a rigorous exercise (Garrod, 2003). Thus, a safeguard
(minimum panel size and attrition rate/critical level monitoring) is in place in the case of
high attrition rates during the convergence stage. A balance between academic and pro-
fessional experts as well as their global distribution was desired and relatively equal represen-
tation throughout the Delphi exercise was maintained.
Phase 2 ā panel expertise: A purposive sampling method was employed to select the
relevant experts concerning the study topic: (1) ecotourism professionals from government,
private industry and non-governmental organisations and (2) academics engaged in eco-
tourism research and education. It is assumed that these groups, together (heterogeneous
panel), will contribute rich and varied data and information for the Delphi study. The
inclusion of both professional and academic experts is substantiated by Alberts (2007),
Briedenhann and Butts (2006), Donohoe and Needham (2009), Sunstein (2006), and
Vaugeois et al. (2005) as a means to achieve a balance between differing approaches to
and perspectives on āknowledgeā, to mitigate the existent divide between research and
practitioner communities (knowledge sharing, communications, priorities, epistemology,
etc.), and to achieve an inclusive Delphi outcome. An intriguing dichotomy is present,
and it was expected that consultations with these groups would provide meaningful
results. For this study, the āexpertā must be proļ¬cient in English and they must satisfy a
minimum of one of the selection criteria (parameters for participant recruitment and
selection).
(1) Current or previous experience in the public or governmental sector related to eco-
tourism and/or nature-based tourism activities (minimum 5 years)
(2) Current or previous experience in the private sector related to ecotourism and/or
nature-based tourism activities (minimum 5 years)
(3) Evidence of professional productivity in terms of peer-reviewed or professional
publications and research and/or participation in academic or industry symposia
(4) Teaching portfolio that includes courses dedicated to tourism and/or ecotourism
(minimum 5 years)
(5) Other ā potential participants are to be afforded the opportunity to provide insight
into ecotourism expert qualities and/or attributes that may have been overlooked
and that should be considered for participant selection.
The researcher recognises that the language requirement is a very important element in
this study as it introduced a potential bias ā particularly because of the desired diversity of
āvoicesā. However, English is the primary language of the researcher and for this reason, as
well as logistical reasons such as time and budget, multiple language editions of the survey
were not developed. Instead, English served as the unifying language for the panel and
where possible, language assistance was provided. For example, support was provided to
English-as-a-second-language participants in locations such as Canada, Brazil, China,
Greece, and Mexico so to avoid unnecessary language-based exclusion or attrition. The
irony of only using English in a study of cultural sensitivity is acknowledged.
Phase 3 ā expert recruitment: Portals through which ecotourism experts were identiļ¬ed
and recruited include international peer-reviewed publications and organisations where
ecotourism experts publish and/or seek membership. Peer-reviewed journals provide
Journal of Ecotourism 9
Downloaded
by
[University
of
Florida],
[Holly
Donohoe]
at
08:26
04
January
2012
11. publication portals where ecotourism experts publish their research ļ¬ndings. For example,
the Journal of Ecotourism, the Journal of Sustainable Tourism, the International Journal of
Tourism Research, and Tourism Management are established publications where ecotour-
ism research is frequently published. Expertise is established through a rigorous peer-
review process. In addition to the selection criteria, peer-review serves as an additional
measure of expertise. International organisations where ecotourism experts seek member-
ship provide complementary portals for recruitment. The International Ecotourism
Society (TIES) and the International Ecotourism Club (ECOCLUB) are global organis-
ations where ecotourism experts (both academics and professionals) seek membership.
Unique to these organisations is a strong representation of ecotourism professionals from
a diversity of locations and cultural backgrounds. While academic experts are also rep-
resented in the membership ledgers, the [total] diversity of ecotourism expert members is
the strength and utility of these portals. In total, six portals are included as they offer unpar-
alleled access to ecotourism experts generally, and potential research participants speciļ¬-
cally. In the absence of these select portals, the international pool of ecotourism experts
would be difļ¬cult to access and the challenge of constructing the desired Delphi panel
would be magniļ¬ed.
Phase 4 ā recruitment strategies: Using the expert portals, direct and assisted strategies
were used to identify potential participants and to create a potential participants database.
First, individuals with ecotourism research experience are identiļ¬ed through a strategic
review of papers published in the aforementioned journals within the last 5 years (2002ā
2007). This direct approach was expected to produce 100 individuals but it produced
160. By way of the TIES portal, additional ecotourism experts were identiļ¬ed from the
membership and expert lists found on the organisationsā public website. Individuals and
their contact information were harvested and entered into the potential research participant
database. In addition to these direct methods, TIES and ECOCLUB assisted by circulating a
ācall for participantsā to their membership through electronic or print newsletters and/or
posting the ācallā on their website. Interested individuals were invited to submit an
āexpression of interestā to the researcher. This assisted approach was expected and did
produce a pool of approximately 600 potential participants for the Delphi exercise. The
lists were cross-referenced to remove duplications and the ļ¬nal pool included a total of
706 individuals.
Phase 5 ā panelist screening and selection: Potential participants were contacted by
email with an invitation to participate. The invitation contained a description of the research
and the participant selection criteria as well as a link to an Internet-based survey. The survey
required potential participants to indicate which, if any, of the expert selection criteria they
satisfy, to indicate their English language proļ¬ciency, to identify their nationality, to self-
identify as an academic, professional, or combination thereof, and to provide their contact
information. Respondents were provided with a 14-day period in which to respond. A
total of 159 individuals responded while 157 satisļ¬ed the expert selection criteria.
Phase 6 ā panel composition: While creating a panel that is statistically representative
of the Worldās population of cultures is likely to be laborious and time consuming, it was
not likely to add great(er) value to the study. Instead, the decision was taken to develop a
panel comprising a diversity of individuals from a variety of locations. To establish diver-
sity, two measures were employed. First, the number of nationalities, organised by conti-
nent, provided a simple diversity measure (Figure 2). A diversity of nationalities was
desired (i.e. at least 25) while a lack of diversity (i.e. few nationalities or a western predica-
tion) was to serve as indication that cultural richness had not been achieved. This was not
the case and no additional recruitment was required. While this measure may be considered
10 H.M. Donohoe
Downloaded
by
[University
of
Florida],
[Holly
Donohoe]
at
08:26
04
January
2012
12. subjective by some, statistical sampling is acknowledged by the GDT as unnecessary for
establishing validity (Day & Bobeva, 2005). Instead, representativeness and validity is
established by other rules of thumb that are best described on the basis of their appropriate-
ness to the research at hand (Linstone & Turoff, 1975; Miller, 2001; Spenceley, 2005). The
panelistās locations provide a secondary measure of panel composition. The locations were
marked on an Internet-based map and the link was shared with participants. For both the
researcher and the participant group, the map provided an interactive and public view of
the ālaboratoryā. A map with markers across the globe, while a very basic indication,
was meant to complement the aforementioned diversity measure.
Phase 7 ā initiate Delphi rounds: Once the participants were selected and the desired
balance was achieved, the participants were sent a formal email invitation to participate. A
link to the round one Internet-based survey was provided in the letter. The Internet
comprised the primary communication medium in the preparatory stage. The Delphi invita-
tions were administered by email, and the round one survey was administered using an
Internet-based survey software provider. The advantage of using the Internet is that it is
a convenient, economical, and quick method for transmitting communications and
collecting data (Denzin, 2004; Donohoe, 2008). It is an increasingly attractive option for
Delphi administrators who struggle to mitigate the traditionally long waits between
Delphi iterations when surface mail is the primary communication method (Day &
Bobeva, 2005; Donohoe, 2009). While surface mail was offered as an alternative to
participants who do not have the Internet access or who prefer the traditional paper-and-
pen approach to surveys, this offer was not accepted by any of the Delphi panelists.
Therefore, a critical design decision was made to continue Internet-based communications
and e-surveys through subsequent rounds.
As the expert panel construction is considered the most important Delphi design feature,
this very detailed and strategic approach was considered methodologically imperative.
The care to detail and process afforded in this stage, as guided by the GDT, produced a
Delphi panel consisting of 157 ecotourism experts (47 professional, 22 academic, 25
both academic and professional) from over 40 countries (Figure 2).
It is important to note that the time required to complete the preparation stage varies
considerably across Delphi studies. Deļ¬ning the research problem and purpose can take
Figure 2. Delphi panel composition.
Journal of Ecotourism 11
Downloaded
by
[University
of
Florida],
[Holly
Donohoe]
at
08:26
04
January
2012
13. months of preparation, review, and preliminary investigation. The recruitment process can
also vary depending on the challenges that may present when identifying, approaching, and
inviting potential participants to join the panel. A standard time frame for the preparation
stage is therefore not deļ¬ned by the GDT or elsewhere in the literature. In this case, the
total time for the preparation stage was 1 year with the majority of this period dedicated
to deļ¬ning the research problem and process. Once the panel recruitment process was
deļ¬ned, it took approximately 2 months to complete the seven-phase process described
here. This would have been much longer if traditional communications methods were
used; the use of the Internet expedited communications and recruitment. A Delphi study
requires more time and effort than many of the methodological alternatives available to
the ecotourism researcher. Therefore, it is recommended that the research architect
budget an appropriate amount of time for the design decisions and activities required
during the Delphi study preparation stage and that the Internet be considered as a communi-
cation and time management tool.
The convergence stage
The GDT recognises that preparatory effort is necessary before initiating the convergence
stage, but it does not distinguish this work as a separate stage (Day & Bobeva, 2005).
Instead, the GDT suggests that the initial base of knowledge be āgeneratedā through the
ļ¬rst round of the study and that it be complimented with a synthesis of the key issues ident-
iļ¬ed in the literature. Therefore, the ļ¬rst round was a scoping exercise designed to get
participants reļ¬ecting about the relationship between cultural sensitivity and ecotourism.
Round one asked: What are the three most important issues surrounding contemporary
ecotourism?; How important is it for ecotourism to be explicitly sensitive to culture and
cultural differences around the globe?; How appropriate is the working deļ¬nition of cultural
sensitivity for ecotourism (deļ¬nition provided)?; and What cultural variables or attributes
need to ļ¬nd expression in the deļ¬nition? Participants were provided with a one-month
period in which to respond. A response rate of 60% was achieved (Table 2) and the
results revealed that a majority agree (81%) that cultural sensitivity is a very important eco-
tourism issue that is highly relevant to the contemporary context but neglected in theory and
praxis. The working deļ¬nition for culturally sensitive ecotourism provided to the panelists
was purposefully broad and loosely structured so as to allow the group to inform the shape
and content of the deļ¬nition over the course of the Delphi exercise. As expected, the ļ¬rst
deļ¬nition resulted in a ļ¬urry of valuable comments, suggestions, and critiques that were
used to enhance the deļ¬nition and inform the subsequent Delphi rounds.
The GDT suggests that subsequent rounds may be considered āevaluation and exten-
sionā of the results produced in the ļ¬rst āscopingā round. Therefore, the focus of the remain-
der of the study was on developing a group consensus deļ¬nition for culturally sensitive
ecotourism. This included two survey rounds that successfully captured hundreds of com-
ments and suggestions from the Delphi panel. In each round, an enhanced deļ¬nition was
introduced and feedback was collected regarding its appropriateness. In order to assess
the level of convergence between rounds, simple mean scores and standard deviation
were calculated (Table 2). This approach is substantiated by the GDT as well as throughout
the Delphi literature and in particular, the Delphi research within the tourism domain (Green
et al., 1990; MoeĢller & Shafer, 1994). As the deļ¬nition was enhanced, the research revealed
a convergence of group opinion and an increase in convergence between rounds. Between
rounds two and three, the deļ¬nitional enhancements were minimal, no signiļ¬cant change in
the mean scores was observed, but the increase in convergence (reduction in standard
12 H.M. Donohoe
Downloaded
by
[University
of
Florida],
[Holly
Donohoe]
at
08:26
04
January
2012
14. deviation value) was noteworthy. Thus, it was decided that consensus had been reached and
that further rounds would not produce additional convergence of opinion.
It is important to note that the design of the data collection instrument evolved between
rounds. The GDT suggests that ācreativityā should guide any data collection mechanism and
that different ways of conceptualising the issue and structuring questions must be tried so as
to match the expertise of the panel and their preferred communication mode. The GDT
provides standard guidance on structuring the survey questions in accordance with well-
proven survey design practices. Moreover, the GDT suggests that the key is formulating
clear, concise, and unambiguous questions, together with clear instructions for the panelists.
It is also suggested that the researcher assume a commonsense approach to survey design
where care is afforded an aesthetically pleasing and a standard and technically easy format
so as to facilitate participation and mitigate attrition. For example, they suggest including
questions and scales that are easily identiļ¬able by the participant and where the participant
does not require additional training in order to complete the survey. As recommended by the
GDT, this study employed a combination of open and closed-ended questions and the
majority of consensus building activities and measures were based on a ļ¬ve-point
Likert-type scale where panelists judged the appropriateness of the deļ¬nition from āļ¬veā
for very appropriate to āoneā not appropriate (Table 2).
It is also important to note that the time required to complete the convergence stage
varies across Delphi studies. Contributing factors include the time between rounds, the
number of survey rounds required to achieve consensus, administration challenges, and
attrition. While the literature contains reports of Delphiās that required several months to
several years (Brown, 2007; Hanaļ¬n et al., 2007; Hurd & McLean, 2004; Vaugeois
et al., 2005), 1 year appears to be the average length of time required for Delphi adminis-
tration (Landeta, 2006). In this case, the survey rounds were administered between January
and July 2008. The total time required to complete the convergence stage was 6 months.
The consensus stage
The convergence stage produced a deļ¬nition that is considered the most appropriate to an
international panel of ecotourism experts. The process assures that the deļ¬nition has under-
gone testing and that a consensus judgement has been rendered regarding its value and
appropriateness. Before declaring āconsensusā, a draft ļ¬nal report was circulated to the
Delphi panelists with a request for ļ¬nal comments. A few panelists submitted ļ¬nal rec-
ommendations and these were integrated into the ļ¬nal consensus deļ¬nition (Figure 3). A
ļ¬nal Delphi report was then prepared and disseminated to the panelists, supporting organ-
isations, and other interested stakeholders. The ļ¬nal result is a deļ¬nition for culturally
sensitive ecotourism.
Discussion: critical issues for Delphi administration
The Delphi is a legitimate research technique that is proving useful in a growing range of
disciplines and ļ¬elds. In the ecotourism case, it has been shown that the Delphi is an effec-
tive technique for the generation of opinion from a geographically dispersed group of
ecotourism experts (Garrod & Fyall, 2005; Miller, 2001). The GDT introduced by Day
and Bobeva (2005) can be a valuable source of guidance for Delphi architects in all
stages of ecotourism research administration. In this case, the GDT has provided a useful
instrument for moving through each of the Delphi stages and the researcher was able to
carefully consider and operationalise the utility, structure, and substance of a Delphi for
Journal of Ecotourism 13
Downloaded
by
[University
of
Florida],
[Holly
Donohoe]
at
08:26
04
January
2012
15. the study of cultural sensitivity and ecotourism. In doing so, the exercise revealed a number
of GDT beneļ¬ts. First, the GDT provides a comprehensive summary of the critical issues
that must be considered when designing a Delphi study. The āchecklistā format of the GDT
facilitates critical reļ¬ection on the research problem, the choice of methodology, and the
associated research design and administration decisions. It offers a range of options as
well as a set of recommendations which have proven to be particularly useful for the
ļ¬rst-time Delphi architect. Second, the GDT describes a set of āoptionalā iterations
where researchers can validate the research purpose (scoping round) and the research pro-
ducts (draft ļ¬nal report circulation). These options provide researchers with an additional
set of tools that can ensure that the methodology is a ābest-ļ¬tā for the research, that the
method will produce meaningful data and by extension, and that the results of the
Delphi study can be substantiated. As such, the GDT provides researchers with a reference
standard for designing, adapting, and implementing a Delphi study.
In addition to the panel design, instrument design, communication method, time
requirements, and convergence measures, there are other design and implementation
issues that have been revealed by the implementation of the Delphi study. Four issues
were selected for discussion here as they are judged to be highly relevant to the ecotourism
research context and for the methodogical reļ¬nement process.
Issue 1: analysis of the Delphi results
The GDT warns about the potential data analysis problems that can arise. However, the dis-
cussion is limited to a speciļ¬c data collection instrument and little guidance is provided for
those using a breadth of instruments and techniques (as is recommended by the GDT). This
study revealed the complexity of processing a large, rich, and varied data set comprising
numerical and non-numerical data. At this time, there are few tools available for processing
data of this kind and this presents the Delphi administrator with additional time constraints
Figure 3. Deļ¬nitional enhancement process.
14 H.M. Donohoe
Downloaded
by
[University
of
Florida],
[Holly
Donohoe]
at
08:26
04
January
2012
16. (Day & Bobeva, 2005). As it is recommended that the time between rounds be kept to a
minimum ā less than 2 months, the researcher maintained strict deadlines so as to avoid
attrition between rounds, to avoid āsurvey fatigueā, and to mitigate temporal delay.
However, the data and recommended timelines presented the researcher with considerable
challenges in maintaining the research schedule and delivering the analysis reports to the
panelist with the subsequent round. Although deadline adherence served to mitigate attri-
tion and to expedite the Delphi process, additional time or ļ¬exibility could have allowed
for greater data analysis creativity and the identiļ¬cation of multiple perspectives. In this
regard, the GDT provided limited guidance on time management.
Issue 2: timing of the Delphi surveys
The GDT recommends that the surveys be dispatched so as to capture the maximum avail-
ability of the panelists and to achieve an acceptable response rate. This approach has been
substantiated in the literature, and the work of Briedenhann and Butts (2006) conļ¬rmed the
importance of knowing the situational context so as to avoid distractions and attrition. Their
study of rural tourism in Africa coincided with the World Forum on Sustainable Develop-
ment in Johannesburg and this caused a decline in participation as panelists abandoned the
Delphi to attend the event. For this study, care was afforded the timing of the surveys so that
they did not coincide with the active ecotourism season for professionals and the active
research or vacation seasons for academics. The surveys were initiated in January and a
high response rate was achieved and maintained throughout. Although attrition did
occur, the response rate improved and attrition rates decreased as the rounds progressed.
This may be evidence of the panelists level of commitment to the study and/or the cultural
sensitivity/ecotourism issue. It may also be the result of the efforts to inform the panelists of
the Delphi process, goals, and timelines at the outset so as to mitigate confusion and disil-
lusionment. It is therefore recommended that researchers become informed of the context
for administration and that the Delphi be timed to avoid known distractions so as to
achieve a desirable response rate. It is also recommended that panel include individuals
who are interested in the research topic and that they are well informed so as to prevent frus-
tration with the process and loss of interest over the course of the exercise.
Issue 3: communications
The GDT recommends that the researcher maintain a high level of communication for a
number of reasons: it supports participant commitment, it enables an adequate level of
response, and it mitigates attrition. For this study, signiļ¬cant time and effort were dedicated
to communications. Through all research stages, the Internet served as the primary com-
munication medium. Potential Delphi participants were identiļ¬ed through Internet
resources (online journals and databases), and all communications were conducted
through email. This made communications with a global network of experts possible,
rapid, and efļ¬cient while recruitment costs were reduced as postage and travel were not
required. A website was also created to serve as the central research communication
portal. The interactive online map and the survey summary reports were posted and
shared through this page. Participants frequented the webpage and they reported that
added value as a very useful communication portal for the study. The Delphi surveys
were administered through a āuser-friendlyā online survey provider. Postage costs for
survey delivery and return were eliminated and data collection and analysis costs were
minimal. While the time required for a Delphi study is substantial, the GDT suggests
Journal of Ecotourism 15
Downloaded
by
[University
of
Florida],
[Holly
Donohoe]
at
08:26
04
January
2012
17. that the time required can be decreased through the use of e-surveys and internet-based
tools. Although this Delphi study was completed in a 6-month period, the reliance on
Internet-based communications revealed several challenges ā both expected and unex-
pected. Technological challenges presented the most signiļ¬cant barrier to participant
recruitment and data collection. Select participants reported difļ¬culties such as an inability
to open survey links, to download the survey, or to submit their survey responses. In most
cases, difļ¬culty was the result of user error and administrator assistance served to resolve
the issue. In other cases, computer hardware such as operating systems, Internet connec-
tions, and technical failures were the source for difļ¬culty. For some, Internet access was
not available (infrastructure limited in remote locations and developing countries); it was
interrupted by electricity failures or unreliable connections (or in one case, cables
damaged by terrorists). Technological failures at the research centre also caused delay
when the server collapsed and the network remained inaccessible for 7 days. Finally,
some email addresses were inactive and this resulted in a signiļ¬cant amount of returned
emails. These challenges are not unique to the ecotourism case, but given the nature of
ecotourism operations and research, it is often the case that experts are located in areas
where Internet communications are not always possible. To maximise communications
and time savings as well as to mitigate technological challenges, it is recommended that
the researcher offer a pen-and-paper survey alternative, invest in developing clear directions
for survey completion, and pilot test online surveys and communications and that they
make themselves or an assistant available for technical support throughout the duration
of an Internet-based Delphi study.
Issue 4: panel management
The GDT identiļ¬es panel stability as a signiļ¬cant challenge for Delphi administrators.
Panel stability is understood to represent minimal attrition rates, preferred panel size man-
agement, and preferred panel composition or balance (Garrod & Fyall, 2005). A lack of
panel stability presents a considerable barrier to maintaining the expert group and achieving
the Delphiās objectives (Garrod, 2003). In this case, the researcher consulted the Delphi lit-
erature for a more comprehensive set of recommendations. The addition of new members,
in order to replace withdrawals, is not a recommended mitigation measure. Murray (1979,
p. 155) suggested that it damages the āvery core of the Delphi procedureā and āthe results
that emerge must be suspectā. Instead, it is recommended that Delphi architects develop an
initial expert sample list that reļ¬ects the predetermined expert selection criteria, establish a
minimum requisite group size and/or a preferred group size, and develop a panel stability
management plan accordingly (Choi & Sirakaya, 2006; Gordon, 1994). Garrod and Fyall
(2005) recommended that panel stability be assessed periodically throughout the process
with a quality control measure based on predetermined panel composition preferences
and criteria. Based on their experiences, Pan, Vega, Vella, Archer, and Parlett (1995,
p. 32) advocated that the āsample size should be as large as possible to allow for subsequent
drop-outs, yet small enough to ensure the respondents are all experts in their ļ¬eldsā. In the
worst case, Garrod (2003) recommended Delphi termination if the desired panel compo-
sition and size are compromised by attrition between rounds. Given the nature of the
research, the importance of the panel composition, and the established panel size
minimum, the latter is particularly pertinent. These recommendations complemented the
guidance provided by the GDT and were therefore integrated into the Delphi study
described here. For ecotourism researchers, a similar path would serve to avoid confusion
and study compromise.
16 H.M. Donohoe
Downloaded
by
[University
of
Florida],
[Holly
Donohoe]
at
08:26
04
January
2012
18. Conclusion
The review of the design and implementation of this Delphi study has illuminated the
potential utility of the GDT for ecotourism research. It makes possible the sharing of
values across a broad ecotourism stakeholder spectrum, the convergence of group
opinion on contemporary and relevant ecotourism issues, and the delivery of outcomes
that may have important implications for ecotourism management. Furthermore, bringing
together ecotourism experts is a particularly arduous task given the nature of the environ-
ments in which ecotourism activities occur. Experts are spread across the globe and are
often located in remote areas. Bringing these individuals together to discuss pertinent
issues presents logistical and ļ¬nancial challenges. The Delphi offers a constructed space
for examining ecotourism free from the aforementioned challenges. While the examination
of the GDT has illuminated other potential challenges associated with Delphi design and
implementation, the researcher is optimistic that the guidance provided by the GDT and
complimented by a literature review, that the level of care afforded the research design,
and that study implementation controls will mitigate potential challenges and maximise
potential opportunities. The GDT discussed in this paper has been proposed by Day and
Bobeva (2005) as a Delphi guide. Its utility as a guide has been assessed and it has been
found to be practical and user-friendly. Furthermore, its utility for research design may
be enhanced, its limitations can be mitigated, and its potential challenges can be addressed
when additional resources are consulted. For these reasons, it is suggested that ecotourism
researchers consult the GDT for guidance when considering, designing, or conducting a
Delphi study.
As the ecotourism research domain moves through its ācoming of ageā (Weaver &
Lawton, 2007, p. 1168), it is important to consider not only emergent themes and knowl-
edge voids, but also its methodological evolution. In Backman and Moraisās (2001)
review of the methodological approaches used in the ecotourism literature, they conclude
that the current methodological state of the domain is relatively immature and narrowly
focused. The Delphi technique, unlike the commonly applied interview, participant obser-
vation, or content analysis methods, is unique because it has not been the focus of signiļ¬-
cant attention or critical review in the ecotourism literature. In fact, Garrod (2003) reported
that the ecotourism literature is very lean in this regard. This paper represents an attempt to
address this void and to contribute to ecotourismās methodological ācoming of ageā. It is
hoped that it will ignite interest in the Delphi method, critical discourse related to its suit-
ability for ecotourism research and that it will inspire other researchers to beneļ¬t from its
advantages, attributes, and applications.
Acknowledgements
The author wishes to express gratitude to Delphi participants for their expertise and contributions and
to TIES and the ECOCLUB for their support.
References
Alberts, D.J. (2007). Stakeholders or subject matter experts, who should be consulted? Energy Policy,
35, 2336ā2236.
Archer, B. (1980). Forecasting demand: Quantitative and intuitive techniques. Tourism Management,
1(1), 5ā12.
Austin, D.R., Lee, Y., & Getz, D.A. (2008). A Delphi study of trends in special and inclusive
recreation. Leisure/Loisir, 32(1), 163ā182.
Journal of Ecotourism 17
Downloaded
by
[University
of
Florida],
[Holly
Donohoe]
at
08:26
04
January
2012
19. Backman, K.F., & Morais, D.B. (2001). Methodological approaches used in the literature. In D.B.
Weaver (Ed.), The encyclopedia of ecotourism (pp. 597ā610). Wallingford, Oxon: CABI Publishing.
Braden, K., & Prudnikova, N. (2008). The challenge of ecotourism development in the Altay region of
Russia. Tourism Geographies, 10(1), 1ā21.
Briedenhann, J., & Butts, S. (2006). The application of the Delphi technique to rural tourism project
evaluation. Current Issues in Tourism, 9(2), 171ā190.
Brown, C.A. (2007). The opt-in/opt-out feature in a multi-stage Delphi method study. International
Journal of Social Research Methodology, 10(2), 135ā144.
Cantrill, J.A., Sibbald, B., & Buetow, S. (1996). The Delphi and nominal group techniques in health
services research. International Journal of Pharmacy Practice, 4(2), 67ā74.
Carrier, J.G., & Macleod, D.V.L. (2005). Bursting the bubble: The socio-cultural context of ecotour-
ism. Journal of the Royal Anthropological Institute, 11, 315ā334.
Cater, E. (2006). Ecotourism as a Western construct. Journal of Ecotourism, 5(1&2), 23ā39.
Chevron, J.R. (1998). The Delphi process: A strategic branding methodology. Journal of Consumer
Marketing, 15(3), 254ā264.
Choi, H.C., & Sirakaya, E. (2006). Sustainability indicators for managing community tourism.
Tourism Management, 27, 1274ā1289.
Day, J., & Bobeva, M. (2005). A generic toolkit for the successful management of Delphi studies. The
Electronic Journal of Business Research Methodology, 3(2), 103ā116.
Delbecq, A.L., Van de Ven, A.H., & Gustafson, D.H. (1975). Group techniques for program plan-
ning: A guide to nominal and Delphi processes. Glenview, IL: Scott, Foresman and Co.
de la Barre, S. (2005). Not āecotourismā?: Wilderness tourism in Canadaās Yukon territory. Journal of
Ecotourism, 4(2), 92ā107.
de Meyrick, J. (2003). The Delphi method and health research. Health Education, 103(1), 7ā16.
Denzin, N.K. (2004). Prologue: Online environments and interpretive social research. In M.D. Johns,
S.-L.S. Chen, & G.J. Hall (Eds.), Online social research: Methods, issues, and ethics (pp. 1ā12).
New York: Peter Lang.
Denzin, N.K., & Lincoln, Y.S. (2005). Paradigms and perspectives in contention. In N.K. Denzin &
Y.S. Lincoln (Eds.), The Sage handbook of qualitative research (3rd ed., pp. 183ā190). Thousand
Oaks, CA: Sage Publications, Inc.
Donohoe, H. (2009). Internet-based Delphi administration: The evolution of methodological best-
practices for tourism research. Travel and Tourism Research Association 40th Annual
Conference Proceedings, Honolulu, HI.
Donohoe, H.M. (2008). Internet-based leisure research: Opportunities and constraints (Pesquisa No
Lazer Associados aĢ Internet: Oportunidades e Entraves). Motriz. Revista de EducacĢ§aĢo FıĢsica,
UNESP, 14(1), 1ā8.
Donohoe, H.M., & Needham, R.D. (2009). Moving best practice forward: Delphi characteristics,
advantages, potential problems, and solutions. International Journal of Tourism Research,
11(5), 415ā437.
Duboff, R.S. (2007, September 28). The wisdom of (expert) crowds. Harvard Business Review.
Retrieved January 3, 2008, from http://hbr.org/2007/09/the-wisdom-of-expert-crowds/ar/1
Garrod, B. (2003). Deļ¬ning marine ecotourism: A Delphi study. In B. Garrod & J.C. Wilson (Eds.),
Marine ecotourism: Issues and experiences (pp. 17ā36). Clevedon: Channel View Publications.
Garrod, B., & Fyall, A. (2005). Revisiting Delphi: The Delphi technique in tourism research. In B.W.
Ritchie, P. Burns, & C. Palmer (Eds.), Tourism research methods: Integrating theory and practice
(pp. 85ā98). Wallingford: CAB International.
Gokhale, A.A. (2001). Environmental initiative prioritization with a Delphi approach: A case study.
Environmental Management, 28(2), 187ā193.
Gordon, T.J. (1994). The Delphi method. AC/UNU millennium project: Futures research method-
ology. Retrieved November 23, 2007, from http://www.futurovenezuela.org/_curso/5-delphi.pdf.
Green, H., Hunter, C., & Moore, B. (1990). The application of the Delphi technique in tourism. Annals
of Tourism Research, 17, 270ā279.
Habermas, J. (1990). Discourse ethics on a program of philosophical justiļ¬cation. In C. Lenhardt &
S.W. Nicholsen (Eds.), Moral consciousness and communicative action. Cambridge, MA: MIT
Press.
Hanaļ¬n, S., Brooks, A.-M., Carroll, E., Fitzgerald, E., Gabhainn, S.N., & Sixsmith, J. (2007).
Achieving consensus in developing a national set of child well-being indicators. Social
Indicators Research, 80, 79ā104.
18 H.M. Donohoe
Downloaded
by
[University
of
Florida],
[Holly
Donohoe]
at
08:26
04
January
2012
20. Hayes, T. (2007). Delphi study of the future of marketing of higher education. Journal of Business
Research, 60, 927ā931.
Holey, E.A., Feeley, J.L., Dixon, J., & Whittaker, V.J. (2007). An exploration of the use of simple
statistics to measure consensus and stability in Delphi studies. BMC Medical Research
Methodology, 7, 1ā6.
Honey, M. (1999). Ecotourism and sustainable development: Who owns paradise? Washington, DC:
Island Press.
Humberstone, B. (2004). Standpoint research: Multiple versions of reality in tourism theorising and
research. In J. Phillimore & L. Goodson (Eds.), Qualitative research in tourism: Ontologies,
epistemologies and methodologies (pp. 119ā136). London: Routledge.
Hurd, A., & McLean, D. (2004). An analysis of the perceived competencies of CEOs in public park
and recreation agencies. Managing Leisure, 9, 96ā110.
Jamal, T., Borges, M., & Stronza, A. (2006). The institutionalisation of ecotourism: certiļ¬cation,
cultural equity and praxis. Journal of Ecotourism, 5(3), 145ā175.
Kaynak, E., & Macauley, J.A. (1984). The Delphi technique in the measurement of tourism market
potential: The case of Nova Scotia. Tourism Management, 5(2), 87ā101.
Kaynak, E., & Marandu, E.E. (2006). Tourism market potential analysis in Botswana: A Delphi study.
Journal of Travel Research, 45, 227ā237.
Kincheloe, J.L., & McLaren, P. (2005). Rethinking critical theory and qualitative research. In N.K.
Denzin & Y.S. Lincoln (Eds.), The Sage handbook of qualitative research (3rd ed.,
pp. 303ā342). Thousand Oaks, CA: Sage Publications, Inc.
Kuo, N.-W., & Chiu, T. (2006). The assessment of agritourism policy based on SEA combination with
HIA. Land Use Policy, 23, 560ā570.
Landeta, J. (2006). Current validity of the Delphi method in social sciences. Technological
Forecasting and Social Change, 73, 467ā482.
Linstone, H.A. (1978). The Delphi technique. In J. Fowles (Ed.), Handbook of futures research.
London: Greenwood Place.
Linstone, H.A., & Turoff, M. (1975). The Delphi method: Techniques and applications. London:
Addison-Wesley Publishing Company, Inc.
Linstone, H.A., & Turoff, M. (2002). General applications: Introduction. In M. Turoff & H.A.
Linstone (Eds.), The Delphi approach: Techniques and applications (pp. 71ā79). Reading,
MA: Addison-Wesley Publishing Company Inc. Retrieved November 11, 2007, from http://
www.is.njit.edu/pubs/delphibook.
Lloyd, J., La Lopa, J.M., & Braunlich, C.G. (2000). Predicting changes in Hong Kongās hotel industry
given the change in sovereignty from Britain to China in 1997. Journal of Travel Research, 38,
405ā410.
McCool, S.F., & Moisey, R.N. (Eds.). (2001). Tourism, recreation and sustainability: Linking culture
and the environment. Wallingford: CABI Publishing.
Miller, G. (2001). The development of indicators for sustainable tourism: Results of a Delphi survey
of tourism researchers. Tourism Management, 22, 351ā362.
MoeĢller, G.H., & Shafer, E.L. (1994). The Delphi technique: A tool for long-range travel and tourism
planning. In J.R.B. Ritchie & C.R. Goeldner (Eds.), Travel, tourism and hospitality research: A
handbook for managers and researchers (2nd ed., pp. 473ā480). New York: John Wiley & Sons,
Inc.
Mullen, P.M. (2003). Delphi myths and reality. Journal of Health Organization and Management,
17(1), 37ā52.
Murray, T.J. (1979). Delphi methodologies: A review and critique. Urban Systems, 4, 153ā158.
Needham, R.D., & de LoeĢ, R. (1990). The policy Delphi: Purpose, structure, and application. The
Canadian Geographer, 34(2), 133ā142.
Nyiri, P. (2006). Scenic spots: Chinese tourism, the state, and cultural authority. Seattle, WA:
University of Washington Press.
OāConnor, P., & Frew, A.J. (2004). An evaluation of methodology for hotel electronic channels of
distribution. Hospitality Management, 23, 179ā199.
Okoli, C., & Pawlowski, S.D. (2004). The Delphi method as a research tool: An example, design
considerations and applications. Information and Management Systems, 42, 15ā29.
Pan, S.Q., Vega, M., Vella, A.J., Archer, B.H., & Parlett, G.R. (1995). Mini-Delphi approach: An
improvement on single round techniques. Progress in Tourism and Hospitality Research, 2(1),
27ā39.
Journal of Ecotourism 19
Downloaded
by
[University
of
Florida],
[Holly
Donohoe]
at
08:26
04
January
2012
21. Phillimore, J., & Goodson, L. (2004). Progress in qualitative research in tourism. In J. Phillimore & L.
Goodson (Eds.), Qualitative research in tourism: Ontologies, epistemologies and methodologies
(pp. 3ā29). London: Routledge.
Powell, C. (2003). The Delphi technique: Myths and realities. Journal of Advanced Nursing, 41(4),
376ā382.
Rowe, G., & Wright, G. (1999). The Delphi technique as forecasting tool: Issues and analysis.
International Journal of Forecasting, 15, 353ā375.
Sackman, H. (1975). Delphi assessment, expert opinion, forecasting, and group processes. Santa
Monica, CA: Rand Corporation.
Saukko, L. (2005). Methodologies for cultural studies. In N.K. Denzin & Y.S. Lincoln (Eds.), The
Sage handbook of qualitative research (3rd ed., pp. 343ā356). Thousand Oaks, CA: Sage
Publications, Inc.
Schmidt, R. (1997). Managing Delphi surveys using non-parametric statistical techniques. Decision
Sciences, 28(3), 763ā774.
Seely, R.L., Iglarsh, H., & Edgell, D. (1980). Utilising the Delphi technique at international confer-
ences: A method for forecasting international tourism conditions. Travel Research Journal, 1,
30ā35.
Sofeild, T. (2007). Tourism and natural landscapes: Universal principles or culturally determined
values? In H.M. Donohoe & W.J. Li (Eds.), Leisure in context: Implications for human develop-
ment and well-being (pp. 150ā159). Hangzhou: Zhejiang University Press.
Spenceley, A. (2005). Nature-based tourism and environmental sustainability in South Africa. Journal
of Sustainable Tourism, 13(2), 136ā170.
Stark, J.C. (2002). Ethics and ecotourism: Connections and conļ¬icts. Philosophy and Geography,
5(1), 101ā113.
Sunstein, C.R. (2006). Infotopia: How many minds produce knowledge. Oxford: Oxford University
Press.
Tsaur, S., Lin, Y., & Lin, J. (2006). Evaluating ecotourism sustainability from the integrated perspec-
tive of resource, community and tourism. Tourism Management, 27, 640ā653.
Vaugeois, N., Rollins, R., Hammer, K., Randall, C., Clark, B., & Balmer, K. (2005). Realizing
improved knowledge transfer in the leisure industry: A new agenda and strategies to link research-
ers and professionals, Travel and Tourism Research Association Canada Chapter Conference
Proceedings, Kelowna, BC, Canada.
Vivanco, L. (2002). Escaping from reality. The Ecologist, 32(2), 26ā30.
Weaver, D.B., & Lawton, L.J. (2007). Twenty years on: The state of contemporary ecotourism
research. Tourism Management, 28, 1168ā1179.
Weber, K., & Ladkin, A. (2003). The convention industry in Australia and the United Kingdom: Key
issues and competitive forces. Journal of Travel Research, 42, 125ā132.
Wolfers, J., & Zitzwitz, E. (2004). Prediction markets. Journal of Economic Perspectives, 18(2),
107ā126.
Wright, T.S.A. (2006). Giving āTeethā to an environmental policy: A Delphi study at Dalhousie
University. Journal of Cleaner Production, 14, 761ā768.
Yeong, Y.W., Keng, K.A., & Leng, T.L. (1989). A Delphi forecast for the Singapore tourism industry:
Future scenario and marketing implications. European Journal of Marketing, 23(11), 15ā26.
20 H.M. Donohoe
Downloaded
by
[University
of
Florida],
[Holly
Donohoe]
at
08:26
04
January
2012