“Measuring customer satisfaction in the context of a project-based organization”

Tuomas J. Ahola, Helsinki University of Technology Jaakko Kujala, Helsinki University of Technology
ABSTRACT

There is a lack of research that focuses on the suitability of the concept of customer satisfaction
and the current methods used for measuring it in organizations operating in business-to-business
(B2B) markets, such as project-based organizations (PBO), which typically provide customized
product and service offerings to a rather limited customer base. In this theoretical paper, we
discuss how certain characteristics of PBOs should be taken into consideration when measuring
customer satisfaction. In addition, we discuss the suitability of different customer satisfaction
measurement (CSM) methods for PBOs. We argue that PBOs can expect to benefit from survey-
based CSM methods. However, survey-based methods do not often suffice alone and standard
instruments should not be directly transferred from a B2C context to a B2B context. In addition,
we argue that the match between the selected CSM methods and the salient features of the PBO
in question should always be considered.

customer loyalty which is vital for long-term revenues (Gronholdt et. al, 2000; Lam et al.,
2004).
• CSM enables the supplying organization to compare the performance of its different
business units in different time periods and locations (Jones & Sasser, 1995).
(2/37)
• CSM is useful for assessing the effectiveness of efforts to redesign elements of the service
delivery system (Chase & Bowen, 1991; Juran & Gryna, 1988).
• Customer satisfaction can be used as a basis for customer segmentation
(Athanassopoulos, 2000).
• According to McColl-Kennedy and Schneider (2000), measuring customer satisfaction is
not a neutral act, but an intervention. The opinions of the customer whose satisfaction is
measured can be affected by the measurement process
• CSM can be used by the supplier as a symbolic activity for demonstrating customer-
oriented behaviour (Kujala & Ahola, 2004).

Striving for high customer satisfaction can be considered important, since increased customer
satisfaction has been linked to customer behaviour that positively affects business results (e.g.
Kotler, 1994; Rust & Oliver, 1994; Vavra, 1997; Lam et al., 2004).

We did not find any academic papers focusing on CSM in a project-based B2B context and only
a rather limited number of papers that focused on CSM in a general B2B context. We presume
that in practice, many project-based organizations among other organizations operating in B2B
markets are measuring customer satisfaction with methods developed for B2C market contexts
without clearly understanding that several of the underlying assumptions inherent in these
methods are not valid in a B2B context (Rossomme, 2003; Kujala & Ahola, 2004). According to
Piercy (1996), this may lead to considering CSM as a superficial and unnecessary activity which
is likely to limit the utilization of the CSM results.

(3/37)
The research focus in this paper is to point out the key differences between traditional, product or

question has great implications towards measuring the satisfaction of the customer.

One should also consider how the concept of satisfaction is defined. Oliver (1997) defines
satisfaction as “pleasurable fulfillment”. He further links this fulfillment to an act of consumption
by the consumer. Oliver’s, like most existing definitions of satisfaction, view satisfaction as a
function of what is expected and what is experienced by the customer. But as stated earlier, the
customer is not clearly defined in a B2B context. It is important to differentiate whether CSM is
focusing on the satisfaction of the customer organization as a whole or the satisfaction of certain
individuals within that organization. These are clearly two different issues, since in the context of
a project delivery, there are several individuals involved from both the buyer and seller sides.
Another issue is the experience of consumption linked to most definitions of satisfaction. In a
B2B project delivery, not all individuals from the purchasing organization that are involved in the
project participate directly to the consumption, or use of the delivered project.

(5/37)
It is rather straightforward to reason that in a project-based B2B context, to ensure the
continuation of a business relationship, the supplier needs to keep several individuals from the
customer’s side satisfied. However, it is also typically the case that certain individuals from the
purchasing organization have more influencing power in project purchase decisions
1
, and thus
their satisfaction matters more that the satisfaction of those individuals who do not possess such
power (Silk & Kawani, 1982). Therefore, it can be argued that the aggregate satisfaction of the
buying organization is a rather complex function of satisfactions of individual, from an
importance viewpoint, non-equal persons within that organization. To contrast this to a B2C
context, there it is often enough if the satisfaction of one individual can be ensured to keep the
business relationship alive.

Several researchers argue that in a B2B context, measuring the satisfaction of one key informant
suffices (e.g., Pennings, 1979; Phillips, 1981; Conant et al., 1990). It can be argued that in a

organization is the nature of the organization’s offering. In a B2C context, the offering is
typically either a product or service and thus the CSM typically focuses on measuring the
customers satisfaction towards some elements (or attributes) of this offering. Which attributes
should be measured and how each attribute’s relative importance should be calculated is under
continuous scientific debate. In the context of B2B project deliveries, the offering often contains
a complex product (or hardware) component, and a service component including all the
interpersonal interactions related to conducting business between two organizations. The complex
technical nature of project product, which is intended for generating revenues for customer over
long period of time, makes it difficult for the customer to give an informed opinion how satisfied
(7/37)
he is to specific product features. When comparing B2B project-deliveries to B2C product or
service offerings, it is in the former case even less clear, which attributes should be measured and
which of these two attribute-laden components (hardware and service) is more important towards
the aggregate satisfaction of the customer.

According to Granovetter (1985), embeddedness, or social bonding has great implications for
economic activity. Individual persons do not make purchase decisions solely by technical
reasoning, but interpersonal social ties significantly affect these decisions. Interpersonal social
ties and trust play a greater role in interorganizational economic exchange than they do in B2C
commerce, since B2B exchanges tend to involve significant amounts of money, risk, and persons.
According to (Crosby et al., 1990), interpersonal relationships may be strengthened by
investments to social resources such as respect, concern and advice, and according to Bolton et
al. (2003) social bonds may even be stronger than structural bonds in an interorganizational
business relationship. Social bonding between individuals is linked to past experiences between
them, and one component of these experiences is the satisfaction towards the behaviour of the
other person. Following this line of reasoning, one can argue that in B2B commerce, customer
satisfaction is affected significantly by factors that are not directly linked to the product or service
component of the offering, but to how persons from two different organizations get along. These
social bonds play a very crucial role in project-based business, since the definition of the project
is often influenced via negotiations (Cova et al., 2002). If the chemistry is not there, for example,

normally in excess of 50 customers per surveyed segment. Surveys are used to assess the
satisfaction of customers towards physical products and intangible services and in addition to the
traditional B2C context, they are also utilized in a B2B context (Bearden et al, 1993; Kujala &
Ahola, 2004). Most academic papers have aimed at contributing to extant knowledge by defining
the various attributes of the product or service offering to measure (e.g. Parasuraman et al., 1985;
Bolton & Drew, 1991; Bitner et al., 1990), how they should be measured e.g. should you measure
the customer’s perception of the offering (Cronin & Taylor, 1992 & 1994) or the difference
between customers expectations and experiences (Parasuraman et al., 1985 & 1988 & 1991), and
to how the relative importance of these attributes should be calculated (e.g. Green & Tull, 1975).

Customer satisfaction surveys usually fulfil three organizational needs:

• They enable the organization to compare the performance of its different business units in
different time periods and locations (Jones & Sasser, 1995).
• They can produce rich information for generating continuous quality improvement
activities, when they are examined carefully and used within a consistent framework (Lin
& Jones, 1997).
(10/37)
• They are useful for assessing the effectiveness of efforts to redesign elements of the
service delivery system (Chase & Bowen, 1991; Juran & Gryna, 1988).

There are, however, some inherent problems regarding to survey-based CS measurement. Firstly,
the amount of attributes in a survey is always limited, i.e. a survey cannot contain all possible
attributes affecting customer satisfaction, because this would make the questionnaire very long,
severely limiting the amount of customers willing to complete it. Secondly, customers have to
aggregate their product or service consumption experiences in a potentially problematic way. For
example, when rating “the friendliness of staff”, a customer has to aggregate her answer is she
has encountered friendly and less friendly staff. Thirdly, the operationalization of the satisfaction
attributes is problematic, since customers always interpret the questions contained in the survey
in more or less different ways. This decreases the reliability of the results.

supplier have the advantage that the measures are developed with managerial knowledge and
expertise of the specific business context (Jeffries & Sells, 2000). However, their validity and
reliability is hard to verify. In addition, the results obtained with the use of customized surveys
cannot be compared with results obtained with standardized scales. This can make benchmarking
of CS difficult or impossible between organizations and if the supplier uses different customized
surveys for each of its sub-business units, comparison of CS may not even be possible between
them.
(12/37)

Critical incident technique (CIT) and sequential incident technique (SIT)

As a method for assessing customer satisfaction CIT (and SIT) allows the detailed description of
problematic customer incidents as the customers perceive them (Flanagan, 1954). CIT also
allows the categorization of the critical incidents into homogenous groups, and thus makes
limited generalization of the results possible. The underlying assumption behind critical incident
technique (CIT) is that customer satisfaction is strongly affected by very positive or negative
experiences in the product/service delivery and consumption processes. These two techniques
have been used primarily in the B2C service context (Bitner et al, 1990), but have also been
successfully applied in B2B contexts (Lockshin & McDougall, 1998; Franco et al., 2004). By
systematically documenting critical incidents, the supplier organization can gather valuable
information regarding to the events that have very significantly affected the satisfaction of its
customers. Sequential incident technique (SIT) is very similar, but it also records usual events
that do not affect customer satisfaction in the same degree than critical incidents do. Both
techniques use qualitative interviews to collect customer incidents. Collected data is then sorted
into groups containing similar incidents.

The main strengths of CIT and SIT are as follows:

• They have the ability to record incidents, which significantly affect customer satisfaction
in a manner that is very natural to the customers (Flanagan, 1954; Nyquist & Booms,


• They encourage the customer to discuss various elements and incidents affecting
customer satisfaction in great detail. This information can then be used by the supplier to
improve its operations.
• Because in a B2B project-delivery context, the supplier is typically in contact with several
customer personnel, interviews allow the supplier to gain a holistic view of how different
individuals perceive the supplier and which persons within the customer organization are
most important for aggregate satisfaction of the customer organization.
• They can promote interpersonal bonding between customer and supplier personnel.
• The interviewing situation involves two-way interaction and gives the supplying
organization the possibility to explain why it has behaved in certain ways. For example, a
person within the customer organization can be dissatisfied with the supplier organization,
but if the supplier is capable of giving a solid explanation of what has happened and why
as well as provide a solution that will correct the situation, a considerable positive impact
towards the satisfaction of the customer is possible.

A major drawback of using interviews for measuring CS is the method’s very limited ability to
produce generalizable results, since interview data that is not based on strictly structured, formal
interviews is so context dependent that it often cannot be generalized. Another key weakness of
(15/37)
the interview method is the fact that it is very resource consuming. Both the supplier and the
customer need to allocate a significant amount of their employees’ time to the process.

From an epistemological viewpoint, the customer satisfaction interview is based on an
interpretative or constructivist view of reality. The supplier aims to understand how satisfied
different individuals at the customer organization are, what are the reasons behind satisfaction or
dissatisfaction and how these different individuals, with differing possibilities to affect the
customer organization’s decision making, and their satisfaction contribute towards the
satisfaction of the customer organization as a whole. In this case, the measurement of CS includes
the identification of key individuals and their relative importance and the measurement /

Customer base Large Medium / small Medium / small
Sample size Statistically significant
proportion of customer
base
Limited sample of customer
base
Limited sample of customer
base
Sample
homogeneity
Homogenous segments
can be identified
Typically a group of
homogeneous customer
personnel (e.g. end-product
users)
Heterogeneous customer
base with differing needs
Analysis of
data
Quantitative Mostly qualitative – content
analysis, grouping incidents
into heterogeneous
categories
Qualitative
Typical
industry
context where
Mass-produced
consumer goods and

discuss and record incidents
Selection and skills of the
interviewers
Reliability of
results depends
on
Stability of measures
and response bias
The ability to generalize
incidents into homogeneous
categories
Selection of interviewers and
informants
Limitations -Limited number of
attributes
-Forced aggregation of
experiences
-Differing
interpretations of
survey questions
-Results can often not be
generalized
-Difficult to analyze data
with statistical methods
-Very time consuming for
supplier and customer
-Hard or impossible to
generalize results

Indirect measures of customer satisfaction

In the following, we aim to generate testable hypotheses based on our analysis of various CSM
approaches and special features of project-based organizations. These hypotheses are created
both to be empirically tested in further research, and to demonstrate which managerial
implications can be drawn from our analytical work. The hypotheses are formulated to be tested
in the context of B2B PBOs that receive a significant amount of their revenues from the delivery
of complex projects (e.g. power plants, ships, buildings, etc.) to external customers (other
organizations than the producing organization).

Project-based organizations generally have a rather limited number of customers, which makes
the use of standardized survey -based CSM and statistical analysis of the results difficult to apply.
Additionally, if the customer base can not easily be segmented into homogenous groups that have
similar characteristics, requirements and expectations, this further complicates the situation.
Further, project-based organizations, which rely solely on survey-based CSM, can be expected to
prefer the use of customized surveys, which may be even tailored on project basis. It should be
noted that in our hypotheses the implementation includes using information gathered in the
decision making process. Thus implementation implicitly includes also validity and reliability of
the information gathered, and how successful implementation has been from managerial point of
view.

(20/37)
H1: The implementation of standardized customer satisfaction surveys positively correlates with
the size of the customer base.

H2: The implementation of standardized customer satisfaction surveys positively correlates with
homogeneity of the customer base.

H3: The implementation of survey-based CSM as the only method for measuring customer
satisfaction positively correlates with the use of customized surveys.

Our review of different CSM methods and discussions about the differences between project-

personal relationships, which can be expected to make the use of interviews and direct contacts
with customer the most suitable way to measure customer satisfaction.

H6: The size, complexity and length of an average project delivery positively correlates with the
implementation of interview-based approaches for CSM.

If the supplier’s customer contact personnel are familiar with the critical incident technique (CIT)
or sequential incident technique (SIT), they are able to record and analyze incidents having direct
influence on customer satisfaction or process quality. Based on our review of academic papers,
(22/37)
we would expect that these two methods are less commonly known and deployed than survey and
interview methods. Thus, the use of these approaches can be expected to depend on how well
learning from and between projects is organized as a part of the quality management system of
the supplier organization. Individual project managers are focused on successful project
completion may have only a little interest on recording and analyzing incidents that do not
directly contribute to this target.

H7: Implementation of CIT and/or SIT positively correlates with front line employee quality
training and pervasiveness of the quality management system.

An important advantage of CIT and SIT is the usability of the data gathered with them for
improving service levels (Edvardsson & Roos, 2001). This is supported by Durvasula et al.
(2000), who discovered that effective service recovery, such as problem handling and complaint
handling is associated with high levels of customer satisfaction.

H8: Implementation of CIT and/or SIT positively correlates with customer satisfaction towards
the services provided by the supplier.
2

One of the differences between project work and other industrial services is the degree of

operating. Institutional perspective (Scott, 2001) complements traditional research approaches by
focusing on the role of customer satisfaction surveys as institutionalized practice, which takes
into account the symbolic value of customer surveys. The technical value of survey methodology
(24/37)