Redefining IT Performance and IT Effectiveness

Performance refers to the ability to acquire resources necessary for organizational survival.  Organizational performance results from a combination of industry or environmental conditions, the strategy that an organization’s decision makers choose, and the structure in place to support the strategy.  Performance is a proxy measure that indicates legitimacy by resource suppliers and perceived organizational effectiveness. Performance measurement consists of an assessment tool to measure effectiveness, provides information to managers for decision-making, and helps them to analyze organizational efficiency at the operative and strategic levels. 

Performance issues relate to the disciplines of general systems theory elaborated by Ludwig von Bertalanffy in the 1920s.  This theory included various disciplines such as behavioral science (sociology), economics (management accounting), information technology, mathematics (operations research), and organization theory.  These core disciplines for agency and management theories form a suitable umbrella for HR management, public administration, and management control systems. The literature on management philosophies provides an examination of these disciplines with an emphasis on corporate culture and power, Taylor’s scientific management, Mayo’s humanistic management, or quality management.

From a performance standpoint, three major components relate to management and agency theories: analysis, evaluation, and measures.  Several methods facilitate a performance analysis: expert systems, data mining, factor analysis, geographic information systems, ratio analysis, statistical regression, structural equation modeling, and productivity theory (data envelopment analysis, total factor productivity, and stochastic frontier analysis).  Whereas performance measures result from various frameworks such as the balanced scorecard and performance pyramid, performance evaluation includes strategic management issues that cover the alignment between incentive means of knowledge workers and corporate strategic goals and processes.

Early traditional frameworks of organizational performance such as Du Pont’s pyramid of financial measures (1920s) were a single all-encompassing approach. Du Pont’s framework for example, only focuses on financial performance. In contrast, emerging tools integrate the complexity and dynamic aspects of organizations by considering various dimensions of performance.  In this line of reasoning, performance measurement covers processes, system dynamics, and strategies that characterize business.   

Even though the overall performance of the information systems function seems to be difficult to conceptualize and measure, two approaches can be distinguished in research into the business value of IT investments: variance and process approaches. The variance approach focuses on the relationship between IT investments and organizational performance by considering financial measures such as lower costs, higher revenues, and improved market share.  The variance approach examines the “what” question: What is the relationship between IT investments and organizational performance? In contrast, the process approach focuses on the “how” question: How do IT investments improve organizational performance?

The process approach combines the returns on investments with process and organizational changes. The process approach analyzes the impact of IT on organizations from efficiency, effectiveness, and strategic IT alignment standpoints.  IT efficiency is the IS function highlighting the relationship between IT expenditures (or IS capabilities) and IT assets (or IS function outputs such as systems performance, information effectiveness, and services performance).  IS capabilities are inputs such as hardware, software, human skills, and management processes that serve to translate IT expenditures into IT assets.  Researchers use various metrics to assess IT efficiency: availability of systems and applications, number of help desk tickets, mean time between failure, and license usage.  These metrics comment on efficiency of systems, applications, and networks, unlike other performance variables that focus on engineering performance.

Prior scholars pointed out the limitation of IT efficiency measurements to assess IT effectiveness.  The influence of IT on organizations moves gradually from an efficiency production factor toward the maximization of the business value of IT investments (or IT effectiveness).  Enterprises generally invest on IT for two reasons: (a) to capture information to support corporate processes, and (b) to enable business change.  These scholars advised that the contribution of IT is to be specific (by supporting defined business processes) and generic (by enabling undefined business change).  They added that the measurement models of the IT business value should differ from performance models but close to capability models.

Within the context of strategic IT planning, some of the prior research attempted to investigate the linkages between IT investment projects and the associated business value using selected financial measures related to performance and productivity.  Some other studies attempted to measure the business impact of IT in organizations by market expansion, cost avoidance, customer value, efficiency, and profitability.  Some other research compared two analytical models (linear and nonlinear) and two conceptual (contingency-based and resource-centered) frameworks to assess the business value of IT using both financial objectives (expense and revenue) and perceived measures (firm’s perceived profitability).

Drawing upon the theoretical input-output model, Chang and King (2005) developed an instrument that explored the role of the IS function on business process effectiveness and organizational performance. Silvius (2006) proposed a multivariate value framework to assess the impact of IT on an organization. Yeniyurt (2003) proposed a performance measurement framework for global corporations drawing upon methods involving both financial and non-financial variables such as Skandia navigator, economic value added, and balanced scorecard. Yeniyurt’s non-financial variables for the organizational performance and effectiveness construct are customer satisfaction, innovation, internal processes, and organizational culture and climate.

The research on IT project planning process can be subdivided into strategic and operational perspectives. Some research on IT project planning explored the strategic aspects and the identification of projects that match with corporate objectives. Some other studies focused on the analysis and selection of a project from several capital expenditures’ alternatives (or capital budgeting of IT investments). The traditional capital budget methods are based on the calculation of the cash flows input and outputs. Seven traditional budgeting models are used to evaluate capital projects: (a) payback method, (b) return on investment, (c) cost-benefit ratio, (d) profitability index, (e) net present value, (f) economic value added, and (g) internal rate of return.

These traditional capital budget methods are limited to valuate IT projects because of (a) their inability to cope with risk, uncertainty, and flexibility, (b) they overlook the cost to train users, the learning curve to adapt to new technologies, and the socials subsystems costs and benefits of the IT projects, and (c) their inability to quantify intangible benefits such as improving knowledge, customer service, or decision making. These shortcomings are especially clear with IT investments done under conditions of uncertainty in today’s global economy which requires dynamic capabilities and strategic flexibility. The real option approach have been proposed as an alternative to the deterministic capital budget methodologies and the extension of the financial option theory to the options on real (non-financial) assets.

Some References:

Chang, J. C., & King, W. R. (2005). Measuring the performance of information systems: A functional scorecard. Journal of Management Information Systems, 22(1), 85-115.

Silvius, A. J. G. (2006). Does ROI Matter? Insights into the True Business Value of IT. Electronic Journal of Information Systems Evaluation, 9(2), 93-104.

Yeniyurt, S. (2003). A literature review and integrative performance measurement framework for multinational companies. Marketing Intelligence & Planning, 21(3), 134-142.