From page 219 of Virtual Economies: Design and Analysis by the MIT Press:
Quote:
The virtual scorecard provides a great starting point for understanding the management of a live operation, but it is not the full picture of a virtual economy. The reason is that complex virtual economies are not the same thing as firms with hierarchies and chains of command. Nor are they simply e-commerce websites with customer-supplier interactions that can be accurately captured with measures such as ARPU and time spent. Virtual economies are economies, which means that they can play host to complex networks of interactions not easily captured by the kinds of measures normally used in performance management and web analytics.
An important case in point is measuring and acting on the extent to which the virtual economy is providing users with fun experiences (what we call content). In the virtual scorecard example, we measured the time spent in the market interface and assumed that more time spent implies more entertainment gained from the economy. A better measure would be a regular survey that asks a panel of users to rate how much fun they are having in the economy. This is more expensive but still doable and part of standard performance management. But here's the real problem: What if the measure shows that satisfaction with the economy is going down? How do we know what is going wrong and where to intervene?
We could always just ask the users where the problem is and apply a fix there. For example, new users in the online community Gaia Online were complaining that they didn't have enough virtual currency to afford the items traded on Gaia's user-to-user marketplace. The publisher heard the complaints and responded by giving the users more currency. But all this meant was there was now even more currency competing for the same number of items, driving goods prices even higher. New users were left almost as disappointed as before.
A better way to manage the health of a complex virtual economy is to track economic indicators, understand their causal relationships with each other as well as KPIs such as user satisfaction, and carry out interventions where appropriate. Economic indicators are measures that describe an aspect of an entire economy as a number. Economic indicators can also be calculated for specific segments of the economy, such as user cohorts or goods categories.
An important case in point is measuring and acting on the extent to which the virtual economy is providing users with fun experiences (what we call content). In the virtual scorecard example, we measured the time spent in the market interface and assumed that more time spent implies more entertainment gained from the economy. A better measure would be a regular survey that asks a panel of users to rate how much fun they are having in the economy. This is more expensive but still doable and part of standard performance management. But here's the real problem: What if the measure shows that satisfaction with the economy is going down? How do we know what is going wrong and where to intervene?
We could always just ask the users where the problem is and apply a fix there. For example, new users in the online community Gaia Online were complaining that they didn't have enough virtual currency to afford the items traded on Gaia's user-to-user marketplace. The publisher heard the complaints and responded by giving the users more currency. But all this meant was there was now even more currency competing for the same number of items, driving goods prices even higher. New users were left almost as disappointed as before.
A better way to manage the health of a complex virtual economy is to track economic indicators, understand their causal relationships with each other as well as KPIs such as user satisfaction, and carry out interventions where appropriate. Economic indicators are measures that describe an aspect of an entire economy as a number. Economic indicators can also be calculated for specific segments of the economy, such as user cohorts or goods categories.
Italics are from the original; boldface, mine
