eKC Home | Certification Program

We will review the importance of different stakeholder perspectives in measuring knowledge and describe the metrics that can be utilized in processes of knowledge production, transfer, acquisition, management and claim.

You will learn to:

identify the measurement perspectives of different organizational roles
recognize the role metrics plays in knowledge management
understand measures in knowledge production
recognize possible measures in knowledge transfer and knowledge acquisition
interpret metrics for knowledge management

Knowledge management is a strategic initiative.

This way of thinking is often a paradigm shift for senior management. Most do not see a connection between KM and strategic planning. Senior management needs to realize that KM is already happening within their organization -- in an ad hoc way, for better or for worse.

By creating a knowledge plan, tying it into the strategic plan, and then redesigning and managing the firm's knowledge processes, a firm has the potential of out-changing, out-performing, and out-maneuvering the competition.

Although KM is as old as the first tribe, the idea of developing a knowledge plan to govern a company's Knowledge Management is relatively new. The strategic goals of the team or organization drive the plan, and fulfillment of that plan requires a method for improving the knowledge processes.

Chief Financial Officers are the collectors of the mission statement, goals, business plans, financial plans, etc. Their interests center on the costs of KM and the savings that can be realized.

A business owner, large or small, is interested in KM from a variety of perspectives as, on a much larger scale, shareholders of a publicly held large industry are. His or her view may center on:

Earnings per share
Return on Investment (ROI)
Return on Equity (ROE)
Growth rank in the industry
Total shareholder return
Market share
Competitiveness

Market value
Return on assets
Value-added per employee
Revenue per employee
New product sales
Revenue growth

Customers are usually looking for quality in goods or services, getting their needs satisfied, or perhaps increasing the speed of the service delivery.

Employees are also looking for quality in the workplace environment. They want to work in an empowered environment where their talents and needs get recognized and utilized. Employee concerns also center on health, finances, status, time with family and friends, and responsibility, all of which will be satisfied in a thriving, successful, growing, and innovative business.

These “satisfiers” for the various stakeholders can all be the beginnings of a metrics design that will measure successful knowledge production and innovation.

Measurement and Metrics

Metrics are numbers or units. Examples include inches, minutes, numbers sold, dollars, etc. Metrics are used to measure things or compare two or more things. Metrics can be used to measure such things as size, speed, sales, costs, etc.

Metrics allow their user to test the validity of their product or the validity or our knowledge claims about Knowledge Management. Measurements can lead to greater understanding. The use of metrics separates snake-oil salesmen from those who are genuinely trying to develop KM instruments and tools. If you can't measure the success or failure of something based on clear metrics, you will have difficulty "selling" it to your boss or others. However, creating metrics for the sake of creating metrics has no value. The value of metrics is in the fulfillment of its function. Metrics need to be revisited, improved, discarded, or replaced on a regular basis.

In KM, metrics can be designed to measure four things about knowledge claims:

Number of new knowledge claims
Rate at which they are being generated
Quality of the claims
Economic impact of knowledge

Meta-metrics allows us to determine the quality of the metrics we are using.

Business Knowledge Metrics

There are many varied metrics that can be utilized in the knowledge process. We arrive at metrics after goals have been set. A metric measures a result against the goal for that result. Typically in the business world, a positive change to the bottom line is the most common and important metric.

In a simplified example, the goal of an engineering department is to cut in half the number of steps needed to assemble a widget; the steps are actually cut from 10 down to 7. The measurement of that metric (each step) indicates that the success rate is 60% (a reduction of 3 instead of 5).

A second goal might be to reduce the cost of making that widget from $1.50 to $1.00 (1/3 reduction). In this case, the three-step cut (from above) resulted in an actual reduction in cost of 1/2, or a reduction from $1.50 to $0.75. Therefore, the metric (cost) measurement would be looking for a 33% reduction, and it would find a 50% reduction, for a success rate of 150%.

In both cases, the metric measures the result of the activity against what the anticipated outcome is. These metrics are clear and can be accurate because they are based only on numbers.

Metrics designed to measure the success of Knowledge Management goals are, in most cases more difficult to write and analyze because so much of KM does not deal only in numbers. Still, it is possible to design successful and meaningful metrics for the field of KM. Suggested metrics for each area of KM are found below.

Knowledge Production

The ultimate goal of knowledge production is increasing the rate of successful innovations with the same or fewer resources. These are some metrics that could be used to measure success in knowledge production.

Number of successful ideas (ideas that result in a profitable product) vs. cost (time and money)
Rate of idea generation
Idea quality (knowledge quality)
Originality
Rate of developing new techniques
Cost of developing these new knowledge claims

An innovation can be the improvement or invention of anything -- things and processes. A bread machine can be an innovation. Reducing the steps in creating a bread machine is also an innovation which can be measured metrically. Both can have a positive impact on the success of the business process.

Knowledge Transfer or Diffusion

Knowledge transfer requires a knowledge broadcaster and a knowledge receiver. An instructor engages in knowledge transfer. A good instructor is one who transfers knowledge to the greatest number of people in the shortest period. Metrics used to gauge success might include:

Successfully passing a test on the material being taught
Speed at which trainees can do new tasks

Again, it is important to remember that metrics mean nothing if they are not tied to specific goals for an activity or process.

Knowledge Acquisition

Ways to acquire business knowledge include: purchasing a company to gain access to the knowledge in its documents and employees, research and imitation, corporate intelligence about another company's product or techniques, reverse-engineering a product, and imitating best practices from other companies or other teams within your company. Some metrics regarding knowledge acquisition include:

Rate of acquisition
- Speed at which an engineering team can reverse-engineer a product and replicate it.
- Rate at which a person or collective acquires knowledge from a new source or purchased company
- Proof that knowledge has been acquired successfully
Quality of the source -- Book, record, gadget, painting, blueprint, person, etc.
Accuracy of the source -- Person, records, etc.

Retrieval time

Knowledge Management

Having weekly lessons-learned meetings, increasing the quality of dialogue within teams and the organization, using an Intranet to share knowledge -- these are some KM initiatives that might be employed in an organization. They, too, can be measured against metrics.

Once again, KM metrics must link to the fulfillment of a goal that is meaningful to the organization. In the case of a business or business process, this typically means the financial balance sheet.

Managers will not spend thousands of dollars on KM if you cannot show them that they will get that money back, and then some, within a reasonable period.

Does an increase in the rate of knowledge transfer have a positive impact on the bottom line? Does the change in the rate of idea generation have an impact on the bottom line?
If so, how much?
Was the cost it took to improve knowledge transfer justified?
By training the members of the team to improve communication and listening skills, was concept generation increased?
How effectively did the new help desk software increase the satisfaction of the user in finding an answer, in contrast to talking to a human directly?
How much time did it take compared to calling an expert?
Does the help desk justify its cost?

Knowledge Claims

Knowledge claims have a distinct set of metrics. The quality of a knowledge claim is based on all of the following:

Leverage -- the ability of actionable knowledge to leverage on other knowledge within the agent to help the agent successfully act in its environment with the least resources.
Understandability -- The ability to comprehend.
Parsimony Measure -- Minimum number of symbols and/or words needed to convey the knowledge to the receiver.
Predictive Power -- How far in the future can this knowledge claim predict events accurately?
Explanatory Power -- The ability to explain the existence of a thing or event.
Predictive Accuracy -- Time/Accuracy ratio.
Effectiveness -- Helping a person or organization respond to conditions in the environment
Originality -- Was this a primary or secondary source? If it was primary, how unique (original) is the knowledge claim?

And, finally, one successful metric for knowledge production is benchmarking. Business processes, knowledge processes, and KM processes can be benchmarked and compared to other businesses or other industries.

Next module