WHAT IS A BENCHMARK?

Benchmark
From Wikipedia, the free encyclopedia.
A benchmark is a point of reference for a measurement. The term presumably originates from the practice of making dimensional height measurements of an object on a workbench using a graduated scale or similar tool, and using the surface of the workbench as the origin for the measurements.


--------------------------------------------------------------------------------
In surveying, benchmarks are landmarks of reliable, precisely-known altitude, and are often man-made objects, such as features of permanent structures that are unlikely to change, or special-purpose "monuments", which are typically small concrete obelisks, approximately 3 feet tall and 1 foot at the base, set permanently into the earth.


--------------------------------------------------------------------------------

In computing, a benchmark is the result of running a computer program, or a set of programs, in order to assess the relative performance of an object, by running a number of standard tests and trials against it. The term is also commonly used for specially-designed benchmarking programs themselves. Benchmarking is usually associated with assessing performance characteristics of computer hardware, e.g., the floating point operation performance of a CPU, but there are circumstances when the technique is also applicable to software. Software benchmarks are, for example, run against compilers or database management systems.

Benchmarks provide a method of comparing the performance of various subsystems across different chip/system architectures.

As computer architecture advanced, it became more and more difficult to compare the performance of various computer systems simply by looking at their specifications. Therefore, tests were developed that could be performed on different systems, allowing the results from these tests to be compared across different architectures. For example, Intel Pentium 4 processors have a higher hertz rating than AMD Athlon XP processors for the same computational speed, in other words a 'slower' AMD processors could be as fast on benchmark tests as a higher hertz rated Intel processors.

Benchmarks are designed to mimic a particular type of workload on a component or system. "Synthetic" benchmarks do this by specially-created programs that impose the workload on the component. "Application" benchmarks, instead, run actual real-world programs on the system. Whilst application benchmarks usually give a much better measure of real-world performance on a given system, synthetic benchmarks still have their use for testing out individual components, like a hard disk or networking device.

Computer manufacturers have a long history of trying to set up their systems to give unrealistically high performance on benchmark tests that is not replicated in real usage. For instance, during the 1980s some compilers could detect a specific mathematical operation used in a well-known floating-point benchmark and replace the operation with a mathematically-equivalent operation that was much faster. However, such a transformation was rarely useful outside the benchmark. Manufacturers commonly report only those benchmarks (or aspects of benchmarks) that show their products in the best light. They also have been known to mis-represent the significance of benchmarks, again to show their products in the best possible light. Taken together, these practices are called bench-marketing.

Users are recommended to take benchmarks, particularly those provided by manufacturers themselves, with ample quantities of salt. If performance is really critical, the only benchmark that matters is the actual workload that the system is to be used for. If that is not possible, benchmarks that resemble real workloads as closely as possible should be used, and even then used with skepticism. It is quite possible for system A to outperform system B when running program "furble" on workload X (the workload in the benchmark), and the order to be reversed with the same program on your own workload.

Benchmarking

Benchmarking is a process used in management and particularly strategic management, in which businesses use industry leaders as a model in developing their business practices. This involves determining where you need to improve, finding an organization that is exceptional in this area, then studying the company and applying it's best practices in your firm. Benchmarking systematicly studies the absolute best firms, then uses their best practices as the standard of comparison, a standard to meet or even surpass.

Benchmarking recognizes that no company is exceptional at everything. That is why it is an ongoing process involving firms from any industry and any country. It is not a one-shot event. There is no room for complacency. Benchmarking requires that you constantly search for better solutions. The rationnale is, If you continuosly search for best practices in the best firms around the world, you should become an exceptional company.

Every function and task of your business can be benchmarked, from production, to marketing, to purchasing, to information technology management, to customer service.

Some authors call benchmarking "best practices benchmarking" or "process benchmarking". This is to distinguish it from what they call "competitive benchmarking". Competitive benchmarking is used in competitor analysis. When researching your direct competitors you also research the best company in the industry (even if it serves a different location or market segment and is therefore not a direct competitor). This benchmark company is then used as a standard of comparison when assessing your direct competition and yourself.

A process similar to benchmarking is also used in technical product testing and in land surveying. See the article benchmark for these applications.

Contents
1 Procedure

2 Cost of benchmarking

3 See also

4 Lists of related articles

5 References


Procedure
1) Identify your problem areas - Because benchmarking can be applied to any business process or function, a range of research techniques may be required. They include: informal conversations with customers, employees, or suppliers; exploratory research techniques such as focus groups; or indepth marketing research, quantitative research, surveys, questionnaires, reengineering analysis, process mapping, quality control variance reports, or financial ratio analysis.

2) Identify organizations that are leaders in these areas - Look for the very best in any industry and in any country. Consult customers, suppliers, financial analysts, trade associations, and magazines to determine which companies are worthy of study.

3) Study their best practices - An initial study can be done at a good university library or online. This will give you an overview, however more detailed information will require an in-person visit. Phone the CEO and ask if a group of your managers and employees can visit their operations for an hour. Be forthright as to the purpose of the visit. Most CEOs will be flattered and agree to the request. Make it clear that any information obtained from the visit will be shared with them. Determine what subject areas will be off-limits. Ask if camera or video recorders are acceptable. Prepare two lists well in advance: a list of your objectives, and a list of questions. Choose 2 to 5 visitors, people that are closest to the issue, that will be responsible for implementing any recommendations, and cover a broad range of functional responsibilities. Occasionally an outside consultant is included in the visit team so as to provide an alternative perspective. Meet with your employees to explain the purpose of the visit and assign one or two questions to each employee. Explain what subject areas are off limits. Ask them to think about how the visit could benefit their area, and ask them to device more questions. Stay away from questions that could cause legal problems (eg., price fixing or new product development). Send a confirmation letter one week before the visit stating the date, time, and location of the visit, the number of visiters and their positions, your objectives, and a list of possible questions. Visits are typically 1 to 3 hours long. When at the site, provide a token gift to show that you appreciate the opportunity, keep focused on your objectives, give praise where it is due, and do not criticize. Look for anything remarkable or unexpected. As soon as you get back to your office (or hotel), have an immediate debriefing. Discuss what you have learnt and how you can apply it. Make sure that every visitor has an action plan detailing how they will be implementing the new information in their job. Some formal analysis (such as process mapping) of the benchmarked process may be necessary. After several weeks, phone back the CEO to express your appreciation and give concrete examples of how the knowledge gained from the visit will be used in your company. Send them a copy of any written reports about the visit before they are distributed. This allows them to correct inaccuracies and modify sensitive or propriortory information.

4) Implement the best practices - Delegate responsibility for actions to individuals or cross-functional teams. Set measurable goals that are to be accomplished within a specified time frame. Monitor the results. Get key personnel to give you a brief (one page) summary of how the immplementation is proceeding. Spread the information through out the entire organization.

5) Repeat - Benchmarking is an ongoing process. Best practices can always be made better.


Cost of benchmarking
Benchmarking is a moderately expensive process, but most companies find that it more than pays for itself. The three main types of costs are:

Visit costs - This includes hotel rooms, travel costs, meals, a token gift, and lost labour time.
Time costs - Members of the benchmarking team will be investing time in researching problems, finding exceptional companies to study, visits, and implementation. This will take them away from their regular tasks for part of each day so additional staff might be required.
Benchmarking database costs - Organizations that institutionalize benchmarking into their daily procedures find it is useful to create and maintain a database of best practices and the companies associated with each best practice.