View source for Computer performance

{{short description|Amount of useful work accomplished by a computer}}
In [[computing]], '''computer performance''' is the amount of useful work accomplished by a [[computer system]]. Outside of specific contexts, computer performance is estimated in terms of accuracy, [[efficiency]] and speed of executing [[computer program]] instructions. When it comes to high computer performance, one or more of the following factors might be involved:
* Short [[Response time (technology)|response time]] for a given piece of work.
* High [[throughput]] (rate of processing work tasks).
* Low utilization of [[computing resource]]s.
** Fast (or highly compact) [[data compression]] and decompression.
* [[High availability]] of the computing system or application.
* High [[Bandwidth (computing)|bandwidth]].
* Short [[data transmission]] time.
==Technical and non-technical definitions==
The performance of any computer system can be evaluated in measurable, technical terms, using one or more of the metrics listed above. This way the performance can be
* Compared relative to other systems or the same system before/after changes
* In absolute terms, e.g. for fulfilling a contractual obligation
Whilst the above definition relates to a scientific, technical approach, the following definition given by [[Arnold Allen (mathematician)|Arnold Allen]] would be useful for a non-technical audience:
<blockquote>''The word ''performance'' in computer performance means the same thing that performance means in other contexts, that is, it means "How well is the computer doing the work it is supposed to do?"''<ref>Computer Performance Analysis with Mathematica by Arnold O. Allen, Academic Press, 1994. ''$1.1 Introduction, pg 1.''</ref></blockquote>
===As an aspect of software quality===
[[Computer software]] performance, particularly [[software application]] response time, is an aspect of [[software quality]] that is important in [[human–computer interaction]]s.
== Performance engineering ==
Performance engineering within systems engineering encompasses the set of roles, skills, activities, practices, tools, and deliverables applied at every phase of the systems development life cycle which ensures that a solution will be designed, implemented, and operationally supported to meet the performance requirements defined for the solution.
Performance engineering continuously deals with trade-offs between types of performance.  Occasionally a [[CPU design]]er can find a way to make a [[Central processing unit|CPU]] with better overall performance by improving one of the aspects of performance, presented below, without sacrificing the CPU's performance in other areas.  For example, building the CPU out of better, faster [[transistor]]s.
However, sometimes pushing one type of performance to an extreme leads to a CPU with worse overall performance, because other important aspects were sacrificed to get one impressive-looking number, for example, the chip's [[clock rate]] (see the [[megahertz myth]]).
=== Application performance engineering ===
{{Main|Application performance engineering}}
Application Performance Engineering (APE) is a specific methodology within [[performance engineering]] designed to meet the challenges associated with application performance in increasingly distributed mobile, cloud and terrestrial IT environments. It includes the roles, skills, activities, practices, tools and deliverables applied at every phase of the application lifecycle that ensure an application will be designed, implemented and operationally supported to meet non-functional performance requirements.
== Aspects of performance ==
Computer performance [[Software metric|metrics]] (things to measure) include [[availability]], [[Response time (technology)|response time]], [[channel capacity]], [[Latency (engineering)|latency]], [[completion time]], [[service time]], [[Bandwidth (computing)|bandwidth]], [[throughput]], [[relative efficiency]], [[scalability]], [[performance per watt]], [[Data compression|compression ratio]], [[instruction path length]] and [[speed up]]. [[CPU]] benchmarks are available.<ref>{{citation|title=Measuring Program Similarity: Experiments with SPEC CPU Benchmark Suites|year=2005 |pages=10–20 |citeseerx=10.1.1.123.501 }}</ref>
=== Availability ===
{{Main|Availability}}
Availability of a system is typically measured as a factor of its reliability - as reliability increases, so does availability (that is, less [[downtime]]). Availability of a system may also be increased by the strategy of focusing on increasing testability and maintainability and not on reliability. Improving maintainability is generally easier than reliability. Maintainability estimates (repair rates) are also generally more accurate. However, because the uncertainties in the reliability estimates are in most cases very large, it is likely to dominate the availability (prediction uncertainty) problem, even while maintainability levels are very high.
=== Response time ===
{{Main|Response time (technology)}}
Response time is the total amount of time it takes to respond to a request for service. In computing, that service can be any unit of work from a simple [[computer data storage|disk IO]] to loading a complex [[web page]]. The response time is the sum of three numbers:<ref>{{cite book  | last = Wescott | first = Bob | title = The Every Computer Performance Book, Chapter 3: Useful laws | publisher = [[CreateSpace]] | date = 2013  | isbn = 978-1482657753}}</ref>
* Service time - How long it takes to do the work requested.
* Wait time - How long the request has to wait for requests queued ahead of it before it gets to run.
* Transmission time – How long it takes to move the request to the computer doing the work and the response back to the requestor.
=== Processing speed ===
{{Main|Instructions per second|FLOPS}}
Most consumers pick a computer architecture (normally [[Intel]] [[IA-32]] architecture) to be able to run a large base of pre-existing, pre-compiled software.  Being relatively uninformed on computer benchmarks, some of them pick a particular CPU based on operating frequency (see [[megahertz myth]]).
Some system designers building parallel computers pick CPUs based on the speed per dollar.
=== Channel capacity ===
{{Main|Channel capacity}}
Channel capacity is the tightest upper bound on the rate of [[information]] that can be reliably transmitted over a [[channel (communications)|communications channel]]. By the [[noisy-channel coding theorem]], the channel capacity of a given channel is the limiting information rate (in units of [[information entropy|information]] per unit time) that can be achieved with arbitrarily small error probability.<ref>{{cite web |url=http://www.cs.ucl.ac.uk/staff/S.Bhatti/D51-notes/node31.html |author=Saleem Bhatti |title=Channel capacity |work=Lecture notes for M.Sc. Data Communication Networks and Distributed Systems D51 -- Basic Communications and Networks |url-status=dead |archive-url=https://web.archive.org/web/20070821212637/http://www.cs.ucl.ac.uk/staff/S.Bhatti/D51-notes/node31.html |archive-date=2007-08-21 }}</ref><ref>{{cite web | url = http://www.st-andrews.ac.uk/~www_pa/Scots_Guide/iandm/part8/page1.html | title = Signals look like noise! | author = Jim Lesurf | work = Information and Measurement, 2nd ed.}}</ref>
[[Information theory]], developed by [[Claude E. Shannon]] during [[World War II]], defines the notion of channel capacity and provides a mathematical model by which one can compute it. The key result states that the capacity of the channel, as defined above, is given by the maximum of the [[mutual information]] between the input and output of the channel, where the maximization is with respect to the input distribution.<ref>{{cite book| author = Thomas M. Cover, Joy A. Thomas | title = Elements of Information Theory | publisher = John Wiley & Sons, New York |year=2006}}</ref>
=== Latency ===
{{Main|Latency (engineering)}}
Latency is a time delay between the cause and the effect of some physical change in the system being observed. Latency is a result of the limited velocity with which any physical interaction can take place. This velocity is always lower or equal to speed of light. Therefore, every physical system that has non-zero spatial dimensions will experience some sort of latency.
The precise definition of latency depends on the system being observed and the nature of stimulation. In communications, the lower limit of latency is determined by the medium being used for communications. In reliable two-way communication systems, latency limits the maximum rate that information can be transmitted, as there is often a limit on the amount of information that is "in-flight" at any one moment. In the field of human-machine interaction, perceptible latency (delay between what the user commands and when the computer provides the results) has a strong effect on user satisfaction and usability.
Computers run sets of instructions called a process. In operating systems, the execution of the process can be postponed if other processes are also executing. In addition, the operating system can schedule when to perform the action that the process is commanding. For example, suppose a process commands that a computer card's voltage output be set high-low-high-low and so on at a rate of 1000&nbsp;Hz. The operating system may choose to adjust the scheduling of each transition (high-low or low-high) based on an internal clock. The latency is the delay between the process instruction commanding the transition and the hardware actually transitioning the voltage from high to low or low to high.
System designers building [[real-time computing]] systems want to guarantee worst-case response. That is easier to do when the CPU has low [[interrupt latency]] and when it has a deterministic response.
=== Bandwidth ===
{{Main|Bandwidth (computing)}}
In computer networking, bandwidth is a measurement of bit-rate of available or consumed data communication resources, expressed in bits per second or multiples of it (bit/s, kbit/s, Mbit/s, Gbit/s, etc.).
Bandwidth sometimes defines the net bit rate (aka. peak bit rate, information rate, or physical layer useful bit rate), channel capacity, or the maximum throughput of a logical or physical communication path in a digital communication system. For example, bandwidth tests measure the maximum throughput of a computer network. The reason for this usage is that according to Hartley's law, the maximum data rate of a physical communication link is proportional to its bandwidth in hertz, which is sometimes called frequency bandwidth, spectral bandwidth, RF bandwidth, signal bandwidth or analog bandwidth.
=== Throughput ===
{{Main|Throughput}}
In general terms, throughput is the rate of production or the rate at which something can be processed.
In communication networks, throughput is essentially synonymous to digital bandwidth consumption. In [[wireless network]]s or [[cellular communication networks]], the [[system spectral efficiency]] in bit/s/Hz/area unit, bit/s/Hz/site or bit/s/Hz/cell, is the maximum system throughput (aggregate throughput) divided by the analog bandwidth and some measure of the system coverage area.
In integrated circuits, often a block in a [[data flow diagram]] has a single input and a single output, and operates on discrete packets of information. Examples of such blocks are [[Fast Fourier transform|FFT]] modules or [[binary multiplier]]s. Because the units of throughput are the reciprocal of the unit for [[propagation delay]], which is 'seconds per message' or 'seconds per output', throughput can be used to relate a computational device performing a dedicated function such as an [[ASIC]] or [[embedded processor]] to a communications channel, simplifying system analysis.
=== Scalability ===
{{Main|Scalability}}
Scalability is the ability of a system, network, or process to handle a growing amount of work in a capable manner or its ability to be enlarged to accommodate that growth.
=== Power consumption ===
The amount of [[electric power]] used by the computer ([[power consumption]]). This becomes especially important for systems with limited power sources such as solar, batteries, and human power.
==== Performance per watt ====
{{Main|Performance per watt}}
System designers building [[parallel computing|parallel computers]], such as [[Google search technology#Production hardware|Google's hardware]], pick CPUs based on their speed per watt of power, because the cost of powering the CPU outweighs the cost of the CPU itself.<ref>{{cite web |title=EEMBC -- the Embedded Microprocessor Benchmark Consortium |url=http://www.eembc.org/benchmark/consumer.asp?HTYPE=SIM |url-status=dead |archive-url=https://web.archive.org/web/20050327005323/http://www.eembc.org/benchmark/consumer.asp?HTYPE=SIM |archive-date=2005-03-27 |access-date=2009-01-21}}[http://news.cnet.com/Power+could+cost+more+than+servers,+Google+warns/2100-1010_3-5988090.html]</ref>
For spaceflight computers, the processing speed per watt ratio is a more useful performance criterion than raw processing speed due to limited on-board resources of power.<ref>D. J. Shirley; and M. K. McLelland. [https://digitalcommons.usu.edu/cgi/viewcontent.cgi?article=2656&context=smallsat "The Next-Generation SC-7 RISC Spaceflight Computer"]. p. 2.</ref>
=== Compression ratio ===
{{Main|Data compression}}
Compression is useful because it helps reduce resource usage, such as data storage space or transmission capacity. Because compressed data must be decompressed to use, this extra processing imposes computational or other costs through decompression; this situation is far from being a free lunch. Data compression is subject to a space–time complexity trade-off.
=== Size and weight ===
This is an important performance feature of mobile systems, from the smart phones you keep in your pocket to the portable embedded systems in a spacecraft.
=== Environmental impact ===
{{Further|Green computing}}
The effect of computing on the environment, during manufacturing and recycling as well as during use. Measurements are taken with the objectives of reducing waste, reducing hazardous materials, and minimizing a computer's [[ecological footprint]].
=== Transistor count ===
{{Main|Transistor count}}
The number of [[transistor]]s on an [[integrated circuit]] (IC). Transistor count is the most common measure of IC complexity.
== Benchmarks ==
{{Main|Benchmark (computing)}}
Because there are so many programs to test a CPU on all aspects of performance, [[benchmark (computing)|benchmark]]s were developed.
The most famous benchmarks are the SPECint and [[SPECfp]] benchmarks developed by [[Standard Performance Evaluation Corporation]] and the [[Certification Mark]] benchmark developed by the Embedded Microprocessor Benchmark Consortium [[EEMBC]].
== Software performance testing ==
{{Main|Software performance testing}}
In software engineering, performance testing is, in general, conducted to determine how a system performs in terms of responsiveness and stability under a particular workload. It can also serve to investigate, measure, validate, or verify other quality attributes of the system, such as scalability, reliability, and resource usage.
Performance testing is a subset of performance engineering, an emerging computer science practice which strives to build performance into the implementation, design, and architecture of a system.
=== Profiling (performance analysis)===
{{Main|Profiling (computer programming)}}
In [[software engineering]], profiling ("program profiling", "software profiling") is a form of [[dynamic program analysis]] that measures, for example, the space (memory) or time [[Computational complexity theory|complexity of a program]], the [[instruction set simulator|usage of particular instructions]], or frequency and duration of function calls.  The most common use of profiling information is to aid program [[optimization (computer science)|optimization]].
Profiling is achieved by instrumenting either the program [[source code]] or its binary executable form using a tool called a ''profiler'' (or ''code profiler''). A number of different techniques may be used by profilers, such as event-based, statistical, instrumented, and simulation methods.
== Performance tuning ==
{{Main|Performance tuning}}
Performance tuning is the improvement of [[system]] performance. This is typically a computer application, but the same methods can be applied to economic markets, bureaucracies or other complex systems. The motivation for such activity is called a performance problem, which can be real or anticipated. Most systems will respond to increased [[Load (computing)|load]] with some degree of decreasing performance. A system's ability to accept a higher load is called [[scalability]], and modifying a system to handle a higher load is synonymous to performance tuning.
Systematic tuning follows these steps:
# Assess the problem and establish numeric values that categorize acceptable behavior.
# Measure the performance of the system before modification.
# Identify the part of the system that is critical for improving the performance. This is called the [[bottleneck (software)|bottleneck]].
# Modify that part of the system to remove the bottleneck.
# Measure the performance of the system after modification.
# If the modification makes the performance better, adopt it. If the modification makes the performance worse, put it back to the way it was.
== Perceived performance ==
{{Main|Perceived performance}}
Perceived performance, in computer engineering, refers to how quickly a software feature appears to perform its task. The concept applies mainly to [[User acceptance test|user acceptance]] aspects.
The amount of time an application takes to start up, or a file to download, is not made faster by showing a startup screen (see Splash screen) or a file progress dialog box. However, it satisfies some human needs: it appears faster to the user as well as provides a visual cue to let them know the system is handling their request.
In most cases, increasing real performance increases perceived performance, but when real performance cannot be increased due to physical limitations, techniques can be used to increase perceived performance.
{{anchor|Performance Equation}}
== Performance Equation ==
The total amount of time ('''t''') required to execute a particular benchmark program is
: '''<math>t=\tfrac{NC}{f}</math>''' , or equivalently
: '''<math>P=\tfrac{If}{N}</math>'''<ref name="shrinking">Paul DeMone.
"The Incredible Shrinking CPU".
2004.
[http://www.realworldtech.com/page.cfm?ArticleID=RWT062004172947&p=5] {{Webarchive|url=https://web.archive.org/web/20120531101531/http://www.realworldtech.com/page.cfm?ArticleID=RWT062004172947&p=5|date=2012-05-31}}</ref>
where
*<math display="inline">P = \frac{1}{t}</math> is "the performance" in terms of time-to-execute
*<math display="inline">N</math> is the number of instructions actually executed (the [[instruction path length]]). The [[instruction set#Code density|code density]] of the [[instruction set]] strongly affects '''N'''. The value of '''N''' can either be determined '''exactly''' by using an [[instruction set simulator]] (if available) or by estimation—itself based partly on estimated or actual frequency distribution of input variables and by examining generated [[machine code]] from an [[High-level programming language|HLL]] compiler. It cannot be determined from the number of lines of HLL source code. N is not affected by other processes running on the same processor. The significant point here is that hardware normally does not keep track of (or at least make easily available) a value of N for executed programs. The value can therefore only be accurately determined by instruction set simulation, which is rarely practiced.
*<math display="inline">f</math> is the clock frequency in cycles per second.
*<math display="inline">C = \frac{1}{I}</math> is the average [[cycles per instruction]] (CPI) for this benchmark.
*<math display="inline">I = \frac{1}{C}</math> is the average [[instructions per cycle]] (IPC) for this benchmark.
Even on one machine, a different compiler or the same compiler with different [[compiler optimization]] switches can change N and CPI—the benchmark executes faster if the new compiler can improve N or C without making the other worse, but often there is a trade-off between them—is it better, for example, to use a few complicated instructions that take a long time to execute, or to use instructions that execute very quickly, although it takes more of them to execute the benchmark?
{{anchor|brainiac}}
A CPU designer is often required to implement a particular [[instruction set]], and so cannot change N.
Sometimes a designer focuses on improving performance by making significant improvements in f (with techniques such as deeper pipelines and faster caches), while (hopefully) not sacrificing too much C—leading to a [[speed-demon]] CPU design.
Sometimes a designer focuses on improving performance by making significant improvements in CPI (with techniques such as [[out-of-order execution]], [[superscalar]] CPUs, larger caches, caches with improved hit rates, improved [[branch predictor|branch prediction]], [[speculative execution]], etc.), while (hopefully) not sacrificing too much clock frequency—leading to a brainiac CPU design.<ref>
[https://www.rose-hulman.edu/~hudson/Ec332/Papers/03Brainiacs.pdf "Brainiacs, Speed Demons, and Farewell"]
by Linley Gwennap
</ref>
For a given instruction set (and therefore fixed N) and semiconductor process, the maximum single-thread performance (1/t) requires a balance between brainiac techniques and speedracer techniques.<ref name="shrinking" />
==See also==
* [[Algorithmic efficiency]]
* [[Computer performance by orders of magnitude]]
* [[Network performance]]
* [[Latency oriented processor architecture]]
* [[Optimization (computer science)]]
* [[RAM update rate]]
* Complete [[instruction set]]
* [[Hardware acceleration]]
* [[Speedup]]
* [[Cache replacement policies]]
* [https://www.laptopicker.com/how-to-optimize-my-pc-hardware/ Understanding Your PC Hardware]
* [[Relative efficiency]]
==References==
{{Reflist}}
{{Authority control}}
[[Category:Computer performance| ]]