View source for Downtime

{{Short description|Period when a system is unavailable or unable to provide or perform its primary function}}
{{Other uses}}
The term '''downtime''' (also '''(system) outage''' or '''(system) drought''' colloquially) is used to refer to periods when a system is unavailable. The '''unavailability''' is the proportion of a time-span that a [[system]] is unavailable or [[offline]].
This is usually a result of the system [[failure|failing to function]] because of an unplanned event, or because of routine [[repair and maintenance|maintenance]] (a planned event).
The terms are commonly applied to [[telecommunications network|network]]s and [[server (computing)|servers]]. The common reasons for unplanned outages are system failures (such as a [[Crash (computing)|crash]]) or communications failures (commonly known as '''network outage''' or '''network drought''' colloquially). For outages due to issues with general [[computer systems]], the term '''computer outage''' (also '''IT outage''' or '''IT drought''') can be used.
The term is also commonly applied in industrial environments in relation to failures in industrial production equipment. Some facilities measure the downtime incurred during a work shift, or during a 12- or 24-hour period. Another common practice is to identify each downtime event as having an operational, electrical or mechanical origin.
The opposite of downtime is [[uptime]].
==Types==
Industry standards for the term "Outage Duration" or "Maintenance Duration" can have different point of initiation and completion thus the following clarification should be used to avoid conflicts in contract execution:
# "Turnkey" this is the most engrossing of all outage types.  Outage or Maintenance starts with operator of the plant or equipment pressing the shutdown or stop button to initiate a halt in operation.  Unless otherwise noted, Outage or Maintenance is considered completed when the plant or equipment is back in normal operation ready to begin manufacturing or ready be synchronized with system or grid or ready to perform duties as pump or compressor.
# "Breaker to Breaker" This Outage or Maintenance starts with operator of the plant or equipment removing the power circuit (Main power breaker at "off" or "disengaged" or "On-Cooldown"), not the control circuit from operation.  This still would allow for the equipment to be cooled down or brought to ambient such that outage/maintenance work can be prepared or initiated.  Depending on equipment types, "Breaker to Breaker" outage can be advantageous if contracting out controls related maintenance as this type of maintenance work can be performed while main equipment is still on cool-down or on stand-by.  Unless otherwise noted, this type of outage is considered complete when power circuit is re-energized via engaging of the power breaker.
# "Completion of [[Lockout-tagout|Lock-out/Tag-out]]" This Outage or Maintenance (sometimes mistaken for "Off-Cooldown" but not the same) starts with operator of the plant or equipment removing the power circuit, disengaging the control circuit and performing other neutralization of potential power and hazard sources (typically called Lock-Out, Tag-Out  "LOTO")  This point of maintenance period is typically the last phase of the outage initiation stage before actual work starts on the facility, plant or equipment.  Safety briefing should always follow the LOTO activity, before any work is conducted.  Unless otherwise noted, this type of outage is considered complete when the equipment has reached mechanical completion and ready to be placed on slow-roll for many heavy rotating equipment, Bump-test or rotation check for motors, etc., but must follow return or work permit per LOTO procedures.
Any on-line testing, performance testing and tuning required should not count towards the outage duration as these activities are typically conducted after the completion of outage or maintenance event and are out of control of most maintenance contractors.
==Characteristics==
Unplanned downtime may be the result of an equipment malfunction, etc.
===Telecommunication outage classifications===
Downtime can be caused by failure in
hardware (physical equipment),
(logic controlling equipment),
interconnecting equipment (such as cables, facilities, routers,...),
transmission (wireless, microwave, satellite), and/or
capacity (system limits).
The failures can occur because of
damage,
failure,
design,
procedural (improper use by humans),
engineering (how to use and deployment),
overload (traffic or system resources stressed beyond designed limits),
environment (support systems like power and HVAC),
(outages designed into the system for a purpose such as software upgrades and equipment growth),
other (none of the above but known), or
unknown.
The failures can be the responsibility of
customer/service provider,
vendor/supplier,
utility,
government,
contractor,
end customer,
public individual,
act of nature,
other (none of the above but known), or
unknown.
==Impact==
Outages caused by system failures can have a serious impact on the users of computer/network systems, in particular those industries that rely on a nearly 24-hour service:
* [[Medical informatics]]
* [[Nuclear power]] and other [[infrastructure]]
* [[Bank]]s and other [[financial institution]]s
* [[Aeronautics]], [[airline]]s
* [[News agency|News reporting]]
* [[E-commerce]] and [[online transaction processing]]
* [[Virtual world|Persistent online games]]
Also affected can be the users of an [[Internet service provider|ISP]] and other customers of a telecommunication network.
Corporations can lose business due to network outage or they may default on a contract, resulting in financial losses. According to [[Veeam Software|Veeam]] 2019 cloud data management report organizations encounter unplanned downtime, on average, 5-10 times per year with the average cost of one hour of downtime being $102,450.<ref>{{Cite web|url=https://go.veeam.com/data-protection-trends-executive-brief|title=2021 Data Protection Trends Executive Brief|website=Veeam Software}}</ref>
Those people or organizations that are affected by downtime can be more sensitive to particular aspects:
* some are more affected by the length of an outage - it matters to them how much time it takes to recover from a problem
* others are sensitive to the timing of an outage - outages during peak hours affect them the most
The most demanding users are those that require [[high availability]].
==Famous outages==
{{Recentism|date=May 2013}}
On [[Mother's Day]], Sunday, May 8, 1988, a fire broke out in the main switching room of the Hinsdale Central Office of the [[Illinois Bell]] telephone company. One of the largest [[Telephone exchange|switching]] systems in the state, the facility processed more than 3.5 million calls each day while serving 38,000 customers, including numerous businesses, hospitals, and Chicago's O'Hare and Midway Airports.<ref>{{Cite journal|url=http://catless.ncl.ac.uk/Risks/6.82.html#subj2.1|title=Risks of Single Point Failures: The Hinsdale Fire |first2=Chuck |last2=Weinstock |first3=Patrick |last3=Townson |first1=Peter G.|last1=Neumann|date=May 11, 1988|journal=The RISKS Digest |volume=6|issue=82 |archive-url=https://web.archive.org/web/20221006172553/http://catless.ncl.ac.uk/Risks/6.82.html#subj2.1 |archive-date=October 6, 2022 |via=The Catless Web Server}} Excerpted from TELECOM Digest. '''8''' (76).</ref>
Virtually the entire [[AT&T Corporation|AT&T]] network of [[4ESS]] toll tandems switches went in and out of service over and over again on January 15, 1990, disrupting long-distance service for the entire United States.  The problem dissipated by itself when traffic slowed down. A software bug was found.<ref>{{cite web|url=http://www.phworld.org/history/attcrash.htm |title=The Crash of the AT&T Network in 1990 |agency=The Risks Digest |date=February 26, 1990 |first1=Peter G. |last1=Neumann |website=Telephone World |url-status=dead |archive-url=https://web.archive.org/web/20221219151129/https://telephoneworld.org/landline-telephone-history/the-crash-of-the-att-network-in-1990/ |archive-date= Dec 19, 2022 }}</ref>
AT&T lost its [[Frame Relay]] network for 26 hours on April 13, 1998.<ref>{{Cite web|url=https://www.keysight.com/upload/cmc_upload/All/sp-insight.pdf|title=Preventing IP Network Service Outages|website=Agilent Technologies |date=March 15, 2002 |url-status=dead |archive-url= https://web.archive.org/web/20180928044428/https://www.keysight.com/upload/cmc_upload/All/sp-insight.pdf |archive-date= Sep 28, 2018 }}</ref> This affected many thousands of customers, and bank transactions were one casualty.  AT&T failed to meet the [[service level agreement]] on their contracts with customers and had to refund<ref>{{Cite journal|url=http://catless.ncl.ac.uk/Risks/19.72.html#subj6.1|title= AT&T Announces Cause of Frame Relay Network Outage |first2=Steve |last2=Bellovin |first3= Jim |last3=Byrnes |first4=Ruthlyn |last4=Newell |first1=Peter G.|last1=Neumann|date=May 7, 1998|journal=The RISKS Digest |volume=19|issue=72|via=The Catless Web Server}}</ref> 6,600 [[customer account]]s, costing millions of dollars.
[[Xbox Live]] had intermittent downtime during the 2007–2008 holiday season which lasted thirteen days.<ref>{{cite web |url=https://www.engadget.com/2008/01/03/xbox-live-outage-day-13-still-up-and-down-still-preventing-fu/ |title=Xbox Live outage, day 13: still up and down, still preventing fun from being had |first1=Ryan |last1=Block |date=2008-01-03 |publisher=Engadget |access-date=2011-04-27 |url-status=live |archive-url=https://web.archive.org/web/20120127074021/https://www.engadget.com/2008/01/03/xbox-live-outage-day-13-still-up-and-down-still-preventing-fu/ |archive-date= Jan 27, 2012 }}</ref> Increased demand from Xbox 360 purchasers (the largest number of new user sign-ups in the history of Xbox Live) was given as the reason for the downtime; in order to make amends for the service issues, Microsoft offered their users the opportunity to receive a free game.<ref>{{cite web |url-status=dead |url=http://www.pcworld.com/article/141018/microsoft_offers_free_game_for_xbox_live_holiday_problems.html |title=Microsoft offers free game for Xbox Live holiday problems |archive-url=https://web.archive.org/web/20111201212234/http://www.pcworld.com/article/141018/microsoft_offers_free_game_for_xbox_live_holiday_problems.html |archive-date=2011-12-01 |website=[[PC World (magazine)|PC World]] |date=January 4, 2008 |first1=Peter |last1=Cohen |agency=Macworld }}</ref>
[[Sony]]'s [[PlayStation Network outage|PlayStation Network April 2011 outage]], began on April 20, 2011, and was gradually restored on May 14, 2011, starting in the [[United States]]. This outage is the longest amount of time the PSN has been offline since its inception in 2006.  Sony has stated the problem was caused by an external intrusion which resulted in the confiscation of personal information. Sony reported on April 26, 2011, that a large amount of user data had been obtained by the same hack that resulted in the downtime.<ref>{{Cite web|title=Restoration of PlayStation®Network and Qriocity Services begins |date=May 15, 2011 |url=http://www.sony.com/en/SonyInfo/News/Press/201105/11-0515E/index.html|access-date=2021-10-22|website=Sony Group Portal - Sony Global Headquarters|language=en}}</ref>
[[Telstra]]'s Ryde switch failed in late 2011 after water egressed into the electrical switch board from continuing wet weather. The Ryde switch is one of the largest by area switches in Australia, and affected more than 720,000 services.{{cn|date=November 2023}}
The [[Miami]] datacenter of ServerAxis went offline unannounced on February 29, 2016, and was never restored. This impacted multiple providers and hundreds of websites. The outage impacted coverage of the [[2016 NCAA Division I women's basketball tournament]] as WBBState, one of the affected sites, was by far the most comprehensive provider of women's basketball statistics available.<ref>{{Cite web |last=Levy |first=Ian |date=2016-03-16 |title=A Website Went Offline And Took Most Of Women's College Basketball Analytics With It |url=https://fivethirtyeight.com/features/a-website-went-offline-and-took-most-of-womens-college-basketball-analytics-with-it/ |website=[[FiveThirtyEight]] |language=en-US |url-status=live |archive-url=https://web.archive.org/web/20230930191634/https://fivethirtyeight.com/features/a-website-went-offline-and-took-most-of-womens-college-basketball-analytics-with-it/ |archive-date= Sep 30, 2023 }}</ref>
The game platform [[Roblox]] had an outage around October 2021, during their [[Chipotle Mexican Grill|Chipotle]] Event. Many users thought it was because of the event, because it received massive reception, as users could get a free Chipotle burrito during it. The outage was Roblox's longest downtime, lasting 3 days.<ref>{{cite web | url=https://www.ign.com/articles/roblox-servers-down-chipotle-free-burritos | title=Roblox's Servers Are Back Online &#91;Update&#93; | date=29 October 2021 |website=IGN |first1= Logan |last1=Plant |url-status=live |archive-url=https://web.archive.org/web/20231017191715/https://www.ign.com/articles/roblox-servers-down-chipotle-free-burritos |archive-date= Oct 17, 2023 }}</ref><ref>{{cite web | url=https://www.msn.com/en-us/news/technology/is-roblox-down-why-the-gaming-platform-isn-e2-80-99t-working-today-with-thousands-of-users-reporting-login-problems/ar-AAQ5oWh | title=Is Roblox down? Why the gaming platform isn't working today with thousands of users reporting login problems | website=[[MSN]] |first1=Alex |last1=Finnis |url-status=dead |archive-url= https://web.archive.org/web/20211115154939/https://www.msn.com/en-us/news/technology/is-roblox-down-why-the-gaming-platform-isn-e2-80-99t-working-today-with-thousands-of-users-reporting-login-problems/ar-AAQ5oWh |archive-date= Nov 15, 2021 }}</ref><ref>{{cite web | url=https://www.theverge.com/2021/10/30/22754107/roblox-down-outage-chipotle-server-issues-status | title=Roblox was down all weekend, and not because of Chipotle | date=30 October 2021 }}</ref>
On July 8, 2022, Rogers suffered a [[2022 Rogers Communications outage|major nationwide outage]] in [[Canada]]. This simultaneously affected cell phone and internet access, causing 911 calls, interbank transactions to fail and also disrupting government services.
On July 19th, 2024, [[CrowdStrike]] issued a [[2024 CrowdStrike incident|faulty device driver update]] for their Falcon software, resulting in Windows PCs, servers, and virtual machines to crash and boot loop. The incident unintentionally affected approximately 8.5 million [[Microsoft Windows|Windows]] machines worldwide, including critical infrastructure such as 911 services in various states. It is considered to be the largest outage in the history of [[information technology]].<ref>{{Cite news |last=Milmo |first=Dan |last2=Kollewe |first2=Julia |last3=Quinn |first3=Ben |last4=Taylor |first4=Josh |last5=Ibrahim |first5=Mimi |date=2024-07-20 |title=Slow recovery from IT outage begins as experts warn of future risks |url=https://www.theguardian.com/australia-news/article/2024/jul/19/microsoft-windows-pcs-outage-blue-screen-of-death |access-date=2024-07-21 |work=The Guardian |language=en-GB |issn=0261-3077}}</ref><ref>{{Cite web |last=Weston |first=David |date=2024-07-20 |title=Helping our customers through the CrowdStrike outage |url=https://blogs.microsoft.com/blog/2024/07/20/helping-our-customers-through-the-crowdstrike-outage/ |access-date=2024-07-21 |website=The Official Microsoft Blog |language=en-US}}</ref>
==Service levels==
In [[service level agreement]]s, it is common to mention a percentage value (per month or per year) that is calculated by dividing the sum of all downtimes timespans by the total time of a reference time span (e.g. a month). 0% downtime means that the server was available all the time.
For Internet servers downtimes above 1% per year or worse can be regarded as unacceptable as this means a downtime of more than 3 days per year. For e-commerce and other industrial use any value above 0.1% is usually considered unacceptable.<ref>{{Cite web|last=Cohen|first=Gad|title=Downtime, Outages and Failures - Understanding Their True Costs|url=https://www.evolven.com/blog/downtime-outages-and-failures-understanding-their-true-costs.html|access-date=2021-10-22|website=www.evolven.com|language=en-gb}}</ref>
==Response and reduction of impact==
It is the duty of the network designer to make sure that a network outage does not happen. When it does happen, a well-designed system will further reduce the effects of an outage by having localized outages which can be detected and fixed as soon as possible.
A process needs to be in place to detect a malfunction - [[network monitoring]] - and to restore the network to a working condition - this generally involves a [[help desk]] team that can [[troubleshoot]] a problem, one composed of trained engineers; a separate help desk team is usually necessary in order to field user input, which can be particularly demanding during a downtime.
A [[Network management model|network management]] system can be used to detect faulty or degrading components prior to customer complaints, with proactive fault rectification.
[[Risk management]] techniques can be used to determine the impact of network outages on an organisation and what actions may be required to minimise risk.  Risk may be minimised by using reliable components, by performing maintenance, such as upgrades, by using [[Redundancy (engineering)|redundant systems]] or by having a [[contingency plan]] or [[business continuity planning|business continuity plan]].
Technical means can reduce errors with [[error correcting code]]s, [[Retransmission (data networks)|retransmission]], [[redundancy check|checksum]]s, or [[diversity scheme]].
One of the biggest causes of downtime is misconfiguration, where a planned change goes wrong. Typically organisations rely on manual effort to manage the process of configuration backups, but this requires highly skilled engineers with the time to manage the process across a multi-vendor network. Automation tools are available to manage backups, but there are very few solutions that handle configuration recovery which is needed to minimize the overall impact of the outage.<ref>{{Cite web|title=Why Machine Downtime Tracking Matters?|url=https://evocon.com/kb/machine-downtime-tracking/|access-date=2021-10-22|website=Evocon|date=10 September 2018 |language=en-US}}</ref>
==Planning==
A planned outage is the result of a planned activity by the system owner and/or by a [[service provider]]. These outages, often scheduled during the [[maintenance window]], can be used to perform tasks including the following:
* Deferred maintenance, e.g., a deferred hardware repair or a deferred restart to clean up a garbled memory
* Diagnostics to isolate a detected fault
* Hardware fault repair
* Fixing an error or omission in a configuration database or omission in a recent configuration database change
* Fixing an error in application database or an error in a recent application database change
* Software patching/software updates to fix a software fault.
Outages can also be planned as a result of a predictable natural event, such as [[Sun outage]].
Maintenance downtimes have to be carefully scheduled in industries that rely on computer systems. In many cases, system-wide downtimes can be averted using what is called a "rolling upgrade" - the process of incrementally taking down parts of the system for upgrade, without affecting the overall functionality.
==Avoidance==
For most websites, [[website monitoring]] is available.  Website monitoring (synthetic or passive) is a service that "monitors" downtime and users on the site.
==Other usage==
Downtime can also refer to time when human capital or other assets go down.  For instance, if employees are in meetings or unable to perform their work due to another constraint, they are down.  This can be equally expensive, and can be the result of another asset (i.e. computer/systems) being down.  This is also commonly known as "[[dead time]]".
Downtime is also generalized in a personal sense, being used to refer to a period of [[sleep]] or [[recreation]].<ref>{{cite web|url=https://thenaturalnurturer.com/rest-relaxation-why-downtime-is-important-for-kids/|title=Rest & Relaxation: Why "Downtime" Is Important For Kids|date=19 September 2016}}</ref><ref>{{cite web|url=https://www.lifehack.org/articles/productivity/the-importance-of-scheduling-downtime.html|title=The Importance of Scheduling Downtime|date=25 August 2008}}</ref><ref>{{cite web|url=https://www.webmd.com/sleep-disorders/features/emotions-cognitive|title=What Lack of Sleep Does to Your Mind|quote=Many people think of sleep simply as a luxury -- a little downtime.}}</ref>
This term is used also in factories or industrial use. See [[total productive maintenance]] (TPM).
==Measuring downtime==
There are many external services which can be used to monitor the uptime and downtime as well as availability of a service or a host.
==See also==
* [[High availability]]
* [[Uptime]]
* [[Mean down time]]
* [[#Planning|Planned downtime]]
* [[Carrier grade]]
==References==
{{Reflist}}
==External links==
*{{Wiktionary-inline}}
[[Category:Engineering failures]]
[[Category:Maintenance]]
[[Category:System administration]]
[[it:Tempo di fermo]]