Rippowam

Systems

Hidden Infrastructure

On visible interfaces, missing alarms, inherited systems, and the work required to keep failure local.

Journal/Systems

A card touches a terminal and nothing happens. The screen holds, then returns an error too general to locate the problem. The customer tries again while the merchant checks the cable, the connection, and the small symbols along the top of the display. A second card may work, or the first may be accepted thirty seconds later. An ordinary purchase has briefly exposed a question with no visible address.

When the transaction succeeds, the experience appears almost self-contained. A card or phone is presented, an approval arrives, and the purchase is finished from the perspective of the people at the counter. The gesture compresses a larger arrangement into a few seconds. The interface is not the system. It is the place where the system agrees to become legible.

The transaction

At a high level, a card payment links a holder and merchant to acquiring and issuing institutions through a card network. The terminal sends the request toward the merchant's acquirer; the network carries it to the issuer, which determines whether the transaction may proceed under the relevant account, credit, authentication, and risk controls. An approval travels back to the terminal. Clearing and settlement occur through related but later processes, reconciling obligations and moving funds among the institutions involved.

Commercial and technical routes vary across networks, processors, banks, wallets, countries, and payment types. The architecture is still recognizable. One visible response depends on equipment, software, identity, communications, rules, financial institutions, security controls, and a settlement mechanism that neither customer nor merchant needs to observe. The customer receives one message because the system has been designed to conceal most of its own coordination.

Infrastructure is still found in roads, bridges, ports, pipes, and power plants, but the category no longer ends at concrete. Federal critical-function frameworks include payment and settlement, medical care, communications, identity management, information technology, water, electricity, transportation, and other activities whose disruption would travel well beyond the asset where it began. Many of these functions are entered through a credential, a protocol, or a screen, even when their operation remains inseparable from buildings, equipment, rights of way, power, and people. The useful distinction is between interface and architecture. A terminal needs power and communications; the network needs equipment, software, maintenance, and electricity; the bank depends on data centers, staff, regulators, liquidity, and other institutions. A local fault can therefore appear somewhere that seems unrelated to its origin. Infrastructure, in this sense, is the arrangement on which other important activity has come to depend. Its criticality becomes clearer by following the consequences of absence than by measuring the visibility of the asset itself.

The missing alarm

Complex operations require more than data. They require an architecture of attention that directs the right evidence toward the people responsible for interpreting and acting on it. The August 2003 Northeast blackout supplies a severe example. During the afternoon, FirstEnergy's alarm and logging software failed. Operators did not know that the alarms were unavailable. Some valid real-time information continued to exist, and other monitoring organizations continued receiving accurate data about the grid. The local interface had stopped performing one of its most consequential functions: indicating that conditions had changed and that attention should move.

The absence of alarms was mistaken for the absence of a problem.

Had the operators known the alarm system was unavailable, the final investigation noted that they could have attempted repetitive manual scanning across many displays and status points. They did not know the operating condition had changed. Information remained in the environment while situational understanding deteriorated.

The alarm failure did not cause the blackout by itself. The official investigation documented vegetation contact, line trips, incomplete reliability tools, weak coordination, and other technical and organizational failures. Its narrower lesson is more useful here. A system can continue collecting and transmitting valid information while losing the means to make that information timely, intelligible, and actionable for the people accountable for the outcome.

Automation changes the location of human work rather than eliminating it. Sensors and software can observe more variables, enforce consistency, and detect patterns no person could monitor continuously. That delegation raises the importance of interface design, escalation, training, exception handling, and the operator's mental model when conditions leave the normal range. NASA human-factors research describes operators in complex sociotechnical systems integrating fragmented indications into an understanding deeper than the interface or formal procedure alone can provide. Experienced operators may recognize an unusual sequence, distinguish a noisy measurement from a meaningful deviation, or understand a degraded mode that the official description barely acknowledges. They may also become fatigued, anchored to a prior explanation, or overconfident in familiar patterns. Serious design allocates observation, judgment, authority, and recovery across machine and person, especially when either side is least reliable.

The burden of continuity

Important systems seldom arrive as clean-sheet designs. They accumulate. A current interface may sit above a decades-old core; a new analytics layer may depend on data shaped for another generation of software; a hospital, bank, airline, factory, or public agency may combine equipment and protocols introduced across several eras. The resulting architecture is less like a finished diagram than an inherited structure altered while occupied.

A 2025 Government Accountability Office review of federal technology illustrates the problem in a bounded setting. It identified critical legacy systems using unsupported components, older programming languages, and scarce specialist skills, with known cybersecurity or mission risks and incomplete modernization plans. The precise findings concern federal agencies, but the operating tension travels. Age alone does not make a system obsolete. Some older systems still perform narrow functions reliably and encode decades of rules and exceptions. Their risk may lie in supportability, security, adaptability, or dependence on a shrinking number of people who understand them. Modernization behaves as an operating transition even when procured as a technology project. Data must be reconciled, embedded rules rediscovered, users served, and recovery maintained while the architecture changes beneath them. A technically elegant replacement can still fail if it misunderstands what the inherited system was actually doing. Some of the burden is code; some is contractual or regulatory history; some is institutional habit. Earlier success can turn a temporary arrangement into something too consequential to interrupt casually.

Continuity also requires choices that look inefficient during ordinary operation. Redundant equipment, spare capacity, backup communications, recovery exercises, and separated operating paths consume money and attention before they produce any visible return. Resilience has a carrying cost. That cost needs to be justified; the label resilient cannot do the work by itself. Additional capacity can preserve function, introduce complexity, or do both. Isolation can contain a failure while reducing flexibility. The governing task is to decide which disruptions may be tolerated, which must be confined, and where preparation is worth its recurring burden.

Interdependence makes that judgment difficult because consequences refuse to remain inside the category where a fault began. Power supports communications, water pumping, refrigeration, transport, medical care, and payments. Communications support the coordination required to restore physical assets. A software interruption can become a delayed shipment, missed payroll, inaccessible building, or unavailable medication. What appears local in the architecture can become widely distributed in consequence.

Dependency therefore changes the operating standard. Once other work cannot proceed without a system, maintenance, security, staffing, change control, incident communication, recovery testing, and the preservation of operating knowledge are part of the service itself. Preserving every legacy arrangement, funding every redundancy, or insulating critical systems from economics would create different forms of fragility. The obligation is to understand the function well enough to change its form without losing the conditions other people rely upon.

Much of this work produces no event. A backup remains unused, a recovery procedure is rehearsed, an interface presents the right warning, or a specialist transfers knowledge before leaving. These activities can look secondary until the primary path fails. They are what keep a local fault from becoming someone else's emergency. Most days, nothing announces that work. The rest of the system simply continues.

Words & PhotosRyan Bonifacino

Notes

  1. Kees van Hee, Anneke Kosse, Peter Wierts, and Jacob Wijngaard, Let's Speak the Same Language: A Formally Defined Model to Describe and Compare Payment System Architectures, BIS Working Papers No. 1259, April 2025. The card-payment discussion uses the paper's high-level account of the card holder, merchant terminal, acquiring and issuing payment-service providers, card network, authorization path, and settlement bank. The model is a formal simplification and does not imply that every card payment follows one identical commercial or technical route.
  2. Cybersecurity and Infrastructure Security Agency, "National Critical Functions Set" and "Critical Infrastructure Security and Resilience". These materials support the wider treatment of infrastructure through physical and virtual assets, systems, networks, and functions including payment, medical care, communications, information technology, water, electricity, transportation, and public safety. The federal frameworks are used as breadth-setting references, not as the only valid definition of infrastructure.
  3. U.S.-Canada Power System Outage Task Force, Final Report on the August 14, 2003 Blackout in the United States and Canada: Causes and Recommendations, April 2004. The report documents the FirstEnergy alarm and logging failure, operators' lack of awareness that alarms were unavailable, the continued availability of some valid data, and degraded situational awareness. The blackout was multi-causal; the alarm failure is used only as a bounded example of attention architecture.
  4. Randy Mumaw, "Human Factors Discovered: Stories from the Front Lines", NASA Technical Reports Server, presentation acquired 2020. The article uses this source narrowly for fragmented indications, operator mental models, and the need to integrate evidence beyond what an interface or procedure provides. It does not generalize aerospace findings to every operator or system.
  5. U.S. Government Accountability Office, Information Technology: Agencies Need to Plan for Modernizing Critical Decades-Old Legacy Systems, GAO-25-107795, July 17, 2025; and National Institute of Standards and Technology, "Dependencies and Cascading Effects", Disaster Resilience Framework workshop draft, February 11, 2015. The GAO report supports the bounded discussion of unsupported technology, older languages, cybersecurity and mission risk, scarce specialist skills, and incomplete modernization planning in federal agencies. The NIST chapter supports the discussion of dependencies, cascading effects, additional capacity, redundancy, isolation, and recovery planning. Neither source is generalized into a prevalence estimate or universal prescription for private industry, and the NIST document's workshop-draft status is preserved.

Continue reading

Houses

Capital With a Long Memory

Remembered conduct, repeated dealings, and what the second transaction knows.

Builders

The Builder's Bias

How operating experience changes the questions an owner asks.

Enterprise

The Dignity of Boring Businesses

Maintenance, repetition, skilled work, and the companies that keep ordinary life usable.

Source

A Name Older Than the Road

A regional name that outlived a settlement and survived an incomplete record.

Systems

Companies Beneath Companies

Embedded providers, recurring need, switching, and capability built one exception at a time.

Ownership

A House Is Not a Fund

Balance-sheet ownership, institutional form, and the discipline of keeping unlike assets coherent.