Approve the test script before the day
Direct Answer Block: Why every liquid cooling commissioning script must be frozen, version-controlled, and signed with explicit pass/fail bounds before test day begins. In mission-critical data center commissioning and operations, engineers enforce this boundary using deterministic physical equations, calibrated field instrumentation, and auditable sign-off gates rather than relying on vendor brochure summaries, uncalibrated software plots, or verbal assurances. Every measurement and state transition must be recorded with synchronized UTC timestamps and explicit pass/fail thresholds before a signature is granted.

A script is a set of steps with an expected result, a stop, and a revision, approved before the window. A script written during the window to match what the plant did is a diary. Diaries do not support a certificate.
The expected result comes from the specification then in force, or from a limit the owner wrote and can point to. It does not come from the first calm hour. If the expected result cell is blank, do not start. Blank is not "use engineering judgment on the night." Judgment on the night is how a pass line moves.
Say who witnesses and who may accept a document instead. Those are different verbs. A letter that a setting was adjusted is not a witnessed changeover. If you did not witness every line, say what was sampled. "We saw enough" is not a sampling rule.
Stops belong in the script, not in someone's memory. The stop is the first safe state, not a root cause. You will not know the cause in the first minute. Writing one there is how the report inherits a guess.
1. First-Principles Engineering Formulation and Governing Mathematics
In high-density data center environments, verbal summaries and qualitative status tiles routinely obscure incipient physical failures. Whether auditing a 120 kW liquid-cooled rack boundary, evaluating a 1-second telemetry time series, or verifying plant redundancy prior to handover, the governing physical relationship for this protocol is expressed as:
When any term on the left-hand or right-hand side of this relationship is left unmeasured, inferred from an uncalibrated nameplate, or averaged across an unverified time window, the resulting engineering record cannot support a valid operational signature. Field engineers must verify each physical variable independently using NIST/ISO 17025 traceable instruments before proceeding to the next gate in the sequence.
| Audit Verification Parameter | Unacceptable / Defective Field Practice | Auditable First-Principles Requirement | Mandatory Action on Failure |
|---|---|---|---|
| Primary Boundary Definition | Implied by vendor brochure or slide deck | Explicitly bounded on stamped P&ID / SLD with tag IDs | Halt acceptance; issue formal boundary clarification |
| Measurement & Telemetry Proof | Single uncalibrated display reading or 60-sec average | Calibrated Class A/0.5S sensor with 1-sec UTC timestamp | Reject dataset; classify point as 0x40 Stale or 0x80 Bad |
| Pass / Fail Acceptance Gate | Qualitative "Looks OK" or "Within Normal Range" | Quantitative numerical corridor cited from revision in force | Log Category A/B defect; withhold retainage or permit |
| Custody & Sign-Off Record | Single bundled "Accepted" signature at end of meeting | Scoped, dated sentence naming exact proved & unproved lines | Strike unsupported claims; issue Conditional Refusal |
2. Deep-Dive Technical Protocol and Field Implementation
The following field protocol—adapted directly from Volume 03: Finish a Liquid Cooling Verification: Seven Sheets for One Load-Bank Test—establishes the step-by-step verification mechanics required on the data hall floor:
No dynamic commissioning test may be executed from memory or oral instructions. A test conducted without a pre-approved, step-by-step test script is merely an informal equipment demonstration. It cannot support an engineering completion certificate.
A script that the testing team drafted but which the Owner’s Project Engineer has not formally reviewed and signed is merely an internal working document. It carries no contractual authority. Executing against an unapproved draft leaves the testing team fully exposed to commercial disputes and legal liability if an equipment failure occurs.
COMMISSIONING DOCUMENT HIERARCHY
| 1. CODES & STANDARDS | ASHRAE 0-2019, ASHRAE 90.4, NFPA 70, IEC 60364, BSRIA BG 29. |
|---|---|
| 2. CONTRACT SPECIFICATIONS & BOD | CSI Div 23 & 26, Owner Project Requirements, As-Built P&IDs. |
| 3. COMMISSIONING PLAN (Cx Plan) | Governance rules, level gates, witness protocols, schedules. |
| 4. FUNCTIONAL TEST PROCEDURES (FPT) | Pre-approved, step-by-step scripts with exact numeric bands. |
| 5. MASTER FIELD SHEETS (INK LOGS) | Floor records signed in ink; raw time-series sensor files. |
The Anatomy of an Auditable Test Script
Every Functional Test Procedure (FPT) and Integrated System Test (IST) script must be structured with consistent, auditable engineering sections:
- Test Scope and Boundary: The specific mechanical and electrical components under test, and the exact physical boundary separating them from adjacent operational equipment.
- Governing Reference Documents: Direct citations of the governing P&ID revision, electrical single-line diagram (SLD), and equipment submittals. Every tolerance must reference a specific contract clause or manufacturer standard; an acceptance tolerance with no cited source is merely an unverified opinion.
- Instrumentation and Calibration Schedule: The specific instrument classes, serial numbers, and calibration certificate IDs required to execute the procedure.
- Prerequisite Verification Gate: Written verification that all preceding level requirements (such as static uncrating, de-energized megger checks, and hydrostatic pressure holds) are fully signed off.
- Safety Controls and Environmental Boundaries: Mandatory Personal Protective Equipment (PPE), arc-flash boundary definitions, spill containment staging, and verified emergency stop paths.
- Step-by-Step Execution Sequence: A sequential table detailing every manual action, the exact physical feedback to observe, the expected numerical tolerance band, and dedicated sign-off cells for measured data and witness initials.
- Explicit Stop Conditions: Clear operational limits that mandate an immediate test halt if breached (such as secondary coolant temperature exceeding 45.0°C, secondary pressure dropping below 100 kPa, or supply phase voltage unbalance exceeding 2.0%).
STANDARD TEST SCRIPT STEP STRUCTURE
| STEP | ACTION COMMAND | EXPECTED PHYSICAL RESULT | ACCEPTANCE BAND | MEASURE | SIGN / TIME |
|---|---|---|---|---|---|
| 4.2 | Ramp Load Bank to 50% (Step 2: 600 kW total) Dwell: 45 minutes | Active power stabilizes Secondary supply T increases Heat balance closure ratio | 600 kW +/- 1.5% 36.0°C +/- 0.5°C 97.0% to 101.5% | [ ] [ ] [ ] | Init: ____ Time: ____ Sheet 3 ref |
Freezing Script Revisions on the Floor
The Commissioning Kickoff Meeting establishes the specific approval status required before a script may be executed on the floor. Only scripts bearing the formal status stamp "APPROVED FOR EXECUTION - REV [X]" are permitted in the data hall.
If during testing the mechanical plant exhibits unexpected dynamics that the approved script did not anticipate, modifying the script on the floor with a pencil note is strictly prohibited. Uncontrolled redlining allows the test procedure to quietly adapt to whatever the plant happens to do.
When a procedural modification is necessary:
- The Test Lead pauses the test.
- A formal Engineering Change Notice (ECN) is drafted documenting the deviation, its technical justification, and the amended steps.
- The ECN is signed by the Test Lead, the Contractor Lead, and the Owner’s Project Engineer.
- The script revision is incremented (e.g., Rev 1.0 to Rev 1.1) and re-issued.
If the owner refuses the ECN, testing resumes under the original script, and the failure of the plant to meet the original expected result is logged on the master punch list.
The Mandatory Script Suite for a High-Density Liquid Hall
A complete verification campaign requires a comprehensive suite of distinct, standalone scripts. Attempting to combine multiple operational verifications into a single high-level procedure produces an unwieldy document that inevitably glosses over critical safety and performance checks.
The verification campaign requires nine dedicated test scripts:
Phase 1: Static Pre-Start Checklists (Level 2 / Level 3)
- Script 01: Physical Uncrating, Alignment & Visual QA: Verification of load-bank chassis integrity, hose routing, fitting seals, and seismic tie-downs.
- Script 02: De-Energized Electrical Megger & Phase Rotation: Insulation resistance testing (≥ 100 MΩ at 1000V DC), ground continuity (≤ 0.1 Ω), and confirmation of clockwise A-B-C phase sequence.
- Script 03: Secondary Loop Flushing, Hydrostatic Hold & Air Purge: Chemical flushing velocity verification (≥ 1.5 m/s per BSRIA BG 29/2020), hydrostatic pressure hold (1.5× working pressure for 2 hours with zero decay), and automatic deaeration.
Phase 2: Single-System Functional Testing (Level 4 FPT)
- Script 04: Autonomous CDU Startup, Flow & Pressure Baseline: Verification of canned pump VFD ramp profiles, manual differential-pressure setpoint tracking, and cold system resistance.
- Script 05: CDU N+1 Pump Failover & Autonomous Control: Physical tripping of the primary pump VFD; verification of standby pump auto-start within < 1.5 s with zero loop cavitation.
- Script 06: Alarm Injection & Point-to-Point Telemetry: Physical injection of simulated faults (secondary low flow, expansion tank low level, leak detection rope wetting) and cross-checking BMS telemetry against local instrumentation.
Phase 3: Integrated System Testing (Level 5 IST)
- Script 07: Cold No-Heat Baseline Pump Curve: Hydraulic head versus flow profiling across 4 manual valve throttling points with automated VFD control locked off.
- Script 08: Multi-Tier Thermal Heat Ladder & Overload Dwell: Stepped thermal execution (25%, 50%, 75%, 100% full design load, and 110% overload margin) with minimum 45-minute dwells and a 4-hour full-load soak.
- Script 09: Dynamic Transient Failure & Emergency Shutdown: 2N primary power feed cut, facility chilled water loss ride-through, and controlled de-energization shutdown sequence.
The "One Script, One Signature" Principle
Every script in the suite stands as an independent contractual document with its own dedicated sign-off block. Combining a successful cold pump curve and a failed thermal overload step into a single combined signature creates a fundamentally ambiguous record.
Each script must produce its own unambiguous verdict: PASS, FAIL, or NOT EXECUTED. Any unresolved failure within a script immediately generates an entry in the master Issues Log.
3. Field Verification Checklist and Instrumentation Hygiene
To execute this protocol without introducing measurement bias or scope ambiguity, the responsible engineer enforces four mandatory field rules:
- Instrument Traceability Before Observation: Verify that every thermometer (4-wire Pt100 RTD Class A), pressure transducer (≤ ± 0.1% span), flowmeter, balometer, or power analyzer carries a current calibration certificate and passes a pre-test zero/span check on the day of the walk.
- Synchronized Time-Series Capture: Align all digital loggers and gateway pollers to a single UTC reference clock (via IEEE 1588v2 PTP at ± 1 ms for electrical Sequence of Events, or NTP at ± 10 ms for thermal/hydraulic dwells). Never merge unsynchronized CSV files by row index.
- Explicit Isolation of Disturbances vs. Control Handles: Hold upstream supply temperatures, parallel cabinet loads, and containment door states constant during the observation dwell so that transient plant drift is never mistaken for steady-state equipment performance.
- Written Record Prior to Signature: Record every boundary tag, measured value, uncertainty tolerance, and open limitation in ink on the controlled project sheet before leaving the equipment line-up.
Frequently Asked Questions (FAQ)
Why does "Approve the test script before the day" matter during high-density data center commissioning and operations?
Why every liquid cooling commissioning script must be frozen, version-controlled, and signed with explicit pass/fail bounds before test day begins. When project teams bypass this rule to meet a commercial schedule milestone, unverified physical assumptions—such as hidden hydraulic pressure drops, unlogged polling gaps, unblanked bypass leakage, or uncalibrated sensor offsets—transfer directly into live 24/7 production as latent single points of failure.
How do you quantitatively verify compliance with "data center commissioning script approval" on the physical floor?
Compliance is verified by executing the governing mathematical balance (∀ Step k: Scriptk = (Precondition, Action, Pass Band [Lk, Uk], Abort Trigger)) using independent, calibrated field instruments across a stabilized steady-state dwell window (minimum 15 to 30 minutes for thermal and hydraulic systems), and confirming that the discrepancy between primary and secondary measurements remains strictly within the pre-declared acceptance band.
What should an engineer do if a contractor or vendor submits a report that violates this protocol?
The reviewing engineer must refuse to sign the broad acceptance sentence, mark the specific line item as Unproved / Defective on the official audit sheet, and either require a witnessed re-test under a corrected script or isolate the unproved scope under a time-bounded Conditional Acceptance with a 150% to 200% financial retainage escrow holdback.
From Finish a Liquid Cooling Verification: Seven Sheets for One Load-Bank Test, Tristan, Data Center Field Guides.
Related: One system at a time.
Master the Complete Protocol: Finish a Liquid Cooling Verification: Seven Sheets for One Load-Bank Test
ASHRAE Guideline 0 Levels 1-5, Integrated Systems Testing (IST), and Contractor Pass Separation
Stop relying on contractor spreadsheets and vendor marketing claims. This field guide provides the complete, auditable verification binder—including physical equations, inspection stations, acceptance thresholds, and signed refusal templates.