Models in, verdicts out, within a day

Every generator that joins an interconnection queue submits dynamic models of its plant, usually in three simulation tools, at several points in the process. The operator has to test each package against its own rulebook before the models go into studies. By hand that is a few engineer-days per package, the answer can depend on who ran it, and the manufacturer's model binaries get run on engineers' own workstations.

The operator's model quality test suite defines the scenarios a model must pass, such as faults, voltage and frequency disturbances, protection limits, phase jumps and weak-grid conditions, and what a passing response looks like. Every submission has to be tested the same way, and the operator's engineers have to be able to see why a case passed or failed.

We encoded the operator's suite, about 20 cases, plus a smaller grid-forming suite. The operator's systems post a model package to our API. We run every case in all three tools, PSCAD, PSS/E and TSAT, benchmark each model against its EMT model, score pass or fail against written criteria, and return the report and the raw results through the same API.

Each run happens in a fresh virtual machine that is destroyed after the run, so no model ever shares an environment with another. Staff can open any run in the dashboard and override a verdict in the rare case where judgment is needed.

The same engine runs suites for six regions' rulebooks, about 150 test cases in all.

A package that takes two to three engineer-days by hand runs in under four hours. On six real model packages run against a regional suite, the automated verdicts matched the engineer's verdict on every case. Individual scenarios finish in five to fifteen minutes.

Under the service agreement the operator sends dozens of packages a day and gets every report back within a day. Model quality is the slowest, most inconsistent step in the queue, and it repeats at every milestone. Testing every package the same way, inside the operator's own workflow, turns it into a step that finishes overnight.