Grid Simulation

N7 Grid Simulation using Smart Meter Data

Derrick Oswald

Motivation:

• smart meter data:

• 15 minute measurements

• 50 houses per transformer service area

• 5000 service areas

• 96×50×5000 = 24 million/day

• low voltage network: 1-10GB CIM file (uncompressed)

Use-cases:

• which transformers can service these customers

• what if every tenth house gets a Tesla

• can the infrastructure support another apartment building

Architecture

Processes

• Ingest – head-end-system smart meter data into Cassandra

• CIM Generation – generation of network snapshot CIM files

• Topological Analysis – node/breaker to bus/branch & islands

• Export – export topological islands (transformer service areas)

• Baseline simulation – generate time series for everything

• Project/use-case simulation

I'll cover the end-to-end process workflow, including: Ingest – which moves head-end-system smart meter data into Cassandra CIM Generation – that creates a network snapshot, in the form of CIM files (Common Information Model) Topological Analysis – to convert from a node/breaker to a bus/branch model, and also identifies topological islands (or transformer service areas) Export – to partition the large CIM files into these topological islands for downstream use Baseline simulation – to run load-flow calculations, using the available meter data, which generates time series for voltages, currents and powers for all network components in the valid time period of the meter data and Project or Use-Case simulations for a number of scenarios of interest

Ingest

Smart meter data is collected from customer meters through the concentrators into the head-end-system. The head-end system generates (near) CSV files, one row per day – where the difference is at daylight savings time changes, where twice a year there are four more or less readings in a day. The system normally provides apparent power – because it's based on billing system requirements – but sometimes active and reactive power are available. Currently, readings are captured at 15 minute intervals approximately 300MB per year per substation. The ingest phase demultiplexes the daily reading records, and joins them to a CIM EnergyConsumer mRID via a mapping table which maps one or more Meter ID values, (which for Swtzerland are CH#####) to a CIM mRID. Records are summed over multiple meters at one mRID, and stored as one Cassandra row per 15 minute reading per EnergyConsumer. There is a plan to aggregate these into hourly, three hour, and daily values, to be used for quicker client side access when drilling down from overview to specific measurements.

CIM Generation

The GIS system is a very rich source of data; every station, transformer, cable, switch, pole, trench, etc. both external and internal is maintained in it. We use version CIM16. Version CIM17 a.k.a. CIM100 is also possible. Full network generation is performed at a once per month cadence. Any topology or value changes in the GIS during the month are ignored – this may lead to slight inconsistencies with the meter data that is constantly being built out. The process merges equipment libraries to attach electrical parameters for cables and transformers. Electrical parameters for these standard components are not stored in the GIS – there is only a foreign key reference used to join to the equipment libraries. A customer specific XML template is used to accommodate schema differences between customers. To speed up the process it is performed in parallel across many GIS instances in east-west “stripes”. One instance does the northernmost stripe, another the second most northern stripe, and so on. Elements that cross stripe boundaries are emitted in multiple stripes, so this requires “de-duplication” (based on mRID) when the stripes are read into Spark. The process generates CIM files totaling approximately 8-10GB per snapshot. When assets are included, as for another use-case, the files are even bigger. The files are zipped to conserve space with about a 95% compression ratio, resulting in approximately 400MB sets that are stored in per-customer Amazon S3 buckets.

CIMReader

Topological Analysis

CIM Export

Baseline Simulation

Finally we get to the simulation phase. Basically a GridLAB-D model file (a .glm file) is generated for each topological island. The program merges parallel “reinforcing” cables, and ganged transformers. A SWING node is added to the transformer primary. Player files (which can be considered initial conditions at each 15 minute interval) are created from the Cassandra smart meter records. GridLAB-D is invoked to perform load-flow analysis with the given .glm file, reading in the player files and producing recorder files as output. Any power factor (cosφ) is inherent in the complex player and recorder values. The program also stashes GeoJSON data for nodes, edges and polgons in Cassandra for later use as geometric objects by the web-client software to display the results.

Generate Players

• identify loads using Spark query for EnergyConsumer

• query Cassandra to create “player” files (CSV) for loads from smart meter data

• conversion from energy ⇒ power (×4)

• back shift one 15 minute period

• output time stamp and complex value at 15 minute intervals

• time-series duration based on CIM cadence

To generate player files, we first query the CIM data using Spark SQL to find the mRID of EnergyConsumers and the topological node they are associated with. Then we query Cassandra by mRID for each consumer's meter data, to create “player” files in CSV format for each customer load. The readings are converted from energy to power (basically multiplying by four to go from Watt-Hours per fifteen minutes to Watts), They are also back shifted in time, because the meter reading is taken at the end of the 15 minute period, and we require the value at the beginning of the period. The span of the player time-series corresponds to the cadence of CIM file generation - one month.

Player Query

Spark SQL Query
select
    c.ConductingEquipment.Equipment.PowerSystemResource.IdentifiedObject.mRID mrid,
    'energy' type,
    concat(c.ConductingEquipment.Equipment.PowerSystemResource.IdentifiedObject.mRID, '_load') name,
    t.TopologicalNode parent,
    'constant_power' property,
    'Watt' unit,
    n.TopologicalIsland island
from
    EnergyConsumer c,
    Terminal t,
    TopologicalNode n
where
    c.ConductingEquipment.Equipment.PowerSystemResource.PSRType == 'PSRType_HouseService' and
    c.ConductingEquipment.Equipment.PowerSystemResource.IdentifiedObject.mRID = t.ConductingEquipment and
    t.TopologicalNode = n.IdentifiedObject.mRID

Query Result Fragment
mrid   type   name        parent          property       unit island
HAS160 energy HAS160_load HAS160_topo     constant_power Watt TRA173_terminal_2_island
HAS166 energy HAS166_load HAS166_topo     constant_power Watt TRA161_TRA162_terminal_2_island
HAS23  energy HAS23_load  HAS23_topo      constant_power Watt TRA161_TRA162_terminal_2_island
HAS49  energy HAS49_load  HAS49_fuse_topo constant_power Watt TRA153_terminal_2_island
…

GridLAB-D .glm Fragment
…
object load {
    name "HAS160_load_object";
    parent "HAS160_topo";
    phases AN;
    nominal_voltage 400.0V; };
object player {
    name "HAS160_load";
    parent "HAS160_load_object";
    property "constant_power_A";
    file "input_data/HAS160_load.csv"; };
…

Recorders

• Spark queries for “recorder” files for:

· PowerTransformer (N7) apparent power (VA)

· PowerTransformer (N7) current (A)

· BusbarSection (container PSRType_DistributionBox) voltage (V)

· BusbarSection (container PSRType_DistributionBox) apparent power (VA)

· EnergyConsumer (house service) voltage (V)

· EnergyConsumer (house service) apparent power (VA)

· ACLineSegment (N7) current (A)

• after simulation, read recorders, insert into Cassandra

• aggregate over 1, 3 and 24 hours

• time-to-live, 15 minute values “live” a year, daily values forever

Spark queries are also used to add “recorder” file specifications for: apparent power and current of PowerTransformers busbar apparent power and voltages energy consumer apparent power and voltages and cable currents GridLAB-D generates a CSV format file for each output recorder. All CSV files are read into Spark, with 15 minute interval values. The records are tagged with the mRID and units based on the CSV filename. Aggregations are performed over 1, 3 and 24 hours. The records are inserted in bulk into Cassandra. Disk size is proportional to number of elements and duration of the simulation. An average size transformer service area for one month is about 500 million records. When stored in Cassandra it consumes approximately 30MB of disk space. Currently the bottleneck is insertions per second per Cassandra node. Cassandra's “time-to-live” feature is used to age-out the detailed results over time, leaving only the aggregate values.

Recorder Query

Spark SQL Query
select
    concat (p.ConductingEquipment.Equipment.PowerSystemResource.IdentifiedObject.mRID, '_power_recorder') name,
    p.ConductingEquipment.Equipment.PowerSystemResource.IdentifiedObject.mRID mrid,
    p.ConductingEquipment.Equipment.PowerSystemResource.IdentifiedObject.mRID parent,
    'power' type,
    'power_out' property,
    'VA' unit,
    n.TopologicalIsland island
from
    PowerTransformer p,
    Terminal t,
    TopologicalNode n
where
    t.ConductingEquipment = p.ConductingEquipment.Equipment.PowerSystemResource.IdentifiedObject.mRID and
    t.ACDCTerminal.sequenceNumber > 1 and
    t.TopologicalNode = n.IdentifiedObject.mRID

Query Result Fragment
name                  mrid   parent        type  property  unit island
TRA168_power_recorder TRA168 TRA168_TRA169 power power_out VA   TRA168_TRA169_terminal_2_island
TRA180_power_recorder TRA180 TRA180        power power_out VA   TRA180_terminal_2_island
…

GridLAB-D .glm Fragment
…
object transformer {
    name "TRA168_TRA169";
    phases AN;
    from "MUI4786_topo";
    to "PIN2448_topo";
    configuration "_1260kVA16000$400V590e-5+396e-4jΩ"; };
object recorder {
    name "TRA168_power_recorder";
    parent "TRA168_TRA169";
    property "power_out_A";
    interval "900";
    file "output_data/TRA168_power_recorder.csv"; };
…

Postprocessing

• events are business rules for “problem” conditions

• include a severity for color coding at the client

· node voltage deviations ±6% at any time, severity 1

· node voltage deviations ±10% at any time, severity 2

· edge current exceeding 75% for 14 consecutive hours, severity 1

· edge current exceeding 90% for 3 consecutive hours, severity 2

· edge current exceeding 110% at any time, severity 2

· transformer power exceeding 75% for 14 consecutive hours, severity 1

· transformer power exceeding 90% for 3 consecutive hours, severity 2

· transformer power exceeding 110% at any time, severity 2

• save events in Cassandra tagged by transformer service area

• coincidence/diversity factor, load factor, responsibility factor

Dashboard

Use-case: N-1

The N-1 redundancy use-case is basically "Can neighbouring service areas support a problem area?" One starts by assembling the problem in the browser, pulling in transformer service areas from CIM zip blobs stored earlier, from the desired snapshot. The user then adjusts some switch normalOpen/open states and saves it as a project (a “what-if” scenario). A simulation with the new topology is performed and saved in the “project” keyspace in Cassandra. The client software displays a map-based drill-down analysis dashboard of the project. In this and the following use-case, the baseline simulation can be superimposed in these time-series charts. The intent is to automate this analysis over the whole distribution grid eventually.

Use-case: Future DER

The future distributed energy resource use-case is basically "What happens when local generation and batteries become significant?" Again, one starts by assembling the problem in the browser, pulling in a transformer service area, and then the user attaches solar generation and/or battery storage to some energy consumer(s) and saves it as a project. The user then attaches synthetic profile data to the new equipment. These synthetic profiles are at the front-line coal face of current development. A simulation with the added elements is performed and saved in the "project" keyspace in Cassandra. The client software displays a map-based drill-down analysis dashboard as before. The intent is to simulate adding DER to all statistically likely locations over the whole distribution grid eventually.

Use-case: New Consumer

The new consumer use-case is basically "Can the network support a new consumer? If so, from which distribution box and what are the new element dimensions?" Again, one starts by assembling the problem in the browser, pulling in a transformer service area, and then the user adds new cables and energy consumer(s) and saves it as a project. The user then attaches synthetic profile data, or alternatively, links data from a similar existing smart meter to the new consumers. A simulation with the added elements is performed and saved in the "project" keyspace in Cassandra. The client software displays a map-based drill-down analysis dashboard as before. The intent is to provide dimensioning and connection help to planning engineers.

Summary

• provides distribution level simulation with:

· Apache Spark

· Apache Cassandra

· CIM

· smart meter data

• uses topological partitioning

• performs baseline load-flow simulation

• allows project/use-cases:

· N-1

· new distributed energy resources

· new consumer