JSON Representation

The PyPES JSON file format is broken into four categories at the top level:

1. String Node ID list

2. String Connection ID list

3. String VirtualTag ID list

4. Key-value entries for each of actual node or connection objects (see Representing Nodes and Representing Connections), where keys are the string IDs from 1-3

Here is the simplest example of this structure, with two nodes, one connection, and no virtual tags:

{
    "nodes": ["PowerGrid", "WWTP"],
    "connections": ["GridToPlant"],
    "virtual_tags": {},
    "PowerGrid": {
        "type": "Network",
        "contents": ["Electricity", "NaturalGas"],
        "tags": {},
        "nodes": [],
        "connections": []
    },
    "WWTP": {
        "input_contents": ["UntreatedSewage", "Electricity"],
        "output_contents": ["TreatedSewage"],
        "tags": {},
        "nodes": [],
        "connections": []
    },
    "GridToPlant": {
        "type": "Wire",
        "source": "PowerGrid",
        "destination": "WWTP",
        "contents": "Electricity",
        "bidirectional": true,
        "tags": {}
    }
}

For integration with data-driven modeling applications, Tag objects that represent real-world sensors can be added to any other object. As a basica example, if there was an electrical meter with a database column name “grid_to_plant_kW”, then we would add the following tag specification to the connection:

"GridToPlant": {
    "type": "Wire",
    "source": "PowerGrid",
    "destination": "WWTP",
    "contents": "Electricity",
    "bidirectional": true,
    "tags": {
        "grid_to_plant_kW": {
            "type": "Flow",
            "units": "kWh",
            "contents": "Electricity",
            "totalized": false
        }
    }
}

One other thing to note about this connection is that “bidirectional” is set to true. In the real world this means that electricity exports are allowed. PyPES will also take this into account (e.g., when querying all connections entering a node, it will return conncections that leave the node with “bidirectional”=``true``).

Certain types of nodes, like the “WWTP” Facility and “PowerGrid” Network objects above, can have nodes and connections nested inside them. They take on the same structure as the top level. For example we could fill in the wastewater treatment facility with some basic processes:

"WWTP": {
    "nodes": ["ProcessA", "ProcessB"],
    "connections": ["AtoB"],
    "ProcessA": {
        "type": "Clarification",
        "input_contents": "UntreatedSewage",
        "output_contents": "PrimaryEffluent",

    }
    "ProcessB": {

    },
    "AtoB": {
        "type": "Pipe",
        "source": "ProcessA",
        "destination": "ProcessB",
        "contents": "Electricity"
    }
}

The following sections will detail how to represent different types of nodes (Representing Nodes), connections (Representing Connections), and tags (Representing Tags) so that the meaning of fields such as “type”, “num_units”, “contents”, etc. is clear.

Putting all the above together, we have the following valid PyPES JSON representation:

{
    "nodes": ["PowerGrid", "WWTP"],
    "connections": ["GridToPlant"],
    "virtual_tags": {},
    "PowerGrid": {
        "type": "Network",
        "contents": ["Electricity", "NaturalGas"],
        "tags": {},
        "nodes": [],
        "connections": []
    },
    "WWTP": {
        "input_contents": ["UntreatedSewage", "Electricity"],
        "output_contents": ["TreatedSewage"],
        "nodes": ["ProcessA", "ProcessB"],
        "connections": ["AtoB"],
        "ProcessA": {
            "type": "Clarification",
            "input_contents": "UntreatedSewage",
            "output_contents": "PrimaryEffluent",
            "num_units": 4,
            "flowrate": {
                "min": null,
                "max": null,
                "avg": 2,
                "units": "MGD"
            }
        }
        "ProcessB": {
            "type": "Aeration",
            "contents": ["PrimaryEffluent", "WasteActivatedSludge"],
            "num_units": 8,
            "flowrate": {
                "min": null,
                "max": null,
                "avg": 1.5,
                "units": "MGD"
            }
        },
        "AtoB": {
            "type": "Pipe",
            "source": "ProcessA",
            "destination": "ProcessB",
            "contents": "Electricity"
        },
        "tags": {}
    },
    "GridToPlant": {
        "type": "Wire",
        "source": "PowerGrid",
        "destination": "WWTP",
        "contents": "Electricity",
        "bidirectional": true,
        "tags": {
            "grid_to_plant_kW": {
                "type": "Flow",
                "units": "kWh",
                "contents": "Electricity",
                "totalized": false
            }
        }
    }
}

A more complex example is avaiable at wrrf_sample.json.

Representing Nodes

The most generic structure of a node is outlined above in JSON Representation, but that just scratches the surface. There are numerous types of nodes, and each one has a number attributes.

For example, a Digestion object must have id, tags, input_contents, output_contents, flowrate, num_units, and digester_type.

And a complete JSON representation of a digester might be:

"AnaerobicDigester": {
    "type": "Digestion",
    "input_contents": "ThickenedSludgeBlend",
    "output_contents": "Biogas",
    "digester_type": "Anaerobic",
    "volume (cubic meters)": 2500,
    "num_units": 3,
    "flowrate": {
        "min": null,
        "max": null,
        "avg": null,
        "units": "MGD"
    },
    "tags": {}
}

Some of the attributes can be left null (as we see in the above example with flowrate). Optional attributes have default values (in the case of flowrate the default is null), so it would be equally correct to leave flowrate out of the JSON file entirely. Below are two tables: first is all the Node subclasses and the second the attributes of each subclass.

Description of all potential attributes of Node class and subclasses

Attribute	Type	Description
id	str	unique global identifier for the node in the WRRF database
tags	dict of Tag or VirtualTag	dictionary of all tags directly associated with this node.
input_contents	list of ContentsType	Contents entering the node (e.g., biogas or waste activated sludge)
output_contents	list of ContentsType	Contents leaving the node (e.g., biogas or waste activated sludge)
nodes	dict of Node	dictionary of all nodes that are children of this node
connections	dict of Connection	dictionary of all connections that are children of this node
elevation	Quantity [1]	facility/process elevation above sea level
flow_rate	tuple of Quantity [1]	min, max, and average flow rate of a single unit
num_units	int	number of identical parallel processes
volume	Quantity [1]	volume of a single unit of the process
horsepower	Quantity [1]	maximum horsepower of a single pump
pump_type	PumpType [2]	type of pump (either constant or variable frequency drive (VFD))
pump_curve	func	function for the efficiency curve of a pump
electrical_efficiency	func	function for the electrical efficiency of a cogenerator
thermal_efficiency	func	function for the thermal efficiency of a boiler or cogenerator
digester_type	DigesterType [3]	type of digester (either aerobic or anaerobic)
gen_capacity	tuple of Quantity [1]	min, max, and average generation capacity of a single engine
capacity	Quantity [1]	maximum battery storage capacity
discharge_rate	Quantity [1]	maximum discharge rate of the battery

Matrix of attributes for each Node subclass

	id	tags	input_contents	output_contents	nodes	connections	elevation	flow_rate	num_units	volume	horsepower	pump_type	pump_curve	electrical_efficiency	thermal_efficiency	digester_type	gen_capacity	capacity	discharge_rate
Node	✓	✓	✓	✓
Network	✓	✓	✓	✓	✓	✓
Facility	✓	✓	✓	✓	✓	✓	✓	✓
Pump	✓	✓	✓	✓			✓	✓	✓		✓	✓	✓
Tank	✓	✓	✓	✓			✓			✓
Reservoir	✓	✓	✓	✓			✓			✓
Screening	✓	✓	✓	✓				✓	✓
Clarification	✓	✓	✓	✓				✓	✓	✓
Aeration	✓	✓	✓	✓				✓	✓	✓
Filtration	✓	✓	✓	✓				✓	✓	✓
Chlorination	✓	✓	✓	✓				✓	✓	✓
Thickening	✓	✓	✓	✓				✓	✓	✓
Digestion	✓	✓	✓	✓				✓	✓	✓						✓
Conditioning	✓	✓	✓	✓				✓	✓
Cogeneration	✓	✓	✓	✓					✓					✓	✓		✓
Boiler	✓	✓	✓	✓					✓						✓		✓
Flaring	✓	✓	✓
Battery	✓	✓	✓	✓														✓	✓

An example of how to define all the potential attributes is available in wrrf_sample.json.

Representing Connections

The most generic structure of a node is outlined above in JSON Representation, but that just scratches the surface. There are numerous types of connections, and each one has a number attributes.

Description of all potential attributes of Connection class and subclasses

Attribute	Type	Description
id	str	unique global identifier for the node in the WRRF database
tags	dict of Tag or VirtualTag	dictionary of all tags directly associated with this node.
contents	list of ContentsType	Contents moving through the connection (e.g., biogas or waste activated sludge)
source	Node	Starting point of the connection
destination	Node	Endpoint of the connection
bidirectional	bool	whether flow can go from destination to source. False by default
exit_point	Node	The child node from which this connection leaves its source.
entry_point	Node	The child node at which this connection enters its destination.
flow_rate	tuple of Quantity [1]	min, max, and average flow rate of a single unit
diameter	Quantity [1]	inner diameter of a pipe
friction_coeff	float	Friction coefficient for a pipe
pressure	Quantity [1]	Minimum, maximum, and average pressure in a pipe
heating_values	tuple of Quantity [1]	The lower and higher heating values of the gas in the pipe.

Matrix of attributes for each Connection subclass

	id	tags	contents	source	destination	bidirectional	exit_point	entry_point	flow_rate	diameter	friction_coeff	pressure	heating_values
Connection	✓	✓	✓	✓	✓	✓	✓	✓
Wire	✓	✓	✓	✓	✓	✓	✓	✓
Pipe	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓

An example of how to define all the potential attributes is available in wrrf_sample.json.

Representing Tags

Both nodes and connections can have tags nested inside them. These tags represent sensors or data points at the facility. Two types of tags exist in PyPES: Tag and VirtualTag objects.

Basic Tag

As with other objects, the tags dictionary is nested within the corresponding Node or Connection, and the keys are the string IDs for the facility’s database column names.

The JSON format to represent a single tag in PyPES is as follows:

"TankVolume": {
    "type": "Volume",
    "units": "gallons",
    "contents": "FatOilGrease"
}

Where type is a string from TagType Enum, units is a Pint parseable string, and contents_type a string form ContentsType Enum.

Multiple tags can be attached to a node. For example:

"FOGTank": {
    "type": "Tank",
    "contents": "FatOilGrease",
    "tags": {
        "TankVolume": {
            "type": "Volume",
            "units": "gallons",
            "contents": "FatOilGrease"
        },
        "TankLevel": {
            "type": "Level",
            "units": "feet",
            "contents": "FatOilGrease"
        }
    }
}

VirtualTag

The VirtualTag class provides users with a powerful way to combine data from existing sensors to create new “virtual” sensors. Specifically, a user can define their own function or lambda expression that takes in other Tag or VirtualTag objects and computes the resulting data.

These combinations can be quite simple. For example, let’s calculate the cogenerator efficiency from the following plant configuration:

"WWTP": {
    "nodes": ["AnaerobicDigester", "Cogenerator", "VirtualDemand"],
    "connections": ["CogenElecToFacility", "DigesterToCogenerator"],
    "virtual_tags": {}
    "Cogenerator": {
        "Cogenerator": {
        "type": "Cogeneration",
        "num_units": 1,
        "input_contents": ["Biogas", "NaturalGas"],
        "output_contents": ["Electricity", "Heat"],
        "tags": {}
    },
    "AnaerobicDigester": {
        "type": "Digestion",
        "input_contents": "ThickenedSludgeBlend",
        "output_contents": "Biogas",
        "digester_type": "Anaerobic",
        "volume (cubic meters)": 2500,
        "num_units": 2,
        "tags": {}
    },
    "VirtualDemand": {
        "type": "Network",
        "contents": ["Electricity", "Heat"],
        "tags": {},
        "nodes": [],
        "connections": []
    },
    "CogenElecToFacility": {
        "type": "Wire",
        "source": "Cogenerator",
        "destination": "VirtualDemand",
        "contents": "Electricity",
        "bidirectional": false,
        "tags": {
            "ElectricityGeneration": {
                "type": "Flow",
                "units": "kilowatt",
                "contents": "Electricity",
                "totalized": false
            }
        }
    },
    "DigesterToCogenerator": {
        "type": "Pipe",
        "source": "AnaerobicDigester",
        "destination": "Cogenerator",
        "contents": "Biogas",
        "heating_values": {
            "lower": 600,
            "higher": 700,
            "units": "BTU/scf"
        },
        "tags": {
            "Digester1GasFlow": {
                "type": "Flow",
                "units": "SCFM",
                "source_unit_id": 1,
                "dest_unit_id": "total",
                "contents": "Biogas",
                "totalized": false
            }
            "Digester2GasFlow": {
                "type": "Flow",
                "units": "SCFM",
                "source_unit_id": 2,
                "dest_unit_id": "total",
                "contents": "Biogas",
                "totalized": false
            }
        }
    }
}

First, we need to add a VirtualTag that sums the biogas flows into the cogenerator. Since the child Tag objects are all inside “DigesterToCogenerator” Pipe, this can be done at the lowest level:

"DigesterToCogenerator": {
    "type": "Pipe",
    "source": "AnaerobicDigester",
    "destination": "Cogenerator",
    "contents": "Biogas",
    "heating_values": {
        "lower": 600,
        "higher": 700,
        "units": "BTU/scf"
    },
    "tags": {
        "Digester1GasFlow": {
            "type": "Flow",
            "units": "SCFM",
            "source_unit_id": 1,
            "dest_unit_id": "total",
            "contents": "Biogas",
            "totalized": false
        }
        "Digester2GasFlow": {
            "type": "Flow",
            "units": "SCFM",
            "source_unit_id": 2,
            "dest_unit_id": "total",
            "contents": "Biogas",
            "totalized": false
        },
        "Digester3GasFlow": {
            "type": "Flow",
            "units": "SCFM",
            "source_unit_id": 3,
            "dest_unit_id": "total",
            "contents": "Biogas",
            "totalized": false
        }
    },
    "virtual_tags": {
        "BiogasProductionCombined": {
            "tags": ["Digester1GasFlow", "Digester2GasFlow", "Digester3GasFlow"],
            "operations": "lambda x, y, z: x + y + z",
            "units": "SCFM"
        }
    }
}

Then, another VirtualTag could be used to divide the biogas production by electricity generation. This tag would be located at the facility level since it is combining data from two different Connection objects.

"WWTP": {
    "nodes": ["AnaerobicDigester", "Cogenerator", "VirtualDemand"],
    "connections": ["CogenElecToFacility", "DigesterToCogenerator"],
    "virtual_tags": {
        "ElectricalEfficiency": {
            "tags": ["BiogasProductionCombined", "ElectricityGeneration"],
            "operations": "lambda x, y: x / y",
            "units": "SCFM / kW"
        }
    }
}

Members of VirtualTag class

Attribute	Type	Description
id	str	unique global identifier for the tag in the WRRF database
units	Unit [1]	Units for the tag. Can be null, e.g., if a Boolean variable.
tag_type	TagType	Type of data saved under the tag. E.g., InfluentFlow or RunTime.
tags	list of Tag	tags to combine according to specified operations
unary_operations	list of str	Unary operations to apply before combining tags
binary_operations	list or str	Binary operations to apply before combining tags
parent_id	str	Identifier for the parent object (either a Node or Connection)
totalized	bool	True if data is totalized, or a running summation reset periodically (usually daily). False otherwise.
contents	ContentsType	Contents measured by the tag (e.g., biogas or primary effluent)

TagType Enum

Members of TagType enum

Member	Description	Default Units
Flow	flow of any contents (electricity, water, gas, sludge)	kW (electricity) or m3 d-1 (fluids)
Volume	volume of fluid in a tank	m3
Level	height of fluid in a tank	m
Pressure	force per unit area	Pa
Temperature	average kinetic energy of particles	K
RunTime	binary variable indicating if equipment is on (1) or off (0)	unitless
RunStatus	fraction of time equipment is on	unitless
VSS	concentration of volatile suspended solids	mg L-1
TSS	concentration of total suspended solids	mg L-1
TDS	concentration of total dissolved solids	mg L-1
COD	chemical oxygen demand	mg L-1
BOD	biochemical oxygen demand	mg L-1
pH	measure of acidity	unitless
Conductivity	ease with which an electric charge moves	Siemens m-1
Turbidity	the opaqueness of a fluid	NTU
Rotation	number of revolutions (useful for motors/pumps)	RPM
Efficiency	fraction of a quantity retained during a process (e.g., conversion of heat to electricity)	unitless
StateOfCharge	fraction of max capacity currently available in a battery	unitless
InFlow	flow into a node	kW (electricity) or m3 d-1 (fluids)
OutFlow	flow out of a node	kW (electricity) or m3 d-1 (fluids)
NetFlow	net flow (i.e., flow in minus flow out) of a node	kW (electricity) or m3 d-1 (fluids)

ContentsType Enum

A fundamental facet of process engineering is the conversion of reactants to products. In PyPES, these changes are represented by the ContentsType enum. For consistency, both input_contents and output_contents are represented as lists even when there is only a single member. For example, a Cogeneration object typically has input_contents=[ContentsType.Biogas, ContentsType.NaturalGas] and output_contents=[ContentsType.Electricity] Each node has distinct input and output contents, while a connection is assumed to have a single ContentsType list since it is simply transporting the contents.

As a shorthand, instead of specifying input_contents and output_contents separately, the user can simply enter contents and the value will be set to both attributes. I.e., the below are two ways to represent the same node in JSON format:

"PowerGrid": {
    "type": "Network",
    "contents": ["Electricity", "NaturalGas"],
    "tags": {},
    "nodes": [],
    "connections": []
}

"PowerGrid": {
    "type": "Network",
    "input_contents": ["Electricity", "NaturalGas"],
    "output_contents": ["Electricity", "NaturalGas"],
    "tags": {},
    "nodes": [],
    "connections": []
}

Supported ContentsType values are shown in the table below.

Members of ContentsType enum

hline Member	Description
UntreatedSewage	raw wastewater before any treatment
PrimaryEffluent	liquid outflow from primary clarification
SecondaryEffluent	liquid outflow from secondary clarification
TertiaryEffluent	outflow from tertiary treatment
TreatedSewage	fully treated (i.e., disinfected) plant effluent
DrinkingWater	conventionally treated or desalinated water
PotableReuse	water recovered at potable standards
NonpotableReuse	water recovered at non-potable standards
Biogas	mix of CH4 and CO2 produced by digesters
NaturalGas	fossil CH4 purchased from the grid
GasBlend	a blend of fossil natural gas and biogas
FatOilGrease	fats, oils, and greases (FOG)
PrimarySludge	settled solids from primary clarification
TPS	thickened primary sludge
WasteActivatedSludge	settled solids from secondary clarification
TWAS	thickened waste activated sludge
Scum	solids skimmed off the top of aeration basins
FoodWaste	organic food waste delivered to the facility
SludgeBlend	a mix of unthickened solids (e.g., scum and primary sludge)
ThickenedSludgeBlend	a mix of thickened solids (e.g., TPS and TWAS)
Electricity	includes grid purchases and on-site generation
Brine	saline waste stream from desalination process
Seawater	water from the ocean
SurfaceWater	water from rivers, streams, or lakes
Groundwater	water pumped from underground
Stormwater	stormwater runoff (for separated drainage systems)
Heat	heat energy generated by on-site processes
Oil	oil of any form (e.g., combustion or lubrication)
Grease	grease for any purpose (e.g., digestion or lubrication)

num_units Attribute

There are often identical parallel processes in a treatment train. The num_units attribute exists to ease the effort needed to represent these identical objects.

For example, the anaerobic digester from VirtualTag was specified with three digesters in parallel. That example shows how flow from the three digesters can be tracked separately by specifying a source_unit_id or dest_unit_id for each Tag.

"num_units": 3

By default num_units is set to 1, meaning there is only a single instance and no parallel processes.

elevation Attribute

Elevations can be assigned to some nodes to help calculate the headloss or pressure drop throughout the system. Currently, elevation is only supported by Facility, Pump, Tank, and Reservoir nodes, but that could easily be extended in the future.

By default the elevation attribute is null. To assign an elevation to a node, simply add the following entry to its JSON dictionary:

"elevation (meters)": 10

Currently, the units are hardcoded as meters, but this will soon be modified to match the dictionary-style unit-parsing from flow_rate Attribute and gen_capacity Attribute.

volume Attribute

Elevations can be assigned to some nodes to help calculate the headloss or pressure drop throughout the system. Currently, elevation is only supported by Tank, Reservoir, Aeration, Filtration, Chlorination, Thickening, Digestion, and Conditioning nodes, but that could easily be extended in the future.

By default the volume attribute is null. To assign a volume to a node, simply add the following entry to its JSON dictionary:

"volume (cubic meters)": 10

Currently, the units are hardcoded as cubic meters, but this will soon be modified to match the dictionary-style unit-parsing from flow_rate Attribute and gen_capacity Attribute.

flow_rate Attribute

Flow rates in PyPES are defined as tuples of the form (min, max, avg). Any member of the tuple can be left as None. For example, if there is only a minimum of maximum flow rate. Units must be specified (e.g., m³ / day or ft³ / min) as text strings. The average flow rate can also be used to keep track of design flow, which will be between min and max. Note that this is the flow rate for a single unit in a parallel process, so the combined flow rate would require multiplying by num_units.

Both “flowrate” and “flow_rate” are acceptable keys to use in the JSON representation. So the following two represent the same flow_rate attribute:

"flowrate": {
    "min": 0,
    "max": 1000,
    "avg": 100,
    "units": "m3pd"
}

"flowrate": {
    "min": 0,
    "max": 1000,
    "avg": 100,
    "units": "m3pd"
}

Outside of JSON, node.Node.set_flow_rate can be used to change a Node or Connection objects flow_rate attribute.

gen_capacity Attribute

Generation capacity of a boiler or cogenerator is represented as a (min, max, avg) tuple just like the flow_rate Attribute:. Note that this is the generation capacity for a single engine if there are multiple, so the combined flow rate would require multiplying by num_units.

Both “generation_capacity” and “gen_capacity” are acceptable keys to use in the JSON representation.

"generation_capacity": {
    "min": 200,
    "max": 650,
    "avg": 450,
    "units": "kW"
}

Since both heat and electricity are forms of energy, the generation capacity for the Cogenerator is tied to the electricity generation and for the Boiler to heat generation. Then, for the Cogenerator the heat generation capacity can be calculated by converting from electricity to heat using electrical_efficiency and thermal_efficiency.

heating_values Attribute

capacity Attribute

The capacity attribute provides the maximum storage capacity for a Battery object. To assign a value to the capacity in JSON:

"capacity (kWh)": 2000

Currently, the units are hardcoded as kilowatt-hours, but this will soon be modified to match the dictionary-style unit-parsing from flow_rate Attribute and gen_capacity Attribute.

discharge_rate Attribute

The discharge_rate attribute provides the maximum rate of discharge (and at the moment charge, but these will be separated in the future) for a Battery object. To assign a value to the discharge_rate in JSON:

"discharge_rate (kW)": 10

Currently, the units are hardcoded as kilowatts, but this will soon be modified to match the dictionary-style unit-parsing from flow_rate Attribute and gen_capacity Attribute.

[1] (1,2,3,4,5,6,7,8,9,10,11,12)

from Pint

[2]

PumpType is an enum that contains two members: Constant or VFD

[3]

DigesterType is an enum that contains two members: Aerobic or Anaerobic