For my National Energy Market (NEM) industry project I’ve soured data from the below data sources. These sources are managed by the Australian Energy Market Operator (AEMO).
Within the below report sites are data repositories that report on generation output, financial metrics as well as operational status of the NEM. With these insights you can begin to understand how the NEM operates day to day and what events are encountered and responded to.
AEMO Market Data NEMWEB
I’ve selected what I believe to be the most interesting data sources that could be extracted and loaded into a database and presented on a BI tool (i.e. Power BI or Grafana).
Note – I later found the NEMWEB GitHub site which listed all the same data sources for their data project confirming I was on the right track. For my project I’ve included a few more sources that I believe can gain other operational insights.
| NEMWEB Data source (NEM_Name) | Purpose / Insights / Sources |
| actual_generation (Next_Day_Actual_Gen) | Energy output (MWh (but really MW)) per DUID at 5-min intervals for non-scheduled generators in the NEM. Updated daily. Reports on previous days power generation. – Current source – Archive source Use case(s): – Analysis of non-scheduled power generation |
| auction_units (Auction_Units_Reports) | Provides details of registered generation and load units bidding parameters for NEM auctions – Current source – Archive source N/A Use case(s): – Price forecasting and simulation – Competition analysis |
| dispatch_price (Dispatchprices_PRE_AP) | RRP and interconnector prices at dispatch interval. Note before administered price (Pre-AP) – Current source – Archive source Use cases(s): – Call out price spike as market stress indicator – Price and revenue modelling |
| dispatch_scada (Dispatch_SCADA) | 5-minute SCADA snapshots of scheduled/semi-scheduled DUID in the NEM – Current source – Archive source Use case(s): – Compare actual versus target versus under versus over generation events. |
| duid_metadata (NEM Registration and Exemption List.xlxs) | Station names, regions, fuel types, capacities – Current source (Page: “PU and Scheduled Loads” ) Use case(s): – Gives context to Dispatch Unit Identification (duid) sites. i.e. Name, location, technology, category. |
| market_notice (Market_Notice) | Operator alerts in the NEM — planned and unplanned outages, curtailments, constraint lifts – Current source – Archive source (No data) Use cases(s): – Identify outage and constraint events – Correlate generation shortfalls with price spikes using the underlying notices. |
| rooftop_pv_actual (ROOFTOP_PV) | PV (Solar) generation (behind the meter) estimates by NEM region. – Current source – Archive source Use cases(s): – Understand daily and seasonal rooftop PV generation patterns. |
| tradingis_interconnector (TradingIS_Reports) | Regional flow and limits between NEM zones at 30-minute intervals – Current source – Archive source Use cases(s): – Identify interconnector utilisation trends over time. |