CHALLENGE:
- Integrate localized traditional weather data with that of renewable energy to a media audience of 2.5 million through 19 channels.
- Create a robust and flexible digital framework, usable and extensible by non-expert Python programmers, including creation of a Python-based Domain Specific Language (DSL).
SOLUTION:
- Use an agile approach including rapid iterations with continuous client feedback through full read/write access to the development repository and issue tracker.
- Use of test-driven development for a large portion of the project providing code with a high level of test coverage and a high degree of confidence in its reliability.
- Code developed through pair programming, particularly during the early stages of design iteration, resulted in a well-thought-out, consistent, easy-to-use API.
RESULT:
- Delivered a robust infrastructure framework to retrieve data from multiple sources at regular intervals.
- Store and provide processed interpretations in relatable terms on defined timetables to targeted audiences.
- Supported the ultimate objective to educate and develop a culture of responsible energy consumption by giving visibility to climate and energy usage ‘at a family level’ in the context of routine weather reports.
Educating and developing a culture of responsible energy consumption
The EnergizAIR project is presented both to show the innovative technology and methodology to solve the challenge, and to demonstrate Enthought’s time-proven expertise in designing and delivering solutions. The EnergizAIR project ran from 2010-2012, with phase 2 continuing into 2015.
EnergizAIR is a European Commission, Intelligent Energy Europe project designed to give visibility to the energy provided by renewable resources in the context of, and integral with, weather reports. The ultimate objective was to educate and develop a culture of responsible energy consumption by giving visibility to climate and energy usage ‘at a family level’, in the context of routine weather reports.
The project integrated traditional weather data with that of renewable energy for a media audience of 2.5 million through 19 channels.
No existing code enabled building a thoroughly tested, well-documented robust solution.
The task was to create a flexible, extensible infrastructure framework, together with working example code showing its use. The EnergizAIR team added specific details (e.g., which wind farms to use, how to retrieve data for those wind farms, among others).
The Enthought role was to create the data-management framework, required to:
- Retrieve raw energy data and meteorological data from various online sources at regular intervals
- Process raw data: in particular, aggregating over time (e.g., the mean daily energy production for January 2012) and space (e.g. the total energy production for the Lyon region of France)
- Store raw and processed data in a reusable and easily searchable format
- Provide interpretations of processed data in ‘real-world’ terms (e.g., “in Belgium in March 2012, the average household equipped with solar panels produced enough energy to run a refrigerator and 2 televisions”)
- Create and distribute reports in various formats on a regular basis (e.g., a daily email sent to a particular email address, a weekly summary in XML format posted to an FTP server, among other reports.)
An agile approach allowed us to craft the eventual deliverables:
- Rapid iterations, together with continuous feedback from the client, allowed us to converge quickly on a working solution. The customer had full read and write access to the development repository and issue tracker.
-
Use of a test-driven development for a large portion of the project resulted in code with a high level of test coverage and a high degree of confidence in its reliability.
-
Code developed through pair programming, particularly during the early stages of design iteration, resulted in a well-thought-out, consistent, easy-to-use API.
Delivering a system maintainable and extensible by the EnergizAIR team
A major challenge was to produce a solution that was flexible and extensible while remaining readable and requiring minimal Python knowledge to extend and modify. Part of our solution was to create a Python-based Domain Specific Language (DSL). Code examples that set up applications for retrieving photovoltaic, meteorological and wind data from various French and Belgian sources, and publishing daily and weekly reports by email, are available upon request.
Ultimately 2 profiles were created to enable use and management of the system – one for communications and one for IT experts. Project success was evidenced by its presentation at 21 conferences to audiences exceeding 2000.
Visit the project site here.
Quoting the lessons learned from the project website; “When planning a project, one should never underestimate the time and resources needed for IT aspects. Their management takes time… When a project has that big an IT component, it is important to have someone with the same profile on the partners’ team, which can coordinate the IT people”.
The EnergizAIR Infrastructure framework and key interfaces. Enthought responsibility on the project is indicated in the central, grey box. The solution revolved around a number of Traits-based Python class hierarchies, designed to be easily accessible and comprehensible to the programmers who would have to maintain and use the code.