On-Prem Solution for the Selection of Wind Energy Models - Databricks

On-Prem Solution for the Selection of Wind Energy Models

The renewable energy industry has only recently started to rely on data-driven models on applications that have traditionally required complex physical solutions. In this talk, we would like to show how we leverage Spark, Keras and (in our case, on-prem) high performance computing (HPC) infrastructure to  potentially tackle common and interesting problems in the wind-related industry (saving hours of CPU-consuming simulations).

We use:

  • Apache Spark and Hive for data preparation and a combination of different data sources (some of them in the range of the petabytes scale).
  • Keras for model training/generation.
  • HPC for coordination and node-wide training of hyperparameters.


« back
About Ana Maria Martinez Fernandez

Ana M. Martinez works as a large-scale Data Specialist on Vestas Wind Systems A/S, the energy industry's global partner on sustainable energy solutions. She received her Ph.D. in the area of probabilistic machine learning in 2012. She was part of the research team and core developer of the AMIDST Toolbox (Analysis of MassIve Data STreams) http://www.amidsttoolbox.com/. She has worked on the field of machine learning and supervised classification on large amounts of data using probabilistic graphical models at different universities (Aalborg/Monash University and University of Castilla-La Mancha).