Data science and research

Data science and & research

All our employees have a PhD in mathematics, mathematical statistics, physics, or computer science. We also have a broad network of professionals from academia with an excellent track record in their fields. This allows us to build advanced mathematical and statistical models and solve challenging problems that arise in power trading and finance. We have a unique mix of competences that brings together academic research, knowledge of the industry, and top-level programming skills.

We use data science and research to develop our products and services, and we are looking forward to taking on research projects for our customers.

Evaluation of forecast accuracy. Suppose one has five wind power forecast suppliers who all claim to deliver "the best", most accurate wind power production forecast on the market. Can one verify these claims with any degree of certainty?

This is not as easy as doing a simple t-test. What does "most accurate" really mean? Should one use mean square error as a measure of accuracy? Do forecast errors really have Gaussian distribution? What if the forecast errors are not identically distributed (e.g. the error variance depends on the value of the forecast), or not independent (e.g. there is a systematic bias in the weather forecast and the error terms for consecutive hours are correlated)? And if so, which statistical tests could be appropriate?

Or should one instead use relative error as the measure of the forecast accuracy? After all, the error cost is calculated based on absolute errors, not squared errors. In other words, to minimize the error cost one should devise a special method to compare forecast accuracy, and theoretical knowledge becomes more and more scarce as one departs from the "standard" Gaussian independent identically distributed errors assumption and root mean square deviation metrics...

Finally, should one look at individual windmills, or is it more important to choose a forecast that performs "uniformly better" for all windmills? The forecast quality varies, for example, based on geographical location of the windmill: it is quite common, during winters, for windmills in the SE1 electricity region (northern part of Sweden), to accumulate ice frost on their blades - which reduces the aerodynamic efficiency and can lower the electricity production by up to 20% - a less frequent problem for windmills in the SE4 electricity region (southern part of Sweden), since winters are much milder in that area. Or, in the end, should one aim at obtaining the most accurate aggregate forecast?

And how does one compare two forecast suppliers if the forecasts get progressively less accurate the further ahead one wants to predict? One hour ahead forecast has completely different properties than one day ahead forecast.

These are all complicated questions, and one requires competence and expert knowledge to answer them. The solution to some of the problems might constitute a scientific publication in itself, or a topic of research for a PhD student.

We have a service that evaluates forecast suppliers and a product that uses the theory of forecast combinations to derive a superior forecast - these tools address the forecast evaluation problem described above in great detail. We can do research on similar topics, taking into account peculiarities of your data.

Risk management. Errors cost a lot. If one makes a high electricity production forecast and the regulation prices are extremely up-regulated, the error cost can be as much as €5,000 per hour for each 10 MWh of the forecast error.¹ Is there a way to avoid losses by adjusting the forecast² in order to reduce the risk of being "on the wrong side of the error"? And if yes, how much should the forecast correction be? If there is no correction, for example, there might be rare but very large losses when the prices are extremely up-regulated and the forecast happened to be too high. If, on the other hand, the correction is too large, then the combined loss from the days when the prices are balanced or down-regulated (and the forecast is higher than reality) will be larger than the potential loss from hours with extreme up-regulated prices.

These kinds of questions are common in Extreme Value Theory - a branch of mathematical statistics that studies very rare but highly important, due to the devastating nature of their consequences, events.

The risk management problem described above is oversimplified: one has to consider the frequency and magnitude of the down-regulated prices as well.³ However, one can reduce the losses caused by extreme up- or down-regulated prices by being able to anticipate a slightly higher probability of occurrence of such events using statistical models, and by using EVT to determine the optimal amount of the forecast correction.

The two research questions described above are just an example. More generally, we offer the following services:

• Data science - gathering, cleaning, and organizing data; performing exploratory data analysis, finding patterns and trends; identifying data-analytics problems; analyzing data to discover actionable insights that can lead to competitive advantage; interpreting the data, building mathematical and statistical models, hypothesis testing; using visualization techniques to make reports and communicate the findings to non-experts.
• Research - advanced topics involving the use of mathematical statistics and probability theory, extreme value theory, computer intensive statistical methods, design of experiments, data science, analysis of big data, high-throughput screening, multiple testing; building mathematical models; conducting research towards academic publications; scientific literature review; scientific writing.
• Teaching - designing courses and organizing workshops to teach: VBA programming, writing VBA scripts, developing Excel addins; working with databases, database design and optimization, making efficient SQL queries; statistical data analysis, how to do exploratory data analysis, make plots, and use data visualization techniques to make inferences; computer intensive statistical methods; experimental planning and design of experiments; artificial intelligence and machine learning. We can make a course that targets your needs.
• Consulting - automation of internal processes and optimization; improving the quality of forecasts; setting up cost-efficient and high-performance IT infrastructure to handle big data; recommendations regarding the use of power trading software; assessing the quality of decision making; and we can even help you to become a balance responsible power trading company.

We do research related to other challenging problems that may not necessarily fit in the categories described above, and not necessarily in the domain of power trading.

If it involves big data and requires statistical analysis, data science, or intensive computations - we are excited to be involved and will do our best to deliver exceptional quality results.