What The Weather Can Teach Us About Open Data

We spoke with Ken Mylne, Head of Weather Science Numerical Modelling at the Met Office


Ken Mylne is the Head of Weather Science Numerical Modelling at the Met Office, the United Kingdom’s national weather service. Naturally, these services rely heavily on data to predict what the weather is most likely to be like days and even weeks in advance. The Met Office also works with data to predict the potential impact of global warming some way down the line, and it is sophisticated in how it handles and analyses the vast datasets it both records itself and is given by international partners.

We sat down with Ken to discuss the necessity for open data in the weather industry, the general shift towards open data in business, and the role of government in driving open data projects forward.

Hi Ken, so how important is it for organizations to open up their datasets, in your industry and in every industry?

Well, I think it’s probably fair to say that meteorology has been doing that for generations. And the reason for that is that weather does not know national boundaries – it’s a global business. We can only forecast the weather for the UK by knowing what the weather is doing over in the United States, and over the whole of the world, actually. Now, certainly, to be able to predict more than a day or so ahead, we need to know what’s happening a long way upstream.

So, sharing data has been at the core of meteorology for a hundred years or more. We have a United Nations agency, called the World Meteorological Organization, that has 191 member states’ met services, which has the prime purpose of the sharing of meteorological data.

When you’re sharing data internationally, in different communities, are there challenges around that? Are there standard practices that need to be followed?

Of course, yeah. And technology is at the forefront of that. But, yes, standard practices – we’ve been defining standard codes, standard ways of providing and sharing data, since way before most industries had even thought about the need for that sort of thing, because of this need to exchange data freely. So, the World Meteorological Organization is central to defining codes for sharing data. And the Met Office, as a major member of WMO, has been very involved in that for many decades, and still is.

Find out more about data storage and data sharing at the Chief Data Officer Summit this May 9-10 in San Francisco. Check out the full schedule here.

Can open data and data privacy coexist? As data privacy has become a more important thing do you think that’s possible?

It’s changing, and it’s partly changing through evolution of business models – how organizations like the Met Office work. Historically, because we as an organization are a service organization, as well as providing public service forecasts, the way that the government of the UK has asked us to work is that we also operate a commercial business. We sell weather services. And so, historically, we’ve tended to be quite protective of our data, in order to make sure that we keep some commercial opportunities for ourselves.

But, over the last few years, particularly with the growth of open data strategies and things, we’re changing that strategy. We still have a commercial operation, but we’re making far more of our data open and freely available. We’re following more the line that the Americans have followed for a long period of time – to make that data freely available to the industry as a whole. And then our services, particularly our commercial business operation, using exactly the same data that is available to everybody else, but focusing on how to turn that into value for their customers.

So, it’s a continually evolving pattern, but we are making far more data available openly than we used to. That’s not to say there’s not still some data that we keep private, we have to. We also provide meteorological services to the military, for example, so there’s lots of data for them that clearly has to be kept private. And then there’s the other data that we provide for specific customers, of course, which is private. And we can’t, just because of the sheer volume of data we provide, make everything truly open, so we have some data that we just keep internally.

In general, what do you think the biggest benefit of the increased awareness and use of open data in society has been?

Well, I think that more and more people are looking at how to get value out of data that’s available. If you make data freely available it gives the analytics industry the opportunity to find value that we might never have thought of within it.

And, finally, do you think governments are moving forward with their open data programs?

Yes, and our whole change of strategy is consistent with that. There is generally a view that making more data available, making it more open, gives opportunities for industry to find new value in information that we might never have found otherwise, and getting different organizations working closely together.

So, just to give one example, we are involved in a partnership in the UK called the Natural Hazards Partnership, which is bringing together lots of different government organizations such as the British Geological Survey, the Met Office of course, and the Environment Agency, working together sharing data in common formats to make all our data more easily accessible. But, also, to see how we can address problems with natural hazards in a common framework so that everybody can benefit from it in the same way. So, that’s an example of different government organizations working closely together, around data, making it more freely available, and working on common standards and things. Of course, having common standards helps everyone to access it better. 


Read next:

Why We Need Data Visualization To Understand Unstructured Data