This article is from the source 'guardian' and was first published or seen on . It last changed over 40 days ago and won't be checked again for changes.
You can find the current article at its original source at http://www.guardian.co.uk/news/2013/jul/23/met-office-space-weather-forecasting
The article has changed 3 times. There is an RSS feed of changes available.
| Version 1 | Version 2 |
|---|---|
| Met Office adds space weather to its forecasting | Met Office adds space weather to its forecasting |
| (2 months later) | |
| Solar flares and coronal mass ejections (CMEs) will now form part of the Met Office's forecasting, as the result of a joint project between UK and US weather and science agencies. | Solar flares and coronal mass ejections (CMEs) will now form part of the Met Office's forecasting, as the result of a joint project between UK and US weather and science agencies. |
| Data collected by the Met Office, the British Geological Survey, NASA and the US National Oceanic and Atmospheric Administration (NOAA) will be combined, and then analysed collaboratively by forecasters. | Data collected by the Met Office, the British Geological Survey, NASA and the US National Oceanic and Atmospheric Administration (NOAA) will be combined, and then analysed collaboratively by forecasters. |
| The move follows the inclusion of "severe space weather" in the National Risk Register, an document published annually by the Cabinet Office that sets out major risks to UK national security as defined by likelihood and potential severity. | The move follows the inclusion of "severe space weather" in the National Risk Register, an document published annually by the Cabinet Office that sets out major risks to UK national security as defined by likelihood and potential severity. |
| The first results of the project came in March 2013, coinciding with the solar maximum - the most active period in the 11 year solar cycle, during which CMEs are roughly 15 times more frequent than during the solar minima. | The first results of the project came in March 2013, coinciding with the solar maximum - the most active period in the 11 year solar cycle, during which CMEs are roughly 15 times more frequent than during the solar minima. |
| The main threat posed by such events comes when a solar flare or CME disrupts the Earth's magnetosphere, causing - in extreme cases - a severe geomagnetic storm. | The main threat posed by such events comes when a solar flare or CME disrupts the Earth's magnetosphere, causing - in extreme cases - a severe geomagnetic storm. |
| The two principal data feeds come via NOAA (from its GOES and ACE satellites) provide information on radiation flux, magnetic field strength, and the speed and density of the solar wind. | The two principal data feeds come via NOAA (from its GOES and ACE satellites) provide information on radiation flux, magnetic field strength, and the speed and density of the solar wind. |
| Depending on the specific data type, feeds are updated either every one minute, five minutes or one hour. | Depending on the specific data type, feeds are updated either every one minute, five minutes or one hour. |
| Open source technologies will form the backbone of the analytical and predictive system, including the NOSQL database system MongoDB, the use of which was instrumental in allowing the Met Office to develop these capabilities in only three months. | Open source technologies will form the backbone of the analytical and predictive system, including the NOSQL database system MongoDB, the use of which was instrumental in allowing the Met Office to develop these capabilities in only three months. |
| The advantage offered by MongoDB over the relational database systems used by the Met Office for traditional weather forecasting is that it handles the incoming data feeds more quickly and easily. | The advantage offered by MongoDB over the relational database systems used by the Met Office for traditional weather forecasting is that it handles the incoming data feeds more quickly and easily. |
| In fact the Met Office had previously experimented with using the same system for both tasks, but the data complexity and speed requirements associated with space weather made this impossible. | In fact the Met Office had previously experimented with using the same system for both tasks, but the data complexity and speed requirements associated with space weather made this impossible. |
| "We had previously done a pilot of the system which we were demonstrating to government to prove capability, which we had done using a traditional relational database, and we had encountered all kinds of problems", said James Tomkins, Data Services Portfolio Technical Lead at the Met Office. | "We had previously done a pilot of the system which we were demonstrating to government to prove capability, which we had done using a traditional relational database, and we had encountered all kinds of problems", said James Tomkins, Data Services Portfolio Technical Lead at the Met Office. |
| "Image data in particular gave us a lot of difficulties, and we had never previously implemented any NoSQL in the Met Office", said Tomkins. | "Image data in particular gave us a lot of difficulties, and we had never previously implemented any NoSQL in the Met Office", said Tomkins. |
| Once the various data feeds are combined using time stamps, the Met Office uses a complex event processing tool - Esper - to identify solar events as quickly as possible and deliver the information to forecasters. | Once the various data feeds are combined using time stamps, the Met Office uses a complex event processing tool - Esper - to identify solar events as quickly as possible and deliver the information to forecasters. |
| "We're getting data very frequently, and the traditional model of storing data in a database and then running a query to check for breaching of a threshold, was simply going to be too slow. | "We're getting data very frequently, and the traditional model of storing data in a database and then running a query to check for breaching of a threshold, was simply going to be too slow. |
| "What Esper does, is it turns a database query on its head. Rather than storing your data and querying it, you store your query - in our case the thresholding - and then as the data comes you just parse it through what is in effect a very complex filter, and it will identify anomalies very quickly", said Tomkins. | "What Esper does, is it turns a database query on its head. Rather than storing your data and querying it, you store your query - in our case the thresholding - and then as the data comes you just parse it through what is in effect a very complex filter, and it will identify anomalies very quickly", said Tomkins. |
| The time gaps between prediction and occurrence are often less than one hour, meaning this ability to read, link and analyse data very rapidly is crucial. | The time gaps between prediction and occurrence are often less than one hour, meaning this ability to read, link and analyse data very rapidly is crucial. |
| "We're now running our own version of the CME prediction model - called Enlil - which is running on a supercomputer in Exeter. | "We're now running our own version of the CME prediction model - called Enlil - which is running on a supercomputer in Exeter. |
| "Once a CME has been emitted, we can run a forecast model to try and predict how large it is, when it might arrive with us, what its polarity will be, and therefore trying to give an estimate of whether it's going to be catastrophic or whether it will bounce off the magnetosphere and not affect us at all", said Tomkins. | "Once a CME has been emitted, we can run a forecast model to try and predict how large it is, when it might arrive with us, what its polarity will be, and therefore trying to give an estimate of whether it's going to be catastrophic or whether it will bounce off the magnetosphere and not affect us at all", said Tomkins. |
| How could a solar even affect us? | How could a solar even affect us? |
| There are four main ways in which such an event could have a terrestrial impact. First, astronauts caught in a storm outside of the protective layers of the atmosphere would receive elevated doses of radiation. | There are four main ways in which such an event could have a terrestrial impact. First, astronauts caught in a storm outside of the protective layers of the atmosphere would receive elevated doses of radiation. |
| The same effect can occur with passengers of high altitude aircraft, and although the relative dose would be much lower, aviation authorities will be using warnings from the Met Office and others to adjust flight paths. | The same effect can occur with passengers of high altitude aircraft, and although the relative dose would be much lower, aviation authorities will be using warnings from the Met Office and others to adjust flight paths. |
| Second, communications reliant on bouncing signals off the ionosphere can be severely disrupted, including radio signals and military detection systems such as over-the-horizon radar. GPS signals are also affected in a similar way, and location data can be out by many miles during a solar event. | Second, communications reliant on bouncing signals off the ionosphere can be severely disrupted, including radio signals and military detection systems such as over-the-horizon radar. GPS signals are also affected in a similar way, and location data can be out by many miles during a solar event. |
| Satellites themselves can also be damaged, either through disruption of orbit resulting from changes to atmospheric density brought about by increased solar radiation, or through damage to microchips caused by the build of positive and negative charge in different parts of equipment. | Satellites themselves can also be damaged, either through disruption of orbit resulting from changes to atmospheric density brought about by increased solar radiation, or through damage to microchips caused by the build of positive and negative charge in different parts of equipment. |
| Finally, ground-level power grids could be damaged by the interactions between aerial wires and solar event-forced magnetic field disruption. | Finally, ground-level power grids could be damaged by the interactions between aerial wires and solar event-forced magnetic field disruption. |
| Estimations of the extent of such damage vary, however, with the US Federal Energy Regulatory Commission (FERC) and North American Electric Reliability Corp (NERC) in a notable and longstanding disagreement over severity. | Estimations of the extent of such damage vary, however, with the US Federal Energy Regulatory Commission (FERC) and North American Electric Reliability Corp (NERC) in a notable and longstanding disagreement over severity. |
| Stay tuned to our big data series for more examples of the ways analytical and predictive capabilities are changing the way we interact with the world, or join the debate either in the comments below or directly via Twitter at @jburnmurdoch | Stay tuned to our big data series for more examples of the ways analytical and predictive capabilities are changing the way we interact with the world, or join the debate either in the comments below or directly via Twitter at @jburnmurdoch |
| Our editors' picks for the day's top news and commentary delivered to your inbox each morning. |