Recode Weather

Why TDCF migration is needed?

Your bulletins should be in TDCF

“The Commission agreed on the deadline of November 2014 to stop the parallel distribution of TAC and TDCF data for the category 1 as well as the category 2 (satellite observations) and 4 (marine data). After November 2014, the TAC may be used only for the exchange of data between two NMHS under bilateral agreement.”

(WMO-No. 1070. Commission for Basic Systems Extraordinary session – Final report, 2010)

Recode Weather is designed to help you resolve some of the severe issues with development of your National Migration Plan, allowing to generate BUFR observations without an immediate upgrade of observing hardware/software, and at the same time to provide backward compatibility for legacy systems based on Traditional Alphanumeric Codes until those are upgraded or replaced.

In accordance with the WMO TDCF Migration Plan, (Table Driven Code Forms such as BUFR) was completed in November 2010. By November 2014 dual dissemination of BUFR and TAC bulletins might be terminated. After this date, meteorological observations such as SYNOP, TEMP, PILOT, CLIMAT should not be disseminated to GTS network (UK, Ireland and Netherland has already announced termination of dissemination of TAC SYNOP bulletins). Instead, observations all over the world should be transmitted in BUFR/CREX.

TDCF Migration is a process of preparing every National Meteorological Service for this step.

If you do not migrate…

Besides category 1 data migration which is due this year, WMO plans to have all other meteorological data being interchanged in TDCF in the following years. Therefore without reliable support of TDCF in your system you will not be able to communicate with rest of the meteorological world, to understand the data received as well as to send data to others.

Keep up with others and be prepared

Indeed Migration is long and difficult process consisting of several steps and at the end it most likely requires to upgrade and/or to change of your current hardware and software. This could be very costly and quite often even not possible due to missing support for legacy software.

Recode Weather is designed to help National Meteorological Services in their process of migration to Table Driven Code Forms (TDCF) and to play a key role in developing the National Migration Plan. It is based on “Migration by Zones” concept which allows continuous use of your current systems until they are upgraded or retired.

With BUFR migration tool your way to TDCF could be much smoother and less complicated. BUFR migration tool offers you standalone solution for transparent, fully-automated bi-directional conversion while adding proper metadata. TDCF descriptor tables and TM templates can be updated without software change.

Key benefits of Recode Weather

  • Flexible way towards TDCF codes — it brings bidirectional bridging between TDCF and TAC which allows you to generate TDCF bulletins without an immediate upgrade of observing hardware/software (leading to cost saving). You can issue your observations in BUFR/CREX simply by re-creating them from existing TAC codes with additional metadata or data.
  • Support legacy systems — continue using your TAC-consuming systems until they are upgraded or retired as scheduled, because TDCF data arriving from other RTH/NMC’s can be converted back to their TAC for legacy systems, where possible. Please consider that existing systems might not be easy to upgrade because:
    • they are not maintained or developed any longer,
    • they cannot be adapted to new compilers or operating systems,
    • original developers are no longer available,
    • implementing the change is too expensive.

Re-creating old codes from BUFR prolongs usability of your current systems to allow proper development of new tools and migration when ready part-by-part.

  • Service Oriented Architecture — the BUFR Migration Tool provides REST, SOAP or file input/output interfaces, so it can be easily integrated with your message switching system.
  • External SQL database integration — together with Moving Weather switching system it allows direct compilation of BUFR bulletins out of your external observation database, as well as to decode TDCF data and store it directly into your SQL tables.
  • Open for updates — system is driven by XML-based configuration, new formats can be simply supported just by adding new transformation rules.
  • Generated data fully conforms to relevant WMO standards and recommendations — Output is compliant with WMO IPET-DRC templates. Input is backward compatible with various BUFR editions and template revisions. Therefore you can be sure you are producing TDCF messages in accordance with current standards while you can still understand other parties not on the same level of compliance.

Do you need your Message Switch upgraded as well?

In case you consider upgrading also your Automated Message Switching System, we are happy to offer you our Moving Weather which provides you significant advantages! First of all it has BUFR migration tool already integrated with Graphical user interface, so you can control the process directly from message switch. Moreover, since Moving Weather is best and most advanced WMO message switching system available, it is already compatible with WIS capability (WMO Information System). It can be upgraded to WIS system Discover Weather at any time you need it.

We are proud to propose to you easy and understandable guidance on how to migrate to TDCF in a very convenient and simple way, using using the most user-friendly tool available.

Try BUFR migration tool NOW for free!

See what you should do

Goal of TDCF migration

“Table-driven Code Forms (TDCF) are the solution to satisfy the demands of rapidly evolving science and technology, in view of their self-description, flexibility and expandability.”

(The Fourteenth WMO Congress (Cg-XIV), 2003)

The goal of the TDCF migration is the replacement of data exchange in Traditional Alphanumeric Codes (TAC) by Table-Driven Code Forms (TDCF). TDCF offers many advantages comparing to TAC such as: new data types, higher resolution and precision of data, metadata, flexibility and expand-ability. In near future all data in meteological network should be encoded in TDCF.

There are two codes for data encoding: BUFR and CREX.

  • BUFR – the Binary Universal Form for the Representation of meteorological data – is universal and portable binary format to store observation data and metadata, also offering packing of data.
  • CREX – Character form for the Representation and EXchange of data – is alphanumeric code that provides human readability, and is only a temporary step to BUFR and should be used where digital network is not available.

TDCF codes “are the ideal codes for observations and the best adapted to the fast scientific and technological evolution of the twenty-first century”. (Summary of the Plan for Migration to Table-Driven Code Forms (TDCF), WMO, 2002).

If you do not migrate…

Besides category 1 data migration which is due this year, WMO plans to have all other meteorological data being interchanged in TDCF in the following years. Therefore without reliable support of TDCF in your system you will not be able to communicate with rest of the meteorological world, to understand the data received as well as to send data to others.

WMO migration status

Below is the current WMO plan for Migration, showing where National Meteorological Services are supposed to be right now:

 

Migration process explanation

According to WMO Migration Plan, WMO members should:

  1. Define a Migration Contact Point;
  2. Establish a National Migration to TDCF Steering Group (MTSG);
  3. Identify impacts of migration on national operation;
  4. Produce a national migration plan;
  5. Plan their requests for equipment and software (resources commitment);
  6. Start a national training program on TDCF;
  7. As needed, modify or replace software used for observation, encoding, data concentration, dissemination systems, input data processing, message switching, decoding, visualization and archiving;
  8. Evaluate the implications, due to the migration process, on WMO Members’ resources for development and operation;
  9. Reserve the budget resources necessary to implement the migration.

BUFR migration tool is ready to help you with probably the most expensive and difficult step on your way to TDCF. It allows you to generate BUFR observations and data without an immediate upgrade of the observing hardware/software, support current legacy systems until they are upgraded or retired as scheduled.

Migration problems

The most direct way to migrate your data to TDCF is to produce TDCF data from measurement systems. Migration shall not affect operation of existing systems (both producing & processing) – however, there are practical issues in upgrading legacy systems for new codes, such as they might not be developed any longer, they cannot be ported for new compilers or operating systems, original developers are no longer available/reachable, or implement change is too expensive.

Solution is in bidirectional bridging of BUFR <—> TAC formats, which allows co-existence new and old systems. This will reduce need upgrade or change your systems immediately, as data currently generated in traditional forms will be automatically encoded with proper Metadata to BUFR. The other way round by providing old codes from BUFR prolongs usability of your current systems to allow proper development of new tools and migration when ready part-by-part.

Metadata enrichment

It is important to note that plain Conversion is not a Migration. Because resulting data will miss additional precision and important metadata. Therefore TDCF Migration Tool provides more by Enriching BUFR with proper metadata or additional data, in order to provide real value and step forward.

For example with for SYNOPs additional metadata contain station coordinates, station name, elevation, height of each sensor above ground, type of sensors, etc. Since SYNOP (in FM-12 alphanumeric code) contains data in sufficient precision but it does not carry proper metadata, their are added in TDCF Migration Tool and during encoding in order to produce fully rich & valid BUFR with Synoptic observation.*

Software features

Highlights

BUFR migration tool helps you resolve major issues with Migration and will be key component for developing your National Migration Plan. With BUFR migration tool is your way to TDCF can be much more easy and less painful as it solves backward compatibility issues.

  • Offers you easy and flexible way towards TDCF codes. It brings bidirectional bridging TDCF <—> TAC which allows you to generate TDCF observations without an immediate upgrade of the observing hardware/software (i.e. cost reduction).
  • Being able to immediately your observations in BUFR/CREX simply by re-creating them from TAC codes with additional metadata or data. In addition, it allows you to use current TAC-based legacy consuming systems until they are upgraded or retired as scheduled, as the TDCF data arriving from other RTH/NMC’s will be converted back to their TAC (where possible) for legacy systems. This will reduce the benefit of metadata you will be receiving from others, on the other hand it closes the gap between current and new technology.
  • Transparent, fully-automated standalone encoding and decoding solution. Integration with your present systems will be thanks to Service Oriented Architecture (SOA) as easy as it could be. You can use REST or SOAP web service, file-based input/output (tool watches certain directory, where if you place a file, it will pick it up, convert, and it will place the result to outgoing directory), C++ binding, command line interface or Python binding (to support flexible scripting tool these functions are accessible from Python as API — so other wrappers for processing can be easily created as need).
    Moreover, when used in conjunction with Moving Weather it is also possible to make use of SQL interface which allows direct compilation of BUFR bulletins out of your observation database, as well as to decode data directly into your SQL tables.
  • Supporting REST and SOAP calls makes things very simple, for example converting SYNOP (from AWS collecting point) to BUFR is easy by invoking URL such as: http://tdcf.your-met-office.domain/convert?SMGR21+LGAT+++170000+AAXX+17004+++16667+12965+03304+… Where in response to this URL request a equivalent BUFR code will be sent for download. This of course works the other way round too: by passing BUFR (as URL-encoded or POST) and obtaining SYNOP as URL response. Network-oriented calls makes it simple to integrate in network environment and even can operate over proxy servers or as secured service (SSL).
  • Fast and easy expansibility — system is created to be driven by configuration. Conversions are programmed based on the XML functionality: the module features a XSLT-Processor and the conversion programs are actually XSL transformations. Thanks to this it is possible to support new formats just by adding new transformation rules.
  • Generated data fully conforms to WMO standard and recommendations — with the compliance to WMO IPET-DRC templates on output, the backward compatibility on input, and with configurable formats you can be sure you are producing TDCF messages in accordance with the up-to-date standard. The conversion module is easily upgradable by new templates.
  • BUFR migration tool is fully multi-platform system – it is available for Microsoft Windows, Linux, SUN Solaris (both 32-bit and 64-bit).
  • Incoming observations coded in TAC, such as SYNOP, AWS, PILOT, TEMP, SHIP, CLIMAT or others (not limited to Category 1), will be converted to their BUFR/CREX equivalents according to the latest WMO templates instantly, and will become available along with the old forms. In your NWS, you will then be able to feed processing systems with format they need.

Technical details

Compatible solution

Supported formats

TDCF formats:

  • BUFR (Binary Universal Form for the Representation of Meteorological Data)
  • CREX (Character Form for the Representation and Exchange of Data)
  • XML (Extensible Markup Language)*

TDCF schemes:

  • EUMETNET AMDAR
  • TM 307073
  • TM 307080
  • TM 307086
  • TM 307090
  • TM 307091
  • TM 308009
  • TM 308013
  • TM 309050
  • TM 309051
  • TM 309052
  • TM 309053
  • TM 309054

TAC code forms:

  • AMDAR
  • SYNOP
  • SYNOP MOBIL
  • SHIP
  • TEMP
  • TEMP SHIP
  • TEMP MOBIL
  • CLIMAT
  • CLIMAT TEMP
  • BUOY
  • PILOT

 

Compliance

  • WMO Manual No. 306 (Manual on Codes)
  • WMO ET-DRC TDCF Templates
  • WMO Manual No. 386 (Manual on GTS, incl. Attachment II)

Interface – integration

  • SOAP WS
  • C++ binding
  • Python binding
  • File I/O
  • Command line interface (CLI)


Extensibility

  • Fast and easy expansibility.
  • Based on the XML functionality.
  • The tool features a XSLT-Processor and the conversion programs are actually XSL transformations.
  • Supports new formats just by adding new transformation rules.
  • We will be providing you with updates on experimental or pre-operational descriptors as they appear or are discussed by the IPET-DRC.

Operational support and guarantee

  • Operational support of the BUFR migration tool is planned for minimum 10 years. We will keep supporting to date, this tool will be needed and used.
  • Providing updates to descriptors, standard template mappings as needed, as well as software update where necessary.
  • When there are other codes/templates which are in-use in your office, we will – as part of installation – create a template mapping for you.

Requirements

Operating systems:

  • GNU/Linux® (recommended: Red Hat Enterprise Linux 5 or CentOS 5)
  • Microsoft® Windows®
  • Solaris®

Capabilities

  • Each report of an input TAC bulletin becomes a data subset in the output TDCF bulletin and reversely each data subset of an input TDCF bulletin becomes a report in the output TAC bulletin.
  • It is possible to perform a bundle conversion of TAC TEMP, PILOT part A, B, C, D reports into a single TDCF bulletin.
  • Conversion of BUFR TEMP, PILOT bulletins to TAC produces several bulletins according to the present pressure levels observations.
  • Headings of output WMO bulletins are produced automatically based on input file WMO heading, data type and intrinsic data.
  • Automatic WMO format end to end conversion takes care of BUFR <—> TAC, BUFR <—> CREX, as well as CREX <—> TAC conversions. It is as simple as specifying the input data and the desired output format.
  • BUFR editions 3 and 4 are supported both on input and output; compressed data sections are supported only on input.
  • CREX edition 3 is supported both on input and output.

Try the software

You can evaluate software online after a free registration.

Do you use Moving Weather? Write us to get more information TDCF integration in Moving Weather.