cc: "'shipl@cma.gov.cn'" , Fred Branski , Steve Foreman , "'Arimatea_J.HQONE.WMOGVA@gateway.wmo.ch'" , Richter Bernd , Carolin Richter , Adrian Simmons , Phil Jones , Fuchs Tobias , Howard Diamond , "Rock, Diana" , "hkaranja@meteo.go.ke" , kanemba , Walter Smith , LI Xiang , "'Little, Chris'" , "'bez@mecom.ru'" , "'jacques.anquetil@meteo.fr'" , "'tsunoda@met.kishou.go.jp'" date: Tue, 18 Aug 2009 09:22:18 -0400 from: Richard Thigpen subject: Re: GTS Problems [SEC=UNCLASSIFIED] to: Kelvin Wong Dear All, As of this morning's 12Z flight it appears that we have solved the Dar problem. For some reason they were no longer inserting NNNN between sections and we coached them into inserting it. So thanks to all for assisting. It took us a bit over 6 weeks to solve this so I am hopeful that you will come up with a better problem resolution procedure. Cheers Dick Kelvin Wong wrote: > Dear all, > > I am back at work today and have gone through all the discussions of the problems with TEMP reports at Dar Es Salaam. Thanks you to all who have contributed to resolving this problem thus far. I will contact individual centres with respect to the switching problems identified. Based on the latest available routeing catalogues I have found some inconsistencies in some individual centres. In this case it appears that the delivery problem is more likely to be at the source with only part A of the TEMP report USTN01 HDTA being delivered but not the other parts U(K,L,E)TN01 HDTA. > > Let me describe some procedures which should have taken place in normal operational problem: > > 1. Dar Es Salaam, Tanzania is in the zone of responsibility of RTH Nairobi and accordingly its RTH Focal Point would escalate this incident to its operational centre in the first place. Depending on the available NMC contacts at Dar Es Salaam with RTH Nairobi and the urgency of the matter a direct phone call appears to be appropriate after repeated attempts by other means of communication. > > 2. Any in-depth investigation to the source of the problem would still be handled by the responsible RTH Focal Point together with other RTH/MTN centres to which the responsible RTH is directly or indirectly connected to as the operational centres are normally data centres not manned by professional staff outside office hours. All chains of routeing tables should be examined by the RTH Focal Points. In this case it would be: > > Offenbach > Toulouse > Exeter > Washington, Melbourne > Tokyo, etc. or > Offenbach > Prague, etc. > Cairo > Moscow, etc. > > 3. Switching problems do happen but single points of failure are normally limited as each RTH are most likely connected to more than one RTH on the MTN. RTH Focal Points should ensure that essential data products are routed to all RTH centres to which they are directly connected. > > 4. Due to the difference in time zones and spoken languages in different regions the responsible RTH is always in the best position to resolve such problems. I agree with Peiliang that Focal Points for National Centres should also be made available at WMO web site. I believe most RTH/MTN centres would have some sort of informal list of contacts in their region for internal operational use. The matter now is to have a formal list of contacts nominated by the PRs and the continuous effort to update the list. > > Kind regards, > Kelvin > --------------------------------------------- > Kelvin Wong ( K.Wong@bom.gov.au ) > RTH Focal Point, WMC Melbourne > Bureau of Meteorology http://www.bom.gov.au > GPO Box 1289, Melbourne, Vic 3001, Australia > Phone: +61 3 96694227 Fax: +61 3 96694128 > --------------------------------------------- > > > -----Original Message----- > From: shipl@cma.gov.cn [mailto:shipl@cma.gov.cn] > Sent: Thursday, 13 August 2009 16:25 > To: Fred Branski; Richard Thigpen; Steve Foreman; LI Xiang; Jose Arimatea de Sousa Brito > Cc: Richter Bernd; Kelvin Wong; Carolin Richter; Adrian Simmons; Phil Jones; Fuchs Tobias; Howard Diamond; Rock, Diana; hkaranja@meteo.go.ke; kanemba; Walter Smith > Subject: Re: AW: GTS Problems > > Dear Fred and colleagues, > > I analysed the story and consulted with my colleague Ms. Li Xiang, who is our RTH focal point. We believe that there is something missing from our problem escalating process, and we need to fix it. > > Currently, the general data flow is > - A NMC collects data from its domestic observation stations, and distributes the data to the associated RTH. > - The RTH distributes the data which is collected from its domestic observation stations and the NMCs located in its responsibility zone to other RTHs and NMCs. > > When a problem occurs, it is suggested to report the problem to all the links (the NMC, concened RTH and MTN centers) of the distribution at the same time. So that, the concerned centers can make check at their own side, and share the progress and results. In this way, even if we have the contact point for one center on leave, it is still possible to get response from its upstream or downstream links(or centers). This would be helpful to get to the bottom of the problem and have it fixed in time. > > And, at WMO web site (http://www.wmo.int/pages/prog/www/ois/RTHFocalPts/RTHzones_en.html), we now already have the contacts for RTHs. To get in touch with the concerned NMC during a problem investigation, it is also suggested to publish the contact points for NMCs on this page. So we need to re-habilitate the RTH focal-points, as we agreed at the recent MG meeting. And it seems a list of GTS operation focal-points at NC level would be necessary. If we could get feedback directly from Dar es Salaam, the whole story would be different. > > Regards, > > Peiliang > > ----- Original Message ----- > From: "Fred Branski" > To: "Richard Thigpen" ; "Steve Foreman" ; "Peiliang Shi" > Cc: "Richter Bernd" ; "Kelvin Wong" ; "Carolin Richter" ; "Adrian Simmons" ; "Phil Jones" ; "Fuchs Tobias" ; "Howard Diamond" ; "Rock, Diana" ; ; "kanemba" ; "Walter Smith" > Sent: Tuesday, August 11, 2009 10:36 PM > Subject: Re: AW: GTS Problems > > > Hi Peiliang, Steve, Dick and others, > > First, thanks Dick for another example of the issue of resolving > problems with exchange of information. I have some comments to add but > first, for ISS in particular this goes back to our discussions at the MG > meeting and the need for improved resolution processes. We need to > follow up on this more but for now there are some good examples here of > what may be part of the recurring issue. > > All RTHs have real time mechanisms for addressing operational problems. > The issue is how can we effectively interface these mechanisms with "all > appropriate" user identified problems. In particular, I'm talking about > the mechanisms that identify and track to resolution problems identified > in real time. I'm not fully familiar with all the details of this > particular problem occurrence, but it seems the problem may have been > reported through and "worked on" through "second level" problem > resolution activities instead of as a real time operational problem. I > believe one reason this occurs often because the impact of a problem is > not fully understood by the organization it was reported to. Also, it > may simply be an issue of the user who is requesting assistance not > fully understanding either how to engage our problem resolution > activities or not understanding how to characterize the impact so that > those working on the issue can apply an appropriate level of importance. > > I look forward to your views on this and further follow up but first > let's get this particular problem fixed. > > TIA! > > Cheers, Fred > ========================================================== > Fred Branski > WMO CBS President > U.S. National Weather Service > Office of the Chief Information Officer > Intl Data & Requirements Liaison > 1325 East-West Highway, Room 17456 > Silver Spring, MD 20910 U.S.A. > Phone: +1 301 713 3538 ext 121 Fax: +1 301 713 9450 > Cell: +1 240 355 4468 > ========================================================== > > > -----Original Message----- > From: Richard Thigpen [mailto:thigpenr@erols.com] > Sent: Tuesday, 11 August 2009 00:36 > To: Richter Bernd > Cc: Fred Branski; Kelvin Wong; Carolin Richter; Adrian Simmons; Phil Jones; Fuchs Tobias; Howard Diamond; Rock, Diana; hkaranja@meteo.go.ke; kanemba > Subject: Re: AW: GTS Problems > > > Thanks Bernd, > So this points to Nairobi or to Dar es Salaam. Lets see what Henry in > Nairobi finds. Todays report by e mail shows all four parts. And yes > Bernd they did not report for two days. Perhaps they do not work on > week ends? > Hope you're having a good holiday. > Dick > > > -----Original Message----- > From: Richter Bernd [mailto:Bernd.Richter@dwd.de] > Sent: Tuesday, 11 August 2009 00:23 > To: Richard Thigpen; Fred Branski > Cc: Kelvin Wong; Carolin Richter; Adrian Simmons; Phil Jones; Fuchs Tobias; Howard Diamond; Bernd.Richter@dwd.de; Rock, Diana; hkaranja@meteo.go.ke > Subject: AW: GTS Problems > > > Dear Dick, > > I just got the situation checked: > > USTN01 HTDA is received in EDZW more on a now-and-then-basis: > We received it today 12Z, the previous was received 07 Aug 12Z. > > There are however no TTBB-Part received ! > > Our routeing lists switches this bulletin, among others, to EGRR > > So from our point of view this seems at least not to be a problem of > data losses at EDZW - if a switching problem ever ... > > Hope this helps, Bernd. > > > -----Ursprüngliche Nachricht----- > Von: Richard Thigpen [mailto:thigpenr@erols.com] Gesendet: Montag, > 10. August 2009 15:29 > An: Fred Branski > Cc: Kelvin Wong; Carolin Richter; Adrian Simmons; Phil Jones; Fuchs > Tobias; Howard Diamond; Bernd.Richter@dwd.de; Rock, Diana; > hkaranja@meteo.go.ke > Betreff: GTS Problems > > Hello Fred, > I trust all is well with you. And enjoying the weather in Washington! > Remember we asked for your assistance in your CBS role in resolving > GTS problems? I just wanted to share one with you an example of why > we need a systematic approach. > > Dar es Salaam was down for a long time until we sent them technical > assistance to repair the equipment and a supply of radiosondes. For > the past month however, we cannot receive their reports over GTS. > Kelvin Wong got us started tracking the problem but Kelvin is now on > vacation. He helped us confirmed it was getting to Nairobi. By the > way I am receiving e mail copies of the report that confirm they are > actually operating. > > Then we found and corrected a switching problem at Offenbach but now > Bernd is on holiday. Then we found and corrected a switching problem > at UKMO but now Diana is on holiday. We are only receiving the first > part (the TTAA) of the report. At this point I cannot get any > response from any of the respective RTH as to who is getting what. I > do not believe that UKMO is receiving all parts, I cannot be sure > what Offenbach is receiving and I do not know what Nairobi is > receiving or sending. We have been trying to resolve this for over a > month. At $120 per sounding! > > Kelvin especially cooperative and helpful as are all the others when > we can get responses from them. But the fact remains that after a > frustrating month we are still not getting the data. > > So this is why we asked for CBS assistance. > Thanks for your cooperation and assistance. > Dick > >