Update #4: SNPP/JPSS1/GCOM is now available at CBU for Push and Pull subscribers. PDA and NDE at the OPS site is down for the Oracle database upgrade with an expected Return To Service time of 2100Z – 2200Z.

Update #3: Reminder – The COOP activity will begin tomorrow May 24, 2022 @ 1500Z and follow the estimated schedule below.

PDA staff will be able to help with CBU connectivity issues from 0930 EDT / 1330 UTC until 1800 EDT / 2200 UTC. Send email to PDA_DHS@noaa.gov and you will be contacted with Google Meet details or a phone number.  

Update #2: High Level Estimated COOP / NSOF Outage timeline.

Tuesday

5/24/2022 @ 1500Z – Begin COOP failover activity to CBU backup site.

                                    SNPP/JPSS1/GCOM data will become available at CBU for Push and Pull subscribers.

                                    GOES-16/17 data will be turned off.

5/24/2022 @ 1600Z – The PDA OPS environment will begin shutdown for Oracle database upgrade.

                                    PDA subscribers and providers will no longer be able to login to PDA OPS.

5/24/2022 @ 2000Z to ~2200Z – The PDA OPS environment will be up and operational. A notification will be sent when database maintenance is complete.

                                    PDA consumers and providers will be able to login to PDA OPS.

                                    Backlogged Legacy products created during the outage will become available.

 GOES-16/17 data will be turned on.  

Wednesday

5/25/2022 @ 1500Z – Begin COOP failback activity to the PDA OPS site.

5/25/2022 @ 1630Z -  All product generation at CBU should be completed.             

Update #1: In order to ensure a safe restart of GOES after such significant maintenance, we need to stop dataflow to PDA during the PDA OPS 4-6 hour maintenance on May 24. That data will not be recovered after the outage, but GOES data will start again as soon as the database comes back online.

Topic: Planned PDA Partial Failover to CBU and Full Outage of PDA OPS at NSOF 

Date/Time Issued:  May 24, 2022 1635Z

Product(s) or Data Impacted:  All products distributed from PDA. Some products will be available at CBU and some will be unavailable for part of the maintenance window.  

Date/Time of the Initial Impact: May 24, 2022 1400Z

Date/Time of Expected End:  May 25, 2022 1800Z

Length of the Outage: Failover to CBU ~24 hours; Full outage at NSOF OPS ~4-6 hours

Details/Specifics of Change:  

On May 24, a significant database maintenance activity will take place during business hours that will require a period of about 4-6 hours of complete OPS PDA downtime.  Delivery of GOES-16/17, Metop, POES and other data provided from the OPS PDA will cease distribution during that affected time.  When the OPS PDA system is returned to service, it is anticipated that users will receive products that were ingested/created during the outage time. However, there will be a significant backlog on PDA, and it may take several hours for the OPS PDA at NSOF to receive and distribute all the products produced during the outage period in both ESPC and NCCF. After the OPS PDA outage period, latency will be impacted as the backlog of products are processed and distributed. Some products from externals may be lost, as well as some products NESDIS creates that are missing needed input data. 

In order to minimize the outage impact, NESDIS plans to do this maintenance in conjunction with planned Continuity of Operations (COOP) activities. There will be a partial activation of the Consolidated Backup (CBU) site at Fairmont, WV. Only data and products from S-NPP, NOAA-20 and GCOM-W1 will be available.  This failover is expected to begin with the first orbit received after 1400Z on May 24 (the day of the outage in OPS) and end with the next orbit received after 1400Z on May 25 (the day after the PDA OPS outage). The exact times of the failover will be announced closer to the event.   It is noted that while in this data failover configuration, S-NPP, NOAA-20 and GCOM-W1 data and products will ONLY be at CBU for that period of about 24 hours and then will be returned to the nominal PDA OPS system at NSOF.

At this time we expect little to no duplicate data will be distributed between PDA OPS and CBU; therefore, pull users will be safe to pull data from both sites starting at the time of partial failover and until about 1.5 hours after failback end time while the final NDE products are created at CBU.

PDA staff will be available to help troubleshoot users’ issues with failover from 0930 EDT / 1330 UTC until 1800 EDT / 2200 UTC on Tue May 24 and from 1000 EDT / 1400 UTC until 1400 EDT / 1800 UTC on Wed May 25 for failback. Email PDA_DHS@noaa.gov and someone will provide the Google Meet details or phone number.  

A declaration of Critical Weather (CWD) before or during this event will not postpone or change the date of the COOP. The COOP date is firm and JPSS data will only be available at CBU on those dates. If the database upgrade is not ready to implement on those dates, the PDA OPS database maintenance activity will be postponed to a later date. In this scenario the PDA OPS environment will not be shutdown, allowing non-JPSS data to be processed during the COOP.

Contact Information for Further  Information: ESPC Operations at ESPCOperations@noaa.gov and 301-817-3880.

Web Site(s) for applicable information: N/A

This message was sent by ESPC.Notification@noaa.gov. You have been sent this and other notifications because you have opted in to receive it. If for any reason, you wish to unsubscribe, please contact ESPC Help Desk at ESPCOperations@noaa.gov or (301) 817-3880. Please note: it may take up to two business days to process your unsubscribe request.