Case #:PM_MYD_20229_NRT  Opening date: 08/17/2020  Last update: 09/15/2020 |
On August 16th, 2020 (2020.229) at 09:26:40 Aqua received a yellow CDH alarm that was associated with a data value mismatch error and caused by data of incorrect length. MOT was unable to recover any data from the partitions 6,16, and 9 on the SSR. Partition 6 contains housekeeping and GBAD ephemeris and attitude data. Hence, this will affect the processing of the Aqua data on both NRT and OPS streams, starting 2020.229 09:26:35, pending the resolution of this issue. This issue is currently under investigation and more information will be posted below to this note when available. The image below shows MODIS Aqua NRT for DOY 2020.229 where the data loss has impacted the global browse image for the majority of the day.
Aqua Top-of-the-atmosphere Visible Radiance Bands (MYD02CRS) NRT
Day 2020.229 (08/16/2020)
*** UPDATE: 09.15.20. This issue has been closed, please refer to the latest post on the data recovery details
*** UPDATE: 09.02.20. As of September 2, 21:00 the Aqua FOT was able to recover the FMU/SSR as of 246/20:12:00 UTC and Aqua has resumed nominal operations. Effort is in place to recover some of the lost data from 2020 (2020.229) at 09:26:40 till 246/21:00:00 UTC from Direct Broadcast (DB) communities. Once EDOS pieces together the L0 from the different DB stations and sends them to MODAPS, MODAPS will fill in this gap, with as much Aqua coverage as possible.
For now, Aqua forward production in C6 OPS and NRT will resume. Some of the combined products like BRDF/Albedo and MAIAC will remain on-hold till MODAPS have been able to backfill the gap between 2020.229 - 09:26:40 to 2020.246-21:00:00 UTC. We will keep you posted on the status of processing of Aqua and combined Terra+Aqua MODIS product for the anomaly period (8/16 - 9/2) using the data collected from DB stations.
*** UPDATE: 08.28.20 NASA Anomaly Report - Aqua Solid State Recorder Anomaly. Aqua Flight Operation Team (FOT) continues to develop procedures to recover from the Formatter Multiplex Unit (FMU) anomaly. Procedures to perform a software reset of the FMU have been developed and tested along with procedures to power cycle the FMU. Conducted independent peer review on recovery procedures on 8/28. Finalized Instrument configuration for the reset activities. Run-for-record simulation planned for 8/31, software reset on 9/2 and if necessary, power cycle on 9/3.
*** UPDATE 08.21.20: No Aqua MODIS science data has been generated since Aug 16 2020, 09:26 due to the Formatter Multiplexer Unit (FMU) anomaly. FOT is working with the IOT to implement an FMU reset in the coming week, with a soft reset of FMU tentatively planned on 8/26, and if needed a hard reset on 8/27. As a part of the recovery process, MODIS instrument may need to transition to a standby or safe mode (SV and nadir doors closed). This continues to be investigated and we will keep the science team posted as we get more information. At this time, the MODIS instrument is in nominal operations.
*** UPDATE 08.18.20: On day 229/09:26:40, Aqua experienced an anomaly with the Formatter-Multiplexer Unit (FMU). Aqua FOT is continuing to investigate this anomaly and an FMU reset activity is currently being planned. MCST are awaiting to hear back from FOT about the exact timing/details of this activity. As a result of this anomaly, there is an anticipated impact on the MODIS science data. The MODIS instrument continues to be in nominal operations and the health/telemetry is been monitored by the IOT.
Occurrence: All MODIS Aqua products for specified date and time periods.