MADIS RT RUN-TIME STATUS for Tue Mar 28 16:32:01 GMT 2023 on vm-bldr-madis-process1a
INFO RT VM STATUS for Node vm-bldr-madis-process1a Time Tue Mar 28 16:32:01 GMT 2023
PASS METAR processing tbl file: 128320 Mar 21 21:03 madis/proc/decode/data/fxa/LDAD/data/NonFedAWOSStation.tbl
PASS Station update flag file: 0 May 3 2019 madis/proc/decode/awips/fxa/GO-UPDATE-STATIONS
PASS SNOTEL files Last updates: hourly<120min 102 daily<1800min 102 monthly<45000min 52
PASS Metar station table file: 1121683 May 3 2019 madis/proc/sfc/iqc1/fslparms/lsfc.tbl
PASS Accept/reject|autorej files Cnts: AO: 3 IQC: 3 NON-SFC: 20
PASS Persistent processes Cnts: pqact (1+): 1 ldmd (1+): 23 DECODE (5): 5
PASS Dcmetr files CNT-LDAD.raw: 806
PASS RSAS values range check Total chk: 317840 Outside of range (<100): 0
PASS Aircraft data Check: REC-CNT>300 #TAMDAR 20230328_1500:425 20230328_1400:593 #PUBLIC-DP 20230328_1500:1802 20230328_1400:1887 #PUBLIC-DP-profile 20230328_1500:37 20230328_1400:44
PASS Load resource m15 < 9 cores 98% (goal > 60%) -- % of the time M15 < 9 Samples: 991 #10=+ 0 #9 12 #8 120 #7 494
PASS Open file resource check: open file < 5000 Max Files: 2833 #<1000: 342 #<10000 991
PASS Process count check: processes < 1000 Max Processes: 594
PASS AO Throughput 0 -- check: #"conflicts" <20 delayed %Delayed: 0% #Sample: 198 #Delayed: 0
INFO AO runtime avg sfcanl 14 seconds 989/66
INFO AO runtime avg sfcncdf 1 seconds 66/51
INFO AO runtime avg sfcver 78 seconds 4731/60
INFO AO runtime avg sfcqcmsg 21 seconds 1445/66
INFO AO runtime avg sfcqcout meso 33 seconds 6149/183
INFO AO runtime avg sfcqcout_late meso 83 seconds 20487/245
INFO AO runtime avg FSL_qcout5m_ 44 seconds 8925/199
INFO AO runtime avg FSL_analrun15m_ 113 seconds 7487/66
PASS IQC Throughput 0 -- check: #"conflicts" <30 delayed %Delayed: 0% #Sample: 995 #Delayed: 5 Competed timely: 926
INFO IQC runtime avg qcstg1_2 5 seconds 88076/17427
INFO IQC runtime avg qcstg1_2 meso 20 seconds 39552/1975
INFO IQC runtime avg qcstg1_2_late meso 21 seconds 28915/1366
INFO IQC runtime avg sfcing 30 seconds 29645/986
INFO IQC runtime avg sfchqc 63 seconds 62256/985
PASS QC applied max value Max applied check should be more than 107, 123 is good: 123, // temperatureQCA(99999)
PASS RT Throughput -238828 secs -- check: Avg time < 180 sec. #Hrs: 16 Max secs: 2009995
PASS '0' length data Station table check: cnt '0' length files #: 0
PASS '0' length master Station table check: cnt '0' length files #: 0
PASS netcdf-4.1 lib NetCDF lib link not in routerStoreNetCDF
WARN Shefdecoder issues. Missing stations in either hydro and mesonet: check 20230328_1500: #CPYRMA: 0 #NJWXNET: 636
PASS QC MAPS SEA-LEVEL PRESSURE (MB) 24.97 % OBS FAILED CHECK ( 8432 OF 33774) AVGTHR 1.7946 RATIO 0.100E+06 AVGEPS 0.2546E-01 FRMSE 42.084 TRMSE 99999.
PASS QC DEW POINT DEPRESSION (K) 6.68 % OBS FAILED CHECK ( 4344 OF 65017) AVGTHR 5.3066 RATIO 0.380 AVGEPS 0.1634E-01 FRMSE 14.719 TRMSE 2.0173
PASS QC POTENTIAL TEMPERATURE (K) 7.17 % OBS FAILED CHECK ( 5028 OF 70094) AVGTHR 3.6462 RATIO 0.100E+06 AVGEPS 0.2614E-01 FRMSE 1090.2 TRMSE 99999.
PASS QC DEW POINT TEMPERATURE (K) 5.85 % OBS FAILED CHECK ( 3806 OF 65030) AVGTHR 3.9614 RATIO 0.356 AVGEPS 0.1727E-01 FRMSE 18.017 TRMSE 1.4110
PASS QC U COMPONENT OF SFC WIND (M/S) 1.83 % OBS FAILED CHECK ( 1185 OF 64787) AVGTHR 4.4176 RATIO 0.321 AVGEPS 0.1422E-01 FRMSE 8.2667 TRMSE 1.4163
PASS QC V COMPONENT OF SFC WIND (M/S) 2.69 % OBS FAILED CHECK ( 1740 OF 64787) AVGTHR 4.5458 RATIO 0.333 AVGEPS 0.1468E-01 FRMSE 7.9521 TRMSE 1.5151
PASS QC 3H ALTIMETER CHANGE (MB) 1.32 % OBS FAILED CHECK ( 398 OF 30130) AVGTHR 1.2112 RATIO 0.236 AVGEPS 0.1630E-01 FRMSE 51.382 TRMSE 0.28527
PASS QC NWS SEA-LEVEL PRESSURE (MB) 6.99 % OBS FAILED CHECK ( 52 OF 744) AVGTHR 1.5939 RATIO 0.410 AVGEPS 0.6660E-01 FRMSE 4.8672 TRMSE 0.65318
PASS QC ALTIMETER (MB) 23.83 % OBS FAILED CHECK ( 8049 OF 33774) AVGTHR 1.9345 RATIO 0.100E+06 AVGEPS 0.2594E-01 FRMSE 43.081 TRMSE 99999.
PASS MXST_P limit 270479 MXST_P: 280000
PASS AO Possible Seg fault 0 #logs checked: 2267
PASS IQC Possible Seg fault 0 #logs checked: 4628
PASS NONSFC Possible Seg fault 0 #logs checked: 809
PASS MADIS DB Connection ok madis/proc/storage/log/MadisDbUpload/MadisDataUpload_20230328_398496.log
INFO MADIS DB Connection cnts Info:491 [2023-Mar-28 16:31:11.536962] - LastUploadStats:mode:upload_sfc,pid:32528,aborted:0,fileCount:1,wholeFileErrors:0,lines:42031,inserts:30859,updates:11171,duplicates:0,errors:4,stationsAdded:0,stationsUpdated:2,publishLines:0,publishErrors:0,notes:./202303281630_receipt.sfcd2.txt,run ms:61807
PASS AFWS files current # in last hour: 20
PASS madis.noaa.gov.stations current # updated in last day: 12
PASS madis.noaa.gov.trans current # updated in last day: 24
WARN MSAS ANAL PRODUCT -- SFCNCDF # Today Day: 23087
madis/proc/sfc/ao/output/sfcncdf.out.230870200_4:
madis/proc/sfc/ao/output/sfcncdf.out.230870300_4:
madis/proc/sfc/ao/output/sfcncdf.out.230870500_4:
madis/proc/sfc/ao/output/sfcncdf.out.230870900_1:
madis/proc/sfc/ao/output/sfcncdf.out.230871100_1:
madis/proc/sfc/ao/output/sfcncdf.out.230871200_4:
madis/proc/sfc/ao/output/sfcncdf.out.230871300_2:
madis/proc/sfc/ao/output/sfcncdf.out.230871300_4:
madis/proc/sfc/ao/output/sfcncdf.out.230871400_4:
WARN MSAS ANAL PRODUCT -- SFCNCDF # Prior Day: 23086
madis/proc/sfc/ao/output/sfcncdf.out.230851600_4:
madis/proc/sfc/ao/output/sfcncdf.out.230851700_4:
madis/proc/sfc/ao/output/sfcncdf.out.230851800_4:
madis/proc/sfc/ao/output/sfcncdf.out.230851900_4:
madis/proc/sfc/ao/output/sfcncdf.out.230852000_4:
madis/proc/sfc/ao/output/sfcncdf.out.230852100_4:
madis/proc/sfc/ao/output/sfcncdf.out.230852300_4:
madis/proc/sfc/ao/output/sfcncdf.out.230860000_2:
madis/proc/sfc/ao/output/sfcncdf.out.230860000_3:
madis/proc/sfc/ao/output/sfcncdf.out.230861200_4:
madis/proc/sfc/ao/output/sfcncdf.out.230861300_2:
madis/proc/sfc/ao/output/sfcncdf.out.230861400_4:
madis/proc/sfc/ao/output/sfcncdf.out.230861700_4:
madis/proc/sfc/ao/output/sfcncdf.out.230861800_4:
madis/proc/sfc/ao/output/sfcncdf.out.230861900_4:
madis/proc/sfc/ao/output/sfcncdf.out.230862000_4:
madis/proc/sfc/ao/output/sfcncdf.out.230862100_4:
madis/proc/sfc/ao/output/sfcncdf.out.230862200_2:
madis/proc/sfc/ao/output/sfcncdf.out.230862200_4:
madis/proc/sfc/ao/output/sfcncdf.out.230862300_4:
madis/proc/sfc/ao/output/sfcncdf.out.230870200_4:
madis/proc/sfc/ao/output/sfcncdf.out.230870300_4:
madis/proc/sfc/ao/output/sfcncdf.out.230870500_4:
madis/proc/sfc/ao/output/sfcncdf.out.230870900_1:
madis/proc/sfc/ao/output/sfcncdf.out.230871100_1:
madis/proc/sfc/ao/output/sfcncdf.out.230871200_4:
madis/proc/sfc/ao/output/sfcncdf.out.230871300_2:
madis/proc/sfc/ao/output/sfcncdf.out.230871300_4:
madis/proc/sfc/ao/output/sfcncdf.out.230871400_4:
WARN SFCANL MATRIX FAILURE # Today Day: 23087
madis/proc/sfc/ao/output/sfcanl.out.230870900_1:
madis/proc/sfc/ao/output/sfcanl.out.230871100_1:
madis/proc/sfc/ao/output/sfcanl.out.230871300_4:
WARN SFCANL MATRIX FAILURE # Prior Day: 23086
madis/proc/sfc/ao/output/sfcanl.out.230860000_2:
madis/proc/sfc/ao/output/sfcanl.out.230860000_3:
madis/proc/sfc/ao/output/sfcanl.out.230862200_2:
COMPARE Mesonet data providers cnt 48 for 20230328_1400:
COMPARE DECODE STN TBLS 149 Newest: 214481 Mar 28 16:30 LDAD/data/RAWSStation.txt Oldest: 8848 Sep 12 2019 LDAD/data/AWS-GANetStation.txt
COMPARE PRE-DECODE STN TBLS 149 Newest: 214481 Mar 28 16:30 LDAD/data/master/RAWSStation.txt Oldest: 1568 Sep 12 2019 LDAD/data/master/AWS-KSNetStation.txt
COMPARE pqsend entries 6 (Only on primary, 1 for each feedtype/distr)
COMPARE Surface QC text products 7320 hour (15): Newest: Mar 28 16:27 madis/proc/sfc/ao/qc122/qcsumry16.dat Oldest:
INFO No NETWORK Stats available for (2308714):
COMPARE Rough compare sizes for same hour on known good system. The ext fields are #recNum and file-size. Note: Max #recNum has to be < MXST_P 280000
AO-20230328_14 sfc/ao/qcms_data/buoy/20230328_1400 935 1208404
AO-20230328_14 sfc/ao/qcms_data/crn/20230328_1400 246 436072
AO-20230328_14 sfc/ao/qcms_data/hcn/20230328_1400 102 199844
AO-20230328_14 sfc/ao/qcms_data/hfmetar/20230328_1400 11989 18868156
AO-20230328_14 sfc/ao/qcms_data/hydro/20230328_1400 10496 13095180
AO-20230328_14 sfc/ao/qcms_data/mesonet/20230328_1400 248923 548158188
AO-20230328_14 sfc/ao/qcms_data/metar/20230328_1400 8254 12279308
AO-20230328_14 sfc/ao/qcms_data/nepp/20230328_1400 72 177472
AO-20230328_14 sfc/ao/qcms_data/rwis/20230328_1400 5989 21372188
AO-20230328_14 sfc/ao/qcms_data/urbanet/20230328_1400 5935 16144692
NS-20230328_14 nonsfc/madis_data/LDAD/profiler/netCDF/20230328_1400 47 2407952
NS-20230328_14 nonsfc/madis_data/LDAD/profiler/netCDF/20230328_1430 37 1904512
NS-20230328_14 nonsfc/madis_data/LDAD/snow/netCDF/20230328_1400 2927 3219064
NS-20230328_14 nonsfc/madis_data/point/acars/netcdf/20230328_1400 38370 44154296
NS-20230328_14 nonsfc/madis_data/point/acarsProfiles/netcdf/20230328_1400 360 52436276
NS-20230328_14 nonsfc/madis_data/point/raob/netcdf/20230328_1400 1 180508
AO-20230328_13 sfc/ao/qcms_data/buoy/20230328_1300 918 1189092
AO-20230328_13 sfc/ao/qcms_data/crn/20230328_1300 246 436072
AO-20230328_13 sfc/ao/qcms_data/hcn/20230328_1300 102 199844
AO-20230328_13 sfc/ao/qcms_data/hfmetar/20230328_1300 12009 18899276
AO-20230328_13 sfc/ao/qcms_data/hydro/20230328_1300 11118 13811724
AO-20230328_13 sfc/ao/qcms_data/mesonet/20230328_1300 248710 547690440
AO-20230328_13 sfc/ao/qcms_data/metar/20230328_1300 8359 12433028
AO-20230328_13 sfc/ao/qcms_data/nepp/20230328_1300 83 193444
AO-20230328_13 sfc/ao/qcms_data/rwis/20230328_1300 5974 21321488
AO-20230328_13 sfc/ao/qcms_data/urbanet/20230328_1300 5926 16120752
NS-20230328_13 nonsfc/madis_data/LDAD/profiler/netCDF/20230328_1300 48 2458296
NS-20230328_13 nonsfc/madis_data/LDAD/profiler/netCDF/20230328_1330 38 1954856
NS-20230328_13 nonsfc/madis_data/LDAD/snow/netCDF/20230328_1300 3582 3876684
NS-20230328_13 nonsfc/madis_data/point/acars/netcdf/20230328_1300 39351 45276560
NS-20230328_13 nonsfc/madis_data/point/acarsProfiles/netcdf/20230328_1300 417 60732056
DECODE decode/data/fxa/LDAD/crn/netCDF/20230328_1400 246 316448
DECODE decode/data/fxa/LDAD/hcn/netCDF/20230328_1400 102 139672
DECODE decode/data/fxa/LDAD/hfmetar/netCDF/20230328_1400 11989 13283188
DECODE decode/data/fxa/LDAD/hydro/netCDF/20230328_1400 10698 8817912
DECODE decode/data/fxa/LDAD/mesonet/netCDF/20230328_1400 250828 310489536
DECODE decode/data/fxa/LDAD/nepp/netCDF/20230328_1400 72 111828
DECODE decode/data/fxa/LDAD/rwis/netCDF/20230328_1400 6020 15985724
DECODE decode/data/fxa/LDAD/snow/netCDF/20230328_1400 2955 2338584
DECODE decode/data/fxa/LDAD/urbanet/netCDF/20230328_1400 5935 5342532
IQC sfc/iqc1/qcms_data/buoy/20230328_1400 650 884644
IQC sfc/iqc1/qcms_data/crn/20230328_1400 153 301036
IQC sfc/iqc1/qcms_data/hcn/20230328_1400 0 69692
IQC sfc/iqc1/qcms_data/hfmetar/20230328_1400 3536 5715288
IQC sfc/iqc1/qcms_data/hydro/20230328_1400 4265 5917068
IQC sfc/iqc1/qcms_data/mesonet/20230328_1400 87637 193974132
IQC sfc/iqc1/qcms_data/metar/20230328_1400 3826 5796716
IQC sfc/iqc1/qcms_data/nepp/20230328_1400 0 72928
IQC sfc/iqc1/qcms_data/rwis/20230328_1400 1187 5141428
IQC sfc/iqc1/qcms_data/urbanet/20230328_1400 1222 3608112
IQC sfc/iqc2/qcms_data/buoy/20230328_1400 11 158740
IQC sfc/iqc2/qcms_data/crn/20230328_1400 40 136960
IQC sfc/iqc2/qcms_data/hcn/20230328_1400 0 69692
IQC sfc/iqc2/qcms_data/hfmetar/20230328_1400 2830 4616752
IQC sfc/iqc2/qcms_data/hydro/20230328_1400 2153 3484044
IQC sfc/iqc2/qcms_data/mesonet/20230328_1400 61894 137442504
IQC sfc/iqc2/qcms_data/metar/20230328_1400 1572 2496860
IQC sfc/iqc2/qcms_data/nepp/20230328_1400 0 72928
IQC sfc/iqc2/qcms_data/rwis/20230328_1400 2807 10617028
IQC sfc/iqc2/qcms_data/urbanet/20230328_1400 499 1684932
IQC sfc/iqc3/qcms_data/buoy/20230328_1400 64 218948
IQC sfc/iqc3/qcms_data/crn/20230328_1400 19 106468
IQC sfc/iqc3/qcms_data/hcn/20230328_1400 78 169220
IQC sfc/iqc3/qcms_data/hfmetar/20230328_1400 2557 4191964
IQC sfc/iqc3/qcms_data/hydro/20230328_1400 1439 2661516
IQC sfc/iqc3/qcms_data/mesonet/20230328_1400 35867 80287212
IQC sfc/iqc3/qcms_data/metar/20230328_1400 1373 2205524
IQC sfc/iqc3/qcms_data/nepp/20230328_1400 0 72928
IQC sfc/iqc3/qcms_data/rwis/20230328_1400 1100 4847368
IQC sfc/iqc3/qcms_data/urbanet/20230328_1400 2206 6225552
IQC sfc/iqc4/qcms_data/buoy/20230328_1400 150 316644
IQC sfc/iqc4/qcms_data/crn/20230328_1400 33 126796
IQC sfc/iqc4/qcms_data/hcn/20230328_1400 90 184532
IQC sfc/iqc4/qcms_data/hfmetar/20230328_1400 2521 4135948
IQC sfc/iqc4/qcms_data/hydro/20230328_1400 2547 3937932
IQC sfc/iqc4/qcms_data/mesonet/20230328_1400 46184 102943344
IQC sfc/iqc4/qcms_data/metar/20230328_1400 1263 2044484
IQC sfc/iqc4/qcms_data/nepp/20230328_1400 0 72928
IQC sfc/iqc4/qcms_data/rwis/20230328_1400 239 1937188
IQC sfc/iqc4/qcms_data/urbanet/20230328_1400 2615 7313492
IQC sfc/iqc5/qcms_data/buoy/20230328_1400 154 321188
IQC sfc/iqc5/qcms_data/crn/20230328_1400 27 118084
IQC sfc/iqc5/qcms_data/hcn/20230328_1400 0 69692
IQC sfc/iqc5/qcms_data/hfmetar/20230328_1400 2328 3835640
IQC sfc/iqc5/qcms_data/hydro/20230328_1400 2175 3509388
IQC sfc/iqc5/qcms_data/mesonet/20230328_1400 47848 106597488
IQC sfc/iqc5/qcms_data/metar/20230328_1400 1121 1836596
IQC sfc/iqc5/qcms_data/nepp/20230328_1400 72 177472
IQC sfc/iqc5/qcms_data/rwis/20230328_1400 1372 5766728
IQC sfc/iqc5/qcms_data/urbanet/20230328_1400 1214 3586832