News
Resources
Support
Related Links
  MADIS LDAD Filter

How to filter stations out of the CSV files

Several of the mesonets available from MADIS are national in scope, and contain thousands of stations. To cut down on the data volume, you can filter out the stations that are outside your geographic area of interest, by specifying the corners of a latitude/longitude rectangle. Only stations inside the rectangle will be sent into LDAD. If the mesonet is MesoWest (also called UTMESNET), there's an additional option to include only those subproviders that are specified by the user. The filtering scheme will also work with any mesonet obtained from a source other than MADIS, as long as the mesonet meets this requirement: the first column of the data files must contain the same site ID's as found in the first column of the station table. Note that some of the mesonets from MADIS do not meet this criteria, and cannot be filtered with this scheme, but these are all regional and shouldn't need filtering. Here is the list of which mesonets can and cannot be filtered:

MesonetCan be FilteredHas Header
   1-min ASOSYesNo
   AFAYesNo
   AIRNowYesNo
   AKDOTNoNo
   AK-MesoYesNo
   APGNoNo
   APRSWXNETYesNo
   ARLFRDYesNo
   AWSYesNo
   AWXYesNo
   CA-HydroYesNo
   CAICYesNo
   CODOTYesNo
   CO_E-470YesNo
   DCNetYesYes
   DDMETYesNo
   DEOSYesNo
   FLDOTYesNo
   FL-MesoYesNo
   GADOTYesNo
   GLDNWSYesNo
   GLOBEYesNo
   GoMOOSYesNo
   GPSMETYesYes
   HADSYesNo
   IADOTNoYes
   IEMYesNo
   INDOTYesNo
   INTERNETYesNo
   ITDYesNo
   KSDOTYesYes
   KYTC-RWISYesNo
   LCRAYesNo
   LSU-JSUNoYes
   MAPYesNo
   MDDOTNoNo
   MEDOTNoNo
   MesoWestYesNo
   MISCYesNo
   MNDOTNoNo
   MOComAgNetYesNo
   MQT-MesoYesNo
   NC-ECONetYesNo
   NDDOTNoNo
   NEDORYesYes
   NERRSYesNo
   NHDOTYesNo
   NJWxNetYesNo
   NonFedAWOSYesNo
   NOS-NWLONYesNo
   NOS-PORTSYesNo
   NWS-COOPYesNo
   OHDOTYesNo
   OK-MesoNoYes
   PADEPYesNo
   RAWSYesNo
   RDMTRYesNo
   SFWMDYesNo
   UDFCDYesYes
   UrbaNetYesNo
   USouthALYesNo
   VADOTYesNo
   VTDOTYesNo
   WIDOTYesNo
   WT-MesoYesNo
   WxFlowYesNo
   WXforYouYesNo
   WYDOTNoNo

The way this works is that you choose which mesonets you want to filter, create the list of stations to be included by running a script, then process incoming data files with a second script that will filter the unwanted stations and send files containing only the included stations into LDAD.

Here are the details:

* The MADIS team would like to thank Clark Safford of FFC for contributing the original software on which this filtering scheme is based. (Please contact us, however, and not Clark, with any questions.)


Last updated 16 March 2017