Scribd Logo
Upload

Welcome to Scribd!

  • ODM Tools Python: Open Source Software For Managing Environmental Sensor Data

    Uploaded by

    Consortium of Universities for the Advancement of Hydrologic Science, Inc.
    738 views1 page
    Amber Spackman Jones1 Jeffery S. Horsburgh2 Stephanie Reeder1 Jacob Meline1 The use of in situ sensors is resulting in environmental data collected at high frequencies, for extended durations, and with spatial distributions that require infrastructure for data storage and management. In particular, data from environmental sensors often need quality control and post processing performed to remove anomalous values, account for sensor drift, and arrive at an ‘approved’ dataset. The Observations Data Model (ODM), which is part of the Consortium of Universities for the Advancement of Hydrologic Science, Inc. (CUAHSI) Hydrologic Information System (HIS), is a data model for organizing, storing, and describing point observations data. We have developed ODM Tools Python as an open source software application for managing sensor data stored in an ODM database. Previous versions of ODM Tools included functionality to export data series and associated metadata, plot and summarize a single data series, generate derivative data series, and edit data series using a set of simple tools. We have re-designed and deployed a new version of ODM Tools in Python that adds a modernized graphical user interface with dockable windows, multiple platform support (Windows, Linux, and Mac), multiple database support (Microsoft SQL Server, MySQL, PostgreSQL), enhanced plotting capability, and support for automated scripting of quality control edits. Automated scripting is accomplished through a Python script editor and console integrated with the ODM Tools Python interface. Scripting records the corrections and adjustments performed on a data series in the quality control process, ensuring that editing steps performed using the visual tools in the graphical user interface are traceable and reproducible. Furthermore, the improved ability of ODM Tools Python to plot multiple data series simultaneously with various plot types, facilitates comparisons between a data series that is being edited with time series for other variables at the same site (internal consistency) or for the same variable at other sites (external consistency). 1 Utah Water Research Laboratory, Utah State University, Logan, Utah, USA 2 Department of Civil and Environmental Engineering and Utah Water Research Laboratory, Utah State University, Logan, Utah, USA

    Original Title

    ODM Tools Python: Open Source Software for Managing Environmental Sensor Data

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Report this Document
    Amber Spackman Jones1 Jeffery S. Horsburgh2 Stephanie Reeder1 Jacob Meline1 The use of in situ sensors is resulting in environmental data collected at high frequencies, for extended durations, and with spatial distributions that require infrastructure for data storage and management. In particular, data from environmental sensors often need quality control and post processing performed to remove anomalous values, account for sensor drift, and arrive at an ‘approved’ dataset. The Observations Data Model (ODM), which is part of the Consortium of Universities for the Advancement of Hydrologic Science, Inc. (CUAHSI) Hydrologic Information System (HIS), is a data model for organizing, storing, and describing point observations data. We have developed ODM Tools Python as an open source software application for managing sensor data stored in an ODM database. Previous versions of ODM Tools included functionality to export data series and associated metadata, plot and summarize a single data series, generate derivative data series, and edit data series using a set of simple tools. We have re-designed and deployed a new version of ODM Tools in Python that adds a modernized graphical user interface with dockable windows, multiple platform support (Windows, Linux, and Mac), multiple database support (Microsoft SQL Server, MySQL, PostgreSQL), enhanced plotting capability, and support for automated scripting of quality control edits. Automated scripting is accomplished through a Python script editor and console integrated with the ODM Tools Python interface. Scripting records the corrections and adjustments performed on a data series in the quality control process, ensuring that editing steps performed using the visual tools in the graphical user interface are traceable and reproducible. Furthermore, the improved ability of ODM Tools Python to plot multiple data series simultaneously with various plot types, facilitates comparisons between a data series that is being edited with time series for other variables at the same site (internal consistency) or for the same variable at other sites (external consistency). 1 Utah Water Research Laboratory, Utah State University, Logan, Utah, USA 2 Department of Civil and Environmental Engineering and Utah Water Research Laboratory, Utah State University, Logan, Utah, USA

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    738 views1 page

    ODM Tools Python: Open Source Software For Managing Environmental Sensor Data

    Uploaded by

    Consortium of Universities for the Advancement of Hydrologic Science, Inc.
    Amber Spackman Jones1 Jeffery S. Horsburgh2 Stephanie Reeder1 Jacob Meline1 The use of in situ sensors is resulting in environmental data collected at high frequencies, for extended durations, and with spatial distributions that require infrastructure for data storage and management. In particular, data from environmental sensors often need quality control and post processing performed to remove anomalous values, account for sensor drift, and arrive at an ‘approved’ dataset. The Observations Data Model (ODM), which is part of the Consortium of Universities for the Advancement of Hydrologic Science, Inc. (CUAHSI) Hydrologic Information System (HIS), is a data model for organizing, storing, and describing point observations data. We have developed ODM Tools Python as an open source software application for managing sensor data stored in an ODM database. Previous versions of ODM Tools included functionality to export data series and associated metadata, plot and summarize a single data series, generate derivative data series, and edit data series using a set of simple tools. We have re-designed and deployed a new version of ODM Tools in Python that adds a modernized graphical user interface with dockable windows, multiple platform support (Windows, Linux, and Mac), multiple database support (Microsoft SQL Server, MySQL, PostgreSQL), enhanced plotting capability, and support for automated scripting of quality control edits. Automated scripting is accomplished through a Python script editor and console integrated with the ODM Tools Python interface. Scripting records the corrections and adjustments performed on a data series in the quality control process, ensuring that editing steps performed using the visual tools in the graphical user interface are traceable and reproducible. Furthermore, the improved ability of ODM Tools Python to plot multiple data series simultaneously with various plot types, facilitates comparisons between a data series that is being edited with time series for other variables at the same site (internal consistency) or for the same variable at other sites (external consistency). 1 Utah Water Research Laboratory, Utah State University, Logan, Utah, USA 2 Department of Civil and Environmental Engineering and Utah Water Research Laboratory, Utah State University, Logan, Utah, USA

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    of 1

    !"# %&&'( )*+,&-. !

    /0- 1&2340 1&5+6730


    5&3 #7-789-8 :-;93&-<0-+7' 10-(&3 "7+7
    !"#$%&#$"'( LnvlronmenLal daLa are belng collecLed aL
    hlgh frequencles, for exLended durauons, and wlLh spaual
    dlsLrlbuuons requlrlng lnfrasLrucLure for daLa managemenL.
    1hese daLa oen need quallLy conLrol and posL processlng
    performed Lo remove anomalous values, accounL for drl,
    and arrlve aL an 'approved' daLaseL. SLandard Lools for
    performlng daLa CACC enhance Lhe ablllLy Lo Lrack daLa
    provenance and repeaL edlLs Lo faclllLaLe daLa reuse.
    *'+,-./$'0 1-&!,2"-3(
    "+!: a daLa model for
    organlzlng, sLorlng, and
    descrlblng polnL observauons
    daLa orlglnally developed as
    parL of Lhe CuAPSl PlS.
    %$4*&.$5&#$"'(
    laclllLaLes lnspecuon of daLa Lo ldenufy and assess pauerns
    and problems.
    Muluple serles can be ploued slmulLaneously Lo faclllLaLe
    comparlsons wlLh oLher varlables (lnLernal conslsLency) and
    oLher slLes (exLernal conslsLency).
    LdlLed daLa lmmedlaLely avallable for lnspecuon aer
    modlcauon.
    varlous ploL Lypes lmplemenLed.

    ,+$#$'0 &'+ &*#"!&#,+ 46-$7#$'0(
    All edlLs performed on a daLa serles are auLomaucally recorded uslng a
    yLhon scrlpL edlLor and console lnLegraLed wlLh Lhe CuM 1ools yLhon
    lnLerface.
    1ools are bullL-ln for common edlung Lasks, and Lhe yLhon console and
    scrlpL edlLor can be used Lo perform advanced edlung Lasks.
    Lach acuon ls LranslaLed lnLo execuLable code LhaL ls capLured ln Lhe scrlpL.
    Scrlpung ensures LhaL edlung sLeps are Lraceable and reproduclble.
    AnnoLauons Lo Lhe edlLs can also be made as commenLs ln Lhe scrlpL.

    =<>03 1/74?<7- @&-0(A @05503* 1B C&3(>238,A 1+0/,7-90 DB E00F03A @74&> #0'9-0
    G+7, H7+03 E0(0734, D7>&37+&3*A G+7, 1+7+0 G-9;03(9+*
    "7,' 4"*-6, -,7"4$#"-/( CuM 1ools yLhon
    releases and code are avallable on ClLPu8
    hups://glLhub.com/uCPlC/CuM1oolsyLhon
    1hls pro[ecL ls funded by nauonal
    Sclence loundauon granL LS-1208732.
    Any oplnlons, ndlngs, and concluslons
    or recommendauons expressed ln Lhls
    maLerlal are Lhose of Lhe auLhor(s) and
    do noL necessarlly reecL Lhe vlews of
    Lhe nauonal Sclence loundauon.
    "+! #889: 7;<=8> ls an open source soware appll-
    cauon for managlng daLa sLored ln an CuM daLabase.
    1he 8?@A@>B9 "+! #889: soware was developed
    as parL of Lhe CuAPSl PlS. We have re-deslgned
    and deployed a new verslon ln yLhon.
    $!7-"%,+ 1*'6#$"'&.$#/(
    !C9DE9F 79BG8?H 4CEE8?<( Wlndows, Mac, and Llnux
    !C9DE9F -+I!4 4CEE8?<( Can connecL Lo CuM daLabases
    deployed wlLh SCLServer, MySCL, or osLgreSCL.
    JCF?; B>K ,LE8?<( Slmple lLers are lmplemenLed for daLa
    serles selecuon. uaLa and assoclaLed meLadaLa can be
    exporLed from Lhe daLabase.
    +8MNBO9F 2@>K8P:( varlous componenLs can be dlsplayed,
    hldden, and docked for user exlblllLy.
    CommlL LdlLs Lo
    Lhe uaLabase
    Common
    Ldlung 1ools
    Serles Selecuon
    lllLers 1lme Serles
    Selecuon
    uynamlc
    Zoomlng and
    annlng
    Muluple
    loL 1ypes
    uaLe 8ange
    8esLrlcuons
    loL ulsplay
    Cpuons lllLers on
    uaLa olnLs
    1abular uaLa
    Selecuon
    uynamlc uaLa
    Ldlung ulsplay
    yLhon Code
    Console
    yLhon
    ScrlpL LdlLor
    2"-31."2(
    new serles can be derlved wlLh a meLadaLa componenL
    modled (e.g., quallLy conLrol level, varlable) and Lhe
    remalnlng meLadaLa reLalned.
    Ceneraung and preservlng provenance for hlgher level
    daLa producLs (l.e., quallLy conLrol level 1, 2, eLc.)
    ls sLreamllned.
    8aw daLa are always preserved-
    edlLs are performed on a copy
    of Lhe daLabase sLored ln
    memory.
    1he edlung scrlpL can be saved
    for fuLure reference and execuuon.
    users can compare, revlew, and modlfy
    Lhe edlung sLeps.

    You might also like