hec.datastore

Provides classes for cataloging, storing, retrieving, and deleting data using various types of data stores.

Comprises the classes:

class DataStoreException(builtins.Exception):

Base class for all data store exceptions

class CwmsDataStore(AbstractDataStore):

Class to facilitate cataloging, storing, retrieving, and deleting data in CWMS databases.

Requires installation of the cwms-python >= '0.8.2'.

CwmsDataStore(**kwargs: Any)

Creates and returns a new CwmsDataStore object.

Equivalent of calling CwmsDataStore.open(name, **kwargs)

Arguments:
  • api_key (Optional[str], must be passed by name): The API key for this data store. Must be specified to write to or delete from this data store. Defaults to None. If None:
    • The value of environment variable "cda_api_key" is used, if it exists.
  • description (Optional[str], must be passed by name): The description assocaited with the data store. Defaults to None
  • end_time (Optional[Any], must be passed by name): Specifies the end time of the data store's time window. Must be an HecTime object or a valid input to the HecTime constructor. Defaults to None
  • name (Optional[str], must be passed by name): The API root (base URL). Defaults to None. If None:
    • The value of environment variable "cda_api_root" is used, if it exists.
    • If the environment variable "cda_api_root" is not set, the default, the default value used in cwms.api.init_session(api_root=None) is used.
  • office (Optional[str], must be passed by name): The default CWMS office for the data store. If None or not specified, each access method will have to have an office specified.
  • read_only (Optional[bool], must be passed by name): Specifies whether to open the data store in read-only mode. Defaults to True
  • start_time (Optional[Any], must be passed by name): Specifies the start time of the data store's time window. Must be an HecTime object or a valid input to the HecTime constructor. Defaults to None
  • store_rule (Optional[str], must be passed by name): Specifies the default behavior to use when storing data. If specified, it must be one of the following (case insensitive). Defaults to 'REPLACE_ALL'.
    • 'DELETE_INSERT' - delete all existing data in the incoming time window, then store the incoming data
    • 'DO_NOT_REPLACE' - store only non-existing data
    • 'REPLACE_ALL' - store existing and non-existing data
    • 'REPLACE_MISSING_VALUES_ONLY' - store incoming data only where existing data is missing
    • 'REPLACE_WITH_NON_MISSING' - store only non-missing incoming data
  • time_zone (Optional[str], must be passed by name): The default time zone for the data store. Defaults to the local time zone.
  • trim (Optional[bool], must be passed by name): Specifies the data store's default setting to trim missing values from the beginning and end of any regular time series data set retrieved. Defaults to True.
  • units: (Optional[str], must be passed by name): "EN" or "SI", specifying English or metric unit system as the default unit system for the data store. Defaults to "EN"
  • vertical_datum: (Optional[str], must be passed by name): "NGVD29", "NAVD88", or "NATIVE", specifying the data store's default vertical datum for retrieving elevation data. Defaults to "NATIVE"
def catalog(self, data_type: Optional[str] = None, **kwargs: Any) -> List[str]:

Retrieves CWMS Identifiers for the specified data type, optionally with extents for specific data types.

Arguments:
  • data_type (str): Must be one of the following (case insensitive):
    • 'LOCATION'
    • 'RATING'
    • 'RATING_SPECIFICATION'
    • 'RATING_TEMPLATE'
    • 'TIMESERIES'

  • pattern (Optional[str], must be passed by name): An extended wildcard pattern to use for matching Identifiers. regex takes precedence if both are specified. Defaults to None.

    

    Pattern Examples
    

    patternmatches
    

    abcthe literal string "abc"
    

    ab{2,3}c{2}1 "a" followed by 2 or 3 "b" followed by 2 "c"
    

    a*b?c1 "a" followed by zero or more characters followed by "b" followed by 1 character followed by 1 "c"
    

    [abc]1 "a" or "b" or "c"
    

    ^[abc]$beginning of string followed by 1 "a" or "b" or "c" followed by end of string
    

    [!abc]1 character other than "a" or "b" or "c"
    

    [_a-z0-9]*zero or more of characters "_" or "a" through "z" or "0" through "9"
    

    [!a-z0-9]{1,5}1..5 characters other than "a" through "z" or "0" through "9"
    

    (abc|def)either "abc" or "def"
    


    • 

  • regex (Optional[str], must be passed by name):  Regular expression to use for matching Identifiers. Takes precedence over pattern if both are specified. Defaults to None.
    • 

  • bounding_office (Optional[str]), must be passed by name, LOCATION and TIMESERIES only):  Specifies cataloging only Identifiers that are physically located within the boundaries of the specified office.
Can be a wildcard pattern. Matching is affected by case_sensitive.
    • 

  • case_sensitive (Optional[bool], must be passed by name):  Specifies whether and pattern or regular expression matching is case-sensitive.
    • 

  • category (Optional[str], must be passed by name, LOCATION only):  Specifies cataloging only locations in a location group belonging to the specified catgory(ies). Can be a wildcard pattern.
Matching is affected by case_sensitive. Note that specifying category or group, or including "aliases" in fields will slow down the catalog operation.
    • 

  • fields (Optional[str]), must be passed by name):  A comma-separated list of fields to include in the catalog. Valid fields for data_type are listed below.
The catalog will include the fields in the order specified. Defaults to identifier.

      

    • TIMESERIES:

        

      • identifier: The time series identifier
        • 

      • office: The CWMS office for the time series
        • 

      • name: Same as identifier
        • 

      • time-zone: The time zone of the location of the time seroies
        • 

      • interval: The interval of the time series
        • 

      • offset:  The offset into each interval of regular time series (in minutes), or  if interval is irregular
        • 

      • earliest-time: The earliest time in the database for this time series, or  if no data
        • 

      • latest-time: The latest time in the database for this time series of  if no data
        • 

      • last-update: The most recent time this time series has been updated, or  of no data
        • 

      • 

    • RATING:

        

      • identifier: The rating specification
        • 

      • office: The CWMS office for the rating
        • 

      • name: Same as identifier
        • 

      • location: The location identifier
        • 

      • template: The rating template identifier
        • 

      • specification-version: The specification version
        • 

      • description: The description for the rating specification
        • 

      • lookup-methods: A comma-delimited string of effective date lookup methods (order = within-time-range, before-time-range, after-time-range)
        • 

      • specification-active: Whether the specification is active
        • 

      • auto-flags: True or False values for auto-update, auto-activate and auto-migrate-extensions, respectively
        • 

      • rounding-specs: A delimited string of rounding specifications for each indepedent and dependent parameter. Independent parameter roundings are comma-delimited; the dependent parameter rounding is semicolon-delimited.
        • 

      • effective-date: The effective date of the rating
        • 

      • create-date: The creation date date of the rating
        • 

      • units: The native units of the rating. Independent parameter units are comma-delimited; the dependent parameter unit is semicolon-delimited.
        • 

      • active: Whether the rating is active
        • 

      • type: The rating type
        • 

      • 

    • RATING_SPECIFICATION:

        

      • identifier: The rating specification
        • 

      • office: The CWMS office for the rating specification
        • 

      • name: Same as identifier
        • 

      • location: The location identifier
        • 

      • template: The rating template identifier
        • 

      • version: The specification version
        • 

      • description: The description for the rating specification
        • 

      • lookup-methods: A comma-delimited string of effective date lookup methods (order = within-time-range, before-time-range, after-time-range)
        • 

      • active: Whether the specification is active
        • 

      • auto-flags: True or False values for auto-update, auto-activate and auto-migrate-extensions, respectively
        • 

      • rounding-specs: A delimited string of rounding specifications for each indepedent and dependent parameter. Independent parameter roundings are comma-delimited; the dependent parameter rounding is semicolon-delimited.
        • 

      • effective-date: A comma-delimited string of effective date/times for ratings with this specification
        • 

      • 

    • RATING_TEMPLATE:

        

      • identifier: The rating template identifier
        • 

      • office: The CWMS office for the rating template
        • 

      • name: Same as identifier
        • 

      • independent-parameters: A comma-delimited string of independent parameter names
        • 

      • dependent-parameter: The dependent parameter name
        • 

      • version The template version
        • 

      • lookup-methods: A semicolon-delimited string of comma-delimited lookup methods for each independent parameter (order = in-range, below-range, above-range)
        • 

      • rating-ids: A list of rating ids that use the template. The rating ids are specified as the string representation of a list (i.e., an actual list
can be obtained by passing this field to eval())
        • 

      • 

    • LOCATION:

        

      • identifier: The location identifier
        • 

      • office: The CWMS office for the location
        • 

      • name: Same as identifier
        • 

      • nearest-city: The name of the city or town closest to the location
        • 

      • public-name: The public name of the location
        • 

      • long-name: The long name of the location
        • 

      • kind: The kind of location - constrained to:

          

        • SITE
          • 

        • EMBANKMENT
          • 

        • OVERFLOW
          • 

        • TURBINE
          • 

        • STREAM
          • 

        • PROJECT
          • 

        • STREAMGAGE
          • 

        • BASIN
          • 

        • OUTLET
          • 

        • LOCK
          • 

        • GATE
          • 

        • 

      • time-zone: The time zone of the location
        • 

      • latitude: The latitude of the location
        • 

      • longitude: The latitude of the location
        • 

      • published-latitude: The published latitude of the location
        • 

      • published-longitude: The published longitude of the location
        • 

      • horizontal-datum: The horizontal datum associated with the latitude and longitude
        • 

      • elevation: The elevation of the location
        • 

      • unit: The unit of the elevation
        • 

      • vertical-datum: The vertical datum of the elevation
        • 

      • nation: The nation containing the location
        • 

      • state: The state/province containing the location
        • 

      • county: The county containing the location
        • 

      • bounding-office: The CWMS office whose boundary includes the location
        • 

      • map-label: The map label of the location
        • 

      • active: Whether the location is active
        • 

      • aliases: The aliases associated with the location. Note that including this slows down the catalog operation. The aliases are specified as the string
representation of a dictionary (i.e., an actual dictionary can be obtained by passing this field to eval()). The dictionary keys are the location
categories and groups for each alias, and the values are the aliases. The keys are of the form {category}-{group} (e.g., "Angency Aliases-NWS Handbook 5 ID")
        • 

      • description: The description of the location
        • 

      • type: The unconstrained type of the location (cf kind)
        • 

      • 

    • 

  • group (Optional[str], must be passed by name, LOCATION only):  Specifies cataloging only locations in the specivied location group(s). Can be a wildcard pattern.
Matching is affected by case_sensitive. Note that specifying category or group, or including "aliases" in fields will slow down the catalog operation.
    • 

  • header (Optional[bool], must be passed by name):  Specifies whether to include a header line in the catalog that identifies the fields
    • 

  • kind (Optional[str], must be passed by name, LOCATION only):  Specifies cataloging only locations of the specified location kind. Can be a wildcard pattern.
Matching is affected by case_sensitive.
    • 

  • limit (Optional[int], must be passed by name, LOCATION and TIMESERIES only):  The maximum number of Identifiers to return. If None, no limit is imposed. Defaults to None.
    • 

  • office (Optional[str], must be passed by name):  The CWMS office to generate the catalog for. Defaults to None, which uses the data store's default office.
    • 

  • units (Optional[str], must be passed by name, LOCATION and TIMESERIES only):  The unit system ("EN" or "SI") to return the elevation values in. Defaults to None.
    • 

  • vertical_datum (Optional[str], must be passed by name, LOCATION and TIMESERIES only):  The vertical datum ("NGVD29", "NAVD88", or "LOCAL") to return the elevation values in. Defaults to None (Native datum).
    • 




Raises:



    

  • DataStoreException:  if the data store is not open or an invalid data_type is specified
    • 




Returns:




  
List[str]: The CWMS Identifiers that match the specified parameters, up to the specified limit, if any







                            

                            

                                

            
        def
        close(self) -> None:

        
    

    
    
            
Explicitly closes the data store, preventing further access.



The data store is implicitly closed upon deletion or exiting a context manager ('with' block)





                            

                            

                                

            
        def
        delete(self, identifier: str, **kwargs: Any) -> None:

        
    

    
    
            
Deletes a data set from the data store.



Currently only time series data may be deleted.



Arguments:



    

  • identifier (str):  The name of the data set to delete. Must be a valid one of the following:

      

    • time series identifier: start and end times specify the time window of time series values to delete, inclusive
      • 

    • 

  • delete_action (Optional[str], must be passed by name):  Defaults to None. If specified, any time window is ignored and it must be one of (case insensitive):

      

    • 'DELETE_ALL': delete all data and the identifier for the object
      • 

    • 'DELETE_DATA': delete all data for the identifier but does not delete the identifier
      • 

    • 'DELETE_KEY': delete only the object identifier - will fail if any data is associated with the identifier
      • 

    • 

  • end_time (Optional[Any], must be passed by name):  Specifies the end of the time window to delete data. Ignored if delete_action is specified. Must be an HecTime object or a valid input to the HecTime constructor.
Defaults to the end of the data store's time window.
    • 

  • office (Optional[str], must be passed by name):  The CWMS office to delete data for. Defaults to None, which uses the data store's default office.
    • 

  • start_time (Optional[Any], must be passed by name):  Specifies the start of the time window to delete data. Ignored if delete_action is specified. Must be an HecTime object or a valid input to the HecTime constructor.
Defaults to the start of the data store's time window.
    • 

  • version_time (Optional[Any], must be passed by name):  Specifies the version date/time of the data to delete (time series types only). Must be an HecTime object or a valid input to the HecTime constructor.
Defaults to the None, meaning non-versioned data.
    • 






                            

                            

                                

            
        def
        get_extents(self, identifier: str, **kwargs: Any) -> List[hec.HecTime]:

        
    

    
    
            
Retrieves the data extents for the specified identifier



Arguments:



    

  • identifier (str):  The identifier to retrieve the extents for
    • 




Returns:




  
List[HecTime]: The earliest time, latest time, and latest update time







                            

                            

                                

            
        def
        get_vertical_datum_info(self, identifier: str, **kwargs: Any) -> Optional[str]:

        
    

    
    
            
Retrieves the vertical datum information for a CWMS location



Arguments:



    

  • identifier (str):  The location identifier
    • 

  • office (Optional[str]):  The CWMS office for the location. Defaults to the data store's default office.
    • 

  • format (Optional[str]):  The output format for the information. Must be one of 'JSON' or 'XML' (case insensitive). Defaults to 'JSON'
    • 




Returns:




  
Optional[str]: The vertical datum information in the specified format







                            

                            

                                

            is_open: bool

        
    

    
    
            
Whether this data store is open for cataloging and reading.



A data store is open from its construction or call to open() until a call to close() or deletiion



Operations:




  
Read Only







                            

                            

                                

            native_data_store: Any

        
    

    
    
            
The cwms module imported by this data store



Operations:




  
Read-Only







                            

                            

                                

                    
@staticmethod


        def
        open(name: Optional[str] = None, **kwargs: Any) -> CwmsDataStore:

        
    

    
    
            
Creates and returns a new CwmsDataStore object.



Equivalent of calling CwmsDataStore(**kwargs) with name in kwargs



Arguments:



    

  • name (Optional[str]):  The API root (base URL). Defaults to None. If None:

      

    • The value of environment variable "cda_api_root" is used, if it exists.
      • 

    • If the environment variable "cda_api_root" is not set, the default, the default value used in cwms.api.init_session() is used.
      • 

    • 

  • api_key (Optional[str], must be passed by name):  The API key for this data store. Must be specified to write to or delete from this data store. Defaults to None. If None:

      

    • The value of environment variable "cda_api_key" is used, if it exists.
      • 

    • 

  • description (Optional[str], must be passed by name):  The description assocaited with the data store. Defaults to None
    • 

  • end_time (Optional[Any], must be passed by name):  Specifies the end time of the data store's time window. Must be an HecTime object or a valid input to the HecTime constructor. Defaults to None
    • 

  • office (Optional[str], must be passed by name):  The default CWMS office for the data store. If None or not specified, each access method will have to have an office specified.
    • 

  • read_only (Optional[bool], must be passed by name):  Specifies whether to open the data store in read-only mode. Defaults to True
    • 

  • start_time (Optional[Any], must be passed by name):  Specifies the start time of the data store's time window. Must be an HecTime object or a valid input to the HecTime constructor. Defaults to None
    • 

  • store_rule (Optional[str], must be passed by name):  Specifies the default behavior to use when storing data. If specified, it must be one of the following (case insensitive). Defaults to 'REPLACE_ALL'.

      

    • 'DELETE_INSERT' - delete all existing data in the incoming time window, then store the incoming data
      • 

    • 'DO_NOT_REPLACE' - store only non-existing data
      • 

    • 'REPLACE_ALL' - store existing and non-existing data
      • 

    • 'REPLACE_MISSING_VALUES_ONLY' - store incoming data only where existing data is missing
      • 

    • 'REPLACE_WITH_NON_MISSING' - store only non-missing incoming data
      • 

    • 

  • time_zone (Optional[str], must be passed by name):  The default time zone for the data store. Defaults to the local time zone.
    • 

  • trim (Optional[bool], must be passed by name):  Specifies the data store's default setting to trim missing values from the beginning and end of any regular time series data set retrieved.
Defaults to True.
    • 

  • units (Optional[str], must be passed by name):  "EN" or "SI", specifying English or metric unit system as the default unit system for the data store. Defaults to "EN"
    • 

  • vertical_datum (Optional[str], must be passed by name):  "NGVD29", "NAVD88", or "NATIVE", specifying the data store's default vertical datum for retrieving elevation data. Defaults to "NATIVE"
    • 






                            

                            

                                

            
        def
        retrieve(self, identifier: str, **kwargs: Any) -> Any:

        
    

    
    
            
Retrieves a data set from the CWMS database.



Currently only Location, AbstractRatingSet TimeSeries objects may be retrieved.



To retrieve all data for a TimeSeries, specifiy start_time=None and end_time=None



Arguments:



    

  • Positional Arguments: 
    

      

    • identifier (str): The name of the data set to retrieve. Must be a valid identifier for one of the retrievable data set types.
      • 

    • 

  • Keyword Arguments (Optional, must be passed by name): 
    

      

    • All Identifiers:
      

        

      • office (str): The CWMS office to retrieve data for. Defaults to None, which uses the data store's default office.
        • 

      • 

    • Location Identifiers:
      

        

      • units (str): "EN" or "SI", specifying to retrieve data in English or metric units. Defaults to None, which uses the default unit system for the data store
        • 

      • vertical_datum (str): "NGVD29", "NAVD88", or "NATIVE", specifying the vertical datum to retrieve elevation data for. Defaults to None, which uses the data store's default vertical datum
        • 

      • 

    • Rating Set Identifiers:
      

        

      • effective_time (datetime): Retrieves a LocalRatingSet object with only one rating (at the specified effective_time). Causes the method argument
to be ignored, with "EAGER" retrieval implied. Used by the library to populate the rating points of individual TableRating objects that were previously
retrieved with the "LAZY" method.
        • 

      • method (str): The method used to retrieve the rating set from the database. Restricted to "EAGER", "LAZY", and "REFERENCE". Defaults to "LAZY".

          

        • If "REFERENCE",  a ReferenceRatingSet is retrieved, where all values are sent to the database to be rated.
          • 

        • If "EAGER", a LocalRatingSet is retrieved, where all ratings are performed in python code. Rating points for all included
TableRating objects are retrieved when the rating set is retrieved.
          • 

        • If "LAZY", a LocalRatingSet is retrieved, where all ratings are performed in python code. Rating points for all included
TableRating objects are not retrieved until when the individual TableRating objects are first used.
          • 

        • 

      • 

    • Time Series Identifiers:
      

        

      • start_time (Any): Specifies the start of the time window to retrieve data. Must be an HecTime object or a valid input to the HecTime constructor.
Defaults to the start of the data store's time window. If None or not specified and the data store's time window doesn't have a start time, the current time minus 24 hours is used
        • 

      • end_time (Any): Specifies the end of the time window to retrieve data. Must be an HecTime object or a valid input to the HecTime constructor.
Defaults to the end of the data store's time window. If None or not specified and the data store's time window doesn't have an end time, the current time is used
        • 

      • trim (bool): Specifies whether to trim missing values from the beginning and end of any regular time series data set retrieved.
Defaults to the data store's trim setting.
        • 

      • units (str): "EN" or "SI", specifying to retrieve data in English or metric units. Defaults to None, which uses the default unit system for the data store
        • 

      • version_time (Any)): Specifies the version date/time of the data to retrieve (time series types only). Must be an HecTime object or a valid input to the HecTime constructor.
Defaults to the None, which uses the data store's default vertical datum.
        • 

      • vertical_datum (str): "NGVD29", "NAVD88", or "NATIVE", specifying the vertical datum to retrieve elevation data for. Defaults to None, which uses the data store's default vertical datum
        • 

      • 

    • 




Returns:




  
Any: The Location, AbstractRatingSet TimeSeries object







                            

                            

                                

            
        def
        store(self, obj: object, **kwargs: Any) -> None:

        
    

    
    
            
Stores a data set to the data store.



Currently only locations and time series data may be stored.



Arguments:



    

  • obj (object):  The data set to store.

      

    • For locations, must be a Location object or the name of the location
      • 

    • 

  • Location Arguments: 
    


      

    • name (Optional[str], must be passed by name): The location's name, Defaults to the Location object's name
      • 

    • office (Optional[str], must be passed by name): The location's office. Defaults to the Location object's office
      • 

    • kind (str, must be passed by name): The location's kind. Must be on of:

        

      • SITE
        • 

      • EMBANKMENT
        • 

      • OVERFLOW
        • 

      • TURBINE
        • 

      • STREAM
        • 

      • PROJECT
        • 

      • STREAMGAGE
        • 

      • BASIN
        • 

      • OUTLET
        • 

      • LOCK
        • 

      • GATE
        • 

      • 

    • active (Optional[bool], must be passed by name): Whether the location is active in the datbase. Defaults to True
      • 

    • latitude (Optional[float], must be passed by name): The location's latitude. Defaults to the Location object's latitude.
      • 

    • longitude (Optional[float], must be passed by name): The location's longitude. Defaults to the Location object's longitude
      • 

    • horizontal_datum (Optional[str], must be passed by name): The location's horizontal datum
      • 

    • elevation (Optional[float], must be passed by name): The location's elevation. Defaults to the Location object's elevation
      • 

    • elevation_unit (Optional[str], must be passed by name): The unit of the location's elevation. Defaults to the Location object's elevation unit
      • 

    • vertical_datum (Optional[str], must be passed by name): The location's vertical datum. Defaults to the location's vertical datum
      • 

    • nation (Optional[str], must be passed by name): The location's nation. Defaults to "US"
      • 

    • state (Optional[str], must be passed by name): The location's state. Defaults to None
      • 

    • county (Optional[str], must be passed by name): The locations's county. Defaults to None
      • 

    • bounding_office (Optional[str], must be passed by name): The CWMS office whose boundar includes the location. Defaults to None
      • 

    • nearest_city (Optional[str], must be passed by name): The city nearest the location. Defaults to None
      • 

    • time_zone (Optional[str], must be passed by name): The location's time zone. Defaults to the location's time zone
      • 

    • public_name (Optional[str], must be passed by name): The location's public name. Defaults to None
      • 

    • long_name (Optional[str], must be passed by name): The location's long name. Defaults to None
      • 

    • description (Optional[str], must be passed by name): The location's description. Defaults to None
      • 

    • map_label (Optional[str], must be passed by name): The location's map label. Defaults to None
      • 

    • published_latitude (Optional[float], must be passed by name): The published latitude of the location. Defaults to None
      • 

    • published_longitude (Optional[float], must be passed by name): The published longitude of the location. Defaults to None
      • 

    • type (Optional[str], must be passed by name): An unconstrained type for the location. Defaults to None

       
      • 

    


    The following items are required to be specified either in the Location object or arguments:
    
      
    
    • name
      • 
    
    • office
      • 
    
    • kind
      • 
    
    • latitude
      • 
    
    • longitude
      • 
    
    • horizontal_datum
      • 
    
    • time_zone
      • 
    
    

    


    • 

  • TimeSeries Arguments: 
    

      

    • as_lrts (Optional[bool], must be passed by name): Specifies whether to store the time series as LRTS is its interval starts with "~". Defaults to whether the time series is a local regular time series
      • 

    • override_protecteion (Optional[bool], must be passed by name): Specifies whether to store non-protected values over existing protected values. (Protected values always overwrite existing values and non-protected existing values are alway overwritten) Defaults to False
      • 

    • store_rule (Optional[str], must be passed by name): Specifies the default behavior to use when storing data. If specified, it must be one of the following (case insensitive). Defaults to 'REPLACE_ALL'.

        

      • 'DELETE_INSERT' - delete all existing data in the incoming time window, then store the incoming data
        • 

      • 'DO_NOT_REPLACE' - store only non-existing data
        • 

      • 'REPLACE_ALL' - store existing and non-existing data
        • 

      • 'REPLACE_MISSING_VALUES_ONLY' - store incoming data only where existing data is missing
        • 

      • 'REPLACE_WITH_NON_MISSING' - store only non-missing incoming data
        • 

      • 

    • 






                            

                            

                                
Inherited Members

                                

                                    
AbstractDataStore

                                
end_time

                
name

                
office

                
is_read_only

                
start_time

                
time_window

                
time_zone

                
trim

                
units

                
vertical_datum


            

                                

                            

                

                

                    

            
    class
    DssDataStore(AbstractDataStore):

        
    

    
    
            
Class to facilitate cataloging, storing, retrieving, and deleting data in HEC-DSS files.



Requires installation of the 'hecdss' package >= '0.1.28'.






Note on using DssDataStore objects with rating objectsDssDataStore objects can be used to store and retrieve
LocalRatingSet objects. Since these objects are not
identifiable with a single pathname, rating specification identifiers (which are mapped to multiple pathnames) are used
to retrieve LocalRatingSet objects from the data store. The portions
of rating sets (i.e., rating templates, rating specifications, and individual ratings) may be cataloged with the appropiate
identifiers mathched by the pattern or regex keyword arguments to the catalog() method:

For cataloging purposes, the data_type parameter must be specified (i.e., "RATING_TEMPLATE",
"RATING_SPECIFICATION" or "RATING"). The catalog() method normally returns
identifiers for these data types, but specifying pathnames=True causes pathnames to be returned instead, as shown
in the command-line session below.

All rating related pathnames have D pathname parts that start with "Rating-". The mapping of the CWMS-style
identifiers onto HEC-DSS pathnames is:






    Object Type
    A Part
    B Part
    C Part
    D Part
    E Part
    F Part
    Record Body




    Rating Template
    Office ID
    
    Parameters ID
    "Rating‑Template"
    Template Version
    
    Rating Template XML




    Rating Specification
    Office ID
    Location ID
    Parameters ID
    "Rating‑Specification"
    Template Version
    Specification Version
    Rating Specification XML




    Rating
    Office ID
    Location ID
    Parameters ID
    "Rating‑Body‑Effective‑Time"
    Template Version
    Specification Version
    Rating XML minus any TableRating points (for lazy loading)




    Table Rating Points
    Office ID
    Location ID
    Parameters ID
    "Rating‑Points‑Effective‑Time"
    Template Version
    Specification Version
    Rating XML including any TableRating points (for populating after lazy loading)




For example, in the image below, LocalRatingSet with the following rating 
specification identifiers have been stored to an empty HEC-DSS file. Both rating sets were for office SWT.

    

  • COUN.Count-Conduit_Gates,Opening-Conduit_Gates,Elev;Flow-Conduit_Gates.Standard.Production/
    • 

  • KEYS.Elev;Stor.Linear.Production/
    • 









The following command line session demonstrates cataloging the file and retrieving one of the rating sets:


>>> from hec import DssDataStore
>>> dss = DssDataStore.open(r"U:\Devl\git\hec-python-library\test\resources\rating\local_rating_set.dss")
16:50:45.507      -----DSS---zopen   Existing file opened,  File: U:\Devl\git\hec-python-library\test\resources\rating\local_rating_set.dss
16:50:45.507                         Handle 3;  Process: 18724;  DSS Versions - Software: 7-IU, File:  7-IU
16:50:45.508                         Single-user advisory access mode
>>>
>>> for item in dss.catalog("RATING_TEMPLATE"): print(item)
...
Count-Conduit_Gates,Opening-Conduit_Gates,Elev;Flow-Conduit_Gates.Standard
Elev;Stor.Linear
>>>
>>> for item in dss.catalog("RATING_TEMPLATE", pathnames=True): print(item)
...
/SWT//Count-Conduit_Gates,Opening-Conduit_Gates,Elev;Flow-Conduit_Gates/Rating-Template/Standard//
/SWT//Elev;Stor/Rating-Template/Linear//
>>>
>>> for item in dss.catalog("RATING_SPECIFICATION"): print(item)
...
COUN.Count-Conduit_Gates,Opening-Conduit_Gates,Elev;Flow-Conduit_Gates.Standard.Production
KEYS.Elev;Stor.Linear.Production
>>>
>>> for item in dss.catalog("RATING_SPECIFICATION", pathnames=True): print(item)
...
/SWT/COUN/Count-Conduit_Gates,Opening-Conduit_Gates,Elev;Flow-Conduit_Gates/Rating-Specification/Standard/Production/
/SWT/KEYS/Elev;Stor/Rating-Specification/Linear/Production/
>>>
>>> for item in dss.catalog("RATING"): print(item)
...
COUN.Count-Conduit_Gates,Opening-Conduit_Gates,Elev;Flow-Conduit_Gates.Standard.Production
KEYS.Elev;Stor.Linear.Production
>>>
>>> for item in dss.catalog("RATING", pathnames=True): print(item)
...
/SWT/COUN/Count-Conduit_Gates,Opening-Conduit_Gates,Elev;Flow-Conduit_Gates/Rating-Body-2012-04-26T05:00:00Z/Standard/Production/
/SWT/COUN/Count-Conduit_Gates,Opening-Conduit_Gates,Elev;Flow-Conduit_Gates/Rating-Body-2012-04-27T05:00:00Z/Standard/Production/
/SWT/KEYS/Elev;Stor/Rating-Body-2009-01-14T06:00:00Z/Linear/Production/
/SWT/KEYS/Elev;Stor/Rating-Body-2011-10-19T05:00:00Z/Linear/Production/
/SWT/KEYS/Elev;Stor/Rating-Body-2020-08-01T05:00:00Z/Linear/Production/
>>>
>>> rs = dss.retrieve("KEYS.Elev;Stor.Linear.Production", office="SWT")
>>> print(type(rs))
<class 'hec.rating.local_rating_set.LocalRatingSet'>
>>>
>>> for effective_time in rs.ratings: print(effective_time.isoformat(), rs.ratings[effective_time])
...
2009-01-14T06:00:00+00:00 <hec.rating.table_rating.TableRating object at 0x0000019CBC556EB0>
2011-10-19T05:00:00+00:00 <hec.rating.table_rating.TableRating object at 0x0000019CBC59C220>
2020-08-01T05:00:00+00:00 <hec.rating.table_rating.TableRating object at 0x0000019CBC600880>








                            

                                

            
        DssDataStore(**kwargs: Any)

        
    

    
    
            
Creates and returns a new DssDataStore object.



Equivalent of calling DssDataStore.open(name, **kwargs)



Arguments:



    

  • description (Optional[str], must be passed by name):  The description assocaited with the data store. Defaults to None
    • 

  • end_time (Optional[Any], must be passed by name):  Specifies the end time of the data store's time window. Must be an HecTime object or a valid input to the HecTime constructor. Defaults to None
    • 

  • name (str, must be passed by name):  The name of the HEC-DSS file to open.
    • 

  • read_only (Optional[bool], must be passed by name):  Specifies whether to open the data store in read-only mode. Defaults to True
    • 

  • start_time (Optional[Any], must be passed by name):  Specifies the start time of the data store's time window. Must be an HecTime object or a valid input to the HecTime constructor. Defaults to None
    • 

  • store_rule (Optional[str], must be passed by name):  Must be one of the following (case insensitive). Defaults to 'REPLACE_ALL'.

      

    • 'DELETE_INSERT'
      • 

    • 'DO_NOT_REPLACE'
      • 

    • 'REPLACE_ALL'
      • 

    • 'REPLACE_MISSING_VALUES_ONLY'
      • 

    • 'REPLACE_WITH_NON_MISSING'
      • 

    • 

  • trim (Optional[bool], must be passed by name):  Specifies the data store's default setting to trim missing values from the beginning and end of any regular time series data set retrieved.
Defaults to True.
    • 






                            

                            

                                

            
        def
        catalog(self, data_type: Optional[str] = None, **kwargs: Any) -> List[str]:

        
    

    
    
            
Retrieves pathnames that match specified conditions.



If data_type is "RATING", "RATING-SPECIFICATION", or "RATING-TEMPLATE", CWMS-style identifiers are returned instead of pathnames
unless the pathnames parameter is specifed as True. (See Note on use with rating objects)



Arguments:



    

  • data_type (Optional[str]):  The type of data to retrieve pathnames for. Defaults to None, which specifies all data types.
If specified, must be one of (case insensitive):

      

    • 'ARRAY'
      • 

    • 'GRID'
      • 

    • 'LOCATION'
      • 

    • 'PAIRED_DATA'
      • 

    • 'RATING'
      • 

    • 'RATING_SPECIFICATION'
      • 

    • 'RATING_TEMPLATE'
      • 

    • 'TEXT'
      • 

    • 'TIMESERIES'
      • 

    • 'TIMESERIES_PROFILE'
      • 

    • 'TIN'
      • 

    • 

  • pattern (Optional[str], must be passed by name):  Wildcard pattern (using * and ?) to use for matching pathnames. regex takes precedence if both are specified. Defaults to None (all pathnames are returned).
    


    If data_type is "RATING", "RATING-SPECIFICATION", or "RATING-TEMPLATE", the pattern matches the CWMS-style identifiers instead of pathnames.
    


    

    Pattern Examples
    

    patternmatches
    

    abcthe literal string "abc"
    

    ab{2,3}c{2}1 "a" followed by 2 or 3 "b" followed by 2 "c"
    

    a*b?c1 "a" followed by zero or more characters followed by "b" followed by 1 character followed by 1 "c"
    

    [abc]1 "a" or "b" or "c"
    

    ^[abc]$beginning of string followed by 1 "a" or "b" or "c" followed by end of string
    

    [!abc]1 character other than "a" or "b" or "c"
    

    [_a-z0-9]*zero or more of characters "_" or "a" through "z" or "0" through "9"
    

    [!a-z0-9]{1,5}1..5 characters other than "a" through "z" or "0" through "9"
    

    (abc|def)either "abc" or "def"
    


    • 

  • regex (Optional[str], must be passed by name):  Regular expression to use for matching pathnames. Takes precedence over pattern if both are specified. Defaults to None (all pathnames are retuned).
If data_type is "RATING", "RATING-SPECIFICATION", or "RATING-TEMPLATE", the regex matches the CWMS-style identifiers instead of pathnames.
    • 

  • case_sensitive (Optional[bool], must be passed by name):  Specifies whether pattern or regex matching is case-sensitive. Defaults to False.
    • 

  • condensed (Optional[bool], must be passed by name):  Specifies whether to return a condensed catalog (D-part = time range for time series). Defaults to True
    • 

  • office (Optional[str], must be passed by name):  Specifies the office if data_type is "RATING", "RATING-SPECIFICATION", or "RATING-TEMPLATE". Defaults to None (all offices are matched).
    • 

  • pathnames (Optional[bool]), must be passed by name:  Specifies whether to return pathnames instead of identifiers if data_type is "RATING", "RATING-SPECIFICATION", or "RATING-TEMPLATE". Defaults to False.
    • 




Raises:



    

  • DataStoreException:  if the data store is not open or an invalid data_type is specified
    • 




Returns:




  
List[str]: The pathnames that match the specified parameters







                            

                            

                                

            
        def
        close(self) -> None:

        
    

    
    
            
Explicitly closes the data store, preventing further access.



The data store is implicitly closed upon deletion or exiting a context manager ('with' block)





                            

                            

                                

            
        def
        delete(self, identifier: str, **kwargs: Any) -> None:

        
    

    
    
            
Deletes a data set from the data store.



Currently only time series data may be deleted. To delete all data for a time series, specifiy start_time=None and end_time=None



Arguments:



    

  • identifier (str):  The name of the data set to delete:

      

    • TIMESERIES: A pathname in the dataset. The D part (block start date) is ignored.
      • 

    • 

  • end_time (Optional[Any], must be passed by name):  Specifies the end of the time window to delete data. Must be an HecTime object or a valid input to the HecTime constructor.
Defaults to the end of the data store's time window. If None or not specified and the data store's time window doesn't have an end time, all data on or after the start time will be deleted.
    • 

  • start_time (Optional[Any], must be passed by name):  Specifies the start of the time window to delete data. Must be an HecTime object or a valid input to the HecTime constructor.
Defaults to the start of the data store's time window. If None or not specified and the data store's time window doesn't have a start time, all data up to and on the end time will be deleted.
    • 






                            

                            

                                

                    
@staticmethod


        def
        decode_ints(ints: list[int]) -> str:

        
    

    
    
    

                            

                            

                                

                    
@staticmethod


        def
        encode_str(s: str) -> list[int]:

        
    

    
    
    

                            

                            

                                

            
        def
        get_extents(self, identifier: str, **kwargs: Any) -> List[hec.HecTime]:

        
    

    
    
            
Retrieves the data extents for the specified identifier



Arguments:



    

  • identifier (str):  The identifier to retrieve the extents for
    • 




Returns:




  
List[HecTime]: The earliest time and latest time for the identifier







                            

                            

                                

            is_open: bool

        
    

    
    
            
Whether this data store is open for cataloging and reading.



A data store is open from its construction or call to open() until a call to close() or deletiion



Operations:




  
Read Only







                            

                            

                                

            native_data_store: Any

        
    

    
    
            
The underlying HecDss object from the imported library used by this data store



Operations:




  
Read-Only







                            

                            

                                

                    
@staticmethod


        def
        open(name: Optional[str] = None, **kwargs: Any) -> DssDataStore:

        
    

    
    
            
Creates and returns a new DssDataStore object.



Equivalent of calling DssDataStore(**kwargs) with name in kwargs



Arguments:



    

  • name (str):  The name of the HEC-DSS file to open.
    • 

  • description (Optional[str], must be passed by name):  The description assocaited with the data store. Defaults to None
    • 

  • end_time (Optional[Any], must be passed by name):  Specifies the end time of the data store's time window. Must be an HecTime object or a valid input to the HecTime constructor. Defaults to None
    • 

  • read_only (Optional[bool], must be passed by name):  Specifies whether to open the data store in read-only mode. Defaults to True
    • 

  • start_time (Optional[Any], must be passed by name):  Specifies the start time of the data store's time window. Must be an HecTime object or a valid input to the HecTime constructor. Defaults to None
    • 

  • store_rule (Optional[str], must be passed by name):  Must be one of the following (case insensitive). Defaults to 'REPLACE_ALL'.

      

    • 'DELETE_INSERT'
      • 

    • 'DO_NOT_REPLACE'
      • 

    • 'REPLACE_ALL'
      • 

    • 'REPLACE_MISSING_VALUES_ONLY'
      • 

    • 'REPLACE_WITH_NON_MISSING'
      • 

    • 

  • trim (Optional[bool], must be passed by name):  Specifies the data store's default setting to trim missing values from the beginning and end of any regular time series data set retrieved.
Defaults to True.
    • 






                            

                            

                                

            
        def
        retrieve(self, identifier: str, **kwargs: Any) -> Any:

        
    

    
    
            
Retrieves a data set from the data store.



Currently only LocalRatingSet, PairedData and TimeSeries objects can be retrieved.
(See Note on use with rating objects)



To retrieve all data for a TimeSeries, specifiy start_time=None and end_time=None



Arguments:



    

  • Positional Arguments: 
    

      

    • identifier (str): The name of the data set to retrieve:
      

        

      • LocalRatingSet: A valid rating specification identifier. NOTE: LocalRatingSet objects cannot be retrieved by pathname.
        • 

      • PairedData: A valid paired-data pathname
        • 

      • TimeSeries: A valid time series pathname. NOTE: Unlike the Java HEC-DSS library, the D part (block start date) is always ignored.
Set the datastore time window or use start_time and end_time keyword arguments to retrieve data for a time window.
        • 

      • 

    • 

  • Keyword Arguments (Optional, must be passed by name): 
    

      

    • LocalRatingSet Identifiers:
      

        

      • effective_time (datetime): Retrieves a LocalRatingSet object with only one rating (at the specified effective_time).
Causes the method argument to be ignored, with "EAGER" retrieval implied. Used by the library to populate the rating points of individual
TableRatingSet objects that were previously retrieved with the "LAZY" method.
        • 

      • method (str): The method used to retrieve the rating set from the HEC-DSS file. Restricted to "EAGER" and "LAZY".

          

        • If "EAGER", a LocalRatingSet is retrieved with rating points for all included
TableRating objects are retrieved fully populated.
          • 

        • If "LAZY", a LocalRatingSet is retrieved omitting the rating points for all included
TableRating objects until they are first needed.
          • 

        • 

      • 

    • PairedData Identifiers:
      

        

      • dep_rounding (UsgsRounder|str): Rounds all of the dependent parameter values using the specified rounder or rounding spec after retrieval from the HEC-DSS file.
        • 

      • ind_rounding (UsgsRounder|str): Rounds all of the independent parameter values using the specified rounder or rounding spec after retrieval from the HEC-DSS file.
        • 

      • rounding (UsgsRounder|str): Rounds all of the independent and dependent parameter values using the specified rounder or rounding spec after retrieval from the HEC-DSS file.
        • 

      • 

    • TimeSeries Identifiers:
      

        

      • end_time (Any): Specifies the end of the time window to retrieve data. Must be an HecTime object or a valid input to the HecTime constructor.
Defaults to the end of the data store's time window. If None or not specified and the data store's time window doesn't have an end time, all data on or after the start time will be retrieved.
        • 

      • start_time (Any): Specifies the start of the time window to retrieve data. Must be an HecTime object or a valid input to the HecTime constructor.
Defaults to the start of the data store's time window. If None or not specified and the data store's time window doesn't have a start time, all data up to and on the end time will be retrieved.
        • 

      • trim (bool): Specifies whether to trim missing values from the beginning and end of any regular time series data set retrieved.
Defaults to the data store's trim setting.
        • 

      • 

    • 




Returns (Any): The retrieved LocalRatingSet, PairedData or TimeSeries object.





                            

                            

                                

                    
@staticmethod


        def
        set_message_level(level: int = 4) -> None:

        
    

    
    
            
Sets the HEC-DSS message level for all DssDataStore objects



Arguments:



    

  • level (int, optional):  Defaults to 4

      

    • 0: No output
      • 

    • 1: Critcal output only
      • 

    • 2: Terse (includes file open and close)
      • 

    • 4: General (includes read and write)
      • 

    • 5: User Diagnostic
      • 

    • 11: Internal Diagnostic
      • 

    • 13: Internal Debug
      • 

    • 






                            

                            

                                

            
        def
        store(self, obj: object, **kwargs: Any) -> None:

        
    

    
    
            
Stores a data set to the data store.



Currently only time series and rating sets may be stored.



Arguments:



    

  • obj (object):  The data set to store
    • 






                            

                            

                                
Inherited Members

                                

                                    
AbstractDataStore

                                
end_time

                
name

                
office

                
is_read_only

                
start_time

                
time_window

                
time_zone

                
trim

                
units

                
vertical_datum


            

                                

                            

                

                

                    

            
    class
    AbstractDataStore(abc.ABC):

        
    

    
    
            
Abstract base class for data store classes





                            

                                

                    
@abstractmethod


        def
        catalog(self, data_type: Optional[str] = None, **kwargs: Any) -> List[str]:

        
    

    
    
    

                            

                            

                                

                    
@abstractmethod


        def
        close(self) -> None:

        
    

    
    
            
Explicitly closes the data store, preventing further access.



The data store is implicitly closed upon deletion or exiting a context manager ('with' block)





                            

                            

                                

                    
@abstractmethod


        def
        delete(self, identifier: str, **kwargs: Any) -> None:

        
    

    
    
    

                            

                            

                                

            end_time: Optional[hec.HecTime]

        
    

    
    
            
The end time of the data store's time window (if any)



Operations:




  
Read/Write







                            

                            

                                

                    
@abstractmethod


        def
        get_extents(self, identifier: str, **kwargs: Any) -> List[hec.HecTime]:

        
    

    
    
    

                            

                            

                                

            native_data_store: Any

        
    

    
    
    

                            

                            

                                

            name: str

        
    

    
    
            
The name of the data store as provided in the constructor or open() method



Operations:




  
Read Only







                            

                            

                                

            office: Optional[str]

        
    

    
    
            
The office associated with the data store, if any



Operations:




  
Read/Write







                            

                            

                                

                    
@staticmethod

        
@abstractmethod


        def
        open(	name: Optional[str] = None,	**kwargs: Any) -> AbstractDataStore:

        
    

    
    
    

                            

                            

                                

            is_open: bool

        
    

    
    
            
Whether this data store is open for cataloging and reading.



A data store is open from its construction or call to open() until a call to close() or deletiion



Operations:




  
Read Only







                            

                            

                                

            is_read_only: bool

        
    

    
    
            
Whether this data store is open for storing and deleting. Meaningless if is_open is False



Operations:




  
Read/Write







                            

                            

                                

                    
@abstractmethod


        def
        retrieve(self, identifier: str, **kwargs: Any) -> Any:

        
    

    
    
    

                            

                            

                                

            start_time: Optional[hec.HecTime]

        
    

    
    
            
The start time of the data store's time window (if any)



Operations:




  
Read/Write







                            

                            

                                

                    
@abstractmethod


        def
        store(self, obj: object, **kwargs: Any) -> None:

        
    

    
    
    

                            

                            

                                

            time_window: Tuple[Optional[hec.HecTime], Optional[hec.HecTime]]

        
    

    
    
            
The data store's time window (if any)



    

  • Getting: same as (ds.start_time, ds.end_time)
    • 

  • Setting: same as ds.start_time, ds.end_time = tw
    • 




Operations:




  
Read/Write







                            

                            

                                

            time_zone: str

        
    

    
    
            
The time zone associated with the data store



When setting the time zone, any time window values are labeled with the specified time zone and not converted to the time zone



Operations:




  
Read/Write







                            

                            

                                

            trim: bool

        
    

    
    
            
Whether the datastore will trim missing values from the edges of retrieved regular time series



Operations:




  
Read/Write







                            

                            

                                

            units: str

        
    

    
    
            
The unit system ('EN' or 'SI') associated with the data store



Operations:




  
Read/Write







                            

                            

                                

            vertical_datum: Optional[str]

        
    

    
    
            
The vertical datum ('NGVD29', 'NAVD88', 'OTHER') associated with the data store



Returns:




  
Optional[str]: _description_