This repository has been archived by the owner on Oct 11, 2024. It is now read-only.
Releases: 52North/sos-importer
Releases · 52North/sos-importer
0.4.2
0.4.1
0.4.0
Features
- Rename Core module to Wizard
- Support for SOS 2.0 incl. Binding definition
- Start Screen offers button to see all dependency licenses
- Support for sensors with multiple outputs
- Introduced import strategies:
- SingleObservation:
Default strategy
- SweArrayObservationWithSplitExtension:
Readshunksize
# lines and imports each time series using an SWEArrayObservation in combination with the SplitExtension of the 52North SOS implementation. Hence, this strategy works only in combination with 52North implementation. Other impl. might work, too, but not as expected. Hunksize and import strategy are both optional<AdditionalMetadata><Metadata>
elements.
- SingleObservation:
- Support for date information extraction from file name using two new OPTIONAL attributes in element
<DataFile>
:- "regExDateInfoInFileName" for extracting date information from file names.
- "dateInfoPattern" for parsing the date information into a
java.util.Date
.
- Date information extraction from last modified date using two new OPTIONAL attributes:
- "useDateFromLastModifiedDate" for enabling this feature
- "lastModifiedDelta" for moving the date n days back (this attribute is OPTIONAL for this feature, too.)
- Ignore lines with regular expressions feature: 0..infinity elements can be added to the element. Each element will be used as regular expression and applied to each line of the data file before parsing.
- Handling of data files containing several sample runs. A sample run contains additional metadata like its size (number of performed measurements) and a date. The required attributes are:
- "sampleStartRegEx" - the start of a new sample (MUST match the whole line).
- "sampleDateOffset" - the offset of the line containing the date of the sample from the start line.
- "sampleDateExtractionRegEx" - the regular expression to extract the date information from the line containing the date information of the current sample. The expression MUST result in ONE group. This group will be parsed to a
java.util.Date
using "sampleDatePattern" attribute. - "sampleDatePattern" - the pattern used to parse the date information of the current pattern.
- "sampleDataOffset" - the offset in lines from sample beginning till the first lines with data.
- "sampleSizeOffset" - the offset in lines from sample beginning till the line containing the sample size in lines with data.
- "sampleSizeRegEx" - the regular expression to extract the sample size. The regular expression MUST result in ONE group which contains an integer value.
- Setting of timeout buffer for the insertion of SweArrayObservations:
With the attribute "insertSweArrayObservationTimeoutBuffer" of<SosMetadata>
it is possible to define an additional timeout buffer for connect and socket timeout when using import strategy "SweArrayObservationWithSplitExtension". Scale is in milliseconds, e.g.1000
→ 1s more connect and socket timeout. The size of this value is related to the set-up of the SOS server, importer, and theHUNK_SIZE
value.
The current OX-F SimpleHttpClient implementation uses a default value of 5s, hence setting this to25,000
results in 30s connection and socket timeout. - More details can be found in the release notes.
Fixed Bugs/Issues
- #06: Hardcoded time zone in test
- #10: NPE during feeding if binding value is not set
- #11: !BadLocationException in the case of having empty lines in csv file
- #20: Current GUI is broken when using sample based files with minor inconsistencies
- #24: Fix/ignore line and column: Solved two NPEs while ignoring lines or columns
- #25: Fix/timezone-bug-parse-timestamps: Solved bug while parsing time stamps
- #NN: Fix bug with timestamps of sample files
- #NN: Fix bug with incrementing lastline causing data loss
- #NN: Fix bug with data files without headerline
- #NN: NSAMParser: Fix bug with timestamp extraction
- #NN: NSAMParser: Fix bug with skipLimit
- #NN: NSAMParser: Fix bug with empty lines, line ending, time series encoding
- #NN: fix/combinationpanel: On step 3 it was not possible to enter parse patterns for position and date & time
- #NN: fix problem with textfield for CSV file when switching to German
- #NN: fix problem with multiple sensors in CSV file and register sensor
- 878
- "Too many columns issue"
- Fixed issues