Dataset Errata: Difference between revisions

=== Gold Parsing Errors ===

=== Lichen Gold ===

 

==== elevation verbatimElevation ====

 

This is open to debate, but I think Elevation should be a pure numeric field, assumed to be in meters. Therefore, it should not be expressed as "750 m", but rather as "750". verbatimElevation, of course, should retain the "m" if it was present on the label. (Note that Darwin Core apparently does not have a field called "elevation", but rather MinimumElevationInMeters, and MaximumElevationInMeters, both numeric fields.) Not sure if this is something to change on the labels, but worth being aware of. I think parsing programs should generate the Darwin Core fields. ('''Daryl''') <br> ('''Bryan:''' Odd to not have "elevation" I agree with the use of verbatimElevation. If "elevation" is filled it is numeric.) <br> ('''Deb''': What are the ramifications then? Does the lichen set need to be fixed in this regard? or just ignore derived columns and expect letters like m or mi or ft in verbatimElevation field?)--[[User:Dpaul|Dpaul]] 16:40, 12 June 2013 (EDT)  

 

 

 

-- Gold Parsed TENN-L-0000001_lg.csv lists country as "USA", but on the .txt label, it is "U.S.A." (with periods). Same with Gold Parsed TENN-L-0000035_lg.csv and others.'''(Daryl''') &nbsp; <br> ('''Bryan''': Agreed. Should be fixed to match the label.) <br> ('''Ed''': fixed, note that TENN-L-0000035_lg.txt has "U. S. A. " with spaces, thus conserved format)

-- Gold Parsed TENN-L-0000005_lg.csv leaves country blank, but the label shows it as "USA". Again, maybe this is OK, but it should be consistent. ('''Daryl)''' <br> ('''Bryan''': Agreed. Should be fixed to match the OCR label.) <br> ('''Ed''': Fixed, country had county value) <br> <br> <br> Inconsistency and errors in TENN Lichen Gold Parsed dateIdentified. Examples:

-- TENN-L-0000015_lg.csv has dateIdentified in the wrong format, neither verbatim, nor standard DarwinCore format: Verbatim would be: Nov. 12, 1939, DarwinCore would be: 1939-11-12, Listed is: 1939-November-12. <br> '''(Bryan''': I think excel may have imposed it's own format and changed the records. If the column type is set to "text" Excel will not transform to a new format. ) <br> ('''Ed''': The verbatimEventDate (not dateIdentified is in the correct format, but if you open file in excel it will convert the display to match program's defaults)).  

-- TENN-L-0000017_lg.csv omits dateIdentified, though it is on the label as 3 Feb. 1963&nbsp; <br> ('''Bryan''': Agreed. Should be fixed to match the label.) <br> ('''Ed''': That is the collection date, not the dateIdentified. Format is correct in verbatimDate)  

-- TENN-L-0000019_lg.csv has 1954-Aug-8, but on the label it is "8 Aug 1954", again neither verbatim nor DarwinCore (1954-08-08). ('''Daryl)'''&nbsp; <br> ('''Bryan''': Agreed. Should be fixed to match the label.)  

==== comma, space, period, apostrophe in verbatimCoordinates errors ====

Gold Parsed NY01075760_lg.csv replaces the comma with a space, and replaces an apostrophe (') with a double quote (") in verbatimCoordinates: 38°42'20"N, 83°08'25'W is rendered as 38°42'20""N 83°08'25""W. (Note also that the double quote is replaced with two double quotes. This may be necessary to preserve the quote-delimited, comma separated fields, but could cause some problems when uploading to a database. Not presented here as an error, but we should be aware of possible implications.)&nbsp; <br> ('''Bryan''': Agreed. Should be fixed to match the Label except, the double quoted double quote I think is needed for CSV readers to identify fileds. I am not sure)  

Gold Parsed NY01075764_lg.csv has a similar problem where a single space is replaced with a double space in verbatimCoordinates.&nbsp; <br> ('''Bryan:''' Agreed. Should be fixed to match the label as best as possible. If it is not clear follow the OCR file.)  

'''Gold Parsed CSV Files''' There are more errors in gold csv files. (Qianjin) '''(Bryan: I agree with Qianjin's edits except as noted below)'''  

NY01075768_lg country (canada), it should be (ca.)&nbsp;  

NY01075779_lg habitat concatenation (Bryan: "on Protoblastenia rupestris" appears before the location and habitate section of the label. However, Habitate says "dolomite rock along lake shore and adjacent Thuja forest; on Protoblastenia rupestris". There was a period after "forest". The period was removed and a ";" added. Then the "on Protoblastenia rupestris" from the earlier part of the label was concatinated.&nbsp;  

NY01075780_lg NEW YOUR BOTANICAL GARDEN (Bryan: the label said "GARDEN". The OCR said "CARDEN". SIlver should be "CARDEN" Gold should be "GARDEN"  

NY01075823_lg identifiedBy (Bryan:&nbsp;?? I do not see the problem)  

TENN-L-0000012_lg verbatimLocality (apria -s ) in the text file; but it is (apricas) in the csv file. (Bryan: on the label the word is hyphenated and split across lines. SO, if we ignore the "New Line" the word is "apria -cas" and not what Qianjin listed. so, folloing the rule for gold of making perfect OCR, character by character, the csv shoudl say&nbsp;"apria -cas")  

<br> '''Silver Parsed CSV Files''' '''(Bryan: I do not get most of these. There should be OCR errors in silver. We do need to stay true to the OCR output.)&nbsp;'''  

NY01075762_lg misspelling in habitat misspelling in verbatimLocality (Bryan: I think the problem is that habitate is concatinated with the substraight "Abies [7a[5amzfem—Betu[a papyrzfem forest over granite adjacent to waterfall, parasite on Peltigera scabrosa". the parasite part is from another part of the label. It shoudl be on a new row.)