Guidance When First Considering iDigBio Data Ingestion

Contact Info

For assistance, contact data@idigbio.org

Below are what we ask of the data to make it fit for use in the cyberinfrastructure we are building:

Processing steps, each step has a start and an end, signifying that it has moved to the next step.

negotiating - in the process of evaluating provider's interest in ingestion
- begin with email invitation to providers (in institutions, aggregators) to invite them to send their data to iDigbio specimen data portal
- open a redmine ticket in category=Data Mobilizing
- ends with data exported by provider, ready for inspection and ingestion
mobilizing - in the process of evaluating data being fit for ingestion
- begins with exported data and cursory inspection
- ends with data passing inspection and passing to ingestion state, redmine ticket change to category=Data
ingesting - in the process of ingesting provider's data
- begins with redmine ticket change to category=Data
- ends with
  - data successfully being ingested
  - report sent back to data mobilizing staff
  - redmine ticket set to Status= Closed
evaluating - in the process of evaluating a failure to be ingested
- begins with ingestion failure
  - evaluate ingestion failure, send back to mobilizing state for corrections or
  - evaluate ingestion failure, make corrections
- ends with re-submission to ingesting step

3 kinds of data are submitted for ingestion:

format:
- IPT
- DwC-A
- CSV (save teh data in UTF-8 format to preserve diacritics)
- Symbiota feeds,
plus an RSS feed for ready access and update is recommended, otherwise email the files to us
metadata:
- each specimen record needs to have a unique (within the dataset) identifier in the occurrenceID field.
- name the fields as close to Darwin Core as possible, and additionally use the MISC field names (local iDigbio extensions to DarwinCore)
you need to have permission to submit the data
data recommendations for optimal searchability:
- put dates in ISO 8601 format, i.e., YYYY-MM-DD
- put elevation in METERS units in the elevation field without the units
- no unescaped newline characters
- parse out genus, species, infraspecific epithet if already aggregated into a scientific name
- take caution to preserve diacritics in people and place names when saving or exporting for ingestion (save the data in UTF8 format).

each media record needs to have a GUID: a persistent globally unique identifier or at least a unique (within the dataset) identifier in the occurrenceID field.
1. if submitting media records with specimen data records, put the specimen data record into the ??? field
we need there to be Audubon Core metadata file, with one record to go with each media record, and we can provide coaching to help you create that file. The more you can flesh out the details of the image, the more likely it will be to be highly retrievable.
just like the ownership of catalog records, the media records need to provided freely and with permission, and each record needs to have at least Creative Commons permission = "CC BY"

The methods for linking the catalog records to the media records are in this document, as well as explanation about creating GUIDs for the records:

Details about data ingestion requirements and guidelines are here:

Additional info about image format is here:

If you need to learn about acceptable Creative Commons licenses in iDigBio: