IDigBio API: Difference between revisions

From iDigBio
Jump to navigation Jump to search
(Moved and changed quick start.)
No edit summary
Line 9: Line 9:
[https://www.idigbio.org/ Integrated Digitized Biocollections (iDigBio)] is the National Resource for Advancing Digitization of Biodiversity Collections (ADBC) funded by the National Science Foundation. Through ADBC, data and images for millions of biological specimens are being made available in electronic format for the research community, government agencies, students, educators, and the general public. iDigBio is a data aggregator. This means that data is provided to iDigBio through various publishing mechanisms.
[https://www.idigbio.org/ Integrated Digitized Biocollections (iDigBio)] is the National Resource for Advancing Digitization of Biodiversity Collections (ADBC) funded by the National Science Foundation. Through ADBC, data and images for millions of biological specimens are being made available in electronic format for the research community, government agencies, students, educators, and the general public. iDigBio is a data aggregator. This means that data is provided to iDigBio through various publishing mechanisms.


Many consumers of the iDigBio aggregated data will choose to use the [https://www.idigbio.org/portal iDigBio Portal web site]. Although our portal has many features, to facilitate integration of iDigBio data with other web sites, services, or research uses, iDigBio provides APIs in order to provide direct access to our data. We ourselves make use of these APIs in our portal and other data services so any functionality you see there, as well as functions not available through the portal, can be done through APIs yourself.
Many consumers of the iDigBio aggregated data will choose to use the [https://www.idigbio.org/portal iDigBio Portal web site]. Although our portal has many features and is the easiest interface to use, to facilitate integration of iDigBio data with other web sites, services, or research uses, iDigBio provides APIs in order to provide direct access to our data. We ourselves make use of these APIs in our portal and other data services so any functionality you see there, as well as functions not available through the portal, can be done through APIs yourself.


The iDigBio APIs are an abstraction layer for retrieving data from the iDigBio back-end data systems. This abstraction allows reuse and mashup of aggregated data without needing to understand the complex underlying details of the back-end data storage. Currently, the public APIs support HTTP GET and POST requests for data read operations only. The iDigBio APIs are RESTful web services that delivers data primarily as [https://www.json.org/ JSON] documents.
The iDigBio APIs are an abstraction layer for retrieving data from the iDigBio back-end data systems. This abstraction allows reuse and mashup of aggregated data without needing to understand the complex underlying details of the back-end data storage. Currently, the public APIs support HTTP GET and POST requests for data read operations only. The iDigBio APIs are RESTful web services that delivers data primarily as [https://www.json.org/ JSON] documents.
Line 27: Line 27:
You can join via the web at: http://lists.ufl.edu/cgi-bin/wa?A0=IDIGBIO-API-USERS-L
You can join via the web at: http://lists.ufl.edu/cgi-bin/wa?A0=IDIGBIO-API-USERS-L


== Specification ==
== Searching iDigBio ==
 
The [https://github.com/idigbio/idigbio-search-api/wiki iDigBio API v2 / iDigBio Search API specification] includes detailed information about the Search API endpoints, parameters, query format, values, and returns. It is available in the github wiki associated with the [https://github.com/iDigBio/idigbio-search-api code repository]. The iDigBio Search API was released to production in June 2015.


iDigBio transitioned from version 1 to version 2 (v1 to v2) of the API. All v1 endpoints and documentation should be considered deprecated, even if they continue to function. Programmers who are interested in accessing data directly in iDigBio via an API should consult the v2 API documentation. '''The v2 API is also known as the iDigBio Search API.'''
There are several APIs you can use to retrieve data from iDigBio:


The now-deprecated [[iDigBio API v1 Specification]] includes detailed information about the old v1 API endpoints, parameters, values, and returns. In most cases, the only reason to use the v1 API was to retrieve raw, unprocessed versions of a record (e.g. no iDigBio data quality enhancements) or to access previous versions of a particular record.
# Search API
# Download API
# Original v1 API


== Examples ==
=== Search API ===
 
See [https://github.com/iDigBio/idigbio-search-api/wiki/Examples iDigBio API v2 / iDigBio Search API Examples] in the [https://github.com/iDigBio/idigbio-search-api/wiki idigbio-search-api github wiki].


The version 2 Search API was released in mid 2015 and is the current API that the portal and other iDigBio services are based on. Full documentation for this API is available in the [https://github.com/idigbio/idigbio-search-api/wiki idbsearch-api Github wiki] for the source code. The search API allows you to retrieve limited sets of data in response to custom queries and is typically the one people use.


== Searching iDigBio ==
=== Download API ===


=== Search Portal and Bulk Record Downloads ===
The Download API can be used to generated [https://en.wikipedia.org/wiki/Darwin_Core_Archive Darwin Core archives] containing any amount of data up to everything that is in iDigBio. This API is documented on the [[iDigBio Download API]] wiki page. This API is useful when the size of the data you want exceeds what you can get through the search API. Be aware that this is not an interactive API; requests will result in a queued job being started to make the Darwin Core archive and you will be informed when it is ready to download later.


The easiest method for searching iDigBio is to use the Portal website search features, not the API. The portal also provides bulk download capabilities for acquiring larger sets of data.  See: https://www.idigbio.org/portal
=== Other APIs ===
 
=== Search API ===
 
The [https://github.com/idigbio/idigbio-search-api/wiki iDigBio Search API] has be released and the June 2015 version of the portal is based on this API.


=== Other APIs ===
[[iDigBio Download API]]


[https://docs.google.com/document/d/1C75oUWk7Wbbo81-ldh-ogOurNjbIQ-ZLPqSQVR-hc5A/edit?usp=sharing Media API Working Docs]
[https://docs.google.com/document/d/1C75oUWk7Wbbo81-ldh-ogOurNjbIQ-ZLPqSQVR-hc5A/edit?usp=sharing Media API Working Docs]
Line 62: Line 55:


Instead of generating large queries in the portal or with the API, it may be easier to download iDigBio data from regular dumps of all data in iDigBio. These dumps are available from [http://s.idigbio.org/idigbio-static-downloads?max-keys=10000 s.idigbio.org]. This is an S3-compatible object store (although it can be treated as a web server if that is easier) that contains .zip files of each record set in iDigBio named by recordset uuid as well as files containing all data across all records sets.
Instead of generating large queries in the portal or with the API, it may be easier to download iDigBio data from regular dumps of all data in iDigBio. These dumps are available from [http://s.idigbio.org/idigbio-static-downloads?max-keys=10000 s.idigbio.org]. This is an S3-compatible object store (although it can be treated as a web server if that is easier) that contains .zip files of each record set in iDigBio named by recordset uuid as well as files containing all data across all records sets.
== Specification ==
The [https://github.com/idigbio/idigbio-search-api/wiki iDigBio API v2 / iDigBio Search API specification] includes detailed information about the Search API endpoints, parameters, query format, values, and returns. It is available in the github wiki associated with the [https://github.com/iDigBio/idigbio-search-api code repository]. The iDigBio Search API was released to production in June 2015.
iDigBio transitioned from version 1 to version 2 (v1 to v2) of the API. All v1 endpoints and documentation should be considered deprecated, even if they continue to function. Programmers who are interested in accessing data directly in iDigBio via an API should consult the v2 API documentation. '''The v2 API is also known as the iDigBio Search API.'''
The now-deprecated [[iDigBio API v1 Specification]] includes detailed information about the old v1 API endpoints, parameters, values, and returns. In most cases, the only reason to use the v1 API was to retrieve raw, unprocessed versions of a record (e.g. no iDigBio data quality enhancements) or to access previous versions of a particular record.
== Examples ==
See [https://github.com/iDigBio/idigbio-search-api/wiki/Examples iDigBio API v2 / iDigBio Search API Examples] in the [https://github.com/iDigBio/idigbio-search-api/wiki idigbio-search-api github wiki].


== Client Libraries ==
== Client Libraries ==

Revision as of 17:05, 9 July 2018


iDigBio API Overview

This document serves as the starting page of official documentation for the iDigBio Application Programming Interfaces (APIs).

Integrated Digitized Biocollections (iDigBio) is the National Resource for Advancing Digitization of Biodiversity Collections (ADBC) funded by the National Science Foundation. Through ADBC, data and images for millions of biological specimens are being made available in electronic format for the research community, government agencies, students, educators, and the general public. iDigBio is a data aggregator. This means that data is provided to iDigBio through various publishing mechanisms.

Many consumers of the iDigBio aggregated data will choose to use the iDigBio Portal web site. Although our portal has many features and is the easiest interface to use, to facilitate integration of iDigBio data with other web sites, services, or research uses, iDigBio provides APIs in order to provide direct access to our data. We ourselves make use of these APIs in our portal and other data services so any functionality you see there, as well as functions not available through the portal, can be done through APIs yourself.

The iDigBio APIs are an abstraction layer for retrieving data from the iDigBio back-end data systems. This abstraction allows reuse and mashup of aggregated data without needing to understand the complex underlying details of the back-end data storage. Currently, the public APIs support HTTP GET and POST requests for data read operations only. The iDigBio APIs are RESTful web services that delivers data primarily as JSON documents.

Quick Start Example

The search API is our most useful for most people. Below is a simple example of searching for all records in a given genus to provide you with an example of how this API looks. You can simply click the link in your browser to make the API call and get results:

https://search.idigbio.org/v2/search/records?rq={"genus":"acer"}

(You might want to install a JSON viewing plugin in your browser such as JSONView for Chrome.)

iDigBio API Mailing List

Please join our list at idigbio-api-users-l@lists.ufl.edu if you are using our APIs. Besides providing notification of API changes, the mailing list gives direct access to the iDigBio API developers and is a good place to ask questions or give and receive feedback.

You can join via the web at: http://lists.ufl.edu/cgi-bin/wa?A0=IDIGBIO-API-USERS-L

Searching iDigBio

There are several APIs you can use to retrieve data from iDigBio:

  1. Search API
  2. Download API
  3. Original v1 API

Search API

The version 2 Search API was released in mid 2015 and is the current API that the portal and other iDigBio services are based on. Full documentation for this API is available in the idbsearch-api Github wiki for the source code. The search API allows you to retrieve limited sets of data in response to custom queries and is typically the one people use.

Download API

The Download API can be used to generated Darwin Core archives containing any amount of data up to everything that is in iDigBio. This API is documented on the iDigBio Download API wiki page. This API is useful when the size of the data you want exceeds what you can get through the search API. Be aware that this is not an interactive API; requests will result in a queued job being started to make the Darwin Core archive and you will be informed when it is ready to download later.

Other APIs

Media API Working Docs

Specimen Web Map Module

The iDigBio Portal Search page map is also provided as a stand-alone module for use in any third-party web site. Go to the Specimen Map page for details on using the map.

Full Recordset Downloads

Instead of generating large queries in the portal or with the API, it may be easier to download iDigBio data from regular dumps of all data in iDigBio. These dumps are available from s.idigbio.org. This is an S3-compatible object store (although it can be treated as a web server if that is easier) that contains .zip files of each record set in iDigBio named by recordset uuid as well as files containing all data across all records sets.


Specification

The iDigBio API v2 / iDigBio Search API specification includes detailed information about the Search API endpoints, parameters, query format, values, and returns. It is available in the github wiki associated with the code repository. The iDigBio Search API was released to production in June 2015.

iDigBio transitioned from version 1 to version 2 (v1 to v2) of the API. All v1 endpoints and documentation should be considered deprecated, even if they continue to function. Programmers who are interested in accessing data directly in iDigBio via an API should consult the v2 API documentation. The v2 API is also known as the iDigBio Search API.

The now-deprecated iDigBio API v1 Specification includes detailed information about the old v1 API endpoints, parameters, values, and returns. In most cases, the only reason to use the v1 API was to retrieve raw, unprocessed versions of a record (e.g. no iDigBio data quality enhancements) or to access previous versions of a particular record.

Examples

See iDigBio API v2 / iDigBio Search API Examples in the idigbio-search-api github wiki.


Client Libraries

Client libraries, packages, and modules are pieces of software that make it easier to interface with the iDigBio API. The following list includes the known iDigBio libraries that are currently available.

ridigbio R Package for Search API

R is a free software environment for statistical computing and graphics. It compiles and runs on a wide variety of UNIX platforms, Windows and MacOS.

The production version of this package is in CRAN. The latest development version is available at https://github.com/idigbio/ridigbio

The ridigbio R package is an "official" client library in the iDigBio code repository.

iDigBio Python Library for Search API

The Python client library is available via PyPI:

https://pypi.python.org/pypi/idigbio

or just use pip:

pip install idigbio

The client code can also be found at github if you wish to help develop the library:

https://github.com/idigbio/idigbio-python-client/

The iDigBio Python library is an "official" client library in the iDigBio code repository.

idigbio_client Ruby Gem

The idigbio_client ruby gem is available:

https://rubygems.org/gems/idigbio_client

which can be installed via:

gem install idigbio_client

The idigbio_client ruby gem is a "community-supported" client library with software repository currently at:

https://github.com/GlobalNamesArchitecture/idigbio_client

Suggestions?

If you are in need of other client libraries or iDigBio API features, please use the feedback button to submit your request.