Apache Spark

iDigBio and Data Carpentry go to Africa

Location: BIS (TDWG) 2015 Biodiversity Information Standards:
An amazing 2 weeks in Nairobi, Kenya.
by Deb Paul, input from Libby Ellwood and Matt Collins.

Exploring unique values in iDigBio using Apache Spark

Data exploration for large datasets is always challenging. Often you are left with deciding between subsetting the dataset (randomly or on some facet), making slow progress waiting for results just to find that something needs to be fixed, or optimizing code for performance when you don't even know if the result is going to be interesting. Having a high-performance system capable of ad-hoc investigation has always been difficult and/or expensive.

Subscribe to RSS - Apache Spark
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
1 + 12 =
Solve this simple math problem and enter the result. E.g. for 1+3, enter 4.