2013 AOCR Hackathon Wiki: Difference between revisions

From iDigBio
Jump to navigation Jump to search
No edit summary
 
(98 intermediate revisions by 6 users not shown)
Line 1: Line 1:
{| class="wikitable" style="float:right;"
! colspan="2" style="background:#D58B28;width:200px;font-size:10pt" | Digitizing the Past and Present for the Future
|-
| colspan="2" style="text-align:center;font-size:7pt" | [[Image:IDigBio Logo RGB.png|center|300px|iDigBio Logo RGB.png]]<br />
|-
!colspan="2" style="background:#D58B28;text-align:center;font-size:9pt" | Quick Links for iDigBio AOCR Hackathon Workshop
|-
|[https://docs.google.com/document/d/1dNbEOdL1stn-ztKDWTzfL1T4VPzg20ZZ76zV5ueFrnY/edit# iDigBio AOCR Hackathon Workshop Agenda]
|-
|[https://www.idigbio.org/biblio?f%5bkeyword%5d=458 iDigBio AOCR Hackathon Workshop Biblio Entries]
|-
|[https://www.idigbio.org/content/update-idigbio-augmenting-ocr-working-group iDigBio AOCR Hackathon Workshop Report]
|}
[[Category:Workshop]]
= Welcome to the 2013 iDigBio AOCR Hackathon Wiki =
= Welcome to the 2013 iDigBio AOCR Hackathon Wiki =
*Short URL to this hackathon wiki http://tinyurl.com/aocrhackathonwiki
*Short URL to this hackathon wiki http://tinyurl.com/aocrhackathonwiki
*Those participating in the first iDigBio AOCR Hackathon need an iDigBio account.
*Those participating in the first iDigBio AOCR Hackathon need an iDigBio account.
*Note: This wiki page undergoing frequent updates and some participants have wiki edit permissions and will add to / update / edit these pages before, during and after the hackathon.
*Note: This wiki page undergoing frequent updates and some participants have wiki edit permissions and will add to / update / edit these pages before, during and after the hackathon.
*[[Augmenting_OCR| AOCR Working Group Wiki]]
*[[IDigBio_Augmenting_OCR_Workshop#Workshop_Presentations| AOCR October 2012 Working Group Meeting Presentations]]
*[[iConference 2013 iDigBio AOCR WG Wiki]]
=== [https://www.dropbox.com/s/p8zjm0ajaj838tw/HackathonIconfAgendaFinal.docx Day-By-Day Schedule] ===
*download to have active hyperlinks
* [https://docs.google.com/document/d/1dNbEOdL1stn-ztKDWTzfL1T4VPzg20ZZ76zV5ueFrnY/edit# Day-By-Day Online in GoogleDoc Format]
**NOTE: editable by participants.
=== Group Notes ===
*Please use [http://tinyurl.com/AOCRhackNotes AOCRhackNotes] to Capture your ideas, insights collaboratively at our hackathon.
**We are excited to use this process and have found it extremely useful for collaborating effectively!
== Meetings ==
*11 Jan 2013 via AdobeConnect, 2 - 3 PM EST
**Notes at Google Doc: http://tinyurl.com/aocrhackmeet1
*17 Jan 2013 via AdobeConnect, 2 - 3 PM EST
**Notes at Google Doc: http://tinyurl.com/aocrhackmeet1
*25 Jan 2013 via AdobeConnect, 2 - 3 PM EST
**Notes at Google Doc: http://tinyurl.com/aocrhackmeet1
*1 Feb 2013 via AdobeConnect, 2 - 3 PM EST
**Notes at Google Doc: http://tinyurl.com/aocrhackmeet1
*8 Feb 2013 via AdobeConnect, 3 - 4 PM EST
**Notes at Google Doc: http://tinyurl.com/aocrhackmeet1
== [[Presentations & Reports]] ==
*[[Presentations & Reports#Deborah_Paul|Deborah Paul: 1st iDigBio - BRIT Hackathon]]
*Alex Thompson
*Daryl Lafferty
*Ben Brumfield
*Phuc Nguyen
*[[Presentations & Reports#Robert_Anglin|Robert Anglin: Regular Expression-Based Parsing of Tesseract Output from Lichen Herbarium Labels]]
*Bryan Heidorn & Qianjin Zhang
*Dmitry Mozzherin
*[[Presentations & Reports#Paul_Schroder|Paul & Robin Schroeder: iDigBio Hackathon: iDigBio OCR Hackathon Initial Results - Rest API]]
*John Pickering
*Jason Best
==Photos==
*[https://www.facebook.com/media/set/?set=a.513738458670105.110787.215120891865198&type=3 Hackathon Facebook Photo Album]


== Links to Logistics, Communication, and Participant Information ==
== Links to Logistics, Communication, and Participant Information ==
*[[2013 Hackathon Participants| Participant List]]
*[[2013 Hackathon Participants| Participant List]]
*See [http://tinyurl.com/aocrHackerPhotos Participant Photos!] and small bio bits.
**Editable by you!
*[[Participant Related Projects]]
**[[Participant Related Projects#EoL | EoL]]
**[[Participant Related Projects#GNA | GNA]]
**[[Participant Related Projects#ICSI | ICSI]]
**[[Participant Related Projects#MaCC | MaCC]]
**[[Participant Related Projects#Tolkin | Tolkin]]
**[[Participant Related Projects#SALIX | SALIX]]
**[[Participant Related Projects#LABELX | LABELX]]
**[[Participant Related Projects#Symbiota | Symbiota]]
**[[Participant Related Projects#ScioQualis | ScioQualis]]
**[[Participant Related Projects#BiSciCol | BiSciCol]]
**[[Participant Related Projects#LBCC_TCN | LBCC TCN]]
**[[Participant Related Projects#Discover Life | Discover Life]]
**[[Participant Related Projects#From The Page | From The Page]]
**[[Participant Related Projects#DarwinCore Parser | DarwinCore Parser]]
**[[Participant Related Projects#Apiary Project | The Apiary Project]]
**[[Participant Related Projects#USCD Computer Vision | USCD Computer Vision]]
**[[Participant Related Projects#Biodiversity Heritage Library | Biodiversity Heritage Library]]
*[http://tinyurl.com/aocrHack  Call for Participation]
*[http://tinyurl.com/aocrHack  Call for Participation]
*[http://tinyurl.com/idigbioAOCRHackathon  Application Form]
*[http://tinyurl.com/idigbioAOCRHackathon  Application Form]
*[[2013 Hackathon Logistics| Travel, Food, Lodging, Connectivity Logistics]]
*[[2013 Hackathon Logistics| Travel, Food, Lodging, Connectivity Logistics]]
*2013 Hackathon Listserv, a mailing list for Hackathon Participants at aocr-hackathon-l@lists.ufl.edu
*2013 Hackathon Listserv, a mailing list for Hackathon Participants at aocr-hackathon-l@lists.ufl.edu
=== Remote Participation ===
*[http://idigbio.adobeconnect.com/augmentocr Remote Participation via AdobeConnect]
**Join us on Wednesday morning 9 AM - Noon CST to hear our hackathon participants report back to the group and share their progress on parsing algorithms.
**Please sign in 15 - 20 minutes early to learn how Adobe Connect works.
**[[Web Conferencing#Audio_Troubleshooting|Help with AdobeConnect]]
**Want to try your skills at parsing? Share your User Interface ideas? Got mad skills for developing image-analysis and image segmentation algorithms? Know where the great authority files (data dictionaries) are? Do tell, via remote participation or in our online group notes doc.


== Overview of the Challenge ==
== Overview of the Challenge ==
*2013 iDigBio AOCR [[Hackathon Challenge]]
*2013 iDigBio AOCR [[Hackathon Challenge]]
**overall description of [[Hackathon Challenge#the_challenge| The Challenge]]
**overall description of [[Hackathon Challenge#the_challenge| The Challenge]]
**[[Hackathon Challenge#the_specific_task| The Specific Task]]: parse OCR output to find values for these [[Hackathon Challenge#core_data_elements| core data elements]]
**[[Hackathon Challenge#the_specific_task| The Specific Task]]: parse OCR output to find values for these [[2013 hackathon data elements]]
**[[Hackathon Challenge#metrics_and_evaluation| Metrics and Evaluation]] to be used
**[[Hackathon Challenge#metrics_and_evaluation| Metrics and Evaluation]] to be used
  [[2013 hackathon data elements| "core" fields]] 
**[[Hackathon Challenge#three_data_sets| Three Data Sets]]
 
***There are three data sets, that is, three different sets of images of museum specimen labels. Participants, working alone or in groups, may work on one or more data sets as they choose. The sets have been ranked, easy, medium, hard, as an estimate of how difficult it might be to successfully get good parsed data from the OCR output from each data set.
== Three Data Sets ==
**[[Hackathon Challenge#accessing_the_data| Accessing the Data]]
There are three data sets, that is, three different sets of images of museum specimen labels. Participants, working alone or in groups, may work on one or more data sets as they choose. The sets have been ranked, easy, medium, hard, as an estimate of how difficult it might be to successfully get good parsed data from the OCR output from each data set.
 
:::; Set 1 (easy) : 10,000 Lichens, Bryophyte and Climate Change TCN, lichen and bryophyte packet labels. These are considered easy because these jpg images are of the label only and data on the label is mostly typed or printed with little or no handwriting present.
:::; Set 2 (medium) : 5,000 Botanical Research Institute of Texas (BRIT) Herbarium and 5,000 New York Botanical Garden Herbarium specimen sheets. These are full sheets and again, most have been pre-selected to focus on labels containing mostly print or typed text and little handwriting. Note there are exceptions in order to make a more realistic (and more difficult) data set.
:::; Set 3 (hard): Several thousand images from the Essig Museum and the CalBug project. The gold set has not yet been created for these (in progress). Silver set creation needs to be discussed.


== Frequently Asked Questions ==
== Frequently Asked Questions ==
*[[Hackathon FAQ]]
*[[Hackathon FAQ]]


== Deliverables & Outputs ==
*report on progress beyond parsing, accomplishments, and in-the-works
*summary report on progress, including metrics evaluation
*Development of [[OCR SaaS]] for use by the entire community.
*code from participants (at github)
*all participant's talks posted to the hackathon wiki, including links to their code, comments.
*Participants talks and summary reports added to the iDigBio Biblio.
*collected conversation / feedback from whole group (report back summary)
*Social Media
**photographs
**post to Facebook
**blog post to iDigBio
*white paper on the hackathon process.
*[[Media:HackathonGeneralWorkflowGraphic.pdf|Hackathon Workflow v1 by Robin Schroeder]]
*[[REST API Documentation by Paul Schroeder]]
*[http://manuscripttranscription.blogspot.com/2013/02/improving-ocr-inputs-from-ocr-outputs.html Improving OCR Inputs from OCR Outputs?] by Ben W. Brumfield. "This is a transcript of my talk at the iDigBio Augmenting OCR Hackathon, presenting preliminary results of my efforts before the event."
*[http://manuscripttranscription.blogspot.com/2013/02/idigbio-augmenting-ocr-hackathon.html iDigBio Augmenting OCR Hackathon] blog post by participant Ben W. Brumfield (FromThePage.com).
*[http://manuscripttranscription.blogspot.com/2013_02_01_archive.html Results of the "Ocrocrop" Approach to Improving OCR] report by participant Ben W. Brumfield (FromThePage.com).


== Choosing Images and Parsing Decisions ==
*[[Image Selection and Processing Protocols]]
*[[Text Transcription Issues]]


link to page summarizing the rules we followed to transcribe the gold set (and others)
== Issues that need work ==
 
*[http://tinyurl.com/OCRHackathonWishList Our AOCR wg Wish List]
[[Text Transcription Issues]]
*[[Known OCR, ML, NLP Issues]] and challenges
 
*Human-in-the-loop: [[User Interface Wish List]]
[[Known OCR, ML, NLP Issues]] and challenges
*Development of integrated web services that return OCR output to providers. See the A[[OCR_SaaS]]
 
Human-in-the-loop: [[User Interface Wish List]]
    
    
<pre>*Thank you NESCent, Hilmar Lapp and the HIP working group for this model.</pre>
<pre>*Thank you NESCent, Hilmar Lapp and the HIP working group for this model.</pre>
----
[[Augmenting OCR|Back to the aOCR Wiki]]

Latest revision as of 16:32, 9 July 2015

Digitizing the Past and Present for the Future
iDigBio Logo RGB.png

Quick Links for iDigBio AOCR Hackathon Workshop
iDigBio AOCR Hackathon Workshop Agenda
iDigBio AOCR Hackathon Workshop Biblio Entries
iDigBio AOCR Hackathon Workshop Report

Welcome to the 2013 iDigBio AOCR Hackathon Wiki

Day-By-Day Schedule

Group Notes

  • Please use AOCRhackNotes to Capture your ideas, insights collaboratively at our hackathon.
    • We are excited to use this process and have found it extremely useful for collaborating effectively!

Meetings

Presentations & Reports

Photos

Links to Logistics, Communication, and Participant Information

Remote Participation

  • Remote Participation via AdobeConnect
    • Join us on Wednesday morning 9 AM - Noon CST to hear our hackathon participants report back to the group and share their progress on parsing algorithms.
    • Please sign in 15 - 20 minutes early to learn how Adobe Connect works.
    • Help with AdobeConnect
    • Want to try your skills at parsing? Share your User Interface ideas? Got mad skills for developing image-analysis and image segmentation algorithms? Know where the great authority files (data dictionaries) are? Do tell, via remote participation or in our online group notes doc.

Overview of the Challenge

Frequently Asked Questions

Deliverables & Outputs

Choosing Images and Parsing Decisions

Issues that need work

*Thank you NESCent, Hilmar Lapp and the HIP working group for this model.

Back to the aOCR Wiki

Retrieved from "http://www.idigbio.org/wiki/index.php?title=2013_AOCR_Hackathon_Wiki&oldid=16415"