How we developed a data exchange format: Lessons learned from Camtrap DP

Transcript

1. How we developed a
   data exchange format
   Lessons learned from Camtrap DP
   TDWG 2023
   13 October 2023
   Peter Desmet & Jakub Bubnicki
   0000-0002-8442-8025 & 0000-0002-2064-3113
   

   View Slide

2. - “Camera Trap Data Package”
   - Model and format to exchange
   camera trapping data
   - Designed to capture all essential
   data and metadata of a single
   camera trap study
   What is Camtrap DP?
   Desmet et al. (2021)
   bit.ly/camtrap-dp-tdwg-2021
   

   View Slide

3. 📈 Massive amounts of
   camera trap data
   ✅ Data management
   ✅ Data processing
   ❌ Data exchange,
   harmonization, publication
   ❌ Suitable standards
   Why Camtrap DP?
   Bubnicki et al. (2023)
   doi.org/10.32942/X2BC8J
   Agouti Wildlife
   Insights
   TRAP
   PER
   eMam
   mal
   

   View Slide

4. 1. Camtrap DP should allow
   easy and interoperable
   data exchange
   2. Camtrap DP should be
   developed openly and
   collaboratively
   Two guiding principles
   

   View Slide

5. Use a simple data model
   - This is hard! 🤯
   - Three tables:
   - Deployments
   - Media
   - Observations
   - Supports wide range of:
   - Deployment designs
   - Classification techniques
   - Analytical use cases
   

   View Slide

6. Build on Frictionless Standards
   - Open, generic specifications
   (JSON Schemas) to describe:
   - Datasets (Data Package)
   - Data files (Data Resource)
   - Table fields (Table Schema)
   - Simple, machine-usable &
   extensible
   - Camtrap DP = Frictionless DP
   - Existing software can be used
   to read and validate data
   specs.frictionlessdata.io
   

   View Slide

7. {
   "name": "mediaID",
   "description": "Unique identifier of the media file.",
   "skos:broadMatch": "http://purl.org/dc/terms/identifier",
   "type": "string",
   "constraints": {
   "required": true,
   "unique": true
   }
   }, {
   "name": "deploymentID",
   "description": "Identifier of the deployment …",
   "skos:broadMatch": "http://rs.tdwg.org/dwc/terms/parentEventID",
   "type": "string",
   "constraints": {
   "required": true
   }
   }, {
   "name": "captureMethod",
   "description": "Method used to capture the media file.",
   "skos:broadMatch":
   "http://rs.tdwg.org/ac/terms/resourceCreationTechnique",
   "type": "string",
   "constraints": {
   "enum": ["motionDetection", "timeLapse"]
   }
   }, {
   "name": "timestamp",
   "description": "Date and time at which the media file …",
   "skos:exactMatch": "http://ns.adobe.com/xap/1.0/CreateDate",
   "type": "datetime",
   "format": "%Y-%m-%dT%H:%M:%S%z",
   "constraints": {
   "required": true
   },
   }
   Reuse existing standards
   - Frictionless Standards:
   - Metadata terms, csv dialect,
   field names, data types,
   required values, controlled
   values, etc.
   - TDWG and other standards:
   - Darwin Core
   - Audiovisual Core
   - Humboldt Extension
   - Dublin Core
   - DataCite Metadata Schema
   - Camtrap DP = Domain-specific
   and highly interoperable
   

   View Slide

8. Develop openly
   - Code on GitHub:
   - Open source (MIT license)
   - Versioned (incl. semantic)
   - Collaboration on GitHub:
   - Suggestions as issues
   - Review, discuss, implement
   - Automated tests
   - Documentation on GitHub:
   - Website (updates automatically)
   - Example dataset
   github.com/tdwg/camtrap-dp
   

   View Slide

9. Build into software
   - Entire ecosystem:
   - Management: Agouti, Trapper,
   other systems
   - Analysis: Camtraptor
   - Publication: GBIF IPT
   - Challenging to coordinate 😅
   - Feedback from adopters
   agouti.eu
   os-conservation.org/projects/trapper
   inbo.github.io/camtraptor
   gbif.org/ipt
   

   View Slide

10. Reyserhove et al. (2023)
    doi.org/10.35035/doc-0qzp-2x37
    - Short, sticky name
    - Outreach:
    - Conferences & GBIF webinar
    - GBIF publication guide
    - Co-authors on paper
    - Use cases
    - Give it time, build trust
    Embrace participation
    

    View Slide

11. - Versioning:
    - Datasets point to a version of
    Camtrap DP
    - Camtraptor provides conversion
    - People:
    - Committed maintainers
    - TDWG community
    - Funding:
    - LifeWatch, OSCF, NLBIF
    - Biodiversa+
    Think about maintenance
    

    View Slide

12. Summary
    - Use a simple data model
    - Build on Frictionless Standards
    - Reuse existing standards
    - Develop openly
    - Built into software
    - Embrace participation
    - Think about maintenance
    

    View Slide

13. Thank you
    camtrap-dp.tdwg.org
    Desmet P & Bubnicki J (2023) How we
    developed a data exchange format:
    Lessons learned from Camtrap DP.
    Presentation at TDWG 2023.
    https://bit.ly/camtrap-dp-tdwg-2023
    Open Science
    Conservation Fund
    

    View Slide