-
Notifications
You must be signed in to change notification settings - Fork 1
Data Processing Steps and Data Dictionaries
Eric Fegraus edited this page Jul 20, 2018
·
1 revision
# NEEDS TO BE UPDATED TO DESCRIBE THE CURRENT PROCESS/WORKFLOW AND DATASET IMPORT
Data processing scripts located in the rscripts folder transform and standardize the camera trap data to produce a series of files that are used by ShinyCam
team_rate_of_detection.csvrate_of_detection_MARIN_total.csv
Camera trap data associated with the TEAM project are currently available in one CSV file, named Terrestrial_Vertebrate.csv.
| Form Value | Definition | Data Type | Example |
|---|---|---|---|
| ID | Unique ID for each photo in dataset. | Numeric | 426758 |
| Site Name | Name of site where photo was taken. | String | Yasuni |
| Sampling Unit Name | String | CT-YAS-1-01 | |
| Latitude | Latitude of deployment location. | Numeric | -0.6092808 |
| Longitude | Longitude of deployment location | Numeric | -76.45798 |
| Sampling Period | 2011.01 | ||
| Photo Type | Type of photo that was taken (e.g. Animal, Unknown) | String | Animal |
| Photo Date | Date photo was taken. | Date (yyyy-mm-dd) | 2011-04-10 |
| Photo Time | Time photo was taken. | Time (hh:mm:ss) | 23:18:17 |
| Raw Name | Name of JPG file. | String | IMG_0002.JPG |
| Photo Notes | Notes relating to photo. | String | |
| Photo ID URL | URL to download photo. | String | |
| Class | Biological taxonomy level. | String | MAMMALIA |
| Order | Biological taxonomy level. | String | RODENTIA |
| Family | Biological taxonomy level. | String | CUNICULIDAE |
| Genus | Biological taxonomy level. | String | Cuniculus |
| Species | Biological taxonomy level. | String | paca |
| Number of Animals | Count of animals in image. | Numeric | 1 |
| Person Identifying the Photo | First and last name of person who identified photo. | String | John Doe |
| Camera Serial Number | Serial number of the camera track. | String | P900DD114014965 |
| Memory Card Serial Number | String | CT-YAS-01 | |
| Camera Notes | String | ||
| Camera Start Date and Time | The date the camera was activated and starting recorded observations. | Date (yyyy-mm-dd hh:mm:ss) | 2011-04-10 16:00:00 |
| Camera End Date and Time | The date the camera was picked up. | Date (yyyy-mm-dd hh:mm:ss) | 2011-05-10 07:43:55 |
| Person setting up the Camera | First and last name of person setting up camera. | String | John Doe |
| Person picking up the Camera | First and last name of person picking up the camera. | String | John Doe |
| Camera Make | Manufacturer of camera. | String | |
| Camera Model | Model of camera. | String | |
| Flash | Numeric | ||
| Exposure Time | Length of time digital sensor in camera exposed to light. In fraction of second. | String | 1/30 |
| Sequence Info | String | P900DD11014965 | |
| Triplet Number | String | 1 of 10 | |
| Moon Phase | Shape of the moon when photo was taken. | String | First Quarter |
| Temperature | Temperature when photo was taken, in Celsius. | Numeric | 20 |
| Protocol Version | String | Terrestrial Vertebrate-1 | |
| Data Set Creator Institutions | Research institution of data set creator. | String | Smithsonian Tropical Research Institute |
| Data Set Creator Scientist | First and last name of person who collected data. | String | John Doe |
| Data Set Contact | First and last name of contact person. | String | John Doe |
| Data Level | String | Level 0 |
Camera trap data associated with the Marin project are available for each conservation area of interest in the form of three CSV files, labeled: Camera, Deployments, and Images.
This file consists of technical specs of the camera equipment deployed in the field.
| Form Value | Definition | Data Type | Values List | Example | Required |
|---|---|---|---|---|---|
| Project ID | An internal identification for the project. It should be a relatively short name that makes the project unique among other prorjects for the user or organization | String | None | San Pablo Reserve | Yes |
| Camera ID | An internal identification that makes this camera unique among other camera traps. For example, some camera trap projects will label a camera with a code CT103. | String | None | CT104 | Yes |
| Make | Manufacturer of the camera | String | None | Reconyx | Yes |
| Model | The model of the camera | String | None | H850 | No |
| Serial Number | The serial number of the camera trap. This can also be used as the Camera id. | String | None | 12348571 | No |
| Year Purchased | Year the camera was purchased. | Date (yyyy-mm-dd hh:mm:ss) | None | 2016-05-11 06:55:11 | No |
This file consists of experimental details for each camera trap deployment.
| Form Value | Definition | Data Type | Values List | Example | Reqiured |
|---|---|---|---|---|---|
| Project ID | An internal identification for the project. It should be a relatively short name that makes the project unique among other prorjects for the user or organization | String | IDE | Yes | |
| Deployment ID | Unique identifier for the camera deployment (within the project) | String | UND-012-D0001 | Yes | |
| Event | Camera trap arrays can be grouped into sampling ‘events’ (not to be confused with events of animal detection by camera traps, see footnote). There are many uses for an event and is designed to be flexibly used. Common events could be seasons (wet and dry), months, years or other types of logical groupings when field sampling occurs. | String | No | ||
| Array Name | A name for a logical grouping of deployments. This could be for thematic or logistical reasons. For example, all deployments north of the river will be in array "N" and those south will be in array "S" | String | No | ||
| Deployment Location ID | A unique name for the location of the deploymentID. This name should be consistent across projects and time periods. This is useful for repeat sampling at the same location or when 2 or more cameras are associated with one location. | String | UND-012 | Yes | |
| Longitude Resolution | Latitude of Deployment Location ID in decimal degrees using datum WGS84 | Numeric | -57.99257 | Yes | |
| Latitude Resolution | Longitude Deployment Location ID in decimal degrees using datum WGS84 | Numeric | -23.73154 | Yes | |
| Camera Deployment Begin Date | The date the camera was activated and starting recorded observations. This also marks the date a new SD card was inserted into a camera that is continuously operating. | Date (yyyy-mm-dd hh:mm:ss) | 2016-04-08 06:45:31 | Yes | |
| Camera Deployment End Date | The date the camera was picked up OR the date the SD card was retrieved. | Date (yyyy-mm-dd hh:mm:ss) | 2016-05-11 06:55:11 | Yes | |
| Bait Type | Type of bait used (if any)Type of bait that was used with camera; standardized list with No Bait and Other Bait options | String | None, Scent, Meat, Visual, Acoustic, Other | None | Yes |
| Bait Description | Text field to describe baiting method and Other Bait type if necessary | String | |||
| Feature Type | Type of feature, if any, that camera deployment is associated with | String | Road paved, Road dirt, Trail hiking, Trail game, Road underpass, Road overpass, Road bridge, Culvert, Burrow, Nest site, Carcass, Water source, Fruiting tree, Other | ||
| Feature Type methodology | String | ||||
| Camera ID | Camera ID (see Cameras tab) | String | Yes | ||
| Quiet Period Setting | Time specified between shutter triggers when activity in the sensor will not trigger the shutter. Specified in minutes and fraction of minutes | Numeric | 0 to some maximum number | Yes | |
| Restriction on access | An optional capacity to put restrictions on images coming from a deployment | String | No | ||
| Camera Failure Details | Details the status of the camera after the named deployment; standardized list | String | Camera Functioning,Unknown Failure,Vandalism/Theft,Memory Card/Film Failure,Camera Hardware Failure,Wildlife Damage | Camera Functioning | Yes |
This file consists of metadata of wildlife photos taken by the camera traps, as well as information produced by wildlife experts after examining the photos.
| Form Value | Definition | Data Type | Values List | Example | Required |
|---|---|---|---|---|---|
| Project ID | See Cameras | String | IDE | Yes | |
| Deployment ID | See Deployments | String | UND-012-D0002 | Yes | |
| Image ID | Unique identifier for the image (within the project) | String or numeric | IMG1 | Yes | |
| Location | Location of the image via a relative path | String | /images/CT-011/image01.jpg | No | |
| Photo Type | Type of photo that was taken (e.g., Animal, Set-up) | String or numeric | Start, End, Setup, Blank, Animal, Staff, Unknown, Unidentifiable, Timelapse, Pickup | Yes | |
| Photo Type Identified by | First and last name of the person who identified the photo type | String | John Doe | Yes | |
| Genus Species | Latin name for a species (Genus and species) in the image | String | IUCN Red List for mammals and Birdlife International for birds + additional domestic and human types (e.g., tourist, park staff, etc) | Yes, if Photo Type = Animal | |
| Uncertainty | The uncertainty for the image identification by the photo identifier | String | Absolutely sure, Pretty sure, Not sure, Dont know, Other | No | |
| IUCN Identification Number | Identification number for IUCN or Birdlife or other supported taxonomic authority | String | No | ||
| Date_Time Captured | Date and time an image was taken | Date (yyyy-mm-dd hh:mm:ss) | 2016-04-18 06:45:31 | Yes | |
| Age | Age of animal | String | No | ||
| Sex | Sex of animal | String | No | ||
| Individual ID | Individual animal ID | String | No | ||
| Count | Number of animal per species in the image | Numeric | Yes | ||
| Animal recognizable (Y/N) | Was the animal recognizable? | String | Yes,No | No | |
| individual Animal notes | Notes about the animal and/or the image itself | String | No |
GIS shapefiles are currently used by the ShinyCamp app to define park boundaries associated with the Marin Project. See here for more information relating to shapefiles.
The ShinyCam app ingests data relating to species name, taxonomy, and conservation status from two CSV files:
taxonomy_scientific_name_20160813.csvtaxonomy_red_list_status_20160813.csv