/afscgap

Python-based tools for NOAA AFSC GAP marine species surveys data

Primary LanguagePythonBSD 3-Clause "New" or "Revised" LicenseBSD-3-Clause

Python Tools for AFSC GAP

Group Badges
Status build workflow status docs workflow status Project Status: Active – The project has reached a stable, usable state and is being actively developed.
Usage Python 3.7+ Pypi Badge License Binder
Publication pyOpenSci DOI
Archive Open in Code Ocean DOI

Python-based tool chain ("Pyafscgap.org") for working with the public bottom trawl data from the NOAA AFSC GAP. This provides information from multiple survey programs about where certain species were seen and when under what conditions, information useful for research in ocean health.

See webpage, project Github, and example notebook.



Quickstart

Taking your first step is easy!

Explore the data in a UI: To learn about the datasets, try out an in-browser visual analytics app at https://app.pyafscgap.org without writing any code.

Try out a tutorial in your browser: Learn from and modify an in-depth tutorial notebook in a free hosted academic environment (all without installing any local software).

Jump into code: Ready to build your own scripts? Here's an example querying for Pacific cod in the Gulf of Alaska for 2021:

import afscgap  # install with pip install afscgap
query = afscgap.Query()
query.filter_year(eq=2021)
query.filter_srvy(eq='GOA')
query.filter_scientific_name(eq='Gadus macrocephalus')
results = query.execute()

Continue your exploration in the developer docs.



Installation

Ready to take it to your own machine? Install the open source tools for accessing the AFSC GAP via Pypi / Pip:

$ pip install afscgap

The library's only dependency is requests and Pandas / numpy are not expected but supported. The above will install the release version of the library. However, you can also install the development version via:

$ pip install git+https://github.com/SchmidtDSE/afscgap.git@main

Installing directly from the repo provides the "edge" version of the library which should be treated as pre-release.



Purpose

Unofficial Python-based tool set for interacting with bottom trawl surveys from the Ground Fish Assessment Program (GAP). It offers:

  • Pythonic access to the official NOAA AFSC GAP API service.
  • Tools for inference of the "negative" observations not provided by the API service.
  • Visualization tools for quickly exploring and creating comparisons within the datasets, including for audiences with limited programming experience.

Note that GAP are an excellent collection of datasets produced by the Resource Assessment and Conservation Engineering (RACE) Division of the Alaska Fisheries Science Center (AFSC) as part of the National Oceanic and Atmospheric Administration's Fisheries organization (NOAA Fisheries).

Please see our objectives documentation for additional information about the purpose, developer needs addressed, and goals of the project.



Usage

This library provides access to the AFSC GAP data with optional zero catch ("absence") record inference.


Examples / tutorial

One of the best ways to learn is through our examples / tutorials series. For more details see our usage guide.


API Docs

Full formalized API documentation is available as generated by pdoc in CI / CD.


Data structure

Detailed information about our data structures and their relationship to the data structures found in NOAA's upstream database is available in our data model documentation.


Absence vs presence data

By default, the NOAA API service will only return information on hauls matching a query. So, for example, requesting data on Pacific cod will only return information on hauls in which Pacific cod is found. This can complicate the calculation of important metrics like catch per unit effort (CPUE). That in mind, one of the most important features in afscgap is the ability to infer "zero catch" records as enabled by set_presence_only(False). See more information in our inference docs.


Data quality and completeness

There are a few caveats for working with these data that are important for researchers to understand. These are detailed in our limitations docs.



License

We are happy to make this library available under the BSD 3-Clause license. See LICENSE for more details. (c) 2023 Regents of University of California. See the Eric and Wendy Schmidt Center for Data Science and the Environment at UC Berkeley.



Developing

Intersted in contributing to the project or want to bulid manually? Please see our build docs for details.



People

Sam Pottinger is the primary contact with additional development from Giulia Zarpellon. Additionally some acknowledgements:

This is a project of the The Eric and Wendy Schmidt Center for Data Science and the Environment at UC Berkeley where Kevin Koy is Executive Director. Please contact us via dse@berkeley.edu.



Open Source

We are happy to be part of the open source community.

At this time, the only open source dependency used by this microlibrary is Requests which is available under the Apache v2 License from Kenneth Reitz and other contributors.

In addition to Github-provided Github Actions, our build and documentation systems also use the following but are not distributed with or linked to the project itself:

Next, the visualization tool has additional dependencies as documented in the visualization readme.

Finally, note that the website uses assets from The Noun Project under the NounPro plan. If used outside of https://pyafscgap.org, they may be subject to a different license.

Thank you to all of these projects for their contribution.



Version history

Annotated version history:

  • 1.0.4: Minor documentation fypo fix.
  • 1.0.3: Documentation edits for journal article.
  • 1.0.2: Minor documentation touch ups for pyopensci.
  • 1.0.1: Minor documentation fix.
  • 1.0.0: Release with pyopensci.
  • 0.0.9: Fix with issue for certain import modalities and the http module.
  • 0.0.8: New query syntax (builder / chaining) and units conversions.
  • 0.0.7: Visual analytics tools.
  • 0.0.6: Performance and size improvements.
  • 0.0.5: Changes to documentation.
  • 0.0.4: Negative / zero catch inference.
  • 0.0.3: Minor updates in documentation.
  • 0.0.2: License under BSD.
  • 0.0.1: Initial release.

The hauls community file was last updated on Nov 1, 2023. See 108 for details.