Showing preview only (1,538K chars total). Download the full file or copy to clipboard to get everything.
Repository: nasa/EMIT-Data-Resources
Branch: main
Commit: 2ae671f12803
Files: 40
Total size: 90.5 MB
Directory structure:
gitextract_4n5_jjs8/
├── .gitignore
├── CHANGE_LOG.md
├── CITATION.cff
├── CODE_OF_CONDUCT.md
├── CONTRIBUTING.md
├── LICENSE
├── README.md
├── data/
│ ├── cuprite_bbox.geojson
│ ├── emit_data_urls.txt
│ ├── isla_gaviota.geojson
│ ├── methane_tutorial/
│ │ ├── emit20220815t042838_ch4_target
│ │ ├── methane_inout_points.csv
│ │ ├── plume_bbox.geojson
│ │ └── rad_band_ratio_points.csv
│ ├── min_uncert_urls.txt
│ ├── mineral_grouping_matrix_20230503.csv
│ ├── results_urls.txt
│ ├── rgb_browse_urls.txt
│ └── sample_coords.csv
├── guides/
│ ├── Getting_EMIT_Data_using_EarthData_Search.md
│ └── Streaming_cloud_optimized_geotiffs_using_QGIS.md
├── python/
│ ├── how-tos/
│ │ ├── How_to_Convert_to_ENVI.ipynb
│ │ ├── How_to_Direct_S3_Access.ipynb
│ │ ├── How_to_Extract_Area.ipynb
│ │ ├── How_to_Extract_Points.ipynb
│ │ ├── How_to_Orthorectify.ipynb
│ │ ├── How_to_find_EMIT_data_using_CMR_API.ipynb
│ │ ├── How_to_find_and_access_EMIT_data.ipynb
│ │ └── How_to_use_EMIT_Quality_data.ipynb
│ ├── modules/
│ │ ├── emit_tools.py
│ │ └── tutorial_utils.py
│ └── tutorials/
│ ├── Exploring_EMIT_L2A_Reflectance.html
│ ├── Exploring_EMIT_L2A_Reflectance.ipynb
│ ├── Finding_EMIT_L2B_Data.ipynb
│ ├── Generating_Methane_Spectral_Fingerprint.ipynb
│ ├── Visualizing_Methane_Plume_Timeseries.ipynb
│ └── Working_with_EMIT_L2B_Mineralogy.ipynb
└── setup/
├── prerequisites.md
├── setup_instructions.md
└── workshop_setup.md
================================================
FILE CONTENTS
================================================
================================================
FILE: .gitignore
================================================
# python internals
*.ipynb_checkpoints
*__pycache__*
# local development
*.img*
*.envi*
*.hdr*
*.nc*
*.tif*
*.pptx*
/data/example_out.csv
/data/isla_gaviota_3.geojson
/data/outputs
/data/envi
/python/daac_data_download_python
/python/emit_utils
/ignore
/data/emit_asset_urls.txt
/.vscode/
================================================
FILE: CHANGE_LOG.md
================================================
# Change Log
All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](http://keepachangelog.com/)
and this project adheres to [Semantic Versioning](http://semver.org/).
_________________________________________________________________________
## 2024-06-17
> ### Added
>
> - Add CITATION.cff file to provide a citation for the repository
> ### Changed
>
> - Removed use of `nan` for fill_values to be more consistent with EMIT datasets. This means that users will have to manually assign them before plotting.
> - Updated notebooks to include the new fill_value behavior, and assign them to `np.nan` for visualizations
> - Replaced `raw_spatial_crop` with `spatial_subset` function to subset EMIT data spatially before applying orthorectification
> - Fixed typos in Earthdata Search Guide
## 2024-03-22
> ### Added
>
> - `geojson` with plume bounding box for `Visualizing Methane Plume Timeseries` notebook
> ### Changed
>
> - Updates to instructions in `Visualizing Methane Plume Timeseries` notebook
> - Remove horizontal lines from notebooks for web-book creation
## 2024-03-13
> ### Added
>
> - Added new `Visualizing Methane Plume Timeseries` notebook
> - Added new `Generating Methane Spectral Fingerprint` notebook
> - Added new `tutorial_utils.py` module which has functions specific to the new CH4 notebooks
> ### Changed
>
> - Updated `setup_instructions.md` to provide more up to date Python setup instructions
> - Updated `README.md` to include new notebooks
> - Added some functions to `emit_tools.py` to support the new methane tutorial notebooks
> - Minor updates to `How to Extract Area` and `How to Extract Points` notebooks to improve visualizations
## 2023-12-01
> ### Changed
>
> - Applied automatic formatting to `emit_tools.py`
> - Fixed GLT index adjustment in `raw_spatial_crop` function
> - Reimplemented `PointerXY` stream so interactive plot from `Exploring EMIT L2A Reflectance` shows spectra at current cursor location
## 2023-11-27
> ### Changed
>
> - Updated `emit_tools.py` to fix an issue with `raw_spatial_crop`
> - Updated some plotting visuals within notebooks
> - Improved interactive plots in `Exploring_EMIT_L2A_Reflectance`
> - Renamed `CONTRIBUTE.md` to `CONTRIBUTING.md`
## 2023-10-23
> ### Changed
>
> - Updated `emit_tools.py` to fix an issue with application of the GLT to elevation
## 2023-10-12
> ### Changed
>
> - Updated data download method in how-to and tutorial notebooks
## 2023-08-04
> ### Changed
>
> - Updated `How to Direct S3 Access` notebook to include `earthaccess`
## 2023-07-06
> ### Added
>
> - `How_to_find_and_access_EMIT_data` Notebook - A guide for how to use `earthaccess` Python library to find and open/download EMIT data.
> ### Changed
>
> - **Default behavior of `emit_xarray` function from `emit_tools.py` no longer orthorectifies, must add parameter `ortho=True`**
> - Removed 'bands' dimension in favor of 'wavelengths' for 'radiance' and 'reflectance' when using `emit_xarray` function from `emit_tools.py`
> - Updated `write_envi` function from `emit_tools.py`, it should work for L1A,L1B, and L2A products.
> - various other minor changes to `emit_tools.py`
> - Updated notebooks to support changes made to `emit_tools.py`
> - Improved RGB Image creation and brightness adjustment in `Exploring_EMIT_L2A_Reflectance` Tutorial
> - Each notebook now uses the `earthaccess` python library to handle NASA Earthdata Login and downloads of needed files
> - Updated `setup_instructions.md` and recommended python environment `emit_tutorials_windows.yml`
> - Updated `setup_instructions.md` to recommend mamba
## 2023-05-30
> ### Changed
>
> - Restructured repository layout to match LP DAAC standards
## 2023-05-16
>
> ### Changed
>
> - Updated `emit_tools.py` to work for HTTPFileSystem URIs
## 2023-04-11
>
> ### Changed
>
> - The orthorectification process within `emit_tools.py` was creating a lat/lon grid based upon the upper left pixel coordinates. Corrected this to be pixel centers.
## 2023-03-29
> ### Added
>
> - CHANGE_LOG.md
> - CODE_OF_CONDUCT.md
> - CONTRIBUTING.md
> - LICENSE.md
>
> ### Changed
>
> - content in CHANGE_LOG.md
>
> ### Fixed
>
> -
================================================
FILE: CITATION.cff
================================================
cff-version: 1.2.0
title: EMIT Data Resources
message: >-
If you use this collection of resources in your research
we would appreciate an appropriate citation.
type: software
authors:
- given-names: Erik A
family-names: Bolch
affiliation: >-
KBR, Inc., Contractor to the U.S. Geological Survey,
Earth Resources Observation and Science Center,
National Aeronautics and Space Administration Land
Processes Distributed Active Archive Center
orcid: 'https://orcid.org/0000-0003-2470-4048'
- given-names: Aaron M
family-names: Friesz
affiliation: >-
KBR, Inc., Contractor to the U.S. Geological Survey,
Earth Resources Observation and Science Center,
National Aeronautics and Space Administration Land
Processes Distributed Active Archive Center
orcid: 'https://orcid.org/0000-0003-4096-3824'
- given-names: Mahsa
family-names: Jami
affiliation: >-
KBR, Inc., Contractor to the U.S. Geological Survey,
Earth Resources Observation and Science Center,
National Aeronautics and Space Administration Land
Processes Distributed Active Archive Center
orcid: 'https://orcid.org/0000-0002-3594-3004'
- given-names: Brianna M
family-names: Lind
affiliation: >-
KBR, Inc., Contractor to the U.S. Geological Survey,
Earth Resources Observation and Science Center,
National Aeronautics and Space Administration Land
Processes Distributed Active Archive Center
orcid: 'https://orcid.org/0000-0002-5306-9963'
- given-names: Philip G
family-names: Brodrick
affiliation: >-
Jet Propulsion Laboratory, California Institute of
Technology
orcid: 'https://orcid.org/0000-0001-9497-7661'
- given-names: K Dana
family-names: Chadwick
affiliation: >-
Jet Propulsion Laboratory, California Institute of
Technology
orcid: 'https://orcid.org/0000-0002-5633-4865'
- given-names: Cole K
family-names: Krehbiel
affiliation: >-
National Aeronautics and Space Administration Land
Processes Distributed Active Archive Center,
U.S. Geological Survey, Earth Resources
Observation and Science Center
orcid: 'https://orcid.org/0000-0003-2645-3449'
repository-code: 'https://github.com/nasa/EMIT-Data-Resources'
abstract: >-
This repository contains guides and Jupyter Notebooks to
help users access and work with data from the Earth
Surface Mineral Dust Source Investigation (EMIT) mission.
license: Apache-2.0
================================================
FILE: CODE_OF_CONDUCT.md
================================================
# Code of Conduct
## 1. Our Commitment
We are dedicated to fostering a respectful environment for everyone contributing to this project. We expect all participants to treat each other with respect, professionalism, and kindness.
## 2. Expected Behavior
- Be respectful and considerate of others.
- Engage in constructive discussions and offer helpful feedback.
- Gracefully accept constructive criticism.
## 3. Unacceptable Behavior
The following behaviors will not be tolerated:
- Harassment, discrimination, or intimidation of any kind.
- Offensive, abusive, or derogatory language and actions.
- Personal attacks or insults.
- Trolling or disruptive conduct.
- Sharing inappropriate content.
## 4. Reporting Violations
If you experience or witness any behavior that violates this Code of Conduct, please report it by contacting the project maintainers. All reports will be reviewed confidentially.
## 5. Enforcement
Violations of this Code of Conduct may result in actions such as warnings, temporary bans, or permanent exclusion from participation at the discretion of the maintainers.
## Contact Info
Email: <LPDAAC@usgs.gov>
Voice: +1-866-573-3222
Organization: Land Processes Distributed Active Archive Center (LP DAAC)¹
Website: <https://lpdaac.usgs.gov/>
Date last modified: 01-22-2025
¹Work performed under USGS contract G15PD00467 for NASA contract NNG14HH33I.
================================================
FILE: CONTRIBUTING.md
================================================
# Contributing to Our GitHub
This page is your one-stop shop for uncovering how to contribute to our Github!
## We Want Your Help!
No, really, we do! Please come and participate in our community and lets do science together! Depending on your level of interaction with the Land Processes Data Active Archive Center (LP DAAC) and the LP DAAC GitHub, visitors to the site can be described as:
- A **community member**: anyone in the open science community who visits a LP DAAC site, utilizes LP DAAC online tools, or attends a LP DAAC event.
- A **participant**: anyone who posts a comment or poses a question in the *GitHub Discussion Space*, reports a site bug or requests a new resource in *GitHub Issues*, or attends a LP DAAC event and utilizes any virtual chat features during that event.
- A **contributor**: anyone who forks this GitHub repository and submits pull requests to make additions or changes to the posted content.
Everyone reading this page is a community member, and we hope everyone will post comments and join discussions as a participant. Contributors are welcome, particularly to help find and point to other open science resources.
## Ways to Contribute to the GitHub
There are two main ways to contribute to the LP DAAC GitHub.
- **Suggest a change, addition, or deletion to what is already on the GitHub using [Issues](https://github.com/nasa/Transform-to-Open-Science/issues).** Issues can be about any LP DAAC plans, timelines, and content.
- Before submitting a new [issue](https://github.com/nasa/LPDAAC-Data-Resources/issues), check to make sure [a similar issue isn't already open](https://github.com/nasa/LPDAAC-Data-Resources/issues). If one is, contribute to that issue thread with your feedback.
- When submitting a bug report, please try to provide as much detail as possible, i.e. a screenshot or [gist](https://gist.github.com/) that demonstrates the problem, the technology you are using, and any relevant links.
- Issues labeled :sparkles:[`help wanted`](https://github.com/nasa/LPDAAC-Data-Resources/labels/help%20wanted):sparkles: make it easy for you to find ways you can contribute today.
- **Become a contributor!** [Fork](https://docs.github.com/en/get-started/quickstart/fork-a-repo) the repository and [make commits](https://docs.github.com/en/get-started/quickstart/contributing-to-projects#making-and-pushing-changes) to add resources and additional materials. Here are some ways you can contribute:
- by reporting bugs
- by suggesting new features
- by translating content to a new language
- by writing or editing documentation
- by writing specifications
- by writing code and documentation (**no pull request is too small**: fix typos, add code comments, clean up inconsistent whitespace)
- by closing [issues](https://github.com/nasa/LPDAAC-Data-Resources/issues)
In the spirit of open source software, everyone is encouraged to help improve this project!
## New to GitHub? Start here!
You can [sign up for GitHub here](https://github.com/)! The NASA Transform to Open Science Team has made a short video demonstrating how to make an easy pull request [here](https://youtu.be/PHoScPeMWHI).
For a more in-depth start, we suggest *[Getting Started with Git and GitHub: The Complete Beginner’s Guide](https://towardsdatascience.com/getting-started-with-git-and-github-6fcd0f2d4ac6)* and *[The Beginners Guide to Git and GitHub](https://www.freecodecamp.org/news/the-beginners-guide-to-git-github/)*. We've summarized some of the most important points below.
### Making a Change
*This section is attributed to [NumFOCUS](https://github.com/numfocus/getting-started-with-open-source/blob/master/CONTRIBUTING.md) and [Adrienne Friend](https://github.com/adriennefriend/imposter-syndrome-disclaimer).*
Once you've identified something you'd like to help with you're ready to make a change to the project repository!
1. First, describe what you're planning to do as a comment to the issue, (and this might mean making a new issue).
[This blog](https://www.igvita.com/2011/12/19/dont-push-your-pull-requests/) is a nice explanation of why putting this work in up front is so useful to everyone involved.
2. Fork this repository to your profile.
You can now do whatever you want with this copy of the project. You won't mess up anyone else's work so you're super safe.
Make sure to [keep your fork up to date]( https://github.com/KirstieJane/STEMMRoleModels/wiki/Syncing-your-fork-to-the-original-repository-via-the-browser) with the master repository.
3. Make the changes you've discussed.
Try to keep the changes focused rather than changing lots of things at once. If you feel tempted to branch out then please *literally* branch out: create separate branches for different updates to make the next step much easier!
4. Submit a pull request.
A member of the executive team will review your changes, have a bit of discussion and hopefully merge them in!
N.B. you don't have to be ready to merge to make a pull request! We encourage you to submit a pull request as early as you want to. They help us to keep track of progress and help you to get earlier feedback.
## Development Model
For accepting new contributions, TOPS uses the [forking workflow](https://guides.github.com/activities/forking/). As the first step of your contribution, you'll want to fork this repository, make a local clone of it, add your contribution, and then create a pull request back to the LP DAAC repository.
All documentation should be written using Markdown and Github Markdown-supported HTML.
## Attribution
These contributing guidelines are adapted from the NASA Transform to Open Science github, available at https://github.com/nasa/Transform-to-Open-Science/blob/main/CONTRIBUTING.md.
================================================
FILE: LICENSE
================================================
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
================================================
FILE: README.md
================================================
# EMIT-Data-Resources
Welcome to the EMIT-Data-Resources repository. This repository provides guides, short how-tos, and tutorials to help users access and work with data from the [Earth Surface Mineral Dust Source Investigation (EMIT) mission](https://lpdaac.usgs.gov/data/get-started-data/collection-overview/missions/emit-overview/). In the interest of open science this repository has been made public but is still under active development. All notebooks and scripts should be functional, however, changes or additions may be made. Make sure to consult the [CHANGE_LOG.md](CHANGE_LOG.md) for the most recent changes to the repository. Contributions from all parties are welcome.
---
## EMIT Background
The [EMIT](https://earth.jpl.nasa.gov/emit/) Project delivers space-based measurements of surface mineralogy of the Earth’s arid dust source regions. These measurements are used to initialize the compositional makeup of dust sources in Earth System Models (ESMs). The dust cycle, which describe the generation, lofting, transport, and deposition of mineral dust, plays an important role in ESMs. Dust composition is presently the largest uncertainty factor in quantifying the magnitude of aerosol direct radiative forcing. By understanding the composition of mineral dust sources, EMIT aims to constrain the sign and magnitude of dust-related radiative forcing at regional and global scales. During its one-year mission on the International Space Station (ISS), EMIT will make measurements over the sunlit Earth’s dust source regions that fall within ±52° latitude. EMIT will schedule up to five visits (three on average) of each arid target region and only acquisitions not dominated by cloud cover will be downlinked. EMIT-based maps of the relative abundance of source minerals will advance the understanding of the current and future impacts of mineral dust in the Earth system.
EMIT Data Products are distributed by the [LP DAAC](https://lpdaac.usgs.gov/). Learn more about EMIT data products from [EMIT Product Pages](https://lpdaac.usgs.gov/product_search/?query=emit&status=Operational&view=cards&sort=title) and search for and download EMIT data products using [NASA EarthData Search](https://search.earthdata.nasa.gov/search?q=%22EMIT%22)
---
## Prerequisites/Setup Instructions
This repository requires that users set up a compatible Python environment and download the EMIT granules used. See the `setup_instuctions.md` file in the `./setup/` folder.
## Repository Contents
Below are the resources available for EMIT Data.
|Name|Type|Summary|
|:---|:---|:---|
|[Getting EMIT Data using EarthData Search](guides/Getting_EMIT_Data_using_EarthData_Search.md)|Markdown Guide|A thorough walkthrough for using [EarthData Search](https://search.earthdata.nasa.gov/search) to find and download EMIT data|
|[Streaming NASA Earthdata Cloud-Optimized GeoTIFFs using QGIS](guides/Streaming_cloud_optimized_geotiffs_using_QGIS.md)|Markdown Guide|A walkthrough to set up QGIS to stream cloud-optimized geotiff files from NASA Earthdata|
|[Exploring EMIT L2A Reflectance](python/tutorials/Exploring_EMIT_L2A_Reflectance.ipynb)|Jupyter Notebook|Explore EMIT L2A Reflectance data using interactive plots|
|[Visualizing Methane Plume Timeseries](python/tutorials/Visualizing_Methane_Plume_Timeseries.ipynb)|Jupyter Notebook|Find EMIT L2B CH4 Plume Data and build a timeseries of CH4 plume complexes|
|[Generating_Methane_Spectral_Fingerprint](python/tutorials/Generating_Methane_Spectral_Fingerprint.ipynb)|Jupyter Notebook|Extract Radiance Spectra and build an in-plume/out-of-plume ratio to compare with CH4 absorption coefficient|
|[Finding_EMIT_L2B_Data](python/tutorials/Finding_EMIT_L2B_Data.ipynb)|Jupyter Notebook|Use the `earthaccess` Python library to find EMIT L2B Mineral Identification Band Depth and Uncertainty data|
|[Working with EMIT L2B Mineralogy](python/tutorials/Working_with_EMIT_L2B_Mineralogy.ipynb)|Jupyter Notebook|Work with the EMIT L2B Mineral Identification Band Depth and Uncertainty Data and aggregate individual spectral library constituents into the EMIT-10 minerals and estimate abundance|
|[How to find and access EMIT data](python/how-tos/How_to_find_and_access_EMIT_data.ipynb)|Jupyter Notebook|Use the `earthaccess` Python library to find and download or stream EMIT data|
|[How to Convert to ENVI Format](python/how-tos/How_to_Convert_to_ENVI.ipynb)|Jupyter Notebook|Convert from downloaded netCDF4 (.nc) format to .envi format|
|[How to Orthorectify](python/how-tos/How_to_Orthorectify.ipynb)|Jupyter Notebook|Use the geometry lookup table (GLT) included with the EMIT netCDF4 file to project on a geospatial grid (EPSG:4326)|
|[How to Extract Point Data](python/how-tos/How_to_Extract_Points.ipynb)|Jupyter Notebook|Extract spectra using lat/lon coordinates from a .csv and build a dataframe/.csv output|
|[How to Extract Area Data](python/how-tos/How_to_Extract_Area.ipynb)|Jupyter Notebook|Extract an area defined by a .geojson or shapefile|
|[How to use EMIT Quality Data](python/how-tos/How_to_use_EMIT_Quality_data.ipynb)|Jupyter Notebook|Build a mask using an EMIT L2A Mask file and apply it to an L2A Reflectance file|
|[How to use Direct S3 Access with EMIT](python/how-tos/How_to_Direct_S3_Access.ipynb)|Jupyter Notebook|Use S3 from inside AWS us-west2 to access EMIT Data|
|[How to find EMIT Data using NASA's CMR API](python/how-tos/How_to_find_EMIT_data_using_CMR_API.ipynb)|Jupyter Notebook|Use NASA's CMR API to programmatically find EMIT Data|
---
## Helpful Links
+ [JPL EMIT Website](https://earth.jpl.nasa.gov/emit/)
+ [EMIT Tutorial Webinar Series Recordings](https://www.youtube.com/playlist?list=PLO2yB4LGNlWrC5NdxeHMxyAxdwQhSypXe)
+ [LP DAAC EMIT Product Pages](https://lpdaac.usgs.gov/product_search/?query=emit&status=Operational&view=cards&sort=title) - Learn more about available EMIT products
+ [VISIONS Open Data Portal](https://earth.jpl.nasa.gov/emit/data/data-portal/coverage-and-forecasts/) - Learn about current and forecasted EMIT coverage
+ [EMIT on Earth Data Search](https://search.earthdata.nasa.gov/search?q=%22EMIT%22) - Download EMIT Data from NASA
+ [EMIT Github Repository](https://github.com/emit-sds) - Main EMIT Repository
+ [EMIT Utilities Github Repository](https://github.com/emit-sds/emit-utils) - General convenience utilities for working with EMIT data
+ [L2A Reflectance User Guide](https://lpdaac.usgs.gov/documents/1569/EMITL2ARFL_User_Guide_v1.pdf)
+ [L2A Algorithm Theoretical Basis Document](https://lpdaac.usgs.gov/documents/1571/EMITL2A_ATBD_v1.pdf)
+ [EMIT on Slack]( https://forms.gle/XefLVG6e6A7ezwpY9) - Join the EMIT slack community!
---
## Contact Info
Email: <LPDAAC@usgs.gov>
Voice: +1-866-573-3222
Organization: Land Processes Distributed Active Archive Center (LP DAAC)¹
Website: <https://lpdaac.usgs.gov/>
Date last modified: 08-01-2024
¹Work performed under USGS contract G15PD00467 for NASA contract NNG14HH33I.
================================================
FILE: data/cuprite_bbox.geojson
================================================
{"type":"FeatureCollection","features":[{"type":"Feature","properties":{},"geometry":{"coordinates":[[[-117.24309240198033,37.59129385913785],[-117.24309240198033,37.50102626452812],[-117.14631968357332,37.50102626452812],[-117.14631968357332,37.59129385913785],[-117.24309240198033,37.59129385913785]]],"type":"Polygon"}}]}
================================================
FILE: data/emit_data_urls.txt
================================================
https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/EMITL2ARFL.001/EMIT_L2A_RFL_001_20220903T163129_2224611_012/EMIT_L2A_RFL_001_20220903T163129_2224611_012.nc
https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/EMITL2ARFL.001/EMIT_L2A_RFL_001_20220903T163129_2224611_012/EMIT_L2A_MASK_001_20220903T163129_2224611_012.nc
================================================
FILE: data/isla_gaviota.geojson
================================================
{"type":"FeatureCollection","features":[{"type":"Feature","properties":{},"geometry":{"coordinates":[[[-62.147583513919045,-39.88950549416461],[-62.16899895047814,-39.87693893067732],[-62.19419358172446,-39.90641838472922],[-62.20427143422259,-39.94071456822524],[-62.1318368693898,-39.95230375907932],[-62.11609022486114,-39.92091182572591],[-62.125538211578245,-39.895787912197314],[-62.147583513919045,-39.88950549416461]]],"type":"Polygon"}}]}
================================================
FILE: data/methane_tutorial/emit20220815t042838_ch4_target
================================================
001 381.006 0.000000000000000000
002 388.409 0.000000000000000000
003 395.816 0.000000000000000000
004 403.225 0.000000000000000000
005 410.638 0.000000000000000000
006 418.054 0.000000000000000000
007 425.472 0.000000000000000000
008 432.893 0.000000000000000000
009 440.317 0.000000000000000000
010 447.743 0.000000000000000000
011 455.170 0.000000000000000000
012 462.599 0.000000000000000000
013 470.030 0.000000000000000000
014 477.463 0.000000000000000000
015 484.897 0.000000000000000000
016 492.333 0.000000000000000000
017 499.771 0.000000000000000000
018 507.210 0.000000000000000000
019 514.650 0.000000000000000000
020 522.091 0.000000000000000000
021 529.533 0.000000000000000000
022 536.977 0.000000000000000000
023 544.421 0.000000000000000000
024 551.867 0.000000000000000000
025 559.314 0.000000000000000000
026 566.762 0.000000000000000000
027 574.209 0.000000000000000000
028 581.659 0.000000000000000000
029 589.108 0.000000000000000000
030 596.558 0.000000000000000000
031 604.010 0.000000000000000000
032 611.462 0.000000000000000000
033 618.915 0.000000000000000000
034 626.368 0.000000000000000000
035 633.821 0.000000000000000000
036 641.276 0.000000000000000000
037 648.730 0.000000000000000000
038 656.186 0.000000000000000000
039 663.641 0.000000000000000000
040 671.098 0.000000000000000000
041 678.554 0.000000000000000000
042 686.010 0.000000000000000000
043 693.468 0.000000000000000000
044 700.925 0.000000000000000000
045 708.384 0.000000000000000000
046 715.841 0.000000000000000000
047 723.299 0.000000000000000000
048 730.759 0.000000000000000000
049 738.217 0.000000000000000000
050 745.677 0.000000000000000000
051 753.136 0.000000000000000000
052 760.596 0.000000000000000000
053 768.056 0.000000000000000000
054 775.516 0.000000000000000000
055 782.978 0.000000000000000000
056 790.438 0.000000000000000000
057 797.899 0.000000000000000000
058 805.362 0.000000000000000000
059 812.823 0.000000000000000000
060 820.285 0.000000000000000000
061 827.746 0.000000000000000000
062 835.207 0.000000000000000000
063 842.670 0.000000000000000000
064 850.131 0.000000000000000000
065 857.594 0.000000000000000000
066 865.055 0.000000000000000000
067 872.518 0.000000000000000000
068 879.980 0.000000000000000000
069 887.441 0.000000000000000000
070 894.904 0.000000000000000000
071 902.366 0.000000000000000000
072 909.829 0.000000000000000000
073 917.291 0.000000000000000000
074 924.754 0.000000000000000000
075 932.216 0.000000000000000000
076 939.679 0.000000000000000000
077 947.140 0.000000000000000000
078 954.603 0.000000000000000000
079 962.064 0.000000000000000000
080 969.527 0.000000000000000000
081 976.988 0.000000000000000000
082 984.450 0.000000000000000000
083 991.911 0.000000000000000000
084 999.373 0.000000000000000000
085 1006.834 0.000000000000000000
086 1014.295 0.000000000000000000
087 1021.757 0.000000000000000000
088 1029.217 0.000000000000000000
089 1036.678 0.000000000000000000
090 1044.138 0.000000000000000000
091 1051.599 0.000000000000000000
092 1059.060 0.000000000000000000
093 1066.520 0.000000000000000000
094 1073.980 0.000000000000000000
095 1081.440 0.000000000000000000
096 1088.900 0.000000000000000000
097 1096.360 0.000000000000000000
098 1103.818 0.000000000000000000
099 1111.278 0.000000000000000000
100 1118.737 0.000000000000000000
101 1126.196 0.000000000000000000
102 1133.655 0.000000000000000000
103 1141.113 0.000000000000000000
104 1148.572 0.000000000000000000
105 1156.030 0.000000000000000000
106 1163.488 0.000000000000000000
107 1170.946 0.000000000000000000
108 1178.404 0.000000000000000000
109 1185.862 0.000000000000000000
110 1193.318 0.000000000000000000
111 1200.776 0.000000000000000000
112 1208.233 0.000000000000000000
113 1215.690 0.000000000000000000
114 1223.147 0.000000000000000000
115 1230.604 0.000000000000000000
116 1238.060 0.000000000000000000
117 1245.515 0.000000000000000000
118 1252.972 0.000000000000000000
119 1260.428 0.000000000000000000
120 1267.883 0.000000000000000000
121 1275.339 -0.000000069654603082
122 1282.794 -0.000000111146726415
123 1290.250 -0.000000182101968164
124 1297.705 -0.000000305636779985
125 1305.160 -0.000000527336328276
126 1312.614 -0.000000929683246631
127 1320.068 -0.000001671465390813
128 1327.523 -0.000003054706867060
129 1334.976 -0.000005628539263286
130 1342.429 -0.000010447838948433
131 1349.882 -0.000019460938889804
132 1357.335 -0.000036627215903753
133 1364.787 -0.000071207584885208
134 1372.238 -0.000150330219280241
135 1379.691 -0.000374439666851373
136 1387.142 -0.001133920033820656
137 1394.593 -0.003713146872892919
138 1402.043 -0.010163969848653337
139 1409.494 -0.017011214017695429
140 1416.944 -0.017485140828103148
141 1424.393 -0.020481224988834982
142 1431.843 -0.015399980587926871
143 1439.292 -0.007183837386920701
144 1446.740 -0.005160594956011227
145 1454.189 -0.003739930072627756
146 1461.637 -0.002887781246559040
147 1469.085 -0.003163939162842973
148 1476.532 -0.003950452229267255
149 1483.980 -0.002272845531566601
150 1491.426 -0.001322976560280106
151 1498.873 -0.000841631995893533
152 1506.319 -0.000661007257311044
153 1513.765 -0.000482083726915260
154 1521.210 -0.000314814341620599
155 1528.655 -0.000207755538147690
156 1536.101 -0.000146404330602690
157 1543.545 -0.000109917824796095
158 1550.989 -0.000080034561866423
159 1558.433 -0.000098030483352345
160 1565.877 -0.000105905085002603
161 1573.319 -0.000067902940535459
162 1580.762 -0.000057276544744732
163 1588.205 -0.000120224595071614
164 1595.647 -0.000401908209860957
165 1603.089 -0.001576972276221576
166 1610.530 -0.006222410401732700
167 1617.970 -0.021435322927493949
168 1625.410 -0.059910292461777682
169 1632.851 -0.112971337794280702
170 1640.290 -0.120352695021060635
171 1647.730 -0.129676205161983693
172 1655.169 -0.106476159401298842
173 1662.607 -0.144605482127502377
174 1670.046 -0.165405726174070178
175 1677.484 -0.080018266251608433
176 1684.921 -0.091654209149709925
177 1692.358 -0.116862482051208180
178 1699.795 -0.159110003641637521
179 1707.231 -0.188785289744263840
180 1714.667 -0.095120403815047841
181 1722.103 -0.121835208746717272
182 1729.538 -0.155497119666077821
183 1736.973 -0.104063918135378949
184 1744.407 -0.071084409123687700
185 1751.841 -0.050739589759742680
186 1759.275 -0.047096044049794450
187 1766.708 -0.064428685570723135
188 1774.142 -0.093163445946300161
189 1781.574 -0.060473895624796645
190 1789.007 -0.091518120707544448
191 1796.439 -0.070149779655907019
192 1803.870 -0.028500757144960604
193 1811.301 -0.031436803808985371
194 1818.731 -0.028705900563194626
195 1826.161 -0.002531317794645903
196 1833.591 -0.000124534361707594
197 1841.021 -0.000000106797917028
198 1848.449 -0.000000000050185741
199 1855.877 -0.000000013870771764
200 1863.305 -0.000000103434050046
201 1870.733 -0.000000050168210930
202 1878.160 -0.000000105384939200
203 1885.587 -0.000000275024896038
204 1893.013 -0.000000404027025009
205 1900.439 -0.000000186151182586
206 1907.864 -0.000000113526786294
207 1915.289 -0.000010222163086953
208 1922.713 -0.000149251023607931
209 1930.137 -0.000559261915934703
210 1937.561 -0.001180059937729890
211 1944.984 -0.001474258832851471
212 1952.407 -0.000839866126592593
213 1959.829 -0.000802114280937387
214 1967.252 -0.001185584956468647
215 1974.673 -0.000928373925971665
216 1982.095 -0.000796093377431769
217 1989.515 -0.000445132732861969
218 1996.935 -0.000282454611375082
219 2004.355 -0.000274540349301289
220 2011.775 -0.000188231187288176
221 2019.193 -0.000244587775762389
222 2026.612 -0.000164175820921038
223 2034.030 -0.000129539761708694
224 2041.447 -0.000079308880014316
225 2048.865 -0.000065495516761232
226 2056.281 -0.000047772383600339
227 2063.696 -0.000046003731905308
228 2071.112 -0.000070168273049554
229 2078.527 -0.000093521950338807
230 2085.942 -0.000110860062235196
231 2093.356 -0.000154363119462358
232 2100.769 -0.000255881943941033
233 2108.182 -0.000523277565558211
234 2115.594 -0.001375186007124925
235 2123.006 -0.003719877579447772
236 2130.418 -0.009393931436509179
237 2137.829 -0.020910727897321599
238 2145.239 -0.039043649935313118
239 2152.648 -0.068566906252670715
240 2160.058 -0.097573402237012843
241 2167.467 -0.113142164077031565
242 2174.875 -0.099651068695073369
243 2182.283 -0.067035094053992911
244 2189.690 -0.053604723388108656
245 2197.097 -0.329089281850986182
246 2204.503 -0.456604549736337129
247 2211.909 -0.142098157650880630
248 2219.315 -0.221827819797885795
249 2226.719 -0.322910943255505289
250 2234.123 -0.473408806585640218
251 2241.527 -0.572844899580387867
252 2248.930 -0.668838327700018942
253 2256.333 -0.730652529686325547
254 2263.735 -0.757765945995275580
255 2271.137 -0.669960748226233860
256 2278.538 -0.683494057865527971
257 2285.939 -0.804684649946742936
258 2293.339 -1.070588085146407797
259 2300.738 -1.161203428970881246
260 2308.136 -0.608511216055738879
261 2315.534 -0.621358606897385535
262 2322.933 -0.933162539142736835
263 2330.330 -0.736720258289056851
264 2337.726 -1.091126161372567349
265 2345.122 -1.453405372049445088
266 2352.517 -1.231725418739371136
267 2359.913 -0.657273677359269226
268 2367.307 -0.770079200631794691
269 2374.701 -1.199089582930701114
270 2382.093 -0.681602805457481642
271 2389.486 -0.674950280268280189
272 2396.878 -0.662488986727583384
273 2404.270 -0.513653895860329612
274 2411.660 -0.298528928634253010
275 2419.051 -0.303678622373404872
276 2426.440 -0.348436616387659437
277 2433.830 -0.212319449968018131
278 2441.218 -0.166568524018598063
279 2448.606 -0.154442898524042083
280 2455.994 -0.114423875787858245
281 2463.382 -0.077059434305238886
282 2470.768 -0.060319040795876841
283 2478.153 -0.037668219548523431
284 2485.539 -0.029877843993233233
285 2492.924 -0.034757379003344489
================================================
FILE: data/methane_tutorial/methane_inout_points.csv
================================================
ID,in-plume,latitude,longitude
0,1,39.4628,53.7743
1,0,39.4697,53.7792
2,0,39.4726,53.7797
================================================
FILE: data/methane_tutorial/plume_bbox.geojson
================================================
{
"type": "FeatureCollection",
"crs": { "type": "name", "properties": { "name": "urn:ogc:def:crs:OGC:1.3:CRS84" } },
"features": [
{ "type": "Feature", "properties": { "name": "plume_bbox" }, "geometry": { "type": "Polygon", "coordinates": [ [ [ 36.1075296, 31.8092048 ], [ 36.2805017, 31.8092048 ], [ 36.2805017, 32.1274953 ], [ 36.1075296, 32.1274953 ], [ 36.1075296, 31.8092048 ] ] ] } }
]
}
================================================
FILE: data/methane_tutorial/rad_band_ratio_points.csv
================================================
ID,in-plume,x,y
0,0,508,421
1,1,508,420
2,1,508,419
3,0,508,418
4,0,508,417
5,0,508,416
6,0,508,415
7,0,508,414
================================================
FILE: data/min_uncert_urls.txt
================================================
https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/EMITL2BMIN.001/EMIT_L2B_MIN_001_20230427T173309_2311711_010/EMIT_L2B_MINUNCERT_001_20230427T173309_2311711_010.nc
https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/EMITL2BMIN.001/EMIT_L2B_MIN_001_20230804T191650_2321613_007/EMIT_L2B_MINUNCERT_001_20230804T191650_2321613_007.nc
https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/EMITL2BMIN.001/EMIT_L2B_MIN_001_20230808T173953_2322012_011/EMIT_L2B_MINUNCERT_001_20230808T173953_2322012_011.nc
================================================
FILE: data/mineral_grouping_matrix_20230503.csv
================================================
Index,Record,Filename,Name,URL,Group,Library,Calcite,Chlorite,Dolomite,Goethite,Gypsum,Hematite,Illite+Muscovite,Kaolinite,Montmorillonite,Vermiculite,Iron Oxide,Clay,Carbonate,Sulfate,Sulfide,Artifical Materials,Vegetation,Coating,Biotite
1,6858,group.1um/organic_green_plastic_tarp_1um,Plastic_Tarp GDS339 Green W1R1Fa,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Plastic_Tarp_GDS339_Green_ASDFRa_AREF.html,1,splib06,,,,,,,,,,,,,,,,-1,,,
2,744,group.1um/fe3+_hematite.fine.gr.gds76,Hematite.02+Quartz.98 GDS76 W1R1Ha,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Hematite.02+Quartz.98_GDS76_BECKa_AREF.html,1,sprlb06,,,,,,0.02,,,,,,,,,,,,,
3,882,group.1um/fe3+_goethite.medgr.ws222,Goethite WS222 <250um MedGrn W1R1H_ AREF,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Goethite_WS222_Medium_Gr._BECKa_AREF.html,1,sprlb06,,,,1,,,,,,,,,,,,,,,
4,5736,group.1um/fe3+_goethite+qtz.medgr.gds240,Goethite0.02+Quartz GDS240 W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Goethite0.02+Quartz_GDS240_BECKa_AREF.html,1,splib06,,,,0.02,,,,,,,,,,,,,,,
5,1878,group.1um/fe3+_goethite.fingr,Goethite MPCMA2-B FineGr adj W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Goethite_MPCMA2-B_FineGr_adj_BECKb_AREF.html,1,splib06,,,,0.2,,,,,,,,,,,,,,,
6,852,group.1um/fe3+_goethite.medcoarsegr.mpc.trjar,Goethite MPCMA2-C M-Crsgrad2 W1R1Hb AREF,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Goethite_MPCMA2-C_M-Crsgrad2_BECKb_AREF.html,1,sprlb06,,,,0.37,,,,,,,,,,,,,,,
7,894,group.1um/fe3+_goethite.coarsegr,Goethite WS222 coarse grain W1R1H_,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Goethite_WS222_Coarse_Gr._BECKa_AREF.html,1,sprlb06,,,,1,,,,,,,,,,,,,,,
8,6168,group.1um/fe3+_goethite.thincoat,Goethite_Thin_Film WS222 W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Goethite_Thin_Film_WS222_BECKa_AREF.html,1,splib06,,,,0.1,,,,,,,,,,,,,,,
9,5730,group.1um/fe3+_goeth+jarosite,Goeth+qtz.5+Jarosite.5 AMX11 W1R1BbS,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Goeth+qtz.5+Jarosite.5_AMX11_BECKb_AREF.html,1,splib06,,,,0.02,,,,,,,,,,,,,,,
10,6042,group.1um/sulfide_pyrite,Pyrite LV95-6A Weath on Tail W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Pyrite_LV95-6A_Weath_on_Tail_BECKb_AREF.html,1,splib06,,,,,,,,,,,,,,,-1,,,,
11,4476,group.1um/fe3+_sulfate_schwertmannite,Schwertmannite BZ93-1 W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Schwertmannite_BZ93-1_BECKb_AREF.html,1,splib06,,,,,,,,,,,-1,,,-1,,,,,
12,6144,group.1um/Mn-Coating,Blck_Mn_Coat_Tailngs LV95-3 W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Blck_Mn_Coat_Tailngs_LV95-3_BECKb_AREF.html,1,splib06,,,,,,,,,,,,,,,,,,-1,
13,5316,group.1um/fe3+mix_AMD.assemb1,Acid_Mine_Dr Assemb1-Fe3+ W1R1Fb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Acid_Mine_Dr_Assemb1-Fe3+_ASDFRb_AREF.html,1,splib06,,,,,,,,,,,-1,,,,,,,,
14,5322,group.1um/fe3+mix_AMD.assemb2,Acid_Mine_Dr Assemb2-Fe3+ W1R1Fb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Acid_Mine_Dr_Assemb2-Fe3+_ASDFRb_AREF.html,1,splib06,,,,,,,,,,,-1,,,,,,,,
15,2568,group.1um/fe3+_sulfate_kjarosite200,Jarosite GDS99 K 200C Syn W1R1BaS,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Jarosite_GDS99_K_200C_Syn_BECKa_AREF.html,1,splib06,,,,,,,,,,,-1,,,-1,,,,,
16,732,group.1um/fe3+_goethite.lepidocrosite,Lepidocrosite GDS80 (Syn) W1R1Hb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Lepidocrocite_GDS80_(Syn)_BECKb_AREF.html,1,sprlb06,,,,1,,,,,,,,,,,,,,,
17,5634,group.1um/fe2+_chlor+muscphy,Chlorite+Muscovite CU93-65A W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Chlorite+Muscovite_CU93-65A_BECKa_AREF.html,1,splib06,,,,,,,,,,,-1,,,,,,,,
18,900,group.1um/fe2+_goeth+musc,Goethite CU91-252 coatedchip W1R1H_,No available USGS Spectral Library 7 Description Reference,1,sprlb06,,,,0.04,,,,,,,,,,,,,,,
19,5604,group.1um/fe2+fe3+_chlor+goeth.propylzone,Chlor+Goethite CU93-4B Phyl W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Chlor+Goethite_CU93-4B_Phyl_BECKa_AREF.html,1,splib06,,,,0.02,,,,,,,,,,,,,,,
20,1446,group.1um/fe2+generic_nrw.cummingtonite,Cummingtonite HS294.3B W1R1Bc,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Cummingtonite_HS294.3B_BECKc_AREF.html,1,splib06,,,,,,,,,,,-1,,,,,,,,
21,72,group.1um/fe2+generic_nrw.actinolite,Actinolite NMNHR16485 W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Actinolite_NMNHR16485_BECKb_AREF.html,1,splib06,,,,,,,,,,,-1,,,,,,,,
22,42,group.1um/fe2+generic_nrw.hs-actinolite,Actinolite HS22.3B W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Actinolite_HS22.3B_BECKb_AREF.html,1,splib06,,,,,,,,,,,-1,,,,,,,,
23,2334,group.1um/fe2+_pyroxene.hypersthene,Hypersthene NMNHC2368 W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Hypersthene_NMNHC2368_BECKb_AREF.html,1,splib06,,,,,,,,,,,-1,,,,,,,,
24,1548,group.1um/fe2+_pyroxene.diopside,Diopside NMNHR18685 ~160 Pyx W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Diopside_NMNHR18685_~160_Pyx_BECKb_AREF.html,1,splib06,,,,,,,,,,,-1,,,,,,,,
25,2502,group.1um/fe2+generic_med.jadeite,Jadeite HS343.3B W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Jadeite_HS343.3B_BECKb_AREF.html,1,splib06,,,,,,,,,,,-1,,,,,,,,
26,3966,group.1um/fe2+_pyroxene_clino_pigeonite,Pigeonite HS199.3B W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Pigeonite_HS199.3B_BECKc_AREF.html,1,splib06,,,,,,,,,,,-1,,,,,,,,
27,1668,group.1um/epidote,Epidote GDS26.a 75-200um W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Epidote_GDS26.a_75-200um_BECKb_AREF.html,1,splib06,,,,,,,,,,,-1,,,,,,,,
28,5724,group.1um/fe3+bearing1,Fe-Hydroxide SU93-106 amorph W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Fe-Hydroxide_SU93-106_amorph_BECKb_AREF.html,1,splib06,,,,0.2,,,,,,,,,,,,,,,
29,1740,group.1um/fe3+_ferrihydrite,Ferrihydrite GDS75 Syn F6 W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Ferrihydrite_GDS75_Syn_F6_BECKb_AREF.html,1,splib06,,,,,,,,,,,-1,,,,,,,,
30,738,group.1um/fe3+_hematite.med.gr.gds27,Hematite GDS27 W1R1Ha,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Hematite_GDS27_BECKa_AREF.html,1,sprlb06,,,,,,1,,,,,,,,,,,,,
31,6198,group.1um/fe3+_hematite.thincoat,Hematite_Thin_Film GDS27 W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Hematite_Thin_Film_GDS27_BECKa_AREF.html,1,splib06,,,,,,0.05,,,,,,,,,,,,,
32,2100,group.1um/fe3+_hematite.fine.gr.fe2602,Hematite FE2602 W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Hematite_FE2602_BECKb_AREF.html,1,splib06,,,,,,1,,,,,,,,,,,,,
33,906,group.1um/fe3+_hematite.fine.gr.ws161,Hematite WS161 W1R1Hb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Hematite_WS161_BECKb_AREF.html,1,sprlb06,,,,,,0.79,,,,,,,,,,,,,
34,3558,group.1um/fe3+_smectite_nontronite,Nontronite NG-1.a W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Nontronite_NG-1.a_BECKb_AREF.html,1,splib06,,,,,,,,,,,-1,,,,,,,,
35,5328,group.1um/fe2+generic_brd.br5a_actinolite,Actinolite-Hornfels BR93-5a W1R1Bc,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Actinolite-Hornfels_BR93-5a_BECKc_AREF.html,1,splib06,,,,,,,,,,,-1,,,,,,,,
36,846,group.1um/fe2+fe3+mix_with_hematite_br5b,Magnetite_skarn BR93-5B W1R1Hb AREF,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Magnetite_skarn_BR93-5B_BECKb_AREF.html,1,sprlb06,,,,,,0.17,,,,,,,,,,,,,
37,5334,group.1um/fe2+generic_brd.br22c_actinolite,Actinolite-Tremolit BR93-22C W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Actinolite-Tremolit_BR93-22C_BECKb_AREF.html,1,splib06,,,,,,,,,,,-1,,,,,,,,
38,1656,group.1um/fe2+generic_br33a_bioqtzmonz_epidote,Epidote BR93-33a W1R1Bb AREF,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Epidote_BR93-33a_BECKb_AREF.html,1,splib06,,,,,,,,,,,-1,,,,,,,,
39,5490,group.1um/fe2+generic_brd.br36a_chlorite,Biotite-Chlorite_Mx BR93-36A W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Biotite-Chlorite_Mx_BR93-36A_BECKb_AREF.html,1,splib06,,,,,,,,,,,-1,,,,,,,,
40,5460,group.1um/fe2+fe3+_hematite_weathering,Basalt_weathered BR93-43 W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Basalt_weathered_BR93-43_BECKb_AREF.html,1,splib06,,,,,,0.01,,,,,,,,,,,,,
41,5454,group.1um/fe2+generic_basalt_br46b,Basalt_fresh BR93-46B W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Basalt_fresh_BR93-46B_BECKb_AREF.html,1,splib06,,,,,,,,,,,-1,,,,,,,,
42,840,group.1um/fe3+_hematite.lg.gr.br25a,Hematite_Coatd_Qtzt BR93-25A W1R1Hb AREF,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Hematite_Coatd_Qtzt_BR93-25A_BECKa_AREF.html,1,sprlb06,,,,,,0.01,,,,,,,,,,,,,
43,6174,group.1um/fe3+_hematite.med.gr.br25b,Hematite_Coatd_Qtz BR93-25B W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Hematite_Coatd_Qtz_BR93-25B_BECKa_AREF.html,1,splib06,,,,,,0.05,,,,,,,,,,,,,
44,816,group.1um/fe3+_hematite.lg.gr.br25c,Hematite_Coatd_Qtzt BR93-25C W1R1Hb AREF,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Hematite_Coatd_Qtzt_BR93-25C_BECKa_AREF.html,1,sprlb06,,,,,,0.69,,,,,,,,,,,,,
45,3468,group.1um/fe3+_hematite.nano.BR34b2,Nanohematite BR93-34B2 W1R1BbS,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Nanohematite_BR93-34B2_BECKb_AREF.html,1,splib06,,,,,,0.1,,,,,,,,,,,,,
46,1026,group.1um/fe3+_hematite-nano+goethite-fg,nHematit+fg-Goethit 34B2+MPC W1R1Hb,No available USGS Spectral Library 7 Description Reference,1,sprlb06,,,,0.1,,0.05,,,,,,,,,,,,,
47,918,group.1um/fe3+_hematite.nano.BR34b2b,Nanohematite FBR93-34B2b ed1 W1R1Hb,No available USGS Spectral Library 7 Description Reference,1,sprlb06,,,,,,0.1,,,,,,,,,,,,,
48,822,group.1um/fe3+_hematite.lg.gr.br34c,Hematite_Coatd_Qtz BR93-34C W1R1Ha AREF,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Hematite_Coatd_Qtz_BR93-34C_BECKa_AREF.html,1,sprlb06,,,,,,0.63,,,,,,,,,,,,,
49,6210,group.1um/fe3+_sulfate_jarosite_br34a2,Jarosite_on_Qtzite BR93-34A2 W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Jarosite_on_Qtzite_BR93-34A2_BECKa_AREF.html,1,splib06,,,,,,,,,,,-1,,,,,,,,
50,5340,group.1um/fe2+generic_broad_br60b,Actinolite_Dolomit BR93-60B W1R1BbS,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Actinolite_Dolomit_BR93-60B_BECKb_AREF.html,1,splib06,,,,,,,,,,,-1,,,,,,,,
51,6036,group.1um/fe2+generic_vbroad_br20,Phlogopite_Sand_Mix BR93-20 W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Phlogopite_Sand_Mix_BR93-20_BECKa_AREF.html,1,splib06,,,,,,,,,,,-1,,,,,,,,
52,6942,group.1um/fe2+fe3+_water_RTsludge,Renyolds_TnlSldgWet SM93-15w W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Renyolds_TnlSldgWet_SM93-15w_BECKa_AREF.html,1,splib06,,,,,,,,,,,-1,,,,,,,,
53,660,group.1um/fe2+_chlorite.clinochlor,Clinochlore GDS158 Flagst W1R1Hb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Clinochlore_GDS158_Flagstaff_ASDNGb_AREF.html,1,sprlb06,,,,,,,,,,,,,,,,,,,
54,1200,group.1um/fe2+_chlorite.Felow.clinochlor,Clinochlore NMNH83369 W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Clinochlore_NMNH83369_BECKb_AREF.html,1,splib06,,,,,,,,,,,-1,,,,,,,,
55,4890,group.1um/fe2+_chlorite.thuringite,Thuringite SMR-15.c 32um W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Thuringite_SMR-15.c_32um_NIC4aa_RREF.html,1,splib06,,,,,,,,,,,-1,,,,,,,,
56,720,group.1um/copper_carbonate_azurite,Azurite WS316 W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Azurite_WS316_BECKa_AREF.html,1,splib06,,,,,,,,,,,,,,,,,,,
57,3990,group.1um/fe3+copper-hydroxide_pitchlimonite,Pitch_Limonite GDS104 Cu W1R1B?,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Pitch_Limonite_GDS104_Cu_BECKu_AREF.html,1,splib06,,,,,,,,,,,-1,,,,,,,,
58,2964,group.1um/copper_carbonate_malachite,Malachite HS254.3B W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Malachite_HS254.3B_BECKb_AREF.html,1,splib06,,,,,,,,,,,,,,,,,,,
59,5496,group.1um/copper_sulfate_bluefflor,Blue_Efflorscnt_Min SU93-300 W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Blue_Efflorscnt_Min_SU93-300_BECKb_AREF.html,1,splib06,,,,,,,,,,,,,,-1,,,,,
60,6594,group.1um/copper_precipitate_greenslime,Green_Slime SM93-14A Summitv W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Green_Slime_SM93-14A_Summitv_BECKa_AREF.html,1,splib06,,,,,,,,,,,,,,,,,,,
61,3696,group.1um/fe2+_olivine-lrg-gr,Olivine HS285.4B Fo80 W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Olivine_HS285.4B_Fo80_BECKb_AREF.html,1,splib06,,,,,,,,,,,-1,,,,,,,,
62,3666,group.1um/fe2+_olivine-fine-gr,Olivine GDS71.a Fo91 65um W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Olivine_GDS71.a_Fo91_65um_BECKb_AREF.html,1,splib06,,,,,,,,,,,-1,,,,,,,,
63,4278,group.1um/carbonate_rhodochrosite,Rhodochrosite HS338.3B W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Rhodochrosite_HS338.3B_BECKb_AREF.html,1,splib06,,,,,,,,,,,,,,,,,,,
64,4560,group.1um/fe2+generic_carbonate_siderite1,Siderite HS271.3B W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Siderite_HS271.3B_BECKa_AREF.html,1,splib06,,,,,,,,,,,-1,,,,,,,,
65,4362,group.1um/fe2+generic_amphibole_riebeckite,Riebeckite NMNH122689 Amph W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Riebeckite_NMNH122689_Amph_BECKa_AREF.html,1,splib06,,,,,,,,,,,-1,,,,,,,,
66,1386,group.1um/fe3+fe2+_sulfate_coquimbite,Coquimbite GDS22 W1R1Bc,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Coquimbite_GDS22_BECKc_AREF.html,1,splib06,,,,,,,,,,,-1,,,-1,,,,,
67,1374,group.1um/fe3+fe2+_sulfate_copiapite,Copiapite GDS21 W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Copiapite_GDS21_BECKb_AREF.html,1,splib06,,,,,,,,,,,-1,,,-1,,,,,
68,1134,group.1um/copper_chrysocolla,Chrysocolla HS297.3B W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Chrysocolla_HS297.3B_ASDFRb_AREF.html,1,splib06,,,,,,,,,,,,,,,,,,,
69,3498,group.1um/ree_neodymium_oxide,Neodymium_Oxide GDS34 W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Neodymium_Oxide_GDS34_BECKa_AREF.html,1,splib06,,,,,,,,,,,,,,,,,,,
70,4410,group.1um/ree_samarium_oxide,Samarium_Oxide GDS36 W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Samarium_Oxide_GDS36_BECKa_AREF.html,1,splib06,,,,,,,,,,,,,,,,,,,
71,708,group.1um/fe2+generic_axinite,Axinite HS342.3B W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Axinite_HS342.3B_BECKb_AREF.html,1,splib06,,,,,,,,,,,-1,,,,,,,,
72,4728,group.1um/fe2+generic_staurolite,Staurolite HS188.3B W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Staurolite_HS188.3B_BECKc_AREF.html,1,splib06,,,,,,,,,,,-1,,,,,,,,
73,216,group.1um/fe2+generic_almandine,Almandine WS479 Garnet W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Almandine_WS479_Garnet_BECKb_AREF.html,1,splib06,,,,,,,,,,,-1,,,,,,,,
74,678,group.1um/fe2+_pyroxene_augite,Augite NMNH120049 W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Augite_NMNH120049_BECKb_AREF.html,1,splib06,,,,,,,,,,,-1,,,,,,,,
75,810,group.1um/fe2+_pyroxene.bronzite,Bronzite HS9.3B Pyroxene W1R1Bc,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Bronzite_HS9.3B_Pyroxene_BECKc_AREF.html,1,splib06,,,,,,,,,,,-1,,,,,,,,
76,858,group.1um/fe2+generic_sulfate_butlerite,Butlerite GDS25 W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Butlerite_GDS25_BECKa_AREF.html,1,splib06,,,,,,,,,,,-1,,,,,,,,
77,882,group.1um/fe2+_feldspar.bytownite,Bytownite HS106.3B Plagio W1R1Bc,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Bytownite_HS106.3B_Plagio_BECKc_AREF.html,1,splib06,,,,,,,,,,,-1,,,,,,,,
78,1002,group.1um/sulfide_copper_chalcopyrite,Chalcopyrite HS431.3B W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Chalcopyrite_HS431.3B_BECKb_AREF.html,1,splib06,,,,,,,,,,,,,,,-1,,,,
79,1122,group.1um/fe2+_chromite,Chromite HS281.3B W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Chromite_HS281.3B_ASDFRc_AREF.html,1,splib06,,,,,,,,,,,-1,,,,,,,,
80,1164,group.1um/sulfide_cinnabar,Cinnabar HS133.3B W1R1Bc,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Cinnabar_HS133.3B_BECKc_AREF.html,1,splib06,,,,,,,,,,,,,,,-1,,,,
81,1458,group.1um/copper_oxide_cuprite,Cuprite HS127.3B W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Cuprite_HS127.3B_BECKb_AREF.html,1,splib06,,,,,,,,,,,,,,,,,,,
82,6156,group.1um/fe3+mn_desert.varnish1,Desert_Varnish GDS141 W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Desert_Varnish_GDS141_BECKa_AREF.html,1,splib06,,,,,,,,,,,-1,,,,,,,-1,
83,6162,group.1um/fe3+mn_desert.varnish2,Desert_Varnish GDS78A Rhy W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Desert_Varnish_GDS78A_Rhy_BECKa_AREF.html,1,splib06,,,,,,,,,,,-1,,,,,,,-1,
84,1644,group.1um/fe2+_pyroxene_enstatite,Enstatite NMNH128288 W1R1Bc,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Enstatite_NMNH128288_BECKc_AREF.html,1,splib06,,,,,,,,,,,-1,,,,,,,,
85,2088,group.1um/fe2+_pyroxene_hedenbergite,Hedenbergite NMNH119197 W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Hedenbergite_NMNH119197_BECKb_AREF.html,1,splib06,,,,,,,,,,,-1,,,,,,,,
86,2796,group.1um/sulfate_lazurite,Lazurite HS418.3B W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Lazurite_HS418.3B_BECKb_AREF.html,1,splib06,,,,,,,,,,,,,,-1,,,,,
87,2928,group.1um/fe3+_maghemite,Maghemite GDS81 Syn (M-3) W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Maghemite_GDS81_Syn_(M-3)_BECKb_AREF.html,1,splib06,,,,,,0.85,,,,,,,,,,,,,
88,2940,group.1um/magnetite,Magnetite HS195.3B W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Magnetite_HS195.3B_BECKb_AREF.html,1,splib06,,,,,,,,,,,-1,,,,,,,,
89,4158,group.1um/mn_oxide_pyrolusite,Pyrolusite HS138.3B W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Pyrolusite_HS138.3B_BECKb_AREF.html,1,splib06,,,,,,,,,,,,,,,,,,,
90,3042,group.1um/fe2+_feldspar_microcline,Microcline HS103.3B Feldspar W1R1Bc,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Microcline_HS103.3B_Feldspar_BECKc_AREF.html,1,splib06,,,,,,,,,,,-1,,,,,,,,
91,132,group.1um/fe2+_feldspar_albite,Albite HS143.3B Plagioclase W1R1Bc,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Albite_HS143.3B_Plagioclase_BECKc_AREF.html,1,splib06,,,,,,,,,,,-1,,,,,,,,
92,3852,group.1um/fe2+_feldspar_orthoclase,Orthoclase NMNH142137 Fe W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Orthoclase_NMNH142137_Fe_BECKb_AREF.html,1,splib06,,,,,,,,,,,-1,,,,,,,,
93,3882,group.1um/inosilicate_pectolite,Pectolite NMNH94865.a W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Pectolite_NMNH94865.a_BECKb_AREF.html,1,splib06,,,,,,,,,,,,,,,,,,,
94,4314,group.1um/mn2+_rhodonite,Rhodonite NMNHC6148 >250um W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Rhodonite_NMNHC6148_gt250um_BECKb_AREF.html,1,splib06,,,,,,,,,,,,,,,,,,,
95,4776,group.1um/sulfur,Sulfur GDS94 Reagent W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Sulfur_GDS94_Reagent_BECKa_AREF.html,1,splib06,,,,,,,,,,,,,,,,,,,
96,7128,group.2um/organic_vegetation-dry-grass-long,Dry_Long_Grass AV87-2 W1R1Ba,No available USGS Spectral Library 7 Description Reference,2,splib06,,,,,,,,,,,,,,,,,-1,,
97,7248,group.2um/organic_vegetation-dry-grass-golden,Grass_Golden_Dry GDS480 W1R1Fa,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Grass_Golden_Dry_GDS480_ASDFRa_AREF.html,2,splib06,,,,,,,,,,,,,,,,,-1,,
98,6912,group.2um/organic_vegetation-dry-wood,Plywood GDS365 Fresh Pine W1R1Fa,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Plywood_GDS365_Fresh_Pine_ASDFRa_AREF.html,2,splib06,,,,,,,,,,,,,,,,,-1,,
99,6864,group.2um/organic_plastic-tarp1,Plastic_Tarp GDS340 Blu_Wovn W1R1Fa,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Plastic_Tarp_GDS340_Blu_Wovn_ASDFRa_AREF.html,2,splib06,,,,,,,,,,,,,,,,-1,,,
100,6876,group.2um/organic_plastic-vinyl,Plastic_Vinyl GDS372 White W1R1Fa,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Plastic_Vinyl_GDS372_White_ASDFRa_AREF.html,2,splib06,,,,,,,,,,,,,,,,-1,,,
101,6558,group.2um/organic_fiberglass-roofing,Fiberglass GDS337 Grn Roofng W1R1Fa,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Fiberglass_GDS337_Grn_Roofng_ASDFRa_AREF.html,2,splib06,,,,,,,,,,,,,,,,,,,
102,6936,group.2um/organic_plastic-polystyrene,Polystyrene GDS345 BluInsul W1R1Fa,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Polystyrene_GDS345_BluInsul_ASDFRa_AREF.html,2,splib06,,,,,,,,,,,,,,,,-1,,,
103,6816,group.2um/organic_plastic-pete,Plastic_PETE GDS380 Clear W1R1Fa,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Plastic_PETE_GDS380_Clear_ASDFRa_AREF.html,2,splib06,,,,,,,,,,,,,,,,-1,,,
104,6810,group.2um/organic_plastic-pete2,Plastic_PETE GDS379 TrnslBrn W1R1Fa,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Plastic_PETE_GDS379_TrnslBrn_ASDFRa_AREF.html,2,splib06,,,,,,,,,,,,,,,,-1,,,
105,6714,group.2um/organic_plastic-hdpe.1,Plastic_HDPE GDS393 GlosWhTr W1R1Fa,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Plastic_HDPE_GDS393_GlosWhTr_ASDFRa_AREF.html,2,splib06,,,,,,,,,,,,,,,,-1,,,
106,6672,group.2um/organic_paint-1,Painted_Aluminum GDS333 LgGr W1R1Fa,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Painted_Aluminum_GDS333_LgGr_ASDFRa_AREF.html,2,splib06,,,,,,,,,,,,,,,,-1,,,
107,6834,group.2um/organic_plastic-pete3,Plastic_PETE GDS383 Clrbluis W1R1Fa,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Plastic_PETE_GDS383_Clrbluis_ASDFRa_AREF.html,2,splib06,,,,,,,,,,,,,,,,-1,,,
108,6810,group.2um/organic_plastic-pete4,Plastic_PETE GDS379 TrnslBrn W1R1Fa,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Plastic_PETE_GDS379_TrnslBrn_ASDFRa_AREF.html,2,splib06,,,,,,,,,,,,,,,,-1,,,
109,300,group.2um/sulfate_kalun150c,Alunite GDS97 K Syn (150C) W2R4Na,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Alunite_GDS97_K_Syn_(150C)_NIC4a_RREF.html,2,splib06,,,,,,,,,,,,,,-1,,,,,
110,294,group.2um/sulfate_kalun250c,Alunite GDS96 K Syn (250C) W2R4Na,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Alunite_GDS96_K_Syn_(250C)_NIC4a_RREF.html,2,splib06,,,,,,,,,,,,,,-1,,,,,
111,324,group.2um/sulfate_kalun450c,Alunite RES-2 K Syn (450C) W2R4Na,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Alunite_RES-2_K_Syn_(450C)_NIC4a_RREF.html,2,splib06,,,,,,,,,,,,,,-1,,,,,
112,288,group.2um/sulfate_naalun150c,Alunite GDS95 Na Syn (150C) W2R4Na,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Alunite_GDS95_Na_Syn_(150C)_NIC4a_RREF.html,2,splib06,,,,,,,,,,,,,,-1,,,,,
113,336,group.2um/sulfate_naalun300c,Alunite RES-4 Na Syn (300C) W2R4Na,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Alunite_RES-4_Na_Syn_(300C)_NIC4a_RREF.html,2,splib06,,,,,,,,,,,,,,-1,,,,,
114,330,group.2um/sulfate_naalun450c,Alunite RES-3 Na Syn (450C) W2R4Na,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Alunite_RES-3_Na_Syn_(450C)_NIC4a_RREF.html,2,splib06,,,,,,,,,,,,,,-1,,,,,
115,246,group.2um/sulfate_na82alun100c,Alunite GDS82 Na82 W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Alunite_GDS82_Na82_BECKb_AREF.html,2,splib06,,,,,,,,,,,,,,-1,,,,,
116,5382,group.2um/sulfate+kaolingrp_natroalun+dickite,Alunite+Dickite MV99-6-26b W1R1Fc,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Alunite+Dickite_MV99-6-26b_ASDFRc_AREF.html,2,splib06,,,,,,,,0.485,,,,,,-1,,,,,
117,252,group.2um/sulfate_na63alun300c,Alunite GDS83 Na63 W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Alunite_GDS83_Na63_BECKb_AREF.html,2,splib06,,,,,,,,,,,,,,-1,,,,,
118,342,group.2um/sulfate_na40alun400c,Alunite RES-9 Summitv (400C) W2R4Nb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Alunite_RES-9_Summitv_(400C)_ASDNGb_AREF.html,2,splib06,,,,,,,,,,,,,,-1,,,,,
119,276,group.2um/sulfate_alunNa03,Alunite GDS84 Na03 W2R4Na,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Alunite_GDS84_Na03_BECKa_AREF.html,2,splib06,,,,,,,,,,,,,,-1,,,,,
120,348,group.2um/sulfate_alunNa56450c,Alunite RES10 NA56% Syn 450C W2R4Na,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Alunite_RES10_NA56percent_Syn_450C_NIC4a_RREF.html,2,splib06,,,,,,,,,,,,,,-1,,,,,
121,354,group.2um/sulfate_alunNa78.450c,Alunite RES12 NA78% Syn 450C W2R4NaIS,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Alunite_RES12_NA78percent_Syn_450C_NIC4a_RREF.html,2,splib06,,,,,,,,,,,,,,-1,,,,,
122,5394,group.2um/sulfate_alun35K65Na.low,Alunite0.35K+.65Na CU91-217H W2R4Nb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Alunite0.35K+.65Na_CU91-217H_ASDFRb_AREF.html,2,splib06,,,,,,,,,,,,,,-1,,,,,
123,5418,group.2um/sulfate_alun73K27Na.low,Alunite0.73K+.27Na CU91-217D W2R4Na,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Alunite0.73K+.27Na_CU91-217D_NIC4a_RREF.html,2,splib06,,,,,,,,,,,,,,-1,,,,,
124,5412,group.2um/sulfate_alun66K34Na.low,Alunite0.66K+.34Na CU91-217A W2R4Nb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Alunite0.66K+.34Na_CU91-217A_NIC4b_RREF.html,2,splib06,,,,,,,,,,,,,,-1,,,,,
125,5976,group.2um/micagrp_muscovite-med-Al,Muscovite CU91-250A med Al W2R4NbS,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Alunite.5+MuscCU91-250A_AMX4_BECKa_AREF.html,2,splib06,,,,,,,0.59,,,,,,,,,,,,
126,1038,group.2um/calcite.33+kaol.33+mus-AMXr2,Calcite.33+Kaol.33+Mus AMXr2 W1R1Ba,No available USGS Spectral Library 7 Description Reference,2,sprlb06,0.33,,,,,,0.33,0.33,,,,,,,,,,,
127,654,group.2um/micagrp_muscovite-medhigh-Al,Muscovite GDS113 Ruby W1R1Bb splib06 3348,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Muscovite_GDS113_Ruby_ASDNGa_AREF.html,2,sprlb06,,,,,,,1,,,,,,,,,,,,
128,3876,group.2um/micagrp_paragonite,Paragonite GDS109 W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Paragonite_GDS109_BECKa_AREF.html,2,splib06,,,,,,,,,,,,,,,,,,,-1
129,714,group.2um/micagrp_illite,Illite IMt-1.b <2um W1R1BaS,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Illite_IMt-1.b_lt2um_ASDNGa_AREF.html,2,sprlb06,,,,,,,0.99,,,,,,,,,,,,
130,726,group.2um/micagrp_illite.gds4,Illite GDS4.2 Marblehead_WI W1R1H_,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Illite_GDS4.2_Marblehead_ASDNGb_AREF.html,2,sprlb06,,,,,,,0.76,,,,,,,,,,,,
131,3306,group.2um/micagrp_muscovite-low-Al,Muscovite CU93-1 low-Al Phyl W2R4Nb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Muscovite_CU93-1_low-Al_Phyl_NIC4b_RREF.html,2,splib06,,,,,,,0.22,,,,,,,,,,,,
132,3372,group.2um/micagrp_muscoviteFerich,Muscovite GDS116 Tanzania W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Muscovite_GDS116_Tanzania_BECKa_AREF.html,2,splib06,,,,,,,1,,,,,,,,,,,,
133,606,group.2um/kaolgrp_kaolinite_wxl,Kaolinite CM9 (wxl) W1R1Hb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Kaolinite_CM9_ASDNGb_AREF.html,2,sprlb06,,,,,,,,0.97,,,,,,,,,,,
134,672,group.2um/kaolgrp_kaolinite_pxl,Kaolinite KGa-2 (pxl) W1R1Hb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Kaolinite_KGa-2_(pxl)_ASDNGb_AREF.html,2,sprlb06,,,,,,,,1,,,,,,,,,,,
135,684,group.2um/kaolgrp_halloysite,Halloysite NMNH106237 W1R1Ha,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Halloysite_NMNH106237_ASDNGa_AREF.html,2,sprlb06,,,,,,,,0.99,,,,,,,,,,,
136,3462,group.2um/kaolgrp_nacrite,Nacrite GDS88 s06crj3a=c,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Nacrite_GDS88_BECKc_AREF.html,2,splib06,,,,,,,,-1,,,,,,,,,,,
137,708,group.2um/kaolgrp_dickite,Dickite NMNH106242 W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Dickite_NMNH106242_ASDNGb_AREF.html,2,sprlb06,,,,,,,,-1,,,,,,,,,,,
138,618,group.2um/sulfate_gypsum,Gypsum HS333.3B (Selenite) W1R1Ha,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Gypsum_HS333.3B_(Selenite)_ASDNGa_AREF.html,2,sprlb06,,,,,0.995,,,,,,,,,,,,,,
139,5868,group.2um/kaolin.5+smect.5,Kaolin_Smect H89-FR-2 .5Kaol W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Kaolin_Smect_H89-FR-2_.5Kaol_BECKb_AREF.html,2,splib06,,,,,,,,0.5,,,,,,,,,,,
140,5880,group.2um/kaolin.3+smect.7,Kaolin_Smect H89-FR-5 .3Kaol W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Kaolin_Smect_H89-FR-5_.3Kaol_BECKb_AREF.html,2,splib06,,,,,,,,0.3,,,,,,,,,,,
141,588,group.2um/smectite_montmorillonite_na_highswelling,Montmorillonite SWy-1 W1R1Hb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Montmorillonite_SWy-1_ASDNGb_AREF.html,2,sprlb06,,,,,,,,,0.84,,,,,,,,,,
142,54,group.2um/smectite_montmorillonite_fe_swelling,Montmorillonite_Fe GDS759A,No available USGS Spectral Library 7 Description Reference,2,sprlb06,,,,,,,,,0.83,,,,,,,,,,
143,600,group.2um/smectite_montmorillonite_ca_swelling,Montmorillonite SAz-1 W1R1Hb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Montmorillonite_SAz-1_ASDNGb_AREF.html,2,sprlb06,,,,,,,,,0.99,,,,,,,,,,
144,60,group.2um/sioh_hydrated_basaltic_glass,HyBasaltGlassMLUT03-9CMarP&T,No available USGS Spectral Library 7 Description Reference,2,sprlb06,,,,,,,,,,,,,,,,,,,
145,66,group.2um/sulfate_kieserite,Kieserite Batch4 GDS671,No available USGS Spectral Library 7 Description Reference,2,sprlb06,,,,,,,,,,,,,,-1,,,,,
146,150,group.2um/sulfate_szomolnokite,Szomolnokite NMNH104303 UT,No available USGS Spectral Library 7 Description Reference,2,sprlb06,,,,,,,,,,,,,,-1,,,,,
147,192,group.2um/sulfate_szomolnokite_heated,Szomolnokite GDS874 Heat275C,No available USGS Spectral Library 7 Description Reference,2,sprlb06,,,,,,,,,,,,,,-1,,,,,
148,96,group.2um/perchlorate_mg,Perchlorate_Mg GDS851.2 220K,No available USGS Spectral Library 7 Description Reference,2,sprlb06,,,,,,,,,,,,,,,,,,,
149,144,group.2um/perchlorate_na,Perchlorate_Na GDS857 230C r06crj3a=_,No available USGS Spectral Library 7 Description Reference,2,sprlb06,,,,,,,,,,,,,,,,,,,
150,5466,group.2um/smectite_beidellite_gds123,Beidellite+Montmor GDS123 =b,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Beidellite+Montmor_GDS123_ASDFRb_AREF.html,2,splib06,,,,,,,,,,,,,,,,,,,-1
151,5478,group.2um/smectite_beidellite_gds124,Beidellite+Montmor GDS124 =b,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Beidellite_GDS124_ASDFRb_AREF.html,2,splib06,,,,,,,,,,,,,,,,,,,-1
152,990,group.2um/sioh_chalcedony,Chalcedony CU91-6A W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Chalcedony_CU91-6A_BECKa_AREF.html,2,splib06,,,,,,,,,,,,,,,,,,,
153,1488,group.2um/hydroxide_diaspore,Diaspore HS416.3B W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Diaspore_HS416.3B_BECKb_AREF.html,2,splib06,,,,,,,,,,,,,,,,,,,
154,4188,group.2um/pyrophyllite,Pyrophyllite PYS1A fine gr W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Pyrophyllite_PYS1A_gt250um_ASDNGa_AREF.html,2,splib06,,,,,,,,,,,,,,,,,,,
155,6054,group.2um/kaol.75+pyroph.25,Pyrophyl.25+wxlKaol.75 AMX17,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Pyrophyl.25+wxlKaol.75_AMX17_BECKb_AREF.html,2,splib06,,,,,,,,0.73,,,,,,,,,,,
156,6060,group.2um/pyroph.5+mont0.5,Pyrophyl.50+Ca-Mont.50 AMX16 W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Pyrophyl.50+Ca-Mont.50_AMX16_BECKa_AREF.html,2,splib06,,,,,,,,,0.42,,,,,,,,,,
157,5358,group.2um/pyroph.5+alunit.5,Alun366+.50PyroPYS1A GDS222 W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Alun366+.50PyroPYS1A_GDS222_BECKb_AREF.html,2,splib06,,,,,,,,,,,,,,-1,,,,,
158,5370,group.2um/alunite+pyrophyl,Alunite+Pyrophyl SD1093A W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Alunite+Pyrophyl_SD1093A_BECKb_AREF.html,2,splib06,,,,,,,,,,,,,,-1,,,,,
159,6048,group.2um/pyroph+tr.musc,Pyrophyl+Muscovite JH_PYRM1 W1R1Fb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Pyrophyl+Muscovite_JH_PYRM1_ASDFRb_AREF.html,2,splib06,,,,,,,0.04,,,,,,,,,,,,
160,5988,group.2um/musc+pyroph,Muscovite+Pyrophyl JH_PYRP1 W1R1Fb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Muscovite+Pyrophyl_JH_PYRP1_ASDFRb_AREF.html,2,splib06,,,,,,,0.15,,,,,,,,,,,,
161,5352,group.2um/alunite+musc+pyroph,Alun0.3+Musc0.4+Pyro0.3 AMX1 W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Alun0.3+Musc0.4+Pyro0.3_AMX1_BECKa_AREF.html,2,splib06,,,,,,,0.4,,,,,,,,,,,,
162,5406,group.2um/alunite.5+kaol.5,Alunite0.5+Kaol_KGa-1 AMX3 W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Alunite0.5+Kaol_KGa-1_AMX3_BECKb_AREF.html,2,splib06,,,,,,,,0.475,,,,,,,,,,,
163,5910,group.2um/kaol.75+alun.25,Kaolwxl.75+Alun_HS295 AMX14 W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Kaolwxl.75+Alun_HS295_AMX14_BECKb_AREF.html,2,splib06,,,,,,,,0.73,,,,,,,,,,,
164,5820,group.2um/Kalun+kaol.intmx,Kalun+kaol+gth mv2-ar3 W1R1Fb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Kalun+kaol+gth_mv2-ar3_ASDFRb_AREF.html,2,splib06,,,,,,,,0.4,,,,,,,,,,,
165,5364,group.2um/Na-alun+kaol.intmx,Alun_Na+Kaol+Hemat MV00-11a W1R1Fc,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Alun_Na+Kaol+Hemat_MV00-11a_ASDFRc_AREF.html,2,splib06,,,,,,,,0.32,,,,,,,,,,,
166,5832,group.2um/kaolin.5+muscov.medAl,Kaol.5+MuscCU91-250A AMX13 W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Kaol.5+MuscCU91-250A_AMX13_BECKb_AREF.html,2,splib06,,,,,,,0.295,0.485,,,,,,,,,,,
167,5838,group.2um/kaolin.5+muscov.medhighAl,Kaol_Wxl+0.5Musc_Ruby AMX12 W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Kaol_Wxl+0.5Musc_Ruby_AMX12_BECKa_AREF.html,2,splib06,,,,,,,0.5,0.485,,,,,,,,,,,
168,870,group.2um/kaolin+musc.intimat,Kaol+Musc_intimate CU93-5C W1R1Ha AREF,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Kaol+Muscov_intimate_CU93-5C_ASDNGa_AREF.html,2,sprlb06,,,,,,,0.1,0.485,,,,,,,,,,,
169,876,group.2um/musc+jarosite.intimat,Muscov+Jaros CU93-314 coatng W1R1Hb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Muscov+Jaros_CU93-314_coatng_ASDNGb_AREF.html,2,sprlb06,,,,,,,0.27,,,,,,,,,,,,
170,5748,group.2um/sulfate-mix_gypsum+jar+illite.intmix,Gyp+jar+ill BRCM1 Marysvale W1R1Fb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Gyp+jar+ill_BRCM1_Marysvale_ASDFRb_AREF.html,2,splib06,,,,,0.33,,0.33,,,,,,,,,,,,
171,5766,group.2um/sulfate-mix_gyp+jar+musc.amix,Gyp.5+Kjar.3+hAlmsc.2 AMX24 W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Gyp.5+Kjar.3+hAlmsc.2_AMX24_BECKa_AREF.html,2,splib06,,,,,0.497,,0.2,,,,,,,,,,,,
172,5760,group.2um/sulfate-mix_gyp+jar+musc+dick.amix,Gyp.4+jr20+msc.2+dic.2 AMX23 W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Gyp.4+jr20+msc.2+dic.2_AMX23_BECKa_AREF.html,2,splib06,,,,,0.398,,0.2,,,,,,,,,,,,
173,5754,group.2um/musc+gyp+jar+dick.amix,Gyp.3+jr.1+msc.4+dic.2 AMX22 W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Gyp.3+jr.1+msc.4+dic.2_AMX22_BECKa_AREF.html,2,splib06,,,,,0.298,,0.4,,,,,,,,,,,,
174,5670,group.2um/dick+musc+gyp+jar.amix,Dic.4+msc.3+gyp.2+jr.1 AMX20 W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Dic.4+msc.3+gyp.2+jr.1_AMX20_BECKa_AREF.html,2,splib06,,,,,0.199,,0.3,,,,,,,,,,,,
175,5388,group.2um/alunite.5+musc.5,Alunite.5+MuscCU91-250A AMX4 W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Alunite.5+MuscCU91-250A_AMX4_BECKa_AREF.html,2,splib06,,,,,,,0.295,,,,,,,,,,,,
176,5346,group.2um/alunite.33+kaol.33+musc.33,Alun.33+Kaol.33+Musc.33 AMX2 W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Alun.33+Kaol.33+Musc.33_AMX2_BECKa_AREF.html,2,splib06,,,,,,,0.195,0.313,,,,,,,,,,,
177,5982,group.2um/muscovite+chlorite,Muscovite+Chlorite CU91-253D W2R4Na,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Muscovite+Chlorite_CU91-253D_NIC4a_RREF.html,2,splib06,,0.03,,,,,0.53,,,,,,,,,,,,
178,786,group.2um/smectite_ammonillsmec,Ammonio-Illite/Smectit GDS87 W1R1Hb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Ammonio-Illite-Smect_GDS87_BECKb_AREF.html,2,sprlb06,,,,,,,0.5,,,,,,,,,,,,
179,846,group.2um/feldspar_buddingtonite_ammonium,Buddingtonite GDS85 D-206 W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Buddingtonite_GDS85_D-206_BECKb_AREF.html,2,splib06,,,,,,,,,,,,,,,,,,,
180,5502,group.2um/feldspar_buddington.namont,Buddingtnt+Na-Mont CU93-260B W2R4Nb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Buddingtnt+Na-Mont_CU93-260B_NIC4b_RREF.html,2,splib06,,,,,,,,,0.1,,,,,,,,,,
181,5502,group.2um/feldspar_buddington.namont2,Buddingtnt+Na-Mont CU93-260B W2R4Nb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Buddingtnt+Na-Mont_CU93-260B_NIC4b_RREF.html,2,splib06,,,,,,,,,0.1,,,,,,,,,,
182,5436,group.2um/sulfate_ammonalunite,Alunite_NH4+Jaro NMNH145596A W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Alunite_NH4+Jaro_NMNH145596A_BECKb_AREF.html,2,splib06,,,,,,,,,,,,,,-1,,,,,
183,390,group.2um/sulfate_ammonjarosite,Ammonio-Jarosite SCR-NHJ W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Ammonio-Jarosite_SCF-NHJ_BECKa_AREF.html,2,splib06,,,,,,,,,,,,,,-1,,,,,
184,612,group.2um/carbonate_dolomite,Dolomite HS102.3B W1R1Hb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Dolomite_HS102.3B_ASDNGb_AREF.html,2,sprlb06,,,0.996,,,,,,,,,,,,,,,,
185,72,group.2um/actinolite,Actinolite NMNHR16485 W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Actinolite_NMNHR16485_BECKb_AREF.html,2,splib06,,,,,,,,,,,,,,,,,,,
186,402,group.2um/amphibole,Amphibole NMNH78662 W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Amphibole_NMNH78662_BECKb_AREF.html,2,splib06,,,,,,,,,,,,,,,,,,,
187,774,group.2um/micagrp_biotite,Biotite HS28.3B W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Biotite_HS28.3B_BECKa_AREF.html,2,splib06,,,,,,,,,,,,,,,,,,,-1
188,912,group.2um/micagrp_biotite-phlogopite,Biotite-phlogopite P23-5 Sam W1R1H_,No available USGS Spectral Library 7 Description Reference,2,sprlb06,,,,,,,,,,,,,,,,,,,-1
189,5664,group.2um/chlorite-skarn,Chlorite_Serpentine BR93-22B W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Chlorite_Serpentine_BR93-22B_BECKb_AREF.html,2,splib06,,0.61,,,,,,,,,,,,,,,,,
190,1080,group.2um/chlorite,Chlorite SMR-13.b 60-104um W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Chlorite_SMR-13.b_60-104um_BECKa_AREF.html,2,splib06,,1,,,,,,,,,,,,,,,,,
191,252,group.2um/prehnite+.50chlorite,Prehnite+.50Chlorite AMIX,No available USGS Spectral Library 7 Description Reference,2,sprlb06,,0.5,,,,,,,,,,,,,,,,,
192,258,group.2um/prehnite+.67chlorite,Prehnite+.67Chlorite AMIX,No available USGS Spectral Library 7 Description Reference,2,sprlb06,,0.67,,,,,,,,,,,,,,,,,
193,264,group.2um/prehnite+.75chlorite,Prehnite+.75Chlorite AMIX,No available USGS Spectral Library 7 Description Reference,2,sprlb06,,0.75,,,,,,,,,,,,,,,,,
194,276,group.2um/carbonate_aragonite,Aragonite GDS898 Moracco,No available USGS Spectral Library 7 Description Reference,2,sprlb06,,,,,,,,,,,,,-1,,,,,,
195,660,group.2um/chlorite_clinochlore,Clinochlore GDS158 Flagst W1R1Hb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Clinochlore_GDS158_Flagstaff_ASDNGb_AREF.html,2,sprlb06,,1,,,,,,,,,,,,,,,,,
196,1212,group.2um/chlorite_clinochlore.nmnh83369,Clinochlore NMNH83369 =bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Clinochlore_NMNH83369_BECKb_AREF.html,2,splib06,,1,,,,,,,,,,,,,,,,,
197,648,group.2um/chlorite_clinochlore.fe.gds157,Clinochlore_Fe GDS157.b W1R1Ha,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Clinochlore_Fe_GDS157.b_ASDFRa_AREF.html,2,sprlb06,,1,,,,,,,,,,,,,,,,,
198,1260,group.2um/chlorite_clinochlore.fe.sc-cca-1,Clinochlore_Fe SC-CCa-1.b =aa_,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Clinochlore_Fe_SC-CCa-1.b_BECKa_AREF.html,2,splib06,,1,,,,,,,,,,,,,,,,,
199,1338,group.2um/chlorite_cookeite-car-1.a,Cookeite CAr-1.a 104-150um =bb_,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Cookeite_CAr-1.a_104-150um_BECKb_AREF.html,2,splib06,,0.94,,,,,,,,,,,,,,,,,
200,1368,group.2um/chlorite_cookeite-car-1.c,Cookeite CAr-1.c <30um =cc_,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Cookeite_CAr-1.c_lt30um_BECKb_AREF.html,2,splib06,,0.81,,,,,,,,,,,,,,,,,
201,4872,group.2um/chlorite_thuringite,Thuringite SMR-15.b 32um W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Thuringite_SMR-15.b_80um_BECKa_AREF.html,2,splib06,,1,,,,,,,,,,,,,,,,,
202,1668,group.2um/epidote,Epidote GDS26.a 75-200um W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Epidote_GDS26.a_75-200um_BECKb_AREF.html,2,splib06,,,,,,,,,,,,,,,,,,,
203,2274,group.2um/hornblende,Hornblende_Fe HS115.3B W1R1Bc,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Hornblende_Fe_HS115.3B_BECKc_AREF.html,2,splib06,,,,,,,,,,,,,,,,,,,
204,2538,group.2um/sulfate_jarosite-Na,Jarosite GDS24 Na W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Jarosite_GDS24_Na_BECKb_AREF.html,2,splib06,,,,,,,,,,,,,,-1,,,,,
205,2568,group.2um/sulfate_jarosite-K,Jarosite GDS99 K 200C Syn W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Jarosite_GDS99_K_200C_Syn_BECKa_AREF.html,2,splib06,,,,,,,,,,,,,,-1,,,,,
206,2604,group.2um/sulfate_jarosite-lowT,Jarosite SJ-1 H3O 10-20% W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Jarosite_SJ-1_H3O_10-20percent_BECKb_AREF.html,2,splib06,,,,,,,,,,,,,,-1,,,,,
207,3930,group.2um/phlogopite,Phlogopite HS23.3B W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Phlogopite_HS23.3B_BECKa_AREF.html,2,splib06,,,,,,,-1,,,,,,,,,,,,
208,5142,group.2um/tremolite.or.talc,Tremolite HS18.3 W1R1Bc,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Tremolite_HS18.3_BECKc_AREF.html,2,splib06,,,,,,,,,,,,,,,,,,,
209,4800,group.2um/talc.crsgrnd,Talc GDS23 74-250um fr W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Talc_GDS23_ASDNGa_AREF.html,2,splib06,,,,,,,,,,,,,,,,,,,
210,4830,group.2um/talc.fngrnd,Talc WS659 W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Talc_WS659_BECKb_AREF.html,2,splib06,,,,,,,,,,,,,,,,,,,
211,5538,group.2um/talc+calcite.parkcity,Calcite+Talc PC99-1G W1R1Fc,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Calcite+Talc_PC99-1G_ASDFRc_AREF.html,2,splib06,0.5,,,,,,,,,,,,,,,,,,
212,5682,group.2um/talc+carbonate.parkcity,Dolomit+Calcite+Talc PC99-1E W1R1Fc,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Dolomit+Calcite+Talc_PC99-1E_ASDFRc_AREF.html,2,splib06,0.33,,0.33,,,,,,,,,,,,,,,,
213,5568,group.2um/carbonate_calcite+0.2Ca-mont,Calcite.8+Ca-Montmor.2 AMX15 W1R1BbS,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Calcite.8+Ca-Montmor.2_AMX15_BECKb_AREF.html,2,splib06,0.796,,,,,,,,0.17,,,,,,,,,,
214,5922,group.2um/carbonate_magnesite,Magnesite+Hydroma HS47.3B W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Magnesite+Hydromag_HS47.3B_BECKa_AREF.html,2,splib06,,,,,,,,,,,,,-1,,,,,,
215,666,group.2um/carbonate_calcite,Calcite WS272 W1R1Ha,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Calcite_WS272_ASDNGa_AREF.html,2,sprlb06,0.995,,,,,,,,,,,,,,,,,,
216,5532,group.2um/carbonate_calcite+dolomite.5,Calcite+Dolomite.5 AMX8 W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Calcite.5+Dolomite.5_AMX8_BECKb_AREF.html,2,splib06,0.497,,0.498,,,,,,,,,,,,,,,,
217,5694,group.2um/carbonate_dolo+.5ca-mont,Dolomite.50+Ca-Montmor AMX10 W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Dolomite.5+Ca-Mont.5_AMX10_BECKb_AREF.html,2,splib06,,,0.5,,,,,,0.42,,,,,,,,,,
218,5514,group.2um/calcite.25+dolom.25+Na-mont.5,Calc.25+dolo.25+mont.5 AMX18 W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Calc.25+Dolo.25+Na-Mon_AMX18_BECKb_AREF.html,2,splib06,0.249,,0.249,,,,,,0.42,,,,,,,,,,
219,5508,group.2um/carbonate_calcite.25+dolom.25+Ca-mont.5,Calc.25+Dolo.25+Ca-Mon AMX7 W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Calc.25+Dolo.25+Ca-Mont_AMX7_BECKb_AREF.html,2,splib06,0.249,,0.249,,,,,,0.42,,,,,,,,,,
220,4560,group.2um/carbonate_siderite,Siderite HS271.3B W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Siderite_HS271.3B_BECKa_AREF.html,2,splib06,,,,,,,,,,,,,-1,,,,,,
221,3558,group.2um/smectite_nontronite_swelling,Nontronite NG-1.a W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Nontronite_NG-1.a_BECKb_AREF.html,2,splib06,,,,,,,,,-1,,,,,,,,,,
222,696,group.2um/clay_hectorite,Hectorite SHCa-1.Ac-B W1R1Hb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Hectorite_SHCa-1.Ac-B_BECKb_AREF.html,2,sprlb06,,,,,,,,,-1,,,,,,,,,,
223,702,group.2um/saponite.or.talc,Saponite SapCa-1.AcB W1R1Hb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Saponite_SapCa-1.AcB_BECKb_AREF.html,2,sprlb06,,,,,,,,,-1,,,,,,,,,,
224,690,group.2um/sepiolite,Sepiolite SepNev-1.AcB W1R1Ha,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Sepiolite_SepNev-1.AcB_BECKa_AREF.html,2,sprlb06,,,,,,,,,,,,,,,,,,,
225,5574,group.2um/calcite+0.2Na-mont,Calcite.80+Mont_Swy-1 GDS212 W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Calcite.80+Mont_Swy-1_GDS212_BECKa_AREF.html,2,splib06,0.796,,,,,,,,0.17,,,,,,,,,,
226,5526,group.2um/calcite+0.5Ca-mont,Calcite+.50Ca-Mont AMX6 W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Calcite.5+.Ca-Mont.5_AMX6_BECKb_AREF.html,2,splib06,0.497,,,,,,,,0.42,,,,,,,,,,
227,5520,group.2um/carbonate_calcite+0.3muscovite,Calcite+.33Muscov AMX5 Ruby W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Calcite+.33Muscov_AMX5_Ruby_BECKa_AREF.html,2,splib06,0.667,,,,,,0.33,,,,,,,,,,,,
228,30,group.2um/organic_drygrass+.17Na-mont,Grass_dry+0.17%Na-Mont AMX35 W1R1Ba AREF,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Grass_dry.83+.17NaMont_AMX35_BECKb_AREF.html,2,sprlb06,,,,,,,,,0.14,,,,,,,,,,
229,5688,group.2um/carbonate_dolomite.5+Na-mont.5,Dolomite.5+Na-mont.5 AMX21 W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Dolomite.5+Na-Mont.5_AMX21_BECKb_AREF.html,2,splib06,,,0.498,,,,,,0.42,,,,,,,,,,
230,5544,group.2um/carbonate_smectite_calcite.33+Ca-mont.67,Calcite.33+Ca-mont.67 AMX19 W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Calcite.33+Ca-Mont.67_AMX19_BECKb_AREF.html,2,splib06,0.328,,,,,,,,0.66,,,,,,,,,,
231,5580,group.2um/carbonate_calcite+0.2kaolwxl,Calcite.80wt+Kaol_CM9 GDS213 W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Calcite.80wt+Kaol_CM9_GDS213_BECKa_AREF.html,2,splib06,0.796,,,,,,,0.19,,,,,,,,,,,
232,5562,group.2um/carbonate_calcite0.7+kaol0.3,Calcite.7+Kaolwxl.3 AMX9 W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Calcite.7+Kaolin-wxl.3_AMX9_BECKa_AREF.html,2,splib06,0.696,,,,,,,0.19,,,,,,,,,,,
233,1314,group.2um/phyllosilicate.clintonite,Clintonite NMNH126553 W1R1Bc,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Clintonite_NMNH126553_BECKc_AREF.html,2,splib06,,,,,,,-1,,,,,,,,,,,,
234,3870,group.2um/palygorskite,Palygorskite PFL-1 Attapulg W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Palygorskite_PFL-1_Attapulg_BECKb_AREF.html,2,splib06,,,,,,,-1,,,,,,,,,,,,
235,5298,group.2um/zoisite,Zoisite HS347.3B W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Zoisite_HS347.3B_BECKb_AREF.html,2,splib06,,,,,,,,,,,,,,,,,,,
236,1626,group.2um/elbaite,Elbaite NMNH94217-1.c <74 W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Elbaite_NMNH94217-1.c_lt74_BECKb_AREF.html,2,splib06,,,,,,,,,,,,,,,,,,,
237,1824,group.2um/hydroxide_gibbsite,Gibbsite WS214 W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Gibbsite_WS214_BECKb_AREF.html,2,splib06,,,,,,,,,,,,,,,,,,,
238,3546,group.2um/nitrate_niter,Niter GDS43 (K-Saltpeter) W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Niter_GDS43_(K-Saltpeter)_BECKa_AREF.html,2,splib06,,,,,,,,,,,,,,,,,,,
239,5040,group.2um/topaz,Topaz Mt._Antero_#5 W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Topaz_Mt_Antero_%235_BECKb_AREF.html,2,splib06,,,,,,,,,,,,,,,,,,,
240,4278,group.2um/carbonate_rhodochrosite,Rhodochrosite HS338.3B W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Rhodochrosite_HS338.3B_BECKb_AREF.html,2,splib06,,,,,,,,,,,,,-1,,,,,,
241,5268,group.2um/carbonate_witherite,Witherite HS273.3B W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Witherite_HS273.3B_BECKa_AREF.html,2,splib06,,,,,,,,,,,,,-1,,,,,,
242,4026,group.2um/prehnite,Prehnite GDS613.a <60um =aa_,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Prehnite_GDS613.a_lt60um_ASDFRa_AREF.html,2,splib06,,,,,,,,,,,,,,,,,,,
243,3126,group.2um/prehnite+muscovite,Mizzonite BM1931 12 Scapolte W1R1Bc,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Mizzonite_BM1931-12_Scapolte_BECKc_AREF.html,2,splib06,,,,,,,0.1,,,,,,,,,,,,
244,3978,group.2um/borate_pinnoite,Pinnoite NMNH123943 W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Pinnoite_NMNH123943_BECKa_AREF.html,2,splib06,,,,,,,,,,,,,,,,,,,
245,3996,group.2um/polyhalite,Polyhalite NMNH92669-4 W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Polyhalite_NMNH92669-4_BECKa_AREF.html,2,splib06,,,,,,,,,,,,,,,,,,,
246,4368,group.2um/rivadavite,Rivadavite NMNH170164 Amph W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Rivadavite_NMNH170164_BECKa_AREF.html,2,splib06,,,,,,,,,,,,,,,,,,,
247,2298,group.2um/howlite,Howlite GDS155 W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Howlite_GDS155_BECKa_AREF.html,2,splib06,,,,,,,,,,,,,,,,,,,
248,5190,group.2um/ulexite,Ulexite HS441.3B W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Ulexite_HS441.3B_BECKa_AREF.html,2,splib06,,,,,,,,,,,,,,,,,,,
249,6114,group.2um/white.crust.starkeyite,White_Crust LV30 starkeyite W1R1Fc,https://crustal.usgs.gov/speclab/data/HTMLmetadata/White_Crust_LV30_starkeyite_ASDFRc_AREF.html,2,splib06,,,,,,,,,,,,,,,,,,,
250,414,group.2um/zeolite_analcime,Analcime GDS1 Zeolite W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Analcime_GDS1_Zeolite_BECKa_AREF.html,2,splib06,,,,,,,,,,,,,,,,,,,
251,834,group.2um/hydroxide_brucite,Brucite HS247.3B W1R1Bc,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Brucite_HS247.3B_BECKc_AREF.html,2,splib06,,,,,,,,,,,,,,,,,,,
252,942,group.2um/chloride_carnallite,Carnallite NMNH98011 W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Carnallite_NMNH98011_BECKb_AREF.html,2,splib06,,,,,,,,,,,,,,,,,,,
253,1320,group.2um/borate_colemanite,Colemanite GDS143 W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Colemanite_GDS143_BECKb_AREF.html,2,splib06,,,,,,,,,,,,,,,,,,,
254,1470,group.2um/datolite,Datolite HS442.3B W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Datolite_HS442.3B_BECKb_AREF.html,2,splib06,,,,,,,,,,,,,,,,,,,
255,2310,group.2um/hydrogrossular,Hydrogrossular NMNH120555 W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Hydrogrossular_NMNH120555_BECKb_AREF.html,2,splib06,,,,,,,,,,,,,,,,,,,
256,2862,group.2um/micagrp_lepidolite,Lepidolite NMNH88526-1 W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Lepidolite_NMNH88526-1_BECKa_AREF.html,2,splib06,,,,,,,-1,,,,,,,,,,,,
257,2976,group.2um/micagrp_margarite,Margarite GDS106 W1R1Bc,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Margarite_GDS106_BECKc_AREF.html,2,splib06,,,,,,,-1,,,,,,,,,,,,
258,3012,group.2um/sulfate_mascagnite,Mascagnite GDS65.b (fr) W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Mascagnite_GDS65.b_(fr)_BECKa_AREF.html,2,splib06,,,,,,,,,,,,,,-1,,,,,
259,3114,group.2um/sulfate_mirabilite-wet_soil,Mirabilite GDS150 Na2SO4 W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Mirabilite_GDS150_Na2SO4_BECKa_AREF.html,2,splib06,,,,,,,,,,,,,,-1,,,,,
260,3474,group.2um/zeolite_natrolite,Natrolite HS169.3B Zeolite W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Natrolite_HS169.3B_Zeolite_BECKb_AREF.html,2,splib06,,,,,,,,,,,,,,,,,,,
261,4338,group.2um/richterite,Richterite NMNH150800 HCl W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Richterite_NMNH150800_HCl_BECKb_AREF.html,2,splib06,,,,,,,,,,,,,,,,,,,
262,1152,group.2um/serpentine_chrysotile.coarse-fibrous,Chrysotile ML99-12A Coar Fib W1R1Fb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Chrysotile_ML99-12A_Coar_Fib_ASDFRb_AREF.html,2,splib06,,,,,,,,,,,,,,,,,,,
263,1158,group.2um/serpentine_chrysotile.med-fine-fibrous,Chrysotile ML99-12C Fine Fib W1R1Fb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Chrysotile_ML99-12C_Fine_Fib_ASDFRb_AREF.html,2,splib06,,,,,,,,,,,,,,,,,,,
264,1146,group.2um/serpentine_chrysotile.fine-fibrous,Chrysotile HS323.1B W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Chrysotile_HS323.1B_BECKa_AREF.html,2,splib06,,,,,,,,,,,,,,,,,,,
265,1440,group.2um/serpentine_cronstedtite,Cronstedtite M3542 =bb_,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Cronstedtite_M3542_BECKb_AREF.html,2,splib06,,,,,,,,,,,,,,,,,,,
266,4380,group.2um/micagrp_illite.roscoelite,Roscoelite EN124 W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Roscoelite_EN124_BECKb_AREF.html,2,splib06,,,,,,,0.15,,,,,,,,,,,,
267,4764,group.2um/carbonate_strontianite,Strontianite HS272.3B W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Strontianite_HS272.3B_BECKb_AREF.html,2,splib06,,,,,,,,,,,,,-1,,,,,,
268,4788,group.2um/sulfate_syngenite,Syngenite GDS139 W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Syngenite_GDS139_BECKa_AREF.html,2,splib06,,,,,,,,,,,,,,-1,,,,,
269,1134,group.2um/chrysocolla,Chrysocolla HS297.3B W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Chrysocolla_HS297.3B_ASDFRb_AREF.html,2,splib06,,,,,,,,,,,,,,,,,,,
270,720,group.2um/carbonate_azurite,Azurite WS316 W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Azurite_WS316_BECKa_AREF.html,2,splib06,,,,,,,,,,,,,-1,,,,,,
271,2964,group.2um/carbonate_malachite,Malachite HS254.3B W1R1Bb,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Malachite_HS254.3B_BECKb_AREF.html,2,splib06,,,,,,,,,,,,,-1,,,,,,
272,6522,group.2um/cyanide-CdK,Cyanide_Cadmium_Potassium W2R4Na RREF,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Cyanide_Cadmium_Potassium_BECKa_AREF.html,2,splib06,,,,,,,,,,,,,,,,,,,
273,6528,group.2um/cyanide-NiK,Cyanide_Nickel_Potassium W2R4Na RREF,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Cyanide_Nickel_Potassium_BECKa_AREF.html,2,splib06,,,,,,,,,,,,,,,,,,,
274,6534,group.2um/cyanide-trihydrate,Cyanide_Trihydrate CT1 W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Cyanide_Trihydrate_CT1_BECKa_AREF.html,2,splib06,,,,,,,,,,,,,,,,,,,
275,6552,group.2um/cyanide-KFe,Cyanide_Potassium_Ferro W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Cyanide_Potassium_Ferro_BECKa_AREF.html,2,splib06,,,,,,,,,,,,,,,,,,,
276,6540,group.2um/cyanide-ZnK,Cyanide_Zinc_Potassium W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Cyanide_Zinc_Potassium_BECKa_AREF.html,2,splib06,,,,,,,,,,,,,,,,,,,
277,6690,group.2um/organic_plastic.grnhs.roof,Plastic GGA-54 Grnhouse Roof W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Plastic_GGA-54_Grnhouse_Roof_BECKa_AREF.html,2,splib06,,,,,,,,,,,,,,,1,-1,,,
278,6606,group.2um/organic_benzene+montswy,Montmorill+Benzene SWy 1ml W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Montmorill+Benzene_SWy_1ml_BECKa_AREF.html,2,splib06,,,,,,,,,0.756,,,,,,,-1,,,
279,6624,group.2um/organic_trichlor+montswy,Montmorill+Trichlor SWy 1ml W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Montmorill+Trichlor_SWy_1ml_BECKa_AREF.html,2,splib06,,,,,,,,,0.756,,,,,,,-1,,,
280,6618,group.2um/organic_toluene+montswy,Montmorill+Toluene SWy .5ml W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Montmorill+Toluene_SWy_.5ml_BECKa_AREF.html,2,splib06,,,,,,,,,0.798,,,,,,,-1,,,
281,6630,group.2um/organic_unleaded.gas+montswy,Montmorill+Unleaded_Gas SWy W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Montmorill+Unleaded_Gas_SWy_BECKa_AREF.html,2,splib06,,,,,,,,,0.756,,,,,,,-1,,,
282,6612,group.2um/organic_tce+montswy,Montmorill+TCE SWy saturated W1R1Ba,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Montmorill+TCE_SWy_saturated_BECKa_AREF.html,2,splib06,,,,,,,,,0.756,,,,,,,-1,,,
283,5310,group.2um/zunyite,Zunyite GDS-241B <150um W1R1B?,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Zunyite_GDS241B_lt150um_BECKu_AREF.html,2,splib06,,,,,,,,,,,,,,,,,,,
284,7008,group.2um/sulfate-mix_gypsum.trace.dust+debris-WTC01-2,WTC_Dust_Debris WTC01-2 W1R1Fa,https://crustal.usgs.gov/speclab/data/HTMLmetadata/WTC_Dust_Debris_WTC01-2_ASDFRa_AREF.html,2,splib06,,,,,0.1,,,,,,,,,,,-1,,,
285,7014,group.2um/sulfate-mix_gypsum.trace.dust+debris-WTC01-28,WTC_Dust_Debris WTC01-28 W1R1Fa,https://crustal.usgs.gov/speclab/data/HTMLmetadata/WTC_Dust_Debris_WTC01-28_ASDFRa_AREF.html,2,splib06,,,,,0.1,,,,,,,,,,,-1,,,
286,6498,group.2um/chrysotile.gypsum.wtc01-8,Coated_Steel_Girder WTC01-8 W1R1Fa,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Coated_Steel_Girder_WTC01-8_ASDFRa_AREF.html,2,splib06,,,,,0.1,,,,,,,,,,,-1,,,
287,4008,group.2um/portlandite,Portlandite GDS525 Ca(OH)2 W1R1Fa,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Portlandite_GDS525_Ca(OH)2_ASDFRa_AREF.html,2,splib06,,,,,,,,,,,,,,,,,,,
288,234,group.2um/nitrate_potassium,Potassium Nitrate GDS43,No available USGS Spectral Library 7 Description Reference,2,sprlb06,,,,,,,,,,,,,,,,,,,
289,240,group.2um/nitrate_sodium,Sodium Nitrate GDS883,No available USGS Spectral Library 7 Description Reference,2,sprlb06,,,,,,,,,,,,,,,,,,,
290,624,group.2um/micagrp_vermiculite_GDS458,Vermiculite GDS458 W1R1Hc,No available USGS Spectral Library 7 Description Reference,2,sprlb06,,,,,,,,,,0.16,,,,,,,,,
291,630,group.2um/micagrp_vermiculite_ALB4SA00,Vermiculite ALB4SA00 W1R1Hc,No available USGS Spectral Library 7 Description Reference,2,sprlb06,,,,,,,,,,0.26,,,,,,,,,
292,636,group.2um/micagrp_vermiculite_WS682,Vermiculite WS682 W1R1Hc,No available USGS Spectral Library 7 Description Reference,2,sprlb06,,,,,,,,,,0.68,,,,,,,,,
293,642,group.2um/micagrp_vermiculite_WS681,Vermiculite WS681 W1R1Hc,https://crustal.usgs.gov/speclab/data/HTMLmetadata/Vermiculite_WS681_BECKc_AREF.html,2,sprlb06,,,,,,,,,,0.42,,,,,,,,,
294,1008,group.2um/methane-gas-2.37um-experimental,CO2+O2+CH4 TRANSM ModtranZ20 r06crj3a=,No available USGS Spectral Library 7 Description Reference,2,sprlb06,,,,,,,,,,,,,,,,,,,
================================================
FILE: data/results_urls.txt
================================================
https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/EMITL2BMIN.001/EMIT_L2B_MIN_001_20230427T173309_2311711_010/EMIT_L2B_MIN_001_20230427T173309_2311711_010.nc
https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/EMITL2BMIN.001/EMIT_L2B_MIN_001_20230804T191650_2321613_007/EMIT_L2B_MIN_001_20230804T191650_2321613_007.nc
https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/EMITL2BMIN.001/EMIT_L2B_MIN_001_20230808T173953_2322012_011/EMIT_L2B_MIN_001_20230808T173953_2322012_011.nc
================================================
FILE: data/rgb_browse_urls.txt
================================================
https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-public/EMITL2ARFL.001/EMIT_L2A_RFL_001_20230427T173309_2311711_010/EMIT_L2A_RFL_001_20230427T173309_2311711_010.png
https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-public/EMITL2ARFL.001/EMIT_L2A_RFL_001_20230804T191650_2321613_007/EMIT_L2A_RFL_001_20230804T191650_2321613_007.png
https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-public/EMITL2ARFL.001/EMIT_L2A_RFL_001_20230808T173953_2322012_011/EMIT_L2A_RFL_001_20230808T173953_2322012_011.png
================================================
FILE: data/sample_coords.csv
================================================
ID,Category,Latitude,Longitude
0,1,-39.94,-62.36
1,1,-39.75,-61.74
2,3,-40,-62.1
3,2,-39.89,-61.85
4,3,-39.38,-62.03
================================================
FILE: guides/Getting_EMIT_Data_using_EarthData_Search.md
================================================
# Getting Started with Earth Surface Mineral Dust Source Investigation (EMIT) data in Earthdata Search
**Note:** Users must have [NASA Earthdata Login](https://urs.earthdata.nasa.gov/home) to download data
## Step 1. Go to [Earthdata Search](https://search.earthdata.nasa.gov/search) and Login
Click **Earthdata Login** in the upper right portion of the Earthdata Search Landing Page
Enter your Earthdata Login credentials (username/password)
## Step 2. Finding EMIT Data
1. In the top left portion of the screen type in "EMIT" (quotes will limit results to an exact match). This will search for all available EMIT collections.
2. Currently the LP DAAC manages two EMIT collections.
EMIT L2A Estimated Surface Reflectance and Uncertainty and Masks 60m V001 and EMIT L1B At-Sensor Calibrated Radiance and Geolocation Data 60m V001.
Select the collection of granules that you want to view. If you are unsure which collection to choose the ***View Collection Details*** offers more information about each data product. The current state of the collection includes a very limited data release with only a handful of scenes from select locations.
## Step 3. Download EMIT Data
To view the scenes and location, click on the image to focus it on the map the location is indicated by a green box. A preview of the scene is shown on the right.
1. If you are interested in downloading all EMIT scenes discovered by Earthdata search, select the ***Download All*** icon at the bottom of the search results page.
2. If you are interested in only a single scene, click the download arrow on the card (or list row) for the select scene. You will then be able to download the select files related to that image.
You can also access S3 information (e.g., AWS region, bucket, temporary credentials for S3 access, and file names)
3. To download multiple granules, click on the green + symbol to add files to the project. Click on the green button on the bottom of the page that says **Download**. This will take you to the options page for customizing the download.
## Step 4. Data Access
On the next page click the **Direct Download** option and click the green **Download Data** on the bottom left side of the page.
## Step 5. Download Status
The final page of instructions show the download links for data access in the cloud. You should see four tabs: Download Files, AWS S3 Access, Download Script, and Browse Imagery.
The **Download Files** tab provides the *https://* links for downloading the files locally
The **AWS S3 Access** tab provides the *s3://* links, which is what we would use to access the data directly in region (us-west-2) within the AWS cloud.
---
**Please visit our [FAQ](https://forum.earthdata.nasa.gov/) page if you have general questions or visit on the [Earthdata Forum](https://forum.earthdata.nasa.gov/) to ask your own question.**
---
## Contact Info
Email: <LPDAAC@usgs.gov>
Voice: +1-866-573-3222
Organization: Land Processes Distributed Active Archive Center (LP DAAC)¹
Website: <https://lpdaac.usgs.gov/>
Date last modified: 06-05-2024
¹Work performed under USGS contract 140G0121D0001 for NASA contract NNG14HH33I.
================================================
FILE: guides/Streaming_cloud_optimized_geotiffs_using_QGIS.md
================================================
# Streaming NASA Earthdata Cloud-Optimized Geotiffs using QGIS
**Note:** Users must have [NASA Earthdata Login](https://urs.earthdata.nasa.gov/home) to stream data.
QGIS can be used to stream NASA Earthdata cloud-optimized geotiff files. To do this QGIS uses https and the vsicurl virtual file system handler. This guide will show you how to configure QGIS to do this. There are 4 steps needed.
1. Create a `.netrc` file
2. Add Custom Environment Variables
3. Restart QGIS
4. Add Data
## 1. Create a .netrc file
Under the hood QGIS uses gdal and vsicurl to stream data via https. For NASA Earthdata, this requires a `.netrc` file to store your Earthdata login credentials. If you already have one, you can skip this step. This file should be located in your home directory. Choose one of the methods below to create the file and enter your credentials.
### A. Manual Set Up
- Download the [.netrc template file](https://github.com/nasa/LPDAAC-Data-Resources/tree/main/data/.netrc) and save it in your *home/user/* directory, where *user* is your personal user directory. For example: `C:\Users\user\.netrc` or `home/user/.netrc.`
- Open the `.netrc` file in a text editor and replace <USERNAME> with your NASA Earthdata Login username and <PASSWORD> with your NASA Earthdata Login password.
After editing, the file should look something like this:
or you can also have everything on a single line separated by spaces, like:
### B. Command Line
**For Linux/MacOS:**
To Create a .netrc file, enter the following in the command line, replacing <USERNAME> and <PASSWORD> with your NASA Earthdata username and password. This will create a file in your home directory or append your NASA credentials to an existing file.
```bash
echo "machine urs.earthdata.nasa.gov login <USERNAME> password <PASSWORD>" >>~/.netrc
```
**For Windows:**
To Create a .netrc file, enter the following in the command line, replacing <USERNAME> and <PASSWORD> with your NASA Earthdata username and password. This will create a file in your home directory or append your NASA credentials to an existing file.
```cmd
echo machine urs.earthdata.nasa.gov login <USERNAME> password <PASSWORD> >> %userprofile%\.netrc
```
You can verify that the file is correct by opening with a text editor. It should look like an example in one of the figures above.
## 2. Environment Settings
Next, while still in the **Settings** menu, select **System** on the left-hand side, then scroll down to **Environment**. Select the check box and enter the Variables and Values in the table below by clicking the plus sign to add a new variable. These variables, set up a place for cookies to be stored and read from, prevent GDAL from reading all files in the directory, specify the extensions GDAL is allowed to access over HTTPS using the vsicurl virtual file system handler, and allow use of unsafe SSL connections (use with caution).
|Variable|Value|
| ----------- | ----------- |
|GDAL_HTTP_COOKIEFILE|~/cookies.txt|
|GDAL_HTTP_COOKIEJAR|~/cookies.txt|
|GDAL_DISABLE_READDIR_ON_OPEN|EMPTY_DIR|
|CPL_VSIL_CURL_ALLOWED_EXTENSIONS|TIF|
|GDAL_HTTP_UNSAFESSL|YES|
> **Note: These settings may affect streaming from other data sources.**
## 3. Restart QGIS
Restart QGIS to load the new environment settings.
## 4. Add Some Data
To add some raster data, select the add raster data button from the toolbar, select 'Protocol: HTTP(S), cloud, etc' as the source type, then enter a URI for a cloud-optimized geotiff file and press the add button. For example:
`https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/EMITL2BCH4ENH.001/EMIT_L2B_CH4ENH_001_20240423T074559_2411405_018/EMIT_L2B_CH4ENH_001_20240423T074559_2411405_018.tif`
This should add the data to your map, and will work with any NASA Earthdata hosted cloud-optimized geotiff file.
>**If these instructions do not work, please verify that your `.netrc` file has the correct username and password, and is formatted as shown in [Section 1](#1-create-a-netrc-file).**
## Contact Info
Email: <LPDAAC@usgs.gov>
Voice: +1-866-573-3222
Organization: Land Processes Distributed Active Archive Center (LP DAAC)¹
Website: <https://lpdaac.usgs.gov/>
Date last modified: 06-25-2024
¹Work performed under USGS contract 140G0121D0001 for NASA contract NNG14HH33I.
================================================
FILE: python/how-tos/How_to_Convert_to_ENVI.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# How To: Convert EMIT .nc to .envi\n",
"\n",
"There are currently 2 similar methods to convert the EMIT netCDF4 files to `.envi` format. Note these only support L1B Radiance, L1B Obs, L2A Reflectance, L2A Reflectance Uncertainty, or L2A Mask to .envi. They do not yet support the L2B Mineral or L2B Mineral Uncertainty products.\n",
"1. The `write_envi` function in EMIT tools. This function is still being developed but will currently:\n",
" - Write a GLT output to use for orthocorrection later \n",
" - Functions from `emit_tools` can be used beforehand to orthorectify if so desired\n",
"2. The `reformat.py` script available in the [emit-sds/emit-utils](https://github.com/emit-sds/emit-utils) repository can be used to convert EMIT netCDF files (as delivered to the LP DAAC) to ENVI files. This script also can apply the included GLT to orthorectify the image if desired.\n",
"\n",
"This jupyter notebook walks through how to use both methods to provide users with programmatic routes to accomplish their EMIT reformatting workflows.\n",
"\n",
"**Requirements:**\n",
"+ A NASA [Earthdata Login](https://urs.earthdata.nasa.gov/) account is required to download EMIT data \n",
"+ Selected the `emit_tutorials` environment as the kernel for this notebook.\n",
" + For instructions on setting up the environment, follow the the `setup_instructions.md` included in the `/setup/` folder of the repository. \n",
"\n",
"**Learning Objectives**\n",
"+ How to use the `write_envi` function from `emit_tools` module to convert an EMIT netCDF4 to a `.envi` file.\n",
"+ How to use the `reformat.py` function from the `emit-utils` repository to convert an EMIT netCDF4 to a `.envi` file."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Setup\n",
"\n",
"Import packages"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"import earthaccess"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Authenticate using Earthdata Login and Download the required Granules\n",
"\n",
"Login to your NASA Earthdata account and create a `.netrc` file using the `login` function from the `earthaccess` library. If you do not have an Earthdata Account, you can create one [here](https://urs.earthdata.nasa.gov/home). "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"earthaccess.login(persist=True)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"For this notebook we will download the files necessary using `earthaccess`. You can also access the data in place or stream it, but this can slow due to the file sizes. Provide a URL for an EMIT L2A Reflectance granule."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"url = 'https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/EMITL2ARFL.001/EMIT_L2A_RFL_001_20220903T163129_2224611_012/EMIT_L2A_RFL_001_20220903T163129_2224611_012.nc'"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Get an HTTPS Session using your earthdata login, set a local path to save the file, and download the granule asset - This may take a while, the reflectance file is approximately 1.8 GB. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Get Https Session using Earthdata Login Info\n",
"fs = earthaccess.get_fsspec_https_session()\n",
"# Retrieve granule asset ID from URL (to maintain existing naming convention)\n",
"granule_asset_id = url.split('/')[-1]\n",
"# Define Local Filepath\n",
"fp = f'../../data/{granule_asset_id}'\n",
"# Download the Granule Asset if it doesn't exist\n",
"if not os.path.isfile(fp):\n",
" fs.download(url, fp)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now lets create an output folder where we will save the `.envi` files."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"outpath = '../../data/envi' \n",
"if not os.path.exists(outpath):\n",
" os.makedirs(outpath)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Method 1: Using `write_envi` from the `emit_tools` module.\n",
"\n",
"Import the necessary packages for this method. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import sys\n",
"sys.path.append('../modules/')\n",
"import emit_tools as et"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Open the granule using the `emit_xarray` function. We can orthorectify here if so desired."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"ds = et.emit_xarray(fp, ortho=True)\n",
"ds"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now, write the dataset as an `.envi` output. If we chose not to orthorectify, you can include a `glt` file to orthorectify later."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"et.write_envi(ds, outpath, overwrite=False, extension='.img', interleave='BIL', glt_file=False)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Method 2: Using reformat.py from emit-utils\n",
"\n",
"### 2.1 Clone and Install emit-utils\n",
"\n",
"Clone the [emit-utils](https://github.com/emit-sds/emit-utils) repository."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"!git clone https://github.com/emit-sds/emit-utils.git ../emit_utils/"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This will copy the `emit-utils` repository to a folder within this repository. \n",
"\n",
"After you have copied it, use `pip` package manager to install the directory as a package to ensure you have all of the dependencies and be used in the command line. \n",
"\n",
"> **This requires that some dependencies already be installed to work properly on Windows. If you have created the Python environment described in the [setup instructions](../../setup/setup_instructions.md) it should work.**"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"!pip install --editable ../emit_utils"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"After successfully installing `emit-utils`, you can use the scripts contained within as part of your workflows. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 2.2 Executing the Reformat Script\n",
"\n",
"Before calling the `reformat.py` script, make sure you have an output directory for the `.envi` files that will be produced."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"import os\n",
"outpath = '../../data/envi' \n",
"if not os.path.exists(outpath):\n",
" os.makedirs(outpath)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now, execute the `reformat.py` script contained in the emit-utils repository. When executing this script, provide the path to the `.nc` file, followed by the directory to place the `.envi` files in. If you wish to apply the GLT or orthorectify, include `--orthorectify` as an argument."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"!python ../emit_utils/emit_utils/reformat.py ../../data/EMIT_L2A_RFL_001_20220903T163129_2224611_012.nc ../../data/envi/ --orthorectify"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This will orthorectify the image, create an ENVI header, and save it in `.envi` format inside the `../data/envi` folder."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Contact Info: \n",
"\n",
"Email: LPDAAC@usgs.gov \n",
"Voice: +1-866-573-3222 \n",
"Organization: Land Processes Distributed Active Archive Center (LP DAAC)¹ \n",
"Website: <https://lpdaac.usgs.gov/> \n",
"Date last modified: 07-06-2023 \n",
"\n",
"¹Work performed under USGS contract G15PD00467 for NASA contract NNG14HH33I. "
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.13"
},
"vscode": {
"interpreter": {
"hash": "ee32eb811eece122811842209709e899a392a6a8deb39746eb88e988164e27f9"
}
}
},
"nbformat": 4,
"nbformat_minor": 4
}
================================================
FILE: python/how-tos/How_to_Direct_S3_Access.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# How to: Use Direct S3 Access to work with EMIT Data\n",
"\n",
"**Summary** \n",
"\n",
"EMIT datasets are available through NASA's Earthdata Cloud. NASA Earthdata on Cloud is always free and accessible via either HTTPS or direct S3 bucket access. With direct S3 access, you can bring your \"code to the data\", making your processing faster and scalable. Direct S3 access to NASA Earthdata on Cloud is only available if your Amazon Web Services (AWS) instance is set up in the `us-west-2` region. This notebook explains how to utilize direct S3 access to EMIT data. \n",
"\n",
"**Requirements**\n",
"\n",
"+ A NASA [Earthdata Login](https://urs.earthdata.nasa.gov/) account is required to download EMIT data\n",
"+ A configured `.netrc` file to access NASA Earth Data\n",
" + This will be configured in the notebook using the `earthaccess` python library. \n",
"+ Selected the `emit_tutorials` environment as the kernel for this notebook.\n",
" + For instructions on setting up the environment, follow the `setup_instructions.md` included in the `/setup/` folder of the repository. \n",
"+ **xarray v2022.12.0** or newer to read s3 files multiple times when open with s3fs\n",
" + Version can be checked in Python using `xarray.__version__`\n",
" + To update, activate your environment and enter `pip install xarray==2022.12.0` in command line.\n",
"\n",
"**Learning Objectives** \n",
"+ How to use Direct S3 Access to EMIT Data\n",
"+ How to add this functionality to any notebook from this repository"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Import the required packages."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# Import Packages\n",
"import os\n",
"import sys\n",
"import requests\n",
"import s3fs\n",
"import netCDF4 as nc\n",
"from osgeo import gdal\n",
"import numpy as np\n",
"import xarray as xr\n",
"import hvplot.xarray\n",
"import holoviews as hv\n",
"import earthaccess\n",
"sys.path.append('../modules/')\n",
"from emit_tools import emit_xarray"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Overview of s3 Access\n",
"\n",
"NASA Earthdata Cloud data in S3 can be directly accessed via temporary credentials; this access is limited to requests made within the US West (Oregon) (code: us-west-2) AWS region. Direct S3 access is achieved by passing NASA supplied temporary credentials (token) to AWS so we can interact with S3 objects from applicable Earthdata Cloud buckets. Your NASA Earthdata login credentials can easily be managed using the `earthaccess` library. We can create the necessary `.netrc` file that stores the Earthdata Login credentials locally with the following code cell. Enter your username and password if prompted, these will be used to sign into NASA Earthdata and retrieve the necessary temporary credentials allowing you to utilize s3 access."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# The persist option creates and stores your username and password in a .netrc file\n",
"earthaccess.login(persist=True)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"For now, each NASA DAAC has different AWS credentials endpoints. Below are some of the credential endpoints to various DAACs:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"s3_cred_endpoint = {\n",
" 'podaac':'https://archive.podaac.earthdata.nasa.gov/s3credentials',\n",
" 'gesdisc': 'https://data.gesdisc.earthdata.nasa.gov/s3credentials',\n",
" 'lpdaac':'https://data.lpdaac.earthdatacloud.nasa.gov/s3credentials',\n",
" 'ornldaac': 'https://data.ornldaac.earthdata.nasa.gov/s3credentials',\n",
" 'ghrcdaac': 'https://data.ghrc.earthdata.nasa.gov/s3credentials'\n",
"}\n",
"s3_cred_endpoint"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Create a function to make a request to an endpoint for temporary credentials. Remember, each DAAC has their own endpoint and credentials are not usable for cloud data from other DAACs."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# Define Function \n",
"def get_temp_creds(provider):\n",
" return requests.get(s3_cred_endpoint[provider]).json()\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# Get Credentials\n",
"temp_creds_req = get_temp_creds('lpdaac')\n",
"#temp_creds_req"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Set up an s3fs Session for Direct Access\n",
"\n",
"`s3fs` sessions are used for authenticated access to s3 bucket and allows for typical file-system style operations. Below we create session by passing in the temporary credentials we recieved from our temporary credentials endpoint."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# Pass Authentication to s3fs\n",
"fs_s3 = s3fs.S3FileSystem(anon=False, \n",
" key=temp_creds_req['accessKeyId'], \n",
" secret=temp_creds_req['secretAccessKey'], \n",
" token=temp_creds_req['sessionToken'])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Below we specify the s3 URL to the data asset in Earthdata Cloud. This URL can be found via Earthdata Search or programmatically through the CMR and CMR-STAC APIs."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# Define S3 URL \n",
"s3_url = 's3://lp-prod-protected/EMITL2ARFL.001/EMIT_L2A_RFL_001_20220903T163129_2224611_012/EMIT_L2A_RFL_001_20220903T163129_2224611_012.nc'"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Open with the netCDF file using the s3fs package, then load the cloud asset into an xarray dataset, or use directly with `emit_xarray` function from `emit_tools`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# Open s3 url\n",
"fp = fs_s3.open(s3_url, mode='rb')\n",
"# Open dataset with xarray\n",
"ds = xr.open_dataset(fp) #Note this only opens the root group (reflectance)\n",
"ds"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# Open and Orthorectify\n",
"ds = emit_xarray(fp, ortho=True)\n",
"ds"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"#Plot Spatially\n",
"ds.reflectance.data[ds.reflectance.data == -9999] = np.nan # Mask out fill_values\n",
"ds.sel(wavelengths=650, method='nearest').hvplot.image(cmap='viridis', aspect = 'equal', frame_width=500, rasterize=True)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"#Plot Spectra at a Location\n",
"ds['reflectance'].data[:,:,ds['good_wavelengths'].data==0] = np.nan # Mask out bad bands\n",
"ds.sel(longitude=-61.833,latitude=-39.710,method='nearest').hvplot.line(y='reflectance',x='wavelengths', color='black', frame_width=400)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## S3 Access for any Notebook in this Repository\n",
"\n",
"Add the two code blocks below to any how-to or tutorial notebook, by replacing the block that sets the local filepath(s) as `fp` with the two blocks below. The first block imports the additional packages required and retrieves temporary s3 credentials. The second uses `s3fs` to open the desired s3 URL and create an object readable by `xarray`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import requests\n",
"import s3fs\n",
"import earthaccess\n",
"\n",
"# Use Earthaccess to Manage NASA Earthdata Credentials and create .netrc if necessary\n",
"earthaccess.login(persist=True)\n",
"\n",
"# Get Temporary Credentials/Token\n",
"temp_creds_req = requests.get('https://data.lpdaac.earthdatacloud.nasa.gov/s3credentials').json() # use lpdaac credential endpoint for EMIT data\n",
"\n",
"# Create s3fs session\n",
"fs_s3 = s3fs.S3FileSystem(anon=False, \n",
" key=temp_creds_req['accessKeyId'], \n",
" secret=temp_creds_req['secretAccessKey'], \n",
" token=temp_creds_req['sessionToken'])"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Set s3 url and open\n",
"s3_url = 's3://lp-prod-protected/EMITL2ARFL.001/EMIT_L2A_RFL_001_20220903T163129_2224611_012/EMIT_L2A_RFL_001_20220903T163129_2224611_012.nc' # S3 URL to L2A Reflectance File used throughout tutorial\n",
"#s3_url_mask = 's3://lp-prod-protected/EMITL2ARFL.001/EMIT_L2A_RFL_001_20220903T163129_2224611_012/EMIT_L2A_MASK_001_20220903T163129_2224611_012.nc' # Only used for Quality How-to\n",
"fp = fs_s3.open(s3_url, mode='rb')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now you should be able to proceed as normal through the other Jupyter notebooks at the step opening the file with `emit_xarray` from the `emit_tools` module or `xarray.open_dataset` depending on the notebook."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Contact Info: \n",
"\n",
"Email: LPDAAC@usgs.gov \n",
"Voice: +1-866-573-3222 \n",
"Organization: Land Processes Distributed Active Archive Center (LP DAAC)¹ \n",
"Website: <https://lpdaac.usgs.gov/> \n",
"Date last modified: 08-03-2023 \n",
"\n",
"¹Work performed under USGS contract G15PD00467 for NASA contract NNG14HH33I. "
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.13"
},
"vscode": {
"interpreter": {
"hash": "3292b2aceff7d39327a7519422d4180a7c9b133202090f26e797e3dd8f2c7877"
}
}
},
"nbformat": 4,
"nbformat_minor": 4
}
================================================
FILE: python/how-tos/How_to_Extract_Area.ipynb
================================================
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"# How to: Extracting EMIT Spectra using a Shapefile/GeoJSON\n",
"\n",
"**Summary** \n",
"\n",
"In this notebook we will open a netCDF4 file from the Earth Surface Minteral Dust Source Investigation (EMIT) as an `xarray.Dataset`. We will then extract extract or clip to an area using a `.geojson` file (will also work with shapefile). The workflows outlined here will work with reflectance L2A or radiance L1B data.\n",
"\n",
"**Requirements:**\n",
"+ A NASA [Earthdata Login](https://urs.earthdata.nasa.gov/) account is required to download EMIT data \n",
"+ Selected the `emit_tutorials` environment as the kernel for this notebook.\n",
" + For instructions on setting up the environment, follow the the `setup_instructions.md` included in the `/setup/` folder of the repository. \n",
"\n",
"**Learning Objectives** \n",
"- How to open and EMIT Dataset as an `xarray.Dataset`\n",
"- How to extract values or clip an EMIT dataset to a region of interest\n",
"- How to write a new netCDF4 output using the clipped data"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Import the required Python libraries."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# Import Packages\n",
"import os\n",
"import earthaccess\n",
"import numpy as np\n",
"import xarray as xr\n",
"from osgeo import gdal\n",
"import rasterio as rio\n",
"import rioxarray as rxr\n",
"import hvplot.xarray\n",
"import hvplot.pandas\n",
"import holoviews as hv\n",
"import geopandas as gp\n",
"import sys\n",
"sys.path.append('../modules/')\n",
"from emit_tools import emit_xarray"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Login to your NASA Earthdata account and create a `.netrc` file using the `login` function from the `earthaccess` library. If you do not have an Earthdata Account, you can create one [here](https://urs.earthdata.nasa.gov/home). "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"earthaccess.login(persist=True)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"For this notebook we will download the files necessary using `earthaccess`. You can also access the data in place or stream it, but this can slow due to the file sizes. Provide a URL for an EMIT L2A Reflectance granule."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"url = 'https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/EMITL2ARFL.001/EMIT_L2A_RFL_001_20220903T163129_2224611_012/EMIT_L2A_RFL_001_20220903T163129_2224611_012.nc'"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Get an HTTPS Session using your earthdata login, set a local path to save the file, and download the granule asset - This may take a while, the reflectance file is approximately 1.8 GB. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Get requests https Session using Earthdata Login Info\n",
"fs = earthaccess.get_requests_https_session()\n",
"# Retrieve granule asset ID from URL (to maintain existing naming convention)\n",
"granule_asset_id = url.split('/')[-1]\n",
"# Define Local Filepath\n",
"fp = f'../../data/{granule_asset_id}'\n",
"# Download the Granule Asset if it doesn't exist\n",
"if not os.path.isfile(fp):\n",
" with fs.get(url,stream=True) as src:\n",
" with open(fp,'wb') as dst:\n",
" for chunk in src.iter_content(chunk_size=64*1024*1024):\n",
" dst.write(chunk)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Open the file downloaded and defined as `fp`. To do this, we will use the `emit_xarray` function from the `emit_tools` module. This module contains a few helpful functions that can be used with EMIT data."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"ds = emit_xarray(fp, ortho=True)\n",
"ds"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Using the `read_file()` function from `geopandas`, read in the `.geojson` file containing the polygon you wish to extract."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"shape = gp.read_file('../../data/isla_gaviota.geojson')\n",
"shape"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"First, mask out fill-values by setting them to `np.nan`, then plot the polygon we've loaded overlayed on a plot of the dataset."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# Set fill values to NaN\n",
"ds.reflectance.data[ds.reflectance.data == -9999] = np.nan\n",
"# Select band and plot with polygon\n",
"ds_850 = ds.sel(wavelengths=850,method='nearest')\n",
"ds_850.hvplot.image(cmap='viridis', frame_height=600, frame_width=600, geo=True, crs='EPSG:4326').opts(title=f\"Reflectance at {ds_850.wavelengths.data:.3f} ({ds_850.wavelengths.units})\")*shape.hvplot(color='#d95f02', alpha=0.5, geo=True, crs='EPSG:4326')"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Use the `clip` function from `rasterio` to clip the dataset to polygons from the `geopandas.geodataframe`. Setting `all_touched` to `True` will include pixels that intersected with the edges of the polygon. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"clipped = ds.rio.clip(shape.geometry.values,shape.crs, all_touched=True)\n",
"clipped"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"To view the clipped image, select a band from the `clipped` dataset and plot it spatially."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"clipped_850 = clipped.sel(wavelengths=850,method='nearest')\n",
"clipped_850.hvplot.image(cmap='viridis', frame_height=600, geo=True, tiles='ESRI').opts(\n",
" title=f'Reflectance at {clipped_850.wavelengths.data:.3f} ({clipped_850.wavelengths.units})',\n",
" xlabel='Longitude', ylabel='Latitude')"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Now we can save the clipped `xarray.Dataset` as a netCDF4 output that can be reopened using the `xarray.open_dataset` function. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"clipped.to_netcdf('../../data/clipped_data.nc')\n",
"# Example for Opening \n",
"#ds = xr.open_dataset('../../data/clipped_data.nc')"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Contact Info: \n",
"\n",
"Email: LPDAAC@usgs.gov \n",
"Voice: +1-866-573-3222 \n",
"Organization: Land Processes Distributed Active Archive Center (LP DAAC)¹ \n",
"Website: <https://lpdaac.usgs.gov/> \n",
"Date last modified: 03-13-2024 \n",
"\n",
"¹Work performed under USGS contract G15PD00467 for NASA contract NNG14HH33I. "
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.13"
},
"vscode": {
"interpreter": {
"hash": "3292b2aceff7d39327a7519422d4180a7c9b133202090f26e797e3dd8f2c7877"
}
}
},
"nbformat": 4,
"nbformat_minor": 4
}
================================================
FILE: python/how-tos/How_to_Extract_Points.ipynb
================================================
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"# How to: Extracting EMIT Spectra at Specified Coordinates\n",
"\n",
"**Summary** \n",
"\n",
"In this notebook we will open a netCDF4 file from the Earth Surface Minteral Dust Source Investigation (EMIT) as an `xarray.Dataset`. We will then extract the spectra at point coordinates from a `.csv` as a dataframe, then save and plot the data.\n",
"\n",
"**Requirements:**\n",
"+ A NASA [Earthdata Login](https://urs.earthdata.nasa.gov/) account is required to download EMIT data \n",
"+ Selected the `emit_tutorials` environment as the kernel for this notebook.\n",
" + For instructions on setting up the environment, follow the the `setup_instructions.md` included in the `/setup/` folder of the repository. \n",
"\n",
"**Learning Objectives**\n",
"+ How to open an EMIT file as an `xarray.Dataset`\n",
"+ How to extract spectra at coordinates listed in a `.csv` file "
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Import the required Python libraries."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import Packages\n",
"import sys\n",
"import os\n",
"import earthaccess\n",
"import numpy as np\n",
"import pandas as pd\n",
"import xarray as xr\n",
"import hvplot.pandas\n",
"import hvplot.xarray\n",
"import holoviews as hv\n",
"sys.path.append('../modules/')\n",
"from emit_tools import emit_xarray"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Login to your NASA Earthdata account and create a `.netrc` file using the `login` function from the `earthaccess` library. If you do not have an Earthdata Account, you can create one [here](https://urs.earthdata.nasa.gov/home). "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"earthaccess.login(persist=True)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"For this notebook we will download the files necessary using `earthaccess`. You can also access the data in place or stream it, but this can slow due to the file sizes. Provide a URL for an EMIT L2A Reflectance granule."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"url = 'https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/EMITL2ARFL.001/EMIT_L2A_RFL_001_20220903T163129_2224611_012/EMIT_L2A_RFL_001_20220903T163129_2224611_012.nc'"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Get an HTTPS Session using your earthdata login, set a local path to save the file, and download the granule asset - This may take a while, the reflectance file is approximately 1.8 GB."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Get requests https Session using Earthdata Login Info\n",
"fs = earthaccess.get_requests_https_session()\n",
"# Retrieve granule asset ID from URL (to maintain existing naming convention)\n",
"granule_asset_id = url.split('/')[-1]\n",
"# Define Local Filepath\n",
"fp = f'../../data/{granule_asset_id}'\n",
"# Download the Granule Asset if it doesn't exist\n",
"if not os.path.isfile(fp):\n",
" with fs.get(url,stream=True) as src:\n",
" with open(fp,'wb') as dst:\n",
" for chunk in src.iter_content(chunk_size=64*1024*1024):\n",
" dst.write(chunk)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Open the file downloaded and defined as `fp`. To do this, we will use the `emit_tools` module which contains a few helpful functions that can be used with EMIT data. Use the `ortho=True` option to orthorectify the dataset."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"ds = emit_xarray(fp, ortho=True)\n",
"ds"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Now open the `.csv` included in the `/data/` directory as a `pandas.dataframe`.\n",
"\n",
"> Note: The category values here are arbitrary and included as an example of an additional column users may want."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"points = pd.read_csv('../../data/sample_coords.csv')\n",
"points"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Make a plot to visualize the points we're going to select on the dataset. Here we use the reflectance values at 850nm as our basemap. To mask fill-values, assign them to `np.nan`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Assign fill-values to NaN\n",
"ds.reflectance.data[ds.reflectance.data == -9999] = np.nan\n",
"# Select a single band and plot points on top\n",
"ds.sel(wavelengths=850,method='nearest').hvplot.image(cmap='greys', frame_height=600, frame_width=600, geo=True, crs='EPSG:4326').opts(title=f\"Example Points\")*\\\n",
"points.hvplot.points(x='Longitude',y='Latitude', by='ID', color=hv.Cycle('Dark2'), geo=True, crs='EPSG:4326')"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Set the `points` dataframe index as `ID` to utilize our existing point ID's as an index."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"points = points.set_index(['ID'])"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Convert the dataframe to an `xarray.Dataset`"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"xp = points.to_xarray()\n",
"xp"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Select the data from our EMIT dataset using the Latitude and Longitude coordinates from our point dataset, then convert the output to a `pandas.dataframe`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"extracted = ds.sel(latitude=xp.Latitude,longitude=xp.Longitude, method='nearest').to_dataframe()\n",
"extracted"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"The output is a longform dataframe using the `'ID'` field as an index. This is missing our `'Category'` column from our original dataframe. Use the `pd.join` function to add the `'Category'` column to our dataset using `'ID'` as an index."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"df = extracted.join(points['Category'], on=['ID'])\n",
"df"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Now we have a dataframe containing our initial data, in addition to the extracted point data. This a a good place to save an output as a `.csv`. Go ahead and do that below."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"df.to_csv('../../data/example_out.csv')"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"We can use our dataframe to plot the reflectance data we extracted, but first, mask the reflectance values of `-0.01`, which represent deep water vapor absorption regions. To do this, assign values where the reflectance = `-0.01` to `np.nan`. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Data values are slightly different than -0.01 because of floating point precision, which requires a tolerance to be set\n",
"df['reflectance'] = df['reflectance'].where(abs(df['reflectance'] - (-0.01)) > 1e-8, np.nan)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Plot the data using `hvplot`. We can use `by=` to separate the reflectances by their `ID`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"df.hvplot(x='wavelengths',y='reflectance', by=['ID'], color=hv.Cycle('Dark2'), frame_height=400, frame_width=600).opts(title='Example Points - Reflectance', xlabel='Wavelengths (nm)',ylabel='Reflectance')"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Contact Info: \n",
"\n",
"Email: LPDAAC@usgs.gov \n",
"Voice: +1-866-573-3222 \n",
"Organization: Land Processes Distributed Active Archive Center (LP DAAC)¹ \n",
"Website: <https://lpdaac.usgs.gov/> \n",
"Date last modified: 03-13-2024 \n",
"\n",
"¹Work performed under USGS contract G15PD00467 for NASA contract NNG14HH33I. "
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.13"
},
"vscode": {
"interpreter": {
"hash": "3292b2aceff7d39327a7519422d4180a7c9b133202090f26e797e3dd8f2c7877"
}
}
},
"nbformat": 4,
"nbformat_minor": 4
}
================================================
FILE: python/how-tos/How_to_Orthorectify.ipynb
================================================
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"# How To: Orthorectify EMIT Data\n",
"\n",
">**Warning: This notebook uses a lot of memory, if using 2i2c, you will need a 29.7 GB, 3.75 CPU instance.**\n",
"\n",
"EMIT Data is provided in non-orthorectified format to reduce data size. The `location` group contains latitude and longitude values of each pixel as well as a geometric lookup table (GLT) that can be used to orthocorrect the imagery. The GLT is an array that provides relative downtrack and crosstrack reference locations from the raw scene to facilitate fast projection of the dataset.\n",
"\n",
"This notebook will walk through using the `emit_tools` module to do the orthorectification process as well as a in depth walkthrough of the process.\n",
"\n",
"If you are planning to convert the EMIT `netCDF4` files to `.envi` format, the `reformat.py` available in the [emit-sds/emit-utils](https://github.com/emit-sds/emit-utils) repository can be used to orthorectify the imagery during the conversion process.\n",
"\n",
"**Requirements:**\n",
"+ A NASA [Earthdata Login](https://urs.earthdata.nasa.gov/) account is required to download EMIT data \n",
"+ Selected the `emit_tutorials` environment as the kernel for this notebook.\n",
" + For instructions on setting up the environment, follow the the `setup_instructions.md` included in the `/setup/` folder of the repository. \n",
"\n",
"**Learning Objectives**\n",
"+ How to open an EMIT file as an `xarray.Dataset`\n",
"+ How to orthorectify an EMIT file"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## 1. Using emit_xarray to Orthorectify\n",
"\n",
"Import the required Python libraries."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import Packages\n",
"import os\n",
"import earthaccess\n",
"import netCDF4 as nc\n",
"from osgeo import gdal\n",
"import numpy as np\n",
"import xarray as xr\n",
"import hvplot.xarray\n",
"import holoviews as hv\n",
"import sys\n",
"sys.path.append('../modules/')\n",
"from emit_tools import emit_xarray"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Login to your NASA Earthdata account and create a `.netrc` file using the `login` function from the `earthaccess` library. If you do not have an Earthdata Account, you can create one [here](https://urs.earthdata.nasa.gov/home). "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"earthaccess.login(persist=True)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"For this notebook we will download the files necessary using `earthaccess`. You can also access the data in place or stream it, but this can slow due to the file sizes. Provide a URL for an EMIT L2A Reflectance granule."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"url = 'https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/EMITL2ARFL.001/EMIT_L2A_RFL_001_20220903T163129_2224611_012/EMIT_L2A_RFL_001_20220903T163129_2224611_012.nc'"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Get an HTTPS Session using your earthdata login, set a local path to save the file, and download the granule asset - This may take a while, the reflectance file is approximately 1.8 GB. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Get requests https Session using Earthdata Login Info\n",
"fs = earthaccess.get_requests_https_session()\n",
"# Retrieve granule asset ID from URL (to maintain existing naming convention)\n",
"granule_asset_id = url.split('/')[-1]\n",
"# Define Local Filepath\n",
"fp = f'../../data/{granule_asset_id}'\n",
"# Download the Granule Asset if it doesn't exist\n",
"if not os.path.isfile(fp):\n",
" with fs.get(url,stream=True) as src:\n",
" with open(fp,'wb') as dst:\n",
" for chunk in src.iter_content(chunk_size=64*1024*1024):\n",
" dst.write(chunk)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Use the `emit_xarray` function imported from the `emit_tools` module to read in the data.\n",
"\n",
"The `emit_xarray` function will open and and place data from the groups of an EMIT dataset into an `xarray.Dataset`. It includes an `ortho` option which is set to `False` by default. This function reads the root group, `location` group, and `sensor_band_parameters` groups from the EMIT dataset. It then flattens all of the variables contained within those 3 groups into a single `xarray_dataset`. When `ortho=True`, it uses the GLT layers to build a lat/lon grid and places the values from the `reflectance` root group into the grid. The lat and lon grid is used as dimensional coordinates in the output `xarray.dataset` while the `wavelengths` and `fwhm` from the `sensor_band_parameters` group are used as non-dimensional coordinates. The `reflectance` data from the root group of the netCDF is then orthorectified and added as a variable in the `xarray.Dataset`.\n",
"\n",
"Read in a dataset using the `emit_xarray` function."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"ds = emit_xarray(fp, ortho=True)\n",
"ds"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's visualize a red band. First assign fill-values to `np.nan`, then select and plot a band using the `sel` function from `xarray` and the `hvplot` library. When we orthorectify (place data on a grid and rotate it) we end up with a bunch of extra pixels that have no data. These are filled with -9999, which will distort our visualization unless we assign them to `np.nan` or mask them."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Assign fill-values to nan\n",
"ds.reflectance.data[ds.reflectance.data == -9999] = np.nan\n",
"# Get red band and plot example\n",
"ds.sel(wavelengths=650,method='nearest').hvplot.image(cmap='viridis', aspect = 'equal', frame_width=500, rasterize=True)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"The orthorectified dataset can also be saved as a netCDF4 output that can be reopened using the `xarray.open_dataset` function if so desired using the cell below."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"ds.to_netcdf('../../data/example_emit_ortho_out.nc')\n",
"# Example for Opening \n",
"# ds = xr.open_dataset('../data/example_emit_ortho_out.nc')"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Now open a dataset with `ortho=False`. Notice that the data is not on a Lat/Lon grid, instead each downtrack and crosstrack pixel center is geotagged with a latitude and longitude."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"ds = emit_xarray(fp,ortho=False)\n",
"ds"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"We can see the difference in structure and the plotted data if the dataset is orthorectified or not. It's also not necessary to mask fill-values generated by the orthorectification process."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Get red band and plot example\n",
"ds.sel(wavelengths=850,method='nearest').hvplot.image(cmap='viridis', aspect = 'equal', frame_width=500, rasterize=True)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## 2. Detailed Explanation of Orthorectification and xarray.Dataset Construction\n",
"\n",
"As mentioned, the orthorectification process for EMIT data utilizes a premade geometric lookup table (GLT) that is included in the `location` group of the file. `glt_x` and `glt_y` contain the pixel indices from the raw dataset to construct an orthocorrected array. These indices data can be pulled from the raw dataset to populate an orthocorrected array using numpy broadcasting.\n",
"\n",
"To do this, first read in the `location` group data. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"loc = xr.open_dataset(fp, engine = 'h5netcdf', group='location')"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Next, build a numpy array using the GLT arrays from the `location` group. Use the `stack` function to stack the x and y GLT arrays and use the function `nan_to_num` to set the NaN values to the `GLT_NODATA_VALUE`, which is 0 for EMIT datasets. After doing this, you can check the shape of the array to see the expected height and width in pixels of the image after orthorectification."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"GLT_NODATA_VALUE=0\n",
"glt_array = np.nan_to_num(np.stack([loc['glt_x'].data,loc['glt_y'].data],axis=-1),nan=GLT_NODATA_VALUE).astype(int)\n",
"glt_array.shape"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"After we have a GLT array, we need to open and create a numpy array from the reflectance dataset. Open the dataset using `xarray` and assign the data to an array. Here you can check the shape of the array to see the dimensions of the uncorrected data."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"ds = xr.open_dataset(fp,engine = 'h5netcdf')\n",
"ds_array = ds['reflectance'].data\n",
"ds_array.shape"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Now that we have both arrays, we need to assign a fill value for positions that have no data. We can then construct an empty array with the dimensions we desire (GLT shape) and populate it with values from the dataset. To do this we first use np.zeros to create an array of all zeros then add the `fill_value` to the array of zeros to change the zeros to the `fill_value`.\n",
"\n",
"Next we can build an array of valid locations by omitting the portions of the array containing the `GLT_NODATA_VALUES`.\n",
"\n",
"After that, change the indexing of the GLT array which is one based to zero based by subtracting 1 from them.\n",
"\n",
"Lastly, we can use broadcasting/indexing to pull through the values we want from the dataset into our new output array."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Build Output Dataset\n",
"fill_value = -9999\n",
"out_ds = np.zeros((glt_array.shape[0], glt_array.shape[1], ds_array.shape[-1]), dtype=np.float32) + fill_value\n",
"valid_glt = np.all(glt_array != GLT_NODATA_VALUE, axis=-1)\n",
"# Adjust for One based Index\n",
"glt_array[valid_glt] -= 1 \n",
"# Use indexing/broadcasting to populate array cells with 0 values\n",
"out_ds[valid_glt, :] = ds_array[glt_array[valid_glt, 1], glt_array[valid_glt, 0], :]\n",
"out_ds.shape"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"After this step, we have an array of values that is orthorectified, but if we want to have a grid of lat/lon values we still need to calculate them using the geotransform. We can retrieve this information from the dataset attributes (`ds.attrs['geotransform']`)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"GT = ds.geotransform\n",
"GT"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"The geotransform consists of 2 formulas with 6 coefficients used to calculate the xy-grid to project the dataset. \n",
"\n",
"`x_geo = GT[0] + x * GT[1] + y * GT[2]` \n",
"\n",
"`y_geo = GT[3] + x * GT[4] + y * GT[5]`\n",
"\n",
"`GT[0]` - The x-coordinate of the upper-left corner of the upper-left pixel \n",
"`GT[1]` - W-E pixel width \n",
"`GT[2]` - Row rotation (zero for North up images) \n",
"`GT[3]` - The y-coordinate of the upper-left corner of the upper-left pixel \n",
"`GT[4]` - Column rotation (zero for North up images) \n",
"`GT[5]` - N-S pixel height (negative value for a north-up image) \n",
"\n",
"Create empty arrays for the x and y (lon and lat) based on the dimensions of the dataset.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Create Array for Lat and Lon and fill\n",
"dim_x = loc.glt_x.shape[1]\n",
"dim_y = loc.glt_x.shape[0]\n",
"lon = np.zeros(dim_x)\n",
"lat = np.zeros(dim_y)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"After we have an empty array with the correct dimensions, we can populate it with values using the geotransform formula to build a lat/lon grid for plotting the orthorectified data. We can remove the `GT[2]` and `GT[4]` terms from the equation since our orthorectified image is North up. We also want to shift the geotransform by half a pixel since we want to build an array of pixel centers. We do this by adding half a pixel width to `GT[0]` and `GT[3]`. Calclutate the latitude and longitude for each row (x_dim) and column (y_dim) of the dataset."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"for x in np.arange(dim_x):\n",
" x_geo = (GT[0]+0.5*GT[1]) + x * GT[1]\n",
" lon[x] = x_geo\n",
"for y in np.arange(dim_y):\n",
" y_geo = (GT[3]+0.5*GT[5]) + y * GT[5]\n",
" lat[y] = y_geo\n",
"\n",
"lat, lon"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"As a last step we can place this and remaining information from the EMIT dataset, such as attributes and wavelengths into an `xarray.Dataset` if desired. To do this, build dictionaries for the coordinates and variables components of the dataset. In this example, the variable would be `reflectance`. First, we'll want to read in the `sensor_band_parameters` group to get the wavelength and fwhm info, then build the dictionaries.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"wvl = xr.open_dataset(fp, engine = 'h5netcdf', group='sensor_band_parameters')\n",
"coords = {'lat':(['lat'],lat), 'lon':(['lon'],lon), **wvl.variables} ## ** upacks the existing dictionary from the wvl dataset.\n",
"data_vars = {'reflectance':(['lat','lon','bands'], out_ds)}"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Now build an `xarray.Dataset` using the dictionaries, then add attributes to the dataset from the non-orthocorrected data. We also can add the crs using the `rio.write_crs` function, to allow `rasterio` and `rioxarray` to recognize the CRS for future use.\n",
"\n",
">Note: the wavelength and fwhm are included as non-dimensional coordinates because they are independent of Lat/Lon."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"out_xr = xr.Dataset(data_vars=data_vars, coords=coords, attrs= ds.attrs)\n",
"out_xr['reflectance'].attrs = ds['reflectance'].attrs\n",
"out_xr.coords['lat'].attrs = loc['lat'].attrs\n",
"out_xr.coords['lon'].attrs = loc['lon'].attrs\n",
"out_xr.rio.write_crs(ds.spatial_ref,inplace=True) # Add CRS in easily recognizable format\n",
"out_xr"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Now we have an orthorectified dataset with coordinates that can be plotted. We can also mask the `fill_values` of -9999 to make a clearer figure."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"out_xr['reflectance'].data[out_xr['reflectance'].data == fill_value] = np.nan\n",
"out_xr.sel(bands=63).hvplot.image(cmap='viridis', aspect = 'equal', frame_width=500, rasterize=True)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Contact Info: \n",
"\n",
"Email: LPDAAC@usgs.gov \n",
"Voice: +1-866-573-3222 \n",
"Organization: Land Processes Distributed Active Archive Center (LP DAAC)¹ \n",
"Website: <https://lpdaac.usgs.gov/> \n",
"Date last modified: 05-30-2023 \n",
"\n",
"¹Work performed under USGS contract G15PD00467 for NASA contract NNG14HH33I. "
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.13"
},
"vscode": {
"interpreter": {
"hash": "3292b2aceff7d39327a7519422d4180a7c9b133202090f26e797e3dd8f2c7877"
}
}
},
"nbformat": 4,
"nbformat_minor": 4
}
================================================
FILE: python/how-tos/How_to_find_EMIT_data_using_CMR_API.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# How to: Find and Access EMIT Data\n",
"\n",
"**Summary** \n",
"\n",
"This notebook will explain how to access Earth Surface Minteral Dust Source Investigation (EMIT) data programmaticly using NASA's CMR API. The Common Metadata Repository (CMR) is a metadata system that catalogs Earth Science data and associated metadata records. The CMR Application Programming Interface (API) provides programatic search capabilities through CMR's vast metadata holdings using various parameters and keywords. When querying NASA's CMR, there is a limit of 1 million granules matched and only 2000 granules returned per page. \n",
"\n",
"**Requirements:**\n",
"+ A NASA [Earthdata Login](https://urs.earthdata.nasa.gov/) account is required to download EMIT data \n",
"+ Selected the `emit_tutorials` environment as the kernel for this notebook.\n",
" + For instructions on setting up the environment, follow the the `setup_instructions.md` included in the `/setup/` folder of the repository. \n",
"\n",
"**Learning Objectives** \n",
"- How to find EMIT data using NASA's CMR API\n",
"- How to download programmatically "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Import the required packages"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"import os\n",
"import requests\n",
"import earthaccess\n",
"import pandas as pd\n",
"import datetime as dt\n",
"import geopandas\n",
"from shapely.geometry import MultiPolygon, Polygon, box"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Obtaining the Concept ID\n",
"\n",
"NASA EarthData's unique ID for this dataset (called Concept ID) is needed for searching the dataset. The dataset Digital Object Identifier or DOI can be used to obtain the Concept ID. DOIs can be found by clicking the `Citation` link on the LP DAAC's [EMIT Product Pages](https://lpdaac.usgs.gov/product_search/?query=emit&view=cards&sort=title)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"doi = '10.5067/EMIT/EMITL2ARFL.001'# EMIT L2A Reflectance\n",
"\n",
"# CMR API base url\n",
"cmrurl='https://cmr.earthdata.nasa.gov/search/' \n",
"\n",
"doisearch = cmrurl + 'collections.json?doi=' + doi\n",
"concept_id = requests.get(doisearch).json()['feed']['entry'][0]['id']\n",
"print(concept_id)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This is the unique NASA-given concept ID for the EMIT L2A Reflectance dataset, which can be used to retrieve relevant files (or granules)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Searching using CMR API\n",
"\n",
"When searching the CMR API, users can provide spatial bounds and date-time ranges to narrow their search. These spatial bounds can be either, points, a bounding box, or a polygon. \n",
"\n",
"Specify start time and dates and reformat them to the structure necessary for searching CMR."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# Temporal Bound - Year, month, day. Hour, minutes, and seconds (ZULU) can also be included \n",
"start_date = dt.datetime(2022, 9, 3)\n",
"end_date = dt.datetime(2022, 9, 3, 23, 23, 59) \n",
"\n",
"# CMR formatted start and end times\n",
"dt_format = '%Y-%m-%dT%H:%M:%SZ'\n",
"temporal_str = start_date.strftime(dt_format) + ',' + end_date.strftime(dt_format)\n",
"print(temporal_str)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The CMR API only allows 2000 results to be shown at a time. Using `page_num` allows a user to loop through the search result pages. The sections below walk through using Points, Bounding Boxes, and Polygons to spatially constrain a search made using the CMR API. \n",
"\n",
"### Search using Points\n",
"\n",
"To search using a point we specify a latitude and longitude."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# Search using a Point\n",
"\n",
"lon = -62.1123\n",
"lat = -39.89402\n",
"point_str = str(lon) +','+ str(lat)\n",
"\n",
"page_num = 1\n",
"page_size = 2000 # CMR page size limit\n",
"\n",
"granule_arr = []\n",
"\n",
"while True:\n",
" \n",
" # defining parameters\n",
" cmr_param = {\n",
" \"collection_concept_id\": concept_id, \n",
" \"page_size\": page_size,\n",
" \"page_num\": page_num,\n",
" \"temporal\": temporal_str,\n",
" \"point\":point_str\n",
" }\n",
"\n",
" granulesearch = cmrurl + 'granules.json'\n",
" response = requests.post(granulesearch, data=cmr_param)\n",
" granules = response.json()['feed']['entry']\n",
" \n",
" if granules:\n",
" for g in granules:\n",
" granule_urls = ''\n",
" granule_poly = ''\n",
" \n",
" # read cloud cover\n",
" cloud_cover = g['cloud_cover']\n",
" \n",
" # reading bounding geometries\n",
" if 'polygons' in g:\n",
" polygons= g['polygons']\n",
" multipolygons = []\n",
" for poly in polygons:\n",
" i=iter(poly[0].split (\" \"))\n",
" ltln = list(map(\" \".join,zip(i,i)))\n",
" multipolygons.append(Polygon([[float(p.split(\" \")[1]), float(p.split(\" \")[0])] for p in ltln]))\n",
" granule_poly = MultiPolygon(multipolygons)\n",
" \n",
" # Get https URLs to .nc files and exclude .dmrpp files\n",
" granule_urls = [x['href'] for x in g['links'] if 'https' in x['href'] and '.nc' in x['href'] and '.dmrpp' not in x['href']]\n",
" # Add to list\n",
" granule_arr.append([granule_urls, cloud_cover, granule_poly])\n",
" \n",
" page_num += 1\n",
" else: \n",
" break\n",
"print(granule_arr)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Search using a bounding box\n",
"For this we'll use a bounding box along the coast of Argentina with a bottom left corner of -62.1123 Longitude, -39.89402 Latitude, and a top right corner of -61.70801 Longitude and -39.57769 Latitude."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# Search Using a Bounding Box\n",
"bound = (-62.1123, -39.89402, -61.70801, -39.57769) \n",
"bound_str = ','.join(map(str,bound))\n",
"\n",
"page_num = 1\n",
"page_size = 2000 # CMR page size limit\n",
"\n",
"granule_arr = []\n",
"\n",
"while True:\n",
" \n",
" # defining parameters\n",
" cmr_param = {\n",
" \"collection_concept_id\": concept_id, \n",
" \"page_size\": page_size,\n",
" \"page_num\": page_num,\n",
" \"temporal\": temporal_str,\n",
" \"bounding_box[]\":bound_str\n",
" }\n",
"\n",
" granulesearch = cmrurl + 'granules.json'\n",
" response = requests.post(granulesearch, data=cmr_param)\n",
" granules = response.json()['feed']['entry']\n",
" \n",
" if granules:\n",
" for g in granules:\n",
" granule_urls = ''\n",
" granule_poly = ''\n",
" \n",
" # read cloud cover\n",
" cloud_cover = g['cloud_cover']\n",
" \n",
" # reading results bounding geometries\n",
" if 'polygons' in g:\n",
" polygons= g['polygons']\n",
" multipolygons = []\n",
" for poly in polygons:\n",
" i=iter(poly[0].split (\" \"))\n",
" ltln = list(map(\" \".join,zip(i,i)))\n",
" multipolygons.append(Polygon([[float(p.split(\" \")[1]), float(p.split(\" \")[0])] for p in ltln]))\n",
" granule_poly = MultiPolygon(multipolygons)\n",
" \n",
" # Get https URLs to .nc files and exclude .dmrpp files\n",
" granule_urls = [x['href'] for x in g['links'] if 'https' in x['href'] and '.nc' in x['href'] and '.dmrpp' not in x['href']]\n",
" # Add to list\n",
" granule_arr.append([granule_urls, cloud_cover, granule_poly])\n",
" \n",
" page_num += 1\n",
" else: \n",
" break\n",
"print(granule_arr)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Search a Polygon\n",
"\n",
"A polygon can also be used to spatially search using the CMR API. A shapefile, geojson, or other format can be opened as a geopandas dataframe, then reformatted to a geojson format to be sent as a parameter in the CMR search. Note that very complex shapefiles must be simplified, there is a 5000 coordinate limit."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# Search using a Polygon\n",
"polygon = geopandas.read_file('../../data/isla_gaviota.geojson')\n",
"geojson = {\"shapefile\": (\"isla_gaviota.geojson\", polygon.geometry.to_json(), \"application/geo+json\")}\n",
"\n",
"page_num = 1\n",
"page_size = 2000 # CMR page size limit\n",
"\n",
"granule_arr = []\n",
"\n",
"while True:\n",
" \n",
" # defining parameters\n",
" cmr_param = {\n",
" \"collection_concept_id\": concept_id, \n",
" \"page_size\": page_size,\n",
" \"page_num\": page_num,\n",
" \"temporal\": temporal_str,\n",
" \"simplify-shapefile\": 'true' # this is needed to bypass 5000 coordinates limit of CMR\n",
" }\n",
"\n",
" granulesearch = cmrurl + 'granules.json'\n",
" response = requests.post(granulesearch, data=cmr_param, files=geojson)\n",
" granules = response.json()['feed']['entry']\n",
" \n",
" if granules:\n",
" for g in granules:\n",
" granule_urls = ''\n",
" granule_poly = ''\n",
" \n",
" # read granule title and cloud cover\n",
" granule_name = g['title']\n",
" cloud_cover = g['cloud_cover']\n",
" \n",
" # reading bounding geometries\n",
" if 'polygons' in g:\n",
" polygons= g['polygons']\n",
" multipolygons = []\n",
" for poly in polygons:\n",
" i=iter(poly[0].split (\" \"))\n",
" ltln = list(map(\" \".join,zip(i,i)))\n",
" multipolygons.append(Polygon([[float(p.split(\" \")[1]), float(p.split(\" \")[0])] for p in ltln]))\n",
" granule_poly = MultiPolygon(multipolygons)\n",
" \n",
" # Get https URLs to .nc files and exclude .dmrpp files\n",
" granule_urls = [x['href'] for x in g['links'] if 'https' in x['href'] and '.nc' in x['href'] and '.dmrpp' not in x['href']]\n",
" # Add to list\n",
" granule_arr.append([granule_urls, cloud_cover, granule_poly])\n",
" \n",
" page_num += 1\n",
" else: \n",
" break\n",
" \n",
"print(granule_arr)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"> Note: At the time this tutorial was made, all 3 searches, point, bounding box, and polygon should result in the same assets being returned.\n",
"\n",
"### Creating a Dataframe with the resulting Links\n",
"\n",
"A `pandas.dataframe` can be used to store the download URLs and geometries of each file. The EMIT L2A Reflectance and Uncertainty and Mask collection contains 3 assets per granule (reflectance, reflectance uncertainty, and masks). We can see when printing this list, that there are three assets that correspond to a single polygon. For the next step we will place these into a dataframe and 'explode' the dataframe to place each of these in a separate row. If we only want a subset of these assets, we can filter them out. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# creating a pandas dataframe\n",
"cmr_results_df = pd.DataFrame(granule_arr, columns=[\"asset_url\", \"cloud_cover\", \"granule_poly\"])\n",
"# Drop granules with empty geometry - if any exist\n",
"cmr_results_df = cmr_results_df[cmr_results_df['granule_poly'] != '']\n",
"# Expand so each row contains a single url \n",
"cmr_results_df = cmr_results_df.explode('asset_url')\n",
"# Name each asset based on filename\n",
"cmr_results_df.insert(0,'asset_name', cmr_results_df.asset_url.str.split('/',n=-1).str.get(-1))\n",
"\n",
"cmr_results_df"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"At this stage we can filter based on the assets that we want or the cloud cover. For this example lets say we are only interested in the Reflectance and the Mask. To filter by asset, we can match strings included in the asset name. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"cmr_results_df = cmr_results_df[cmr_results_df.asset_name.str.contains('_RFL_') | cmr_results_df.asset_name.str.contains('MASK')]\n",
"cmr_results_df"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"After filtering down to the assets you want, you can output a text file with the asset urls or save the entire dataframe, then use a utility such as wget or the DAAC Data Download Tool to download the files. To download you will need to set up NASA Earthdata Login authentication using a .netrc file. \n",
"\n",
"Save the asset urls to a textfile in the `/data/` folder."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# Save text file of asset urls\n",
"cmr_results_dfs = cmr_results_df[:-1].drop_duplicates(subset=['asset_url']) # Remove any duplicates\n",
"cmr_results_df.to_csv('../../data/emit_asset_urls.txt', columns = ['asset_url'], index=False, header = False)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Downloading Files using the list of URLS/Text File\n",
"\n",
"To download the files using Python, you can run the cell below. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Define input filepath\n",
"url_list_filepath = \"../../data/emit_asset_urls.txt\"\n",
"# Define output directory\n",
"output_directory = \"../../data/\"\n",
"\n",
"# Open Text file\n",
"with open(url_list_filepath, \"r\") as file:\n",
" file_list = file.read().splitlines()\n",
" file.close()\n",
"\n",
"# EDL Authentication/Create .netrc if necessary\n",
"earthaccess.login(persist=True)\n",
"# Get requests https Session using Earthdata Login Info\n",
"fs = earthaccess.get_requests_https_session()\n",
"# Retrieve granule asset ID from URL (to maintain existing naming convention)\n",
"for url in file_list:\n",
" granule_asset_id = url.split(\"/\")[-1]\n",
" # Define Local Filepath\n",
" fp = f\"{output_directory}{granule_asset_id}\"\n",
" # Download the Granule Asset if it doesn't exist\n",
" print(f\"Downloading {granule_asset_id}...\")\n",
" if not os.path.isfile(fp):\n",
" with fs.get(url, stream=True) as src:\n",
" with open(fp, \"wb\") as dst:\n",
" for chunk in src.iter_content(chunk_size=64 * 1024 * 1024):\n",
" dst.write(chunk)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"To download using wget, use the following in the command line."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"!wget -P ../../data/ -i ../../data/emit_asset_urls.txt"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Contact Info: \n",
"\n",
"Email: LPDAAC@usgs.gov \n",
"Voice: +1-866-573-3222 \n",
"Organization: Land Processes Distributed Active Archive Center (LP DAAC)¹ \n",
"Website: <https://lpdaac.usgs.gov/> \n",
"Date last modified: 03-22-2024 \n",
"\n",
"¹Work performed under USGS contract G15PD00467 for NASA contract NNG14HH33I. "
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.13"
},
"vscode": {
"interpreter": {
"hash": "3292b2aceff7d39327a7519422d4180a7c9b133202090f26e797e3dd8f2c7877"
}
}
},
"nbformat": 4,
"nbformat_minor": 4
}
================================================
FILE: python/how-tos/How_to_find_and_access_EMIT_data.ipynb
================================================
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# How to: Find and Access EMIT Data\n",
"\n",
"**Summary** \n",
"\n",
"There are currently 4 ways to find EMIT data:\n",
"\n",
"1. [EarthData Search](https://search.earthdata.nasa.gov/search)\n",
"2. [NASA's CMR API](https://www.earthdata.nasa.gov/eosdis/science-system-description/eosdis-components/cmr) (`earthaccess` uses this)\n",
"3. [NASA's CMR-STAC API](https://cmr.earthdata.nasa.gov/search/site/docs/search/stac)\n",
"3. [Visions Open Access Data Portal](https://earth.jpl.nasa.gov/emit/data/data-portal/coverage-and-forecasts/)\n",
"\n",
"This notebook will explain how to access Earth Surface Mineral Dust Source Investigation (EMIT) data programmaticly using the [earthaccess python library](https://github.com/nsidc/earthaccess). `earthaccess` is an easy to use library that reduces finding and downloading or streaming data over https or s3 to only a few lines of code. `earthaccess` searches NASA's Common Metadata Repository (CMR), a metadata system that catalogs Earth Science data and associated metadata records, then can be used to download granules or generate lists granule search result URLs.\n",
"\n",
"**Requirements:**\n",
"- A NASA [Earthdata Login](https://urs.earthdata.nasa.gov/) account is required to download EMIT data \n",
"- *No Python setup requirements if connected to the workshop cloud instance!*\n",
"- **Local Only** Set up Python Environment - See **setup_instructions.md** in the `/setup/` folder to set up a local compatible Python environment\n",
"\n",
"**Learning Objectives** \n",
"- How to get information about data collections using `earthaccess`\n",
"- How to search and access EMIT data using `earthaccess`"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Setup\n",
"Import the required packages"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"import os\n",
"import earthaccess\n",
"import numpy as np\n",
"import pandas as pd\n",
"import geopandas as gp\n",
"from shapely.geometry.polygon import orient\n",
"import xarray as xr\n",
"import sys\n",
"sys.path.append('../modules/')\n",
"from emit_tools import emit_xarray\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Authentication\n",
"\n",
"`earthaccess` creates and leverages Earthdata Login tokens to authenticate with NASA systems. Earthdata Login tokens expire after a month. To retrieve a token from Earthdata Login, you can either enter your username and password each time you use `earthaccess`, or use a `.netrc` file. A `.netrc` file is a configuration file that is commonly used to store login credentials for remote systems. If you don't have a `.netrc` or don't know if you have one or not, you can use the `persist` argument with the `login` function below to create or update an existing one, then use it for authentication.\n",
"\n",
"If you do not have an Earthdata Account, you can create one [here](https://urs.earthdata.nasa.gov/home). "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"auth = earthaccess.login(persist=True)\n",
"print(auth.authenticated)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"If you receive a message that your token has expired, use `refresh_tokens()` like below to generate a new one."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"# auth.refresh_tokens"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Searching for Collections\n",
"\n",
"The EMIT mission produces several collections or datasets available via the LP DAAC cloud archive.\n",
"\n",
"To view what's available, we can use the `search_datasets` function and with the `keyword` and and `provider` arguments. The `provider` is the data location, in this case `LPCLOUD`. Specifying the provider isn't necessary, but the \"emit\" keyword can be found in metadata for some other datasets, and additional collections may be returned.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Retrieve Collections\n",
"collections = earthaccess.search_datasets(provider='LPCLOUD', keyword='emit')\n",
"# Print Quantity of Results\n",
"print(f'Collections found: {len(collections)}')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"If you print the `collections` object you can explore all of the json metadata."
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [],
"source": [
"# # Print collections\n",
"# collections"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We can also create a list of the `short-name`, `concept-id`, and `version` of each result collection using list comprehension. These fields are important for specifying and searching for data within collections. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"collections_info = [\n",
" {\n",
" 'short_name': c.summary()['short-name'],\n",
" 'collection_concept_id': c.summary()['concept-id'],\n",
" 'version': c.summary()['version'],\n",
" 'entry_title': c['umm']['EntryTitle']\n",
" }\n",
" for c in collections\n",
"]\n",
"pd.set_option('display.max_colwidth', 150)\n",
"collections_info = pd.DataFrame(collections_info)\n",
"collections_info"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The collection `concept-id` is the best way to search for data within a collection, as this is unique to each collection. The `short-name` can be used as well, however the `version` should be passed as well as there can be multiple versions available with the same short name. After finding the collection you want to search, you can use the `concept-id` to search for granules within that collection."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Searching for Granules\n",
"\n",
"A `granule` can be thought of as a unique spatiotemporal grouping within a collection. To search for `granules`, we can use the `search_data` function from `earthaccess` and provide the arguments for our search. Its possible to specify search products using several criteria shown in the table below:\n",
"\n",
"|dataset origin and location|spatio temporal parameters|dataset metadata parameters|\n",
"|:---|:---|:---|\n",
"|archive_center|bounding_box|concept_id\n",
"|data_center|temporal|entry_title\n",
"|daac|point|keyword\n",
"|provider|polygon|version\n",
"|cloud_hosted|line|short_name\n",
"\n",
"### Point Search\n",
"\n",
"In this case, we specify the `shortname`, `point` coordinates, `temporal` range, and min and max `cloud_cover` percentages, as well as `count`, which limits the maximum number of results returned. "
]
},
{
"cell_type": "code",
"execution_count": 27,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# Search example using a Point\n",
"results = earthaccess.search_data(\n",
" short_name='EMITL2ARFL',\n",
" point=(-62.1123,-39.89402),\n",
" temporal=('2022-09-03','2022-09-04'),\n",
" cloud_cover=(0,90),\n",
" count=100\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Bounding Box Search\n",
"\n",
"You can also use a bounding box to search. To do this we will first open a geojson file containing our region of interest (ROI) then simplify it to a bounding box by getting the bounds and putting them into a Python object called a tuple. We will use the `total_bounds` property to get the bounding box of our ROI, and add that to a Python tuple, which is the expected data type for the bounding_box parameter `earthaccess` `search_data`."
]
},
{
"cel
gitextract_4n5_jjs8/
├── .gitignore
├── CHANGE_LOG.md
├── CITATION.cff
├── CODE_OF_CONDUCT.md
├── CONTRIBUTING.md
├── LICENSE
├── README.md
├── data/
│ ├── cuprite_bbox.geojson
│ ├── emit_data_urls.txt
│ ├── isla_gaviota.geojson
│ ├── methane_tutorial/
│ │ ├── emit20220815t042838_ch4_target
│ │ ├── methane_inout_points.csv
│ │ ├── plume_bbox.geojson
│ │ └── rad_band_ratio_points.csv
│ ├── min_uncert_urls.txt
│ ├── mineral_grouping_matrix_20230503.csv
│ ├── results_urls.txt
│ ├── rgb_browse_urls.txt
│ └── sample_coords.csv
├── guides/
│ ├── Getting_EMIT_Data_using_EarthData_Search.md
│ └── Streaming_cloud_optimized_geotiffs_using_QGIS.md
├── python/
│ ├── how-tos/
│ │ ├── How_to_Convert_to_ENVI.ipynb
│ │ ├── How_to_Direct_S3_Access.ipynb
│ │ ├── How_to_Extract_Area.ipynb
│ │ ├── How_to_Extract_Points.ipynb
│ │ ├── How_to_Orthorectify.ipynb
│ │ ├── How_to_find_EMIT_data_using_CMR_API.ipynb
│ │ ├── How_to_find_and_access_EMIT_data.ipynb
│ │ └── How_to_use_EMIT_Quality_data.ipynb
│ ├── modules/
│ │ ├── emit_tools.py
│ │ └── tutorial_utils.py
│ └── tutorials/
│ ├── Exploring_EMIT_L2A_Reflectance.html
│ ├── Exploring_EMIT_L2A_Reflectance.ipynb
│ ├── Finding_EMIT_L2B_Data.ipynb
│ ├── Generating_Methane_Spectral_Fingerprint.ipynb
│ ├── Visualizing_Methane_Plume_Timeseries.ipynb
│ └── Working_with_EMIT_L2B_Mineralogy.ipynb
└── setup/
├── prerequisites.md
├── setup_instructions.md
└── workshop_setup.md
SYMBOL INDEX (17 symbols across 2 files) FILE: python/modules/emit_tools.py function emit_xarray (line 32) | def emit_xarray(filepath, ortho=False, qmask=None, unpacked_bmask=None): function get_pixel_center_coords (line 125) | def get_pixel_center_coords(ds): function apply_glt (line 149) | def apply_glt(ds_array, glt_array, fill_value=-9999, GLT_NODATA_VALUE=0): function ortho_xr (line 180) | def ortho_xr(ds, GLT_NODATA_VALUE=0, fill_value=-9999): function quality_mask (line 267) | def quality_mask(filepath, quality_bands): function band_mask (line 297) | def band_mask(filepath): function write_envi (line 320) | def write_envi( function envi_header (line 498) | def envi_header(inputpath): function spatial_subset (line 525) | def spatial_subset(ds, gdf): function is_adjacent (line 618) | def is_adjacent(scene: str, same_orbit: list): function merge_emit (line 627) | def merge_emit(datasets: dict, gdf: gpd.GeoDataFrame): function ortho_browse (line 703) | def ortho_browse(url, glt, spatial_ref, geotransform, white_background=T... FILE: python/modules/tutorial_utils.py function convert_bounds (line 20) | def convert_bounds(bbox, invert_y=False): function flattent_column_names (line 32) | def flattent_column_names(df: pd.DataFrame) -> pd.DataFrame: function get_shapely_object (line 39) | def get_shapely_object( function list_metadata_fields (line 75) | def list_metadata_fields(results: List[earthaccess.search.DataGranule]) ... function results_to_geopandas (line 80) | def results_to_geopandas(
Condensed preview — 40 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (1,564K chars).
[
{
"path": ".gitignore",
"chars": 290,
"preview": "# python internals\n*.ipynb_checkpoints\n*__pycache__*\n\n# local development\n*.img*\n*.envi*\n*.hdr*\n*.nc*\n*.tif*\n*.pptx*\n/da"
},
{
"path": "CHANGE_LOG.md",
"chars": 4229,
"preview": "# Change Log\n\nAll notable changes to this project will be documented in this file.\nThe format is based on [Keep a Change"
},
{
"path": "CITATION.cff",
"chars": 2523,
"preview": "cff-version: 1.2.0\ntitle: EMIT Data Resources\nmessage: >-\n If you use this collection of resources in your research\n w"
},
{
"path": "CODE_OF_CONDUCT.md",
"chars": 1394,
"preview": "# Code of Conduct\n\n## 1. Our Commitment\n\nWe are dedicated to fostering a respectful environment for everyone contributin"
},
{
"path": "CONTRIBUTING.md",
"chars": 5898,
"preview": "# Contributing to Our GitHub\n\nThis page is your one-stop shop for uncovering how to contribute to our Github!\n\n## We Wan"
},
{
"path": "LICENSE",
"chars": 11356,
"preview": " Apache License\n Version 2.0, January 2004\n "
},
{
"path": "README.md",
"chars": 6998,
"preview": "# EMIT-Data-Resources \n\nWelcome to the EMIT-Data-Resources repository. This repository provides guides, short how-tos, "
},
{
"path": "data/cuprite_bbox.geojson",
"chars": 324,
"preview": "{\"type\":\"FeatureCollection\",\"features\":[{\"type\":\"Feature\",\"properties\":{},\"geometry\":{\"coordinates\":[[[-117.243092401980"
},
{
"path": "data/emit_data_urls.txt",
"chars": 340,
"preview": "https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/EMITL2ARFL.001/EMIT_L2A_RFL_001_20220903T163129_2224611_01"
},
{
"path": "data/isla_gaviota.geojson",
"chars": 447,
"preview": "{\"type\":\"FeatureCollection\",\"features\":[{\"type\":\"Feature\",\"properties\":{},\"geometry\":{\"coordinates\":[[[-62.1475835139190"
},
{
"path": "data/methane_tutorial/emit20220815t042838_ch4_target",
"chars": 10425,
"preview": "001 381.006 0.000000000000000000\n002 388.409 0.000000000000000000\n003 395.816 0.000000000000000000\n004 403.2"
},
{
"path": "data/methane_tutorial/methane_inout_points.csv",
"chars": 91,
"preview": "ID,in-plume,latitude,longitude\n0,1,39.4628,53.7743\n1,0,39.4697,53.7792\n2,0,39.4726,53.7797\n"
},
{
"path": "data/methane_tutorial/plume_bbox.geojson",
"chars": 395,
"preview": "{\n\"type\": \"FeatureCollection\",\n\"crs\": { \"type\": \"name\", \"properties\": { \"name\": \"urn:ogc:def:crs:OGC:1.3:CRS84\" } },\n\"fe"
},
{
"path": "data/methane_tutorial/rad_band_ratio_points.csv",
"chars": 113,
"preview": "ID,in-plume,x,y\n0,0,508,421\n1,1,508,420\n2,1,508,419\n3,0,508,418\n4,0,508,417\n5,0,508,416\n6,0,508,415\n7,0,508,414\n"
},
{
"path": "data/min_uncert_urls.txt",
"chars": 528,
"preview": "https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/EMITL2BMIN.001/EMIT_L2B_MIN_001_20230427T173309_2311711_01"
},
{
"path": "data/mineral_grouping_matrix_20230503.csv",
"chars": 56902,
"preview": "Index,Record,Filename,Name,URL,Group,Library,Calcite,Chlorite,Dolomite,Goethite,Gypsum,Hematite,Illite+Muscovite,Kaolini"
},
{
"path": "data/results_urls.txt",
"chars": 510,
"preview": "https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/EMITL2BMIN.001/EMIT_L2B_MIN_001_20230427T173309_2311711_01"
},
{
"path": "data/rgb_browse_urls.txt",
"chars": 504,
"preview": "https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-public/EMITL2ARFL.001/EMIT_L2A_RFL_001_20230427T173309_2311711_010/E"
},
{
"path": "data/sample_coords.csv",
"chars": 118,
"preview": "ID,Category,Latitude,Longitude\n0,1,-39.94,-62.36\n1,1,-39.75,-61.74\n2,3,-40,-62.1\n3,2,-39.89,-61.85\n4,3,-39.38,-62.03\n"
},
{
"path": "guides/Getting_EMIT_Data_using_EarthData_Search.md",
"chars": 3721,
"preview": "# Getting Started with Earth Surface Mineral Dust Source Investigation (EMIT) data in Earthdata Search\n\n**Note:** Users "
},
{
"path": "guides/Streaming_cloud_optimized_geotiffs_using_QGIS.md",
"chars": 4550,
"preview": "# Streaming NASA Earthdata Cloud-Optimized Geotiffs using QGIS\n\n**Note:** Users must have [NASA Earthdata Login](https:/"
},
{
"path": "python/how-tos/How_to_Convert_to_ENVI.ipynb",
"chars": 9570,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# How To: Convert EMIT .nc to .envi"
},
{
"path": "python/how-tos/How_to_Direct_S3_Access.ipynb",
"chars": 11303,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# How to: Use Direct S3 Access to w"
},
{
"path": "python/how-tos/How_to_Extract_Area.ipynb",
"chars": 8641,
"preview": "{\n \"cells\": [\n {\n \"attachments\": {},\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# How to: Ext"
},
{
"path": "python/how-tos/How_to_Extract_Points.ipynb",
"chars": 10063,
"preview": "{\n \"cells\": [\n {\n \"attachments\": {},\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# How to: Ext"
},
{
"path": "python/how-tos/How_to_Orthorectify.ipynb",
"chars": 18270,
"preview": "{\n \"cells\": [\n {\n \"attachments\": {},\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# How To: Ort"
},
{
"path": "python/how-tos/How_to_find_EMIT_data_using_CMR_API.ipynb",
"chars": 18176,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# How to: Find and Access EMIT Data"
},
{
"path": "python/how-tos/How_to_find_and_access_EMIT_data.ipynb",
"chars": 18565,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# How to: Find and Access EMIT Data"
},
{
"path": "python/how-tos/How_to_use_EMIT_Quality_data.ipynb",
"chars": 15150,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# How to: Use EMIT Quality Data\\n\","
},
{
"path": "python/modules/emit_tools.py",
"chars": 26279,
"preview": "\"\"\"\nThis Module has the functions related to working with an EMIT dataset. This includes doing things\nlike opening and f"
},
{
"path": "python/modules/tutorial_utils.py",
"chars": 3988,
"preview": "\"\"\"\nThis module contains various functions used within the Exploring EMIT\nL2B CH4 plume complexes notebook for searching"
},
{
"path": "python/tutorials/Exploring_EMIT_L2A_Reflectance.ipynb",
"chars": 31322,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# Exploring L2A Reflectance\\n\",\n "
},
{
"path": "python/tutorials/Finding_EMIT_L2B_Data.ipynb",
"chars": 1128824,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# Finding EMIT L2B Data\\n\",\n \"\\n"
},
{
"path": "python/tutorials/Generating_Methane_Spectral_Fingerprint.ipynb",
"chars": 22761,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# Generating Methane Spectral Finge"
},
{
"path": "python/tutorials/Visualizing_Methane_Plume_Timeseries.ipynb",
"chars": 53377,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# Visualizing Methane Plume Timeser"
},
{
"path": "python/tutorials/Working_with_EMIT_L2B_Mineralogy.ipynb",
"chars": 34536,
"preview": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"id\": \"28e9239d-5a33-4e90-b1e8-d18bc44f4f45\",\n \"metadata\": {},\n \"so"
},
{
"path": "setup/prerequisites.md",
"chars": 1841,
"preview": "# Prerequisites\n\nTo follow along during the workshop, or to run through the notebooks contained within the repository us"
},
{
"path": "setup/setup_instructions.md",
"chars": 2775,
"preview": "# Repository Setup Instructions\n\nThe how-tos and tutorials in this repository require a [NASA Earthdata account](https:/"
},
{
"path": "setup/workshop_setup.md",
"chars": 2930,
"preview": "# Cloud Workspace Setup\n\n> If you plan to use this repository with the **Openscapes 2i2c JupyterHub Cloud Workspace** th"
}
]
// ... and 1 more files (download for full content)
About this extraction
This page contains the full source code of the nasa/EMIT-Data-Resources GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 40 files (90.5 MB), approximately 856.0k tokens, and a symbol index with 17 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.