Getting Started with pySPEDAS: Difference between revisions

From SPEDAS Wiki
Jump to navigation Jump to search
(Changed lang="cmd" to lang="bash" for syntax highlighting)
 
(7 intermediate revisions by the same user not shown)
Line 1: Line 1:
== Welcome to pySPEDAS ==
== Welcome to PySPEDAS ==
 
PySPEDAS is an implementation of the SPEDAS framework in Python.  PySPEDAS has tools for loading, analyzing, and plotting data from over 30 different directly supported satellite missions and ground magnetometer networks, as well as other sources like NASA's CDAWeb multi-mission data archive service, and HAPI (Heliophysics Application Programming Interface) servers.
 
PySPEDAS is a core package of the [https://pyhc.org Python Heliophysics Community (pyhc.org)] PyHC is working to promote standardization, interoperability, and open science principles among the various Python packages in this space (Astropy, SunPy, pysat, SpiceyPy, and others).
 


pySPEDAS is an implementation of the SPEDAS framework in Python.


=== Requirements ===
=== Requirements ===
Line 18: Line 22:


If you're not already familiar with using Python, you may want to visit [https://python.org (python.org)] and read some documentation, work through a few tutorials, and familiarize yourself
If you're not already familiar with using Python, you may want to visit [https://python.org (python.org)] and read some documentation, work through a few tutorials, and familiarize yourself
a bit with the Python ecosystem, before taking the plunge and installing a Python environment.
a bit with the Python ecosystem, before taking the plunge and installing a Python environment. PyHC also has links to some [https://heliopython.org/docs/python_resources/ resources for getting started with Python].


We suggest that you install Python using the Anaconda distribution; step-by-step instructions for installing Anaconda can be found at:
We suggest that you install Python using the Anaconda distribution; step-by-step instructions for installing Anaconda can be found at:
Line 26: Line 30:
* Linux: https://docs.anaconda.com/anaconda/install/linux/
* Linux: https://docs.anaconda.com/anaconda/install/linux/


Once Anaconda is installed, you can use Anaconda Navigator as a graphical interface to create Python virtual environments (described more fully below) with
Once Anaconda is installed, you can use Anaconda Navigator as a graphical interface to create Python virtual environments (described more fully below), or install a variety of development environments and other useful tools.
several options for the version of Python to be used in each environment, by selecting "Environments" from the leftmost pane, then using the "Create" button
 
at the bottom of the second pane. You should get a dropdown menu of Python versions...select the version you want to use, and once it's created, you can activate it for use within Anaconda, or use it in your interactive development environment (see below) when you're setting up a new project.


== Installing an Interactive Development Environment (IDE) ==
== Installing an Interactive Development Environment (IDE) ==
Strictly speaking, all you really need is a Python interpreter and a text editor to create and run PySPEDAS programs.  But a good Python IDE will make it  
Strictly speaking, all you really need is a Python interpreter and a text editor to create and run PySPEDAS programs.  But a good Python IDE will make it  
much easier to install, understand, navigate, and run PySPEDAS code.  An IDE will generally let you create multiple independent projects, each with its own virtual environment.  There will probably be a way to create a new project from a git or other version control repository.  The PySPEDAS code and many usage examples are developed in several repositories on Github (for example, [https://github.com/spedas/pyspedas PySPEDAS], [https://github.com/spedas/pyspedas_examples Pyspedas examples], [https://github.com/spedas/mms_examples MMS examples]). (You may want to consider installing the git command line tools on your machine...some IDEs rely on having git available to connect your project to a Github repository.) Within a project, the IDE provide an area with editing tools where you can view or develop Python code. You will probably be able to highlight routine names and go directly to files where they're implemented, or hover over a routine name and get a tooltip showing the parameters it expects and other documentation.
much easier to install, understand, navigate, and run PySPEDAS code.  An IDE will generally let you create multiple independent projects, each with its own virtual environment.  There will probably be a way to create a new project from a git or other version control repository.  The PySPEDAS code and many usage examples are developed in several repositories on Github (for example, [https://github.com/spedas/pyspedas PySPEDAS], [https://github.com/spedas/pyspedas_examples Pyspedas examples], [https://github.com/spedas/mms_examples MMS examples]). (You may want to consider installing the git command line tools on your machine...some IDEs rely on having git available to connect your project to a Github repository.) Within a project, the IDE provide an area with editing tools where you can view or develop Python code. You will probably be able to highlight routine names and go directly to files where they're implemented, or hover over a routine name and get a tooltip showing the parameters it expects and other documentation.
Line 40: Line 44:
* [https://www.spyder-ide.org/ Spyder]  
* [https://www.spyder-ide.org/ Spyder]  


Downloads, installation instructions, and other documentation is available at each of their respective web sites,
PyHC also has a list of [https://heliopython.org/docs/suggested_tools/ suggested IDEs and other Python development tools].
 
Downloads, installation instructions, and other documentation is available at each of their respective web sites.
 
== Developer install via Github, versus user install via "pip install" ==
 
When you're creating a new project, your IDE probably offers several ways to get started.  The method you choose depends on how you'll be
using PySPEDAS.  If you'll be using PySPEDAS as more of an end user, and just want to install a stable, fully-tested package, you'll probably want to create a fresh virtual environment (described below), then install the PySPEDAS package using "pip install pyspedas".
 
If you are interested in contributing to PySPEDAS itself, or if you want to keep up with the "bleeding edge", and have access to
new features and bug fixes as soon as they're committed to the PySPEDAS repository, you would probably want to use your IDE's option to create a project starting from a Github repository.  If you take this route, you may need to have the Git command-line tools installed on your machine. See [https://git-scm.com the Git web site] for downloads and installation instructions. The URL for the PySPEDAS Github repo is https://github.com/spedas/pyspedas.git .  Copy and paste this URL into your IDE's project setup panel, then follow the rest of your IDE's instructions for creating the project.  If you use this method to get PySPEDAS, you don't need to do the "pip install pyspedas" step...you will already have the PySPEDAS source code cloned directly from the repository.  However, you will still need to install several packages that PySPEDAS depends on.  Your IDE may prompt you to do this as part of creating the project, or you might have to use a command like "pip install -r requirements.txt" to install the dependencies.
 
The PySPEDAS project has several other Github repositories containing code examples, many in the form of Jupyter notebooks:
 
* General PySPEDAS and PyTplot examples: [https://github.com/spedas/pyspedas_examples.git]
* THEMIS examples [https://github.com/spedas/themis_examples.git]
* MMS examples [https://github.com/spedas/mms_examples.git]
 
The easiest way to work with these examples is to create a new project for each set of examples, starting from the Github URL.  Then use "pip install pyspedas" to install pyspedas in this environment, and "pip install jupyter" to get the packages you'll need to run Jupyter notebooks.
 
Alternatively, you can download individual notebooks from Github, and save them in the same project where you installed PySPEDAS.  Make sure that you're downloading the raw notebook file using the link Github offers when you navigate to the file, otherwise the notebook may download in the wrong format, and you won't be able to open it.


== Creating a virtual environment ==
== Creating a virtual environment ==
To avoid potential dependency issues with other Python packages, it’s best to create a dedicated virtual environment in Python for working with PySPEDAS.
To avoid potential dependency issues with other Python packages, it’s best to create a dedicated virtual environment in Python for working with PySPEDAS.


Most IDEs will provide a way to create a new virtual environment when you start a new project.  See your IDE documentation for the steps you'll need to take.


You can create a virtual environment in your terminal with:
If you prefer, you can create a virtual environment in your terminal with:


<syntaxhighlight lang="cmd">
<syntaxhighlight lang="bash">
python -m venv pyspedas
python -m venv pyspedas
</syntaxhighlight>
</syntaxhighlight>
Line 55: Line 81:


=== macOS and Linux ===
=== macOS and Linux ===
<syntaxhighlight lang="cmd">
<syntaxhighlight lang="bash">
source pyspedas/bin/activate
source pyspedas/bin/activate
</syntaxhighlight>
</syntaxhighlight>


=== Windows ===
=== Windows ===
<syntaxhighlight lang="cmd">
<syntaxhighlight lang="bash">
.\pyspedas\Scripts\activate
.\pyspedas\Scripts\activate
</syntaxhighlight>
</syntaxhighlight>


== Installing pySPEDAS ==


The first time you enter your virtual environment, you’ll have to install pyspedas; this is as simple as:
== Installing PySPEDAS ==
 
If you're doing a user install of PySPEDAS (i.e. not from Github), the first time you enter your virtual environment, you’ll have to install pyspedas; this is as simple as using your IDE to open a terminal window (not an interactive Python window!), then typing


<syntaxhighlight lang="cmd">
<syntaxhighlight lang="bash">
pip install pyspedas
pip install pyspedas
</syntaxhighlight>
</syntaxhighlight>
Line 76: Line 103:
If you would like to upgrade your copy of the pySPEDAS libraries inside of your virtual environment, use:
If you would like to upgrade your copy of the pySPEDAS libraries inside of your virtual environment, use:


<syntaxhighlight lang="cmd">
<syntaxhighlight lang="bash">
pip install pyspedas --upgrade
pip install pyspedas --upgrade
</syntaxhighlight>
</syntaxhighlight>
You'll probably want to install the packages for running Jupyter notebooks (for both user installs and Github installs):
<syntaxhighlight lang="bash">
pip install jupyter
</syntaxhighlight>
For certain PySPEDAS tools, there may be additional packages you'll need to install.  For example, if you want to plot EICS/SECS data on top of a map of North America, you'll need the basemap package:
<syntaxhighlight lang="bash">
pip install basemap
</syntaxhighlight>


== Setting your local data directory ==
== Setting your local data directory ==
Line 94: Line 136:
== Checking that everything is working ==
== Checking that everything is working ==


The quickest way to check if everything is working is to load some data; once you’re inside Python in your virtual environment, try:
The quickest way to check if everything is working is to load some data; to do this, use your IDE to start an interactive Python window (note: not a terminal window, as you used for the various "pip install" commands!).  Then type the following Python commands:


<syntaxhighlight lang="python">
<syntaxhighlight lang="python">
Line 112: Line 154:
To get virtual environments working with Jupyter, there are a few extra steps:
To get virtual environments working with Jupyter, there are a few extra steps:


in the virtual environment, type:
If you haven't already installed the jupyter package in your virtual environment, you can get it by typing:


<syntaxhighlight lang="cmd">
<syntaxhighlight lang="bash">
pip install ipykernel
pip install jupyter
python -m ipykernel install --user --name pyspedas --display-name "Python (pySPEDAS)"
</syntaxhighlight>
</syntaxhighlight>


(note: "pyspedas" is the name of your virtual environment)
Then, to start the Jupyter server:


Then once you open the notebook, go to "Kernel" then "Change kernel" and select the one named "Python (pySPEDAS)"
<syntaxhighlight lang="bash">
jupyter notebook
</syntaxhighlight>
 
This should open a page in your browser, where you can navigate to the Juypyter notebook you want to run, and open it.
 
Your IDE may also support running notebooks directly, e.g. by double-clicking them in your project.


== Recorded tutorials, presentation slides, notebooks ==
== Recorded tutorials, presentation slides, notebooks ==
* [https://drive.google.com/drive/u/1/folders/1RZgRtVowhdcMUHuDd1Mv9w3fWCEwYVWH UCLA class and demo, October 5, 2023]
* [https://drive.google.com/drive/u/1/folders/1RZgRtVowhdcMUHuDd1Mv9w3fWCEwYVWH UCLA class and demo, October 5, 2023]


== Examples ==
== Google Colab Examples ==
 
There are two repositories that contain examples of using pyspedas:


MMS Examples
Google offers a service called Google Colaboratory that lets you run Jupyter notebooks (stored in a Google Drive account) in the cloud, with
* [https://github.com/spedas/mms-examples Jupyter notebooks containing MMS examples]
no local Python installation needed.  You will have to be signed in to your own Google account to use the service.  The first time you try to run a Google Colab notebook, you may have to navigate to the Google Colab service from the "open with" prompt, and make sure Google Colab is enabled for your Google account.


Examples for other missions and general webinars
Here are a couple of examples (caveat: at this writing, there may be package version conflicts between PySPEDAS and the Google Colab environment. We expect to have this resolved soon.)
* [https://github.com/spedas/pyspedas_examples Jupyter notebooks containing pySPEDAS examples]


Google Colab Examples (Jupyter notebooks that run in the cloud...no local Python installation required!)
* [https://colab.research.google.com/drive/15GV6f9XJ7JjGD1rrvXZKnGvLq_lLOs5_?authuser=1 Getting Started with PySPEDAS, UCLA class demo, October 5, 2023]
* [https://colab.research.google.com/drive/15GV6f9XJ7JjGD1rrvXZKnGvLq_lLOs5_?authuser=1 Getting Started with PySPEDAS, UCLA class demo, October 5, 2023]
* [https://colab.research.google.com/drive/1kUJP1FM18L03ov3JYZSEvkvCJUp9XcL6?authuser=1 Exploring the Heliosphere with Python]   
* [https://colab.research.google.com/drive/1kUJP1FM18L03ov3JYZSEvkvCJUp9XcL6?authuser=1 Exploring the Heliosphere with Python]   

Latest revision as of 02:39, 29 December 2023

Welcome to PySPEDAS

PySPEDAS is an implementation of the SPEDAS framework in Python. PySPEDAS has tools for loading, analyzing, and plotting data from over 30 different directly supported satellite missions and ground magnetometer networks, as well as other sources like NASA's CDAWeb multi-mission data archive service, and HAPI (Heliophysics Application Programming Interface) servers.

PySPEDAS is a core package of the Python Heliophysics Community (pyhc.org) PyHC is working to promote standardization, interoperability, and open science principles among the various Python packages in this space (Astropy, SunPy, pysat, SpiceyPy, and others).


Requirements

As of PySPEDAS version 1.4.46, Python 3.8 or later is required. Note that PySPEDAS has dependencies on many other Python packages, some of which may need to be updated to work with newly released Python versions, OS versions, CPU architectures, etc. As of this writing (October 2023), PySPEDAS and its dependencies should be compatible with Python 3.8 through 3.10.

Certain packages may still have incompatibilities with Python 3.11 or Python 3.12. Other packages may depend on binary libraries that may not be available "out of the box" for certain OS versions or CPU architectures (for example, Mac laptops with M2 chips). It is possible that "pip install" may fail for such packages, if your environment doesn't have all the necessary compiler tools or prerequisite packages installed. If this happens to you while installing or updating PySPEDAS, please let the development team know, and we'll try to help you get things running.

pySPEDAS releases

Installing Python

If you're not already familiar with using Python, you may want to visit (python.org) and read some documentation, work through a few tutorials, and familiarize yourself a bit with the Python ecosystem, before taking the plunge and installing a Python environment. PyHC also has links to some resources for getting started with Python.

We suggest that you install Python using the Anaconda distribution; step-by-step instructions for installing Anaconda can be found at:

Once Anaconda is installed, you can use Anaconda Navigator as a graphical interface to create Python virtual environments (described more fully below), or install a variety of development environments and other useful tools.


Installing an Interactive Development Environment (IDE)

Strictly speaking, all you really need is a Python interpreter and a text editor to create and run PySPEDAS programs. But a good Python IDE will make it much easier to install, understand, navigate, and run PySPEDAS code. An IDE will generally let you create multiple independent projects, each with its own virtual environment. There will probably be a way to create a new project from a git or other version control repository. The PySPEDAS code and many usage examples are developed in several repositories on Github (for example, PySPEDAS, Pyspedas examples, MMS examples). (You may want to consider installing the git command line tools on your machine...some IDEs rely on having git available to connect your project to a Github repository.) Within a project, the IDE provide an area with editing tools where you can view or develop Python code. You will probably be able to highlight routine names and go directly to files where they're implemented, or hover over a routine name and get a tooltip showing the parameters it expects and other documentation.

Some popular IDEs for Python development include:

PyHC also has a list of suggested IDEs and other Python development tools.

Downloads, installation instructions, and other documentation is available at each of their respective web sites.

Developer install via Github, versus user install via "pip install"

When you're creating a new project, your IDE probably offers several ways to get started. The method you choose depends on how you'll be using PySPEDAS. If you'll be using PySPEDAS as more of an end user, and just want to install a stable, fully-tested package, you'll probably want to create a fresh virtual environment (described below), then install the PySPEDAS package using "pip install pyspedas".

If you are interested in contributing to PySPEDAS itself, or if you want to keep up with the "bleeding edge", and have access to new features and bug fixes as soon as they're committed to the PySPEDAS repository, you would probably want to use your IDE's option to create a project starting from a Github repository. If you take this route, you may need to have the Git command-line tools installed on your machine. See the Git web site for downloads and installation instructions. The URL for the PySPEDAS Github repo is https://github.com/spedas/pyspedas.git . Copy and paste this URL into your IDE's project setup panel, then follow the rest of your IDE's instructions for creating the project. If you use this method to get PySPEDAS, you don't need to do the "pip install pyspedas" step...you will already have the PySPEDAS source code cloned directly from the repository. However, you will still need to install several packages that PySPEDAS depends on. Your IDE may prompt you to do this as part of creating the project, or you might have to use a command like "pip install -r requirements.txt" to install the dependencies.

The PySPEDAS project has several other Github repositories containing code examples, many in the form of Jupyter notebooks:

  • General PySPEDAS and PyTplot examples: [1]
  • THEMIS examples [2]
  • MMS examples [3]

The easiest way to work with these examples is to create a new project for each set of examples, starting from the Github URL. Then use "pip install pyspedas" to install pyspedas in this environment, and "pip install jupyter" to get the packages you'll need to run Jupyter notebooks.

Alternatively, you can download individual notebooks from Github, and save them in the same project where you installed PySPEDAS. Make sure that you're downloading the raw notebook file using the link Github offers when you navigate to the file, otherwise the notebook may download in the wrong format, and you won't be able to open it.

Creating a virtual environment

To avoid potential dependency issues with other Python packages, it’s best to create a dedicated virtual environment in Python for working with PySPEDAS.

Most IDEs will provide a way to create a new virtual environment when you start a new project. See your IDE documentation for the steps you'll need to take.

If you prefer, you can create a virtual environment in your terminal with:

<syntaxhighlight lang="bash"> python -m venv pyspedas </syntaxhighlight>

And enter into that virtual environment by running the 'activate' script with:

macOS and Linux

<syntaxhighlight lang="bash"> source pyspedas/bin/activate </syntaxhighlight>

Windows

<syntaxhighlight lang="bash"> .\pyspedas\Scripts\activate </syntaxhighlight>


Installing PySPEDAS

If you're doing a user install of PySPEDAS (i.e. not from Github), the first time you enter your virtual environment, you’ll have to install pyspedas; this is as simple as using your IDE to open a terminal window (not an interactive Python window!), then typing

<syntaxhighlight lang="bash"> pip install pyspedas </syntaxhighlight>

This should go out and find all of the required libraries and install them inside the virtual environment.

If you would like to upgrade your copy of the pySPEDAS libraries inside of your virtual environment, use:

<syntaxhighlight lang="bash"> pip install pyspedas --upgrade </syntaxhighlight>

You'll probably want to install the packages for running Jupyter notebooks (for both user installs and Github installs):

<syntaxhighlight lang="bash"> pip install jupyter </syntaxhighlight>

For certain PySPEDAS tools, there may be additional packages you'll need to install. For example, if you want to plot EICS/SECS data on top of a map of North America, you'll need the basemap package:

<syntaxhighlight lang="bash"> pip install basemap </syntaxhighlight>



Setting your local data directory

The recommended way of setting your local data directory is to set the 'SPEDAS_DATA_DIR' environment variable. 'SPEDAS_DATA_DIR' acts as a root data directory for all missions, and will also be used by IDL (if you’re running a recent copy of the bleeding edge).

Mission specific data directories (e.g., 'MMS_DATA_DIR') can also be set, and these will override 'SPEDAS_DATA_DIR'

Network mirror for the MMS dataset

If you have a mirror of the MMS dataset on your local network, you may want to set the MMS_MIRROR_DATA_DIR environment variable.

If set, when you use the no_update keyword in the load routines (or if don’t have an internet connection), the load routines will check the mirror for data. Just as in IDL, data files found on the network mirror will be copied to your local data directory before loading them.

Checking that everything is working

The quickest way to check if everything is working is to load some data; to do this, use your IDE to start an interactive Python window (note: not a terminal window, as you used for the various "pip install" commands!). Then type the following Python commands:

<syntaxhighlight lang="python"> import pyspedas pyspedas.mms.fgm() </syntaxhighlight>

This should load some default data (srvy mode, probe 1) for Oct 16, 2015. You can then plot the FGM data with:

<syntaxhighlight lang="python"> from pytplot import tplot tplot('mms1_fgm_b_gse_srvy_l2') </syntaxhighlight>

Using Jupyter notebooks with your virtual environment

To get virtual environments working with Jupyter, there are a few extra steps:

If you haven't already installed the jupyter package in your virtual environment, you can get it by typing:

<syntaxhighlight lang="bash"> pip install jupyter </syntaxhighlight>

Then, to start the Jupyter server:

<syntaxhighlight lang="bash"> jupyter notebook </syntaxhighlight>

This should open a page in your browser, where you can navigate to the Juypyter notebook you want to run, and open it.

Your IDE may also support running notebooks directly, e.g. by double-clicking them in your project.

Recorded tutorials, presentation slides, notebooks

Google Colab Examples

Google offers a service called Google Colaboratory that lets you run Jupyter notebooks (stored in a Google Drive account) in the cloud, with no local Python installation needed. You will have to be signed in to your own Google account to use the service. The first time you try to run a Google Colab notebook, you may have to navigate to the Google Colab service from the "open with" prompt, and make sure Google Colab is enabled for your Google account.

Here are a couple of examples (caveat: at this writing, there may be package version conflicts between PySPEDAS and the Google Colab environment. We expect to have this resolved soon.)

You can also browse the README.md files using GitHub for more information and examples, e.g.,

Comparison of pySPEDAS to SPEDAS

Some examples that demonstrate how to achieve the same results using either IDL SPEDAS or python pySPEDAS.