Using GitHub to run your tests
In this lesson we’ll learn how to have all our tests run every time we push changes to GitHub. This is an example of a continuous integration (CI) workflow. CI workflows are available via third party services such as Travis-CI, which you can link to your GitHub, GitLab, or Bitbucket repositories to build, test, and report on your projects. For this lesson we’ll use GitHub actions to perform the testing.
GitHub actions
GitHub Actions makes it easy to automate all your software workflows, now with world-class CI/CD. Build, test, and deploy your code right from GitHub. Make code reviews, branch management, and issue triaging work the way you want.
– GitHub.com
GitHub actions can perform a large variety of automation of tasks, but for this lesson we will focus only on the ability to install our module and run the tests. All of the actions that are associated with your project are stored in text files within your project. This means that you can edit the actions locally and then push them like any other file.
The actions are stored in a special directory called ./github/workflows/, and the files are stored in the .yml format.
Creating a new action
To create a new action the easiest way is to use the Actions tab of your GitHub repo, and select the New Workflow button.
The New workflow button will take you to a page to choose a template from a long list. This list is semi-smart in that it will look at the files that you have in your repository and suggest a templates based on the language(s) that you are using. The first template that shows up for our example repository is “Python Package using Anaconda” which sounds like what we want so we’ll start with that. Note that you can “set up a workflow yourself” using the small link (this will begin with a blank workflow). Also note that the templates are hosted in GitHub repositories so that you can browse them and copy ideas from one to another. For the “Python Package using Anaconda” the template is in the actions/starter-workflows repository.
After selecting a template we’ll go to the online editor for committing a new file on GitHub. The file (./github/workflows/python-package-conda.yml) is currently as follows:
name: Python Package using Conda
on: [push]
jobs:
build-linux:
runs-on: ubuntu-latest
strategy:
max-parallel: 5
steps:
- uses: actions/checkout@v2
- name: Set up Python 3.10
uses: actions/setup-python@v2
with:
python-version: 3.10
- name: Add conda to system path
run: |
# $CONDA is an environment variable pointing to the root of the miniconda directory
echo $CONDA/bin >> $GITHUB_PATH
- name: Install dependencies
run: |
conda env update --file environment.yml --name base
- name: Lint with flake8
run: |
conda install flake8
# stop the build if there are Python syntax errors or undefined names
flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
# exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
- name: Test with pytest
run: |
conda install pytest
pytest
The first two lines give the name of the action which is just for your own reference, and an on directive. The on directive will trigger this action when [push] happens. This is what we want.
The main part of the action is the jobs: section, which can specify multiple jobs. Currently there is just one job with the name build-linux. Within this job there are multiple steps that are run.
The template is mostly good however:
- since this is the Anaconda template it uses conda for all the build/test. However Anaconda is slow to create new environments and install software;
- there is a bug in this template: the
python-versionshould be a string, not a float; - we don’t want to run the
lintstep.
We will replace conda with pip and can fix the other issues by updating the script to be as follows:
name: Build and Test Python package using pip
on: [push]
jobs:
build-linux:
runs-on: ubuntu-latest
strategy:
max-parallel: 5
steps:
- uses: actions/checkout@v2
- name: Set up Python "3.6"
uses: actions/setup-python@v2
with:
python-version: "3.6"
- name: Install dependencies
run: |
# upgrade pip
python -m pip install --upgrade pip
# install the testing requirements
python -m pip install pytest pytest-cov
# install the dependencies for this
pip install -r requirements.txt
- name: Install module
run: |
python -m pip install -e .
- name: Test with pytest
run: |
pytest --cov=skysim --cov-report=termSince we are now not using conda we should also change the name of the script to be ./github/workflows/python-build-test-pip.yml before we save it. Once we have made these changes and are happy we can press the green “Start commit” button where we’ll be asked for a commit message, and whether we want to commit to the main branch or to a new branch. Lets commit directly to the main branch.
Once we make the commit the action will run. This is because adding/changing a file via the GitHub webpage is considered the same as a push action, which will trigger our workflow.
Viewing the progress/result of an action
To see what actions have been run and why, we navigate to the “Actions” tab. Here you’ll see all the actions that have ever been triggered for your repository, how they were triggered, what the status is, how long it took to run, and what artifacts (if any) were produced. Below is an example showing the completed action that we just created above.
On the left, we see all the jobs that were part of this action, and on the right we have a summary for each. We want to see green ticks as this means that everything is ok. A red X will mean there was a problem somewhere. Either way we can click on one of the jobs to see the different steps of the job, and clicking the “>” next to a step will show all the details of that step. Below we can see the output of the “test with pytest” step.
If an action fails then the repository owner, and anyone watching the repository, will get a notification of the failure. When the action succeeds for the first time after a failure a notification will be sent, but if the action is a success following a success no further notifications will be sent. This means that you can push changes to your repo and you wont be bothered unless there is a problem.
Summary
We have seen how to setup a basic install-test workflow for a python project using a GitHub action. Once you have a working action you can extend it, create additional actions, or change how it triggers. See the documentation for more details and some examples.