Git as a storage

git-test is a Git extension that allows to execute regression tests in your project tracked in Git and attach test reports to Git commits. Under the hood it is a command-line script for running automated tests against commits in a Git repository. It is especially targeted for developers who like their tests to pass on every commit in a branch, not just the branch tip.

git test will test the commits in the specified range, reporting any failures. The results of running tests are also recorded permanently in your repository as Git “notes” (see git-notes(1)).

Let’s see on it in action:

First of all we need download it, install and create an alias:

$ git clone https://github.com/ligurio/git-test && cd git-test && make install`
$ git config --global alias.test '!git-test'

Then define the test that you would like to run; for example,

$ git test add "make -j8 && make test"

The string that you specify can be an arbitrary command; it is run with sh -c. Its exit code should be 0 if the test passes, or non-zero if it fails. The test definition is stored in your Git config.

By default, git test run tests HEAD:

$ git test run

One can even run tests for a range of commits:

git test run --retest bdcd9e8448bfd00bb61cfbe962f1cb492bdf881c d5de104492d97cb10666348bf2256eff8af5602b

Using test default; command: echo PASS
bdcd9e8448bfd00bb61cfbe962f1cb492bdf881c^{tree} known-good
Tree bdcd9e8^{tree} is already known to be good.

commit d5de104492d97cb10666348bf2256eff8af5602b (HEAD)
Author: Sergey Bronnikov <estetus@gmail.com>
Date:   Fri Jul 10 16:41:48 2020 +0300

    make: merge phony targets and introduce install target

PASS
d5de104492d97cb10666348bf2256eff8af5602b^{tree} good
Marked tree d5de104492d97cb10666348bf2256eff8af5602b^{tree} to be good

ALL TESTS SUCCESSFUL

With git test one can easily setup pure man continuous integration system:

  • setup git test on a remote machine
  • clone target Git repository to a remote machine
  • setup pull and run git test and push for a repository using cron(8) with desired time slots
  • restrict access to a master branch for a user that will be used for push to a remote repository
  • test results can be viewed locally with git test results or with web interface using CGit (see about it below).

CGit is a popular web-interface for Git repositories. FreeBSD, Linux kernel and many other projects uses CGit. Git SCM allows to store information in so called notes and CGit allows to show notes that placed to default reference (‘refs/notes/commits’) and attached to certain commits. It is convenient to add various useful information related to commit in Git notes.

There is also a patch that implements support of test results saved to Git notes with non-default reference. Often tests results stored for future analysis in different systems (aka ‘test report systems’). But one is the logic places to store test results is a Git SCM, test report automatically connected to commits and easily available for everyone.

Let’s imagine we have a test report in a Test Anything Protocol format:

$ cat report.tap
1..2
ok 1 - test_0.py::test_list_once
ok 2 - test_0.py::test_list_twice
$ GIT_NOTES_REF=refs/notes/testres git notes add -F report.tap 95aa7d200ee10580c472a1156a11c726046b110f
$ GIT_NOTES_REF=refs/notes/testres git show 95aa7d200ee10580c472a1156a11c726046b110f

commit 95aa7d200ee10580c472a1156a11c726046b110f
Author: Sergey Bronnikov
Date:   Tue Dec 22 10:06:10 2020 +0300

...

Notes (testres):
    1..2
    ok 1 - test_0.py::test_list_once
    ok 2 - test_0.py::test_list_twice

Or list all commits along with attached test reports: git log --show-notes=testres.

If we open our commit in CGit our test report will be available by clicking to a link ‘tests’.

Jenkins CI has a plugin git-notes written in pure Ruby which annotates git commits with Jenkins build information using the awesome git-notes functionality.

Google Git Notes is another plugin for Jenkins CI that provides automatic recording of Jenkins build actions to Git Notes.

git-bug is a distributed, offline-first bug tracker embedded in git, with bridges. It is fully embedded in git: you only need your git repository to have a bug tracker is distributed: use your normal git remote to collaborate, push and pull your bugs. git-bug works offline: you don’t need a network to work with bugs. git-bug integrates with your tooling through the CLI or the GraphQL API.

You can launch a rich Web UI with git bug webui:

An interactive terminal UI is available using the command git bug termui to browse and edit bugs:

Gerrit is a code review systems. Gerrit has a plugin that annotates merged commits using notes on refs/notes/review to store information about code reviews. Read more about it on Gerrit site

git-appraise is a distributed code review system for Git repos. Code reviews are stored inside of the repository as Git objects. Every developer on a team has their own copy of the review history that they can push or pull. When pulling, updates from the remote repo are automatically merged by the tool. The code review data is stored in Git notes, using special format described in schema. git-appraise has a number of integrations: git-appraise-web, git-appraise-eclipse and others.

Gitea since version 1.9.0+ has support for displaying notes created by Git notes:

GitHub supported Git notes in it’s web interface. But support has been removed on August 2014.

Discuss on HackerNews.

Теги: softwareopensourcetestingfeeden