Classes involved in doctesting

This module controls the various classes involved in doctesting.

AUTHORS:

  • David Roe (2012-03-27) – initial version, based on Robert Bradshaw’s code.
class sage.doctest.control.DocTestController(options, args)

Bases: sage.structure.sage_object.SageObject

This class controls doctesting of files.

After creating it with appropriate options, call the run() method to run the doctests.

add_files()

Checks for the flags ‘–all’, ‘–new’ and ‘–sagenb’.

For each one present, this function adds the appropriate directories and files to the todo list.

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults, DocTestController
sage: from sage.env import SAGE_SRC
sage: import os
sage: log_location = os.path.join(SAGE_TMP, 'control_dt_log.log')
sage: DD = DocTestDefaults(all=True, logfile=log_location)
sage: DC = DocTestController(DD, [])
sage: DC.add_files()
Doctesting entire Sage library.
sage: os.path.join(SAGE_SRC, 'sage') in DC.files
True
sage: DD = DocTestDefaults(new = True)
sage: DC = DocTestController(DD, [])
sage: DC.add_files()
Doctesting ...
sage: DD = DocTestDefaults(sagenb = True)
sage: DC = DocTestController(DD, [])
sage: DC.add_files()
Doctesting the Sage notebook.
sage: DC.files[0][-6:]
'sagenb'
cleanup(final=True)

Runs cleanup activities after actually running doctests.

In particular, saves the stats to disk and closes the logfile.

INPUT:

  • final – whether to close the logfile

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults, DocTestController
sage: from sage.env import SAGE_SRC
sage: import os
sage: dirname = os.path.join(SAGE_SRC, 'sage', 'rings', 'infinity.py')
sage: DD = DocTestDefaults()

sage: DC = DocTestController(DD, [dirname])
sage: DC.expand_files_into_sources()
sage: DC.sources.sort(key=lambda s:s.basename)

sage: for i, source in enumerate(DC.sources):
....:     DC.stats[source.basename] = {'walltime': 0.1*(i+1)}
....:

sage: DC.run()
Running doctests with ID ...
Doctesting 1 file.
sage -t .../rings/infinity.py
    [... tests, ... s]
----------------------------------------------------------------------
All tests passed!
----------------------------------------------------------------------
Total time for all tests: ... seconds
    cpu time: ... seconds
    cumulative wall time: ... seconds
0
sage: DC.cleanup()
create_run_id()

Creates the run id.

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults, DocTestController
sage: DC = DocTestController(DocTestDefaults(), [])
sage: DC.create_run_id()
Running doctests with ID ...
expand_files_into_sources()

Expands self.files, which may include directories, into a list of sage.doctest.FileDocTestSource

This function also handles the optional command line option.

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults, DocTestController
sage: from sage.env import SAGE_SRC
sage: import os
sage: dirname = os.path.join(SAGE_SRC, 'sage', 'doctest')
sage: DD = DocTestDefaults(optional='all')
sage: DC = DocTestController(DD, [dirname])
sage: DC.expand_files_into_sources()
sage: len(DC.sources)
9
sage: DC.sources[0].options.optional
True
sage: DD = DocTestDefaults(optional='magma,guava')
sage: DC = DocTestController(DD, [dirname])
sage: DC.expand_files_into_sources()
sage: sorted(list(DC.sources[0].options.optional))
['guava', 'magma']

We check that files are skipped appropriately:

sage: dirname = tmp_dir()
sage: filename = os.path.join(dirname, 'not_tested.py')
sage: with open(filename, 'w') as F:
....:     F.write("#"*80 + "\n\n\n\n## nodoctest\n    sage: 1+1\n    4")
sage: DC = DocTestController(DD, [dirname])
sage: DC.expand_files_into_sources()
sage: DC.sources
[]

The directory sage/doctest/tests contains nodoctest.py but the files should still be tested when that directory is explicitly given (as opposed to being recursed into):

sage: DC = DocTestController(DD, [os.path.join(SAGE_SRC, 'sage', 'doctest', 'tests')])
sage: DC.expand_files_into_sources()
sage: len(DC.sources) >= 10
True
filter_sources()

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults, DocTestController
sage: from sage.env import SAGE_SRC
sage: import os
sage: dirname = os.path.join(SAGE_SRC, 'sage', 'doctest')
sage: DD = DocTestDefaults(failed=True)
sage: DC = DocTestController(DD, [dirname])
sage: DC.expand_files_into_sources()
sage: for i, source in enumerate(DC.sources):
...       DC.stats[source.basename] = {'walltime': 0.1*(i+1)}
sage: DC.stats['sage.doctest.control'] = {'failed':True,'walltime':1.0}
sage: DC.filter_sources()
Only doctesting files that failed last test.
sage: len(DC.sources)
1
load_stats(filename)

Load stats from the most recent run(s).

Stats are stored as a JSON file, and include information on which files failed tests and the walltime used for execution of the doctests.

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults, DocTestController
sage: DC = DocTestController(DocTestDefaults(), [])
sage: import json
sage: filename = tmp_filename()
sage: with open(filename, 'w') as stats_file:
...       json.dump({'sage.doctest.control':{u'walltime':1.0r}}, stats_file)
sage: DC.load_stats(filename)
sage: DC.stats['sage.doctest.control']
{u'walltime': 1.0}

If the file doesn’t exist, nothing happens. If there is an error, print a message. In any case, leave the stats alone:

sage: d = tmp_dir()
sage: DC.load_stats(os.path.join(d))  # Cannot read a directory
Error loading stats from ...
sage: DC.load_stats(os.path.join(d, "no_such_file"))
sage: DC.stats['sage.doctest.control']
{u'walltime': 1.0}
log(s, end='n')

Logs the string s + end (where end is a newline by default) to the logfile and prints it to the standard output.

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults, DocTestController
sage: DD = DocTestDefaults(logfile=tmp_filename())
sage: DC = DocTestController(DD, [])
sage: DC.log("hello world")
hello world
sage: DC.logfile.close()
sage: print open(DD.logfile).read()
hello world

Check that no duplicate logs appear, even when forking (trac ticket #15244):

sage: DD = DocTestDefaults(logfile=tmp_filename())
sage: DC = DocTestController(DD, [])
sage: DC.log("hello world")
hello world
sage: if os.fork() == 0:
....:     DC.logfile.close()
....:     os._exit(0)
sage: DC.logfile.close()
sage: print open(DD.logfile).read()
hello world
run()

This function is called after initialization to set up and run all doctests.

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults, DocTestController
sage: from sage.env import SAGE_SRC
sage: import os
sage: DD = DocTestDefaults()
sage: filename = os.path.join(SAGE_SRC, "sage", "sets", "non_negative_integers.py")
sage: DC = DocTestController(DD, [filename])
sage: DC.run()
Running doctests with ID ...
Doctesting 1 file.
sage -t .../sage/sets/non_negative_integers.py
    [... tests, ... s]
----------------------------------------------------------------------
All tests passed!
----------------------------------------------------------------------
Total time for all tests: ... seconds
    cpu time: ... seconds
    cumulative wall time: ... seconds
0
run_doctests()

Actually runs the doctests.

This function is called by run().

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults, DocTestController
sage: from sage.env import SAGE_SRC
sage: import os
sage: dirname = os.path.join(SAGE_SRC, 'sage', 'rings', 'homset.py')
sage: DD = DocTestDefaults()
sage: DC = DocTestController(DD, [dirname])
sage: DC.expand_files_into_sources()
sage: DC.run_doctests()
Doctesting 1 file.
sage -t .../sage/rings/homset.py
    [... tests, ... s]
----------------------------------------------------------------------
All tests passed!
----------------------------------------------------------------------
Total time for all tests: ... seconds
    cpu time: ... seconds
    cumulative wall time: ... seconds
run_val_gdb(testing=False)

Spawns a subprocess to run tests under the control of gdb or valgrind.

INPUT:

  • testing – boolean; if True then the command to be run will be printed rather than a subprocess started.

EXAMPLES:

Note that the command lines include unexpanded environment variables. It is safer to let the shell expand them than to expand them here and risk insufficient quoting.

sage: from sage.doctest.control import DocTestDefaults, DocTestController
sage: DD = DocTestDefaults(gdb=True)
sage: DC = DocTestController(DD, ["hello_world.py"])
sage: DC.run_val_gdb(testing=True)
exec gdb -x "$SAGE_LOCAL/bin/sage-gdb-commands" --args python "$SAGE_LOCAL/bin/sage-runtests" --serial --timeout=0 hello_world.py
sage: DD = DocTestDefaults(valgrind=True, optional="all", timeout=172800)
sage: DC = DocTestController(DD, ["hello_world.py"])
sage: DC.run_val_gdb(testing=True)
exec valgrind --tool=memcheck --leak-resolution=high --leak-check=full --num-callers=25 --suppressions="$SAGE_LOCAL/lib/valgrind/sage.supp"  --log-file=".../valgrind/sage-memcheck.%p" python "$SAGE_LOCAL/bin/sage-runtests" --serial --timeout=172800 --optional=True hello_world.py
save_stats(filename)

Save stats from the most recent run as a JSON file.

WARNING: This function overwrites the file.

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults, DocTestController
sage: DC = DocTestController(DocTestDefaults(), [])
sage: DC.stats['sage.doctest.control'] = {u'walltime':1.0r}
sage: filename = tmp_filename()
sage: DC.save_stats(filename)
sage: import json
sage: D = json.load(open(filename))
sage: D['sage.doctest.control']
{u'walltime': 1.0}
sort_sources()

This function sorts the sources so that slower doctests are run first.

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults, DocTestController
sage: from sage.env import SAGE_SRC
sage: import os
sage: dirname = os.path.join(SAGE_SRC, 'sage', 'doctest')
sage: DD = DocTestDefaults(nthreads=2)
sage: DC = DocTestController(DD, [dirname])
sage: DC.expand_files_into_sources()
sage: DC.sources.sort(key=lambda s:s.basename)
sage: for i, source in enumerate(DC.sources):
...       DC.stats[source.basename] = {'walltime': 0.1*(i+1)}
sage: DC.sort_sources()
Sorting sources by runtime so that slower doctests are run first....
sage: print "\n".join([source.basename for source in DC.sources])
sage.doctest.util
sage.doctest.test
sage.doctest.sources
sage.doctest.reporting
sage.doctest.parsing
sage.doctest.forker
sage.doctest.control
sage.doctest.all
sage.doctest
test_safe_directory(dir=None)

Test that the given directory is safe to run Python code from.

We use the check added to Python for this, which gives a warning when the current directory is considered unsafe. We promote this warning to an error with -Werror. See sage/tests/cmdline.py for a doctest that this works, see also trac ticket #13579.

TESTS:

sage: from sage.doctest.control import DocTestDefaults, DocTestController
sage: DD = DocTestDefaults()
sage: DC = DocTestController(DD, [])
sage: DC.test_safe_directory()
sage: d = os.path.join(tmp_dir(), "test")
sage: os.mkdir(d)
sage: os.chmod(d, 0o777)
sage: DC.test_safe_directory(d)
Traceback (most recent call last):
...
RuntimeError: refusing to run doctests...
class sage.doctest.control.DocTestDefaults(**kwds)

Bases: sage.structure.sage_object.SageObject

This class is used for doctesting the Sage doctest module.

It fills in attributes to be the same as the defaults defined in SAGE_LOCAL/bin/sage-runtests, expect for a few places, which is mostly to make doctesting more predictable.

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults
sage: D = DocTestDefaults()
sage: D
DocTestDefaults()
sage: D.timeout
-1

Keyword arguments become attributes:

sage: D = DocTestDefaults(timeout=100)
sage: D
DocTestDefaults(timeout=100)
sage: D.timeout
100
sage.doctest.control.run_doctests(module, options=None)

Runs the doctests in a given file.

INPUTS:

  • module – a Sage module, a string, or a list of such.
  • options – a DocTestDefaults object or None.

EXAMPLES:

sage: run_doctests(sage.rings.infinity)
Running doctests with ID ...
Doctesting 1 file.
sage -t .../sage/rings/infinity.py
    [... tests, ... s]
----------------------------------------------------------------------
All tests passed!
----------------------------------------------------------------------
Total time for all tests: ... seconds
    cpu time: ... seconds
    cumulative wall time: ... seconds
sage.doctest.control.skipdir(dirname)

Return True if and only if the directory dirname should not be doctested.

EXAMPLES:

sage: from sage.doctest.control import skipdir
sage: skipdir(sage.env.SAGE_SRC)
False
sage: skipdir(os.path.join(sage.env.SAGE_SRC, "sage", "doctest", "tests"))
True
sage.doctest.control.skipfile(filename)

Return True if and only if the file filename should not be doctested.

EXAMPLES:

sage: from sage.doctest.control import skipfile
sage: skipfile("skipme.c")
True
sage: f = tmp_filename(ext=".pyx")
sage: skipfile(f)
False
sage: open(f, "w").write("# nodoctest")
sage: skipfile(f)
True

Previous topic

Sage’s Doctesting Framework

Next topic

Classes for sources of doctests

This Page