Processes for running doctests

This module controls the processes started by Sage that actually run the doctests.

EXAMPLES:

The following examples are used in doctesting this file:

sage: doctest_var = 42; doctest_var^2
1764
sage: R.<a> = ZZ[]
sage: a + doctest_var
a + 42

AUTHORS:

  • David Roe (2012-03-27) – initial version, based on Robert Bradshaw’s code.
class sage.doctest.forker.DocTestDispatcher(controller)

Bases: sage.structure.sage_object.SageObject

Creates parallel DocTestWorker processes and dispatches doctesting tasks.

dispatch()

Run the doctests for the controller’s specified sources, by calling parallel_dispatch() or serial_dispatch() according to the --serial option.

EXAMPLES:

sage: from sage.doctest.control import DocTestController, DocTestDefaults
sage: from sage.doctest.forker import DocTestDispatcher
sage: from sage.doctest.reporting import DocTestReporter
sage: from sage.doctest.util import Timer
sage: from sage.env import SAGE_SRC
sage: import os
sage: freehom = os.path.join(SAGE_SRC, 'sage', 'modules', 'free_module_homspace.py')
sage: bigo = os.path.join(SAGE_SRC, 'sage', 'rings', 'big_oh.py')
sage: DC = DocTestController(DocTestDefaults(), [freehom, bigo])
sage: DC.expand_files_into_sources()
sage: DD = DocTestDispatcher(DC)
sage: DR = DocTestReporter(DC)
sage: DC.reporter = DR
sage: DC.dispatcher = DD
sage: DC.timer = Timer().start()
sage: DD.dispatch()
sage -t .../sage/modules/free_module_homspace.py
    [... tests, ... s]
sage -t .../sage/rings/big_oh.py
    [... tests, ... s]
parallel_dispatch()

Run the doctests from the controller’s specified sources in parallel.

This creates DocTestWorker subprocesses, while the master process checks for timeouts and collects and displays the results.

EXAMPLES:

sage: from sage.doctest.control import DocTestController, DocTestDefaults
sage: from sage.doctest.forker import DocTestDispatcher
sage: from sage.doctest.reporting import DocTestReporter
sage: from sage.doctest.util import Timer
sage: from sage.env import SAGE_SRC
sage: import os
sage: crem = os.path.join(SAGE_SRC, 'sage', 'databases', 'cremona.py')
sage: bigo = os.path.join(SAGE_SRC, 'sage', 'rings', 'big_oh.py')
sage: DC = DocTestController(DocTestDefaults(), [crem, bigo])
sage: DC.expand_files_into_sources()
sage: DD = DocTestDispatcher(DC)
sage: DR = DocTestReporter(DC)
sage: DC.reporter = DR
sage: DC.dispatcher = DD
sage: DC.timer = Timer().start()
sage: DD.parallel_dispatch()
sage -t .../databases/cremona.py
    [... tests, ... s]
sage -t .../rings/big_oh.py
    [... tests, ... s]
serial_dispatch()

Run the doctests from the controller’s specified sources in series.

There is no graceful handling for signals, no possibility of interrupting tests and no timeout.

EXAMPLES:

sage: from sage.doctest.control import DocTestController, DocTestDefaults
sage: from sage.doctest.forker import DocTestDispatcher
sage: from sage.doctest.reporting import DocTestReporter
sage: from sage.doctest.util import Timer
sage: from sage.env import SAGE_SRC
sage: import os
sage: homset = os.path.join(SAGE_SRC, 'sage', 'rings', 'homset.py')
sage: ideal = os.path.join(SAGE_SRC, 'sage', 'rings', 'ideal.py')
sage: DC = DocTestController(DocTestDefaults(), [homset, ideal])
sage: DC.expand_files_into_sources()
sage: DD = DocTestDispatcher(DC)
sage: DR = DocTestReporter(DC)
sage: DC.reporter = DR
sage: DC.dispatcher = DD
sage: DC.timer = Timer().start()
sage: DD.serial_dispatch()
sage -t .../rings/homset.py
    [... tests, ... s]
sage -t .../rings/ideal.py
    [... tests, ... s]
class sage.doctest.forker.DocTestTask(source)

Bases: object

This class encapsulates the tests from a single source.

This class does not insulate from problems in the source (e.g. entering an infinite loop or causing a segfault), that has to be dealt with at a higher level.

INPUT:

EXAMPLES:

sage: from sage.doctest.forker import DocTestTask
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.doctest.control import DocTestDefaults, DocTestController
sage: from sage.env import SAGE_SRC
sage: import os
sage: filename = os.path.join(SAGE_SRC,'sage','doctest','sources.py')
sage: DD = DocTestDefaults()
sage: FDS = FileDocTestSource(filename,DD)
sage: DTT = DocTestTask(FDS)
sage: DC = DocTestController(DD,[filename])
sage: ntests, results = DTT(options=DD)
sage: ntests >= 300 or ntests
True
sage: sorted(results.keys())
['cputime', 'err', 'failures', 'optionals', 'walltime']
class sage.doctest.forker.DocTestWorker(source, options, funclist=[])

Bases: multiprocessing.process.Process

The DocTestWorker process runs one DocTestTask for a given source. It returns messages about doctest failures (or all tests if verbose doctesting) though a pipe and returns results through a multiprocessing.Queue instance (both these are created in the start() method).

It runs the task in its own process-group, such that killing the process group kills this process together with its child processes.

The class has additional methods and attributes for bookkeeping by the master process. Except in run(), nothing from this class should be accessed by the child process.

INPUT:

  • source – a DocTestSource instance
  • options – an object representing doctest options.
  • funclist – a list of callables to be called at the start of the child process.

EXAMPLES:

sage: from sage.doctest.forker import DocTestWorker, DocTestTask
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.doctest.reporting import DocTestReporter
sage: from sage.doctest.control import DocTestController, DocTestDefaults
sage: from sage.env import SAGE_SRC
sage: filename = os.path.join(SAGE_SRC,'sage','doctest','util.py')
sage: DD = DocTestDefaults()
sage: FDS = FileDocTestSource(filename,DD)
sage: W = DocTestWorker(FDS, DD)
sage: W.start()
sage: DC = DocTestController(DD, filename)
sage: reporter = DocTestReporter(DC)
sage: W.join()  # Wait for worker to finish
sage: result = W.result_queue.get()
sage: reporter.report(FDS, False, W.exitcode, result, "")
    [... tests, ... s]
kill()

Kill this worker. The first time this is called, use SIGHUP. Subsequent times, use SIGKILL. Also close the message pipe if it was still open.

EXAMPLES:

sage: import time
sage: from sage.doctest.forker import DocTestWorker, DocTestTask
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.doctest.reporting import DocTestReporter
sage: from sage.doctest.control import DocTestController, DocTestDefaults
sage: from sage.env import SAGE_SRC
sage: filename = os.path.join(SAGE_SRC,'sage','doctest','tests','99seconds.rst')
sage: DD = DocTestDefaults()
sage: FDS = FileDocTestSource(filename,DD)

We set up the worker to start by blocking SIGHUP, such that killing will fail initially:

sage: from sage.ext.pselect import PSelecter
sage: import signal
sage: def block_hup():
....:     # We never __exit__()
....:     PSelecter([signal.SIGHUP]).__enter__()
sage: W = DocTestWorker(FDS, DD, [block_hup])
sage: W.start()
sage: W.killed
False
sage: W.kill()
sage: W.killed
True
sage: time.sleep(0.2)  # Worker doesn't die
sage: W.kill()         # Worker dies now
sage: time.sleep(0.2)
sage: W.is_alive()
False
read_messages()

In the master process, read from the pipe and store the data read in the messages attribute.

Note

This function may need to be called multiple times in order to read all of the messages.

EXAMPLES:

sage: from sage.doctest.forker import DocTestWorker, DocTestTask
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.doctest.reporting import DocTestReporter
sage: from sage.doctest.control import DocTestController, DocTestDefaults
sage: from sage.env import SAGE_SRC
sage: filename = os.path.join(SAGE_SRC,'sage','doctest','util.py')
sage: DD = DocTestDefaults(verbose=True,nthreads=2)
sage: FDS = FileDocTestSource(filename,DD)
sage: W = DocTestWorker(FDS, DD)
sage: W.start()
sage: while W.rmessages is not None:
....:     W.read_messages()
sage: W.join()
sage: len(W.messages) > 0
True
run()

Runs the DocTestTask under its own PGID.

TESTS:

sage: run_doctests(sage.symbolic.units) # indirect doctest
Running doctests with ID ...
Doctesting 1 file.
sage -t .../sage/symbolic/units.py
    [... tests, ... s]
----------------------------------------------------------------------
All tests passed!
----------------------------------------------------------------------
Total time for all tests: ... seconds
    cpu time: ... seconds
    cumulative wall time: ... seconds
save_result_output()

Annotate self with self.result (the result read through the result_queue and with self.output, the complete contents of self.outtmpfile. Then close the Queue and self.outtmpfile.

EXAMPLES:

sage: from sage.doctest.forker import DocTestWorker, DocTestTask
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.doctest.reporting import DocTestReporter
sage: from sage.doctest.control import DocTestController, DocTestDefaults
sage: from sage.env import SAGE_SRC
sage: filename = os.path.join(SAGE_SRC,'sage','doctest','util.py')
sage: DD = DocTestDefaults()
sage: FDS = FileDocTestSource(filename,DD)
sage: W = DocTestWorker(FDS, DD)
sage: W.start()
sage: W.join()
sage: W.save_result_output()
sage: sorted(W.result[1].keys())
['cputime', 'err', 'failures', 'optionals', 'walltime']
sage: len(W.output) > 0
True
start()

Start the worker and close the writing end of the message pipe.

TESTS:

sage: from sage.doctest.forker import DocTestWorker, DocTestTask
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.doctest.reporting import DocTestReporter
sage: from sage.doctest.control import DocTestController, DocTestDefaults
sage: from sage.env import SAGE_SRC
sage: filename = os.path.join(SAGE_SRC,'sage','doctest','util.py')
sage: DD = DocTestDefaults()
sage: FDS = FileDocTestSource(filename,DD)
sage: W = DocTestWorker(FDS, DD)
sage: W.start()
sage: try:
....:     os.fstat(W.wmessages)
....: except OSError:
....:     print "Write end of pipe successfully closed"
Write end of pipe successfully closed
sage: W.join()  # Wait for worker to finish
class sage.doctest.forker.SageDocTestRunner(*args, **kwds)

Bases: doctest.DocTestRunner

A customized version of DocTestRunner that tracks dependencies of doctests.

INPUT:

  • stdout – an open file to restore for debugging
  • checker – None, or an instance of doctest.OutputChecker
  • verbose – boolean, determines whether verbose printing is enabled.
  • optionflags – Controls the comparison with the expected output. See testmod for more information.

EXAMPLES:

sage: from sage.doctest.parsing import SageOutputChecker
sage: from sage.doctest.forker import SageDocTestRunner
sage: from sage.doctest.control import DocTestDefaults; DD = DocTestDefaults()
sage: import doctest, sys, os
sage: DTR = SageDocTestRunner(SageOutputChecker(), verbose=False, sage_options=DD, optionflags=doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS)
sage: DTR
<sage.doctest.forker.SageDocTestRunner instance at ...>
execute(example, compiled, globs)

Runs the given example, recording dependencies.

Rather than using a basic dictionary, Sage’s doctest runner uses a sage.doctest.util.RecordingDict, which records every time a value is set or retrieved. Executing the given code with this recording dictionary as the namespace allows Sage to track dependencies between doctest lines. For example, in the following two lines

sage: R.<x> = ZZ[]
sage: f = x^2 + 1

the recording dictionary records that the second line depends on the first since the first INSERTS x into the global namespace and the second line RETRIEVES x from the global namespace.

INPUT:

  • example – a doctest.Example instance.
  • compiled – a code object produced by compiling example.source
  • globs – a dictionary in which to execute compiled.

OUTPUT:

  • the output of the compiled code snippet.

EXAMPLES:

sage: from sage.doctest.parsing import SageOutputChecker
sage: from sage.doctest.forker import SageDocTestRunner
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.doctest.util import RecordingDict
sage: from sage.doctest.control import DocTestDefaults; DD = DocTestDefaults()
sage: from sage.env import SAGE_SRC
sage: import doctest, sys, os, hashlib
sage: DTR = SageDocTestRunner(SageOutputChecker(), verbose=False, sage_options=DD, optionflags=doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS)
sage: DTR.running_doctest_digest = hashlib.md5()
sage: filename = os.path.join(SAGE_SRC,'sage','doctest','forker.py')
sage: FDS = FileDocTestSource(filename,DD)
sage: globs = RecordingDict(globals())
sage: 'doctest_var' in globs
False
sage: doctests, extras = FDS.create_doctests(globs)
sage: ex0 = doctests[0].examples[0]
sage: compiled = compile(ex0.source, '<doctest sage.doctest.forker[0]>', 'single', 32768, 1)
sage: DTR.execute(ex0, compiled, globs)
1764
sage: globs['doctest_var']
42
sage: globs.set
set(['doctest_var'])
sage: globs.got
set(['Integer'])

Now we can execute some more doctests to see the dependencies.

sage: ex1 = doctests[0].examples[1]
sage: compiled = compile(ex1.source, '<doctest sage.doctest.forker[1]>', 'single', 32768, 1)
sage: DTR.execute(ex1, compiled, globs)
sage: sorted(list(globs.set))
['R', 'a']
sage: globs.got
set(['ZZ'])
sage: ex1.predecessors
[]
sage: ex2 = doctests[0].examples[2]
sage: compiled = compile(ex2.source, '<doctest sage.doctest.forker[2]>', 'single', 32768, 1)
sage: DTR.execute(ex2, compiled, globs)
a + 42
sage: list(globs.set)
[]
sage: sorted(list(globs.got))
['a', 'doctest_var']
sage: set(ex2.predecessors) == set([ex0,ex1])
True
report_failure(out, test, example, got, globs)

Called when a doctest fails.

INPUT:

  • out – a function for printing
  • test – a doctest.DocTest instance
  • example – a doctest.Example instance in test
  • got – a string, the result of running example
  • globs – a dictionary of globals, used if in debugging mode

OUTPUT:

  • prints a report to out

EXAMPLES:

sage: from sage.doctest.parsing import SageOutputChecker
sage: from sage.doctest.forker import SageDocTestRunner
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.doctest.control import DocTestDefaults; DD = DocTestDefaults()
sage: from sage.env import SAGE_SRC
sage: import doctest, sys, os
sage: DTR = SageDocTestRunner(SageOutputChecker(), verbose=True, sage_options=DD, optionflags=doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS)
sage: filename = os.path.join(SAGE_SRC,'sage','doctest','forker.py')
sage: FDS = FileDocTestSource(filename,DD)
sage: doctests, extras = FDS.create_doctests(globals())
sage: ex = doctests[0].examples[0]
sage: DTR.no_failure_yet = True
sage: DTR.report_failure(sys.stdout.write, doctests[0], ex, 'BAD ANSWER\n', {})
**********************************************************************
File ".../sage/doctest/forker.py", line 11, in sage.doctest.forker
Failed example:
    doctest_var = 42; doctest_var^2
Expected:
    1764
Got:
    BAD ANSWER

If debugging is turned on this function starts an IPython prompt when a test returns an incorrect answer:

sage: import os
sage: os.environ['SAGE_PEXPECT_LOG'] = "1"
sage: sage0.quit()
sage: _ = sage0.eval("import doctest, sys, os, multiprocessing, subprocess")
sage: _ = sage0.eval("from sage.doctest.parsing import SageOutputChecker")
sage: _ = sage0.eval("import sage.doctest.forker as sdf")
sage: _ = sage0.eval("sdf.init_sage()")
sage: _ = sage0.eval("from sage.doctest.control import DocTestDefaults")
sage: _ = sage0.eval("DD = DocTestDefaults(debug=True)")
sage: _ = sage0.eval("ex1 = doctest.Example('a = 17', '')")
sage: _ = sage0.eval("ex2 = doctest.Example('2*a', '1')")
sage: _ = sage0.eval("DT = doctest.DocTest([ex1,ex2], globals(), 'doubling', None, 0, None)")
sage: _ = sage0.eval("DTR = sdf.SageDocTestRunner(SageOutputChecker(), verbose=False, sage_options=DD, optionflags=doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS)")
sage: sage0._prompt = r"debug: "
sage: print sage0.eval("DTR.run(DT, clear_globs=False)") # indirect doctest
**********************************************************************
Line 1, in doubling
Failed example:
    2*a
Expected:
    1
Got:
    34
**********************************************************************
Previously executed commands:
...
sage: sage0.eval("a")
'...17'
sage: sage0._prompt = "sage: "
sage: sage0.eval("quit")
'Returning to doctests...TestResults(failed=1, attempted=2)'
report_overtime(out, test, example, got)

Called when the warn_long option flag is set and a doctest runs longer than the specified time.

INPUT:

  • out – a function for printing
  • test – a doctest.DocTest instance
  • example – a doctest.Example instance in test
  • got – a string, the result of running example

OUTPUT:

  • prints a report to out

EXAMPLES:

sage: from sage.doctest.parsing import SageOutputChecker
sage: from sage.doctest.forker import SageDocTestRunner
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.doctest.control import DocTestDefaults; DD = DocTestDefaults()
sage: from sage.misc.misc import walltime
sage: from sage.env import SAGE_SRC
sage: import doctest, sys, os
sage: DTR = SageDocTestRunner(SageOutputChecker(), verbose=True, sage_options=DD, optionflags=doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS)
sage: filename = os.path.join(SAGE_SRC,'sage','doctest','forker.py')
sage: FDS = FileDocTestSource(filename,DD)
sage: doctests, extras = FDS.create_doctests(globals())
sage: ex = doctests[0].examples[0]
sage: ex.walltime = 1.23
sage: DTR.report_overtime(sys.stdout.write, doctests[0], ex, 'BAD ANSWER\n')
**********************************************************************
File ".../sage/doctest/forker.py", line 11, in sage.doctest.forker
Failed example:
    doctest_var = 42; doctest_var^2
Test ran for 1.23 s
report_start(out, test, example)

Called when an example starts.

INPUT:

OUTPUT:

  • prints a report to out

EXAMPLES:

sage: from sage.doctest.parsing import SageOutputChecker
sage: from sage.doctest.forker import SageDocTestRunner
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.doctest.control import DocTestDefaults; DD = DocTestDefaults()
sage: from sage.env import SAGE_SRC
sage: import doctest, sys, os
sage: DTR = SageDocTestRunner(SageOutputChecker(), verbose=True, sage_options=DD, optionflags=doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS)
sage: filename = os.path.join(SAGE_SRC,'sage','doctest','forker.py')
sage: FDS = FileDocTestSource(filename,DD)
sage: doctests, extras = FDS.create_doctests(globals())
sage: ex = doctests[0].examples[0]
sage: DTR.report_start(sys.stdout.write, doctests[0], ex)
Trying (line 11):    doctest_var = 42; doctest_var^2
Expecting:
    1764
report_success(out, test, example, got)

Called when an example succeeds.

INPUT:

  • out – a function for printing
  • test – a doctest.DocTest instance
  • example – a doctest.Example instance in test
  • got – a string, the result of running example

OUTPUT:

  • prints a report to out
  • if in debugging mode, starts an IPython prompt at the point of the failure

EXAMPLES:

sage: from sage.doctest.parsing import SageOutputChecker
sage: from sage.doctest.forker import SageDocTestRunner
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.doctest.control import DocTestDefaults; DD = DocTestDefaults()
sage: from sage.misc.misc import walltime
sage: from sage.env import SAGE_SRC
sage: import doctest, sys, os
sage: DTR = SageDocTestRunner(SageOutputChecker(), verbose=True, sage_options=DD, optionflags=doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS)
sage: filename = os.path.join(SAGE_SRC,'sage','doctest','forker.py')
sage: FDS = FileDocTestSource(filename,DD)
sage: doctests, extras = FDS.create_doctests(globals())
sage: ex = doctests[0].examples[0]
sage: ex.walltime = 0.0
sage: DTR.report_success(sys.stdout.write, doctests[0], ex, '1764')
ok [0.00 s]
report_unexpected_exception(out, test, example, exc_info)

Called when a doctest raises an exception that’s not matched by the expected output.

If debugging has been turned on, starts an interactive debugger.

INPUT:

  • out – a function for printing
  • test – a doctest.DocTest instance
  • example – a doctest.Example instance in test
  • exc_info – the result of sys.exc_info()

OUTPUT:

  • prints a report to out
  • if in debugging mode, starts PDB with the given traceback

EXAMPLES:

sage: import os
sage: os.environ['SAGE_PEXPECT_LOG'] = "1"
sage: sage0.quit()
sage: _ = sage0.eval("import doctest, sys, os, multiprocessing, subprocess")
sage: _ = sage0.eval("from sage.doctest.parsing import SageOutputChecker")
sage: _ = sage0.eval("import sage.doctest.forker as sdf")
sage: _ = sage0.eval("from sage.doctest.control import DocTestDefaults")
sage: _ = sage0.eval("DD = DocTestDefaults(debug=True)")
sage: _ = sage0.eval("ex = doctest.Example('E = EllipticCurve([0,0]); E', 'A singular Elliptic Curve')")
sage: _ = sage0.eval("DT = doctest.DocTest([ex], globals(), 'singular_curve', None, 0, None)")
sage: _ = sage0.eval("DTR = sdf.SageDocTestRunner(SageOutputChecker(), verbose=False, sage_options=DD, optionflags=doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS)")
sage: sage0._prompt = r"\(Pdb\) "
sage: sage0.eval("DTR.run(DT, clear_globs=False)") # indirect doctest
'... ArithmeticError("Invariants %s define a singular curve."%ainvs)'
sage: sage0.eval("l")
'...if self.discriminant() == 0:...raise ArithmeticError...'
sage: sage0.eval("u")
'...EllipticCurve_field.__init__(self, [field(x) for x in ainvs])'
sage: sage0.eval("p ainvs")
'[0, 0]'
sage: sage0._prompt = "sage: "
sage: sage0.eval("quit")
'TestResults(failed=1, attempted=1)'
run(test, compileflags=None, out=None, clear_globs=True)

Runs the examples in a given doctest.

This function replaces doctest.DocTestRunner.run since it needs to handle spoofing. It also leaves the display hook in place.

INPUT:

  • test – an instance of doctest.DocTest
  • compileflags – the set of compiler flags used to execute examples (passed in to the compile()). If None, they are filled in from the result of doctest._extract_future_flags() applied to test.globs.
  • out – a function for writing the output (defaults to sys.stdout.write()).
  • clear_globs – boolean (default True): whether to clear the namespace after running this doctest.

OUTPUT:

  • f – integer, the number of examples that failed
  • t – the number of examples tried

EXAMPLES:

sage: from sage.doctest.parsing import SageOutputChecker
sage: from sage.doctest.forker import SageDocTestRunner
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.doctest.control import DocTestDefaults; DD = DocTestDefaults()
sage: from sage.env import SAGE_SRC
sage: import doctest, sys, os
sage: DTR = SageDocTestRunner(SageOutputChecker(), verbose=False, sage_options=DD, optionflags=doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS)
sage: filename = os.path.join(SAGE_SRC,'sage','doctest','forker.py')
sage: FDS = FileDocTestSource(filename,DD)
sage: doctests, extras = FDS.create_doctests(globals())
sage: DTR.run(doctests[0], clear_globs=False)
TestResults(failed=0, attempted=4)
summarize(verbose=None)

Print results of testing to self.msgfile and return number of failures and tests run.

INPUT:

  • verbose – whether to print lots of stuff

OUTPUT:

  • returns (f, t), a doctest.TestResults instance giving the number of failures and the total number of tests run.

EXAMPLES:

sage: from sage.doctest.parsing import SageOutputChecker
sage: from sage.doctest.forker import SageDocTestRunner
sage: from sage.doctest.control import DocTestDefaults; DD = DocTestDefaults()
sage: import doctest, sys, os
sage: DTR = SageDocTestRunner(SageOutputChecker(), verbose=False, sage_options=DD, optionflags=doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS)
sage: DTR._name2ft['sage.doctest.forker'] = (1,120)
sage: results = DTR.summarize()
**********************************************************************
1 item had failures:
    1 of 120 in sage.doctest.forker
sage: results
TestResults(failed=1, attempted=120)
update_digests(example)

Update global and doctest digests.

Sage’s doctest runner tracks the state of doctests so that their dependencies are known. For example, in the following two lines

sage: R.<x> = ZZ[]
sage: f = x^2 + 1

it records that the second line depends on the first since the first INSERTS x into the global namespace and the second line RETRIEVES x from the global namespace.

This function updates the hashes that record these dependencies.

INPUT:

EXAMPLES:

sage: from sage.doctest.parsing import SageOutputChecker
sage: from sage.doctest.forker import SageDocTestRunner
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.doctest.control import DocTestDefaults; DD = DocTestDefaults()
sage: from sage.env import SAGE_SRC
sage: import doctest, sys, os, hashlib
sage: DTR = SageDocTestRunner(SageOutputChecker(), verbose=False, sage_options=DD, optionflags=doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS)
sage: filename = os.path.join(SAGE_SRC,'sage','doctest','forker.py')
sage: FDS = FileDocTestSource(filename,DD)
sage: doctests, extras = FDS.create_doctests(globals())
sage: DTR.running_global_digest.hexdigest()
'd41d8cd98f00b204e9800998ecf8427e'
sage: DTR.running_doctest_digest = hashlib.md5()
sage: ex = doctests[0].examples[0]; ex.predecessors = None
sage: DTR.update_digests(ex)
sage: DTR.running_global_digest.hexdigest()
'3cb44104292c3a3ab4da3112ce5dc35c'
update_results(D)

When returning results we pick out the results of interest since many attributes are not pickleable.

INPUT:

  • D – a dictionary to update with cputime and walltime

OUTPUT:

  • the number of failures (or False if there is no failure attribute)

EXAMPLES:

sage: from sage.doctest.parsing import SageOutputChecker
sage: from sage.doctest.forker import SageDocTestRunner
sage: from sage.doctest.sources import FileDocTestSource, DictAsObject
sage: from sage.doctest.control import DocTestDefaults; DD = DocTestDefaults()
sage: from sage.env import SAGE_SRC
sage: import doctest, sys, os
sage: DTR = SageDocTestRunner(SageOutputChecker(), verbose=False, sage_options=DD, optionflags=doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS)
sage: filename = os.path.join(SAGE_SRC,'sage','doctest','forker.py')
sage: FDS = FileDocTestSource(filename,DD)
sage: doctests, extras = FDS.create_doctests(globals())
sage: from sage.doctest.util import Timer
sage: T = Timer().start()
sage: DTR.run(doctests[0])
TestResults(failed=0, attempted=4)
sage: T.stop().annotate(DTR)
sage: D = DictAsObject({'cputime':[],'walltime':[],'err':None})
sage: DTR.update_results(D)
0
sage: sorted(list(D.iteritems()))
[('cputime', [...]), ('err', None), ('failures', 0), ('walltime', [...])]
class sage.doctest.forker.SageSpoofInOut(outfile=None, infile=None)

Bases: sage.structure.sage_object.SageObject

We replace the standard doctest._SpoofOut for three reasons:

  • we need to divert the output of C programs that don’t print through sys.stdout,
  • we want the ability to recover partial output from doctest processes that segfault.
  • we also redirect stdin (usually from /dev/null) during doctests.

This class defines streams self.real_stdin, self.real_stdout and self.real_stderr which refer to the original streams.

INPUT:

  • outfile – (default: os.tmpfile()) a seekable open file object to which stdout and stderr should be redirected.
  • infile – (default: open(os.devnull)) an open file object from which stdin should be redirected.

EXAMPLES:

sage: import os, subprocess
sage: from sage.doctest.forker import SageSpoofInOut
sage: O = os.tmpfile()
sage: S = SageSpoofInOut(O)
sage: try:
....:     S.start_spoofing()
....:     print "hello world"
....: finally:
....:     S.stop_spoofing()
....:
sage: S.getvalue()
'hello world\n'
sage: O.seek(0)
sage: S = SageSpoofInOut(outfile=sys.stdout, infile=O)
sage: try:
....:     S.start_spoofing()
....:     _ = subprocess.check_call("cat")
....: finally:
....:     S.stop_spoofing()
....:
hello world
getvalue()

Gets the value that has been printed to outfile since the last time this function was called.

EXAMPLES:

sage: from sage.doctest.forker import SageSpoofInOut
sage: S = SageSpoofInOut()
sage: try:
....:     S.start_spoofing()
....:     print "step 1"
....: finally:
....:     S.stop_spoofing()
....:
sage: S.getvalue()
'step 1\n'
sage: try:
....:     S.start_spoofing()
....:     print "step 2"
....: finally:
....:     S.stop_spoofing()
....:
sage: S.getvalue()
'step 2\n'
start_spoofing()

Set stdin to read from self.infile and stdout to print to self.outfile.

EXAMPLES:

sage: import os
sage: from sage.doctest.forker import SageSpoofInOut
sage: O = os.tmpfile()
sage: S = SageSpoofInOut(O)
sage: try:
....:     S.start_spoofing()
....:     print "this is not printed"
....: finally:
....:     S.stop_spoofing()
....:
sage: S.getvalue()
'this is not printed\n'
sage: O.seek(0)
sage: S = SageSpoofInOut(infile=O)
sage: try:
....:     S.start_spoofing()
....:     v = sys.stdin.read()
....: finally:
....:     S.stop_spoofing()
....:
sage: v
'this is not printed\n'

We also catch non-Python output:

sage: try:
....:     S.start_spoofing()
....:     retval = os.system('''echo "Hello there"\nif [ $? -eq 0 ]; then\necho "good"\nfi''')
....: finally:
....:     S.stop_spoofing()
....:
sage: S.getvalue()
'Hello there\ngood\n'
stop_spoofing()

Reset stdin and stdout to their original values.

EXAMPLES:

sage: from sage.doctest.forker import SageSpoofInOut
sage: S = SageSpoofInOut()
sage: try:
....:     S.start_spoofing()
....:     print "this is not printed"
....: finally:
....:     S.stop_spoofing()
....:
sage: print "this is now printed"
this is now printed
sage.doctest.forker.dummy_handler(sig, frame)

Dummy signal handler for SIGCHLD (just to ensure the signal isn’t ignored).

TESTS:

sage: import signal
sage: from sage.doctest.forker import dummy_handler
sage: _ = signal.signal(signal.SIGUSR1, dummy_handler)
sage: os.kill(os.getpid(), signal.SIGUSR1)
sage: signal.signal(signal.SIGUSR1, signal.SIG_DFL)
<function dummy_handler at ...>
sage.doctest.forker.init_sage()

Import the Sage library.

This function is called once at the beginning of a doctest run (rather than once for each file). It imports the Sage library, sets DOCTEST_MODE to True, and invalidates any interfaces.

EXAMPLES:

sage: from sage.doctest.forker import init_sage
sage: sage.doctest.DOCTEST_MODE = False
sage: init_sage()
sage: sage.doctest.DOCTEST_MODE
True

Check that pexpect interfaces are invalidated, but still work:

sage: gap.eval("my_test_var := 42;")
'42'
sage: gap.eval("my_test_var;")
'42'
sage: init_sage()
sage: gap('Group((1,2,3)(4,5), (3,4))')
Group( [ (1,2,3)(4,5), (3,4) ] )
sage: gap.eval("my_test_var;")
Traceback (most recent call last):
...
RuntimeError: Gap produced error output...

Check that SymPy equation pretty printer is limited in doctest mode to default width (80 chars):

sage: from sympy import sympify
sage: from sympy.printing.pretty.pretty import PrettyPrinter
sage: s = sympify('+x^'.join(str(i) for i in range(30)))
sage: print PrettyPrinter(settings={'wrap_line':True}).doprint(s)
 29    28    27    26    25    24    23    22    21    20    19    18    17
x   + x   + x   + x   + x   + x   + x   + x   + x   + x   + x   + x   + x   +

 16    15    14    13    12    11    10    9    8    7    6    5    4    3
x   + x   + x   + x   + x   + x   + x   + x  + x  + x  + x  + x  + x  + x  + x

2
  + x
sage.doctest.forker.warning_function(file)

Creates a function that prints warnings to the given file.

INPUT:

  • file – an open file handle.

OUPUT:

  • a function that prings warnings to the given file.

EXAMPLES:

sage: from sage.doctest.forker import warning_function
sage: import os
sage: F = os.tmpfile()
sage: wrn = warning_function(F)
sage: wrn("bad stuff", UserWarning, "myfile.py", 0)
sage: F.seek(0)
sage: F.read()
'doctest:0: UserWarning: bad stuff\n'

Previous topic

Classes for sources of doctests

Next topic

Parsing docstrings

This Page