Classes for sources of doctests

This module defines various classes for sources from which doctests originate, such as files, functions or database entries.

AUTHORS:

  • David Roe (2012-03-27) – initial version, based on Robert Bradshaw’s code.
class sage.doctest.sources.DictAsObject(attrs)

Bases: dict

A simple subclass of dict that inserts the items from the initializing dictionary into attributes.

EXAMPLES:

sage: from sage.doctest.sources import DictAsObject
sage: D = DictAsObject({'a':2})
sage: D.a
2
class sage.doctest.sources.DocTestSource(options)

Bases: object

This class provides a common base class for different sources of doctests.

INPUT:

class sage.doctest.sources.FileDocTestSource(path, options)

Bases: sage.doctest.sources.DocTestSource

This class creates doctests from a file.

INPUT:

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.env import SAGE_SRC
sage: import os
sage: filename = os.path.join(SAGE_SRC,'sage','doctest','sources.py')
sage: FDS = FileDocTestSource(filename,DocTestDefaults())
sage: FDS.basename
'sage.doctest.sources'

TESTS:

sage: TestSuite(FDS).run()
basename()

The basename of this file source, e.g. sage.doctest.sources

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.env import SAGE_SRC
sage: import os
sage: filename = os.path.join(SAGE_SRC,'sage','rings','integer.pyx')
sage: FDS = FileDocTestSource(filename,DocTestDefaults())
sage: FDS.basename
'sage.rings.integer'
create_doctests(namespace)

Returns a list of doctests for this file.

INPUT:

OUTPUT:

  • doctests – a list of doctests defined in this file.
  • extras – a dictionary

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.env import SAGE_SRC
sage: import os
sage: filename = os.path.join(SAGE_SRC,'sage','doctest','sources.py')
sage: FDS = FileDocTestSource(filename,DocTestDefaults())
sage: doctests, extras = FDS.create_doctests(globals())
sage: len(doctests)
40
sage: extras['tab']
False

We give a self referential example:

sage: doctests[17].name
'sage.doctest.sources.FileDocTestSource.create_doctests'
sage: doctests[17].examples[10].source
'doctests[Integer(17)].examples[Integer(10)].source\n'

TESTS:

We check that we correctly process results that depend on 32 vs 64 bit architecture:

sage: import sys
sage: bitness = '64' if sys.maxsize > (1 << 32) else '32'
sage: n = -920390823904823094890238490238484; hash(n) > 0
False # 32-bit
True  # 64-bit
sage: ex = doctests[17].examples[13]
sage: (bitness == '64' and ex.want == 'True  \n') or (bitness == '32' and ex.want == 'False \n')
True

We check that lines starting with a # aren’t doctested:

#sage: raise RuntimeError
in_lib()

Whether this file should be considered part of the Sage library.

Such files aren’t loaded before running tests.

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.env import SAGE_SRC
sage: import os
sage: filename = os.path.join(SAGE_SRC,'sage','rings','integer.pyx')
sage: FDS = FileDocTestSource(filename,DocTestDefaults())
sage: FDS.in_lib
True

You can override the default:

sage: FDS = FileDocTestSource("hello_world.py",DocTestDefaults())
sage: FDS.in_lib
False
sage: FDS = FileDocTestSource("hello_world.py",DocTestDefaults(force_lib=True))
sage: FDS.in_lib
True
printpath()

Whether the path is printed absolutely or relatively depends on an option.

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.env import SAGE_SRC
sage: import os
sage: root = os.path.realpath(os.path.join(SAGE_SRC,'sage'))
sage: filename = os.path.join(root,'doctest','sources.py')
sage: cwd = os.getcwd()
sage: os.chdir(root)
sage: FDS = FileDocTestSource(filename,DocTestDefaults(randorder=0,abspath=False))
sage: FDS.printpath
'doctest/sources.py'
sage: FDS = FileDocTestSource(filename,DocTestDefaults(randorder=0,abspath=True))
sage: FDS.printpath
'.../sage/doctest/sources.py'
sage: os.chdir(cwd)
class sage.doctest.sources.PythonSource

Bases: sage.doctest.sources.SourceLanguage

This class defines the functions needed for the extraction of doctests from python sources.

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.env import SAGE_SRC
sage: import os
sage: filename = os.path.join(SAGE_SRC,'sage','doctest','sources.py')
sage: FDS = FileDocTestSource(filename,DocTestDefaults())
sage: type(FDS)
<class 'sage.doctest.sources.PythonFileSource'>
ending_docstring(line)

Determines whether the input line ends a docstring.

INPUT:

  • line – a string, one line of an input file.

OUTPUT:

  • an object that, when evaluated in a boolean context, gives True or False depending on whether the input line marks the end of a docstring.

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.doctest.util import NestedName
sage: from sage.env import SAGE_SRC
sage: import os
sage: filename = os.path.join(SAGE_SRC,'sage','doctest','sources.py')
sage: FDS = FileDocTestSource(filename,DocTestDefaults())
sage: FDS._init()
sage: FDS.quotetype = "'''"
sage: FDS.ending_docstring("'''")
<_sre.SRE_Match object at ...>
sage: FDS.ending_docstring('\"\"\"')
starting_docstring(line)

Determines whether the input line starts a docstring.

If the input line does start a docstring (a triple quote), then this function updates self.qualified_name.

INPUT:

  • line – a string, one line of an input file

OUTPUT:

  • either None or a Match object.

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.doctest.util import NestedName
sage: from sage.env import SAGE_SRC
sage: import os
sage: filename = os.path.join(SAGE_SRC,'sage','doctest','sources.py')
sage: FDS = FileDocTestSource(filename,DocTestDefaults())
sage: FDS._init()
sage: FDS.starting_docstring("r'''")
<_sre.SRE_Match object at ...>
sage: FDS.ending_docstring("'''")
<_sre.SRE_Match object at ...>
sage: FDS.qualified_name = NestedName(FDS.basename)
sage: FDS.starting_docstring("class MyClass(object):")
sage: FDS.starting_docstring("    def hello_world(self):")
sage: FDS.starting_docstring("        '''")
<_sre.SRE_Match object at ...>
sage: FDS.qualified_name
sage.doctest.sources.MyClass.hello_world
sage: FDS.ending_docstring("    '''")
<_sre.SRE_Match object at ...>
sage: FDS.starting_docstring("class NewClass(object):")
sage: FDS.starting_docstring("    '''")
<_sre.SRE_Match object at ...>
sage: FDS.qualified_name
sage.doctest.sources.NewClass
class sage.doctest.sources.RestSource

Bases: sage.doctest.sources.SourceLanguage

This class defines the functions needed for the extraction of doctests from ReST sources.

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: filename = "sage_doc.rst"
sage: FDS = FileDocTestSource(filename,DocTestDefaults())
sage: type(FDS)
<class 'sage.doctest.sources.RestFileSource'>
ending_docstring(line)

When the indentation level drops below the initial level the block ends.

INPUT:

  • line – a string, one line of an input file

OUTPUT:

  • a boolean, whether the verbatim block is ending.

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: filename = "sage_doc.rst"
sage: FDS = FileDocTestSource(filename,DocTestDefaults())
sage: FDS._init()
sage: FDS.starting_docstring("Hello world::")
True
sage: FDS.ending_docstring("    sage: 2 + 2")
False
sage: FDS.ending_docstring("    4")
False
sage: FDS.ending_docstring("We are now done")
True
parse_docstring(docstring, namespace, start)

Return a list of doctest defined in this docstring.

Code blocks in a REST file can contain python functions with their own docstrings in addition to in-line doctests. We want to include the tests from these inner docstrings, but Python’s doctesting module has a problem if we just pass on the whole block, since it expects to get just a docstring, not the Python code as well.

Our solution is to create a new doctest source from this code block and append the doctests created from that source. We then replace the occurrences of “sage:” and “>>>” occurring inside a triple quote with “safe:” so that the doctest module doesn’t treat them as tests.

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.doctest.parsing import SageDocTestParser
sage: from sage.doctest.util import NestedName
sage: filename = "sage_doc.rst"
sage: FDS = FileDocTestSource(filename,DocTestDefaults())
sage: FDS.parser = SageDocTestParser(False, set(['sage']))
sage: FDS.qualified_name = NestedName('sage_doc')
sage: s = "Some text::\n\n    def example_python_function(a, \
....:      b):\n        '''\n        Brief description \
....:      of function.\n\n        EXAMPLES::\n\n            \
....:      sage: test1()\n            sage: test2()\n        \
....:      '''\n        return a + b\n\n    sage: test3()\n\nMore \
....:      ReST documentation."
sage: tests = FDS.parse_docstring(s, {}, 100)
sage: len(tests)
2
sage: for ex in tests[0].examples:
....:     print ex.sage_source,
test3()
sage: for ex in tests[1].examples:
....:     print ex.sage_source,
test1()
test2()
sig_on_count()
starting_docstring(line)

A line ending with a double quote starts a verbatim block in a ReST file.

This function also determines whether the docstring block should be joined with the previous one, or should be skipped.

INPUT:

  • line – a string, one line of an input file

OUTPUT:

  • either None or a Match object.

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: filename = "sage_doc.rst"
sage: FDS = FileDocTestSource(filename,DocTestDefaults())
sage: FDS._init()
sage: FDS.starting_docstring("Hello world::")
True
sage: FDS.ending_docstring("    sage: 2 + 2")
False
sage: FDS.ending_docstring("    4")
False
sage: FDS.ending_docstring("We are now done")
True
sage: FDS.starting_docstring(".. link")
sage: FDS.starting_docstring("::")
True
sage: FDS.linking
True
class sage.doctest.sources.SourceLanguage

An abstract class for functions that depend on the programming language of a doctest source.

Currently supported languages include Python, ReST and LaTeX.

parse_docstring(docstring, namespace, start)

Return a list of doctest defined in this docstring.

This function is called by DocTestSource._process_doc(). The default implementation, defined here, is to use the sage.doctest.parsing.SageDocTestParser attached to this source to get doctests from the docstring.

INPUT:

  • docstring – a string containing documentation and tests.
  • namespace – a dictionary or sage.doctest.util.RecordingDict.
  • start – an integer, one less than the starting line number

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.doctest.parsing import SageDocTestParser
sage: from sage.doctest.util import NestedName
sage: from sage.env import SAGE_SRC
sage: import os
sage: filename = os.path.join(SAGE_SRC,'sage','doctest','util.py')
sage: FDS = FileDocTestSource(filename,DocTestDefaults())
sage: doctests, _ = FDS.create_doctests({})
sage: for dt in doctests:
....:     FDS.qualified_name = dt.name
....:     dt.examples = dt.examples[:-1] # strip off the sig_on() test
....:     assert(FDS.parse_docstring(dt.docstring,{},dt.lineno-1)[0] == dt)
class sage.doctest.sources.StringDocTestSource(basename, source, options, printpath, lineno_shift=0)

Bases: sage.doctest.sources.DocTestSource

This class creates doctests from a string.

INPUT:

  • basename – string such as ‘sage.doctests.sources’, going into the names of created doctests and examples.
  • source – a string, giving the source code to be parsed for doctests.
  • options – a sage.doctest.control.DocTestDefaults or equivalent.
  • printpath – a string, to be used in place of a filename when doctest failures are displayed.
  • lineno_shift – an integer (default: 0) by which to shift the line numbers of all doctests defined in this string.

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import StringDocTestSource, PythonSource
sage: from sage.structure.dynamic_class import dynamic_class
sage: s = "'''\n    sage: 2 + 2\n    4\n'''"
sage: PythonStringSource = dynamic_class('PythonStringSource',(StringDocTestSource, PythonSource))
sage: PSS = PythonStringSource('<runtime>', s, DocTestDefaults(), 'runtime')
sage: dt, extras = PSS.create_doctests({})
sage: len(dt)
1
sage: extras['tab']
[]
create_doctests(namespace)

Creates doctests from this string.

INPUT:

OUTPUT:

  • doctests – a list of doctests defined by this string
  • tab_locations – either False or a list of linenumbers on which tabs appear.

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import StringDocTestSource, PythonSource
sage: from sage.structure.dynamic_class import dynamic_class
sage: s = "'''\n    sage: 2 + 2\n    4\n'''"
sage: PythonStringSource = dynamic_class('PythonStringSource',(StringDocTestSource, PythonSource))
sage: PSS = PythonStringSource('<runtime>', s, DocTestDefaults(), 'runtime')
sage: dt, tabs = PSS.create_doctests({})
sage: for t in dt:
....:     print t.name, t.examples[0].sage_source
<runtime> 2 + 2
class sage.doctest.sources.TexSource

Bases: sage.doctest.sources.SourceLanguage

This class defines the functions needed for the extraction of doctests from a LaTeX source.

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: filename = "sage_paper.tex"
sage: FDS = FileDocTestSource(filename,DocTestDefaults())
sage: type(FDS)
<class 'sage.doctest.sources.TexFileSource'>
ending_docstring(line, check_skip=True)

Determines whether the input line ends a docstring.

Docstring blocks in tex files are defined by verbatim environments, and can be linked together by adding %link immediately after the end{verbatim}.

Within a verbatim block, you can tell Sage not to process the rest of the block by including a %skip line.

INPUT:

  • line – a string, one line of an input file
  • check_skip – boolean (default True), used internally in starting_docstring.

OUTPUT:

  • a boolean giving whether the input line marks the end of a docstring (verbatim block).

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: filename = "sage_paper.tex"
sage: FDS = FileDocTestSource(filename,DocTestDefaults())
sage: FDS._init()
sage: FDS.ending_docstring(r"\end{verbatim}")
True
sage: FDS.linking
False

Use %link to link with the next verbatim block:

sage: FDS.ending_docstring(r"\end{verbatim}%link")
True
sage: FDS.linking
True

%skip also ends a docstring block:

sage: FDS.ending_docstring("%skip")
True
starting_docstring(line)

Determines whether the input line starts a docstring.

Docstring blocks in tex files are defined by verbatim environments, and can be linked together by adding %link immediately after the end{verbatim}.

Within a verbatim block, you can tell Sage not to process the rest of the block by including a %skip line.

INPUT:

  • line – a string, one line of an input file

OUTPUT:

  • a boolean giving whether the input line marks the start of a docstring (verbatim block).

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: filename = "sage_paper.tex"
sage: FDS = FileDocTestSource(filename,DocTestDefaults())
sage: FDS._init()

We start docstrings with begin{verbatim}:

sage: FDS.starting_docstring(r"\begin{verbatim}")
True
sage: FDS.skipping
False
sage: FDS.ending_docstring("sage: 2+2")
False
sage: FDS.ending_docstring("4")
False

To start ignoring the rest of the verbatim block, use %skip:

sage: FDS.ending_docstring("%skip")
True
sage: FDS.skipping
True
sage: FDS.starting_docstring("sage: raise RuntimeError")
False

You can even pretend to start another verbatim block while skipping:

sage: FDS.starting_docstring(r"\begin{verbatim}")
False
sage: FDS.skipping
True

To stop skipping end the verbatim block:

sage: FDS.starting_docstring(r"\end{verbatim} %link")
False
sage: FDS.skipping
False

Linking works even when the block was ended while skipping:

sage: FDS.linking
True
sage: FDS.starting_docstring(r"\begin{verbatim}")
True
sage.doctest.sources.get_basename(path)

This function returns the basename of the given path, e.g. sage.doctest.sources or doc.ru.tutorial.tour_advanced

EXAMPLES:

sage: from sage.doctest.sources import get_basename
sage: from sage.env import SAGE_SRC
sage: import os
sage: get_basename(os.path.join(SAGE_SRC,'sage','doctest','sources.py'))
'sage.doctest.sources'

Previous topic

Classes involved in doctesting

Next topic

Processes for running doctests

This Page