gavo.rscdef.regtest module

A framework for regression tests within RDs.

The basic idea is that there’s small pieces of python almost-declaratively defining tests for a given piece of data. These things can then be run while (or rather, after) executing dachs val.

class gavo.rscdef.regtest.DataURL(parent, **kwargs)

Bases: Structure

A source document for a regression test.

As string URLs, they specify where to get data from, but the additionally let you specify uploads, authentication, headers and http methods, while at the same time saving you manual escaping of parameters.

The bodies is the path to run the test against. This is interpreted as relative to the RD if there’s no leading slash, relative to the server if there’s a leading slash, and absolute if there’s a scheme.

The attributes are translated to parameters, except for a few pre-defined names. If you actually need those as URL parameters, should at us and we’ll provide some way of escaping these.

We don’t actually parse the URLs coming in here. GET parameters are appended with a & if there’s a ? in the existing URL, with a ? if not. Again, shout if this is too dumb for you (but urlparse really isn’t all that robust either…)

attrSeq = [<gavo.base.structure.DataContent object>, <gavo.base.attrdef.UnicodeAttribute object>, <gavo.base.attrdef.IntAttribute object>, <gavo.base.complexattrs.DictAttribute object>, <gavo.base.attrdef.BooleanAttribute object>, <gavo.base.attrdef.UnicodeAttribute object>, <gavo.rscdef.common.ResdirRelativeAttribute object>, <gavo.base.complexattrs.StructListAttribute object>, <gavo.base.parsecontext.IdAttribute object>, <gavo.rscdef.regtest.DynamicOpenVocAttribute object>, <gavo.base.attrdef.EnumeratedUnicodeAttribute object>, <gavo.base.attrdef.UnicodeAttribute object>, <gavo.rscdef.common.RDAttribute object>]

completedCallbacks = []

getAttribute(name)

Returns an attribute instance from name.

This function will raise a StructureError if no matching attribute definition is found.

getFullId()

getParams()

returns the URL parameters as a sequence of kw, value pairs.

getValue(serverURL)

returns a pair of full request URL and postable payload for this test.

httpURL = '(not retrieved)'

managedAttrs = {'content_': <gavo.base.structure.DataContent object>, 'httpAuthKey': <gavo.base.attrdef.UnicodeAttribute object>, 'httpChunkSize': <gavo.base.attrdef.IntAttribute object>, 'httpHeader': <gavo.base.complexattrs.DictAttribute object>, 'httpHonorRedirects': <gavo.base.attrdef.BooleanAttribute object>, 'httpMethod': <gavo.base.attrdef.UnicodeAttribute object>, 'httpPostMediaType': <gavo.base.attrdef.UnicodeAttribute object>, 'httpUpload': <gavo.base.complexattrs.StructListAttribute object>, 'id': <gavo.base.parsecontext.IdAttribute object>, 'open': <gavo.rscdef.regtest.DynamicOpenVocAttribute object>, 'parSet': <gavo.base.attrdef.EnumeratedUnicodeAttribute object>, 'postPayload': <gavo.rscdef.common.ResdirRelativeAttribute object>, 'rd': <gavo.rscdef.common.RDAttribute object>, 'uploads': <gavo.base.complexattrs.StructListAttribute object>, 'value': <gavo.base.complexattrs.DictAttribute object>}

name_ = 'url'

property postPayload

property rd

retrieveResource(serverURL, timeout)

returns a triple of status, headers, and content for retrieving this URL.

validate()

class gavo.rscdef.regtest.DynamicOpenVocAttribute(name, **kwargs)

Bases: AttributeDef

an attribute that collects arbitrary attributes in a sequence of pairs.

The finished sequence is available as a freeAttrs attribute on the embedding instance. No parsing is done, everything is handled as a string.

feed(ctx, instance, value)

feedObject(instance, value)

getCopy(instance, newParent)

iterParentMethods()

returns an iterator over (name, method) pairs that should be inserted in the parent class.

makeUserDoc()

typeDesc_ = 'any attribute not otherwise used'

class gavo.rscdef.regtest.Keywords(*args, **kwargs)

Bases: Action

A class encapsulating test selection keywords.

There’s a match method that takes a string and returns true if either no keywords are defined or all keywords are present in other (after case folding).

This doubles as an argparse action and as such is “self-parsing” if you will.

match(other)

class gavo.rscdef.regtest.RegTest(*args, **kwargs)

Bases: ProcApp, TestCase

A regression test.

Tests are defined through url and code elements. See `Regression Testing`_ for more information.

XPATH_NAMESPACE_MAP = {'h': 'http://www.w3.org/1999/xhtml', 'm': 'http://www.ivoa.net/xml/mivot', 'o': 'http://www.openarchives.org/OAI/2.0/', 'v': 'http://www.ivoa.net/xml/VOTable/v1.3', 'v1': 'http://www.ivoa.net/xml/VOTable/v1.1', 'v2': 'http://www.ivoa.net/xml/VOTable/v1.2'}

additionalNamesForProcs = {'EqualingRE': <class 'gavo.utils.codetricks.EqualingRE'>}

assertHTTPStatus(expectedStatus)

checks whether the request came back with expectedStatus.

assertHasStrings(*strings)

checks that all its arguments are found within content.

If string arguments are passed, they are utf-8 encoded before comparison. If that’s not what you want, pass bytes yourself.

assertHeader(key, value)

checks that header key has value in the response headers.

keys are compared case-insensitively, values are compared literally.

assertLacksStrings(*strings)

checks that all its arguments are not found within content.

assertValidatesXSD()

checks whether the returned data are XSD valid.

This uses DaCHS built-in XSD validator with the built-in schema files; it hence will in general not retrieve schema files from external sources.

assertXpath(path, assertions)

checks an xpath assertion.

path is an xpath (as understood by lxml), with namespace prefixes statically mapped; there’s currently v2 (VOTable 1.2), v1 (VOTable 1.1), v (whatever VOTable version is the current DaCHS default), h (the namespace of the XHTML elements DaCHS generates), m (the provisional MIVOT namespace) and o (OAI-PMH 2.0).

If you need more prefixes, hack the source and feed back your changes (or just add to self.XPATH_NAMESPACE_MAP locally).

path must match exactly one element.

assertions is a dictionary mapping attribute names to their expected value. Use the key None to check the element content, and match for None if you expect an empty element. To match against a namespaced attribute, you have to give the full URI; prefixes are not applied here. This would look like:

"{http://www.w3.org/2001/XMLSchema-instance}type": "vg:OAIHTTP"

If you need an RE match rather than equality, there’s EqualingRE in your code’s namespace.

attrSeq = [<gavo.base.complexattrs.StructListAttribute object>, <gavo.base.attrdef.UnicodeAttribute object>, <gavo.base.attrdef.UnicodeAttribute object>, <gavo.base.attrdef.UnicodeAttribute object>, <gavo.base.parsecontext.IdAttribute object>, <gavo.base.attrdef.UnicodeAttribute object>, <gavo.base.parsecontext.OriginalAttribute object>, <gavo.base.parsecontext.ReferenceAttribute object>, <gavo.rscdef.common.RDAttribute object>, <gavo.base.complexattrs.StructListAttribute object>, <gavo.base.attrdef.StringSetAttribute object>, <gavo.base.attrdef.NWUnicodeAttribute object>, <gavo.base.attrdef.EnumeratedUnicodeAttribute object>, <gavo.base.complexattrs.StructAttribute object>]

completedCallbacks = []

data = b'<No data retrieved yet>'

property description

fakeForPyUnit()

formalArgs = 'self'

getDataSource()

returns a string pointing people to where data came from.

getFirstVOTableRow(rejectExtras=True)

interprets data as a VOTable and returns the first row as a dictionary

It will normally ensure that only one row is returned. To make it silently discard extra rows, make sure the result is sorted, or you will get randomly failing tests. Database-querying cores (which is where order is an issue) also honor _DBOPTIONS_ORDER).

getFullId()

getUnique(seq)

returns seq[0], asserting at the same time that len(seq) is 1.

The idea is that you can say row = self.getUnique(self.getVOTableRows()) and have a nice test on the side – and no ugly IndexError on an empty respone.

getVOTableRows()

parses the first table in a result VOTable and returns the contents as a sequence of dictionaries.

getXpath(path, element=None)

returns the equivalent of tree.xpath(path) for an lxml etree of the current document or in element, if passed in.

This uses the same namespace conventions as assertXpath.

managedAttrs = {'bind': <gavo.base.complexattrs.StructListAttribute object>, 'bindings': <gavo.base.complexattrs.StructListAttribute object>, 'code': <gavo.base.attrdef.UnicodeAttribute object>, 'deprecated': <gavo.base.attrdef.UnicodeAttribute object>, 'doc': <gavo.base.attrdef.UnicodeAttribute object>, 'id': <gavo.base.parsecontext.IdAttribute object>, 'name': <gavo.base.attrdef.UnicodeAttribute object>, 'original': <gavo.base.parsecontext.OriginalAttribute object>, 'procDef': <gavo.base.parsecontext.ReferenceAttribute object>, 'rd': <gavo.rscdef.common.RDAttribute object>, 'setup': <gavo.base.complexattrs.StructListAttribute object>, 'setups': <gavo.base.complexattrs.StructListAttribute object>, 'tags': <gavo.base.attrdef.StringSetAttribute object>, 'title': <gavo.base.attrdef.NWUnicodeAttribute object>, 'type': <gavo.base.attrdef.EnumeratedUnicodeAttribute object>, 'url': <gavo.base.complexattrs.StructAttribute object>}

name_ = 'regTest'

pointNextToLocation(addToPath='')

arranges for the value of the location header to become the base URL of the next test.

addToPath, if given, is appended to the location header.

If no location header was provided, the test fails.

All this of course only works for tests in sequential regSuites.

property rd

requestTime = None

requiredType = 'regTest'

retrieveData(serverURL, timeout)

returns headers and content when retrieving the resource at url.

Sets the headers and data attributes of the test instance.

runCount = 1

class gavo.rscdef.regtest.RegTestSuite(parent, **kwargs)

Bases: Structure

A suite of regression tests.

attrSeq = [<gavo.base.parsecontext.IdAttribute object>, <gavo.base.attrdef.BooleanAttribute object>, <gavo.base.complexattrs.StructListAttribute object>, <gavo.base.attrdef.NWUnicodeAttribute object>]

completeElement(ctx)

completedCallbacks = []

expand(*args, **kwargs)

hand macro expansion to the RD.

itertests(tags, keywords)

managedAttrs = {'id': <gavo.base.parsecontext.IdAttribute object>, 'regTest': <gavo.base.complexattrs.StructListAttribute object>, 'sequential': <gavo.base.attrdef.BooleanAttribute object>, 'tests': <gavo.base.complexattrs.StructListAttribute object>, 'title': <gavo.base.attrdef.NWUnicodeAttribute object>}

name_ = 'regSuite'

class gavo.rscdef.regtest.TestRunner(suites, serverURL=None, verbose=True, dumpNegative=False, tags=None, timeout=45, failFile=None, nRepeat=1, execDelay=0, nThreads=8, printTitles=False, keywords=None)

Bases: object

A runner for regression tests.

It is constructed with a sequence of suites (RegTestSuite instances) and allows running these in parallel. It honors the suites’ wishes as to being executed sequentially.

classmethod fromRD(rd, **kwargs)

constructs a TestRunner for a single ResourceDescriptor.

classmethod fromSuite(suite, **kwargs)

constructs a TestRunner for a RegTestSuite suite

classmethod fromTest(test, **kwargs)

constructs a TestRunner for a single RegTest

runOneTest(test, threadId, execDelay)

runs test and puts the results in the result queue.

This is usually run in a thread. However, threadId is only used for reporting, so you may run this without threads.

To support sequential execution, if test has a followUp attribute, this followUp is queued after the test has run.

If the execDelay argument is non-zero, the thread delays its execution by that many seconds.

runTests(showDots=False)

executes the tests in a random order and in parallel.

runTestsInOrder()

runs all tests sequentially and in the order they were added.

class gavo.rscdef.regtest.TestStatistics(verbose=True)

Bases: object

A statistics gatherer/reporter for the regression tests.

add(status, runTime, title, payload, srcRD)

adds a test result to the statistics.

status is either OK, FAIL, or ERROR, runTime is the time spent in running the test, title is the test’s title, and payload is “something” associated with failures that should help diagnosing them.

getFailures()

returns a string containing some moderately verbose info on the failures collected.

getReport()

returns a string representation of a short report on how the tests fared.

save(target)

saves the entire test statistics to target.

This is a pickle of basically what’s added with add. No tools for doing something with this are provided so far.

class gavo.rscdef.regtest.Upload(parent, **kwargs)

Bases: Structure

An upload going with a URL.

addToForm(form)

sets up a _Form instance to upload the data.

attrSeq = [<gavo.base.structure.DataContent object>, <gavo.base.attrdef.UnicodeAttribute object>, <gavo.base.parsecontext.IdAttribute object>, <gavo.base.attrdef.UnicodeAttribute object>, <gavo.rscdef.common.ResdirRelativeAttribute object>]

completedCallbacks = []

managedAttrs = {'content_': <gavo.base.structure.DataContent object>, 'fileName': <gavo.base.attrdef.UnicodeAttribute object>, 'id': <gavo.base.parsecontext.IdAttribute object>, 'name': <gavo.base.attrdef.UnicodeAttribute object>, 'source': <gavo.rscdef.common.ResdirRelativeAttribute object>}

name_ = 'httpUpload'

property rd

property source

validate()

gavo.rscdef.regtest.doHTTPRequest(scheme, method, host, path, query, payload, headers, timeout)

creates the HTTP request and retrieves the result.

gavo.rscdef.regtest.getAuthFor(authKey)

returns a header dictionary to authenticate for authKey.

authKey is a key into ~/.gavo/test.creds.

gavo.rscdef.regtest.getHeaderValue(headers, key)

returns the value for key in the httplib headers.

Matching is case-insensitive as required by HTTP. Missing keys raise KeyErrors.

gavo.rscdef.regtest.main(args=None)

user interaction for gavo test.

gavo.rscdef.regtest.parseCommandLine(args=None)

parses the command line for main()

gavo.rscdef.regtest.urlToURL()

converts HTTP (GET) URLs to URL elements.