gavo.rscdesc module

Structure definition of resource descriptors.

The stuff they are describing is not a resource in the VO sense (whatever that is) or in the Dublin Core sense, but simply stuff held together by common metadata. If it’s got the same creator, the same base title, the same keywords, etc., it’s described by one RD.

In the DaCHS, a resource descriptor typically sets up a schema in the database.

class gavo.rscdesc.CachedException(exception, sourcePath)

Bases: object

An exception that occurred while parsing an RD.

This will remain in the cache until the underlying RD is changed.

isDirty()

raiseAgain()

class gavo.rscdesc.PkgResourcePath

Bases: str

A sentinel class used to mark an RD as coming from pkg_resources.

class gavo.rscdesc.RD(srcId, **kwargs)

Bases: Structure, ComputedMetaMixin, ScriptingMixin, StandardMacroMixin, PrivilegesMixin

A resource descriptor.

RDs collect all information about how to parse a particular source (like a collection of FITS images, a catalogue, or whatever), about the database tables the data ends up in, and the services used to access them.

In DaCHS’ RD XML serialisation, they correspond to the root element.

acceptedScriptTypes = {}

addDependency(rd, prereq)

declares that rd needs the RD prereq to properly work.

This is used in the generation of resource records to ensure that, e.g. registered data have added their served-bys to the service resources.

attrSeq = [<gavo.base.meta.MetaAttribute object>, <gavo.base.complexattrs.StructListAttribute object>, <gavo.base.complexattrs.StructListAttribute object>, <gavo.base.complexattrs.StructListAttribute object>, <gavo.base.complexattrs.StructListAttribute object>, <gavo.base.complexattrs.MultiStructListAttribute object>, <gavo.base.complexattrs.StructListAttribute object>, <gavo.base.complexattrs.StructListAttribute object>, <gavo.base.complexattrs.StructListAttribute object>, <gavo.base.complexattrs.StructListAttribute object>, <gavo.base.attrdef.ActionAttribute object>, <gavo.base.complexattrs.StructListAttribute object>, <gavo.base.complexattrs.StructListAttribute object>, <gavo.rscdesc.ResdirAttribute object>, <gavo.base.complexattrs.StructListAttribute object>, <gavo.base.complexattrs.StructAttribute object>, <gavo.base.complexattrs.PropertyAttribute object>, <gavo.base.parsecontext.IdAttribute object>, <gavo.rscdef.common.ProfileListAttribute object>, <gavo.rscdef.common.ProfileListAttribute object>, <gavo.base.attrdef.UnicodeAttribute object>, <gavo.base.complexattrs.StructListAttribute object>, <gavo.base.complexattrs.StructListAttribute object>]

cacheable = False

clearProperty(name)

completedCallbacks = []

copy(parent)

returns a deep copy of self, reparented to parent.

This is a shallow wrapper around change, present for backward compatibility.

getAbsPath(relPath)

returns the absolute path for a resdir-relative relPath.

getById(id, forceType=None)

getDataDescById(id)

getProperty(name, default=<Undefined>)

getRelResdir()

returns the inputsDir-relative resource directory path.

This never has either a leading or a trailing path.

getService(id)

getTableDefById(id)

hasProperty(name)

importModule(ctx)

invalidate()

make the RD fail on every attribute read.

See rscdesc._loadRDIntoCache for why we want this.

isDirty()

returns true if the RD on disk has a timestamp newer than loadedAt.

iterDDs()

macro_RSTcc0(stuffDesignation)

expands to a declaration that stuffDesignation is available under CC-0.

This only works in reStructured text (though it’s still almost readable as source).

You’ll probably want to use the `//procs#license-cc0`_ stream instead of this, as that also sets the rights URI.

macro_RSTccby(stuffDesignation)

expands to a declaration that stuffDesignation is available under CC-BY.

This only works in reStructured text (though it’s still almost readable as source).

You’ll probably want to use the `//procs#license-cc-by`_ stream instead of this, as that also sets the rights URI.

macro_RSTccbysa(stuffDesignation)

expands to a declaration that stuffDesignation is available under CC-BY-SA.

This only works in reStructured text (though it’s still almost readable as source).

You’ll probably want to use the `//procs#license-cc-by-sa`_ stream instead of this, as that also sets the rights URI.

managedAttrs = {'adqlCore': <gavo.base.complexattrs.MultiStructListAttribute object>, 'allProfiles': <gavo.rscdef.common.ProfileListAttribute object>, 'condDesc': <gavo.base.complexattrs.StructListAttribute object>, 'condDescs': <gavo.base.complexattrs.StructListAttribute object>, 'cores': <gavo.base.complexattrs.MultiStructListAttribute object>, 'coverage': <gavo.base.complexattrs.StructAttribute object>, 'customCore': <gavo.base.complexattrs.MultiStructListAttribute object>, 'data': <gavo.base.complexattrs.StructListAttribute object>, 'datalinkCore': <gavo.base.complexattrs.MultiStructListAttribute object>, 'dbCore': <gavo.base.complexattrs.MultiStructListAttribute object>, 'dds': <gavo.base.complexattrs.StructListAttribute object>, 'debugCore': <gavo.base.complexattrs.MultiStructListAttribute object>, 'execute': <gavo.base.complexattrs.StructListAttribute object>, 'fancyQueryCore': <gavo.base.complexattrs.MultiStructListAttribute object>, 'fixedQueryCore': <gavo.base.complexattrs.MultiStructListAttribute object>, 'id': <gavo.base.parsecontext.IdAttribute object>, 'jobs': <gavo.base.complexattrs.StructListAttribute object>, 'macDef': <gavo.base.complexattrs.StructListAttribute object>, 'macDefs': <gavo.base.complexattrs.StructListAttribute object>, 'meta': <gavo.base.meta.MetaAttribute object>, 'meta_': <gavo.base.meta.MetaAttribute object>, 'mixdefs': <gavo.base.complexattrs.StructListAttribute object>, 'mixinDef': <gavo.base.complexattrs.StructListAttribute object>, 'nullCore': <gavo.base.complexattrs.MultiStructListAttribute object>, 'outputTable': <gavo.base.complexattrs.StructListAttribute object>, 'outputTables': <gavo.base.complexattrs.StructListAttribute object>, 'procDef': <gavo.base.complexattrs.StructListAttribute object>, 'procDefs': <gavo.base.complexattrs.StructListAttribute object>, 'productCore': <gavo.base.complexattrs.MultiStructListAttribute object>, 'properties': <gavo.base.complexattrs.PropertyAttribute object>, 'property': <gavo.base.complexattrs.PropertyAttribute object>, 'pythonCore': <gavo.base.complexattrs.MultiStructListAttribute object>, 'readProfiles': <gavo.rscdef.common.ProfileListAttribute object>, 'regSuite': <gavo.base.complexattrs.StructListAttribute object>, 'registryCore': <gavo.base.complexattrs.MultiStructListAttribute object>, 'require': <gavo.base.attrdef.ActionAttribute object>, 'resRec': <gavo.base.complexattrs.StructListAttribute object>, 'resRecs': <gavo.base.complexattrs.StructListAttribute object>, 'resdir': <gavo.rscdesc.ResdirAttribute object>, 'rowmaker': <gavo.base.complexattrs.StructListAttribute object>, 'rowmakers': <gavo.base.complexattrs.StructListAttribute object>, 'schema': <gavo.base.attrdef.UnicodeAttribute object>, 'script': <gavo.base.complexattrs.StructListAttribute object>, 'scripts': <gavo.base.complexattrs.StructListAttribute object>, 'scsCore': <gavo.base.complexattrs.MultiStructListAttribute object>, 'service': <gavo.base.complexattrs.StructListAttribute object>, 'services': <gavo.base.complexattrs.StructListAttribute object>, 'siapCutoutCore': <gavo.base.complexattrs.MultiStructListAttribute object>, 'ssapCore': <gavo.base.complexattrs.MultiStructListAttribute object>, 'table': <gavo.base.complexattrs.StructListAttribute object>, 'tables': <gavo.base.complexattrs.StructListAttribute object>, 'tapCore': <gavo.base.complexattrs.MultiStructListAttribute object>, 'tests': <gavo.base.complexattrs.StructListAttribute object>, 'uploadCore': <gavo.base.complexattrs.MultiStructListAttribute object>}

name_ = 'resource'

onElementComplete()

openRes(relPath, mode='rb')

returns a file object for relPath within self’s resdir.

Deprecated. This is going to go away, use getAbsPath and a context manager.

setProperty(name, value)

updateMetaInDB(connection, reason)

updates this RD’s metadata in the database.

See the grammar in //rds for what you can give for reason.

validate()

class gavo.rscdesc.RDParseContext(doQueries=True, restricted=False, forRD=None)

Bases: ParseContext

is a parse context for RDs.

It defines a couple of attributes that structures can ask for (however, it’s good practice not to rely on their presence in case someone wants to parse XML snippets with a standard parse context, so use getattr(ctx, “doQueries”, True) or somesuch.

property failuresAreCacheable

returns true if failures produced with this context should be cached.

This is not the case with restricted parses.

classmethod fromContext(ctx, forRD=None)

a constructor that makes a context with the parameters taken from the RDParseContext ctx.

class gavo.rscdesc.ResdirAttribute(name, **kwargs)

Bases: UnicodeAttribute

An attribute representing an RD’s resdir.

This is for resource’s resdir attribute, which will by default get the inputsDir prepended. There is the special value “.” that means “the directory containing the RD”, which is what people should do now.

This will only work for RDs.

Note that without that, the resdir defaults to <inputsDir>/<schemaName>.

feedObject(instance, parsedValue)

gavo.rscdesc.canonicalizeRDId(srcId)

returns a standard rd id for srcId.

srcId may be a file system path, or it may be an “id”. The canonical basically is “inputs-relative path without .rd extension”. Everything that’s not within inputs or doesn’t end with .rd is handed through. // is expanded to __system__/. The path to built-in RDs, /resources/inputs, is treated analogous to inputsDir.

gavo.rscdesc.getRD(srcId, doQueries=True, restricted=False, useRD=None)

returns a ResourceDescriptor for srcId.

srcId is something like an input-relative path; you’ll generally omit the extension (unless it’s not the standard .rd).

getRD furnishes the resulting RD with an idmap attribute containing the mapping from id to object collected by the parse context.

The useRD parameter is for _loadRDIntoCache exclusively and is used by it internally. It is strictly an ugly implementation detail.

gavo.rscdesc.getRDInputStream(srcId)

returns a read-open stream for the XML source of the resource descriptor with srcId.

srcId is already normalized; that means that absolute paths must point to a file (sans possibly .rd), relative paths are relative to inputsDir or pkg_resources(/resources/inputs).

This function prefers files with .rd to those without, and inputsDir to pkg_resources (the latter allowing the user to override built-in system RDs).

gavo.rscdesc.refuseBlacklisted(srcId, rdInputPath)

raises an exception if rdInputPath ends with or srcId is equal to one of the strings in [general]rdblacklist.

gavo.rscdesc.setRDDateTimes(ctx, rd, inputFile)

sets the _dataUpdated and _metadataUpdated meta items on rd.