gavo.base.meta module

Code dealing with meta information.

In DaCHS, tree-structured metadata can be attached to tables, services, resources as a whole, and some other structures. This module provides a python mixin to keep and manipulate such metadata.

We deal with VO-style RMI metadata but also allow custom metadata. Custom metadata keys should usually start with _.

See develNotes for some discussion of why this is so messy and an explanation of why in particular addMeta and helpers are a minefield of selections.

The rough plan to do this looks like this:

Metadata is kept in containers that mix in MetaMixin. Meta information is accessed through keys of the form <atom>{.<atom>} The first atom is the primary key. An atom consists exclusively of ascii letters and the underscore.

There’s a meta mixin having the methods addMeta and getMeta that care about finding metadata including handover. For compound metadata, they will split the key and hand over to the parent if the don’t have a meta item for the main key.

To make this complete for DaCHS, this needs to be completemented with magic, “typed” meta classes overriding certain types of behaviour (see the forKeys class decorator for how this works); base.__init__ arranges for base.typedmeta to be imported. To access these magic types, use META_CLASSES_FOR_KEYS (but don’t change that directory except through @meta.forKeys).

class gavo.base.meta.ComputedMetaMixin

Bases: MetaMixin

A MetaMixin for classes that want to implement defaults for unresolvable meta items.

If getMeta would return a NoMetaKey, this mixin’s getMeta will check the presence of a _meta_<key> method (replacing dots with two underscores) and, if it exists, returns whatever it returns. Otherwise, the exception will be propagated.

The _meta_<key> methods should return MetaItems; if something else is returned, it is wrapped in a MetaValue.

On copying such metadata, the copy will retain the value on the original if it has one. This does not work for computed metadata that would be inherited.

getMetaKeys()

class gavo.base.meta.IncludesChildren

Bases: str

a formatted result that already includes all meta children.

This is returned from some of the special meta types’ HTML formatters to keep the HTMLMetaBuilder from adding meta items that are already included in their HTML.

class gavo.base.meta.MetaAttribute(description='Metadata')

Bases: AttributeDef

An attribute magically inserting meta values to Structures mixing in MetaMixin.

We don’t want to keep metadata itself in structures for performance reasons, so we define a parser of our own in here.

create(parent, ctx, name)

property default_

feedObject(instance, value)

getCopy(parent, newParent, ctx)

creates a deep copy of the current meta dictionary and returns it.

This is used when a MetaMixin’s attribute is set to copyable and a meta carrier is copied. As there’s currently no way to make the _metaAttr copyable, this isn’t called by itself. If you must, you can manually call this (_metaAttr.getCopy), but that’d really be an indication the interface needs changes.

Note that the copying semantics is a bit funky: Copied values remain, but on write, sequences are replaced rather than added to.

iterEvents(instance)

makeUserDoc()

typedesc = 'Metadata'

class gavo.base.meta.MetaBuilder

Bases: object

A base class for meta builders.

Builders are passed to a MetaItem’s traverse method or to MetaMixin’s buildRepr method to build representations of the meta information.

You can override startKey, endKey, and enterValue. If you are not doing anything fancy, you can get by by just overriding enterValue and inspecting curAtoms[-1] (which contains the last meta key).

You will want to override getResult.

endKey(key)

enterValue(value)

getResult()

startKey(key)

exception gavo.base.meta.MetaCardError(msg, carrier=None, hint=None, key=None)

Bases: MetaError

is raised when a meta value somehow has the wrong cardinality (e.g., on attempting to stringify a sequence meta).

exception gavo.base.meta.MetaError(msg, carrier=None, hint=None, key=None)

Bases: Error

A base class for metadata-related errors.

MetaErrors have a carrier attribute that should point to the MetaMixin item queried. Metadata propagation makes this a bit tricky, but we should at least try; for setMeta and addMeta, the top-level entry functions manipulate the carrier attributes for this purpose.

To yield useful error messages, leave carrier at its default None only when you really have no idea what the meta will end up on.

class gavo.base.meta.MetaItem(val)

Bases: object

is a collection of homogeneous MetaValues.

All MetaValues within a MetaItem have the same key.

A MetaItem contains a list of children MetaValues; it is usually constructed with just one MetaValue, though. Use the alternative constructor formSequence if you already have a sequence of MetaValues. Or, better, use the ensureMetaItem utility function.

The last added MetaValue is the “active” one that will be changed on _addMeta calls.

addChild(metaValue=None, key=None)

addContent(item)

copy()

returns a deep copy of self.

classmethod fromAtoms(atoms, metaValue)

classmethod fromSequence(seq)

getContent(targetFormat='text', macroPackage=None, acceptSequence=False)

getMeta(key, *args, **kwargs)

isEmpty()

serializeToXMLStan()

traverse(builder)

class gavo.base.meta.MetaMixin

Bases: StructCallbacks

is a mixin for entities carrying meta information.

The meta mixin provides the following methods:

• setMetaParent(m) – sets the name of the meta container enclosing the

  current one. m has to have the Meta mixin as well.

• getMeta(key, propagate=True, raiseOnFail=False, default=None) – returns

  meta information for key or default.

• addMeta(key, metaItem, moreAttrs) – adds a piece of meta information here. Key may be a compound, metaItem may be a text, in which case it will be turned into a proper MetaValue taking key and moreAttrs into account.

• setMeta(key, metaItem) – like addMeta, only previous value(s) are

  overwritten

• delMeta(key) – removes a meta value(s) for key.

When querying meta information, by default all parents are queried as well (propagate=True).

Metadata is not copied when the embedding object is copied. That, frankly, has not been a good design decision, and there should probably be a way to pass copypable=True to the mixin’s attribute definition.

addMeta(key, metaValue, **moreAttrs)

adds metaItem to self under key.

moreAttrs can be additional keyword arguments; these are used by the XML constructor to define formats or to pass extra items to special meta types.

For convenience, this returns the meta container.

buildRepr(key, builder, propagate=True, raiseOnFail=True)

copyMetaFrom(other)

sets a copy of other’s meta items on self.

delMeta(key)

removes a meta item from this meta container.

This will not propagate, i.e., getMeta(key) might still return something unless you give propagate=False.

It is not an error do delete an non-existing meta key.

getAllMetaPairs()

iterates over all meta items this container has.

Each item consists of key, MetaValue. Multiple MetaValues per key may be given.

This will not iterate up, i.e., in general, getMeta will succeed for more keys than what’s given here.

getMeta(key, propagate=True, raiseOnFail=False, default=None, acceptSequence=False)

getMetaKeys()

getMetaParent()

hasMeta(atom, propagate=False)

returns true if this meta carrier has an atom metadatum.

This will not parse atom, so you cannot (yet) pass in hierarchical meta names.

With propagate, it will also ask the meta parents.

In user code, it is generally preferable to just call getMeta with raiseOnFail. This is mainly a shortcut for internal code.

isEmpty()

iterMeta(key, propagate=False)

yields all MetaValues for key.

This will traverse down all branches necessary to yield, in sequence, all MetaValues reachable by key.

If propagation is enabled, the first meta carrier that has at least one item exhausts the iteration.

(this currently doesn’t return an iterator but a sequence; that’s an implementation detail, though. You should only assume whatever comes back is iterable)

keys()

makeOriginal(key)

marks the meta item key, if existing, as original.

This is for when a meta container has copied metadata. DaCHS’ default behaviour is that a subsequent addMeta will clear the copied content. Call this method for the key in question to enable adding to copied metadata.

setMeta(key, value, **moreAttrs)

replaces any previous meta content of key (on this container) with value.

setMetaParent(parent)

setParent(parent)

traverse(builder)

class gavo.base.meta.MetaParser(container, nextParser)

Bases: Parser

A structure parser that kicks in when meta information is parsed from XML.

This parser can also handle the notation with an attribute-less meta tag and lf-separated colon-pairs as content.

addMeta(key, content='', **kwargs)

end_(ctx, name, value)

setMeta(key, content='', **kwargs)

start_(ctx, name, value)

value_(ctx, name, value)

exception gavo.base.meta.MetaSyntaxError(msg, carrier=None, hint=None, key=None)

Bases: MetaError

is raised when a meta key is invalid.

class gavo.base.meta.MetaValue(content='', format='plain')

Bases: MetaMixin

is a piece of meta information about a resource.

The content is always a string.

The text content may be in different formats, notably

• literal

• rst (restructured text)

• plain (the default)

• raw (for embedded HTML, mainly – only use this if you know

  the item will only be embedded into HTML templates).

consecutiveWSPat = re.compile('\\s\\s+')

copy()

returns a deep copy of self.

getContent(targetFormat='text', macroPackage=None)

getExpandedContent(macroPackage)

knownFormats = {'literal', 'plain', 'raw', 'rst'}

paragraphPat = re.compile('\n\\s*\n')

plainWrapper = <gavo.base.meta._NoHyphenWrapper object>

exception gavo.base.meta.MetaValueError(msg, carrier=None, hint=None, key=None)

Bases: MetaError

is raised when a meta value is inapproriate for the key given.

class gavo.base.meta.ModelBasedBuilder(constructors, format='text')

Bases: object

is a meta builder that can create stan-like structures from meta information

It is constructed with with a tuple-tree of keys and DOM constructors; these must work like stan elements, which is, e.g., also true for our registrymodel elements.

Each node in the tree can be one of:

• a meta key and a callable,

• this, and a sequence of child nodes

• this, and a dictionary mapping argument names for the callable

  to meta keys of the node; the arguments extracted in this way are passed in a single dictionary localattrs.

The callable can also be None, which causes the corresponding items to be inlined into the parent (this is for flattening nested meta structures).

The meta key can also be None, which causes the factory to be called exactly once (this is for nesting flat meta structures).

build(metaContainer, macroPackage=None)

exception gavo.base.meta.NoMetaKey(msg, carrier=None, hint=None, key=None)

Bases: MetaError

is raised when a meta key does not exist (and raiseOnFail is True).

class gavo.base.meta.TextBuilder

Bases: MetaBuilder

is a MetaBuilder that recovers a tuple sequence of the meta items in text representation.

enterValue(value)

getResult()

gavo.base.meta.doMetaOverride(container, metaKey, metaValue, extraArgs={})

creates the representation of metaKey/metaValue in container.

This is there for magic meta types, mostly from typedmeta. It inspects META_CLASSES_FOR_KEYS, which in turn is fed by the forKeys class decorator. There’s some hard-coded junk in here that doesn’t quite fit anywhere else yet (but that should be rationalised if more hackery of this kind becomes necessary).

If metaKey does not need any special action, this returns None.

This gets called from one central point in MetaMixin.addMeta, and essentially all magic involved should be concentrated here.

gavo.base.meta.ensureMetaItem(thing, moreAttrs={})

ensures that thing is a MetaItem.

If it is not, thing is turned into a sequence of MetaValues, which is then packed into a MetaItem.

Essentially, if thing is not a MetaValue, it is made into one with moreAttrs. If thing is a list, this recipe is used for all its items.

gavo.base.meta.ensureMetaValue(val, moreAttrs={})

makes a MetaValue out of val and a dict moreAttrs unless val already is a MetaValue.

gavo.base.meta.forKeys(*keys)

gavo.base.meta.getMetaText(ob, key, default=None, **kwargs)

returns the meta item key form ob in text form if present, default otherwise.

You can pass getMeta keyword arguments (except default).

Additionally, there’s acceptSequence; if set to true, this will return the first item of a sequence-valued meta item rather than raising an error.

ob will be used as a macro package if it has an expand method; to use something else as the macro package, pass a macroPackage keyword argument.

gavo.base.meta.getPrimary(metaKey)

gavo.base.meta.makeFallbackMeta(reload=False)

fills CONFIG_META with items from $configDir/defaultmeta.txt.

This is called from the module __init__ once typed meta from base is in (which means that any typed meta defined in other places won’t become magic in fallback; regrettably, we can’t hold off loading fallback meta much longer).

gavo.base.meta.metaRstToHtml(inputString)

gavo.base.meta.parseKey(metaKey)

gavo.base.meta.parseMetaStream(metaContainer, metaStream, clearItems=False)

parser meta key/value pairs from metaStream and adds them to metaContainer.

If clearItems is true, for each key found in the metaStream there’s first a delMeta for that key executed. This is for re-parsing meta streams.

The stream format is:

• continuation lines with backslashes, where any sequence of backslash, (cr?) lf (blank or tab)* is replaced by nothing.

• comments are lines like (ws*)# anything

• empty lines are no-ops

• all other lines are (ws*)<key>(ws*):(ws*)value(ws*)

• if a key starts with !, any meta info for the key is cleared before setting

• a value may be prefixed with rst:, literal:, or raw: to set the format of the meta literal (default is plain).

gavo.base.meta.printMetaTree(metaContainer, curKey='')

gavo.base.meta.stanFactory(tag, **kwargs)

returns a factory for ModelBasedBuilder built from a stan-like “tag”.