Command Line

For experiments, we provide a simple command line tool. Try:

gavostc help

to see what operations it exposes. Here are some examples:

$ gavostc help
Usage: gavostc [options] <command> {<command-args}
  Use command 'help' to see commands available.

Options:
  -h, --help            show this help message and exit
  -e, --dump-exception  Dump exceptions.

Commands include:
conform <srcSTCS>. <dstSTCS>  -- prints srcSTCS in the system of dstSTCS.
help -- outputs help to stdout.
parseUtypes --- reads the output of utypes and prints quoted STC for it.
parseX <srcFile> -- read STC-X from srcFile and output it as STC-S, - for stdin
resprof <srcSTCS> -- make a resource profile for srcSTCS.
utypes <QSTCS> -- prints the utypes for the quoted STC string <QSTCS>.
$ gavostc resprof "Polygon ICRS 20 20 21 19 18 17" | xmlstarlet fo
<?xml version="1.0"?>
<STCResourceProfile
  xmlns="http://www.ivoa.net/xml/STC/stc-v1.30.xsd"
  xmlns:xlink="http://www.w3.org/1999/xlink"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.ivoa.net/xml/STC/stc-v1.30.xsd http://vo.ari.uni-heidelberg.de/docs/schemata/stc-v1.30.xsd http://www.w3.org/1999/xlink http://vo.ari.uni-heidelberg.de/docs/schemata/xlink.xsd">
  <AstroCoordSystem id="thgloml">
    <SpaceFrame id="thdbgwl">
      <ICRS/>
      <UNKNOWNRefPos/>
      <SPHERICAL coord_naxes="2"/>
    </SpaceFrame>
  </AstroCoordSystem>
  <AstroCoordArea coord_system_id="thgloml">
    <Polygon frame_id="thdbgwl" unit="deg">
      <Vertex>
        <Position>
          <C1>20.0</C1>
          <C2>20.0</C2>
        </Position>
      </Vertex>
      <Vertex>
        <Position>
          <C1>21.0</C1>
          <C2>19.0</C2>
        </Position>
      </Vertex>
      <Vertex>
        <Position>
          <C1>18.0</C1>
          <C2>17.0</C2>
        </Position>
      </Vertex>
    </Polygon>
  </AstroCoordArea>
</STCResourceProfile>
$ gavostc resprof "Circle FK5 -10 340 3" | gavostc parseX -
Circle FK5 -10.0 340.0 3.0
$ gavostc conform "Position GALACTIC 3 4 VelocityInterval Velocity 0.01 -0.002 unit deg/cy" "Position FK5"
Position FK5 264.371974024 -24.2795040403 VelocityInterval Velocity 0.00768930497899 0.00737459624525 unit deg/cy
$ gavostc utypes 'Redshift TOPOCENTER VELOCITY "z" Error "e_z" PixSize "p_z"'
AstroCoordSystem.RedshiftFrame.value_type                    = VELOCITY
AstroCoordSystem.RedshiftFrame.DopplerDefinition             = OPTICAL
AstroCoordSystem.RedshiftFrame.ReferencePosition             = TOPOCENTER
AstroCoords.Redshift.Error                                   -> e_z
AstroCoords.Redshift.Value                                   -> z
AstroCoords.Redshift.PixSize                                 -> p_z
$ gavostc utypes 'Redshift TOPOCENTER VELOCITY "z" Error "e_z" PixSize "p_z"'\
| gavostc parseUtypes
Redshift TOPOCENTER VELOCITY "z" Error "e_z" PixSize "p_z"

Programmatic

The library is written in python, and thus currently can only be operated from python programs. It should not be too hard to embed it into C or even Java programs. If you have such needs, contact the author.

See API for details.

from gavo import stc

The Data Model

The STC library turns all input into a tree called AST ("Abstract Syntax Tree", since it abstracts away the details for parsing from whatever serialisation you employ).

The ASTs are following the STC data model quite closely. However, it turned out that -- even with the changes already in place -- this is quite inconvenient to work with, so we will probably change it after we've gathered some experience. It is quite likely that we will enforce a much stricter separation between data and metadata, i.e., unit, error and such will go from the positions to what is now the frame object.

Thus, we don't document the data model fully yet. The gory details are in dm.py. Meanwhile, we will try to maintain the following properties:

• All objects in ASTs are considered immutable, i.e., nobody is supposed to change them once they are constructed.
• An AST object has attributes time, place, freq, redshift, velocity refererring to an objects describing quantities or None if not given. These are called "positions" in the following.
• An AST object has attributes timeAs, areas, freqAs, redshiftAs, velocityAs containing sequences of intervals or geometries of the respective quantities. These sequences are empty if nothing is specified. They are called areas in the following.
• Both positions and areas have a frame attribute giving the frame (for spatial coordinates, these have flavor, nDim, refFrame, equinox, and refPos attributes, quite like in STC).
• Positions have a values attribute containing either a python float or a tuple of floats (for spatial and velocity coordinates). For time coordinates, a datetime.datetime object is used instead of a float
• Positions have a unit attribute. We will keep this even if all other metadata move to the frame object. The unit attribute follows the coordinate values, i.e., they are tuple-valued when the values are tuples. For velocities and redshifts, there is a velTimeUnit as well.
• ASTs have a cooSystem attribute with, in turn, spaceFrame, timeFrame, spectralFrame, and redshiftFrame attributes.
• NULL is consistently represented as None, except when the values would be sequences, in which case NULL is an empty tuple.

Parsing STC-X

To parse an STC-X document, use stc.parseSTCX(literal) -> AST. Thus, you pass in a string containing STC-X and receive a AST structure.

Since STC documents should in general be rather small, there should be no necessity for a streaming API. If you want to read directly from a file, you could use something like:

def parseFromFile(fName):
  f = open(fName)
  stcxLiteral = f.read()
  f.close()
  return stc.parseSTCX(stcxLiteral)

The return value is a sequence of pairs of (tagName, ast), where tagName is the namespace qualified name of the root element of the STC element. The tagName is present since multiple STC trees may be present in one STC-X document. The qualification is in standard W3C form, i.e., {<namespace URI>}<element name>. If you do not care about versioning (and you should not need to with this library), you could find a specific element using a construct like:

def getSTCElement(literal, elementName):
  for rootName, ast in stc.parseSTCX(literal):
    if rootName.endswith('}'+elementName):
      return ast
getSTCElement(open("M81.xml").read(), "ObservationLocation")

Note that the STC library does not contain a validating parser. Invalid STC-X documents will at best give you rather incomprehensible error messages, at worst an AST that has little to do with what was in the document. If you are not sure whether the STC-X you receive is valid, run a schema validator before parsing.

We currently understand a subset of STC-X that matches the expressiveness of STC-S. Most STC-X features that cannot be mapped in STC-X are silently ignored.

Generating STC-X

To generate STC-X, use the stc.getSTCX(ast, rootElmement) -> str function. Since there are quite a few root elements possible, you have to explicitely pass one. You can find root elements in stc.STC. It is probably a good idea to only use ObservatoryLocation, ObservationLocation, and STCResourceProfile right now. Ask the authors if you need something else.

There is the shortcut stc.getSTCXProfile(ast) -> str that is equivalent to stc.getSTCX(ast, stc.STC.STCResourceProfile).

Parsing STC-S

To parse an STC-S string into an AST, use stc.parseSTCS(str) -> ast. The most common exception this may raise is stc.STCSParseError, though others are conceivable.

Generating STC-S

To turn an AST into STC-S, use stc.getSTCS(ast) -> str. If you pass in ASTs that use features not supported by STC-S, you should get an STCNotImplementedError or an STCValueError.

Generating Utypes

For embedding STC into VOTables, utypes are used. To turn an AST object into utypes, use stc.getUtypes(ast) -> dict, dict. The function returns a pair of dictionaries:

• the first dictionary, the "system dict", maps utypes to values. All utypes belong to AstroCoordSystem and into this group.
• the second dictionary, the "columns dict", maps values to utypes.

Of course, the columns dict doesn't make much sense with ASTs actually containing values. To sensibly use it it a way useful for VOTables, you can define your columns' STC using "quoted STC-S". In this format, you have identifiers in double quotes instead of normal STC-S values. Despite the double quotes, only python-compatible identifiers are allowed, i.e., these are not quoted identifiers in the SQL sense. The stc.parseQSTCS(str) -> ast function parses such strings.

Consider:

In [5]:from gavo import stc
In [6]:stc.getUtypes(stc.parseQSTCS(
   ...:'Position ICRS "ra" "dec" Error "e_p" "e_p"'))
Out[6]:
({'AstroCoordSystem.SpaceFrame.CoordFlavor': 'SPHERICAL',
  'AstroCoordSystem.SpaceFrame.CoordRefFrame': 'ICRS',
  'AstroCoordSystem.SpaceFrame.ReferencePosition': 'UNKNOWNRefPos'},
 {'dec': 'AstroCoords.Position2D.Value2.C2',
  'e_p': 'AstroCoords.Position2D.Error2Radius',
  'ra': 'AstroCoords.Position2D.Value2.C1'})

Note that there is no silly "namespace prefix" here. Nobody really knows what those prefixes really mean with utypes. When sticking these things into VOTables, you will currently need to stick an "stc:" in front of those.

Parsing Utypes

When parsing a VOTable, you can gather the utypes encountered to dictionaries as returned by getUtypes. You can then pass these to parseFromUtypes(sysDict, colDict) -> ast. The function does not expect any namespace prefixes on the utypes.

Conforming

You can force two ASTs to be expressed in the same frames, which we call "conforming". As mentioned above, currently only reference frames and equinoxes are conformed right now, i.e., the conversion from Galactic to FK5 1980.0 coordinates should work correctly. Reference positions are ignored, i.e. conforming ICRS TOPOCENTER to ICRS BARYCENTER will not change values.

To convert coordinates in ast1 to the frame defined by ast2, use the stc.conformTo(ast1, ast2) -> ast function. This could look like this:

>>> p = stc.parseSTCS("Circle ICRS 12 12 1")
>>> stc.conformTo(p, stc.parseSTCS("Position GALACTIC"))
>>> stc.conformTo(p, stc.parseSTCS("Position GALACTIC")).areas[0].center
(121.59990883115164, -50.862855782323962)

Conforming also works for units:

>>> stc.conformTo(p, stc.parseSTCS("Position GALACTIC unit rad")).areas[0].center
(2.1223187792285256, -0.8877243003685894)

Transformation

For simple transformations, you can ask DaCHS to give you a function just turning simple positions into positions. For instance,

from gavo import stc

toICRS = stc.getSimple2Converter(
        stc.parseSTCS("Position FK4 B1900.0"),
        stc.parseSTCS("Position ICRS"))
print(toICRS(30, 40))

shows how to build turn positions given in the B1900 equinox (don't sweat the reference system for data that old) to ICRS.

Equivalence

For some applications it is necessary to decide if two STC specifications are equivalent. Python's built-in equivalence operator requires all values in two ASTs to be identical except of the values of id attributes.

Frequently, you want to be more lenient:

• you might decide that unspecified values match anything
• you may ignore certain keys entirely (e.g., the reference position when you're doing extragalactic work or when a parallax error doesn't matter)
• you may want to view certain combinatinons as equivalent (e.g., ICRS and J2000 are quite close)

To support this, the STC library lets you define EquivalencePolicy objects. There is a default equivalence policy ignoring the reference position, defining ICRS and FK5 J2000 as equivalent, and matching Nones to anything. This default policy is available as stc.defaultPolicy. It has a single method, match(sys1, sys2) -> boolean with the obvious semantics. Note, however, that you pass in systems, i.e., ast.cooSystem rather than ASTs themselves.

You can define your own equivalence policies. Tell us if you want that and we'll document it. In the mean time, check stc/eq.py.

Extending STC-S

Here's an example for an extension to STC-S: Let's handle the planetary ephemeris element.

Checking the schema, you'll see only two literals are allowed for the ephemeris: JPL-DE200 and JPL-DE405. So, in stcs._getSTCSGrammar, near the definition of refpos, add:

plEphemeris = Keyword("JPL-DE200") | Keyword("JPL-DE405")

The plan is to allow the optional specification of the ephemeris used after refpos. Now grep for the occurrences of refpos and notice that there are quite a number of them. So, rather than fixing all those rules, we change the refpos rule from:

refpos = (Regex(_reFromKeys(stcRefPositions)))("refpos")

to:

    refpos = ((Regex(_reFromKeys(stcRefPositions)))("refpos")
+ Optional( plEphemeris("plEphemeris") ))

We can test this. In stcstest.STCSSpaceParsesTest, let's add the sample:

("position", "Position ICRS TOPOCENTER JPL-DE200"),

Now, the refpos nodes are handled in the _makeRefpos function, looking like this:

def _makeRefpos(node):
  refposName = node.get("refpos")
  if refposName=="UNKNOWNRefPos":
    refposName = None
  return dm.RefPos(standardOrigin=refposName)

The node passed in here is a pyparsing node. Since in our data model, None is always null/ignored, we can just take the planetary ephemeris if it's present, and the system will do the right thing if it's not there:

def _makeRefpos(node):
  refposName = node.get("refpos")
  if refposName=="UNKNOWNRefPos":
    refposName = None
  return dm.RefPos(standardOrigin=refposName,
    planetaryEphemeris=node.get("plEphemeris"))

Let's test this; testing STC-S to AST parsing takes place in stctest.py, so let's add a method to CoordSysTest:

def testPlanetaryEphemeris(self):
        ast = stcsast.parseSTCS("Time TT TOPOCENTER JPL-DE200")
        self.assertEqual(ast.astroSystem.timeFrame.refPos.planetaryEphemeris,
                "JPL-DE200")

Thus, we can parse the ephemeris spec from STC-S. To generate it, two things need to be done: The DM item must be transformed into the CST the STC-S is built from, and the part of the CST must be flattened out. Both things happen in stcsgen.py. The CST is just nested dictionaries. Refpos handline happens in refPosToCST, so replace:

def refPosToCST(node):
  return {"refpos": node.standardOrigin}

with:

def refPosToCST(node):
  return {
    "refpos": node.standardOrigin,
    "planetaryEphemeris": node.planetaryEphemeris,}

To flatten that out to the finished string, the flatteners need to be told that you want that key noticed. Grepping for repos shows that it's used in several places. So, let's define a "common flattener", which is a function taking a value and the CST node (i.e., a dictionary) the value was taken from and returns a string ready for inclusion into the STC-S. The flattener here would look like this:

def _flattenRefPos(val, node):
  return _joinWithNull([node["refpos"], node["planetaryEphemeris"]])

The _joinWithNull call makes sure that empty specifications do not show up the in result.

This "global" flattener is now entered into _commonFlatteners, a dictionary mapping specific CST keys to flatten functions:

_commonFlatteners = {
...
        "refpos": _flattenRefPos,
}

The most convenient way to test this is to define a round-trip test. These again reside stcstest. Use BaseGenerationTest and add a sample pair like this:

("Redshift BARYCENTER JPL-DE405 3.5",
        "Redshift BARYCENTER JPL-DE405 3.5")

With this, you should be done.