What is disclosed are a novel system and method for inferring types of database queries. In one embodiment a program and associated database schema that includes a type hierarchy is accessed. The program includes query operations to a database that contains relations described by a database schema. types are inferred from definitions in the program by replacing each database relationship in the program by the types in the database schema. A new program is generated with the types that have been inferred with the new program only accessing unary relations in the database. In another embodiment, testing of each of the types that have been inferred is performed for type emptiness. In response to type emptiness being found for a type that have been inferred, a variety of different operations are performing including removing the type, providing a notification regarding the emptiness found for the type, and more.

Patent
   RE46380
Priority
Jul 15 2010
Filed
Nov 25 2015
Issued
Apr 25 2017
Expiry
Jul 15 2031
Assg.orig
Entity
Large
3
7
currently ok
1. A computer-implemented method comprising:
accessing a query program with, the query program containing one or more queries to a database that contains relations described by at least one a database schema, wherein the database schema specifies a column type for every column of every extensional predicate that occurs in the query program;
receiving the database schema and at least one an entity type hierarchy for the database; and
inferring a first type program from definitions in the query program by replacing each use of a database relation an extensional predicate in the query program by its type in the conjunction of the column types of the columns of the extensional predicate, the column types being provided by the database schema, and the wherein each column type is at least portion of a respective second type program,
wherein each of the first type program and the second type program is a derived program using type symbols without other extensional relation symbols, and each of the first type program and the second type program do not contain negation,
wherein the first type program and the second type program uses programs use only extensionals that are monadic extensions, and the first type program and the second type programs do not contain negation;
testing portion portions of the first type program that has been inferred for type emptiness and type inclusion; and
finding type emptiness or type inclusion for a first portion of the first type program, wherein the first portion is also a type program; and
providing at least one of error information and or optimization information regarding the type emptiness and or type inclusion being found for the type first portion of the first type program.
19. A non-transitory computer program product, the computer program product comprising instructions for to cause a computer system to perform the operations of:
accessing a query program with, the query program containing one or more query operations to a database that contains relations described by at least one a database schema, wherein the database schema specifies a column type for every column of every extensional predicate that occurs in the query program;
receiving the database schema and at least one an entity type hierarchy for the database; and
inferring a first type program from definitions in the query program by replacing each use of a database relation an extensional predicate in the query program by its type in the conjunction of the column types of the columns of the extensional predicate, the column types being provided by the database schema, and the wherein each column type is at least portion of a respective second type program,
wherein each of the first type program and the second type program is a derived program using type symbols without other extensional relation symbols, and each of the first type program and the second type program do not contain negation,
wherein the first type program and the second type program uses programs use only extensionals that are monadic extensions, and the first type program and the second type programs do not contain negation;
testing portion portions of the first type program that has been inferred for type emptiness and type inclusion; and
finding type emptiness or type inclusion for a first portion of the first type program, wherein the first portion is also a type program; and
providing at least one of error information and optimization information regarding the type emptiness and or type inclusion being found for the type first portion of the first type program.
16. A system comprising:
a computer system comprising one or more computers each having a memory; and a processor communicatively coupled to the memory; and
a type inferencer communicatively coupled to the memory and the processor computer program loaded in the computer system, wherein the type inferencer is adapted to cause the computer system perform the operations of:
accessing a query program with, the query program containing one or more query operations to a database that contains relations described by at least one a database schema, wherein the database schema specifies a column type for every column of every extensional predicate that occurs in the query program;
receiving the database schema and at least one an entity type hierarchy for the database; and
inferring a first type program from definitions in the query program by replacing each use of a database relation an extensional predicate in the query program by its type in the conjunction of the column types of the columns of the extensional predicate, the column types being provided by the database schema, and the wherein each column type is at least portion of a respective second type program,
wherein each of the first type program and the second type program is a derived program using type symbols without other extensional relation symbols, and each of the first type program and the second type program do not contain negation,
wherein the first type program and the second type program uses programs use only extensionals that are monadic extensions, and the first type program and the second type programs do not contain negation;
testing portion portions of the first type program that has been inferred for type emptiness and type inclusion; and
finding type emptiness or type inclusion for a first portion of the first type program, wherein the first portion is also a type program; and
providing at least one of error information and optimization information regarding the type emptiness and or type inclusion being found for the type first portion of the first type program.
2. The computer-implemented method of claim 1, further comprising:
in response to type emptiness being found for a the first portion of the first type program that has been inferred during determining of the testing portion of portions of the first type program, performing at least one of:
removing the a type test from the query program a first portion of the query program that corresponds to the first portion of the first type program;
providing a notification regarding the emptiness found for the first portion of the first type program; or
providing notification on combining queries in the query program by conjunction without creating empty parts in a combined query;
in response to an empty part of a query being a conjunction, finding a smallest set of query parts that has a conjunction that is empty;
in response to a search for a smallest empty part of a database query that traverses all parts of the database query, pushing a conjunction of an approximation of the smallest empty part and a context on top of a stack of approximations of all contexts where the empty part is being used; and
eliminating empty query parts to achieve virtual method resolution in an object-oriented query language.
3. The computer-implemented method of claim 1, further comprising:
in response to type inclusion being found for a second portion of the first type program that has been inferred during determining of the testing portion of portions of the first type program, performing at least one of:
removing the a type test from the query program a second portion of the query program that corresponds to the second portion of the first type program; and or
providing a notification regarding the type inclusion being found for the second portion of the first type program.
4. The computer-implemented method of claim 1, further comprising wherein:
inferring the first type program includes inferring an inferred type for an intensional predicate in the query program that has a declared type in the query program; and
the method further comprises:
testing the type program that has been inferred for whether a database query is contained in a given portion of the type program; and in response to the database query not being contained in the given portion, performing at least one of:
removing a type test; and
determining that the inferred type does not contain the declared type; and
providing a notification regarding that the database query declared type is not being contained in the given portion inferred type.
5. The computer-implemented method of claim 1, wherein portions of the first type program that has been inferred are represented by a set respective sets of type tuple constraints (TTCs), wherein each of the TTCs includes:
a tuple of type propositions;
an equivalence relation between tuple components, each tuple component being a type proposition; and
a set of inhabitation constraints.
6. The computer-implemented method of claim 5, further comprising:
checking inclusion of a third portion of the first type program that has been inferred represented by a first set of TTCs into another, fourth portion of the first type program represented by a second set of TTCs by:
computing a set of prime implicants of the second set; and
checking each TTC in the first set and finding determining whether a larger TTC exists in the set of prime implicants of the second set of TTCs; and
determining that the third portion is included in the fourth portion if for each TTC in the first set a larger TTC exists in the set of prime implicants of the second set of TTCs.
7. The computer-implemented method of claim 6, further comprising:
computing the set of prime implicants of the second set of TTCs by saturating the second set by exhaustively applying consensus operations to ensure all relevant TTCs are included.
8. The computer-implemented method of claim 7, wherein each of the consensus operations are performed by includes generating a new TTC from two existing TTCs by
receiving two TTCs as operands two TTCs, the tuple of type propositions of each TTC having the same number of type propositions;
receiving a set of indices and, each index in the set identifying a type proposition in a tuple of type propositions by the position of the type proposition in the tuple;
equating in each operand TTC all columns whose indices type propositions at index positions that occur in the set of indices, and preserving all other equalities and inhabitation constraints of the operands each TTC operand, and then for each index position taking disjunctions over columns type propositions indexed in the set of indices and conjunctions over all other columns type propositions as the type propositions of the tuple of type propositions of the new TTC; and
taking generating a union of two or more of the equivalence relations in of the operands operand TTCs as the equivalence relation between tuple components of the new TTC; and
generating a pointwise disjunction of the inhabitation constraints of the operands operand TTCs as the inhabitation constraints of the new TTC.
9. The computer-implemented method of claim 8, further comprising:
reducing a number of consensus operations that need to be performed during saturation by omitting consensus operations where a resulting TTC will be covered by TTCs already present in the set, wherein a TTC τ is covered by a TTC τ′ if τ<:τ′.
10. The computer-implemented method of claim 8, further comprising:
receiving a logical formula that represents a type hierarchy, and representing a component type proposition of a TTC as a binary decision diagram (BDDs BDD), and choosing a BDD variable order by assigning neighboring indices of the set of indices to type symbols that appear in a same conjunct of the logical formula that represents the type hierarchy.
11. The computer-implemented method of claim 1, wherein further comprising:
using the first type program as an approximation is used to find erroneous parts of a database query in the query program that will return an empty set of results, regardless of contents stored in the database.
12. The computer-implemented method of claim 2, wherein the providing notification on combining queries in the query program by conjunction without creating empty parts in a combined query includes depicting compatible types with similar pictures in a user interface.
13. The computer-implemented method of claim 1, wherein the one or more queries to the database contain calls to other query procedures, and the method further comprises:
using the first type program as an approximation is used to optimize these called the query procedures, by and eliminating query parts from the query procedures that will return an empty set of results in a context where they are called, regardless of any contents of the database.
14. The computer-implemented method of claim 13, wherein the context of a procedure in a database query is computed by comprising:
computing the context of each query procedure called by a query program by
traversing a call graph of that query program, and
keeping a stack of approximations of all contexts where the query procedure is being used, and
when entering a procedure in the call graph, pushing a conjunction of an approximation of a body of that procedure and a top of the stack onto the stack as a new context.
15. The computer-implemented method of claim 1, wherein further comprising:
using the first type program as an approximation is used to optimize queries, in the query program by eliminating from the query program query parts that test whether a value is included in a portion of the first type program has a particular declared type in the query program, and the approximation first type program indicates at least one of:
the value will be included in this portion of the first type program have the particular declared type regardless of contents stored in the database; and
the value will not be included in the portion of the first type program have the particular declared type regardless of contents stored in the database.
17. The system of claim 16, wherein the type inferencer is further adapted to perform the operations of:
in response to type emptiness being found for a the first portion of the first type program that has been inferred during determining of the testing portion of portions of the first type program, performing at least one of:
removing a type test from the query program a first portion of the query program that corresponds to the first portion of the first type program;
providing a notification regarding the emptiness found for the first portion of the first type program; or
providing notification on combining queries in the query program by conjunction without creating empty parts in a combined query;
in response to an empty part of a query being a conjunction, finding a smallest set of query parts that has a conjunction that is empty;
in response to a search for a smallest empty part of a database query that traverses all parts of the database query, pushing a conjunction of an approximation of the smallest empty part and a context on top of a stack of approximations of all contexts where the empty part is being used; and
eliminating empty query parts to achieve virtual method resolution in an object-oriented query language.
18. The system of claim 16, wherein the type inferencer is further adapted to perform the operations of:
in response to type inclusion being found for a second portion of the first type program that has been inferred during determining of the testing portion of portions of the first type program, performing at least one of:
removing a type test from the query program a second portion of the query program that corresponds to the second portion of the first type program; and or
providing a notification regarding the type inclusion being found for the second portion of the first type program.
20. The non-transitory computer program product of claim 19, further comprising instructions to cause the computer system to perform the operations of:
in response to type emptiness being found for a the first portion of the first type program that has been inferred during determining of the testing portion of portions of the first type program, performing at least one of:
removing a type test from the query program a first portion of the query program that corresponds to the first portion of the first type program;
providing a notification regarding the emptiness found for the first portion of the first type program; or
providing notification on combining queries in the query program by conjunction without creating empty parts in a combined query;
in response to an empty part of a query being a conjunction, finding a smallest set of query parts that has a conjunction that is itself empty;
in response to a search for a smallest empty part of a database query that traverses all parts of the database query, pushing a conjunction of an approximation of that part and a context on top of a stack; and
eliminating empty query parts to achieve virtual method resolution in an object-oriented query language.
0. 21. The computer-implemented method of claim 1, wherein:
each of the column types is a monadic type found in a set of type symbols.
0. 22. The computer-implemented method of claim 2, further comprising:
in response to finding that the first portion of the query program is a conjunction, finding a smallest set of query parts in the first portion of the query program that has a conjunction that is empty, and providing a notification regarding the emptiness found for the smallest set of query parts that has a conjunction that is empty.


states that the first column of the extensional relation salary will contain values of type employee, and the second one will contain floating point numbers. A database schema provides this column-wise typing information for every extensional relation.

Given the schema and the hierarchy, type inference should infer the following implications from the definitions in the program:
bonus(x, y)→employee(x) custom character float(y)
factor(x, y)→employee(x) custom character float(y)
query(x, y)→senior(x) custom character manager(x) custom character float(y)

Note how these depend on facts stated in the type hierarchy: for example, in inferring the type of factor, we need to know that employee is the union of junior and senior.

Type inference will catch errors: when a term is inferred to have the type ⊥, we know that it denotes the empty relation, independent of the contents of the database. Also, the programmer may wish to declare types for intensional relations in her program, and then it needs to be checked that the inferred type implies the declared type.

The benefits of having such type inference for queries is not confined to merely checking for errors. Precise types are also useful in query optimisation. In particular, the above query can be optimised to:
query(x, y)←manager(x) custom character senior(x) custom character salary(x, z) custom character y=0.15xz

This rewriting relies on the fact that we are asking for a manager, and therefore the disjunct in the definition of bonus that talks about students does not apply: a student is a parttime employee, and managers cannot be parttime. Again notice how the complex type hierarchy is crucial in making that deduction. The elimination of disjuncts based on the types in the calling context is named type specialisation; type specialisation is very similar to virtual method resolution in optimising compilers for object-oriented languages. Similarly, the junior alternative in the definition of factor can be eliminated. Once type specialisation has been applied, we can eliminate a couple of superfluous type tests: for instance, custom character student(x) is implied by manager(x). That is called type erasure. The key to both type specialisation and type erasure is an efficient and complete test for type containment, which takes all the complex facts about types into account.

Overview The general problem we address is as follows. Given the class of all Datalog programs custom character, we identify a sublanguage custom character of type programs. Type inference assigns an upper bound ┌p┐ in the language of type programs to each program p. To say that ┌p┐ is an upper bound is just to say that p (plus the schema and the type hierarchy) implies ┌p┐. The idea of such ‘upper envelopes’ is due to Chaudhuri [8, 9].

In Section 2 we propose a definition of the class of type programs: any program where all extensionals called are monadic. Type programs can, on the other hand, contain (possibly recursive) definitions of intensional predicates of arbitrary arity. We then define a type inference procedure ┌p┐, and prove that its definition is sound in the sense that ┌p┐ is truly an upper bound of p. Furthermore, it is shown that for negation-free programs p, the inferred type ┌p┐ is also optimal: ┌p┐ is the smallest type program that is an upper bound of p.

For negation-free programs, the definition of ┌p┐ is in terms of the semantics of p, so that the syntactic presentation of a query does not affect the inferred type. It follows that the application of logical equivalences by the programmer or by the query optimiser does not affect the result of type inference.

The restriction that programs be negation-free can be relaxed to allow negation to occur in front of sub-programs that already have the shape of a type program. Not much more can be hoped for, since sound and optimal type inference for programs with arbitrary negations is not decidable.

To also do type checking and apply type optimisations, we need an effective way of representing and manipulating type programs. To this end, we identify a normal form for a large and natural class of type programs in Section 3. The class consists of those type programs where the only negations are on monadic extensionals (i.e., entity types). There is a simple syntactic containment check on this representation, inspired by our earlier work reported in [13].

That simple containment check is sound but not complete. A geometric analysis of the incompleteness in Section 4 suggests a generalisation of the celebrated procedure of Quine [22] for computing prime implicants. We present that generalised procedure, and show that the combination of the simple containment check plus the computation of prime implicants yields a complete algorithm for testing containment of type programs.

The algorithm we present is very different from the well-known approach of propositionalisation, which could also be used to decide containment. The merit of our novel algorithm is that it allows an implementation that is efficient in common cases. We discuss that implementation in Section 5. We furthermore present experiments with an industrial database system that confirm our claims of efficiency on many useful queries.

Finally, we discuss related work in Section 6, and conclude in Section 8.

In summary, the original contributions of this paper include:

2. Type Programs

It is customary to write Datalog programs as a sequence of rules as seen in FIG. 1, but this syntax is too imprecise for our purposes. In the remainder of the paper, all Datalog programs will be written as formulae of Stratified Least Fixpoint Logic (SLFP) over signatures without function symbols.

This logic, presented for example in [10], extends first order logic (including equality) with fixpoint definitions of the form [r(x1, . . . , xn)≡φ] ψ. Here, r is an n-ary relation symbol, the xi are pairwise different element variables, and φ and ψ are formulae. Intuitively, the formula defines r by the formula φ, which may contain recursive occurrences of r, and the defined relation can then be used in ψ.

The xi are considered bound in φ and free in ψ, whereas the relation r is free in φ and bound in ψ. To ensure stratification, free relation symbols are not allowed to occur under a negation, which avoids the problematic case of recursion through negation. For example, the formula
[s(x, y)≡x≐y custom character ∃z.e(x, z) custom character s(z, y)]s(x, y)  (1)
defines the relation s to be the reflexive transitive closure of e. By way of illustration (and only in this example), free occurrences of element variables and relation variables have been marked up in bold face. The formula is trivially stratified since no negations occur anywhere.

In this program, the relation symbol s is an intensional predicate since it is given a defining equation. On the other hand, relation e is an extensional, which we assume to be defined as part of a database on which the program is run. We will denote the set of intensional relation symbols as custom character and that of extensional relation symbols as custom character, and require them to be disjoint.

Model-theoretically, the role of the database is played by an interpretation custom character which assigns, to every n-ary relation symbol e ∈ custom character, a set of n-tuples of some non-empty domain. To handle free variables we further need an assignment σ that maps free element variables to domain elements and free relation variables to sets of tuples.

Just as in first order logic, we can then define a modelling judgement custom character custom characterσ φ stating that formula φ is satisfied under interpretation custom character and assignment σ. In particular, stratification ensures that formulae can be interpreted as continuous maps of their free relation variables, so that fixpoint definitions can be semantically interpreted by the least fixpoints of their defining equations.

For a formula φ with a single free element variable x, we set [[φ]] custom character:={c| custom character custom character x:=c φ}, omitting the subscript custom character where unambiguous.

Just as the semantics of the intensional predicates is determined with respect to the semantics of the extensional predicates, the types of the intensional predicates are derived with respect to the types of the extensional predicates. These are provided by a schema custom character which assigns, to every n-ary extensional predicate, an n-tuple of types. As is customary, types are themselves monadic predicates from a set custom character custom character of type symbols. For consistency, we require that the schema assigns a type symbol to itself as its type.

For the program in (1), for example, we might have two type symbols a and b, with the schema specifying that custom character(e)=(a, b) and of course custom character(a)=(a), custom character(b)=(b).

Semantically, we understand this to mean that in every interpretation of the extensional symbols conforming to the schema, the extension of a given column of an extensional symbol should be exactly the same as the extension of the type symbol assigned to it by the schema.

We can define a first order formula, likewise denoted as custom character, which expresses this constraint: For every n-ary relation symbol e ∈ custom character\custom character and every 1≦i≦n where u ∈ custom character is the type assigned to the i-th column of e, the formula custom character contains a conjunct
∀xi.(∃x1, . . . , xi−1, xi+1, . . . , xn.e(x1, . . . , xn)) custom character u(xi)  (2)

Our example schema above, for instance, would be expressed by the formula
(∀x.(∃y.e(x,y)) custom character a(x)) custom character (∀y.(∃x.e(x, y)) custom character b(y))

In the literature, the types assigned by the schema are often not required to fit exactly, but merely to provide an overapproximation of the semantics. We could achieve this by relaxing (2) to use an implication instead of a bi-implication.

Instead, we take a more general approach by allowing arbitrary additional information about the types to be expressed through a hierarchy formula custom character. Our only requirements about this formula are that it be a closed first order formula containing only type symbols (and no other relation symbols).

In particular, the hierarchy may stipulate subtyping relations; in our example, the hierarchy could be ∀x.a(x)→b(x), expressing that a is a subtype of b. Perhaps more interestingly, it can also express disjointness of types, e.g. by stating that ∀x. custom charactera(x) custom character custom characterb(x); we could then deduce that in the definition of the reflexive transitive closure e can, in fact, be iterated at most once. Many other kinds of constraints are just as easily expressed, and provide opportunities for advanced optimisations.

We now approximate programs by type programs, which are programs that only make use of type symbols (and no other extensional relation symbols), and do not contain negation. (Compare this with the hierarchy which can contain negations, but no fixpoint definitions.) To every SLFP formula φ we assign a type program ┌φ┐ by replacing negated subformulae with

which is semantically equivalent to
x≅y custom character a(x) custom character b(y)

We will see in the next section that, in fact, fixpoint definitions can always be eliminated from such type programs, yielding formulae of monadic first order logic.

As the example suggests, the types assigned to programs semantically over-approximate them in the following sense:

Theorem 1 (Soundness). For every program φ we have
custom character, custom character, φcustom character ┌φ┐

That is, every interpretation and assignment satisfying the schema, the hierarchy and φ also satisfies ┌φ┐.

Proof Sketch. We can show by induction on φ the following stronger version of the the—orem: For any interpretation custom character with custom character custom character custom character and any two assignments σ and σ′, which assign the same values to element variables, and where σ(r) σ′(r) for every r ∈ custom character, we have custom character custom character σ′ ┌φ┐ whenever custom character custom character σφ.

From this, of course, it follows that custom character, φ custom character ┌φ┐, and again custom character, custom character, φ custom character ┌φ┐ by monotonicity of entailment.

Perhaps more surprisingly, our type assignment also yields the tightest such over-approximation, in spite of our very lax restrictions on the hierarchy:

Theorem 2 (Optimality). For a negation-free program φ and a type program custom character, if we have
custom character, custom character, φ custom character custom character
then also
custom character, custom character, ┌φ┐ custom character custom character

To prove this theorem, we need a monotonicity result for the type assignment.

Lemma 3. For two negation-free programs φ and ψ, if it is the case that custom character, custom character, φ custom character ψ then also custom character, custom character, ┌φ┐ custom character ┌ψ┐.

An easy way to prove this result is to make use of cartesianisation, which is the semantic equivalent of the typing operator ┌•┐. For a relation R Dn, we define its cartesianisation cart (R)=π1(R) x . . . . xπn(R) to be the cartesian product of the columns of R. Likewise, the cartesianisation cart(custom character) of an interpretation custom character is the interpretation which assigns cart(custom character(e)) to every relation symbol e.

It is then not hard to see that
cart(custom character) custom character σ φ custom character custom character custom character σ┌φ┐  (3)
for any negation-free formula φ, interpretation custom character and assignment σ whenever custom character custom character custom character. Also, cart(custom character) custom character custom character if custom character custom character custom character, and cart(custom character) custom character custom character iff custom character custom character custom character.

From this observation, we easily get the

Proof of Lemma 3. Assume custom character, custom character, φ | ψ, and let custom character, σ be given with custom character custom character custom character, custom character custom character custom character and custom character custom character σ ┌φ┐.

By (3), we have cart(custom character) | custom character, cart(custom character) | custom character and also cart(custom character) ψ σφ, so by assumption cart(custom character) | σ ψ, but then by applying (3) again we get custom character | σ┌ψ┐.

We briefly pause to remark that Lemma 3 also shows that type inference for negation-free programs respects semantic equivalence: If φ and ψ are semantically equivalent (under custom character and custom character), then so are their types. This does, of course, not hold in the presence of negation, for given a type symbol u, we have ┌u(x)┐=u(x) yet ┌ custom charactercustom characteru(x)┐=τ.

Continuing with our development, we can now give the

Proof of Theorem 2. Assume custom character, custom character, φ custom character custom character; by Lemma 3 this means custom character, custom character, ┌φ┐ custom charactercustom character┐. But since custom character is a type program we have ┌custom character┐=custom character, which gives the result.

So far we have handled negation rather crudely, trivially approximating it by τ. In fact, we can allow negations in type programs and amend the definition of ┌•┐ for negations to read

φ = { φ if φ is a type program otherwise

All the above results, including soundness and optimality, go through with this new definition, and the optimality and monotonicity results can be generalised to hold on all programs which contain only harmless negations, i.e. negations where the negated formula is a type program.

There is little hope for an optimality result on an even larger class of programs: Using negation, equivalence checking of two programs can be reduced to emptiness, and a program is empty iff its optimal typing is ⊥. As equivalence is undecidable [23], any type system that is both sound and optimal for all programs would likewise have to be undecidable.

Our proposed definition of type programs is significantly more liberal than previous proposals in the literature. Frühwirth et al. in a seminal paper on types for Datalog [16] propose the use of conjunctive queries, with no negation, no disjunction, no equalities and no existentials.

As an example where the additional power is useful, consider a database that stores the marriage register of a somewhat traditionally minded community. This database might have types male and female, specified to be disjoint by the type hierarchy, and an extensional relation married with schema (female, male). (Note that this means that all male and female entities in the database are married, i.e. occur somewhere in the married relation.)

Using our above terminology, we have custom character={male, female} and custom character=custom character ∪ {married}; custom character is ∀x.custom charactermale(x) custom character custom character female (x).

Let us now define an intensional predicate spouse:
[spouse(x, y)≡married(x, y) custom character married(y, x)]spouse(x, y)

What is the type of spouse(x, y)? In the proposal of [16], we could only assign it (female(x) custom character male(x)) custom character (female(y) custom character male(y)), so both arguments could be either male or female. By contrast, when employing our definition of type programs, the inferred type is
(female(x) custom character male(y)) custom character (male(x) custom character female(y))

This accurately reflects that x and y must be of opposite sex. By properly accounting for equality, we can also infer that the query spouse(x, y) custom character x≐y has type ⊥ under the hierarchy: nobody can be married to themselves.

3. Representing Type Programs

For query optimisation and error checking, the single most important operation on types is containment checking, i.e. we want to decide, for two type programs custom character and custom character′, whether custom charactercustom charactercustom character| custom character′, which we will write custom character | custom charactercustom character′ for short.

As mentioned in the introduction, we are often interested in checking whether a formula is unsatisfiable, that is whether it always gives an empty set of results regardless of the interpretation of the extensional relations. If such a formula occurs in a user program, it is regarded as a type error and the user is alerted to this fact; if it occurs during optimisation, it provides an opportunity for logical simplification. By the results of the previous section, a formula φ containing only harmless negations is unsatisfiable iff its type is equivalent to ⊥, that is iff ┌φ┐ custom character custom character ⊥.

Our type programs only make use of monadic extensionals, and their containment can be decided by a straightforward extension of the well-known decision procedure for monadic first order logic [6]. This construction, however, incurs an exponential blowup in formula size and does not directly yield a practical algorithm.

Under some mild restrictions on the class of type programs and the hierarchy, however, we can represent type programs in a compact manner that allows efficient containment checking in practice.

The class of type programs we deal with falls in between the two classes discussed in the previous section by allowing negation, but only in front of type symbols (and never before equalities). The definition of ┌•┐ is easily changed to accommodate this by setting ┌custom characterφ┐=custom characterφ if φ is of the form u(t) for u ∈ custom character, and τ otherwise. We call a possibly negated application of a type symbol to a variable a type literal.

Furthermore, we require the hierarchy custom character to be of the form ∀x.h(x), where h(x) does not contain quantifiers or equations, and only a single free variable x.

The basic unit of our representation are type propositions, which are Boolean combinations of type symbols, all of them applied to the same variable. In other words, a type proposition is a quantifier-free type program without fixpoint definitions or free relation variables, having precisely one free element variable. For example, the subformula h(x) of custom character is a type proposition.

A type proposition φ behaves essentially the same as the propositional formula |φ| that results from it by replacing every atom u(x) with the propositional letter u; for example, it is easily seen that ∀x.y is (first order) valid iff |φ| is (propositionally) valid [20]. Hence we will sometimes identify the two notationally.

For example, if a and b are type symbols, then a(x) custom character custom character b(x) is a type proposition with |a(x) custom character custom character b(x)|=a custom character custom character b.

For the restricted class of type programs under consideration, fixpoint definitions can be eliminated. To see this, recall that fixpoint definitions can be expanded into infinite disjunctions [10]. It thus suffices to show that the type programs without free relation variables and without fixpoint definitions (which we shall henceforth call flat) form a complete lattice, so that even a prima facie infinite disjunction can in fact be represented by a single formula.

Lemma 4. The flat type programs form a complete lattice under the order custom character custom character.

Proof. The lattice structure is obviously given by the logical operations of conjunction and disjunction. To show completeness, we show that the set of flat type programs is finite (modulo equivalence under custom character custom character).

Notice that every flat type program with m free element variables from X={x1, . . . , xm} can be written in prenex form as

y 1 , , y n · i [ 1 j m ψ i , j 1 k n χ i , k ( y k ) ζ ]
for a set Y={y1, . . . , yn} of element variables, where all the ψi,j and Xi,k are type propositions, and ζ is a conjunction of equations between element variables from X ∪ Y.

Existentials distribute over disjunctions, and also over conjunctions if the variable they bind is only free in one of the conjuncts. Also, note that ∃x.(x≐y custom character φ) is semantically equivalent to φ with y substituted for x. Hence we can further bring the type program into the form

i [ 1 j m ψ i , j ( x j ) 1 k n ( y k · χ i , k ( y k ) ) ζ ]
where ζ′ now contains only variables from X. We will say that a formula of this shape is in solved form.

Clearly, there are only finitely many different formulae in solved form (modulo semantic equivalence), since there are only finitely many type propositions.

Notice that the set of flat type programs is no longer finite if we allow negated equalities. Indeed, for every natural number n, let εn be a formula asserting that there are at least n different individuals; for example, ε2 is ∃x1, x2. custom characterx1≐x2. Then custom characterεn custom character custom character custom characterεn+1 for every n, yet all the εn are semantically distinct; so the lattice is not even of finite height anymore.

Corollary 5. Every type program without free relation variables is semantically equivalent to a flat type program, and hence to a formula of monadic first order logic.

Here is an example of how a type program (with only a single disjunct) can be brought into solved form.
∃z·a(x)custom characterz≐ycustom characterb(y)custom characterc(x)custom characterx≐ycustom character
custom characterc(z)≡{sorting by variable}∃z·a(x)custom characterc(x)custom characterb(y)custom characterx≐ycustom character
z≐ycustom charactercustom characterc(z)≡{pushing in ∃}a(x)custom characterc(x)custom characterb(y)custom characterx≐ycustom character
∃z·z≐ycustom charactercustom characterc(z)≡{eliminating≐under ∃}a(x)custom characterc(x)custom characterb(y)custom character
x≐ycustom charactercustom characterc(y)≡{sorting again}a(x)custom characterc(x)custom characterb(y)custom charactercustom characterc(y)custom characterx≐y

We will now introduce data structures that represent formulae in solved form, and develop a containment checking algorithm that works directly on this representation.

We define an order on type propositions by taking φ<: ψ to mean | |h| custom character |φ|→|ψ|, where |h| is the propositional formula corresponding to custom character as defined above. It is not hard to see that this holds precisely when φcustom charactercustom character ψ. In the remainder, we will identify type propositions that are equivalent under <: so that it becomes a partial order. Where needed, we make equivalence under <: explicit by writing it as ≡custom character.

If t is a single type proposition and C a set of type propositions, we write t<:C (C<:t) to mean that t<:c (c<:t) for some c∈C.

The order <: can be extended to sets of type propositions by defining C<:D to mean that for any d∈D there is some c∈C with c<:d. This order is clearly reflexive and transitive, and is anti-symmetric (and hence a partial order) on lean sets, i.e. sets C where c<:c′ for c, c′∈C implies c=c′. One way of making a set of type propositions lean is to discard its non-minimal elements, which we write as min(C).

We can represent every disjunct of a formula in solved form using the following data structure:

Definition 1. An n-ary type tuple constraint, or TTC, τ is a structure (t1, . . . , tn|p|C) where

If p is a relation, we write p to mean the smallest equivalence relation containing it. The equivalence class of an element x under p is written [x]p, where we omit the subscript p if it is clear from context. We call an equivalence relation q finer (or, more precisely, no coarser) than p if whenever x˜qy then also x˜py.

Whenever we have a pre-TTC τ=(t1, . . . , tn|p|C) that does not satisfy one of the requirements, we can build a TTC that does by setting
∥τ∥:=(∥t1p . . . , ∥tnp|p|,om(C∪{t1, . . . , tn}))
where ∥tiq=custom character{tj|i˜qj}.

For a TTC τ=(t1, . . . , tn|p|C), we define τi:=ti, and τi:=r is the TTC resulting from τ by replacing the component ti (and every component tj where i˜p j) by r and adding r to the inhabitation constraints.

The formula represented by a TTC is easily recovered:

Definition 2. Given a list of n different element variables x1, . . . , xn, we define the type program [τ](x1, . . . , xn) corresponding to a TTC τ=(t1, . . . , tn|p|C) as

( 1 k n t i ( x i ) ) ( i p j x i x j ) ( c C y · c ( y ) )

A TTC is called degenerate if ⊥ is among its inhabitation constraints; it represents an unsatisfiable formula. The equivalence relation is called trivial if it is in fact the identity relation id, and the set of inhabitation constraints is called trivial if it only constrains the component types to be inhabited. When writing a TTC, we will often leave out trivial partitionings or inhabitation constraints.

We extend the order <: to TTCs component-wise:

Definition 3. For two n-ary TTCs τ=(t1, . . . , tn|p|C) and τ′=(t1′, . . . , tn′|p′|C′) we define τ<:τ′ to hold iff

It is routine to check that this defines a partial order with maximal element τTTC=(τ, . . . , τ), with binary meets defined by

( t 1 , , t n p C ) _ ( t 1 , , t n p C ) = ( t 1 t 1 , , t n t n p p C C ) ( 4 )

Since all type programs can be brought into solved form, we can represent them as sets of TTCs.

Definition 4. A lean set of non-degenerate TTCs is called a typing.

The formula represented by a typing is
[T]:=custom character

The order <: is extended to typings by defining T<:T′ to hold if, for every τ∈T there is a τ′∈T′ with τ<:τ′.

To convert any set of TTCs into a typing we can eliminate all non-maximal and degenerate TTCs from it; this operation we will denote by max.

The order on typings is again a partial order with least element ∅ and greatest element {τTTC}. Binary meets can be defined point-wise as
Tcustom characterT′=max({τcustom characterτ′|τ∈T, τ′∈T′}  (5)
and joins are given by
Tcustom characterT′=max(T∪T′)  (6)

We will now show that every type program can be translated to a typing. We already have enough structure to model conjunction and disjunction; existential quantification is also not hard to support.

Definition 5. The i-th existential of a TTC τ=(t1, . . . , tn|p|C), where 1≦i≦n, is defined as
i(τ):=(t1, . . . , ti−1, ti+1, . . . , tn|p′|min(C∪{ti}))
where p′ is the restriction of p to {1, . . . , i−1, i+1, . . . , n}.

The existential of a typing is defined pointwise:
i(T):=max{∃i(τ)|τ∈T}

To see that this models the existential quantifier on type programs, we can easily convince ourselves of the following:

Lemma 6. Let τ be an n-ary TTC, custom character an interpretation and σ a statement. Then custom character | σ [∃i(τ)] iff there is a domain element d such that custom character custom character σ, xi:=d[τ]. An analogous statement holds for typings.

Although typings cannot directly represent type programs with fixpoint definitions or free relation variables, we can translate every closed type program to a typing by eliminating definitions. Let us start with a translation relative to an assignment to free relation symbols.

Definition 6. Let φ be a type program without fixpoint definitions containing the intensional relation variables r1, . . . , rm and the element variables x1, . . . , xn. We translate it to a mapping custom characterφcustom character: Typingm→Typing by recursion on the structure of φ:
custom charactercustom character({right arrow over (T)})=∅
custom characterxi≐xjcustom character({right arrow over (T)})={(τ, . . . , τ|{{ij}})}
custom characteru(xj)custom character({right arrow over (T)})={τTTCj:=u}
custom charactercustom characteru(xj)custom character({right arrow over (T)})={τTTCj:=custom characteru}
custom characterri({right arrow over (x)})custom character({right arrow over (T)})=Ti
custom characterψcustom characterXcustom character({right arrow over (T)})=(ψ)({right arrow over (T)})custom charactercustom characterXcustom character({right arrow over (T)})
custom characterψcustom characterXcustom character({right arrow over (T)})=(ψ)({right arrow over (T)})custom charactercustom characterXcustom character({right arrow over (T)})
custom character∃xicustom character({right arrow over (T)})=∃i(custom characterψcustom character({right arrow over (T)}))

It is easy to check that custom characterφcustom character is a monotonic mapping for every φ with respect to the order <:, and since the set of typings is finite its least fixpoint can be computed by iteration. Thus we can translate every type program φ which does not have free relation variables into a typing custom characterφcustom character which represents it:

Lemma 7. For every type program φ without free relation variables, we have φcustom charactercustom character[custom characterφcustom character].

Proof Sketch. In fact, we can show that, for any type program φ with m free relation variables and every m-tuple {right arrow over (T)} of typings, we have the equivalence φ([{right arrow over (T)}]) custom character [custom characterφcustom character ({right arrow over (T)})], where [{right arrow over (T)}] is the m-tuple of type programs we get from applying [•] to every component of {right arrow over (T)}.

This can be proved by structural induction on φ by showing that [custom charactercustom character] commutes with the logical operators (using Lemma 6 for the case of existentials) and with fixpoint iteration. The lemma then follows for the case m=0.

4. Containment Checking

We are now going to investigate the relation between the orders <: (component-wise or syntactic subtyping) and custom character custom character (semantic subtyping). The former is convenient in that it can be checked component-wise and only involves propositional formulae, so it would be nice if we could establish that T1<:T2 iff [T1]| custom character [T2].

Unfortunately, this is not true. The ordering <: is easily seen to be sound in the following sense:

Theorem 8. For any two typings T and T′, if T<:T′ then [T] custom character custom character [T′].

However, it is not complete. Consider, for example, the typings T={(b custom character c, a)} and T′={(a custom character b, a), (c custom character d, a)}. Clearly we have [T] custom charactercustom character [T′], yet Tcustom characterT′.

If we interpret type symbols as intervals of real numbers, this example has an intuitive geometric interpretation, shown in FIG. 4.

The individual TTCs then become rectangles in the two-dimensional plain, and it is easy to see that the single TTC in T (depicted by the single rectangle in heavy outline filled with vertical lines) is contained in the union of the TTCs in T′ (the two rectangles filled with slanted lines), but not in either one of them in isolation.

A similar problem can arise from the presence of equality constraints: Consider the typings T={(a custom character b, a custom character b|{{1, 2}})} and T′={(a, a), (b, b)}. The former appears in FIG. 5 as a heavy slanted line, which is not contained in either of the two squares representing the TTCs in T′, but only in their union.

Intuitively, the problem is that <: tests containment of TTCs in isolation, whereas the semantic inclusion check considers containments between sets of TTCs.

The following lemma paves the way for a solution:

Lemma 9. The mapping [•] is the lower adjoint of a Galois connection, i.e. there is a mapping (|•|) from type programs to typings such that
[T] custom charactercustom charactercustom character iff T<:(|custom character|)

Proof. As we have established, typings and type programs form complete lattices under their respective orders. Since [•] distributes over joins it must be the lower adjoint of a Galois connection between these lattices.

An immediate consequence of this is that [T1] custom character custom character [T2] implies T1<:(|[T2]|). To check, then, whether T1 is semantically contained in T2, we compute (|[T2]|) and check for component-wise inclusion.

One possible implementation of (|•|) comes directly from the definition of the Galois connection: For any type program custom character, (|custom character|) is the greatest typing T such that [T] custom character custom character custom character, and since there are only finitely many typings and containment is decidable we could perform an exhaustive search to find this T.

Clearly, this algorithm is not ideal, since one of our goals is to avoid having to explicitly decide the relation custom charactercustom character. Inspiration for a better solution comes from a perhaps somewhat unexpected direction, namely the theory of Boolean function minimisation:

Quine [22] introduces an algorithm whose first part determines the set of prime implicants of a Boolean formula, which are conjunctive formulae such that every conjunctive formula that implies the original formula implies one of the prime implicants.

Generalising from conjunctions to TTCs and from propositional formulae to typings, we will introduce a saturation mapping sat that generates the set of all prime implicant TTCs for a typing, which are TTCs such that every TTC that (semantically) entails the typing entails one of them componentwise. Hence whenever [T1] custom character custom character [T2] we have T1<:sat(T2), so sat is an implementation of (|[•]|). The saturation mapping also preserves semantics, so the other direction of the implication holds as well.

We can follow Quine's definitions and proofs almost paragraph by paragraph, starting with the generalisation of the consensus operation:

Definition 7. The consensus τ⊕J τ′ over J of two n-ary TTCs τ and τ′, where J{1, . . . , n} is an index set, is defined by

( t 1 , , t n p C ) J ( t 1 , , t n p C ) = ( u 1 , , u n p p J × J C C )

where

u i = { j J t j j J t j if i J t i t i otherwise

We will sometimes write custom characterJ τ for custom characterj∈Jtj.

Roughly, the consensus operation equates all columns in J and preserves all other equalities and inhabitation constraints of its operands, takes disjunctions over the columns in J, and conjunctions over all the others.

For example, in the example shown in FIG. 5 we can take the consensus of (a, a) and (b, b) over J={1, 2} to obtain the TTC (a custom character b, a custom character b|{{1, 2}}), which is precisely the missing piece to show that T is a subtype of T′.

In the particular case where J={j} is a singleton set, and both partitions and inhabitation constraints are trivial, the consensus takes conjunctions on all columns except j, where it takes a disjunction. This is a generalisation of the behaviour of the consensus operator used in the minimisation of Boolean functions.

For example, in FIG. 4 we can take the consensus of (a custom character b, a) and (c custom character d, a) on J={1} to obtain the TTC (a custom character b custom character c custom character d, a) which covers (b custom character c, a).

We need another kind of consensus to deal with inhabitation constraints:

Definition 8. The existential consensus of two n-ary TTCs τ and τ is defined by

( t 1 , , t n p C ) ( t 1 , , t n p C ) = ( t 1 t 1 , , t n t n p p C C )
where C custom character C′={c custom character c′|c∈C, c′∈C′}.

To see why this is necessary, assume we have type symbols a, b, c with custom charactercustom character ∀x.a(x) custom character b(x)→c(x), and consider typings T={(c|id|a custom character b)} and T′={(c|id|a), (c|id|b)}. Clearly, [T]=c(x1) custom character ∃z.(a(z) custom character b(z))=[T′], yet Tcustom characterT′ since {a custom character b}custom character:{a} and {a custom character b}custom character:{b}. However, if we add the existential consensus (c|id|a custom character b) to T′, we will be able to prove that T<:T′.

It is not hard to verify that adding the consensus of two TTCs to a typing does not change its semantics.

Lemma 10. For two TTCs τ, τ and an index set J we have [τ⊕Jτ′] custom character custom character [{τ, τ′}] and [τ⊕τ′]custom character custom character [{τ, τ′}].

Proof Sketch. Clearly, if J=∅ the consensus is just the meet, so nothing needs to be proved.

Otherwise, let custom character and σ be given such that custom character custom characterσ[τ⊕Jτ′]. Then, for some i∈J, either custom character| σcustom characterj∈Jtj(xi) or custom character| σcustom characterj∈Jtj′; in the former case custom charactercustom characterσ[τ], and in the latter case custom charactercustom characterσ[τ′].

For the existential consensus, assume custom character|σ[τ⊕τ′] and further assume there is some inhabitation constraint c of τ such that custom charactercustom character ┌y.c(y) (for otherwise custom charactercustom characterσ[τ] is immediate). We must have custom charactercustom character ∃y.c(y) custom character c′(y) for all inhabitation constraints c′ of τ′, so certainly custom character| ∃y.c′(y). We then easily see that custom character custom characterσ[τ′].

Remember that typings do not contain non-maximal elements. The following result shows that we do not lose anything by eliminating them.

Lemma 11. Consensus formation is monotonic in the sense that for TTCs σ, σ′, τ, τ′ with σ<:σ′ and τ<:τ′ and for an index set J we have σ⊕Jτ<: σ′⊕Jτ′ and σ⊕τ<:σ′⊕τ′

Proof Sketch. The components of the two TTCs are combined using conjunction and disjunction, which are monotonic with respect to subtyping, and the partitionings and inhabitation constraints are combined using set union which is also monotonic.

Definition 9. An n-ary typing T is said to be saturated if the consensus of any two TTCs contained in it is covered by the typing; i.e. for any τ, τ′∈T and any index set J{1, . . . , n} we have τ⊕Jτ′<:T and τ⊕τ′<:T.

Lemma 12. Every typing T can be converted into a semantically equivalent, saturated typing sat(T) by exhaustive consensus formation.

Proof. We use Algorithm 2 to collect all consensus TTCs of two given TTCs. As shown in Algorithm 1, this is performed for every pair of TTCs in the typing, and this procedure is repeated until a fixpoint is reached.

Notice that in each iteration (except the last) the set of TTCs covered by the typing becomes larger, and since there are only finitely many TTCs, the termination condition must become true eventually.

We now generalise the concepts of implicants and prime implicants.

Definition 10. A TTC τ is an implicant for a typing T′ if we have [τ] custom character custom character [T′]; it is a prime implicant if it is a <:-maximal implicant.

More explicitly, a TTC π is a prime implicant for a typing T′ if

1. π is an implicant of T′

2. For any τ with π<:τ and [τ] custom character custom character[T′], we have τ<:π.

The second condition is equivalent to saying that for any τ with τ<:τ and τcustom character:π we have [τ] custom character custom character [T].

Lemma 13. Every implicant implies a prime implicant.

Proof. The set of all implicants of a typing is finite, so for any implicant τ there is a maximal implicant π with τ<:π, which is then prime by definition.

Remark 14. If π is a prime implicant of T, then π<:T iff π∈T.

Proof. The direction from right to left is trivial. For the other direction, suppose π<:T, i.e. π<:π′ for some π′∈T. Certainly [π′] custom character custom character [T], so π′<:π since π is prime. But this means that π=π′∈T.

We want to show that sat(T) contains all prime implicants of T, so we show that saturation can continue as long as there is some prime implicant not included in the saturated set yet.

Lemma 15. If there is a prime implicant π of T with π∉T, then T is not saturated.

Proof. Let π be a prime implicant of T with π∉T. Consider the set
M:={τ|[τ]custom character custom character[π], τcustom character:T}
This set is not empty (it contains π by the preceding remark) and it is finite, so we can choose a <:-minimal element ψ∈M.

The proof proceeds by considering three cases. In the first, we shall show that a consensus step is possible, thus proving that T is not saturated. In the second, we show that an existential consensus step can be made, again proving non-saturation. Finally, we show that either the first or second case must apply, as their combined negation leads to a contradiction.

We will show that in this case ψ<:T, contrary to our assumptions. We know that [ψ] | custom character [T], so any interpretation custom character and assignment σ with custom character custom character custom character and custom character custom character σ [ψ] will satisfy some TTC τ∈E T. Put differently, if custom character custom character σ(custom charactercustom characterψ) (xi) for all i and custom character custom character ∃y.(custom charactercustom characterc) (y) for all inhabitation constraints c and moreover a satisfies the partition of ψ, then we will find τ∈T with custom charactercustom characterτ[τ].

Observe that for every i the (propositional) formula |h|custom characteri| can be written as a conjunction li, l custom character . . . custom character li, mi of type literals. We can moreover assume that every type symbol occurs in precisely one of these literals, for if, say, type t does not occur, then li, 1 custom character . . . custom character li, mi custom character t custom character li, 1 custom character . . . custom character li, mi custom character custom character is a join reduction of |h|custom characteri|.

Let such a decomposition be fixed and let Li:={1i, 1, , . . . , li, mi}. Since ψij whenever i and j are in the same partition, we can clearly choose the decomposition such that Li=Lj in this case. Likewise, for every inhabitation constraint c we can write |h|custom character|c| as lc, 1 custom character . . . custom character lc, mc, and define Lc by analogy.

We define an interpretation custom character over the domain {x[1], . . . , x[n]}∪C of variables modulo the partition of ψ plus the inhabitation constraints by
custom character(b)={x[i]|b∈Li}∪{c|b∈Lc}
for every type symbol b. The assignment is simply defined by σ(xi)=x[i].

It is easy to see that x[i]∈[[l]] iff l∈Li for all 1≦i≦n, and c∈[[l]] iff l∈Lc for all inhabitation constraints c, from which we deduce custom charactercustom characterσ (custom character custom character ψi) (xi) for all i, and custom charactercustom character ∃y.(custom charactercustom character) (y) for all inhabitation constraints c. By definition, σ satisfies the partition of ψ. So we have some τ∈T with custom character| σ[τ].

We claim that ψ<:T.

Indeed, let an index 1≦i≦n be given. Then we can write τi as a disjunction of conjunctions, such that one of its disjuncts, of the form li, 1custom character . . . custom character li, mi′′, is satisfied under custom character and σ, meaning that x[i]∈[[li,j′]] custom character for all j, so all the li,j′ are in fact in Li, so ψi<:τi.

Since σ satisfies the partition of τ, this partition must be finer than the partition of ψ.

Finally, let an inhabitation constraint d of τ be given. Since custom character custom character ∃y.(custom charactercustom character) (y), the interpretation of custom charactercustom characterd cannot be empty. As above, we can write custom charactercustom characterd in a disjunctive form such that all the literals in one of its disjuncts are non-empty in custom character. So there must be some domain element that occurs in the denotation of all these literals. If it is one of the x[i], then we have ψi<:d, which implies c<:d for some inhabitation constraint c of ψ; if it is an inhabitation constraint c, then we have c<:d directly.

Taking these facts together, we get ψ<:τ, whence ψ<:T. This contradicts our assumption, and we conclude that this subcase cannot occur.

Now we can declare victory:

Theorem 16. If [T1] custom character custom character [T2], then T1<:sat(T2).

Proof. Assume [T1] custom character custom character [T2] and let τ∈T1 be given. Then [τ] custom character custom character [T2], so certainly [τ] custom character custom character [sat(T2)], since saturation does not change the semantics. By Lemma 13 this means that there is a prime implicant π of sat(T2) with τ<:π. By Lemma 15 we must have π∈sat(T2), so τ<: sat(T2). Since this holds for any τ∈T1 we get T1<:sat(T2).

5. Implementation

It is not immediately clear that the representation for type programs proposed in the preceding two sections can be implemented efficiently. While the mapping ┌•┐ from programs to type programs is, of course, easy to implement and linear in the size of the program, the translation of type programs into their corresponding typings involves fixpoint iterations to eliminate definitions. Saturation is also potentially very expensive, since we have to compute the consensus on every combination of columns for every pair of TTCs in a typing, and repeat this operation until a fixpoint is reached.

We report in this section on our experience with a prototype implementation based on Semmle's existing type checking technology as described in [13]. The type checker computes typings for programs and immediately proceeds to saturate them to prepare for containment checking. It uses a number of simple heuristics to determine whether a consensus operation needs to be performed. This drastically reduces the over-head of saturation in practice and makes our type inference algorithm practically usable.

TTCs can be compactly represented by encoding their component type propositions as binary decision diagrams. The same can be done for the hierarchy, so checking containment of type propositions can be done with simple BDD operations.

As with any use of BDDs, it is crucial to find a suitable variable order. We choose a very simple order that tries to assign neighbouring indices to type symbols that appear in the same conjunct of the hierarchy formula custom character. For example, if the hierarchy contains a subtyping statement u1(x)→u2(x) or a disjointness constraint custom character(u1(x) custom character u2(x)), then u1 and u2 will occupy adjacent BDD variables. This heuristic is simple to implement and yields reasonable BDD sizes as shown below.

To mitigate the combinatorial explosion that saturation might bring with it, we avoid useless consensus formation, i.e. we will not form a consensus τ⊕τ′ if τ⊕τ′<:{τ, τ′} or τ⊕τ′ is degenerate. The following lemma provides a number of sufficient conditions for a consensus to be useless:

Lemma 17. Let τ=(t1, . . . , tn|p|C) and τ′=(t1′, . . . , tn′|p′|C′) be two TTCs and J {1, . . . , n} a set of indices. Then the following holds:

Proof. Recall that τ⊕Jτ′=∥(u1, . . . , un|p∪p′∪J×J|C∪C′)∥ where

u i = { j J t j j J t j if i J t i t i otherwise
We prove the individual claims:

Otherwise we have (τcustom characterJτ′)i=(τcustom characterJ′τ′), so overall τcustom characterJτ′<:τcustom characterJ′τ′.

This suggests an improved implementation of Algorithm 2, shown in two parts as Algorithm 3 and Algorithm 4, which straightforwardly exploit the above properties to avoid performing useless consensus operations whose result would be discarded by the maxoperator anyway. In the latter, we make use of an operation PART(ν, i) which returns the equivalence partition index i belongs to in TTC ν.

To show that these improvements make our type inference algorithm feasible in practice, we measure its performance on some typical programs from our intended application domain. The Datalog programs to be typed arise as an intermediate representation of programs written in a high-level object-oriented query language named .QL [14, 15], which are optimised and then compiled to one of several low-level representations for execution.

We measure the time it takes to infer types for the 89 queries in the so-called “Full Analysis” that ships with our source code analysis tool SemmleCode. These queries provide statistics and overview data on structural properties of the analysed code base, compute code metrics, and check for common mistakes or dubious coding practices.

A type inference for the query being compiled is performed at three different points during the optimisation process, giving a total of 267 time measurements. All times were measured on a machine running a Java 1.6 virtual machine under Linux 2.6.28-13 on an Intel Core2 Duo at 2.1 GHz with four gigabytes of RAM, averaging over ten runs, and for each value discarding the lowest and highest reading.

Of the 267 calls to type inference, 65% finish in 0.5 seconds or less, 71% take no more than one second, 93% no more than two, and in 98% of the cases type inference is done in under three seconds. Only two type inferences take longer than four seconds, at around 4.5 seconds each.

The size of the programs for which types are inferred varies greatly, with most programs containing between 500 and 1500 subterms, but some programs are significantly larger at more than 3000 subterms. Interestingly, program size and type inference time are only very weakly correlated (p<0.4), and the correlation between the number of stratification layers (which roughly corresponds to the number of fixpoint computations to be done) and type inference time is also not convincing (ρ<0.6).

The low correlation is evident in FIG. 6 which shows the type inference time plotted on the y-axis versus the depth (i.e., number of layers) of the program on the x-axis. In particular, the two cases in which type inference takes longer than four seconds are both of medium size and not among the deepest either.

This suggests that the asymptotic behaviour of the algorithm in terms of input size is masked by the strong influence of implementation details (at least for the kind of programs we would expect to deal with in our applications), and we can expect significant performance gains from fine tuning the implementation.

Our experiments also show that saturation, while extremely expensive in theory, is quite well-behaved in practice: in 78% of all cases, no new TTCs are added during the first iteration of Algorithm 1, so the typing is already saturated. In another 17% of all cases we need only one more iteration, and we almost never (≈0.01%) need to do more than four iterations.

Although in every individual iteration we potentially have to take the consensus over every combination of columns, our heuristics manage to exclude all combinations of more than one column in 94% of all cases, and sometimes (14%) can even show that no consensus formation is needed at all.

Since our type inference algorithm makes heavy use of BDDs, some statistics about their usage may also be of interest. We use about 300 BDD variables (one for every type symbol), with most of the BDDs constructed during type checking being of rather moderate size: the BDD to represent the type hierarchy (which encodes about 800 constraints automatically derived during translation from .QL to Datalog) occupies around 4000 nodes, while the BDDs for individual type propositions never take more than 100 nodes.

While we have anecdotal evidence to show that the optimisation techniques we have presented in earlier work [13] benefit greatly from combining them with the richer type hierarchies our type system supports, we leave the precise investigation of this matter to future work.

6. Benefits

The present invention handles negated types accurately. Also, the present invention handles inhabitation constraints (i.e., existentials in type programs) precisely.

We achieve pleasing theoretical properties and can support a rich language of type constraints by sacrificing polynomial time guarantees, although our experiments show that this is a reasonable tradeoff for our application area.

While the performance of our prototype implementation is promising, other implementation approaches certainly exist and may be worth exploring. Bachmair et al. [4] develop a decision procedure for monadic first order logic based on the superposition calculus. Since type programs can readily be expressed as monadic first order formulae, their algorithm could be used to decide type containment. Another possibility would be to use Ackermann's translation from monadic first order logic to equality logic [2], and then employ a decision procedure for this latter logic.

7. Review of High Level Flow Diagrams

FIG. 7 is a block diagram illustrating how types of database queries are inferred. Type inferencer 701 receives database schema 703 and query 704 and infers types from definitions in the program by replacing each use of a database relation in the program by the types in the database schema, resulting in type program 706 as described in the section above entitled “2. Type Programs.” The type inferencer furthermore receives type hierarchy 705 to check types in the type program for inclusion as described in the section above entitled “4. Containment Checking,” generating information about type inclusion 708. The type inferencer also uses the type hierarchy to generate emptiness information 710 indicating which types in the type program are empty as described in the section above entitled “2. Type Programs.” In one embodiment the inclusion and emptiness information is used by query optimizer 711 to generate an optimized version of the query that can be run on query engine 712 to query relational data source 713, as described in the section above entitled “1. Introduction,” subsection “Overview.”

FIG. 8 is a flow chart illustrating inclusion testing for types represented by sets of type tuple constraints (TTCs). To test inclusion of a type represented by a set S1 of TTCs 801 in a type represented by a set S2 of TTCs 802, set S2 is first saturated 803 by adding all prime implicants of all TTCs in S2 to the set, yielding a larger set S2804, as described in the section above entitled “4. Containment Checking.” All TTCs in S1 are then examined in turn 805, and for every TTC 807 in S1 it is checked whether there is a larger, covering TTC in S2808, as described in the section above entitled “4. Containment Checking,” subsection “Definition 7.” If no such larger TTC is found 809, the type represented by S1 is not included in S2. Otherwise the TTC is removed from S1 810 and the next TTC is examined. If a covering TTC is found for every TTC in S1 806, then the type represented by S1 is included in the type represented by S2.

FIG. 9 is a flow chart illustrating saturation of sets of type tuple constraints (TTCs) for inclusion testing. Starting with a given set S of TTCs 901, every pair of TTCs from S is considered in turn 902, and a consensus of the two TTCs is computed 903, as described in the section above entitled “3. Representing Type Programs,” subsection “Definition 4.” If S does not contain a TTC larger than the consensus 904, it is added to the set S 905. This is repeated until all pairs of TTCs have been considered 907.

8. Conclusion

We have presented a type inference procedure that assigns an upper envelope to each Datalog program. That envelope is itself a Datalog program that makes calls to monadic extensionals only. The algorithm is able to cope with complex type hierarchies, which may include statements of implication, equivalence and disjointness of entity types.

The type inference procedure is itself an extremely simple syntactic mapping. The hard work goes into an efficient method of checking containment between type programs. We achieve this via a novel adaption of Quine's algorithm for the computation of the prime implicants of a logical formula. Generalising from logical formulae to type programs, we bring types into a saturated form on which containment is easily checked.

As shown by our experiments, the algorithm for inferring a type and saturating it works well on practical examples. While it may still exhibit exponential behaviour in the worst case, such extreme cases do not seem to arise in our application area. Thus our algorithm is a marked improvement over well-known simpler constructions that always require an exponential overhead.

Many avenues for further work remain. Perhaps the most challenging of these is the production of good error messages when type errors are identified: this requires printing the Boolean formulae represented via TTCs in legible form. We have made some progress on this, employing Coudert et al.'s restrict operator on BDDs, which is another application of prime implicants [12].

There is also substantial further engineering work to be done in the implementation. Careful inspection of the statistics show that our use of BDDs is very much unlike their use in typical model checking applications [26], and we believe this could be exploited in the use of a specialised BDD package. For now we are using JavaBDD [25], which is a literal translation of a C-based BDD package into Java.

Finally, we will need to investigate how to best exploit the advanced features offered by our type system. In particular, much experience remains to be gained in how to make it easy and natural for the programmer to specify the constraints making up the type hierarchy, and which kinds of constraints benefit which kinds of programs.

9. Non-Limiting Hardware Examples

Overall, the present invention can be realized in hardware or a combination of hardware and software. The processing system according to a preferred embodiment of the present invention can be realized in a centralized fashion in one computer system, or in a distributed fashion where different elements are spread across several interconnected computer systems and image acquisition sub-systems. Any kind of computer system—or other apparatus adapted for carrying out the methods described herein—is suited. A typical combination of hardware and software is a general-purpose computer system with a computer program that, when loaded and executed, controls the computer system such that it carries out the methods described herein.

An embodiment of the processing portion of the present invention can also be embedded in a computer program product, which comprises all the features enabling the implementation of the methods described herein, and which—when loaded in a computer system—is able to carry out these methods. Computer program means or computer programs in the present context mean any expression, in any language, code or notation, of a set of instructions intended to cause a system having an information processing capability to perform a particular function either directly or after either or both of the following a) conversion to another language, code or, notation; and b) reproduction in a different material form.

FIG. 10 is a block diagram of a computer system useful for implementing the software steps of the present invention. Computer system 1000 includes a display interface 1008 that forwards graphics, text, and other data from the communication infrastructure 1002 (or from a frame buffer not shown) for display on the display unit 1010. Computer system 1000 also includes a main memory 1006, preferably random access memory (RAM), and optionally includes a secondary memory 1012. The secondary memory 1012 includes, for example, a hard disk drive 1014 and/or a removable storage drive 1016, representing a floppy disk drive, a magnetic tape drive, an optical disk drive, etc. The removable storage drive 1016 reads from and/or writes to a removable storage unit 1018 in a manner well known to those having ordinary skill in the art. Removable storage unit 1018, represents a CD, DVD, magnetic tape, optical disk, etc. which is read by and written to by removable storage drive 1016. As will be appreciated, the removable storage unit 1018 includes a computer usable storage medium having stored therein computer software and/or data. The terms “computer program medium,” “computer usable medium,” and “computer readable medium” are used to generally refer to media such as main memory 1006 and secondary memory 1012, removable storage drive 1016, a hard disk installed in hard disk drive 1014, and signals.

Computer system 1000 also optionally includes a communications interface 1024. Communications interface 1024 allows software and data to be transferred between computer system 1000 and external devices. Examples of communications interface 1024 include a modem, a network interface (such as an Ethernet card), a communications port, a PCM-CIA slot and card, etc. Software and data transferred via communications interface 1024 are in the form of signals which may be, for example, electronic, electromagnetic, optical, or other signals capable of being received by communications interface 1024. These signals are provided to communications interface 1024 via a communications path (i.e., channel) 1026. This channel 1026 carries signals and is implemented using wire or cable, fiber optics, a phone line, a cellular phone link, an RF link, and/or other communications channels.

Although specific embodiments of the invention have been disclosed, those having ordinary skill in the art will understand that changes can be made to the specific embodiments without departing from the spirit and scope of the invention. The scope of the invention is not to be restricted, therefore, to the specific embodiments. Furthermore, it is intended that the appended claims cover any and all such applications, modifications, and embodiments within the scope of the present invention.

Each of the following twenty-seven references are hereby incorporated by reference in their entirety.

Schaefer, Max, de Moor, Oege

Patent Priority Assignee Title
10423397, Dec 29 2016 GRAMMATECH, INC. Systems and/or methods for type inference from machine code
10942718, Dec 29 2016 GRAMMATECH, INC Systems and/or methods for type inference from machine code
9996583, May 25 2017 Microsoft Technology Licensing, LLC Differential recursive evaluation of recursive aggregates
Patent Priority Assignee Title
6006214, Dec 04 1996 International Business Machines Corporation Database management system, method, and program for providing query rewrite transformations for nested set elimination in database views
6338055, Dec 07 1998 INNOVATION TECHNOLOGY GROUP, INC Real-time query optimization in a decision support system
7089266, Jun 02 2003 The Board of Trustees of the Leland Stanford Jr. University; BOARD OF TRUSTEES OF THE LELAND STANFORD JR UNIVERSITY Computer systems and methods for the query and visualization of multidimensional databases
20020069193,
20050086639,
20070043702,
20070276787,
/////
Executed onAssignorAssigneeConveyanceFrameReelDoc
Jul 14 2011SCHAEFER, MAXSemmle LimitedASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS 0456900336 pdf
Jul 15 2011MOOR, OEGE DESemmle LimitedASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS 0456900336 pdf
Nov 25 2015Semmle Limited(assignment on the face of the patent)
Nov 29 2019Semmle LimitedGITHUB SOFTWARE UK LTD CHANGE OF NAME SEE DOCUMENT FOR DETAILS 0512440305 pdf
Jan 16 2020GITHUB SOFTWARE UK LTD Microsoft Technology Licensing, LLCASSIGNMENT OF ASSIGNORS INTEREST SEE DOCUMENT FOR DETAILS 0517100252 pdf
Date Maintenance Fee Events
May 26 2017M2551: Payment of Maintenance Fee, 4th Yr, Small Entity.
Jan 23 2020BIG: Entity status set to Undiscounted (note the period is included in the code).
May 12 2021M1552: Payment of Maintenance Fee, 8th Year, Large Entity.


Date Maintenance Schedule
Apr 25 20204 years fee payment window open
Oct 25 20206 months grace period start (w surcharge)
Apr 25 2021patent expiry (for year 4)
Apr 25 20232 years to revive unintentionally abandoned end. (for year 4)
Apr 25 20248 years fee payment window open
Oct 25 20246 months grace period start (w surcharge)
Apr 25 2025patent expiry (for year 8)
Apr 25 20272 years to revive unintentionally abandoned end. (for year 8)
Apr 25 202812 years fee payment window open
Oct 25 20286 months grace period start (w surcharge)
Apr 25 2029patent expiry (for year 12)
Apr 25 20312 years to revive unintentionally abandoned end. (for year 12)