\section{Ontology bootstrapping using OBDA specifications}
\label{bootstrap_spec}
+TODO: only mapping, no duplication, r2rml
\subsection{Structure of OBDA specifications}
+\label{bootstrap_spec_struc}
An OBDA specification consists of several so-called ``maps'', which are
data records containing data and references to each other describing
parts of the OBDA specification in statically defined fields \cite{eng}.
For different aspects of the specification, there are different map types,
while usually several maps exist for each type.
Namely, these are \emph{Entity maps} describing database tables,
-\emph{Attribute maps} describing database columns,
\emph{Identifier maps} describing database primary keys,
+\emph{Attribute maps} describing database columns,
\emph{Relation maps} describing database foreign keys,
\emph{Subtype maps} describing ``is-a'' relationships in the data and
\emph{Translation tables} describing desired translations of data.
The fields of the several types of maps and their interconnection via
references is shown in Figure~\ref{spec_fig_structure}.
+Here, each field specifies, in that order, the field label, the field's
+long name, the bootstrapping steps in which the field is used and the
+field's short name. Fields storing a set of values have both their short
+and their long name suffixed with ``\code{...}''.
+Note that each reference between two fields is denoted with a short field
+name contained in the source of the reference, specifying the field
+in which the reference is stored.
+What the values of the fields of the several types of maps express
+exactly and how they can be used is described in
+Section~\ref{bootstrap_spec_using}.
+For a full description as well of the structure of OBDA specifications
+as of their application and the general idea behind them,
+refer to \cite{eng}.
+How OBDA specifications can in turn be automatically bootstrapped,
+excluding Subtype maps and Translation tables is
+described in Section~\ref{bootstrap_bootstrap}.
\begin{figure}[H]\begin{center}
\includegraphics[scale=1.0]{Images/specification_structure.pdf}
\label{spec_fig_structure}
\end{center}\end{figure}
-While ,
-Subtype maps and Translation tables.
+Entity maps, Identifier maps, Attribute maps and Relation maps
+directly relate to database concepts and each of them describes
+exactly one database table, primary key, column or foreign key,
+respectively, and vice versa.
+Subtype maps and Translation tables, on the other hand,
+represent concepts of the bootstrapping process or data to be added to
+the target ontology and are somewhat harder to obtain:
+Subtype maps represent ``is-a'' relationships in the target ontology
+to be determined from the source data \cite{eng} -- heuristically or
+semi-automatically --, while Translation tables allow for the
+transformation of data values,
+for example from \code{TRUE} to \code{true} or from \code{No} to
+\code{false} \cite{eng}.
+Therefore, they also have to be determined heuristically or semi-
+automatically from the source data --
+considering the database schema only is not sufficient \cite{eng}.
+Note that, because of this, special care has to be taken to keep the maps
+synchronized with the data in case of Subtype maps and Translation tables.
+
+The structural description of OBDA specifications in \cite{eng}
+does not propose a serialization format in which OBDA specifications can
+be stored or read and written by software and human agents.
+How this can be done is subject to this thesis, which introduces the
+\oslboth{} designed exactly for this purpose in Chapter~\ref{osl}.
\subsection{Using OBDA specifiations}
+\label{bootstrap_spec_using}
+TODO: dirm
+TODO: r2rml
+
+As described in in Section~\fullref{back_obdaspecs}, using OBDA
+specifications provides several benefits when concerned in ontology
+bootstrapping.
+Principally, information about the bootstrapping process is collected in
+one place and can be used to manage the tools involved.
+This includes the availability of the URIs to be used in the constructed
+ontology from a central place, which is a great advantage, since URIs
+are central to an ontology TODO.
+Additionally, all information on the database schema of the source
+database is available.
+Using Translation tables, all these information can at will be made
+subject to transformations normalizing or correcting the data
+changing the database \cite{eng}.
+
+By the use of a single specification language like the \oslboth{}
+to store OBDA specifications,
+the expense of converting between different data formats can be
+reduced significantly:
+assumed that there are $n$ different formats to be handled with no means
+provided to convert between them, the converting costs decrease from
+$\mathcal{O}(n^2)$ to $\mathcal{O}(n)$ by introducing a single central
+language TODO.
+
+Besides the structure of OBDA specifications, described in
+Section~\ref{bootstrap_spec_struc}, Skjæveland et al. introduce a set
+of formal rules defining the bootstrapping process and the mapping of the
+source data to the generated ontology \cite{eng}.
+Using an OBDA specification, tools implementing these rules can
+easily and in a well-defined manner bootstrap an ontology and
+a mapping from the source data onto this ontology (or duplicate that
+source data, if the materialized OBDA approach is used, see
+Section~\ref{back_obda}).
+The mapping rules produce RDF triples which can be interpreted by R2RML
+to establish the mapping (see Section~\fullref{back_basic}).
+They are accompanied by SQL statements specifying the
+queries over the source database used to link the ontology to the data.
+If the data source is not a relational database but another form of
+structured data, like \name{CSV} files, a database schema can be
+bootstrapped first by applying additional ``database rules'' \cite{eng}.
+Afterwards, the proceeding can continue as if the data source were a
+database, so this case is neglected in the following.
+
+The rest of this section contains description of the information contained
+in the various types of OBDA specification maps and how it is used during
+bootstrapping, based on the description in \cite{eng}.
+Keep in mind that ``bootstrapping'' here refers to the ontology
+bootstrapping process specified by the OBDA specification, yielding a
+target ontology and mappings
+relating it to the source database -- it does not refer to the
+bootstrapping of the OBDA specification itself.
+The text is meant to give a brief explanatory overview over the bootstrapping
+process using OBDA specifications.
+Thus, it focuses on the information they provide and how they are used to
+link the bootstrapped ontology to the source data, leaving out the
+SQL statements to be used to gain the datasets.
+How exactly the ontology is created is also left out, since this would have
+involved introducing too many technical details, and moreover, the topic is
+also comprehensible by describing the mapping only.
+For a detailed description of the bootstrapping process, including ontology
+creation and all formal rules to be applied, see \cite{eng}.
+For an explanation about how an OBDA specification containing Entity maps,
+Identifier maps, Attribute maps and Relation maps can be bootstrapped from
+a relational database schema, see Section~\ref{bootstrap_bootstrap}.
+For details on URI generation, refer to Section~\fullref{iris}.
+
+\subsubsection{Entity maps}
+Entity maps provide information about the tables contained in the
+source database or in the intermediate database schema to be constructed,
+if the data source is not a database but some other source of structured
+information (see Section~\fullref{back_obda}).
+The information provided by an Entity map includes the table name,
+a label describing the table and a (more detailed) description
+of the table.
+Furthermore, each Entity map references an Identifier map representing
+its primary key and a set of Attribute maps representing its columns.
+Finally, an Entity map provides an OWL class URI identifying the
+represented table uniquely in the resulting ontology.
+As the name suggests, this URI is given to an OWL class which serves as
+OWL type (or more precisely: \code{rdf:type}) for all OWL individuals
+representing the datasets from the respective table in the target ontology
+(see Paragraph~``\nameref{iden}'').
+
+Suppose, for example, that an Entity map for a table ``\code{persons}''
+provides the OWL class URI ``\code{mydb:persons}''.
+Then, in the target ontology, all OWL individuals representing rows in the
+``\code{persons}'' table, will be of \code{rdf:type}
+``\code{mydb:persons}''.
+If a data record in the ``\code{persons}'' table has the identifying URI
+pattern ``\code{mydb:person/\{pno\}}'' (see Paragraph~\nameref{iden}),
+this type information will be expressed by the following RDF triple:\\
+\code{mydb:person/\{pno\} rdf:type mydb:persons}.
+
+\subsubsection{Identifier maps}
+\label{iden}
+Identifier maps describe database primary keys contained in the
+source database or in the intermediate database schema to be constructed,
+if the data source is not a database (see Section~\ref{back_obda}).
+Each Identifier map contains a reference back to the respective
+Entity map, representing the database table the primary key belongs to.
+Furthermore, each Identifier map references a set of Attribute maps,
+representing the database columns the primary key consists of
+(see next paragraph).
+Finally, an Identifier map provides a URI pattern, allowing OWL
+individuals in the bootstrapped target ontology that represent
+datasets to be identified.
+A URI pattern contains placeholders like ``\code{\{\$1\}}'' for
+all primary key columns, which are replaced with the respective column
+names, surrounded by curly braces, during the bootstrapping process,
+to yield a valid R2RML template \cite{r2rml}.
+Since the column name substituted in is still a placeholder,
+the result of this substitution is still a URI pattern and not a URI.
+Since such a URI pattern uniquely identifies a dataset from a given
+database table -- when data values are substituted in --,
+it will be called \emph{identifying URI pattern} in the following.
+
+Consider the URI pattern ``\code{mydb:person/\{\$1\}}''.
+Replacing the placeholder ``\code{\{\$1\}}'' with the column name
+``\code{pno}'' in curly braces yields the following identifying URI
+pattern: \\ ``\code{mydb:person/\{pno\}}''.
+
+\subsubsection{Attribute maps}
+Attribute maps provide information about database columns contained in the
+source database.
+Each Attribute map carries the column name, information whether having a
+value in this column is mandatory for a dataset (in SQL terms: whether it
+has a \code{NOT NULL} constraint), a label describing the column as well as
+an extended description of the column.
+A database column is represented as a relation in the final ontology, thus,
+in OWL terms, as an \code{owl:DataProperty} or an \code{owl:ObjectProperty}.
+The Attribute map provides the URI for this OWL property.
+
+Additionally, it specifies the datatype of values in the
+represented column in the following manner:
+Three fields are provided for this purpose, \code{SQL datatype},
+\code{RDF language} and \code{XSD datatype}.
+If the \code{XSD datatype} field is nonempty, its value is specified to be
+the datatype for values in the column the Attribute map represents
+(note that OWL only knows XSD datatypes TODO).
+Otherwise, if the value of the \code{SQL datatype} is a standard SQL type,
+it will be mapped to an XSD datatype and the resulting type is
+specified as datatype for values in the respective column.
+If neither of the above is the case and the \code{RDF language} field is
+nonempty, values in the respective column will be interpreted as strings
+with the value of the \code{RDF language} field applied as RDF language tag
+(TODO).
+If neither of the above is the case, values in the respective column will be
+interpreted as strings without an RDF language tag.
+
+Finally, an Attribute map specifies whether the column shall be represented
+as an \\ \code{owl:DataProperty} (for non-foreign-key columns) or as an
+\code{owl:ObjectProperty} (for foreign key columns).
+The field specifying that -- \code{Property type} -- also allows, as a
+further distinction of object properties, whether the property's
+target URI should be the column name placed in an URI pattern
+provided by the Attribute map,
+similarly to URI patterns in Identifier maps (see example at the end of this
+paragraph), or if it shall simply be the column name, possibly with a
+translation from a Translation table, specified by the Attribute map,
+applied.
+The former option is useful for example when using custom property URIs
+to express relations between source data and the target ontology.
+If an \code{owl:DataProperty} is generated, it always has the column name
+as target URI, without the use of an URI pattern.
+Note that it is sufficient to have the column name be the target of the
+property, since only a \emph{mapping} to the source data is generated.
+
+Consider an Attribute map representing a column named ``\code{name}'' to
+be mapped to an \\ \code{owl:DataProperty}.
+Suppose the OWL property URI the Attribute map specifies is\\
+``\code{mynamespace:lastName}'' and each dataset containing the
+\code{name} column has the identifying URI pattern
+``\code{mydb:person/\{pno\}}'' (see Paragraph~``\nameref{iden}'').
+Then, during the bootstrapping process
+the following RDF triple will be produced:\\
+\code{mydb:person/\{pno\} mynamespace:lastName "\{name\}"}.\\
+This triple can easily be interpreted by R2RML, which on request then
+retrieves the queried \code{name} from the data source.
+
+Consider, as a more elaborate example,
+an Attribute map representing a column named
+``\code{company}'' and to be mapped to an \code{owl:ObjectProperty} with the
+use of the URI pattern ``\code{http://otherdb/\{\$1\}}''.
+Suppose the OWL property URI the Attribute map specifies is
+``\code{mynamespace:hasSameOwnerAs}'', there is no datatype specified
+and each dataset containing the
+\code{company} column has the identifying URI pattern
+``\code{mydb:company/\{cmpno\}}''
+(see Paragraph~``\nameref{iden}'').
+Then, the following RDF triple will be produced during the bootstrapping
+process:\\
+\code{mydb:company/\{cmpno\} mynamespace:hasSameOwnerAs
+ http://otherdb/\{company\}}.\\
+Note that this only makes sense if R2RML can expand
+``\code{http://otherdb/\{company\}}'' to a valid subject
+for each value in the \code{company} column of the database,
+and if all rows in the respective database table are
+indeed entities having the same owner as the company specified in the
+\code{company} column.
+
+\subsubsection{Relation maps}
+Relation maps represent foreign keys contained in the
+source database.
+Each Relation map references the Entity maps representing the foreign key's
+child table and parent table, respectively.
+Furthermore, it provides the column names of both the foreign key columns
+and the referenced columns and specifies an OWL property URI.
+Relation maps allow the relations expressed in the source data via
+foreign key relationships to be included into the bootstrapped ontology.
+This happens in a simple and straight-forward manner:
+for each foreign key relationship, exactly one triple is generated which
+contains the two identifying
+URI patterns representing the source and the target dataset of the foreign
+key, respectively, and the OWL property URI specified by the Relation map.
+
+Suppose, for example, that a Relation map expressing a relation between
+datasets with the identifying URI patterns (see Paragraph~``\nameref{iden}'')
+``\code{mydb:persons/pno/\{pno\}}'' and
+``\code{mydb:companies/cmpno/\{cmpno\}}'', respectively, and specifying
+the OWL property URI ``\code{mynamespace:isEmployedAt}''.
+This will result in the following RDF triple to be generated during the
+bootstrapping process:\\
+\code{mydb:persons/pno/\{pno\} mynamespace:isEmployedAt
+ mydb:companies/cmpno/\{cmpno\}}
+
+\subsubsection{Subtype maps}
+Subtype maps provide a means to automatically add subclass-superclass
+relationships to the target ontology during the bootstrapping process.
+They specify an Entity map and a column name defining a table and a
+column, respectively, that exist in the source database and contain the
+values to be declared as belonging to the subclass.
+Furthermore, they store a prefix, a suffix and possibly a reference to
+a Translation table which are used to generate a URI for that subclass.
+Finally, they provide the URI of the superclass.
+This can be, for instance, some OWL class being created during the
+bootstrapping process or already existing in some imported ontology.
+The URI generated for the subclass contains the data value of the
+respective database column, thus every dataset gets its own (sub)class.
+During bootstrapping, an RDF triple declaring the value to belong to that
+subclass is generated for each data value of the respective table column in
+the source database.
+The limitation to the desired value only thereby happens by a restriction
+on the SQL statement accompanying the respective triple, not by
+limiting the triple to only cover a specific data value.
+The actual subclass-superclass relationship is expressed during the
+creation of the ontology.
+Note the difference from the previously described mapping rules, which
+produced triples independent from the data values in the source database.
+
+Consider a Subtype map specifying a table column containing the values
+``\code{Purchase}'' and ``\code{Sales}'' with the datasets having the
+identifying URI pattern (see Paragraph~``\nameref{iden}'')
+``\code{mydb:managers/mno/\{mno\}}''.
+Suppose, the Subtype map specifies the prefix\\
+``\code{mydb:manager/of\_department/}'', no suffix, no translation table
+and the supertype URI ``\code{mynamespace:persons}''.
+This will result in the generation of the following triples during the
+bootstrapping process:\\
+\code{mydb:managers/mno/\{mno\} rdf:type mydb:manager/of\_department/Purchase}\\
+\code{mydb:managers/mno/\{mno\} rdf:type mydb:manager/of\_department/Sales},\\
+while ``\code{mydb:manager/of\_department/Purchase}'' \\ and
+``\code{mydb:manager/of\_department/Sales}'' will be subclasses of class\\
+``\code{mynamespace:persons}'' in the target ontology.
+The accompanying SQL statement will ensure that, despite the use of the
+R2RML template ``\code{mydb:managers/mno/\{mno\}}'', not \emph{every}
+manager will be declared as the manager of \emph{every} department.
+
+\subsubsection{Translation tables}
+Translation tables allow for transforming URIs or other strings in
+arbitrary ways, by simply mapping each string to be translated to a
+target string.
+
+They don't reflect in the target ontology in any form but are used
+only during the bootstrapping process.
\subsection{Bootstrapping OBDA specifications}
+\label{bootstrap_bootstrap}
+How OBDA specifications can in turn be bootstrapped from database schemata
+is subject to this thesis and is explained in this section.
+For the description of the software developed to automate this, see
+Chapter~\fullref{program}.
+The description in this section assumes an SQL database as data source.
+However, ontology-based data access and OBDA specifications are not limited
+to SQL databases, as mentioned in Section~\fullref{back_obda} and in
+Section~\fullref{bootstrap_spec_using}.
+Furthermore, this section refers to OBDA specifications without assuming
+any specific format in which they are represented.
+How OBDA specifications are represented internally by the \myprog{}
+software, is described in Section~\fullref{fine}.
+For the description of a format to serialize OBDA specifications
+-- the output format of \myprog{} --,
+refer to Chapter~\fullref{osl}.
+
+Subtype maps and Translation tables are not considered in this approach,
+since they cannot be bootstrapped from schema information only but have to
+be determined from the input data (see Section~\fullref{bootstrap_spec_struc}).
+Thus, the bootstrapped OBDA specification does not contain maps of these
+types. Including them is a significant challenge in its own right and,
+since the use of heuristics or user decisions would be necessary,
+would make the process involve human supervision at least.
+Apart from that, the bootstrapping is an easy and straight-forward task
+which can be carried out fully automatic TODO.
+
+Recall the structure of an OBDA specification explained in
+Section~\fullref{bootstrap_spec_struc}.
+The map types considered in this approach are Entity maps, Attribute maps,
+Identifier maps and Relation maps.
+The assignment of values to their fields is summarized in
+Table~\ref{bootstrap_tab_mapping}, only hinting at how maps are
+generated. Both the generation of the maps and the assignment of values
+to their fields are described in the rest of this section, with one
+exception:
+since the generation of URIs (or IRIs) in the context of OBDA specifications
+is an essential topic which requires some conceptual efforts, it is
+described in a separate section, Section~\ref{iris}.
+
+\begin{table}[H]\begin{centering}
+ \begin{tabular}{p{3.2cm}|p{4.0cm}|p{8.3cm}}
+ \textbf{Map type} & \textbf{Field name} & \textbf{Value} \\ \hline
+ Entity map & Table name & SQL table name \\
+ Entity map & Label & <empty> \\
+ Entity map & \emph{Identifier map} & Identifier map for table \\
+ Entity map & \emph{Attribute maps...} & Attribute maps for table columns \\
+ Entity map & OWL class URI & URI(table) \\
+ Entity map & Description & SQL table description \\ \hline
+ Identifier map & \emph{Entity map} & Entity map for corresponding table \\
+ Identifier map & \emph{Attribute maps...} & Attribute maps for primary key columns \\
+ Identifier map & URI pattern & URIpattern(table) \\ \hline
+ Attribute map & Column name & SQL column name \\
+ Attribute map & SQL datatype & SQL datatype of column \\
+ Attribute map & Mandatory & SQL \code{NOT NULL} property of column\newline(\code{true} or \code{false}) \\
+ Attribute map & Label & <empty> \\
+ Attribute map & OWL property URI & <empty> for foreign key columns,\newline else URI(table, column) \\
+ Attribute map & Property type & ``\code{ObjectProperty}'' for foreign key columns,\newline else ``\code{DataProperty}'' \\
+ Attribute map & \emph{Translation} & <empty> \\
+ Attribute map & URI pattern & <empty> \\
+ Attribute map & RDF language & <empty> \\
+ Attribute map & XSD datatype & <empty> \\
+ Attribute map & Description & SQL column description \\ \hline
+ Relation map & \emph{Source entity map} & Entity map for foreign key child table \\
+ Relation map & Source column & Foreign key child columns\newline(SQL column names) \\
+ Relation map & \emph{Target entity map} & Entity map for foreign key parent table \\
+ Relation map & Target column & Foreign key parent columns\newline (SQL column names) \\
+ Relation map & OWL property URI & URI(table, foreignKey) \\
+ \end{tabular}
+ \caption{Assignment of values to fields of OBDA specification maps}
+ \label{bootstrap_tab_mapping}
+\end{centering}\end{table}
+
+\subsubsection{Entity maps}
+Exactly one Entity map and one Identifier map is generated per table
+contained in the source database.
+The generated Identifier map is referenced by the Entity map's
+\code{Identifier map} field.
+Similarly, exactly one Attribute map is generated per table column
+and these Attribute maps are referenced by the Entity map's
+\code{Attribute maps...} field.
+The Entity map's \code{Table name} field is set to the SQL name of
+the table, the \code{Label} field remains empty.
+An URI identifying the table is generated and stored in the
+Entity map's \code{OWL class URI} field.
+The SQL table description is copied into the Entity map's
+\code{Description} field.
+
+\subsubsection{Identifier maps}
+An Identifier map represents exactly one primary key in the source
+database and is referenced by the Entity map representing the table
+containing the primary key constraint.
+In addition, it references this table in its
+\code{Entity map} field, so that there is a bidirectional referencing.
+The Attribute maps representing the columns constituting the primary
+key are referenced by the Identifier map's \code{Attribute maps...}
+field.
+An URI pattern, allowing datasets (thus, rows in the source database)
+to be identified in the target ontology, is generated and put in
+the Identifier map's \code{URI pattern} field.
+
+\subsubsection{Attribute maps}
+An Attribute map represents exactly one column in the source database
+and is referenced by the Entity map representing the table
+containing the column.
+The Attribute map's \code{Column name} field is set to the SQL column
+name of the column, the \code{SQL datatype} field is set
+to its SQL datatype.
+The \code{Mandatory} field is set to \code{true} if the column has
+the SQL \code{NOT NULL} constraint, otherwise to \code{false}.
+If the column is part of a foreign key, the \code{OWL propert URI}
+field remains empty. Otherwise,
+an URI identifying the column is generated and stored in the
+Attribute map's \code{OWL property URI} field.
+The \code{Property type} field is set to ``\code{ObjectProperty}''
+if the column is part of a foreign key, otherwise to
+``\code{DataProperty}''.
+The SQL column description is copied into the Attribute map's
+\code{Description} field.
+The remaining columns, \code{Label}, \code{Translation},
+\code{URI pattern}, \code{RDF language} and \code{XSD datatype}
+remain empty.
+
+\subsubsection{Relation maps}
+A Relation map represents exactly one foreign key in the source database.
+It contains fields storing the parent and child table of the foreign
+key: the \code{Source entity map} field, referencing the
+Entity map representing the child table of the foreign key, and
+the \code{Target entity map} field, referencing the
+Entity map representing the parent table of the foreign key.
+The SQL column names of the foreign key columns (thus, column names
+in the child table) are copied into the \code{Source column}
+field of the Relation map, and the SQL column names of the referenced
+columns in the parent table are copied into its \code{Target column}
+field.
+Note that, in contrast to Identifier maps representing primary keys,
+it is not referenced by any Entity map (or any other map).
git://git.uio.no / u / philim / db2osl_thesis.git / blobdiff