+
+\chapter{Analyzing Source Code in Eclipse}
+
+\section{The Java model}\label{javaModel}
+The Java model of \name{Eclipse} is its internal representation of a Java project. It
+is light-weight, and has only limited possibilities for manipulating source
+code. It is typically used as a basis for the Package Explorer in \name{Eclipse}.
+
+The elements of the Java model is only handles to the underlying elements. This
+means that the underlying element of a handle does not need to actually exist.
+Hence the user of a handle must always check that it exist by calling the
+\method{exists} method of the handle.
+
+The handles with descriptions is listed in \myref{tab:javaModel}.
+
+\begin{table}[h]
+ \centering
+
+ \newcolumntype{L}[1]{>{\hsize=#1\hsize\raggedright\arraybackslash}X}%
+ % sum must equal number of columns (3)
+ \begin{tabularx}{\textwidth}{| L{0.7} | L{1.1} | L{1.2} |}
+ \hline
+ \textbf{Project Element} & \textbf{Java Model element} &
+ \textbf{Description} \\
+ \hline
+ Java project & \type{IJavaProject} & The Java project which contains all other objects. \\
+ \hline
+ Source folder /\linebreak[2] binary folder /\linebreak[3] external library &
+ \type{IPackageFragmentRoot} & Hold source or binary files, can be a folder
+ or a library (zip / jar file). \\
+ \hline
+ Each package & \type{IPackageFragment} & Each package is below the
+ \type{IPackageFragmentRoot}, sub-packages are not leaves of the package,
+ they are listed directed under \type{IPackageFragmentRoot}. \\
+ \hline
+ Java Source file & \type{ICompilationUnit} & The Source file is always below
+ the package node. \\
+ \hline
+ Types /\linebreak[2] Fields /\linebreak[3] Methods & \type{IType} /
+ \linebreak[0]
+ \type{IField} /\linebreak[3] \type{IMethod} & Types, fields and methods. \\
+ \hline
+ \end{tabularx}
+ \caption{The elements of the Java Model. {\footnotesize Taken from
+ \url{http://www.vogella.com/tutorials/EclipseJDT/article.html}}}
+ \label{tab:javaModel}
+\end{table}
+
+The hierarchy of the Java Model is shown in \myref{fig:javaModel}.
+
+\begin{figure}[h]
+ \centering
+ \begin{tikzpicture}[%
+ grow via three points={one child at (0,-0.7) and
+ two children at (0,-0.7) and (0,-1.4)},
+ edge from parent path={(\tikzparentnode.south west)+(0.5,0) |-
+ (\tikzchildnode.west)}]
+ \tikzstyle{every node}=[draw=black,thick,anchor=west]
+ \tikzstyle{selected}=[draw=red,fill=red!30]
+ \tikzstyle{optional}=[dashed,fill=gray!50]
+ \node {\type{IJavaProject}}
+ child { node {\type{IPackageFragmentRoot}}
+ child { node {\type{IPackageFragment}}
+ child { node {\type{ICompilationUnit}}
+ child { node {\type{IType}}
+ child { node {\type{\{ IType \}*}}
+ child { node {\type{\ldots}}}
+ }
+ child [missing] {}
+ child { node {\type{\{ IField \}*}}}
+ child { node {\type{IMethod}}
+ child { node {\type{\{ IType \}*}}
+ child { node {\type{\ldots}}}
+ }
+ }
+ child [missing] {}
+ child [missing] {}
+ child { node {\type{\{ IMethod \}*}}}
+ }
+ child [missing] {}
+ child [missing] {}
+ child [missing] {}
+ child [missing] {}
+ child [missing] {}
+ child [missing] {}
+ child [missing] {}
+ child { node {\type{\{ IType \}*}}}
+ }
+ child [missing] {}
+ child [missing] {}
+ child [missing] {}
+ child [missing] {}
+ child [missing] {}
+ child [missing] {}
+ child [missing] {}
+ child [missing] {}
+ child [missing] {}
+ child { node {\type{\{ ICompilationUnit \}*}}}
+ }
+ child [missing] {}
+ child [missing] {}
+ child [missing] {}
+ child [missing] {}
+ child [missing] {}
+ child [missing] {}
+ child [missing] {}
+ child [missing] {}
+ child [missing] {}
+ child [missing] {}
+ child [missing] {}
+ child { node {\type{\{ IPackageFragment \}*}}}
+ }
+ child [missing] {}
+ child [missing] {}
+ child [missing] {}
+ child [missing] {}
+ child [missing] {}
+ child [missing] {}
+ child [missing] {}
+ child [missing] {}
+ child [missing] {}
+ child [missing] {}
+ child [missing] {}
+ child [missing] {}
+ child [missing] {}
+ child { node {\type{\{ IPackageFragmentRoot \}*}}}
+ ;
+ \end{tikzpicture}
+ \caption{The Java model of \name{Eclipse}. ``\type{\{ SomeElement \}*}'' means
+ \type{SomeElement} zero or more times. For recursive structures,
+ ``\type{\ldots}'' is used.}
+ \label{fig:javaModel}
+\end{figure}
+
+\section{The Abstract Syntax Tree}
+\name{Eclipse} is following the common paradigm of using an abstract syntax tree for
+source code analysis and manipulation.
+
+When parsing program source code into something that can be used as a foundation
+for analysis, the start of the process follows the same steps as in a compiler.
+This is all natural, because the way a compiler analyzes code is no different
+from how source manipulation programs would do it, except for some properties of
+code that is analyzed in the parser, and that they may be differing in what
+kinds of properties they analyze. Thus the process of translation source code
+into a structure that is suitable for analyzing, can be seen as a kind of
+interrupted compilation process \see{fig:interruptedCompilationProcess}.
+
+\begin{figure}[h]
+ \centering
+ \tikzset{
+ base/.style={anchor=north, align=center, rectangle, minimum height=1.4cm},
+ basewithshadow/.style={base, drop shadow, fill=white},
+ outlined/.style={basewithshadow, draw, rounded corners, minimum
+ width=0.4cm},
+ primary/.style={outlined, font=\bfseries},
+ dashedbox/.style={outlined, dashed},
+ arrowpath/.style={black, align=center, font=\small},
+ processarrow/.style={arrowpath, ->, >=angle 90, shorten >=1pt},
+ }
+ \begin{tikzpicture}[node distance=1.3cm and 3cm, scale=1, every
+ node/.style={transform shape}]
+ \node[base](AuxNode1){\small source code};
+ \node[primary, right=of AuxNode1, xshift=-2.5cm](Scanner){Scanner};
+ \node[primary, right=of Scanner, xshift=0.5cm](Parser){Parser};
+ \node[dashedbox, below=of Parser](SemanticAnalyzer){Semantic\\Analyzer};
+ \node[dashedbox, left=of SemanticAnalyzer](SourceCodeOptimizer){Source
+ Code\\Optimizer};
+ \node[dashedbox, below=of SourceCodeOptimizer
+ ](CodeGenerator){Code\\Generator};
+ \node[dashedbox, right=of CodeGenerator](TargetCodeOptimizer){Target
+ Code\\Optimizer};
+ \node[base, right=of TargetCodeOptimizer](AuxNode2){};
+
+ \draw[processarrow](AuxNode1) -- (Scanner);
+
+ \path[arrowpath] (Scanner) -- node [sloped](tokens){tokens}(Parser);
+ \draw[processarrow](Scanner) -- (tokens) -- (Parser);
+
+ \path[arrowpath] (Parser) -- node (syntax){syntax
+ tree}(SemanticAnalyzer);
+ \draw[processarrow](Parser) -- (syntax) -- (SemanticAnalyzer);
+
+ \path[arrowpath] (SemanticAnalyzer) -- node
+ [sloped](annotated){annotated\\tree}(SourceCodeOptimizer);
+ \draw[processarrow, dashed](SemanticAnalyzer) -- (annotated) --
+ (SourceCodeOptimizer);
+
+ \path[arrowpath] (SourceCodeOptimizer) -- node
+ (intermediate){intermediate code}(CodeGenerator);
+ \draw[processarrow, dashed](SourceCodeOptimizer) -- (intermediate) --
+ (CodeGenerator);
+
+ \path[arrowpath] (CodeGenerator) -- node [sloped](target1){target
+ code}(TargetCodeOptimizer);
+ \draw[processarrow, dashed](CodeGenerator) -- (target1) --
+ (TargetCodeOptimizer);
+
+ \path[arrowpath](TargetCodeOptimizer) -- node [sloped](target2){target
+ code}(AuxNode2);
+ \draw[processarrow, dashed](TargetCodeOptimizer) -- (target2) (AuxNode2);
+ \end{tikzpicture}
+ \caption{Interrupted compilation process. {\footnotesize (Full compilation
+ process borrowed from \emph{Compiler construction: principles and practice}
+ by Kenneth C. Louden\citing{louden1997}.)}}
+ \label{fig:interruptedCompilationProcess}
+\end{figure}
+
+The process starts with a \emph{scanner}, or lexer. The job of the scanner is to
+read the source code and divide it into tokens for the parser. Therefore, it is
+also sometimes called a tokenizer. A token is a logical unit, defined in the
+language specification, consisting of one or more consecutive characters. In
+the Java language the tokens can for instance be the \var{this} keyword, a curly
+bracket \var{\{} or a \var{nameToken}. It is recognized by the scanner on the
+basis of something equivalent of a regular expression. This part of the process
+is often implemented with the use of a finite automata. In fact, it is common to
+specify the tokens in regular expressions, that in turn is translated into a
+finite automata lexer. This process can be automated.
+
+The program component used to translate a stream of tokens into something
+meaningful, is called a parser. A parser is fed tokens from the scanner and
+performs an analysis of the structure of a program. It verifies that the syntax
+is correct according to the grammar rules of a language, that is usually
+specified in a context-free grammar, and often in a variant of the
+\name{Backus--Naur
+Form}\footnote{\url{https://en.wikipedia.org/wiki/Backus-Naur\_Form}}. The
+result coming from the parser is in the form of an \emph{Abstract Syntax Tree},
+AST for short. It is called \emph{abstract}, because the structure does not
+contain all of the tokens produced by the scanner. It only contain logical
+constructs, and because it forms a tree, all kinds of parentheses and brackets
+are implicit in the structure. It is this AST that is used when performing the
+semantic analysis of the code.
+
+As an example we can think of the expression \code{(5 + 7) * 2}. The root of
+this tree would in \name{Eclipse} be an \type{InfixExpression} with the operator
+\var{TIMES}, and a left operand that is also an \type{InfixExpression} with the
+operator \var{PLUS}. The left operand \type{InfixExpression}, has in turn a left
+operand of type \type{NumberLiteral} with the value \var{``5''} and a right
+operand \type{NumberLiteral} with the value \var{``7''}. The root will have a
+right operand of type \type{NumberLiteral} and value \var{``2''}. The AST for
+this expression is illustrated in \myref{fig:astInfixExpression}.
+
+Contrary to the Java Model, an abstract syntax tree is a heavy-weight
+representation of source code. It contains information about properties like
+type bindings for variables and variable bindings for names.
+
+
+\begin{figure}[h]
+ \centering
+ \begin{tikzpicture}[scale=0.8]
+ \tikzset{level distance=40pt}
+ \tikzset{sibling distance=5pt}
+ \tikzstyle{thescale}=[scale=0.8]
+ \tikzset{every tree node/.style={align=center}}
+ \tikzset{edge from parent/.append style={thick}}
+ \tikzstyle{inode}=[rectangle,rounded corners,draw,fill=lightgray,drop
+ shadow,align=center]
+ \tikzset{every internal node/.style={inode}}
+ \tikzset{every leaf node/.style={draw=none,fill=none}}
+
+ \Tree [.\type{InfixExpression} [.\type{InfixExpression}
+ [.\type{NumberLiteral} \var{``5''} ] [.\type{Operator} \var{PLUS} ]
+ [.\type{NumberLiteral} \var{``7''} ] ]
+ [.\type{Operator} \var{TIMES} ]
+ [.\type{NumberLiteral} \var{``2''} ]
+ ]
+ \end{tikzpicture}
+ \caption{The abstract syntax tree for the expression \code{(5 + 7) * 2}.}
+ \label{fig:astInfixExpression}
+\end{figure}
+
+\subsection{The AST in Eclipse}\label{astEclipse}
+In \name{Eclipse}, every node in the AST is a child of the abstract superclass
+\typewithref{org.eclipse.jdt.core.dom}{ASTNode}. Every \type{ASTNode}, among a
+lot of other things, provides information about its position and length in the
+source code, as well as a reference to its parent and to the root of the tree.
+
+The root of the AST is always of type \type{CompilationUnit}. It is not the same
+as an instance of an \type{ICompilationUnit}, which is the compilation unit
+handle of the Java model. The children of a \type{CompilationUnit} is an
+optional \type{PackageDeclaration}, zero or more nodes of type
+\type{ImportDecaration} and all its top-level type declarations that has node
+types \type{AbstractTypeDeclaration}.
+
+An \type{AbstractType\-Declaration} can be one of the types
+\type{AnnotationType\-Declaration}, \type{Enum\-Declaration} or
+\type{Type\-Declaration}. The children of an \type{AbstractType\-Declaration}
+must be a subtype of a \type{BodyDeclaration}. These subtypes are:
+\type{AnnotationTypeMember\-Declaration}, \type{EnumConstant\-Declaration},
+\type{Field\-Declaration}, \type{Initializer} and \type{Method\-Declaration}.
+
+Of the body declarations, the \type{Method\-Declaration} is the most interesting
+one. Its children include lists of modifiers, type parameters, parameters and
+exceptions. It has a return type node and a body node. The body, if present, is
+of type \type{Block}. A \type{Block} is itself a \type{Statement}, and its
+children is a list of \type{Statement} nodes.
+
+There are too many types of the abstract type \type{Statement} to list up, but
+there exists a subtype of \type{Statement} for every statement type of Java, as
+one would expect. This also applies to the abstract type \type{Expression}.
+However, the expression \type{Name} is a little special, since it is both used
+as an operand in compound expressions, as well as for names in type declarations
+and such.
+
+There is an overview of some of the structure of an \name{Eclipse} AST in
+\myref{fig:astEclipse}.
+
+\begin{figure}[h]
+ \centering
+ \begin{tikzpicture}[scale=0.8]
+ \tikzset{level distance=50pt}
+ \tikzset{sibling distance=5pt}
+ \tikzstyle{thescale}=[scale=0.8]
+ \tikzset{every tree node/.style={align=center}}
+ \tikzset{edge from parent/.append style={thick}}
+ \tikzstyle{inode}=[rectangle,rounded corners,draw,fill=lightgray,drop
+ shadow,align=center]
+ \tikzset{every internal node/.style={inode}}
+ \tikzset{every leaf node/.style={draw=none,fill=none}}
+
+ \Tree [.\type{CompilationUnit} [.\type{[ PackageDeclaration ]} [.\type{Name} ]
+ [.\type{\{ Annotation \}*} ] ]
+ [.\type{\{ ImportDeclaration \}*} [.\type{Name} ] ]
+ [.\type{\{ AbstractTypeDeclaration \}+} [.\node(site){\type{\{
+ BodyDeclaration \}*}}; ] [.\type{SimpleName} ] ]
+ ]
+ \begin{scope}[shift={(0.5,-6)}]
+ \node[inode,thescale](root){\type{MethodDeclaration}};
+ \node[inode,thescale](modifiers) at (4.5,-5){\type{\{ IExtendedModifier \}*}
+ \\ {\footnotesize (Of type \type{Modifier} or \type{Annotation})}};
+ \node[inode,thescale](typeParameters) at (-6,-3.5){\type{\{ TypeParameter
+ \}*}};
+ \node[inode,thescale](parameters) at (-5,-5){\type{\{
+ SingleVariableDeclaration \}*} \\ {\footnotesize (Parameters)}};
+ \node[inode,thescale](exceptions) at (5,-3){\type{\{ Name \}*} \\
+ {\footnotesize (Exceptions)}};
+ \node[inode,thescale](return) at (-6.5,-2){\type{Type} \\ {\footnotesize
+ (Return type)}};
+ \begin{scope}[shift={(0,-5)}]
+ \Tree [.\node(body){\type{[ Block ]} \\ {\footnotesize (Body)}};
+ [.\type{\{ Statement \}*} [.\type{\{ Expression \}*} ]
+ [.\type{\{ Statement \}*} [.\type{\ldots} ]]
+ ]
+ ]
+ \end{scope}
+ \end{scope}
+ \draw[->,>=triangle 90,shorten >=1pt](root.east)..controls +(east:2) and
+ +(south:1)..(site.south);
+
+ \draw (root.south) -- (modifiers);
+ \draw (root.south) -- (typeParameters);
+ \draw (root.south) -- ($ (parameters.north) + (2,0) $);
+ \draw (root.south) -- (exceptions);
+ \draw (root.south) -- (return);
+ \draw (root.south) -- (body);
+
+ \end{tikzpicture}
+ \caption{The format of the abstract syntax tree in \name{Eclipse}.}
+ \label{fig:astEclipse}
+\end{figure}
+\todoin{Add more to the AST format tree? \myref{fig:astEclipse}}
+
+\section{The ASTVisitor}\label{astVisitor}
+So far, the only thing that has been addressed is how the data that is going to
+be the basis for our analysis is structured. Another aspect of it is how we are
+going to traverse the AST to gather the information we need, so we can conclude
+about the properties we are analysing. It is of course possible to start at the
+top of the tree, and manually search through its nodes for the ones we are
+looking for, but that is a bit inconvenient. To be able to efficiently utilize
+such an approach, we would need to make our own framework for traversing the
+tree and visiting only the types of nodes we are after. Luckily, this
+functionality is already provided in \name{Eclipse}, by its
+\typewithref{org.eclipse.jdt.core.dom}{ASTVisitor}.
+
+The \name{Eclipse} AST, together with its \type{ASTVisitor}, follows the
+\pattern{Visitor} pattern\citing{designPatterns}. The intent of this design
+pattern is to facilitate extending the functionality of classes without touching
+the classes themselves.
+
+Let us say that there is a class hierarchy of elements. These elements all have
+a method \method{accept(Visitor visitor)}. In its simplest form, the
+\method{accept} method just calls the \method{visit} method of the visitor with
+itself as an argument, like this: \code{visitor.visit(this)}. For the visitors
+to be able to extend the functionality of all the classes in the elements
+hierarchy, each \type{Visitor} must have one visit method for each concrete
+class in the hierarchy. Say the hierarchy consists of the concrete classes
+\type{ConcreteElementA} and \type{ConcreteElementB}. Then each visitor must have
+the (possibly empty) methods \method{visit(ConcreteElementA element)} and
+\method{visit(ConcreteElementB element)}. This scenario is depicted in
+\myref{fig:visitorPattern}.
+
+\begin{figure}[h]
+ \centering
+ \tikzstyle{abstract}=[rectangle, draw=black, fill=white, drop shadow, text
+ centered, anchor=north, text=black, text width=6cm, every one node
+part/.style={align=center, font=\bfseries\itshape}]
+ \tikzstyle{concrete}=[rectangle, draw=black, fill=white, drop shadow, text
+ centered, anchor=north, text=black, text width=6cm]
+ \tikzstyle{inheritarrow}=[->, >=open triangle 90, thick]
+ \tikzstyle{commentarrow}=[->, >=angle 90, dashed]
+ \tikzstyle{line}=[-, thick]
+ \tikzset{every one node part/.style={align=center, font=\bfseries}}
+ \tikzset{every second node part/.style={align=center, font=\ttfamily}}
+
+ \begin{tikzpicture}[node distance=1cm, scale=0.8, every node/.style={transform
+ shape}]
+ \node (Element) [abstract, rectangle split, rectangle split parts=2]
+ {
+ \nodepart{one}{Element}
+ \nodepart{second}{+accept(visitor: Visitor)}
+ };
+ \node (AuxNode01) [text width=0, minimum height=2cm, below=of Element] {};
+ \node (ConcreteElementA) [concrete, rectangle split, rectangle split
+ parts=2, left=of AuxNode01]
+ {
+ \nodepart{one}{ConcreteElementA}
+ \nodepart{second}{+accept(visitor: Visitor)}
+ };
+ \node (ConcreteElementB) [concrete, rectangle split, rectangle split
+ parts=2, right=of AuxNode01]
+ {
+ \nodepart{one}{ConcreteElementB}
+ \nodepart{second}{+accept(visitor: Visitor)}
+ };
+
+ \node[comment, below=of ConcreteElementA] (CommentA) {visitor.visit(this)};
+
+ \node[comment, below=of ConcreteElementB] (CommentB) {visitor.visit(this)};
+
+ \node (AuxNodeX) [text width=0, minimum height=1cm, below=of AuxNode01] {};
+
+ \node (Visitor) [abstract, rectangle split, rectangle split parts=2,
+ below=of AuxNodeX]
+ {
+ \nodepart{one}{Visitor}
+ \nodepart{second}{+visit(ConcreteElementA)\\+visit(ConcreteElementB)}
+ };
+ \node (AuxNode02) [text width=0, minimum height=2cm, below=of Visitor] {};
+ \node (ConcreteVisitor1) [concrete, rectangle split, rectangle split
+ parts=2, left=of AuxNode02]
+ {
+ \nodepart{one}{ConcreteVisitor1}
+ \nodepart{second}{+visit(ConcreteElementA)\\+visit(ConcreteElementB)}
+ };
+ \node (ConcreteVisitor2) [concrete, rectangle split, rectangle split
+ parts=2, right=of AuxNode02]
+ {
+ \nodepart{one}{ConcreteVisitor2}
+ \nodepart{second}{+visit(ConcreteElementA)\\+visit(ConcreteElementB)}
+ };
+
+
+ \draw[inheritarrow] (ConcreteElementA.north) -- ++(0,0.7) -|
+ (Element.south);
+ \draw[line] (ConcreteElementA.north) -- ++(0,0.7) -|
+ (ConcreteElementB.north);
+
+ \draw[inheritarrow] (ConcreteVisitor1.north) -- ++(0,0.7) -|
+ (Visitor.south);
+ \draw[line] (ConcreteVisitor1.north) -- ++(0,0.7) -|
+ (ConcreteVisitor2.north);
+
+ \draw[commentarrow] (CommentA.north) -- (ConcreteElementA.south);
+ \draw[commentarrow] (CommentB.north) -- (ConcreteElementB.south);
+
+
+ \end{tikzpicture}
+ \caption{The Visitor Pattern.}
+ \label{fig:visitorPattern}
+\end{figure}
+
+The use of the visitor pattern can be appropriate when the hierarchy of elements
+is mostly stable, but the family of operations over its elements is constantly
+growing. This is clearly the case for the \name{Eclipse} AST, since the hierarchy of
+type \type{ASTNode} is very stable, but the functionality of its elements is
+extended every time someone needs to operate on the AST. Another aspect of the
+\name{Eclipse} implementation is that it is a public API, and the visitor pattern is an
+easy way to provide access to the nodes in the tree.
+
+The version of the visitor pattern implemented for the AST nodes in \name{Eclipse} also
+provides an elegant way to traverse the tree. It does so by following the
+convention that every node in the tree first let the visitor visit itself,
+before it also makes all its children accept the visitor. The children are only
+visited if the visit method of their parent returns \var{true}. This pattern
+then makes for a prefix traversal of the AST. If postfix traversal is desired,
+the visitors also has \method{endVisit} methods for each node type, that is
+called after the \method{visit} method for a node. In addition to these visit
+methods, there are also the methods \method{preVisit(ASTNode)},
+\method{postVisit(ASTNode)} and \method{preVisit2(ASTNode)}. The
+\method{preVisit} method is called before the type-specific \method{visit}
+method. The \method{postVisit} method is called after the type-specific
+\method{endVisit}. The type specific \method{visit} is only called if
+\method{preVisit2} returns \var{true}. Overriding the \method{preVisit2} is also
+altering the behavior of \method{preVisit}, since the default implementation is
+responsible for calling it.
+
+An example of a trivial \type{ASTVisitor} is shown in
+\myref{lst:astVisitorExample}.
+
+\begin{listing}
+\begin{minted}{java}
+public class CollectNamesVisitor extends ASTVisitor {
+ Collection<Name> names = new LinkedList<Name>();
+
+ @Override
+ public boolean visit(QualifiedName node) {
+ names.add(node);
+ return false;
+ }
+
+ @Override
+ public boolean visit(SimpleName node) {
+ names.add(node);
+ return true;
+ }
+}
+\end{minted}
+\caption{An \type{ASTVisitor} that visits all the names in a subtree and adds
+them to a collection, except those names that are children of any
+\type{QualifiedName}.}
+\label{lst:astVisitorExample}
+\end{listing}
+
+\section{Property collectors}\label{propertyCollectors}
+The prefixes and unfixes are found by property
+collectors\typeref{no.uio.ifi.refaktor.extractors.collectors.PropertyCollector}.
+A property collector is of the \type{ASTVisitor} type, and thus visits nodes of
+type \type{ASTNode} of the abstract syntax tree \see{astVisitor}.
+
+\subsection{The PrefixesCollector}
+The \typewithref{no.uio.ifi.refaktor.extractors.collectors}{PrefixesCollector}
+finds prefixes that makes up the basis for calculating move targets for the
+\refa{Extract and Move Method} refactoring. It visits expression
+statements\typeref{org.eclipse.jdt.core.dom.ExpressionStatement} and creates
+prefixes from its expressions in the case of method invocations. The prefixes
+found is registered with a prefix set, together with all its sub-prefixes.
+
+\subsection{The UnfixesCollector}\label{unfixes}
+The \typewithref{no.uio.ifi.refaktor.extractors.collectors}{UnfixesCollector}
+finds unfixes within a selection. That is prefixes that cannot be used as a
+basis for finding a move target in a refactoring.
+
+An unfix can be a name that is assigned to within a selection. The reason that
+this cannot be allowed, is that the result would be an assignment to the
+\type{this} keyword, which is not valid in Java \see{eclipse_bug_420726}.
+
+Prefixes that originates from variable declarations within the same selection
+are also considered unfixes. This is because when a method is moved, it needs to
+be called through a variable. If this variable is also within the method that is
+to be moved, this obviously cannot be done.
+
+Also considered as unfixes are variable references that are of types that is not
+suitable for moving a methods to. This can be either because it is not
+physically possible to move the method to the desired class or that it will
+cause compilation errors by doing so.
+
+If the type binding for a name is not resolved it is considered and unfix. The
+same applies to types that is only found in compiled code, so they have no
+underlying source that is accessible to us. (E.g. the \type{java.lang.String}
+class.)
+
+Interfaces types are not suitable as targets. This is simply because interfaces
+in Java cannot contain methods with bodies. (This thesis does not deal with
+features of Java versions later than Java 7. Java 8 has interfaces with default
+implementations of methods.) Neither are local types allowed. This accounts for
+both local and anonymous classes. Anonymous classes are effectively the same as
+interface types with respect to unfixes. Local classes could in theory be used
+as targets, but this is not possible due to limitations of the implementation of
+the \refa{Extract and Move Method} refactoring. The problem is that the refactoring is
+done in two steps, so the intermediate state between the two refactorings would
+not be legal Java code. In the case of local classes, the problem is that, in
+the intermediate step, a selection referencing a local class would need to take
+the local class as a parameter if it were to be extracted to a new method. This
+new method would need to live in the scope of the declaring class of the
+originating method. The local class would then not be in the scope of the
+extracted method, thus bringing the source code into an illegal state. One could
+imagine that the method was extracted and moved in one operation, without an
+intermediate state. Then it would make sense to include variables with types of
+local classes in the set of legal targets, since the local classes would then be
+in the scopes of the method calls. If this makes any difference for software
+metrics that measure coupling would be a different discussion.
+
+\begin{listing}
+\begin{multicols}{2}
+\begin{minted}[]{java}
+// Before
+void declaresLocalClass() {
+ class LocalClass {
+ void foo() {}
+ void bar() {}
+ }
+
+ LocalClass inst =
+ new LocalClass();
+ inst.foo();
+ inst.bar();
+}
+\end{minted}
+
+\columnbreak
+
+\begin{minted}[]{java}
+// After Extract Method
+void declaresLocalClass() {
+ class LocalClass {
+ void foo() {}
+ void bar() {}
+ }
+
+ LocalClass inst =
+ new LocalClass();
+ fooBar(inst);
+}
+
+// Intermediate step
+void fooBar(LocalClass inst) {
+ inst.foo();
+ inst.bar();
+}
+\end{minted}
+\end{multicols}
+\caption{When \refa{Extract and Move Method} tries to use a variable with a local type
+as the move target, an intermediate step is taken that is not allowed. Here:
+\type{LocalClass} is not in the scope of \method{fooBar} in its intermediate
+location.}
+\label{lst:extractMethod_LocalClass}
+\end{listing}
+
+The last class of names that are considered unfixes is names used in null tests.
+These are tests that reads like this: if \texttt{<name>} equals \var{null} then
+do something. If allowing variables used in those kinds of expressions as
+targets for moving methods, we would end up with code containing boolean
+expressions like \texttt{this == null}, which would not be meaningful, since
+\var{this} would never be \var{null}.
+
+
+\subsection{The ContainsReturnStatementCollector}
+The
+\typewithref{no.uio.ifi.refaktor.analyze.collectors}{ContainsReturnStatementCollector}
+is a very simple property collector. It only visits the return statements within
+a selection, and can report whether it encountered a return statement or not.
+
+\subsection{The LastStatementCollector}
+The \typewithref{no.uio.ifi.refaktor.analyze.collectors}{LastStatementCollector}
+collects the last statement of a selection. It does so by only visiting the top
+level statements of the selection, and compares the textual end offset of each
+encountered statement with the end offset of the previous statement found.
+
+\section{Checkers}\label{checkers}
+\todoin{Check out ExtractMethodAnalyzer from ExtractMethodRefactoring}
+The checkers are a range of classes that checks that text selections complies
+with certain criteria. All checkers operates under the assumption that the code
+they check is free from compilation errors. If a
+\typewithref{no.uio.ifi.refaktor.analyze.analyzers}{Checker} fails, it throws a
+\type{CheckerException}. The checkers are managed by the
+\type{LegalStatementsChecker}, which does not, in fact, implement the
+\type{Checker} interface. It does, however, run all the checkers registered with
+it, and reports that all statements are considered legal if no
+\type{CheckerException} is thrown. Many of the checkers either extends the
+\type{PropertyCollector} or utilizes one or more property collectors to verify
+some criteria. The checkers registered with the \type{LegalStatementsChecker}
+are described next. They are run in the order presented below.
+
+\subsection{The CallToProtectedOrPackagePrivateMethodChecker}
+This checker is designed to prevent an error that can occur in situations where
+a method is declared in one class, but overridden in another. If a text
+selection contains a call to a method like this, and the seletion is extracted
+to a new method, the subsequent movement of this method could cause the code to
+break.
+
+The code breaks in situations where the method call in the selection is to a
+method that has the \code{protected} modifier, or it does not have any access
+modifiers, i.e. it is package-private. The method is not public, so the
+\MoveMethod refactoring must make it public, making the moved method able to
+call it from its new location. The problem is that the, now public, method is
+overridden in a subclass, where it has a protected or package-private status.
+This makes the compiler complain that the subclass is trying to reduce the
+visibility of a method declared in its superclass. This is not allowed in Java,
+and for good reasons. It would make it possible to make a subclass that could
+not be a substitute for its superclass.
+
+The workings of the \type{CallToProtectedOrPackagePrivateMethod\-Checker} is
+therefore very simple. It looks for calls to methods that are either protected
+or package-private within the selection, and throws an
+\type{IllegalExpressionFoundException} if one is found.
+
+The problem this checker helps to avoid, is a little subtle. The problem does
+not arise in the class where the change is done, but in a class derived from it.
+This shows that classes acting as superclasses are especially fragile to
+introducing errors in the context of automated refactoring. This is also shown
+in bug\ldots \todoin{File Eclipse bug report}
+
+\subsection{The InstantiationOfNonStaticInnerClassChecker}
+When a non-static inner class is instatiated, this must happen in the scope of
+its declaring class. This is because it must have access to the members of the
+declaring class. If the inner class is public, it is possible to instantiate it
+through an instance of its declaring class, but this is not handled by the
+\type{MoveInstanceMethodProcessor} in Eclipse when moving a method. Therefore,
+performing a move on a method that instantiates a non-static inner class, will
+break the code if the instantiation is not handled properly. For this reason,
+the \type{InstantiationOfNonStaticInnerClassChecker} does not validate
+selections that contains instantiations of non-static inner classes. This
+problem is also related to bug\ldots \todoin{File Eclipse bug report}
+
+\subsection{The EnclosingInstanceReferenceChecker}
+The purpose of this checker is to verify that the names in a selection is not
+referencing any enclosing instances. This is for making sure that all references
+is legal in a method that is to be moved. Theoretically, some situations could
+be easily solved my passing a reference to the referenced class with the moved
+method (e.g. when calling public methods), but the dependency on the
+\type{MoveInstanceMethodProcessor} prevents this.
+
+The
+\typewithref{no.uio.ifi.refaktor.analyze.analyzers}{EnclosingInstanceReferenceChecker}
+is a modified version of the
+\typewithref{org.eclipse.jdt.internal.corext.refactoring.structure.MoveInstanceMethod\-Processor}{EnclosingInstanceReferenceFinder}
+from the \type{MoveInstanceMethodProcessor}. Wherever the
+\type{EnclosingInstanceReferenceFinder} would create a fatal error status, the
+checker throws a \type{CheckerException}.
+
+It works by first finding all of the enclosing types of a selection. Thereafter
+it visits all its simple names to check that they are not references to
+variables or methods declared in any of the enclosing types. In addition the
+checker visits \var{this}-expressions to verify that no such expressions is
+qualified with any name.
+
+\subsection{The ReturnStatementsChecker}\label{returnStatementsChecker}
+The checker for return statements is meant to verify that if a text selection
+contains a return statement, then every possible execution path within the
+selection ends in a return statement. This property is important regarding the
+\ExtractMethod refactoring. If it holds, it means that a method could be
+extracted from the selection, and a call to it could be substituted for the
+selection. If the method has a non-void return type, then a call to it would
+also be a valid return point for the calling method. If its return value is of
+the void type, then the \type{ExtractMethodRefactoring} of \name{Eclipse}
+appends an empty return statement to the back of the method call. Therefore, the
+analysis does not discriminate on either kinds of return statements, with or
+without a return value.
+
+The property description implies that if the selection is free from return
+statements, then the checker validates. So this is the first thing the checker
+investigates.
+
+If the checker proceedes any further, it is because the selection contains one
+or more return statements. The next test is therefore to check if the last
+statement of the selection ends in either a return or a throw statement. If the
+last statement of the selection ends in a return statement, then all execution
+paths within the selection should end in either this, or another, return
+statement. This is also true for a throw statement, since it causes an immediate
+exit from the current block, together with all outer blocks in its control flow
+that does not catch the thrown exception.
+
+Return statements can be either explicit or implicit. An \emph{explicit} return
+statement is formed by using the \code{return} keyword, while an \emph{implicit}
+return statement is a statement that is not formed by the \code{return} keyword,
+but must be the last statement of a method that can have any side effects. This
+can happen in methods with a void return type. An example is a statement that is
+inside one or more blocks. The last statement of a method could for instance be
+an if-statement, but the last statement that is executed in the method, and that
+can have any side effects, may be located inside the block of the else part of
+the if-statement.
+
+The responsibility for checking that the last statement of the selection
+eventually ends in a return or throw statement, is put on the
+\type{LastStatementOfSelectionEndsInReturnOrThrowChecker}. For every node
+visited, if it is a statement, it does a test to see if the statement is a
+return, a throw or if it is an implicit return statement. If this is the case,
+no further checking is done. This checking is done in the \code{preVisit2}
+method \see{astVisitor}. If the node is not of a type that is being handled by
+its type specific visit method, the checker performs a simple test. If the node
+being visited is not the last statement of its parent that is also enclosed by
+the selection, an \type{IllegalStatementFoundException} is thrown. This ensures
+that all statements are taken care of, one way or the other. It also ensures
+that the checker is conservative in the way it checks for legality of the
+selection.
+
+To examine if a statement is an implicit return statement, the checker first
+finds the last statement declared in its enclosing method. If this statement is
+the same as the one under investigation, it is considered an implicit return
+statement. If the statements are not the same, the checker does a search to see
+if statement examined is also the last statement of the method that can be
+reached. This includes the last statement of a block statement, a labeled
+statement, a synchronized statement or a try statement, that in turn is the last
+statement enclosed by the statement types listed. This search goes through all
+the parents of a statement until a statement is found that is not one of the
+mentioned acceptable parent statements. If the search ends in a method
+declaration, then the statement is considered to be the last reachable statement
+of the method, and thus also an implicit return statement.
+
+There are two kinds of statements that are handled explicitly. It is
+if-statements and try-statements. Block, labeled and do-statements are handled
+by fall-through to the other two. Do-statements are considered equal to blocks
+in this context, since their bodies are always evaluated at least one time. If-
+and try-statements are visited only if they are the last node of their parent
+within the selection.
+
+For if-statements, the rule is that if the then-part does not contain any return
+or throw statements, it is considered illegal. If it does contain a return or
+throw, its else-part is checked. If the else-part is non-existent, or it does
+not contain any return or throw statements, it is considered illegal. If the
+statement is not regarded illegal, its children are visited.
+
+Try-statements are handled much the same way as if-statements. Its body must
+contain a return or throw. The same applies to its catch clauses and finally
+body.
+
+If the checker does not complain at any point, the selection is considered valid
+with respect to return statements.
+
+\subsection{The AmbiguousReturnValueChecker}
+This checker verifies that there are no \emph{ambiguous return statements} in a
+selection. The problem with ambiguous return statements arise when a selection
+is chosen to be extracted into a new method, but it needs to return more than
+one value from that method. This problem occurs in two situations. The first
+situation arise when there is more than one local variable that is both assigned
+to within a selection and also referenced after the selection. The other
+situation occur when there is only one such assignment, but there is also one or
+more return statements in the selection.
+
+First the checker needs to collect some data. Those data are the binding keys
+for all simple names that are assigned to within the selection, including
+variable declarations, but excluding fields. The checker also collects whether
+there exists a return statement in the selection or not. No further checks of
+return statements are needed, since, at this point, the selection is already
+checked for illegal return statements \see{returnStatementsChecker}.
+
+After the binding keys of the assignees are collected, the checker searches the
+part of the enclosing method that is after the selection for references whose
+binding keys are among the collected keys. If more than one unique referral is
+found, or only one referral is found, but the selection also contains a return
+statement, we have a situation with an ambiguous return value, and an exception
+is thrown.
+
+%\todoin{Explain why we do not need to consider variables assigned inside
+%local/anonymous classes. (The referenced variables need to be final and so
+%on\ldots)}
+
+\subsection{The IllegalStatementsChecker}
+This checker is designed to check for illegal statements.
+
+Any use of the \var{super} keyword is prohibited, since its meaning is altered
+when moving a method to another class.
+
+For a \emph{break} statement, there is two situations to consider: A break
+statement with or without a label. If the break statement has a label, it is
+checked that whole of the labeled statement is inside the selection. Since a
+label does not have any binding information, we have to search upwards in the
+AST to find the \type{LabeledStatement} that corresponds to the label from the
+break statement, and check that it is contained in the selection. If the break
+statement does not have a label attached to it, it is checked that its innermost
+enclosing loop or switch statement also is inside the selection.
+
+The situation for a \emph{continue} statement is the same as for a break
+statement, except that it is not allowed inside switch statements.
+
+Regarding \emph{assignments}, two types of assignments is allowed: Assignment to
+a non-final variable and assignment to an array access. All other assignments is
+regarded illegal.
+
+\todoin{Finish\ldots}
+
+
+\chapter{Benchmarking}
+\todoin{Better name than ``benchmarking''?}
+This part of the master project is located in the \name{Eclipse} project
+\code{no.uio.ifi.refaktor.benchmark}. The purpose of it is to run the equivalent
+of the \type{SearchBasedExtractAndMoveMethodChanger}
+\see{searchBasedExtractAndMoveMethodChanger} over a larger software project,
+both to test its robustness but also its effect on different software metrics.
+
+\section{The benchmark setup}
+The benchmark itself is set up as a \name{JUnit} test case. This is a convenient
+setup, and utilizes the \name{JUnit Plugin Test Launcher}. This provides us a
+with a fully functional \name{Eclipse} workbench. Most importantly, this gives
+us access to the Java Model of \name{Eclipse} \see{javaModel}.
+
+\subsection{The ProjectImporter}
+The Java project that is going to be used as the data for the benchmark, must be
+imported into the JUnit workspace. This is done by the
+\typewithref{no.uio.ifi.refaktor.benchmark}{ProjectImporter}. The importer
+require the absolute path to the project description file. It is named
+\code{.project} and is located at the root of the project directory.
+
+The project description is loaded to find the name of the project to be
+imported. The project that shall be the destination for the import is created in
+the workspace, on the base of the name from the description. Then an import
+operation is created, based on both the source and destination information. The
+import operation is run to perform the import.
+
+I have found no simple API call to accomplish what the importer does, which
+tells me that it may not be too many people performing this particular action.
+The solution to the problem was found on \name{Stack
+Overflow}\footnote{\url{https://stackoverflow.com/questions/12401297}}. It
+contains enough dirty details to be considered inconvenient to use, if not
+wrapping it in a class like my \type{ProjectImporter}. One would probably have
+to delve into the source code for the import wizard to find out how the import
+operation works, if no one had already done it.
+
+\section{Statistics}
+Statistics for the analysis and changes is captured by the
+\typewithref{no.uio.ifi.refaktor.aspects}{StatisticsAspect}. This an
+\emph{aspect} written in \name{AspectJ}.
+
+\subsection{AspectJ}
+\name{AspectJ}\footnote{\url{http://eclipse.org/aspectj/}} is an extension to
+the Java language, and facilitates combining aspect-oriented programming with
+the object-oriented programming in Java.
+
+Aspect-oriented programming is a programming paradigm that is meant to isolate
+so-called \emph{cross-cutting concerns} into their own modules. These
+cross-cutting concerns are functionalities that spans over multiple classes, but
+may not belong naturally in any of them. It can be functionality that does not
+concern the business logic of an application, and thus may be a burden when
+entangled with parts of the source code it does not really belong. Examples
+include logging, debugging, optimization and security.
+
+Aspects are interacting with other modules by defining advices. The concept of
+an \emph{advice} is known from both aspect-oriented and functional
+programming\citing{wikiAdvice2014}. It is a function that modifies another
+function when the latter is run. An advice in AspectJ is somewhat similar to a
+method in Java. It is meant to alter the behavior of other methods, and contains
+a body that is executed when it is applied.
+
+An advice can be applied at a defined \emph{pointcut}. A pointcut picks out one
+or more \emph{join points}. A join point is a well-defined point in the
+execution of a program. It can occur when calling a method defined for a
+particular class, when calling all methods with the same name,
+accessing/assigning to a particular field of a given class and so on. An advice
+can be declared to run both before, after returning from a pointcut, when there
+is thrown an exception in the pointcut or after the pointcut either returns or
+throws an exception. In addition to picking out join points, a pointcut can
+also bind variables from its context, so they can be accessed in the body of an
+advice. An example of a pointcut and an advice is found in
+\myref{lst:aspectjExample}.
+
+\begin{listing}[h]
+\begin{minted}{aspectj}
+pointcut methodAnalyze(
+ SearchBasedExtractAndMoveMethodAnalyzer analyzer) :
+ call(* SearchBasedExtractAndMoveMethodAnalyzer.analyze())
+ && target(analyzer);
+
+after(SearchBasedExtractAndMoveMethodAnalyzer analyzer) :
+ methodAnalyze(analyzer) {
+ statistics.methodCount++;
+ debugPrintMethodAnalysisProgress(analyzer.method);
+}
+\end{minted}
+\caption{An example of a pointcut named \method{methodAnalyze},
+and an advice defined to be applied after it has occurred.}
+\label{lst:aspectjExample}
+\end{listing}
+
+\subsection{The Statistics class}
+The statistics aspect stores statistical information in an object of type
+\type{Statistics}. As of now, the aspect needs to be initialized at the point in
+time where it is desired that it starts its data gathering. At any point in time
+the statistics aspect can be queried for a snapshot of the current statistics.
+
+The \type{Statistics} class also include functionality for generating a report
+of its gathered statistics. The report can be given either as a string or it can
+be written to a file.
+
+\subsection{Advices}
+The statistics aspect contains advices for gathering statistical data from
+different parts of the benchmarking process. It captures statistics from both
+the analysis part and the execution part of the composite \ExtractAndMoveMethod
+refactoring.
+
+For the analysis part, there are advices to count the number of text selections
+analyzed and the number of methods, types, compilation units and packages
+analyzed. There are also advices that counts for how many of the methods there
+is found a selection that is a candidate for the refactoring, and for how many
+methods there is not.
+
+There exists advices for counting both the successful and unsuccessful
+executions of all the refactorings. Both for the \ExtractMethod and \MoveMethod
+refactorings in isolation, as well as for the combination of them.
+
+\section{Optimizations}
+When looking for optimizations to make for the benchmarking process, I used the
+\name{VisualVM}\footnote{\url{http://visualvm.java.net/}} for the Java Virtual
+Machine to both profile the application and also to make memory dumps of its
+heap.
+
+\subsection{Caching}
+When profiling the benchmark process before making any optimizations, it early
+became apparent that the parsing of source code was a place to direct attention
+towards. This discovery was done when only \emph{analyzing} source code, before
+trying to do any \emph{manipulation} of it. Caching of the parsed ASTs seemed
+like the best way to save some time, as expected. With only a simple cache of
+the most recently used AST, the analysis time was speeded up by a factor of
+around
+20. This number depends a little upon which type of system the analysis was
+run.
+
+The caching is managed by a cache manager, that now, by default, utilizes the
+not so well known feature of Java called a \emph{soft reference}. Soft
+references are best explained in the context of weak references. A \emph{weak
+reference} is a reference to an object instance that is only guaranteed to
+persist as long as there is a \emph{strong reference} or a soft reference
+referring the same object. If no such reference is found, its referred object is
+garbage collected. A strong reference is basically the same as a regular Java
+reference. A soft reference has the same guarantees as a week reference when it
+comes to its relation to strong references, but it is not necessarily garbage
+collected whenever there exists no strong references to it. A soft reference
+\emph{may} reside in memory as long as the JVM has enough free memory in the
+heap. A soft reference will therefore usually perform better than a weak
+reference when used for simple caching and similar tasks. The way to use a
+soft/weak reference is to as it for its referent. The return value then has to
+be tested to check that it is not \var{null}. For the basic usage of soft
+references, see \myref{lst:softReferenceExample}. For a more thorough
+explanation of weak references in general, see\citing{weakRef2006}.
+
+\begin{listing}[h]
+\begin{minted}{java}
+// Strong reference
+Object strongRef = new Object();
+
+// Soft reference
+SoftReference<Object> softRef =
+ new SoftReference<Object>(new Object());
+
+// Using the soft reference
+Object obj = softRef.get();
+if (obj != null) {
+ // Use object here
+}
+\end{minted}
+\caption{Showing the basic usage of soft references. Weak references is used the
+ same way. {\footnotesize (The references are part of the \code{java.lang.ref}
+package.)}}
+\label{lst:softReferenceExample}
+\end{listing}
+
+The cache based on soft references has no limit for how many ASTs it caches. It
+is generally not advisable to keep references to ASTs for prolonged periods of
+time, since they are expensive structures to hold on to. For regular plugin
+development, \name{Eclipse} recommends not creating more than one AST at a time to
+limit memory consumption. Since the benchmarking has nothing to do with user
+experience, and throughput is everything, these advices are intentionally
+ignored. This means that during the benchmarking process, the target \name{Eclipse}
+application may very well work close to its memory limit for the heap space for
+long periods during the benchmark.
+
+\subsection{Memento}
+\todoin{Write}
+
+
+\chapter{Technicalities}
+
+\section{Source code organization}
+All the parts of this master project is under version control with
+\name{Git}\footnote{\url{http://git-scm.com/}}.
+
+The software written is organized as some \name{Eclipse} plugins. Writing a plugin is
+the natural way to utilize the API of \name{Eclipse}. This also makes it possible to
+provide a user interface to manually run operations on selections in program
+source code or whole projects/packages.
+
+When writing a plugin in \name{Eclipse}, one has access to resources such as the
+current workspace, the open editor and the current selection.
+
+The thesis work is contained in the following Eclipse projects:
+
+\begin{description}
+ \item[no.uio.ifi.refaktor] \hfill \\ This is the main Eclipse plugin
+ project, and contains all of the business logic for the plugin.
+
+ \item[no.uio.ifi.refaktor.tests] \hfill \\
+ This project contains the tests for the main plugin.
+
+ \item[no.uio.ifi.refaktor.examples] \hfill \\
+ Contains example code used in testing. It also contains code for managing
+ this example code, such as creating an Eclipse project from it before a test
+ run.
+
+ \item[no.uio.ifi.refaktor.benchmark] \hfill \\
+ This project contains code for running search based versions of the
+ composite refactoring over selected Eclipse projects.
+
+ \item[no.uio.ifi.refaktor.releng] \hfill \\
+ Contains the rmap, queries and target definitions needed by by Buckminster
+ on the Jenkins continuous integration server.
+
+\end{description}
+
+\subsection{The no.uio.ifi.refaktor project}
+
+\subsubsection{no.uio.ifi.refaktor.analyze}
+This package, and its subpackages, contains code that is used for analyzing Java
+source code. The most important subpackages are presented below.
+
+\begin{description}
+ \item[no.uio.ifi.refaktor.analyze.analyzers] \hfill \\
+ This package contains source code analyzers. These are usually responsible
+ for analyzing text selections or running specialized analyzers for different
+ kinds of entities. Their structure are often hierarchical. This means that
+ you have an analyzer for text selections, that in turn is utilized by an
+ analyzer that analyzes all the selections of a method. Then there are
+ analyzers for analyzing all the methods of a type, all the types of a
+ compilation unit, all the compilation units of a package, and, at last, all
+ of the packages in a project.
+
+ \item[no.uio.ifi.refaktor.analyze.checkers] \hfill \\
+ A package containing checkers. The checkers are classes used to validate
+ that a selection can be further analyzed and chosen as a candidate for a
+ refactoring. Invalidating properties can be such as usage of inner classes
+ or the need for multiple return values.
+
+ \item[no.uio.ifi.refaktor.analyze.collectors] \hfill \\
+ This package contains the property collectors. Collectors are used to gather
+ properties from a text selection. This is mostly properties regarding
+ referenced names and their occurrences. It is these properties that makes up
+ the basis for finding the best candidates for a refactoring.
+\end{description}
+
+\subsubsection{no.uio.ifi.refaktor.change}
+This package, and its subpackages, contains functionality for manipulate source
+code.
+
+\begin{description}
+ \item[no.uio.ifi.refaktor.change.changers] \hfill \\
+ This package contains source code changers. They are used to glue together
+ the analysis of source code and the actual execution of the changes.
+
+ \item[no.uio.ifi.refaktor.change.executors] \hfill \\
+ The executors that are responsible for making concrete changes are found in
+ this package. They are mostly used to create and execute one or more Eclipse
+ refactorings.
+
+ \item[no.uio.ifi.refaktor.change.processors] \hfill \\
+ Contains a refactoring processor for the \MoveMethod refactoring. The code
+ is stolen and modified to fix a bug. The related bug is described in
+ \myref{eclipse_bug_429416}.
+
+\end{description}
+
+\subsubsection{no.uio.ifi.refaktor.handlers}
+This package contains handlers for the commands defined in the plugin manifest.
+
+\subsubsection{no.uio.ifi.refaktor.prefix}
+This package contains the \type{Prefix} type that is the data representation of
+the prefixes found by the \type{PrefixesCollector}. It also contains the prefix
+set for storing and working with prefixes.
+
+\subsubsection{no.uio.ifi.refaktor.statistics}
+The package contains statistics functionality. Its heart is the statistics
+aspect that is responsible for gathering statistics during the execution of the
+\ExtractAndMoveMethod refactoring.
+
+\begin{description}
+ \item[no.uio.ifi.refaktor.statistics.reports] \hfill \\
+ This package contains a simple framework for generating reports from the
+ statistics data generated by the aspect. Currently, the only available
+ report type is a simple text report.
+
+\end{description}
+
+
+\subsubsection{no.uio.ifi.refaktor.textselection}
+This package contains the two custom text selections that are used extensively
+throughout the project. One of them is just a subclass of the other, to support
+the use of the memento pattern to optimize the memory usage during benchmarking.
+
+\subsubsection{no.uio.ifi.refaktor.debugging}
+The package contains a debug utility class. I addition to this, the package
+\code{no.uio.ifi.refaktor.utils.aspects} contains a couple of aspects used for
+debugging purposes.
+
+\subsubsection{no.uio.ifi.refaktor.utils}
+Utility package that contains all the functionality that has to do with parsing
+of source code. It also has utility classes for looking up handles to methods
+and types et cetera.
+
+\begin{description}
+ \item[no.uio.ifi.refaktor.utils.caching] \hfill \\
+ This package contains the caching manager for compilation units, along with
+ classes for different caching strategies.
+
+ \item[no.uio.ifi.refaktor.utils.nullobjects] \hfill \\
+ Contains classes for creating different null objects. Most of the classes is
+ used to represent null objects of different handle types. These null objects
+ are returned from various utility classes instead of returning a \var{null}
+ value when other values are not available.
+
+\end{description}
+
+\section{Continuous integration}
+The continuous integration server
+\name{Jenkins}\footnote{\url{http://jenkins-ci.org/}} has been set up for the
+project\footnote{A work mostly done by the supervisor.}. It is used as a way to
+run tests and perform code coverage analysis.
+
+To be able to build the \name{Eclipse} plugins and run tests for them with Jenkins, the
+component assembly project
+\name{Buckminster}\footnote{\url{http://www.eclipse.org/buckminster/}} is used,
+through its plugin for Jenkins. Buckminster provides for a way to specify the
+resources needed for building a project and where and how to find them.
+Buckminster also handles the setup of a target environment to run the tests in.
+All this is needed because the code to build depends on an \name{Eclipse}
+installation with various plugins.
+
+\subsection{Problems with AspectJ}
+The Buckminster build worked fine until introducing AspectJ into the project.
+When building projects using AspectJ, there are some additional steps that needs
+to be performed. First of all, the aspects themselves must be compiled. Then the
+aspects needs to be woven with the classes they affect. This demands a process
+that does multiple passes over the source code.
+
+When using AspectJ with \name{Eclipse}, the specialized compilation and the
+weaving can be handled by the \name{AspectJ Development
+Tools}\footnote{\url{https://www.eclipse.org/ajdt/}}. This works all fine, but
+it complicates things when trying to build a project depending on \name{Eclipse}
+plugins outside of \name{Eclipse}. There is supposed to be a way to specify a
+compiler adapter for javac, together with the file extensions for the file types
+it shall operate. The AspectJ compiler adapter is called
+\typewithref{org.aspectj.tools.ant.taskdefs}{Ajc11CompilerAdapter}, and it works
+with files that has the extensions \code{*.java} and \code{*.aj}. I tried to
+setup this in the build properties file for the project containing the aspects,
+but to no avail. The project containing the aspects does not seem to be built at
+all, and the projects that depends on it complains that they cannot find certain
+classes.
+
+I then managed to write an \name{Ant}\footnote{\url{https://ant.apache.org/}}
+build file that utilizes the AspectJ compiler adapter, for the
+\code{no.uio.ifi.refaktor} plugin. The problem was then that it could no longer
+take advantage of the environment set up by Buckminster. The solution to this
+particular problem was of a ``hacky'' nature. It involves exporting the plugin
+dependencies for the project to an Ant build file, and copy the exported path
+into the existing build script. But then the Ant script needs to know where the
+local \name{Eclipse} installation is located. This is no problem when building
+on a local machine, but to utilize the setup done by Buckminster is a problem
+still unsolved. To get the classpath for the build setup correctly, and here
+comes the most ``hacky'' part of the solution, the Ant script has a target for
+copying the classpath elements into a directory relative to the project
+directory and checking it into Git. When no \code{ECLIPSE\_HOME} property is set
+while running Ant, the script uses the copied plugins instead of the ones
+provided by the \name{Eclipse} installation when building the project. This
+obviously creates some problems with maintaining the list of dependencies in the
+Ant file, as well as remembering to copy the plugins every time the list of
+dependencies change.
+
+The Ant script described above is run by Jenkins before the Buckminster setup
+and build. When setup like this, the Buckminster build succeeds for the projects
+not using AspectJ, and the tests are run as normal. This is all good, but it
+feels a little scary, since the reason for Buckminster not working with AspectJ
+is still unknown.
+
+The problems with building with AspectJ on the Jenkins server lasted for a
+while, before they were solved. This is reflected in the ``Test Result Trend''
+and ``Code Coverage Trend'' reported by Jenkins.
+
+
+\chapter{Eclipse Bugs Found}
+\newcommand{\submittedBugReport}[1]{The submitted bug report can be found on
+ \url{#1}.}