[ifi-stolz-refaktor.git] / thesis / master-thesis-erlenkr.tex

\documentclass[USenglish]{ifimaster}
\usepackage{import}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc,url}
\usepackage{lmodern} % using Latin Modern to be able to use bold typewriter font
\urlstyle{sf}
\usepackage{babel,textcomp,csquotes,ifimasterforside,varioref,graphicx}
\usepackage[style=numeric-comp,backend=bibtex]{biblatex}
\usepackage{amsthm}
\usepackage{todonotes}
\usepackage{verbatim}
\usepackage{minted}
\usemintedstyle{bw}
\usepackage{perpage} %the perpage package
\MakePerPage{footnote} %the perpage package command

\theoremstyle{plain}
\newtheorem*{wordDef}{Definition}

\graphicspath{ {./figures/} }

\newcommand{\definition}[1]{\begin{wordDef}#1\end{wordDef}}
\newcommand{\see}[1]{(see section \ref{#1})}
\newcommand{\explanation}[3]{\noindent\textbf{\textit{#1}}\\*\emph{When:} 
#2\\*\emph{How:} #3\\*[-7px]}
\newcommand{\type}[1]{\texttt{\textbf{#1}}}
\newcommand{\typeref}[1]{\footnote{\type{#1}}}
\newcommand{\typewithref}[2]{\type{#2}\typeref{#1.#2}}
\newcommand{\method}[1]{\type{#1}}
\newcommand{\methodref}[2]{\footnote{\type{#1}\method{\##2()}}}
\newcommand{\methodwithref}[2]{\method{#2}\footnote{\type{#1}\method{\##2()}}}
\newcommand{\var}[1]{\type{#1}}


\title{Refactoring}
\subtitle{An unfinished essay}
\author{Erlend Kristiansen}

\bibliography{bibliography/master-thesis-erlenkr-bibliography}

\begin{document}
\ififorside
\frontmatter{}


\chapter*{Abstract}
Empty document.

\tableofcontents{}
\listoffigures{}
\listoftables{}

\chapter*{Preface}

To make it clear already from the beginning: The discussions in this report must 
be seen in the context of object oriented programming languages, and Java in 
particular, since that is the language in which most of the examples will be 
given. All though the techniques discussed may be applicable to languages from 
other paradigms, they will not be the subject of this report.

\mainmatter

\chapter{What is Refactoring?}

This question is best answered by first defining the concept of a 
\emph{refactoring}, what it is to \emph{refactor}, and then discuss what aspects 
of programming that make people want to refactor their code.

\section{Defining refactoring}
Martin Fowler, in his masterpiece on refactoring \cite{refactoring}, defines a 
refactoring like this:

\begin{quote}
  \emph{Refactoring} (noun): a change made to the \todo{what does he mean by 
  internal?} internal structure of software to make it easier to understand and 
  cheaper to modify without changing its observable 
  behavior.~\cite{refactoring} % page 53
\end{quote}

\noindent This definition assign additional meaning to the word 
\emph{refactoring}, beyond the composition of the prefix \emph{re-}, usually 
meaning something like ``again'' or ``anew'', and the word \emph{factoring}, 
that can mean to determine the \emph{factors} of something. Where a 
\emph{factor} would be close to the mathematical definition of something that 
divides a quantity, without leaving a remainder. Fowler is mixing the 
\emph{motivation} behind refactoring into his definition.  Instead it could be 
made clean, only considering the mechanical and behavioral aspects of 
refactoring. That is to factor the program again, putting it together in a 
different way than before, while preserving the behavior of the program. An 
alternative definition could then be: 

\definition{A refactoring is a transformation
done to a program without altering its external behavior.}

From this we can conclude that a refactoring primarily changes how the 
\emph{code} of a program is perceived by the \emph{programmer}, and not the 
\emph{behavior} experienced by any user of the program. Although the logical 
meaning is preserved, such changes could potentially alter the program's 
behavior when it comes to performance gain or -penalties. So any logic depending 
on the performance of a program could make the program behave differently after 
a refactoring.

In the extreme case one could argue that such a thing as \emph{software 
obfuscation} is to refactor. If we where to define it as a refactoring, it could 
be defined as a composite refactoring \see{intro_composite}, consisting of, for 
instance, a series of rename refactorings. (But it could of course be much more 
complex, and the mechanics of it would not exactly be carved in stone.) To 
perform some serious obfuscation one would also take advantage of techniques not 
found among established refactorings, such as removing whitespace. This might 
not even generate a different syntax tree for languages not sensitive to 
whitespace, placing it in the gray area of what kind of transformations is to be 
considered refactorings.

Finally, to \emph{refactor} is (quoting Martin Fowler)
\begin{quote}
  \ldots to restructure software by applying a series of refactorings without 
  changing its observable behavior.~\cite{refactoring} % page 54, definition
\end{quote}

\section{The etymology of 'refactoring'}
It is a little difficult to pinpoint the exact origin of the word 
``refactoring'', as it seems to have evolved as part of a colloquial 
terminology, more than a scientific term. There is no authoritative source for a 
formal definition of it. 

According to Martin Fowler~\cite{etymology-refactoring}, there may also be more 
than one origin of the word. The most well-known source, when it comes to the 
origin of \emph{refactoring}, is the Smalltalk\footnote{\emph{Smalltalk}, 
object-oriented, dynamically typed, reflective programming language.}\todo{find 
reference to Smalltalk website or similar?} community and their infamous 
\emph{Refactoring 
Browser}\footnote{\url{http://st-www.cs.illinois.edu/users/brant/Refactory/RefactoringBrowser.html}} 
described in the article \emph{A Refactoring Tool for 
Smalltalk}~\cite{refactoringBrowser1997}, published in 1997.  
Allegedly~\cite{etymology-refactoring}, the metaphor of factoring programs was 
also present in the Forth\footnote{\emph{Forth} -- stack-based, extensible 
programming language, without type-checking. See \url{http://www.forth.org}} 
community, and the word ``refactoring'' is mentioned in a book by Leo Brodie, 
called \emph{Thinking Forth}~\cite{brodie1984}, first published in 
1984\footnote{\emph{Thinking Forth} was first published in 1984 by the 
\emph{Forth Interest Group}.  Then it was reprinted in 1994 with minor 
typographical corrections, before it was transcribed into an electronic edition 
typeset in \LaTeX\ and published under a Creative Commons licence in 2004. The 
edition cited here is the 2004 edition, but the content should essentially be as 
in 1984.}. The exact word is only printed one place\footnote{p. 232}, but the 
term \emph{factoring} is prominent in the book, that also contains a whole 
chapter dedicated to (re)factoring, and how to keep the (Forth) code clean and 
maintainable.

\begin{quote}
  \ldots good factoring technique is perhaps the most important skill for a 
  Forth programmer.~\cite{brodie1984}
\end{quote}

\noindent Brodie also express what \emph{factoring} means to him:

\begin{quote}
  Factoring means organizing code into useful fragments. To make a fragment 
  useful, you often must separate reusable parts from non-reusable parts. The  
  reusable parts become new definitions. The non-reusable parts become arguments 
  or parameters to the definitions.~\cite{brodie1984}
\end{quote}

Fowler claims that the usage of the word \emph{refactoring} did not pass between 
the \emph{Forth} and \emph{Smalltalk} communities, but that it emerged 
independently in each of the communities.

\todo{more history?}

\section{Motivation -- Why people refactor}
To get a grasp of what refactoring is all about, we can try to answer this 
question: \emph{Why do people refactor?} Possible answers could include: ``To 
remove duplication'' or ``to break up long methods''.  Practitioners of the art 
of Design Patterns~\cite{dp} could say that they do it to introduce a 
long-needed pattern into their program's design.  So it is safe to say that 
peoples' intentions are to make their programs \emph{better} in some sense. But 
what aspects of the programs are becoming improved?

As already mentioned, people often refactor to get rid of duplication. Moving 
identical or similar code into methods, and maybe pushing those up or down in 
their class hierarchies. Making template methods for overlapping 
algorithms/functionality and so on.  It's all about gathering what belongs 
together and putting it all in one place.  And the result? The code is easier to 
maintain. When removing the implicit coupling between the code snippets, the 
location of a bug is limited to only one place, and new functionality need only 
to be added this one place, instead of a number of places people might not even 
remember.

The same people find out that their program contains a lot of long and 
hard-to-grasp methods. Then what do they do? They begin dividing their methods 
into smaller ones, using the \emph{Extract Method} 
refactoring~\cite{refactoring}.  Then they may discover something about their 
program that they weren't aware of before; revealing bugs they didn't know about 
or couldn't find due to the complex structure of their program. \todo{Proof?} 
Making the methods smaller and giving good names to the new ones clarifies the 
algorithms and enhances the \emph{understandability} of the program 
\see{magic_number_seven}. This makes simple refactoring an excellent method for 
exploring unknown program code, or code that you had forgotten that you wrote!

The word \emph{simple} came up in the last section. In fact, most primitive 
refactorings are simple. The true power of them are revealed first when they are 
combined into larger --- higher level --- refactorings, called \emph{composite 
refactorings} \see{intro_composite}. Often the goal of such a series of 
refactorings is a design pattern. Thus the \emph{design} can be evolved 
throughout the lifetime of a program, opposed to designing up-front.  It's all 
about being structured and taking small steps to improve a program's design.

Many refactorings are aimed at lowering the coupling between different classes 
and different layers of logic. \todo{which refactorings?} Say for instance that 
the coupling between the user interface and the business logic of a program is 
lowered.  Then the business logic of the program could much easier be the target 
of automated tests, increasing the productivity in the software development 
process. It is also easier to distribute (e.g. between computers) the different 
components of a program if they are sufficiently decoupled.

Another effect of refactoring is that with the increased separation of concerns 
coming out of many refactorings, the \emph{performance} is improved.  When 
profiling programs, the problem parts are narrowed down to smaller parts of the 
code, which are easier to tune, and optimization can be performed only where 
needed and in a more effective way.

Last, but not least, and this should probably be the best reason to refactor, is 
to refactor to \emph{facilitate a program change}. If one has managed to keep 
one's code clean and tidy, and the code is not bloated with design patterns that 
is not ever going to be needed, then some refactoring might be needed to 
introduce a design pattern that is appropriate for the change that is going to 
happen.

Refactoring program code --- with a goal in mind --- can give the code itself 
more value. That is in the form of robustness to bugs, understandability and 
maintainability. With the first as an obvious advantage, but with the following 
two being also very important for software development. By incorporating 
refactoring in the development process, bugs are found faster, new functionality 
is added more easily and code is easier to understand by the next person exposed 
to it, which might as well be the person who wrote it. The consequence of this, 
is that refactoring can increase the average productivity of the development 
process, and thus also add to the monetary value of a business in the long run.  
Where this last point also should open the eyes of some nearsighted managers who 
seldom see beyond the next milestone.

\section{The magical number seven}\label{magic_number_seven}
\emph{The magical number seven, plus or minus two: some limits on our capacity 
for processing information}~\cite{miller1956} is an article by George A. Miller 
that was published in the journal \emph{Psychological Review} in 1956. It 
presents evidence that support that the capacity of the number of objects a 
human being can hold in its working memory is roughly seven, plus or minus two 
objects. This number varies a bit depending on the nature and complexity of the 
objects, but is according to Miller ``\ldots never changing so much as to be 
unrecognizable.''

Miller's article culminates in the section called \emph{Recoding}, a term he 
borrows from communication theory. The central result in this section is that by 
recoding information, the capacity of the amount of information that a human can 
process at a time is increased. By \emph{recoding}, Miller means to group 
objects together in chunks and give each chunk a new name that it can be 
remembered by. By organizing objects into patterns of ever growing depth, one 
can memorize and process a much larger amount of data than if it were to be 
represented as its basic pieces. This grouping and renaming is analogous to how 
many refactorings work, by grouping pieces of code and give them a new name.  
Examples are the central \emph{Extract Method} and \emph{Extract Class} 
refactorings~\cite{refactoring}.

\begin{quote}
  \ldots recoding is an extremely powerful weapon for increasing the amount of 
  information that we can deal with.~\cite{miller1956}
\end{quote}

An example from the article address the problem of memorizing a sequence of 
binary digits. Let us say we have the following sequence\footnote{The example 
  presented here is slightly modified (and shortened) from what is presented in 
  the original article~\cite{miller1956}, but it is essentially the same.} of 
16 binary digits: ``1010001001110011''. Most of us will have a hard time 
memorizing this sequence by only reading it once or twice. Imagine if we instead 
translate it to this sequence: ``A273''. If you have a background from computer 
science, it will be obvious that the latest sequence is the first sequence 
recoded to be represented by digits with base 16. Most people should be able to 
memorize this last sequence by only looking at it once.

Another result from the Miller article is that when the amount of information a 
human must interpret increases, it is crucial that the translation from one code 
to another must be almost automatic for the subject to be able to remember the 
translation, before he or she is presented with new information to recode. Thus 
learning and understanding how to best organize certain kinds of data is 
essential to efficiently handle that kind of data in the future. This is much 
like when children learn to read. First they must learn how to recognize 
letters. Then they can learn distinct words, and later read sequences of words 
that form whole sentences. Eventually, most of them will be able to read whole 
books and briefly retell the important parts of its content. This suggest that 
the use of design patterns~\cite{dp} is a good idea when reasoning about 
computer programs. With extensive use of design patterns when creating complex 
program structures, one does not always have to read whole classes of code to 
comprehend how they function, it may be sufficient to only see the name of a 
class to almost fully understand its responsibilities.

\begin{quote}
  Our language is tremendously useful for repackaging material into a few chunks 
  rich in information.~\cite{miller1956}
\end{quote}

Without further evidence, these results at least indicates that refactoring 
source code into smaller units with higher cohesion and, when needed, 
introducing appropriate design patterns, should aid in the cause of creating 
computer programs that are easier to maintain and has code that is easier (and 
better) understood.

\section{Notable contributions to the refactoring literature}
\todo{Update with more contributions}
\begin{description}
  \item[1992] William F. Opdyke submits his doctoral dissertation called 
    \emph{Refactoring Object-Oriented Frameworks}~\cite{opdyke1992}. This 
    work defines a set of refactorings, that are behavior preserving given that 
    their preconditions are met. The dissertation is focused on the automation 
    of refactorings.
  \item[1999] Martin Fowler et al.: \emph{Refactoring: Improving the Design of 
    Existing Code}~\cite{refactoring}. This is maybe the most influential text 
    on refactoring. It bares similarities with Opdykes thesis~\cite{opdyke1992} 
    in the way that it provides a catalog of refactorings. But Fowler's book is 
    more about the craft of refactoring, as he focuses on establishing a 
    vocabulary for refactoring, together with the mechanics of different 
    refactorings and when to perform them. His methodology is also founded on 
    the principles of test-driven development.
  \item[todo] \emph{Refactoring to Patterns}\todo{include}
\end{description}

\section{Tool support}
\todo{write, section vs. subsection}

\section{Relation to design patterns}
\todo{write, section vs. subsection, refactoring to patterns?}
\begin{comment}

\section{Classification of refactorings} 
% only interesting refactorings
% with 2 detailed examples? One for structured and one for intra-method?
% Is replacing Bubblesort with Quick Sort considered a refactoring?

\subsection{Structural refactorings}

\subsubsection{Primitive refactorings}

% Composing Methods
\explanation{Extract Method}{You have a code fragment that can be grouped 
together.}{Turn the fragment into a method whose name explains the purpose of 
the method.}

\explanation{Inline Method}{A method's body is just as clear as its name.}{Put 
the method's body into the body of its callers and remove the method.}

\explanation{Inline Temp}{You have a temp that is assigned to once with a simple 
expression, and the temp is getting in the way of other refactorings.}{Replace 
all references to that temp with the expression}

% Moving Features Between Objects
\explanation{Move Method}{A method is, or will be, using or used by more 
features of another class than the class on which it is defined.}{Create a new 
method with a similar body in the class it uses most. Either turn the old method 
into a simple delegation, or remove it altogether.}

\explanation{Move Field}{A field is, or will be, used by another class more than 
the class on which it is defined}{Create a new field in the target class, and 
change all its users.}

% Organizing Data
\explanation{Replace Magic Number with Symbolic Constant}{You have a literal 
number with a particular meaning.}{Create a constant, name it after the meaning, 
and replace the number with it.}

\explanation{Encapsulate Field}{There is a public field.}{Make it private and 
provide accessors.}

\explanation{Replace Type Code with Class}{A class has a numeric type code that 
does not affect its behavior.}{Replace the number with a new class.}

\explanation{Replace Type Code with Subclasses}{You have an immutable type code 
that affects the behavior of a class.}{Replace the type code with subclasses.}

\explanation{Replace Type Code with State/Strategy}{You have a type code that 
affects the behavior of a class, but you cannot use subclassing.}{Replace the 
type code with a state object.}

% Simplifying Conditional Expressions
\explanation{Consolidate Duplicate Conditional Fragments}{The same fragment of 
code is in all branches of a conditional expression.}{Move it outside of the 
expression.}

\explanation{Remove Control Flag}{You have a variable that is acting as a 
control flag fro a series of boolean expressions.}{Use a break or return 
instead.}

\explanation{Replace Nested Conditional with Guard Clauses}{A method has 
conditional behavior that does not make clear the normal path of 
execution.}{Use guard clauses for all special cases.}

\explanation{Introduce Null Object}{You have repeated checks for a null 
value.}{Replace the null value with a null object.}

\explanation{Introduce Assertion}{A section of code assumes something about the 
state of the program.}{Make the assumption explicit with an assertion.}

% Making Method Calls Simpler
\explanation{Rename Method}{The name of a method does not reveal its 
purpose.}{Change the name of the method}

\explanation{Add Parameter}{A method needs more information from its 
caller.}{Add a parameter for an object that can pass on this information.}

\explanation{Remove Parameter}{A parameter is no longer used by the method 
body.}{Remove it.}

%\explanation{Parameterize Method}{Several methods do similar things but with 
%different values contained in the method.}{Create one method that uses a 
%parameter for the different values.}

\explanation{Preserve Whole Object}{You are getting several values from an 
object and passing these values as parameters in a method call.}{Send the whole 
object instead.}

\explanation{Remove Setting Method}{A field should be set at creation time and 
never altered.}{Remove any setting method for that field.}

\explanation{Hide Method}{A method is not used by any other class.}{Make the 
method private.}

\explanation{Replace Constructor with Factory Method}{You want to do more than 
simple construction when you create an object}{Replace the constructor with a 
factory method.}

% Dealing with Generalization
\explanation{Pull Up Field}{Two subclasses have the same field.}{Move the field 
to the superclass.}

\explanation{Pull Up Method}{You have methods with identical results on 
subclasses.}{Move them to the superclass.}

\explanation{Push Down Method}{Behavior on a superclass is relevant only for 
some of its subclasses.}{Move it to those subclasses.}

\explanation{Push Down Field}{A field is used only by some subclasses.}{Move the 
field to those subclasses}

\explanation{Extract Interface}{Several clients use the same subset of a class's 
interface, or two classes have part of their interfaces in common.}{Extract the 
subset into an interface.}

\explanation{Replace Inheritance with Delegation}{A subclass uses only part of a 
superclasses interface or does not want to inherit data.}{Create a field for the 
superclass, adjust methods to delegate to the superclass, and remove the 
subclassing.}

\explanation{Replace Delegation with Inheritance}{You're using delegation and 
are often writing many simple delegations for the entire interface}{Make the 
delegating class a subclass of the delegate.}

\subsubsection{Composite refactorings}

% Composing Methods
% \explanation{Replace Method with Method Object}{}{}

% Moving Features Between Objects
\explanation{Extract Class}{You have one class doing work that should be done by 
two}{Create a new class and move the relevant fields and methods from the old 
class into the new class.}

\explanation{Inline Class}{A class isn't doing very much.}{Move all its features 
into another class and delete it.}

\explanation{Hide Delegate}{A client is calling a delegate class of an 
object.}{Create Methods on the server to hide the delegate.}

\explanation{Remove Middle Man}{A class is doing to much simple delegation.}{Get 
the client to call the delegate directly.}

% Organizing Data
\explanation{Replace Data Value with Object}{You have a data item that needs 
additional data or behavior.}{Turn the data item into an object.}

\explanation{Change Value to Reference}{You have a class with many equal 
instances that you want to replace with a single object.}{Turn the object into a 
reference object.}

\explanation{Encapsulate Collection}{A method returns a collection}{Make it 
return a read-only view and provide add/remove methods.}

% \explanation{Replace Array with Object}{}{}

\explanation{Replace Subclass with Fields}{You have subclasses that vary only in 
methods that return constant data.}{Change the methods to superclass fields and 
eliminate the subclasses.}

% Simplifying Conditional Expressions
\explanation{Decompose Conditional}{You have a complicated conditional 
(if-then-else) statement.}{Extract methods from the condition, then part, an 
else part.}

\explanation{Consolidate Conditional Expression}{You have a sequence of 
conditional tests with the same result.}{Combine them into a single conditional 
expression and extract it.}

\explanation{Replace Conditional with Polymorphism}{You have a conditional that 
chooses different behavior depending on the type of an object.}{Move each leg 
of the conditional to an overriding method in a subclass. Make the original 
method abstract.}

% Making Method Calls Simpler
\explanation{Replace Parameter with Method}{An object invokes a method, then 
passes the result as a parameter for a method. The receiver can also invoke this 
method.}{Remove the parameter and let the receiver invoke the method.}

\explanation{Introduce Parameter Object}{You have a group of parameters that 
naturally go together.}{Replace them with an object.}

% Dealing with Generalization
\explanation{Extract Subclass}{A class has features that are used only in some 
instances.}{Create a subclass for that subset of features.}

\explanation{Extract Superclass}{You have two classes with similar 
features.}{Create a superclass and move the common features to the 
superclass.}

\explanation{Collapse Hierarchy}{A superclass and subclass are not very 
different.}{Merge them together.}

\explanation{Form Template Method}{You have two methods in subclasses that 
perform similar steps in the same order, yet the steps are different.}{Get the 
steps into methods with the same signature, so that the original methods become 
the same. Then you can pull them up.}


\subsection{Functional refactorings}

\explanation{Substitute Algorithm}{You want to replace an algorithm with one 
that is clearer.}{Replace the body of the method with the new algorithm.}

\end{comment}

\section{The impact on software quality}

\subsection{What is meant by quality?}
The term \emph{software quality} has many meanings. It all depends on the 
context we put it in. If we look at it with the eyes of a software developer, it 
usually mean that the software is easily maintainable and testable, or in other 
words, that it is \emph{well designed}. This often correlates with the 
management scale, where \emph{keeping the schedule} and \emph{customer 
satisfaction} is at the center. From the customers point of view, in addition to 
good usability, \emph{performance} and \emph{lack of bugs} is always 
appreciated, measurements that are also shared by the software developer. (In 
addition, such things as good documentation could be measured, but this is out 
of the scope of this document.)

\subsection{The impact on performance}
\begin{quote}
  Refactoring certainly will make software go more slowly, but it also makes the 
  software more amenable to performance tuning.~\cite{refactoring} % page 69
\end{quote}

\noindent There is a common belief that refactoring compromises performance, due 
to increased degree of indirection and that polymorphism is slower than 
conditionals.

In a survey, Demeyer~\cite{demeyer2002} disproves this view in the case of 
polymorphism. He is doing an experiment on, what he calls, ``Transform Self Type 
Checks'' where you introduce a new polymorphic method and a new class hierarchy 
to get rid of a class' type checking of a ``type attribute``. He uses this kind 
of transformation to represent other ways of replacing conditionals with 
polymorphism as well. The experiment is performed on the C++ programming 
language and with three different compilers and platforms. \todo{But is the 
result better?} Demeyer concludes that, with compiler optimization turned on, 
polymorphism beats middle to large sized if-statements and does as well as 
case-statements.  (In accordance with his hypothesis, due to similarities 
between the way C++ handles polymorphism and case-statements.)

\begin{quote}
  The interesting thing about performance is that if you analyze most programs, 
  you find that they waste most of their time in a small fraction of the code.  
  ~\cite{refactoring}
\end{quote}

\noindent So, although an increased amount of method calls could potentially 
slow down programs, one should avoid premature optimization and sacrificing good 
design, leaving the performance tuning until after profiling\footnote{For and 
  example of a Java profiler, check out VisualVM: 
  \url{http://visualvm.java.net/}} the software and having isolated the actual 
  problem areas.

\section{Composite refactorings} \label{intro_composite}
\todo{motivation, examples, manual vs automated?, what about refactoring in a 
very large code base?}
Generally, when thinking about refactoring, at the mechanical level, there are 
essentially two kinds of refactorings. There are the \emph{primitive} 
refactorings, and the \emph{composite} refactorings. A primitive refactoring can 
be defined like this:

\definition{A primitive refactoring is a refactoring that cannot be expressed in 
terms of other refactorings.}

\noindent Examples are the \emph{Pull Up Field} and \emph{Pull Up Method} 
refactorings~\cite{refactoring}, that moves members up in their class 
hierarchies.

A composite refactoring is more complex, and can be defined like this:

\definition{A composite refactoring is a refactoring that can be expressed in 
terms of two or more primitive refactorings.}

\noindent An example of a composite refactoring is the \emph{Extract Superclass} 
refactoring~\cite{refactoring}. In its simplest form, it is composed of the 
previously described primitive refactorings, in addition to the \emph{Pull Up 
Constructor Body} refactoring~\cite{refactoring}. It works by creating an 
abstract superclass that the target class(es) inherits from, then by applying 
\emph{Pull Up Field}, \emph{Pull Up Method} and \emph{Pull Up Constructor Body} 
on the members that are to be members of the new superclass. For an overview of 
the \emph{Extract Superclass} refactoring, see figure 
\ref{fig:extractSuperclass}.

\begin{figure}[h]
  \centering
  \includegraphics[angle=270,width=\linewidth]{extractSuperclassItalic.pdf}
  \caption{The Extract Superclass refactoring}
  \label{fig:extractSuperclass}
\end{figure}

\section{Manual vs. automated refactorings}
Refactoring is something every programmer does, even if he or she does not known 
the term \emph{refactoring}. Every refinement of source code that does not alter 
the program's behavior is a refactoring. For small refactorings, such as 
\emph{Extract Method}, executing it manually is a manageable task, but is still 
prone to errors. Getting it right the first time is not easy, considering the 
signature and all the other aspects of the refactoring that has to be in place.  

Take for instance the renaming of classes, methods and fields. For complex 
programs these refactorings are almost impossible to get right.  Attacking them 
with textual search and replace, or even regular expressions, will fall short on 
these tasks. Then it is crucial to have proper tool support that can perform 
them automatically. Tools that can parse source code and thus has semantic 
knowledge about which occurrences of which names that belongs to what construct 
in the program. For even trying to perform one of these complex task manually, 
one would have to be very confident on the existing test suite \see{tdd}.

\section{Correctness of refactorings}
\todo{Volker's example?}
For automated refactorings to be truly useful, they must show a high degree of 
behavior preservation. This might seem obvious, but there are examples of 
refactorings in existing tools that break programs. \todo{More than Eclipse?} I 
will now present an example of an \emph{Extract Method} refactoring followed by 
a \emph{Move Method} refactoring that breaks a program. The following piece of 
code shows the target for the composed refactoring:

\begin{minted}[linenos]{java}
public class C {
    public X x = new X();

    public void f() {
        x.m(this);
        x.n();
    }
}
\end{minted}

\noindent The next piece of code shows the destination of the refactoring. Note 
that the method \method{m(C c)} of class \type{C} assigns to the field \var{x} 
of the argument \var{c} that has type \type{C}:

\begin{minted}{java}
public class X {
    public void m(C c) {
        c.x = new X();
    }
    public void n() {}
}
\end{minted}

The refactoring sequence works by extracting line 5 and 6 from the original 
class \type{C} into a method \method{f} with the statements from those lines as 
its method body. The method is then moved to the class \type{X}. The result is 
shown in the following two pieces of code:

\begin{minted}[linenos]{java}
public class C {
    public X x = new X();

    public void f() {
        x.f(this);
    }
}
\end{minted}

\begin{minted}[linenos]{java}
public class X {
    public void m(C c) {
        c.x = new X();
    }
    public void n() {}
    public void f(C c) {
        m(c);
        n();
    }
}
\end{minted}

Since, after the refactoring, the method \method{f} of class \type{C} calls the 
method \method{f} of class \type{X}, the program breaks.  (See line 5 of the 
version of class \type{C} after the refactoring.) Before the refactoring, the 
methods \method{m} and \method{n} of class \type{X} are called on different 
object instances (see line 5 and 6 of the original class \type{C}). After, they 
are called on the same object, and the statement on line 3 in the class \type{X} 
after the refactoring no longer have any effect in our example.

\section{Test Driven Development}\label{tdd}

\section{Software metrics}

%\part{The project}
%\chapter{Planning the project}
%\part{Conclusion}
%\chapter{Results}                   



\chapter{\ldots}
\todo{write}
\section{The problem statement}
\section{Choosing the target language}
Choosing which programming language to use as the target for manipulation is not 
a very difficult task. The language have to be an object-oriented programming 
language, and it must have existing tool support for refactoring. The 
\emph{Java} programming language\footnote{\url{https://www.java.com/}} is the 
dominating language when it comes to examples in the literature of refactoring, 
and is thus a natural choice. Java is perhaps, currently the most influential 
programming language in the world, with its \emph{Java Virtual Machine} that 
runs on all of the most popular architectures and also supports\footnote{They 
compile to java bytecode.} dozens of other programming languages, with 
\emph{Scala}, \emph{Clojure} and \emph{Groovy} as the most prominent ones. Java 
is currently the language that every other programming language is compared 
against. It is also the primary language of the author of this thesis.

\section{Choosing the tools}
When choosing a tool for manipulating Java, there are certain criterias that 
have to be met. First of all, the tool should have some existing refactoring 
support that this thesis can build upon. Secondly it should provide some kind of 
framework for parsing and analyzing Java source code. Third, it should itself be 
open source. This is both because of the need to be able to browse the code for 
the existing refactorings that is contained in the tool, and also because open 
source projects hold value in them selves. Another important aspect to consider 
is that open source projects of a certain size, usually has large communities of 
people connected to them, that are commited to answering questions regarding the 
use and misuse of the products, that to a large degree is made by the cummunity 
itself.

There is a certain class of tools that meet these criterias, namely the class of 
\emph{IDEs}\footnote{\emph{Integrated Development Environment}}. These are 
proagrams that is ment to support the whole production cycle of a cumputer 
program, and the most popular IDEs that support Java, generally have quite good 
refactoring support.

The main contenders for this thesis is the \emph{Eclipse 
IDE}\footnote{\url{http://www.eclipse.org/}}, with the \emph{Java development 
tools} (JDT), the \emph{IntelliJ IDEA Community 
Edition}\footnote{\url{http://www.jetbrains.com/idea/}} and the \emph{NetBeans 
IDE}\footnote{\url{https://netbeans.org/}}. Eclipse and NetBeans are both free, 
open source and community driven, while the IntelliJ IDEA has an open sourced 
community edition that is free of charge, but also offer an \emph{Ultimate 
Edition} with an extended set of features, at additional cost. All three IDEs 
supports adding plugins to extend their functionality and tools that can be used 
to parse and analyze Java source code. \todo{investigate if this is true} But 
one of the IDEs stand out as a favorite, and that is the \emph{Eclipse IDE}.  
This is the most popular~\cite{javaReport2011} among them and seems to be de 
facto standard IDE for Java development regardless of platform.

\chapter{Refactorings in Eclipse JDT: Design, Shortcomings and Wishful 
Thinking}\label{ch:jdt_refactorings}

This chapter will deal with some of the design behind refactoring support in 
Eclipse, and the JDT in specific. After which it will follow a section about 
shortcomings of the refactoring API in terms of composition of refactorings. The 
chapter will be concluded with a section telling some of the ways the 
implementation of refactorings in the JDT could have worked to facilitate 
composition of refactorings.

\section{Design}
The refactoring world of Eclipse can in general be separated into two parts: The 
language independent part and the part written for a specific programming 
language -- the language that is the target of the supported refactorings.  
\todo{What about the language specific part?}

\subsection{The Language Toolkit}
The Language Toolkit, or LTK for short, is the framework that is used to 
implement refactorings in Eclipse. It is language independent and provides the 
abstractions of a refactoring and the change it generates, in the form of the 
classes \typewithref{org.eclipse.ltk.core.refactoring}{Refactoring} and 
\typewithref{org.eclipse.ltk.core.refactoring}{Change}. (There is also parts of 
the LTK that is concerned with user interaction, but they will not be discussed 
here, since they are of little value to us and our use of the framework.)

\subsubsection{The Refactoring Class}
The abstract class \type{Refactoring} is the core of the LTK framework. Every 
refactoring that is going to be supported by the LTK have to end up creating an 
instance of one of its subclasses. The main responsibilities of subclasses of 
\type{Refactoring} is to implement template methods for condition checking 
(\methodwithref{org.eclipse.ltk.core.refactoring.Refactoring}{checkInitialConditions} 
and 
\methodwithref{org.eclipse.ltk.core.refactoring.Refactoring}{checkFinalConditions}), 
in addition to the 
\methodwithref{org.eclipse.ltk.core.refactoring.Refactoring}{createChange} 
method that creates and returns an instance of the \type{Change} class.

If the refactoring shall support that others participate in it when it is 
executed, the refactoring has to be a processor-based 
refactoring\typeref{org.eclipse.ltk.core.refactoring.participants.ProcessorBasedRefactoring}.  
It then delegates to its given 
\typewithref{org.eclipse.ltk.core.refactoring.participants}{RefactoringProcessor} 
for condition checking and change creation.

\subsubsection{The Change Class}
This class is the base class for objects that is responsible for performing the 
actual workspace transformations in a refactoring. The main responsibilities for 
its subclasses is to implement the 
\methodwithref{org.eclipse.ltk.core.refactoring.Change}{perform} and 
\methodwithref{org.eclipse.ltk.core.refactoring.Change}{isValid} methods. The 
\method{isValid} method verifies that the change object is valid and thus can be 
executed by calling its \method{perform} method. The \method{perform} method 
performs the desired change and returns an undo change that can be executed to 
reverse the effect of the transformation done by its originating change object. 

\subsubsection{Executing a Refactoring}\label{executing_refactoring}
The life cycle of a refactoring generally follows two steps after creation: 
condition checking and change creation. By letting the refactoring object be 
handled by a 
\typewithref{org.eclipse.ltk.core.refactoring}{CheckConditionsOperation} that
in turn is handled by a 
\typewithref{org.eclipse.ltk.core.refactoring}{CreateChangeOperation}, it is 
assured that the change creation process is managed in a proper manner.

The actual execution of a change object has to follow a detailed life cycle.  
This life cycle is honored if the \type{CreateChangeOperation} is handled by a 
\typewithref{org.eclipse.ltk.core.refactoring}{PerformChangeOperation}. If also 
an undo manager\typeref{org.eclipse.ltk.core.refactoring.IUndoManager} is set 
for the \type{PerformChangeOperation}, the undo change is added into the undo 
history.

\section{Shortcomings}
This section is introduced naturally with a conclusion: The JDT refactoring 
implementation does not facilitate composition of refactorings. 
\todo{refine}This section will try to explain why, and also identify other 
shortcomings of both the usability and the readability of the JDT refactoring 
source code.

I will begin at the end and work my way toward the composition part of this 
section.

\subsection{Absence of Generics in Eclipse Source Code}
This section is not only concerning the JDT refactoring API, but also large 
quantities of the Eclipse source code. The code shows a striking absence of the 
Java language feature of generics. It is hard to read a class' interface when 
methods return objects or takes parameters of raw types such as \type{List} or 
\type{Map}. This sometimes results in having to read a lot of source code to 
understand what is going on, instead of relying on the available interfaces. In 
addition, it results in a lot of ugly code, making the use of typecasting more 
of a rule than an exception.

\subsection{Composite Refactorings Will Not Appear as Atomic Actions}

\subsubsection{Missing Flexibility from JDT Refactorings}
The JDT refactorings are not made with composition of refactorings in mind. When 
a JDT refactoring is executed, it assumes that all conditions for it to be 
applied successfully can be found by reading source files that has been 
persisted to disk. They can only operate on the actual source material, and not 
(in-memory) copies thereof. This constitutes a major disadvantage when trying to 
compose refactorings, since if an exception occur in the middle of a sequence of 
refactorings, it can leave the project in a state where the composite 
refactoring was executed only partly. It makes it hard to discard the changes 
done without monitoring and consulting the undo manager, an approach that is not 
bullet proof.

\subsubsection{Broken Undo History}
When designing a composed refactoring that is to be performed as a sequence of 
refactorings, you would like it to appear as a single change to the workspace.  
This implies that you would also like to be able to undo all the changes done by 
the refactoring in a single step. This is not the way it appears when a sequence 
of JDT refactorings is executed. It leaves the undo history filled up with 
individual undo actions corresponding to every single JDT refactoring in the 
sequence. This problem is not trivial to handle in Eclipse. (See section 
\ref{hacking_undo_history}.)

\section{Wishful Thinking}



\chapter{Composite Refactorings in Eclipse}

\section{A Simple Ad Hoc Model}
As pointed out in chapter \ref{ch:jdt_refactorings}, the Eclipse JDT refactoring 
model is not very well suited for making composite refactorings. Therefore a 
simple model using changer objects (of type \type{RefaktorChanger}) is used as 
an abstraction layer on top of the existing Eclipse refactorings.

\section{The Extract and Move Method Refactoring}
%The Extract and Move Method Refactoring is implemented mainly using these 
%classes:
%\begin{itemize}
%  \item \type{ExtractAndMoveMethodChanger}
%  \item \type{ExtractAndMoveMethodPrefixesExtractor}
%  \item \type{Prefix}
%  \item \type{PrefixSet}
%\end{itemize}

\subsection{The Building Blocks}
This is a composite refactoring, and hence is built up using several primitive 
refactorings. These basic building blocks are, as its name implies, the Extract 
Method Refactoring \cite{refactoring} and the Move Method Refactoring 
\cite{refactoring}. In Eclipse, the implementations of these refactorings are 
found in the classes 
\typewithref{org.eclipse.jdt.internal.corext.refactoring.code}{ExtractMethodRefactoring} 
and 
\typewithref{org.eclipse.jdt.internal.corext.refactoring.structure}{MoveInstanceMethodProcessor}, 
where the last class is designed to be used together with the processor-based 
\typewithref{org.eclipse.ltk.core.refactoring.participants}{MoveRefactoring}.

\subsubsection{The ExtractMethodRefactoring Class}
This class is quite simple in its use. The only parameters it requires for 
construction is a compilation 
unit\typeref{org.eclipse.jdt.core.ICompilationUnit}, the offset into the source 
code where the extraction shall start, and the length of the source to be 
extracted. Then you have to set the method name for the new method together with 
which access modifier that shall be used and some not so interesting parameters.

\subsubsection{The MoveInstanceMethodProcessor Class}
For the Move Method the processor requires a little more advanced input than  
the class for the Extract Method. For construction it requires a method 
handle\typeref{org.eclipse.jdt.core.IMethod} from the Java Model for the method 
that is to be moved. Then the target for the move have to be supplied as the 
variable binding from a chosen variable declaration. In addition to this, one 
have to set some parameters regarding setters/getters and delegation.

To make a whole refactoring from the processor, one have to construct a 
\type{MoveRefactoring} from it.

\subsection{The ExtractAndMoveMethodChanger Class}
The \typewithref{no.uio.ifi.refaktor.changers}{ExtractAndMoveMethodChanger} 
class, that is a subclass of the class 
\typewithref{no.uio.ifi.refaktor.changers}{RefaktorChanger}, is the class 
responsible for composing the \type{ExtractMethodRefactoring} and the 
\type{MoveRefactoring}. Its constructor takes a project 
handle\typeref{org.eclipse.core.resources.IProject}, the method name for the new 
method and a \typewithref{no.uio.ifi.refaktor.utils}{SmartTextSelection}.

A \type{SmartTextSelection} is basically a text 
selection\typeref{org.eclipse.jface.text.ITextSelection} object that enforces 
the providing of the underlying document during creation. I.e. its 
\methodwithref{no.uio.ifi.refaktor.utils.SmartTextSelection}{getDocument} method 
will never return \type{null}.

Before extracting the new method, the possible targets for the move operation is 
found with the help of an
\typewithref{no.uio.ifi.refaktor.extractors}{ExtractAndMoveMethodPrefixesExtractor}.  
The possible targets is computed from the prefixes that the extractor returns 
from its
\methodwithref{no.uio.ifi.refaktor.extractors.ExtractAndMoveMethodPrefixesExtractor}{getSafePrefixes} 
method. The changer then choose the most suitable target by finding the most 
frequent occurring prefix among the safe ones. The target is the type of the 
first part of the prefix.

After finding a suitable target, the \type{ExtractAndMoveMethodChanger} first 
creates an \type{ExtractMethodRefactoring} and performs it as explained in 
section \ref{executing_refactoring} about the execution of refactorings. Then it 
creates and performs the \type{MoveRefactoring} in the same way, based on the 
changes done by the Extract Method refactoring.

\subsection{The ExtractAndMoveMethodPrefixesExtractor Class}
This extractor extracts properties needed for building the Extract and Move 
Method refactoring. It searches through the given selection to find safe 
prefixes, and those prefixes form a base that can be used to compute possible 
targets for the move part of the refactoring.  It finds both the candidates, in 
the form of prefixes, and the non-candidates, called unfixes. All prefixes (and 
unfixes) are represented by a 
\typewithref{no.uio.ifi.refaktor.extractors}{Prefix}, and they are collected 
into prefix sets.\typeref{no.uio.ifi.refaktor.extractors.PrefixSet}. 

The prefixes and unfixes are found by property 
collectors\typeref{no.uio.ifi.refaktor.extractors.collectors.PropertyCollector}.  
A property collector follows the visitor pattern \cite{dp} and is of the 
\typewithref{org.eclipse.jdt.core.dom}{ASTVisitor} type.  An \type{ASTVisitor} 
visits nodes in an abstract syntax tree that forms the Java document object 
model. The tree consists of nodes of type 
\typewithref{org.eclipse.jdt.core.do}{ASTNode}.

\subsubsection{The PrefixesCollector}
The \typewithref{no.uio.ifi.refaktor.extractors.collectors}{PrefixesCollector} 
is of type \type{PropertyCollector}. It visits expression 
statements\typeref{org.eclipse.jdt.core.dom.ExpressionStatement} and creates 
prefixes from its expressions in the case of method invocations. The prefixes 
found is registered with a prefix set, together with all its sub-prefixes.
\todo{Rewrite in the case of changes to the way prefixes are found}

\subsubsection{The UnfixesCollector}
The \typewithref{no.uio.ifi.refaktor.extractors.collectors}{UnfixesCollector} 
finds unfixes within the selection. An unfix is a name that is assigned to 
within the selection. The reason that this cannot be allowed, is that the result 
would be an assignment to the \type{this} keyword, which is not valid in Java.

\subsubsection{Computing Safe Prefixes}
A safe prefix is a prefix that does not enclose an unfix. A prefix is enclosing 
an unfix if the unfix is in the set of its sub-prefixes. As an example, 
\texttt{``a.b''} is enclosing \texttt{``a''}, as is \texttt{``a''}. The safe 
prefixes is unified in a \type{PrefixSet} and can be fetched calling the 
\method{getSafePrefixes} method of the 
\type{ExtractAndMoveMethodPrefixesExtractor}.

\subsection{The Prefix Class}
\todo{?}
\subsection{The PrefixSet Class}

\subsection{Hacking the Refactoring Undo 
History}\label{hacking_undo_history}
\todo{Where to put this section?}

As an attempt to make multiple subsequent changes to the workspace appear as a 
single action (i.e. make the undo changes appear as such), I tried to alter 
the undo changes\typeref{org.eclipse.ltk.core.refactoring.Change} in the history 
of the refactorings.  

My first impulse was to remove the, in this case, last two undo changes from the 
undo manager\typeref{org.eclipse.ltk.core.refactoring.IUndoManager} for the 
Eclipse refactorings, and then add them to a composite 
change\typeref{org.eclipse.ltk.core.refactoring.CompositeChange} that could be 
added back to the manager. The interface of the undo manager does not offer a 
way to remove/pop the last added undo change, so a possible solution could be to 
decorate \cite{dp} the undo manager, to intercept and collect the undo changes 
before delegating to the \method{addUndo} 
method\methodref{org.eclipse.ltk.core.refactoring.IUndoManager}{addUndo} of the 
manager. Instead of giving it the intended undo change, a null change could be 
given to prevent it from making any changes if run. Then one could let the 
collected undo changes form a composite change to be added to the manager.

There is a technical challenge with this approach, and it relates to the undo 
manager, and the concrete implementation 
UndoManager2\typeref{org.eclipse.ltk.internal.core.refactoring.UndoManager2}.  
This implementation is designed in a way that it is not possible to just add an 
undo change, you have to do it in the context of an active 
operation\typeref{org.eclipse.core.commands.operations.TriggeredOperations}.  
One could imagine that it might be possible to trick the undo manager into 
believing that you are doing a real change, by executing a refactoring that is 
returning a kind of null change that is returning our composite change of undo 
refactorings when it is performed.

Apart from the technical problems with this solution, there is a functional 
problem: If it all had worked out as planned, this would leave the undo history 
in a dirty state, with multiple empty undo operations corresponding to each of 
the sequentially executed refactoring operations, followed by a composite undo 
change corresponding to an empty change of the workspace for rounding of our 
composite refactoring. The solution to this particular problem could be to 
intercept the registration of the intermediate changes in the undo manager, and 
only register the last empty change.

Unfortunately, not everything works as desired with this solution. The grouping 
of the undo changes into the composite change does not make the undo operation 
appear as an atomic operation. The undo operation is still split up into 
separate undo actions, corresponding to the change done by its originating
refactoring. And in addition, the undo actions has to be performed separate in 
all the editors involved. This makes it no solution at all, but a step toward 
something worse.

There might be a solution to this problem, but it remains to be found. The 
design of the refactoring undo management is partly to be blamed for this, as it 
it is to complex to be easily manipulated.


\backmatter{}
\printbibliography
\listoftodos
\end{document}