1 \documentclass[USenglish,11pt]{ifimaster}
3 \usepackage[utf8]{inputenc}
4 \usepackage[T1]{fontenc,url}
5 \usepackage{lmodern} % using Latin Modern to be able to use bold typewriter font
12 \usepackage{tikz-qtree}
13 \usetikzlibrary{shapes,snakes,trees,arrows,shadows,positioning,calc}
14 \usepackage{babel,textcomp,csquotes,ifimasterforside}
17 \usepackage[hidelinks]{hyperref}
19 \usepackage[xindy]{glossaries}
21 \usepackage[style=alphabetic,backend=biber,doi=false,isbn=false]{biblatex}
23 \usepackage{mathtools}
25 % use 'disable' before printing:
26 \usepackage[]{todonotes}
34 \def\mintedframesep{11pt}
36 \usepackage{perpage} %the perpage package
37 \MakePerPage{footnote} %the perpage package command
39 \theoremstyle{definition}
40 \newtheorem*{wordDef}{Definition}
41 \newtheorem*{theorem}{Theorem}
43 \graphicspath{ {./figures/} }
45 \newcommand{\citing}[1]{~\cite{#1}}
46 %\newcommand{\myref}[1]{\cref{#1} on \cpageref{#1}}
47 \newcommand{\myref}[1]{\vref{#1}}
48 \newcommand{\Myref}[1]{\Vref{#1}}
50 %\newcommand{\glossref}[1]{\textsuperscript{(\glsrefentry{#1})}}
51 %\newcommand{\gloss}[1]{\gls{#1}\glossref{#1}}
52 %\newcommand{\glosspl}[1]{\glspl{#1}\glossref{#1}}
53 \newcommand{\gloss}[1]{\gls{#1}}
54 \newcommand{\glosspl}[1]{\glspl{#1}}
56 \newcommand{\definition}[1]{\begin{wordDef}#1\end{wordDef}}
57 \newcommand{\see}[1]{(see \myref{#1})}
58 \newcommand{\explanation}[3]{\noindent\textbf{\textit{#1}}\\*\emph{When:}
59 #2\\*\emph{How:} #3\\*[-7px]}
61 %\newcommand{\type}[1]{\lstinline{#1}}
62 \newcommand{\code}[1]{\texttt{\textbf{#1}}}
63 \newcommand{\type}[1]{\code{#1}}
64 \newcommand{\typeref}[1]{\footnote{\type{#1}}}
65 \newcommand{\typewithref}[2]{\type{#2}\typeref{#1.#2}}
66 \newcommand{\method}[1]{\type{#1}}
67 \newcommand{\methodref}[2]{\footnote{\type{#1}\method{\##2()}}}
68 \newcommand{\methodwithref}[2]{\method{#2}\footnote{\type{#1}\method{\##2()}}}
69 \newcommand{\var}[1]{\type{#1}}
71 \newcommand{\name}[1]{#1}
72 \newcommand{\tit}[1]{\emph{#1}}
73 \newcommand{\refa}[1]{\emph{#1}}
74 \newcommand{\pattern}[1]{\emph{#1}}
75 \newcommand{\metr}[1]{\emph{#1}}
76 \newcommand{\ExtractMethod}{\refa{Extract Method}\xspace}
77 \newcommand{\MoveMethod}{\refa{Move Method}\xspace}
78 \newcommand{\ExtractAndMoveMethod}{\refa{Extract and Move Method}\xspace}
80 \newcommand{\m}[1]{$#1$}
82 \newcommand\todoin[2][]{\todo[inline, caption={#2}, #1]{
83 \begin{minipage}{\textwidth-4pt}#2\end{minipage}}}
85 \title{Automated Composition of Refactorings}
86 \subtitle{Implementing and evaluating a search-based Extract and Move Method
88 \author{Erlend Kristiansen}
91 \newglossaryentry{profiling}
94 description={is to run a computer program through a profiler/with a profiler
97 \newglossaryentry{profiler}
100 description={A profiler is a program for analyzing performance within an
101 application. It is used to analyze memory consumption, processing time and
102 frequency of procedure calls and such}
104 \newglossaryentry{xUnit}
106 name={xUnit framework},
107 description={An xUnit framework is a framework for writing unit tests for a
108 computer program. It follows the patterns known from the JUnit framework for
109 Java\citing{fowlerXunit}
111 plural={xUnit frameworks}
113 \newglossaryentry{softwareObfuscation}
115 name={software obfuscation},
116 description={makes source code harder to read and analyze, while preserving
119 \newglossaryentry{extractClass}
121 name=\refa{Extract Class},
122 description={The \refa{Extract Class} refactoring works by creating a class,
123 for then to move members from another class to that class and access them from
124 the old class via a reference to the new class}
126 \newglossaryentry{designPattern}
128 name={design pattern},
129 description={A design pattern is a named abstraction, that is meant to solve a
130 general design problem. It describes the key aspects of a common problem and
131 identifies its participators and how they collaborate},
132 plural={design patterns}
134 \newglossaryentry{enclosingClass}
136 name={enclosing class},
137 description={An enclosing class is the class that surrounds any specific piece
138 of code that is written in the inner scope of this class},
140 \newglossaryentry{mementoPattern}
142 name={memento pattern},
143 description={The memento pattern is a software design pattern that is used to
144 capture an object's internal state so that it can be restored to this state
145 later\citing{designPatterns}},
147 %\newglossaryentry{extractMethod}
149 % name=\refa{Extract Method},
150 % description={The \refa{Extract Method} refactoring is used to extract a
151 %fragment of code from its context and into a new method. A call to the new
152 %method is inlined where the fragment was before. It is used to break code into
153 %logical units, with names that explain their purpose}
155 %\newglossaryentry{moveMethod}
157 % name=\refa{Move Method},
158 % description={The \refa{Move Method} refactoring is used to move a method from
159 % one class to another. This is useful if the method is using more features of
160 % another class than of the class which it is currently defined. Then all calls
161 % to this method must be updated, or the method must be copied, with the old
162 %method delegating to the new method}
165 \bibliography{bibliography/master-thesis-erlenkr-bibliography}
166 \DefineBibliographyStrings{english}{%
167 bibliography = {References},
169 \newbibmacro{string+doi}[1]{%
170 \iffieldundef{doi}{#1}{\href{http://dx.doi.org/\thefield{doi}}{#1}}}
171 \DeclareFieldFormat{title}{\usebibmacro{string+doi}{\mkbibemph{#1}}}
172 \DeclareFieldFormat[article]{title}{\usebibmacro{string+doi}{\mkbibquote{#1}}}
174 % UML comment in TikZ:
175 % ref: https://tex.stackexchange.com/questions/103688/folded-paper-shape-tikz
177 \pgfdeclareshape{umlcomment}{
178 \inheritsavedanchors[from=rectangle] % this is nearly a rectangle
179 \inheritanchorborder[from=rectangle]
180 \inheritanchor[from=rectangle]{center}
181 \inheritanchor[from=rectangle]{north}
182 \inheritanchor[from=rectangle]{south}
183 \inheritanchor[from=rectangle]{west}
184 \inheritanchor[from=rectangle]{east}
185 % ... and possibly more
186 \backgroundpath{% this is new
187 % store lower right in xa/ya and upper right in xb/yb
188 \southwest \pgf@xa=\pgf@x \pgf@ya=\pgf@y
189 \northeast \pgf@xb=\pgf@x \pgf@yb=\pgf@y
190 % compute corner of ‘‘flipped page’’
191 \pgf@xc=\pgf@xb \advance\pgf@xc by-10pt % this should be a parameter
192 \pgf@yc=\pgf@yb \advance\pgf@yc by-10pt
193 % construct main path
194 \pgfpathmoveto{\pgfpoint{\pgf@xa}{\pgf@ya}}
195 \pgfpathlineto{\pgfpoint{\pgf@xa}{\pgf@yb}}
196 \pgfpathlineto{\pgfpoint{\pgf@xc}{\pgf@yb}}
197 \pgfpathlineto{\pgfpoint{\pgf@xb}{\pgf@yc}}
198 \pgfpathlineto{\pgfpoint{\pgf@xb}{\pgf@ya}}
201 \pgfpathmoveto{\pgfpoint{\pgf@xc}{\pgf@yb}}
202 \pgfpathlineto{\pgfpoint{\pgf@xc}{\pgf@yc}}
203 \pgfpathlineto{\pgfpoint{\pgf@xb}{\pgf@yc}}
204 \pgfpathlineto{\pgfpoint{\pgf@xc}{\pgf@yc}}
209 \tikzstyle{comment}=[%
222 %\interfootnotelinepenalty=10000
224 % Space between table rows
225 \renewcommand{\arraystretch}{1.3}
227 \newcommand{\spancols}[2]{\multicolumn{#1}{@{}l@{}}{#2}}
229 \newcolumntype{L}[1]{>{\hsize=#1\hsize\raggedright\arraybackslash}X}%
230 \newcolumntype{R}[1]{>{\hsize=#1\hsize\raggedleft\arraybackslash}X}%
234 %\pagenumbering{arabic}
239 %\setcounter{page}{3}
242 \todoin{\textbf{Remove all todos (including list) before delivery/printing!!!
243 Can be done by removing ``draft'' from documentclass.}}
244 \todoin{Write abstract}
252 %\setcounter{page}{13}
254 \chapter{Introduction}
256 \section{Motivation and structure}
258 For large software projects, complex program source code is an issue. It impacts
259 the cost of maintenance in a negative way. It often stalls the implementation of
260 new functionality and other program changes. The code may be difficult to
261 understand, the changes may introduce new bugs that are hard to find and its
262 complexity can simply keep people from doing code changes in fear of breaking
263 some dependent piece of code. All these problems are related, and often lead to
264 a vicious circle that slowly degrades the overall quality of a project.
266 More specifically, and in an object-oriented context, a class may depend on a
267 number of other classes. Sometimes these intimate relationships are appropriate,
268 and sometimes they are not. Inappropriate \emph{coupling} between classes can
269 make it difficult to know whether or not a change that is aimed at fixing a
270 specific problem also alters the behavior of another part of a program.
272 One of the tools that are used to fight complexity and coupling in program
273 source code is \emph{refactoring}. The intention for this master's thesis is
274 therefore to create an automated composite refactoring that reduces coupling
275 between classes. The refactoring shall be able to operate automatically in all
276 phases of a refactoring, from performing analysis to executing changes. It is
277 also a requirement that it should be able to process large quantities of source
278 code in a reasonable amount of time.
281 \todoin{Structure. Write later\ldots}
284 \section{What is refactoring?}
286 This question is best answered by first defining the concept of a
287 \emph{refactoring}, what it is to \emph{refactor}, and then discuss what aspects
288 of programming make people want to refactor their code.
290 \subsection{Defining refactoring}
291 Martin Fowler, in his classic book on refactoring\citing{refactoring}, defines a
292 refactoring like this:
295 \emph{Refactoring} (noun): a change made to the internal
296 structure\footnote{The structure observable by the programmer.} of software to
297 make it easier to understand and cheaper to modify without changing its
298 observable behavior.~\cite[p.~53]{refactoring}
301 \noindent This definition assigns additional meaning to the word
302 \emph{refactoring}, beyond the composition of the prefix \emph{re-}, usually
303 meaning something like ``again'' or ``anew'', and the word \emph{factoring},
304 that can mean to isolate the \emph{factors} of something. Here a \emph{factor}
305 would be close to the mathematical definition of something that divides a
306 quantity, without leaving a remainder. Fowler is mixing the \emph{motivation}
307 behind refactoring into his definition. Instead it could be more refined, formed
308 to only consider the \emph{mechanical} and \emph{behavioral} aspects of
309 refactoring. That is to factor the program again, putting it together in a
310 different way than before, while preserving the behavior of the program. An
311 alternative definition could then be:
313 \definition{A \emph{refactoring} is a transformation
314 done to a program without altering its external behavior.}
316 From this we can conclude that a refactoring primarily changes how the
317 \emph{code} of a program is perceived by the \emph{programmer}, and not the
318 \emph{behavior} experienced by any user of the program. Although the logical
319 meaning is preserved, such changes could potentially alter the program's
320 behavior when it comes to performance gain or -penalties. So any logic depending
321 on the performance of a program could make the program behave differently after
324 In the extreme case one could argue that \gloss{softwareObfuscation} is
325 refactoring. It is often used to protect proprietary software. It restrains
326 uninvited viewers, so they have a hard time analyzing code that they are not
327 supposed to know how works. This could be a problem when using a language that
328 is possible to decompile, such as Java.
330 Obfuscation could be done composing many, more or less randomly chosen,
331 refactorings. Then the question arises whether it can be called a
332 \emph{composite refactoring} or not \see{compositeRefactorings}? The answer is
333 not obvious. First, there is no way to describe the mechanics of software
334 obfuscation, because there are infinitely many ways to do that. Second,
335 obfuscation can be thought of as \emph{one operation}: Either the code is
336 obfuscated, or it is not. Third, it makes no sense to call software obfuscation
337 \emph{a refactoring}, since it holds different meaning to different people.
339 This last point is important, since one of the motivations behind defining
340 different refactorings, is to establish a \emph{vocabulary} for software
341 professionals to use when reasoning about and discussing programs, similar to
342 the motivation behind \glosspl{designPattern}\citing{designPatterns}.
344 So for describing \emph{software obfuscation}, it might be more appropriate to
345 define what you do when performing it rather than precisely defining its
346 mechanics in terms of other refactorings.
349 \subsection{The etymology of 'refactoring'}
350 It is a little difficult to pinpoint the exact origin of the word
351 ``refactoring'', as it seems to have evolved as part of a colloquial
352 terminology, more than a scientific term. There is no authoritative source for a
353 formal definition of it.
355 According to Martin Fowler\citing{etymology-refactoring}, there may also be more
356 than one origin of the word. The most well-known source, when it comes to the
357 origin of \emph{refactoring}, is the
358 Smalltalk\footnote{\label{footNote}Programming language} community and their
359 infamous \name{Refactoring
360 Browser}\footnote{\url{http://st-www.cs.illinois.edu/users/brant/Refactory/RefactoringBrowser.html}}
361 described in the article \tit{A Refactoring Tool for
362 Smalltalk}\citing{refactoringBrowser1997}, published in 1997.
363 Allegedly\citing{etymology-refactoring}, the metaphor of factoring programs was
364 also present in the Forth\textsuperscript{\ref{footNote}} community, and the
365 word ``refactoring'' is mentioned in a book by Leo Brodie, called \tit{Thinking
366 Forth}\citing{brodie2004}, first published in 1984\footnote{\tit{Thinking Forth}
367 was first published in 1984 by the \name{Forth Interest Group}. Then it was
368 reprinted in 1994 with minor typographical corrections, before it was
369 transcribed into an electronic edition typeset in \LaTeX\ and published under a
370 Creative Commons licence in
371 2004. The edition cited here is the 2004 edition, but the content should
372 essentially be as in 1984.}. The exact word is only printed one
373 place~\cite[p.~232]{brodie2004}, but the term \emph{factoring} is prominent in
374 the book, that also contains a whole chapter dedicated to (re)factoring, and how
375 to keep the (Forth) code clean and maintainable.
378 \ldots good factoring technique is perhaps the most important skill for a
379 Forth programmer.~\cite[p.~172]{brodie2004}
382 \noindent Brodie also express what \emph{factoring} means to him:
385 Factoring means organizing code into useful fragments. To make a fragment
386 useful, you often must separate reusable parts from non-reusable parts. The
387 reusable parts become new definitions. The non-reusable parts become arguments
388 or parameters to the definitions.~\cite[p.~172]{brodie2004}
391 Fowler claims that the usage of the word \emph{refactoring} did not pass between
392 the \name{Forth} and \name{Smalltalk} communities, but that it emerged
393 independently in each of the communities.
395 \subsection{Reasons for refactoring}
396 There are many reasons why people want to refactor their programs. They can for
397 instance do it to remove duplication, break up long methods or to introduce
398 design patterns into their software systems. The shared trait for all these are
399 that peoples' intentions are to make their programs \emph{better}, in some
400 sense. But what aspects of their programs are becoming improved?
402 As just mentioned, people often refactor to get rid of duplication. They are
403 moving identical or similar code into methods, and are pushing methods up or
404 down in their class hierarchies. They are making template methods for
405 overlapping algorithms/functionality, and so on. It is all about gathering what
406 belongs together and putting it all in one place. The resulting code is then
407 easier to maintain. When removing the implicit coupling\footnote{When
408 duplicating code, the duplicate pieces of code might not be coupled, apart
409 from representing the same functionality. So if this functionality is going to
410 change, it might need to change in more than one place, thus creating an
411 implicit coupling between multiple pieces of code.} between code snippets, the
412 location of a bug is limited to only one place, and new functionality need only
413 to be added to this one place, instead of a number of places people might not
416 A problem you often encounter when programming, is that a program contains a lot
417 of long and hard-to-grasp methods. It can then help to break the methods into
418 smaller ones, using the \ExtractMethod refactoring\citing{refactoring}. Then
419 you may discover something about a program that you were not aware of before;
420 revealing bugs you did not know about or could not find due to the complex
421 structure of your program. Making the methods smaller and giving good names to
422 the new ones clarifies the algorithms and enhances the \emph{understandability}
423 of the program \see{magic_number_seven}. This makes refactoring an excellent
424 method for exploring unknown program code, or code that you had forgotten that
427 Most primitive refactorings are simple, and usually involves moving code
428 around\citing{kerievsky2005}. The motivation behind them may first be revealed
429 when they are combined into larger --- higher level --- refactorings, called
430 \emph{composite refactorings} \see{compositeRefactorings}. Often the goal of
431 such a series of refactorings is a design pattern. Thus the design can
432 \emph{evolve} throughout the lifetime of a program, as opposed to designing
433 up-front. It is all about being structured and taking small steps to improve a
436 Many software design pattern are aimed at lowering the coupling between
437 different classes and different layers of logic. One of the most famous is
438 perhaps the \pattern{Model-View-Controller}\citing{designPatterns} pattern. It
439 is aimed at lowering the coupling between the user interface, the business logic
440 and the data representation of a program. This also has the added benefit that
441 the business logic could much easier be the target of automated tests, thus
442 increasing the productivity in the software development process.
444 Another effect of refactoring is that with the increased separation of concerns
445 coming out of many refactorings, the \emph{performance} can be improved. When
446 profiling programs, the problematic parts are narrowed down to smaller parts of
447 the code, which are easier to tune, and optimization can be performed only where
448 needed and in a more effective way\citing{refactoring}.
450 Last, but not least, and this should probably be the best reason to refactor, is
451 to refactor to \emph{facilitate a program change}. If one has managed to keep
452 one's code clean and tidy, and the code is not bloated with design patterns that
453 are not ever going to be needed, then some refactoring might be needed to
454 introduce a design pattern that is appropriate for the change that is going to
457 Refactoring program code --- with a goal in mind --- can give the code itself
458 more value. That is in the form of robustness to bugs, understandability and
459 maintainability. Having robust code is an obvious advantage, but
460 understandability and maintainability are both very important aspects of
461 software development. By incorporating refactoring in the development process,
462 bugs are found faster, new functionality is added more easily and code is easier
463 to understand by the next person exposed to it, which might as well be the
464 person who wrote it. The consequence of this, is that refactoring can increase
465 the average productivity of the development process, and thus also add to the
466 monetary value of a business in the long run. The perspective on productivity
467 and money should also be able to open the eyes of the many nearsighted managers
468 that seldom see beyond the next milestone.
470 \subsection{The magical number seven}\label{magic_number_seven}
471 The article \tit{The magical number seven, plus or minus two: some limits on our
472 capacity for processing information}\citing{miller1956} by George A. Miller,
473 was published in the journal \name{Psychological Review} in 1956. It presents
474 evidence that support that the capacity of the number of objects a human being
475 can hold in its working memory is roughly seven, plus or minus two objects. This
476 number varies a bit depending on the nature and complexity of the objects, but
477 is according to Miller ``\ldots never changing so much as to be
480 Miller's article culminates in the section called \emph{Recoding}, a term he
481 borrows from communication theory. The central result in this section is that by
482 recoding information, the capacity of the amount of information that a human can
483 process at a time is increased. By \emph{recoding}, Miller means to group
484 objects together in chunks, and give each chunk a new name that it can be
488 \ldots recoding is an extremely powerful weapon for increasing the amount of
489 information that we can deal with.~\cite[p.~95]{miller1956}
492 By organizing objects into patterns of ever growing depth, one can memorize and
493 process a much larger amount of data than if it were to be represented as its
494 basic pieces. This grouping and renaming is analogous to how many refactorings
495 work, by grouping pieces of code and give them a new name. Examples are the
496 fundamental \ExtractMethod and \refa{Extract Class}
497 refactorings\citing{refactoring}.
499 An example from the article addresses the problem of memorizing a sequence of
500 binary digits. The example presented here is a slightly modified version of the
501 one presented in the original article\citing{miller1956}, but it preserves the
502 essence of it. Let us say we have the following sequence of
503 16 binary digits: ``1010001001110011''. Most of us will have a hard time
504 memorizing this sequence by only reading it once or twice. Imagine if we instead
505 translate it to this sequence: ``A273''. If you have a background from computer
506 science, it will be obvious that the latter sequence is the first sequence
507 recoded to be represented by digits in base 16. Most people should be able to
508 memorize this last sequence by only looking at it once.
510 Another result from the Miller article is that when the amount of information a
511 human must interpret increases, it is crucial that the translation from one code
512 to another must be almost automatic for the subject to be able to remember the
513 translation, before \heshe is presented with new information to recode. Thus
514 learning and understanding how to best organize certain kinds of data is
515 essential to efficiently handle that kind of data in the future. This is much
516 like when humans learn to read. First they must learn how to recognize letters.
517 Then they can learn distinct words, and later read sequences of words that form
518 whole sentences. Eventually, most of them will be able to read whole books and
519 briefly retell the important parts of its content. This suggest that the use of
520 design patterns is a good idea when reasoning about computer programs. With
521 extensive use of design patterns when creating complex program structures, one
522 does not always have to read whole classes of code to comprehend how they
523 function, it may be sufficient to only see the name of a class to almost fully
524 understand its responsibilities.
527 Our language is tremendously useful for repackaging material into a few chunks
528 rich in information.~\cite[p.~95]{miller1956}
531 Without further evidence, these results at least indicate that refactoring
532 source code into smaller units with higher cohesion and, when needed,
533 introducing appropriate design patterns, should aid in the cause of creating
534 computer programs that are easier to maintain and have code that is easier (and
537 \subsection{Notable contributions to the refactoring literature}
540 \item[1992] William F. Opdyke submits his doctoral dissertation called
541 \tit{Refactoring Object-Oriented Frameworks}\citing{opdyke1992}. This work
542 defines a set of refactorings, that are behavior preserving given that their
543 preconditions are met. The dissertation is focused on the automation of
545 \item[1999] Martin Fowler et al.: \tit{Refactoring: Improving the Design of
546 Existing Code}\citing{refactoring}. This is maybe the most influential text
547 on refactoring. It bares similarities with Opdykes thesis\citing{opdyke1992}
548 in the way that it provides a catalog of refactorings. But Fowler's book is
549 more about the craft of refactoring, as he focuses on establishing a
550 vocabulary for refactoring, together with the mechanics of different
551 refactorings and when to perform them. His methodology is also founded on
552 the principles of test-driven development.
553 \item[2005] Joshua Kerievsky: \tit{Refactoring to
554 Patterns}\citing{kerievsky2005}. This book is heavily influenced by Fowler's
555 \tit{Refactoring}\citing{refactoring} and the ``Gang of Four'' \tit{Design
556 Patterns}\citing{designPatterns}. It is building on the refactoring
557 catalogue from Fowler's book, but is trying to bridge the gap between
558 \emph{refactoring} and \emph{design patterns} by providing a series of
559 higher-level composite refactorings, that makes code evolve toward or away
560 from certain design patterns. The book is trying to build up the reader's
561 intuition around \emph{why} one would want to use a particular design
562 pattern, and not just \emph{how}. The book is encouraging evolutionary
563 design \see{relationToDesignPatterns}.
566 \subsection{Tool support (for Java)}\label{toolSupport}
567 This section will briefly compare the refactoring support of the three IDEs
568 \name{Eclipse}\footnote{\url{http://www.eclipse.org/}}, \name{IntelliJ
569 IDEA}\footnote{The IDE under comparison is the \name{Community Edition},
570 \url{http://www.jetbrains.com/idea/}} and
571 \name{NetBeans}\footnote{\url{https://netbeans.org/}}. These are the most
572 popular Java IDEs\citing{javaReport2011}.
574 All three IDEs provide support for the most useful refactorings, like the
575 different extract, move and rename refactorings. In fact, Java-targeted IDEs are
576 known for their good refactoring support, so this did not appear as a big
579 The IDEs seem to have excellent support for the \ExtractMethod refactoring, so
580 at least they have all passed the first ``refactoring
581 rubicon''\citing{fowlerRubicon2001,secondRubicon2012}.
583 Regarding the \MoveMethod refactoring, the \name{Eclipse} and \name{IntelliJ}
584 IDEs do the job in very similar manners. In most situations they both do a
585 satisfying job by producing the expected outcome. But they do nothing to check
586 that the result does not break the semantics of the program \see{correctness}.
587 The \name{NetBeans} IDE implements this refactoring in a somewhat
588 unsophisticated way. For starters, the refactoring's default destination for the
589 move, is the same class as the method already resides in, although it refuses to
590 perform the refactoring if chosen. But the worst part is, that if moving the
591 method \method{f} of the class \type{C} to the class \type{X}, it will break the
592 code. The result is shown in \myref{lst:moveMethod_NetBeans}.
596 \begin{minted}[samepage]{java}
609 \begin{minted}[samepage]{java}
619 \caption{Moving method \method{f} from \type{C} to \type{X}.}
620 \label{lst:moveMethod_NetBeans}
623 \name{NetBeans} will try to create code that call the methods \method{m} and \method{n}
624 of \type{X} by accessing them through \var{c.x}, where \var{c} is a parameter of
625 type \type{C} that is added the method \method{f} when it is moved. (This is
626 seldom the desired outcome of this refactoring, but ironically, this ``feature''
627 keeps \name{NetBeans} from breaking the code in the example from \myref{correctness}.)
628 If \var{c.x} for some reason is inaccessible to \type{X}, as in this case, the
629 refactoring breaks the code, and it will not compile. \name{NetBeans} presents a
630 preview of the refactoring outcome, but the preview does not catch it if the IDE
631 is about break the program.
633 The IDEs under investigation seem to have fairly good support for primitive
634 refactorings, but what about more complex ones, such as
635 \gloss{extractClass}\citing{refactoring}? \name{IntelliJ} handles this in a
636 fairly good manner, although, in the case of private methods, it leaves unused
637 methods behind. These are methods that delegate to a field with the type of the
638 new class, but are not used anywhere. \name{Eclipse} has added its own quirk to
639 the \refa{Extract Class} refactoring, and only allows for \emph{fields} to be
640 moved to a new class, \emph{not methods}. This makes it effectively only
641 extracting a data structure, and calling it \refa{Extract Class} is a little
642 misleading. One would often be better off with textual extract and paste than
643 using the \refa{Extract Class} refactoring in \name{Eclipse}. When it comes to
644 \name{NetBeans}, it does not even show an attempt on providing this refactoring.
646 \subsection{The relation to design patterns}\label{relationToDesignPatterns}
648 Refactoring and design patterns have at least one thing in common, they are both
649 promoted by advocates of \emph{clean code}\citing{cleanCode} as fundamental
650 tools on the road to more maintainable and extendable source code.
653 Design patterns help you determine how to reorganize a design, and they can
654 reduce the amount of refactoring you need to do
655 later.~\cite[p.~353]{designPatterns}
658 Although sometimes associated with
659 over-engineering\citing{kerievsky2005,refactoring}, design patterns are in
660 general assumed to be good for maintainability of source code. That may be
661 because many of them are designed to support the \emph{open/closed principle} of
662 object-oriented programming. The principle was first formulated by Bertrand
663 Meyer, the creator of the Eiffel programming language, like this: ``Modules
664 should be both open and closed.''\citing{meyer1988} It has been popularized,
665 with this as a common version:
668 Software entities (classes, modules, functions, etc.) should be open for
669 extension, but closed for modification.\footnote{See
670 \url{http://c2.com/cgi/wiki?OpenClosedPrinciple} or
671 \url{https://en.wikipedia.org/wiki/Open/closed_principle}}
674 Maintainability is often thought of as the ability to be able to introduce new
675 functionality without having to change too much of the old code. When
676 refactoring, the motivation is often to facilitate adding new functionality. It
677 is about factoring the old code in a way that makes the new functionality being
678 able to benefit from the functionality already residing in a software system,
679 without having to copy old code into new. Then, next time someone shall add new
680 functionality, it is less likely that the old code has to change. Assuming that
681 a design pattern is the best way to get rid of duplication and assist in
682 implementing new functionality, it is reasonable to conclude that a design
683 pattern often is the target of a series of refactorings. Having a repertoire of
684 design patterns can also help in knowing when and how to refactor a program to
685 make it reflect certain desired characteristics.
688 There is a natural relation between patterns and refactorings. Patterns are
689 where you want to be; refactorings are ways to get there from somewhere
690 else.~\cite[p.~107]{refactoring}
693 This quote is wise in many contexts, but it is not always appropriate to say
694 ``Patterns are where you want to be\ldots''. \emph{Sometimes}, patterns are
695 where you want to be, but only because it will benefit your design. It is not
696 true that one should always try to incorporate as many design patterns as
697 possible into a program. It is not like they have intrinsic value. They only add
698 value to a system when they support its design. Otherwise, the use of design
699 patterns may only lead to a program that is more complex than necessary.
702 The overuse of patterns tends to result from being patterns happy. We are
703 \emph{patterns happy} when we become so enamored of patterns that we simply
704 must use them in our code.~\cite[p.~24]{kerievsky2005}
707 This can easily happen when relying largely on up-front design. Then it is
708 natural, in the very beginning, to try to build in all the flexibility that one
709 believes will be necessary throughout the lifetime of a software system.
710 According to Joshua Kerievsky ``That sounds reasonable --- if you happen to be
711 psychic.''~\cite[p.~1]{kerievsky2005} He is advocating what he believes is a
712 better approach: To let software continually evolve. To start with a simple
713 design that meets today's needs, and tackle future needs by refactoring to
714 satisfy them. He believes that this is a more economic approach than investing
715 time and money into a design that inevitably is going to change. By relying on
716 continuously refactoring a system, its design can be made simpler without
717 sacrificing flexibility. To be able to fully rely on this approach, it is of
718 utter importance to have a reliable suit of tests to lean on \see{testing}. This
719 makes the design process more natural and less characterized by difficult
720 decisions that has to be made before proceeding in the process, and that is
721 going to define a project for all of its unforeseeable future.
723 \subsection{The impact on software quality}
725 \subsubsection{What is software quality?}
726 The term \emph{software quality} has many meanings. It all depends on the
727 context we put it in. If we look at it with the eyes of a software developer, it
728 usually means that the software is easily maintainable and testable, or in other
729 words, that it is \emph{well designed}. This often correlates with the
730 management scale, where \emph{keeping the schedule} and \emph{customer
731 satisfaction} is at the center. From the customers point of view, in addition to
732 good usability, \emph{performance} and \emph{lack of bugs} is always
733 appreciated, measurements that are also shared by the software developer. (In
734 addition, such things as good documentation could be measured, but this is out
735 of the scope of this document.)
737 \subsubsection{The impact on performance}
739 Refactoring certainly will make software go more slowly\footnote{With todays
740 compiler optimization techniques and performance tuning of e.g. the Java
741 virtual machine, the penalties of object creation and method calls are
742 debatable.}, but it also makes the software more amenable to performance
743 tuning.~\cite[p.~69]{refactoring}
746 \noindent There is a common belief that refactoring compromises performance, due
747 to increased degree of indirection and that polymorphism is slower than
750 In a survey, Demeyer\citing{demeyer2002} disproves this view in the case of
751 polymorphism. He did an experiment on, what he calls, ``Transform Self Type
752 Checks'' where you introduce a new polymorphic method and a new class hierarchy
753 to get rid of a class' type checking of a ``type attribute``. He uses this kind
754 of transformation to represent other ways of replacing conditionals with
755 polymorphism as well. The experiment is performed on the C++ programming
756 language and with three different compilers and platforms. Demeyer concludes
757 that, with compiler optimization turned on, polymorphism beats middle to large
758 sized if-statements and does as well as case-statements. (In accordance with
759 his hypothesis, due to similarities between the way C++ handles polymorphism and
763 The interesting thing about performance is that if you analyze most programs,
764 you find that they waste most of their time in a small fraction of the
765 code.~\cite[p.~70]{refactoring}
768 \noindent So, although an increased amount of method calls could potentially
769 slow down programs, one should avoid premature optimization and sacrificing good
770 design, leaving the performance tuning until after \gloss{profiling} the
771 software and having isolated the actual problem areas.
773 \subsection{Composite refactorings}\label{compositeRefactorings}
774 Generally, when thinking about refactoring, at the mechanical level, there are
775 essentially two kinds of refactorings. There are the \emph{primitive}
776 refactorings, and the \emph{composite} refactorings.
778 \definition{A \emph{primitive refactoring} is a refactoring that cannot be
779 expressed in terms of other refactorings.}
781 \noindent Examples are the \refa{Pull Up Field} and \refa{Pull Up
782 Method} refactorings\citing{refactoring}, that move members up in their class
785 \definition{A \emph{composite refactoring} is a refactoring that can be
786 expressed in terms of two or more other refactorings.}
788 \noindent An example of a composite refactoring is the \refa{Extract
789 Superclass} refactoring\citing{refactoring}. In its simplest form, it is composed
790 of the previously described primitive refactorings, in addition to the
791 \refa{Pull Up Constructor Body} refactoring\citing{refactoring}. It works
792 by creating an abstract superclass that the target class(es) inherits from, then
793 by applying \refa{Pull Up Field}, \refa{Pull Up Method} and
794 \refa{Pull Up Constructor Body} on the members that are to be members of
795 the new superclass. If there are multiple classes in play, their interfaces may
796 need to be united with the help of some rename refactorings, before extracting
797 the superclass. For an overview of the \refa{Extract Superclass}
798 refactoring, see \myref{fig:extractSuperclass}.
802 \includegraphics[angle=270,width=\linewidth]{extractSuperclassItalic.pdf}
803 \caption{The Extract Superclass refactoring, with united interfaces.}
804 \label{fig:extractSuperclass}
807 \subsection{Manual vs. automated refactorings}
808 Refactoring is something every programmer does, even if \heshe does not known
809 the term \emph{refactoring}. Every refinement of source code that does not alter
810 the program's behavior is a refactoring. For small refactorings, such as
811 \ExtractMethod, executing it manually is a manageable task, but is still prone
812 to errors. Getting it right the first time is not easy, considering the method
813 signature and all the other aspects of the refactoring that has to be in place.
815 Consider the renaming of classes, methods and fields. For complex programs these
816 refactorings are almost impossible to get right. Attacking them with textual
817 search and replace, or even regular expressions, will fall short on these tasks.
818 Then it is crucial to have proper tool support that can perform them
819 automatically. Tools that can parse source code and thus have semantic knowledge
820 about which occurrences of which names belong to what construct in the program.
821 For even trying to perform one of these complex task manually, one would have to
822 be very confident on the existing test suite \see{testing}.
824 \subsection{Correctness of refactorings}\label{correctness}
825 For automated refactorings to be truly useful, they must show a high degree of
826 behavior preservation. This last sentence might seem obvious, but there are
827 examples of refactorings in existing tools that break programs. In an ideal
828 world, every automated refactoring would be ``complete'', in the sense that it
829 would never break a program. In an ideal world, every program would also be free
830 from bugs. In modern IDEs the implemented automated refactorings are working for
831 \emph{most} cases, that is enough for making them useful.
833 I will now present an example of a \emph{corner case} where a program breaks
834 when a refactoring is applied. The example shows an \ExtractMethod refactoring
835 followed by a \MoveMethod refactoring that breaks a program in both the
836 \name{Eclipse} and \name{IntelliJ} IDEs\footnote{The \name{NetBeans} IDE handles this
837 particular situation without altering the program's behavior, mainly because
838 its \refa{Move Method} refactoring implementation is a bit flawed in other ways
839 \see{toolSupport}.}. The target and the destination for the composed
840 refactoring is shown in \myref{lst:correctnessExtractAndMove}. Note that the
841 method \method{m(C c)} of class \type{X} assigns to the field \var{x} of the
842 argument \var{c} that has type \type{C}.
846 \begin{minted}[linenos,frame=topline,label={Refactoring
847 target},framesep=\mintedframesep]{java}
849 public X x = new X();
861 \begin{minted}[frame=topline,label={Method
862 destination},framesep=\mintedframesep]{java}
866 // If m is called from
867 // c, then c.x no longer
874 \caption{The target and the destination for the composition of the Extract
875 Method and \refa{Move Method} refactorings.}
876 \label{lst:correctnessExtractAndMove}
880 The refactoring sequence works by extracting line 6 through 8 from the original
881 class \type{C} into a method \method{f} with the statements from those lines as
882 its method body (but with the comment left out, since it will no longer hold any
883 meaning). The method is then moved to the class \type{X}. The result is shown
884 in \myref{lst:correctnessExtractAndMoveResult}.
886 Before the refactoring, the methods \method{m} and \method{n} of class \type{X}
887 are called on different object instances (see line 6 and 8 of the original class
888 \type{C} in \cref{lst:correctnessExtractAndMove}). After the refactoring, they
889 are called on the same object, and the statement on line
890 3 of class \type{X} (in \cref{lst:correctnessExtractAndMoveResult}) no longer
891 has the desired effect in our example. The method \method{f} of class \type{C}
892 is now calling the method \method{f} of class \type{X} (see line 5 of class
893 \type{C} in \cref{lst:correctnessExtractAndMoveResult}), and the program now
894 behaves different than before.
898 \begin{minted}[linenos]{java}
900 public X x = new X();
910 \begin{minted}[linenos]{java}
925 \caption{The result of the composed refactoring.}
926 \label{lst:correctnessExtractAndMoveResult}
929 The bug introduced in the previous example is of such a nature\footnote{Caused
930 by aliasing. See \url{https://en.wikipedia.org/wiki/Aliasing_(computing)}}
931 that it is very difficult to spot if the refactored code is not covered by
932 tests. It does not generate compilation errors, and will thus only result in
933 a runtime error or corrupted data, which might be hard to detect.
935 \subsection{Refactoring and the importance of testing}\label{testing}
937 If you want to refactor, the essential precondition is having solid
938 tests.\citing{refactoring}
941 When refactoring, there are roughly three classes of errors that can be made.
942 The first class of errors are the ones that make the code unable to compile.
943 These \emph{compile-time} errors are of the nicer kind. They flash up at the
944 moment they are made (at least when using an IDE), and are usually easy to fix.
945 The second class are the \emph{runtime} errors. Although they take a bit longer
946 to surface, they usually manifest after some time in an illegal argument
947 exception, null pointer exception or similar during the program execution.
948 These kind of errors are a bit harder to handle, but at least they will show,
949 eventually. Then there are the \emph{behavior-changing} errors. These errors are
950 of the worst kind. They do not show up during compilation and they do not turn
951 on a blinking red light during runtime either. The program can seem to work
952 perfectly fine with them in play, but the business logic can be damaged in ways
953 that will only show up over time.
955 For discovering runtime errors and behavior changes when refactoring, it is
956 essential to have good test coverage. Testing in this context means writing
957 automated tests. Manual testing may have its uses, but when refactoring, it is
958 automated unit testing that dominate. For discovering behavior changes it is
959 especially important to have tests that cover potential problems, since these
960 kind of errors does not reveal themselves.
962 Unit testing is not a way to \emph{prove} that a program is correct, but it is a
963 way to make you confident that it \emph{probably} works as desired. In the
964 context of test-driven development (commonly known as TDD), the tests are even a
965 way to define how the program is \emph{supposed} to work. It is then, by
966 definition, working if the tests are passing.
968 If the test coverage for a code base is perfect, then it should, theoretically,
969 be risk-free to perform refactorings on it. This is why automated tests and
970 refactoring are such a great match.
972 \subsubsection{Testing the code from correctness section}
973 The worst thing that can happen when refactoring is to introduce changes to the
974 behavior of a program, as in the example on \myref{correctness}. This example
975 may be trivial, but the essence is clear. The only problem with the example is
976 that it is not clear how to create automated tests for it, without changing it
979 Unit tests, as they are known from the different \glosspl{xUnit} around, are
980 only suitable to test the \emph{result} of isolated operations. They can not
981 easily (if at all) observe the \emph{history} of a program.
983 This problem is still open.
987 Assuming a sequential (non-concurrent) program:
990 tracematch (C c, X x) {
992 call(* X.m(C)) && args(c) && cflow(within(C));
994 call(* X.n()) && target(x) && cflow(within(C));
996 set(C.x) && target(c) && !cflow(m);
1000 { assert x == c.x; }
1004 %\begin{minted}{java}
1005 %tracematch (X x1, X x2) {
1007 % call(* X.m(C)) && target(x1);
1009 % call(* X.n()) && target(x2);
1011 % set(C.x) && !cflow(m) && !cflow(n);
1015 % { assert x1 != x2; }
1021 \section{The Project}
1022 In this section we look at the work that shall be done for this project, its
1023 building stones and some of the methodologies used.
1025 \subsection{Project description}
1026 The aim of this master's project will be to explore the relationship between the
1027 \ExtractMethod and the \MoveMethod refactorings. This will be done by composing
1028 the two into a composite refactoring. The refactoring will be called the
1029 \ExtractAndMoveMethod refactoring.
1031 The two primitive \ExtractMethod and \MoveMethod refactorings must already be
1032 implemented in a tool, so the \ExtractAndMoveMethod refactoring is going to be
1033 built on top of those.
1035 The composition of the \ExtractMethod and \MoveMethod refactorings springs
1036 naturally out of the need to move procedures closer to the data they manipulate.
1037 This composed refactoring is not well described in the literature, but it is
1038 implemented in at least one tool called
1039 \name{CodeRush}\footnote{\url{https://help.devexpress.com/\#CodeRush/CustomDocument3519}},
1040 that is an extension for \name{MS Visual
1041 Studio}\footnote{\url{http://www.visualstudio.com/}}. In CodeRush it is called
1042 \refa{Extract Method to
1043 Type}\footnote{\url{https://help.devexpress.com/\#CodeRush/CustomDocument6710}},
1044 but I choose to call it \ExtractAndMoveMethod, since I feel this better
1045 communicates which primitive refactorings it is composed of.
1047 The project will consist of implementing the \ExtractAndMoveMethod refactoring,
1048 as well as executing it over a larger code base, as a case study. To be able to
1049 execute the refactoring automatically, I have to make it analyze code to
1050 determine the best selections to extract into new methods.
1052 \subsection{The premises}
1053 Before we can start manipulating source code and write a tool for doing so, we
1054 need to decide on a programming language for the code we are going to
1055 manipulate. Also, since we do not want to start from scratch by implementing
1056 primitive refactorings ourselves, we need to choose an existing tool that
1057 provides the needed refactorings. In addition to be able to perform changes, we
1058 need a framework for analyzing source code for the language we select.
1060 \subsubsection{Choosing the target language}
1061 Choosing which programming language the code that shall be manipulated shall be
1062 written in, is not a very difficult task. We choose to limit the possible
1063 languages to the object-oriented programming languages, since most of the
1064 terminology and literature regarding refactoring comes from the world of
1065 object-oriented programming. In addition, the language must have existing tool
1066 support for refactoring.
1068 The \name{Java} programming language\footnote{\url{https://www.java.com/}} is
1069 the dominating language when it comes to example code in the literature of
1070 refactoring, and is thus a natural choice. Java is perhaps, currently the most
1071 influential programming language in the world, with its \name{Java Virtual
1072 Machine} that runs on all of the most popular architectures and also supports
1073 dozens of other programming languages\footnote{They compile to Java bytecode.},
1074 with \name{Scala}, \name{Clojure} and \name{Groovy} as the most prominent ones.
1075 Java is currently the language that every other programming language is compared
1076 against. It is also the primary programming language for the author of this
1079 \subsubsection{Choosing the tools}
1080 When choosing a tool for manipulating Java, there are certain criteria that
1081 have to be met. First of all, the tool should have some existing refactoring
1082 support that this thesis can build upon. Secondly it should provide some kind of
1083 framework for parsing and analyzing Java source code. Third, it should itself be
1084 open source. This is both because of the need to be able to browse the code for
1085 the existing refactorings that is contained in the tool, and also because open
1086 source projects hold value in them selves. Another important aspect to consider
1087 is that open source projects of a certain size, usually has large communities of
1088 people connected to them, that are committed to answering questions regarding the
1089 use and misuse of the products, that to a large degree is made by the community
1092 There is a certain class of tools that meet these criteria, namely the class of
1093 \emph{IDEs}\footnote{\emph{Integrated Development Environment}}. These are
1094 programs that is meant to support the whole production cycle of a computer
1095 program, and the most popular IDEs that support Java, generally have quite good
1096 refactoring support.
1098 The main contenders for this thesis is the \name{Eclipse IDE}, with the
1099 \name{Java development tools} (JDT), the \name{IntelliJ IDEA Community Edition}
1100 and the \name{NetBeans IDE} \see{toolSupport}. \name{Eclipse} and
1101 \name{NetBeans} are both free, open source and community driven, while the
1102 \name{IntelliJ IDEA} has an open sourced community edition that is free of
1103 charge, but also offer an \name{Ultimate Edition} with an extended set of
1104 features, at additional cost. All three IDEs supports adding plugins to extend
1105 their functionality and tools that can be used to parse and analyze Java source
1106 code. But one of the IDEs stand out as a favorite, and that is the \name{Eclipse
1107 IDE}. This is the most popular\citing{javaReport2011} among them and seems to be
1108 de facto standard IDE for Java development regardless of platform.
1110 \subsection{The primitive refactorings}
1111 The refactorings presented here are the primitive refactorings used in this
1112 project. They are the abstract building blocks used by the \ExtractAndMoveMethod
1115 \paragraph{The Extract Method refactoring}
1116 The \refa{Extract Method} refactoring is used to extract a fragment of code
1117 from its context and into a new method. A call to the new method is inlined
1118 where the fragment was before. It is used to break code into logical units, with
1119 names that explain their purpose.
1121 An example of an \ExtractMethod refactoring is shown in
1122 \myref{lst:extractMethodRefactoring}. It shows a method containing calls to the
1123 methods \method{foo} and \method{bar} of a type \type{X}. These statements are
1124 then extracted into the new method \method{fooBar}.
1127 \begin{multicols}{2}
1128 \begin{minted}[samepage,frame=topline,label={Before},framesep=\mintedframesep]{java}
1139 \begin{minted}[samepage,frame=topline,label={After},framesep=\mintedframesep]{java}
1151 \caption{An example of an \ExtractMethod refactoring.}
1152 \label{lst:extractMethodRefactoring}
1155 \paragraph{The Move Method refactoring}
1156 The \refa{Move Method} refactoring is used to move a method from one class to
1157 another. This can be appropriate if the method is using more features of another
1158 class than of the class which it is currently defined.
1160 \Myref{lst:moveMethodRefactoring} shows an example of this refactoring. Here a
1161 method \method{fooBar} is moved from the class \type{C} to the class \type{X}.
1164 \begin{multicols}{2}
1165 \begin{minted}[samepage,frame=topline,label={Before},framesep=\mintedframesep]{java}
1184 \begin{minted}[samepage,frame=topline,label={After},framesep=\mintedframesep]{java}
1201 \caption{An example of a \MoveMethod refactoring.}
1202 \label{lst:moveMethodRefactoring}
1205 \subsection{The Extract and Move Method refactoring}
1206 The \ExtractAndMoveMethod refactoring is a composite refactoring composed of the
1207 primitive \ExtractMethod and \MoveMethod refactorings. The effect of this
1208 refactoring on source code is the same as when extracting a method and moving it
1209 to another class. Conceptually, this is done without an intermediate step. In
1210 practice, as we shall see later, an intermediate step may be necessary.
1212 An example of this composite refactoring is shown in
1213 \myref{lst:extractAndMoveMethodRefactoring}. The example joins the examples from
1214 \cref{lst:extractMethodRefactoring} and \cref{lst:moveMethodRefactoring}. This
1215 means that the selection consisting of the consecutive calls to the methods
1216 \method{foo} and \method{bar}, is extracted into a new method \method{fooBar}
1217 located in the class \type{X}.
1220 \begin{multicols}{2}
1221 \begin{minted}[samepage,frame=topline,label={Before},framesep=\mintedframesep]{java}
1237 \begin{minted}[samepage,frame=topline,label={After},framesep=\mintedframesep]{java}
1254 \caption{An example of the \ExtractAndMoveMethod refactoring.}
1255 \label{lst:extractAndMoveMethodRefactoring}
1258 \subsection{Research questions}
1259 The main question that I seek an answer to in this thesis is:
1262 Is it possible to automate the analysis and execution of the
1263 \ExtractAndMoveMethod refactoring, and do so for all of the code of a larger
1267 \noindent The secondary questions will then be:
1269 \paragraph{Can we do this efficiently?} Can we automate the analysis and
1270 execution of the refactoring so it can be run in a reasonable amount of time?
1271 And what does \emph{reasonable} mean in this context?
1273 And, assuming the refactoring does in fact improve the quality of source code:
1275 \paragraph{How can the automation of the refactoring be helpful?} What is the
1276 usefulness of the refactoring in a software development setting? In what parts
1277 of the development process can the refactoring play a role?
1279 \subsection{Methodology}
1281 \subsubsection{Evolutionary design}
1282 In the programming work for this project, it have tried to use a design strategy
1283 called evolutionary design, also known as continuous or incremental
1284 design\citing{wiki_continuous_2014}. It is a software design strategy
1285 advocated by the Extreme Programming community. The essence of the strategy is
1286 that you should let the design of your program evolve naturally as your
1287 requirements change. This is seen in contrast with up-front design, where
1288 design decisions are made early in the process.
1290 The motivation behind evolutionary design is to keep the design of software as
1291 simple as possible. This means not introducing unneeded functionality into a
1292 program. You should defer introducing flexibility into your software, until it
1293 is needed to be able to add functionality in a clean way.
1295 Holding up design decisions, implies that the time will eventually come when
1296 decisions have to be made. The flexibility of the design then relies on the
1297 programmer's abilities to perform the necessary refactoring, and \his confidence
1298 in those abilities. From my experience working on this project, I can say that
1299 this confidence is greatly enhanced by having automated tests to rely on
1302 The choice of going for evolutionary design developed naturally. As Fowler
1303 points out in his article \tit{Is Design Dead?}, evolutionary design much
1304 resembles the ``code and fix'' development strategy\citing{fowler_design_2004}.
1305 A strategy that most of us have practiced in school. This was also the case when
1306 I first started this work. I had to learn the inner workings of Eclipse and its
1307 refactoring-related plugins. That meant a lot of fumbling around with code I did
1308 not know, in a trial and error fashion. Eventually I started writing tests for
1309 my code, and my design began to evolve.
1311 \subsubsection{Test-driven development}\label{tdd}
1312 As mentioned before, the project started out as a classic code and fix
1313 developmen process. My focus was aimed at getting something to work, rather than
1314 doing so according to best practice. This resulted in a project that got out of
1315 its starting blocks, but it was not accompanied by any tests. Hence it was soon
1316 difficult to make any code changes with the confidence that the program was
1317 still correct afterwards (assuming it was so before changing it). I always knew
1318 that I had to introduce some tests at one point, but this experience accelerated
1319 the process of leading me onto the path of testing.
1321 I then wrote tests for the core functionality of the plugin, and thus gained
1322 more confidence in the correctness of my code. I could now perform quite drastic
1323 changes without ``wetting my pants``. After this, nearly all of the semantic
1324 changes done to the business logic of the project, or the addition of new
1325 functionality, was made in a test-driven manner. This means that before
1326 performing any changes, I would define the desired functionality through a set
1327 of tests. I would then run the tests to check that they were run and that they
1328 did not pass. Then I would do any code changes necessary to make the tests
1329 pass. The definition of how the program is supposed to operate is then captured
1330 by the tests. However, this does not prove the correctness of the analysis
1331 leading to the test definitions.
1333 \subsubsection{Continuous integration}
1336 \section{Related Work}
1338 \subsection{Safer refactorings}
1341 \subsection{The compositional paradigm of refactoring}
1342 This paradigm builds upon the observation of Vakilian et
1343 al.\citing{vakilian2012}, that of the many automated refactorings existing in
1344 modern IDEs, the simplest ones are dominating the usage statistics. The report
1345 mainly focuses on \name{Eclipse} as the tool under investigation.
1347 The paradigm is described almost as the opposite of automated composition of
1348 refactorings \see{compositeRefactorings}. It works by providing the programmer
1349 with easily accessible primitive refactorings. These refactorings shall be
1350 accessed via keyboard shortcuts or quick-assist menus\footnote{Think
1351 quick-assist with Ctrl+1 in \name{Eclipse}} and be promptly executed, opposed to in the
1352 currently dominating wizard-based refactoring paradigm. They are meant to
1353 stimulate composing smaller refactorings into more complex changes, rather than
1354 doing a large upfront configuration of a wizard-based refactoring, before
1355 previewing and executing it. The compositional paradigm of refactoring is
1356 supposed to give control back to the programmer, by supporting \himher with an
1357 option of performing small rapid changes instead of large changes with a lesser
1358 degree of control. The report authors hope this will lead to fewer unsuccessful
1359 refactorings. It also could lower the bar for understanding the steps of a
1360 larger composite refactoring and thus also help in figuring out what goes wrong
1361 if one should choose to op in on a wizard-based refactoring.
1363 Vakilian and his associates have performed a survey of the effectiveness of the
1364 compositional paradigm versus the wizard-based one. They claim to have found
1365 evidence of that the \emph{compositional paradigm} outperforms the
1366 \emph{wizard-based}. It does so by reducing automation, which seem
1367 counterintuitive. Therefore they ask the question ``What is an appropriate level
1368 of automation?'', and thus questions what they feel is a rush toward more
1369 automation in the software engineering community.
1373 \chapter{The search-based Extract and Move Method refactoring}
1374 In this chapter I will delve into the workings of the search-based
1375 \ExtractAndMoveMethod refactoring. We will see the choices it must make along
1376 the way and why it chooses a text selection as a candidate for refactoring or
1379 After defining some concepts, I will introduce an example that will be used
1380 throughout the chapter to illustrate how the refactoring works in some simple
1383 \section{The inputs to the refactoring}
1384 For executing an \ExtractAndMoveMethod refactoring, there are two simple
1385 requirements. The first thing the refactoring needs is a text selection, telling
1386 it what to extract. Its second requirement is a target for the subsequent move
1389 The extracted method must be called instead of the selection that makes up its
1390 body. Also, the method call has to be performed via a variable, since the method
1391 is not static. Therefore, the move target must be a variable in the scope of the
1392 extracted selection. The actual new location for the extracted method will be
1393 the class representing the type of the move target variable. But, since the
1394 method also must be called through a variable, it makes sense to define the move
1395 target to be either a local variable or a field in the scope of the text
1398 \section{Defining a text selection}
1399 A text selection, in our context, is very similar to what you think of when
1400 selecting a bit of text in your editor or other text processing tool with your
1401 mouse or keyboard. It is an abstract construct that is meant to capture which
1402 specific portion of text we are about to deal with.
1404 To be able to clearly reason about a text selection done to a portion of text in
1405 a computer file, that consist of pure text, we put up the following definition.
1407 \definition{A \emph{text selection} in a text file is defined by two
1408 non-negative integers, in addition to a reference to the file itself. The first
1409 integer is an offset into the file, while the second reference is the length of
1410 the text selection.}
1412 This means that the selected text consist of a number of characters equal to the
1413 length of the selection, where the first character is found at the specified
1416 \section{Where we look for text selections}
1418 \subsection{Text selections are found in methods}
1419 The text selections we are interested in are those that surrounds program
1420 statements. Therefore, the place we look for selections that can form candidates
1421 for an execution of the \ExtractAndMoveMethod refactoring, is within the body of
1424 \paragraph{On ignoring static methods}
1425 In this project we are not analyzing static methods for candidates to the
1426 \ExtractAndMoveMethod refactoring. The reason for this is that in the cases
1427 where we want to perform the refactoring for a selection within a static method,
1428 the first step is to extract the selection into a new method. Hence this method
1429 also become static, since it must be possible to call it from a static context.
1430 It would then be difficult to move the method to another class, make it
1431 non-static and calling it through a variable. To avoid these obstacles, we
1432 simply ignore static methods.
1434 \begin{listing}[htb]
1435 \def\charwidth{5.8pt}
1436 \def\indent{2*\charwidth}
1437 \def\lineheight{\baselineskip}
1438 \def\mintedtop{2*\lineheight+5.8pt}
1440 \begin{tikzpicture}[overlay, yscale=-1, xshift=3.8pt+\charwidth*31]
1441 \tikzstyle{overlaybox}=[fill=lightgray,opacity=0.2]
1443 \draw[overlaybox] (\indent,\mintedtop+\lineheight*4) rectangle
1444 +(23*\charwidth,17*\lineheight);
1447 \draw[overlaybox] (2*\indent,\mintedtop+5*\lineheight) rectangle
1448 +(15*\charwidth,3*\lineheight);
1449 \draw[overlaybox] (2*\indent,\mintedtop+15*\lineheight) rectangle
1450 +(15*\charwidth,3*\lineheight);
1451 \draw[overlaybox] (2*\indent,\mintedtop+19*\lineheight) rectangle
1452 +(15*\charwidth,\lineheight);
1454 \begin{multicols}{2}
1455 \begin{minted}[linenos,frame=topline,label=Clean,framesep=\mintedframesep]{java}
1457 A a; B b; boolean bool;
1459 void method(int val) {
1483 \begin{minted}[frame=topline,label={With statement
1484 sequences},framesep=\mintedframesep]{java}
1486 A a; B b; boolean bool;
1488 void method(int val) {
1511 \caption{Classes \type{A} and \type{B} are both public. The methods
1512 \method{foo} and \method{bar} are public members of class \type{A}.}
1513 \label{lst:grandExample}
1516 \subsection{The possible text selections of a method body}
1518 The number of possible text selections that can be made from the text in a
1519 method body, are equal to all the sub-sequences of characters within it. For our
1520 purposes, analyzing program source code, we must define what it means for a text
1521 selection to be valid.
1523 \definition{A \emph{valid text selection} is a text selection that contains all
1524 of one or more consecutive program statements.}
1526 For a sequence of statements, the text selections that can be made from it, are
1527 equal to all its sub-sequences. \Myref{lst:textSelectionsExample} show an
1528 example of all the text selections that can be made from the code in
1529 \myref{lst:grandExample}, lines 16-18. For convenience and the clarity of this
1530 example, the text selections are represented as tuples with the start and end
1531 line of all selections: $\{(16), (17), (18), (16,17), (16,18), (17,18)\}$.
1533 \begin{listing}[htb]
1534 \def\charwidth{5.7pt}
1535 \def\indent{4*\charwidth}
1536 \def\lineheight{\baselineskip}
1537 \def\mintedtop{\lineheight-1pt}
1539 \begin{tikzpicture}[overlay, yscale=-1]
1540 \tikzstyle{overlaybox}=[fill=lightgray,opacity=0.2]
1543 \draw[overlaybox] (2*\charwidth,\mintedtop) rectangle
1544 +(16*\charwidth,\lineheight);
1547 \draw[overlaybox] (2*\charwidth,\mintedtop+\lineheight) rectangle
1548 +(16*\charwidth,\lineheight);
1551 \draw[overlaybox] (2*\charwidth,\mintedtop+2*\lineheight) rectangle
1552 +(16*\charwidth,\lineheight);
1554 \draw[overlaybox] (\indent-3*\charwidth,\mintedtop) rectangle
1555 +(18*\charwidth,2*\lineheight);
1557 \draw[overlaybox] (3*\charwidth,\mintedtop+\lineheight) rectangle
1558 +(14*\charwidth,2*\lineheight);
1561 \draw[overlaybox] (\indent,\mintedtop) rectangle
1562 +(12*\charwidth,3*\lineheight);
1564 % indent should be 5 spaces
1565 \begin{minted}[linenos,firstnumber=16]{java}
1570 \caption{Example of how the text selections generator would generate text
1571 selections based on a lists of statements. Each highlighted rectangle
1572 represents a text selection.}
1573 \label{lst:textSelectionsExample}
1576 Each nesting level of a method body can have many such sequences of statements.
1577 The outermost nesting level has one such sequence, and each branch contains
1578 their own sequence of statements. \Myref{lst:grandExample} has a version of some
1579 code where all such sequences of statements are highlighted for a method body.
1581 To complete our example of possible text selections, I will now list all
1582 possible text selections for the method in \myref{lst:grandExample}, by nesting
1583 level. There are 23 of them in total.
1586 \item[Level 1 (10 selections)] \hfill \\
1587 $\{(5,9), (11), (12), (14,21), (5,11), (5,12), (5,21), (11,12),
1588 (11,21), \\(12,21)\}$
1590 \item[Level 2 (13 selections)] \hfill \\
1591 $\{(6), (7), (8), (6,7), (6,8), (7,8), (16), (17), (18), (16,17), (16,18), \\
1595 \subsubsection{The complexity}\label{sec:complexity}
1596 The complexity of how many text selections that needs to be analyzed for a body
1597 of in total $n$ statements, is bounded by $O(n^2)$. A body of statements is here
1598 all the statements in all nesting levels of a sequence of statements. A method
1599 body (or a block) is a body of statements. To prove that the complexity is
1600 bounded by $O(n^2)$, I present a couple of theorems and proves them.
1603 The number of text selections that need to be analyzed for each list of
1604 statements of length $n$, is exactly
1607 \sum_{i=1}^{n} i = \frac{n(n+1)}{2}
1608 \label{eq:complexityStatementList}
1610 \label{thm:numberOfTextSelection}
1614 For $n=1$ this is trivial: $\frac{1(1+1)}{2} = \frac{2}{2} = 1$. One statement
1615 equals one selection.
1617 For $n=2$, you get one text selection for the first statement, one selection
1618 for the second statement, and one selection for the two of them combined.
1619 This equals three selections. $\frac{2(2+1)}{2} = \frac{6}{2} = 3$.
1621 For $n=3$, you get 3 selections for the two first statements, as in the case
1622 where $n=2$. In addition you get one selection for the third statement itself,
1623 and two more statements for the combinations of it with the two previous
1624 statements. This equals six selections. $\frac{3(3+1)}{2} = \frac{12}{2} = 6$.
1626 Assume that for $n=k$ there exists $\frac{k(k+1)}{2}$ text selections. Then we
1627 want to add selections for another statement, following the previous $k$
1628 statements. So, for $n=k+1$, we get one additional selection for the statement
1629 itself. Then we get one selection for each pair of the new selection and the
1630 previous $k$ statements. So the total number of selections will be the number
1631 of already generated selections, plus $k$ for every pair, plus one for the
1632 statement itself: $\frac{k(k+1)}{2} + k +
1633 1 = \frac{k(k+1)+2k+2}{2} = \frac{k(k+1)+2(k+1)}{2} = \frac{(k+1)(k+2)}{2} =
1634 \frac{(k+1)((k+1)+1)}{2} = \sum_{i=1}^{k+1} i$
1637 %\definition{A \emph{body of statements} is a sequence of statements where every
1638 %statement may have sub-statements.}
1641 The number of text selections for a body of statements is maximized if all the
1642 statements are at the same level.
1643 \label{thm:textSelectionsMaximized}
1647 Assume we have a body of, in total, $k$ statements. Then, the sum of the
1648 lengths of all the lists of statements in the body, is also $k$. Let
1649 $\{l,\ldots,m,(k-l-\ldots-m)\}$ be the lengths of the lists of statements in
1650 the body, with $l+\ldots+m<k \Rightarrow \forall i \in \{l,\ldots,m\} : i < k$.
1652 Then, the number of text selections that are generated for the $k$ statements
1658 \frac{l(l+1)}{2} + \ldots + \frac{m(m+1)}{2} +
1659 \frac{(k-l-\ldots-m)((k-l-\ldots-m)+ 1)}{2} = \\
1660 \frac{l^2+l}{2} + \ldots + \frac{m^2+m}{2} + \frac{k^2 - 2kl - \ldots - 2km +
1661 l^2 + \ldots + m^2 + k - l - \ldots - m}{2} = \\
1662 \frac{2l^2 - 2kl + \ldots + 2m^2 - 2km + k^2 + k}{2}
1666 \noindent It then remains to show that this inequality holds:
1669 \frac{2l^2 - 2kl + \ldots + 2m^2 - 2km + k^2 + k}{2} < \frac{k(k+1)}{2} =
1673 \noindent By multiplication by $2$ on both sides, and by removing the equal
1677 2l^2 - 2kl + \ldots + 2m^2 - 2km < 0
1680 Since $\forall i \in \{l,\ldots,m\} : i < k$, we have that $\forall i \in
1681 \{l,\ldots,m\} : 2ki > 2i^2$, so all the pairs of parts on the form $2i^2-2ki$
1682 are negative. In sum, the inequality holds.
1686 Therefore, the complexity for the number of selections that needs to be analyzed
1687 for a body of $n$ statements is $O\bigl(\frac{n(n+1)}{2}\bigr) = O(n^2)$.
1689 \section{Disqualifying a selection}
1690 Certain text selections would lead to broken code if used as input to the
1691 \ExtractAndMoveMethod refactoring. To avoid this, we have to check all text
1692 selections for such conditions before they are further analyzed. This section
1693 is therefore going to present some properties that make a selection unsuitable
1694 for our refactoring.
1696 \subsection{A call to a protected or package-private method}
1697 If a text selection contains a call to a protected or package-private method, it
1698 would not be safe to move it to another class. The reason for this, is that we
1699 cannot know if the called method is being overridden by some subclass of the
1700 \gloss{enclosingClass}, or not.
1702 Imagine that the protected method \method{foo} is declared in class \m{A},
1703 and overridden in class \m{B}. The method \method{foo} is called from within a
1704 selection done to a method in \m{A}. We want to extract and move this selection
1705 to another class. The method \method{foo} is not public, so the \MoveMethod
1706 refactoring must make it public, making the extracted method able to call it
1707 from the extracted method's new location. The problem is, that the now public
1708 method \method{foo} is overridden in a subclass, where it has a protected
1709 status. This makes the compiler complain that the subclass \m{B} is trying to
1710 reduce the visibility of a method declared in its superclass \m{A}. This is not
1711 allowed in Java, and for good reasons. It would make it possible to make a
1712 subclass that could not be a substitute for its superclass.
1714 The problem this check helps to avoid, is a little subtle. The problem does not
1715 arise in the class where the change is done, but in a class derived from it.
1716 This shows that classes acting as superclasses are especially fragile to
1717 introducing errors in the context of automated refactoring.
1719 This is also shown in bug\ldots \todoin{File Eclipse bug report}
1722 \subsection{A double class instance creation}
1723 The following is a problem caused solely by the underlying \MoveMethod
1724 refactoring. The problem occurs if two classes are instantiated such that the
1725 first constructor invocation is an argument to a second, and that the first
1726 constructor invocation takes an argument that is built up using a field. As an
1727 example, say that \var{name} is a field of the enclosing class, and we have the
1728 expression \code{new A(new B(name))}. If this expression is located in a
1729 selection that is moved to another class, \var{name} will be left untouched,
1730 instead of being prefixed with a variable of the same type as it is declared in.
1731 If \var{name} is the destination for the move, it is not replaced by
1732 \code{this}, or removed if it is a prefix to a member access
1733 (\code{name.member}), but it is still left by itself.
1735 Situations like this would lead to code that will not compile. Therefore, we
1736 have to avoid them by not allowing selections to contain such double class
1737 instance creations that also contains references to fields.
1739 \todoin{File Eclipse bug report}
1742 \subsection{Instantiation of non-static inner class}
1743 When a non-static inner class is instantiated, this must happen in the scope of
1744 its declaring class. This is because it must have access to the members of the
1745 declaring class. If the inner class is public, it is possible to instantiate it
1746 through an instance of its declaring class, but this is not handled by the
1747 underlying \MoveMethod refactoring.
1749 Performing a move on a method that instantiates a non-static inner class, will
1750 break the code if the instantiation is not handled properly. For this reason,
1751 selections that contains instantiations of non-static inner classes are deemed
1752 unsuitable for the \ExtractAndMoveMethod refactoring.
1754 \subsection{References to enclosing instances of the enclosing class}
1755 The title of this section may be a little hard to grasp at first. What it means
1756 is that there is a (non-static) class \m{C} that is declared in the scope of
1757 possibly multiple other classes. And there is a statement in the body of a
1758 method declared in class \m{C}, that contains a reference to one or more
1759 instances of these enclosing classes of \m{C}.
1761 The problem with this, is that these references may not be valid if they are
1762 moved to another class. Theoretically, some situations could easily be solved by
1763 passing, to the moved method, a reference to the instance where the problematic
1764 referenced member is declared. This should work in the case where this member is
1765 publicly accessible. This is not done in the underlying \MoveMethod refactoring,
1766 so it cannot be allowed in the \ExtractAndMoveMethod refactoring either.
1768 \subsection{Inconsistent return statements}
1769 To verify that a text selection is consistent with respect to return statements,
1770 we must check that if a selection contains a return statement, then every
1771 possible execution path within the selection ends in either a return or a throw
1772 statement. This property is important regarding the \ExtractMethod refactoring.
1773 If it holds, it means that a method could be extracted from the selection, and a
1774 call to it could be substituted for the selection. If the method has a non-void
1775 return type, then a call to it would also be a valid return point for the
1776 calling method. If its return value is of the void type, then the \ExtractMethod
1777 refactoring will append an empty return statement to the back of the method
1778 call. Therefore, the analysis does not discriminate on either kinds of return
1779 statements, with or without a return value.
1781 A throw statement is accepted anywhere a return statement is required. This is
1782 because a throw statement causes an immediate exit from the current block,
1783 together with all outer blocks in its control flow that does not catch the
1786 Return statements can be either explicit or implicit. An \emph{explicit} return
1787 statement is formed by using the \code{return} keyword, while an \emph{implicit}
1788 return statement is a statement that is not formed using \code{return}, but must
1789 be the last statement of a method that can have any side effects. This can
1790 happen in methods with a void return type. An example is a statement that is
1791 inside one or more blocks. The last statement of a method could for instance be
1792 a synchronized statement, but the last statement that is executed in the method,
1793 and that can have any side effects, may be located inside the body of the
1794 synchronized statement.
1796 We can start the check for this property by looking at the last statement of a
1797 selection to see if it is a return statement (explicit or implicit) or a throw
1798 statement. If this is the case, then the property holds, assuming the selected
1799 code does not contain any compilation errors. All execution paths within the
1800 selection should end in either this, or another, return or throw statement.
1801 \todoin{State somewhere that we assume no compilation errors?}
1803 If the last statement of the selection is not a return or throw, the execution
1804 of it must eventually end in one for the selection to be legal. This means that
1805 all branches of the last statement of every branch must end in a return or
1806 throw. Given this recursive definition, there are only five types of statements
1807 that are guaranteed to end in a return or throw if their child branches does.
1808 All other statements would have to be considered illegal. The first three:
1809 Block-statements, labeled statements and do-statements are all kinds of
1810 fall-through statements that always gets their body executed. Do-statements
1811 would not make much sense if written such that they
1812 always ends after the first round of execution of their body, but that is not
1813 our concern. The remaining two statements that can end in a return or throw are
1814 if-statements and try-statements.
1816 For an if-statement, the rule is that if its then-part does not contain any
1817 return or throw statements, this is considered illegal. If the then-part does
1818 contain a return or throw, the else-part is checked. If its else-part is
1819 non-existent, or it does not contain any return or throw statements, the
1820 statement is considered illegal. If an if-statement is not considered illegal,
1821 the bodies of its two parts must be checked.
1823 Try-statements are handled much the same way as if-statements. The body of a
1824 try-statement must contain a return or throw. The same applies to its catch
1825 clauses and finally body.
1827 \subsection{Ambiguous return values}
1828 The problem with ambiguous return values arise when a selection is chosen to be
1829 extracted into a new method, but it needs to return more than one value from
1832 This problem occurs in two situations. The first situation arise when there is
1833 more than one local variable that is both assigned to within a selection and
1834 also referenced after the selection. The other situation occur when there is
1835 only one such assignment, but the selection also contain return statements.
1837 Therefore we must examine the selection for assignments to local variables that
1838 are referenced after the text selection. Then we must verify that not more than
1839 one such reference is done, or zero if any return statements are found.
1841 \subsection{Illegal statements}
1842 An illegal statement may be a statement that is of a type that is never allowed,
1843 or it may be a statement of a type that is only allowed if certain conditions
1846 Any use of the \var{super} keyword is prohibited, since its meaning is altered
1847 when moving a method to another class.
1849 For a \emph{break} statement, there are two situations to consider: A break
1850 statement with or without a label. If the break statement has a label, it is
1851 checked that whole of the labeled statement is inside the selection. If the
1852 break statement does not have a label attached to it, it is checked that its
1853 innermost enclosing loop or switch statement also is inside the selection.
1855 The situation for a \emph{continue} statement is the same as for a break
1856 statement, except that it is not allowed inside switch statements.
1858 Regarding \emph{assignments}, two types of assignments are allowed: Assignments
1859 to non-final variables and assignments to array access. All other assignments
1860 are regarded illegal.
1862 \todoin{Expand with more illegal statements and/or conclude that I did not have
1863 time to analyze all statement types.}
1865 \section{Disqualifying selections from the
1866 example}\label{sec:disqualifyingExample}
1867 Among the selections we found for the code in \myref{lst:grandExample}, not many
1868 of them must be disqualified on the basis of containing something illegal. The
1869 only statement causing trouble is the break statement in line 18. None of the
1870 selections on nesting level 2 can contain this break statement, since the
1871 innermost switch statement is not inside any of these selections.
1873 This means that the text selections $(18)$, $(16,18)$ and $(17,18)$ can be
1874 excluded from further consideration, and we are left with the following
1878 \item[Level 1 (10 selections)] \hfill \\
1879 $\{(5,9), (11), (12), (14,21), (5,11), (5,12), (5,21), (11,12),
1880 (11,21), \\(12,21)\}$
1882 \item[Level 2 (10 selections)] \hfill \\
1883 $\{(6), (7), (8), (6,7), (6,8), (7,8), (16), (17), (16,17), (20)\}$
1886 \section{Finding a move target}
1887 In the analysis needed to perform the \ExtractAndMoveMethod refactoring
1888 automatically, the selection we choose is found among all the selections that
1889 has a possible move target. Therefore, the best possible move target must be
1890 found for all the candidate selections, so that we are able to sort out the
1891 selection that is best suited for the refactoring.
1893 To find the best move target for a specific text selection, we first need to
1894 find all the possible targets. Since the target must be a local variable or a
1895 field, we are basically looking for names within the selection; names that
1896 represents references to variables.
1898 The names we are looking for, we call prefixes. This is because we are not
1899 interested in names that occur in the middle of a dot-separated sequence of
1900 names. We are only interested in names that constitutes prefixes of other names,
1901 possibly themselves. The reason for this, is that two lexically equal names need
1902 not be referencing the same variable, if they themselves are not referenced via
1903 the same prefix. Consider the two method calls \code{a.x.foo()} and
1904 \code{b.x.foo()}. Here, the two references to \code{x}, in the middle of the
1905 qualified names both preceding \code{foo()}, are not referencing the same
1906 variable. Even though the variables may share the type, and the method
1907 \method{foo} thus is the same for both, we would not know through which of the
1908 variables \var{a} or \var{b} we should call the extracted method.
1910 The possible move targets are then the prefixes that are not among a subset of
1911 the prefixes that are not valid move targets \see{s:unfixes}. Also, prefixes
1912 that are just simple names, and have only one occurrence, are left out. This is
1913 because they are not going to have any positive effect on coupling between
1914 classes, and are only going to increase the complexity of the code.
1916 For finding the best move target among these safe prefixes, a simple heuristic
1917 is used. It is as simple as choosing the prefix that is most frequently
1918 referenced within the selection.
1920 \section{Unfixes}\label{s:unfixes}
1921 The prefixes that are not valid as move targets are called unfixes.
1923 An unfix can be a name that is assigned to within a selection. The reason that
1924 this cannot be allowed, is that the result would be an assignment to the
1925 \type{this} keyword, which is not valid in Java \see{eclipse_bug_420726}.
1927 Prefixes that originates from variable declarations within the same selection
1928 are also considered unfixes. This is because when a method is moved, it needs to
1929 be called through a variable. If this variable is also declared within the
1930 method that is to be moved, this obviously cannot be done.
1932 Also considered as unfixes are variable references that are of types that are
1933 not suitable for moving methods to. This can either be because it is not
1934 physically possible to move a method to the desired class or that it will cause
1935 compilation errors by doing so.
1937 If the type binding for a name is not resolved it is considered and unfix. The
1938 same applies to types that is only found in compiled code, so they have no
1939 underlying source that is accessible to us. (E.g. the \type{java.lang.String}
1942 Interfaces types are not suitable as targets. This is simply because interfaces
1943 in Java cannot contain methods with bodies. (This thesis does not deal with
1944 features of Java versions later than Java 7. Java 8 has interfaces with default
1945 implementations of methods.)
1947 Neither are local types allowed. This accounts for both local and anonymous
1948 classes. Anonymous classes are effectively the same as interface types with
1949 respect to unfixes. Local classes could in theory be used as targets, but this
1950 is not possible due to limitations of the way the \refa{Extract and Move Method}
1951 refactoring has to be implemented. The problem is that the refactoring is done
1952 in two steps, so the intermediate state between the two refactorings would not
1953 be legal Java code. In the intermediate step for the case where a local class is
1954 the move target, the extracted method would need to take the local class as a
1955 parameter. This new method would need to live in the scope of the declaring
1956 class of the originating method. The local class would then not be in the scope
1957 of the extracted method, thus bringing the source code into an illegal state.
1958 One could imagine that the method was extracted and moved in one operation,
1959 without an intermediate state. Then it would make sense to include variables
1960 with types of local classes in the set of legal targets, since the local classes
1961 would then be in the scopes of the method calls. If this makes any difference
1962 for software metrics that measure coupling would be a different discussion.
1965 \begin{listing}[htb]
1966 \begin{multicols}{2}
1967 \begin{minted}[frame=topline,label=Before,framesep=\mintedframesep]{java}
1968 void declaresLocalClass() {
1983 \begin{minted}[frame=topline,label={After Extract
1984 Method},framesep=\mintedframesep]{java}
1985 void declaresLocalClass() {
1996 // Intermediate step
1997 void fooBar(LocalClass inst) {
2003 \caption{When the \refa{Extract and Move Method} tries to use a variable with a
2004 local type as the move target, an intermediate step is performed that is not
2005 allowed. Here: \type{LocalClass} is not in the scope of \method{fooBar} in its
2006 intermediate location.}
2007 \label{lst:extractMethod_LocalClass}
2010 The last class of names that are considered unfixes are names used in null
2011 tests. These are tests that reads like this: if \code{<name>} equals \var{null}
2012 then do something. If allowing variables used in those kinds of expressions as
2013 targets for moving methods, we would end up with code containing boolean
2014 expressions like \code{this == null}, which would not be meaningful, since
2015 \var{this} would never be \var{null}.
2017 \section{Finding the example selections that have possible targets}
2018 We now pick up the thread from \myref{sec:disqualifyingExample} where we have a
2019 set of text selections that needs to be analyzed to find out if some of them are
2020 suitable targets for the \ExtractAndMoveMethod refactoring.
2022 We start by analyzing the text selections for nesting level 2, because these
2023 results can be used to reason about the selections for nesting level 1. First we
2024 have all the single-statement selections.
2027 \item[Selections $(6)$, $(8)$ and $(20)$.] \hfill \\
2028 All these selections have a prefix that contains a possible target, namely
2029 the variable \var{a}. The problem is that the prefixes are only one segment
2030 long, and their frequency counts are only 1 as well. None of these
2031 selections are therefore considered as suitable candidates for the
2034 \item[Selection $(7)$.] \hfill \\
2035 This selection contains the unfix \var{a}, and no other possible targets.
2036 The reason for \var{a} being an unfix is that it is assigned to within the
2037 selection. Selection $(7)$ is therefore unsuited as a refactoring candidate.
2039 \item[Selections $(16)$ and $(17)$.] \hfill \\
2040 These selections both have a possible target. The target for both selections
2041 is the variable \var{b}. Both the prefixes have frequency 1. We denote this
2042 with the new tuples $((16), \texttt{b.a}, f(1))$ and $((17), \texttt{b.a},
2043 f(1))$. They contain the selection, the prefix with the target and the
2044 frequency for this prefix.
2048 Then we have all the text selections from level 2 that are composed of multiple
2052 \item[Selections $(6,7)$, $(6,8)$ and $(7,8)$.] \hfill \\
2053 All these selections are disqualified for the reason that they contain the
2054 unfix \var{a}, due to the assignment, and no other possible move targets.
2056 \item[Selection $(16,17)$.] \hfill \\
2057 This selection is the last selection that is not analyzed on nesting level
2058 2. It contains only one possible move target, and that is the variable
2059 \var{b}. It also contains only one prefix \var{b.a}, with frequency count
2060 2. Therefore we have a new candidate $((16,17), \texttt{b.a}, f(2))$.
2064 Moving on to the text selections for nesting level 1, starting with the
2065 single-statement selections:
2068 \item[Selection $(5,9)$.] \hfill \\
2069 This selection contains two variable references that must be analyzed to see
2070 if they are possible move candidates. The first one is the variable
2071 \var{bool}. This variable is of type \type{boolean}, that is a primary type
2072 and therefore not possible to make any changes to. The second variable is
2073 \var{a}. The variable \var{a} is an unfix in $(5,9)$, for the same reason as
2074 in the selections $(6,7)$, $(7,8)$ and $(6,8)$. So selection $(5,9)$
2075 contains no possible move targets.
2077 \item[Selections $(11)$ and $(12)$.] \hfill \\
2078 These selections are disqualified for the same reasons as selections $(6)$
2079 and $(8)$. Their prefixes are one segment long and are referenced only one
2082 \item[Selection $(14,21)$] \hfill \\
2083 This is the switch statement from \myref{lst:grandExample}. It contains the
2084 relevant variable references \var{val}, \var{a} and \var{b}. The variable
2085 \var{val} is a primary type, just as \var{bool}. The variable \var{a} is
2086 only found in one statement, and in a prefix with only one segment, so it is
2087 not considered to be a possible move target. The only variable left is
2088 \var{b}. Just as in the selection $(16,17)$, \var{b} is part of the prefix
2089 \code{b.a}, that has 2 appearances. We have found a new candidate $((14,21),
2090 \texttt{b.a}, f(2))$.
2094 It remains to see if we get any new candidates by analyzing the multi-statement
2095 text selections for nesting level 1:
2098 \item[Selections $(5,11)$ and $(5,12)$.] \hfill \\
2099 These selections are disqualified for the same reason as $(5,9)$. The only
2100 possible move target \var{a} is an unfix.
2102 \item[Selection $(5,21)$.] \hfill \\
2103 This is whole of the method body in \myref{lst:grandExample}. The variables
2104 \var{a}, \var{bool} and \var{val} are either an unfix or primary types. The
2105 variable \var{b} is the only possible move target, and we have, again, the
2106 prefix \code{b.a} as the center of attention. The refactoring candidate is
2107 $((5,21), \texttt{b.a}, f(2))$.
2109 \item[Selection $(11,12)$.] \hfill \\
2110 This small selection contains the prefix \code{a} with frequency 2, and no
2111 unfixes. The candidate is $((11,12), \texttt{a}, f(2))$.
2113 \item[Selection $(11,21)$] \hfill \\
2114 This selection contains the selection $(11,12)$ in addition to the switch
2115 statement. The selection has two possible move targets. The first one is
2116 \var{b}, in a prefix with frequency 2. The second is \var{a}, although it
2117 is in a simple prefix, it is referenced 3 times, and is therefore chosen
2118 as the target for the selection. The new candidate is $((11,21),
2121 \item[Selection $(12,21)$.] \hfill \\
2122 This selection is similar to the previous $(11,21)$, only that \var{a} now
2123 has frequency count 2. This means that the prefixes \code{a} and
2124 \code{b.a} both have the count 2. Of the two, \code{b.a} is preferred,
2125 since it has more segments than \code{a}. Thus the candidate for this
2126 selection is $((12,21), \texttt{b.a}, f(2))$.
2130 For the method in \myref{lst:grandExample} we therefore have the following 8
2131 candidates for the \ExtractAndMoveMethod refactoring: $\{((16), \texttt{b.a},
2132 f(1)), ((17), \texttt{b.a}, f(1)), ((16,17), \texttt{b.a}, f(2)), ((14,21),
2133 \texttt{b.a}, f(2)), \\ ((5,21), \texttt{b.a}, f(2)), ((11,12), \texttt{a},
2134 f(2)), ((11,21), \texttt{a}, f(3)), ((12,21), \texttt{b.a}, f(2))\}$.
2136 It now only remains to specify an order for these candidates, so we can choose
2137 the most suitable one to refactor.
2140 \section{Choosing the selection}\label{sec:choosingSelection}
2141 When choosing a selection between the text selections that have possible move
2142 targets, the selections need to be ordered. The criteria below are presented in
2143 the order they are prioritized. If not one selection is favored over the other
2144 for a concrete criterion, the selections are evaluated by the next criterion.
2147 \item The first criterion that is evaluated is whether a selection contains
2148 any unfixes or not. If selection \m{A} contains no unfixes, while selection
2149 \m{B} does, selection \m{A} is favored over selection \m{B}. This is
2150 because, if we can, we want to avoid moving such as assignments and variable
2151 declarations. This is done under the assumption that, if possible, avoiding
2152 selections containing unfixes will make the code moved a little cleaner.
2154 \item The second criterion that is evaluated is whether a selection contains
2155 multiple possible targets or not. If selection \m{A} has only one possible
2156 target, and selection \m{B} has multiple, selection \m{A} is favored. If
2157 both selections have multiple possible targets, they are considered equal
2158 with respect to this criterion. The rationale for this heuristic is that we
2159 would prefer not to introduce new couplings between classes when performing
2160 the \ExtractAndMoveMethod refactoring.
2162 \item When evaluating this criterion, this is with the knowledge that
2163 selection \m{A} and \m{B} both have one possible target, or multiple
2164 possible targets. Then, if the move target candidate of selection \m{A} has
2165 a higher reference count than the target candidate of selection \m{B},
2166 selection \m{A} is favored. The reason for this is that we would like to
2167 move the selection that gets rid of the most references to another class.
2169 \item The last criterion is that if the frequencies of the targets chosen for
2170 both selections are equal, the selection with the target that is part of the
2171 prefix with highest number of segments is favored. This is done to favor
2176 If none of the above mentioned criteria favors one selection over another, the
2177 selections are considered to be equally good candidates for the
2178 \ExtractAndMoveMethod refactoring.
2180 \section{Concluding the example}
2181 For choosing one of the remaining selections, we need to order our candidates
2182 after the criteria in the previous section. Below we have the candidates ordered
2183 in descending order, with the ``best'' candidate first:
2185 \begin{multicols}{2}
2187 \item $((16,17), \texttt{b.a}, f(2))$
2188 \item $((11,12), \texttt{a}, f(2))$
2189 \item $((16), \texttt{b.a}, f(1))$
2190 \item $((17), \texttt{b.a}, f(1))$
2193 % Many possible targets
2194 \item $((11,21), \texttt{a}, f(3))$
2195 \item $((5,21), \texttt{b.a}, f(2))$
2196 \item $((12,21), \texttt{b.a}, f(2))$
2197 \item $((14,21), \texttt{b.a}, f(2))$
2222 The candidates ordered 5-8 all have unfixes in them, therefore they are ordered
2223 last. None of the candidates ordered 1-4 have multiple possible targets. The
2224 only interesting discussion is now why $(16,17)$ was favored over $(11,12)$.
2225 This is because the prefix \code{b.a} enclosing the move target of selection
2226 $(16,17)$ has one more segment than the prefix \code{a} of $(11,12)$.
2228 The selection is now extracted into a new method \method{gen\_123} and then
2229 moved to class \type{B}, since that is the type of the variable \var{b} that is
2230 the target from the chosen refactoring candidate. The name of the method has a
2231 randomly generated suffix. In the refactoring implementation, the extracted
2232 methods follow the pattern \code{generated\_<long>}, where \code{<long>} is a
2233 pseudo-random long value. This is shortened here to make the example readable.
2234 The result after the refactoring is shown in \myref{lst:grandExampleResult}.
2236 \begin{listing}[htb]
2237 \begin{multicols}{2}
2238 \begin{minted}[linenos]{java}
2240 A a; B b; boolean bool;
2242 void method(int val) {
2265 \begin{minted}[]{java}
2269 public void gen_123(C c) {
2277 \caption{The result after refactoring the class \type{C} of
2278 \myref{lst:grandExample} with the \ExtractAndMoveMethod refactoring with
2279 $((16,17), \texttt{b.a}, f(2))$ as input.}
2280 \label{lst:grandExampleResult}
2283 \chapter{Refactorings in Eclipse JDT: Design and
2284 Shortcomings}\label{ch:jdt_refactorings}
2286 This chapter will deal with some of the design behind refactoring support in
2287 \name{Eclipse}, and the JDT in specific. After which it will follow a section
2288 about shortcomings of the refactoring API in terms of composition of
2292 The refactoring world of \name{Eclipse} can in general be separated into two parts: The
2293 language independent part and the part written for a specific programming
2294 language -- the language that is the target of the supported refactorings.
2295 \todo{What about the language specific part?}
2297 \subsection{The Language Toolkit}
2298 The Language Toolkit\footnote{The content of this section is a mixture of
2299 written material from
2300 \url{https://www.eclipse.org/articles/Article-LTK/ltk.html} and
2301 \url{http://www.eclipse.org/articles/article.php?file=Article-Unleashing-the-Power-of-Refactoring/index.html},
2302 the LTK source code and my own memory.}, or LTK for short, is the framework that
2303 is used to implement refactorings in \name{Eclipse}. It is language independent and
2304 provides the abstractions of a refactoring and the change it generates, in the
2305 form of the classes \typewithref{org.eclipse.ltk.core.refactoring}{Refactoring}
2306 and \typewithref{org.eclipse.ltk.core.refactoring}{Change}.
2308 There are also parts of the LTK that is concerned with user interaction, but
2309 they will not be discussed here, since they are of little value to us and our
2310 use of the framework. We are primarily interested in the parts that can be
2313 \subsubsection{The Refactoring Class}
2314 The abstract class \type{Refactoring} is the core of the LTK framework. Every
2315 refactoring that is going to be supported by the LTK have to end up creating an
2316 instance of one of its subclasses. The main responsibilities of subclasses of
2317 \type{Refactoring} is to implement template methods for condition checking
2318 (\methodwithref{org.eclipse.ltk.core.refactoring.Refactoring}{checkInitialConditions}
2320 \methodwithref{org.eclipse.ltk.core.refactoring.Refactoring}{checkFinalConditions}),
2322 \methodwithref{org.eclipse.ltk.core.refactoring.Refactoring}{createChange}
2323 method that creates and returns an instance of the \type{Change} class.
2325 If the refactoring shall support that others participate in it when it is
2326 executed, the refactoring has to be a processor-based
2327 refactoring\typeref{org.eclipse.ltk.core.refactoring.participants.ProcessorBasedRefactoring}.
2328 It then delegates to its given
2329 \typewithref{org.eclipse.ltk.core.refactoring.participants}{RefactoringProcessor}
2330 for condition checking and change creation. Participating in a refactoring can
2331 be useful in cases where the changes done to programming source code affects
2332 other related resources in the workspace. This can be names or paths in
2333 configuration files, or maybe one would like to perform additional logging of
2334 changes done in the workspace.
2336 \subsubsection{The Change Class}
2337 This class is the base class for objects that is responsible for performing the
2338 actual workspace transformations in a refactoring. The main responsibilities for
2339 its subclasses is to implement the
2340 \methodwithref{org.eclipse.ltk.core.refactoring.Change}{perform} and
2341 \methodwithref{org.eclipse.ltk.core.refactoring.Change}{isValid} methods. The
2342 \method{isValid} method verifies that the change object is valid and thus can be
2343 executed by calling its \method{perform} method. The \method{perform} method
2344 performs the desired change and returns an undo change that can be executed to
2345 reverse the effect of the transformation done by its originating change object.
2347 \subsubsection{Executing a Refactoring}\label{executing_refactoring}
2348 The life cycle of a refactoring generally follows two steps after creation:
2349 condition checking and change creation. By letting the refactoring object be
2351 \typewithref{org.eclipse.ltk.core.refactoring}{CheckConditionsOperation} that
2352 in turn is handled by a
2353 \typewithref{org.eclipse.ltk.core.refactoring}{CreateChangeOperation}, it is
2354 assured that the change creation process is managed in a proper manner.
2356 The actual execution of a change object has to follow a detailed life cycle.
2357 This life cycle is honored if the \type{CreateChangeOperation} is handled by a
2358 \typewithref{org.eclipse.ltk.core.refactoring}{PerformChangeOperation}. If also
2359 an undo manager\typeref{org.eclipse.ltk.core.refactoring.IUndoManager} is set
2360 for the \type{PerformChangeOperation}, the undo change is added into the undo
2363 \section{Shortcomings}
2364 This section is introduced naturally with a conclusion: The JDT refactoring
2365 implementation does not facilitate composition of refactorings.
2366 \todo{refine}This section will try to explain why, and also identify other
2367 shortcomings of both the usability and the readability of the JDT refactoring
2370 I will begin at the end and work my way toward the composition part of this
2373 \subsection{Absence of Generics in Eclipse Source Code}
2374 This section is not only concerning the JDT refactoring API, but also large
2375 quantities of the \name{Eclipse} source code. The code shows a striking absence of the
2376 Java language feature of generics. It is hard to read a class' interface when
2377 methods return objects or takes parameters of raw types such as \type{List} or
2378 \type{Map}. This sometimes results in having to read a lot of source code to
2379 understand what is going on, instead of relying on the available interfaces. In
2380 addition, it results in a lot of ugly code, making the use of typecasting more
2381 of a rule than an exception.
2383 \subsection{Composite Refactorings Will Not Appear as Atomic Actions}
2385 \subsubsection{Missing Flexibility from JDT Refactorings}
2386 The JDT refactorings are not made with composition of refactorings in mind. When
2387 a JDT refactoring is executed, it assumes that all conditions for it to be
2388 applied successfully can be found by reading source files that have been
2389 persisted to disk. They can only operate on the actual source material, and not
2390 (in-memory) copies thereof. This constitutes a major disadvantage when trying to
2391 compose refactorings, since if an exception occurs in the middle of a sequence
2392 of refactorings, it can leave the project in a state where the composite
2393 refactoring was only partially executed. It makes it hard to discard the changes
2394 done without monitoring and consulting the undo manager, an approach that is not
2397 \subsubsection{Broken Undo History}
2398 When designing a composed refactoring that is to be performed as a sequence of
2399 refactorings, you would like it to appear as a single change to the workspace.
2400 This implies that you would also like to be able to undo all the changes done by
2401 the refactoring in a single step. This is not the way it appears when a sequence
2402 of JDT refactorings is executed. It leaves the undo history filled up with
2403 individual undo actions corresponding to every single JDT refactoring in the
2404 sequence. This problem is not trivial to handle in \name{Eclipse}
2405 \see{hacking_undo_history}.
2409 \chapter{Composite Refactorings in Eclipse}
2411 \section{A Simple Ad Hoc Model}
2412 As pointed out in \myref{ch:jdt_refactorings}, the \name{Eclipse} JDT refactoring model
2413 is not very well suited for making composite refactorings. Therefore a simple
2414 model using changer objects (of type \type{RefaktorChanger}) is used as an
2415 abstraction layer on top of the existing \name{Eclipse} refactorings, instead of
2416 extending the \typewithref{org.eclipse.ltk.core.refactoring}{Refactoring} class.
2418 The use of an additional abstraction layer is a deliberate choice. It is due to
2419 the problem of creating a composite
2420 \typewithref{org.eclipse.ltk.core.refactoring}{Change} that can handle text
2421 changes that interfere with each other. Thus, a \type{RefaktorChanger} may, or
2422 may not, take advantage of one or more existing refactorings, but it is always
2423 intended to make a change to the workspace.
2425 \subsection{A typical \type{RefaktorChanger}}
2426 The typical refaktor changer class has two responsibilities, checking
2427 preconditions and executing the requested changes. This is not too different
2428 from the responsibilities of an LTK refactoring, with the distinction that a
2429 refaktor changer also executes the change, while an LTK refactoring is only
2430 responsible for creating the object that can later be used to do the job.
2432 Checking of preconditions is typically done by an
2433 \typewithref{no.uio.ifi.refaktor.analyze.analyzers}{Analyzer}. If the
2434 preconditions validate, the upcoming changes are executed by an
2435 \typewithref{no.uio.ifi.refaktor.change.executors}{Executor}.
2437 \section{The Extract and Move Method Refactoring}
2438 %The Extract and Move Method Refactoring is implemented mainly using these
2441 % \item \type{ExtractAndMoveMethodChanger}
2442 % \item \type{ExtractAndMoveMethodPrefixesExtractor}
2443 % \item \type{Prefix}
2444 % \item \type{PrefixSet}
2447 \subsection{The Building Blocks}
2448 This is a composite refactoring, and hence is built up using several primitive
2449 refactorings. These basic building blocks are, as its name implies, the
2450 \ExtractMethod refactoring\citing{refactoring} and the \MoveMethod
2451 refactoring\citing{refactoring}. In \name{Eclipse}, the implementations of these
2452 refactorings are found in the classes
2453 \typewithref{org.eclipse.jdt.internal.corext.refactoring.code}{ExtractMethodRefactoring}
2455 \typewithref{org.eclipse.jdt.internal.corext.refactoring.structure}{MoveInstanceMethodProcessor},
2456 where the last class is designed to be used together with the processor-based
2457 \typewithref{org.eclipse.ltk.core.refactoring.participants}{MoveRefactoring}.
2459 \subsubsection{The ExtractMethodRefactoring Class}
2460 This class is quite simple in its use. The only parameters it requires for
2461 construction is a compilation
2462 unit\typeref{org.eclipse.jdt.core.ICompilationUnit}, the offset into the source
2463 code where the extraction shall start, and the length of the source to be
2464 extracted. Then you have to set the method name for the new method together with
2465 its visibility and some not so interesting parameters.
2467 \subsubsection{The MoveInstanceMethodProcessor Class}
2468 For the \refa{Move Method}, the processor requires a little more advanced input than
2469 the class for the \refa{Extract Method}. For construction it requires a method
2470 handle\typeref{org.eclipse.jdt.core.IMethod} for the method that is to be moved.
2471 Then the target for the move have to be supplied as the variable binding from a
2472 chosen variable declaration. In addition to this, one have to set some
2473 parameters regarding setters/getters, as well as delegation.
2475 To make a working refactoring from the processor, one have to create a
2476 \type{MoveRefactoring} with it.
2478 \subsection{The ExtractAndMoveMethodChanger}
2480 The \typewithref{no.uio.ifi.refaktor.changers}{ExtractAndMoveMethodChanger}
2481 class is a subclass of the class
2482 \typewithref{no.uio.ifi.refaktor.changers}{RefaktorChanger}. It is responsible
2483 for analyzing and finding the best target for, and also executing, a composition
2484 of the \refa{Extract Method} and \refa{Move Method} refactorings. This particular changer is
2485 the one of my changers that is closest to being a true LTK refactoring. It can
2486 be reworked to be one if the problems with overlapping changes are resolved. The
2487 changer requires a text selection and the name of the new method, or else a
2488 method name will be generated. The selection has to be of the type
2489 \typewithref{no.uio.ifi.refaktor.utils}{CompilationUnitTextSelection}. This
2490 class is a custom extension to
2491 \typewithref{org.eclipse.jface.text}{TextSelection}, that in addition to the
2492 basic offset, length and similar methods, also carry an instance of the
2493 underlying compilation unit handle for the selection.
2496 \type{ExtractAndMoveMethodAnalyzer}}\label{extractAndMoveMethodAnalyzer}
2497 The analysis and precondition checking is done by the
2498 \typewithref{no.uio.ifi.refaktor.analyze.analyzers}{ExtractAnd\-MoveMethodAnalyzer}.
2499 First is check whether the selection is a valid selection or not, with respect
2500 to statement boundaries and that it actually contains any selections. Then it
2501 checks the legality of both extracting the selection and also moving it to
2502 another class. This checking of is performed by a range of checkers
2503 \see{checkers}. If the selection is approved as legal, it is analyzed to find
2504 the presumably best target to move the extracted method to.
2506 For finding the best suitable target the analyzer is using a
2507 \typewithref{no.uio.ifi.refaktor.analyze.collectors}{PrefixesCollector} that
2508 collects all the possible candidate targets for the refactoring. All the
2509 non-candidates is found by an
2510 \typewithref{no.uio.ifi.refaktor.analyze.collectors}{UnfixesCollector} that
2511 collects all the targets that will give some kind of error if used. (For
2512 details about the property collectors, see \myref{propertyCollectors}.) All
2513 prefixes (and unfixes) are represented by a
2514 \typewithref{no.uio.ifi.refaktor.extractors}{Prefix}, and they are collected
2515 into sets of prefixes. The safe prefixes is found by subtracting from the set of
2516 candidate prefixes the prefixes that is enclosing any of the unfixes. A prefix
2517 is enclosing an unfix if the unfix is in the set of its sub-prefixes. As an
2518 example, \code{``a.b''} is enclosing \code{``a''}, as is \code{``a''}. The safe
2519 prefixes is unified in a \type{PrefixSet}. If a prefix has only one occurrence,
2520 and is a simple expression, it is considered unsuitable as a move target. This
2521 occurs in statements such as \code{``a.foo()''}. For such statements it bares no
2522 meaning to extract and move them. It only generates an extra method and the
2525 The most suitable target for the refactoring is found by finding the prefix with
2526 the most occurrences. If two prefixes have the same occurrence count, but they
2527 differ in the number of segments, the one with most segments is chosen.
2530 \type{ExtractAndMoveMethodExecutor}}\label{extractAndMoveMethodExecutor}
2531 If the analysis finds a possible target for the composite refactoring, it is
2533 \typewithref{no.uio.ifi.refaktor.change.executors}{ExtractAndMoveMethodExecutor}.
2534 It is composed of the two executors known as
2535 \typewithref{no.uio.ifi.refaktor.change.executors}{ExtractMethodRefactoringExecutor}
2537 \typewithref{no.uio.ifi.refaktor.change.executors}{MoveMethodRefactoringExecutor}.
2538 The \type{ExtractAndMoveMethodExecutor} is responsible for gluing the two
2539 together by feeding the \type{MoveMethod\-RefactoringExecutor} with the
2540 resources needed after executing the extract method refactoring.
2542 \subsubsection{The \type{ExtractMethodRefactoringExecutor}}
2543 This executor is responsible for creating and executing an instance of the
2544 \type{ExtractMethodRefactoring} class. It is also responsible for collecting
2545 some post execution resources that can be used to find the method handle for the
2546 extracted method, as well as information about its parameters, including the
2547 variable they originated from.
2549 \subsubsection{The \type{MoveMethodRefactoringExecutor}}
2550 This executor is responsible for creating and executing an instance of the
2551 \type{MoveRefactoring}. The move refactoring is a processor-based refactoring,
2552 and for the \refa{Move Method} refactoring it is the \type{MoveInstanceMethodProcessor}
2555 The handle for the method to be moved is found on the basis of the information
2556 gathered after the execution of the \refa{Extract Method} refactoring. The only
2557 information the \type{ExtractMethodRefactoring} is sharing after its execution,
2558 regarding find the method handle, is the textual representation of the new
2559 method signature. Therefore it must be parsed, the strings for types of the
2560 parameters must be found and translated to a form that can be used to look up
2561 the method handle from its type handle. They have to be on the unresolved
2562 form.\todo{Elaborate?} The name for the type is found from the original
2563 selection, since an extracted method must end up in the same type as the
2566 When analyzing a selection prior to performing the \refa{Extract Method} refactoring, a
2567 target is chosen. It has to be a variable binding, so it is either a field or a
2568 local variable/parameter. If the target is a field, it can be used with the
2569 \type{MoveInstanceMethodProcessor} as it is, since the extracted method still is
2570 in its scope. But if the target is local to the originating method, the target
2571 that is to be used for the processor must be among its parameters. Thus the
2572 target must be found among the extracted method's parameters. This is done by
2573 finding the parameter information object that corresponds to the parameter that
2574 was declared on basis of the original target's variable when the method was
2575 extracted. (The extracted method must take one such parameter for each local
2576 variable that is declared outside the selection that is extracted.) To match the
2577 original target with the correct parameter information object, the key for the
2578 information object is compared to the key from the original target's binding.
2579 The source code must then be parsed to find the method declaration for the
2580 extracted method. The new target must be found by searching through the
2581 parameters of the declaration and choose the one that has the same type as the
2582 old binding from the parameter information object, as well as the same name that
2583 is provided by the parameter information object.
2587 SearchBasedExtractAndMoveMethodChanger}\label{searchBasedExtractAndMoveMethodChanger}
2589 \typewithref{no.uio.ifi.refaktor.change.changers}{SearchBasedExtractAndMoveMethodChanger}
2590 is a changer whose purpose is to automatically analyze a method, and execute the
2591 \ExtractAndMoveMethod refactoring on it if it is a suitable candidate for the
2594 First, the \typewithref{no.uio.ifi.refaktor.analyze.analyzers}{SearchBasedExtractAndMoveMethodAnalyzer} is used
2595 to analyze the method. If the method is found to be a candidate, the result from
2596 the analysis is fed to the \type{ExtractAndMoveMethodExecutor}, whose job is to
2597 execute the refactoring \see{extractAndMoveMethodExecutor}.
2599 \subsubsection{The SearchBasedExtractAndMoveMethodAnalyzer}
2600 This analyzer is responsible for analyzing all the possible text selections of a
2601 method and then choose the best result out of the analysis results that is, by
2602 the analyzer, considered to be the potential candidates for the Extract and Move
2605 Before the analyzer is able to work with the text selections of a method, it
2606 needs to generate them. To do this, it parses the method to obtain a
2607 \type{MethodDeclaration} for it \see{astEclipse}. Then there is a statement
2608 lists creator that creates statements lists of the different groups of
2609 statements in the body of the method declaration. A text selections generator
2610 generates text selections of all the statement lists for the analyzer to work
2613 \paragraph{The statement lists creator}
2614 is responsible for generating lists of statements for all the possible nesting
2615 levels of statements in the method. The statement lists creator is implemented
2616 as an AST visitor \see{astVisitor}. It generates lists of statements by visiting
2617 all the blocks in the method declaration and stores their statements in a
2618 collection of statement lists. In addition, it visits all of the other
2619 statements that can have a statement as a child, such as the different control
2620 structures and the labeled statement.
2622 The switch statement is the only kind of statement that is not straight forward
2623 to obtain the child statements from. It stores all of its children in a flat
2624 list. Its switch case statements are included in this list. This means that
2625 there are potential statement lists between all of these case statements. The
2626 list of statements from a switch statement is therefore traversed, and the
2627 statements between the case statements are grouped as separate lists.
2629 The highlighted part of \myref{lst:grandExample} shows an example of how the
2630 statement lists creator would separate a method body into lists of statements.
2632 \paragraph{The text selections generator} generates text selections for each
2633 list of statements from the statement lists creator. The generator generates a
2634 text selection for every sub-sequence of statements in a list. For a sequence of
2635 statements, the first statement and the last statement span out a text
2638 In practice, the text selections are calculated by only one traversal of the
2639 statement list. There is a set of generated text selections. For each statement,
2640 there is created a temporary set of selections, in addition to a text selection
2641 based on the offset and length of the statement. This text selection is added to
2642 the temporary set. Then the new selection is added with every selection from the
2643 set of generated text selections. These new selections are added to the
2644 temporary set. Then the temporary set of selections is added to the set of
2645 generated text selections. The result of adding two text selections is a new
2646 text selection spanned out by the two addends.
2650 \def\charwidth{5.7pt}
2651 \def\indent{4*\charwidth}
2652 \def\lineheight{\baselineskip}
2653 \def\mintedtop{\lineheight}
2655 \begin{tikzpicture}[overlay, yscale=-1]
2656 \tikzstyle{overlaybox}=[fill=lightgray,opacity=0.2]
2658 \draw[overlaybox] (2*\charwidth,\mintedtop) rectangle
2659 +(18*\charwidth,\lineheight);
2661 \draw[overlaybox] (2*\charwidth,\mintedtop+\lineheight) rectangle
2662 +(18*\charwidth,\lineheight);
2664 \draw[overlaybox] (2*\charwidth,\mintedtop+3*\lineheight) rectangle
2665 +(18*\charwidth,\lineheight);
2667 \draw[overlaybox] (\indent-3*\charwidth,\mintedtop) rectangle
2668 +(20*\charwidth,2*\lineheight);
2670 \draw[overlaybox] (3*\charwidth,\mintedtop+\lineheight) rectangle
2671 +(16*\charwidth,3*\lineheight);
2673 \draw[overlaybox] (\indent,\mintedtop) rectangle
2674 +(14*\charwidth,4*\lineheight);
2676 \begin{minted}{java}
2682 \caption{Example of how the text selections generator would generate text
2683 selections based on a lists of statements. Each highlighted rectangle
2684 represents a text selection.}
2685 \label{lst:textSelectionsExample}
2687 \todoin{fix \myref{lst:textSelectionsExample}? Text only? All
2688 sub-sequences\ldots}
2691 \paragraph{Finding the candidate} for the refactoring is done by analyzing all
2692 the generated text selection with the \type{ExtractAndMoveMethodAnalyzer}
2693 \see{extractAndMoveMethodAnalyzer}. If the analyzer generates a useful result,
2694 an \type{ExtractAndMoveMethodCandidate} is created from it, that is kept in a
2695 list of potential candidates. If no candidates are found, the
2696 \type{NoTargetFoundException} is thrown.
2698 Since only one of the candidates can be chosen, the analyzer must sort out which
2699 candidate to choose. The sorting is done by the static \method{sort} method of
2700 \type{Collections}. The comparison in this sorting is done by an
2701 \type{ExtractAndMoveMethodCandidateComparator}.
2702 \todoin{Write about the
2703 ExtractAndMoveMethodCandidateComparator/FavorNoUnfixesCandidateComparator}
2706 \subsection{The Prefix Class}
2707 This class exists mainly for holding data about a prefix, such as the expression
2708 that the prefix represents and the occurrence count of the prefix within a
2709 selection. In addition to this, it has some functionality such as calculating
2710 its sub-prefixes and intersecting it with another prefix. The definition of the
2711 intersection between two prefixes is a prefix representing the longest common
2712 expression between the two.
2714 \subsection{The PrefixSet Class}
2715 A prefix set holds elements of type \type{Prefix}. It is implemented with the
2716 help of a \typewithref{java.util}{HashMap} and contains some typical set
2717 operations, but it does not implement the \typewithref{java.util}{Set}
2718 interface, since the prefix set does not need all of the functionality a
2719 \type{Set} requires to be implemented. In addition It needs some other
2720 functionality not found in the \type{Set} interface. So due to the relatively
2721 limited use of prefix sets, and that it almost always needs to be referenced as
2722 such, and not a \type{Set<Prefix>}, it remains as an ad hoc solution to a
2725 There are two ways adding prefixes to a \type{PrefixSet}. The first is through
2726 its \method{add} method. This works like one would expect from a set. It adds
2727 the prefix to the set if it does not already contain the prefix. The other way
2728 is to \emph{register} the prefix with the set. When registering a prefix, if the
2729 set does not contain the prefix, it is just added. If the set contains the
2730 prefix, its count gets incremented. This is how the occurrence count is handled.
2732 The prefix set also computes the set of prefixes that is not enclosing any
2733 prefixes of another set. This is kind of a set difference operation only for
2736 \subsection{Hacking the Refactoring Undo
2737 History}\label{hacking_undo_history}
2738 \todoin{Where to put this section?}
2740 As an attempt to make multiple subsequent changes to the workspace appear as a
2741 single action (i.e. make the undo changes appear as such), I tried to alter
2742 the undo changes\typeref{org.eclipse.ltk.core.refactoring.Change} in the history
2743 of the refactorings.
2745 My first impulse was to remove the, in this case, last two undo changes from the
2746 undo manager\typeref{org.eclipse.ltk.core.refactoring.IUndoManager} for the
2747 \name{Eclipse} refactorings, and then add them to a composite
2748 change\typeref{org.eclipse.ltk.core.refactoring.CompositeChange} that could be
2749 added back to the manager. The interface of the undo manager does not offer a
2750 way to remove/pop the last added undo change, so a possible solution could be to
2751 decorate\citing{designPatterns} the undo manager, to intercept and collect the
2752 undo changes before delegating to the \method{addUndo}
2753 method\methodref{org.eclipse.ltk.core.refactoring.IUndoManager}{addUndo} of the
2754 manager. Instead of giving it the intended undo change, a null change could be
2755 given to prevent it from making any changes if run. Then one could let the
2756 collected undo changes form a composite change to be added to the manager.
2758 There is a technical challenge with this approach, and it relates to the undo
2759 manager, and the concrete implementation
2760 \typewithref{org.eclipse.ltk.internal.core.refactoring}{UndoManager2}. This
2761 implementation is designed in a way that it is not possible to just add an undo
2762 change, you have to do it in the context of an active
2763 operation\typeref{org.eclipse.core.commands.operations.TriggeredOperations}.
2764 One could imagine that it might be possible to trick the undo manager into
2765 believing that you are doing a real change, by executing a refactoring that is
2766 returning a kind of null change that is returning our composite change of undo
2767 refactorings when it is performed. But this is not the way to go.
2769 Apart from the technical problems with this solution, there is a functional
2770 problem: If it all had worked out as planned, this would leave the undo history
2771 in a dirty state, with multiple empty undo operations corresponding to each of
2772 the sequentially executed refactoring operations, followed by a composite undo
2773 change corresponding to an empty change of the workspace for rounding of our
2774 composite refactoring. The solution to this particular problem could be to
2775 intercept the registration of the intermediate changes in the undo manager, and
2776 only register the last empty change.
2778 Unfortunately, not everything works as desired with this solution. The grouping
2779 of the undo changes into the composite change does not make the undo operation
2780 appear as an atomic operation. The undo operation is still split up into
2781 separate undo actions, corresponding to the change done by its originating
2782 refactoring. And in addition, the undo actions has to be performed separate in
2783 all the editors involved. This makes it no solution at all, but a step toward
2786 There might be a solution to this problem, but it remains to be found. The
2787 design of the refactoring undo management is partly to be blamed for this, as it
2788 it is to complex to be easily manipulated.
2793 \chapter{Analyzing Source Code in Eclipse}
2795 \section{The Java model}\label{javaModel}
2796 The Java model of \name{Eclipse} is its internal representation of a Java project. It
2797 is light-weight, and has only limited possibilities for manipulating source
2798 code. It is typically used as a basis for the Package Explorer in \name{Eclipse}.
2800 The elements of the Java model is only handles to the underlying elements. This
2801 means that the underlying element of a handle does not need to actually exist.
2802 Hence the user of a handle must always check that it exist by calling the
2803 \method{exists} method of the handle.
2805 The handles with descriptions is listed in \myref{tab:javaModel}, while the
2806 hierarchy of the Java Model is shown in \myref{fig:javaModel}.
2809 \caption{The elements of the Java Model\citing{vogelEclipseJDT2012}.}
2810 \label{tab:javaModel}
2812 % sum must equal number of columns (3)
2813 \begin{tabularx}{\textwidth}{@{} L{0.7} L{1.1} L{1.2} @{}}
2815 \textbf{Project Element} & \textbf{Java Model element} &
2816 \textbf{Description} \\
2818 Java project & \type{IJavaProject} & The Java project which contains all other objects. \\
2820 Source folder /\linebreak[2] binary folder /\linebreak[3] external library &
2821 \type{IPackageFragmentRoot} & Hold source or binary files, can be a folder
2822 or a library (zip / jar file). \\
2824 Each package & \type{IPackageFragment} & Each package is below the
2825 \type{IPackageFragmentRoot}, sub-packages are not leaves of the package,
2826 they are listed directed under \type{IPackageFragmentRoot}. \\
2828 Java Source file & \type{ICompilationUnit} & The Source file is always below
2829 the package node. \\
2831 Types / Fields /\linebreak[3] Methods & \type{IType} / \type{IField}
2832 /\linebreak[3] \type{IMethod} & Types, fields and methods. \\
2840 \begin{tikzpicture}[%
2841 grow via three points={one child at (0,-0.7) and
2842 two children at (0,-0.7) and (0,-1.4)},
2843 edge from parent path={(\tikzparentnode.south west)+(0.5,0) |-
2844 (\tikzchildnode.west)}]
2845 \tikzstyle{every node}=[draw=black,thick,anchor=west]
2846 \tikzstyle{selected}=[draw=red,fill=red!30]
2847 \tikzstyle{optional}=[dashed,fill=gray!50]
2848 \node {\type{IJavaProject}}
2849 child { node {\type{IPackageFragmentRoot}}
2850 child { node {\type{IPackageFragment}}
2851 child { node {\type{ICompilationUnit}}
2852 child { node {\type{IType}}
2853 child { node {\type{\{ IType \}*}}
2854 child { node {\type{\ldots}}}
2857 child { node {\type{\{ IField \}*}}}
2858 child { node {\type{IMethod}}
2859 child { node {\type{\{ IType \}*}}
2860 child { node {\type{\ldots}}}
2865 child { node {\type{\{ IMethod \}*}}}
2874 child { node {\type{\{ IType \}*}}}
2885 child { node {\type{\{ ICompilationUnit \}*}}}
2898 child { node {\type{\{ IPackageFragment \}*}}}
2913 child { node {\type{\{ IPackageFragmentRoot \}*}}}
2916 \caption{The Java model of \name{Eclipse}. ``\type{\{ SomeElement \}*}'' means
2917 ``\type{SomeElement} zero or more times``. For recursive structures,
2918 ``\type{\ldots}'' is used.}
2919 \label{fig:javaModel}
2922 \section{The Abstract Syntax Tree}
2923 \name{Eclipse} is following the common paradigm of using an abstract syntax tree for
2924 source code analysis and manipulation.
2926 When parsing program source code into something that can be used as a foundation
2927 for analysis, the start of the process follows the same steps as in a compiler.
2928 This is all natural, because the way a compiler analyzes code is no different
2929 from how source manipulation programs would do it, except for some properties of
2930 code that is analyzed in the parser, and that they may be differing in what
2931 kinds of properties they analyze. Thus the process of translation source code
2932 into a structure that is suitable for analyzing, can be seen as a kind of
2933 interrupted compilation process \see{fig:interruptedCompilationProcess}.
2938 base/.style={anchor=north, align=center, rectangle, minimum height=1.4cm},
2939 basewithshadow/.style={base, drop shadow, fill=white},
2940 outlined/.style={basewithshadow, draw, rounded corners, minimum
2942 primary/.style={outlined, font=\bfseries},
2943 dashedbox/.style={outlined, dashed},
2944 arrowpath/.style={black, align=center, font=\small},
2945 processarrow/.style={arrowpath, ->, >=angle 90, shorten >=1pt},
2947 \begin{tikzpicture}[node distance=1.3cm and 3cm, scale=1, every
2948 node/.style={transform shape}]
2949 \node[base](AuxNode1){\small source code};
2950 \node[primary, right=of AuxNode1, xshift=-2.5cm](Scanner){Scanner};
2951 \node[primary, right=of Scanner, xshift=0.5cm](Parser){Parser};
2952 \node[dashedbox, below=of Parser](SemanticAnalyzer){Semantic\\Analyzer};
2953 \node[dashedbox, left=of SemanticAnalyzer](SourceCodeOptimizer){Source
2955 \node[dashedbox, below=of SourceCodeOptimizer
2956 ](CodeGenerator){Code\\Generator};
2957 \node[dashedbox, right=of CodeGenerator](TargetCodeOptimizer){Target
2959 \node[base, right=of TargetCodeOptimizer](AuxNode2){};
2961 \draw[processarrow](AuxNode1) -- (Scanner);
2963 \path[arrowpath] (Scanner) -- node [sloped](tokens){tokens}(Parser);
2964 \draw[processarrow](Scanner) -- (tokens) -- (Parser);
2966 \path[arrowpath] (Parser) -- node (syntax){syntax
2967 tree}(SemanticAnalyzer);
2968 \draw[processarrow](Parser) -- (syntax) -- (SemanticAnalyzer);
2970 \path[arrowpath] (SemanticAnalyzer) -- node
2971 [sloped](annotated){annotated\\tree}(SourceCodeOptimizer);
2972 \draw[processarrow, dashed](SemanticAnalyzer) -- (annotated) --
2973 (SourceCodeOptimizer);
2975 \path[arrowpath] (SourceCodeOptimizer) -- node
2976 (intermediate){intermediate code}(CodeGenerator);
2977 \draw[processarrow, dashed](SourceCodeOptimizer) -- (intermediate) --
2980 \path[arrowpath] (CodeGenerator) -- node [sloped](target1){target
2981 code}(TargetCodeOptimizer);
2982 \draw[processarrow, dashed](CodeGenerator) -- (target1) --
2983 (TargetCodeOptimizer);
2985 \path[arrowpath](TargetCodeOptimizer) -- node [sloped](target2){target
2987 \draw[processarrow, dashed](TargetCodeOptimizer) -- (target2) (AuxNode2);
2989 \caption{Interrupted compilation process. {\footnotesize (Full compilation
2990 process borrowed from \emph{Compiler construction: principles and practice}
2991 by Kenneth C. Louden\citing{louden1997}.)}}
2992 \label{fig:interruptedCompilationProcess}
2995 The process starts with a \emph{scanner}, or lexer. The job of the scanner is to
2996 read the source code and divide it into tokens for the parser. Therefore, it is
2997 also sometimes called a tokenizer. A token is a logical unit, defined in the
2998 language specification, consisting of one or more consecutive characters. In
2999 the Java language the tokens can for instance be the \var{this} keyword, a curly
3000 bracket \var{\{} or a \var{nameToken}. It is recognized by the scanner on the
3001 basis of something equivalent of a regular expression. This part of the process
3002 is often implemented with the use of a finite automata. In fact, it is common to
3003 specify the tokens in regular expressions, that in turn is translated into a
3004 finite automata lexer. This process can be automated.
3006 The program component used to translate a stream of tokens into something
3007 meaningful, is called a parser. A parser is fed tokens from the scanner and
3008 performs an analysis of the structure of a program. It verifies that the syntax
3009 is correct according to the grammar rules of a language, that is usually
3010 specified in a context-free grammar, and often in a variant of the
3012 Form}\footnote{\url{https://en.wikipedia.org/wiki/Backus-Naur\_Form}}. The
3013 result coming from the parser is in the form of an \emph{Abstract Syntax Tree},
3014 AST for short. It is called \emph{abstract}, because the structure does not
3015 contain all of the tokens produced by the scanner. It only contain logical
3016 constructs, and because it forms a tree, all kinds of parentheses and brackets
3017 are implicit in the structure. It is this AST that is used when performing the
3018 semantic analysis of the code.
3020 As an example we can think of the expression \code{(5 + 7) * 2}. The root of
3021 this tree would in \name{Eclipse} be an \type{InfixExpression} with the operator
3022 \var{TIMES}, and a left operand that is also an \type{InfixExpression} with the
3023 operator \var{PLUS}. The left operand \type{InfixExpression}, has in turn a left
3024 operand of type \type{NumberLiteral} with the value \var{``5''} and a right
3025 operand \type{NumberLiteral} with the value \var{``7''}. The root will have a
3026 right operand of type \type{NumberLiteral} and value \var{``2''}. The AST for
3027 this expression is illustrated in \myref{fig:astInfixExpression}.
3029 Contrary to the Java Model, an abstract syntax tree is a heavy-weight
3030 representation of source code. It contains information about properties like
3031 type bindings for variables and variable bindings for names.
3036 \begin{tikzpicture}[scale=0.8]
3037 \tikzset{level distance=40pt}
3038 \tikzset{sibling distance=5pt}
3039 \tikzstyle{thescale}=[scale=0.8]
3040 \tikzset{every tree node/.style={align=center}}
3041 \tikzset{edge from parent/.append style={thick}}
3042 \tikzstyle{inode}=[rectangle,rounded corners,draw,fill=lightgray,drop
3043 shadow,align=center]
3044 \tikzset{every internal node/.style={inode}}
3045 \tikzset{every leaf node/.style={draw=none,fill=none}}
3047 \Tree [.\type{InfixExpression} [.\type{InfixExpression}
3048 [.\type{NumberLiteral} \var{``5''} ] [.\type{Operator} \var{PLUS} ]
3049 [.\type{NumberLiteral} \var{``7''} ] ]
3050 [.\type{Operator} \var{TIMES} ]
3051 [.\type{NumberLiteral} \var{``2''} ]
3054 \caption{The abstract syntax tree for the expression \code{(5 + 7) * 2}.}
3055 \label{fig:astInfixExpression}
3058 \subsection{The AST in Eclipse}\label{astEclipse}
3059 In \name{Eclipse}, every node in the AST is a child of the abstract superclass
3060 \typewithref{org.eclipse.jdt.core.dom}{ASTNode}. Every \type{ASTNode}, among a
3061 lot of other things, provides information about its position and length in the
3062 source code, as well as a reference to its parent and to the root of the tree.
3064 The root of the AST is always of type \type{CompilationUnit}. It is not the same
3065 as an instance of an \type{ICompilationUnit}, which is the compilation unit
3066 handle of the Java model. The children of a \type{CompilationUnit} is an
3067 optional \type{PackageDeclaration}, zero or more nodes of type
3068 \type{ImportDecaration} and all its top-level type declarations that has node
3069 types \type{AbstractTypeDeclaration}.
3071 An \type{AbstractType\-Declaration} can be one of the types
3072 \type{AnnotationType\-Declaration}, \type{Enum\-Declaration} or
3073 \type{Type\-Declaration}. The children of an \type{AbstractType\-Declaration}
3074 must be a subtype of a \type{BodyDeclaration}. These subtypes are:
3075 \type{AnnotationTypeMember\-Declaration}, \type{EnumConstant\-Declaration},
3076 \type{Field\-Declaration}, \type{Initializer} and \type{Method\-Declaration}.
3078 Of the body declarations, the \type{Method\-Declaration} is the most interesting
3079 one. Its children include lists of modifiers, type parameters, parameters and
3080 exceptions. It has a return type node and a body node. The body, if present, is
3081 of type \type{Block}. A \type{Block} is itself a \type{Statement}, and its
3082 children is a list of \type{Statement} nodes.
3084 There are too many types of the abstract type \type{Statement} to list up, but
3085 there exists a subtype of \type{Statement} for every statement type of Java, as
3086 one would expect. This also applies to the abstract type \type{Expression}.
3087 However, the expression \type{Name} is a little special, since it is both used
3088 as an operand in compound expressions, as well as for names in type declarations
3091 There is an overview of some of the structure of an \name{Eclipse} AST in
3092 \myref{fig:astEclipse}.
3096 \begin{tikzpicture}[scale=0.8]
3097 \tikzset{level distance=50pt}
3098 \tikzset{sibling distance=5pt}
3099 \tikzstyle{thescale}=[scale=0.8]
3100 \tikzset{every tree node/.style={align=center}}
3101 \tikzset{edge from parent/.append style={thick}}
3102 \tikzstyle{inode}=[rectangle,rounded corners,draw,fill=lightgray,drop
3103 shadow,align=center]
3104 \tikzset{every internal node/.style={inode}}
3105 \tikzset{every leaf node/.style={draw=none,fill=none}}
3107 \Tree [.\type{CompilationUnit} [.\type{[ PackageDeclaration ]} [.\type{Name} ]
3108 [.\type{\{ Annotation \}*} ] ]
3109 [.\type{\{ ImportDeclaration \}*} [.\type{Name} ] ]
3110 [.\type{\{ AbstractTypeDeclaration \}+} [.\node(site){\type{\{
3111 BodyDeclaration \}*}}; ] [.\type{SimpleName} ] ]
3113 \begin{scope}[shift={(0.5,-6)}]
3114 \node[inode,thescale](root){\type{MethodDeclaration}};
3115 \node[inode,thescale](modifiers) at (4.5,-5){\type{\{ IExtendedModifier \}*}
3116 \\ {\footnotesize (Of type \type{Modifier} or \type{Annotation})}};
3117 \node[inode,thescale](typeParameters) at (-6,-3.5){\type{\{ TypeParameter
3119 \node[inode,thescale](parameters) at (-5,-5){\type{\{
3120 SingleVariableDeclaration \}*} \\ {\footnotesize (Parameters)}};
3121 \node[inode,thescale](exceptions) at (5,-3){\type{\{ Name \}*} \\
3122 {\footnotesize (Exceptions)}};
3123 \node[inode,thescale](return) at (-6.5,-2){\type{Type} \\ {\footnotesize
3125 \begin{scope}[shift={(0,-5)}]
3126 \Tree [.\node(body){\type{[ Block ]} \\ {\footnotesize (Body)}};
3127 [.\type{\{ Statement \}*} [.\type{\{ Expression \}*} ]
3128 [.\type{\{ Statement \}*} [.\type{\ldots} ]]
3133 \draw[->,>=triangle 90,shorten >=1pt](root.east)..controls +(east:2) and
3134 +(south:1)..(site.south);
3136 \draw (root.south) -- (modifiers);
3137 \draw (root.south) -- (typeParameters);
3138 \draw (root.south) -- ($ (parameters.north) + (2,0) $);
3139 \draw (root.south) -- (exceptions);
3140 \draw (root.south) -- (return);
3141 \draw (root.south) -- (body);
3144 \caption{The format of the abstract syntax tree in \name{Eclipse}.}
3145 \label{fig:astEclipse}
3147 \todoin{Add more to the AST format tree? \myref{fig:astEclipse}}
3149 \section{The ASTVisitor}\label{astVisitor}
3150 So far, the only thing that has been addressed is how the data that is going to
3151 be the basis for our analysis is structured. Another aspect of it is how we are
3152 going to traverse the AST to gather the information we need, so we can conclude
3153 about the properties we are analysing. It is of course possible to start at the
3154 top of the tree, and manually search through its nodes for the ones we are
3155 looking for, but that is a bit inconvenient. To be able to efficiently utilize
3156 such an approach, we would need to make our own framework for traversing the
3157 tree and visiting only the types of nodes we are after. Luckily, this
3158 functionality is already provided in \name{Eclipse}, by its
3159 \typewithref{org.eclipse.jdt.core.dom}{ASTVisitor}.
3161 The \name{Eclipse} AST, together with its \type{ASTVisitor}, follows the
3162 \pattern{Visitor} pattern\citing{designPatterns}. The intent of this design
3163 pattern is to facilitate extending the functionality of classes without touching
3164 the classes themselves.
3166 Let us say that there is a class hierarchy of elements. These elements all have
3167 a method \method{accept(Visitor visitor)}. In its simplest form, the
3168 \method{accept} method just calls the \method{visit} method of the visitor with
3169 itself as an argument, like this: \code{visitor.visit(this)}. For the visitors
3170 to be able to extend the functionality of all the classes in the elements
3171 hierarchy, each \type{Visitor} must have one visit method for each concrete
3172 class in the hierarchy. Say the hierarchy consists of the concrete classes
3173 \type{ConcreteElementA} and \type{ConcreteElementB}. Then each visitor must have
3174 the (possibly empty) methods \method{visit(ConcreteElementA element)} and
3175 \method{visit(ConcreteElementB element)}. This scenario is depicted in
3176 \myref{fig:visitorPattern}.
3180 \tikzstyle{abstract}=[rectangle, draw=black, fill=white, drop shadow, text
3181 centered, anchor=north, text=black, text width=6cm, every one node
3182 part/.style={align=center, font=\bfseries\itshape}]
3183 \tikzstyle{concrete}=[rectangle, draw=black, fill=white, drop shadow, text
3184 centered, anchor=north, text=black, text width=6cm]
3185 \tikzstyle{inheritarrow}=[->, >=open triangle 90, thick]
3186 \tikzstyle{commentarrow}=[->, >=angle 90, dashed]
3187 \tikzstyle{line}=[-, thick]
3188 \tikzset{every one node part/.style={align=center, font=\bfseries}}
3189 \tikzset{every second node part/.style={align=center, font=\ttfamily}}
3191 \begin{tikzpicture}[node distance=1cm, scale=0.8, every node/.style={transform
3193 \node (Element) [abstract, rectangle split, rectangle split parts=2]
3195 \nodepart{one}{Element}
3196 \nodepart{second}{+accept(visitor: Visitor)}
3198 \node (AuxNode01) [text width=0, minimum height=2cm, below=of Element] {};
3199 \node (ConcreteElementA) [concrete, rectangle split, rectangle split
3200 parts=2, left=of AuxNode01]
3202 \nodepart{one}{ConcreteElementA}
3203 \nodepart{second}{+accept(visitor: Visitor)}
3205 \node (ConcreteElementB) [concrete, rectangle split, rectangle split
3206 parts=2, right=of AuxNode01]
3208 \nodepart{one}{ConcreteElementB}
3209 \nodepart{second}{+accept(visitor: Visitor)}
3212 \node[comment, below=of ConcreteElementA] (CommentA) {visitor.visit(this)};
3214 \node[comment, below=of ConcreteElementB] (CommentB) {visitor.visit(this)};
3216 \node (AuxNodeX) [text width=0, minimum height=1cm, below=of AuxNode01] {};
3218 \node (Visitor) [abstract, rectangle split, rectangle split parts=2,
3221 \nodepart{one}{Visitor}
3222 \nodepart{second}{+visit(ConcreteElementA)\\+visit(ConcreteElementB)}
3224 \node (AuxNode02) [text width=0, minimum height=2cm, below=of Visitor] {};
3225 \node (ConcreteVisitor1) [concrete, rectangle split, rectangle split
3226 parts=2, left=of AuxNode02]
3228 \nodepart{one}{ConcreteVisitor1}
3229 \nodepart{second}{+visit(ConcreteElementA)\\+visit(ConcreteElementB)}
3231 \node (ConcreteVisitor2) [concrete, rectangle split, rectangle split
3232 parts=2, right=of AuxNode02]
3234 \nodepart{one}{ConcreteVisitor2}
3235 \nodepart{second}{+visit(ConcreteElementA)\\+visit(ConcreteElementB)}
3239 \draw[inheritarrow] (ConcreteElementA.north) -- ++(0,0.7) -|
3241 \draw[line] (ConcreteElementA.north) -- ++(0,0.7) -|
3242 (ConcreteElementB.north);
3244 \draw[inheritarrow] (ConcreteVisitor1.north) -- ++(0,0.7) -|
3246 \draw[line] (ConcreteVisitor1.north) -- ++(0,0.7) -|
3247 (ConcreteVisitor2.north);
3249 \draw[commentarrow] (CommentA.north) -- (ConcreteElementA.south);
3250 \draw[commentarrow] (CommentB.north) -- (ConcreteElementB.south);
3254 \caption{The Visitor Pattern.}
3255 \label{fig:visitorPattern}
3258 The use of the visitor pattern can be appropriate when the hierarchy of elements
3259 is mostly stable, but the family of operations over its elements is constantly
3260 growing. This is clearly the case for the \name{Eclipse} AST, since the hierarchy of
3261 type \type{ASTNode} is very stable, but the functionality of its elements is
3262 extended every time someone needs to operate on the AST. Another aspect of the
3263 \name{Eclipse} implementation is that it is a public API, and the visitor pattern is an
3264 easy way to provide access to the nodes in the tree.
3266 The version of the visitor pattern implemented for the AST nodes in \name{Eclipse} also
3267 provides an elegant way to traverse the tree. It does so by following the
3268 convention that every node in the tree first let the visitor visit itself,
3269 before it also makes all its children accept the visitor. The children are only
3270 visited if the visit method of their parent returns \var{true}. This pattern
3271 then makes for a prefix traversal of the AST. If postfix traversal is desired,
3272 the visitors also has \method{endVisit} methods for each node type, that is
3273 called after the \method{visit} method for a node. In addition to these visit
3274 methods, there are also the methods \method{preVisit(ASTNode)},
3275 \method{postVisit(ASTNode)} and \method{preVisit2(ASTNode)}. The
3276 \method{preVisit} method is called before the type-specific \method{visit}
3277 method. The \method{postVisit} method is called after the type-specific
3278 \method{endVisit}. The type specific \method{visit} is only called if
3279 \method{preVisit2} returns \var{true}. Overriding the \method{preVisit2} is also
3280 altering the behavior of \method{preVisit}, since the default implementation is
3281 responsible for calling it.
3283 An example of a trivial \type{ASTVisitor} is shown in
3284 \myref{lst:astVisitorExample}.
3287 \begin{minted}{java}
3288 public class CollectNamesVisitor extends ASTVisitor {
3289 Collection<Name> names = new LinkedList<Name>();
3292 public boolean visit(QualifiedName node) {
3298 public boolean visit(SimpleName node) {
3304 \caption{An \type{ASTVisitor} that visits all the names in a subtree and adds
3305 them to a collection, except those names that are children of any
3306 \type{QualifiedName}.}
3307 \label{lst:astVisitorExample}
3310 \section{Property collectors}\label{propertyCollectors}
3311 The prefixes and unfixes are found by property
3312 collectors\typeref{no.uio.ifi.refaktor.extractors.collectors.PropertyCollector}.
3313 A property collector is of the \type{ASTVisitor} type, and thus visits nodes of
3314 type \type{ASTNode} of the abstract syntax tree \see{astVisitor}.
3316 \subsection{The PrefixesCollector}
3317 The \typewithref{no.uio.ifi.refaktor.extractors.collectors}{PrefixesCollector}
3318 finds prefixes that makes up the basis for calculating move targets for the
3319 \refa{Extract and Move Method} refactoring. It visits expression
3320 statements\typeref{org.eclipse.jdt.core.dom.ExpressionStatement} and creates
3321 prefixes from its expressions in the case of method invocations. The prefixes
3322 found is registered with a prefix set, together with all its sub-prefixes.
3324 \subsection{The UnfixesCollector}\label{unfixes}
3325 The \typewithref{no.uio.ifi.refaktor.extractors.collectors}{UnfixesCollector}
3326 finds unfixes within a selection.
3327 \todoin{Give more technical detail?}
3331 \subsection{The ContainsReturnStatementCollector}
3332 \todoin{Remove section?}
3334 \typewithref{no.uio.ifi.refaktor.analyze.collectors}{ContainsReturnStatementCollector}
3335 is a very simple property collector. It only visits the return statements within
3336 a selection, and can report whether it encountered a return statement or not.
3338 \subsection{The LastStatementCollector}
3339 The \typewithref{no.uio.ifi.refaktor.analyze.collectors}{LastStatementCollector}
3340 collects the last statement of a selection. It does so by only visiting the top
3341 level statements of the selection, and compares the textual end offset of each
3342 encountered statement with the end offset of the previous statement found.
3344 \section{Checkers}\label{checkers}
3345 The checkers are a range of classes that checks that text selections complies
3346 with certain criteria. All checkers operates under the assumption that the code
3347 they check is free from compilation errors. If a
3348 \typewithref{no.uio.ifi.refaktor.analyze.analyzers}{Checker} fails, it throws a
3349 \type{CheckerException}. The checkers are managed by the
3350 \type{LegalStatementsChecker}, which does not, in fact, implement the
3351 \type{Checker} interface. It does, however, run all the checkers registered with
3352 it, and reports that all statements are considered legal if no
3353 \type{CheckerException} is thrown. Many of the checkers either extends the
3354 \type{PropertyCollector} or utilizes one or more property collectors to verify
3355 some criteria. The checkers registered with the \type{LegalStatementsChecker}
3356 are described next. They are run in the order presented below.
3358 \subsection{The CallToProtectedOrPackagePrivateMethodChecker}
3359 This checker is used to check that at selection does not contain a call to a
3360 method that is protected or package-private. Such a method either has the access
3361 modifier \code{protected} or it has no access modifier.
3363 The workings of the \type{CallToProtectedOrPackagePrivateMethod\-Checker} is
3364 very simple. It looks for calls to methods that are either protected or
3365 package-private within the selection, and throws an
3366 \type{IllegalExpressionFoundException} if one is found.
3368 \subsection{The DoubleClassInstanceCreationChecker}
3369 The \type{DoubleClassInstanceCreationChecker} checks that there are no double
3370 class instance creations where the inner constructor call take and argument that
3371 is built up using field references.
3373 The checker visits all nodes of type \type{ClassInstanceCreation} within a
3374 selection. For all of these nodes, if its parent also is a class instance
3375 creation, it accepts a visitor that throws a
3376 \type{IllegalExpressionFoundException} if it encounters a name that is a field
3379 \subsection{The InstantiationOfNonStaticInnerClassChecker}
3380 The \type{InstantiationOfNonStaticInnerClassChecker} checks that selections
3381 does not contain instantiations of non-static inner classes. The
3382 \type{MoveInstanceMethodProcessor} in \name{Eclipse} does not handle such
3383 instantiations gracefully when moving a method. This problem is also related to
3384 bug\ldots \todoin{File Eclipse bug report}
3386 \subsection{The EnclosingInstanceReferenceChecker}
3387 The purpose of this checker is to verify that the names in a text selection are
3388 not referencing any enclosing instances. In theory, the underlying problem could
3389 be solved in some situations, but our dependency on the
3390 \type{MoveInstanceMethodProcessor} prevents this.
3393 \typewithref{no.uio.ifi.refaktor.analyze.analyzers}{EnclosingInstanceReferenceChecker}
3394 is a modified version of the
3395 \typewithref{org.eclipse.jdt.internal.corext.refactoring.structure.MoveInstanceMethod\-Processor}{EnclosingInstanceReferenceFinder}
3396 from the \type{MoveInstanceMethodProcessor}. Wherever the
3397 \type{EnclosingInstanceReferenceFinder} would create a fatal error status, the
3398 checker will throw a \type{CheckerException}.
3400 The checker works by first finding all of the enclosing types of a selection.
3401 Thereafter, it visits all the simple names of the selection to check that they
3402 are not references to variables or methods declared in any of the enclosing
3403 types. In addition, the checker visits \var{this}-expressions to verify that no
3404 such expressions are qualified with any name.
3406 \subsection{The ReturnStatementsChecker}\label{returnStatementsChecker}
3407 The checker for return statements is meant to verify that a text selection is
3408 consistent regarding return statements.
3410 If the selection is free from return statements, then the checker validates. So
3411 this is the first thing the checker investigates.
3413 If the checker proceeds any further, it is because the selection contains one
3414 or more return statements. The next test is therefore to check if the last
3415 statement of the selection ends in either a return or a throw statement. The
3416 responsibility for checking that the last statement of the selection eventually
3417 ends in a return or throw statement, is put on the
3418 \type{LastStatementOfSelectionEndsInReturnOrThrowChecker}. For every node
3419 visited, if the node is a statement, it does a test to see if the statement is a
3420 return, a throw or if it is an implicit return statement. If this is the case,
3421 no further checking is done. This checking is done in the \code{preVisit2}
3422 method \see{astVisitor}. If the node is not of a type that is being handled by
3423 its type-specific visit method, the checker performs a simple test. If the node
3424 being visited is not the last statement of its parent that is also enclosed by
3425 the selection, an \type{IllegalStatementFoundException} is thrown. This ensures
3426 that all statements are taken care of, one way or the other. It also ensures
3427 that the checker is conservative in the way it checks for legality of the
3430 To examine if a statement is an implicit return statement, the checker first
3431 finds the last statement declared in its enclosing method. If this statement is
3432 the same as the one under investigation, it is considered an implicit return
3433 statement. If the statements are not the same, the checker does a search to see
3434 if the statement examined is also the last statement of the method that can be
3435 reached. This includes the last statement of a block statement, a labeled
3436 statement, a synchronized statement or a try statement, that in turn is the last
3437 statement enclosed by one of the statement types listed. This search goes
3438 through all the parents of a statement until a statement is found that is not
3439 one of the mentioned acceptable parent statements. If the search ends in a
3440 method declaration, then the statement is considered to be the last reachable
3441 statement of the method, and thus it is an implicit return statement.
3443 There are two kinds of statements that are handled explicitly: If-statements and
3444 try-statements. Block, labeled and do-statements are handled by fall-through to
3447 If-statements are handled explicitly by overriding their type-specific visit
3448 method. If the then-part does not contain any return or throw statements an
3449 \type{IllegalStatementFoundException} is thrown. If it does contain a return or
3450 throw, its else-part is checked. If the else-part is non-existent, or it does
3451 not contain any return or throw statements an exception is thrown. If no
3452 exception is thrown while visiting the if-statement, its children are visited.
3454 A try-statement is checked very similar to an if-statement. Its body must
3455 contain a return or throw. The same applies to its catch clauses and finally
3456 body. Failure to validate produces an \type{IllegalStatementFoundException}.
3458 If the checker does not complain at any point, the selection is considered valid
3459 with respect to return statements.
3461 \subsection{The AmbiguousReturnValueChecker}
3462 This checker verifies that there are no ambiguous return values in a selection.
3464 First, the checker needs to collect some data. Those data are the binding keys
3465 for all simple names that are assigned to within the selection, including
3466 variable declarations, but excluding fields. The checker also collects whether
3467 there exists a return statement in the selection or not. No further checks of
3468 return statements are needed, since, at this point, the selection is already
3469 checked for illegal return statements \see{returnStatementsChecker}.
3471 After the binding keys of the assignees are collected, the checker searches the
3472 part of the enclosing method that is after the selection for references whose
3473 binding keys are among the collected keys. If more than one unique referral is
3474 found, or only one referral is found, but the selection also contains a return
3475 statement, we have a situation with an ambiguous return value, and an exception
3478 %\todoin{Explain why we do not need to consider variables assigned inside
3479 %local/anonymous classes. (The referenced variables need to be final and so
3482 \subsection{The IllegalStatementsChecker}
3483 This checker is designed to check for illegal statements.
3485 Notice that labels in break and continue statements needs some special
3486 treatment. Since a label does not have any binding information, we have to
3487 search upwards in the AST to find the \type{LabeledStatement} that corresponds
3488 to the label from the break or continue statement, and check that it is
3489 contained in the selection. If the break or continue statement does not have a
3490 label attached to it, it is checked that its innermost enclosing loop or switch
3491 statement (break statements only) also is contained in the selection.
3493 \todoin{Follow the development in the semantics section\ldots}
3495 \chapter{Case Studies}
3497 In this chapter I am going to present a few case studies. This is done to give
3498 an impression of how the search-based \ExtractAndMoveMethod refactoring
3499 performs when giving it a larger project to take on. I will try to answer where
3500 it lacks, in terms of completeness, as well as showing its effect on refactored
3503 The first and primary case, is refactoring source code from the \name{Eclipse
3504 JDT UI} project. The project is chosen because it is a real project, still in
3505 development, with a large code base that is written by many different people
3506 through several years. The code is installed in thousands of \name{Eclipse}
3507 applications worldwide, and must be seen as a good representative for
3508 professionally written Java source code. It is also the home for most of the JDT
3511 For the second case, the \ExtractAndMoveMethod refactoring is fed the
3512 \code{no.uio.ifi.refaktor} project. This is done as a variation of the
3513 ``dogfooding'' methodology, where you use your own tools to do your job, also
3514 referred to as ``eating your own dog
3515 food''\citing{harrisonDogfooding2006}.
3518 For conducting these experiments, three tools are used. Two of the ``tools''
3519 both uses Eclipse as their platform. The first is our own tool,
3520 written to be able to run the \ExtractAndMoveMethod refactoring as a batch
3521 process, analyzing and refactoring many methods after each other. The second is
3522 JUnit, that is used for running the projects own unit tests on the target code
3523 both before and after it is refactored. The last tool that is used is a code
3524 quality management tool, called \name{SonarQube}. It can be used to perform
3525 different tasks for assuring code quality, but we are only going to take
3526 advantage of one of its main features, namely Quality profiles.
3528 A quality profile is used to define a set of coding rules that a project is
3529 supposed to comply with. Failure to following these rules will be recorded as
3530 so-called ``issues'', marked as having one of several degrees of severities,
3531 ranging from ``info'' to ``blocker'', where the latter one is the most severe.
3532 The measurements done for these case studies are therefore not presented as
3533 fine-grained software metrics results, but rather as the number of issues for
3536 In addition to the coding rules defined through quality profiles, \name{SonarQube}
3537 calculates the complexity of source code. The metric that is used is cyclomatic
3538 complexity, developed by Thomas J. McCabe in
3539 1976\citing{mccabeComplexity1976}. In this metric, functions have an initial
3540 complexity of 1, and whenever the control flow of a function splits, the
3541 complexity increases by
3542 one\footnote{\url{http://docs.codehaus.org/display/SONAR/Metric+definitions}}.
3543 \name{SonarQube} discriminates between functions and accessors. Accessors
3544 are methods that are recognized as setters or getters. Accessors are not counted
3545 in the complexity analysis.
3547 \section{The \name{SonarQube} quality profile}
3548 The quality profile that is used with \name{SonarQube} in these case studies has got
3549 the name \name{IFI Refaktor Case Study} (version 6). The rules defined in the
3550 profile are chosen because they are the available rules found in \name{SonarQube} that
3551 measures complexity and coupling. Now follows a description of the rules in the
3552 quality profile. The values that are set for these rules are listed in
3553 \myref{tab:qualityProfile1}.
3556 \item[Avoid too complex class] is a rule that measures cyclomatic complexity
3557 for every statement in the body of a class, except for setters and getter.
3558 The threshold value set is its default value of 200.
3560 \item[Classes should not be coupled to too many other classes ] is a rule that
3561 measures how many other classes a class depends upon. It does not count the
3562 dependencies of nested classes. It is meant to promote the Single
3563 Responsibility Principle. Although not explicitly stated, the rule's metric
3564 resembles the \metr{Coupling between object classes} (CBO) metric that is
3565 described by Chidamber and Kemerer in their article \tit{A Metrics Suite for
3566 Object Oriented Design}\citing{metricsSuite1994}. The max value for the rule
3567 is chosen on the background of an empirical study by Raed Shatnawi, that
3568 concludes that the number 9 is the most useful threshold for the CBO
3569 metric\citing{shatnawiQuantitative2010}. This study is also performed on
3570 Eclipse source code, so this threshold value should be particularly well
3571 suited for the Eclipse JDT UI case in this chapter.
3573 \item[Control flow statements \ldots{} should not be nested too deeply] is
3574 a rule that is meant to counter ``Spaghetti code''. It measures the nesting
3575 level of if, for, while, switch and try statements. The nesting levels start
3576 at 1. The max value set is its default value of 3.
3578 \item[Methods should not be too complex] is a rule that measures cyclomatic
3579 complexity the same way as the ``Avoid too complex class'' rule. The max
3580 value used is 10, which ``seems like a reasonable, but not magical, upper
3581 limit``\citing{mccabeComplexity1976}.
3583 \item[Methods should not have too many lines] is a rule that simply measures
3584 the number of lines in methods. The threshold value of 20 is used for this
3585 metric. This is based on my own subjective opinions, as the default value of
3586 100 seems a bit too loose.
3588 \item[NPath Complexity] is a rule that measures the number of possible
3589 execution paths through a function. The value used is the default value of
3590 200, that seems like a recognized threshold for this metric.
3592 \item[Too many methods] is a rule that measures the number of methods in a
3593 class. The threshold value used is the default value of 10.
3599 \caption{The \name{IFI Refaktor Case Study} quality profile (version 6).}
3600 \label{tab:qualityProfile1}
3602 \begin{tabularx}{\textwidth}{@{}>{\bfseries}L{1.5}R{0.5}@{}}
3604 \textbf{Rule} & \textbf{Max value} \\
3606 Avoid too complex class & 200 \\
3607 Classes should not be coupled to too many other classes (Single
3608 Responsibility Principle) & 9 \\
3609 Control flow statements \ldots{} should not be nested too deeply &
3611 Methods should not be too complex & 10 \\
3612 Methods should not have too many lines & 20 \\
3613 NPath Complexity & 200 \\
3614 Too many methods & 10 \\
3621 A precondition for the source code that is going to be the target for a series
3622 of \ExtractAndMoveMethod refactorings, is that it is organized as an Eclipse
3623 project. It is also assumed that the code is free from compilation errors.
3625 \section{The experiment}
3626 For a given project, the first job that is done, is to refactor its source code.
3627 The refactoring batch job produces three things: The refactored project,
3628 statistics gathered during the execution of the series of refactorings, and an
3629 error log describing any errors happening during this execution. See
3630 \myref{sec:benchmarking} for more information about how the refactorings are
3633 After the refactoring process is done, the before- and after-code is analyzed
3634 with \name{SonarQube}. The analysis results are then stored in a database and
3635 displayed through a \name{SonarQube} server with a web interface.\todoin{How
3636 long are these results going to be publicly available?}
3638 The before- and after-code is also tested with their own unit tests. This is
3639 done to discover any changes in the semantic behavior of the refactored code,
3640 within the limits of these tests.
3642 \section{Case 1: The Eclipse JDT UI project}
3643 This case is the ultimate test for our \ExtractAndMoveMethod refactoring. The
3644 target source code is massive. With its over 300,000 lines of code and over
3645 25,000 methods, it is formidable task to perform automated changes on it. There
3646 should be plenty of situations where things can go wrong, and, as we shall see
3649 I will start by presenting some statistics from the refactoring execution,
3650 before I pick apart the \name{SonarQube} analysis and conclude by commenting on
3651 the results from the unit tests. The configuration for the experiment is
3652 specified in \myref{tab:configurationCase1}.
3655 \caption{Configuration for Case 1.}
3656 \label{tab:configurationCase1}
3658 \begin{tabularx}{\textwidth}{@{}>{\bfseries}L{0.67}L{1.33}@{}}
3660 \spancols{2}{Benchmark data} \\
3662 Launch configuration & CaseStudy.launch \\
3663 Project & no.uio.ifi.refaktor.benchmark \\
3664 Repository & gitolite@git.uio.no:ifi-stolz-refaktor \\
3665 Commit & 43c16c04520746edd75f8dc2a1935781d3d9de6c \\
3667 \spancols{2}{Input data} \\
3669 Project & org.eclipse.jdt.ui \\
3670 Repository & git://git.eclipse.org/gitroot/jdt/eclipse.jdt.ui.git \\
3671 Commit & f218388fea6d4ec1da7ce22432726c244888bb6b \\
3672 Branch & R3\_8\_maintenance \\
3673 Tests suites & org.eclipse.jdt.ui.tests.AutomatedSuite,
3674 org.eclipse.jdt.ui.tests.refactoring.all.\-AllAllRefactoringTests \\
3679 \subsection{Statistics}
3680 The statistics gathered during the refactoring execution is presented in
3681 \myref{tab:case1Statistics}.
3684 \caption{Statistics after batch refactoring the Eclipse JDT UI project with
3685 the \ExtractAndMoveMethod refactoring.}
3686 \label{tab:case1Statistics}
3688 \begin{tabularx}{\textwidth}{@{}>{\bfseries}L{1.5}R{0.5}@{}}
3690 \spancols{2}{Time used} \\
3692 Total time & 98m38s \\
3693 Analysis time & 14m41s (15\%) \\
3694 Change time & 74m20s (75\%) \\
3695 Miscellaneous tasks & 9m37s (10\%) \\
3697 \spancols{2}{Numbers of each type of entity analyzed} \\
3700 Compilation units & 2,097 \\
3703 Text selections & 591,500 \\
3705 \spancols{2}{Numbers for \ExtractAndMoveMethod refactoring candidates} \\
3707 Methods chosen as candidates & 2,552 \\
3708 Methods NOT chosen as candidates & 25,115 \\
3709 Candidate selections (multiple per method) & 36,843 \\
3711 \spancols{2}{\ExtractAndMoveMethod refactorings executed} \\
3713 Fully executed & 2,469 \\
3714 Not fully executed & 83 \\
3715 Total attempts & 2,552 \\
3717 \spancols{2}{Primitive refactorings executed} \\
3718 \spancols{2}{\small \ExtractMethod refactorings} \\
3720 Performed & 2,483 \\
3721 Not performed & 69 \\
3722 Total attempts & 2,552 \\
3724 \spancols{2}{\small \MoveMethod refactorings} \\
3727 Not performed & 14 \\
3728 Total attempts & 2,483 \\
3734 \subsubsection{Execution time}
3735 I consider the total execution time of approximately 1.5 hours as being
3736 acceptable. It clearly makes the batch process unsuitable for doing any
3737 on-demand analysis or changes, but it is good enough for running periodic jobs,
3738 like over-night analysis.
3740 As the statistics show, 75\% of the total time goes into making the actual code
3741 changes. The time consumers are here the primitive \ExtractMethod and
3742 \MoveMethod refactorings. Included in the change time is the parsing and
3743 precondition checking done by the refactorings, as well as textual changes done
3744 to files on disk. All this parsing and disk access is time-consuming, and
3745 constitute a large part of the change time.
3747 In comparison, the pure analysis time, used to find suitable candidates, only
3748 make up for 15\% of the total time consumed. This includes analyzing almost
3749 600,000 text selections, while the number of attempted executions of the
3750 \ExtractAndMoveMethod refactoring are only about 2,500. So the number of
3751 executed primitive refactorings are approximately 5,000. Assuming the time used
3752 on miscellaneous tasks are used mostly for parsing source code for the analysis,
3753 we can say that the time used for analyzing code is at most 25\% of the total
3754 time. This means that for every primitive refactoring executed, we can analyze
3755 around 360 text selections. So, with an average of about 21 text selections per
3756 method, it is reasonable to say that we can analyze over 15 methods in the time
3757 it takes to perform a primitive refactoring.
3759 \subsubsection{Refactoring candidates}
3760 Out of the 27,667 methods that was analyzed, 2,552 methods contained selections
3761 that was considered candidates for the \ExtractAndMoveMethod refactoring. This
3762 is roughly 9\% off the methods in the project. These 9\% of the methods had on
3763 average 14.4 text selections that was considered considered possible refactoring
3766 \subsubsection{Executed refactorings}
3767 2,469 out of 2,552 attempts on executing the \ExtractAndMoveMethod refactoring
3768 was successful, giving a success rate of 96.7\%. The failure rate of 3.3\% stem
3769 from situations where the analysis finds a candidate selection, but the change
3770 execution fails. This failure could be an exception that was thrown, and the
3771 refactoring aborts. It could also be the precondition checking for one of the
3772 primitive refactorings that gives us an error status, meaning that if the
3773 refactoring proceeds, the code will contain compilation errors afterwards,
3774 forcing the composite refactoring to abort. This means that if the
3775 \ExtractMethod refactoring fails, no attempt is done for the \MoveMethod
3776 refactoring. \todo{Redundant information? Put in benchmark chapter?}
3778 Out of the 2,552 \ExtractMethod refactorings that was attempted executed, 69 of
3779 them failed. This give a failure rate of 2.7\% for the primitive refactoring. In
3780 comparison, the \MoveMethod refactoring had a failure rate of 0.6 \% of the
3781 2,483 attempts on the refactoring.
3783 \subsection{\name{SonarQube} analysis}
3784 Results from the \name{SonarQube} analysis is shown in
3785 \myref{tab:case1ResultsProfile1}.
3788 \caption{Results for analyzing the Eclipse JDT UI project, before and after
3789 the refactoring, with \name{SonarQube} and the \name{IFI Refaktor Case Study}
3790 quality profile. (Bold numbers are better.)}
3791 \label{tab:case1ResultsProfile1}
3793 \begin{tabularx}{\textwidth}{@{}>{\bfseries}L{1.5}R{0.25}R{0.25}@{}}
3795 \textnormal{Number of issues for each rule} & Before & After \\
3797 Avoid too complex class & 81 & \textbf{79} \\
3798 Classes should not be coupled to too many other classes (Single
3799 Responsibility Principle) & \textbf{1,098} & 1,199 \\
3800 Control flow statements \ldots{} should not be nested too deeply & 1,375 &
3802 Methods should not be too complex & 1,518 & \textbf{1,452} \\
3803 Methods should not have too many lines & 3,396 & \textbf{3,291} \\
3804 NPath Complexity & 348 & \textbf{329} \\
3805 Too many methods & \textbf{454} & 520 \\
3807 Total number of issues & 8,270 & \textbf{8,155} \\
3810 \spancols{3}{Complexity} \\
3812 Per function & 3.6 & \textbf{3.3} \\
3813 Per class & \textbf{29.5} & 30.4 \\
3814 Per file & \textbf{44.0} & 45.3 \\
3816 Total complexity & \textbf{84,765} & 87,257 \\
3819 \spancols{3}{Numbers of each type of entity analyzed} \\
3821 Files & 1,926 & 1,926 \\
3822 Classes & 2,875 & 2,875 \\
3823 Functions & 23,744 & 26,332 \\
3824 Accessors & 1,296 & 1,019 \\
3825 Statements & 162,768 & 165,145 \\
3826 Lines of code & 320,941 & 329,112 \\
3828 Technical debt (in days) & \textbf{1,003.4} & 1,032.7 \\
3833 \subsubsection{Diversity in the number of entities analyzed}
3834 The analysis performed by \name{SonarQube} is reporting fewer methods than found
3835 by the pre-refactoring analysis. \name{SonarQube} discriminates between
3836 functions (methods) and accessors, so the 1,296 accessors play a part in this
3837 calculation. \name{SonarQube} also has the same definition as our plugin when
3838 it comes to how a class is defined. Therefore is seems like \name{SonarQube}
3839 misses 277 classes that our plugin handles. This can explain why the {SonarQube}
3840 report differs from our numbers by approximately 2,500 methods,
3842 \subsubsection{Complexity}
3843 On all complexity rules that works on the method level, the number of issues
3844 decreases with between 3.1\% and 6.5\% from before to after the refactoring. The
3845 average complexity of a method decreases from 3.6 to 3.3, which is an
3846 improvement of about 8.3\%. So, on the method level, the refactoring must be
3847 said to have a slightly positive impact.
3849 The improvement in complexity on the method level is somewhat traded for
3850 complexity on the class level. The complexity per class metric is worsen by 3\%
3851 from before to after. The issues for the ``Too many methods'' rule also
3852 increases by 14.5\%. These numbers indicate that the refactoring makes quite a
3853 lot of the classes a little more complex overall. This is the expected outcome,
3854 since the \ExtractAndMoveMethod refactoring introduces almost 2,500 new methods
3857 The only number that can save the refactoring's impact on complexity on the
3858 class level, is the ``Avoid too complex class'' rule. It improves with 2.5\%,
3859 thus indicating that the complexity is moderately better distributed between the
3860 classes after the refactoring than before.
3862 \subsubsection{Coupling}
3863 One of the hopes when starting this project, was to be able to make a
3864 refactoring that could lower the coupling between classes. Better complexity at
3865 the method level is a not very unexpected byproduct of dividing methods into
3866 smaller parts. Lowering the coupling on the other hand, is a far greater task.
3867 This is also reflected in the results for the only coupling rule defined in the
3868 \name{SonarQube} quality profile, namely the ``Classes should not be coupled to
3870 other classes (Single Responsibility Principle)'' rule.
3872 The number of issues for the coupling rule is 1,098 before the refactoring, and
3873 1,199 afterwards. This is an increase in issues of 9.2\%, and a blow for this
3874 project. These numbers can be interpreted two ways. The first possibility is
3875 that our assumptions are wrong, and that increasing indirection does not
3876 decrease coupling between classes. The other possibility is that our analysis
3877 and choices of candidate text selections are not good enough. I vote for the
3878 second possibility. (Voting against the public opinion may also be a little
3881 What probably happens is, that many of the times the \ExtractAndMoveMethod
3882 refactoring is performed, the \MoveMethod refactoring ``drags'' with it
3883 references to classes that are unknown to the method destination. If it happens
3884 to be so lucky that it removes a dependency from one class, it might as well
3885 introduce three new dependencies to another class. In those situations that a
3886 class does not know about the originating class of a moved method, the
3887 \MoveMethod refactoring most certainly will introduce a dependency. This is
3889 bug\footnote{\url{https://bugs.eclipse.org/bugs/show\_bug.cgi?id=228635}} in the
3890 refactoring, making it pass an instance of the originating class as a reference
3891 to the moved method, regardless of whether the reference is used in the method
3894 There is also the possibility that the heuristics used to find candidate text
3895 selections are not good enough, they most certainly are not. I wish I had more
3896 time to fine-tune them, and to complete the analysis part of the project, but
3897 this is simply not the case. This becomes even clearer when analyzing the unit
3898 test results for the after-code.
3900 \subsubsection{Totals}
3901 On the bright side, the total number of issues is lower after the refactoring
3902 than it was before. Before the refactoring, the total number of issues is
3903 8,270, and after it is 8,155. An improvement of only 1.4\%.
3905 Then \name{SonarQube} tells me that the total complexity has increased by
3906 2.9\%, and that the (more questionable) ``technical debt'' has increased from
3907 1,003.4 to 1,032.7 days, also a deterioration of 2.9\%. Although these numbers
3908 are similar, no correlation has been found between them.
3910 \subsection{Unit tests}
3911 The tests that have been run for the \name{Eclipse JDT UI} project, are the
3912 tests in the test suites specified as the main test suites on the JDT UI wiki
3913 page on how to contribute to the
3914 project\footnote{\url{https://wiki.eclipse.org/JDT\_UI/How\_to\_Contribute\#Unit\_Testing}}.
3916 \subsubsection{Before the refactoring}
3917 Running the tests for the before-code of Eclipse JDT UI yielded 4 errors and 3
3918 failures for the \type{AutomatedSuite} test suite (2,007 test cases), and 2
3920 3 failures for the \type{AllAllRefactoringTests} test suite (3,816 test cases).
3922 \subsubsection{After the refactoring}
3923 The test results for the after-code of the Eclipse JDT UI project is another
3924 story. The reason for this is that during the setup for the unit tests, Eclipse
3925 now reports that the project contains 322 fatal errors, and a lot of errors that
3926 probably follows from these. This is another blow for this master's project.
3928 It has now been shown that the \ExtractAndMoveMethod refactoring, in its current
3929 state, produces code that does not compile. Had these errors originated from
3930 only one bug, it would not have been much of a problem, but this is not the
3931 case. By only looking at some random compilation problems in the refactored code,
3932 I came up with at least four different bugs \todo{write bug reports?} that
3933 caused those problems. I then stopped looking for more, since some of the bugs
3934 would take more time to fix than I could justify using on them at this point.
3936 The only thing that can be said in my defence, is that all the compilation
3937 errors could have been avoided if the type of situations that causes them was
3938 properly handled by the primitive refactorings, that again are supported by the
3939 Eclipse JDT UI project. All of the four randomly found bugs that I mentioned
3940 before, are also weaknesses of the \MoveMethod refactoring. If the primitive
3941 refactorings had detected the up-coming errors
3942 in their precondition checking phase, the refactorings would have been aborted,
3943 since this is how the \ExtractAndMoveMethod refactoring handles such situations.
3945 Of course, taking all possible situations into account is an immense task. This
3946 is one of the reasons for the failure. A complete analysis is too big of a task
3947 for this master's project to handle. Looking at it now, this comes as no
3948 surprise, since the task is obviously also too big for the creators of the
3949 primitive \MoveMethod refactoring. This shows that the underlying primitive
3950 refactorings are not complete enough to be fully relied upon for avoiding
3953 Considering all these problems, it is difficult to know how to interpret the
3954 unit test results from after refactoring the Eclipse JDT UI. The
3955 \type{AutomatedSuite} reported 565 errors and 5 failures. Three of the failures
3956 were the same as reported before the refactoring took place, so two of them are
3957 new. For these two cases it is not immediately apparent what makes them behave
3958 differently. The program is so complex that to analyze it to find this out, we
3959 might need more powerful methods than just manually analyzing its source code.
3960 This is somewhat characteristic for imperative programming: The programs are
3961 often hard to analyze and understand.
3963 For the \type{AllAllRefactoringTests} test suite, the three failures are gone,
3964 but the two errors have grown to 2,257 errors. I will not try to analyze those
3967 What I can say, is that it is likely that the \ExtractAndMoveMethod refactoring
3968 has introduced some unintended behavioral changes. Let us say that the
3969 refactoring introduces at least two behavior-altering changes for every 2,500
3970 executions. More than that is difficult to say about the behavior-preserving
3971 properties of the \ExtractAndMoveMethod refactoring, at this point.
3973 \subsection{Conclusions}
3974 After automatically analyzing and executing the \ExtractAndMoveMethod
3975 refactoring for all the methods in the Eclipse JDT UI project, the results does
3976 not look that promising. For this case, the refactoring seems almost unusable as
3977 it is now. The error rate and measurements done tells us this.
3979 The refactoring makes the code a little less complex at the method level. But
3980 this is merely a side effect of extracting methods, and holds little scientific
3981 value. When it comes to the overall complexity, it is increased, although it is
3982 slightly better spread among the classes.
3984 The analysis done before the \ExtractAndMoveMethod refactoring, is currently not
3985 complete enough to make the refactoring useful. It introduces too many errors in
3986 the code, and the code may change it's behavior. It also remains to prove that
3987 large scale refactoring with it can decrease coupling between classes. A better
3988 analysis may prove this, but in its present state, the opposite is the fact. The
3989 coupling measurements done by \name{SonarQube} shows this.
3991 On the bright side, the performance of the refactoring process is not that bad.
3992 It shows that it is possible to make a tool the way we do, if we can make the
3993 tool do anything useful. As long as the analysis phase is not going to involve
3994 anything that uses to much disk access, a lot of analysis can be done in a
3995 reasonable amount of time.
3997 The time used on performing the actual changes excludes a trial and error
3998 approach with the tools used in this master's project. In a trial and error
3999 approach, you could for instance be using the primitive refactorings used in
4000 this project to refactor code, and only then make decisions based on the effect,
4001 possibly shown by traditional software metrics. The problem with the approach
4002 taken in this project, compared to a trial and error approach, is that using
4003 heuristics beforehand is much more complicated. But on the other hand, a trial
4004 and error approach would still need to face the challenges of producing code
4005 that does compile without errors. If using refactorings that could produce
4006 in-memory changes, a trial and error approach could be made more efficient.
4008 \section{Case 2: The \type{no.uio.ifi.refaktor} project}
4009 In this case we will see a form of the ``dogfooding'' methodology used, when
4010 refactoring our own \type{no.uio.ifi.refaktor} project with the
4011 \ExtractAndMoveMethod refactoring.
4013 In this case I will try to point out some differences from case 1, and how they
4014 impact the execution of the benchmark. The refaktor project is 39 times smaller
4015 than the Eclipse JDT UI project, measured in lines of code. This will make
4016 things a bit more transparent. It will therefore be interesting to see if this
4017 case can shed light on any aspect of our project that was lost in the larger
4020 The configuration for the experiment is specified in
4021 \myref{tab:configurationCase2}.
4024 \caption{Configuration for Case 2.}
4025 \label{tab:configurationCase2}
4027 \begin{tabularx}{\textwidth}{@{}>{\bfseries}L{0.67}L{1.33}@{}}
4029 \spancols{2}{Benchmark data} \\
4031 Launch configuration & CaseStudyDogfooding.launch \\
4032 Project & no.uio.ifi.refaktor.benchmark \\
4033 Repository & gitolite@git.uio.no:ifi-stolz-refaktor \\
4034 Commit & 43c16c04520746edd75f8dc2a1935781d3d9de6c \\
4036 \spancols{2}{Input data} \\
4038 Project & no.uio.ifi.refaktor \\
4039 Repository & gitolite@git.uio.no:ifi-stolz-refaktor \\
4040 Commit & 43c16c04520746edd75f8dc2a1935781d3d9de6c \\
4042 Test configuration & no.uio.ifi.refaktor.tests/ExtractTest.launch \\
4047 \subsection{Statistics}
4048 The statistics gathered during the refactoring execution is presented in
4049 \myref{tab:case2Statistics}.
4052 \caption{Statistics after batch refactoring the \type{no.uio.ifi.refaktor}
4053 project with the \ExtractAndMoveMethod refactoring.}
4054 \label{tab:case2Statistics}
4056 \begin{tabularx}{\textwidth}{@{}>{\bfseries}L{1.5}R{0.5}@{}}
4058 \spancols{2}{Time used} \\
4060 Total time & 1m15s \\
4061 Analysis time & 0m18s (24\%) \\
4062 Change time & 0m47s (63\%) \\
4063 Miscellaneous tasks & 0m10s (14\%) \\
4065 \spancols{2}{Numbers of each type of entity analyzed} \\
4068 Compilation units & 154 \\
4071 Text selections & 8,609 \\
4073 \spancols{2}{Numbers for \ExtractAndMoveMethod refactoring candidates} \\
4075 Methods chosen as candidates & 58 \\
4076 Methods NOT chosen as candidates & 1,012 \\
4077 Candidate selections (multiple per method) & 227 \\
4079 \spancols{2}{\ExtractAndMoveMethod refactorings executed} \\
4081 Fully executed & 53 \\
4082 Not fully executed & 5 \\
4083 Total attempts & 58 \\
4085 \spancols{2}{Primitive refactorings executed} \\
4086 \spancols{2}{\small \ExtractMethod refactorings} \\
4089 Not performed & 2 \\
4090 Total attempts & 58 \\
4092 \spancols{2}{\small \MoveMethod refactorings} \\
4095 Not performed & 3 \\
4096 Total attempts & 56 \\
4102 \subsubsection{Differences}
4103 There are some differences between the two projects that make them a little
4104 difficult to compare by performance.
4106 \paragraph{Different complexity.}
4107 Although the JDT UI project is 39 times greater than the refaktor project in
4108 terms of lines of code, it is only about 26 times its size measured in numbers
4109 of methods. This means that the methods in the refaktor project are smaller in
4110 average than in the JDT project. This is also reflected in the \name{SonarQube}
4111 report, where the complexity per method for the JDT project is 3.6, while the
4112 refaktor project has a complexity per method of 2.1.
4114 \paragraph{Number of selections per method.}
4115 The analysis for the JDT project processed 21 text selections per method in
4116 average. This number for the refaktor project is only 8 selections per method
4117 analyzed. This is a direct consequence of smaller methods.
4119 \paragraph{Different candidates to methods ratio.}
4120 The differences in how the projects are factored are also reflected in the
4121 ratios for how many methods that are chosen as candidates compared to the total
4122 number of methods analyzed. For the JDT project, 9\% of the methods was
4123 considered to be candidates, while for the refaktor project, only 5\% of the
4126 \paragraph{The average number of possible candidate selection.}
4127 For the methods that are chosen as candidates, the average number of possible
4128 candidate selections for these methods differ quite much. For the JDT project,
4129 the number of possible candidate selections for these methods were 14.44
4130 selections per method, while the candidate methods in the refaktor project had
4131 only 3.91 candidate selections to choose from, in average.
4133 \subsubsection{Execution time}
4134 The differences in complexity, and the different candidate methods to total
4135 number of methods ratios, is shown in the distributions of the execution times.
4136 For the JDT project, 75\% of the total time was used on the actual changes,
4137 while for the refaktor project, this number was only 63\%.
4139 For the JDT project, the benchmark used on average 0.21 seconds per method in
4140 the project, while for the refaktor project it used only 0.07 seconds per
4141 method. So the process used 3 times as much time per method for the JDT project
4142 than for the refaktor project.
4144 While the JDT project is 39 times larger than the refaktor project measured in
4145 lines of code, the benchmark used about 79 times as long time on it than for the
4146 refaktor project. Relatively, this is about twice as long.
4148 Since the details of these execution times are not that relevant to this
4149 master's project, only their magnitude, I will leave them here.
4151 \subsubsection{Executed refactorings}
4152 For the composite \ExtractAndMoveMethod refactoring performed in case 2, 53
4153 successful attempts out of 58 gives a success rate of 91.4\%. This is 5.3
4154 percentage points worse than for case 1.
4156 \subsection{\name{SonarQube} analysis}
4157 Results from the \name{SonarQube} analysis is shown in
4158 \myref{tab:case2ResultsProfile1}.
4160 Not much is to be said about these results. The trends in complexity and
4161 coupling are the same. We end up a little worse after the refactoring process
4165 \caption{Results for analyzing the \var{no.uio.ifi.refaktor} project, before
4166 and after the refactoring, with \name{SonarQube} and the \name{IFI Refaktor
4167 Case Study} quality profile. (Bold numbers are better.)}
4168 \label{tab:case2ResultsProfile1}
4170 \begin{tabularx}{\textwidth}{@{}>{\bfseries}L{1.5}R{0.25}R{0.25}@{}}
4172 \textnormal{Number of issues for each rule} & Before & After \\
4174 Avoid too complex class & 1 & 1 \\
4175 Classes should not be coupled to too many other classes (Single
4176 Responsibility Principle) & \textbf{29} & 34 \\
4177 Control flow statements \ldots{} should not be nested too deeply & 24 &
4179 Methods should not be too complex & 17 & \textbf{15} \\
4180 Methods should not have too many lines & 41 & \textbf{40} \\
4181 NPath Complexity & 3 & 3 \\
4182 Too many methods & \textbf{13} & 15 \\
4184 Total number of issues & \textbf{128} & 129 \\
4187 \spancols{3}{Complexity} \\
4189 Per function & 2.1 & 2.1 \\
4190 Per class & \textbf{12.5} & 12.9 \\
4191 Per file & \textbf{13.8} & 14.2 \\
4193 Total complexity & \textbf{2,089} & 2,148 \\
4196 \spancols{3}{Numbers of each type of entity analyzed} \\
4198 Files & 151 & 151 \\
4199 Classes & 167 & 167 \\
4200 Functions & 987 & 1,045 \\
4201 Accessors & 35 & 30 \\
4202 Statements & 3,355 & 3,416 \\
4203 Lines of code & 8,238 & 8,460 \\
4205 Technical debt (in days) & \textbf{19.0} & 20.7 \\
4210 \subsection{Unit tests}
4211 The tests used for this case are the same that has been developed throughout the
4214 The code that was refactored for this case suffered from some of the problems
4215 discovered in case 1. This means that the after-code for case 2 also contained
4216 compilation errors, but they were not as many. The code contained only 6 errors
4217 that made the code not compile.
4219 All of the errors made, originated from the same bug. It is a bug that happens
4220 in situation where a class instance creation is moved from between packages, and
4221 the class for the instance is package-private. The \MoveMethod refactoring does
4222 not detect that there will be a visibility problem, and neither does it promote
4223 the package-private class to be public.
4225 Since the errors was easy to fix manually, I corrected them and ran the unit
4226 tests as planned. Before the refactoring, all tests passed. All tests also
4227 passed after the refactoring, with the six error corrections. Since the
4228 corrections done is not of a kind that could make the behavior of the program
4229 change, it is likely that the refactorings done to the
4230 \type{no.uio.ifi.refaktor} project did not change its behavior. This is also
4231 supported by the informal experiment presented next.
4233 \subsection{An informal experiment}
4234 To complete the task of ``eating my own dog food'', I conducted an informal
4235 experiment where I used the refactored version of the \type{no.uio.ifi.refaktor}
4236 project, with the corrections, to again refaktor ``itself''.
4238 The experiment produced code containing the same six errors as after the
4239 previous experiment. I also compared the after-code from the two experiments
4240 with a diff-tool. The only differences found was different method names. This is
4241 expected, since the method names are randomly generated by the
4242 \ExtractAndMoveMethod refactoring.
4244 The outcome of this simple experiment makes me more confident that the
4245 \ExtractAndMoveMethod refactoring made only behavior-preserving changes to the
4246 \type{no.uio.ifi.refaktor} project, apart from the compilation errors.
4248 \subsection{Conclusions}
4249 The differences in complexity between the Eclipse JDT UI project and the
4250 \type{no.uio.ifi.refaktor} project, clearly influenced the differences in their
4251 execution times. This is mostly because fewer of the methods were chosen to be
4252 refactored for the refaktor project than for the JDT project. What this makes
4253 difficult, is to know if there are any severe performance penalties associated
4254 with refactoring on a large project compared to a small one.
4256 The trends in the \name{SonarQube} analysis are the same for this case as for
4257 the previous one. This gives more confidence in the these results.
4259 By refactoring our own code and using it again to refactor our code, we showed
4260 that it is possible to write an automated composite refactoring that works for
4261 many cases. That it probably did not alter the behavior of a smaller project
4262 shows us nothing more than that though, and might just be a coincidence.
4265 \todoin{Write? Or wrap up in final conclusions?}
4267 \chapter{Benchmarking}\label{sec:benchmarking}
4268 This part of the master's project is located in the \name{Eclipse} project
4269 \code{no.uio.ifi.refaktor.benchmark}. The purpose of it is to run the equivalent
4270 of the \type{SearchBasedExtractAndMoveMethodChanger}
4271 \see{searchBasedExtractAndMoveMethodChanger} over a larger software project,
4272 both to test its robustness but also its effect on different software metrics.
4274 \section{The benchmark setup}
4275 The benchmark itself is set up as a \name{JUnit} test case. This is a convenient
4276 setup, and utilizes the \name{JUnit Plugin Test Launcher}. This provides us with
4277 a fully functional \name{Eclipse} workbench. Most importantly, this gives us
4278 access to the Java Model of \name{Eclipse} \see{javaModel}.
4280 \subsection{The ProjectImporter}
4281 The Java project that is going to be used as the data for the benchmark, must be
4282 imported into the JUnit workspace. This is done by the
4283 \typewithref{no.uio.ifi.refaktor.benchmark}{ProjectImporter}. The importer
4284 require the absolute path to the project description file. It is named
4285 \code{.project} and is located at the root of the project directory.
4287 The project description is loaded to find the name of the project to be
4288 imported. The project that shall be the destination for the import is created in
4289 the workspace, on the base of the name from the description. Then an import
4290 operation is created, based on both the source and destination information. The
4291 import operation is run to perform the import.
4293 I have found no simple API call to accomplish what the importer does, which
4294 tells me that it may not be too many people performing this particular action.
4295 The solution to the problem was found on \name{Stack
4296 Overflow}\footnote{\url{https://stackoverflow.com/questions/12401297}}. It
4297 contains enough dirty details to be considered inconvenient to use, if not
4298 wrapping it in a class like my \type{ProjectImporter}. One would probably have
4299 to delve into the source code for the import wizard to find out how the import
4300 operation works, if no one had already done it.
4302 \section{Statistics}
4303 Statistics for the analysis and changes is captured by the
4304 \typewithref{no.uio.ifi.refaktor.aspects}{StatisticsAspect}. This an
4305 \emph{aspect} written in \name{AspectJ}.
4307 \subsection{AspectJ}
4308 \name{AspectJ}\footnote{\url{http://eclipse.org/aspectj/}} is an extension to
4309 the Java language, and facilitates combining aspect-oriented programming with
4310 the object-oriented programming in Java.
4312 Aspect-oriented programming is a programming paradigm that is meant to isolate
4313 so-called \emph{cross-cutting concerns} into their own modules. These
4314 cross-cutting concerns are functionalities that spans over multiple classes, but
4315 may not belong naturally in any of them. It can be functionality that does not
4316 concern the business logic of an application, and thus may be a burden when
4317 entangled with parts of the source code it does not really belong. Examples
4318 include logging, debugging, optimization and security.
4320 Aspects are interacting with other modules by defining advices. The concept of
4321 an \emph{advice} is known from both aspect-oriented and functional
4322 programming\citing{wikiAdvice2014}. It is a function that modifies another
4323 function when the latter is run. An advice in AspectJ is somewhat similar to a
4324 method in Java. It is meant to alter the behavior of other methods, and contains
4325 a body that is executed when it is applied.
4327 An advice can be applied at a defined \emph{pointcut}. A pointcut picks out one
4328 or more \emph{join points}. A join point is a well-defined point in the
4329 execution of a program. It can occur when calling a method defined for a
4330 particular class, when calling all methods with the same name,
4331 accessing/assigning to a particular field of a given class and so on. An advice
4332 can be declared to run both before, after returning from a pointcut, when there
4333 is thrown an exception in the pointcut or after the pointcut either returns or
4334 throws an exception. In addition to picking out join points, a pointcut can
4335 also bind variables from its context, so they can be accessed in the body of an
4336 advice. An example of a pointcut and an advice is found in
4337 \myref{lst:aspectjExample}.
4340 \begin{minted}{aspectj}
4341 pointcut methodAnalyze(
4342 SearchBasedExtractAndMoveMethodAnalyzer analyzer) :
4343 call(* SearchBasedExtractAndMoveMethodAnalyzer.analyze())
4344 && target(analyzer);
4346 after(SearchBasedExtractAndMoveMethodAnalyzer analyzer) :
4347 methodAnalyze(analyzer) {
4348 statistics.methodCount++;
4349 debugPrintMethodAnalysisProgress(analyzer.method);
4352 \caption{An example of a pointcut named \method{methodAnalyze},
4353 and an advice defined to be applied after it has occurred.}
4354 \label{lst:aspectjExample}
4357 \subsection{The Statistics class}
4358 The statistics aspect stores statistical information in an object of type
4359 \type{Statistics}. As of now, the aspect needs to be initialized at the point in
4360 time where it is desired that it starts its data gathering. At any point in time
4361 the statistics aspect can be queried for a snapshot of the current statistics.
4363 The \type{Statistics} class also include functionality for generating a report
4364 of its gathered statistics. The report can be given either as a string or it can
4365 be written to a file.
4367 \subsection{Advices}
4368 The statistics aspect contains advices for gathering statistical data from
4369 different parts of the benchmarking process. It captures statistics from both
4370 the analysis part and the execution part of the composite \ExtractAndMoveMethod
4373 For the analysis part, there are advices to count the number of text selections
4374 analyzed and the number of methods, types, compilation units and packages
4375 analyzed. There are also advices that counts for how many of the methods there
4376 is found a selection that is a candidate for the refactoring, and for how many
4377 methods there is not.
4379 There exists advices for counting both the successful and unsuccessful
4380 executions of all the refactorings. Both for the \ExtractMethod and \MoveMethod
4381 refactorings in isolation, as well as for the combination of them.
4383 \section{Optimizations}
4384 When looking for optimizations to make for the benchmarking process, I used the
4385 \name{VisualVM}\footnote{\url{http://visualvm.java.net/}} \gloss{profiler} for
4386 the Java Virtual Machine to both profile the application and also to make memory
4389 \subsection{Caching}
4390 When \gloss{profiling} the benchmark process before making any optimizations, it
4391 early became apparent that the parsing of source code was a place to direct
4392 attention towards. This discovery was done when only \emph{analyzing} source
4393 code, before trying to do any \emph{manipulation} of it. Caching of the parsed
4394 ASTs seemed like the best way to save some time, as expected. With only a simple
4395 cache of the most recently used AST, the analysis time was speeded up by a
4396 factor of around 20. This number depends a little upon which type of system the
4399 The caching is managed by a cache manager, that now, by default, utilizes the
4400 not so well known feature of Java called a \emph{soft reference}. Soft
4401 references are best explained in the context of weak references. A \emph{weak
4402 reference} is a reference to an object instance that is only guaranteed to
4403 persist as long as there is a \emph{strong reference} or a soft reference
4404 referring the same object. If no such reference is found, its referred object is
4405 garbage collected. A strong reference is basically the same as a regular Java
4406 reference. A soft reference has the same guarantees as a week reference when it
4407 comes to its relation to strong references, but it is not necessarily garbage
4408 collected whenever there exists no strong references to it. A soft reference
4409 \emph{may} reside in memory as long as the JVM has enough free memory in the
4410 heap. A soft reference will therefore usually perform better than a weak
4411 reference when used for simple caching and similar tasks. The way to use a
4412 soft/weak reference is to as it for its referent. The return value then has to
4413 be tested to check that it is not \var{null}. For the basic usage of soft
4414 references, see \myref{lst:softReferenceExample}. For a more thorough
4415 explanation of weak references in general, see\citing{weakRef2006}.
4418 \begin{minted}{java}
4420 Object strongRef = new Object();
4423 SoftReference<Object> softRef =
4424 new SoftReference<Object>(new Object());
4426 // Using the soft reference
4427 Object obj = softRef.get();
4432 \caption{Showing the basic usage of soft references. Weak references is used the
4433 same way. {\footnotesize (The references are part of the \code{java.lang.ref}
4435 \label{lst:softReferenceExample}
4438 The cache based on soft references has no limit for how many ASTs it caches. It
4439 is generally not advisable to keep references to ASTs for prolonged periods of
4440 time, since they are expensive structures to hold on to. For regular plugin
4441 development, \name{Eclipse} recommends not creating more than one AST at a time to
4442 limit memory consumption. Since the benchmarking has nothing to do with user
4443 experience, and throughput is everything, these advices are intentionally
4444 ignored. This means that during the benchmarking process, the target \name{Eclipse}
4445 application may very well work close to its memory limit for the heap space for
4446 long periods during the benchmark.
4448 \subsection{Candidates stored as mementos}
4449 When performing large scale analysis of source code for finding candidates to
4450 the \ExtractAndMoveMethod refactoring, memory is an issue. One of the inputs to
4451 the refactoring is a variable binding. This variable binding indirectly retains
4452 a whole AST. Since ASTs are large structures, this quickly leads to an
4453 \type{OutOfMemoryError} if trying to analyze a large project without optimizing
4454 how we store the candidates data. This means that the JVM cannot allocate more
4455 memory for out benchmark, and it exists disgracefully.
4457 A possible solution could be to just allow the JVM to allocate even more memory,
4458 but this is not a dependable solution. The allocated memory could easily
4459 supersede the physical memory of a machine, and that would make the benchmark go
4462 Thus, the candidates data must be stored in another format. Therefore, we use
4463 the \gloss{mementoPattern} to store the variable binding information. This is
4464 done in a way that makes it possible to retrieve the variable binding at a later
4465 point. The data that is stored to achieve this, is the key to the original
4466 variable binding. In addition to the key, we know which method and text
4467 selection the variable is referenced in, so that we can find it by parsing the
4468 source code and search for it when it is needed.
4470 \section{Handling failures}
4474 \chapter{Technicalities}
4476 \section{Source code organization}
4477 All the parts of this master's project is under version control with
4478 \name{Git}\footnote{\url{http://git-scm.com/}}.
4480 The software written is organized as some \name{Eclipse} plugins. Writing a plugin is
4481 the natural way to utilize the API of \name{Eclipse}. This also makes it possible to
4482 provide a user interface to manually run operations on selections in program
4483 source code or whole projects/packages.
4485 When writing a plugin in \name{Eclipse}, one has access to resources such as the
4486 current workspace, the open editor and the current selection.
4488 The thesis work is contained in the following Eclipse projects:
4491 \item[no.uio.ifi.refaktor] \hfill \\ This is the main Eclipse plugin
4492 project, and contains all of the business logic for the plugin.
4494 \item[no.uio.ifi.refaktor.tests] \hfill \\
4495 This project contains the tests for the main plugin.
4497 \item[no.uio.ifi.refaktor.examples] \hfill \\
4498 Contains example code used in testing. It also contains code for managing
4499 this example code, such as creating an Eclipse project from it before a test
4502 \item[no.uio.ifi.refaktor.benchmark] \hfill \\
4503 This project contains code for running search based versions of the
4504 composite refactoring over selected Eclipse projects.
4506 \item[no.uio.ifi.refaktor.releng] \hfill \\
4507 Contains the rmap, queries and target definitions needed by by Buckminster
4508 on the Jenkins continuous integration server.
4512 \subsection{The no.uio.ifi.refaktor project}
4514 \subsubsection{no.uio.ifi.refaktor.analyze}
4515 This package, and its sub-packages, contains code that is used for analyzing
4516 Java source code. The most important sub-packages are presented below.
4519 \item[no.uio.ifi.refaktor.analyze.analyzers] \hfill \\
4520 This package contains source code analyzers. These are usually responsible
4521 for analyzing text selections or running specialized analyzers for different
4522 kinds of entities. Their structure are often hierarchical. This means that
4523 you have an analyzer for text selections, that in turn is utilized by an
4524 analyzer that analyzes all the selections of a method. Then there are
4525 analyzers for analyzing all the methods of a type, all the types of a
4526 compilation unit, all the compilation units of a package, and, at last, all
4527 of the packages in a project.
4529 \item[no.uio.ifi.refaktor.analyze.checkers] \hfill \\
4530 A package containing checkers. The checkers are classes used to validate
4531 that a selection can be further analyzed and chosen as a candidate for a
4532 refactoring. Invalidating properties can be such as usage of inner classes
4533 or the need for multiple return values.
4535 \item[no.uio.ifi.refaktor.analyze.collectors] \hfill \\
4536 This package contains the property collectors. Collectors are used to gather
4537 properties from a text selection. This is mostly properties regarding
4538 referenced names and their occurrences. It is these properties that makes up
4539 the basis for finding the best candidates for a refactoring.
4542 \subsubsection{no.uio.ifi.refaktor.change}
4543 This package, and its sub-packages, contains functionality for manipulate source
4547 \item[no.uio.ifi.refaktor.change.changers] \hfill \\
4548 This package contains source code changers. They are used to glue together
4549 the analysis of source code and the actual execution of the changes.
4551 \item[no.uio.ifi.refaktor.change.executors] \hfill \\
4552 The executors that are responsible for making concrete changes are found in
4553 this package. They are mostly used to create and execute one or more Eclipse
4556 \item[no.uio.ifi.refaktor.change.processors] \hfill \\
4557 Contains a refactoring processor for the \MoveMethod refactoring. The code
4558 is stolen and modified to fix a bug. The related bug is described in
4559 \myref{eclipse_bug_429416}.
4563 \subsubsection{no.uio.ifi.refaktor.handlers}
4564 This package contains handlers for the commands defined in the plugin manifest.
4566 \subsubsection{no.uio.ifi.refaktor.prefix}
4567 This package contains the \type{Prefix} type that is the data representation of
4568 the prefixes found by the \type{PrefixesCollector}. It also contains the prefix
4569 set for storing and working with prefixes.
4571 \subsubsection{no.uio.ifi.refaktor.statistics}
4572 The package contains statistics functionality. Its heart is the statistics
4573 aspect that is responsible for gathering statistics during the execution of the
4574 \ExtractAndMoveMethod refactoring.
4577 \item[no.uio.ifi.refaktor.statistics.reports] \hfill \\
4578 This package contains a simple framework for generating reports from the
4579 statistics data generated by the aspect. Currently, the only available
4580 report type is a simple text report.
4585 \subsubsection{no.uio.ifi.refaktor.textselection}
4586 This package contains the two custom text selections that are used extensively
4587 throughout the project. One of them is just a subclass of the other, to support
4588 the use of the memento pattern to optimize the memory usage during benchmarking.
4590 \subsubsection{no.uio.ifi.refaktor.debugging}
4591 The package contains a debug utility class. I addition to this, the package
4592 \code{no.uio.ifi.refaktor.utils.aspects} contains a couple of aspects used for
4595 \subsubsection{no.uio.ifi.refaktor.utils}
4596 Utility package that contains all the functionality that has to do with parsing
4597 of source code. It also has utility classes for looking up handles to methods
4598 and types et cetera.
4601 \item[no.uio.ifi.refaktor.utils.caching] \hfill \\
4602 This package contains the caching manager for compilation units, along with
4603 classes for different caching strategies.
4605 \item[no.uio.ifi.refaktor.utils.nullobjects] \hfill \\
4606 Contains classes for creating different null objects. Most of the classes is
4607 used to represent null objects of different handle types. These null objects
4608 are returned from various utility classes instead of returning a \var{null}
4609 value when other values are not available.
4613 \section{Continuous integration}
4614 The continuous integration server
4615 \name{Jenkins}\footnote{\url{http://jenkins-ci.org/}} has been set up for the
4616 project\footnote{A work mostly done by the supervisor.}. It is used as a way to
4617 run tests and perform code coverage analysis.
4619 To be able to build the \name{Eclipse} plugins and run tests for them with Jenkins, the
4620 component assembly project
4621 \name{Buckminster}\footnote{\url{http://www.eclipse.org/buckminster/}} is used,
4622 through its plugin for Jenkins. Buckminster provides for a way to specify the
4623 resources needed for building a project and where and how to find them.
4624 Buckminster also handles the setup of a target environment to run the tests in.
4625 All this is needed because the code to build depends on an \name{Eclipse}
4626 installation with various plugins.
4628 \subsection{Problems with AspectJ}
4629 The Buckminster build worked fine until introducing AspectJ into the project.
4630 When building projects using AspectJ, there are some additional steps that needs
4631 to be performed. First of all, the aspects themselves must be compiled. Then the
4632 aspects needs to be woven with the classes they affect. This demands a process
4633 that does multiple passes over the source code.
4635 When using AspectJ with \name{Eclipse}, the specialized compilation and the
4636 weaving can be handled by the \name{AspectJ Development
4637 Tools}\footnote{\url{https://www.eclipse.org/ajdt/}}. This works all fine, but
4638 it complicates things when trying to build a project depending on \name{Eclipse}
4639 plugins outside of \name{Eclipse}. There is supposed to be a way to specify a
4640 compiler adapter for javac, together with the file extensions for the file types
4641 it shall operate. The AspectJ compiler adapter is called
4642 \typewithref{org.aspectj.tools.ant.taskdefs}{Ajc11CompilerAdapter}, and it works
4643 with files that has the extensions \code{*.java} and \code{*.aj}. I tried to
4644 setup this in the build properties file for the project containing the aspects,
4645 but to no avail. The project containing the aspects does not seem to be built at
4646 all, and the projects that depends on it complains that they cannot find certain
4649 I then managed to write an \name{Ant}\footnote{\url{https://ant.apache.org/}}
4650 build file that utilizes the AspectJ compiler adapter, for the
4651 \code{no.uio.ifi.refaktor} plugin. The problem was then that it could no longer
4652 take advantage of the environment set up by Buckminster. The solution to this
4653 particular problem was of a ``hacky'' nature. It involves exporting the plugin
4654 dependencies for the project to an Ant build file, and copy the exported path
4655 into the existing build script. But then the Ant script needs to know where the
4656 local \name{Eclipse} installation is located. This is no problem when building
4657 on a local machine, but to utilize the setup done by Buckminster is a problem
4658 still unsolved. To get the classpath for the build setup correctly, and here
4659 comes the most ``hacky'' part of the solution, the Ant script has a target for
4660 copying the classpath elements into a directory relative to the project
4661 directory and checking it into Git. When no \code{ECLIPSE\_HOME} property is set
4662 while running Ant, the script uses the copied plugins instead of the ones
4663 provided by the \name{Eclipse} installation when building the project. This
4664 obviously creates some problems with maintaining the list of dependencies in the
4665 Ant file, as well as remembering to copy the plugins every time the list of
4666 dependencies change.
4668 The Ant script described above is run by Jenkins before the Buckminster setup
4669 and build. When setup like this, the Buckminster build succeeds for the projects
4670 not using AspectJ, and the tests are run as normal. This is all good, but it
4671 feels a little scary, since the reason for Buckminster not working with AspectJ
4674 The problems with building with AspectJ on the Jenkins server lasted for a
4675 while, before they were solved. This is reflected in the ``Test Result Trend''
4676 and ``Code Coverage Trend'' reported by Jenkins.
4680 \chapter{Conclusions and Future Work}
4683 \section{Future work}
4689 \chapter{Eclipse Bugs Found}
4690 \newcommand{\submittedBugReport}[1]{The submitted bug report can be found on
4693 \section{Eclipse bug 420726: Code is broken when moving a method that is
4694 assigning to the parameter that is also the move
4695 destination}\label{eclipse_bug_420726}
4697 was found when analyzing what kinds of names that was to be considered as
4698 \emph{unfixes} \see{unfixes}.
4700 \subsection{The bug}
4701 The bug emerges when trying to move a method from one class to another, and when
4702 the target for the move (must be a variable, local or field) is both a parameter
4703 variable and also is assigned to within the method body. \name{Eclipse} allows this to
4704 happen, although it is the sure path to a compilation error. This is because we
4705 would then have an assignment to a \var{this} expression, which is not allowed
4707 \submittedBugReport{https://bugs.eclipse.org/bugs/show\_bug.cgi?id=420726}
4709 \subsection{The solution}
4710 The solution to this problem is to add all simple names that are assigned to in
4711 a method body to the set of unfixes.
4713 \section{Eclipse bug 429416: IAE when moving method from anonymous
4714 class}\label{eclipse_bug_429416}
4716 this bug during a batch change on the \type{org.eclipse.jdt.ui} project.
4718 \subsection{The bug}
4719 This bug surfaces when trying to use the \refa{Move Method} refactoring to move a
4720 method from an anonymous class to another class. This happens both for my
4721 simulation as well as in \name{Eclipse}, through the user interface. It only occurs
4722 when \name{Eclipse} analyzes the program and finds it necessary to pass an instance of
4723 the originating class as a parameter to the moved method. I.e. it want to pass a
4724 \var{this} expression. The execution ends in an
4725 \typewithref{java.lang}{IllegalArgumentException} in
4726 \typewithref{org.eclipse.jdt.core.dom}{SimpleName} and its
4727 \method{setIdentifier(String)} method. The simple name is attempted created in
4729 \methodwithref{org.eclipse.jdt.internal.corext.refactoring.structure.\\MoveInstanceMethodProcessor}{createInlinedMethodInvocation}
4730 so the \type{MoveInstanceMethodProcessor} was early a clear suspect.
4732 The \method{createInlinedMethodInvocation} is the method that creates a method
4733 invocation where the previous invocation to the method that was moved was. From
4734 its code it can be read that when a \var{this} expression is going to be passed
4735 in to the invocation, it shall be qualified with the name of the original
4736 method's declaring class, if the declaring class is either an anonymous class or
4737 a member class. The problem with this, is that an anonymous class does not have
4738 a name, hence the term \emph{anonymous} class! Therefore, when its name, an
4739 empty string, is passed into
4740 \methodwithref{org.eclipse.jdt.core.dom.AST}{newSimpleName} it all ends in an
4741 \type{IllegalArgumentException}.
4742 \submittedBugReport{https://bugs.eclipse.org/bugs/show\_bug.cgi?id=429416}
4744 \subsection{How I solved the problem}
4745 Since the \type{MoveInstanceMethodProcessor} is instantiated in the
4746 \typewithref{no.uio.ifi.refaktor.change.executors}{MoveMethod\-RefactoringExecutor},
4747 and only need to be a
4748 \typewithref{org.eclipse.ltk.core.refactoring.participants}{MoveProcessor}, I
4749 was able to copy the code for the original move processor and modify it so that
4750 it works better for me. It is now called
4751 \typewithref{no.uio.ifi.refaktor.change.processors}{ModifiedMoveInstanceMethodProcessor}.
4752 The only modification done (in addition to some imports and suppression of
4753 warnings), is in the \method{createInlinedMethodInvocation}. When the declaring
4754 class of the method to move is anonymous, the \var{this} expression in the
4755 parameter list is not qualified with the declaring class' (empty) name.
4757 \section{Eclipse bug 429954: Extracting statement with reference to local type
4758 breaks code}\label{eclipse_bug_429954}
4760 was discovered when doing some changes to the way unfixes is computed.
4762 \subsection{The bug}
4763 The problem is that \name{Eclipse} is allowing selections that references variables of
4764 local types to be extracted. When this happens the code is broken, since the
4765 extracted method must take a parameter of a local type that is not in the
4766 methods scope. The problem is illustrated in
4767 \myref{lst:extractMethod_LocalClass}, but there in another setting.
4768 \submittedBugReport{https://bugs.eclipse.org/bugs/show\_bug.cgi?id=429954}
4770 \subsection{Actions taken}
4771 There are no actions directly springing out of this bug, since the Extract
4772 Method refactoring cannot be meant to be this way. This is handled on the
4773 analysis stage of our \refa{Extract and Move Method} refactoring. So names representing
4774 variables of local types is considered unfixes \see{unfixes}.
4775 \todoin{write more when fixing this in legal statements checker}