1 \documentclass[USenglish,11pt]{ifimaster}
3 \usepackage[utf8]{inputenc}
4 \usepackage[T1]{fontenc,url}
5 \usepackage{lmodern} % using Latin Modern to be able to use bold typewriter font
12 \usepackage{tikz-qtree}
13 \usetikzlibrary{shapes,snakes,trees,arrows,shadows,positioning,calc}
14 \usepackage{babel,textcomp,csquotes,ifimasterforside}
17 \usepackage[hidelinks]{hyperref}
19 \usepackage[xindy]{glossaries}
21 \usepackage[style=alphabetic,backend=biber]{biblatex}
23 \usepackage{mathtools}
25 % use 'disable' before printing:
26 \usepackage[]{todonotes}
33 \usepackage{perpage} %the perpage package
34 \MakePerPage{footnote} %the perpage package command
36 \theoremstyle{definition}
37 \newtheorem*{wordDef}{Definition}
38 \newtheorem*{theorem}{Theorem}
40 \graphicspath{ {./figures/} }
42 \newcommand{\citing}[1]{~\cite{#1}}
43 %\newcommand{\myref}[1]{\cref{#1} on \cpageref{#1}}
44 \newcommand{\myref}[1]{\vref{#1}}
45 \newcommand{\Myref}[1]{\Vref{#1}}
47 %\newcommand{\glossref}[1]{\textsuperscript{(\glsrefentry{#1})}}
48 %\newcommand{\gloss}[1]{\gls{#1}\glossref{#1}}
49 %\newcommand{\glosspl}[1]{\glspl{#1}\glossref{#1}}
50 \newcommand{\gloss}[1]{\gls{#1}}
51 \newcommand{\glosspl}[1]{\glspl{#1}}
53 \newcommand{\definition}[1]{\begin{wordDef}#1\end{wordDef}}
54 \newcommand{\see}[1]{(see \myref{#1})}
55 \newcommand{\explanation}[3]{\noindent\textbf{\textit{#1}}\\*\emph{When:}
56 #2\\*\emph{How:} #3\\*[-7px]}
58 %\newcommand{\type}[1]{\lstinline{#1}}
59 \newcommand{\code}[1]{\texttt{\textbf{#1}}}
60 \newcommand{\type}[1]{\code{#1}}
61 \newcommand{\typeref}[1]{\footnote{\type{#1}}}
62 \newcommand{\typewithref}[2]{\type{#2}\typeref{#1.#2}}
63 \newcommand{\method}[1]{\type{#1}}
64 \newcommand{\methodref}[2]{\footnote{\type{#1}\method{\##2()}}}
65 \newcommand{\methodwithref}[2]{\method{#2}\footnote{\type{#1}\method{\##2()}}}
66 \newcommand{\var}[1]{\type{#1}}
68 \newcommand{\name}[1]{#1}
69 \newcommand{\tit}[1]{\emph{#1}}
70 \newcommand{\refa}[1]{\emph{#1}}
71 \newcommand{\pattern}[1]{\emph{#1}}
72 \newcommand{\metr}[1]{\emph{#1}}
73 \newcommand{\ExtractMethod}{\refa{Extract Method}\xspace}
74 \newcommand{\MoveMethod}{\refa{Move Method}\xspace}
75 \newcommand{\ExtractAndMoveMethod}{\refa{Extract and Move Method}\xspace}
77 \newcommand{\m}[1]{$#1$}
79 \newcommand\todoin[2][]{\todo[inline, caption={#2}, #1]{
80 \begin{minipage}{\textwidth-4pt}#2\end{minipage}}}
82 \title{Automated Composition of Refactorings}
83 \subtitle{Composing the Extract and Move Method refactorings in Eclipse}
84 \author{Erlend Kristiansen}
87 \newglossaryentry{profiling}
90 description={is to run a computer program through a profiler/with a profiler
93 \newglossaryentry{profiler}
96 description={A profiler is a program for analyzing performance within an
97 application. It is used to analyze memory consumption, processing time and
98 frequency of procedure calls and such}
100 \newglossaryentry{xUnit}
102 name={xUnit framework},
103 description={An xUnit framework is a framework for writing unit tests for a
104 computer program. It follows the patterns known from the JUnit framework for
105 Java\citing{fowlerXunit}
107 plural={xUnit frameworks}
109 \newglossaryentry{softwareObfuscation}
111 name={software obfuscation},
112 description={makes source code harder to read and analyze, while preserving
115 \newglossaryentry{extractClass}
117 name=\refa{Extract Class},
118 description={The \refa{Extract Class} refactoring works by creating a class,
119 for then to move members from another class to that class and access them from
120 the old class via a reference to the new class}
122 \newglossaryentry{designPattern}
124 name={design pattern},
125 description={A design pattern is a named abstraction, that is meant to solve a
126 general design problem. It describes the key aspects of a common problem and
127 identifies its participators and how they collaborate},
128 plural={design patterns}
130 \newglossaryentry{enclosingClass}
132 name={enclosing class},
133 description={An enclosing class is the class that surrounds any specific piece
134 of code that is written in the inner scope of this class},
136 \newglossaryentry{mementoPattern}
138 name={memento pattern},
139 description={The memento pattern is a software design pattern that is used to
140 capture an object's internal state so that it can be restored to this state
141 later\citing{designPatterns}},
143 %\newglossaryentry{extractMethod}
145 % name=\refa{Extract Method},
146 % description={The \refa{Extract Method} refactoring is used to extract a
147 %fragment of code from its context and into a new method. A call to the new
148 %method is inlined where the fragment was before. It is used to break code into
149 %logical units, with names that explain their purpose}
151 %\newglossaryentry{moveMethod}
153 % name=\refa{Move Method},
154 % description={The \refa{Move Method} refactoring is used to move a method from
155 % one class to another. This is useful if the method is using more features of
156 % another class than of the class which it is currently defined. Then all calls
157 % to this method must be updated, or the method must be copied, with the old
158 %method delegating to the new method}
161 \bibliography{bibliography/master-thesis-erlenkr-bibliography}
162 \DefineBibliographyStrings{english}{%
163 bibliography = {References},
166 % UML comment in TikZ:
167 % ref: https://tex.stackexchange.com/questions/103688/folded-paper-shape-tikz
169 \pgfdeclareshape{umlcomment}{
170 \inheritsavedanchors[from=rectangle] % this is nearly a rectangle
171 \inheritanchorborder[from=rectangle]
172 \inheritanchor[from=rectangle]{center}
173 \inheritanchor[from=rectangle]{north}
174 \inheritanchor[from=rectangle]{south}
175 \inheritanchor[from=rectangle]{west}
176 \inheritanchor[from=rectangle]{east}
177 % ... and possibly more
178 \backgroundpath{% this is new
179 % store lower right in xa/ya and upper right in xb/yb
180 \southwest \pgf@xa=\pgf@x \pgf@ya=\pgf@y
181 \northeast \pgf@xb=\pgf@x \pgf@yb=\pgf@y
182 % compute corner of ‘‘flipped page’’
183 \pgf@xc=\pgf@xb \advance\pgf@xc by-10pt % this should be a parameter
184 \pgf@yc=\pgf@yb \advance\pgf@yc by-10pt
185 % construct main path
186 \pgfpathmoveto{\pgfpoint{\pgf@xa}{\pgf@ya}}
187 \pgfpathlineto{\pgfpoint{\pgf@xa}{\pgf@yb}}
188 \pgfpathlineto{\pgfpoint{\pgf@xc}{\pgf@yb}}
189 \pgfpathlineto{\pgfpoint{\pgf@xb}{\pgf@yc}}
190 \pgfpathlineto{\pgfpoint{\pgf@xb}{\pgf@ya}}
193 \pgfpathmoveto{\pgfpoint{\pgf@xc}{\pgf@yb}}
194 \pgfpathlineto{\pgfpoint{\pgf@xc}{\pgf@yc}}
195 \pgfpathlineto{\pgfpoint{\pgf@xb}{\pgf@yc}}
196 \pgfpathlineto{\pgfpoint{\pgf@xc}{\pgf@yc}}
201 \tikzstyle{comment}=[%
214 %\interfootnotelinepenalty=10000
216 % Space between table rows
217 \renewcommand{\arraystretch}{1.3}
219 \newcommand{\spancols}[2]{\multicolumn{#1}{@{}l@{}}{#2}}
221 \newcolumntype{L}[1]{>{\hsize=#1\hsize\raggedright\arraybackslash}X}%
222 \newcolumntype{R}[1]{>{\hsize=#1\hsize\raggedleft\arraybackslash}X}%
225 \pagenumbering{roman}
231 \todoin{\textbf{Remove all todos (including list) before delivery/printing!!!
232 Can be done by removing ``draft'' from documentclass.}}
233 \todoin{Write abstract}
241 The discussions in this report must be seen in the context of object oriented
242 programming languages, and Java in particular, since that is the language in
243 which most of the examples will be given. All though the techniques discussed
244 may be applicable to languages from other paradigms, they will not be the
245 subject of this report.
249 \chapter{What is Refactoring?}
251 This question is best answered by first defining the concept of a
252 \emph{refactoring}, what it is to \emph{refactor}, and then discuss what aspects
253 of programming make people want to refactor their code.
255 \section{Defining refactoring}
256 Martin Fowler, in his classic book on refactoring\citing{refactoring}, defines a
257 refactoring like this:
260 \emph{Refactoring} (noun): a change made to the internal
261 structure\footnote{The structure observable by the programmer.} of software to
262 make it easier to understand and cheaper to modify without changing its
263 observable behavior.~\cite[p.~53]{refactoring}
266 \noindent This definition assigns additional meaning to the word
267 \emph{refactoring}, beyond the composition of the prefix \emph{re-}, usually
268 meaning something like ``again'' or ``anew'', and the word \emph{factoring},
269 that can mean to isolate the \emph{factors} of something. Here a \emph{factor}
270 would be close to the mathematical definition of something that divides a
271 quantity, without leaving a remainder. Fowler is mixing the \emph{motivation}
272 behind refactoring into his definition. Instead it could be more refined, formed
273 to only consider the \emph{mechanical} and \emph{behavioral} aspects of
274 refactoring. That is to factor the program again, putting it together in a
275 different way than before, while preserving the behavior of the program. An
276 alternative definition could then be:
278 \definition{A \emph{refactoring} is a transformation
279 done to a program without altering its external behavior.}
281 From this we can conclude that a refactoring primarily changes how the
282 \emph{code} of a program is perceived by the \emph{programmer}, and not the
283 \emph{behavior} experienced by any user of the program. Although the logical
284 meaning is preserved, such changes could potentially alter the program's
285 behavior when it comes to performance gain or -penalties. So any logic depending
286 on the performance of a program could make the program behave differently after
289 In the extreme case one could argue that \gloss{softwareObfuscation} is
290 refactoring. It is often used to protect proprietary software. It restrains
291 uninvited viewers, so they have a hard time analyzing code that they are not
292 supposed to know how works. This could be a problem when using a language that
293 is possible to decompile, such as Java.
295 Obfuscation could be done composing many, more or less randomly chosen,
296 refactorings. Then the question arises whether it can be called a
297 \emph{composite refactoring} or not \see{compositeRefactorings}? The answer is
298 not obvious. First, there is no way to describe the mechanics of software
299 obfuscation, because there are infinitely many ways to do that. Second,
300 obfuscation can be thought of as \emph{one operation}: Either the code is
301 obfuscated, or it is not. Third, it makes no sense to call software obfuscation
302 \emph{a refactoring}, since it holds different meaning to different people.
304 This last point is important, since one of the motivations behind defining
305 different refactorings, is to establish a \emph{vocabulary} for software
306 professionals to use when reasoning about and discussing programs, similar to
307 the motivation behind \glosspl{designPattern}\citing{designPatterns}.
309 So for describing \emph{software obfuscation}, it might be more appropriate to
310 define what you do when performing it rather than precisely defining its
311 mechanics in terms of other refactorings.
314 \section{The etymology of 'refactoring'}
315 It is a little difficult to pinpoint the exact origin of the word
316 ``refactoring'', as it seems to have evolved as part of a colloquial
317 terminology, more than a scientific term. There is no authoritative source for a
318 formal definition of it.
320 According to Martin Fowler\citing{etymology-refactoring}, there may also be more
321 than one origin of the word. The most well-known source, when it comes to the
322 origin of \emph{refactoring}, is the
323 Smalltalk\footnote{\label{footNote}Programming language} community and their
324 infamous \name{Refactoring
325 Browser}\footnote{\url{http://st-www.cs.illinois.edu/users/brant/Refactory/RefactoringBrowser.html}}
326 described in the article \tit{A Refactoring Tool for
327 Smalltalk}\citing{refactoringBrowser1997}, published in 1997.
328 Allegedly\citing{etymology-refactoring}, the metaphor of factoring programs was
329 also present in the Forth\textsuperscript{\ref{footNote}} community, and the
330 word ``refactoring'' is mentioned in a book by Leo Brodie, called \tit{Thinking
331 Forth}\citing{brodie2004}, first published in 1984\footnote{\tit{Thinking Forth}
332 was first published in 1984 by the \name{Forth Interest Group}. Then it was
333 reprinted in 1994 with minor typographical corrections, before it was
334 transcribed into an electronic edition typeset in \LaTeX\ and published under a
335 Creative Commons licence in
336 2004. The edition cited here is the 2004 edition, but the content should
337 essentially be as in 1984.}. The exact word is only printed one
338 place~\cite[p.~232]{brodie2004}, but the term \emph{factoring} is prominent in
339 the book, that also contains a whole chapter dedicated to (re)factoring, and how
340 to keep the (Forth) code clean and maintainable.
343 \ldots good factoring technique is perhaps the most important skill for a
344 Forth programmer.~\cite[p.~172]{brodie2004}
347 \noindent Brodie also express what \emph{factoring} means to him:
350 Factoring means organizing code into useful fragments. To make a fragment
351 useful, you often must separate reusable parts from non-reusable parts. The
352 reusable parts become new definitions. The non-reusable parts become arguments
353 or parameters to the definitions.~\cite[p.~172]{brodie2004}
356 Fowler claims that the usage of the word \emph{refactoring} did not pass between
357 the \name{Forth} and \name{Smalltalk} communities, but that it emerged
358 independently in each of the communities.
360 \section{Motivation -- Why people refactor}
361 There are many reasons why people want to refactor their programs. They can for
362 instance do it to remove duplication, break up long methods or to introduce
363 design patterns into their software systems. The shared trait for all these are
364 that peoples' intentions are to make their programs \emph{better}, in some
365 sense. But what aspects of their programs are becoming improved?
367 As just mentioned, people often refactor to get rid of duplication. They are
368 moving identical or similar code into methods, and are pushing methods up or
369 down in their class hierarchies. They are making template methods for
370 overlapping algorithms/functionality, and so on. It is all about gathering what
371 belongs together and putting it all in one place. The resulting code is then
372 easier to maintain. When removing the implicit coupling\footnote{When
373 duplicating code, the duplicate pieces of code might not be coupled, apart
374 from representing the same functionality. So if this functionality is going to
375 change, it might need to change in more than one place, thus creating an
376 implicit coupling between multiple pieces of code.} between code snippets, the
377 location of a bug is limited to only one place, and new functionality need only
378 to be added to this one place, instead of a number of places people might not
381 A problem you often encounter when programming, is that a program contains a lot
382 of long and hard-to-grasp methods. It can then help to break the methods into
383 smaller ones, using the \ExtractMethod refactoring\citing{refactoring}. Then
384 you may discover something about a program that you were not aware of before;
385 revealing bugs you did not know about or could not find due to the complex
386 structure of your program. \todo{Proof?} Making the methods smaller and giving
387 good names to the new ones clarifies the algorithms and enhances the
388 \emph{understandability} of the program \see{magic_number_seven}. This makes
389 refactoring an excellent method for exploring unknown program code, or code that
390 you had forgotten that you wrote.
392 Most primitive refactorings are simple, and usually involves moving code
393 around\citing{kerievsky2005}. The motivation behind them may first be revealed
394 when they are combined into larger --- higher level --- refactorings, called
395 \emph{composite refactorings} \see{compositeRefactorings}. Often the goal of
396 such a series of refactorings is a design pattern. Thus the design can
397 \emph{evolve} throughout the lifetime of a program, as opposed to designing
398 up-front. It is all about being structured and taking small steps to improve a
401 Many software design pattern are aimed at lowering the coupling between
402 different classes and different layers of logic. One of the most famous is
403 perhaps the \pattern{Model-View-Controller}\citing{designPatterns} pattern. It
404 is aimed at lowering the coupling between the user interface, the business logic
405 and the data representation of a program. This also has the added benefit that
406 the business logic could much easier be the target of automated tests, thus
407 increasing the productivity in the software development process.
409 Another effect of refactoring is that with the increased separation of concerns
410 coming out of many refactorings, the \emph{performance} can be improved. When
411 profiling programs, the problematic parts are narrowed down to smaller parts of
412 the code, which are easier to tune, and optimization can be performed only where
413 needed and in a more effective way\citing{refactoring}.
415 Last, but not least, and this should probably be the best reason to refactor, is
416 to refactor to \emph{facilitate a program change}. If one has managed to keep
417 one's code clean and tidy, and the code is not bloated with design patterns that
418 are not ever going to be needed, then some refactoring might be needed to
419 introduce a design pattern that is appropriate for the change that is going to
422 Refactoring program code --- with a goal in mind --- can give the code itself
423 more value. That is in the form of robustness to bugs, understandability and
424 maintainability. Having robust code is an obvious advantage, but
425 understandability and maintainability are both very important aspects of
426 software development. By incorporating refactoring in the development process,
427 bugs are found faster, new functionality is added more easily and code is easier
428 to understand by the next person exposed to it, which might as well be the
429 person who wrote it. The consequence of this, is that refactoring can increase
430 the average productivity of the development process, and thus also add to the
431 monetary value of a business in the long run. The perspective on productivity
432 and money should also be able to open the eyes of the many nearsighted managers
433 that seldom see beyond the next milestone.
435 \section{The magical number seven}\label{magic_number_seven}
436 The article \tit{The magical number seven, plus or minus two: some limits on our
437 capacity for processing information}\citing{miller1956} by George A. Miller,
438 was published in the journal \name{Psychological Review} in 1956. It presents
439 evidence that support that the capacity of the number of objects a human being
440 can hold in its working memory is roughly seven, plus or minus two objects. This
441 number varies a bit depending on the nature and complexity of the objects, but
442 is according to Miller ``\ldots never changing so much as to be
445 Miller's article culminates in the section called \emph{Recoding}, a term he
446 borrows from communication theory. The central result in this section is that by
447 recoding information, the capacity of the amount of information that a human can
448 process at a time is increased. By \emph{recoding}, Miller means to group
449 objects together in chunks, and give each chunk a new name that it can be
453 \ldots recoding is an extremely powerful weapon for increasing the amount of
454 information that we can deal with.~\cite[p.~95]{miller1956}
457 By organizing objects into patterns of ever growing depth, one can memorize and
458 process a much larger amount of data than if it were to be represented as its
459 basic pieces. This grouping and renaming is analogous to how many refactorings
460 work, by grouping pieces of code and give them a new name. Examples are the
461 fundamental \ExtractMethod and \refa{Extract Class}
462 refactorings\citing{refactoring}.
464 An example from the article addresses the problem of memorizing a sequence of
465 binary digits. The example presented here is a slightly modified version of the
466 one presented in the original article\citing{miller1956}, but it preserves the
467 essence of it. Let us say we have the following sequence of
468 16 binary digits: ``1010001001110011''. Most of us will have a hard time
469 memorizing this sequence by only reading it once or twice. Imagine if we instead
470 translate it to this sequence: ``A273''. If you have a background from computer
471 science, it will be obvious that the latter sequence is the first sequence
472 recoded to be represented by digits in base 16. Most people should be able to
473 memorize this last sequence by only looking at it once.
475 Another result from the Miller article is that when the amount of information a
476 human must interpret increases, it is crucial that the translation from one code
477 to another must be almost automatic for the subject to be able to remember the
478 translation, before \heshe is presented with new information to recode. Thus
479 learning and understanding how to best organize certain kinds of data is
480 essential to efficiently handle that kind of data in the future. This is much
481 like when humans learn to read. First they must learn how to recognize letters.
482 Then they can learn distinct words, and later read sequences of words that form
483 whole sentences. Eventually, most of them will be able to read whole books and
484 briefly retell the important parts of its content. This suggest that the use of
485 design patterns is a good idea when reasoning about computer programs. With
486 extensive use of design patterns when creating complex program structures, one
487 does not always have to read whole classes of code to comprehend how they
488 function, it may be sufficient to only see the name of a class to almost fully
489 understand its responsibilities.
492 Our language is tremendously useful for repackaging material into a few chunks
493 rich in information.~\cite[p.~95]{miller1956}
496 Without further evidence, these results at least indicate that refactoring
497 source code into smaller units with higher cohesion and, when needed,
498 introducing appropriate design patterns, should aid in the cause of creating
499 computer programs that are easier to maintain and have code that is easier (and
502 \section{Notable contributions to the refactoring literature}
503 \todoin{Thinking Forth?}
506 \item[1992] William F. Opdyke submits his doctoral dissertation called
507 \tit{Refactoring Object-Oriented Frameworks}\citing{opdyke1992}. This work
508 defines a set of refactorings, that are behavior preserving given that their
509 preconditions are met. The dissertation is focused on the automation of
511 \item[1999] Martin Fowler et al.: \tit{Refactoring: Improving the Design of
512 Existing Code}\citing{refactoring}. This is maybe the most influential text
513 on refactoring. It bares similarities with Opdykes thesis\citing{opdyke1992}
514 in the way that it provides a catalog of refactorings. But Fowler's book is
515 more about the craft of refactoring, as he focuses on establishing a
516 vocabulary for refactoring, together with the mechanics of different
517 refactorings and when to perform them. His methodology is also founded on
518 the principles of test-driven development.
519 \item[2005] Joshua Kerievsky: \tit{Refactoring to
520 Patterns}\citing{kerievsky2005}. This book is heavily influenced by Fowler's
521 \tit{Refactoring}\citing{refactoring} and the ``Gang of Four'' \tit{Design
522 Patterns}\citing{designPatterns}. It is building on the refactoring
523 catalogue from Fowler's book, but is trying to bridge the gap between
524 \emph{refactoring} and \emph{design patterns} by providing a series of
525 higher-level composite refactorings, that makes code evolve toward or away
526 from certain design patterns. The book is trying to build up the reader's
527 intuition around \emph{why} one would want to use a particular design
528 pattern, and not just \emph{how}. The book is encouraging evolutionary
529 design \see{relationToDesignPatterns}.
532 \section{Tool support (for Java)}\label{toolSupport}
533 This section will briefly compare the refactoring support of the three IDEs
534 \name{Eclipse}\footnote{\url{http://www.eclipse.org/}}, \name{IntelliJ
535 IDEA}\footnote{The IDE under comparison is the \name{Community Edition},
536 \url{http://www.jetbrains.com/idea/}} and
537 \name{NetBeans}\footnote{\url{https://netbeans.org/}}. These are the most
538 popular Java IDEs\citing{javaReport2011}.
540 All three IDEs provide support for the most useful refactorings, like the
541 different extract, move and rename refactorings. In fact, Java-targeted IDEs are
542 known for their good refactoring support, so this did not appear as a big
545 The IDEs seem to have excellent support for the \ExtractMethod refactoring, so
546 at least they have all passed the first ``refactoring
547 rubicon''\citing{fowlerRubicon2001,secondRubicon2012}.
549 Regarding the \MoveMethod refactoring, the \name{Eclipse} and \name{IntelliJ}
550 IDEs do the job in very similar manners. In most situations they both do a
551 satisfying job by producing the expected outcome. But they do nothing to check
552 that the result does not break the semantics of the program \see{correctness}.
553 The \name{NetBeans} IDE implements this refactoring in a somewhat
554 unsophisticated way. For starters, the refactoring's default destination for the
555 move, is the same class as the method already resides in, although it refuses to
556 perform the refactoring if chosen. But the worst part is, that if moving the
557 method \method{f} of the class \type{C} to the class \type{X}, it will break the
558 code. The result is shown in \myref{lst:moveMethod_NetBeans}.
562 \begin{minted}[samepage]{java}
575 \begin{minted}[samepage]{java}
585 \caption{Moving method \method{f} from \type{C} to \type{X}.}
586 \label{lst:moveMethod_NetBeans}
589 \name{NetBeans} will try to create code that call the methods \method{m} and \method{n}
590 of \type{X} by accessing them through \var{c.x}, where \var{c} is a parameter of
591 type \type{C} that is added the method \method{f} when it is moved. (This is
592 seldom the desired outcome of this refactoring, but ironically, this ``feature''
593 keeps \name{NetBeans} from breaking the code in the example from \myref{correctness}.)
594 If \var{c.x} for some reason is inaccessible to \type{X}, as in this case, the
595 refactoring breaks the code, and it will not compile. \name{NetBeans} presents a
596 preview of the refactoring outcome, but the preview does not catch it if the IDE
597 is about break the program.
599 The IDEs under investigation seem to have fairly good support for primitive
600 refactorings, but what about more complex ones, such as
601 \gloss{extractClass}\citing{refactoring}? \name{IntelliJ} handles this in a
602 fairly good manner, although, in the case of private methods, it leaves unused
603 methods behind. These are methods that delegate to a field with the type of the
604 new class, but are not used anywhere. \name{Eclipse} has added its own quirk to
605 the \refa{Extract Class} refactoring, and only allows for \emph{fields} to be
606 moved to a new class, \emph{not methods}. This makes it effectively only
607 extracting a data structure, and calling it \refa{Extract Class} is a little
608 misleading. One would often be better off with textual extract and paste than
609 using the \refa{Extract Class} refactoring in \name{Eclipse}. When it comes to
610 \name{NetBeans}, it does not even show an attempt on providing this refactoring.
612 \section{The relation to design patterns}\label{relationToDesignPatterns}
614 Refactoring and design patterns have at least one thing in common, they are both
615 promoted by advocates of \emph{clean code}\citing{cleanCode} as fundamental
616 tools on the road to more maintainable and extendable source code.
619 Design patterns help you determine how to reorganize a design, and they can
620 reduce the amount of refactoring you need to do
621 later.~\cite[p.~353]{designPatterns}
624 Although sometimes associated with
625 over-engineering\citing{kerievsky2005,refactoring}, design patterns are in
626 general assumed to be good for maintainability of source code. That may be
627 because many of them are designed to support the \emph{open/closed principle} of
628 object-oriented programming. The principle was first formulated by Bertrand
629 Meyer, the creator of the Eiffel programming language, like this: ``Modules
630 should be both open and closed.''\citing{meyer1988} It has been popularized,
631 with this as a common version:
634 Software entities (classes, modules, functions, etc.) should be open for
635 extension, but closed for modification.\footnote{See
636 \url{http://c2.com/cgi/wiki?OpenClosedPrinciple} or
637 \url{https://en.wikipedia.org/wiki/Open/closed_principle}}
640 Maintainability is often thought of as the ability to be able to introduce new
641 functionality without having to change too much of the old code. When
642 refactoring, the motivation is often to facilitate adding new functionality. It
643 is about factoring the old code in a way that makes the new functionality being
644 able to benefit from the functionality already residing in a software system,
645 without having to copy old code into new. Then, next time someone shall add new
646 functionality, it is less likely that the old code has to change. Assuming that
647 a design pattern is the best way to get rid of duplication and assist in
648 implementing new functionality, it is reasonable to conclude that a design
649 pattern often is the target of a series of refactorings. Having a repertoire of
650 design patterns can also help in knowing when and how to refactor a program to
651 make it reflect certain desired characteristics.
654 There is a natural relation between patterns and refactorings. Patterns are
655 where you want to be; refactorings are ways to get there from somewhere
656 else.~\cite[p.~107]{refactoring}
659 This quote is wise in many contexts, but it is not always appropriate to say
660 ``Patterns are where you want to be\ldots''. \emph{Sometimes}, patterns are
661 where you want to be, but only because it will benefit your design. It is not
662 true that one should always try to incorporate as many design patterns as
663 possible into a program. It is not like they have intrinsic value. They only add
664 value to a system when they support its design. Otherwise, the use of design
665 patterns may only lead to a program that is more complex than necessary.
668 The overuse of patterns tends to result from being patterns happy. We are
669 \emph{patterns happy} when we become so enamored of patterns that we simply
670 must use them in our code.~\cite[p.~24]{kerievsky2005}
673 This can easily happen when relying largely on up-front design. Then it is
674 natural, in the very beginning, to try to build in all the flexibility that one
675 believes will be necessary throughout the lifetime of a software system.
676 According to Joshua Kerievsky ``That sounds reasonable --- if you happen to be
677 psychic.''~\cite[p.~1]{kerievsky2005} He is advocating what he believes is a
678 better approach: To let software continually evolve. To start with a simple
679 design that meets today's needs, and tackle future needs by refactoring to
680 satisfy them. He believes that this is a more economic approach than investing
681 time and money into a design that inevitably is going to change. By relying on
682 continuously refactoring a system, its design can be made simpler without
683 sacrificing flexibility. To be able to fully rely on this approach, it is of
684 utter importance to have a reliable suit of tests to lean on \see{testing}. This
685 makes the design process more natural and less characterized by difficult
686 decisions that has to be made before proceeding in the process, and that is
687 going to define a project for all of its unforeseeable future.
691 \section{Classification of refactorings}
692 % only interesting refactorings
693 % with 2 detailed examples? One for structured and one for intra-method?
694 % Is replacing Bubblesort with Quick Sort considered a refactoring?
696 \subsection{Structural refactorings}
698 \subsubsection{Primitive refactorings}
701 \explanation{Extract Method}{You have a code fragment that can be grouped
702 together.}{Turn the fragment into a method whose name explains the purpose of
705 \explanation{Inline Method}{A method's body is just as clear as its name.}{Put
706 the method's body into the body of its callers and remove the method.}
708 \explanation{Inline Temp}{You have a temp that is assigned to once with a simple
709 expression, and the temp is getting in the way of other refactorings.}{Replace
710 all references to that temp with the expression}
712 % Moving Features Between Objects
713 \explanation{Move Method}{A method is, or will be, using or used by more
714 features of another class than the class on which it is defined.}{Create a new
715 method with a similar body in the class it uses most. Either turn the old method
716 into a simple delegation, or remove it altogether.}
718 \explanation{Move Field}{A field is, or will be, used by another class more than
719 the class on which it is defined}{Create a new field in the target class, and
720 change all its users.}
723 \explanation{Replace Magic Number with Symbolic Constant}{You have a literal
724 number with a particular meaning.}{Create a constant, name it after the meaning,
725 and replace the number with it.}
727 \explanation{Encapsulate Field}{There is a public field.}{Make it private and
730 \explanation{Replace Type Code with Class}{A class has a numeric type code that
731 does not affect its behavior.}{Replace the number with a new class.}
733 \explanation{Replace Type Code with Subclasses}{You have an immutable type code
734 that affects the behavior of a class.}{Replace the type code with subclasses.}
736 \explanation{Replace Type Code with State/Strategy}{You have a type code that
737 affects the behavior of a class, but you cannot use subclassing.}{Replace the
738 type code with a state object.}
740 % Simplifying Conditional Expressions
741 \explanation{Consolidate Duplicate Conditional Fragments}{The same fragment of
742 code is in all branches of a conditional expression.}{Move it outside of the
745 \explanation{Remove Control Flag}{You have a variable that is acting as a
746 control flag fro a series of boolean expressions.}{Use a break or return
749 \explanation{Replace Nested Conditional with Guard Clauses}{A method has
750 conditional behavior that does not make clear the normal path of
751 execution.}{Use guard clauses for all special cases.}
753 \explanation{Introduce Null Object}{You have repeated checks for a null
754 value.}{Replace the null value with a null object.}
756 \explanation{Introduce Assertion}{A section of code assumes something about the
757 state of the program.}{Make the assumption explicit with an assertion.}
759 % Making Method Calls Simpler
760 \explanation{Rename Method}{The name of a method does not reveal its
761 purpose.}{Change the name of the method}
763 \explanation{Add Parameter}{A method needs more information from its
764 caller.}{Add a parameter for an object that can pass on this information.}
766 \explanation{Remove Parameter}{A parameter is no longer used by the method
769 %\explanation{Parameterize Method}{Several methods do similar things but with
770 %different values contained in the method.}{Create one method that uses a
771 %parameter for the different values.}
773 \explanation{Preserve Whole Object}{You are getting several values from an
774 object and passing these values as parameters in a method call.}{Send the whole
777 \explanation{Remove Setting Method}{A field should be set at creation time and
778 never altered.}{Remove any setting method for that field.}
780 \explanation{Hide Method}{A method is not used by any other class.}{Make the
783 \explanation{Replace Constructor with Factory Method}{You want to do more than
784 simple construction when you create an object}{Replace the constructor with a
787 % Dealing with Generalization
788 \explanation{Pull Up Field}{Two subclasses have the same field.}{Move the field
791 \explanation{Pull Up Method}{You have methods with identical results on
792 subclasses.}{Move them to the superclass.}
794 \explanation{Push Down Method}{Behavior on a superclass is relevant only for
795 some of its subclasses.}{Move it to those subclasses.}
797 \explanation{Push Down Field}{A field is used only by some subclasses.}{Move the
798 field to those subclasses}
800 \explanation{Extract Interface}{Several clients use the same subset of a class's
801 interface, or two classes have part of their interfaces in common.}{Extract the
802 subset into an interface.}
804 \explanation{Replace Inheritance with Delegation}{A subclass uses only part of a
805 superclasses interface or does not want to inherit data.}{Create a field for the
806 superclass, adjust methods to delegate to the superclass, and remove the
809 \explanation{Replace Delegation with Inheritance}{You're using delegation and
810 are often writing many simple delegations for the entire interface}{Make the
811 delegating class a subclass of the delegate.}
813 \subsubsection{Composite refactorings}
816 % \explanation{Replace Method with Method Object}{}{}
818 % Moving Features Between Objects
819 \explanation{Extract Class}{You have one class doing work that should be done by
820 two}{Create a new class and move the relevant fields and methods from the old
821 class into the new class.}
823 \explanation{Inline Class}{A class isn't doing very much.}{Move all its features
824 into another class and delete it.}
826 \explanation{Hide Delegate}{A client is calling a delegate class of an
827 object.}{Create Methods on the server to hide the delegate.}
829 \explanation{Remove Middle Man}{A class is doing to much simple delegation.}{Get
830 the client to call the delegate directly.}
833 \explanation{Replace Data Value with Object}{You have a data item that needs
834 additional data or behavior.}{Turn the data item into an object.}
836 \explanation{Change Value to Reference}{You have a class with many equal
837 instances that you want to replace with a single object.}{Turn the object into a
840 \explanation{Encapsulate Collection}{A method returns a collection}{Make it
841 return a read-only view and provide add/remove methods.}
843 % \explanation{Replace Array with Object}{}{}
845 \explanation{Replace Subclass with Fields}{You have subclasses that vary only in
846 methods that return constant data.}{Change the methods to superclass fields and
847 eliminate the subclasses.}
849 % Simplifying Conditional Expressions
850 \explanation{Decompose Conditional}{You have a complicated conditional
851 (if-then-else) statement.}{Extract methods from the condition, then part, an
854 \explanation{Consolidate Conditional Expression}{You have a sequence of
855 conditional tests with the same result.}{Combine them into a single conditional
856 expression and extract it.}
858 \explanation{Replace Conditional with Polymorphism}{You have a conditional that
859 chooses different behavior depending on the type of an object.}{Move each leg
860 of the conditional to an overriding method in a subclass. Make the original
863 % Making Method Calls Simpler
864 \explanation{Replace Parameter with Method}{An object invokes a method, then
865 passes the result as a parameter for a method. The receiver can also invoke this
866 method.}{Remove the parameter and let the receiver invoke the method.}
868 \explanation{Introduce Parameter Object}{You have a group of parameters that
869 naturally go together.}{Replace them with an object.}
871 % Dealing with Generalization
872 \explanation{Extract Subclass}{A class has features that are used only in some
873 instances.}{Create a subclass for that subset of features.}
875 \explanation{Extract Superclass}{You have two classes with similar
876 features.}{Create a superclass and move the common features to the
879 \explanation{Collapse Hierarchy}{A superclass and subclass are not very
880 different.}{Merge them together.}
882 \explanation{Form Template Method}{You have two methods in subclasses that
883 perform similar steps in the same order, yet the steps are different.}{Get the
884 steps into methods with the same signature, so that the original methods become
885 the same. Then you can pull them up.}
888 \subsection{Functional refactorings}
890 \explanation{Substitute Algorithm}{You want to replace an algorithm with one
891 that is clearer.}{Replace the body of the method with the new algorithm.}
895 \section{The impact on software quality}
897 \subsection{What is software quality?}
898 The term \emph{software quality} has many meanings. It all depends on the
899 context we put it in. If we look at it with the eyes of a software developer, it
900 usually means that the software is easily maintainable and testable, or in other
901 words, that it is \emph{well designed}. This often correlates with the
902 management scale, where \emph{keeping the schedule} and \emph{customer
903 satisfaction} is at the center. From the customers point of view, in addition to
904 good usability, \emph{performance} and \emph{lack of bugs} is always
905 appreciated, measurements that are also shared by the software developer. (In
906 addition, such things as good documentation could be measured, but this is out
907 of the scope of this document.)
909 \subsection{The impact on performance}
911 Refactoring certainly will make software go more slowly\footnote{With todays
912 compiler optimization techniques and performance tuning of e.g. the Java
913 virtual machine, the penalties of object creation and method calls are
914 debatable.}, but it also makes the software more amenable to performance
915 tuning.~\cite[p.~69]{refactoring}
918 \noindent There is a common belief that refactoring compromises performance, due
919 to increased degree of indirection and that polymorphism is slower than
922 In a survey, Demeyer\citing{demeyer2002} disproves this view in the case of
923 polymorphism. He did an experiment on, what he calls, ``Transform Self Type
924 Checks'' where you introduce a new polymorphic method and a new class hierarchy
925 to get rid of a class' type checking of a ``type attribute``. He uses this kind
926 of transformation to represent other ways of replacing conditionals with
927 polymorphism as well. The experiment is performed on the C++ programming
928 language and with three different compilers and platforms. Demeyer concludes
929 that, with compiler optimization turned on, polymorphism beats middle to large
930 sized if-statements and does as well as case-statements. (In accordance with
931 his hypothesis, due to similarities between the way C++ handles polymorphism and
935 The interesting thing about performance is that if you analyze most programs,
936 you find that they waste most of their time in a small fraction of the
937 code.~\cite[p.~70]{refactoring}
940 \noindent So, although an increased amount of method calls could potentially
941 slow down programs, one should avoid premature optimization and sacrificing good
942 design, leaving the performance tuning until after \gloss{profiling} the
943 software and having isolated the actual problem areas.
945 \section{Composite refactorings}\label{compositeRefactorings}
946 \todo{motivation, examples, manual vs automated?, what about refactoring in a
947 very large code base?}
948 Generally, when thinking about refactoring, at the mechanical level, there are
949 essentially two kinds of refactorings. There are the \emph{primitive}
950 refactorings, and the \emph{composite} refactorings.
952 \definition{A \emph{primitive refactoring} is a refactoring that cannot be
953 expressed in terms of other refactorings.}
955 \noindent Examples are the \refa{Pull Up Field} and \refa{Pull Up
956 Method} refactorings\citing{refactoring}, that move members up in their class
959 \definition{A \emph{composite refactoring} is a refactoring that can be
960 expressed in terms of two or more other refactorings.}
962 \noindent An example of a composite refactoring is the \refa{Extract
963 Superclass} refactoring\citing{refactoring}. In its simplest form, it is composed
964 of the previously described primitive refactorings, in addition to the
965 \refa{Pull Up Constructor Body} refactoring\citing{refactoring}. It works
966 by creating an abstract superclass that the target class(es) inherits from, then
967 by applying \refa{Pull Up Field}, \refa{Pull Up Method} and
968 \refa{Pull Up Constructor Body} on the members that are to be members of
969 the new superclass. If there are multiple classes in play, their interfaces may
970 need to be united with the help of some rename refactorings, before extracting
971 the superclass. For an overview of the \refa{Extract Superclass}
972 refactoring, see \myref{fig:extractSuperclass}.
976 \includegraphics[angle=270,width=\linewidth]{extractSuperclassItalic.pdf}
977 \caption{The Extract Superclass refactoring, with united interfaces.}
978 \label{fig:extractSuperclass}
981 \section{Manual vs. automated refactorings}
982 Refactoring is something every programmer does, even if \heshe does not known
983 the term \emph{refactoring}. Every refinement of source code that does not alter
984 the program's behavior is a refactoring. For small refactorings, such as
985 \ExtractMethod, executing it manually is a manageable task, but is still prone
986 to errors. Getting it right the first time is not easy, considering the method
987 signature and all the other aspects of the refactoring that has to be in place.
989 Consider the renaming of classes, methods and fields. For complex programs these
990 refactorings are almost impossible to get right. Attacking them with textual
991 search and replace, or even regular expressions, will fall short on these tasks.
992 Then it is crucial to have proper tool support that can perform them
993 automatically. Tools that can parse source code and thus have semantic knowledge
994 about which occurrences of which names belong to what construct in the program.
995 For even trying to perform one of these complex task manually, one would have to
996 be very confident on the existing test suite \see{testing}.
998 \section{Correctness of refactorings}\label{correctness}
999 For automated refactorings to be truly useful, they must show a high degree of
1000 behavior preservation. This last sentence might seem obvious, but there are
1001 examples of refactorings in existing tools that break programs. In an ideal
1002 world, every automated refactoring would be ``complete'', in the sense that it
1003 would never break a program. In an ideal world, every program would also be free
1004 from bugs. In modern IDEs the implemented automated refactorings are working for
1005 \emph{most} cases, that is enough for making them useful.
1007 I will now present an example of a \emph{corner case} where a program breaks
1008 when a refactoring is applied. The example shows an \ExtractMethod refactoring
1009 followed by a \MoveMethod refactoring that breaks a program in both the
1010 \name{Eclipse} and \name{IntelliJ} IDEs\footnote{The \name{NetBeans} IDE handles this
1011 particular situation without altering the program's behavior, mainly because
1012 its \refa{Move Method} refactoring implementation is a bit flawed in other ways
1013 \see{toolSupport}.}. The target and the destination for the composed
1014 refactoring is shown in \myref{lst:correctnessExtractAndMove}. Note that the
1015 method \method{m(C c)} of class \type{X} assigns to the field \var{x} of the
1016 argument \var{c} that has type \type{C}.
1019 \begin{multicols}{2}
1020 \begin{minted}[linenos]{java}
1021 // Refactoring target
1023 public X x = new X();
1035 \begin{minted}[]{java}
1036 // Method destination
1038 public void m(C c) {
1040 // If m is called from
1041 // c, then c.x no longer
1048 \caption{The target and the destination for the composition of the Extract
1049 Method and \refa{Move Method} refactorings.}
1050 \label{lst:correctnessExtractAndMove}
1054 The refactoring sequence works by extracting line 6 through 8 from the original
1055 class \type{C} into a method \method{f} with the statements from those lines as
1056 its method body (but with the comment left out, since it will no longer hold any
1057 meaning). The method is then moved to the class \type{X}. The result is shown
1058 in \myref{lst:correctnessExtractAndMoveResult}.
1060 Before the refactoring, the methods \method{m} and \method{n} of class \type{X}
1061 are called on different object instances (see line 6 and 8 of the original class
1062 \type{C} in \cref{lst:correctnessExtractAndMove}). After the refactoring, they
1063 are called on the same object, and the statement on line
1064 3 of class \type{X} (in \cref{lst:correctnessExtractAndMoveResult}) no longer
1065 has the desired effect in our example. The method \method{f} of class \type{C}
1066 is now calling the method \method{f} of class \type{X} (see line 5 of class
1067 \type{C} in \cref{lst:correctnessExtractAndMoveResult}), and the program now
1068 behaves different than before.
1071 \begin{multicols}{2}
1072 \begin{minted}[linenos]{java}
1074 public X x = new X();
1084 \begin{minted}[linenos]{java}
1086 public void m(C c) {
1092 public void f(C c) {
1099 \caption{The result of the composed refactoring.}
1100 \label{lst:correctnessExtractAndMoveResult}
1103 The bug introduced in the previous example is of such a nature\footnote{Caused
1104 by aliasing. See \url{https://en.wikipedia.org/wiki/Aliasing_(computing)}}
1105 that it is very difficult to spot if the refactored code is not covered by
1106 tests. It does not generate compilation errors, and will thus only result in
1107 a runtime error or corrupted data, which might be hard to detect.
1109 \section{Refactoring and the importance of testing}\label{testing}
1111 If you want to refactor, the essential precondition is having solid
1112 tests.\citing{refactoring}
1115 When refactoring, there are roughly three classes of errors that can be made.
1116 The first class of errors are the ones that make the code unable to compile.
1117 These \emph{compile-time} errors are of the nicer kind. They flash up at the
1118 moment they are made (at least when using an IDE), and are usually easy to fix.
1119 The second class are the \emph{runtime} errors. Although they take a bit longer
1120 to surface, they usually manifest after some time in an illegal argument
1121 exception, null pointer exception or similar during the program execution.
1122 These kind of errors are a bit harder to handle, but at least they will show,
1123 eventually. Then there are the \emph{behavior-changing} errors. These errors are
1124 of the worst kind. They do not show up during compilation and they do not turn
1125 on a blinking red light during runtime either. The program can seem to work
1126 perfectly fine with them in play, but the business logic can be damaged in ways
1127 that will only show up over time.
1129 For discovering runtime errors and behavior changes when refactoring, it is
1130 essential to have good test coverage. Testing in this context means writing
1131 automated tests. Manual testing may have its uses, but when refactoring, it is
1132 automated unit testing that dominate. For discovering behavior changes it is
1133 especially important to have tests that cover potential problems, since these
1134 kind of errors does not reveal themselves.
1136 Unit testing is not a way to \emph{prove} that a program is correct, but it is a
1137 way to make you confident that it \emph{probably} works as desired. In the
1138 context of test-driven development (commonly known as TDD), the tests are even a
1139 way to define how the program is \emph{supposed} to work. It is then, by
1140 definition, working if the tests are passing.
1142 If the test coverage for a code base is perfect, then it should, theoretically,
1143 be risk-free to perform refactorings on it. This is why automated tests and
1144 refactoring are such a great match.
1146 \subsection{Testing the code from correctness section}
1147 The worst thing that can happen when refactoring is to introduce changes to the
1148 behavior of a program, as in the example on \myref{correctness}. This example
1149 may be trivial, but the essence is clear. The only problem with the example is
1150 that it is not clear how to create automated tests for it, without changing it
1153 Unit tests, as they are known from the different \glosspl{xUnit} around, are
1154 only suitable to test the \emph{result} of isolated operations. They can not
1155 easily (if at all) observe the \emph{history} of a program.
1157 This problem is still open.
1161 Assuming a sequential (non-concurrent) program:
1163 \begin{minted}{java}
1164 tracematch (C c, X x) {
1166 call(* X.m(C)) && args(c) && cflow(within(C));
1168 call(* X.n()) && target(x) && cflow(within(C));
1170 set(C.x) && target(c) && !cflow(m);
1174 { assert x == c.x; }
1178 %\begin{minted}{java}
1179 %tracematch (X x1, X x2) {
1181 % call(* X.m(C)) && target(x1);
1183 % call(* X.n()) && target(x2);
1185 % set(C.x) && !cflow(m) && !cflow(n);
1189 % { assert x1 != x2; }
1195 \chapter{The Project}
1197 \section{Project description}
1198 The aim of this master's project will be to explore the relationship between the
1199 \ExtractMethod and the \MoveMethod refactorings. This will be done by composing
1200 the two into a composite refactoring. The refactoring will be called the
1201 \ExtractAndMoveMethod refactoring.
1203 The two primitive \ExtractMethod and \MoveMethod refactorings must already be
1204 implemented in a tool, so the \ExtractAndMoveMethod refactoring is going to be
1205 built on top of those.
1207 The composition of the \ExtractMethod and \MoveMethod refactorings springs
1208 naturally out of the need to move procedures closer to the data they manipulate.
1209 This composed refactoring is not well described in the literature, but it is
1210 implemented in at least one tool called
1211 \name{CodeRush}\footnote{\url{https://help.devexpress.com/\#CodeRush/CustomDocument3519}},
1212 that is an extension for \name{MS Visual
1213 Studio}\footnote{\url{http://www.visualstudio.com/}}. In CodeRush it is called
1214 \refa{Extract Method to
1215 Type}\footnote{\url{https://help.devexpress.com/\#CodeRush/CustomDocument6710}},
1216 but I choose to call it \ExtractAndMoveMethod, since I feel this better
1217 communicates which primitive refactorings it is composed of.
1219 The project will consist of implementing the \ExtractAndMoveMethod refactoring,
1220 as well as executing it over a larger code base, as a case study. To be able to
1221 execute the refactoring automatically, I have to make it analyze code to
1222 determine the best selections to extract into new methods.
1224 \section{The primitive refactorings}
1225 The refactorings presented here are the primitive refactorings used in this
1226 project. They are the abstract building blocks used by the \ExtractAndMoveMethod
1229 \subsection{The Extract Method refactoring}
1230 The \refa{Extract Method} refactoring is used to extract a fragment of code
1231 from its context and into a new method. A call to the new method is inlined
1232 where the fragment was before. It is used to break code into logical units, with
1233 names that explain their purpose.
1235 An example of an \ExtractMethod refactoring is shown in
1236 \myref{lst:extractMethodRefactoring}. It shows a method containing calls to the
1237 methods \method{foo} and \method{bar} of a type \type{X}. These statements are
1238 then extracted into the new method \method{fooBar}.
1241 \begin{multicols}{2}
1242 \begin{minted}[samepage]{java}
1254 \begin{minted}[samepage]{java}
1267 \caption{An example of an \ExtractMethod refactoring.}
1268 \label{lst:extractMethodRefactoring}
1271 \subsection{The Move Method refactoring}
1272 The \refa{Move Method} refactoring is used to move a method from one class to
1273 another. This can be appropriate if the method is using more features of another
1274 class than of the class which it is currently defined.
1276 \Myref{lst:moveMethodRefactoring} shows an example of this refactoring. Here a
1277 method \method{fooBar} is moved from the class \type{C} to the class \type{X}.
1280 \begin{multicols}{2}
1281 \begin{minted}[samepage]{java}
1301 \begin{minted}[samepage]{java}
1319 \caption{An example of a \MoveMethod refactoring.}
1320 \label{lst:moveMethodRefactoring}
1323 \section{The Extract and Move Method refactoring}
1324 The \ExtractAndMoveMethod refactoring is a composite refactoring composed of the
1325 primitive \ExtractMethod and \MoveMethod refactorings. The effect of this
1326 refactoring on source code is the same as when extracting a method and moving it
1327 to another class. Conseptually, this is done without an intermediate step. In
1328 practice, as we shall see later, an intermediate step may be necessary.
1330 An example of this composite refactoring is shown in
1331 \myref{lst:extractAndMoveMethodRefactoring}. The example joins the examples from
1332 \cref{lst:extractMethodRefactoring} and \cref{lst:moveMethodRefactoring}. This
1333 means that the selection consisting of the consecutive calls to the methods
1334 \method{foo} and \method{bar}, is extracted into a new method \method{fooBar}
1335 located in the class \type{X}.
1338 \begin{multicols}{2}
1339 \begin{minted}[samepage]{java}
1356 \begin{minted}[samepage]{java}
1374 \caption{An example of the \ExtractAndMoveMethod refactoring.}
1375 \label{lst:extractAndMoveMethodRefactoring}
1378 \section{Research questions}
1379 The main question that I seek an answer to in this thesis is:
1382 Is it possible to automate the analysis and execution of the
1383 \ExtractAndMoveMethod refactoring, and do so for all of the code of a larger
1387 \noindent The secondary questions will then be:
1389 \paragraph{Can we do this efficiently?} Can we automate the analysis and
1390 execution of the refactoring so it can be run in a reasonable amount of time?
1391 And what does \emph{reasonable} mean in this context?
1393 And, assuming the refactoring does in fact improve the quality of source code:
1395 \paragraph{How can the automation of the refactoring be helpful?} What is the
1396 usefullness of the refactoring in a software development setting? In what parts
1397 of the development process can the refactoring play a role?
1399 \section{Choosing the target language}
1400 Choosing which programming language the code that shall be manipulated shall be
1401 written in, is not a very difficult task. We choose to limit the possible
1402 languages to the object-oriented programming languages, since most of the
1403 terminology and literature regarding refactoring comes from the world of
1404 object-oriented programming. In addition, the language must have existing tool
1405 support for refactoring.
1407 The \name{Java} programming language\footnote{\url{https://www.java.com/}} is
1408 the dominating language when it comes to example code in the literature of
1409 refactoring, and is thus a natural choice. Java is perhaps, currently the most
1410 influential programming language in the world, with its \name{Java Virtual
1411 Machine} that runs on all of the most popular architectures and also supports
1412 dozens of other programming languages\footnote{They compile to java bytecode.},
1413 with \name{Scala}, \name{Clojure} and \name{Groovy} as the most prominent ones.
1414 Java is currently the language that every other programming language is compared
1415 against. It is also the primary programming language for the author of this
1418 \section{Choosing the tools}
1419 When choosing a tool for manipulating Java, there are certain criteria that
1420 have to be met. First of all, the tool should have some existing refactoring
1421 support that this thesis can build upon. Secondly it should provide some kind of
1422 framework for parsing and analyzing Java source code. Third, it should itself be
1423 open source. This is both because of the need to be able to browse the code for
1424 the existing refactorings that is contained in the tool, and also because open
1425 source projects hold value in them selves. Another important aspect to consider
1426 is that open source projects of a certain size, usually has large communities of
1427 people connected to them, that are committed to answering questions regarding the
1428 use and misuse of the products, that to a large degree is made by the community
1431 There is a certain class of tools that meet these criteria, namely the class of
1432 \emph{IDEs}\footnote{\emph{Integrated Development Environment}}. These are
1433 programs that is meant to support the whole production cycle of a computer
1434 program, and the most popular IDEs that support Java, generally have quite good
1435 refactoring support.
1437 The main contenders for this thesis is the \name{Eclipse IDE}, with the
1438 \name{Java development tools} (JDT), the \name{IntelliJ IDEA Community Edition}
1439 and the \name{NetBeans IDE} \see{toolSupport}. \name{Eclipse} and
1440 \name{NetBeans} are both free, open source and community driven, while the
1441 \name{IntelliJ IDEA} has an open sourced community edition that is free of
1442 charge, but also offer an \name{Ultimate Edition} with an extended set of
1443 features, at additional cost. All three IDEs supports adding plugins to extend
1444 their functionality and tools that can be used to parse and analyze Java source
1445 code. But one of the IDEs stand out as a favorite, and that is the \name{Eclipse
1446 IDE}. This is the most popular\citing{javaReport2011} among them and seems to be
1447 de facto standard IDE for Java development regardless of platform.
1451 \todoin{Rename chapter?}
1453 \section{The inputs to the refactoring}
1454 For executing an \ExtractAndMoveMethod refactoring, there are two simple
1455 requirements. The first thing the refactoring need is a text selection, telling
1456 it what to extract. Its second requirement is a target for the subsequent move
1459 The extracted method must be called instead of the selection that makes up its
1460 body. Also, the method call has to be performed via a variable, since the method
1461 is not static. \todo{Explain why static methods are not considered} Therefore,
1462 the move target must be a variable in the scope of the extracted selection. The
1463 actual new location for the extracted method will be the class representing the
1464 type of the move target variable. But, since the method also must be called
1465 through a variable, it makes sense to define the move target to be either a
1466 local variable or a field in the scope of the text selection.
1468 \section{Finding a move target}
1469 In the analysis needed to perform the \ExtractAndMoveMethod refactoring
1470 automatically, the selection we choose is found among all the selections that
1471 has a possible move target. Therefore, the best possible move target must be
1472 found for all the candidate selections, so that we are able to sort out the
1473 selection that is best suited for the refactoring.
1475 To find the best move target for a specific text selection, we first need to
1476 find all the possible targets. Since the target must be a local variable or a
1477 field, we are basically looking for names within the selection; names that
1478 represents references to variables.
1480 The names we are looking for, we call prefixes. This is because we are not
1481 interested in names that occur in the middle of a dot-separated sequence of
1482 names. We are only interested in names that constitutes prefixes of other names,
1483 possibly themselves. The reason for this, is that two lexically equal names need
1484 not be referencing the same variable, if they themselves are not referenced via
1485 the same prefix. Consider the two method calls \code{a.x.foo()} and
1486 \code{b.x.foo()}. Here, the two references to \code{x}, in the middle of the
1487 qualified names both preceding \code{foo()}, are not referencing the same
1488 variable. Even though the variables may share the type, and the method
1489 \method{foo} thus is the same for both, we would not know through which of the
1490 variables \var{a} or \var{b} we should call the extracted method.
1492 The possible move targets are then the prefixes that are not among a subset of
1493 the prefixes that are not valid move targets \see{s:unfixes}. Also, prefixes
1494 that are just simple names, and have only one occurrence, are left out. This is
1495 because they are not going to have any positive effect on coupling between
1498 For finding the best move target among these safe prefixes, a simple heuristic
1499 is used. It is as simple as choosing the prefix that is most frequently
1500 referenced within the selection.
1502 \section{Unfixes}\label{s:unfixes}
1503 The prefixes that are not valid as move targets are called unfixes.
1505 An unfix can be a name that is assigned to within a selection. The reason that
1506 this cannot be allowed, is that the result would be an assignment to the
1507 \type{this} keyword, which is not valid in Java \see{eclipse_bug_420726}.
1509 Prefixes that originates from variable declarations within the same selection
1510 are also considered unfixes. This is because when a method is moved, it needs to
1511 be called through a variable. If this variable is also declared within the
1512 method that is to be moved, this obviously cannot be done.
1514 Also considered as unfixes are variable references that are of types that are
1515 not suitable for moving methods to. This can either be because it is not
1516 physically possible to move a method to the desired class or that it will cause
1517 compilation errors by doing so.
1519 If the type binding for a name is not resolved it is considered and unfix. The
1520 same applies to types that is only found in compiled code, so they have no
1521 underlying source that is accessible to us. (E.g. the \type{java.lang.String}
1524 Interfaces types are not suitable as targets. This is simply because interfaces
1525 in Java cannot contain methods with bodies. (This thesis does not deal with
1526 features of Java versions later than Java 7. Java 8 has interfaces with default
1527 implementations of methods.)
1529 Neither are local types allowed. This accounts for both local and anonymous
1530 classes. Anonymous classes are effectively the same as interface types with
1531 respect to unfixes. Local classes could in theory be used as targets, but this
1532 is not possible due to limitations of the way the \refa{Extract and Move Method}
1533 refactoring has to be implemented. The problem is that the refactoring is done
1534 in two steps, so the intermediate state between the two refactorings would not
1535 be legal Java code. In the intermediate step for the case where a local class is
1536 the move target, the extracted method would need to take the local class as a
1537 parameter. This new method would need to live in the scope of the declaring
1538 class of the originating method. The local class would then not be in the scope
1539 of the extracted method, thus bringing the source code into an illegal state.
1540 One could imagine that the method was extracted and moved in one operation,
1541 without an intermediate state. Then it would make sense to include variables
1542 with types of local classes in the set of legal targets, since the local classes
1543 would then be in the scopes of the method calls. If this makes any difference
1544 for software metrics that measure coupling would be a different discussion.
1547 \begin{multicols}{2}
1548 \begin{minted}[]{java}
1550 void declaresLocalClass() {
1565 \begin{minted}[]{java}
1566 // After Extract Method
1567 void declaresLocalClass() {
1578 // Intermediate step
1579 void fooBar(LocalClass inst) {
1585 \caption{When the \refa{Extract and Move Method} tries to use a variable with a
1586 local type as the move target, an intermediate step is performed that is not
1587 allowed. Here: \type{LocalClass} is not in the scope of \method{fooBar} in its
1588 intermediate location.}
1589 \label{lst:extractMethod_LocalClass}
1592 The last class of names that are considered unfixes are names used in null
1593 tests. These are tests that reads like this: if \texttt{<name>} equals
1594 \var{null} then do something. If allowing variables used in those kinds of
1595 expressions as targets for moving methods, we would end up with code containing
1596 boolean expressions like \texttt{this == null}, which would not be meaningful,
1597 since \var{this} would never be \var{null}.
1599 \todoin{Describe what a text selection is?}
1601 \section{Choosing the selection}
1602 When choosing a selection between the text selections that have possible move
1603 targets, the selections need to be ordered. The criteria below are presented in
1604 the order they are prioritized. If not one selection is favored over the other
1605 for a concrete criterion, the selections are evaluated by the next criterion.
1608 \item The first criterion that is evaluated is whether a selection contains
1609 any unfixes or not. If selection \m{A} contains no unfixes, while
1610 selection \m{B} does, selection \m{A} is favored over selection
1611 \m{B}. This is done under the assumption that, if possible, avoiding
1612 selections containing unfixes will make the code moved a little
1613 cleaner.\todoin{more arguments?}
1615 \item The second criterion that is evaluated is how many possible targets a
1616 selection contains. If selection \m{A} has only one possible target, and
1617 selection \m{B} has multiple, selection \m{A} is favored. If both
1618 selections have multiple possible targets, they are considered equal with
1619 respect to this criterion. The rationale for this heuristic is that we would
1620 prefer not to introduce new couplings between classes when performing the
1621 \ExtractAndMoveMethod refactoring.
1623 \item When evaluating the last criterion, this is with the knowledge that
1624 selection \m{A} and \m{B} both have one possible target. Then, if
1625 the move target candidate of selection \m{A} has a higher reference count
1626 than the target candidate of selection \m{B}, selection \m{A} is
1627 favored. The reason for this is that we would like to move the selection that
1628 gets rid of the most references to another class.
1632 If none of the above mentioned criteria favors one selection over another, the
1633 selections are considered to be equally good candidates for the
1634 \ExtractAndMoveMethod refactoring.
1636 \section{Disqualifying a selection}
1637 Certain text selections would lead to broken code if used as input to the
1638 \ExtractAndMoveMethod refactoring. To avoid this, we have to check all text
1639 selections for such conditions before they are further analyzed. This sections
1640 is therefore going to present some properties that makes a selection unsuitable
1641 for our refactoring.
1643 \subsection{A call to a protected or package-private method}
1644 If a text selection contains a call to a protected or package-private method, it
1645 would not be safe to move it to another class. The reason for this, is that we
1646 cannot know if the called method is being overridden by some subclass of the
1647 \gloss{enclosingClass}, or not.
1649 Imagine that the protected method \method{foo} is declared in class \m{A},
1650 and overridden in class \m{B}. The method \method{foo} is called from within a
1651 selection done to a method in \m{A}. We want to extract and move this selection
1652 to another class. The method \method{foo} is not public, so the \MoveMethod
1653 refactoring must make it public, making the extracted method able to call it
1654 from the extracted method's new location. The problem is that the, now public,
1655 method \method{foo} is overridden in a subclass, where it has a protected
1656 status. This makes the compiler complain that the subclass \m{B} is trying to
1657 reduce the visibility of a method declared in its superclass \m{A}. This is not
1658 allowed in Java, and for good reasons. It would make it possible to make a
1659 subclass that could not be a substitute for its superclass.
1660 \todoin{Code example?}
1662 The problem this check helps to avoid, is a little subtle. The problem does not
1663 arise in the class where the change is done, but in a class derived from it.
1664 This shows that classes acting as superclasses are especially fragile to
1665 introducing errors in the context of automated refactoring. This is also shown
1666 in bug\ldots \todoin{File Eclipse bug report}
1668 \subsection{A double class instance creation}
1669 The following is a problem caused solely by the underlying \MoveMethod
1670 refactoring. The problem occurs if two classes are instantiated such that the
1671 first constructor invocation is an argument to a second, and that the first
1672 constructor invocation takes an argument that is built up using a field. As an
1673 example, say that \var{name} is a field of the enclosing class, and we have the
1674 expression \code{new A(new B(name))}. If this expression is located in a
1675 selection that is moved to another class, \var{name} will be left untouched,
1676 instead of being prefixed with a variable of the same type as it is declared in.
1677 If \var{name} is the destination for the move, it is not replaced by
1678 \code{this}, or removed if it is a prefix to a member access
1679 (\code{name.member}), but it is still left by itself.
1681 Situations like this would lead to code that will not compile. Therefore, we
1682 have to avoid them by not allowing selections to contain such double class
1683 instance creations that also contains references to fields.
1684 \todoin{File Eclipse bug report}
1686 \subsection{Instantiation of non-static inner class}
1687 When a non-static inner class is instantiated, this must happen in the scope of
1688 its declaring class. This is because it must have access to the members of the
1689 declaring class. If the inner class is public, it is possible to instantiate it
1690 through an instance of its declaring class, but this is not handled by the
1691 underlying \MoveMethod refactoring.
1693 Performing a move on a method that instantiates a non-static inner class, will
1694 break the code if the instantiation is not handled properly. For this reason,
1695 selections that contains instantiations of non-static inner classes are deemed
1696 unsuitable for the \ExtractAndMoveMethod refactoring.
1698 \subsection{References to enclosing instances of the enclosing class}
1699 The title of this section may be a little hard to grasp at first. What it means
1700 is that there is a (non-static) class \m{C} that is declared in the scope of
1701 possibly multiple other classes. And there is a statement in the body of a
1702 method declared in class \m{C}, that contains a reference to one or more
1703 instances of these enclosing classes of \m{C}.
1705 The problem with this, is that these references may not be valid if they are
1706 moved to another class. Theoretically, some situations could easily be solved by
1707 passing, to the moved method, a reference to the instance where the problematic
1708 referenced member is declared. This should work in the case where this member is
1709 publicly accessible. This is not done in the underlying \MoveMethod refactoring,
1710 so it cannot be allowed in the \ExtractAndMoveMethod refactoring either.
1712 \subsection{Inconsistent return statements}
1713 To verify that a text selection is consistent with respect to return statements,
1714 we must check that if a selection contains a return statement, then every
1715 possible execution path within the selection ends in either a return or a throw
1716 statement. This property is important regarding the \ExtractMethod refactoring.
1717 If it holds, it means that a method could be extracted from the selection, and a
1718 call to it could be substituted for the selection. If the method has a non-void
1719 return type, then a call to it would also be a valid return point for the
1720 calling method. If its return value is of the void type, then the \ExtractMethod
1721 refactoring will append an empty return statement to the back of the method
1722 call. Therefore, the analysis does not discriminate on either kinds of return
1723 statements, with or without a return value.
1725 A throw statement is accepted anywhere a return statement is required. This is
1726 because a throw statement causes an immediate exit from the current block,
1727 together with all outer blocks in its control flow that does not catch the
1730 Return statements can be either explicit or implicit. An \emph{explicit} return
1731 statement is formed by using the \code{return} keyword, while an \emph{implicit}
1732 return statement is a statement that is not formed using \code{return}, but must
1733 be the last statement of a method that can have any side effects. This can
1734 happen in methods with a void return type. An example is a statement that is
1735 inside one or more blocks. The last statement of a method could for instance be
1736 a synchronized statement, but the last statement that is executed in the method,
1737 and that can have any side effects, may be located inside the body of the
1738 synchronized statement.
1740 We can start the check for this property by looking at the last statement of a
1741 selection to see if it is a return statement (explicit or implicit) or a throw
1742 statement. If this is the case, then the property holds, assuming the selected
1743 code does not contain any compilation errors. All execution paths within the
1744 selection should end in either this, or another, return or throw statement.
1745 \todoin{State somewhere that we assume no compilation errors?}
1747 If the last statement of the selection is not a return or throw, the execution
1748 of it must eventually end in one for the selection to be legal. This means that
1749 all branches of the last statement of every branch must end in a return or
1750 throw. Given this recursive definition, there are only five types of statements
1751 that are guaranteed to end in a return or throw if their child branches does.
1752 All other statements would have to be considered illegal. The first three:
1753 Block-statements, labeled statements and do-statements are all kinds of
1754 fall-through statements that always gets their body executed. Do-statements
1755 would not make much sense if written such that they
1756 always ends after the first round of execution of their body, but that is not
1757 our concern. The remaining two statements that can end in a return or throw are
1758 if-statements and try-statements.
1760 For an if-statement, the rule is that if its then-part does not contain any
1761 return or throw statements, this is considered illegal. If the then-part does
1762 contain a return or throw, the else-part is checked. If its else-part is
1763 non-existent, or it does not contain any return or throw statements, the
1764 statement is considered illegal. If an if-statement is not considered illegal,
1765 the bodies of its two parts must be checked.
1767 Try-statements are handled much the same way as if-statements. The body of a
1768 try-statement must contain a return or throw. The same applies to its catch
1769 clauses and finally body.
1771 \subsection{Ambiguous return values}
1772 The problem with ambiguous return values arise when a selection is chosen to be
1773 extracted into a new method, but it needs to return more than one value from
1776 This problem occurs in two situations. The first situation arise when there is
1777 more than one local variable that is both assigned to within a selection and
1778 also referenced after the selection. The other situation occur when there is
1779 only one such assignment, but the selection also contain return statements.
1781 Therefore we must examine the selection for assignments to local variables that
1782 are referenced after the text selection. Then we must verify that not more than
1783 one such reference is done, or zero if any return statements are found.
1785 \subsection{Illegal statements}
1786 An illegal statement may be a statement that is of a type that is never allowed,
1787 or it may be a statement of a type that is only allowed if certain conditions
1790 Any use of the \var{super} keyword is prohibited, since its meaning is altered
1791 when moving a method to another class.
1793 For a \emph{break} statement, there are two situations to consider: A break
1794 statement with or without a label. If the break statement has a label, it is
1795 checked that whole of the labeled statement is inside the selection. If the
1796 break statement does not have a label attached to it, it is checked that its
1797 innermost enclosing loop or switch statement also is inside the selection.
1799 The situation for a \emph{continue} statement is the same as for a break
1800 statement, except that it is not allowed inside switch statements.
1802 Regarding \emph{assignments}, two types of assignments are allowed: Assignments
1803 to non-final variables and assignments to array access. All other assignments
1804 are regarded illegal.
1806 \todoin{Expand with more illegal statements and/or conclude that I did not have
1807 time to analyze all statements types.}
1809 \subsection{Grand example}
1810 \todoin{Change title?}
1813 \begin{minted}[linenos]{java}
1821 \caption{The grand example}
1822 \label{lst:grandExample}
1828 \chapter{Refactorings in Eclipse JDT: Design and
1829 Shortcomings}\label{ch:jdt_refactorings}
1831 This chapter will deal with some of the design behind refactoring support in
1832 \name{Eclipse}, and the JDT in specific. After which it will follow a section about
1833 shortcomings of the refactoring API in terms of composition of refactorings. The
1834 chapter will be concluded with a section telling some of the ways the
1835 implementation of refactorings in the JDT could have worked to facilitate
1836 composition of refactorings.
1839 The refactoring world of \name{Eclipse} can in general be separated into two parts: The
1840 language independent part and the part written for a specific programming
1841 language -- the language that is the target of the supported refactorings.
1842 \todo{What about the language specific part?}
1844 \subsection{The Language Toolkit}
1845 The Language Toolkit\footnote{The content of this section is a mixture of
1846 written material from
1847 \url{https://www.eclipse.org/articles/Article-LTK/ltk.html} and
1848 \url{http://www.eclipse.org/articles/article.php?file=Article-Unleashing-the-Power-of-Refactoring/index.html},
1849 the LTK source code and my own memory.}, or LTK for short, is the framework that
1850 is used to implement refactorings in \name{Eclipse}. It is language independent and
1851 provides the abstractions of a refactoring and the change it generates, in the
1852 form of the classes \typewithref{org.eclipse.ltk.core.refactoring}{Refactoring}
1853 and \typewithref{org.eclipse.ltk.core.refactoring}{Change}.
1855 There are also parts of the LTK that is concerned with user interaction, but
1856 they will not be discussed here, since they are of little value to us and our
1857 use of the framework. We are primarily interested in the parts that can be
1860 \subsubsection{The Refactoring Class}
1861 The abstract class \type{Refactoring} is the core of the LTK framework. Every
1862 refactoring that is going to be supported by the LTK have to end up creating an
1863 instance of one of its subclasses. The main responsibilities of subclasses of
1864 \type{Refactoring} is to implement template methods for condition checking
1865 (\methodwithref{org.eclipse.ltk.core.refactoring.Refactoring}{checkInitialConditions}
1867 \methodwithref{org.eclipse.ltk.core.refactoring.Refactoring}{checkFinalConditions}),
1869 \methodwithref{org.eclipse.ltk.core.refactoring.Refactoring}{createChange}
1870 method that creates and returns an instance of the \type{Change} class.
1872 If the refactoring shall support that others participate in it when it is
1873 executed, the refactoring has to be a processor-based
1874 refactoring\typeref{org.eclipse.ltk.core.refactoring.participants.ProcessorBasedRefactoring}.
1875 It then delegates to its given
1876 \typewithref{org.eclipse.ltk.core.refactoring.participants}{RefactoringProcessor}
1877 for condition checking and change creation. Participating in a refactoring can
1878 be useful in cases where the changes done to programming source code affects
1879 other related resources in the workspace. This can be names or paths in
1880 configuration files, or maybe one would like to perform additional logging of
1881 changes done in the workspace.
1883 \subsubsection{The Change Class}
1884 This class is the base class for objects that is responsible for performing the
1885 actual workspace transformations in a refactoring. The main responsibilities for
1886 its subclasses is to implement the
1887 \methodwithref{org.eclipse.ltk.core.refactoring.Change}{perform} and
1888 \methodwithref{org.eclipse.ltk.core.refactoring.Change}{isValid} methods. The
1889 \method{isValid} method verifies that the change object is valid and thus can be
1890 executed by calling its \method{perform} method. The \method{perform} method
1891 performs the desired change and returns an undo change that can be executed to
1892 reverse the effect of the transformation done by its originating change object.
1894 \subsubsection{Executing a Refactoring}\label{executing_refactoring}
1895 The life cycle of a refactoring generally follows two steps after creation:
1896 condition checking and change creation. By letting the refactoring object be
1898 \typewithref{org.eclipse.ltk.core.refactoring}{CheckConditionsOperation} that
1899 in turn is handled by a
1900 \typewithref{org.eclipse.ltk.core.refactoring}{CreateChangeOperation}, it is
1901 assured that the change creation process is managed in a proper manner.
1903 The actual execution of a change object has to follow a detailed life cycle.
1904 This life cycle is honored if the \type{CreateChangeOperation} is handled by a
1905 \typewithref{org.eclipse.ltk.core.refactoring}{PerformChangeOperation}. If also
1906 an undo manager\typeref{org.eclipse.ltk.core.refactoring.IUndoManager} is set
1907 for the \type{PerformChangeOperation}, the undo change is added into the undo
1910 \section{Shortcomings}
1911 This section is introduced naturally with a conclusion: The JDT refactoring
1912 implementation does not facilitate composition of refactorings.
1913 \todo{refine}This section will try to explain why, and also identify other
1914 shortcomings of both the usability and the readability of the JDT refactoring
1917 I will begin at the end and work my way toward the composition part of this
1920 \subsection{Absence of Generics in Eclipse Source Code}
1921 This section is not only concerning the JDT refactoring API, but also large
1922 quantities of the \name{Eclipse} source code. The code shows a striking absence of the
1923 Java language feature of generics. It is hard to read a class' interface when
1924 methods return objects or takes parameters of raw types such as \type{List} or
1925 \type{Map}. This sometimes results in having to read a lot of source code to
1926 understand what is going on, instead of relying on the available interfaces. In
1927 addition, it results in a lot of ugly code, making the use of typecasting more
1928 of a rule than an exception.
1930 \subsection{Composite Refactorings Will Not Appear as Atomic Actions}
1932 \subsubsection{Missing Flexibility from JDT Refactorings}
1933 The JDT refactorings are not made with composition of refactorings in mind. When
1934 a JDT refactoring is executed, it assumes that all conditions for it to be
1935 applied successfully can be found by reading source files that have been
1936 persisted to disk. They can only operate on the actual source material, and not
1937 (in-memory) copies thereof. This constitutes a major disadvantage when trying to
1938 compose refactorings, since if an exception occurs in the middle of a sequence
1939 of refactorings, it can leave the project in a state where the composite
1940 refactoring was only partially executed. It makes it hard to discard the changes
1941 done without monitoring and consulting the undo manager, an approach that is not
1944 \subsubsection{Broken Undo History}
1945 When designing a composed refactoring that is to be performed as a sequence of
1946 refactorings, you would like it to appear as a single change to the workspace.
1947 This implies that you would also like to be able to undo all the changes done by
1948 the refactoring in a single step. This is not the way it appears when a sequence
1949 of JDT refactorings is executed. It leaves the undo history filled up with
1950 individual undo actions corresponding to every single JDT refactoring in the
1951 sequence. This problem is not trivial to handle in \name{Eclipse}
1952 \see{hacking_undo_history}.
1956 \chapter{Composite Refactorings in Eclipse}
1958 \section{A Simple Ad Hoc Model}
1959 As pointed out in \myref{ch:jdt_refactorings}, the \name{Eclipse} JDT refactoring model
1960 is not very well suited for making composite refactorings. Therefore a simple
1961 model using changer objects (of type \type{RefaktorChanger}) is used as an
1962 abstraction layer on top of the existing \name{Eclipse} refactorings, instead of
1963 extending the \typewithref{org.eclipse.ltk.core.refactoring}{Refactoring} class.
1965 The use of an additional abstraction layer is a deliberate choice. It is due to
1966 the problem of creating a composite
1967 \typewithref{org.eclipse.ltk.core.refactoring}{Change} that can handle text
1968 changes that interfere with each other. Thus, a \type{RefaktorChanger} may, or
1969 may not, take advantage of one or more existing refactorings, but it is always
1970 intended to make a change to the workspace.
1972 \subsection{A typical \type{RefaktorChanger}}
1973 The typical refaktor changer class has two responsibilities, checking
1974 preconditions and executing the requested changes. This is not too different
1975 from the responsibilities of an LTK refactoring, with the distinction that a
1976 refaktor changer also executes the change, while an LTK refactoring is only
1977 responsible for creating the object that can later be used to do the job.
1979 Checking of preconditions is typically done by an
1980 \typewithref{no.uio.ifi.refaktor.analyze.analyzers}{Analyzer}. If the
1981 preconditions validate, the upcoming changes are executed by an
1982 \typewithref{no.uio.ifi.refaktor.change.executors}{Executor}.
1984 \section{The Extract and Move Method Refactoring}
1985 %The Extract and Move Method Refactoring is implemented mainly using these
1988 % \item \type{ExtractAndMoveMethodChanger}
1989 % \item \type{ExtractAndMoveMethodPrefixesExtractor}
1990 % \item \type{Prefix}
1991 % \item \type{PrefixSet}
1994 \subsection{The Building Blocks}
1995 This is a composite refactoring, and hence is built up using several primitive
1996 refactorings. These basic building blocks are, as its name implies, the
1997 \ExtractMethod refactoring\citing{refactoring} and the \MoveMethod
1998 refactoring\citing{refactoring}. In \name{Eclipse}, the implementations of these
1999 refactorings are found in the classes
2000 \typewithref{org.eclipse.jdt.internal.corext.refactoring.code}{ExtractMethodRefactoring}
2002 \typewithref{org.eclipse.jdt.internal.corext.refactoring.structure}{MoveInstanceMethodProcessor},
2003 where the last class is designed to be used together with the processor-based
2004 \typewithref{org.eclipse.ltk.core.refactoring.participants}{MoveRefactoring}.
2006 \subsubsection{The ExtractMethodRefactoring Class}
2007 This class is quite simple in its use. The only parameters it requires for
2008 construction is a compilation
2009 unit\typeref{org.eclipse.jdt.core.ICompilationUnit}, the offset into the source
2010 code where the extraction shall start, and the length of the source to be
2011 extracted. Then you have to set the method name for the new method together with
2012 its visibility and some not so interesting parameters.
2014 \subsubsection{The MoveInstanceMethodProcessor Class}
2015 For the \refa{Move Method}, the processor requires a little more advanced input than
2016 the class for the \refa{Extract Method}. For construction it requires a method
2017 handle\typeref{org.eclipse.jdt.core.IMethod} for the method that is to be moved.
2018 Then the target for the move have to be supplied as the variable binding from a
2019 chosen variable declaration. In addition to this, one have to set some
2020 parameters regarding setters/getters, as well as delegation.
2022 To make a working refactoring from the processor, one have to create a
2023 \type{MoveRefactoring} with it.
2025 \subsection{The ExtractAndMoveMethodChanger}
2027 The \typewithref{no.uio.ifi.refaktor.changers}{ExtractAndMoveMethodChanger}
2028 class is a subclass of the class
2029 \typewithref{no.uio.ifi.refaktor.changers}{RefaktorChanger}. It is responsible
2030 for analyzing and finding the best target for, and also executing, a composition
2031 of the \refa{Extract Method} and \refa{Move Method} refactorings. This particular changer is
2032 the one of my changers that is closest to being a true LTK refactoring. It can
2033 be reworked to be one if the problems with overlapping changes are resolved. The
2034 changer requires a text selection and the name of the new method, or else a
2035 method name will be generated. The selection has to be of the type
2036 \typewithref{no.uio.ifi.refaktor.utils}{CompilationUnitTextSelection}. This
2037 class is a custom extension to
2038 \typewithref{org.eclipse.jface.text}{TextSelection}, that in addition to the
2039 basic offset, length and similar methods, also carry an instance of the
2040 underlying compilation unit handle for the selection.
2043 \type{ExtractAndMoveMethodAnalyzer}}\label{extractAndMoveMethodAnalyzer}
2044 The analysis and precondition checking is done by the
2045 \typewithref{no.uio.ifi.refaktor.analyze.analyzers}{ExtractAnd\-MoveMethodAnalyzer}.
2046 First is check whether the selection is a valid selection or not, with respect
2047 to statement boundaries and that it actually contains any selections. Then it
2048 checks the legality of both extracting the selection and also moving it to
2049 another class. This checking of is performed by a range of checkers
2050 \see{checkers}. If the selection is approved as legal, it is analyzed to find
2051 the presumably best target to move the extracted method to.
2053 For finding the best suitable target the analyzer is using a
2054 \typewithref{no.uio.ifi.refaktor.analyze.collectors}{PrefixesCollector} that
2055 collects all the possible candidate targets for the refactoring. All the
2056 non-candidates is found by an
2057 \typewithref{no.uio.ifi.refaktor.analyze.collectors}{UnfixesCollector} that
2058 collects all the targets that will give some kind of error if used. (For
2059 details about the property collectors, see \myref{propertyCollectors}.) All
2060 prefixes (and unfixes) are represented by a
2061 \typewithref{no.uio.ifi.refaktor.extractors}{Prefix}, and they are collected
2062 into sets of prefixes. The safe prefixes is found by subtracting from the set of
2063 candidate prefixes the prefixes that is enclosing any of the unfixes. A prefix
2064 is enclosing an unfix if the unfix is in the set of its sub-prefixes. As an
2065 example, \texttt{``a.b''} is enclosing \texttt{``a''}, as is \texttt{``a''}. The
2066 safe prefixes is unified in a \type{PrefixSet}. If a prefix has only one
2067 occurrence, and is a simple expression, it is considered unsuitable as a move
2068 target. This occurs in statements such as \texttt{``a.foo()''}. For such
2069 statements it bares no meaning to extract and move them. It only generates an
2070 extra method and the calling of it.
2072 The most suitable target for the refactoring is found by finding the prefix with
2073 the most occurrences. If two prefixes have the same occurrence count, but they
2074 differ in length, the longest of them is chosen.
2076 \todoin{Clean up sections/subsections.}
2079 \type{ExtractAndMoveMethodExecutor}}\label{extractAndMoveMethodExecutor}
2080 If the analysis finds a possible target for the composite refactoring, it is
2082 \typewithref{no.uio.ifi.refaktor.change.executors}{ExtractAndMoveMethodExecutor}.
2083 It is composed of the two executors known as
2084 \typewithref{no.uio.ifi.refaktor.change.executors}{ExtractMethodRefactoringExecutor}
2086 \typewithref{no.uio.ifi.refaktor.change.executors}{MoveMethodRefactoringExecutor}.
2087 The \type{ExtractAndMoveMethodExecutor} is responsible for gluing the two
2088 together by feeding the \type{MoveMethod\-RefactoringExecutor} with the
2089 resources needed after executing the extract method refactoring.
2090 %\see{postExtractExecution}.
2092 \subsubsection{The \type{ExtractMethodRefactoringExecutor}}
2093 This executor is responsible for creating and executing an instance of the
2094 \type{ExtractMethodRefactoring} class. It is also responsible for collecting
2095 some post execution resources that can be used to find the method handle for the
2096 extracted method, as well as information about its parameters, including the
2097 variable they originated from.
2099 \subsubsection{The \type{MoveMethodRefactoringExecutor}}
2100 This executor is responsible for creating and executing an instance of the
2101 \type{MoveRefactoring}. The move refactoring is a processor-based refactoring,
2102 and for the \refa{Move Method} refactoring it is the \type{MoveInstanceMethodProcessor}
2105 The handle for the method to be moved is found on the basis of the information
2106 gathered after the execution of the \refa{Extract Method} refactoring. The only
2107 information the \type{ExtractMethodRefactoring} is sharing after its execution,
2108 regarding find the method handle, is the textual representation of the new
2109 method signature. Therefore it must be parsed, the strings for types of the
2110 parameters must be found and translated to a form that can be used to look up
2111 the method handle from its type handle. They have to be on the unresolved
2112 form.\todo{Elaborate?} The name for the type is found from the original
2113 selection, since an extracted method must end up in the same type as the
2116 When analyzing a selection prior to performing the \refa{Extract Method} refactoring, a
2117 target is chosen. It has to be a variable binding, so it is either a field or a
2118 local variable/parameter. If the target is a field, it can be used with the
2119 \type{MoveInstanceMethodProcessor} as it is, since the extracted method still is
2120 in its scope. But if the target is local to the originating method, the target
2121 that is to be used for the processor must be among its parameters. Thus the
2122 target must be found among the extracted method's parameters. This is done by
2123 finding the parameter information object that corresponds to the parameter that
2124 was declared on basis of the original target's variable when the method was
2125 extracted. (The extracted method must take one such parameter for each local
2126 variable that is declared outside the selection that is extracted.) To match the
2127 original target with the correct parameter information object, the key for the
2128 information object is compared to the key from the original target's binding.
2129 The source code must then be parsed to find the method declaration for the
2130 extracted method. The new target must be found by searching through the
2131 parameters of the declaration and choose the one that has the same type as the
2132 old binding from the parameter information object, as well as the same name that
2133 is provided by the parameter information object.
2137 SearchBasedExtractAndMoveMethodChanger}\label{searchBasedExtractAndMoveMethodChanger}
2139 \typewithref{no.uio.ifi.refaktor.change.changers}{SearchBasedExtractAndMoveMethodChanger}
2140 is a changer whose purpose is to automatically analyze a method, and execute the
2141 \ExtractAndMoveMethod refactoring on it if it is a suitable candidate for the
2144 First, the \typewithref{no.uio.ifi.refaktor.analyze.analyzers}{SearchBasedExtractAndMoveMethodAnalyzer} is used
2145 to analyze the method. If the method is found to be a candidate, the result from
2146 the analysis is fed to the \type{ExtractAndMoveMethodExecutor}, whose job is to
2147 execute the refactoring \see{extractAndMoveMethodExecutor}.
2149 \subsubsection{The SearchBasedExtractAndMoveMethodAnalyzer}
2150 This analyzer is responsible for analyzing all the possible text selections of a
2151 method and then choose the best result out of the analysis results that is, by
2152 the analyzer, considered to be the potential candidates for the Extract and Move
2155 Before the analyzer is able to work with the text selections of a method, it
2156 needs to generate them. To do this, it parses the method to obtain a
2157 \type{MethodDeclaration} for it \see{astEclipse}. Then there is a statement
2158 lists creator that creates statements lists of the different groups of
2159 statements in the body of the method declaration. A text selections generator
2160 generates text selections of all the statement lists for the analyzer to work
2163 \paragraph{The statement lists creator}
2164 is responsible for generating lists of statements for all the possible nesting
2165 levels of statements in the method. The statement lists creator is implemented
2166 as an AST visitor \see{astVisitor}. It generates lists of statements by visiting
2167 all the blocks in the method declaration and stores their statements in a
2168 collection of statement lists. In addition, it visits all of the other
2169 statements that can have a statement as a child, such as the different control
2170 structures and the labeled statement.
2172 The switch statement is the only kind of statement that is not straight forward
2173 to obtain the child statements from. It stores all of its children in a flat
2174 list. Its switch case statements are included in this list. This means that
2175 there are potential statement lists between all of these case statements. The
2176 list of statements from a switch statement is therefore traversed, and the
2177 statements between the case statements are grouped as separate lists.
2179 \Myref{lst:statementListsExample} shows an example of how the statement lists
2180 creator would generate lists for a simple method.
2183 \def\charwidth{5.7pt}
2184 \def\indent{4*\charwidth}
2185 \def\lineheight{\baselineskip}
2186 \def\mintedtop{\lineheight}
2188 \begin{tikzpicture}[overlay, yscale=-1]
2189 \tikzstyle{overlaybox}=[fill=lightgray,opacity=0.2]
2190 \draw[overlaybox] (0,\mintedtop+\lineheight) rectangle
2191 +(22*\charwidth,10*\lineheight);
2192 \draw[overlaybox] (\indent,\mintedtop+2*\lineheight) rectangle
2193 +(13*\charwidth,\lineheight);
2194 \draw[overlaybox] (2*\indent,\mintedtop+6*\lineheight) rectangle
2195 +(13*\charwidth,2*\lineheight);
2196 \draw[overlaybox] (2*\indent,\mintedtop+9*\lineheight) rectangle
2197 +(13*\charwidth,\lineheight);
2199 \begin{minted}{java}
2213 \caption{Example of how the statement lists creator would group a simple method
2214 into lists of statements. Each highlighted rectangle represents a list.}
2215 \label{lst:statementListsExample}
2218 \paragraph{The text selections generator} generates text selections for each
2219 list of statements from the statement lists creator. The generator generates a
2220 text selection for every sub-sequence of statements in a list. For a sequence of
2221 statements, the first statement and the last statement span out a text
2224 In practice, the text selections are calculated by only one traversal of the
2225 statement list. There is a set of generated text selections. For each statement,
2226 there is created a temporary set of selections, in addition to a text selection
2227 based on the offset and length of the statement. This text selection is added to
2228 the temporary set. Then the new selection is added with every selection from the
2229 set of generated text selections. These new selections are added to the
2230 temporary set. Then the temporary set of selections is added to the set of
2231 generated text selections. The result of adding two text selections is a new
2232 text selection spanned out by the two addends.
2235 \def\charwidth{5.7pt}
2236 \def\indent{4*\charwidth}
2237 \def\lineheight{\baselineskip}
2238 \def\mintedtop{\lineheight}
2240 \begin{tikzpicture}[overlay, yscale=-1]
2241 \tikzstyle{overlaybox}=[fill=lightgray,opacity=0.2]
2243 \draw[overlaybox] (2*\charwidth,\mintedtop) rectangle
2244 +(18*\charwidth,\lineheight);
2246 \draw[overlaybox] (2*\charwidth,\mintedtop+\lineheight) rectangle
2247 +(18*\charwidth,\lineheight);
2249 \draw[overlaybox] (2*\charwidth,\mintedtop+3*\lineheight) rectangle
2250 +(18*\charwidth,\lineheight);
2252 \draw[overlaybox] (\indent-3*\charwidth,\mintedtop) rectangle
2253 +(20*\charwidth,2*\lineheight);
2255 \draw[overlaybox] (3*\charwidth,\mintedtop+\lineheight) rectangle
2256 +(16*\charwidth,3*\lineheight);
2258 \draw[overlaybox] (\indent,\mintedtop) rectangle
2259 +(14*\charwidth,4*\lineheight);
2261 \begin{minted}{java}
2267 \caption{Example of how the text selections generator would generate text
2268 selections based on a lists of statements. Each highlighted rectangle
2269 represents a text selection.}
2270 \label{lst:textSelectionsExample}
2272 \todoin{fix \myref{lst:textSelectionsExample}? Text only? All
2273 sub-sequences\ldots}
2275 \paragraph{Finding the candidate} for the refactoring is done by analyzing all
2276 the generated text selection with the \type{ExtractAndMoveMethodAnalyzer}
2277 \see{extractAndMoveMethodAnalyzer}. If the analyzer generates a useful result,
2278 an \type{ExtractAndMoveMethodCandidate} is created from it, that is kept in a
2279 list of potential candidates. If no candidates are found, the
2280 \type{NoTargetFoundException} is thrown.
2282 Since only one of the candidates can be chosen, the analyzer must sort out which
2283 candidate to choose. The sorting is done by the static \method{sort} method of
2284 \type{Collections}. The comparison in this sorting is done by an
2285 \type{ExtractAndMoveMethodCandidateComparator}.
2286 \todoin{Write about the
2287 ExtractAndMoveMethodCandidateComparator/FavorNoUnfixesCandidateComparator}
2289 \paragraph{The complexity} of how many text selections that needs to be analyzed
2290 for a total of $n$ statements is bounded by $O(n^2)$.
2293 The number of text selections that need to be analyzed for each list of
2294 statements of length $n$, is exactly
2297 \sum_{i=1}^{n} i = \frac{n(n+1)}{2}
2298 \label{eq:complexityStatementList}
2300 \label{thm:numberOfTextSelection}
2304 For $n=1$ this is trivial: $\frac{1(1+1)}{2} = \frac{2}{2} = 1$. One statement
2305 equals one selection.
2307 For $n=2$, you get one text selection for the first statement, one selection
2308 for the second statement, and one selection for the two of them combined.
2309 This equals three selections. $\frac{2(2+1)}{2} = \frac{6}{2} = 3$.
2311 For $n=3$, you get 3 selections for the two first statements, as in the case
2312 where $n=2$. In addition you get one selection for the third statement itself,
2313 and two more statements for the combinations of it with the two previous
2314 statements. This equals six selections. $\frac{3(3+1)}{2} = \frac{12}{2} = 6$.
2316 Assume that for $n=k$ there exists $\frac{k(k+1)}{2}$ text selections. Then we
2317 want to add selections for another statement, following the previous $k$
2318 statements. So, for $n=k+1$, we get one additional selection for the statement
2319 itself. Then we get one selection for each pair of the new selection and the
2320 previous $k$ statements. So the total number of selections will be the number
2321 of already generated selections, plus $k$ for every pair, plus one for the
2322 statement itself: $\frac{k(k+1)}{2} + k +
2323 1 = \frac{k(k+1)+2k+2}{2} = \frac{k(k+1)+2(k+1)}{2} = \frac{(k+1)(k+2)}{2} =
2324 \frac{(k+1)((k+1)+1)}{2} = \sum_{i=1}^{k+1} i$
2327 %\definition{A \emph{body of statements} is a sequence of statements where every
2328 %statement may have sub-statements.}
2329 \todoin{Define ``body of statements''?}
2332 The number of text selections for a body of statements is maximized if all the
2333 statements are at the same level.
2334 \label{thm:textSelectionsMaximized}
2338 Assume we have a body of, in total, $k$ statements. Then, the sum of the
2339 lengths of all the lists of statements in the body, is also $k$. Let
2340 $\{l,\ldots,m,(k-l-\ldots-m)\}$ be the lengths of the lists of statements in
2341 the body, with $l+\ldots+m<k \Rightarrow \forall i \in \{l,\ldots,m\} : i < k$.
2343 Then, the number of text selections that are generated for the $k$ statements
2349 \frac{l(l+1)}{2} + \ldots + \frac{m(m+1)}{2} +
2350 \frac{(k-l-\ldots-m)((k-l-\ldots-m)+ 1)}{2} = \\
2351 \frac{l^2+l}{2} + \ldots + \frac{m^2+m}{2} + \frac{k^2 - 2kl - \ldots - 2km +
2352 l^2 + \ldots + m^2 + k - l - \ldots - m}{2} = \\
2353 \frac{2l^2 - 2kl + \ldots + 2m^2 - 2km + k^2 + k}{2}
2357 \noindent It then remains to show that this inequality holds:
2360 \frac{2l^2 - 2kl + \ldots + 2m^2 - 2km + k^2 + k}{2} < \frac{k(k+1)}{2} =
2364 \noindent By multiplication by $2$ on both sides, and by removing the equal
2368 2l^2 - 2kl + \ldots + 2m^2 - 2km < 0
2371 Since $\forall i \in \{l,\ldots,m\} : i < k$, we have that $\forall i \in
2372 \{l,\ldots,m\} : 2ki > 2i^2$, so all the pairs of parts on the form $2i^2-2ki$
2373 are negative. In sum, the inequality holds.
2377 Therefore, the complexity for the number of selections that needs to be analyzed
2378 for a body of $n$ statements is $O\bigl(\frac{n(n+1)}{2}\bigr) = O(n^2)$.
2382 \subsection{Finding the IMethod}\label{postExtractExecution}
2383 \todoin{Rename section. Write??}
2387 \subsection{The Prefix Class}
2388 This class exists mainly for holding data about a prefix, such as the expression
2389 that the prefix represents and the occurrence count of the prefix within a
2390 selection. In addition to this, it has some functionality such as calculating
2391 its sub-prefixes and intersecting it with another prefix. The definition of the
2392 intersection between two prefixes is a prefix representing the longest common
2393 expression between the two.
2395 \subsection{The PrefixSet Class}
2396 A prefix set holds elements of type \type{Prefix}. It is implemented with the
2397 help of a \typewithref{java.util}{HashMap} and contains some typical set
2398 operations, but it does not implement the \typewithref{java.util}{Set}
2399 interface, since the prefix set does not need all of the functionality a
2400 \type{Set} requires to be implemented. In addition It needs some other
2401 functionality not found in the \type{Set} interface. So due to the relatively
2402 limited use of prefix sets, and that it almost always needs to be referenced as
2403 such, and not a \type{Set<Prefix>}, it remains as an ad hoc solution to a
2406 There are two ways adding prefixes to a \type{PrefixSet}. The first is through
2407 its \method{add} method. This works like one would expect from a set. It adds
2408 the prefix to the set if it does not already contain the prefix. The other way
2409 is to \emph{register} the prefix with the set. When registering a prefix, if the
2410 set does not contain the prefix, it is just added. If the set contains the
2411 prefix, its count gets incremented. This is how the occurrence count is handled.
2413 The prefix set also computes the set of prefixes that is not enclosing any
2414 prefixes of another set. This is kind of a set difference operation only for
2417 \subsection{Hacking the Refactoring Undo
2418 History}\label{hacking_undo_history}
2419 \todoin{Where to put this section?}
2421 As an attempt to make multiple subsequent changes to the workspace appear as a
2422 single action (i.e. make the undo changes appear as such), I tried to alter
2423 the undo changes\typeref{org.eclipse.ltk.core.refactoring.Change} in the history
2424 of the refactorings.
2426 My first impulse was to remove the, in this case, last two undo changes from the
2427 undo manager\typeref{org.eclipse.ltk.core.refactoring.IUndoManager} for the
2428 \name{Eclipse} refactorings, and then add them to a composite
2429 change\typeref{org.eclipse.ltk.core.refactoring.CompositeChange} that could be
2430 added back to the manager. The interface of the undo manager does not offer a
2431 way to remove/pop the last added undo change, so a possible solution could be to
2432 decorate\citing{designPatterns} the undo manager, to intercept and collect the
2433 undo changes before delegating to the \method{addUndo}
2434 method\methodref{org.eclipse.ltk.core.refactoring.IUndoManager}{addUndo} of the
2435 manager. Instead of giving it the intended undo change, a null change could be
2436 given to prevent it from making any changes if run. Then one could let the
2437 collected undo changes form a composite change to be added to the manager.
2439 There is a technical challenge with this approach, and it relates to the undo
2440 manager, and the concrete implementation
2441 \typewithref{org.eclipse.ltk.internal.core.refactoring}{UndoManager2}. This
2442 implementation is designed in a way that it is not possible to just add an undo
2443 change, you have to do it in the context of an active
2444 operation\typeref{org.eclipse.core.commands.operations.TriggeredOperations}.
2445 One could imagine that it might be possible to trick the undo manager into
2446 believing that you are doing a real change, by executing a refactoring that is
2447 returning a kind of null change that is returning our composite change of undo
2448 refactorings when it is performed. But this is not the way to go.
2450 Apart from the technical problems with this solution, there is a functional
2451 problem: If it all had worked out as planned, this would leave the undo history
2452 in a dirty state, with multiple empty undo operations corresponding to each of
2453 the sequentially executed refactoring operations, followed by a composite undo
2454 change corresponding to an empty change of the workspace for rounding of our
2455 composite refactoring. The solution to this particular problem could be to
2456 intercept the registration of the intermediate changes in the undo manager, and
2457 only register the last empty change.
2459 Unfortunately, not everything works as desired with this solution. The grouping
2460 of the undo changes into the composite change does not make the undo operation
2461 appear as an atomic operation. The undo operation is still split up into
2462 separate undo actions, corresponding to the change done by its originating
2463 refactoring. And in addition, the undo actions has to be performed separate in
2464 all the editors involved. This makes it no solution at all, but a step toward
2467 There might be a solution to this problem, but it remains to be found. The
2468 design of the refactoring undo management is partly to be blamed for this, as it
2469 it is to complex to be easily manipulated.
2474 \chapter{Analyzing Source Code in Eclipse}
2476 \section{The Java model}\label{javaModel}
2477 The Java model of \name{Eclipse} is its internal representation of a Java project. It
2478 is light-weight, and has only limited possibilities for manipulating source
2479 code. It is typically used as a basis for the Package Explorer in \name{Eclipse}.
2481 The elements of the Java model is only handles to the underlying elements. This
2482 means that the underlying element of a handle does not need to actually exist.
2483 Hence the user of a handle must always check that it exist by calling the
2484 \method{exists} method of the handle.
2486 The handles with descriptions is listed in \myref{tab:javaModel}, while the
2487 hierarchy of the Java Model is shown in \myref{fig:javaModel}.
2490 \caption{The elements of the Java Model\citing{vogelEclipseJDT2012}.}
2491 \label{tab:javaModel}
2493 % sum must equal number of columns (3)
2494 \begin{tabularx}{\textwidth}{@{} L{0.7} L{1.1} L{1.2} @{}}
2496 \textbf{Project Element} & \textbf{Java Model element} &
2497 \textbf{Description} \\
2499 Java project & \type{IJavaProject} & The Java project which contains all other objects. \\
2501 Source folder /\linebreak[2] binary folder /\linebreak[3] external library &
2502 \type{IPackageFragmentRoot} & Hold source or binary files, can be a folder
2503 or a library (zip / jar file). \\
2505 Each package & \type{IPackageFragment} & Each package is below the
2506 \type{IPackageFragmentRoot}, sub-packages are not leaves of the package,
2507 they are listed directed under \type{IPackageFragmentRoot}. \\
2509 Java Source file & \type{ICompilationUnit} & The Source file is always below
2510 the package node. \\
2512 Types / Fields /\linebreak[3] Methods & \type{IType} / \type{IField}
2513 /\linebreak[3] \type{IMethod} & Types, fields and methods. \\
2521 \begin{tikzpicture}[%
2522 grow via three points={one child at (0,-0.7) and
2523 two children at (0,-0.7) and (0,-1.4)},
2524 edge from parent path={(\tikzparentnode.south west)+(0.5,0) |-
2525 (\tikzchildnode.west)}]
2526 \tikzstyle{every node}=[draw=black,thick,anchor=west]
2527 \tikzstyle{selected}=[draw=red,fill=red!30]
2528 \tikzstyle{optional}=[dashed,fill=gray!50]
2529 \node {\type{IJavaProject}}
2530 child { node {\type{IPackageFragmentRoot}}
2531 child { node {\type{IPackageFragment}}
2532 child { node {\type{ICompilationUnit}}
2533 child { node {\type{IType}}
2534 child { node {\type{\{ IType \}*}}
2535 child { node {\type{\ldots}}}
2538 child { node {\type{\{ IField \}*}}}
2539 child { node {\type{IMethod}}
2540 child { node {\type{\{ IType \}*}}
2541 child { node {\type{\ldots}}}
2546 child { node {\type{\{ IMethod \}*}}}
2555 child { node {\type{\{ IType \}*}}}
2566 child { node {\type{\{ ICompilationUnit \}*}}}
2579 child { node {\type{\{ IPackageFragment \}*}}}
2594 child { node {\type{\{ IPackageFragmentRoot \}*}}}
2597 \caption{The Java model of \name{Eclipse}. ``\type{\{ SomeElement \}*}'' means
2598 ``\type{SomeElement} zero or more times``. For recursive structures,
2599 ``\type{\ldots}'' is used.}
2600 \label{fig:javaModel}
2603 \section{The Abstract Syntax Tree}
2604 \name{Eclipse} is following the common paradigm of using an abstract syntax tree for
2605 source code analysis and manipulation.
2607 When parsing program source code into something that can be used as a foundation
2608 for analysis, the start of the process follows the same steps as in a compiler.
2609 This is all natural, because the way a compiler analyzes code is no different
2610 from how source manipulation programs would do it, except for some properties of
2611 code that is analyzed in the parser, and that they may be differing in what
2612 kinds of properties they analyze. Thus the process of translation source code
2613 into a structure that is suitable for analyzing, can be seen as a kind of
2614 interrupted compilation process \see{fig:interruptedCompilationProcess}.
2619 base/.style={anchor=north, align=center, rectangle, minimum height=1.4cm},
2620 basewithshadow/.style={base, drop shadow, fill=white},
2621 outlined/.style={basewithshadow, draw, rounded corners, minimum
2623 primary/.style={outlined, font=\bfseries},
2624 dashedbox/.style={outlined, dashed},
2625 arrowpath/.style={black, align=center, font=\small},
2626 processarrow/.style={arrowpath, ->, >=angle 90, shorten >=1pt},
2628 \begin{tikzpicture}[node distance=1.3cm and 3cm, scale=1, every
2629 node/.style={transform shape}]
2630 \node[base](AuxNode1){\small source code};
2631 \node[primary, right=of AuxNode1, xshift=-2.5cm](Scanner){Scanner};
2632 \node[primary, right=of Scanner, xshift=0.5cm](Parser){Parser};
2633 \node[dashedbox, below=of Parser](SemanticAnalyzer){Semantic\\Analyzer};
2634 \node[dashedbox, left=of SemanticAnalyzer](SourceCodeOptimizer){Source
2636 \node[dashedbox, below=of SourceCodeOptimizer
2637 ](CodeGenerator){Code\\Generator};
2638 \node[dashedbox, right=of CodeGenerator](TargetCodeOptimizer){Target
2640 \node[base, right=of TargetCodeOptimizer](AuxNode2){};
2642 \draw[processarrow](AuxNode1) -- (Scanner);
2644 \path[arrowpath] (Scanner) -- node [sloped](tokens){tokens}(Parser);
2645 \draw[processarrow](Scanner) -- (tokens) -- (Parser);
2647 \path[arrowpath] (Parser) -- node (syntax){syntax
2648 tree}(SemanticAnalyzer);
2649 \draw[processarrow](Parser) -- (syntax) -- (SemanticAnalyzer);
2651 \path[arrowpath] (SemanticAnalyzer) -- node
2652 [sloped](annotated){annotated\\tree}(SourceCodeOptimizer);
2653 \draw[processarrow, dashed](SemanticAnalyzer) -- (annotated) --
2654 (SourceCodeOptimizer);
2656 \path[arrowpath] (SourceCodeOptimizer) -- node
2657 (intermediate){intermediate code}(CodeGenerator);
2658 \draw[processarrow, dashed](SourceCodeOptimizer) -- (intermediate) --
2661 \path[arrowpath] (CodeGenerator) -- node [sloped](target1){target
2662 code}(TargetCodeOptimizer);
2663 \draw[processarrow, dashed](CodeGenerator) -- (target1) --
2664 (TargetCodeOptimizer);
2666 \path[arrowpath](TargetCodeOptimizer) -- node [sloped](target2){target
2668 \draw[processarrow, dashed](TargetCodeOptimizer) -- (target2) (AuxNode2);
2670 \caption{Interrupted compilation process. {\footnotesize (Full compilation
2671 process borrowed from \emph{Compiler construction: principles and practice}
2672 by Kenneth C. Louden\citing{louden1997}.)}}
2673 \label{fig:interruptedCompilationProcess}
2676 The process starts with a \emph{scanner}, or lexer. The job of the scanner is to
2677 read the source code and divide it into tokens for the parser. Therefore, it is
2678 also sometimes called a tokenizer. A token is a logical unit, defined in the
2679 language specification, consisting of one or more consecutive characters. In
2680 the Java language the tokens can for instance be the \var{this} keyword, a curly
2681 bracket \var{\{} or a \var{nameToken}. It is recognized by the scanner on the
2682 basis of something equivalent of a regular expression. This part of the process
2683 is often implemented with the use of a finite automata. In fact, it is common to
2684 specify the tokens in regular expressions, that in turn is translated into a
2685 finite automata lexer. This process can be automated.
2687 The program component used to translate a stream of tokens into something
2688 meaningful, is called a parser. A parser is fed tokens from the scanner and
2689 performs an analysis of the structure of a program. It verifies that the syntax
2690 is correct according to the grammar rules of a language, that is usually
2691 specified in a context-free grammar, and often in a variant of the
2693 Form}\footnote{\url{https://en.wikipedia.org/wiki/Backus-Naur\_Form}}. The
2694 result coming from the parser is in the form of an \emph{Abstract Syntax Tree},
2695 AST for short. It is called \emph{abstract}, because the structure does not
2696 contain all of the tokens produced by the scanner. It only contain logical
2697 constructs, and because it forms a tree, all kinds of parentheses and brackets
2698 are implicit in the structure. It is this AST that is used when performing the
2699 semantic analysis of the code.
2701 As an example we can think of the expression \code{(5 + 7) * 2}. The root of
2702 this tree would in \name{Eclipse} be an \type{InfixExpression} with the operator
2703 \var{TIMES}, and a left operand that is also an \type{InfixExpression} with the
2704 operator \var{PLUS}. The left operand \type{InfixExpression}, has in turn a left
2705 operand of type \type{NumberLiteral} with the value \var{``5''} and a right
2706 operand \type{NumberLiteral} with the value \var{``7''}. The root will have a
2707 right operand of type \type{NumberLiteral} and value \var{``2''}. The AST for
2708 this expression is illustrated in \myref{fig:astInfixExpression}.
2710 Contrary to the Java Model, an abstract syntax tree is a heavy-weight
2711 representation of source code. It contains information about properties like
2712 type bindings for variables and variable bindings for names.
2717 \begin{tikzpicture}[scale=0.8]
2718 \tikzset{level distance=40pt}
2719 \tikzset{sibling distance=5pt}
2720 \tikzstyle{thescale}=[scale=0.8]
2721 \tikzset{every tree node/.style={align=center}}
2722 \tikzset{edge from parent/.append style={thick}}
2723 \tikzstyle{inode}=[rectangle,rounded corners,draw,fill=lightgray,drop
2724 shadow,align=center]
2725 \tikzset{every internal node/.style={inode}}
2726 \tikzset{every leaf node/.style={draw=none,fill=none}}
2728 \Tree [.\type{InfixExpression} [.\type{InfixExpression}
2729 [.\type{NumberLiteral} \var{``5''} ] [.\type{Operator} \var{PLUS} ]
2730 [.\type{NumberLiteral} \var{``7''} ] ]
2731 [.\type{Operator} \var{TIMES} ]
2732 [.\type{NumberLiteral} \var{``2''} ]
2735 \caption{The abstract syntax tree for the expression \code{(5 + 7) * 2}.}
2736 \label{fig:astInfixExpression}
2739 \subsection{The AST in Eclipse}\label{astEclipse}
2740 In \name{Eclipse}, every node in the AST is a child of the abstract superclass
2741 \typewithref{org.eclipse.jdt.core.dom}{ASTNode}. Every \type{ASTNode}, among a
2742 lot of other things, provides information about its position and length in the
2743 source code, as well as a reference to its parent and to the root of the tree.
2745 The root of the AST is always of type \type{CompilationUnit}. It is not the same
2746 as an instance of an \type{ICompilationUnit}, which is the compilation unit
2747 handle of the Java model. The children of a \type{CompilationUnit} is an
2748 optional \type{PackageDeclaration}, zero or more nodes of type
2749 \type{ImportDecaration} and all its top-level type declarations that has node
2750 types \type{AbstractTypeDeclaration}.
2752 An \type{AbstractType\-Declaration} can be one of the types
2753 \type{AnnotationType\-Declaration}, \type{Enum\-Declaration} or
2754 \type{Type\-Declaration}. The children of an \type{AbstractType\-Declaration}
2755 must be a subtype of a \type{BodyDeclaration}. These subtypes are:
2756 \type{AnnotationTypeMember\-Declaration}, \type{EnumConstant\-Declaration},
2757 \type{Field\-Declaration}, \type{Initializer} and \type{Method\-Declaration}.
2759 Of the body declarations, the \type{Method\-Declaration} is the most interesting
2760 one. Its children include lists of modifiers, type parameters, parameters and
2761 exceptions. It has a return type node and a body node. The body, if present, is
2762 of type \type{Block}. A \type{Block} is itself a \type{Statement}, and its
2763 children is a list of \type{Statement} nodes.
2765 There are too many types of the abstract type \type{Statement} to list up, but
2766 there exists a subtype of \type{Statement} for every statement type of Java, as
2767 one would expect. This also applies to the abstract type \type{Expression}.
2768 However, the expression \type{Name} is a little special, since it is both used
2769 as an operand in compound expressions, as well as for names in type declarations
2772 There is an overview of some of the structure of an \name{Eclipse} AST in
2773 \myref{fig:astEclipse}.
2777 \begin{tikzpicture}[scale=0.8]
2778 \tikzset{level distance=50pt}
2779 \tikzset{sibling distance=5pt}
2780 \tikzstyle{thescale}=[scale=0.8]
2781 \tikzset{every tree node/.style={align=center}}
2782 \tikzset{edge from parent/.append style={thick}}
2783 \tikzstyle{inode}=[rectangle,rounded corners,draw,fill=lightgray,drop
2784 shadow,align=center]
2785 \tikzset{every internal node/.style={inode}}
2786 \tikzset{every leaf node/.style={draw=none,fill=none}}
2788 \Tree [.\type{CompilationUnit} [.\type{[ PackageDeclaration ]} [.\type{Name} ]
2789 [.\type{\{ Annotation \}*} ] ]
2790 [.\type{\{ ImportDeclaration \}*} [.\type{Name} ] ]
2791 [.\type{\{ AbstractTypeDeclaration \}+} [.\node(site){\type{\{
2792 BodyDeclaration \}*}}; ] [.\type{SimpleName} ] ]
2794 \begin{scope}[shift={(0.5,-6)}]
2795 \node[inode,thescale](root){\type{MethodDeclaration}};
2796 \node[inode,thescale](modifiers) at (4.5,-5){\type{\{ IExtendedModifier \}*}
2797 \\ {\footnotesize (Of type \type{Modifier} or \type{Annotation})}};
2798 \node[inode,thescale](typeParameters) at (-6,-3.5){\type{\{ TypeParameter
2800 \node[inode,thescale](parameters) at (-5,-5){\type{\{
2801 SingleVariableDeclaration \}*} \\ {\footnotesize (Parameters)}};
2802 \node[inode,thescale](exceptions) at (5,-3){\type{\{ Name \}*} \\
2803 {\footnotesize (Exceptions)}};
2804 \node[inode,thescale](return) at (-6.5,-2){\type{Type} \\ {\footnotesize
2806 \begin{scope}[shift={(0,-5)}]
2807 \Tree [.\node(body){\type{[ Block ]} \\ {\footnotesize (Body)}};
2808 [.\type{\{ Statement \}*} [.\type{\{ Expression \}*} ]
2809 [.\type{\{ Statement \}*} [.\type{\ldots} ]]
2814 \draw[->,>=triangle 90,shorten >=1pt](root.east)..controls +(east:2) and
2815 +(south:1)..(site.south);
2817 \draw (root.south) -- (modifiers);
2818 \draw (root.south) -- (typeParameters);
2819 \draw (root.south) -- ($ (parameters.north) + (2,0) $);
2820 \draw (root.south) -- (exceptions);
2821 \draw (root.south) -- (return);
2822 \draw (root.south) -- (body);
2825 \caption{The format of the abstract syntax tree in \name{Eclipse}.}
2826 \label{fig:astEclipse}
2828 \todoin{Add more to the AST format tree? \myref{fig:astEclipse}}
2830 \section{The ASTVisitor}\label{astVisitor}
2831 So far, the only thing that has been addressed is how the data that is going to
2832 be the basis for our analysis is structured. Another aspect of it is how we are
2833 going to traverse the AST to gather the information we need, so we can conclude
2834 about the properties we are analysing. It is of course possible to start at the
2835 top of the tree, and manually search through its nodes for the ones we are
2836 looking for, but that is a bit inconvenient. To be able to efficiently utilize
2837 such an approach, we would need to make our own framework for traversing the
2838 tree and visiting only the types of nodes we are after. Luckily, this
2839 functionality is already provided in \name{Eclipse}, by its
2840 \typewithref{org.eclipse.jdt.core.dom}{ASTVisitor}.
2842 The \name{Eclipse} AST, together with its \type{ASTVisitor}, follows the
2843 \pattern{Visitor} pattern\citing{designPatterns}. The intent of this design
2844 pattern is to facilitate extending the functionality of classes without touching
2845 the classes themselves.
2847 Let us say that there is a class hierarchy of elements. These elements all have
2848 a method \method{accept(Visitor visitor)}. In its simplest form, the
2849 \method{accept} method just calls the \method{visit} method of the visitor with
2850 itself as an argument, like this: \code{visitor.visit(this)}. For the visitors
2851 to be able to extend the functionality of all the classes in the elements
2852 hierarchy, each \type{Visitor} must have one visit method for each concrete
2853 class in the hierarchy. Say the hierarchy consists of the concrete classes
2854 \type{ConcreteElementA} and \type{ConcreteElementB}. Then each visitor must have
2855 the (possibly empty) methods \method{visit(ConcreteElementA element)} and
2856 \method{visit(ConcreteElementB element)}. This scenario is depicted in
2857 \myref{fig:visitorPattern}.
2861 \tikzstyle{abstract}=[rectangle, draw=black, fill=white, drop shadow, text
2862 centered, anchor=north, text=black, text width=6cm, every one node
2863 part/.style={align=center, font=\bfseries\itshape}]
2864 \tikzstyle{concrete}=[rectangle, draw=black, fill=white, drop shadow, text
2865 centered, anchor=north, text=black, text width=6cm]
2866 \tikzstyle{inheritarrow}=[->, >=open triangle 90, thick]
2867 \tikzstyle{commentarrow}=[->, >=angle 90, dashed]
2868 \tikzstyle{line}=[-, thick]
2869 \tikzset{every one node part/.style={align=center, font=\bfseries}}
2870 \tikzset{every second node part/.style={align=center, font=\ttfamily}}
2872 \begin{tikzpicture}[node distance=1cm, scale=0.8, every node/.style={transform
2874 \node (Element) [abstract, rectangle split, rectangle split parts=2]
2876 \nodepart{one}{Element}
2877 \nodepart{second}{+accept(visitor: Visitor)}
2879 \node (AuxNode01) [text width=0, minimum height=2cm, below=of Element] {};
2880 \node (ConcreteElementA) [concrete, rectangle split, rectangle split
2881 parts=2, left=of AuxNode01]
2883 \nodepart{one}{ConcreteElementA}
2884 \nodepart{second}{+accept(visitor: Visitor)}
2886 \node (ConcreteElementB) [concrete, rectangle split, rectangle split
2887 parts=2, right=of AuxNode01]
2889 \nodepart{one}{ConcreteElementB}
2890 \nodepart{second}{+accept(visitor: Visitor)}
2893 \node[comment, below=of ConcreteElementA] (CommentA) {visitor.visit(this)};
2895 \node[comment, below=of ConcreteElementB] (CommentB) {visitor.visit(this)};
2897 \node (AuxNodeX) [text width=0, minimum height=1cm, below=of AuxNode01] {};
2899 \node (Visitor) [abstract, rectangle split, rectangle split parts=2,
2902 \nodepart{one}{Visitor}
2903 \nodepart{second}{+visit(ConcreteElementA)\\+visit(ConcreteElementB)}
2905 \node (AuxNode02) [text width=0, minimum height=2cm, below=of Visitor] {};
2906 \node (ConcreteVisitor1) [concrete, rectangle split, rectangle split
2907 parts=2, left=of AuxNode02]
2909 \nodepart{one}{ConcreteVisitor1}
2910 \nodepart{second}{+visit(ConcreteElementA)\\+visit(ConcreteElementB)}
2912 \node (ConcreteVisitor2) [concrete, rectangle split, rectangle split
2913 parts=2, right=of AuxNode02]
2915 \nodepart{one}{ConcreteVisitor2}
2916 \nodepart{second}{+visit(ConcreteElementA)\\+visit(ConcreteElementB)}
2920 \draw[inheritarrow] (ConcreteElementA.north) -- ++(0,0.7) -|
2922 \draw[line] (ConcreteElementA.north) -- ++(0,0.7) -|
2923 (ConcreteElementB.north);
2925 \draw[inheritarrow] (ConcreteVisitor1.north) -- ++(0,0.7) -|
2927 \draw[line] (ConcreteVisitor1.north) -- ++(0,0.7) -|
2928 (ConcreteVisitor2.north);
2930 \draw[commentarrow] (CommentA.north) -- (ConcreteElementA.south);
2931 \draw[commentarrow] (CommentB.north) -- (ConcreteElementB.south);
2935 \caption{The Visitor Pattern.}
2936 \label{fig:visitorPattern}
2939 The use of the visitor pattern can be appropriate when the hierarchy of elements
2940 is mostly stable, but the family of operations over its elements is constantly
2941 growing. This is clearly the case for the \name{Eclipse} AST, since the hierarchy of
2942 type \type{ASTNode} is very stable, but the functionality of its elements is
2943 extended every time someone needs to operate on the AST. Another aspect of the
2944 \name{Eclipse} implementation is that it is a public API, and the visitor pattern is an
2945 easy way to provide access to the nodes in the tree.
2947 The version of the visitor pattern implemented for the AST nodes in \name{Eclipse} also
2948 provides an elegant way to traverse the tree. It does so by following the
2949 convention that every node in the tree first let the visitor visit itself,
2950 before it also makes all its children accept the visitor. The children are only
2951 visited if the visit method of their parent returns \var{true}. This pattern
2952 then makes for a prefix traversal of the AST. If postfix traversal is desired,
2953 the visitors also has \method{endVisit} methods for each node type, that is
2954 called after the \method{visit} method for a node. In addition to these visit
2955 methods, there are also the methods \method{preVisit(ASTNode)},
2956 \method{postVisit(ASTNode)} and \method{preVisit2(ASTNode)}. The
2957 \method{preVisit} method is called before the type-specific \method{visit}
2958 method. The \method{postVisit} method is called after the type-specific
2959 \method{endVisit}. The type specific \method{visit} is only called if
2960 \method{preVisit2} returns \var{true}. Overriding the \method{preVisit2} is also
2961 altering the behavior of \method{preVisit}, since the default implementation is
2962 responsible for calling it.
2964 An example of a trivial \type{ASTVisitor} is shown in
2965 \myref{lst:astVisitorExample}.
2968 \begin{minted}{java}
2969 public class CollectNamesVisitor extends ASTVisitor {
2970 Collection<Name> names = new LinkedList<Name>();
2973 public boolean visit(QualifiedName node) {
2979 public boolean visit(SimpleName node) {
2985 \caption{An \type{ASTVisitor} that visits all the names in a subtree and adds
2986 them to a collection, except those names that are children of any
2987 \type{QualifiedName}.}
2988 \label{lst:astVisitorExample}
2991 \section{Property collectors}\label{propertyCollectors}
2992 The prefixes and unfixes are found by property
2993 collectors\typeref{no.uio.ifi.refaktor.extractors.collectors.PropertyCollector}.
2994 A property collector is of the \type{ASTVisitor} type, and thus visits nodes of
2995 type \type{ASTNode} of the abstract syntax tree \see{astVisitor}.
2997 \subsection{The PrefixesCollector}
2998 The \typewithref{no.uio.ifi.refaktor.extractors.collectors}{PrefixesCollector}
2999 finds prefixes that makes up the basis for calculating move targets for the
3000 \refa{Extract and Move Method} refactoring. It visits expression
3001 statements\typeref{org.eclipse.jdt.core.dom.ExpressionStatement} and creates
3002 prefixes from its expressions in the case of method invocations. The prefixes
3003 found is registered with a prefix set, together with all its sub-prefixes.
3005 \subsection{The UnfixesCollector}\label{unfixes}
3006 The \typewithref{no.uio.ifi.refaktor.extractors.collectors}{UnfixesCollector}
3007 finds unfixes within a selection.
3008 \todoin{Give more technical detail?}
3012 \subsection{The ContainsReturnStatementCollector}
3013 \todoin{Remove section?}
3015 \typewithref{no.uio.ifi.refaktor.analyze.collectors}{ContainsReturnStatementCollector}
3016 is a very simple property collector. It only visits the return statements within
3017 a selection, and can report whether it encountered a return statement or not.
3019 \subsection{The LastStatementCollector}
3020 The \typewithref{no.uio.ifi.refaktor.analyze.collectors}{LastStatementCollector}
3021 collects the last statement of a selection. It does so by only visiting the top
3022 level statements of the selection, and compares the textual end offset of each
3023 encountered statement with the end offset of the previous statement found.
3025 \section{Checkers}\label{checkers}
3026 \todoin{Check out ExtractMethodAnalyzer from ExtractMethodRefactoring}
3027 The checkers are a range of classes that checks that text selections complies
3028 with certain criteria. All checkers operates under the assumption that the code
3029 they check is free from compilation errors. If a
3030 \typewithref{no.uio.ifi.refaktor.analyze.analyzers}{Checker} fails, it throws a
3031 \type{CheckerException}. The checkers are managed by the
3032 \type{LegalStatementsChecker}, which does not, in fact, implement the
3033 \type{Checker} interface. It does, however, run all the checkers registered with
3034 it, and reports that all statements are considered legal if no
3035 \type{CheckerException} is thrown. Many of the checkers either extends the
3036 \type{PropertyCollector} or utilizes one or more property collectors to verify
3037 some criteria. The checkers registered with the \type{LegalStatementsChecker}
3038 are described next. They are run in the order presented below.
3040 \subsection{The CallToProtectedOrPackagePrivateMethodChecker}
3041 This checker is used to check that at selection does not contain a call to a
3042 method that is protected or package-private. Such a method either has the access
3043 modifier \code{protected} or it has no access modifier.
3045 The workings of the \type{CallToProtectedOrPackagePrivateMethod\-Checker} is
3046 very simple. It looks for calls to methods that are either protected or
3047 package-private within the selection, and throws an
3048 \type{IllegalExpressionFoundException} if one is found.
3050 \subsection{The DoubleClassInstanceCreationChecker}
3051 The \type{DoubleClassInstanceCreationChecker} checks that there are no double
3052 class instance creations where the inner constructor call take and argument that
3053 is built up using field references.
3055 The checker visits all nodes of type \type{ClassInstanceCreation} within a
3056 selection. For all of these nodes, if its parent also is a class instance
3057 creation, it accepts a visitor that throws a
3058 \type{IllegalExpressionFoundException} if it enclounters a name that is a field
3061 \subsection{The InstantiationOfNonStaticInnerClassChecker}
3062 The \type{InstantiationOfNonStaticInnerClassChecker} checks that selections
3063 does not contain instantiations of non-static inner classes. The
3064 \type{MoveInstanceMethodProcessor} in \name{Eclipse} does not handle such
3065 instantiations gracefully when moving a method. This problem is also related to
3066 bug\ldots \todoin{File Eclipse bug report}
3068 \subsection{The EnclosingInstanceReferenceChecker}
3069 The purpose of this checker is to verify that the names in a text selection are
3070 not referencing any enclosing instances. In theory, the underlying problem could
3071 be solved in some situations, but our dependency on the
3072 \type{MoveInstanceMethodProcessor} prevents this.
3075 \typewithref{no.uio.ifi.refaktor.analyze.analyzers}{EnclosingInstanceReferenceChecker}
3076 is a modified version of the
3077 \typewithref{org.eclipse.jdt.internal.corext.refactoring.structure.MoveInstanceMethod\-Processor}{EnclosingInstanceReferenceFinder}
3078 from the \type{MoveInstanceMethodProcessor}. Wherever the
3079 \type{EnclosingInstanceReferenceFinder} would create a fatal error status, the
3080 checker will throw a \type{CheckerException}.
3082 The checker works by first finding all of the enclosing types of a selection.
3083 Thereafter, it visits all the simple names of the selection to check that they
3084 are not references to variables or methods declared in any of the enclosing
3085 types. In addition, the checker visits \var{this}-expressions to verify that no
3086 such expressions are qualified with any name.
3088 \subsection{The ReturnStatementsChecker}\label{returnStatementsChecker}
3089 The checker for return statements is meant to verify that a text selection is
3090 consistent regarding return statements.
3092 If the selection is free from return statements, then the checker validates. So
3093 this is the first thing the checker investigates.
3095 If the checker proceedes any further, it is because the selection contains one
3096 or more return statements. The next test is therefore to check if the last
3097 statement of the selection ends in either a return or a throw statement. The
3098 responsibility for checking that the last statement of the selection eventually
3099 ends in a return or throw statement, is put on the
3100 \type{LastStatementOfSelectionEndsInReturnOrThrowChecker}. For every node
3101 visited, if the node is a statement, it does a test to see if the statement is a
3102 return, a throw or if it is an implicit return statement. If this is the case,
3103 no further checking is done. This checking is done in the \code{preVisit2}
3104 method \see{astVisitor}. If the node is not of a type that is being handled by
3105 its type-specific visit method, the checker performs a simple test. If the node
3106 being visited is not the last statement of its parent that is also enclosed by
3107 the selection, an \type{IllegalStatementFoundException} is thrown. This ensures
3108 that all statements are taken care of, one way or the other. It also ensures
3109 that the checker is conservative in the way it checks for legality of the
3112 To examine if a statement is an implicit return statement, the checker first
3113 finds the last statement declared in its enclosing method. If this statement is
3114 the same as the one under investigation, it is considered an implicit return
3115 statement. If the statements are not the same, the checker does a search to see
3116 if the statement examined is also the last statement of the method that can be
3117 reached. This includes the last statement of a block statement, a labeled
3118 statement, a synchronized statement or a try statement, that in turn is the last
3119 statement enclosed by one of the statement types listed. This search goes
3120 through all the parents of a statement until a statement is found that is not
3121 one of the mentioned acceptable parent statements. If the search ends in a
3122 method declaration, then the statement is considered to be the last reachable
3123 statement of the method, and thus it is an implicit return statement.
3125 There are two kinds of statements that are handled explicitly: If-statements and
3126 try-statements. Block, labeled and do-statements are handled by fall-through to
3129 If-statements are handled explicitly by overriding their type-specific visit
3130 method. If the then-part does not contain any return or throw statements an
3131 \type{IllegalStatementFoundException} is thrown. If it does contain a return or
3132 throw, its else-part is checked. If the else-part is non-existent, or it does
3133 not contain any return or throw statements an exception is thrown. If no
3134 exception is thrown while visiting the if-statement, its children are visited.
3136 A try-statement is checked very similar to an if-statement. Its body must
3137 contain a return or throw. The same applies to its catch clauses and finally
3138 body. Failure to validate produces an \type{IllegalStatementFoundException}.
3140 If the checker does not complain at any point, the selection is considered valid
3141 with respect to return statements.
3143 \subsection{The AmbiguousReturnValueChecker}
3144 This checker verifies that there are no ambiguous return values in a selection.
3146 First, the checker needs to collect some data. Those data are the binding keys
3147 for all simple names that are assigned to within the selection, including
3148 variable declarations, but excluding fields. The checker also collects whether
3149 there exists a return statement in the selection or not. No further checks of
3150 return statements are needed, since, at this point, the selection is already
3151 checked for illegal return statements \see{returnStatementsChecker}.
3153 After the binding keys of the assignees are collected, the checker searches the
3154 part of the enclosing method that is after the selection for references whose
3155 binding keys are among the collected keys. If more than one unique referral is
3156 found, or only one referral is found, but the selection also contains a return
3157 statement, we have a situation with an ambiguous return value, and an exception
3160 %\todoin{Explain why we do not need to consider variables assigned inside
3161 %local/anonymous classes. (The referenced variables need to be final and so
3164 \subsection{The IllegalStatementsChecker}
3165 This checker is designed to check for illegal statements.
3167 Notice that labels in break and continue statements needs some special
3168 treatment. Since a label does not have any binding information, we have to
3169 search upwards in the AST to find the \type{LabeledStatement} that corresponds
3170 to the label from the break or continue statement, and check that it is
3171 contained in the selection. If the break or continue statement does not have a
3172 label attached to it, it is checked that its innermost enclosing loop or switch
3173 statement (break statements only) also is contained in the selection.
3175 \todoin{Follow the development in the semantics section\ldots}
3177 \chapter{Case Studies}
3179 In this chapter I am going to present a few case studies. This is done to give
3180 an impression of how the \ExtractAndMoveMethod refactoring performs when giving
3181 it a larger project to take on. I will try to answer where it lacks, in terms of
3182 completeness, as well as showing its effect on refactored source code.
3184 The first and primary case, is refactoring source code from the \name{Eclipse
3185 JDT UI} project. The project is chosen because it is a real project, still in
3186 development, with a large code base that is written by many different people
3187 through several years. The code is installed in thousands of \name{Eclipse}
3188 applications worldwide, and must be seen as a good representative for
3189 professionally written Java source code. It is also the home for most of the JDT
3192 For the second case, the \ExtractAndMoveMethod refactoring is fed the
3193 \code{no.uio.ifi.refaktor} project. This is done as a variation of the
3194 ``dogfooding'' methodology, where you use your own tools to do your job, also
3195 referred to as ``eating your own dog
3196 food''\citing{harrisonDogfooding2006}.
3199 For conducting these experiments, three tools are used. Two of the ``tools''
3200 both uses Eclipse as their platform. The first is our own tool,
3201 written to be able to run the \ExtractAndMoveMethod refactoring as a batch
3202 prosess, analyzing and refactoring many methods after each other. The second is
3203 JUnit, that is used for running the projects own unit tests on the target code
3204 both before and after it is refactored. The last tool that is used is a code
3205 quality management tool, called \name{SonarQube}. It can be used to perform
3206 different tasks for assuring code quality, but we are only going to take
3207 advantage of one of its main features, namely Quality profiles.
3209 A quality profile is used to define a set of coding rules that a project is
3210 supposed to comply with. Failure to following these rules will be recorded as
3211 so-called ``issues'', marked as having one of several degrees of severities,
3212 ranging from ``info'' to ``blocker'', where the latter one is the most severe.
3213 The measurements done for these case studies are therefore not presented as
3214 fine-grained software metrics results, but rather as the number of issues for
3217 In addition to the coding rules defined through quality profiles, \name{SonarQube}
3218 calculates the complexity of source code. The metric that is used is cyclomatic
3219 complexity, developed by Thomas J. McCabe in
3220 1976\citing{mccabeComplexity1976}. In this metric, functions have an initial
3221 complexity of 1, and whenever the control flow of a function splits, the
3222 complexity increases by
3223 one\footnote{\url{http://docs.codehaus.org/display/SONAR/Metric+definitions}}.
3224 \name{SonarQube} discriminates between functions and accessors. Accessors
3225 are methods that are recognized as setters or getters. Accessors are not counted
3226 in the complexity analysis.
3228 \section{The \name{SonarQube} quality profile}
3229 The quality profile that is used with \name{SonarQube} in these case studies has got
3230 the name \name{IFI Refaktor Case Study} (version 6). The rules defined in the
3231 profile are chosen because they are the available rules found in \name{SonarQube} that
3232 measures complexity and coupling. Now follows a description of the rules in the
3233 quality profile. The values that are set for these rules are listed in
3234 \myref{tab:qualityProfile1}.
3237 \item[Avoid too complex class] is a rule that measures cyclomatic complexity
3238 for every statement in the body of a class, except for setters and getter.
3239 The threshold value set is its default value of 200.
3241 \item[Classes should not be coupled to too many other classes ] is a rule that
3242 measures how many other classes a class depends upon. It does not count the
3243 dependencies of nested classes. It is meant to promote the Single
3244 Responsibility Principle. Although not explicitly stated, the rule's metric
3245 resembles the \metr{Coupling between object classes} (CBO) metric that is
3246 described by Chidamber and Kemerer in their article \tit{A Metrics Suite for
3247 Object Oriented Design}\citing{metricsSuite1994}. The max value for the rule
3248 is chosen on the background of an empirical study by Raed Shatnawi, that
3249 concludes that the number 9 is the most useful threshold for the CBO
3250 metric\citing{shatnawiQuantitative2010}. This study is also performed on
3251 Eclipse source code, so this threshold value should be particularly well
3252 suited for the Eclipse JDT UI case in this chapter.
3254 \item[Control flow statements \ldots{} should not be nested too deeply] is
3255 a rule that is meant to counter ``Spaghetti code''. It measures the nesting
3256 level of if, for, while, switch and try statements. The nesting levels start
3257 at 1. The max value set is its default value of 3.
3259 \item[Methods should not be too complex] is a rule that measures cyclomatic
3260 complexity the same way as the ``Avoid too complex class'' rule. The max
3261 value used is 10, which ``seems like a reasonable, but not magical, upper
3262 limit``\citing{mccabeComplexity1976}.
3264 \item[Methods should not have too many lines] is a rule that simply measures
3265 the number of lines in methods. The threshold value of 20 is used for this
3266 metric. This is based on my own subjective opinions, as the default value of
3267 100 seems a bit too loose.
3269 \item[NPath Complexity] is a rule that measures the number of possible
3270 execution paths through a function. The value used is the default value of
3271 200, that seems like a recognized threshold for this metric.
3273 \item[Too many methods] is a rule that measures the number of methods in a
3274 class. The threshold value used is the default value of 10.
3280 \caption{The \name{IFI Refaktor Case Study} quality profile (version 6).}
3281 \label{tab:qualityProfile1}
3283 \begin{tabularx}{\textwidth}{@{}>{\bfseries}L{1.5}R{0.5}@{}}
3285 \textbf{Rule} & \textbf{Max value} \\
3287 Avoid too complex class & 200 \\
3288 Classes should not be coupled to too many other classes (Single
3289 Responsibility Principle) & 9 \\
3290 Control flow statements \ldots{} should not be nested too deeply &
3292 Methods should not be too complex & 10 \\
3293 Methods should not have too many lines & 20 \\
3294 NPath Complexity & 200 \\
3295 Too many methods & 10 \\
3302 A precondition for the source code that is going to be the target for a series
3303 of \ExtractAndMoveMethod refactorings, is that it is organized as an Eclipse
3304 project. It is also assumed that the code is free from compilation errors.
3306 \section{The experiment}
3307 For a given project, the first job that is done, is to refactor its source code.
3308 The refactoring batch job produces three things: The refactored project,
3309 statistics gathered during the execution of the series of refactorings, and an
3310 error log describing any errors happening during this execution. See
3311 \myref{sec:benchmarking} for more information about how the refactorings are
3314 After the refactoring process is done, the before- and after-code is analyzed
3315 with \name{SonarQube}. The analysis results are then stored in a database and
3316 displayed through a \name{SonarQube} server with a web interface.\todoin{How
3317 long are these results going to be publicly available?}
3319 The before- and after-code is also tested with their own unit tests. This is
3320 done to discover any changes in the semantic behavior of the refactored code,
3321 within the limits of these tests.
3323 \section{Case 1: The Eclipse JDT UI project}
3324 This case is the ultimate test for our \ExtractAndMoveMethod refactoring. The
3325 target sorce code is massive. With its over 300,000 lines of code and over
3326 25,000 methods, it is formidable task to perform automated changes on it. There
3327 should be plenty of situations where things can go wrong, and, as we shall se
3330 I will start by presenting some statistics from the refactoring execution,
3331 before I pick apart the \name{SonarQube} analysis and conclude by commenting on
3332 the results from the unit tests. The configuration for the experiment is
3333 specified in \myref{tab:configurationCase1}.
3336 \caption{Configuration for Case 1.}
3337 \label{tab:configurationCase1}
3339 \begin{tabularx}{\textwidth}{@{}>{\bfseries}L{0.67}L{1.33}@{}}
3341 \spancols{2}{Benchmark data} \\
3343 Launch configuration & CaseStudy.launch \\
3344 Project & no.uio.ifi.refaktor.benchmark \\
3345 Repository & gitolite@git.uio.no:ifi-stolz-refaktor \\
3346 Commit & 43c16c04520746edd75f8dc2a1935781d3d9de6c \\
3348 \spancols{2}{Input data} \\
3350 Project & org.eclipse.jdt.ui \\
3351 Repository & git://git.eclipse.org/gitroot/jdt/eclipse.jdt.ui.git \\
3352 Commit & f218388fea6d4ec1da7ce22432726c244888bb6b \\
3353 Branch & R3\_8\_maintenance \\
3354 Tests suites & org.eclipse.jdt.ui.tests.AutomatedSuite,
3355 org.eclipse.jdt.ui.tests.refactoring.all.\-AllAllRefactoringTests \\
3360 \subsection{Statistics}
3361 The statistics gathered during the refactoring execution is presented in
3362 \myref{tab:case1Statistics}.
3365 \caption{Statistics after batch refactoring the Eclipse JDT UI project with
3366 the \ExtractAndMoveMethod refactoring.}
3367 \label{tab:case1Statistics}
3369 \begin{tabularx}{\textwidth}{@{}>{\bfseries}L{1.5}R{0.5}@{}}
3371 \spancols{2}{Time used} \\
3373 Total time & 98m38s \\
3374 Analysis time & 14m41s (15\%) \\
3375 Change time & 74m20s (75\%) \\
3376 Miscellaneous tasks & 9m37s (10\%) \\
3378 \spancols{2}{Numbers of each type of entity analyzed} \\
3381 Compilation units & 2,097 \\
3384 Text selections & 591,500 \\
3386 \spancols{2}{Numbers for \ExtractAndMoveMethod refactoring candidates} \\
3388 Methods chosen as candidates & 2,552 \\
3389 Methods NOT chosen as candidates & 25,115 \\
3390 Candidate selections (multiple per method) & 36,843 \\
3392 \spancols{2}{\ExtractAndMoveMethod refactorings executed} \\
3394 Fully executed & 2,469 \\
3395 Not fully executed & 83 \\
3396 Total attempts & 2,552 \\
3398 \spancols{2}{Primitive refactorings executed} \\
3399 \spancols{2}{\small \ExtractMethod refactorings} \\
3401 Performed & 2,483 \\
3402 Not performed & 69 \\
3403 Total attempts & 2,552 \\
3405 \spancols{2}{\small \MoveMethod refactorings} \\
3408 Not performed & 14 \\
3409 Total attempts & 2,483 \\
3415 \subsubsection{Execution time}
3416 I consider the total exection time of approximately 1.5 hours as being
3417 acceptable. It clearly makes the batch process unsuitable for doing any
3418 on-demand analysis or changes, but it is good enough for running periodic jobs,
3419 like over-night analysis.
3421 As the statistics show, 75\% of the total time goes into making the actual code
3422 changes. The time consumers are here the primitive \ExtractMethod and
3423 \MoveMethod refactorings. Included in the change time is the parsing and
3424 precondition checking done by the refactorings, as well as textual changes done
3425 to files on disk. All this parsing and disk access is time-consuming, and
3426 constitute a large part of the change time.
3428 In comparison, the pure analysis time, used to find suitable candidates, only
3429 make up for 15\% of the total time consumed. This includes analyzing almost
3430 600,000 text selections, while the number of attempted executions of the
3431 \ExtractAndMoveMethod refactoring are only about 2,500. So the number of
3432 executed primitive refactorings are approximately 5,000. Assuming the time used
3433 on miscellaneous tasks are used mostly for parsing source code for the analysis,
3434 we can say that the time used for analyzing code is at most 25\% of the total
3435 time. This means that for every primitive refactoring executed, we can analyze
3436 around 360 text selections. So, with an average of about 21 text selections per
3437 method, it is reasonable to say that we can analyze over 15 methods in the time
3438 it takes to perform a primitive refactoring.
3440 \subsubsection{Refactoring candidates}
3441 Out of the 27,667 methods that was analyzed, 2,552 methods contained selections
3442 that was considered candidates for the \ExtractAndMoveMethod refactoring. This
3443 is roughly 9\% off the methods in the project. These 9\% of the methods had on
3444 average 14.4 text selections that was considered considered possible refactoring
3447 \subsubsection{Executed refactorings}
3448 2,469 out of 2,552 attempts on executing the \ExtractAndMoveMethod refactoring
3449 was successful, giving a success rate of 96.7\%. The failure rate of 3.3\% stem
3450 from situations where the analysis finds a candidate selection, but the change
3451 execution fails. This failure could be an exception that was thrown, and the
3452 refactoring aborts. It could also be the precondition checking for one of the
3453 primitive refactorings that gives us an error status, meaning that if the
3454 refactoring proceeds, the code will contain compilation errors afterwards,
3455 forcing the composite refactoring to abort. This means that if the
3456 \ExtractMethod refactoring fails, no attempt is done for the \MoveMethod
3457 refactoring. \todo{Redundant information? Put in benchmark chapter?}
3459 Out of the 2,552 \ExtractMethod refactorings that was attempted executed, 69 of
3460 them failed. This give a failure rate of 2.7\% for the primitive refactoring. In
3461 comparison, the \MoveMethod refactoring had a failure rate of 0.6 \% of the
3462 2,483 attempts on the refactoring.
3464 \subsection{\name{SonarQube} analysis}
3465 Results from the \name{SonarQube} analysis is shown in
3466 \myref{tab:case1ResultsProfile1}.
3469 \caption{Results for analyzing the Eclipse JDT UI project, before and after
3470 the refactoring, with \name{SonarQube} and the \name{IFI Refaktor Case Study}
3471 quality profile. (Bold numbers are better.)}
3472 \label{tab:case1ResultsProfile1}
3474 \begin{tabularx}{\textwidth}{@{}>{\bfseries}L{1.5}R{0.25}R{0.25}@{}}
3476 \textnormal{Number of issues for each rule} & Before & After \\
3478 Avoid too complex class & 81 & \textbf{79} \\
3479 Classes should not be coupled to too many other classes (Single
3480 Responsibility Principle) & \textbf{1,098} & 1,199 \\
3481 Control flow statements \ldots{} should not be nested too deeply & 1,375 &
3483 Methods should not be too complex & 1,518 & \textbf{1,452} \\
3484 Methods should not have too many lines & 3,396 & \textbf{3,291} \\
3485 NPath Complexity & 348 & \textbf{329} \\
3486 Too many methods & \textbf{454} & 520 \\
3488 Total number of issues & 8,270 & \textbf{8,155} \\
3491 \spancols{3}{Complexity} \\
3493 Per function & 3.6 & \textbf{3.3} \\
3494 Per class & \textbf{29.5} & 30.4 \\
3495 Per file & \textbf{44.0} & 45.3 \\
3497 Total complexity & \textbf{84,765} & 87,257 \\
3500 \spancols{3}{Numbers of each type of entity analyzed} \\
3502 Files & 1,926 & 1,926 \\
3503 Classes & 2,875 & 2,875 \\
3504 Functions & 23,744 & 26,332 \\
3505 Accessors & 1,296 & 1,019 \\
3506 Statements & 162,768 & 165,145 \\
3507 Lines of code & 320,941 & 329,112 \\
3509 Technical debt (in days) & \textbf{1,003.4} & 1,032.7 \\
3514 \subsubsection{Diversity in the number of entities analyzed}
3515 The analysis performed by \name{SonarCube} is reporting fewer methods than found
3516 by the pre-refactoring analysis. \name{SonarQube} discriminates between
3517 functions (methods) and accessors, so the 1,296 accessors play a part in this
3518 calculation. \name{SonarQube} also has the same definition as our plugin when
3519 it comes to how a class is defined. Therefore is seems like \name{SonarQube}
3520 misses 277 classes that our plugin handles. This can explain why the {SonarQube}
3521 report differs from our numbers by approximately 2,500 methods,
3523 \subsubsection{Complexity}
3524 On all complexity rules that works on the method level, the number of issues
3525 decreases with between 3.1\% and 6.5\% from before to after the refactoring. The
3526 average complexity of a method decreases from 3.6 to 3.3, which is an
3527 improvement of about 8.3\%. So, on the method level, the refactoring must be
3528 said to have a slightly positive impact.
3530 The improvement in complexity on the method level is somewhat traded for
3531 complexity on the class level. The complexity per class metric is worsen by 3\%
3532 from before to after. The issues for the ``Too many methods'' rule also
3533 increases by 14.5\%. These numbers indicate that the refactoring makes quite a
3534 lot of the classes a little more complex overall. This is the expected outcome,
3535 since the \ExtractAndMoveMethod refactoring introduces almost 2,500 new methods
3538 The only number that can save the refactoring's impact on complexity on the
3539 class level, is the ``Avoid too complex class'' rule. It improves with 2.5\%,
3540 thus indicating that the complexity is moderately better distributed between the
3541 classes after the refactoring than before.
3543 \subsubsection{Coupling}
3544 One of the hopes when starting this project, was to be able to make a
3545 refactoring that could lower the coupling between classes. Better complexity at
3546 the method level is a not very unexpected byproduct of dividing methods into
3547 smaller parts. Lowering the coupling on the other hand, is a far greater task.
3548 This is also reflected in the results for the only coupling rule defined in the
3549 \name{SonarQube} quality profile, namely the ``Classes should not be coupled to
3551 other classes (Single Responsibility Principle)'' rule.
3553 The number of issues for the coupling rule is 1,098 before the refactoring, and
3554 1,199 afterwards. This is an increase in issues of 9.2\%, and a blow for this
3555 project. These numbers can be interpreted two ways. The first possibility is
3556 that our assumptions are wrong, and that increasing indirection does not
3557 decrease coupling between classes. The other possibility is that our analysis
3558 and choices of candidate text selections are not good enough. I vote for the
3559 second possibility. (Voting againts the public opinion may also be a little
3562 What probably happens is, that many of the times the \ExtractAndMoveMethod
3563 refactoring is performed, the \MoveMethod refactoring ``drags'' with it
3564 references to classes that are unknown to the method destination. If it happens
3565 to be so lucky that it removes a dependency from one class, it might as well
3566 introduce three new dependencies to another class. In those situations that a
3567 class does not know about the originating class of a moved method, the
3568 \MoveMethod refactroing most certainly will introduce a dependency. This is
3570 bug\todoin{https://bugs.eclipse.org/bugs/show\_bug.cgi?id=228635} in the
3571 refactoring, making it pass an instance of the originating class as a reference
3572 to the moved method, regardless of whether the reference is used in the method
3575 There is also the possibility that the heuristics used to find candidate text
3576 selections are not good enough, they most certainly are not. I wish I had more
3577 time to fine-tune them, and to complete the analysis part of the project, but
3578 this is simply not the case. This becomes even clearer when analyzing the unit
3579 test results for the after-code.
3581 \subsubsection{Totals}
3582 On the bright side, the total number of issues is lower after the refactoring
3583 than it was before. Before the refactoring, the total number of issues is
3584 8,270, and after it is 8,155. An improvement of only 1.4\%.
3586 Then \name{SonarQube} tells me that the total complexity has increased by
3587 2.9\%, and that the (more questionable) ``technical debt'' has increased from
3588 1,003.4 to 1,032.7 days, also a deterioration of 2.9\%. Although these numbers
3589 are similar, no correlation has been found between them.
3591 \subsection{Unit tests}
3592 The tests that have been run for the \name{Eclipse JDT UI} project, are the
3593 tests in the test suites specified as the main test suites on the JDT UI wiki
3594 page on how to contribute to the
3595 project\footnote{\url{https://wiki.eclipse.org/JDT\_UI/How\_to\_Contribute\#Unit\_Testing}}.
3597 \subsubsection{Before the refactoring}
3598 Running the tests for the before-code of Eclipse JDT UI yielded 4 errors and 3
3599 failures for the \type{AutomatedSuite} test suite (2,007 test cases), and 2
3601 3 failures for the \type{AllAllRefactoringTests} test suite (3,816 test cases).
3603 \subsubsection{After the refactoring}
3604 The test results for the after-code of the Eclipse JDT UI project is another
3605 story. The reason for this, and this somehow slipped for me until I was going to
3606 run the unit test for the code, is that Eclipse now reports that the project
3607 contains 322 fatal errors, and a lot of errors that probably follows from these.
3608 This is another blow for this master's project.
3610 It has now been shown that the \ExtractAndMoveMethod refactoring, in its current
3611 state, produces code that does not compile. Had these errors originated from
3612 only one bug, it would not have been much of a problem, but this is not the
3613 case. By only looking at some random compilation problems in the refactore code,
3614 I came up with at least four different bugs \todo{write bug reports} that caused
3615 those problems. I then stopped looking for more, since some of the bugs would
3616 take more time to fix than I could justify using on them at this point.
3618 The only thing that can be said in my defence, is that all the compilation
3619 errors could have been avoided if the type of situations that causes them was
3620 properly handled by the primitive refactorings, that again are supported by the
3621 Eclipse JDT UI project. All of the four randomly found bugs that I menitioned
3622 before, are also weaknesses of the \MoveMethod refactoring. If the primitive
3623 refactorings had detected the up-coming errors
3624 in their precondition checking phase, the refactorings would have been aborted,
3625 since this is how the \ExtractAndMoveMethod refactoring handles such situations.
3627 Of course, taking all possible situations into account is an immense task. This
3628 is one of the reasons for the failure. A complete analysis is too big of a task
3629 for this master's project to handle. Looking at it now, this comes as no
3630 surprise, since the task is obviously also too big for the creators of the
3631 primitive \MoveMethod refactoring. This shows that the underlying primitive
3632 refactorings are not complete enough to be fully relied upon for avoiding
3635 Considering all these problems, it is difficult to know how to interpret the
3636 unit test results from after refactoring the Eclipse JDT UI. The
3637 \type{AutomatedSuite} reported 565 errors and 5 failures. Three of the failures
3638 were the same as reported before the refactoring took place, so two of them are
3639 new. For these two cases it is not immediately apparent what makes them behave
3640 differently. The program is so complex that to analyze it to find this out, we
3641 might need more powerful methods than just manually analyzing its source code.
3642 This is somewhat characteristic for imperative programming: The programs are
3643 often hard to analyze and understand.
3645 For the \type{AllAllRefactoringTests} test suite, the three failures are gone,
3646 but the two errors have grown to 2,257 errors. I will not try to analyze those
3649 What I can say, is that it is likely that the \ExtractAndMoveMethod refactoring
3650 has introduced some unintended behavioral changes. Let us say that the
3651 refactoring introduces at least two behavior-altering changes for every 2,500
3652 executions. More than that is difficult to say about the behavior-preserving
3653 properties of the \ExtractAndMoveMethod refactoring, at this point.
3655 \subsection{Conclusions}
3656 After automatically analyzing and executing the \ExtractAndMoveMethod
3657 refactoring for all the methods in the Eclipse JDT UI project, the results does
3658 not look that promising. For this case, the refactoring seems almost unusable as
3659 it is now. The error rate and measurements done tells us this.
3661 The refactoring makes the code a little less complex at the method level. But
3662 this is merely a side effect of extracting methods, and holds little scientific
3663 value. When it comes to the overall complexity, it is increased, although it is
3664 slightly better spread among the classes.
3666 The analysis done before the \ExtractAndMoveMethod refactoring, is currently not
3667 complete enough to make the refactoring useful. It introduces too many errors in
3668 the code, and the code may change it's behavior. It also remains to prove that
3669 large scale refactoring with it can decrease coupling between classes. A better
3670 analysis may prove this, but in its present state, the opposite is the fact. The
3671 coupling measurements done by \name{SonarQube} shows this.
3673 On the bright side, the performance of the refactoring process is not that bad.
3674 It shows that it is possible to make a tool the way we do, if we can make the
3675 tool do anything useful. As long as the analysis phase is not going to involve
3676 anything that uses to much disk access, a lot of analysis can be done in a
3677 reasonable amount of time.
3679 The time used on performing the actual changes excludes a trial and error
3680 approach with the tools used in this master's project. In a trial and error
3681 approach, you could for instance be using the primitive refactorings used in
3682 this project to refactor code, and only then make decisions based on the effect,
3683 possibly shown by traditional software metrics. The problem with the approach
3684 taken in this project, compared to a trial and error approach, is that using
3685 heuristics beforehand is much more complicated. But on the other hand, a trial
3686 and error approach would still need to face the challenges of producing code
3687 that does compile without errors. If using refactorings that could produce
3688 in-memory changes, a trial and error approach could be made more efficient.
3689 \section{Case 2: Dogfooding}
3690 The configuration for the experiment is specified in
3691 \myref{tab:configurationCase2}.
3694 \caption{Configuration for Case 2.}
3695 \label{tab:configurationCase2}
3697 \begin{tabularx}{\textwidth}{@{}>{\bfseries}L{0.67}L{1.33}@{}}
3699 \spancols{2}{Benchmark data} \\
3701 Launch configuration & CaseStudyDogfooding.launch \\
3702 Project & no.uio.ifi.refaktor.benchmark \\
3703 Repository & gitolite@git.uio.no:ifi-stolz-refaktor \\
3704 Commit & 43c16c04520746edd75f8dc2a1935781d3d9de6c \\
3706 \spancols{2}{Input data} \\
3708 Project & no.uio.ifi.refaktor \\
3709 Repository & gitolite@git.uio.no:ifi-stolz-refaktor \\
3710 Commit & 43c16c04520746edd75f8dc2a1935781d3d9de6c \\
3712 Test configration & no.uio.ifi.refaktor.tests/ExtractTest.launch \\
3716 \subsection{Statistics}
3717 The statistics gathered during the refactoring execution is presented in
3718 \myref{tab:case2Statistics}.
3721 \caption{Statistics after batch refactoring the \type{no.uio.ifi.refaktor}
3722 project with the \ExtractAndMoveMethod refactoring.}
3723 \label{tab:case2Statistics}
3725 \begin{tabularx}{\textwidth}{@{}>{\bfseries}L{1.5}R{0.5}@{}}
3727 \spancols{2}{Time used} \\
3729 Total time & 1m15s \\
3730 Analysis time & 0m18s (24\%) \\
3731 Change time & 0m47s (63\%) \\
3732 Miscellaneous tasks & 0m10s (14\%) \\
3734 \spancols{2}{Numbers of each type of entity analyzed} \\
3737 Compilation units & 154 \\
3740 Text selections & 8,609 \\
3742 \spancols{2}{Numbers for \ExtractAndMoveMethod refactoring candidates} \\
3744 Methods chosen as candidates & 58 \\
3745 Methods NOT chosen as candidates & 1,012 \\
3746 Candidate selections (multiple per method) & 227 \\
3748 \spancols{2}{\ExtractAndMoveMethod refactorings executed} \\
3750 Fully executed & 53 \\
3751 Not fully executed & 5 \\
3752 Total attempts & 58 \\
3754 \spancols{2}{Primitive refactorings executed} \\
3755 \spancols{2}{\small \ExtractMethod refactorings} \\
3758 Not performed & 2 \\
3759 Total attempts & 58 \\
3761 \spancols{2}{\small \MoveMethod refactorings} \\
3764 Not performed & 3 \\
3765 Total attempts & 56 \\
3771 \subsubsection{Execution time}
3773 \subsubsection{Refactoring candidates}
3775 \subsubsection{Executed refactorings}
3777 \subsection{\name{SonarQube} analysis}
3778 Results from the \name{SonarQube} analysis is shown in
3779 \myref{tab:case2ResultsProfile1}.
3782 \caption{Results for analyzing the \var{no.uio.ifi.refaktor} project, before
3783 and after the refactoring, with \name{SonarQube} and the \name{IFI Refaktor
3784 Case Study} quality profile. (Bold numbers are better.)}
3785 \label{tab:case2ResultsProfile1}
3787 \begin{tabularx}{\textwidth}{@{}>{\bfseries}L{1.5}R{0.25}R{0.25}@{}}
3789 \textnormal{Number of issues for each rule} & Before & After \\
3791 Avoid too complex class & 1 & 1 \\
3792 Classes should not be coupled to too many other classes (Single
3793 Responsibility Principle) & \textbf{29} & 34 \\
3794 Control flow statements \ldots{} should not be nested too deeply & 24 &
3796 Methods should not be too complex & 17 & \textbf{15} \\
3797 Methods should not have too many lines & 41 & \textbf{40} \\
3798 NPath Complexity & 3 & 3 \\
3799 Too many methods & \textbf{13} & 15 \\
3801 Total number of issues & \textbf{128} & 129 \\
3804 \spancols{3}{Complexity} \\
3806 Per function & 2.1 & 2.1 \\
3807 Per class & \textbf{12.5} & 12.9 \\
3808 Per file & \textbf{13.8} & 14.2 \\
3810 Total complexity & \textbf{2,089} & 2,148 \\
3813 \spancols{3}{Numbers of each type of entity analyzed} \\
3815 Files & 151 & 151 \\
3816 Classes & 167 & 167 \\
3817 Functions & 987 & 1,045 \\
3818 Accessors & 35 & 30 \\
3819 Statements & 3,355 & 3,416 \\
3820 Lines of code & 11,452 & 11,730 \\
3822 Technical debt (in days) & \textbf{19.0} & 20.7 \\
3827 \subsubsection{Diversity in the number of entities analyzed}
3829 \subsubsection{Complexity}
3831 \subsubsection{Coupling}
3833 \subsubsection{Totals}
3835 \subsection{Unit tests}
3837 \subsubsection{Before the refactoring}
3839 \subsubsection{After the refactoring}
3841 \subsection{Conclusions}
3843 \chapter{Benchmarking}\label{sec:benchmarking}
3844 \todoin{Better name than ``benchmarking''?}
3845 This part of the master's project is located in the \name{Eclipse} project
3846 \code{no.uio.ifi.refaktor.benchmark}. The purpose of it is to run the equivalent
3847 of the \type{SearchBasedExtractAndMoveMethodChanger}
3848 \see{searchBasedExtractAndMoveMethodChanger} over a larger software project,
3849 both to test its robustness but also its effect on different software metrics.
3851 \section{The benchmark setup}
3852 The benchmark itself is set up as a \name{JUnit} test case. This is a convenient
3853 setup, and utilizes the \name{JUnit Plugin Test Launcher}. This provides us with
3854 a fully functional \name{Eclipse} workbench. Most importantly, this gives us
3855 access to the Java Model of \name{Eclipse} \see{javaModel}.
3857 \subsection{The ProjectImporter}
3858 The Java project that is going to be used as the data for the benchmark, must be
3859 imported into the JUnit workspace. This is done by the
3860 \typewithref{no.uio.ifi.refaktor.benchmark}{ProjectImporter}. The importer
3861 require the absolute path to the project description file. It is named
3862 \code{.project} and is located at the root of the project directory.
3864 The project description is loaded to find the name of the project to be
3865 imported. The project that shall be the destination for the import is created in
3866 the workspace, on the base of the name from the description. Then an import
3867 operation is created, based on both the source and destination information. The
3868 import operation is run to perform the import.
3870 I have found no simple API call to accomplish what the importer does, which
3871 tells me that it may not be too many people performing this particular action.
3872 The solution to the problem was found on \name{Stack
3873 Overflow}\footnote{\url{https://stackoverflow.com/questions/12401297}}. It
3874 contains enough dirty details to be considered inconvenient to use, if not
3875 wrapping it in a class like my \type{ProjectImporter}. One would probably have
3876 to delve into the source code for the import wizard to find out how the import
3877 operation works, if no one had already done it.
3879 \section{Statistics}
3880 Statistics for the analysis and changes is captured by the
3881 \typewithref{no.uio.ifi.refaktor.aspects}{StatisticsAspect}. This an
3882 \emph{aspect} written in \name{AspectJ}.
3884 \subsection{AspectJ}
3885 \name{AspectJ}\footnote{\url{http://eclipse.org/aspectj/}} is an extension to
3886 the Java language, and facilitates combining aspect-oriented programming with
3887 the object-oriented programming in Java.
3889 Aspect-oriented programming is a programming paradigm that is meant to isolate
3890 so-called \emph{cross-cutting concerns} into their own modules. These
3891 cross-cutting concerns are functionalities that spans over multiple classes, but
3892 may not belong naturally in any of them. It can be functionality that does not
3893 concern the business logic of an application, and thus may be a burden when
3894 entangled with parts of the source code it does not really belong. Examples
3895 include logging, debugging, optimization and security.
3897 Aspects are interacting with other modules by defining advices. The concept of
3898 an \emph{advice} is known from both aspect-oriented and functional
3899 programming\citing{wikiAdvice2014}. It is a function that modifies another
3900 function when the latter is run. An advice in AspectJ is somewhat similar to a
3901 method in Java. It is meant to alter the behavior of other methods, and contains
3902 a body that is executed when it is applied.
3904 An advice can be applied at a defined \emph{pointcut}. A pointcut picks out one
3905 or more \emph{join points}. A join point is a well-defined point in the
3906 execution of a program. It can occur when calling a method defined for a
3907 particular class, when calling all methods with the same name,
3908 accessing/assigning to a particular field of a given class and so on. An advice
3909 can be declared to run both before, after returning from a pointcut, when there
3910 is thrown an exception in the pointcut or after the pointcut either returns or
3911 throws an exception. In addition to picking out join points, a pointcut can
3912 also bind variables from its context, so they can be accessed in the body of an
3913 advice. An example of a pointcut and an advice is found in
3914 \myref{lst:aspectjExample}.
3917 \begin{minted}{aspectj}
3918 pointcut methodAnalyze(
3919 SearchBasedExtractAndMoveMethodAnalyzer analyzer) :
3920 call(* SearchBasedExtractAndMoveMethodAnalyzer.analyze())
3921 && target(analyzer);
3923 after(SearchBasedExtractAndMoveMethodAnalyzer analyzer) :
3924 methodAnalyze(analyzer) {
3925 statistics.methodCount++;
3926 debugPrintMethodAnalysisProgress(analyzer.method);
3929 \caption{An example of a pointcut named \method{methodAnalyze},
3930 and an advice defined to be applied after it has occurred.}
3931 \label{lst:aspectjExample}
3934 \subsection{The Statistics class}
3935 The statistics aspect stores statistical information in an object of type
3936 \type{Statistics}. As of now, the aspect needs to be initialized at the point in
3937 time where it is desired that it starts its data gathering. At any point in time
3938 the statistics aspect can be queried for a snapshot of the current statistics.
3940 The \type{Statistics} class also include functionality for generating a report
3941 of its gathered statistics. The report can be given either as a string or it can
3942 be written to a file.
3944 \subsection{Advices}
3945 The statistics aspect contains advices for gathering statistical data from
3946 different parts of the benchmarking process. It captures statistics from both
3947 the analysis part and the execution part of the composite \ExtractAndMoveMethod
3950 For the analysis part, there are advices to count the number of text selections
3951 analyzed and the number of methods, types, compilation units and packages
3952 analyzed. There are also advices that counts for how many of the methods there
3953 is found a selection that is a candidate for the refactoring, and for how many
3954 methods there is not.
3956 There exists advices for counting both the successful and unsuccessful
3957 executions of all the refactorings. Both for the \ExtractMethod and \MoveMethod
3958 refactorings in isolation, as well as for the combination of them.
3960 \section{Optimizations}
3961 When looking for optimizations to make for the benchmarking process, I used the
3962 \name{VisualVM}\footnote{\url{http://visualvm.java.net/}} \gloss{profiler} for
3963 the Java Virtual Machine to both profile the application and also to make memory
3966 \subsection{Caching}
3967 When \gloss{profiling} the benchmark process before making any optimizations, it
3968 early became apparent that the parsing of source code was a place to direct
3969 attention towards. This discovery was done when only \emph{analyzing} source
3970 code, before trying to do any \emph{manipulation} of it. Caching of the parsed
3971 ASTs seemed like the best way to save some time, as expected. With only a simple
3972 cache of the most recently used AST, the analysis time was speeded up by a
3973 factor of around 20. This number depends a little upon which type of system the
3976 The caching is managed by a cache manager, that now, by default, utilizes the
3977 not so well known feature of Java called a \emph{soft reference}. Soft
3978 references are best explained in the context of weak references. A \emph{weak
3979 reference} is a reference to an object instance that is only guaranteed to
3980 persist as long as there is a \emph{strong reference} or a soft reference
3981 referring the same object. If no such reference is found, its referred object is
3982 garbage collected. A strong reference is basically the same as a regular Java
3983 reference. A soft reference has the same guarantees as a week reference when it
3984 comes to its relation to strong references, but it is not necessarily garbage
3985 collected whenever there exists no strong references to it. A soft reference
3986 \emph{may} reside in memory as long as the JVM has enough free memory in the
3987 heap. A soft reference will therefore usually perform better than a weak
3988 reference when used for simple caching and similar tasks. The way to use a
3989 soft/weak reference is to as it for its referent. The return value then has to
3990 be tested to check that it is not \var{null}. For the basic usage of soft
3991 references, see \myref{lst:softReferenceExample}. For a more thorough
3992 explanation of weak references in general, see\citing{weakRef2006}.
3995 \begin{minted}{java}
3997 Object strongRef = new Object();
4000 SoftReference<Object> softRef =
4001 new SoftReference<Object>(new Object());
4003 // Using the soft reference
4004 Object obj = softRef.get();
4009 \caption{Showing the basic usage of soft references. Weak references is used the
4010 same way. {\footnotesize (The references are part of the \code{java.lang.ref}
4012 \label{lst:softReferenceExample}
4015 The cache based on soft references has no limit for how many ASTs it caches. It
4016 is generally not advisable to keep references to ASTs for prolonged periods of
4017 time, since they are expensive structures to hold on to. For regular plugin
4018 development, \name{Eclipse} recommends not creating more than one AST at a time to
4019 limit memory consumption. Since the benchmarking has nothing to do with user
4020 experience, and throughput is everything, these advices are intentionally
4021 ignored. This means that during the benchmarking process, the target \name{Eclipse}
4022 application may very well work close to its memory limit for the heap space for
4023 long periods during the benchmark.
4025 \subsection{Candidates stored as mementos}
4026 When performing large scale analysis of source code for finding candidates to
4027 the \ExtractAndMoveMethod refactoring, memory is an issue. One of the inputs to
4028 the refactoring is a variable binding. This variable binding indirectly retains
4029 a whole AST. Since ASTs are large structures, this quickly leads to an
4030 \type{OutOfMemoryError} if trying to analyze a large project without optimizing
4031 how we store the candidates data. This means that the JVM cannot allocate more
4032 memory for out benchmark, and it exists disgracefully.
4034 A possible solution could be to just allow the JVM to allocate even more memory,
4035 but this is not a dependable solution. The allocated memory could easily
4036 supersede the physical memory of a machine, and that would make the benchmark go
4039 Thus, the candidates data must be stored in another format. Therefore, we use
4040 the \gloss{mementoPattern} to store the variable binding information. This is
4041 done in a way that makes it possible to retrieve the variable binding at a later
4042 point. The data that is stored to achieve this, is the key to the original
4043 variable binding. In addition to the key, we know which method and text
4044 selection the variable is referenced in, so that we can find it by parsing the
4045 source code and search for it when it is needed.
4047 \section{Handling failures}
4051 \chapter{Technicalities}
4053 \section{Source code organization}
4054 All the parts of this master's project is under version control with
4055 \name{Git}\footnote{\url{http://git-scm.com/}}.
4057 The software written is organized as some \name{Eclipse} plugins. Writing a plugin is
4058 the natural way to utilize the API of \name{Eclipse}. This also makes it possible to
4059 provide a user interface to manually run operations on selections in program
4060 source code or whole projects/packages.
4062 When writing a plugin in \name{Eclipse}, one has access to resources such as the
4063 current workspace, the open editor and the current selection.
4065 The thesis work is contained in the following Eclipse projects:
4068 \item[no.uio.ifi.refaktor] \hfill \\ This is the main Eclipse plugin
4069 project, and contains all of the business logic for the plugin.
4071 \item[no.uio.ifi.refaktor.tests] \hfill \\
4072 This project contains the tests for the main plugin.
4074 \item[no.uio.ifi.refaktor.examples] \hfill \\
4075 Contains example code used in testing. It also contains code for managing
4076 this example code, such as creating an Eclipse project from it before a test
4079 \item[no.uio.ifi.refaktor.benchmark] \hfill \\
4080 This project contains code for running search based versions of the
4081 composite refactoring over selected Eclipse projects.
4083 \item[no.uio.ifi.refaktor.releng] \hfill \\
4084 Contains the rmap, queries and target definitions needed by by Buckminster
4085 on the Jenkins continuous integration server.
4089 \subsection{The no.uio.ifi.refaktor project}
4091 \subsubsection{no.uio.ifi.refaktor.analyze}
4092 This package, and its subpackages, contains code that is used for analyzing Java
4093 source code. The most important subpackages are presented below.
4096 \item[no.uio.ifi.refaktor.analyze.analyzers] \hfill \\
4097 This package contains source code analyzers. These are usually responsible
4098 for analyzing text selections or running specialized analyzers for different
4099 kinds of entities. Their structure are often hierarchical. This means that
4100 you have an analyzer for text selections, that in turn is utilized by an
4101 analyzer that analyzes all the selections of a method. Then there are
4102 analyzers for analyzing all the methods of a type, all the types of a
4103 compilation unit, all the compilation units of a package, and, at last, all
4104 of the packages in a project.
4106 \item[no.uio.ifi.refaktor.analyze.checkers] \hfill \\
4107 A package containing checkers. The checkers are classes used to validate
4108 that a selection can be further analyzed and chosen as a candidate for a
4109 refactoring. Invalidating properties can be such as usage of inner classes
4110 or the need for multiple return values.
4112 \item[no.uio.ifi.refaktor.analyze.collectors] \hfill \\
4113 This package contains the property collectors. Collectors are used to gather
4114 properties from a text selection. This is mostly properties regarding
4115 referenced names and their occurrences. It is these properties that makes up
4116 the basis for finding the best candidates for a refactoring.
4119 \subsubsection{no.uio.ifi.refaktor.change}
4120 This package, and its subpackages, contains functionality for manipulate source
4124 \item[no.uio.ifi.refaktor.change.changers] \hfill \\
4125 This package contains source code changers. They are used to glue together
4126 the analysis of source code and the actual execution of the changes.
4128 \item[no.uio.ifi.refaktor.change.executors] \hfill \\
4129 The executors that are responsible for making concrete changes are found in
4130 this package. They are mostly used to create and execute one or more Eclipse
4133 \item[no.uio.ifi.refaktor.change.processors] \hfill \\
4134 Contains a refactoring processor for the \MoveMethod refactoring. The code
4135 is stolen and modified to fix a bug. The related bug is described in
4136 \myref{eclipse_bug_429416}.
4140 \subsubsection{no.uio.ifi.refaktor.handlers}
4141 This package contains handlers for the commands defined in the plugin manifest.
4143 \subsubsection{no.uio.ifi.refaktor.prefix}
4144 This package contains the \type{Prefix} type that is the data representation of
4145 the prefixes found by the \type{PrefixesCollector}. It also contains the prefix
4146 set for storing and working with prefixes.
4148 \subsubsection{no.uio.ifi.refaktor.statistics}
4149 The package contains statistics functionality. Its heart is the statistics
4150 aspect that is responsible for gathering statistics during the execution of the
4151 \ExtractAndMoveMethod refactoring.
4154 \item[no.uio.ifi.refaktor.statistics.reports] \hfill \\
4155 This package contains a simple framework for generating reports from the
4156 statistics data generated by the aspect. Currently, the only available
4157 report type is a simple text report.
4162 \subsubsection{no.uio.ifi.refaktor.textselection}
4163 This package contains the two custom text selections that are used extensively
4164 throughout the project. One of them is just a subclass of the other, to support
4165 the use of the memento pattern to optimize the memory usage during benchmarking.
4167 \subsubsection{no.uio.ifi.refaktor.debugging}
4168 The package contains a debug utility class. I addition to this, the package
4169 \code{no.uio.ifi.refaktor.utils.aspects} contains a couple of aspects used for
4172 \subsubsection{no.uio.ifi.refaktor.utils}
4173 Utility package that contains all the functionality that has to do with parsing
4174 of source code. It also has utility classes for looking up handles to methods
4175 and types et cetera.
4178 \item[no.uio.ifi.refaktor.utils.caching] \hfill \\
4179 This package contains the caching manager for compilation units, along with
4180 classes for different caching strategies.
4182 \item[no.uio.ifi.refaktor.utils.nullobjects] \hfill \\
4183 Contains classes for creating different null objects. Most of the classes is
4184 used to represent null objects of different handle types. These null objects
4185 are returned from various utility classes instead of returning a \var{null}
4186 value when other values are not available.
4190 \section{Continuous integration}
4191 The continuous integration server
4192 \name{Jenkins}\footnote{\url{http://jenkins-ci.org/}} has been set up for the
4193 project\footnote{A work mostly done by the supervisor.}. It is used as a way to
4194 run tests and perform code coverage analysis.
4196 To be able to build the \name{Eclipse} plugins and run tests for them with Jenkins, the
4197 component assembly project
4198 \name{Buckminster}\footnote{\url{http://www.eclipse.org/buckminster/}} is used,
4199 through its plugin for Jenkins. Buckminster provides for a way to specify the
4200 resources needed for building a project and where and how to find them.
4201 Buckminster also handles the setup of a target environment to run the tests in.
4202 All this is needed because the code to build depends on an \name{Eclipse}
4203 installation with various plugins.
4205 \subsection{Problems with AspectJ}
4206 The Buckminster build worked fine until introducing AspectJ into the project.
4207 When building projects using AspectJ, there are some additional steps that needs
4208 to be performed. First of all, the aspects themselves must be compiled. Then the
4209 aspects needs to be woven with the classes they affect. This demands a process
4210 that does multiple passes over the source code.
4212 When using AspectJ with \name{Eclipse}, the specialized compilation and the
4213 weaving can be handled by the \name{AspectJ Development
4214 Tools}\footnote{\url{https://www.eclipse.org/ajdt/}}. This works all fine, but
4215 it complicates things when trying to build a project depending on \name{Eclipse}
4216 plugins outside of \name{Eclipse}. There is supposed to be a way to specify a
4217 compiler adapter for javac, together with the file extensions for the file types
4218 it shall operate. The AspectJ compiler adapter is called
4219 \typewithref{org.aspectj.tools.ant.taskdefs}{Ajc11CompilerAdapter}, and it works
4220 with files that has the extensions \code{*.java} and \code{*.aj}. I tried to
4221 setup this in the build properties file for the project containing the aspects,
4222 but to no avail. The project containing the aspects does not seem to be built at
4223 all, and the projects that depends on it complains that they cannot find certain
4226 I then managed to write an \name{Ant}\footnote{\url{https://ant.apache.org/}}
4227 build file that utilizes the AspectJ compiler adapter, for the
4228 \code{no.uio.ifi.refaktor} plugin. The problem was then that it could no longer
4229 take advantage of the environment set up by Buckminster. The solution to this
4230 particular problem was of a ``hacky'' nature. It involves exporting the plugin
4231 dependencies for the project to an Ant build file, and copy the exported path
4232 into the existing build script. But then the Ant script needs to know where the
4233 local \name{Eclipse} installation is located. This is no problem when building
4234 on a local machine, but to utilize the setup done by Buckminster is a problem
4235 still unsolved. To get the classpath for the build setup correctly, and here
4236 comes the most ``hacky'' part of the solution, the Ant script has a target for
4237 copying the classpath elements into a directory relative to the project
4238 directory and checking it into Git. When no \code{ECLIPSE\_HOME} property is set
4239 while running Ant, the script uses the copied plugins instead of the ones
4240 provided by the \name{Eclipse} installation when building the project. This
4241 obviously creates some problems with maintaining the list of dependencies in the
4242 Ant file, as well as remembering to copy the plugins every time the list of
4243 dependencies change.
4245 The Ant script described above is run by Jenkins before the Buckminster setup
4246 and build. When setup like this, the Buckminster build succeeds for the projects
4247 not using AspectJ, and the tests are run as normal. This is all good, but it
4248 feels a little scary, since the reason for Buckminster not working with AspectJ
4251 The problems with building with AspectJ on the Jenkins server lasted for a
4252 while, before they were solved. This is reflected in the ``Test Result Trend''
4253 and ``Code Coverage Trend'' reported by Jenkins.
4256 \chapter{Methodology}
4258 \section{Evolutionary design}
4259 In the programming work for this project, it have tried to use a design strategy
4260 called evolutionary design, also known as continuous or incremental
4261 design\citing{wiki_continuous_2014}. It is a software design strategy
4262 advocated by the Extreme Programming community. The essence of the strategy is
4263 that you should let the design of your program evolve naturally as your
4264 requirements change. This is seen in contrast with up-front design, where
4265 design decisions are made early in the process.
4267 The motivation behind evolutionary design is to keep the design of software as
4268 simple as possible. This means not introducing unneeded functionality into a
4269 program. You should defer introducing flexibility into your software, until it
4270 is needed to be able to add functionality in a clean way.
4272 Holding up design decisions, implies that the time will eventually come when
4273 decisions have to be made. The flexibility of the design then relies on the
4274 programmer's abilities to perform the necessary refactoring, and \his confidence
4275 in those abilities. From my experience working on this project, I can say that
4276 this confidence is greatly enhanced by having automated tests to rely on
4279 The choice of going for evolutionary design developed naturally. As Fowler
4280 points out in his article \tit{Is Design Dead?}, evolutionary design much
4281 resembles the ``code and fix'' development strategy\citing{fowler_design_2004}.
4282 A strategy that most of us have practiced in school. This was also the case when
4283 I first started this work. I had to learn the inner workings of Eclipse and its
4284 refactoring-related plugins. That meant a lot of fumbling around with code I did
4285 not know, in a trial and error fashion. Eventually I started writing tests for
4286 my code, and my design began to evolve.
4288 \section{Test-driven development}\label{tdd}
4289 As mentioned before, the project started out as a classic code and fix
4290 developmen process. My focus was aimed at getting something to work, rather than
4291 doing so according to best practice. This resulted in a project that got out of
4292 its starting blocks, but it was not accompanied by any tests. Hence it was soon
4293 difficult to make any code changes with the confidence that the program was
4294 still correct afterwards (assuming it was so before changing it). I always knew
4295 that I had to introduce some tests at one point, but this experience accelerated
4296 the process of leading me onto the path of testing.
4298 I then wrote tests for the core functionality of the plugin, and thus gained
4299 more confidence in the correctness of my code. I could now perform quite drastic
4300 changes without ``wetting my pants``. After this, nearly all of the semantic
4301 changes done to the business logic of the project, or the addition of new
4302 functionality, was made in a test-driven manner. This means that before
4303 performing any changes, I would define the desired functionality through a set
4304 of tests. I would then run the tests to check that they were run and that they
4305 did not pass. Then I would do any code changes necessary to make the tests
4306 pass. The definition of how the program is supposed to operate is then captured
4307 by the tests. However, this does not prove the correctness of the analysis
4308 leading to the test definitions.
4310 \section{Continuous integration}
4314 \chapter{Eclipse Bugs Found}
4315 \newcommand{\submittedBugReport}[1]{The submitted bug report can be found on
4318 \section{Eclipse bug 420726: Code is broken when moving a method that is
4319 assigning to the parameter that is also the move
4320 destination}\label{eclipse_bug_420726}
4322 was found when analyzing what kinds of names that was to be considered as
4323 \emph{unfixes} \see{unfixes}.
4325 \subsection{The bug}
4326 The bug emerges when trying to move a method from one class to another, and when
4327 the target for the move (must be a variable, local or field) is both a parameter
4328 variable and also is assigned to within the method body. \name{Eclipse} allows this to
4329 happen, although it is the sure path to a compilation error. This is because we
4330 would then have an assignment to a \var{this} expression, which is not allowed
4332 \submittedBugReport{https://bugs.eclipse.org/bugs/show\_bug.cgi?id=420726}
4334 \subsection{The solution}
4335 The solution to this problem is to add all simple names that are assigned to in
4336 a method body to the set of unfixes.
4338 \section{Eclipse bug 429416: IAE when moving method from anonymous
4339 class}\label{eclipse_bug_429416}
4341 this bug during a batch change on the \type{org.eclipse.jdt.ui} project.
4343 \subsection{The bug}
4344 This bug surfaces when trying to use the \refa{Move Method} refactoring to move a
4345 method from an anonymous class to another class. This happens both for my
4346 simulation as well as in \name{Eclipse}, through the user interface. It only occurs
4347 when \name{Eclipse} analyzes the program and finds it necessary to pass an instance of
4348 the originating class as a parameter to the moved method. I.e. it want to pass a
4349 \var{this} expression. The execution ends in an
4350 \typewithref{java.lang}{IllegalArgumentException} in
4351 \typewithref{org.eclipse.jdt.core.dom}{SimpleName} and its
4352 \method{setIdentifier(String)} method. The simple name is attempted created in
4354 \methodwithref{org.eclipse.jdt.internal.corext.refactoring.structure.\\MoveInstanceMethodProcessor}{createInlinedMethodInvocation}
4355 so the \type{MoveInstanceMethodProcessor} was early a clear suspect.
4357 The \method{createInlinedMethodInvocation} is the method that creates a method
4358 invocation where the previous invocation to the method that was moved was. From
4359 its code it can be read that when a \var{this} expression is going to be passed
4360 in to the invocation, it shall be qualified with the name of the original
4361 method's declaring class, if the declaring class is either an anonymous class or
4362 a member class. The problem with this, is that an anonymous class does not have
4363 a name, hence the term \emph{anonymous} class! Therefore, when its name, an
4364 empty string, is passed into
4365 \methodwithref{org.eclipse.jdt.core.dom.AST}{newSimpleName} it all ends in an
4366 \type{IllegalArgumentException}.
4367 \submittedBugReport{https://bugs.eclipse.org/bugs/show\_bug.cgi?id=429416}
4369 \subsection{How I solved the problem}
4370 Since the \type{MoveInstanceMethodProcessor} is instantiated in the
4371 \typewithref{no.uio.ifi.refaktor.change.executors}{MoveMethod\-RefactoringExecutor},
4372 and only need to be a
4373 \typewithref{org.eclipse.ltk.core.refactoring.participants}{MoveProcessor}, I
4374 was able to copy the code for the original move processor and modify it so that
4375 it works better for me. It is now called
4376 \typewithref{no.uio.ifi.refaktor.change.processors}{ModifiedMoveInstanceMethodProcessor}.
4377 The only modification done (in addition to some imports and suppression of
4378 warnings), is in the \method{createInlinedMethodInvocation}. When the declaring
4379 class of the method to move is anonymous, the \var{this} expression in the
4380 parameter list is not qualified with the declaring class' (empty) name.
4382 \section{Eclipse bug 429954: Extracting statement with reference to local type
4383 breaks code}\label{eclipse_bug_429954}
4385 was discovered when doing some changes to the way unfixes is computed.
4387 \subsection{The bug}
4388 The problem is that \name{Eclipse} is allowing selections that references variables of
4389 local types to be extracted. When this happens the code is broken, since the
4390 extracted method must take a parameter of a local type that is not in the
4391 methods scope. The problem is illustrated in
4392 \myref{lst:extractMethod_LocalClass}, but there in another setting.
4393 \submittedBugReport{https://bugs.eclipse.org/bugs/show\_bug.cgi?id=429954}
4395 \subsection{Actions taken}
4396 There are no actions directly springing out of this bug, since the Extract
4397 Method refactoring cannot be meant to be this way. This is handled on the
4398 analysis stage of our \refa{Extract and Move Method} refactoring. So names representing
4399 variables of local types is considered unfixes \see{unfixes}.
4400 \todoin{write more when fixing this in legal statements checker}
4402 \chapter{Conclusions and Future Work}
4405 \section{Future work}
4407 \chapter{Related Work}
4409 \section{Safer refactorings}
4412 \section{The compositional paradigm of refactoring}
4413 This paradigm builds upon the observation of Vakilian et
4414 al.\citing{vakilian2012}, that of the many automated refactorings existing in
4415 modern IDEs, the simplest ones are dominating the usage statistics. The report
4416 mainly focuses on \name{Eclipse} as the tool under investigation.
4418 The paradigm is described almost as the opposite of automated composition of
4419 refactorings \see{compositeRefactorings}. It works by providing the programmer
4420 with easily accessible primitive refactorings. These refactorings shall be
4421 accessed via keyboard shortcuts or quick-assist menus\footnote{Think
4422 quick-assist with Ctrl+1 in \name{Eclipse}} and be promptly executed, opposed to in the
4423 currently dominating wizard-based refactoring paradigm. They are meant to
4424 stimulate composing smaller refactorings into more complex changes, rather than
4425 doing a large upfront configuration of a wizard-based refactoring, before
4426 previewing and executing it. The compositional paradigm of refactoring is
4427 supposed to give control back to the programmer, by supporting \himher with an
4428 option of performing small rapid changes instead of large changes with a lesser
4429 degree of control. The report authors hope this will lead to fewer unsuccessful
4430 refactorings. It also could lower the bar for understanding the steps of a
4431 larger composite refactoring and thus also help in figuring out what goes wrong
4432 if one should choose to op in on a wizard-based refactoring.
4434 Vakilian and his associates have performed a survey of the effectiveness of the
4435 compositional paradigm versus the wizard-based one. They claim to have found
4436 evidence of that the \emph{compositional paradigm} outperforms the
4437 \emph{wizard-based}. It does so by reducing automation, which seem
4438 counterintuitive. Therefore they ask the question ``What is an appropriate level
4439 of automation?'', and thus questions what they feel is a rush toward more
4440 automation in the software engineering community.