+\section{The input}
+A precondition for the source code that is going to be the target for a series
+of \ExtractAndMoveMethod refactorings, is that it is organized as an Eclipse
+project. It is also assumed that the code is free from compilation errors.
+
+\section{The experiment}
+For a given project, the first job that is done, is to refactor its source code.
+The refactoring batch job produces three things: The refactored project,
+statistics gathered during the execution of the series of refactorings, and an
+error log describing any errors happening during this execution. See
+\myref{sec:benchmarking} for more information about how the refactorings are
+performed.
+
+After the refactoring process is done, the before- and after-code is analyzed
+with \name{SonarQube}. The analysis results are then stored in a database and
+displayed through a \name{SonarQube} server with a web interface.\todoin{How
+long are these results going to be publicly available?}
+
+The before- and after-code is also tested with their own unit tests. This is
+done to discover any changes in the semantic behavior of the refactored code,
+within the limits of these tests.
+
+\section{Case 1: The Eclipse JDT UI project}
+This case is the ultimate test for our \ExtractAndMoveMethod refactoring. The
+target source code is massive. With its over 300,000 lines of code and over
+25,000 methods, it is formidable task to perform automated changes on it. There
+should be plenty of situations where things can go wrong, and, as we shall see
+later, they do.
+
+I will start by presenting some statistics from the refactoring execution,
+before I pick apart the \name{SonarQube} analysis and conclude by commenting on
+the results from the unit tests. The configuration for the experiment is
+specified in \myref{tab:configurationCase1}.
+
+\begin{table}[htb]
+ \caption{Configuration for Case 1.}
+ \label{tab:configurationCase1}
+ \centering
+ \begin{tabularx}{\textwidth}{@{}>{\bfseries}L{0.67}L{1.33}@{}}
+ \toprule
+ \spancols{2}{Benchmark data} \\
+ \midrule
+ Launch configuration & CaseStudy.launch \\
+ Project & no.uio.ifi.refaktor.benchmark \\
+ Repository & gitolite@git.uio.no:ifi-stolz-refaktor \\
+ Commit & 43c16c04520746edd75f8dc2a1935781d3d9de6c \\
+ \midrule
+ \spancols{2}{Input data} \\
+ \midrule
+ Project & org.eclipse.jdt.ui \\
+ Repository & git://git.eclipse.org/gitroot/jdt/eclipse.jdt.ui.git \\
+ Commit & f218388fea6d4ec1da7ce22432726c244888bb6b \\
+ Branch & R3\_8\_maintenance \\
+ Tests suites & org.eclipse.jdt.ui.tests.AutomatedSuite,
+ org.eclipse.jdt.ui.tests.refactoring.all.\-AllAllRefactoringTests \\
+
+ \bottomrule
+ \end{tabularx}
+\end{table}
+\subsection{Statistics}
+The statistics gathered during the refactoring execution is presented in
+\myref{tab:case1Statistics}.
+
+\begin{table}[htb]
+ \caption{Statistics after batch refactoring the Eclipse JDT UI project with
+ the \ExtractAndMoveMethod refactoring.}
+ \label{tab:case1Statistics}
+ \centering
+ \begin{tabularx}{\textwidth}{@{}>{\bfseries}L{1.5}R{0.5}@{}}
+ \toprule
+ \spancols{2}{Time used} \\
+ \midrule
+ Total time & 98m38s \\
+ Analysis time & 14m41s (15\%) \\
+ Change time & 74m20s (75\%) \\
+ Miscellaneous tasks & 9m37s (10\%) \\
+ \midrule
+ \spancols{2}{Numbers of each type of entity analyzed} \\
+ \midrule
+ Packages & 110 \\
+ Compilation units & 2,097 \\
+ Types & 3,152 \\
+ Methods & 27,667 \\
+ Text selections & 591,500 \\
+ \midrule
+ \spancols{2}{Numbers for \ExtractAndMoveMethod refactoring candidates} \\
+ \midrule
+ Methods chosen as candidates & 2,552 \\
+ Methods NOT chosen as candidates & 25,115 \\
+ Candidate selections (multiple per method) & 36,843 \\
+ \midrule
+ \spancols{2}{\ExtractAndMoveMethod refactorings executed} \\
+ \midrule
+ Fully executed & 2,469 \\
+ Not fully executed & 83 \\
+ Total attempts & 2,552 \\
+ \midrule
+ \spancols{2}{Primitive refactorings executed} \\
+ \spancols{2}{\small \ExtractMethod refactorings} \\
+ \midrule
+ Performed & 2,483 \\
+ Not performed & 69 \\
+ Total attempts & 2,552 \\
+ \midrule
+ \spancols{2}{\small \MoveMethod refactorings} \\
+ \midrule
+ Performed & 2469 \\
+ Not performed & 14 \\
+ Total attempts & 2,483 \\
+
+ \bottomrule
+ \end{tabularx}
+\end{table}
+
+\subsubsection{Execution time}
+I consider the total execution time of approximately 1.5 hours as being
+acceptable. It clearly makes the batch process unsuitable for doing any
+on-demand analysis or changes, but it is good enough for running periodic jobs,
+like over-night analysis.
+
+As the statistics show, 75\% of the total time goes into making the actual code
+changes. The time consumers are here the primitive \ExtractMethod and
+\MoveMethod refactorings. Included in the change time is the parsing and
+precondition checking done by the refactorings, as well as textual changes done
+to files on disk. All this parsing and disk access is time-consuming, and
+constitute a large part of the change time.
+
+In comparison, the pure analysis time, used to find suitable candidates, only
+make up for 15\% of the total time consumed. This includes analyzing almost
+600,000 text selections, while the number of attempted executions of the
+\ExtractAndMoveMethod refactoring are only about 2,500. So the number of
+executed primitive refactorings are approximately 5,000. Assuming the time used
+on miscellaneous tasks are used mostly for parsing source code for the analysis,
+we can say that the time used for analyzing code is at most 25\% of the total
+time. This means that for every primitive refactoring executed, we can analyze
+around 360 text selections. So, with an average of about 21 text selections per
+method, it is reasonable to say that we can analyze over 15 methods in the time
+it takes to perform a primitive refactoring.
+
+\subsubsection{Refactoring candidates}
+Out of the 27,667 methods that was analyzed, 2,552 methods contained selections
+that was considered candidates for the \ExtractAndMoveMethod refactoring. This
+is roughly 9\% off the methods in the project. These 9\% of the methods had on
+average 14.4 text selections that was considered considered possible refactoring
+candidates.
+
+\subsubsection{Executed refactorings}
+2,469 out of 2,552 attempts on executing the \ExtractAndMoveMethod refactoring
+was successful, giving a success rate of 96.7\%. The failure rate of 3.3\% stem
+from situations where the analysis finds a candidate selection, but the change
+execution fails. This failure could be an exception that was thrown, and the
+refactoring aborts. It could also be the precondition checking for one of the
+primitive refactorings that gives us an error status, meaning that if the
+refactoring proceeds, the code will contain compilation errors afterwards,
+forcing the composite refactoring to abort. This means that if the
+\ExtractMethod refactoring fails, no attempt is done for the \MoveMethod
+refactoring. \todo{Redundant information? Put in benchmark chapter?}
+
+Out of the 2,552 \ExtractMethod refactorings that was attempted executed, 69 of
+them failed. This give a failure rate of 2.7\% for the primitive refactoring. In
+comparison, the \MoveMethod refactoring had a failure rate of 0.6 \% of the
+2,483 attempts on the refactoring.
+
+\subsection{\name{SonarQube} analysis}
+Results from the \name{SonarQube} analysis is shown in
+\myref{tab:case1ResultsProfile1}.
+
+\begin{table}[htb]
+ \caption{Results for analyzing the Eclipse JDT UI project, before and after
+ the refactoring, with \name{SonarQube} and the \name{IFI Refaktor Case Study}
+ quality profile. (Bold numbers are better.)}
+ \label{tab:case1ResultsProfile1}
+ \centering
+ \begin{tabularx}{\textwidth}{@{}>{\bfseries}L{1.5}R{0.25}R{0.25}@{}}
+ \toprule
+ \textnormal{Number of issues for each rule} & Before & After \\
+ \midrule
+ Avoid too complex class & 81 & \textbf{79} \\
+ Classes should not be coupled to too many other classes (Single
+ Responsibility Principle) & \textbf{1,098} & 1,199 \\
+ Control flow statements \ldots{} should not be nested too deeply & 1,375 &
+ \textbf{1,285} \\
+ Methods should not be too complex & 1,518 & \textbf{1,452} \\
+ Methods should not have too many lines & 3,396 & \textbf{3,291} \\
+ NPath Complexity & 348 & \textbf{329} \\
+ Too many methods & \textbf{454} & 520 \\
+ \midrule
+ Total number of issues & 8,270 & \textbf{8,155} \\
+ \midrule
+ \midrule
+ \spancols{3}{Complexity} \\
+ \midrule
+ Per function & 3.6 & \textbf{3.3} \\
+ Per class & \textbf{29.5} & 30.4 \\
+ Per file & \textbf{44.0} & 45.3 \\
+ \midrule
+ Total complexity & \textbf{84,765} & 87,257 \\
+ \midrule
+ \midrule
+ \spancols{3}{Numbers of each type of entity analyzed} \\
+ \midrule
+ Files & 1,926 & 1,926 \\
+ Classes & 2,875 & 2,875 \\
+ Functions & 23,744 & 26,332 \\
+ Accessors & 1,296 & 1,019 \\
+ Statements & 162,768 & 165,145 \\
+ Lines of code & 320,941 & 329,112 \\
+ \midrule
+ Technical debt (in days) & \textbf{1,003.4} & 1,032.7 \\
+ \bottomrule
+ \end{tabularx}
+\end{table}
+
+\subsubsection{Diversity in the number of entities analyzed}
+The analysis performed by \name{SonarQube} is reporting fewer methods than found
+by the pre-refactoring analysis. \name{SonarQube} discriminates between
+functions (methods) and accessors, so the 1,296 accessors play a part in this
+calculation. \name{SonarQube} also has the same definition as our plugin when
+it comes to how a class is defined. Therefore is seems like \name{SonarQube}
+misses 277 classes that our plugin handles. This can explain why the {SonarQube}
+report differs from our numbers by approximately 2,500 methods,
+
+\subsubsection{Complexity}
+On all complexity rules that works on the method level, the number of issues
+decreases with between 3.1\% and 6.5\% from before to after the refactoring. The
+average complexity of a method decreases from 3.6 to 3.3, which is an
+improvement of about 8.3\%. So, on the method level, the refactoring must be
+said to have a slightly positive impact.
+
+The improvement in complexity on the method level is somewhat traded for
+complexity on the class level. The complexity per class metric is worsen by 3\%
+from before to after. The issues for the ``Too many methods'' rule also
+increases by 14.5\%. These numbers indicate that the refactoring makes quite a
+lot of the classes a little more complex overall. This is the expected outcome,
+since the \ExtractAndMoveMethod refactoring introduces almost 2,500 new methods
+into the project.
+
+The only number that can save the refactoring's impact on complexity on the
+class level, is the ``Avoid too complex class'' rule. It improves with 2.5\%,
+thus indicating that the complexity is moderately better distributed between the
+classes after the refactoring than before.
+
+\subsubsection{Coupling}
+One of the hopes when starting this project, was to be able to make a
+refactoring that could lower the coupling between classes. Better complexity at
+the method level is a not very unexpected byproduct of dividing methods into
+smaller parts. Lowering the coupling on the other hand, is a far greater task.
+This is also reflected in the results for the only coupling rule defined in the
+\name{SonarQube} quality profile, namely the ``Classes should not be coupled to
+too many
+other classes (Single Responsibility Principle)'' rule.
+
+The number of issues for the coupling rule is 1,098 before the refactoring, and
+1,199 afterwards. This is an increase in issues of 9.2\%, and a blow for this
+project. These numbers can be interpreted two ways. The first possibility is
+that our assumptions are wrong, and that increasing indirection does not
+decrease coupling between classes. The other possibility is that our analysis
+and choices of candidate text selections are not good enough. I vote for the
+second possibility. (Voting against the public opinion may also be a little
+bold.)
+
+What probably happens is, that many of the times the \ExtractAndMoveMethod
+refactoring is performed, the \MoveMethod refactoring ``drags'' with it
+references to classes that are unknown to the method destination. If it happens
+to be so lucky that it removes a dependency from one class, it might as well
+introduce three new dependencies to another class. In those situations that a
+class does not know about the originating class of a moved method, the
+\MoveMethod refactoring most certainly will introduce a dependency. This is
+because there is a
+bug\footnote{\url{https://bugs.eclipse.org/bugs/show\_bug.cgi?id=228635}} in the
+refactoring, making it pass an instance of the originating class as a reference
+to the moved method, regardless of whether the reference is used in the method
+body or not.
+
+There is also the possibility that the heuristics used to find candidate text
+selections are not good enough, they most certainly are not. I wish I had more
+time to fine-tune them, and to complete the analysis part of the project, but
+this is simply not the case. This becomes even clearer when analyzing the unit
+test results for the after-code.
+
+\subsubsection{Totals}
+On the bright side, the total number of issues is lower after the refactoring
+than it was before. Before the refactoring, the total number of issues is
+8,270, and after it is 8,155. An improvement of only 1.4\%.
+
+Then \name{SonarQube} tells me that the total complexity has increased by
+2.9\%, and that the (more questionable) ``technical debt'' has increased from
+1,003.4 to 1,032.7 days, also a deterioration of 2.9\%. Although these numbers
+are similar, no correlation has been found between them.
+
+\subsection{Unit tests}
+The tests that have been run for the \name{Eclipse JDT UI} project, are the
+tests in the test suites specified as the main test suites on the JDT UI wiki
+page on how to contribute to the
+project\footnote{\url{https://wiki.eclipse.org/JDT\_UI/How\_to\_Contribute\#Unit\_Testing}}.
+
+\subsubsection{Before the refactoring}
+Running the tests for the before-code of Eclipse JDT UI yielded 4 errors and 3
+failures for the \type{AutomatedSuite} test suite (2,007 test cases), and 2
+errors and
+3 failures for the \type{AllAllRefactoringTests} test suite (3,816 test cases).
+
+\subsubsection{After the refactoring}
+The test results for the after-code of the Eclipse JDT UI project is another
+story. The reason for this is that during the setup for the unit tests, Eclipse
+now reports that the project contains 322 fatal errors, and a lot of errors that
+probably follows from these. This is another blow for this master's project.
+
+It has now been shown that the \ExtractAndMoveMethod refactoring, in its current
+state, produces code that does not compile. Had these errors originated from
+only one bug, it would not have been much of a problem, but this is not the
+case. By only looking at some random compilation problems in the refactored code,
+I came up with at least four different bugs \todo{write bug reports?} that
+caused those problems. I then stopped looking for more, since some of the bugs
+would take more time to fix than I could justify using on them at this point.
+
+The only thing that can be said in my defence, is that all the compilation
+errors could have been avoided if the type of situations that causes them was
+properly handled by the primitive refactorings, that again are supported by the
+Eclipse JDT UI project. All of the four randomly found bugs that I mentioned
+before, are also weaknesses of the \MoveMethod refactoring. If the primitive
+refactorings had detected the up-coming errors
+in their precondition checking phase, the refactorings would have been aborted,
+since this is how the \ExtractAndMoveMethod refactoring handles such situations.
+
+Of course, taking all possible situations into account is an immense task. This
+is one of the reasons for the failure. A complete analysis is too big of a task
+for this master's project to handle. Looking at it now, this comes as no
+surprise, since the task is obviously also too big for the creators of the
+primitive \MoveMethod refactoring. This shows that the underlying primitive
+refactorings are not complete enough to be fully relied upon for avoiding
+compilation errors.
+
+Considering all these problems, it is difficult to know how to interpret the
+unit test results from after refactoring the Eclipse JDT UI. The
+\type{AutomatedSuite} reported 565 errors and 5 failures. Three of the failures
+were the same as reported before the refactoring took place, so two of them are
+new. For these two cases it is not immediately apparent what makes them behave
+differently. The program is so complex that to analyze it to find this out, we
+might need more powerful methods than just manually analyzing its source code.
+This is somewhat characteristic for imperative programming: The programs are
+often hard to analyze and understand.
+
+For the \type{AllAllRefactoringTests} test suite, the three failures are gone,
+but the two errors have grown to 2,257 errors. I will not try to analyze those
+errors.
+
+What I can say, is that it is likely that the \ExtractAndMoveMethod refactoring
+has introduced some unintended behavioral changes. Let us say that the
+refactoring introduces at least two behavior-altering changes for every 2,500
+executions. More than that is difficult to say about the behavior-preserving
+properties of the \ExtractAndMoveMethod refactoring, at this point.
+
+\subsection{Conclusions}
+After automatically analyzing and executing the \ExtractAndMoveMethod
+refactoring for all the methods in the Eclipse JDT UI project, the results does
+not look that promising. For this case, the refactoring seems almost unusable as
+it is now. The error rate and measurements done tells us this.
+
+The refactoring makes the code a little less complex at the method level. But
+this is merely a side effect of extracting methods, and holds little scientific
+value. When it comes to the overall complexity, it is increased, although it is
+slightly better spread among the classes.
+
+The analysis done before the \ExtractAndMoveMethod refactoring, is currently not
+complete enough to make the refactoring useful. It introduces too many errors in
+the code, and the code may change it's behavior. It also remains to prove that
+large scale refactoring with it can decrease coupling between classes. A better
+analysis may prove this, but in its present state, the opposite is the fact. The
+coupling measurements done by \name{SonarQube} shows this.
+
+On the bright side, the performance of the refactoring process is not that bad.
+It shows that it is possible to make a tool the way we do, if we can make the
+tool do anything useful. As long as the analysis phase is not going to involve
+anything that uses to much disk access, a lot of analysis can be done in a
+reasonable amount of time.
+
+The time used on performing the actual changes excludes a trial and error
+approach with the tools used in this master's project. In a trial and error
+approach, you could for instance be using the primitive refactorings used in
+this project to refactor code, and only then make decisions based on the effect,
+possibly shown by traditional software metrics. The problem with the approach
+taken in this project, compared to a trial and error approach, is that using
+heuristics beforehand is much more complicated. But on the other hand, a trial
+and error approach would still need to face the challenges of producing code
+that does compile without errors. If using refactorings that could produce
+in-memory changes, a trial and error approach could be made more efficient.
+
+\section{Case 2: The \type{no.uio.ifi.refaktor} project}
+In this case we will see a form of the ``dogfooding'' methodology used, when
+refactoring our own \type{no.uio.ifi.refaktor} project with the
+\ExtractAndMoveMethod refactoring.
+
+In this case I will try to point out some differences from case 1, and how they
+impact the execution of the benchmark. The refaktor project is 39 times smaller
+than the Eclipse JDT UI project, measured in lines of code. This will make
+things a bit more transparent. It will therefore be interesting to see if this
+case can shed light on any aspect of our project that was lost in the larger
+case 1.
+
+The configuration for the experiment is specified in
+\myref{tab:configurationCase2}.
+
+\begin{table}[htb]
+ \caption{Configuration for Case 2.}
+ \label{tab:configurationCase2}
+ \centering
+ \begin{tabularx}{\textwidth}{@{}>{\bfseries}L{0.67}L{1.33}@{}}
+ \toprule
+ \spancols{2}{Benchmark data} \\
+ \midrule
+ Launch configuration & CaseStudyDogfooding.launch \\
+ Project & no.uio.ifi.refaktor.benchmark \\
+ Repository & gitolite@git.uio.no:ifi-stolz-refaktor \\
+ Commit & 43c16c04520746edd75f8dc2a1935781d3d9de6c \\
+ \midrule
+ \spancols{2}{Input data} \\
+ \midrule
+ Project & no.uio.ifi.refaktor \\
+ Repository & gitolite@git.uio.no:ifi-stolz-refaktor \\
+ Commit & 43c16c04520746edd75f8dc2a1935781d3d9de6c \\
+ Branch & master \\
+ Test configuration & no.uio.ifi.refaktor.tests/ExtractTest.launch \\
+ \bottomrule
+ \end{tabularx}
+\end{table}
+
+\subsection{Statistics}
+The statistics gathered during the refactoring execution is presented in
+\myref{tab:case2Statistics}.
+
+\begin{table}[htb]
+ \caption{Statistics after batch refactoring the \type{no.uio.ifi.refaktor}
+project with the \ExtractAndMoveMethod refactoring.}
+ \label{tab:case2Statistics}
+ \centering
+ \begin{tabularx}{\textwidth}{@{}>{\bfseries}L{1.5}R{0.5}@{}}
+ \toprule
+ \spancols{2}{Time used} \\
+ \midrule
+ Total time & 1m15s \\
+ Analysis time & 0m18s (24\%) \\
+ Change time & 0m47s (63\%) \\
+ Miscellaneous tasks & 0m10s (14\%) \\
+ \midrule
+ \spancols{2}{Numbers of each type of entity analyzed} \\
+ \midrule
+ Packages & 33 \\
+ Compilation units & 154 \\
+ Types & 168 \\
+ Methods & 1,070 \\
+ Text selections & 8,609 \\
+ \midrule
+ \spancols{2}{Numbers for \ExtractAndMoveMethod refactoring candidates} \\
+ \midrule
+ Methods chosen as candidates & 58 \\
+ Methods NOT chosen as candidates & 1,012 \\
+ Candidate selections (multiple per method) & 227 \\
+ \midrule
+ \spancols{2}{\ExtractAndMoveMethod refactorings executed} \\
+ \midrule
+ Fully executed & 53 \\
+ Not fully executed & 5 \\
+ Total attempts & 58 \\
+ \midrule
+ \spancols{2}{Primitive refactorings executed} \\
+ \spancols{2}{\small \ExtractMethod refactorings} \\
+ \midrule
+ Performed & 56 \\
+ Not performed & 2 \\
+ Total attempts & 58 \\
+ \midrule
+ \spancols{2}{\small \MoveMethod refactorings} \\
+ \midrule
+ Performed & 53 \\
+ Not performed & 3 \\
+ Total attempts & 56 \\
+
+ \bottomrule
+ \end{tabularx}
+\end{table}
+
+\subsubsection{Differences}
+There are some differences between the two projects that make them a little
+difficult to compare by performance.
+
+\paragraph{Different complexity.}
+Although the JDT UI project is 39 times greater than the refaktor project in
+terms of lines of code, it is only about 26 times its size measured in numbers
+of methods. This means that the methods in the refaktor project are smaller in
+average than in the JDT project. This is also reflected in the \name{SonarQube}
+report, where the complexity per method for the JDT project is 3.6, while the
+refaktor project has a complexity per method of 2.1.
+
+\paragraph{Number of selections per method.}
+The analysis for the JDT project processed 21 text selections per method in
+average. This number for the refaktor project is only 8 selections per method
+analyzed. This is a direct consequence of smaller methods.
+
+\paragraph{Different candidates to methods ratio.}
+The differences in how the projects are factored are also reflected in the
+ratios for how many methods that are chosen as candidates compared to the total
+number of methods analyzed. For the JDT project, 9\% of the methods was
+considered to be candidates, while for the refaktor project, only 5\% of the
+methods was chosen.
+
+\paragraph{The average number of possible candidate selection.}
+For the methods that are chosen as candidates, the average number of possible
+candidate selections for these methods differ quite much. For the JDT project,
+the number of possible candidate selections for these methods were 14.44
+selections per method, while the candidate methods in the refaktor project had
+only 3.91 candidate selections to choose from, in average.
+
+\subsubsection{Execution time}
+The differences in complexity, and the different candidate methods to total
+number of methods ratios, is shown in the distributions of the execution times.
+For the JDT project, 75\% of the total time was used on the actual changes,
+while for the refaktor project, this number was only 63\%.
+
+For the JDT project, the benchmark used on average 0.21 seconds per method in
+the project, while for the refaktor project it used only 0.07 seconds per
+method. So the process used 3 times as much time per method for the JDT project
+than for the refaktor project.
+
+While the JDT project is 39 times larger than the refaktor project measured in
+lines of code, the benchmark used about 79 times as long time on it than for the
+refaktor project. Relatively, this is about twice as long.
+
+Since the details of these execution times are not that relevant to this
+master's project, only their magnitude, I will leave them here.
+
+\subsubsection{Executed refactorings}
+For the composite \ExtractAndMoveMethod refactoring performed in case 2, 53
+successful attempts out of 58 gives a success rate of 91.4\%. This is 5.3
+percentage points worse than for case 1.
+
+\subsection{\name{SonarQube} analysis}
+Results from the \name{SonarQube} analysis is shown in
+\myref{tab:case2ResultsProfile1}.
+
+Not much is to be said about these results. The trends in complexity and
+coupling are the same. We end up a little worse after the refactoring process
+than before.
+
+\begin{table}[htb]
+ \caption{Results for analyzing the \var{no.uio.ifi.refaktor} project, before
+ and after the refactoring, with \name{SonarQube} and the \name{IFI Refaktor
+ Case Study} quality profile. (Bold numbers are better.)}
+ \label{tab:case2ResultsProfile1}
+ \centering
+ \begin{tabularx}{\textwidth}{@{}>{\bfseries}L{1.5}R{0.25}R{0.25}@{}}
+ \toprule
+ \textnormal{Number of issues for each rule} & Before & After \\
+ \midrule
+ Avoid too complex class & 1 & 1 \\
+ Classes should not be coupled to too many other classes (Single
+ Responsibility Principle) & \textbf{29} & 34 \\
+ Control flow statements \ldots{} should not be nested too deeply & 24 &
+ \textbf{21} \\
+ Methods should not be too complex & 17 & \textbf{15} \\
+ Methods should not have too many lines & 41 & \textbf{40} \\
+ NPath Complexity & 3 & 3 \\
+ Too many methods & \textbf{13} & 15 \\
+ \midrule
+ Total number of issues & \textbf{128} & 129 \\
+ \midrule
+ \midrule
+ \spancols{3}{Complexity} \\
+ \midrule
+ Per function & 2.1 & 2.1 \\
+ Per class & \textbf{12.5} & 12.9 \\
+ Per file & \textbf{13.8} & 14.2 \\
+ \midrule
+ Total complexity & \textbf{2,089} & 2,148 \\
+ \midrule
+ \midrule
+ \spancols{3}{Numbers of each type of entity analyzed} \\
+ \midrule
+ Files & 151 & 151 \\
+ Classes & 167 & 167 \\
+ Functions & 987 & 1,045 \\
+ Accessors & 35 & 30 \\
+ Statements & 3,355 & 3,416 \\
+ Lines of code & 8,238 & 8,460 \\
+ \midrule
+ Technical debt (in days) & \textbf{19.0} & 20.7 \\
+ \bottomrule
+ \end{tabularx}
+\end{table}
+
+\subsection{Unit tests}
+The tests used for this case are the same that has been developed throughout the
+master's project.
+
+The code that was refactored for this case suffered from some of the problems
+discovered in case 1. This means that the after-code for case 2 also contained
+compilation errors, but they were not as many. The code contained only 6 errors
+that made the code not compile.
+
+All of the errors made, originated from the same bug. It is a bug that happens
+in situation where a class instance creation is moved from between packages, and
+the class for the instance is package-private. The \MoveMethod refactoring does
+not detect that there will be a visibility problem, and neither does it promote
+the package-private class to be public.
+
+Since the errors was easy to fix manually, I corrected them and ran the unit
+tests as planned. Before the refactoring, all tests passed. All tests also
+passed after the refactoring, with the six error corrections. Since the
+corrections done is not of a kind that could make the behavior of the program
+change, it is likely that the refactorings done to the
+\type{no.uio.ifi.refaktor} project did not change its behavior. This is also
+supported by the informal experiment presented next.
+
+\subsection{An informal experiment}
+To complete the task of ``eating my own dog food'', I conducted an informal
+experiment where I used the refactored version of the \type{no.uio.ifi.refaktor}
+project, with the corrections, to again refaktor ``itself''.
+
+The experiment produced code containing the same six errors as after the
+previous experiment. I also compared the after-code from the two experiments
+with a diff-tool. The only differences found was different method names. This is
+expected, since the method names are randomly generated by the
+\ExtractAndMoveMethod refactoring.
+
+The outcome of this simple experiment makes me more confident that the
+\ExtractAndMoveMethod refactoring made only behavior-preserving changes to the
+\type{no.uio.ifi.refaktor} project, apart from the compilation errors.
+
+\subsection{Conclusions}
+The differences in complexity between the Eclipse JDT UI project and the
+\type{no.uio.ifi.refaktor} project, clearly influenced the differences in their
+execution times. This is mostly because fewer of the methods were chosen to be
+refactored for the refaktor project than for the JDT project. What this makes
+difficult, is to know if there are any severe performance penalties associated
+with refactoring on a large project compared to a small one.