Object Oriented Design}\citing{metricsSuite1994}. The max value for the rule
is chosen on the background of an empirical study by Raed Shatnawi, that
concludes that the number 9 is the most useful threshold for the CBO
- metric\citing{shatnawiQuantitative2010}.
+ metric\citing{shatnawiQuantitative2010}. This study is also performed on
+ Eclipse source code, so this threshold value should be particularly well
+ suited for the Eclipse JDT UI case in this chapter.
\item[Control flow statements \ldots{} should not be nested too deeply] is
a rule that is meant to counter ``Spaghetti code''. It measures the nesting
within the limits of these tests.
\section{Case 1: The Eclipse JDT UI project}
+This case is the ultimate test for our \ExtractAndMoveMethod refactoring. The
+target sorce code is massive. With its over 300,000 lines of code and over
+25,000 methods, it is formidable task to perform automated changes on it. There
+should be plenty of cases where things can go wrong, and, as we shall se later,
+they do.
+
+I will start by presenting some statistics from the refactoring execution,
+before I pick apart the \name{SonarQube} analysis and conclude by commenting on
+the result from the unit tests.
+
+\subsection{Statistics}
+The statistics gathered during the refactoring execution is presented in
+\myref{tab:case1Statistics}.
+
+\subsubsection{Execution time}
+I consider the total exection time of approximately 1.5 hours as being
+acceptable. It clearly makes the batch process unsuitable for doing any
+on-demand analysis or changes, but it is good enough for running periodic jobs,
+like over-night analysis.
+
+As the statistics show, 75\% of the total time goes into making the actual code
+changes. The time consumers are here the primitive \ExtractMethod and
+\MoveMethod refactorings. Included in the change time is the parsing and
+precondition checking done by the refactorings, as well as textual changes done
+to files on disk. All this parsing and disk access is time-consuming, and
+constitute a large part of the change time.
+
+In comparison, the pure analysis time, used to find suitable candidates, only
+make up for 15\% of the total time consumed. This includes analyzing almost
+600,000 text selections, while the number of attempted executions of the
+\ExtractAndMoveMethod refactoring are only about 2,500. So the number of
+executed primitive refactorings are approximately 5,000. Assuming the time used
+on miscellaneous tasks are used mostly for parsing source code for the analysis,
+we can say that the time used for analyzing code is at most 25\% of the total
+time. This means that for every primitive refactoring executed, we can analyze
+around 360 text selections. So, with an average of about 21 text selections per
+method, it is reasonable to say that we can analyze over 15 methods in the time
+it takes to perform a primitive refactoring.
+
+\subsubsection{Refactoring candidates}
+Out of the 27,667 methods that was analyzed, 2,552 methods contained selections
+that was considered candidates for the \ExtractAndMoveMethod refactoring. This
+is roughly 9\% off the methods in the project. These 9\% of the methods had on
+average 14.4 text selections that was considered considered possible refactoring
+candidates.
+
+\subsubsection{Executed refactorings}
+2,469 out of 2,552 attempts on executing the \ExtractAndMoveMethod refactoring
+was successful, giving a success rate of 96.7\%. The failure rate of 3.3\% stem
+from situations where the analysis finds a candidate selection, but the change
+execution fails. This failure could be an exception that was thrown, and the
+refactoring aborts. It could also be the precondition checking for one of the
+primitive refactorings that gives us an error status, meaning that if the
+refactoring proceeds, the code will contain compilation errors afterwards,
+forcing the composite refactoring to abort. This means that if the
+\ExtractMethod refactoring fails, no attempt is done for the \MoveMethod
+refactoring. \todo{Redundant information? Put in benchmark chapter?}
+
+Out of the 2,552 \ExtractMethod refactorings that was attempted executed, 69 of
+them failed. This give a failure rate of 2.7\% for the primitive refactoring. In
+comparison, the \MoveMethod refactoring had a failure rate of 0.6 \% of the
+2,483 attempts on the refactoring.
+
+\subsection{\name{SonarQube} analysis}
+The \name{SonarCube} analysis reports fewer methods than found by the
+pre-refactoring analysis. \name{SonarQube} discriminates between functions
+(methods) and accessors, so the 1,296 accessors play a part in this calculation.
+\name{SonarQube} also has the same definition as our plugin when it comes to how
+a class is defined. Therefore is seems like \name{SonarQube} misses 277 classes
+that our plugin handles. This can explain why the {SonarQube} report differs
+from our numbers by approximately 2,500 methods,
+
+\subsubsection{Complexity}
+On all complexity rules that works on the method level, the number of issues
+decreases with between 3.1\% and 6.5\% from before to after the refactoring. The
+average complexity of a method decreases from 3.6 to 3.3, which is an
+improvement of about 8.3\%. So, on the method level, the refactoring must be
+said to have a slightly positive impact.
+
+The improvement in complexity on the method level is somewhat traded for
+complexity on the class level. The complexity per class metric is worsen by 3\%
+from before to after. The issues for the ``Too many methods'' rule also
+increases by 14.5\%. These numbers indicate that the refactoring makes quite a
+lot of the classes a little more complex overall. This is the expected outcome,
+since the \ExtractAndMoveMethod refactoring introduces almost 2,500 new methods
+into the project.
+
+The only number that can save the refactoring's impact on complexity on the
+class level, is the ``Avoid too complex class'' rule. It improves with 2.5\%,
+thus indicating that the complexity is moderately better distributed between the
+classes after the refactoring than before.
+
+\subsubsection{Coupling}
+One of the hopes when starting this project, was to be able to make a
+refactoring that could lower the coupling between classes. Better complexity at
+the method level is not a very unexpected bi-product of dividing methods into
+smaller parts. Lowering the coupling on the other hand, is a far greater task.
+This is also reflected in the results for the only coupling rule defined in the
+\name{SonarQube} quality profile, namely the ``Classes should not be coupled to too many
+other classes (Single Responsibility Principle)'' rule.
+
+The number of issues for the coupling rule is 1,098 before the refactoring, and
+1,199 afterwards. This is an increase in issues of 9.2\%, and a blow for this
+project. These numbers can be interpreted two ways. The first possibility is
+that our assumptions are wrong, and that increasing indirection does not
+decrease coupling between classes. The other possibility is that our analysis
+and choices of candidate text selections are not good enough. I vote for the
+second possibility. (Voting againts the public opinion may also be a little
+bold.)
+
+\todoin{finish}
+
-\begin{table}[htb]
- \caption{Parameters for Case 1.}
- \label{tab:dataCase1}
- \centering
- \begin{tabularx}{\textwidth}{@{}>{\bfseries}L{0.67}L{1.33}@{}}
- \toprule
- \spancols{2}{Benchmark data} \\
- \midrule
- Launch configuration & CaseStudy.launch \\
- Project & no.uio.ifi.refaktor.benchmark \\
- Repository & gitolite@git.uio.no:ifi-stolz-refaktor \\
- Commit & 43c16c04520746edd75f8dc2a1935781d3d9de6c \\
- \midrule
- \spancols{2}{Input data} \\
- \midrule
- Project & org.eclipse.jdt.ui \\
- Repository & git://git.eclipse.org/gitroot/jdt/eclipse.jdt.ui.git \\
- Commit & f218388fea6d4ec1da7ce22432726c244888bb6b \\
-
- \bottomrule
- \end{tabularx}
-\end{table}
\begin{table}[htb]
\caption{Statistics after batch refactoring the Eclipse JDT UI project with
\end{tabularx}
\end{table}
+\begin{table}[htb]
+ \caption{Parameters for Case 1.}
+ \label{tab:dataCase1}
+ \centering
+ \begin{tabularx}{\textwidth}{@{}>{\bfseries}L{0.67}L{1.33}@{}}
+ \toprule
+ \spancols{2}{Benchmark data} \\
+ \midrule
+ Launch configuration & CaseStudy.launch \\
+ Project & no.uio.ifi.refaktor.benchmark \\
+ Repository & gitolite@git.uio.no:ifi-stolz-refaktor \\
+ Commit & 43c16c04520746edd75f8dc2a1935781d3d9de6c \\
+ \midrule
+ \spancols{2}{Input data} \\
+ \midrule
+ Project & org.eclipse.jdt.ui \\
+ Repository & git://git.eclipse.org/gitroot/jdt/eclipse.jdt.ui.git \\
+ Commit & f218388fea6d4ec1da7ce22432726c244888bb6b \\
+ Branch & R3\_8\_maintenance \\
+
+ \bottomrule
+ \end{tabularx}
+\end{table}
\section{Case 2: Dogfooding}
\chapter{Benchmarking}\label{sec:benchmarking}
selection the variable is referenced in, so that we can find it by parsing the
source code and search for it when it is needed.
+\section{Handling failures}
+\todoin{write}
+
\chapter{Technicalities}
git://git.uio.no / ifi-stolz-refaktor.git / commitdiff