\subsubsection{Coupling}
One of the hopes when starting this project, was to be able to make a
refactoring that could lower the coupling between classes. Better complexity at
-the method level is a not very unexpected bi-product of dividing methods into
+the method level is a not very unexpected byproduct of dividing methods into
smaller parts. Lowering the coupling on the other hand, is a far greater task.
This is also reflected in the results for the only coupling rule defined in the
\name{SonarQube} quality profile, namely the ``Classes should not be coupled to
introduce three new dependencies to another class. In those situations that a
class does not know about the originating class of a moved method, the
\MoveMethod refactroing most certainly will introduce a dependency. This is
-because there is a bug in the refactoring, making it pass an instance of the
-originating class as a reference to the moved method, regardless of whether the
-reference is used in the method body or not.
+because there is a
+bug\todoin{https://bugs.eclipse.org/bugs/show\_bug.cgi?id=228635} in the
+refactoring, making it pass an instance of the originating class as a reference
+to the moved method, regardless of whether the reference is used in the method
+body or not.
There is also the possibility that the heuristics used to find candidate text
selections are not good enough, they most certainly are not. I wish I had more
are similar, no correlation has been found between them.
\subsection{Unit tests}
-\todoin{write}
-
-\subsection{Summary}
-\todoin{write}
-
+The tests that have been run for the \name{Eclipse JDT UI} project, are the
+tests in the test suites specified as the main test suites on the JDT UI wiki
+page on how to contribute to the
+project\footnote{\url{https://wiki.eclipse.org/JDT\_UI/How\_to\_Contribute\#Unit\_Testing}}.
+
+\subsubsection{Before the refactoring}
+Running the tests for the before-code of Eclipse JDT UI yielded 4 errors and 3
+failures for the \type{AutomatedSuite} test suite (2,007 test cases), and 2
+errors and
+3 failures for the \type{AllAllRefactoringTests} test suite (3,816 test cases).
+
+\subsubsection{After the refactoring}
+The test results for the after-code of the Eclipse JDT UI project is another
+story. The reason for this, and this somehow slipped for me until I was going to
+run the unit test for the code, is that Eclipse now reports that the project
+contains 322 fatal errors, and a lot of errors that probably follows from these.
+This is another blow for this master's project.
+
+It has now been shown that the \ExtractAndMoveMethod refactoring, in its current
+state, produces code that does not compile. Had these errors originated from
+only one bug, it would not have been much of a problem, but this is not the
+case. By only looking at some random compilation problems in the refactore code,
+I came up with at least four different bugs \todo{write bug reports} that caused
+those problems. I then stopped looking for more, since some of the bugs would
+take more time to fix than I could justify using on them at this point.
+
+The only thing that can be said in my defence, is that all the compilation
+errors could have been avoided if the type of situations that causes them was
+properly handled by the primitive refactorings, that again are supported by the
+Eclipse JDT UI project. All of the four randomly found bugs that I menitioned
+before, are also weaknesses of the \MoveMethod refactoring. If the primitive
+refactorings had detected the up-coming errors
+in their precondition checking phase, the refactorings would have been aborted,
+since this is how the \ExtractAndMoveMethod refactoring handles such situations.
+
+Of course, taking all possible situations into account is an immense task. This
+is one of the reasons for the failure. A complete analysis is too big of a task
+for this master's project to handle. Looking at it now, this comes as no
+surprise, since the task is obviously also too big for the creators of the
+primitive \MoveMethod refactoring. This shows that the underlying primitive
+refactorings are not complete enough to be fully relied upon for avoiding
+compilation errors.
+
+Considering all these problems, it is difficult to know how to interpret the
+unit test results from after refactoring the Eclipse JDT UI. The
+\type{AutomatedSuite} reported 565 errors and 5 failures. Three of the failures
+were the same as reported before the refactoring took place, so two of them are
+new. For these two cases it is not immediately apparent what makes them behave
+differently. The program is so complex that to analyze it to find this out, we
+might need more powerful methods than just manually analyzing its source code.
+This is somewhat characteristic for imperative programming: The programs are
+often hard to analyze and understand.
+
+For the \type{AllAllRefactoringTests} test suite, the three failures are gone,
+but the two errors have grown to 2,257 errors. I will not try to analyze those
+errors.
+
+What I can say, is that it is likely that the \ExtractAndMoveMethod refactoring
+has introduced some unintended behavioral changes. Let us say that the
+refactoring introduces at least two behavior-altering changes for every 2,500
+executions. More than that is difficult to say about the behavior-preserving
+properties of the \ExtractAndMoveMethod refactoring, at this point.
+
+\subsection{Conclusions}
+After automatically analyzing and executing the \ExtractAndMoveMethod
+refactoring for all the methods in the Eclipse JDT UI project, the results does
+not look that promising. For this case, the refactoring seems almost unusable as
+it is now. The error rate and measurements done tells us this.
+
+The refactoring makes the code a little less complex at the method level. But
+this is merely a side effect of extracting methods, and holds little scientific
+value. When it comes to the overall complexity, it is increased, although it is
+slightly better spread among the classes.
+
+The analysis done before the \ExtractAndMoveMethod refactoring, is currently not
+complete enough to make the refactoring useful. It introduces too many errors in
+the code, and the code may change it's behavior. It also remains to prove that
+large scale refactoring with it can decrease coupling between classes. A better
+analysis may prove this, but in its present state, the opposite is the fact. The
+coupling measurements done by \name{SonarQube} shows this.
+
+On the bright side, the performance of the refactoring process is not that bad.
+It shows that it is possible to make a tool the way we do, if we can make the
+tool do anything useful. As long as the analysis phase is not going to involve
+anything that uses to much disk access, a lot of analysis can be done in a
+reasonable amount of time.
+
+The time used on performing the actual changes excludes a trial and error
+approach with the tools used in this master's project. In a trial and error
+approach, you could for instance be using the primitive refactorings used in
+this project to refactor code, and only then make decisions based on the effect,
+possibly shown by traditional software metrics. The problem with the approach
+taken in this project, compared to a trial and error approach, is that using
+heuristics beforehand is much more complicated. But on the other hand, a trial
+and error approach would still need to face the challenges of producing code
+that does compile without errors. If using refactorings that could produce
+in-memory changes, a trial and error approach could be made more efficient.
\begin{table}[htb]
\caption{Parameters for Case 1.}
Repository & git://git.eclipse.org/gitroot/jdt/eclipse.jdt.ui.git \\
Commit & f218388fea6d4ec1da7ce22432726c244888bb6b \\
Branch & R3\_8\_maintenance \\
+ Tests suites & org.eclipse.jdt.ui.tests.AutomatedSuite,
+ org.eclipse.jdt.ui.tests.refactoring.all.\-AllAllRefactoringTests \\
\bottomrule
\end{tabularx}
git://git.uio.no / ifi-stolz-refaktor.git / commitdiff