Solution: TDD

At first the developing of the unit test:

    @Test
    void reduceTest() {
        Fraction f0 = new Fraction(2, 4);
        f0.reduce();
        assertEquals(1, f0.getNumerator());
        assertEquals(2, f0.getDenominator());

        Fraction f1 = new Fraction(18, 9);
        f1.reduce();
        assertEquals(2, f1.getNumerator());
        assertEquals(1, f1.getDenominator());

        Fraction f2 = new Fraction(3, -9);
        f2.reduce();
        assertEquals(1, f2.getNumerator());
        assertEquals(-3, f2.getDenominator());

        Fraction f3 = new Fraction(-3, -8);  // irreducible
        f3.reduce();
        assertEquals(-3, f3.getNumerator());
        assertEquals(-8, f3.getDenominator());
    }

Unit test tests negative and positive fractions with all variations of numerator and denominator

The method itself:

public void reduce()
{
    double maxFactor, factor;

    if (Math.abs(numerator) >= Math.abs(denominator))  // get maximum factor
        maxFactor = Math.abs(denominator);
    else
        maxFactor = Math.abs(numerator);

    for (factor = 2; factor <= maxFactor; factor++)  // reduce
        while (numerator % factor == 0 && denominator % factor == 0)
        {
            numerator /= factor;  // numerator = numerator / factor;
            denominator /= factor;
        }
    }
} 

/*
public void reduce()    // another solution from the web
{
    // http://www.linux-related.de/index.html?/coding/alg_euklid.htm
    double u = Math.abs(numerator), v =  Math.abs(denominator), tmp;
    while (u > 0)
    {
        tmp = u;
        u = v % u;
        v = tmp;
    }
    numerator /= v;
    denominator /= v;
}
*/

Two variations of method reduce(). The second variation also works fine with floating point numerator and denominater.

The modified event handler in controller.java

public class Controller {
    ...
    private void handleButtonAction(ActionEvent event) {        // Event handler referenced in fxml file

        Double numerator = Double.parseDouble(inputTextFieldNum.getText());
        Double denominator = Double.parseDouble(inputTextFieldDenom.getText());

        Fraction fraction = new Fraction(numerator, denominator);           // Separation of GUI and logic layer
        outputTextField.setText(Double.toString(fraction.quotient()));

        fraction.reduce();
        inputTextFieldNum.setText(Double.toString(fraction.getNumerator()));
        inputTextFieldDenom.setText(Double.toString(fraction.getDenominator()));
    }
}

The fraction is reduced after calculation of quotient.