Code Duplication Example

A typical code duplication situation happens when you want to combine an algorithm (e.g. a search-loop) with objects of different types that do not have a common super-class. Assume you must implement a search in documents of type CSV, PDF and EXCEL:

final List<CsvDocument> csvDocuments = ....;
final List<PdfDocument> pdfDocuments = ....;
final List<ExcelDocument> excelDocuments = ....;

final List<String> authors = new AuthorExtractor().withNamepartFromDocumentsYoungerThan(
        "a",
        Date.from(Instant.now()),
        csvDocuments,
        excelDocuments,
        pdfDocuments);
        

This is a search for authors whose name contains "a" and the documents are younger than date/time of the search execution. The AuthorExtractor class will follow shortly.

The document-classes come from an external library, you can not change their source code. Lets assume all three classes deliver, in some way,

• author name
• creation date

Here are some some simple mock implementations, simulating document-types for the duplication example below:

ATTENTION: Simulation of an external class that you can not change!

public class CsvDocument
{
    public String getAuthorFullName()    {
        return "Anonymous";
    }
    
    public Date getCreatedAt()    {
        return Date.from(Instant.now().minus(Period.of(0, 0, 1)));    // 1 day ago;
    }
}

ATTENTION: Simulation of an external class that you can not change!

public class PdfDocument
{
    public String authorFirstName()    {
        return "John";
    }
    
    public String authorLastName()    {
        return "Warnock";
    }
    
    public Date createdAt()    {
        return Date.from(Instant.now().minus(Period.of(0, 0, 1)));    // 1 day ago;
    }
}

ATTENTION: Simulation of an external class that you can not change!

public class ExcelDocument
{
    public String getAuthor()    {
        return "Bill Gates";
    }
    
    public Date getCreationDate()    {
        return Date.from(Instant.now().minus(Period.of(0, 0, 1)));    // 1 day ago
    }
}

As you can see, the three document-classes expose different methods to get author and date, the PDF even provides two methods for the author's name while the others have it in just one. All of these example classes have mock author names and mock creation dates (one day old), we will use just one instance of each.

Here comes fast written code that implements the withNamepartFromDocumentsYoungerThan() search algorithm announced above:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46

public class AuthorExtractor { public List<String> withNamepartFromDocumentsYoungerThan( String namePart, Date date, List<CsvDocument> csvDocuments, List<ExcelDocument> excelDocuments, List<PdfDocument> pdfDocuments) { final List<String> authors = new ArrayList<String>(); for (CsvDocument csvDocument : csvDocuments) { final String authorFullName = csvDocument.getAuthorFullName(); final Date createdAt = csvDocument.getCreatedAt(); // START code duplication 1 if (authorFullName.contains(namePart) && createdAt.after(date)) { authors.add(authorFullName); } // END code duplication 1 } for (ExcelDocument excelDocument : excelDocuments) { final String author = excelDocument.getAuthor(); final Date creationDate = excelDocument.getCreationDate(); // START code duplication 1 if (author.contains(namePart) && creationDate.after(date)) { authors.add(author); } // END code duplication 1 } for (PdfDocument pdfDocument : pdfDocuments) { final String authorFirstName = pdfDocument.authorFirstName(); final String authorLastName = pdfDocument.authorLastName(); final String authorFirstAndLastName = authorFirstName+" "+authorLastName; final Date createdAt = pdfDocument.createdAt(); // START code duplication 1 if (authorFirstAndLastName.contains(namePart) && createdAt.after(date)) { authors.add(authorFirstAndLastName); } // END code duplication 1 } return authors; } }

I have marked the code duplications with comments. They are there because you need three loops for each of the document types, and inside each loop you have different calls to retrieve author/date, but then the same if-condition to filter out the wanted authors. Essentially even the three loops are code duplications, there should be just one.

Here is a main class that you can use for trying out this application:

public class CodeDuplication
{
    public static void main(String[] args) throws Exception {
        final List<CsvDocument> csvDocuments = new ArrayList<>();
        csvDocuments.add(new CsvDocument());
        final List<ExcelDocument> excelDocuments = new ArrayList<>();
        excelDocuments.add(new ExcelDocument());
        final List<PdfDocument> pdfDocuments = new ArrayList<>();
        pdfDocuments.add(new PdfDocument());
        
        final List<String> authors1 = new AuthorExtractor().withNamepartFromDocumentsYoungerThan(
                "a",
                Date.from(Instant.now().minus(Period.of(0, 0, 2))),	// 2 days ago,
                csvDocuments,
                excelDocuments,
                pdfDocuments);
        System.out.println(authors1);
        
        final List<String> authors2 = new AuthorExtractor().withNamestartFromDocumentsOlderEqual(
                "A",
                Date.from(Instant.now()),
                csvDocuments,
                excelDocuments,
                pdfDocuments);
        System.out.println(authors2);
    }
}

As code duplications are like viruses, it will get worse when the next query withNamestartFromDocumentsOlderEqual() or others get implemented, see the extended AuthorExtractor below, and mind that the first method was copy-pasted with slight modifications.

public class AuthorExtractor
{
    public List<String> withNamepartFromDocumentsYoungerThan(
            String namePart,
            Date date,
            List<CsvDocument> csvDocuments,
            List<ExcelDocument> excelDocuments,
            List<PdfDocument> pdfDocuments)
    {
        final List<String> authors = new ArrayList<String>();
        
        for (CsvDocument csvDocument : csvDocuments)    {
            final String authorFullName = csvDocument.getAuthorFullName();
            final Date createdAt = csvDocument.getCreatedAt();
            // START code duplication 1
            if (authorFullName.contains(namePart) && createdAt.after(date))    {
                authors.add(authorFullName);
            }
            // END code duplication 1
        }
        
        for (ExcelDocument excelDocument : excelDocuments)    {
            final String author = excelDocument.getAuthor();
            final Date creationDate = excelDocument.getCreationDate();
            // START code duplication 1
            if (author.contains(namePart) && creationDate.after(date))    {
                authors.add(author);
            }
            // END code duplication 1
        }
        
        for (PdfDocument pdfDocument : pdfDocuments)    {
            final String authorFirstName = pdfDocument.authorFirstName();
            final String authorLastName = pdfDocument.authorLastName();
            final String authorFirstAndLastName = authorFirstName+" "+authorLastName;
            final Date createdAt = pdfDocument.createdAt();
            // START code duplication 1
            if (authorFirstAndLastName.contains(namePart) && createdAt.after(date))    {
                authors.add(authorFirstAndLastName);
            }
            // END code duplication 1
        }
        
        return authors;
    }
    
    // START code duplication 2
    public List<String> withNamestartFromDocumentsOlderEqual(
            String nameStart,
            Date date,
            List<CsvDocument> csvDocuments,
            List<ExcelDocument> excelDocuments,
            List<PdfDocument> pdfDocuments)
    {
        final List<String> authors = new ArrayList<String>();
        
        for (CsvDocument csvDocument : csvDocuments)    {
            final String authorFullName = csvDocument.getAuthorFullName();
            final Date createdAt = csvDocument.getCreatedAt();
            // START code duplication 3
            if (authorFullName.startsWith(nameStart) && (createdAt.equals(date) || createdAt.before(date)))    {
                authors.add(authorFullName);
            }
            // END code duplication 3
        }
        
        for (ExcelDocument excelDocument : excelDocuments)    {
            final String author = excelDocument.getAuthor();
            final Date creationDate = excelDocument.getCreationDate();
            // START code duplication 3
            if (author.startsWith(nameStart) && (creationDate.equals(date) || creationDate.before(date)))    {
                authors.add(author);
            }
            // END code duplication 3
        }
        
        for (PdfDocument pdfDocument : pdfDocuments)    {
            final String authorFirstName = pdfDocument.authorFirstName();
            final String authorLastName = pdfDocument.authorLastName();
            final String authorFirstAndLastName = authorFirstName+" "+authorLastName;
            final Date createdAt = pdfDocument.createdAt();
            // START code duplication 3
            if (authorFirstAndLastName.startsWith(nameStart) && (createdAt.equals(date) || createdAt.before(date)))    {
                authors.add(authorFirstAndLastName);
            }
            // END code duplication 3
        }
        
        return authors;
    }
    // END code duplication 2
}

Duplicate code manifests itself in Long Methods. I call it "Having Déjà vu while reading the source".

Running the CodeDuplication.main() entry point should yield this output:

[Bill Gates, John Warnock]
[Anonymous]

Before refactoring the code duplications we should write a unit-test that asserts this result. The class CodeDuplication can be easily turned into such a test.

There are several steps and different ways how AuthorExtractor can be improved. In my next Blog I will show how I would do it.