Blog-Archiv

Montag, 30. Januar 2023

Java serialVersionUID Test App

In nearly every Java class that implements Serializable you find the serialVersionUID. Why do developers define that field? Is it needed? What happens if you change its value? What happens on serialization if you add or remove a field or method in the enclosing class?

This Blog tries to give some answers to these questions. It also refers to one of my past Blogs.

Recommendation

Excerpt from Serializable JavaDoc:

It is strongly recommended that all serializable classes explicitly declare serialVersionUID values, since the default serialVersionUID computation is highly sensitive to class details that may vary depending on compiler implementations, and can thus result in unexpected InvalidClassExceptions during deserialization.
....
serialVersionUID fields are not useful as inherited members.

If you do not define the serialVersionUID field with a value on a Serializable class, the JVM will compute it automatically at runtime, and check the compatibility of classes on object serialization by comparing these values.

That doesn't mean that every new compilation of a class will cause a new serialVersionUID value. But adding or removing non-static fields or methods will cause that (statics do not get serialized - except the explicit serialVersionUID).

Example Code

Here is example code of a serializable class. It is about the static field on line 5:

import java.io.Serializable;

public class TransferExample implements Serializable
{
    private static final long serialVersionUID = 1;
    
    private String name;
    private int number;
    
    public TransferExample(String name, int number) {
        this.name = name;
        this.number = number;
    }

    @Override
    public String toString() {
        return super.toString()+": "+name+" "+number;
    }
}

I will use an object of this class to try out what happens if you read a serialized object into a class that has been modified since serialization. The scenario is a server A that has a different TransferExample class version than server B and tries to send a TransferExample object to server B.

If both class versions have a hardcoded serialVersionUID of same value, the data-exchange will always succeed, even if the sending and receiving classes are different. This is the reason why the Serializable JavaDoc (see above) recommends to always define a serialVersionUID.

In another scenario, when a serialized object got persisted into a database or the file-system, reading it out into a modified class version will also succeed (with hardcoded serialVersionUID), but data will be lost in case the receiving class has less properties than the class of the persisted object had. Not a problem? At least you have to know about that data loss ...

Test Code

In reality, there are many class-modification cases to consider: removing a property, adding a property, removing the serialVersionUID, adding it, changing it, all these possibilities combined should be tested to get an imagination what could happen at runtime.

For brevity I will just try out removing a property with or without having defined a serialVersionUID.

Following class provides in-memory serialization. It contains a method to turn an Object into a byte[] array, and another method to turn a byte[] array back into an Object. I will use that to write bytes to a file, and read bytes from a file.

import java.io.*;

public final class Serializer
{
    public static byte[] serialize(Object object) throws IOException {
        final ByteArrayOutputStream byteStream = new ByteArrayOutputStream();
        final ObjectOutputStream objectStream = new ObjectOutputStream(byteStream);
        objectStream.writeObject(object);    // is now in outputStream
        objectStream.close();
        return byteStream.toByteArray();
    }   

    public static Object deserialize(byte[] bytes) throws IOException, ClassNotFoundException {
        final InputStream inputStream = new ByteArrayInputStream(bytes);
        return new ObjectInputStream(inputStream).readObject();
    }   

    private Serializer() {}
}

The scenario of a server A sending to server B is not so short and easy to implement. So I will replace server B by a file. The test-app will serialize an object into a file, then read it it from that file and output it (1st launch). In case the file exists, it will not write the object but only read it (2nd launch). This is the situation where we can modify the class before, and then see what happens on serialization of objects having incompatible classes.

Thus the follwing test-app should be run once to write an example into a serialization-file, afterwards the class should be modified in some way, then the test-app should be run again to see what it does when reading from the already existing serialization-file. When the 2nd launch succeeds, the test-app will automatically remove the file to make place for another experiment. Here is its code:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
import java.io.*;

public class SerialVersionUidTest
{
    private final static String FILE_NAME = "TransferExample.ser";
    
    public static void main(String[] args) throws Exception {
        final Object toSerialize = new TransferExample("Oberon", 1);
        new SerialVersionUidTest(toSerialize);
    }

    public SerialVersionUidTest(Object toSerialize) throws IOException, ClassNotFoundException {
        final File file = new File(FILE_NAME);
        final boolean fileExists = file.exists();
        
        if (fileExists == false) {
            System.err.println("Creating nonexisting file "+file.getAbsolutePath());
            writeObjectToFile(file, toSerialize);
        }
        
        final Object deserialized = readObjectFromFile(file);
        System.out.println("Deserialized object is: "+deserialized);
        
        if (fileExists == true) {
            file.delete();    // make ready for next launch
            System.err.println("Deleted file "+file.getAbsolutePath());
        }
    }
    
    private void writeObjectToFile(File file, Object object) throws IOException {
        System.out.println("Writing to "+file+" ....");
        
        byte[] serializedBytes = Serializer.serialize(object);
        try (OutputStream outputStream = new FileOutputStream(file)) {
            outputStream.write(serializedBytes);
        }   // auto-close stream
    }

    private Object readObjectFromFile(File file) throws IOException, ClassNotFoundException {
        System.out.println("Reading from file "+file+" ....");
        
        try (InputStream inputStream = new FileInputStream(file)) {
            byte[] deserializedBytes = inputStream.readAllBytes();
            return Serializer.deserialize(deserializedBytes);
        }   // auto-close stream
    }
}

Test Executions

Running this as Java class yields on first launch:

Creating nonexisting file /tmp/TransferExample.ser
Writing to TransferExample.ser ....
Reading from file TransferExample.ser ....
Deserialized object is: TransferExample@3930015a: Oberon 1

Now I comment-out the number property:

public class TransferExample implements Serializable
{
    private static final long serialVersionUID = 1;
    
    private String name;
    //private int number;
    
    public TransferExample(String name, int number) {
        this.name = name;
    //    this.number = number;
    }

    @Override
    public String toString() {
        return super.toString()+": "+name; //+" "+number;
    }
}

As you can see I removed the number field and thus forced a new compilation of the class. Running the test now (2nd launch) yields:

Reading from file TransferExample.ser ....
Deserialized object is: TransferExample@67117f44: Oberon
Deleted file /tmp/TransferExample.ser

The serialization worked (because of the explicit serialVersionUID), but the number value got lost. You must know if this is a problem for your application. In case of data-exchange between servers it won't be a problem in most cases, the receiving server will get a zero value for number (JVM initial value for int data types). No exception will happen, although the classes are incompatible.


Now let's try this out without the serialVersionUID definition. Here is the altered TransferExample:

public class TransferExample implements Serializable
{
    //private static final long serialVersionUID = 1;
    
    private String name;
    private int number;
    
    public TransferExample(String name, int number) {
        this.name = name;
        this.number = number;
    }

    @Override
    public String toString() {
        return super.toString()+": "+name+" "+number;
    }
}

This time I commented out the serialVersionUID field. On 1st launch, the test-app outputs the following:

Creating nonexisting file /tmp/TransferExample.ser
Writing to TransferExample.ser ....
Reading from file TransferExample.ser ....
Deserialized object is: TransferExample@6ea6d14e: Oberon 1

Now let's remove the number property again, like done before (see above):

public class TransferExample implements Serializable
{
    //private static final long serialVersionUID = 1;
    
    private String name;
    //private int number;
    
    ....
    

Don't forget to compile the changed source code.
The next (2nd) launch yields the following:

Reading from file TransferExample.ser ....
Exception in thread "main" java.io.InvalidClassException: TransferExample; 
		local class incompatible: 
		stream classdesc serialVersionUID = 2910957717821065909,
		local class serialVersionUID = -4472615520725387357
	at java.base/java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:689)
	at java.base/java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1903)
	at java.base/java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1772)
	at java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2060)
	at java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1594)
	at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:430)
	at Serializer.deserialize(Serializer.java:17)
	at SerialVersionUidTest.readObjectFromFile(SerialVersionUidTest.java:46)
	at SerialVersionUidTest.<init>(SerialVersionUidTest.java:23)
	at SerialVersionUidTest.main(SerialVersionUidTest.java:11)

This time the JVM itself computed serialVersionUID values. Because I removed a property and thus created another version of the TransferExample class, its computed value is different from that of the first version.

On deserialization, the JVM always compares the serialVersionUID values of both sides and throws an InvalidClassException if they are different.

Conclusion

Serialization of objects is used in two contexts: (1) persistence, and (2) data-exchange between JVM-instances at runtime. Case (2) mostly won't be a problem due to the tolerant behavior when a serialVersionUID was defined, so hardcoding it is a good solution. This also applies to case (1) if lost property values do not matter, maybe because these data are not used any more.




Sonntag, 22. Januar 2023

Change LINUX Firefox Snap to Traditional Package

I got slow performance issues with Ubuntu 22.04 ugrade. After I encountered also drag & drop problems in Firefox, I decided to do something.

Remove Firefox Snap

Snap is a new Ubuntu package manager that offers applications directly from the provider, with all neeed library-versions included ("containerized"). There is a big discussion about snap being a proprietary Canonical (Ubuntu) software, while flatpak is the open source solution and thus to be preferred (remember that Ubuntu builds on Debian LINUX which is 100% open source). There are several scripts and Blogs on the Internet that enable you to remove snap completely from your Ubuntu and install flatpak instead.

As I just wanted a faster (and maybe more stable) Firefox browser, I did not remove snap entirely, but tried to remove the snap-Firefox and instead install Firefox traditionally. Here are the commands I launched, in order.

Mind that when you are looking at commands in a browser and you are about to remove that browser, you need to copy all these command-lines to some text file before terminating the browser, because it could happen that the browser is not available anymore afterwards!

List all snaps on system:

$ snap list

When Firefox is not in that list - no need to continue!

Get rid of firefox as snap:

$ sudo snap remove firefox
$ sudo apt purge firefox
$ rm -rf $HOME/snap/firefox

This takes a while, because snap is ....

Avoid Firefox getting reinstalled as snap.
There are numerous solutions for this on the web, I ended up doing them all, here is one:

$ sudo vi /etc/apt/preferences.d/firefox-no-snap
i
Package: firefox*
Pin: release o=Ubuntu*
Pin-Priority: -1
<ESCAPE>
:x
The input i stands for "insert" and is the vi-command for starting to write text.
The <ESCAPE> key finishes the insertion-mode of the editor.
:x is the vi-command for saving and terminating.

And here is another one:

$ sudo vi /etc/apt/preferences.d/mozilla-firefox
i
Package: *
Pin: release o=LP-PPA-mozillateam
Pin-Priority: 1001
<ESCAPE>
:x

Install Firefox Traditionally

Tell your LINUX where the Mozilla/Firefox installation site is:

$ sudo add-apt-repository ppa:mozillateam/ppa

Reinstall Firefox traditionally from its Mozilla PPA ("Personal Package Archive"):

$ sudo apt install -t 'o=LP-PPA-mozillateam' firefox
I am not sure if the -t option is needed, but I am not willing to try it out. (Why are there so many ways to do things?)

Make sure Firefox will be upgraded in future:

$ sudo vi /etc/apt/apt.conf.d/51unattended-upgrades-firefox
i
Unattended-Upgrade::Allowed-Origins:: "LP-PPA-mozillateam:${distro_codename}";
<ESCAPE>
:x

Mind that when you do this before reinstalling Firefox, you would see this error:

$ sudo apt install firefox
....
Package firefox is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

E: Package 'firefox' has no installation candidate

In that case you should remove /etc/apt/apt.conf.d/51unattended-upgrades-firefox, install Firefox, then create the file again.

Resume

Both snap and flatpak are container-systems, that means an application built with one of them has all its needed libraries in its container. Container-Systems are well-known from cloud computing, e.g. docker containers even hold the needed operating-system. Before container-systems, installed applications had to use the libraries that were present in the underlying operating-system. Of course there was a dependency management for that, meaning a newly installed application may also have caused the installation of newer libraries. But this opened problems for older applications in case some new library was not backward-compatible.

Reusing system-libraries would optimize disk consumption. If you have some application that itself has 2 MB and uses only pre-installed libraries, this is the best case. But if you use containers, you may end up with lots of applications that use the same set of libraries, and they do not share them. Disk consumption may climb from 2 MB to 90 MB for such an application, as some wise guy found out. Now, if you install that application traditionally, and maybe just few of the required libraries are present, the space consumption may rise to even 600 MB in some cases! Why is that? Because dependencies are not so cleanly modelled. Separation of concerns is a subtle topic, and reality deviates from the ideal.

But why do containerization? Because of dependency hell, also called DLL hell (Microsoft), in contrast to isolation heaven. Libraries live, applications live, all of them permanently release new versions. Thus permanently upgrading software is a MUST, not a SHOULD or a CAN. You can not preserve applications for eternity. Even their underlying operating-system and hardware-interfaces may change in course of time.




Mittwoch, 11. Januar 2023

Domain Driven Design Terms

In times of cloud computing and 12-factor apps, Eric Evan's Domain Driven Design has become quite popular. Here is my personal collection of term descriptions from DDD. You may like to read also another Internet Blog about this.

Top Level Terms

Domain The user reality: A sphere of knowledge, influence, or activity.
Model Software abstraction of the domain. An incomplete distillation of knowledge.
See also Martin Fowler's articles about domain model and its aenemic counterpart.
Ubiquituos Language Terms and sentences about the domain, understood by both software- and business-experts.
Context Makes domain terms meaningful and unambiguous.
Bounded Context Context boundary description concerning multiple models.

Mindmap

Mindmap Terms

Model-Driven Design Design a portion of the software system to reflect the domain model in a very literal way, so that mapping is obvious.
Layered Architecture Isolate the expression of the domain model and the business logic, and eliminate any dependency on infrastructure, user interface, or even application logic that is not business logic.
Entities When an object is distinguished by its identity, rather than its attributes, make this primary to its definition in the model.
An entity may contain business logic for its natural responsibility when it belongs to no other entity or value object.
Value Objects When you care only about the attributes and logic of an element of the model, classify it as a value object. Treat the value object as immutable. Don’t give a value object any identity.
A value object may contain business logic for its natural responsibility when it belongs to no other entity or value object.
Domain Events Model information about state-changing activity in the domain as a series of discrete events. Represent each event as a domain object.
Services When a significant transformation in the domain is not a natural responsibility of an entity or value object, add an operation to the model, as a standalone interface, declared as a service. Give it a name as part of the ubiquitous language.
Aggregates CAUTION: different from UML Aggregations!

Cluster the entities and value objects into aggregates and define boundaries around each. Choose one entity to be the root of each aggregate, and allow external objects to hold references to the root only. Use aggregate boundaries to govern transactions and distribution.
Repositories Query framework. For each type of aggregate that needs global access, create the illusion of an in-memory collection of all objects of that aggregate’s root type.
Factories Shift the responsibility for creating instances aggregates or complex objects to a separate object.
Continuous Integration Institute a process of merging all code and other implementation artifacts frequently, with automated tests to flag fragmentation quickly.
Context Map To plot strategy, we need a realistic, large-scale view of model development extending across our project and others we integrate with. Identify each model in play on the project and define its bounded context. Name each bounded context, and make the names part of the ubiquitous language.
Shared Kernel Designate, with an explicit boundary, some subset of the domain model that different teams agree to share. Within this boundary, include, along with this subset of the model, the subset of code or of the database design associated with that part of the model. This explicitly shared stuff has special status, and shouldn’t be changed without consultation with the other team.
Customer/Supplier Team Establish a clear customer/supplier relationship between the teams, meaning downstream (dependent) priorities factor into upstream planning.
Conformist Eliminate the complexity of translation between bounded contexts. Choosing conformity enormously simplifies integration.
Anticorruption Layer When control or communication is not adequate to pull off a shared kernel, partner or customer/supplier relationship, translation becomes more complex. As a downstream client, create an isolating layer to provide your system with functionality of the upstream system in terms of your own domain model.
Open Host Service Define a protocol that gives access to your bounded context subsystem as a set of services.
Published Language The translation between the models of two bounded contexts requires a common language.
Separate Ways If two sets of functionality have no significant relationship, declare the bounded context to have no connection to the other at all, allowing developers to find simple, specialized solutions within this small scope.
Big Ball of Mud Parts of systems where models are mixed and boundaries are inconsistent. Do not try to apply sophisticated modeling within this context.

More Terms

Hands-on Modelers Any technical person contributing to the model must spend some time touching the code.
Model-Driven Design (MDD) Draw from the model the terminology used in the (program) design, and the basic assignment of responsibilities. The code becomes an expression of the model.
Intention-revealing Interfaces Name classes and operations to describe their effect and purpose, without reference to the means by which they do what they promise. This relieves the client developer of the need to understand the internals.
Standalone Classes Low coupling, try to reduce dependencies to other classes.
Side-Effect-Free Functions "Pure" functions , do no perform a state change.
Commands Change state, should not return a value (void).
Assertions State pre- and post-conditions of operations, and invariants of classes and aggregates.
Closure of Operations "Fluent Interface" on value objects, i.e. functions that return "this". (Taken from mathematics.)