mapjfx 1.13.2 using OpenLayers 4.3.2

I just released mapjfx version 1.13.2 it should be found shortly at maven central, the artifact coordinates are:

  <dependency>
    <groupId>com.sothawo</groupId>
    <artifactId>mapjfx</artifactId>
    <version>1.13.2</version>
  </dependency>

The source is available at GitHub.

Now uses OpenLayers 4.3.2.

Comments and contributions welcome.

a simple web based chat application built with Kotlin, Vaadin, Spring Boot and Apache Kafka

Intro

In this post I show how to combine some language / frameworks and libraries / tools to build a web-based scalable chat application. I chose the following combination of tools:

As I am bad in creating cool names for projects I just put together the first letters of the used tools and named this whole thing kovasbak. The complete source code and project is available on GitHub.

What it will look like

The following screenshot shows four browser windows with four users chatting:

Running the backend

The first thing that I have to do is to get Apache Kafka running. I downloaded the actual version (0.11.0.0) from the Apache Kafka website and unpacked the download in a local directory. According to the Kafka documentation I started first zookeeper and then one Kafka broker:

./bin/zookeeper-server-start.sh config/zookeeper.properties &
./bin/kafka-server-start.sh config/server.properties &

I am just using the default values, that gets Kafka runnning on port 9092.

Setting up the project

I am using Java 1.8.0_131 and IntelliJ IDEA, but the project is totally maven based, so you can use the IDE / editor of your choice. To create the project, I used the Spring Intializr integration in IntelliJ, but of course you can create the project by using the Spring Initializr website.

I just selected Kotlin as language, Java version 1.8, Spring Boot 1.5.4 and additionally selected web/vaadin and io/kafka.

After creating the project you end up with the following pom.xml, I only added the highlighted lines to be able to have server-push (more on that later):

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>com.sothawo</groupId>
  <artifactId>kovasbak</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <packaging>jar</packaging>

  <name>kovasbak</name>
  <description>a simple chat system built with Kotlin, Vaadin, spring Boot and Apache Kafka</description>

  <parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>1.5.4.RELEASE</version>
    <relativePath/> <!-- lookup parent from repository -->
  </parent>

  <properties>
    <kotlin.compiler.incremental>true</kotlin.compiler.incremental>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
    <java.version>1.8</java.version>

    <kotlin.version>1.1.3</kotlin.version>
    <vaadin.version>8.0.6</vaadin.version>
  </properties>

  <dependencies>
    <dependency>
      <groupId>org.springframework.kafka</groupId>
      <artifactId>spring-kafka</artifactId>
    </dependency>
    <dependency>
      <groupId>com.vaadin</groupId>
      <artifactId>vaadin-spring-boot-starter</artifactId>
    </dependency>
    <dependency>
      <groupId>com.vaadin</groupId>
      <artifactId>vaadin-push</artifactId>
    </dependency>
    <dependency>
      <groupId>org.jetbrains.kotlin</groupId>
      <artifactId>kotlin-stdlib-jre8</artifactId>
      <version>${kotlin.version}</version>
    </dependency>
    <dependency>
      <groupId>org.jetbrains.kotlin</groupId>
      <artifactId>kotlin-reflect</artifactId>
      <version>${kotlin.version}</version>
    </dependency>

    <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-test</artifactId>
      <scope>test</scope>
    </dependency>
  </dependencies>

  <dependencyManagement>
    <dependencies>
      <dependency>
        <groupId>com.vaadin</groupId>
        <artifactId>vaadin-bom</artifactId>
        <version>${vaadin.version}</version>
        <type>pom</type>
        <scope>import</scope>
      </dependency>
    </dependencies>
  </dependencyManagement>

  <build>
    <sourceDirectory>${project.basedir}/src/main/kotlin</sourceDirectory>
    <testSourceDirectory>${project.basedir}/src/test/kotlin</testSourceDirectory>
    <plugins>
      <plugin>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-maven-plugin</artifactId>
      </plugin>
      <plugin>
        <artifactId>kotlin-maven-plugin</artifactId>
        <groupId>org.jetbrains.kotlin</groupId>
        <version>${kotlin.version}</version>
        <configuration>
          <compilerPlugins>
            <plugin>spring</plugin>
          </compilerPlugins>
          <jvmTarget>1.8</jvmTarget>
        </configuration>
        <executions>
          <execution>
            <id>compile</id>
            <phase>compile</phase>
            <goals>
              <goal>compile</goal>
            </goals>
          </execution>
          <execution>
            <id>test-compile</id>
            <phase>test-compile</phase>
            <goals>
              <goal>test-compile</goal>
            </goals>
          </execution>
        </executions>
        <dependencies>
          <dependency>
            <groupId>org.jetbrains.kotlin</groupId>
            <artifactId>kotlin-maven-allopen</artifactId>
            <version>${kotlin.version}</version>
          </dependency>
        </dependencies>
      </plugin>
    </plugins>
  </build>


</project>

The code

In this post I will only show the relevant lines from the code, I will skip package and import statements, the full code is available at GitHub.

The application class

The application class created by the initializr just gets one additional line:

@SpringBootApplication
@EnableKafka
class KovasbakApplication

fun main(args: Array<String>) {
    SpringApplication.run(KovasbakApplication::class.java, *args)
}

The @EnableKafka annotation is used to tell Spring Boot to pull in the kafka related classes and libs.

The UI classes

ChatDisplay

The ChatDisplay is the Panel displaying the chat messages. I first used a TextArea, but had problems with programmatically scrolling to the bottom. So I created this small class that uses a Label to display the data:

class ChatDisplay : Panel() {
    val text: Label

    init {
        setSizeFull()
        text = Label().apply { contentMode = ContentMode.HTML }
        content = VerticalLayout().apply { addComponent(text) }
    }

    fun addMessage(user: String, message: String) {
        text.value = when {
            text.value.isNullOrEmpty() -> "<em>$user:</em> $message"
            else -> text.value + "<br/><em>$user:</em> $message"
        }
        scrollTop = Int.MAX_VALUE
    }
}

ChatUI

This is the main UI class:

@SpringUI
@PreserveOnRefresh
@Push
class ChatUI : UI(), KafkaConnectorListener {

    lateinit var user: String
    val chatDisplay = ChatDisplay()
    val userLabel = Label()

    @Autowired
    lateinit var kafkaConnector: KafkaConnector

    // skipping content here....

    companion object {
        val log: Logger = LoggerFactory.getLogger(ChatUI::class.java)
    }
}

It is marked as a Vaadin UI with @SpringUI, @PreserveOnRefresh keeps the session when the browser is reloaded, and @Push marks this for server-push when new messages arrive from Kafka. The class implements an interface KafkaConnectorListener which is described together with the KafkaConnector class.

The ChatUI has the following fields:

  • user: the name of the user that is chatting
  • chatDisplay: the display panel for the messages
  • userLabel: sits at the bottom left to show the name of the user
  • kafkaConnector: used for sending the own messages and to register for getting the messages from kafka

It further has a companion object containing the Logger. I now show the methods of the class:

override fun init(vaadinRequest: VaadinRequest?) {
    kafkaConnector.addListener(this)
    content = VerticalLayout().apply {
        setSizeFull()
        addComponents(chatDisplay, createInputs())
        setExpandRatio(chatDisplay, 1F)
    }
    askForUserName()
}

private fun createInputs(): Component {
    return HorizontalLayout().apply {
        setWidth(100F, Sizeable.Unit.PERCENTAGE)
        val messageField = TextField().apply { setWidth(100F, Sizeable.Unit.PERCENTAGE) }
        val button = Button("Send").apply {
            setClickShortcut(ShortcutAction.KeyCode.ENTER)
            addClickListener {
                kafkaConnector.send(user, messageField.value)
                messageField.apply { clear(); focus() }
            }
        }
        addComponents(userLabel, messageField, button)
        setComponentAlignment(userLabel, Alignment.MIDDLE_LEFT)
        setExpandRatio(messageField, 1F)
    }
}

This sets up the basic layout with the ChatDisplay and the other UI elements, registers the ChatUI with the KafkaConnector. The click handler for the send button is set up so that the user name and the content of the message TextField are sent to the KafkaConnector (see marked line).

After setting up the layout, the user is asked for her name with the following method:

private fun askForUserName() {
    addWindow(Window("your user:").apply {
        isModal = true
        isClosable = false
        isResizable = false
        content = VerticalLayout().apply {
            val nameField = TextField().apply { focus() }
            addComponent(nameField)
            addComponent(Button("OK").apply {
                setClickShortcut(ShortcutAction.KeyCode.ENTER)
                addClickListener {
                    user = nameField.value
                    if (!user.isNullOrEmpty()) {
                        close()
                        userLabel.value = user
                        log.info("user entered: $user")
                    }
                }
            })
        }
        center()
    })
}

This shows a modal window where the user’s name must be entered.

There is a method that is called when the UI is disposed:

override fun detach() {
    kafkaConnector.removeListener(this)
    super.detach()
    log.info("session ended for user $user")
}

The code used to send the actual message to the kafka connector was already shown, the last thing in this class is the code that is called from the KafkaConnector when new messages arrive:

override fun chatMessage(user: String, message: String) {
    access { chatDisplay.addMessage(user, message) }
}

The received data is added to the chatDisplay, but this is wrapped as a Runnable in the UI.access() method for two reasons:

  1. the code is asynchronously from a different thread and must be wrapped to be run on the UI thread.
  2. Executing the code in access() in combination with the @Push annotation on the class results in a server push to the client which is necessary so that the new messages are immediately shown.

The Kafka connector class

All communication with Kafka is wrapped in a Spring Component (thus being a singleton) which just has the following code:

interface KafkaConnectorListener {
    fun chatMessage(user: String, message: String)
}

@Component
class KafkaConnector {

    val listeners = mutableListOf<KafkaConnectorListener>()

    fun addListener(listener: KafkaConnectorListener) {
        listeners += listener
    }

    fun removeListener(listener: KafkaConnectorListener) {
        listeners -= listener
    }

    @Autowired
    lateinit var kafka: KafkaTemplate<String, String>

    fun send(user: String, message: String) {
        log.info("$user sending message \"$message\"")
        kafka.send("kovasbak-chat", user, message)
    }

    @KafkaListener(topics = arrayOf("kovasbak-chat"))
    fun receive(consumerRecord: ConsumerRecord<String?, String?>) {
        val key: String = consumerRecord.key() ?: "???"
        val value: String = consumerRecord.value() ?: "???"
        log.info("got kafka record with key \"$key\" and value \"$value\"")
        listeners.forEach { listener -> listener.chatMessage(key, value) }
    }

    companion object {
        val log: Logger = LoggerFactory.getLogger(KafkaConnector::class.java)
    }
}

First I defined the KafkaConnectorListener interface which the ChatUI class implements so they can be registered for new messages.

The KafkaConnector has a list of listeners and the methods to add and remove listeners. Nothing special here.

For sending a new message to kafka, the send method uses the injected KafkaTemplate (which comes from the spring-kafka library) to send the data to kafka by using the username as key and the message text as payload. The topic name that is used is kovasbak-chat.

By marking the receive method with @KafkaListener the method is called every time when a message in kafka arrives from any client. The data is parsed for the username and message body and the it is sent to all the registered clients. And finally there is a companion object with a Logger.

The configuration

spring.kafka.consumer.group-id=${random.uuid}
spring.kafka.consumer.auto-offset-reset=latest
spring.kafka.bootstrap-servers=localhost:9092

I use a random kafka consumer-group id so that each instance of my webapp gets all messages, I am not interested in old messages and define the host and port of the kafka broker.

Fire it up

You can either run the program from within the IDE or go to the command line and:

mvn package
java -jar target/kovasbak-0.0.1-SNAPSHOT.jar

you can then as well start a second instance on a different port like and access the servers on both localhost:8080 and localhost:8081

java -jar target/kovasbak-0.0.1-SNAPSHOT.jar --server.port=8081

Conclusion

To sum it up: with just a handful of code lines we have a scalable web-based chat-service which uses a scalable backend for message processing.

mapjfx 1.13.1 using OpenLayers 4.2.0

I just released mapjfx version 1.13.1 it should be found shortly at maven central, the artifact coordinates are:

  <dependency>
    <groupId>com.sothawo</groupId>
    <artifactId>mapjfx</artifactId>
    <version>1.13.1</version>
  </dependency>

The source is available at GitHub.

Now uses OpenLayers 4.2.0.

Comments and contributions welcome.

the shortest code to throw a NullPointerException

If I were to throw a NullpointerException from within my code I normally would code something like this:

if(somethingIsNull) {
  throw new NullPointerException();
}

While reading the code examples on https://github.com/reactive-streams/reactive-streams-jvm I just saw this definitely shorter possibility which I did not know up to now:

if(somethingIsNull) {
  throw null;
}

mapjfx 1.13.0 adds the possibility to exclude URLs from being cached

I just released mapjfx version 1.13.0 it should be found shortly at maven central, the artifact coordinates are:

  <dependency>
    <groupId>com.sothawo</groupId>
    <artifactId>mapjfx</artifactId>
    <version>1.13.0</version>
  </dependency>

The source is available at GitHub.

The offline cache now can be configured with a collection of Strings (Java RegExp patterns) so that URLs matching any of these will not be cached.

 

Comments and contributions welcome.

rxjava2 producer-consumer example with backpressure implementation

This post is a small example how to implement a producer and consumer using rxjava2. The producer produces a sequence of Integer objects which are consumed by the consumer. The producer only produces as much items as are requested by the reactive stream. The reader of this post should be familiar with the reactive streams concept.

project setup

I used a maven project, the only dependency needed for this example besides logging is rxjava:

<dependency>
    <groupId>io.reactivex.rxjava2</groupId>
    <artifactId>rxjava</artifactId>
    <version>2.0.8</version>
</dependency>

the producer

The producer is initialized with the integer values defining the range that is to be produced. It extends the Flowable<Integer> class and therefore must implement the subscribeActual method. In this method a new Subscription object is created which keeps track of the current value to be produced and which emits the values to the subscriber on request. By implementing the Subscription as an anonymous class it is easily possible to access the subscriber that the subscription belongs to. Care must be taken as the call to the subscribers onNext() method can result in a new recursive call to request() by the subscriber.

/**
 * class to produce Integer values using a Flowable.
 */
class TestProducer extends Flowable<Integer> {
    static final Logger logger = LoggerFactory.getLogger(TestProducer.class);
    final int from, to;

    public TestProducer(int from, int to) {
        this.from = from;
        this.to = to;
    }

    @Override
    protected void subscribeActual(Subscriber subscriber) {
        subscriber.onSubscribe(new Subscription() {

            /** the next value. */
            public int next = from;
            /** cancellation flag. */
            private volatile boolean cancelled = false;
            private volatile boolean isProducing = false;
            private AtomicLong outStandingRequests = new AtomicLong(0);

            @Override
            public void request(long n) {
                if (!cancelled) {

                    outStandingRequests.addAndGet(n);

                    // check if already fulfilling request to prevent call  between request() an subscriber .onNext()
                    if (isProducing) {
                        return;
                    }

                    // start producing
                    isProducing = true;

                    while (outStandingRequests.get() > 0) {
                        if (next > to) {
                            logger.info("producer finished");
                            subscriber.onComplete();
                            break;
                        }
                        subscriber.onNext(next++);
                        outStandingRequests.decrementAndGet();
                    }
                    isProducing = false;
                }
            }

            @Override
            public void cancel() {
                cancelled = true;
            }
        });
    }
}

the consumer

The consumer extends DefaultSubscriber<Integer>. On start, it requests one item, and after processing the next item, it requests the next one. To build up a little backpressure on the producer, there’s a little delay every 5 items.

/**
 * TestConsumer derived from DefaultSubscriber. requests one item at a time.
 */
class TestConsumer extends DefaultSubscriber<Integer> {

    private static final Logger logger = LoggerFactory.getLogger(TestConsumer.class);

    @Override
    protected void onStart() {
        request(1);
    }

    @Override
    public void onNext(Integer i) {
        logger.info("consuming {}", i);
        if (0 == (i % 5)) {
            try {
                Thread.sleep(500);
            } catch (InterruptedException ignored) {
                // can be ignored, just used for pausing
            }
        }
        request(1);
    }

    @Override
    public void onError(Throwable throwable) {
        logger.error("error received", throwable);
    }

    @Override
    public void onComplete() {
        logger.info("consumer finished");
    }
}

 putting it together

In the main method of the test program, a producer and a consumer are created. Then the producer is set to run on the Schedulers.computation() threadpool and emit its value on the Schedulers.single() thread to make producing and consuming parallel. The consumer does a blocking subscribe to keep the main thread running until all items are processed.

public static void main(String[] args) {
    try {
        final TestProducer testProducer = new TestProducer(1, 1_000);
        final TestConsumer testConsumer = new TestConsumer();

        testProducer
                .subscribeOn(Schedulers.computation())
                .observeOn(Schedulers.single())
                .blockingSubscribe(testConsumer);

    } catch (Throwable t) {
        t.printStackTrace();
    }
}

Basically that is all that is needed for a simple reactive producer-consumer implementation.

mapjfx 1.12.2 – no default caching

I just released mapjfx version 1.12.2 it should be found shortly at maven central, the artifact coordinates are:

  <dependency>
    <groupId>com.sothawo</groupId>
    <artifactId>mapjfx</artifactId>
    <version>1.12.2</version>
  </dependency>

The source is available at GitHub.

The test program in the library and the mapjfx-demo now do not use the offline cache by default.

The caching seems to get problems the more the backing servers are using https. And Bing for example uses https calls to validate the token, which should not be cached.

Alas it is not possible only to cache the map image data, because only from the url being retrieved, there is no possibility to decide wether it’s image data or something other.

So the best seems to be not use the cache by default.

Comments and contributions welcome.

mapjfx 1.12.1 with fixes to the code for caching data

I just released mapjfx version 1.12.1 it should be found shortly at maven central, the artifact coordinates are:

  <dependency>
    <groupId>com.sothawo</groupId>
    <artifactId>mapjfx</artifactId>
    <version>1.12.1</version>
  </dependency>

The source is available at GitHub.

I implemented some improvemtnes to the code that does the caching of map images.

 

Comments and contributions welcome.

mapjfx 1.12.0 has a custom event for selecting a map area

I just released mapjfx version 1.12.0 it should be found shortly at maven central, the artifact coordinates are:

  <dependency>
    <groupId>com.sothawo</groupId>
    <artifactId>mapjfx</artifactId>
    <version>1.12.0</version>
  </dependency>

The source is available at GitHub.

The MapView now triggers a custom event when the user drags a rectangle while holding the cmd key (on Mac OSX) or the ctrl key (on Windows). This can be used to set the extent of the map to the selected extent, sample code from the mapjfx-demo app:

mapView.addEventHandler(MapViewEvent.MAP_EXTENT, event -> {
            event.consume();
            mapView.setExtent(event.getExtent());
});

 

Comments and contributions welcome.

mapjfx 1.11.0 using OpenLayers 4.0.1

I just released mapjfx version 1.11.0 it should be found shortly at maven central, the artifact coordinates are:

  <dependency>
    <groupId>com.sothawo</groupId>
    <artifactId>mapjfx</artifactId>
    <version>1.11.0</version>
  </dependency>

The source is available at GitHub.

Now uses OpenLayers 4.0.1.

Comments and contributions welcome.