Apache ZooKeeper the Stackable Way

devop |

Welcome to the second part of our series "Hello, Stackable Data Platform" an introduction to the Stackable Data Platform - it's about Apache ZooKeeper.

This document describes how to use:

Apache ZooKeeper “is an effort to develop and maintain an open-source server which enables highly reliable distributed coordination”

Install Apache ZooKeeper itself

As in the forerunner Stackable Building-Block Authorization-as-Code we'll use helmfile to get started quickly:

# Stackable Operator for Apache ZooKeeper

# https://docs.stackable.tech/home/getting_started.html#_installing_stackable_operators

repositories:
  # helm repo add stackable-devel https://repo.stackable.tech/repository/helm-dev/
  - name: stackable-devel
    url: 'https://repo.stackable.tech/repository/helm-dev/'

releases:
  # helm install zookeeper-operator stackable-devel/zookeeper-operator --version 0.9.0-mr338
  - name: zookeeper-operator
    devel: true
    chart: stackable-devel/zookeeper-operator
    # https://repo.stackable.tech/#browse/search/helm=name.raw%3Dzookeeper-operator
    version: "0.9.0-mr338"
    installed: true

Tip: Please find the helm commands provided as inline documentation, in case you prefer plain helm commands.

$ kubectl apply -f k8s/zookeeper-cluster.yaml
zookeepercluster.zookeeper.stackable.tech/stackable-zookeeper created

$ kubectl get pods
NAME                                                        READY   STATUS              RESTARTS   AGE
stackable-zookeeper-operator-9c4694444-rhvvh                1/1     Running             0          8m44s
zookeeper-stackablezookeeper-server-default-zookeep-2r7lf   1/1     Running             0          2m8s
zookeeper-stackablezookeeper-server-default-zookeep-6r42q   1/1     Running             0          69s
zookeeper-stackablezookeeper-server-default-zookeep-ndc5c   0/1     ContainerCreating   0          23s

Test the ZooKeeper Installation

The easiest way seems to jump into the ZooKeeper pod.

Inspired by ZooKeeper Getting Started Guide you could run basic checks inside the pod like:

$ ZOOKEEPER_NAME=$(kubectl get pods \
  --selector app.kubernetes.io/name=zookeeper -o json \
  | jq -r '.items[0].metadata.name')

$ kubectl exec --stdin --tty ${ZOOKEEPER_NAME} \
  -- /bin/bash -c "bin/zkCli.sh -server 127.0.0.1:2181"

You should get a similar output with a prompt at the end:

...
Connecting to 127.0.0.1:2181
...
Welcome to ZooKeeper!
...
[zk: 127.0.0.1:2181(CONNECTED) 0]

You can try such a command with the following output:

[zk: 127.0.0.1:2181(CONNECTED) 0] ls /
[zookeeper]

More ZooKeeper commands to explore the installation:

  • create /zk_test my_data
  • get /zk_test
  • delete /zk_test

First Contact with Spring Boot

A basic Spring Cloud Zookeeper app only needs a few dependecies:

dependencies {
    implementation 'org.springframework.cloud:spring-cloud-starter-zookeeper'
    implementation 'org.apache.curator:curator-framework:5.1.0'
}

...then you can list the ZooKeeper content with a CommandLineRunner:

@SpringBootApplication
public class ZooKeeperList implements CommandLineRunner {

    private static final Logger LOG = LoggerFactory.getLogger(ZooKeeperList.class);

    public static void main(String[] args) {
        SpringApplication.run(ZooKeeperList.class, args);
    }

    @Override
    public void run(String... args) throws Exception {
        CuratorFramework curator = CuratorFrameworkFactory.builder()
                .retryPolicy(new RetryOneTime(500))
                .connectString("localhost:2181").build();
        curator.start();
        List<String> children = curator.getChildren().forPath("/");
        for (String child : children) {
            LOG.info("Found child: {}", child);
        }
    }
}

Without further ado, this results in:

...
Found child: zookeeper
Found child: druid
Found child: zk_test
...

Hint: For more details please check Spring Cloud Zookeeper - documentation

👏 Congratulations! The first tool of your modern data platform is up and running. The next installment of this series will tackle an Apache Kafka installation. Stay tuned...

Additional resources

🙌 Photo by Anthony Yin on Unsplash