Apache Pulsar manager

Apache Pulsar manager is a web-based GUI management tool for managing and monitoring Pulsar.

A Pulsar Manager is capable of managing multiple Pulsar clusters. A Pulsar instance or a group of Pulsar clusters is defined as an Environment in Pulsar Manager. You can create as many environments as you can. The following diagram illustrates the overall architecture of a Pulsar Manager.

A running Pulsar Manager instance is comprised of two components: one is the frontend which provides the GUI to manage and monitor Pulsar clusters, the other one is the backend which interacts with Pulsar clusters for collecting metrics, forwarding admin requests to the requested Pulsar cluster or brokers.

Since Pulsar Manager directly interacts with Pulsar brokers and bookies for management, it is required to deploy the Pulsar Manager into the same network as your Pulsar clusters. So the Pulsar Manager backend is able to talk to the brokers and bookies in your Pulsar cluster.

Install

Use Docker

Start Pulsar standalone.

docker pull apachepulsar/pulsar:latest
docker run -d -it \
    -p 6650:6650 \
    -p 8080:8080 \
    -v pulsardata:/pulsar/data \
    -v pulsarconf:/pulsar/conf \
    --name pulsar-standalone \
    apachepulsar/pulsar:latest \
    bin/pulsar standalone

Start Pulsar Manager in a separate container.

NOTE: the command links the pulsar-manager container with the pulsar standalone container so they are in the same network.

docker pull apachepulsar/pulsar-manager:v0.2.0
docker run -it \
    -p 9527:9527 -p 7750:7750 \
    -e SPRING_CONFIGURATION_FILE=/pulsar-manager/pulsar-manager/application.properties \
    --link pulsar-standalone \
    apachepulsar/pulsar-manager:v0.2.0

NOTE: Enable bookkeeper visual manager(Optional), update the field bkvm.enabled to true for the file bkvm.conf.

docker pull apachepulsar/pulsar-manager:v0.2.0
docker run -it \
    -p 9527:9527 -p 7750:7750 \
    -e SPRING_CONFIGURATION_FILE=/pulsar-manager/pulsar-manager/application.properties \
    -v $PWD/bkvm.conf:/pulsar-manager/pulsar-manager/bkvm.conf \
    --link pulsar-standalone \
    apachepulsar/pulsar-manager:v0.2.0

SPRING_CONFIGURATION_FILE: Default configuration file for spring.

Use Docker Compose

Alternatively, you can start a standalone and Pulsar Manager using the docker compose file provided in this repository.

Clone this git repoistory.

https://github.com/apache/pulsar-manager

Go to the docker directory.
```
cd docker
```
Start the docker composed cluster.
```
docker-compose up
```

Deploy Pulsar Manager to Kubernetes

The Pulsar Manager can be deployed as part of Pulsar Helm Chart.

Install the Pulsar cluster with Pulsar Manager

helm repo add apache https://pulsar.apache.org/charts
helm repo update
helm install <release-name> apache/pulsar

Access the Pulsar Manager.

The default values will create a ClusterIP for the Pulsar Manager you can use to interact with cluster. To find the IP address of the Pulsar Manager, use the following command:
```
kubectl get service -n <k8s-namespace>
```
After find the ip address of the Pulsar Manager, you can access the Pulsar Manager at http://${pulsar-manager-cluster-ip}/#/environments.

Deploy from bin package

wget https://dist.apache.org/repos/dist/release/pulsar/pulsar-manager/pulsar-manager-0.2.0/apache-pulsar-manager-0.2.0-bin.tar.gz
tar -zxvf apache-pulsar-manager-0.2.0-bin.tar.gz
cd pulsar-manager
tar -xvf pulsar-manager.tar
cd pulsar-manager
cp -r ../dist ui
./bin/pulsar-manager

Now, you can access it at the following address: frontend => http://localhost:7750/ui/index.html.

For details, see Set the administrator account and password.

Enable BookKeeper visual manager(optional)

Update the configuration file pulsar-manager/bkvm.conf, and set bkvm.enabled to true.

bkvm address => http://localhost:7750/bkvm

Build from source code

Prerequisites

Java JDK 1.8
Node 10.15.3 or later
Npm 6.4.1 or later
Pulsar 2.4.0 or later

Build instructions

Clone the source code.

git clone https://github.com/apache/pulsar-manager

Build and start the backend.

cd pulsar-manager
./gradlew build -x test
cd build/distributions
unzip pulsar-manager.zip or tar -zxvf pulsar-manager.tar
./pulsar-manager/bin/pulsar-manager

Open a new terminal, build and start the front end.

cd pulsar-manager/front-end
npm install --save
npm run dev

After running these steps, the Pulsar Manager is running locally at http://127.0.0.1:9527/#/environments.

Access Pulsar Manager

Access Pulsar manager UI at http://${frontend-end-ip}/#/environments.

If you started Pulsar Manager using docker or docker-compose, the Pulsar Manager is running at port 9527. You can access the Pulsar Manager UI at http://127.0.0.1:9527/#/environments.

If you are deploying Pulsar Manager 0.1.0 using the released container, you can log in the Pulsar Manager UI using the following credentials.
- Account: pulsar
- Password: pulsar
If you are deploying Pulsar Manager using the latest code, you can create a super-user using the following command. Then you can use the super user credentials to log in the Pulsar Manager UI.
```
CSRF_TOKEN=$(curl http://backend-service:7750/pulsar-manager/csrf-token)
curl \
    -H "X-XSRF-TOKEN: $CSRF_TOKEN" \
    -H "Cookie: XSRF-TOKEN=$CSRF_TOKEN;" \
    -H 'Content-Type: application/json' \
    -X PUT http://backend-service:7750/pulsar-manager/users/superuser \
    -d '{"name": "admin", "password": "apachepulsar", "description": "test", "email": "username@test.org"}'
```
- backend-service: The IP address or domain name of the backend service.
- password: The password should be more than or equal to 6 digits.
Create an environment.

An environment represents a Pulsar instance or a group of clusters you want to manage. A Pulsar Manager is capable of managing multiple environments.
- Click "New Environment" button to add an environment.
- Input the "Environment Name". The environment name is used for identifying an environment.
- Input the "Service URL". The Service URL is the admin service url of your Pulsar cluster.
  - You need to make sure the service url that Pulsar Manager is able to access. In this example, both pulsar container and pulsar-manager container are linked. So you can use pulsar container name as the domain name of the pulsar standalone cluster. Thus you can type http://pulsar-standalone:8080.
- Input the "Bookie URL". In this example, you can type http://pulsar-standalone:6650

Configure Pulsar Manager

Back end

For more information about the back end, see pulsar-manager-backend.

Front end

For more information about the front end, see pulsar-manager-frontend.

Features

Tenants Management
Namespaces Management
Topics Management
Subscriptions Management
Brokers Management
Clusters Management
Dynamic environments with multiple changes
Support JWT Auth

Log in

Use the default account (pulsar) and the default password (pulsar) to log in.

Configure environment

The puslar-manager supports multiple environment configurations and can manage multiple environments conveniently.

Here, the service URL represents the service IP address of the broker. If you run Pulsar manager in the standalone mode, it should be set to "http://127.0.0.1:8080". You can easily find it in the client.conf file of your pulsar-manager.

And the bookie URL represents the service IP address of the bookkeeper. If you run Pulsar manager in the standalone mode, it should be set to "http://127.0.0.1:6650".

Manage tenants

Manage namespaces

Manage topics

Manage subscriptions

Manage clusters

Manage brokers

Topics monitoring

The pulsar-manager can monitor topics and subscriptions.

Manage token

Casdoor

Casdoor Installation

You can use casdoor to realize sso.

Casdoor can connect to Pulsar-manager simply.

Because the code for connecting the casdoor has been added in Pulsar-manager, we need to configure the casdoor in the back-end and front-end.

Step1. Deploy Casdoor

Firstly, the Casdoor should be deployed.

You can refer to the Casdoor official documentation for the Casdoor

After a successful deployment, you need to ensure:

The Casdoor server is successfully running on http://localhost:8000.
Open your favorite browser and visit http://localhost:7001, you will see the login page of Casdoor.
Input admin and 123 to test login functionality is working fine.

Then you can quickly implement a casdoor based login page in your app with the following steps.

step2. Configure Casdoor

Configure casdoor can refer to casdoor(Configure casdoor's browser better not use one browser with your develop browser).

You also should configure the organization, and application, you also can refer to casdoor.

step2.1 you should create an organization

step2.2 you should create an application

Step3. Configure back-end code

You should configure casdoor's Configuration in the Line 154 of pulsar-manager/src/main/resources/application.properties

casdoor.endpoint=http://localhost:8000
casdoor.clientId=<client id in previous step>
casdoor.clientSecret=<client Secret in previous step>
casdoor.certificate=<client certificate in previous step>
casdoor.organizationName=pulsar
casdoor.applicationName=app-pulsar

Step4. Configure front-end code

You also need configure casdoor's Configuration in the Line 50 of pulsar-manager/front-end/src/main.js

const config = {
  serverUrl: "http://localhost:7001",
  clientId: "6ba06c1e1a30929fdda7",
  organizationName: "pulsar",
  appName: "app-plusar",
  redirectPath: "/#callback",
};

Now you can use Casdoor.

Development

Default Test database HerdDB

Introduction

Pulsar Manager bundles JDBC Drivers for HerdDB. The default configuration starts and embedded in-memory only HerdDB database.

HerdDB can be used in production, you just have to use the correct JDBC URL. Follow the instructions in application.properties to switch the connection to a standalone HerdDB service or cluster.

The JDBC URL will look like this: jdbc:herddb:server:localhost:7000

In cluster mode HerdDB uses Apache BookKeeper and Apache ZooKeeper to store data and metadata, you can share your ZooKeeper cluster and the Bookies bundled with Pulsar.

The JDBC URL will look like this: jdbc:herddb:zookeeper:localhost:2181/herddb

In order to start and setup an HerdDB database follow the instructions on the HerdDB documentation.

Fanatics/pulsar-manager