Microservices Tracing

One request, two Kafka hops, three services — one trace.

A small distributed system built to show OpenTelemetry tracing across HTTP, PostgreSQL and Kafka. Two Spring Boot services use auto-instrumentation; a Go consumer creates its spans by hand. Every hop lands in the same trace in Uptrace. Runs locally with Docker Compose.

View on GitHub Read the README
Architecture diagram: an HTTP request enters the Spring Boot API service, which writes to PostgreSQL and publishes to Kafka; a Spring Boot consumer and a Go consumer read from Kafka; all three export traces to the OpenTelemetry collector and on to Uptrace

Start in three steps

You need Docker, Java 23 with Maven, and Go. Taskfile is optional but saves typing.

1 · Bring up the infrastructure

docker-compose up -d starts PostgreSQL, Zookeeper, Kafka, Kafka UI, the OpenTelemetry collector, ClickHouse and Uptrace.

2 · Run the three services

One terminal each: task start-service-1, task start-service-2, task start-service-3. Without Taskfile: mvn clean compile package then java -jar for the two Spring apps, and go run cmd/consumer/main.go for the Go one.

3 · Send a request, open Uptrace

POST a student to localhost:8080/api/v1/student with curl or the bundled test.http. Then open Uptrace at localhost:14318 and Kafka UI at localhost:8090.

curl -X POST http://localhost:8080/api/v1/student \
  -H "Content-Type: application/json" \
  -d '{"firstname": "Gaspard", "lastname": "Proust"}'

How it works

Two ways of getting spans out of a service, side by side in one system.

Auto-instrumentation on the Java side

The API service and the Spring Kafka consumer pull in opentelemetry-spring-boot-starter. HTTP handling, JPA calls to PostgreSQL and Kafka produce/consume get spans without touching the business code. The trace context rides along in Kafka message headers.

Uptrace overview screen listing the services and their traced operations

Manual spans on the Go side

The Go consumer uses the OpenTelemetry SDK directly with confluent-kafka-go. For every message it reads the trace context from the headers, opens its own span, and exports it over OTLP gRPC — so the Go hop shows up as a child of the same trace the Java services started.

A single trace in Uptrace showing spans from the HTTP request through the database insert, Kafka produce, and the consumer services

Errors stay attached to the request

When something fails downstream, the error log and the failing span are part of the same trace as the original HTTP call, so you can walk from the 500 back to the line that raised it.

Uptrace log view showing an error entry with its trace id and attributes

The whole waterfall in Uptrace

Uptrace sits on ClickHouse and receives everything through the OpenTelemetry collector on ports 4317/4318. Latency, error rate and span details per service, one screen.

Uptrace trace waterfall with a red error span highlighted among the request's spans

The flow, hop by hop

What each service does and the spans it produces.

  1. api-service (Spring Boot) — receives POST /api/v1/student, inserts the row into PostgreSQL, publishes the same data to the student topic. Spans: HTTP POST, Database Insert, Kafka Produce.
  2. spring-app-consumer (Spring Boot) — consumes from student, updates the row with the student's fees, publishes the enriched entity to student-billing. Spans: Kafka Consume, Message Processing, linked to the original trace via headers.
  3. go-consumer (Go) — consumes from student-billing and logs the message. Span: Kafka Consume, manually created and linked to the same trace.

Infrastructure from docker-compose.yml: PostgreSQL, Confluent Kafka 7.7.1 + Zookeeper, Kafka UI, OpenTelemetry Collector 0.111, Uptrace 1.7.7, ClickHouse 24.7.