Home

Integrating Event Streaming with Work Breakdown Structure — Procore

When Procore began moving toward service-oriented architecture, the Budget team faced a major performance bottleneck. Work Breakdown Structure (WBS) data was no longer directly queryable from the database — instead, the Budget tool had to fetch it through a public service layer and merge it at runtime, slowing report response times from seconds to frustratingly long waits. The solution: stream WBS events into Kafka, ingest them into a Budget-owned WBS table, and query that table directly in reports. This allowed the Budget tool to remain isolated from the WBS domain while restoring performance.

Back to Projects

WBS Codes on the Budget

Tech Stack: Kafka KafkaConnect Ruby Rails Postgres Debezium Outbox Pattern HTTP Sinks Event Streaming
My Role
  • Completed and productionized the event stream integration initially designed by a Principal engineer.
  • Configured Kafka, KafkaConnect, and Debezium to capture WBS changes and ingest them into a Budget-specific table.
  • Built logic to conditionally execute queries against the new store.
  • Monitored response times with DataDog and SumoLogic dashboards.
  • Documented the implementation extensively in Confluence, enabling other teams to adopt similar patterns.
Impact
  • Cut Budget report response times by 50–100% (from ~5s to 1–2s).
  • Proved out a domain-specific data store pattern, influencing other Financials teams to use event streaming.
  • Strengthened Procore’s broader shift toward microservices and service isolation.
Technical Highlights
  • Implemented batching to handle high-volume event floods caused by nested WBS schemas.
  • Reduced duplication by consolidating nested events, enabling stable Kafka throughput.
  • Applied the Outbox pattern to safely stream changes between domains.
  • Delivered a simplified, report-ready WBS schema purpose-built for Budget.
A Story
The first time I turned on the WBS event stream in production, it nearly overwhelmed the system. We underestimated how many nested events each WBS change could trigger. In real time, I had to design and implement batching and deduplication strategies to stabilize ingestion. Once resolved, seeing the Budget report drop from 5 seconds to under 2 was an incredible moment — and it marked one of my first experiences delivering a Staff-level systems solution.
Docs