As organizations scale their data operations, one truth becomes clear: Data Quality and Observability (DQO) are no longer “nice to have” — they are mission-critical.
At Lightup, we’ve taken a deep dive into the architecture of our platform and discovered more opportunities than we originally imagined. Each layer — from ingestion to visualization — presents unique challenges at scale, and we’ve been systematically upgrading every one of them.
The result? A new wave of features that radically improve both automation and control — enabling faster time-to-value and safer enterprise-wide collaboration.
Why Not Just Write SQL or Use dbt?
We hear this a lot. And if that works for you on a one-off basis at a small scale, great.
The real challenges show up at scale. Scale really compounds the complexity of Data Quality metric authorship, management, and maintenance.
Why? As your data environment grows, complexity grows exponentially at every layer:
- Data sources proliferate across dispersed data warehouses, multiple lakehouses, object stores, streaming platforms, and SaaS tools.
- Metric computation balloons to billions of rows, making full table scans unaffordable and untenable. These hard-coded metrics sliced by business dimensions (per store, per product, per brand, etc.) become impractical to implement manually.
- Anomaly detection monitors get noisy without advanced, dimensional logic.
- Incident workflows require cross-team collaboration, demanding tight integration with tools like ServiceNow, Jira, Teams, and PagerDuty.
- Remediations and reporting need to be automated and scalable across teams.
That’s where Lightup’s newest features come in.
What’s New in Lightup?
Here’s a preview of some of the latest enhancements in Lightup:
- Virtual Tables
- Unsupervised Learning for Sliced Metric Monitors
- Enhanced reporting with richer email alerts, CSV exports, and customizable dashboards
- Zero-Config Auto Metrics
- Data Quality for SaaS applications
- Kafka supported as a data source for streaming Data Quality checks
- Data Lineage
- Genie, Lightup’s AI Assistant
- LLM-powered duplicate data and PII detection
- Unstructured Data Quality
Many of these features already leverage AI and LLMs. And there are many more on the way…
Webinar
We’re excited to share a roundup of our latest feature updates and a product demo in our rescheduled webinar on Wednesday, August 6, 2025, at 10:00 am – 11:00 am PT.
You’ll get a front-row seat to see how Lightup is pushing the boundaries of scalable Data Quality and Observability for enterprises. Plus, we’ll reveal what’s coming soon and what to expect next from Lightup.
Know someone who should attend? Forward this blog to them and we’ll see you there!
