I think you are describing Xata Agent :) https://github.com/xataio/agent It watches metrics/logs, queries pg_stat_stetements for slow queries, the sends slack notification with suggestions, including indexes, but can be more varied than that.
We use event triggers both in pgroll [0] and pgstream [1] to capture DDL changes. It's a feature of pgroll that it captures the schema migration history, regardless if you do the changes via pgroll or with another tool. And pgstream uses it to put the schema changes in the logical replication stream.
One thing to be aware is that on many Postgres DBaaS EVENT TRIGGERS are not allowed, because they generally require superuser. But RDS and Aurora do support them, we (Xata) support them of course, and I think Supabase is working on adding support for them.
I would say it's primarily testing: either testing schema changes or testing your application changes. Copy-on-Write branches help because you can spin up an environment fast with relevant data from production (anonymized).
You can try with data subsetting tools, a few that I know are: greenmask, jailer, tonic condenser, and I think there are more. They are not exactly for your use case, but might help you a bit. The problem is most of these tools can walk foreign keys, but only in one directions.
Also, since you said FK are not fully implemented, then there just won't be enough information for them to work with. So you can either work on fixing the foreign keys or have more complex scripts.
Yeah due to the unavailability of FK's, I'd have to somehow map certain ID columns to the tables they're for, accounting for slight naming convention differences.
As it happens, we've just launched our new Xata platform (https://xata.io/) which has some of the key Neon features: instant copy-on-write branching and separation of storage and compute. As an extra twist, we also can do anonymization (PII masking) between your production database and developer branches.
The way we do copy-on-write branches is a bit different. We haven't done any modifications to Postgres but do it completely at the storage layer, which is a distributed system in itself. This also brings some I/O performance opportunities.
While Xata has been around for a while, we're just launching this new platform, and it is in Private Beta. But we are happy to work with you if you are interested.
Several components are open source as their own projects (see below) which will allow you to reproduce most of the features on top of regular Postgres. But the storage part is not open source. We are considering a simpler implementation of it that would be realistic to self-host and still do copy-on-write branching.
These are the open source components:
* pgstream for the anonymization from the production branch
I think when people look at Neon, the Aurora-style disaggregated compute/data architecture allowing highly scalable read replicas on cloud storage is the defining feature, and it's the only such project that offers it for Postgres. So the storage part is the point.
The PII masking aspect is very interesting and something we couldn't get when we decided on DBLab a month ago. What does the deployment model within AWS look like?
If you want to deploy the whole platform inside your own AWS account, we have a Bring Your Own Cloud model: https://xata.io/byoc
If you want to get anonymization from your RDS/Aurora instance and into Xata branches, then you run only a CLI command (`xata clone`) which does something similar to pg_dump/pg_restore but with masking. It is based on our pgstream open source project.
1. Would you sign BAA (for HIPAA) for the Pay As You Go plan? Can't find that anywhere on your site except for that Lite is HIPAA compliant (https://lite.xata.io/security).
2. FYI, couldn't request access via the BYOC form so I sent an email as per the error: There was an error, please try again or contact us at info@xata.io.
Hi @tudorg - do the Xata copy-on-write branches work like Neon in that you effectively get an isolated Postgres cluster, allowing you to test roles, extensions, creating/dropping DBs, altering schema & data of existing DBs, etc? I looked in the docs but it wasn’t clear to me.
Yes, exactly, you get a new isolated Postgres database, just started from essentially a snapshot at the time of the fork. You can test any types of changes.
Do you support http or websocket connections like https://github.com/neondatabase/serverless? In my experience neon is ultra fast that way in serverless environments like 1-5ms per query with network roundtrip.
(Disclaimer: I work at Xata.)
Just wanted to mention that we also support anonymization, in case that’s something you're looking into:
https://xata.io/postgres-data-masking
This is great, I like in particular that there are extensions plugins. I’ll be looking at integrating this in the Xata Agent (https://github.com/xataio/agent) as custom tooling.
Xata.io looks very interesting!!! I was thinking about building an intelligent agent for pg-mcp as my net project but it looks like you did a lot of the hard work already. When thinking about the "AI Stack" I usually separate concerns like this:
