Varigence builds automation software for the modern data stack, empowering teams to rapidly design, deploy, and own data solutions across the MSFT ecosystem.
Joined September 2009
- Tweets 1,985
- Following 346
- Followers 468
- Likes 179
1,245 Photos and videos
23m
The row that violates your schema should fail the build, not surface in the lakehouse at 2am.
Declare types, nullability, and allowed values in metadata, and BimlFlex turns them into validation in the build. Break a rule and it fails before deploy.
varigence.com/bimlflex?utm_s…
2
Vendor concentration sits on the risk register for years marked 'mitigating controls in progress.'
With BimlFlex your warehouse lives as metadata. If a target stops being the right bet, regenerate native code for the other.
varigence.com/bimlflex?utm_s…
10
'i own this code,' said no one who licensed a closed runtime they were not allowed to look inside.
you rented the verb. they kept the noun.
7
A pipeline that only runs on a vendor's runtime is a liability you re-price at every renewal. With Informatica, the day the renewal triples, your options are pay or rebuild. Own the generated code and the renewal is just a renewal.
25
10h
An unenforceable schema promise is the change-order fight you have in month three. A Slack thread is not an artifact anyone can hold you to.
BimlFlex makes the contract a real deliverable: the shape, the rules, the Snowflake DDL it generates.
varigence.com/bimlflex?utm_s…
7
12h
nobody:
absolutely nobody:
a hashdiff at 2am: actually, these two rows are completely different and i will not be elaborating
6
15h
Modernizing SSIS shouldn't mean trading one black box for another. BimlFlex generates native ADF pipelines and SQL from your metadata, landing in your tenant as code you own and can read. No runtime to license once the move is done.
varigence.com/bimlflex?utm_s…
13
17h
AI-written ETL that ships SQL nobody can audit is a liability dressed as a feature.
If you can't read what ran, you can't defend it to a regulator or debug it at 2am. Generated code you can open is the only kind that survives 'why did this row change.'
1
13
19h
changelog: "lakehouse vs warehouse, explained"
v1: drew the box-and-arrow diagram
v2: same diagram, slower
v3: same diagram, with a metaphor
v4: same diagram. it's friday. they're going to ask again monday.
10
23h
The warehouse bill is a code-quality problem wearing a procurement costume.
You renegotiate credits once a year. You pay for every full scan that should have been incremental on every load, forever.
The cheapest query is the one the generated SQL never made you run twice.
10
Jun 14
the pipeline ran green for six months. nobody touched it. nobody thanked it.
then someone added one nullable column and the whole thing remembered it was load-bearing duct tape the entire time.
21
Jun 13
the three stages of opening someone else's ssis package:
1. "ok this isn't so bad"
2. *opens the data flow*
3. quiet. just quiet.
25
Jun 13
Every consultancy rebuilds the same loaders on every engagement. New client, same staging logic, no leverage.
BimlFlex turns your accumulated patterns into a metadata layer that ships with each project. The first three weeks of re-coding come back as margin.
6
Jun 13
Lineage you document after the fact is a guess. Lineage you generate is a fact.
Build a Data Vault in the Databricks lakehouse with BimlFlex and the trail is generated with it: every satellite traces to its hub, hub to source.
Audit-ready, not later. varigence.com/bimlflex?utm_s…
15
Jun 13
An LLM can produce SQL that looks right. It can't promise the same input gives the same output tomorrow.
Deterministic generation can: same metadata, same code, every run. Regeneration becomes a safe CI step, and drift shows in the diff, not production.
varigence.com/bimlflex?utm_s…
14
Jun 13
Moving an SSIS estate to Fabric does not have to mean rewriting packages.
Keep the metadata, change the target. BimlFlex regenerates the same model into native Fabric Lakehouse notebooks and Warehouse T-SQL. The patterns that were right in SSIS are still right in Fabric.
31
Jun 13
You've sat through the demo where AI writes production SQL on stage. It looks great until someone has to audit what actually ran.
AI is good at suggesting a mapping, bad at emitting the same correct SQL every run. Keep AI on the design side. Keep generation on the build side.
21
Jun 13
the four horsemen of the broken pipeline:
timezone, who arrives one hour off.
nullable, who promised it would never be empty.
encoding, who turned every name into a question mark.
and that one regex, who answers to no one.
1
19
Jun 13
AI's right job in a warehouse build isn't writing SQL nobody owns. It's the design side: the mapping, the column matches, where a human confirms intent.
Keep AI on design. Keep generation on the build side, deterministic: same metadata, same code.
varigence.com/bimlflex?utm_s…
15
Jun 12
A data contract isn't written once. It changes when the schema does.
Leave it in a wiki and the only version that matters is the one in someone's head. Put it in the generator and a build that would break it doesn't ship. The rule travels with the schema.
10