PII Protection Upgrade! 🛡️
We’re thrilled to unveil the latest feature for the Streamdal pipeline engine:
✨ Automatic Detection and Masking of PII ✨
Much less manual hassle! With a few clicks, safeguard sensitive info like email addresses or phone numbers automatically in your data pipelines. 🚀
🔐 Streamlined Security: Effortlessly obscure PII data to and from external systems.
🔧 Simplified Workflows: Greatly simplify custom steps; this feature detects PII dynamically in payloads!
🎯 Precision Matters: Tested rigorously to detect and mask PII, even in complex JSON structures.
📹 Curious? @jacobheric got a quick video walkthrough for you!
#DataSecurity#PII
💡 Ecosystem highlight of the week: prisma-extension-streamdal
@streamdal is an open-source tool to embed privacy controls into app code. With this Prisma Client extension, you can code native data pipelines for Prisma Client queries.
👉 Check it out at github.com/streamdal/prisma-…
💡Streamdal is all about automating mundane and annoying tasks…
That's why we're excited to introduce our newest enhancement 🎉: a Terraform provider used for stamping down pipelines and various data rules - no more manual clicking around in a web UI!
👀Check out our blog post by our Lead Software Engineer, Mark Gregan (@marqueetag) for a step-by-step guide on getting your pipeline setup with our Terraform provider (the link is in the following thread) 🔗👇
#OpenSource#Terraform#CodeNative#PII#DataPrivacy
🔍 Curious about how Streamdal's mission to democratize real-time data pipelines connects with our latest update?
🌐 Dive into our blog post to uncover the magic! ✨
🌟 Discover how our open-source shim for KafkaJS is revolutionizing data pipelines, empowering any application to seamlessly integrate sophisticated functionalities with just a few configuration tweaks 🚀💻
💡 Ready to streamline your data operations and unlock new possibilities? Check the links in the following thread 🔗 👇
#OpenSource#Kafka#JavaSkript#DataEngineering#DataQuality
🔍 Curious about how Streamdal's mission to democratize real-time data pipelines connects with our latest update?
🌐 Dive into our blog post to uncover the magic! ✨
🌟 Discover how our open-source shim for KafkaJS is revolutionizing data pipelines, empowering any application to seamlessly integrate sophisticated functionalities with just a few configuration tweaks 🚀💻
💡 Ready to streamline your data operations and unlock new possibilities? Check the links in the following thread 🔗 👇
#OpenSource#Kafka#JavaSkript#DataEngineering#DataQuality
Your favorite libraries, but now with Streamdal!
We are on a roll with these, and plan to continue to pump these out regularly:
🌟streamdal/ibm-sarama
🌟streamdal/kafkajs
🌟streamdal/segmentio-kafka-go
🌟streamdal/confluent-kafka-python
🌟streamdal/confluent-kafka-go
🔮 Imagine a dev world where data privacy and compliance changes are made immediately and directly in your app, development is empowered, and costs are slashed.
👀Check out the future of data privacy - our CTO Dan Selans (@skey) breaks down the code-native concept
Find the link in the thread to get an idea about this modern approach to dealing with data privacy-related issues 🔗 👇
#DataPrivacy#SoftwareDevelopment#SoftwareEngineering
🛡️Tired of playing catch-up with data security drama?
Streamdal CTO, Dan Selans (@skey), lays down the facts on Upstream Data Security. Learn what it’s all about and why you should care about it.
Dive into the insights 🔗👇
#DataPrivacy#PII#DataEngineering#Governance
🚨 Video Walkthrough Alert
Join our lead engineer, @jacobheric, as he takes you through the newest feature on the Streamdal Code-Native platform: HTTP Requests.
Explore this exciting addition 👇
#CodeNative
Ready for full Node mode? Whether you need to observe, operate on data, or are looking for something clean and fresh for your project, check out our Node SDK.
Similar to our other SDKs, it’s open-source and used to enable data observability and code-native data pipelines. You’ll find this tech in it:
✅Protobuf - Typescript/Javascript gRPC and protocol buffers for Node.
✅Vitest (@vitest_dev) & Sinon - Great for testing!
And of course, a lot more 🔗👇
#FOSS#FossFeb#opensource#Node#protobuf
Thrilled to share a recent endeavor – a comprehensive video walkthrough showcasing the ease of extracting, truncating, and adding metadata with Streamdal 🎥✨
This powerful technique allows you to easily filter out messages or data that may not be intended for downstream endpoints such as Logstash, Snowflake, S3, Postgres, and more.
Check it out! 🚀
#DataEngineering#DevOps#DataManagement#Engineering
More @golang inspiration for those developers young and old, learned or aspiring!
To visualize read/write, observe data, and sync rules for workflows and pipelines, we needed a way to communicate with the frontend.
We built that server functionality in Go.
Like all of our components, the server is open-source. Some particulars you’ll see:
✅gRPC (@grpcio) Protobuf - We use both unary and streaming functionality for communication.
✅Redis (@Redisinc) - We specifically use redis for Pubsub and its traditional key/value functionality.
✅Go-director - This library enables us to declare for loops as injectable dependencies, which greatly simplifies testing.
The code 🔗👇
#FOSS#FossFeb#opensource#go#redis
🤔 Data anonymization vs redaction when handling PII - Is one more ideal than the other?
They both play an important role, but it's all about context!
🔴 Redact PII for sharing & compliance.
🟢 Anonymize PII for long-term storage & utility.
You should redact PII or PHI fields in data when:
✅Sending data to 3rd parties
✅Complying with regulations like HIPAA
✅Ensuring readability while preserving data context
You should anonymize or obfuscate PII or PHI fields in data when:
✅Storing for long-term retention
✅Complying with regulations like GDPR
✅Needing to preserve data utility
Your choice hinges on data and regulations. Tools like Streamdal give you the best of both worlds. #DataProtection#PII