@apachewayang profile picture
Apache Wayang @apachewayang

Apache Wayang is the API of AI, cross-platform data processing to train large models. Star / fork us on GitHub: github.com/apache/incubator-…

Joined April 2021
  • Tweets 77
  • Following 28
  • Followers 93
5 Photos and videos
Apache Wayang isn’t just limited to FedML and Data Federation; it also enhances our taste and lifestyle. Solving real world data science problems! youtube.com/watch?v=4f24QSaj…

Whisky Clustering with Apache Projects: Groovy, Commons Math, Ignite,...

Calling all developers with a penchant for fine whiskey! Join Dr. P...

youtube.com
1
1
91
Apache Wayang retweeted
Alexander Alten
@mapredit
5 Feb 2025
Replying to @apachewayang
@apachewayang 1.0.0 build successful! The backbone for AI, decentralized data processing to access data in silos a dev can't reach typically.
1
1
1
59
Apache Wayang retweeted
Alexander Alten
@mapredit
5 Feb 2025
Replying to @apachewayang
@apachewayang v. 1.0.0 successfully compiled. The backbone for ASI / AGI infrastructure, devs can now reach data which is typically not in reach, like protected silos, and execute processing at the source, in-situ. FedAI is coming soon!
mvn clean install screenshot

ALT mvn clean install screenshot
1
2
73
LLM's struggle to show their business benefit. Why? Data access. AI needs the right data, in a suitable format at the right time. Read how @apachewayang helps to solve that: wayang.apache.org/blog/wayan…

Wayang and the Federated AI | Apache Wayang™

AI systems and applications are widely used nowadays, from assisting grammar spellings to

wayang.apache.org
1
71
Our newest blogpost, written by Wayang developer Juri Petersen, explains how python works with Apache Wayang, and how we enable JSON conform UDF within a wayangPlan. #data #opensource #apachewayang wayang.apache.org/blog/wayan…

Pywayang - Apache Wayang's Python API | Apache Wayang™

In the vast landscape of data processing, efficiency and flexibility are

wayang.apache.org
41
Read part 1 of the blog series, written by Dr. Mirko Kämpf from ecolytiq, about implementing a cross-organization decentralized data processing scenario. #apachewayang #kafka #data wayang.apache.org/blog/kafka…

Apache Kafka meets Wayang - Part 1 | Apache Wayang™

Intro

wayang.apache.org
2
55
EarthObservation / ESA / Copernicus: AGORA plugin (Apache Wayang): meetingorganizer.copernicus.…
1
1
55
Integrate AI with IoT, the federated way: youtube.com/watch?v=dOnsEfVG… #iot #spark #bigdata

Apache Wayang: Bringing Big Data to the IoT - Jorge-Arnulfo Quiane-...

Thank you to all of of our ApacheCon@Home 2021 sponsors, including:...

youtube.com
1
37
We updated our website -> wayang.apache.org <-. More documentation, much more easier to contribute and add PRs.

Apache Wayang | Cross-Platform Data Processing

Unifying Data Processing Engines for Cross-Platform Analytics

wayang.apache.org
1
2
59
Apache Wayang now enables AI/ML training and inference in Spark for federated learning at the enterprise level. The team just merged PR388: github.com/apache/incubator-…

Add three ML operators which map to SparkML by joker-star-l · Pull Request #388 · apache/wayang

For clustering task: KMeansOperator For classification task: DecisionTreeClassificationOperator For regression task: LinearRegressionOperator

github.com
2
3
215
We merged 380: #380 Added a new plugin to support different jdbc platforms and multiple instances of same DBMS. Wayang now adds support for multiple spark and jdbc context in one wayangPlan(). github.com/apache/incubator-…

Closes #380 Added a new plugin to support different jdbc platform and multiple instances of same...

The current architecture of wayang looks like this I created a new plugin that can reduce code redundancy to add different jdbc platforms and also support multiple instances of same DBMS by simply...

github.com
1
27
ML4ALL implementation merged, it's the code to our reseach paper, now merged and fully operable. github.com/apache/incubator-…

ML4all module by zkaoudi · Pull Request #372 · apache/wayang

First versio of ML4all module. It contains the abstraction we discuss in the paper and two implementations, one for kmeans and one for SGD. More improvements to come.

github.com
2
2
50
Now you can run kmeans natively on sparkML! This improvement speeds ML operations massively up. Additionally, with wayang you can have multiple _different_ spark version in one plan, that makes wayang really powerful for cross-spark ML. github.com/apache/incubator-…

naive kmeans operator on sparkml by joker-star-l · Pull Request #356 · apache/wayang

I implemented a naive kmeans operator on sparkml, and add two test cases. It is still in development. Maybe you can give me some advice on the code and design. Thanks!

github.com
1
1
146
Apache Wayang: A Unified Data Analytics Framework - research paper now released with Sigmod Record => sigmodrecord.org/publication…
2
8
412
Wayang AgoraEO Plugin: The Framework for Scalable EO Workflows "We recently proposed AgoraEO, a decentralized, open, and unified ecosystem, where users can find EO elements, compose cross-platform EO pipelines, and execute them efficiently. ui.adsabs.harvard.edu/abs/20…
2
3
77
Apache Wayang retweeted
Apache - The ASF@TheASF
13 Aug 2023
#CommunityOverCode NA, our flagship event, is only TWO MONTHS away! Join us in Halifax 🇨🇦 for four days of sessions, with tracks focusing on Search, Big Data, Streaming, IoT, Community, Fintech, and more. 🎟️ Register now! bit.ly/3D9fKK7
4
18
4,472
Apache Wayang retweeted
Apache Doris@doris_apache
12 Aug 2023
#DynamicPartitioning in Apache Doris It means to automatically create and delete data partitions based on the user-specified rules. You can enable it upon data creation and specify the partition TTL (time to live) in the statement. #database #DataScience #Analytics #SQL
1
8
1,000
Apache Wayang retweeted
Scalytics@scalyticsAI
19 Jun 2023
Apache Wayang enables cross platform data processing, star it via: github.com/apache/incubator-… #dataprocessing #bigdata #analytics #hybridcloud #developers via @apachewayang

GitHub - apache/wayang: Apache Wayang is the first cross-platform data processing system.

Apache Wayang is the first cross-platform data processing system. - apache/wayang

github.com
1
1
44
Wayang can now support simple and basic SPJ queries for decoupled JDBC sources and CSV data sources. github.com/apache/incubator-…

SQL API extensions to support joins by kbeedkar · Pull Request #308 · apache/wayang

This PR makes extends the SQL API to also handle joins. Algebraic Joins are transpiled to Wayang Join Operator Since the JoinOperator outputs a Tuple2, the join is fu...

github.com
1
90
Apache Wayang retweeted
Paul King@paulk_asert
14 Apr 2023
Since the latest JDK21ea supports virtual threads as standard (not preview), I updated the Groovy/GPars meets virtual threads blog post: groovy.apache.org/blog/gpars… Why not give it a try! #groovylang @ApacheGroovy @mittie @vaclav_pech
GPars logo

ALT GPars logo

3
18
41
6,337
Load more