Had an awesome time at re:Supply RoboCode last night, thanks to
@SchematicVC! 🚀
Explore the future of robotics as we unravel opportunities, tackle challenges, and integrate LLMs into today's robotic landscapes. 🧵
1. Robots Are Hard to Simulate
•Simulation is a crucial step before real-world deployment—a step often overlooked by eager startups, resulting in unforeseen problems during deployment. NVIDIA's IsaacSim seems to the go-to platform for these essential tasks, but setting up a simulation environment is complex, costly, and often doesn't capture real-world edge cases.
•Edge Cases and Testing Strategy: Establishing realistic edge cases is vital. Simulations should be used to identify what and where to test in physical environments.
•Platform Choice and Speed of Iteration: Software simulation allows for faster iterations than hardware testing. This advantage should be leveraged to make development more agile.
•Challenges in Simulation: A key focus is simulating adversarial or constrained environments, particularly in defense, remote deployments, and situations with limited bandwidth or impractical cloud processing.
•DALUS AI: Uses your phone to create a NERF of your environment using gaussian splatting. Integrate the resulting opacity and mesh values into your simulation environment, drastically increasing realism. Essential for addressing real-world issues like sensor mis-calibration, unexpected human interactions, and sensor obstructions.
More info:
dalus.ai/
2. Deployment and Integration Strategy
• Market Approach: Mature enterprises are essentially betting their business on your robots. As such, companies are cautious and require a lot of assurance and credibility from robotic solutions. This necessitates addressing all aspects of the deployment process, not just technological readiness.
• Complete Solution Requirement: Customers expect 100% of the problem to be solved; "99.99%" is not enough. A comprehensive solution extends beyond technology to include integration, support, change management, and maintenance.
3. Robotics as a Service (RaaS)
•Incentive Structure Flaws: Startups in the RaaS sector often face pressure to over-engineer or introduce unnecessary complexity. This results in inflated costs and increased implementation challenges, yielding minimal returns on essential functionality.
•Simplicity in Design: There is a significant gap in creating straightforward, maintainable robotic systems. The current ecosystem lacks incentives for simplification, leading to "robot graveyards," where projects are abandoned due to impracticality.
@samanfarid from
@goformic is building the plug-and-play robotics platform for SMBs
4. Current Challenges with ROS (Robot Operating System)
•ROS Limitations: Though valuable for prototyping, ROS falls short of production standards due to reliability and integration issues. We need a shift towards a more robust alternative that can handle production-level demands.
•Platform Ambitions: Companies like Basis Robotics aim to enhance ROS by controlling the entire stack. This promises a better developer experience, deterministic testing, and comprehensive replay functionality.
More info:
basisrobotics.tech/
5. Developer Tools and Robotics-Specific IDEs
•Need for Robotics-Specific Tooling: Current DX (Developer Experience) and instrumentation for robotics are insufficient. There’s potential to improve the DX by addressing these tooling gaps.
•Copilots for Robotics: Tools like
@FidLabs, who offer a VSCode extension optimized for C and embedded environments, are becoming essential to address the complex needs of robotics development. They also effectively aggregate tons of robotics-specific knowledge, especially for niche or challenging imports into existing tooling, would helps streamline development and reduce redundancy.
• "Last Inch" Deployment: Interface tools that bridge traditional algorithms and newer foundation models are where significant value lies. This “last inch” of interface integration is essential for seamless deployment.
6. Collaborative and Decentralized Robotics Development
•Multi-Robot Systems: Moving from individual robotic control to orchestrating fleets of robots highlights the need for inter-robot communication, synchronization, and low-latency decision-making.
•Challenges with Cloud Processing: Latency and availability concerns make cloud processing infeasible for real-time applications in robotics, furthering the need for on-device intelligence and edge computing.