Spectro Cloud Palette EdgeAI builds and manages Kubernetes-based AI software stacks

Spectro Cloud announced Palette EdgeAI to simplify how organizations deploy and manage AI workloads at scale across simple to complex edge locations, such as retail, healthcare, industrial automation, oil and gas, automotive/connected cars, and more.

Palette’s EdgeAI extends Spectro Cloud’s core Palette Edge Kubernetes management platform that addresses the unique challenges of deploying and managing edge environments at scale, specifically:

• Limited on-site specialist IT expertise at the edge locations
• Increased security risk due to the distributed nature of edge infrastructure, software stack and communications.
• Inconsistent connectivity
• Costly and disruptive operational tasks, including security fixes, feature patches and updates

With a record number of organizations embracing the potential of running AI workloads at the edge, these challenges are exacerbated. Gartner suggests that “by 2027, deep learning will be included in over 65% of edge use cases, up from less than 10% in 2021”.

Activities that are achievable in the data center or cloud, such as deploying daily updates to a large language model (LLM), are costly or even impossible across thousands of devices and locations. Furthermore, the exposed security posture of edge locations is problematic given that AI workloads often handle sensitive data and critical intellectual property.

“More and more of our customers are exploring AI at the edge as their primary mechanism to deliver modern, rich applications and transform the customer experience”, said Jim Melton, Head of Cloud Strategy & Programs, Digital Velocity, CDW. “The need to simplify deployment and provide comprehensive management for AI-optimized infrastructure at the edge is real and solutions such as Palette EdgeAI squarely addresses those challenges”.

The new Palette EdgeAI solution offers a rich suite of capabilities to address specific requirements throughout the lifecycle of edge infrastructure and AI software stacks.It:

• Deploys and manages complete AI-ready infrastructure stacks in edge computing environments, from the customer’s preferred OS and Kubernetes distribution, to AI model engines like Kubeflow and LocalAI, including easy “plug-and-play” device onboarding.
• Secures edge infrastructure to protect sensitive intellectual property and model data, with hardened configurations, SBOM scans, full-disk encryption and robust access controls. Palette offers FIPS compliance for highly regulated industries.
• Improves model accessibility, with integrated access to model marketplaces, including Hugging Face and an enterprise’s own private repositories. Operators can incorporate their chosen models as part of the AI stack ‘Cluster Profile’, or blueprint.
• Makes it easy to deploy models to any number of edge locations automatically with a click. Palette will deploy the model along with the infrastructure stack and regularly reconcile the state of the stack to ensure it is in line with policy.
• Enables operators to upgrade and roll back model versions deployed in each edge cluster with a click, including Over-The-Air (OTA), zero-downtime upgrades and designing canary deployments across the edge estate, with advanced model observability.
• Simplifies distributed inferencing, enabling organizations to leverage multiple edge nodes for parallel execution and reduced model latency.
• Unlocks federated training, accelerating model improvement with on-device learning using local data.
• Reduces edge infrastructure costs, by enabling workloads to run with high availability even on limited edge hardware.

Palette’s unique fault-tolerant architecture allows workloads to be deployed on two-node Kubernetes clusters, instead of the usual three-node — a huge saving across multiple sites. 2-node HA now also available on all Palette solutions.
“The edge is the natural environment for AI inference workloads,” said Tenry Fu, CEO of Spectro Cloud. “Our mission is to simplify innovation for our customers and we have been working with organizations that are already disrupting their industries, reaping the benefits of AI at the edge”.

In the healthcare sector, RapidAI uses Palette Edge to deploy its AI applications into hospitals, giving clinicians deeper clinical context to quickly and accurately triage and diagnose conditions, such as strokes and embolisms, for better patient outcomes.

“At RapidAI our business is built on continuous AI innovation, helping clinicians right in the hospital,” said Amit Phadnis, Chief Innovation and Technology Officer at RapidAI. “When it comes to deploying our applications securely and easily to the edge, we trust Spectro Cloud’s Palette.”

Palette’s EdgeAI will be generally available in Q4 2023 with rapid future capabilities throughout 2024.

Spectro Cloud also announced a new round of investment, led by Qualcomm Ventures. The investment will accelerate Spectro Cloud’s innovation in edge computing, AI and enterprise infrastructure management.

“Qualcomm is uniquely positioned within the edge ecosystem and with the adoption of AI, edge has become a necessity,” said Dev Singh, VP of Business Development, Qualcomm Technologies. “Across Industrial, Enterprise, Utilities and Retail, we are seeing a need to dynamically orchestrate AI workloads across edge and the cloud, simplify edge deployments and manage upgrades with no downtime to build the next-generation of resilient, high-performing applications.”