So, you want to build an AI that watches a video feed, listens to users, and responds with natural speech. Sounds cool, right? Now try building it.<\/p>\n\n\n\n
You’ll need a speech-to-text API. A vision model. A language model. A text-to-speech service. WebRTC for real-time streaming. WebSockets for low-latency communication. Then you stitch them all together, pray the latency stays under two seconds, and debug the async chaos when audio and video fall out of sync.<\/p>\n\n\n\n
We’ve been there. That’s exactly why we built Orga AI.<\/strong><\/p>\n\n\n\n Unified SDKs. A seamless API flow. Vision, voice, and conversation \u2014 processed together in under 700 milliseconds. Support for 40+ languages out of the box. And yes, you can have it running in under 30 minutes.<\/p>\n\n\n\n Orga AI is a real-time conversational AI platform. Users turn on their camera, speak naturally, and show the AI what’s happening. The AI watches, listens, and responds with its voice \u2014 in their language.<\/p>\n\n\n\n Here’s what that looks like: Orga provides a suite of SDKs from client to server side that work together to connect your application to our APIs. Our Next.js starter scaffolds a complete app with video, audio, and AI conversation already wired up.<\/p>\n\n\n Open localhost:3000.<\/strong> You’ll see a working demo: camera preview, voice input, AI responses. Customize the personality, plug in your logic, and ship.<\/p>\n\n\n\n To keep your API key secure, you\u2019ll first need to create a small backend service. This backend acts as a proxy between your app and Orga. Its job is to fetch ICE servers and an ephemeral token from the Orga API. Your client SDK (React or React Native) will call this endpoint before establishing its own connection<\/p>\n\n\n Once this backend endpoint is running, your frontend SDK can call it to retrieve the session configuration. No need to expose your API key in the client.<\/p>\n\n\n\n That’s it for the backend. Next, we\u2019ll see how to call this endpoint from the frontend and establish the connection. <\/p>\n\n\n\n Already have a React project? Drop in the SDK.<\/p>\n\n\n When setting up the provider, this is where you tell the client SDK which backend endpoint to call. That endpoint returns the ephemeral token and the ICE servers, which the SDK then uses to establish a secure connection to Orga, without ever exposing your API key. <\/p>\n\n\n Then wrap your app with the provider:<\/p>\n\n\n Now you\u2019re ready to create your main file and import the useOrgaAI<\/strong> hook. It exposes everything you need: startSession()<\/strong>, endSession()<\/strong>, and real-time state like connectionState<\/strong>.<\/p>\n\n\nWhat Is Orga AI?<\/h2>\n\n\n\n
User <\/strong>(pointing their phone camera): “My smart hub won’t connect. The light keeps blinking orange.”
Orga AI <\/strong>(watching the blinking pattern): “I see that orange blink \u2014 your hub lost its network settings. Show me the back and I’ll walk you through a reset.”
No typing. No screenshots. No “please describe your issue in detail.” The AI sees the problem and talks through the solution like a colleague would.
That’s the experience you can ship with Orga.<\/p>\n\n\n\nGetting Started with the Orga SDKs<\/strong><\/h2>\n\n\n\n
The Orga client SDKs integrate with any React-based framework. Pick your stack and follow along.<\/p>\n\n\n\nNext.js (Fastest Setup)<\/h3>\n\n\n\n
npx<\/span> @orga-ai<\/span>\/create-orga-next-app my-app\ncd my-app\nnpm install\nnpm run dev<\/code><\/span>Code language:<\/span> CSS<\/span> (<\/span>css<\/span>)<\/span><\/small><\/pre>\n\n\nBackend Proxy (Node)<\/strong><\/h3>\n\n\n\n
import<\/span> 'dotenv\/config'<\/span>;\nimport<\/span> express from<\/span> 'express'<\/span>;\nimport<\/span> cors from<\/span> 'cors'<\/span>;\nimport<\/span> { OrgaAI } from<\/span> '@orga-ai\/node'<\/span>;\nconst<\/span> app = express();\napp.use(cors()); const<\/span> orga = new<\/span> OrgaAI({\napiKey<\/span>: process.env.ORGA_API_KEY!\n});\napp.get('\/api\/orga-client-secrets'<\/span>, async<\/span> (_req, res) => {\ntry<\/span> {\nconst<\/span> { ephemeralToken, iceServers } = await<\/span> orga.getSessionConfig();\nres.json({ ephemeralToken, iceServers });\n} catch<\/span> (error) {\nconsole<\/span>.error('Failed to get session config:'<\/span>, error);\nres.status(500<\/span>).json({ error<\/span>: 'Internal server error'<\/span> });\n}\n});\napp.listen(5000<\/span>, () => console<\/span>.log('Proxy running on http:\/\/localhost:5000'<\/span>));<\/code><\/span>Code language:<\/span> JavaScript<\/span> (<\/span>javascript<\/span>)<\/span><\/small><\/pre>\n\n\nReact (Vite, CRA, or Your Own Setup)<\/strong><\/h3>\n\n\n\n
npm<\/span> install<\/span> @orga-ai<\/span>\/react<\/code><\/span>Code language:<\/span> CSS<\/span> (<\/span>css<\/span>)<\/span><\/small><\/pre>\n\n\n'use client'<\/span> \nimport<\/span> { OrgaAI, OrgaAIProvider } from<\/span> '@orga-ai\/react'<\/span>;\nOrgaAI.init({\nlogLevel<\/span>: 'debug'<\/span>,\nmodel<\/span>: 'orga-1-beta'<\/span>,\nvoice<\/span>: 'alloy'<\/span>,\nfetchSessionConfig<\/span>: async<\/span> () => {\nconst<\/span> res = await<\/span> fetch('http:\/\/localhost:5000\/api\/orga-client-secrets'<\/span>);\nif<\/span> (!res.ok) throw<\/span> new<\/span> Error<\/span>('Failed to fetch session config'<\/span>);\nconst<\/span> { ephemeralToken, iceServers } = await<\/span> res.json();\nreturn<\/span> { ephemeralToken, iceServers };\n},\n});\nexport<\/span> function<\/span> OrgaClientProvider<\/span>({ children }: { children: React.ReactNode }<\/span>) <\/span>{\nreturn<\/span> {children};\n}<\/code><\/span>Code language:<\/span> JavaScript<\/span> (<\/span>javascript<\/span>)<\/span><\/small><\/pre>\n\n\nimport<\/span> type { ReactNode } from<\/span> 'react'<\/span>;\nimport<\/span> { OrgaClientProvider } from<\/span> '.\/providers\/OrgaClientProvider'<\/span>;\nexport<\/span> default<\/span> function<\/span> RootLayout<\/span>({ children }: { children: ReactNode }<\/span>) <\/span>{\nreturn<\/span> (\n<html<\/span> lang<\/span>=\"en\"<\/span>><\/span>\n<body<\/span>><\/span>\n<OrgaClientProvider<\/span>><\/span>{children}<\/OrgaClientProvider<\/span>><\/span>\n<\/body<\/span>><\/span>\nTypeScript\n<\/html<\/span>><\/span><\/span>\n);\n}<\/code><\/span>Code language:<\/span> JavaScript<\/span> (<\/span>javascript<\/span>)<\/span><\/small><\/pre>\n\n\n'use client'<\/span>\n \nimport<\/span> {\n useOrgaAI,\n OrgaVideo,\n OrgaAudio,\n} from<\/span> '@orga-ai\/react'<\/span>;\n \nexport<\/span> default<\/span> function<\/span> Home<\/span>(<\/span>) <\/span>{\n const<\/span> {\n startSession,\n endSession,\n connectionState,\n toggleCamera,\n toggleMic,\n isCameraOn,\n isMicOn,\n userVideoStream,\n aiAudioStream,\n } = useOrgaAI();\n \n const<\/span> isConnected = connectionState === 'connected'<\/span>;\n const<\/span> isIdle = connectionState === 'disconnected'<\/span>;\n \n return<\/span> (\n <main<\/span> className<\/span>=\"mx-auto flex max-w-2xl flex-col gap-6 p-8\"<\/span>><\/span>\n <header<\/span>><\/span>\n <h1<\/span> className<\/span>=\"text-3xl font-bold\"<\/span>><\/span>Orga React SDK Quick Start<\/h1<\/span>><\/span>\n <p<\/span> className<\/span>=\"text-gray-600\"<\/span>><\/span>Status: {connectionState}<\/p<\/span>><\/span>\n <\/header<\/span>><\/span>\n \n <section<\/span> className<\/span>=\"grid grid-cols-2 gap-4\"<\/span>><\/span>\n <button<\/span>\n className<\/span>=\"rounded bg-blue-600 px-4 py-2 text-white disabled:opacity-50\"<\/span>\n disabled<\/span>={!isIdle}<\/span>\n onClick<\/span>={()<\/span> =><\/span> startSession()}\n >\n Start Session\n <\/button<\/span>><\/span>\n <button<\/span>\n className<\/span>=\"rounded bg-red-600 px-4 py-2 text-white disabled:opacity-50\"<\/span>\n disabled<\/span>={!isConnected}<\/span>\n onClick<\/span>={()<\/span> =><\/span> endSession()}\n >\n End Session\n <\/button<\/span>><\/span>\n <button<\/span>\n className<\/span>=\"rounded border px-4 py-2 disabled:opacity-50\"<\/span>\n disabled<\/span>={!isConnected}<\/span>\n onClick<\/span>={toggleCamera}<\/span>\n ><\/span>\n {isCameraOn ? 'Camera On' : 'Camera Off'}\n <\/button<\/span>><\/span>\n <button<\/span>\n className<\/span>=\"rounded border px-4 py-2 disabled:opacity-50\"<\/span>\n disabled<\/span>={!isConnected}<\/span>\n onClick<\/span>={toggleMic}<\/span>\n ><\/span>\n {isMicOn ? 'Mic On' : 'Mic Off'}\n <\/button<\/span>><\/span>\n <\/section<\/span>><\/span>\n \n <OrgaVideo<\/span> stream<\/span>={userVideoStream}<\/span> className<\/span>=\"h-64 w-full rounded bg-black\"<\/span> \/><\/span>\n <OrgaAudio<\/span> stream<\/span>={aiAudioStream}<\/span> \/><\/span>\n <\/main<\/span>><\/span><\/span>\n );\n}\n<\/code><\/span>Code language:<\/span>