Foundation module

Architecture that keeps intelligence accountable.

The TARZO platform separates device execution, AI reasoning, memory, and account integrations so the assistant can be powerful without allowing a chat surface to bypass safety policy.

Voice

Voice orchestration

Wake-word gateway, streaming speech routing, TTS policy, and interruption handling.

AI reasoning

Gemini orchestration with tool selection, grounded response policies, and audit context.

Memory

Long-term memory

SQLite facts plus vector retrieval boundaries for conversations, preferences, and projects.

Automation

Automation broker

Permissioned desktop and Android actions with risk classification and confirmation gates.

Vision

Vision services

Screen, camera, OCR, and image-understanding requests routed through explicit consent.

Connectors

Connected accounts

Gmail, Outlook, calendar, Slack, and Notion actions through user-authorized connectors.

Internet

Knowledge & web

Live web and news retrieval rendered as cited source cards rather than opaque answers.

Security

Safety control plane

JWT session boundaries, device permissions, action approvals, and audit logging.

Folder structure

Clear service boundaries

tarzo-ai/
├── apps/
│   ├── desktop/          # Electron + React + TypeScript
│   ├── mobile/           # Flutter Android client
│   └── web-console/      # This secure operations console
├── services/
│   ├── api/              # FastAPI, JWT, REST + WebSocket gateway
│   ├── ai/               # Gemini tools, policy, prompting, streaming
│   ├── voice/            # Wake word, Whisper routing, ElevenLabs/Piper
│   ├── automation/       # Windows and Android command executors
│   ├── memory/           # SQLite metadata + ChromaDB retrieval
│   └── plugins/          # Signed plugin loading and isolation
├── packages/
│   ├── contracts/        # Shared OpenAPI models and event contracts
│   └── security/         # Risk classification and audit utilities
└── infra/                # Docker, observability, backup, deployment

Security posture

Policy before privilege

Tool calls are created as audited plans, with confirmation required for destructive system or external effects.
JWT identity scopes every conversation, memory record, calendar action, and private workspace key.
Desktop and Android agents expose narrow, signed command contracts instead of arbitrary shell access.
Secrets stay in server-side functions or service vaults; browser clients never receive provider keys.

SQLite + ChromaDB

Database model

Record	Responsibility
users	Identity, account preferences, encrypted device enrollment metadata
conversations	Conversation headers and AI session lifecycle
messages	User, assistant, tool, and system messages with immutable audit data
memory_items	Long-term facts with source, confidence, retention, and consent flags
automation_runs	Requested action plans, risk status, confirmations, and execution results
reminders	Scheduled reminders, recurrence, delivery channel, and completion state
plugin_registry	Installed plugin manifests, scopes, health, and version pinning

FastAPI contract

Backend API surface

Method	Endpoint	Purpose
POST	/v1/auth/token	Issue and refresh JWT session tokens
POST	/v1/assistant/messages	Submit a message and stream a tool-aware assistant response
POST	/v1/automation/plans	Create a classified action plan; dangerous plans require confirmation
POST	/v1/automation/plans/:id/confirm	Approve a previously reviewed action plan
GET	/v1/memory/search	Semantic memory retrieval scoped to the authenticated user
POST	/v1/reminders	Create a scheduled, user-scoped reminder
POST	/v1/vision/analyze	Consent-scoped OCR, screenshot, or camera analysis
GET	/v1/plugins	List verified plugins and their allowed capabilities