Cognidamesh is a blueprint for building your own open-source AI agent teams for software development, scientific research, OSINT, pentesting, countering disinformation, and administrative and financial management. It maps the essential capabilities, the components to choose and how they fit together.
AN AI TEAM BLUEPRINT · OPEN SOURCE · SELF-HOSTED
A shared foundation for different missions. These are the uses the blueprint aims to enable; each team brings its own expertise, data and specialist tools.
Observe how a team works: map tasks, tools, decisions and hand-offs, then capture its expertise and recurring workflows. Generate a complementary AI agent team with defined roles, skills, tool access and shared memory. Test it on real tasks and progressively automate as much of the workflow as possible, measuring time saved and output quality while people supervise exceptions and key decisions.
Build an AI software factory: coordinate agents to specify, design, implement, test, review and deploy software. It can also reverse engineer existing software to understand its behaviour, document interfaces and support migration or interoperability. Keep findings traceable and validate implementations against the intended behaviour, within the authorised scope.
Review literature, compare hypotheses and support reproducible experiments. Keep methods, sources and results connected.
Collect public information, cross-check sources and build documented investigations with traceable evidence.
Coordinate authorised security assessments, analyse findings and organise remediation within a defined scope.
Trace claims to their sources, compare accounts and document contradictions and uncertainty for human review.
Sort documents, prepare correspondence, track cases and deadlines, and coordinate internal workflows. Keep access scoped to each role and have people validate official submissions.
Support investment management in stock markets and cryptocurrencies: monitor markets and portfolios, compare performance and prepare investment decisions. Prepare budgets, reconcile invoices and transactions, track cash flow and flag discrepancies. Keep supporting records traceable and require human approval for payments and financial commitments.
Assemble an AI agent team to turn a shopping brief into a coordinated purchasing workflow: a discovery agent finds products, an analyst compares prices, availability and delivery terms, and an order agent prepares checkout. A follow-up agent tracks deliveries and handles return requests. Share context across the team, work within the buyer’s budget and preferences, and require approval before placing orders.
A blueprint for assembling your own stack, rather than a ready-to-install code distribution. Each block presents a function, recommended components and alternatives. The goal remains a 100% open-source foundation; proprietary and hosted options are identified.
Give the team goals, roles and a shared work queue. Delegate tasks, organise hand-offs and keep people in charge of priorities.
MISSION → ROLES → TASKSPaperclip provides the organisational layer that brings individual agents into a team.
Turn a mission into actions through tools and skills. Choose a runtime for each role and connect it to the orchestrator through an adapter.
ONE TEAM, COMPLEMENTARY RUNTIMESHermes is our main choice for knowledge work. OpenCode, pi and Codex CLI are open-source coding options. Claude Code and Bob Shell are proprietary alternatives, connected through the available adapters.
Keep findings, sources, decisions and context across sessions. Agents need to retrieve useful knowledge and build on what their team has already learned.
RETAIN → RECALL → REFLECTHindsight is currently the best fit for our blueprint: retain, recall and reflect operations, with a dedicated memory bank per organisation.
Find information, read web pages and follow evolving sources. Feed the team relevant material with its provenance.
SEARCH → READ → CROSS-CHECKSearXNG provides self-hosted search; Camofox and CloakBrowser support browsing and collection. Tavily offers a hosted search and extraction API. CloakBrowser’s wrapper is MIT, but its binary has separate terms: these options are not all fully open source.
Supply the models that agents use to reason, handle routine tasks and create embeddings. Keep the inference layer under your control.
SELF-HOSTED INFERENCEvLLM and TensorFold are inference engines to evaluate against your models and hardware. The open-source LiteLLM core provides API routing. Check model formats, memory requirements and compatibility with the agent runtimes.
These variants, sometimes called “obliterated”, modify model weights to reduce or remove certain learned refusal behaviours, including restrictions that may censor legitimate requests. Abliteration targets refusal-related directions in the model’s internal representations; it does not guarantee the removal of every restriction or bias. Fewer refusals do not guarantee accuracy or stronger capabilities: evaluate answer quality and tool use. The base model’s licence still applies.
The harness — the software that controls the agent’s execution — must enforce security outside the LLM. Add an independent guardrail that assesses the intent and context of requests before execution, then checks proposed actions and tool calls against the user’s permissions and authorised scope. Allow, block or escalate accordingly. Combine this with least-privilege access, sandboxing, audit logs and human approval for sensitive actions. Intent analysis can make mistakes: it complements enforced permissions and isolation, and must not be the only defence.
Example model card ↗Run services with explicit permissions, networking and persistent storage. Reproduce the environment on a server, VM or in containers.
YOUR ENVIRONMENT, YOUR CONTROLPodman and Quadlet suit a single host; Kubernetes orchestrates containers across a cluster, while OpenShift provides an integrated Kubernetes platform. Terraform defines and provisions infrastructure; Ansible configures systems. Current Terraform releases use the BSL; OpenTofu is the open-source alternative.
Combine reusable skills — instructions, procedures and validation criteria — with tools exposed through the Model Context Protocol (MCP). Select them for each role and connect compatible MCP servers to the agent runtime.
SKILLS → MCP TOOLS → VALIDATIONBMAD structures software delivery; Strix supports security assessments. Scrapling provides web extraction with an MCP server. REA combines investigation skills and MCP tools for reverse engineering applications and binaries. Cua provides tools for computer use and isolated desktop environments. Chrome DevTools MCP supports browser automation, debugging and performance analysis. Check each runtime’s support, platform requirements and tool licences, including REA’s analysis engines. Scope access and approvals in the harness.
Adapt a model to your domain, vocabulary and tasks using curated training examples. Keep training separate from production inference.
DATA → TRAIN → EVALUATE → DEPLOYUnsloth supports LLM fine-tuning, including LoRA and QLoRA for supported models. Check model and GPU compatibility, dataset rights and model licences. Version datasets and adapters, compare against the base model on a held-out evaluation set, then export to a compatible inference engine. Recheck tool use and harness guardrails before deployment; fine-tuning and abliteration serve different purposes.
Read each block as a capability, then as a component choice. This is an assembly map for your future team.
Functional map: validate connections for each runtime. Hermes carries the research integrations; Hermes and coding agents share Hindsight memory. Search, inference and hosting alternatives are detailed in the blocks; they are not all required or all open source. Skills and MCP tools — BMAD, Strix, Scrapling, REA, computer-use and Chrome DevTools MCP — connect according to supported formats, platforms and permissions. The harness checks intent and enforces permissions, isolation and approvals. Coding and reverse-engineering agents drive Unsloth retraining in a workflow separate from production inference; only evaluated models move to inference.
flowchart TB
accTitle: Cognidamesh open-source blueprint
accDescr: Architecture diagram connecting human direction and Paperclip to agent runtimes, harness guardrails, skills and MCP tools, research services, Hindsight memory and inference. Coding and reverse-engineering agents drive Unsloth retraining and send evaluated models to inference.
H["Mission and human direction"] --> P["Orchestration · Paperclip"]
P --> A["Harness · Intent checks<br/>Permissions, isolation, approvals"]
A --> C["Coding and reverse engineering<br/>OpenCode, pi, Codex CLI<br/>Claude Code, Bob Shell"]
A --> R["Execution and research<br/>Hermes"]
P -->|activity| M["Memory management · Hindsight"]
R <--> M
C <--> M
R --> W["Search and collection<br/>SearXNG, Camofox<br/>CloakBrowser, Tavily"]
R --> G["Inference · vLLM, TensorFold, LiteLLM"]
C --> G
M --> G
S["Skills and MCP tools<br/>BMAD, Strix, Scrapling, REA<br/>computer-use · Cua<br/>Chrome DevTools MCP"] -.-> R
S -.-> C
C -->|drives| T["Retraining · Unsloth<br/>Data → training → evaluation"]
T -->|validated model| G
subgraph F["Services"]
P
A
R
C
M
W
G
S
T
end
classDef coordination fill:#d5f879,stroke:#8fa54f,color:#19281f,stroke-width:1px;
classDef core fill:#263c2e,stroke:#476345,color:#ffffff,stroke-width:1px;
classDef service fill:#e9efdf,stroke:#b3c49d,color:#263c2e,stroke-width:1px;
classDef data fill:#ffffff,stroke:#a6b993,color:#263c2e,stroke-width:1px;
class H,P,A coordination;
class R,C core;
class W,G,S service;
class M,T data;
style F fill:#f9faf4,stroke:#b3c49d,stroke-dasharray:5 5,color:#476345;
The runtime is the agent’s execution engine. Assemble a team using the open-source clients or proprietary alternatives below, then add the tools and skills each role needs.
Our primary runtime for knowledge work: combine research skills, web tools and Hindsight memory. Use separate profiles for agents and a dedicated container for each organisation.
Research · tools · durable memoryFor a 100% open-source team, select runtimes, services and models with licences that meet that goal. Claude Code and Bob Shell are optional proprietary alternatives. A client’s licence does not determine the model’s licence; downloadable weights alone do not make a model open source.
Follow the capabilities in order. Start with a small, useful team and expand from evidence.
Choose the software product, research question, investigation or management workflow. Specify roles, expected outputs and what people must review.
Select an orchestrator, runtimes and a model stack. Check licences, hardware needs and API compatibility.
Choose skills and MCP servers for each role: BMAD for delivery, REA for reverse engineering, Scrapling for extraction, computer-use and Chrome DevTools MCP for interaction and testing, or Strix for assessments. Connect sources and memory, then validate tool permissions and results.
Test on representative tasks, review quality and document what works. Revisit the component choices as the ecosystem evolves.
Start on a server or VM with Podman, or distribute services across Kubernetes or OpenShift. Define infrastructure with Terraform or OpenTofu, then configure systems with Ansible.
Bring together the orchestrator, agent runtimes, tools and memory. Size CPU, RAM, storage and optional GPUs around the models and workloads you choose.
This architectural blueprint evolves with the ecosystem. Validate the recommended components against your own workloads.
Copy and paste this into your favourite AI coding agent.
Read https://cognidamesh.org/agents.md and use it to build, test and deploy a similar AI infrastructure in my project. Respect existing project instructions; ask me only for missing requirements, access and deployment approvals.The linked AGENTS.md is a public implementation guide in the open Markdown format. Your agent will adapt it to your project, hardware and access. If it cannot read the URL, download the guide and attach it to your request.