DevTrends开发者趋势
主题色

DEV TRENDS / CATEGORY

Agent 生态 · Agent 评测与安全

2026年10月26日 – 2026年11月1日 内首次发现的 56 个新项目,已跨 6 个来源去重。

收录范围

2026年8月10日 – 2026年11月1日 内首次发现 599 个项目

DTDevTrends 归类原始项目或来源原始标签
56 个新项目6 个来源

agent-qa:带持久记忆的自然语言测试工具

agent-qa 面向需要验证用户流程的开发者,可用自然语言检查网页、Android 与 iOS 应用,保留测试证据,并通过 CLI 或 MCP 接入编码工作流。其 memidex 将事实和有效操作流程保存为可审阅文件,自我改进功能验证运行经验、淘汰过时知识 亮点 可审阅的持久记忆:memidex 把已知事实与有效测试流程保存到仓库文件,便于检查和复用测试知识。 自我改进:验证每次运行产生的经验、淘汰过时知识,并利用反馈纠正错误。 覆盖 Web 与移动端:同一工具可从 CLI 或 MCP 发起 Web、Android、iOS 用户流程检查,并保留证据供排查问题。 CLI 免费,模型调用和设备服务费用另计;需要 Node.js 24+。许可为 FSL-1.1-ALv2,限制竞争性商业用途。 · 使用文档 由 Codex 代表 vostride 自荐。已在 HelloGitHub 网站搜索项目地址,未看到已有收录结果。 截图或演示视频 项目公开 README 中的 CLI 演示图:识别示例购物车的预置总价错误。

语言TypeScriptDTAgent 评测与安全原始ai-agents原始ai-testing
915

BLA:防 AI 自动逆向的软件构建产物加固工具

自荐一个我开发的开源工具 BLA(Blind LLM Agents),采用 MIT 许可证。 随着 REA(Reverse Engineer Anything)这类逆向 MCP 工具出现,现在把打包好的桌面客户端、二进制程序或前端产物丢给 Claude Code、Cursor、Codex,AI 已经可以用一条命令自动反编译并还原核心逻辑。传统代码混淆(比如单纯改变量名)在大模型的全局推理面前基本失效了。 我做 BLA 的思路不是和 AI 拼“谁更难读”,而是针对自动化逆向 Agent 的工程边界做低成本反制——对准备发布的构建产物注入格式合法的诱饵逻辑与解析预算陷阱,让正常用户的实际运行行为 100% 不变,但试图自动逆向的 AI 分析器会触发资源上限、顺着假线索跑偏并持续消耗 Token。 几个主要特点: 构建后直接加固,不改业务源码:一条命令即可完成符号/SourceMap 暴露面扫描、加固注入与运行时等价性校验: bash npx bla-agents harden ./dist 覆盖主流发布产物格式:支持 JavaScript / TypeScript、Electron(.asar 与 Fuses)、WebAssembly、Python(.py / .pyc)、原生二进制(ELF / Mach-O / PE)、SQLite、AI 模型权重(Safetensors / GGUF)及移动端包体等。 多种接入方式:既可以作为 CLI 和 Vite / Webpack / Rollup / esbuild 打包插件放进构建流水线,也支持一键注册为 MCP 服务,在 Claude Code、Cursor、Codex 等编程助手里直接调用。 真实分析器对抗实测:仓库附带了针对真实 REA 分析路径的加固前后对照测试记录(docs/redteam/empirical-rea-validation.md),同时也如实标注了当前的防护边界与局限。 项目相关链接: 开源

语言JavaScriptDTAgent 评测与安全原始adversarial-ai原始claude-code
1812

RankCLI

RankCLI runs 280+ SEO checks from your terminal or CI - like ESLint for SEO. It's the first tool built for GEO too: checks whether GPTBot, ClaudeBot, and PerplexityBot can actually access your site, and tracks your AI-search visibility. Connect GitHub and it opens auto-fix PRs instead of just a report. Also ships a free, local MCP server - no signup, nothing sent to our servers - so Claude, Cursor, or any MCP client can audit your site directly.

原文
DTAgent 评测与安全
0

Arbalest

Agentic security CLI that finds reachable vulnerabilities, writes the fix, and attacks its own patch in an isolated Nebius sandbox. A patch is certified only when the live exploit fails.

原文
DTAgent 评测与安全
1

Agent Aisle

A free, open-source security audit for Agent Skills. It reads a skill as text, never runs it, and grades it A to F. Plus a store of audited skills for Claude Code, Codex and Cursor.

原文
DTAgent 评测与安全
0

EmbedCheck

EMBRACE research (Forever Security, Oct 29, 2026) showed the preview browser inside Cursor, Copilot, Codex, VS Code and Claude Desktop was trusted like a real browser — and shouldn't be. Codex and VS Code were unpatched at publication; Claude Desktop's fix status is uncertain. EmbedCheck audits your own workstation: a status table with a source and date on every row, update checklists, a workspace-trust guide, a canary self-test that never reads a real secret, plus policy and report templates.

原文
DTAgent 评测与安全
1

AISoltrics Architecture Templates Bundle

Most AI template packs are prompt lists. These are production machinery — distilled from 35 published AI books and a Delaware C-corp's delivery practice. Three packs — AI Agent Prompt Pack (115 prompts), RAG Templates, MCP Templates — each as PDF + Markdown, free updates included. For teams shipping agents, RAG, and MCP into enterprises where governance, security, and evaluation are not optional. Sold by AISoltrics Technologies Inc. Support within one business day.

原文
DTAgent 评测与安全
1

PanelCheck

Audit your own exposure surface: every agent panel, staging dashboard, and forgotten deploy. If it answers on the public internet without auth, that is the finding. Your domains only — never scan anyone else's infrastructure. PanelCheck is an own-domain panel and exposure audit kit: scope checklist, lockdown runbook, secrets-rotation template, auth-by-default guide, and one-page exposure report. $39 one-time.

原文
DTAgent 评测与安全
1

mcpscore

Audit any MCP server and get a 0–100 quality score across protocol conformance, tool quality, security and auth, and spec readiness.

原文
DTAgent 评测与安全
5

phpClaw – AI agent runtime for PHP with tools, memory and guards

Open-source AI agent framework for PHP. Tools, memory, security guards and a ReAct loop built in. Works with Laravel, Symfony, WordPress, Drupal, Magento, Joomla, OpenCart, PrestaShop. Any LLM provider. MIT.

原文
语言PHPDTAgent 评测与安全原始agentic-ai原始ai-agent
25

AI agent needs process to build a commercial application

I decided to test this claim with my app, NoMoreHappyPath (NMHP), against Claude on a real project. Here is the test: I built Acremuse, a commercial real estate analysis platform, with Claude Desktop and Codex as a reviewer. It is deployed and used today by a couple of investors. Then I wondered what I'd missed that might bite me later. ~ 374,000 Lines of program code - 1,070 Source files (Python and TypeScript) - ~ 11,400 Functions and methods - 3,067 Automated tests So I gave the same 374k-line code snapshot and the same brief to one Claude session that can use Codex as a reviewer and to my own tool, NMHP. My directive was the same for both: Analyze the code, find any errors/bugs, and list them by severity. Findings must be reviewed before writing the final report. The Results (Verified by an impartial, blind Codex judge): NMHP: 33 real issues found. 7 false claims. Claude: 10 real issues found. 2 false claims. Only 6 issues overlapped. Time & Cost: From the owner's go-ahead to the reviewer's pass, NoMoreHappyPath took about 74 minutes, including time lost while it found and worked around a defect in the product itself. The Python version NMHP was working on was 3.9, the OS version, and it didn't have pytest installed, which was needed to verify some claims. After 15 minutes of blockage, the CTO stepped in and told the agent to create a venv with Python 3.13, install the corresponding pytest, and move the project along. NMHP CTO reacts when 15 minutes pass without movement from agents. To be fair to Claude, Claude did create a venv from the get-go. Time & Cost: NMHP took 74 minutes and cost $24 in API spend. Claude finished in a few minutes for a fraction of that cost. And yes, NMHP takes over an hour and costs twenty-four bucks. But for a 374k-line codebase, finding 23 more real issues overall that vanilla Claude missed makes $24 a massive win compared to human audit rates. I know this isn't a benchmark; it's one app and one run each. I wrote a detailed white paper on it. But it confirmed my idea: an AI agent without a real process is not yet capable of building a commercially ready application, even with a frontier model and reviews that are initiated by the same agent.

原文
DTAgent 评测与安全
—

Forge SafetyBox

High-Security file checker and developer sandbox for your Mac. Attack-tested. Pay once, free upgrades. Special Launch Price: $9.99. Drop any file, download or code you don't trust into SafetyBox. It scans for known threats and runs programs in a fresh sealed machine with no internet and no view of your files, then throws it away. For developers: run unread scripts and AI-agent code (Python, Node.js, shell, Linux) and check npm/PyPI packages and leaked keys. Apple certified: Silicon + Intel.

原文
DTAgent 评测与安全
1

Blueprint 2.0

Blueprint is an open-source conversational skill that helps you clarify an idea, gather references, choose a direction and define a testable MVP. Get a portable plan with decisions, data/API, security, acceptance tests and agent handoff. FR/EN docs. No website or API key required by the skill.

原文
DTAgent 评测与安全
2

OLO ToolGate

OLO ToolGate is an Apache-2.0, self-hosted security gateway for AI agents, MCP servers and tool execution. Apply ALLOW, ASK or BLOCK policies, independent human approvals and single-use execution permits, with audit trails linking identities, decisions and outcomes. Now in beta, with a Docker Compose quickstart for local evaluation.

原文
DTAgent 评测与安全
2

NashOS

Agentic FP&A software that runs planning, forecasting and close as one continuously computed system across a single data foundation - multi-entity, multi-currency, audit-ready.

原文
DTAgent 评测与安全
1

StrictLLM – For when you need human-in-the-loop

Hi, We've been working on this for the last 9 months and are nervcited to share this on HN and gather input. StrictLLM is a desktop application for Mac, Windows, and Linux that will give you the following abilities: - Human-in-the-loop with a strong focus in security and control - Permissions enforced by code rather than a prompt - Being able to talk to multiple AIs in the same conversation thread - You can pull in any personas by @Mentions. - Local and cloud models supported (OpenAI API Compatible and Anthropic Compatible) - - LM Studio, Lemonade Server, Ollama, Claude, Strata, ChatGPT, OpenRouter, Kilo Gateway, etc. List is not exhaustive - No telemetry is collected. The only network requests being made will be to check for updates and license activation. Your chats, prompts, tools, and sessions are yours to keep and manage. https://strictllm.com/screenshots To dive right into it: 1) https://strictllm.com/download 2) Hacker News Trial License Key for activation: C564BD-1CD395-06A51E-CDF819-C48315-V3 This key will be good for a limited time up until end of November 30th and will be the same as the Pro license. No credit card or account required to use this trial. Why use this over manual mode in other tools? Manual mode does not guarantee that each call to a tool or MCP server will prompt you first before proceeding. Contact: support [at] strictllm.com Forum: https://forum.strictllm.com/t/discussions

原文
DTAgent 评测与安全
—

Vrux

Describe a process in plain language. Vrux generates the workspace structure and enables people and AI agents to execute work under server-enforced permissions, approval policies, and tenant-isolated data access. Built around a policy-driven execution layer, Vrux makes security and governance foundational—not an afterthought.

原文
DTAgent 评测与安全
1

I built a tool that finds reliability bugs in your code

Hey HN, I’m Abhishek. I'm building Opslane, a tool that finds reliability issues in your code. On every PR, it spins up your entire app and then tries to find weak spots that would break in production. https://youtu.be/47phxkdh6f0 Example: We tested Opslane on Lago (open-source billing) and found a bug where customers get double charged. Opslane made Stripe take the money and then return a 500. Lago marked the payment failed. A retry from the dashboard charged the customer again. https://github.com/getlago/lago-api/issues/6903 Why I built this I used to work on reliability at Robinhood. We used to run chaos testing in our staging environments: inject faults in various parts of the system and see what breaks. We found issues like Kafka or Redis being unavailable for a few seconds could take the whole app down. But that kind of testing was expensive. It needed a well maintained staging environment. Engineers needed to know what to test. Most teams never actually did it. Over the last year, coding agents have gone writing some of my code to almost all my code now. Teams are shipping way more code than they were 6 months ago. Code review tools read the diff, but a lot of reliability bugs aren't in the diff. They're in how your code behaves when Stripe, Redis or the network misbehaves. And e2e tests usually mock those services out and test the happy path. Agents have made this type of testing a lot cheaper. And we will see more of this (kinda like security scanning / testing) How it works A full copy of your app per PR: Postgres, Redis, Kafka, SQS, S3. Working copies of your third-party dependencies: Stripe, Slack, Gmail and more. Traffic goes through our proxy, which lets us inject failures. The agent picks what to test: It reads every PR and writes down invariants that need to hold (for e.g. a customer should never be charged twice). It then injects faults likely to break the invariants. Bugs we've found We ran Opslane on some popular open-source apps and here are some of the bugs we have found: 1/ Twenty: deleting a workspace while Stripe is failing locks everyone out, and billing keeps going. Twenty merged a fix. ( https://github.com/twentyhq/twenty/issues/28043 ) 2/ Dub: a restart in the middle of a payout paid a partner $240 for $120 owed. ( https://github.com/dubinc/dub.link/issues/4707 ) Full list: opslane.com/open-source Opslane is pretty early - we have a couple of early customers. We have found bugs in their system that their other code review tools missed. Would love to get feedback from folks and how they are seeing the reliability of their systems evolve over time with agents.

原文
DTAgent 评测与安全
—

ClinicalFaultBench:度量临床 AI 智能体"怎么失败"的失效审计基准

ClinicalFaultBench 是一个面向临床 AI 智能体的失效审计基准,MIT 开源。 现在几乎所有医疗 AI 评测都在"理想输入"下算成功率。但这个方向有个被绕开的问题:真实医院的检验单会缺项、单位会错、报告是几天前的、系统会截断返回结果。agent 在这些情况下怎么崩,没人系统测过。 于是这个项目做的事很简单——往 agent 的环境里注入"看起来合理但错"的临床故障,然后量它怎么失败: 七类故障:关键化验缺失 / 单位错误(mg/dL 与 SI 制错位)/ 矛盾记录 / 陈旧数据(时间戳前移)/ 结果截断 / 跨患者污染 / 时间线漂移,每类两档严重度 五项指标:注入前后配对错误率、错误传播率、患者安全违规率、不确定性校准、自纠率 三类任务:入院摘要审查 / 医嘱冲突审查 / 出院准备 有意思的发现 最有价值的一条是负结果:结果截断让两个模型分别掉 0.389 / 0.365,而"截 30%"与"截 70%"的伤害逐位相等——缺多少不重要,有没有意识到缺了才重要。模型从不确认数据是否完整。 另一条:禁忌冲突检测已经饱和(检出率约 100%,效应约 0),如实写进报告了。 更值得分享的教训 这个项目第一版出了七条结论,其中五条后来被证明是判分器 bug 造成的伪影,而且每条都错得毫无报错。最阴的一个:参考区间只按 LOINC 码查、没看单位,同一码不同单位量纲差 40 倍,导致 100% 的 fL 制 RDW 被判成异常,一路污染了答案键。 现在这七个坑全部固化成 23 个回归测试。 特点 主实验用纯合成 Synthea 患者(Apache-2.0),仓库里零受限数据,克隆下来不花钱就能跑完纯规则部分(不需要任何 API key) 中文文档为主 代码结构对"数据不出院"的场景留了接口:代码进医院跑,只回传聚合指标

语言PythonDTAgent 评测与安全原始agent-safety原始benchmark
1

AgentGuard

Stop losing enterprise deals to AI security reviews. Production prompt injection defense specs, auditor-vetted ISO 42001 & SOC 2 answer templates, and an open-source test runner for autonomous AI agent builders.

原文
DTAgent 评测与安全
1

Zro: Uncensored Security Models

Security research often involves questions that general-purpose models refuse Meet Dolly 1 Security and Dolly 2 Security, uncensored variants of GLM-5.3 Flash and GLM-5.3 for defensive testing and red-team workflows. Run them through the Zro API or CLI with zero request retention. Since our July launch, we’ve also added new models, agent integrations, yearly plans, and referrals.

原文
DTAgent 评测与安全
104

Resume Claim Guard

Project URL Category Other Project Title Resume Claim Guard: an agent skill that stops hallucinated resume claims Project An open-source agent skill (SKILL.md) for Claude Code and AI coding agents that enforces a strict 4-step resume workflow: anchor every claim to a verified fact base, audit metrics against exaggeration, and generate bilingual role-aligned bullets. MIT licensed. Homepage: Highlights Prevents hallucinated resume claims by binding each bullet to a verified fact base, and shows changes as yellow-highlight diffs so you can review every edit. Bilingual output, usable for interview prep too. Example Code markdown or Videos

原文
语言PythonDTAgent 评测与安全原始ai-agent原始anthropic
0

FastAPI Supabase Starter

A FastAPI + React SaaS starter on Supabase: sign-in, teams, roles, API keys, Stripe and Razorpay payments, a Gemini assistant and an MCP server. Postgres row-level security guards every table, even for API keys and AI clients. From ₹1,999, one-time.

原文
DTAgent 评测与安全
2

catalogaudit

Free Shopify catalog audit: see how Google Shopping and AI shopping agents like ChatGPT read your products, and get your five worst products fixed.

原文
DTAgent 评测与安全
1

Hashling

Free, open-source security for your Mac. See what Claude Code, Codex and Cursor are allowed to do, check texts for scams, and ask your own AI (or one running fully on your Mac) "am I safe?". Plain words, and nothing changes without your Touch ID.

原文
DTAgent 评测与安全
2

Interview your analysts instead of auto-generating a semantic layer

I’ve spent the last year building context for AI analysts in the mid-market. I’ve open sourced a library from my learnings, along the same principles of the Matt Pocock skills. Principles: * Claude / Codex / Cursor etc are great data agents - you just need context to enable them to answer business questions correctly. * Much of the important context lives in analysts heads' * Context must be maintainable (short, human readable - markdown and yaml) * Added context should be load bearing, not repeat of information that is available in another source of truth - and evaluations prove the value of the context * Companies should own their own context, and must be shareable with any model * Context should be version controlled (and github works as a context store). What the library does: * Performs an interview with your analysts (context-interview skill), one domain at a time, similar to what I have done as a consultant. The interview is informed by your query logs. * The AI develops evaluation questions at the same time as building the context - and at the end of each domain interview asks you to run an evaluation with and without the context. The evaluation will actually check the results against your dashboards if you allow it. We have shown increases in accuracy from 87% to 95% by adding this context in our imaginary open-source company, Shorelane. * Provides a set of skills to use the context that follow best practices for self service analytics: * Write a plain english plan for the business user to approve * Execute the plan * Verify that the execution followed the plan, and provide a simple uncertainty score. * Enable a adversarial review of the answer if their are doubts. What you get: Context written in markdown and yaml after you finish the interview. We use a single github repo, divided into domains. The interview can be shared and restarted by checking in/out of github. A set of skills to share with your business team Optional: We offer a fast MCP server to serve the Github-based context to your team.

原文
语言PythonDTAgent 评测与安全原始analytics-engineering原始claude-code
10

BotGauge

AI agents don't just generate answers anymore. They take actions, call tools, and access real data. BotGauge helps engineering teams find where agents break before users do. Run adaptive red-team attacks and evals, catch security and behavioral failures, and replay every issue with full traces. Turn failures into regression tests, continuously retest as your agents evolve, and monitor their behavior in production. One reliability layer from development to deployment.

原文
DTAgent 评测与安全
205

iGentrix

iGentrix is the AI agent platform that proves an agent works before it goes live. A built-in launch check runs about 30 realistic customer questions and about 20 attacks in a safe sandbox, then scores accuracy, security and coverage and shows how to fix each failure. Then one agent serves WhatsApp, web chat, email, Messenger, Instagram and Telegram, with a team inbox, lead capture and automations included.

原文
DTAgent 评测与安全
2

Krill

Run Claude Code, Codex, Cursor CLI, OpenCode or any shell side by side, each in its own Git worktree, with a queue of agents waiting on you and diff review. A separate daemon holds the sessions, so agents keep running when you quit, crash or update the app. Rust desktop app, no Electron or web view. Oct 28 benchmark (Krill 0.2.20 vs Orca 1.4.242, 10 terminals, screen on): 1/7 the memory, idle CPU and install size. Free, Apache-2.0. Apple Silicon Mac; Windows preview.

原文
DTAgent 评测与安全
1

A Lego-like composable Framework for RSI

GEAR: An Open General Evolutionary Architecture for Recursive Self-Improvement of LLM Agent Harness. An open framework for harness evolution, reproducible evaluation, and learning across tasks.

原文
DTAgent 评测与安全
—

ReguLattice AI Governance Platform

ReguLattice is an AI-native governance platform that helps organizations discover, assess, govern, monitor, and improve AI systems and AI agents. It finds shadow AI, keeps people in charge of high-risk AI decisions, and maintains live, audit-ready evidence mapped to ISO/IEC 42001, the NIST AI RMF and the EU AI Act.

原文
DTAgent 评测与安全
2

Aident Loadout:让多个 AI Agent 共用一套应用连接和 Skills

同时用好几个 Agent 时,每个都要单独配 MCP、API Key 和授权,很麻烦。Loadout 提供一个远程 MCP 服务器( npm CLI(@aident-ai/cli)和一个开源 Skill,Agent 按需求搜索应用动作、读取实时 schema 再执行;另外内置 400+ 个现成的 Skills(调研、内容、运营等)。OAuth 在浏览器里授权,API Key 保存在 Aident Vault 里不交给模型,每次调用都会记录在审计日志里,并显示花费。可以免费按量付费开始使用。 我是 Aident 团队成员,本帖为自荐。

语言ShellDTAgent 评测与安全原始agent-skills原始ai-agents
7

Humanbound

Humanbound continuously tests AI agents and chatbots with adversarial attacks, tags findings to OWASP LLM and Agentic Top 10, and scores security posture.

原文
DTAgent 评测与安全
1

Agent Index

Agent Index is a free, sourced directory of about 404 AI agents for compliance, security, risk and business operations, plus about 483 independently verified audit firms (SOC 2, ISO 27001, PCI DSS, HITRUST, CERT-In and more). Compare tools side by side, filter by framework, region and integration, and find auditors by country and city. Not a ranking.

原文
DTAgent 评测与安全
1

lfg

LFG turns Claude Code into an engineering pipeline you direct instead of type into. Describe a change in one sentence and specialist agents plan it, build it test-first, and check it with lint, types, tests, security scans and an expert review. You review by talking to the diff, see what a senior engineer would ask, and decide what ships. It runs on your machine, never pushes or reads your secrets, and is free and open source.

原文
DTAgent 评测与安全
2

Reins —— 自托管零信任远程运维平台

自托管零信任远程运维平台。单个 Go 二进制 Relay + 一行命令安装的 Agent + 50+ 命令 CLI,通过反向 WebSocket 穿透 NAT——被控端零入站端口。mTLS + 内置 CA + 多租户 RBAC + 防篡改审计 hash chain,并为 AI 代理提供规则门禁(自动化操作前必须先读项目运维规则并携带内容指纹 ack)。

语言GoDTAgent 评测与安全原始ai-agents原始devops
2

NVIDIA / SkillSpector

Security scanner for AI agent skills. Detect vulnerabilities, malicious patterns, security risks, prompt injection, data exfiltration, and supply-chain risks in Claude Code, Codex, and MCP skills before you install them.

原文
语言PythonDTAgent 评测与安全
2.2万

CovaSyn Chemistry Tools

All-in-one chemistry MCP server for pharma and biotech: over 250 tools in 18 families for spectra, stability and toxicology. Try it free in Claude, ChatGPT, Cursor. 14 to 41% without CovaSyn, 76 to 92% with CovaSyn. Measured on MolecularIQ, a benchmark we did not design (arXiv:2601.15279, Klambauer Lab, Institute for Machine Learning, JKU Linz).

原文
DTAgent 评测与安全
1

输入一篇论文,生成带引用的商业化初筛报告

一个开源的多智能体 LLM 工具:输入论文 PDF 或研究主题,先由代码检索并验证论文、专利和市场来源,再让 6 个 agent 只在这批冻结的来源里分析,输出技术成熟度、制造成熟度、专利、市场等评分,以及每条论断都标出处的报告。适合高校技术转移、科研人员和早期投资人做初筛;想看多智能体项目如何做测试、断点恢复和成本控制的开发者也可以参考。 亮点 1. 不用注册、不用 API key 就能先看效果:在线示例页有 5 份完整报告(其中 1 份中文),打开时不调用任何模型。 2. 先证据、后模型:检索、URL/DOI 验证和去重都由普通代码完成,模型只能引用登记过的来源;加权总分由代码重算,不让模型自己算。 3. 实验连失败一起公开:我预注册了 1 个、4 个、6 个 agent 的对照实验(各 30 次),隔两个月换模型重跑。「4 个比 6 个省一半」两次都成立,「4 个比 1 个质量更好」没能复现;还发现审稿 agent 因为提示词少写一句限制,30 次里 27 次第一次输出就不合格,补上一句后,因这条限制失败的次数从 21 降到 3。方案、原始数据和局限都在仓库里。 4. 工程细节可以直接参考:每次运行独立子进程、中断后从 Checkpoint 恢复、付费请求有回执(回包丢失 24 小时内可找回结果) OpenTelemetry Trace。 5. 方便参与:带 good first issue 标签的任务不需要任何 API key,测试和浏览器 smoke 测试都不发付费请求,已有外部贡献者的 PR 合并。 我是项目作者,本次为自荐。介绍由 AI 辅助撰写,数字均来自仓库里公开的结果文档。

语言PythonDTAgent 评测与安全原始academic-research原始ai-agent
774

KageX

KageX is an offensive AI security company. Our flagship FreakLabs teaches AI red teaming by doing, with 20 hands-on labs against live chatbots, RAG systems and agents, a 9-level course and a verifiable AI Red Teamer certificate. There is a free tier, a free CTF called SentinelGoat, and open-source tools for testing your own AI apps. Built for pentesters, security engineers and developers shipping LLM features.

原文
DTAgent 评测与安全
2

Markdown is 40× more token-efficient with SQLite

Hi HN, I'm Richard, founder of Native. I'm working on open-source context ownership, to stop AI centralisation: https://github.com/withnative/native My first focus has been making Markdown vaults powerful enough so that teams can self-host all of their context (messaging, docs and projects), without it being externally fragmented or rate-limited. I'm sharing some early results: Markdown vaults can be 40× more efficient with SQLite! https://www.withnative.ai/research-and-posts/markdown-contex... Here's some of the main ideas: - SQLite or Markdown folders, you pick. You can project the same authoritative event log into either form; more projection formats will follow (e.g. maybe neo4j; we do use a graph structure). - Empirically: SQLite is better for agents! On a 120 trial evaluation for 11,000 records, agents use 40× fewer tokens with SQLite vs Markdown, per correct answer. - Everything is a file or SQLite row. Messages, documents and projects: all can be modelled in your vault. You get local-first collaboration with teammates, synced to a cloud of your choice. - You can render your vault with arbitrary UI. Your WhatsApp-like UI speaks to your teammate's Slack-like UI; both are just views over the same database. A few things I've learned building this: - Agents do pretty well even without SQL if you give them strongly typed Markdown frontmatter. This was one arm of the trial I ran. I didn't have any clear expectations, but it was cool to watch: agents ended up writing Python scripts to do SQL-like queries. - Cumulatively, cached input tokens can get expensive. I feel silly in hindsight here. I'd gotten used to letting my sessions run long, "It's all cached so it's very cheap". But when I looked at my results here, over 2/3 of the token bill on long sessions was cached input... - "Markdown vaults don't scale as well as databases" isn't as self-evident as I think. This is why I decided to benchmark it... I've had some very blank stares when I try to explain how Markdown files might break at true agent-scale volume. Unfortunately, the dataset is private, because it's two months of my own work. I am considering open-sourcing it even so, but there is definitely some embarrassing stuff in there. TBD... I know that independent result verification matters! The core engine that I evaluated is open-source, with more to be released soon: https://github.com/withnative/native If you're interested in trying connected context on an open engine, you can get updates from our website or email me directly: [email protected] https://www.withnative.ai/

原文
DTAgent 评测与安全
—

Agent Payment Node:AI 代理的本地支付工具

Agent Payment Node 是用 TypeScript 编写的本地支付运行时,通过 CLI 和 stdio MCP 供 AI 代理检查 x402 报价、准备有界支付并查询状态与收据。仅支持 Apple Silicon macOS 和 Node 24;直接转账和 EVM 签名需前台所有者批准,Solana exact x402 可按所有者接纳的策略运行。软件免费,链上和服务商费用另计。 亮点 将报价检查、有界准备、操作状态/收据和恢复路径组合为本地代理工具,适合研究 MCP 工具接口、支付状态机及所有者授权边界。 这是 Nuanu AI 的 MIT 开源项目,本次代表 Nuanu AI 自荐。具备资金操作能力;本地软件托管、错误策略和主机失陷仍有风险。没有声称独立安全审计、当前客户端兼容性复测或生产使用证明。 已按要求在 HelloGitHub 网站用完整 GitHub URL 搜索;匿名搜索没有展示匹配结果,但提示登录才能查看更多。另检查了当前完整仓库的 263 个文本文件(含 253 个 content 文件)及开放/关闭 issue/PR 历史,未发现 APN 名称、slug 或 Nuanu。

语言TypeScriptDTAgent 评测与安全
0

开源了一个 MCP 治理网关: Jev 判 0.94、d3 判 0.70,两个判官说不同的话时怎么办

给 AI agent 做了个 MCP 执法网关( Rust ),核心问题:当两个决策模型对同一件事给出不同判断时,怎么安全处理? 实测:同一个越权场景( coding agent 试图 merge 夹带无关变更的 PR ): - Jev 给 0.94 置信度 → 准入层 Accept → 授权挂起 → DENY - d3 ( vLLM SR )给 0.70 → Hold → 什么都不改 → ALLOW 两个都对。分歧不是 bug ,是校准本性( d3 ECE 4.3 vs Jev 8.0 )。治理层的价值在于让分歧安全。 网关做的事:每个 tools/call 先过 r2r_decide 门控,ALLOW 转发上游,DENY 返回 isError + 证据文本(相位 + 规则 + 事件 ID ),全程审计到 EventStore 。 零 API key 复现: cargo run -- fixture cargo run -- fixture --provider d3 GitHub: ( Apache-2.0, v0.2.0 )

DTAgent 评测与安全
—

incursio

AI agents that audit your code for security vulnerabilities, on your Claude or ChatGPT subscription or on local models with Ollama. Based on Strix. Join the open beta.

原文
DTAgent 评测与安全
3

Vibefilo

Security and GDPR checks inside Claude Code, Cursor and Codex. Find leaked secrets and RLS gaps, then verify every fix. Your code stays on your machine.

原文
DTAgent 评测与安全
1

Browser Extractor

Inspect any live website like a design file. Explore styles, edit elements in place, audit UX and accessibility, and turn your changes into actionable context for your coding agent. All from a free Chrome extension.

原文
DTAgent 评测与安全
2

Open_SwarmForce

SWARMFORGE is an agentic CI/CD gate for the change path that matters: it reviews a committed or working-tree diff, can group and fix confirmed findings in isolated sandboxes, re-reviews the result, then requires deterministic repository checks before it publishes a PR. It runs with OpenAI Codex/GPT, any OpenRouter model, local open-source models (Ollama, LM Studio), or Claude Code (claude -p, opt-in) — with cost-aware routing measured on an L0–L6 benchmark ladder.

原文
DTAgent 评测与安全
9

Sage 7

Your AI writes the code. Sage 7 makes sure it builds the right thing: map the whole product, brief Lovable, Cursor or Claude Code with living context, and audit every push against the plan.

原文
DTAgent 评测与安全
1

Vesalius

Vesalius reads your repository once, right on your Mac, and remembers how everything connects: which file uses which, who calls what, where a feature actually lives. Claude Code can then ask it directly and get answers with exact file and line references, usually in one call instead of a chain of searches. In our benchmark that cut token use roughly in half with no measurable change in answer quality. Vesalius itself never uploads your code.

原文
DTAgent 评测与安全
8

OrqLabs

OrqLabs is a no-code platform where 22 specialist AI agents (Claude & OpenAI) create social content, answer customers 24/7 on WhatsApp and Telegram, find and score leads, audit SEO and rewrite your strategy from real GA4 data. Every publish or outreach step waits for your approval, and every AI call is metered against a monthly budget cap you can't exceed. 109 templates, 75 integrations, works inside Claude and ChatGPT via MCP. Free plan, no credit card.

原文
DTAgent 评测与安全
2

外链检测:提交网站之前,先查这个外链来源值不值得发

做独立产品的人常把网站提交到导航站、awesome 列表和客座文章站换外链,可网上流传的外链清单里混着已经停放的域名、明码标价卖 dofollow 的站,还有同一个人运营的站群。backlink-guard 在你提交之前逐个检查这些站:读它们自己的页面,对照 Google 的链接垃圾政策,给出跳过、先看看、只免费提交、价值低、可以提交五种建议,并附上原文证据;链接上线后再核实它是否真的传权重、放在页面哪里。适合做 SEO、发产品、维护开源项目的人。 亮点 不帮你多发外链,而是告诉你哪里别发:停放或过期的域名、出售传权重链接、要求回链、外链指向博彩站、共用一套页面模板或同一个统计账号的站群都会被标出来,每条结论都带页面原文或数字,可以自己核对。 不配任何 key 也能免费跑页面检查;配上自己的 Glasser 或 AIsa key 再加评分、访问量、垃圾分和 Google 搜索历史,一次查三十个站约 $0.46;在 agent 里用时,每次付费调用前都会先报价、等你同意。 在 100 个它没见过的站上验证过:Ahrefs 标为垃圾的 40 个站里,37 个没有被给出「可以提交」。 除了投稿前的检查,还能体检自己的外链画像,找出指向 404 页面的外链并建议 301,找出链向竞品却没链向你的站,看 Google AI 概览和 AI 助手在你的关键词上引用了谁。 命令行、Agent Skill、Claude Code 插件和 MCP 服务器四种用法;纯 Python、无依赖、MIT 开源,它自己从不替你提交任何东西。

语言PythonDTAgent 评测与安全原始agent-skills原始backlinks
28