feat(llm-gateway): 新增对外 LLM API 代理服务 — 完整的监管注入、内容审查和管理后台

## 新增微服务: llm-gateway (端口 3008)

对外提供与 Anthropic/OpenAI 完全兼容的 API 接口,中间拦截实现:
- API Key 认证:由我们分配 Key 给外部用户,SHA-256 哈希存储
- System Prompt 注入:在请求转发前注入监管合规内容(支持 prepend/append)
- 内容审查过滤:对用户消息进行关键词/正则匹配,支持 block/warn/log 三种动作
- 用量记录:异步批量写入,跟踪 token 消耗和费用估算
- 审计日志:记录每次请求的来源 IP、过滤状态、注入状态等
- 速率限制:基于内存滑动窗口的 RPM 限制

### 技术选型
- Fastify (非 NestJS):纯代理场景无需 DI 容器,路由开销 ~2ms
- SSE 流式管道:零缓冲直通,支持 Anthropic streaming 和 OpenAI streaming
- 规则缓存:30 秒 TTL,避免每次请求查库

### API 端点
- POST /v1/messages — Anthropic Messages API 代理(流式+非流式)
- POST /v1/embeddings — OpenAI Embeddings API 代理
- POST /v1/chat/completions — OpenAI Chat Completions API 代理
- GET /health — 健康检查

## 数据库 (5 张新表)

- gateway_api_keys: 外部用户 API Key(权限、限速、预算、过期时间)
- gateway_injection_rules: 监管内容注入规则(位置、匹配模型、匹配 Key)
- gateway_content_rules: 内容审查规则(关键词/正则、block/warn/log)
- gateway_usage_logs: Token 用量记录(按 Key、模型、提供商统计)
- gateway_audit_logs: 请求审计日志(IP、过滤状态、注入状态)

## Admin 后端 (conversation-service)

4 个 NestJS 控制器,挂载在 /conversations/admin/gateway/ 下:
- AdminGatewayKeysController: Key 的 CRUD + toggle
- AdminGatewayInjectionRulesController: 注入规则 CRUD + toggle
- AdminGatewayContentRulesController: 内容审查规则 CRUD + toggle
- AdminGatewayDashboardController: 仪表盘汇总、用量查询、审计日志查询

5 个 ORM 实体文件对应 5 张数据库表。

## Admin 前端 (admin-client)

新增 features/llm-gateway 模块,Tabs 布局包含 5 个管理面板:
- API Key Tab: 创建/删除/启停 Key,创建时一次性显示完整 Key
- 注入规则 Tab: 配置监管内容(前置/追加到 system prompt)
- 内容审查 Tab: 配置关键词/正则过滤规则
- 用量统计 Tab: 查看 token 消耗、费用、响应时间
- 审计日志 Tab: 查看请求记录、过滤命中、注入状态

菜单项: GatewayOutlined + "LLM 网关",位于"系统总监"和"数据分析"之间。

## 基础设施

- docker-compose.yml: 新增 llm-gateway 服务定义
- kong.yml: 新增 /v1/messages、/v1/embeddings、/v1/chat/completions 路由
  - 超时设置 300 秒(LLM 长响应)
  - CORS 新增 X-Api-Key、anthropic-version、anthropic-beta 头
- init-db.sql: 新增 5 张 gateway 表的建表语句

## 架构说明

内部服务(conversation-service、knowledge-service、evolution-service)继续直连 API,
llm-gateway 仅服务外部用户。两者通过共享 PostgreSQL 数据库关联配置。

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
hailin 2026-02-25 22:32:25 -08:00
parent 021afd8677
commit 6476bd868f
40 changed files with 3724 additions and 2 deletions

View File

@ -426,6 +426,40 @@ services:
networks:
- iconsulting-network
#=============================================================================
# LLM Gateway - 对外 API 代理服务
#=============================================================================
llm-gateway:
build:
context: .
dockerfile: packages/services/llm-gateway/Dockerfile
container_name: iconsulting-llm-gateway
restart: unless-stopped
depends_on:
postgres:
condition: service_healthy
environment:
NODE_ENV: production
PORT: 3008
DATABASE_URL: postgresql://${POSTGRES_USER:-postgres}:${POSTGRES_PASSWORD:-postgres}@postgres:5432/${POSTGRES_DB:-iconsulting}
ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY}
ANTHROPIC_UPSTREAM_URL: ${ANTHROPIC_BASE_URL:-https://api.anthropic.com}
OPENAI_API_KEY: ${OPENAI_API_KEY}
OPENAI_UPSTREAM_URL: ${OPENAI_BASE_URL:-https://api.openai.com}
RULES_CACHE_TTL_MS: 30000
LOG_LEVEL: info
ports:
- "3008:3008"
healthcheck:
test: ["CMD", "wget", "--no-verbose", "--tries=1", "--spider", "http://127.0.0.1:3008/health"]
interval: 30s
timeout: 10s
retries: 3
start_period: 10s
networks:
- iconsulting-network
#=============================================================================
# 前端 Nginx
#=============================================================================

View File

@ -169,6 +169,28 @@ services:
- DELETE
- OPTIONS
#-----------------------------------------------------------------------------
# LLM Gateway - 对外 API 代理服务
# 注意: 需要长超时以支持 LLM 流式响应
#-----------------------------------------------------------------------------
- name: llm-gateway
url: http://llm-gateway:3008
connect_timeout: 60000
write_timeout: 300000
read_timeout: 300000
retries: 2
routes:
- name: llm-gateway-routes
paths:
- /v1/messages
- /v1/embeddings
- /v1/chat/completions
strip_path: false
preserve_host: true
methods:
- POST
- OPTIONS
#===============================================================================
# 全局插件配置
#===============================================================================
@ -198,6 +220,9 @@ plugins:
- Authorization
- X-User-Id
- X-Request-Id
- X-Api-Key
- anthropic-version
- anthropic-beta
exposed_headers:
- X-Request-Id
credentials: true

View File

@ -15,6 +15,7 @@ import { ObservabilityPage } from './features/observability';
import { AssessmentConfigPage } from './features/assessment-config';
import { CollectionConfigPage } from './features/collection-config';
import { SupervisorPage } from './features/supervisor';
import { LLMGatewayPage } from './features/llm-gateway';
function App() {
return (
@ -44,6 +45,7 @@ function App() {
<Route path="tenants" element={<TenantsPage />} />
<Route path="mcp" element={<McpPage />} />
<Route path="supervisor" element={<SupervisorPage />} />
<Route path="llm-gateway" element={<LLMGatewayPage />} />
<Route path="observability" element={<ObservabilityPage />} />
<Route path="settings" element={<SettingsPage />} />
</Route>

View File

@ -0,0 +1,209 @@
import { useQuery, useMutation, useQueryClient } from '@tanstack/react-query';
import { message } from 'antd';
import { gatewayApi } from '../infrastructure/llm-gateway.api';
const KEYS = {
dashboard: 'gateway-dashboard',
keys: 'gateway-keys',
injectionRules: 'gateway-injection-rules',
contentRules: 'gateway-content-rules',
usage: 'gateway-usage',
auditLogs: 'gateway-audit-logs',
};
// ─── Dashboard ───
export function useDashboard() {
return useQuery({
queryKey: [KEYS.dashboard],
queryFn: () => gatewayApi.getDashboard(),
refetchInterval: 30_000,
});
}
// ─── API Keys ───
export function useApiKeys() {
return useQuery({
queryKey: [KEYS.keys],
queryFn: () => gatewayApi.listKeys(),
});
}
export function useCreateKey() {
const qc = useQueryClient();
return useMutation({
mutationFn: gatewayApi.createKey,
onSuccess: () => {
qc.invalidateQueries({ queryKey: [KEYS.keys] });
qc.invalidateQueries({ queryKey: [KEYS.dashboard] });
message.success('API Key 创建成功');
},
onError: () => message.error('创建失败'),
});
}
export function useUpdateKey() {
const qc = useQueryClient();
return useMutation({
mutationFn: ({ id, dto }: { id: string; dto: any }) => gatewayApi.updateKey(id, dto),
onSuccess: () => {
qc.invalidateQueries({ queryKey: [KEYS.keys] });
message.success('更新成功');
},
onError: () => message.error('更新失败'),
});
}
export function useDeleteKey() {
const qc = useQueryClient();
return useMutation({
mutationFn: gatewayApi.deleteKey,
onSuccess: () => {
qc.invalidateQueries({ queryKey: [KEYS.keys] });
qc.invalidateQueries({ queryKey: [KEYS.dashboard] });
message.success('删除成功');
},
onError: () => message.error('删除失败'),
});
}
export function useToggleKey() {
const qc = useQueryClient();
return useMutation({
mutationFn: gatewayApi.toggleKey,
onSuccess: () => {
qc.invalidateQueries({ queryKey: [KEYS.keys] });
qc.invalidateQueries({ queryKey: [KEYS.dashboard] });
},
onError: () => message.error('操作失败'),
});
}
// ─── Injection Rules ───
export function useInjectionRules() {
return useQuery({
queryKey: [KEYS.injectionRules],
queryFn: () => gatewayApi.listInjectionRules(),
});
}
export function useCreateInjectionRule() {
const qc = useQueryClient();
return useMutation({
mutationFn: gatewayApi.createInjectionRule,
onSuccess: () => {
qc.invalidateQueries({ queryKey: [KEYS.injectionRules] });
message.success('注入规则创建成功');
},
onError: () => message.error('创建失败'),
});
}
export function useUpdateInjectionRule() {
const qc = useQueryClient();
return useMutation({
mutationFn: ({ id, dto }: { id: string; dto: any }) => gatewayApi.updateInjectionRule(id, dto),
onSuccess: () => {
qc.invalidateQueries({ queryKey: [KEYS.injectionRules] });
message.success('更新成功');
},
onError: () => message.error('更新失败'),
});
}
export function useDeleteInjectionRule() {
const qc = useQueryClient();
return useMutation({
mutationFn: gatewayApi.deleteInjectionRule,
onSuccess: () => {
qc.invalidateQueries({ queryKey: [KEYS.injectionRules] });
message.success('删除成功');
},
onError: () => message.error('删除失败'),
});
}
export function useToggleInjectionRule() {
const qc = useQueryClient();
return useMutation({
mutationFn: gatewayApi.toggleInjectionRule,
onSuccess: () => {
qc.invalidateQueries({ queryKey: [KEYS.injectionRules] });
},
onError: () => message.error('操作失败'),
});
}
// ─── Content Rules ───
export function useContentRules() {
return useQuery({
queryKey: [KEYS.contentRules],
queryFn: () => gatewayApi.listContentRules(),
});
}
export function useCreateContentRule() {
const qc = useQueryClient();
return useMutation({
mutationFn: gatewayApi.createContentRule,
onSuccess: () => {
qc.invalidateQueries({ queryKey: [KEYS.contentRules] });
message.success('审查规则创建成功');
},
onError: () => message.error('创建失败'),
});
}
export function useUpdateContentRule() {
const qc = useQueryClient();
return useMutation({
mutationFn: ({ id, dto }: { id: string; dto: any }) => gatewayApi.updateContentRule(id, dto),
onSuccess: () => {
qc.invalidateQueries({ queryKey: [KEYS.contentRules] });
message.success('更新成功');
},
onError: () => message.error('更新失败'),
});
}
export function useDeleteContentRule() {
const qc = useQueryClient();
return useMutation({
mutationFn: gatewayApi.deleteContentRule,
onSuccess: () => {
qc.invalidateQueries({ queryKey: [KEYS.contentRules] });
message.success('删除成功');
},
onError: () => message.error('删除失败'),
});
}
export function useToggleContentRule() {
const qc = useQueryClient();
return useMutation({
mutationFn: gatewayApi.toggleContentRule,
onSuccess: () => {
qc.invalidateQueries({ queryKey: [KEYS.contentRules] });
},
onError: () => message.error('操作失败'),
});
}
// ─── Usage & Audit ───
export function useUsageLogs(params?: { keyId?: string; startDate?: string; endDate?: string }) {
return useQuery({
queryKey: [KEYS.usage, params],
queryFn: () => gatewayApi.getUsage(params),
});
}
export function useAuditLogs(params?: { keyId?: string; filtered?: boolean; startDate?: string; endDate?: string }) {
return useQuery({
queryKey: [KEYS.auditLogs, params],
queryFn: () => gatewayApi.getAuditLogs(params),
});
}

View File

@ -0,0 +1 @@
export { LLMGatewayPage } from './presentation/pages/LLMGatewayPage';

View File

@ -0,0 +1,176 @@
import api from '../../../shared/utils/api';
const BASE = '/conversations/admin/gateway';
// ─── Types ───
export interface GatewayApiKey {
id: string;
tenantId: string | null;
keyHash: string;
keyPrefix: string;
name: string;
owner: string;
permissions: {
allowedModels: string[];
allowStreaming: boolean;
allowTools: boolean;
};
rateLimitRpm: number;
rateLimitTpd: number;
monthlyBudget: number | null;
enabled: boolean;
expiresAt: string | null;
lastUsedAt: string | null;
createdBy: string | null;
createdAt: string;
rawKey?: string; // only on creation
}
export interface InjectionRule {
id: string;
name: string;
description: string | null;
position: 'prepend' | 'append';
content: string;
matchModels: string[];
matchKeyIds: string[] | null;
priority: number;
enabled: boolean;
createdAt: string;
updatedAt: string;
}
export interface ContentRule {
id: string;
name: string;
type: 'keyword' | 'regex';
pattern: string;
action: 'block' | 'warn' | 'log';
rejectMessage: string | null;
priority: number;
enabled: boolean;
createdAt: string;
updatedAt: string;
}
export interface UsageLog {
id: string;
apiKeyId: string;
model: string;
provider: string;
inputTokens: number;
outputTokens: number;
totalTokens: number;
costUsd: number | null;
durationMs: number;
statusCode: number;
createdAt: string;
}
export interface AuditLog {
id: string;
apiKeyId: string;
requestMethod: string;
requestPath: string;
requestModel: string | null;
requestIp: string;
contentFiltered: boolean;
filterRuleId: string | null;
injectionApplied: boolean;
responseStatus: number;
durationMs: number;
createdAt: string;
}
export interface DashboardData {
keys: { total: number; active: number };
today: { requests: number; tokens: number; cost: number; filtered: number };
month: { requests: number; tokens: number; cost: number };
}
// ─── API ───
export const gatewayApi = {
// Dashboard
getDashboard: async (): Promise<DashboardData> => {
const res = await api.get(`${BASE}/dashboard`);
return res.data.data;
},
// Keys
listKeys: async () => {
const res = await api.get(`${BASE}/keys`);
return res.data.data;
},
createKey: async (dto: { name: string; owner?: string; rateLimitRpm?: number; rateLimitTpd?: number; monthlyBudget?: number; expiresAt?: string }) => {
const res = await api.post(`${BASE}/keys`, dto);
return res.data.data;
},
updateKey: async (id: string, dto: Partial<GatewayApiKey>) => {
const res = await api.put(`${BASE}/keys/${id}`, dto);
return res.data.data;
},
deleteKey: async (id: string) => {
const res = await api.delete(`${BASE}/keys/${id}`);
return res.data;
},
toggleKey: async (id: string) => {
const res = await api.post(`${BASE}/keys/${id}/toggle`);
return res.data.data;
},
// Injection Rules
listInjectionRules: async () => {
const res = await api.get(`${BASE}/injection-rules`);
return res.data.data;
},
createInjectionRule: async (dto: { name: string; position?: string; content: string; description?: string; matchModels?: string[]; priority?: number }) => {
const res = await api.post(`${BASE}/injection-rules`, dto);
return res.data.data;
},
updateInjectionRule: async (id: string, dto: Partial<InjectionRule>) => {
const res = await api.put(`${BASE}/injection-rules/${id}`, dto);
return res.data.data;
},
deleteInjectionRule: async (id: string) => {
const res = await api.delete(`${BASE}/injection-rules/${id}`);
return res.data;
},
toggleInjectionRule: async (id: string) => {
const res = await api.post(`${BASE}/injection-rules/${id}/toggle`);
return res.data.data;
},
// Content Rules
listContentRules: async () => {
const res = await api.get(`${BASE}/content-rules`);
return res.data.data;
},
createContentRule: async (dto: { name: string; type?: string; pattern: string; action?: string; rejectMessage?: string; priority?: number }) => {
const res = await api.post(`${BASE}/content-rules`, dto);
return res.data.data;
},
updateContentRule: async (id: string, dto: Partial<ContentRule>) => {
const res = await api.put(`${BASE}/content-rules/${id}`, dto);
return res.data.data;
},
deleteContentRule: async (id: string) => {
const res = await api.delete(`${BASE}/content-rules/${id}`);
return res.data;
},
toggleContentRule: async (id: string) => {
const res = await api.post(`${BASE}/content-rules/${id}/toggle`);
return res.data.data;
},
// Usage & Audit
getUsage: async (params?: { keyId?: string; startDate?: string; endDate?: string; limit?: number }) => {
const res = await api.get(`${BASE}/usage`, { params });
return res.data.data;
},
getAuditLogs: async (params?: { keyId?: string; filtered?: boolean; startDate?: string; endDate?: string; limit?: number }) => {
const res = await api.get(`${BASE}/audit-logs`, { params });
return res.data.data;
},
};

View File

@ -0,0 +1,137 @@
import { useState } from 'react';
import { Table, Button, Modal, Form, Input, InputNumber, Switch, Space, Tag, Popconfirm, Typography, message as antMsg } from 'antd';
import { PlusOutlined, CopyOutlined } from '@ant-design/icons';
import { useApiKeys, useCreateKey, useDeleteKey, useToggleKey } from '../../application/useLLMGateway';
import type { GatewayApiKey } from '../../infrastructure/llm-gateway.api';
const { Text } = Typography;
export function ApiKeysTab() {
const { data, isLoading } = useApiKeys();
const createMutation = useCreateKey();
const deleteMutation = useDeleteKey();
const toggleMutation = useToggleKey();
const [createOpen, setCreateOpen] = useState(false);
const [newKeyModal, setNewKeyModal] = useState<{ rawKey: string } | null>(null);
const [form] = Form.useForm();
const handleCreate = async () => {
const values = await form.validateFields();
const result = await createMutation.mutateAsync(values);
setCreateOpen(false);
form.resetFields();
if (result.rawKey) {
setNewKeyModal({ rawKey: result.rawKey });
}
};
const columns = [
{
title: '名称',
dataIndex: 'name',
key: 'name',
},
{
title: 'Key 前缀',
dataIndex: 'keyPrefix',
key: 'keyPrefix',
render: (v: string) => <Text code>{v}...</Text>,
},
{
title: '所属',
dataIndex: 'owner',
key: 'owner',
},
{
title: '限速 (RPM)',
dataIndex: 'rateLimitRpm',
key: 'rateLimitRpm',
},
{
title: '状态',
dataIndex: 'enabled',
key: 'enabled',
render: (v: boolean, record: GatewayApiKey) => (
<Switch
checked={v}
size="small"
onChange={() => toggleMutation.mutate(record.id)}
/>
),
},
{
title: '最后使用',
dataIndex: 'lastUsedAt',
key: 'lastUsedAt',
render: (v: string | null) => v ? new Date(v).toLocaleString() : '-',
},
{
title: '操作',
key: 'actions',
render: (_: any, record: GatewayApiKey) => (
<Popconfirm title="确认删除?" onConfirm={() => deleteMutation.mutate(record.id)}>
<Button type="link" danger size="small"></Button>
</Popconfirm>
),
},
];
return (
<>
<div style={{ display: 'flex', justifyContent: 'flex-end', marginBottom: 16 }}>
<Button type="primary" icon={<PlusOutlined />} onClick={() => setCreateOpen(true)}>
API Key
</Button>
</div>
<Table
columns={columns}
dataSource={data?.items}
rowKey="id"
loading={isLoading}
size="small"
pagination={false}
/>
{/* Create Modal */}
<Modal
title="创建 API Key"
open={createOpen}
onOk={handleCreate}
onCancel={() => setCreateOpen(false)}
confirmLoading={createMutation.isPending}
>
<Form form={form} layout="vertical">
<Form.Item name="name" label="名称" rules={[{ required: true, message: '请输入名称' }]}>
<Input placeholder="例如: 客户A测试Key" />
</Form.Item>
<Form.Item name="owner" label="所属用户/组织">
<Input placeholder="例如: Company A" />
</Form.Item>
<Form.Item name="rateLimitRpm" label="每分钟请求限制" initialValue={60}>
<InputNumber min={1} max={10000} style={{ width: '100%' }} />
</Form.Item>
<Form.Item name="rateLimitTpd" label="每日 Token 限制" initialValue={1000000}>
<InputNumber min={1000} max={100000000} style={{ width: '100%' }} />
</Form.Item>
</Form>
</Modal>
{/* New Key Display Modal */}
<Modal
title="API Key 已创建"
open={!!newKeyModal}
onOk={() => setNewKeyModal(null)}
onCancel={() => setNewKeyModal(null)}
cancelButtonProps={{ style: { display: 'none' } }}
>
<p> Key</p>
<div style={{ background: '#f5f5f5', padding: 12, borderRadius: 6, display: 'flex', alignItems: 'center', gap: 8 }}>
<Text code copyable style={{ flex: 1, wordBreak: 'break-all' }}>
{newKeyModal?.rawKey}
</Text>
</div>
</Modal>
</>
);
}

View File

@ -0,0 +1,55 @@
import { Table, Tag } from 'antd';
import { useAuditLogs } from '../../application/useLLMGateway';
export function AuditLogsTab() {
const { data, isLoading } = useAuditLogs();
const columns = [
{
title: '时间',
dataIndex: 'createdAt',
key: 'createdAt',
render: (v: string) => new Date(v).toLocaleString(),
width: 180,
},
{ title: '方法', dataIndex: 'requestMethod', key: 'requestMethod', width: 70 },
{ title: '路径', dataIndex: 'requestPath', key: 'requestPath', width: 160 },
{ title: '模型', dataIndex: 'requestModel', key: 'requestModel', ellipsis: true },
{ title: 'IP', dataIndex: 'requestIp', key: 'requestIp', width: 130 },
{
title: '内容过滤',
dataIndex: 'contentFiltered',
key: 'contentFiltered',
render: (v: boolean) => v ? <Tag color="red"></Tag> : <Tag color="green"></Tag>,
},
{
title: '注入',
dataIndex: 'injectionApplied',
key: 'injectionApplied',
render: (v: boolean) => v ? <Tag color="blue"></Tag> : '-',
},
{
title: '状态码',
dataIndex: 'responseStatus',
key: 'responseStatus',
render: (v: number) => <Tag color={v < 400 ? 'green' : 'red'}>{v}</Tag>,
},
{
title: '耗时',
dataIndex: 'durationMs',
key: 'durationMs',
render: (v: number) => `${(v / 1000).toFixed(1)}s`,
},
];
return (
<Table
columns={columns}
dataSource={data?.items}
rowKey="id"
loading={isLoading}
size="small"
pagination={{ pageSize: 20, showSizeChanger: false }}
/>
);
}

View File

@ -0,0 +1,109 @@
import { useState } from 'react';
import { Table, Button, Modal, Form, Input, Select, Switch, Popconfirm, Tag } from 'antd';
import { PlusOutlined } from '@ant-design/icons';
import { useContentRules, useCreateContentRule, useDeleteContentRule, useToggleContentRule } from '../../application/useLLMGateway';
import type { ContentRule } from '../../infrastructure/llm-gateway.api';
export function ContentRulesTab() {
const { data, isLoading } = useContentRules();
const createMutation = useCreateContentRule();
const deleteMutation = useDeleteContentRule();
const toggleMutation = useToggleContentRule();
const [createOpen, setCreateOpen] = useState(false);
const [form] = Form.useForm();
const handleCreate = async () => {
const values = await form.validateFields();
await createMutation.mutateAsync(values);
setCreateOpen(false);
form.resetFields();
};
const actionColors: Record<string, string> = { block: 'red', warn: 'orange', log: 'blue' };
const actionLabels: Record<string, string> = { block: '拦截', warn: '警告', log: '记录' };
const columns = [
{ title: '名称', dataIndex: 'name', key: 'name' },
{
title: '类型',
dataIndex: 'type',
key: 'type',
render: (v: string) => <Tag>{v === 'keyword' ? '关键词' : '正则'}</Tag>,
},
{
title: '匹配模式',
dataIndex: 'pattern',
key: 'pattern',
ellipsis: true,
width: 200,
},
{
title: '动作',
dataIndex: 'action',
key: 'action',
render: (v: string) => <Tag color={actionColors[v]}>{actionLabels[v]}</Tag>,
},
{
title: '拒绝消息',
dataIndex: 'rejectMessage',
key: 'rejectMessage',
ellipsis: true,
width: 200,
},
{
title: '状态',
dataIndex: 'enabled',
key: 'enabled',
render: (v: boolean, record: ContentRule) => (
<Switch checked={v} size="small" onChange={() => toggleMutation.mutate(record.id)} />
),
},
{
title: '操作',
key: 'actions',
render: (_: any, record: ContentRule) => (
<Popconfirm title="确认删除?" onConfirm={() => deleteMutation.mutate(record.id)}>
<Button type="link" danger size="small"></Button>
</Popconfirm>
),
},
];
return (
<>
<div style={{ display: 'flex', justifyContent: 'flex-end', marginBottom: 16 }}>
<Button type="primary" icon={<PlusOutlined />} onClick={() => setCreateOpen(true)}>
</Button>
</div>
<Table columns={columns} dataSource={data?.items} rowKey="id" loading={isLoading} size="small" pagination={false} />
<Modal
title="创建内容审查规则"
open={createOpen}
onOk={handleCreate}
onCancel={() => setCreateOpen(false)}
confirmLoading={createMutation.isPending}
>
<Form form={form} layout="vertical">
<Form.Item name="name" label="规则名称" rules={[{ required: true }]}>
<Input placeholder="例如: 禁止政治敏感话题" />
</Form.Item>
<Form.Item name="type" label="匹配类型" initialValue="keyword">
<Select options={[{ label: '关键词匹配', value: 'keyword' }, { label: '正则表达式', value: 'regex' }]} />
</Form.Item>
<Form.Item name="pattern" label="匹配模式" rules={[{ required: true }]}>
<Input placeholder="关键词或正则表达式" />
</Form.Item>
<Form.Item name="action" label="触发动作" initialValue="block">
<Select options={[{ label: '拦截请求', value: 'block' }, { label: '警告并放行', value: 'warn' }, { label: '仅记录', value: 'log' }]} />
</Form.Item>
<Form.Item name="rejectMessage" label="拒绝消息(拦截时返回给用户)">
<Input placeholder="Content blocked by policy." />
</Form.Item>
</Form>
</Modal>
</>
);
}

View File

@ -0,0 +1,99 @@
import { useState } from 'react';
import { Table, Button, Modal, Form, Input, Select, Switch, Space, Popconfirm, Tag } from 'antd';
import { PlusOutlined } from '@ant-design/icons';
import { useInjectionRules, useCreateInjectionRule, useDeleteInjectionRule, useToggleInjectionRule } from '../../application/useLLMGateway';
import type { InjectionRule } from '../../infrastructure/llm-gateway.api';
const { TextArea } = Input;
export function InjectionRulesTab() {
const { data, isLoading } = useInjectionRules();
const createMutation = useCreateInjectionRule();
const deleteMutation = useDeleteInjectionRule();
const toggleMutation = useToggleInjectionRule();
const [createOpen, setCreateOpen] = useState(false);
const [form] = Form.useForm();
const handleCreate = async () => {
const values = await form.validateFields();
await createMutation.mutateAsync(values);
setCreateOpen(false);
form.resetFields();
};
const columns = [
{ title: '名称', dataIndex: 'name', key: 'name' },
{
title: '位置',
dataIndex: 'position',
key: 'position',
render: (v: string) => <Tag color={v === 'prepend' ? 'blue' : 'green'}>{v === 'prepend' ? '前置' : '追加'}</Tag>,
},
{
title: '内容预览',
dataIndex: 'content',
key: 'content',
ellipsis: true,
width: 300,
},
{
title: '匹配模型',
dataIndex: 'matchModels',
key: 'matchModels',
render: (v: string[]) => v?.includes('*') ? <Tag></Tag> : v?.map((m: string) => <Tag key={m}>{m}</Tag>),
},
{
title: '状态',
dataIndex: 'enabled',
key: 'enabled',
render: (v: boolean, record: InjectionRule) => (
<Switch checked={v} size="small" onChange={() => toggleMutation.mutate(record.id)} />
),
},
{
title: '操作',
key: 'actions',
render: (_: any, record: InjectionRule) => (
<Popconfirm title="确认删除?" onConfirm={() => deleteMutation.mutate(record.id)}>
<Button type="link" danger size="small"></Button>
</Popconfirm>
),
},
];
return (
<>
<div style={{ display: 'flex', justifyContent: 'flex-end', marginBottom: 16 }}>
<Button type="primary" icon={<PlusOutlined />} onClick={() => setCreateOpen(true)}>
</Button>
</div>
<Table columns={columns} dataSource={data?.items} rowKey="id" loading={isLoading} size="small" pagination={false} />
<Modal
title="创建注入规则"
open={createOpen}
onOk={handleCreate}
onCancel={() => setCreateOpen(false)}
confirmLoading={createMutation.isPending}
width={600}
>
<Form form={form} layout="vertical">
<Form.Item name="name" label="规则名称" rules={[{ required: true }]}>
<Input placeholder="例如: 合规声明" />
</Form.Item>
<Form.Item name="position" label="注入位置" initialValue="append">
<Select options={[{ label: '追加到 system prompt 末尾', value: 'append' }, { label: '前置到 system prompt 开头', value: 'prepend' }]} />
</Form.Item>
<Form.Item name="content" label="注入内容" rules={[{ required: true }]}>
<TextArea rows={6} placeholder="输入要注入到 system prompt 中的监管内容..." />
</Form.Item>
<Form.Item name="description" label="描述">
<Input placeholder="规则说明(可选)" />
</Form.Item>
</Form>
</Modal>
</>
);
}

View File

@ -0,0 +1,55 @@
import { Table, Tag } from 'antd';
import { useUsageLogs } from '../../application/useLLMGateway';
export function UsageDashboardTab() {
const { data, isLoading } = useUsageLogs();
const columns = [
{
title: '时间',
dataIndex: 'createdAt',
key: 'createdAt',
render: (v: string) => new Date(v).toLocaleString(),
width: 180,
},
{
title: '提供商',
dataIndex: 'provider',
key: 'provider',
render: (v: string) => <Tag color={v === 'anthropic' ? 'purple' : 'green'}>{v}</Tag>,
},
{ title: '模型', dataIndex: 'model', key: 'model', ellipsis: true },
{ title: '输入 Tokens', dataIndex: 'inputTokens', key: 'inputTokens' },
{ title: '输出 Tokens', dataIndex: 'outputTokens', key: 'outputTokens' },
{ title: '总 Tokens', dataIndex: 'totalTokens', key: 'totalTokens' },
{
title: '费用 (USD)',
dataIndex: 'costUsd',
key: 'costUsd',
render: (v: number | null) => v !== null ? `$${Number(v).toFixed(6)}` : '-',
},
{
title: '耗时',
dataIndex: 'durationMs',
key: 'durationMs',
render: (v: number) => `${(v / 1000).toFixed(1)}s`,
},
{
title: '状态码',
dataIndex: 'statusCode',
key: 'statusCode',
render: (v: number) => <Tag color={v === 200 ? 'green' : 'red'}>{v}</Tag>,
},
];
return (
<Table
columns={columns}
dataSource={data?.items}
rowKey="id"
loading={isLoading}
size="small"
pagination={{ pageSize: 20, showSizeChanger: false }}
/>
);
}

View File

@ -0,0 +1,103 @@
import { useState } from 'react';
import { Typography, Tabs, Card, Row, Col, Statistic, Spin } from 'antd';
import { KeyOutlined, EditOutlined, SafetyOutlined, BarChartOutlined, FileSearchOutlined } from '@ant-design/icons';
import { useDashboard } from '../../application/useLLMGateway';
import { ApiKeysTab } from '../components/ApiKeysTab';
import { InjectionRulesTab } from '../components/InjectionRulesTab';
import { ContentRulesTab } from '../components/ContentRulesTab';
import { UsageDashboardTab } from '../components/UsageDashboardTab';
import { AuditLogsTab } from '../components/AuditLogsTab';
const { Title } = Typography;
export function LLMGatewayPage() {
const [activeTab, setActiveTab] = useState('keys');
const { data: dashboard, isLoading: dashLoading } = useDashboard();
return (
<div style={{ padding: 24 }}>
<Title level={4} style={{ marginBottom: 16 }}>LLM </Title>
{/* Dashboard Summary Cards */}
<Row gutter={[16, 16]} style={{ marginBottom: 16 }}>
<Col xs={12} sm={6}>
<Card size="small">
<Statistic
title="活跃 Key"
value={dashLoading ? '-' : `${dashboard?.keys.active || 0} / ${dashboard?.keys.total || 0}`}
prefix={<KeyOutlined />}
/>
</Card>
</Col>
<Col xs={12} sm={6}>
<Card size="small">
<Statistic
title="今日请求"
value={dashboard?.today.requests || 0}
loading={dashLoading}
/>
</Card>
</Col>
<Col xs={12} sm={6}>
<Card size="small">
<Statistic
title="今日 Tokens"
value={dashboard?.today.tokens || 0}
loading={dashLoading}
formatter={(val) => {
const n = Number(val);
return n >= 1000000 ? `${(n / 1000000).toFixed(1)}M` : n >= 1000 ? `${(n / 1000).toFixed(1)}K` : String(n);
}}
/>
</Card>
</Col>
<Col xs={12} sm={6}>
<Card size="small">
<Statistic
title="今日费用"
value={dashboard?.today.cost || 0}
loading={dashLoading}
precision={4}
prefix="$"
/>
</Card>
</Col>
</Row>
{/* Tabs */}
<Card>
<Tabs
activeKey={activeTab}
onChange={setActiveTab}
items={[
{
key: 'keys',
label: <span><KeyOutlined /> API Key</span>,
children: <ApiKeysTab />,
},
{
key: 'injection',
label: <span><EditOutlined /> </span>,
children: <InjectionRulesTab />,
},
{
key: 'content',
label: <span><SafetyOutlined /> </span>,
children: <ContentRulesTab />,
},
{
key: 'usage',
label: <span><BarChartOutlined /> </span>,
children: <UsageDashboardTab />,
},
{
key: 'audit',
label: <span><FileSearchOutlined /> </span>,
children: <AuditLogsTab />,
},
]}
/>
</Card>
</div>
);
}

View File

@ -22,6 +22,7 @@ import {
ExperimentOutlined,
FormOutlined,
EyeOutlined,
GatewayOutlined,
} from '@ant-design/icons';
import { useAuth } from '../hooks/useAuth';
@ -59,6 +60,11 @@ const menuItems: MenuProps['items'] = [
icon: <EyeOutlined />,
label: '系统总监',
},
{
key: '/llm-gateway',
icon: <GatewayOutlined />,
label: 'LLM 网关',
},
{
key: 'analytics-group',
icon: <BarChartOutlined />,

View File

@ -0,0 +1,605 @@
/**
* Admin Gateway Controller
* LLM API API Key / / / /
*
* keys, injection-rules, content-rules, usage, audit-logs, dashboard
* :id
*/
import {
Controller,
Get,
Post,
Put,
Delete,
Param,
Body,
Query,
Headers,
HttpCode,
HttpStatus,
UnauthorizedException,
BadRequestException,
} from '@nestjs/common';
import { InjectRepository } from '@nestjs/typeorm';
import { Repository, IsNull, Between, MoreThanOrEqual } from 'typeorm';
import * as jwt from 'jsonwebtoken';
import { v4 as uuidv4 } from 'uuid';
import { createHash, randomBytes } from 'crypto';
import { GatewayApiKeyORM } from '../../infrastructure/database/postgres/entities/gateway-api-key.orm';
import { GatewayInjectionRuleORM } from '../../infrastructure/database/postgres/entities/gateway-injection-rule.orm';
import { GatewayContentRuleORM } from '../../infrastructure/database/postgres/entities/gateway-content-rule.orm';
import { GatewayUsageLogORM } from '../../infrastructure/database/postgres/entities/gateway-usage-log.orm';
import { GatewayAuditLogORM } from '../../infrastructure/database/postgres/entities/gateway-audit-log.orm';
interface AdminPayload {
id: string;
username: string;
role: string;
tenantId?: string;
}
// ═══════════════════════════════════════════════════════════════
// API Key 管理
// ═══════════════════════════════════════════════════════════════
@Controller('conversations/admin/gateway/keys')
export class AdminGatewayKeysController {
constructor(
@InjectRepository(GatewayApiKeyORM)
private readonly repo: Repository<GatewayApiKeyORM>,
) {}
private verifyAdmin(authorization: string): AdminPayload {
const token = authorization?.replace('Bearer ', '');
if (!token) throw new UnauthorizedException('Missing token');
try {
return jwt.verify(token, process.env.JWT_SECRET || 'your-jwt-secret-key') as AdminPayload;
} catch {
throw new UnauthorizedException('Invalid token');
}
}
@Get()
async listKeys(@Headers('authorization') auth: string) {
const admin = this.verifyAdmin(auth);
const tenantId = admin.tenantId || null;
const keys = await this.repo.find({
where: { tenantId: tenantId ?? IsNull() },
order: { createdAt: 'DESC' },
});
return {
success: true,
data: {
items: keys,
total: keys.length,
enabledCount: keys.filter((k) => k.enabled).length,
},
};
}
@Post()
@HttpCode(HttpStatus.CREATED)
async createKey(
@Headers('authorization') auth: string,
@Body() dto: {
name: string;
owner?: string;
permissions?: { allowedModels?: string[]; allowStreaming?: boolean; allowTools?: boolean };
rateLimitRpm?: number;
rateLimitTpd?: number;
monthlyBudget?: number;
expiresAt?: string;
},
) {
const admin = this.verifyAdmin(auth);
const tenantId = admin.tenantId || null;
if (!dto.name?.trim()) {
throw new BadRequestException('name is required');
}
// Generate a unique API key: sk-gw-<32 random hex chars>
const rawKey = `sk-gw-${randomBytes(16).toString('hex')}`;
const keyHash = createHash('sha256').update(rawKey).digest('hex');
const keyPrefix = rawKey.slice(0, 12);
const entity = this.repo.create({
id: uuidv4(),
tenantId,
keyHash,
keyPrefix,
name: dto.name.trim(),
owner: dto.owner?.trim() || '',
permissions: {
allowedModels: dto.permissions?.allowedModels || ['*'],
allowStreaming: dto.permissions?.allowStreaming !== false,
allowTools: dto.permissions?.allowTools !== false,
},
rateLimitRpm: dto.rateLimitRpm ?? 60,
rateLimitTpd: dto.rateLimitTpd ?? 1000000,
monthlyBudget: dto.monthlyBudget ?? null,
enabled: true,
expiresAt: dto.expiresAt ? new Date(dto.expiresAt) : null,
createdBy: admin.id,
});
const saved = await this.repo.save(entity);
// Return the raw key only on creation (never stored, only the hash)
return {
success: true,
data: {
...saved,
rawKey, // ⚠️ 仅此一次返回完整 Key
},
};
}
@Put(':id')
async updateKey(
@Headers('authorization') auth: string,
@Param('id') id: string,
@Body() dto: {
name?: string;
owner?: string;
permissions?: { allowedModels?: string[]; allowStreaming?: boolean; allowTools?: boolean };
rateLimitRpm?: number;
rateLimitTpd?: number;
monthlyBudget?: number | null;
expiresAt?: string | null;
enabled?: boolean;
},
) {
this.verifyAdmin(auth);
const key = await this.repo.findOne({ where: { id } });
if (!key) return { success: false, error: 'Key not found' };
if (dto.name !== undefined) key.name = dto.name.trim();
if (dto.owner !== undefined) key.owner = dto.owner.trim();
if (dto.permissions) key.permissions = { ...key.permissions, ...dto.permissions };
if (dto.rateLimitRpm !== undefined) key.rateLimitRpm = dto.rateLimitRpm;
if (dto.rateLimitTpd !== undefined) key.rateLimitTpd = dto.rateLimitTpd;
if (dto.monthlyBudget !== undefined) key.monthlyBudget = dto.monthlyBudget;
if (dto.expiresAt !== undefined) key.expiresAt = dto.expiresAt ? new Date(dto.expiresAt) : null;
if (dto.enabled !== undefined) key.enabled = dto.enabled;
const updated = await this.repo.save(key);
return { success: true, data: updated };
}
@Delete(':id')
async deleteKey(
@Headers('authorization') auth: string,
@Param('id') id: string,
) {
this.verifyAdmin(auth);
const key = await this.repo.findOne({ where: { id } });
if (!key) return { success: false, error: 'Key not found' };
await this.repo.delete(id);
return { success: true };
}
@Post(':id/toggle')
@HttpCode(HttpStatus.OK)
async toggleKey(
@Headers('authorization') auth: string,
@Param('id') id: string,
) {
this.verifyAdmin(auth);
const key = await this.repo.findOne({ where: { id } });
if (!key) return { success: false, error: 'Key not found' };
key.enabled = !key.enabled;
const updated = await this.repo.save(key);
return { success: true, data: updated };
}
}
// ═══════════════════════════════════════════════════════════════
// 注入规则管理
// ═══════════════════════════════════════════════════════════════
@Controller('conversations/admin/gateway/injection-rules')
export class AdminGatewayInjectionRulesController {
constructor(
@InjectRepository(GatewayInjectionRuleORM)
private readonly repo: Repository<GatewayInjectionRuleORM>,
) {}
private verifyAdmin(authorization: string): AdminPayload {
const token = authorization?.replace('Bearer ', '');
if (!token) throw new UnauthorizedException('Missing token');
try {
return jwt.verify(token, process.env.JWT_SECRET || 'your-jwt-secret-key') as AdminPayload;
} catch {
throw new UnauthorizedException('Invalid token');
}
}
@Get()
async listRules(@Headers('authorization') auth: string) {
const admin = this.verifyAdmin(auth);
const tenantId = admin.tenantId || null;
const rules = await this.repo.find({
where: { tenantId: tenantId ?? IsNull() },
order: { priority: 'ASC', createdAt: 'ASC' },
});
return {
success: true,
data: {
items: rules,
total: rules.length,
enabledCount: rules.filter((r) => r.enabled).length,
},
};
}
@Post()
@HttpCode(HttpStatus.CREATED)
async createRule(
@Headers('authorization') auth: string,
@Body() dto: {
name: string;
description?: string;
position?: 'prepend' | 'append';
content: string;
matchModels?: string[];
matchKeyIds?: string[];
priority?: number;
},
) {
const admin = this.verifyAdmin(auth);
const tenantId = admin.tenantId || null;
if (!dto.name?.trim()) throw new BadRequestException('name is required');
if (!dto.content?.trim()) throw new BadRequestException('content is required');
const rule = this.repo.create({
id: uuidv4(),
tenantId,
name: dto.name.trim(),
description: dto.description || null,
position: dto.position || 'append',
content: dto.content.trim(),
matchModels: dto.matchModels || ['*'],
matchKeyIds: dto.matchKeyIds || null,
priority: dto.priority ?? 0,
enabled: true,
createdBy: admin.id,
});
const saved = await this.repo.save(rule);
return { success: true, data: saved };
}
@Put(':id')
async updateRule(
@Headers('authorization') auth: string,
@Param('id') id: string,
@Body() dto: {
name?: string;
description?: string | null;
position?: 'prepend' | 'append';
content?: string;
matchModels?: string[];
matchKeyIds?: string[] | null;
priority?: number;
enabled?: boolean;
},
) {
this.verifyAdmin(auth);
const rule = await this.repo.findOne({ where: { id } });
if (!rule) return { success: false, error: 'Rule not found' };
if (dto.name !== undefined) rule.name = dto.name.trim();
if (dto.description !== undefined) rule.description = dto.description;
if (dto.position !== undefined) rule.position = dto.position;
if (dto.content !== undefined) rule.content = dto.content.trim();
if (dto.matchModels !== undefined) rule.matchModels = dto.matchModels;
if (dto.matchKeyIds !== undefined) rule.matchKeyIds = dto.matchKeyIds;
if (dto.priority !== undefined) rule.priority = dto.priority;
if (dto.enabled !== undefined) rule.enabled = dto.enabled;
const updated = await this.repo.save(rule);
return { success: true, data: updated };
}
@Delete(':id')
async deleteRule(@Headers('authorization') auth: string, @Param('id') id: string) {
this.verifyAdmin(auth);
const rule = await this.repo.findOne({ where: { id } });
if (!rule) return { success: false, error: 'Rule not found' };
await this.repo.delete(id);
return { success: true };
}
@Post(':id/toggle')
@HttpCode(HttpStatus.OK)
async toggleRule(@Headers('authorization') auth: string, @Param('id') id: string) {
this.verifyAdmin(auth);
const rule = await this.repo.findOne({ where: { id } });
if (!rule) return { success: false, error: 'Rule not found' };
rule.enabled = !rule.enabled;
const updated = await this.repo.save(rule);
return { success: true, data: updated };
}
}
// ═══════════════════════════════════════════════════════════════
// 内容审查规则管理
// ═══════════════════════════════════════════════════════════════
@Controller('conversations/admin/gateway/content-rules')
export class AdminGatewayContentRulesController {
constructor(
@InjectRepository(GatewayContentRuleORM)
private readonly repo: Repository<GatewayContentRuleORM>,
) {}
private verifyAdmin(authorization: string): AdminPayload {
const token = authorization?.replace('Bearer ', '');
if (!token) throw new UnauthorizedException('Missing token');
try {
return jwt.verify(token, process.env.JWT_SECRET || 'your-jwt-secret-key') as AdminPayload;
} catch {
throw new UnauthorizedException('Invalid token');
}
}
@Get()
async listRules(@Headers('authorization') auth: string) {
const admin = this.verifyAdmin(auth);
const tenantId = admin.tenantId || null;
const rules = await this.repo.find({
where: { tenantId: tenantId ?? IsNull() },
order: { priority: 'ASC', createdAt: 'ASC' },
});
return {
success: true,
data: {
items: rules,
total: rules.length,
enabledCount: rules.filter((r) => r.enabled).length,
},
};
}
@Post()
@HttpCode(HttpStatus.CREATED)
async createRule(
@Headers('authorization') auth: string,
@Body() dto: {
name: string;
type?: 'keyword' | 'regex';
pattern: string;
action?: 'block' | 'warn' | 'log';
rejectMessage?: string;
priority?: number;
},
) {
const admin = this.verifyAdmin(auth);
const tenantId = admin.tenantId || null;
if (!dto.name?.trim()) throw new BadRequestException('name is required');
if (!dto.pattern?.trim()) throw new BadRequestException('pattern is required');
// Validate regex if type is regex
if (dto.type === 'regex') {
try {
new RegExp(dto.pattern);
} catch {
throw new BadRequestException('Invalid regex pattern');
}
}
const rule = this.repo.create({
id: uuidv4(),
tenantId,
name: dto.name.trim(),
type: dto.type || 'keyword',
pattern: dto.pattern.trim(),
action: dto.action || 'block',
rejectMessage: dto.rejectMessage || null,
priority: dto.priority ?? 0,
enabled: true,
createdBy: admin.id,
});
const saved = await this.repo.save(rule);
return { success: true, data: saved };
}
@Put(':id')
async updateRule(
@Headers('authorization') auth: string,
@Param('id') id: string,
@Body() dto: {
name?: string;
type?: 'keyword' | 'regex';
pattern?: string;
action?: 'block' | 'warn' | 'log';
rejectMessage?: string | null;
priority?: number;
enabled?: boolean;
},
) {
this.verifyAdmin(auth);
const rule = await this.repo.findOne({ where: { id } });
if (!rule) return { success: false, error: 'Rule not found' };
if (dto.name !== undefined) rule.name = dto.name.trim();
if (dto.type !== undefined) rule.type = dto.type;
if (dto.pattern !== undefined) {
if (dto.type === 'regex' || rule.type === 'regex') {
try { new RegExp(dto.pattern); } catch { throw new BadRequestException('Invalid regex pattern'); }
}
rule.pattern = dto.pattern.trim();
}
if (dto.action !== undefined) rule.action = dto.action;
if (dto.rejectMessage !== undefined) rule.rejectMessage = dto.rejectMessage;
if (dto.priority !== undefined) rule.priority = dto.priority;
if (dto.enabled !== undefined) rule.enabled = dto.enabled;
const updated = await this.repo.save(rule);
return { success: true, data: updated };
}
@Delete(':id')
async deleteRule(@Headers('authorization') auth: string, @Param('id') id: string) {
this.verifyAdmin(auth);
const rule = await this.repo.findOne({ where: { id } });
if (!rule) return { success: false, error: 'Rule not found' };
await this.repo.delete(id);
return { success: true };
}
@Post(':id/toggle')
@HttpCode(HttpStatus.OK)
async toggleRule(@Headers('authorization') auth: string, @Param('id') id: string) {
this.verifyAdmin(auth);
const rule = await this.repo.findOne({ where: { id } });
if (!rule) return { success: false, error: 'Rule not found' };
rule.enabled = !rule.enabled;
const updated = await this.repo.save(rule);
return { success: true, data: updated };
}
}
// ═══════════════════════════════════════════════════════════════
// 用量统计 + 审计日志 + 仪表盘
// ═══════════════════════════════════════════════════════════════
@Controller('conversations/admin/gateway')
export class AdminGatewayDashboardController {
constructor(
@InjectRepository(GatewayUsageLogORM)
private readonly usageRepo: Repository<GatewayUsageLogORM>,
@InjectRepository(GatewayAuditLogORM)
private readonly auditRepo: Repository<GatewayAuditLogORM>,
@InjectRepository(GatewayApiKeyORM)
private readonly keyRepo: Repository<GatewayApiKeyORM>,
) {}
private verifyAdmin(authorization: string): AdminPayload {
const token = authorization?.replace('Bearer ', '');
if (!token) throw new UnauthorizedException('Missing token');
try {
return jwt.verify(token, process.env.JWT_SECRET || 'your-jwt-secret-key') as AdminPayload;
} catch {
throw new UnauthorizedException('Invalid token');
}
}
@Get('dashboard')
async getDashboard(@Headers('authorization') auth: string) {
this.verifyAdmin(auth);
const now = new Date();
const todayStart = new Date(now.getFullYear(), now.getMonth(), now.getDate());
const monthStart = new Date(now.getFullYear(), now.getMonth(), 1);
// Parallel queries
const [totalKeys, activeKeys, todayUsage, monthUsage, recentAudits] = await Promise.all([
this.keyRepo.count(),
this.keyRepo.count({ where: { enabled: true } }),
this.usageRepo
.createQueryBuilder('u')
.select('COUNT(*)', 'requestCount')
.addSelect('COALESCE(SUM(u.total_tokens), 0)', 'totalTokens')
.addSelect('COALESCE(SUM(u.cost_usd), 0)', 'totalCost')
.where('u.created_at >= :start', { start: todayStart })
.getRawOne(),
this.usageRepo
.createQueryBuilder('u')
.select('COUNT(*)', 'requestCount')
.addSelect('COALESCE(SUM(u.total_tokens), 0)', 'totalTokens')
.addSelect('COALESCE(SUM(u.cost_usd), 0)', 'totalCost')
.where('u.created_at >= :start', { start: monthStart })
.getRawOne(),
this.auditRepo.count({
where: { contentFiltered: true, createdAt: MoreThanOrEqual(todayStart) },
}),
]);
return {
success: true,
data: {
keys: { total: totalKeys, active: activeKeys },
today: {
requests: parseInt(todayUsage?.requestCount || '0'),
tokens: parseInt(todayUsage?.totalTokens || '0'),
cost: parseFloat(todayUsage?.totalCost || '0'),
filtered: recentAudits,
},
month: {
requests: parseInt(monthUsage?.requestCount || '0'),
tokens: parseInt(monthUsage?.totalTokens || '0'),
cost: parseFloat(monthUsage?.totalCost || '0'),
},
},
};
}
@Get('usage')
async getUsage(
@Headers('authorization') auth: string,
@Query('keyId') keyId?: string,
@Query('startDate') startDate?: string,
@Query('endDate') endDate?: string,
@Query('limit') limit?: string,
) {
this.verifyAdmin(auth);
const qb = this.usageRepo.createQueryBuilder('u');
if (keyId) qb.andWhere('u.api_key_id = :keyId', { keyId });
if (startDate) qb.andWhere('u.created_at >= :startDate', { startDate: new Date(startDate) });
if (endDate) qb.andWhere('u.created_at <= :endDate', { endDate: new Date(endDate) });
qb.orderBy('u.created_at', 'DESC');
qb.take(parseInt(limit || '100'));
const [items, total] = await qb.getManyAndCount();
return {
success: true,
data: { items, total },
};
}
@Get('audit-logs')
async getAuditLogs(
@Headers('authorization') auth: string,
@Query('keyId') keyId?: string,
@Query('filtered') filtered?: string,
@Query('startDate') startDate?: string,
@Query('endDate') endDate?: string,
@Query('limit') limit?: string,
) {
this.verifyAdmin(auth);
const qb = this.auditRepo.createQueryBuilder('a');
if (keyId) qb.andWhere('a.api_key_id = :keyId', { keyId });
if (filtered === 'true') qb.andWhere('a.content_filtered = true');
if (startDate) qb.andWhere('a.created_at >= :startDate', { startDate: new Date(startDate) });
if (endDate) qb.andWhere('a.created_at <= :endDate', { endDate: new Date(endDate) });
qb.orderBy('a.created_at', 'DESC');
qb.take(parseInt(limit || '100'));
const [items, total] = await qb.getManyAndCount();
return {
success: true,
data: { items, total },
};
}
}

View File

@ -6,6 +6,11 @@ import { TokenUsageORM } from '../infrastructure/database/postgres/entities/toke
import { AgentExecutionORM } from '../infrastructure/database/postgres/entities/agent-execution.orm';
import { AssessmentDirectiveORM } from '../infrastructure/database/postgres/entities/assessment-directive.orm';
import { CollectionDirectiveORM } from '../infrastructure/database/postgres/entities/collection-directive.orm';
import { GatewayApiKeyORM } from '../infrastructure/database/postgres/entities/gateway-api-key.orm';
import { GatewayInjectionRuleORM } from '../infrastructure/database/postgres/entities/gateway-injection-rule.orm';
import { GatewayContentRuleORM } from '../infrastructure/database/postgres/entities/gateway-content-rule.orm';
import { GatewayUsageLogORM } from '../infrastructure/database/postgres/entities/gateway-usage-log.orm';
import { GatewayAuditLogORM } from '../infrastructure/database/postgres/entities/gateway-audit-log.orm';
import { ConversationPostgresRepository } from '../adapters/outbound/persistence/conversation-postgres.repository';
import { MessagePostgresRepository } from '../adapters/outbound/persistence/message-postgres.repository';
import { TokenUsagePostgresRepository } from '../adapters/outbound/persistence/token-usage-postgres.repository';
@ -22,12 +27,18 @@ import { AdminObservabilityController } from '../adapters/inbound/admin-observab
import { AdminAssessmentDirectiveController } from '../adapters/inbound/admin-assessment-directive.controller';
import { AdminCollectionDirectiveController } from '../adapters/inbound/admin-collection-directive.controller';
import { AdminSupervisorController } from '../adapters/inbound/admin-supervisor.controller';
import {
AdminGatewayKeysController,
AdminGatewayInjectionRulesController,
AdminGatewayContentRulesController,
AdminGatewayDashboardController,
} from '../adapters/inbound/admin-gateway.controller';
import { ConversationGateway } from '../adapters/inbound/conversation.gateway';
import { PaymentModule } from '../infrastructure/payment/payment.module';
@Module({
imports: [TypeOrmModule.forFeature([ConversationORM, MessageORM, TokenUsageORM, AgentExecutionORM, AssessmentDirectiveORM, CollectionDirectiveORM]), PaymentModule],
controllers: [ConversationController, InternalConversationController, AdminMcpController, AdminEvaluationRuleController, AdminAssessmentDirectiveController, AdminCollectionDirectiveController, AdminSupervisorController, AdminConversationController, AdminObservabilityController],
imports: [TypeOrmModule.forFeature([ConversationORM, MessageORM, TokenUsageORM, AgentExecutionORM, AssessmentDirectiveORM, CollectionDirectiveORM, GatewayApiKeyORM, GatewayInjectionRuleORM, GatewayContentRuleORM, GatewayUsageLogORM, GatewayAuditLogORM]), PaymentModule],
controllers: [ConversationController, InternalConversationController, AdminMcpController, AdminEvaluationRuleController, AdminAssessmentDirectiveController, AdminCollectionDirectiveController, AdminSupervisorController, AdminConversationController, AdminObservabilityController, AdminGatewayKeysController, AdminGatewayInjectionRulesController, AdminGatewayContentRulesController, AdminGatewayDashboardController],
providers: [
ConversationService,
ConversationGateway,

View File

@ -0,0 +1,60 @@
import {
Entity,
PrimaryGeneratedColumn,
Column,
Index,
} from 'typeorm';
@Entity('gateway_api_keys')
@Index('idx_gateway_api_keys_tenant', ['tenantId'])
@Index('idx_gateway_api_keys_enabled', ['enabled'])
export class GatewayApiKeyORM {
@PrimaryGeneratedColumn('uuid')
id: string;
@Column({ name: 'tenant_id', type: 'uuid', nullable: true })
tenantId: string | null;
@Column({ name: 'key_hash', type: 'varchar', length: 64 })
keyHash: string;
@Column({ name: 'key_prefix', type: 'varchar', length: 12 })
keyPrefix: string;
@Column({ type: 'varchar', length: 100 })
name: string;
@Column({ type: 'varchar', length: 200, default: '' })
owner: string;
@Column({ type: 'jsonb', default: () => `'{"allowedModels": ["*"], "allowStreaming": true, "allowTools": true}'` })
permissions: {
allowedModels: string[];
allowStreaming: boolean;
allowTools: boolean;
};
@Column({ name: 'rate_limit_rpm', type: 'int', default: 60 })
rateLimitRpm: number;
@Column({ name: 'rate_limit_tpd', type: 'int', default: 1000000 })
rateLimitTpd: number;
@Column({ name: 'monthly_budget', type: 'decimal', precision: 10, scale: 2, nullable: true })
monthlyBudget: number | null;
@Column({ type: 'boolean', default: true })
enabled: boolean;
@Column({ name: 'expires_at', type: 'timestamp', nullable: true })
expiresAt: Date | null;
@Column({ name: 'last_used_at', type: 'timestamp', nullable: true })
lastUsedAt: Date | null;
@Column({ name: 'created_by', type: 'uuid', nullable: true })
createdBy: string | null;
@Column({ name: 'created_at', type: 'timestamp', default: () => 'NOW()' })
createdAt: Date;
}

View File

@ -0,0 +1,47 @@
import {
Entity,
PrimaryGeneratedColumn,
Column,
Index,
} from 'typeorm';
@Entity('gateway_audit_logs')
@Index('idx_gateway_audit_logs_key', ['apiKeyId'])
@Index('idx_gateway_audit_logs_created', ['createdAt'])
export class GatewayAuditLogORM {
@PrimaryGeneratedColumn('uuid')
id: string;
@Column({ name: 'api_key_id', type: 'uuid' })
apiKeyId: string;
@Column({ name: 'request_method', type: 'varchar', length: 10 })
requestMethod: string;
@Column({ name: 'request_path', type: 'varchar', length: 200 })
requestPath: string;
@Column({ name: 'request_model', type: 'varchar', length: 100, nullable: true })
requestModel: string | null;
@Column({ name: 'request_ip', type: 'varchar', length: 50, default: '' })
requestIp: string;
@Column({ name: 'content_filtered', type: 'boolean', default: false })
contentFiltered: boolean;
@Column({ name: 'filter_rule_id', type: 'uuid', nullable: true })
filterRuleId: string | null;
@Column({ name: 'injection_applied', type: 'boolean', default: false })
injectionApplied: boolean;
@Column({ name: 'response_status', type: 'int', default: 200 })
responseStatus: number;
@Column({ name: 'duration_ms', type: 'int', default: 0 })
durationMs: number;
@Column({ name: 'created_at', type: 'timestamp', default: () => 'NOW()' })
createdAt: Date;
}

View File

@ -0,0 +1,49 @@
import {
Entity,
PrimaryGeneratedColumn,
Column,
CreateDateColumn,
UpdateDateColumn,
Index,
} from 'typeorm';
@Entity('gateway_content_rules')
@Index('idx_gateway_content_rules_tenant', ['tenantId'])
@Index('idx_gateway_content_rules_enabled', ['tenantId', 'enabled'])
export class GatewayContentRuleORM {
@PrimaryGeneratedColumn('uuid')
id: string;
@Column({ name: 'tenant_id', type: 'uuid', nullable: true })
tenantId: string | null;
@Column({ type: 'varchar', length: 100 })
name: string;
@Column({ type: 'varchar', length: 20, default: 'keyword' })
type: 'keyword' | 'regex';
@Column({ type: 'text' })
pattern: string;
@Column({ type: 'varchar', length: 20, default: 'block' })
action: 'block' | 'warn' | 'log';
@Column({ name: 'reject_message', type: 'text', nullable: true })
rejectMessage: string | null;
@Column({ type: 'int', default: 0 })
priority: number;
@Column({ type: 'boolean', default: true })
enabled: boolean;
@Column({ name: 'created_by', type: 'uuid', nullable: true })
createdBy: string | null;
@CreateDateColumn({ name: 'created_at' })
createdAt: Date;
@UpdateDateColumn({ name: 'updated_at' })
updatedAt: Date;
}

View File

@ -0,0 +1,52 @@
import {
Entity,
PrimaryGeneratedColumn,
Column,
CreateDateColumn,
UpdateDateColumn,
Index,
} from 'typeorm';
@Entity('gateway_injection_rules')
@Index('idx_gateway_injection_rules_tenant', ['tenantId'])
@Index('idx_gateway_injection_rules_enabled', ['tenantId', 'enabled'])
export class GatewayInjectionRuleORM {
@PrimaryGeneratedColumn('uuid')
id: string;
@Column({ name: 'tenant_id', type: 'uuid', nullable: true })
tenantId: string | null;
@Column({ type: 'varchar', length: 100 })
name: string;
@Column({ type: 'text', nullable: true })
description: string | null;
@Column({ type: 'varchar', length: 10, default: 'append' })
position: 'prepend' | 'append';
@Column({ type: 'text' })
content: string;
@Column({ name: 'match_models', type: 'jsonb', default: () => `'["*"]'` })
matchModels: string[];
@Column({ name: 'match_key_ids', type: 'jsonb', nullable: true })
matchKeyIds: string[] | null;
@Column({ type: 'int', default: 0 })
priority: number;
@Column({ type: 'boolean', default: true })
enabled: boolean;
@Column({ name: 'created_by', type: 'uuid', nullable: true })
createdBy: string | null;
@CreateDateColumn({ name: 'created_at' })
createdAt: Date;
@UpdateDateColumn({ name: 'updated_at' })
updatedAt: Date;
}

View File

@ -0,0 +1,44 @@
import {
Entity,
PrimaryGeneratedColumn,
Column,
Index,
} from 'typeorm';
@Entity('gateway_usage_logs')
@Index('idx_gateway_usage_logs_key', ['apiKeyId'])
@Index('idx_gateway_usage_logs_created', ['createdAt'])
export class GatewayUsageLogORM {
@PrimaryGeneratedColumn('uuid')
id: string;
@Column({ name: 'api_key_id', type: 'uuid' })
apiKeyId: string;
@Column({ type: 'varchar', length: 100 })
model: string;
@Column({ type: 'varchar', length: 20 })
provider: string;
@Column({ name: 'input_tokens', type: 'int', default: 0 })
inputTokens: number;
@Column({ name: 'output_tokens', type: 'int', default: 0 })
outputTokens: number;
@Column({ name: 'total_tokens', type: 'int', default: 0 })
totalTokens: number;
@Column({ name: 'cost_usd', type: 'decimal', precision: 10, scale: 6, nullable: true })
costUsd: number | null;
@Column({ name: 'duration_ms', type: 'int', default: 0 })
durationMs: number;
@Column({ name: 'status_code', type: 'int', default: 200 })
statusCode: number;
@Column({ name: 'created_at', type: 'timestamp', default: () => 'NOW()' })
createdAt: Date;
}

View File

@ -0,0 +1,57 @@
# ===========================================
# iConsulting LLM Gateway Dockerfile
# ===========================================
# 构建阶段
FROM node:20-alpine AS builder
WORKDIR /app
# 安装 pnpm
RUN corepack enable && corepack prepare pnpm@latest --activate
# 复制 workspace 配置
COPY pnpm-workspace.yaml package.json pnpm-lock.yaml ./
COPY packages/services/llm-gateway/package.json ./packages/services/llm-gateway/
# 安装依赖
RUN pnpm install --frozen-lockfile --filter @iconsulting/llm-gateway
# 复制源代码
COPY packages/services/llm-gateway ./packages/services/llm-gateway
# 构建服务
RUN cd packages/services/llm-gateway && npx tsc
# 运行阶段
FROM node:20-alpine AS runner
WORKDIR /app
# 创建非 root 用户
RUN addgroup --system --gid 1001 nodejs && \
adduser --system --uid 1001 gateway
# 复制构建产物和依赖配置
COPY --from=builder /app/packages/services/llm-gateway/dist ./dist
COPY --from=builder /app/packages/services/llm-gateway/package.json ./
# 安装生产依赖
RUN npm install --omit=dev --ignore-scripts
# 设置环境变量
ENV NODE_ENV=production
ENV PORT=3008
# 切换用户
USER gateway
# 暴露端口
EXPOSE 3008
# 健康检查
HEALTHCHECK --interval=30s --timeout=10s --start-period=10s --retries=3 \
CMD wget --no-verbose --tries=1 --spider http://localhost:3008/health || exit 1
# 启动服务
CMD ["node", "dist/main.js"]

View File

@ -0,0 +1,26 @@
{
"name": "@iconsulting/llm-gateway",
"version": "1.0.0",
"description": "LLM API Gateway - Anthropic/OpenAI compatible proxy with regulatory injection and content filtering",
"main": "dist/main.js",
"scripts": {
"build": "tsc",
"dev": "tsx watch src/main.ts",
"start": "node dist/main.js",
"lint": "eslint \"src/**/*.ts\"",
"test": "jest",
"clean": "rm -rf dist"
},
"dependencies": {
"fastify": "^5.2.0",
"pg": "^8.11.0",
"uuid": "^9.0.0"
},
"devDependencies": {
"@types/node": "^20.10.0",
"@types/pg": "^8.10.0",
"@types/uuid": "^9.0.0",
"tsx": "^4.7.0",
"typescript": "^5.3.0"
}
}

View File

@ -0,0 +1,104 @@
-- ===========================================
-- LLM Gateway Database Tables
-- ===========================================
-- 1. API Keys for external users
CREATE TABLE IF NOT EXISTS gateway_api_keys (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
tenant_id UUID,
key_hash VARCHAR(64) NOT NULL UNIQUE,
key_prefix VARCHAR(12) NOT NULL,
name VARCHAR(100) NOT NULL,
owner VARCHAR(200) NOT NULL DEFAULT '',
permissions JSONB NOT NULL DEFAULT '{"allowedModels": ["*"], "allowStreaming": true, "allowTools": true}',
rate_limit_rpm INTEGER NOT NULL DEFAULT 60,
rate_limit_tpd INTEGER NOT NULL DEFAULT 1000000,
monthly_budget DECIMAL(10,2),
enabled BOOLEAN NOT NULL DEFAULT true,
expires_at TIMESTAMP,
last_used_at TIMESTAMP,
created_by UUID,
created_at TIMESTAMP NOT NULL DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_gateway_api_keys_hash ON gateway_api_keys (key_hash);
CREATE INDEX IF NOT EXISTS idx_gateway_api_keys_tenant ON gateway_api_keys (tenant_id);
CREATE INDEX IF NOT EXISTS idx_gateway_api_keys_enabled ON gateway_api_keys (enabled);
-- 2. Injection rules (regulatory content to inject into system prompts)
CREATE TABLE IF NOT EXISTS gateway_injection_rules (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
tenant_id UUID,
name VARCHAR(100) NOT NULL,
description TEXT,
position VARCHAR(10) NOT NULL DEFAULT 'append' CHECK (position IN ('prepend', 'append')),
content TEXT NOT NULL,
match_models JSONB NOT NULL DEFAULT '["*"]',
match_key_ids JSONB,
priority INTEGER NOT NULL DEFAULT 0,
enabled BOOLEAN NOT NULL DEFAULT true,
created_by UUID,
created_at TIMESTAMP NOT NULL DEFAULT NOW(),
updated_at TIMESTAMP NOT NULL DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_gateway_injection_rules_tenant ON gateway_injection_rules (tenant_id);
CREATE INDEX IF NOT EXISTS idx_gateway_injection_rules_enabled ON gateway_injection_rules (tenant_id, enabled);
-- 3. Content filter rules (message auditing)
CREATE TABLE IF NOT EXISTS gateway_content_rules (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
tenant_id UUID,
name VARCHAR(100) NOT NULL,
type VARCHAR(20) NOT NULL DEFAULT 'keyword' CHECK (type IN ('keyword', 'regex')),
pattern TEXT NOT NULL,
action VARCHAR(20) NOT NULL DEFAULT 'block' CHECK (action IN ('block', 'warn', 'log')),
reject_message TEXT,
priority INTEGER NOT NULL DEFAULT 0,
enabled BOOLEAN NOT NULL DEFAULT true,
created_by UUID,
created_at TIMESTAMP NOT NULL DEFAULT NOW(),
updated_at TIMESTAMP NOT NULL DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_gateway_content_rules_tenant ON gateway_content_rules (tenant_id);
CREATE INDEX IF NOT EXISTS idx_gateway_content_rules_enabled ON gateway_content_rules (tenant_id, enabled);
-- 4. Usage logs (token consumption tracking)
CREATE TABLE IF NOT EXISTS gateway_usage_logs (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
api_key_id UUID NOT NULL REFERENCES gateway_api_keys(id) ON DELETE CASCADE,
model VARCHAR(100) NOT NULL,
provider VARCHAR(20) NOT NULL,
input_tokens INTEGER NOT NULL DEFAULT 0,
output_tokens INTEGER NOT NULL DEFAULT 0,
total_tokens INTEGER NOT NULL DEFAULT 0,
cost_usd DECIMAL(10,6),
duration_ms INTEGER NOT NULL DEFAULT 0,
status_code INTEGER NOT NULL DEFAULT 200,
created_at TIMESTAMP NOT NULL DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_gateway_usage_logs_key ON gateway_usage_logs (api_key_id);
CREATE INDEX IF NOT EXISTS idx_gateway_usage_logs_created ON gateway_usage_logs (created_at);
CREATE INDEX IF NOT EXISTS idx_gateway_usage_logs_key_created ON gateway_usage_logs (api_key_id, created_at);
-- 5. Audit logs (request/response auditing)
CREATE TABLE IF NOT EXISTS gateway_audit_logs (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
api_key_id UUID NOT NULL REFERENCES gateway_api_keys(id) ON DELETE CASCADE,
request_method VARCHAR(10) NOT NULL,
request_path VARCHAR(200) NOT NULL,
request_model VARCHAR(100),
request_ip VARCHAR(50) NOT NULL DEFAULT '',
content_filtered BOOLEAN NOT NULL DEFAULT false,
filter_rule_id UUID,
injection_applied BOOLEAN NOT NULL DEFAULT false,
response_status INTEGER NOT NULL DEFAULT 200,
duration_ms INTEGER NOT NULL DEFAULT 0,
created_at TIMESTAMP NOT NULL DEFAULT NOW()
);
CREATE INDEX IF NOT EXISTS idx_gateway_audit_logs_key ON gateway_audit_logs (api_key_id);
CREATE INDEX IF NOT EXISTS idx_gateway_audit_logs_created ON gateway_audit_logs (created_at);
CREATE INDEX IF NOT EXISTS idx_gateway_audit_logs_filtered ON gateway_audit_logs (content_filtered) WHERE content_filtered = true;

View File

@ -0,0 +1,29 @@
export interface GatewayConfig {
port: number;
databaseUrl: string;
anthropicApiKey: string;
anthropicUpstreamUrl: string;
openaiApiKey: string;
openaiUpstreamUrl: string;
rulesCacheTtlMs: number;
logLevel: string;
}
export function loadConfig(): GatewayConfig {
const required = (key: string): string => {
const val = process.env[key];
if (!val) throw new Error(`Missing required env var: ${key}`);
return val;
};
return {
port: parseInt(process.env.PORT || '3008', 10),
databaseUrl: required('DATABASE_URL'),
anthropicApiKey: required('ANTHROPIC_API_KEY'),
anthropicUpstreamUrl: (process.env.ANTHROPIC_UPSTREAM_URL || 'https://api.anthropic.com').replace(/\/$/, ''),
openaiApiKey: required('OPENAI_API_KEY'),
openaiUpstreamUrl: (process.env.OPENAI_UPSTREAM_URL || 'https://api.openai.com').replace(/\/$/, ''),
rulesCacheTtlMs: parseInt(process.env.RULES_CACHE_TTL_MS || '30000', 10),
logLevel: process.env.LOG_LEVEL || 'info',
};
}

View File

@ -0,0 +1,43 @@
import { Pool, PoolClient } from 'pg';
import { GatewayConfig } from './config';
let pool: Pool;
export function initDb(config: GatewayConfig): Pool {
pool = new Pool({
connectionString: config.databaseUrl,
max: 20,
idleTimeoutMillis: 30000,
connectionTimeoutMillis: 5000,
});
pool.on('error', (err) => {
console.error('[DB] Unexpected error on idle client', err);
});
return pool;
}
export function getPool(): Pool {
if (!pool) throw new Error('Database not initialized');
return pool;
}
export async function query<T = any>(text: string, params?: any[]): Promise<T[]> {
const result = await getPool().query(text, params);
return result.rows as T[];
}
export async function queryOne<T = any>(text: string, params?: any[]): Promise<T | null> {
const rows = await query<T>(text, params);
return rows[0] || null;
}
export async function withClient<T>(fn: (client: PoolClient) => Promise<T>): Promise<T> {
const client = await getPool().connect();
try {
return await fn(client);
} finally {
client.release();
}
}

View File

@ -0,0 +1,119 @@
import { query } from '../db';
import { InjectionRule, AnthropicSystem, AnthropicSystemBlock } from '../types';
// ─── Cache ───
let cachedRules: InjectionRule[] = [];
let cacheTimestamp = 0;
let cacheTtlMs = 30_000;
export function setInjectionCacheTtl(ttlMs: number): void {
cacheTtlMs = ttlMs;
}
async function loadRules(): Promise<InjectionRule[]> {
if (Date.now() - cacheTimestamp < cacheTtlMs && cachedRules.length > 0) {
return cachedRules;
}
const rows = await query<any>(
`SELECT id, tenant_id, name, position, content, match_models, match_key_ids, priority, enabled
FROM gateway_injection_rules
WHERE enabled = true
ORDER BY priority ASC`,
);
cachedRules = rows.map((row) => ({
id: row.id,
tenantId: row.tenant_id,
name: row.name,
position: row.position,
content: row.content,
matchModels: row.match_models || ['*'],
matchKeyIds: row.match_key_ids,
priority: row.priority,
enabled: row.enabled,
}));
cacheTimestamp = Date.now();
return cachedRules;
}
// ─── Match logic ───
function matchesModel(ruleModels: string[], requestModel: string): boolean {
if (ruleModels.includes('*')) return true;
return ruleModels.some((pattern) => {
if (pattern.endsWith('*')) {
return requestModel.startsWith(pattern.slice(0, -1));
}
return requestModel === pattern;
});
}
function matchesKey(ruleKeyIds: string[] | null, apiKeyId: string): boolean {
if (!ruleKeyIds) return true; // null = matches all keys
return ruleKeyIds.includes(apiKeyId);
}
// ─── Inject regulatory content into system prompt ───
export async function injectSystemPrompt(
system: AnthropicSystem | undefined,
model: string,
apiKeyId: string,
): Promise<{ system: AnthropicSystem | undefined; applied: boolean; ruleIds: string[] }> {
const allRules = await loadRules();
const applicableRules = allRules.filter(
(r) => matchesModel(r.matchModels, model) && matchesKey(r.matchKeyIds, apiKeyId),
);
if (applicableRules.length === 0) {
return { system, applied: false, ruleIds: [] };
}
const prependRules = applicableRules.filter((r) => r.position === 'prepend');
const appendRules = applicableRules.filter((r) => r.position === 'append');
const prependText = prependRules.map((r) => r.content).join('\n\n');
const appendText = appendRules.map((r) => r.content).join('\n\n');
const ruleIds = applicableRules.map((r) => r.id);
if (!prependText && !appendText) {
return { system, applied: false, ruleIds: [] };
}
// Handle the three possible system formats
if (system === undefined || system === null) {
// No existing system prompt — create one
const combined = [prependText, appendText].filter(Boolean).join('\n\n');
return { system: combined || undefined, applied: true, ruleIds };
}
if (typeof system === 'string') {
// String format
const parts = [prependText, system, appendText].filter(Boolean);
return { system: parts.join('\n\n'), applied: true, ruleIds };
}
if (Array.isArray(system)) {
// Array of content blocks format
const blocks: AnthropicSystemBlock[] = [...system];
if (prependText) {
blocks.unshift({ type: 'text', text: prependText });
}
if (appendText) {
blocks.push({ type: 'text', text: appendText });
}
return { system: blocks, applied: true, ruleIds };
}
// Fallback — unknown format, don't modify
return { system, applied: false, ruleIds: [] };
}
export function clearInjectionRulesCache(): void {
cachedRules = [];
cacheTimestamp = 0;
}

View File

@ -0,0 +1,79 @@
import { query } from '../db';
import { v4 as uuidv4 } from 'uuid';
import { AuditLogEntry } from '../types';
// ─── Batch queue (async, non-blocking) ───
const pendingAudits: AuditLogEntry[] = [];
let flushTimer: ReturnType<typeof setTimeout> | null = null;
const FLUSH_INTERVAL_MS = 5_000;
const BATCH_SIZE = 50;
async function flushAudits(): Promise<void> {
if (pendingAudits.length === 0) return;
const batch = pendingAudits.splice(0, BATCH_SIZE);
try {
const values: any[] = [];
const placeholders: string[] = [];
for (let i = 0; i < batch.length; i++) {
const entry = batch[i];
const offset = i * 11;
placeholders.push(
`($${offset + 1}, $${offset + 2}, $${offset + 3}, $${offset + 4}, $${offset + 5}, $${offset + 6}, $${offset + 7}, $${offset + 8}, $${offset + 9}, $${offset + 10}, $${offset + 11})`,
);
values.push(
uuidv4(),
entry.apiKeyId,
entry.requestMethod,
entry.requestPath,
entry.requestModel,
entry.requestIp,
entry.contentFiltered,
entry.filterRuleId,
entry.injectionApplied,
entry.responseStatus,
entry.durationMs,
);
}
await query(
`INSERT INTO gateway_audit_logs (id, api_key_id, request_method, request_path, request_model, request_ip, content_filtered, filter_rule_id, injection_applied, response_status, duration_ms)
VALUES ${placeholders.join(', ')}`,
values,
);
} catch (err: any) {
console.error('[AuditLogger] Failed to flush audits:', err.message);
if (pendingAudits.length < 500) {
pendingAudits.unshift(...batch);
}
}
}
function scheduleFlush(): void {
if (flushTimer) return;
flushTimer = setTimeout(async () => {
flushTimer = null;
await flushAudits();
if (pendingAudits.length > 0) scheduleFlush();
}, FLUSH_INTERVAL_MS);
}
// ─── Public API ───
export function recordAudit(entry: AuditLogEntry): void {
pendingAudits.push(entry);
if (pendingAudits.length >= BATCH_SIZE) {
flushAudits();
} else {
scheduleFlush();
}
}
export async function shutdown(): Promise<void> {
if (flushTimer) clearTimeout(flushTimer);
await flushAudits();
}

View File

@ -0,0 +1,142 @@
import { query } from '../db';
import { v4 as uuidv4 } from 'uuid';
import { UsageLogEntry } from '../types';
// ─── Cost estimation (USD per million tokens, approximate) ───
const COST_TABLE: Record<string, { input: number; output: number }> = {
'claude-sonnet-4-5': { input: 3.0, output: 15.0 },
'claude-haiku-4-5': { input: 0.8, output: 4.0 },
'claude-opus-4': { input: 15.0, output: 75.0 },
'text-embedding-3-small': { input: 0.02, output: 0 },
'text-embedding-3-large': { input: 0.13, output: 0 },
};
function estimateCost(model: string, inputTokens: number, outputTokens: number): number | null {
// Find matching cost entry by prefix
for (const [prefix, costs] of Object.entries(COST_TABLE)) {
if (model.startsWith(prefix)) {
return (inputTokens * costs.input + outputTokens * costs.output) / 1_000_000;
}
}
return null;
}
// ─── Batch insert queue (async, non-blocking) ───
const pendingLogs: UsageLogEntry[] = [];
let flushTimer: ReturnType<typeof setTimeout> | null = null;
const FLUSH_INTERVAL_MS = 5_000;
const BATCH_SIZE = 50;
async function flushLogs(): Promise<void> {
if (pendingLogs.length === 0) return;
const batch = pendingLogs.splice(0, BATCH_SIZE);
try {
const values: any[] = [];
const placeholders: string[] = [];
for (let i = 0; i < batch.length; i++) {
const entry = batch[i];
const offset = i * 10;
placeholders.push(
`($${offset + 1}, $${offset + 2}, $${offset + 3}, $${offset + 4}, $${offset + 5}, $${offset + 6}, $${offset + 7}, $${offset + 8}, $${offset + 9}, $${offset + 10})`,
);
values.push(
uuidv4(),
entry.apiKeyId,
entry.model,
entry.provider,
entry.inputTokens,
entry.outputTokens,
entry.totalTokens,
entry.costUsd,
entry.durationMs,
entry.statusCode,
);
}
await query(
`INSERT INTO gateway_usage_logs (id, api_key_id, model, provider, input_tokens, output_tokens, total_tokens, cost_usd, duration_ms, status_code)
VALUES ${placeholders.join(', ')}`,
values,
);
} catch (err: any) {
console.error('[UsageTracker] Failed to flush logs:', err.message);
// Re-queue failed entries (at the front, max 500 pending)
if (pendingLogs.length < 500) {
pendingLogs.unshift(...batch);
}
}
}
function scheduleFlush(): void {
if (flushTimer) return;
flushTimer = setTimeout(async () => {
flushTimer = null;
await flushLogs();
if (pendingLogs.length > 0) scheduleFlush();
}, FLUSH_INTERVAL_MS);
}
// ─── Public API ───
export function recordUsage(entry: UsageLogEntry): void {
if (!entry.costUsd && entry.costUsd !== 0) {
entry.costUsd = estimateCost(entry.model, entry.inputTokens, entry.outputTokens);
}
pendingLogs.push(entry);
if (pendingLogs.length >= BATCH_SIZE) {
flushLogs();
} else {
scheduleFlush();
}
}
export function recordFromAnthropicResponse(
apiKeyId: string,
model: string,
usage: { input_tokens?: number; output_tokens?: number } | undefined,
statusCode: number,
durationMs: number,
): void {
recordUsage({
apiKeyId,
model,
provider: 'anthropic',
inputTokens: usage?.input_tokens || 0,
outputTokens: usage?.output_tokens || 0,
totalTokens: (usage?.input_tokens || 0) + (usage?.output_tokens || 0),
costUsd: null,
durationMs,
statusCode,
});
}
export function recordFromOpenAIResponse(
apiKeyId: string,
model: string,
usage: { prompt_tokens?: number; total_tokens?: number } | undefined,
statusCode: number,
durationMs: number,
): void {
recordUsage({
apiKeyId,
model,
provider: 'openai',
inputTokens: usage?.prompt_tokens || 0,
outputTokens: 0,
totalTokens: usage?.total_tokens || 0,
costUsd: null,
durationMs,
statusCode,
});
}
export async function shutdown(): Promise<void> {
if (flushTimer) clearTimeout(flushTimer);
await flushLogs();
}

View File

@ -0,0 +1,73 @@
import Fastify from 'fastify';
import { loadConfig } from './config';
import { initDb, getPool } from './db';
import { registerHealthRoutes } from './routes/health';
import { registerAnthropicRoutes } from './routes/anthropic';
import { registerOpenAIRoutes } from './routes/openai';
import { setContentFilterCacheTtl } from './middleware/content-filter';
import { setInjectionCacheTtl } from './injection/system-prompt-injector';
import { shutdown as shutdownUsageTracker } from './logging/usage-tracker';
import { shutdown as shutdownAuditLogger } from './logging/audit-logger';
async function main() {
// Load configuration
const config = loadConfig();
// Initialize database
initDb(config);
// Set cache TTLs
setContentFilterCacheTtl(config.rulesCacheTtlMs);
setInjectionCacheTtl(config.rulesCacheTtlMs);
// Create Fastify instance
const app = Fastify({
logger: {
level: config.logLevel,
},
bodyLimit: 50 * 1024 * 1024, // 50MB for base64 PDFs
trustProxy: true,
});
// Disable Fastify's default JSON body parsing — we parse manually
// to avoid double-parse overhead and to handle raw body forwarding
app.addContentTypeParser(
'application/json',
{ parseAs: 'string' },
(_req, body, done) => {
done(null, body);
},
);
// Register routes
registerHealthRoutes(app);
registerAnthropicRoutes(app, config);
registerOpenAIRoutes(app, config);
// Graceful shutdown
const shutdown = async (signal: string) => {
console.log(`[LLM Gateway] Received ${signal}, shutting down...`);
await shutdownUsageTracker();
await shutdownAuditLogger();
await app.close();
const pool = getPool();
await pool.end();
process.exit(0);
};
process.on('SIGTERM', () => shutdown('SIGTERM'));
process.on('SIGINT', () => shutdown('SIGINT'));
// Start server
try {
await app.listen({ port: config.port, host: '0.0.0.0' });
console.log(`[LLM Gateway] Running on port ${config.port}`);
console.log(`[LLM Gateway] Anthropic upstream: ${config.anthropicUpstreamUrl}`);
console.log(`[LLM Gateway] OpenAI upstream: ${config.openaiUpstreamUrl}`);
} catch (err) {
console.error('[LLM Gateway] Failed to start:', err);
process.exit(1);
}
}
main();

View File

@ -0,0 +1,213 @@
import { createHash } from 'crypto';
import { FastifyRequest, FastifyReply } from 'fastify';
import { query, queryOne } from '../db';
import { ApiKeyRecord, ApiKeyPermissions } from '../types';
// ─── Cache ───
interface CachedKey {
record: ApiKeyRecord;
cachedAt: number;
}
const keyCache = new Map<string, CachedKey>();
const KEY_CACHE_TTL_MS = 60_000; // 1 minute
// ─── Hash ───
function hashApiKey(key: string): string {
return createHash('sha256').update(key).digest('hex');
}
// ─── Extract API Key from request ───
// Supports both Anthropic (X-Api-Key) and OpenAI (Authorization: Bearer) formats
function extractApiKey(request: FastifyRequest): string | null {
// Anthropic style
const xApiKey = request.headers['x-api-key'];
if (typeof xApiKey === 'string' && xApiKey) return xApiKey;
// OpenAI style
const auth = request.headers['authorization'];
if (typeof auth === 'string' && auth.startsWith('Bearer ')) {
return auth.slice(7);
}
return null;
}
// ─── Lookup API Key record ───
async function lookupApiKey(apiKey: string): Promise<ApiKeyRecord | null> {
const hash = hashApiKey(apiKey);
// Check cache
const cached = keyCache.get(hash);
if (cached && Date.now() - cached.cachedAt < KEY_CACHE_TTL_MS) {
return cached.record;
}
const row = await queryOne<any>(
`SELECT id, tenant_id, key_hash, key_prefix, name, owner,
permissions, rate_limit_rpm, rate_limit_tpd, monthly_budget,
enabled, expires_at, last_used_at
FROM gateway_api_keys
WHERE key_hash = $1`,
[hash],
);
if (!row) return null;
const record: ApiKeyRecord = {
id: row.id,
tenantId: row.tenant_id,
keyHash: row.key_hash,
keyPrefix: row.key_prefix,
name: row.name,
owner: row.owner,
permissions: row.permissions as ApiKeyPermissions,
rateLimitRpm: row.rate_limit_rpm,
rateLimitTpd: row.rate_limit_tpd,
monthlyBudget: row.monthly_budget ? parseFloat(row.monthly_budget) : null,
enabled: row.enabled,
expiresAt: row.expires_at ? new Date(row.expires_at) : null,
lastUsedAt: row.last_used_at ? new Date(row.last_used_at) : null,
};
keyCache.set(hash, { record, cachedAt: Date.now() });
return record;
}
// ─── Rate limit check (simple in-memory sliding window) ───
interface RateWindow {
timestamps: number[];
}
const rateWindows = new Map<string, RateWindow>();
function checkRateLimit(keyId: string, limitRpm: number): boolean {
if (limitRpm <= 0) return true; // no limit
const now = Date.now();
const windowStart = now - 60_000;
let window = rateWindows.get(keyId);
if (!window) {
window = { timestamps: [] };
rateWindows.set(keyId, window);
}
// Remove expired entries
window.timestamps = window.timestamps.filter((t) => t > windowStart);
if (window.timestamps.length >= limitRpm) {
return false; // rate limited
}
window.timestamps.push(now);
return true;
}
// Periodic cleanup of stale rate windows
setInterval(() => {
const cutoff = Date.now() - 120_000;
for (const [key, window] of rateWindows.entries()) {
if (window.timestamps.length === 0 || window.timestamps[window.timestamps.length - 1] < cutoff) {
rateWindows.delete(key);
}
}
}, 60_000);
// ─── Update last_used_at (async, fire-and-forget) ───
function touchLastUsed(keyId: string): void {
query('UPDATE gateway_api_keys SET last_used_at = NOW() WHERE id = $1', [keyId]).catch((err) => {
console.error('[Auth] Failed to update last_used_at:', err.message);
});
}
// ─── Auth middleware ───
export async function authMiddleware(
request: FastifyRequest,
reply: FastifyReply,
): Promise<void> {
const apiKey = extractApiKey(request);
if (!apiKey) {
reply.status(401).send({
error: {
type: 'authentication_error',
message: 'Missing API key. Provide via X-Api-Key header or Authorization: Bearer header.',
},
});
return;
}
const record = await lookupApiKey(apiKey);
if (!record) {
reply.status(401).send({
error: {
type: 'authentication_error',
message: 'Invalid API key.',
},
});
return;
}
if (!record.enabled) {
reply.status(403).send({
error: {
type: 'permission_error',
message: 'API key is disabled.',
},
});
return;
}
if (record.expiresAt && record.expiresAt < new Date()) {
reply.status(403).send({
error: {
type: 'permission_error',
message: 'API key has expired.',
},
});
return;
}
if (!checkRateLimit(record.id, record.rateLimitRpm)) {
reply.status(429).send({
error: {
type: 'rate_limit_error',
message: `Rate limit exceeded. Maximum ${record.rateLimitRpm} requests per minute.`,
},
});
return;
}
// Attach to request for downstream use
(request as any).apiKeyRecord = record;
// Fire-and-forget update
touchLastUsed(record.id);
}
// ─── Model permission check ───
export function isModelAllowed(record: ApiKeyRecord, model: string): boolean {
const allowed = record.permissions?.allowedModels;
if (!allowed || allowed.length === 0 || allowed.includes('*')) return true;
return allowed.some((pattern) => {
if (pattern.endsWith('*')) {
return model.startsWith(pattern.slice(0, -1));
}
return model === pattern;
});
}
export function clearKeyCache(): void {
keyCache.clear();
}

View File

@ -0,0 +1,116 @@
import { query } from '../db';
import { ContentRule, FilterResult, AnthropicMessage, AnthropicContentBlock } from '../types';
// ─── Cache ───
let cachedRules: ContentRule[] = [];
let cacheTimestamp = 0;
let cacheTtlMs = 30_000;
export function setContentFilterCacheTtl(ttlMs: number): void {
cacheTtlMs = ttlMs;
}
async function loadRules(): Promise<ContentRule[]> {
if (Date.now() - cacheTimestamp < cacheTtlMs && cachedRules.length > 0) {
return cachedRules;
}
const rows = await query<any>(
`SELECT id, tenant_id, name, type, pattern, action, reject_message, priority, enabled
FROM gateway_content_rules
WHERE enabled = true
ORDER BY priority ASC`,
);
cachedRules = rows.map((row) => ({
id: row.id,
tenantId: row.tenant_id,
name: row.name,
type: row.type,
pattern: row.pattern,
action: row.action,
rejectMessage: row.reject_message,
priority: row.priority,
enabled: row.enabled,
}));
cacheTimestamp = Date.now();
return cachedRules;
}
// ─── Extract text from messages ───
function extractTextFromContent(content: string | AnthropicContentBlock[]): string {
if (typeof content === 'string') return content;
if (!Array.isArray(content)) return '';
return content
.filter((block) => block.type === 'text' && block.text)
.map((block) => block.text!)
.join(' ');
}
// ─── Check messages against rules ───
export async function checkContent(messages: AnthropicMessage[]): Promise<FilterResult> {
const rules = await loadRules();
if (rules.length === 0) return { blocked: false };
for (const msg of messages) {
if (msg.role !== 'user') continue;
const text = extractTextFromContent(msg.content);
if (!text) continue;
for (const rule of rules) {
let matched = false;
if (rule.type === 'keyword') {
matched = text.toLowerCase().includes(rule.pattern.toLowerCase());
} else if (rule.type === 'regex') {
try {
matched = new RegExp(rule.pattern, 'i').test(text);
} catch {
console.warn(`[ContentFilter] Invalid regex pattern in rule ${rule.id}: ${rule.pattern}`);
continue;
}
}
if (matched) {
if (rule.action === 'block') {
return {
blocked: true,
reason: rule.rejectMessage || 'Content blocked by policy.',
ruleId: rule.id,
action: 'block',
};
}
if (rule.action === 'warn') {
// Log but allow through
console.warn(`[ContentFilter] Warning triggered by rule "${rule.name}" (${rule.id})`);
return {
blocked: false,
ruleId: rule.id,
action: 'warn',
};
}
if (rule.action === 'log') {
console.info(`[ContentFilter] Log match by rule "${rule.name}" (${rule.id})`);
return {
blocked: false,
ruleId: rule.id,
action: 'log',
};
}
}
}
}
return { blocked: false };
}
export function clearContentRulesCache(): void {
cachedRules = [];
cacheTimestamp = 0;
}

View File

@ -0,0 +1,205 @@
import { FastifyRequest, FastifyReply } from 'fastify';
import { GatewayConfig } from '../config';
import { ApiKeyRecord, AnthropicRequestBody, FilterResult } from '../types';
import { isModelAllowed } from '../middleware/auth';
import { checkContent } from '../middleware/content-filter';
import { injectSystemPrompt } from '../injection/system-prompt-injector';
import { recordFromAnthropicResponse } from '../logging/usage-tracker';
import { recordAudit } from '../logging/audit-logger';
import { pipeSSEStream, createStreamUsageTracker } from './stream-pipe';
export function createAnthropicProxy(config: GatewayConfig) {
return async function handleMessages(request: FastifyRequest, reply: FastifyReply): Promise<void> {
const startTime = Date.now();
const apiKeyRecord: ApiKeyRecord = (request as any).apiKeyRecord;
const clientIp = request.ip || request.headers['x-forwarded-for'] as string || 'unknown';
// 1. Parse request body
let body: AnthropicRequestBody;
try {
body = JSON.parse(request.body as string);
} catch {
reply.status(400).send({
error: { type: 'invalid_request_error', message: 'Invalid JSON body.' },
});
return;
}
const model = body.model || 'unknown';
// 2. Check model permission
if (!isModelAllowed(apiKeyRecord, model)) {
reply.status(403).send({
error: {
type: 'permission_error',
message: `Model "${model}" is not allowed for this API key.`,
},
});
return;
}
// 3. Check streaming permission
if (body.stream && apiKeyRecord.permissions?.allowStreaming === false) {
reply.status(403).send({
error: {
type: 'permission_error',
message: 'Streaming is not allowed for this API key.',
},
});
return;
}
// 4. Content filtering
let filterResult: FilterResult = { blocked: false };
if (body.messages) {
filterResult = await checkContent(body.messages);
if (filterResult.blocked) {
recordAudit({
apiKeyId: apiKeyRecord.id,
requestMethod: 'POST',
requestPath: '/v1/messages',
requestModel: model,
requestIp: clientIp,
contentFiltered: true,
filterRuleId: filterResult.ruleId || null,
injectionApplied: false,
responseStatus: 403,
durationMs: Date.now() - startTime,
});
reply.status(403).send({
error: {
type: 'content_policy_violation',
message: filterResult.reason || 'Content blocked by policy.',
},
});
return;
}
}
// 5. Inject regulatory content into system prompt
const injection = await injectSystemPrompt(body.system, model, apiKeyRecord.id);
body.system = injection.system;
// 6. Build upstream request headers
const upstreamHeaders: Record<string, string> = {
'Content-Type': 'application/json',
'anthropic-version': (request.headers['anthropic-version'] as string) || '2023-06-01',
'X-Api-Key': config.anthropicApiKey,
};
// Forward optional headers
const beta = request.headers['anthropic-beta'];
if (beta) upstreamHeaders['anthropic-beta'] = beta as string;
// 7. Forward to upstream
let upstreamResponse: Response;
try {
upstreamResponse = await fetch(`${config.anthropicUpstreamUrl}/v1/messages`, {
method: 'POST',
headers: upstreamHeaders,
body: JSON.stringify(body),
});
} catch (err: any) {
const durationMs = Date.now() - startTime;
recordAudit({
apiKeyId: apiKeyRecord.id,
requestMethod: 'POST',
requestPath: '/v1/messages',
requestModel: model,
requestIp: clientIp,
contentFiltered: filterResult.action === 'warn' || filterResult.action === 'log',
filterRuleId: filterResult.ruleId || null,
injectionApplied: injection.applied,
responseStatus: 502,
durationMs,
});
reply.status(502).send({
error: {
type: 'upstream_error',
message: `Failed to connect to upstream API: ${err.message}`,
},
});
return;
}
const durationMs = Date.now() - startTime;
// 8. Handle response
if (body.stream && upstreamResponse.ok && upstreamResponse.body) {
// Streaming response — pipe SSE directly
reply.raw.writeHead(upstreamResponse.status, {
'Content-Type': 'text/event-stream',
'Cache-Control': 'no-cache',
'Connection': 'keep-alive',
'X-Accel-Buffering': 'no',
});
const usageTracker = createStreamUsageTracker();
await pipeSSEStream(upstreamResponse.body, reply.raw, usageTracker.onDataLine);
// Record usage from stream (async)
const streamUsage = usageTracker.getUsage();
recordFromAnthropicResponse(
apiKeyRecord.id,
model,
{ input_tokens: streamUsage.inputTokens, output_tokens: streamUsage.outputTokens },
upstreamResponse.status,
Date.now() - startTime,
);
recordAudit({
apiKeyId: apiKeyRecord.id,
requestMethod: 'POST',
requestPath: '/v1/messages',
requestModel: model,
requestIp: clientIp,
contentFiltered: filterResult.action === 'warn' || filterResult.action === 'log',
filterRuleId: filterResult.ruleId || null,
injectionApplied: injection.applied,
responseStatus: upstreamResponse.status,
durationMs: Date.now() - startTime,
});
} else {
// Non-streaming response — buffer and forward
const responseText = await upstreamResponse.text();
// Try to extract usage for logging
try {
const responseJson = JSON.parse(responseText);
if (responseJson.usage) {
recordFromAnthropicResponse(
apiKeyRecord.id,
model,
responseJson.usage,
upstreamResponse.status,
durationMs,
);
}
} catch {
// Not JSON — still forward
}
recordAudit({
apiKeyId: apiKeyRecord.id,
requestMethod: 'POST',
requestPath: '/v1/messages',
requestModel: model,
requestIp: clientIp,
contentFiltered: filterResult.action === 'warn' || filterResult.action === 'log',
filterRuleId: filterResult.ruleId || null,
injectionApplied: injection.applied,
responseStatus: upstreamResponse.status,
durationMs,
});
// Forward all response headers from upstream
reply.raw.writeHead(upstreamResponse.status, {
'Content-Type': upstreamResponse.headers.get('content-type') || 'application/json',
});
reply.raw.end(responseText);
}
};
}

View File

@ -0,0 +1,216 @@
import { FastifyRequest, FastifyReply } from 'fastify';
import { GatewayConfig } from '../config';
import { ApiKeyRecord } from '../types';
import { isModelAllowed } from '../middleware/auth';
import { recordFromOpenAIResponse } from '../logging/usage-tracker';
import { recordAudit } from '../logging/audit-logger';
import { pipeSSEStream, createStreamUsageTracker } from './stream-pipe';
export function createOpenAIEmbeddingsProxy(config: GatewayConfig) {
return async function handleEmbeddings(request: FastifyRequest, reply: FastifyReply): Promise<void> {
const startTime = Date.now();
const apiKeyRecord: ApiKeyRecord = (request as any).apiKeyRecord;
const clientIp = request.ip || request.headers['x-forwarded-for'] as string || 'unknown';
let body: any;
try {
body = JSON.parse(request.body as string);
} catch {
reply.status(400).send({
error: { type: 'invalid_request_error', message: 'Invalid JSON body.' },
});
return;
}
const model = body.model || 'unknown';
if (!isModelAllowed(apiKeyRecord, model)) {
reply.status(403).send({
error: {
type: 'permission_error',
message: `Model "${model}" is not allowed for this API key.`,
},
});
return;
}
let upstreamResponse: Response;
try {
upstreamResponse = await fetch(`${config.openaiUpstreamUrl}/v1/embeddings`, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Authorization': `Bearer ${config.openaiApiKey}`,
},
body: JSON.stringify(body),
});
} catch (err: any) {
recordAudit({
apiKeyId: apiKeyRecord.id,
requestMethod: 'POST',
requestPath: '/v1/embeddings',
requestModel: model,
requestIp: clientIp,
contentFiltered: false,
filterRuleId: null,
injectionApplied: false,
responseStatus: 502,
durationMs: Date.now() - startTime,
});
reply.status(502).send({
error: {
type: 'upstream_error',
message: `Failed to connect to upstream API: ${err.message}`,
},
});
return;
}
const durationMs = Date.now() - startTime;
const responseText = await upstreamResponse.text();
try {
const responseJson = JSON.parse(responseText);
if (responseJson.usage) {
recordFromOpenAIResponse(
apiKeyRecord.id,
model,
responseJson.usage,
upstreamResponse.status,
durationMs,
);
}
} catch {
// Not JSON
}
recordAudit({
apiKeyId: apiKeyRecord.id,
requestMethod: 'POST',
requestPath: '/v1/embeddings',
requestModel: model,
requestIp: clientIp,
contentFiltered: false,
filterRuleId: null,
injectionApplied: false,
responseStatus: upstreamResponse.status,
durationMs,
});
reply.raw.writeHead(upstreamResponse.status, {
'Content-Type': upstreamResponse.headers.get('content-type') || 'application/json',
});
reply.raw.end(responseText);
};
}
export function createOpenAIChatProxy(config: GatewayConfig) {
return async function handleChatCompletions(request: FastifyRequest, reply: FastifyReply): Promise<void> {
const startTime = Date.now();
const apiKeyRecord: ApiKeyRecord = (request as any).apiKeyRecord;
const clientIp = request.ip || request.headers['x-forwarded-for'] as string || 'unknown';
let body: any;
try {
body = JSON.parse(request.body as string);
} catch {
reply.status(400).send({
error: { type: 'invalid_request_error', message: 'Invalid JSON body.' },
});
return;
}
const model = body.model || 'unknown';
if (!isModelAllowed(apiKeyRecord, model)) {
reply.status(403).send({
error: {
type: 'permission_error',
message: `Model "${model}" is not allowed for this API key.`,
},
});
return;
}
let upstreamResponse: Response;
try {
upstreamResponse = await fetch(`${config.openaiUpstreamUrl}/v1/chat/completions`, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Authorization': `Bearer ${config.openaiApiKey}`,
},
body: JSON.stringify(body),
});
} catch (err: any) {
recordAudit({
apiKeyId: apiKeyRecord.id,
requestMethod: 'POST',
requestPath: '/v1/chat/completions',
requestModel: model,
requestIp: clientIp,
contentFiltered: false,
filterRuleId: null,
injectionApplied: false,
responseStatus: 502,
durationMs: Date.now() - startTime,
});
reply.status(502).send({
error: {
type: 'upstream_error',
message: `Failed to connect to upstream API: ${err.message}`,
},
});
return;
}
const durationMs = Date.now() - startTime;
// Handle streaming chat completions
if (body.stream && upstreamResponse.ok && upstreamResponse.body) {
reply.raw.writeHead(upstreamResponse.status, {
'Content-Type': 'text/event-stream',
'Cache-Control': 'no-cache',
'Connection': 'keep-alive',
'X-Accel-Buffering': 'no',
});
await pipeSSEStream(upstreamResponse.body, reply.raw);
recordAudit({
apiKeyId: apiKeyRecord.id,
requestMethod: 'POST',
requestPath: '/v1/chat/completions',
requestModel: model,
requestIp: clientIp,
contentFiltered: false,
filterRuleId: null,
injectionApplied: false,
responseStatus: upstreamResponse.status,
durationMs: Date.now() - startTime,
});
} else {
const responseText = await upstreamResponse.text();
recordAudit({
apiKeyId: apiKeyRecord.id,
requestMethod: 'POST',
requestPath: '/v1/chat/completions',
requestModel: model,
requestIp: clientIp,
contentFiltered: false,
filterRuleId: null,
injectionApplied: false,
responseStatus: upstreamResponse.status,
durationMs,
});
reply.raw.writeHead(upstreamResponse.status, {
'Content-Type': upstreamResponse.headers.get('content-type') || 'application/json',
});
reply.raw.end(responseText);
}
};
}

View File

@ -0,0 +1,94 @@
import { Readable } from 'stream';
import { ServerResponse } from 'http';
/**
* Pipe an upstream SSE response body to the client response.
* Uses zero-buffering: chunks are forwarded as they arrive.
*
* Also provides a callback for each SSE data line, allowing
* the caller to extract usage info from the stream without buffering.
*/
export async function pipeSSEStream(
upstreamBody: ReadableStream<Uint8Array>,
clientResponse: ServerResponse,
onDataLine?: (line: string) => void,
): Promise<void> {
const reader = upstreamBody.getReader();
const decoder = new TextDecoder();
try {
while (true) {
const { done, value } = await reader.read();
if (done) break;
const chunk = decoder.decode(value, { stream: true });
// Forward chunk immediately
const canContinue = clientResponse.write(chunk);
// Parse SSE data lines for usage extraction
if (onDataLine) {
const lines = chunk.split('\n');
for (const line of lines) {
if (line.startsWith('data: ')) {
onDataLine(line.slice(6).trim());
}
}
}
// Handle backpressure
if (!canContinue) {
await new Promise<void>((resolve) => clientResponse.once('drain', resolve));
}
}
} catch (err: any) {
// Connection closed by client — expected during cancellation
if (err.code === 'ERR_STREAM_PREMATURE_CLOSE' || clientResponse.destroyed) {
return;
}
throw err;
} finally {
reader.releaseLock();
clientResponse.end();
}
}
/**
* Extract usage from Anthropic's message_delta SSE event.
* The event format: {"type":"message_delta","delta":{...},"usage":{"output_tokens":N}}
* Final usage is in the message_start event's message.usage for input_tokens.
*/
export interface StreamUsage {
inputTokens: number;
outputTokens: number;
}
export function createStreamUsageTracker(): {
onDataLine: (line: string) => void;
getUsage: () => StreamUsage;
} {
const usage: StreamUsage = { inputTokens: 0, outputTokens: 0 };
return {
onDataLine: (line: string) => {
if (line === '[DONE]') return;
try {
const event = JSON.parse(line);
// message_start contains input_tokens
if (event.type === 'message_start' && event.message?.usage) {
usage.inputTokens = event.message.usage.input_tokens || 0;
}
// message_delta contains output_tokens
if (event.type === 'message_delta' && event.usage) {
usage.outputTokens = event.usage.output_tokens || 0;
}
} catch {
// Not valid JSON — ignore
}
},
getUsage: () => usage,
};
}

View File

@ -0,0 +1,13 @@
import { FastifyInstance } from 'fastify';
import { GatewayConfig } from '../config';
import { authMiddleware } from '../middleware/auth';
import { createAnthropicProxy } from '../proxy/anthropic-proxy';
export function registerAnthropicRoutes(app: FastifyInstance, config: GatewayConfig): void {
const handleMessages = createAnthropicProxy(config);
app.post('/v1/messages', {
preHandler: authMiddleware,
handler: handleMessages,
});
}

View File

@ -0,0 +1,15 @@
import { FastifyInstance } from 'fastify';
import { getPool } from '../db';
export function registerHealthRoutes(app: FastifyInstance): void {
app.get('/health', async (_request, reply) => {
try {
const pool = getPool();
await pool.query('SELECT 1');
return { status: 'ok', service: 'llm-gateway', timestamp: new Date().toISOString() };
} catch (err: any) {
reply.status(503);
return { status: 'unhealthy', error: err.message };
}
});
}

View File

@ -0,0 +1,19 @@
import { FastifyInstance } from 'fastify';
import { GatewayConfig } from '../config';
import { authMiddleware } from '../middleware/auth';
import { createOpenAIEmbeddingsProxy, createOpenAIChatProxy } from '../proxy/openai-proxy';
export function registerOpenAIRoutes(app: FastifyInstance, config: GatewayConfig): void {
const handleEmbeddings = createOpenAIEmbeddingsProxy(config);
const handleChatCompletions = createOpenAIChatProxy(config);
app.post('/v1/embeddings', {
preHandler: authMiddleware,
handler: handleEmbeddings,
});
app.post('/v1/chat/completions', {
preHandler: authMiddleware,
handler: handleChatCompletions,
});
}

View File

@ -0,0 +1,146 @@
// ─── API Key Record (from gateway_api_keys table) ───
export interface ApiKeyRecord {
id: string;
tenantId: string | null;
keyHash: string;
keyPrefix: string;
name: string;
owner: string;
permissions: ApiKeyPermissions;
rateLimitRpm: number;
rateLimitTpd: number;
monthlyBudget: number | null;
enabled: boolean;
expiresAt: Date | null;
lastUsedAt: Date | null;
}
export interface ApiKeyPermissions {
allowedModels: string[]; // ["*"] = all, or ["claude-haiku-4-5-*", "text-embedding-3-small"]
allowStreaming: boolean;
allowTools: boolean;
}
// ─── Injection Rule (from gateway_injection_rules table) ───
export interface InjectionRule {
id: string;
tenantId: string | null;
name: string;
position: 'prepend' | 'append';
content: string;
matchModels: string[]; // ["*"] or specific model patterns
matchKeyIds: string[] | null; // null = all keys
priority: number;
enabled: boolean;
}
// ─── Content Filter Rule (from gateway_content_rules table) ───
export interface ContentRule {
id: string;
tenantId: string | null;
name: string;
type: 'keyword' | 'regex';
pattern: string;
action: 'block' | 'warn' | 'log';
rejectMessage: string | null;
priority: number;
enabled: boolean;
}
// ─── Anthropic Types ───
export interface AnthropicSystemBlock {
type: 'text';
text: string;
cache_control?: { type: string };
}
export type AnthropicSystem = string | AnthropicSystemBlock[];
export interface AnthropicRequestBody {
model: string;
system?: AnthropicSystem;
messages: AnthropicMessage[];
max_tokens: number;
stream?: boolean;
tools?: any[];
tool_choice?: any;
temperature?: number;
top_p?: number;
top_k?: number;
stop_sequences?: string[];
metadata?: any;
output_config?: any;
[key: string]: any;
}
export interface AnthropicMessage {
role: 'user' | 'assistant';
content: string | AnthropicContentBlock[];
}
export interface AnthropicContentBlock {
type: string;
text?: string;
source?: any;
[key: string]: any;
}
// ─── OpenAI Types ───
export interface OpenAIEmbeddingRequest {
model: string;
input: string | string[];
encoding_format?: string;
dimensions?: number;
}
export interface OpenAIChatRequest {
model: string;
messages: Array<{ role: string; content: any }>;
stream?: boolean;
max_tokens?: number;
temperature?: number;
[key: string]: any;
}
// ─── Filter Result ───
export interface FilterResult {
blocked: boolean;
reason?: string;
ruleId?: string;
action?: 'block' | 'warn' | 'log';
}
// ─── Usage Log ───
export interface UsageLogEntry {
apiKeyId: string;
model: string;
provider: 'anthropic' | 'openai';
inputTokens: number;
outputTokens: number;
totalTokens: number;
costUsd: number | null;
durationMs: number;
statusCode: number;
}
// ─── Audit Log ───
export interface AuditLogEntry {
apiKeyId: string;
requestMethod: string;
requestPath: string;
requestModel: string | null;
requestIp: string;
contentFiltered: boolean;
filterRuleId: string | null;
injectionApplied: boolean;
responseStatus: number;
durationMs: number;
}

View File

@ -0,0 +1,23 @@
{
"compilerOptions": {
"module": "commonjs",
"moduleResolution": "node",
"declaration": true,
"removeComments": true,
"allowSyntheticDefaultImports": true,
"esModuleInterop": true,
"target": "ES2021",
"sourceMap": true,
"outDir": "./dist",
"baseUrl": "./",
"incremental": true,
"skipLibCheck": true,
"strict": true,
"strictPropertyInitialization": false,
"noImplicitAny": true,
"forceConsistentCasingInFileNames": true,
"noFallthroughCasesInSwitch": true
},
"include": ["src/**/*"],
"exclude": ["node_modules", "dist"]
}

View File

@ -1810,6 +1810,117 @@ COMMENT ON TABLE collection_directives IS '信息收集指令表 - 管理员配
CREATE INDEX idx_collection_directives_tenant ON collection_directives(tenant_id);
CREATE INDEX idx_collection_directives_tenant_enabled ON collection_directives(tenant_id, enabled);
-- ===========================================
-- LLM Gateway 相关表
-- 对外 API 代理服务的配置、用量和审计
-- ===========================================
-- API Keys for external users
CREATE TABLE IF NOT EXISTS gateway_api_keys (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
tenant_id UUID,
key_hash VARCHAR(64) NOT NULL UNIQUE,
key_prefix VARCHAR(12) NOT NULL,
name VARCHAR(100) NOT NULL,
owner VARCHAR(200) NOT NULL DEFAULT '',
permissions JSONB NOT NULL DEFAULT '{"allowedModels": ["*"], "allowStreaming": true, "allowTools": true}',
rate_limit_rpm INTEGER NOT NULL DEFAULT 60,
rate_limit_tpd INTEGER NOT NULL DEFAULT 1000000,
monthly_budget DECIMAL(10,2),
enabled BOOLEAN NOT NULL DEFAULT true,
expires_at TIMESTAMP,
last_used_at TIMESTAMP,
created_by UUID,
created_at TIMESTAMP NOT NULL DEFAULT NOW()
);
COMMENT ON TABLE gateway_api_keys IS 'LLM Gateway - 外部用户 API Key 管理';
CREATE INDEX IF NOT EXISTS idx_gateway_api_keys_hash ON gateway_api_keys (key_hash);
CREATE INDEX IF NOT EXISTS idx_gateway_api_keys_tenant ON gateway_api_keys (tenant_id);
CREATE INDEX IF NOT EXISTS idx_gateway_api_keys_enabled ON gateway_api_keys (enabled);
-- Injection rules (regulatory content injected into system prompts)
CREATE TABLE IF NOT EXISTS gateway_injection_rules (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
tenant_id UUID,
name VARCHAR(100) NOT NULL,
description TEXT,
position VARCHAR(10) NOT NULL DEFAULT 'append' CHECK (position IN ('prepend', 'append')),
content TEXT NOT NULL,
match_models JSONB NOT NULL DEFAULT '["*"]',
match_key_ids JSONB,
priority INTEGER NOT NULL DEFAULT 0,
enabled BOOLEAN NOT NULL DEFAULT true,
created_by UUID,
created_at TIMESTAMP NOT NULL DEFAULT NOW(),
updated_at TIMESTAMP NOT NULL DEFAULT NOW()
);
COMMENT ON TABLE gateway_injection_rules IS 'LLM Gateway - 监管内容注入规则';
CREATE INDEX IF NOT EXISTS idx_gateway_injection_rules_tenant ON gateway_injection_rules (tenant_id);
CREATE INDEX IF NOT EXISTS idx_gateway_injection_rules_enabled ON gateway_injection_rules (tenant_id, enabled);
-- Content filter rules (message auditing)
CREATE TABLE IF NOT EXISTS gateway_content_rules (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
tenant_id UUID,
name VARCHAR(100) NOT NULL,
type VARCHAR(20) NOT NULL DEFAULT 'keyword' CHECK (type IN ('keyword', 'regex')),
pattern TEXT NOT NULL,
action VARCHAR(20) NOT NULL DEFAULT 'block' CHECK (action IN ('block', 'warn', 'log')),
reject_message TEXT,
priority INTEGER NOT NULL DEFAULT 0,
enabled BOOLEAN NOT NULL DEFAULT true,
created_by UUID,
created_at TIMESTAMP NOT NULL DEFAULT NOW(),
updated_at TIMESTAMP NOT NULL DEFAULT NOW()
);
COMMENT ON TABLE gateway_content_rules IS 'LLM Gateway - 内容审查过滤规则';
CREATE INDEX IF NOT EXISTS idx_gateway_content_rules_tenant ON gateway_content_rules (tenant_id);
CREATE INDEX IF NOT EXISTS idx_gateway_content_rules_enabled ON gateway_content_rules (tenant_id, enabled);
-- Usage logs (token consumption tracking)
CREATE TABLE IF NOT EXISTS gateway_usage_logs (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
api_key_id UUID NOT NULL REFERENCES gateway_api_keys(id) ON DELETE CASCADE,
model VARCHAR(100) NOT NULL,
provider VARCHAR(20) NOT NULL,
input_tokens INTEGER NOT NULL DEFAULT 0,
output_tokens INTEGER NOT NULL DEFAULT 0,
total_tokens INTEGER NOT NULL DEFAULT 0,
cost_usd DECIMAL(10,6),
duration_ms INTEGER NOT NULL DEFAULT 0,
status_code INTEGER NOT NULL DEFAULT 200,
created_at TIMESTAMP NOT NULL DEFAULT NOW()
);
COMMENT ON TABLE gateway_usage_logs IS 'LLM Gateway - API 用量记录';
CREATE INDEX IF NOT EXISTS idx_gateway_usage_logs_key ON gateway_usage_logs (api_key_id);
CREATE INDEX IF NOT EXISTS idx_gateway_usage_logs_created ON gateway_usage_logs (created_at);
CREATE INDEX IF NOT EXISTS idx_gateway_usage_logs_key_created ON gateway_usage_logs (api_key_id, created_at);
-- Audit logs (request/response auditing)
CREATE TABLE IF NOT EXISTS gateway_audit_logs (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
api_key_id UUID NOT NULL REFERENCES gateway_api_keys(id) ON DELETE CASCADE,
request_method VARCHAR(10) NOT NULL,
request_path VARCHAR(200) NOT NULL,
request_model VARCHAR(100),
request_ip VARCHAR(50) NOT NULL DEFAULT '',
content_filtered BOOLEAN NOT NULL DEFAULT false,
filter_rule_id UUID,
injection_applied BOOLEAN NOT NULL DEFAULT false,
response_status INTEGER NOT NULL DEFAULT 200,
duration_ms INTEGER NOT NULL DEFAULT 0,
created_at TIMESTAMP NOT NULL DEFAULT NOW()
);
COMMENT ON TABLE gateway_audit_logs IS 'LLM Gateway - 请求审计日志';
CREATE INDEX IF NOT EXISTS idx_gateway_audit_logs_key ON gateway_audit_logs (api_key_id);
CREATE INDEX IF NOT EXISTS idx_gateway_audit_logs_created ON gateway_audit_logs (created_at);
CREATE INDEX IF NOT EXISTS idx_gateway_audit_logs_filtered ON gateway_audit_logs (content_filtered) WHERE content_filtered = true;
-- ===========================================
-- 结束
-- ===========================================