HiClaw单机嵌入式部署下如何从外部访问某Worker的MinIO文件

前置说明

HiClaw是一个定位于治理企业级多Agent协同的平台。
本文想在HiClaw单机embed模式部署时从外部能直接访问某worker的minio文件，方便对其进行二次开发。
以下假设worker的名字叫alice，以及hiclaw部署过程中的内部地址及端口号都保持默认。

1. 在部署机上确认 MinIO 中确实有 `alice` 的文件

1
2
3

docker exec hiclaw-controller sh -lc \
  "mc alias set local http://127.0.0.1:9000 admin your_passwd >/dev/null 2>&1 && \
   mc ls local/hiclaw-storage --recursive | grep '^.*agents/alice/'"

2. 在部署机上确认 Higress 到 MinIO 的文件系统路由已生效

1 2	docker exec hiclaw-controller sh -lc \ "curl -i -H 'Host: fs-local.hiclaw.io' http://127.0.0.1:8080/minio/health/live"

3. 查询 Higress 里是否存在文件系统 domain 和 route

docker exec hiclaw-controller sh -lc '
COOKIE=/tmp/higress.cookie
curl -s -c "$COOKIE" -H "Content-Type: application/json" \
  -d "{\"username\":\"admin\",\"password\":\"your_passwd\"}" \
  http://127.0.0.1:8001/session/login >/dev/null
echo "== domains =="
curl -s -b "$COOKIE" http://127.0.0.1:8001/v1/domains
echo
echo "== routes =="
curl -s -b "$COOKIE" http://127.0.0.1:8001/v1/routes
'

4. 如果缺失，手工补 MinIO 的 service source

docker exec hiclaw-controller sh -lc '
COOKIE=/tmp/higress.cookie
curl -s -c "$COOKIE" -H "Content-Type: application/json" \
  -d "{\"username\":\"admin\",\"password\":\"your_passwd\"}" \
  http://127.0.0.1:8001/session/login >/dev/null

curl -s -b "$COOKIE" -H "Content-Type: application/json" \
  -X POST http://127.0.0.1:8001/v1/service-sources \
  -d "{\"name\":\"minio\",\"type\":\"static\",\"domain\":\"127.0.0.1:9000\",\"port\":9000,\"properties\":{},\"authN\":{\"enabled\":false}}"
'

5. 如果缺失，手工补文件系统 domain

docker exec hiclaw-controller sh -lc '
COOKIE=/tmp/higress.cookie
curl -s -c "$COOKIE" -H "Content-Type: application/json" \
  -d "{\"username\":\"admin\",\"password\":\"your_passwd\"}" \
  http://127.0.0.1:8001/session/login >/dev/null

curl -s -b "$COOKIE" -H "Content-Type: application/json" \
  -X POST http://127.0.0.1:8001/v1/domains \
  -d "{\"name\":\"fs-local.hiclaw.io\",\"enableHttps\":\"off\"}"
'

6. 如果缺失，手工补文件系统 route

docker exec hiclaw-controller sh -lc '
COOKIE=/tmp/higress.cookie
curl -s -c "$COOKIE" -H "Content-Type: application/json" \
  -d "{\"username\":\"admin\",\"password\":\"your_passwd\"}" \
  http://127.0.0.1:8001/session/login >/dev/null

curl -s -b "$COOKIE" -H "Content-Type: application/json" \
  -X POST http://127.0.0.1:8001/v1/routes \
  -d "{\"name\":\"http-filesystem\",\"domains\":[\"fs-local.hiclaw.io\"],\"path\":{\"matchType\":\"PRE\",\"matchValue\":\"/\"},\"services\":[{\"name\":\"minio.static\",\"port\":9000,\"weight\":100}]}"
'

7. 在外部服务器上验证文件系统入口是否可达

1 2	curl --resolve fs-local.hiclaw.io:18080:your_server_ip \ -i http://fs-local.hiclaw.io:18080/minio/health/live

如果机器上配置了代理，优先使用不走代理的版本：

1 2	curl --noproxy '*' --resolve fs-local.hiclaw.io:18080:your_server_ip \ -i http://fs-local.hiclaw.io:18080/minio/health/live

8. 在外部服务器上查看是不是被本地代理劫持

1 2	curl -v --resolve fs-local.hiclaw.io:18080:your_server_ip \ http://fs-local.hiclaw.io:18080/minio/health/live

如果输出里出现类似下面的内容，说明请求先被发给了本地代理：

1 2	Uses proxy env variable http_proxy == 'http://127.0.0.1:7897' Trying 127.0.0.1:7897...

9. 在外部服务器上确认 18080 端口本身是通的

1	nc -vz your_server_ip 18080

10. 直接运行 Python 脚本列出 `alice` 可访问的文件

部署机上可直接运行，脚本会优先自动从 hiclaw-worker-alice 容器环境变量中读取 HICLAW_FS_SECRET_KEY：

1	python3 test_remote_boto.py

这个py文件的内容是：

import os
import socket
import subprocess

import boto3
from botocore.client import Config

PUBLIC_IP = "your_server_ip"
FS_HOST = "fs-local.hiclaw.io"
ENDPOINT_URL = f"http://{FS_HOST}:18080"
BUCKET = "hiclaw-storage"
WORKER_NAME = os.environ.get("WORKER_NAME", "alice")
WORKER_SECRET_KEY = os.environ.get("WORKER_SECRET_KEY") or os.environ.get("HICLAW_FS_SECRET_KEY")
PROXY_KEYS = (
    "http_proxy",
    "https_proxy",
    "HTTP_PROXY",
    "HTTPS_PROXY",
    "all_proxy",
    "ALL_PROXY",
)


def disable_proxies():
    removed = []
    for key in PROXY_KEYS:
        value = os.environ.pop(key, None)
        if value:
            removed.append(f"{key}={value}")

    no_proxy_value = f"{PUBLIC_IP},{FS_HOST}"
    os.environ["NO_PROXY"] = no_proxy_value
    os.environ["no_proxy"] = no_proxy_value

    if removed:
        print("Disabled proxies for this script:")
        for item in removed:
            print(" ", item)
    print("NO_PROXY set to:", no_proxy_value)


def load_secret_from_env():
    if WORKER_SECRET_KEY:
        print("Using worker credentials:")
        print("  access key:", WORKER_NAME)
        print("  secret key source: WORKER_SECRET_KEY/HICLAW_FS_SECRET_KEY")
        return WORKER_SECRET_KEY
    return None


def load_secret_from_docker():
    container_name = f"hiclaw-worker-{WORKER_NAME}"
    try:
        result = subprocess.run(
            ["docker", "exec", container_name, "env"],
            capture_output=True,
            text=True,
            check=True,
        )
    except FileNotFoundError:
        return None
    except subprocess.CalledProcessError:
        return None

    for line in result.stdout.splitlines():
        if line.startswith("HICLAW_FS_SECRET_KEY="):
            secret = line.split("=", 1)[1].strip()
            if secret:
                print("Using worker credentials:")
                print("  access key:", WORKER_NAME)
                print(f"  secret key source: docker exec {container_name} env")
                return secret
    return None


def get_worker_credentials():
    secret_key = load_secret_from_env()
    if not secret_key:
        secret_key = load_secret_from_docker()
    if not secret_key:
        raise RuntimeError(
            "Missing worker secret key. Set WORKER_SECRET_KEY or HICLAW_FS_SECRET_KEY, "
            f"or run this script on the HiClaw deployment server so it can read "
            f"`hiclaw-worker-{WORKER_NAME}` via docker exec."
        )
    return WORKER_NAME, secret_key


def create_client():
    original_getaddrinfo = socket.getaddrinfo
    access_key, secret_key = get_worker_credentials()

    # Emulate `curl --resolve fs-local.hiclaw.io:18080:your_server_ip`
    def patched_getaddrinfo(host, port, family=0, type=0, proto=0, flags=0):
        if host == FS_HOST:
            host = PUBLIC_IP
        return original_getaddrinfo(host, port, family, type, proto, flags)

    socket.getaddrinfo = patched_getaddrinfo

    client = boto3.client(
        "s3",
        endpoint_url=ENDPOINT_URL,
        aws_access_key_id=access_key,
        aws_secret_access_key=secret_key,
        config=Config(
            signature_version="s3v4",
            s3={"addressing_style": "path"},
        ),
        region_name="us-east-1",
    )

    return client


def dump_listing(client, label, prefix=None, max_keys=20):
    print(f"\n=== {label} ===")
    kwargs = {"Bucket": BUCKET, "MaxKeys": max_keys}
    if prefix is not None:
        kwargs["Prefix"] = prefix

    response = client.list_objects_v2(**kwargs)
    print("Name:", response.get("Name"))
    print("Prefix:", repr(response.get("Prefix")))
    print("KeyCount:", response.get("KeyCount"))
    print("IsTruncated:", response.get("IsTruncated"))

    contents = response.get("Contents", [])
    if not contents:
        print("No objects returned")
        return

    for obj in contents:
        print(obj["Key"])


def main():
    disable_proxies()
    client = create_client()
    dump_listing(client, f"agents/{WORKER_NAME}/ 前缀", prefix=f"agents/{WORKER_NAME}/")
    dump_listing(client, "shared/ 前缀", prefix="shared/")


if __name__ == "__main__":
    try:
        main()
    except Exception as exc:
        print("Error:", type(exc).__name__, exc)

其中一个核心点是得到worker的minio的key，即本质上是如下命令：

1	docker exec hiclaw-worker-<worker名称> env \| grep HICLAW_FS_SECRET_KEY

如果是在另一台服务器上运行，建议显式传入 worker key：

1
2
3

export WORKER_NAME=alice
export WORKER_SECRET_KEY='your-alice-secret-key'
python3 test_remote_boto.py