- 新增图像生成接口,支持试用、积分和自定义API Key模式 - 实现生成图片结果异步上传至MinIO存储,带重试机制 - 优化积分预扣除和异常退还逻辑,保障用户积分准确 - 添加获取生成历史记录接口,支持时间范围和分页 - 提供本地字典配置接口,支持模型、比例、提示模板和尺寸 - 实现图片批量上传接口,支持S3兼容对象存储 feat(admin): 增加管理员角色管理与权限分配接口 - 实现角色列表查询、角色创建、更新及删除功能 - 增加权限列表查询接口 - 实现用户角色分配接口,便于统一管理用户权限 - 增加系统字典增删查改接口,支持分类过滤和排序 - 权限控制全面覆盖管理接口,保证安全访问 feat(auth): 完善用户登录注册及权限相关接口与页面 - 实现手机号验证码发送及校验功能,保障注册安全 - 支持手机号注册、登录及退出接口,集成日志记录 - 增加修改密码功能,验证原密码后更新 - 提供动态导航菜单接口,基于权限展示不同菜单 - 实现管理界面路由及日志、角色、字典管理页面访问权限控制 - 添加系统日志查询接口,支持关键词和等级筛选 feat(app): 初始化Flask应用并配置蓝图与数据库 - 创建应用程序工厂,加载配置,初始化数据库和Redis客户端 - 注册认证、API及管理员蓝图,整合路由 - 根路由渲染主页模板 - 应用上下文中自动创建数据库表,保证运行环境准备完毕 feat(database): 提供数据库创建与迁移支持脚本 - 新增数据库创建脚本,支持自动检测是否已存在 - 添加数据库表初始化脚本,支持创建和删除所有表 - 实现RBAC权限初始化,包含基础权限和角色创建 - 新增字段手动修复脚本,添加用户API Key和积分字段 - 强制迁移脚本支持清理连接和修复表结构,初始化默认数据及角色分配 feat(config): 新增系统配置参数 - 配置数据库、Redis、Session和MinIO相关参数 - 添加AI接口地址及试用Key配置 - 集成阿里云短信服务配置及开发模式相关参数 feat(extensions): 初始化数据库、Redis和MinIO客户端 - 创建全局SQLAlchemy数据库实例和Redis客户端 - 配置基于boto3的MinIO兼容S3客户端 chore(logs): 添加示例系统日志文件 - 记录用户请求、验证码发送成功与失败的日志信息
393 lines
13 KiB
Python
393 lines
13 KiB
Python
import json
|
|
from enum import Enum
|
|
from typing import Any, Awaitable, Dict, List, Optional, Union
|
|
|
|
from redis.client import NEVER_DECODE
|
|
from redis.commands.helpers import get_protocol_version
|
|
from redis.exceptions import DataError
|
|
from redis.typing import CommandsProtocol, EncodableT, KeyT, Number
|
|
|
|
VADD_CMD = "VADD"
|
|
VSIM_CMD = "VSIM"
|
|
VREM_CMD = "VREM"
|
|
VDIM_CMD = "VDIM"
|
|
VCARD_CMD = "VCARD"
|
|
VEMB_CMD = "VEMB"
|
|
VLINKS_CMD = "VLINKS"
|
|
VINFO_CMD = "VINFO"
|
|
VSETATTR_CMD = "VSETATTR"
|
|
VGETATTR_CMD = "VGETATTR"
|
|
VRANDMEMBER_CMD = "VRANDMEMBER"
|
|
|
|
# Return type for vsim command
|
|
VSimResult = Optional[
|
|
List[
|
|
Union[
|
|
List[EncodableT], Dict[EncodableT, Number], Dict[EncodableT, Dict[str, Any]]
|
|
]
|
|
]
|
|
]
|
|
|
|
|
|
class QuantizationOptions(Enum):
|
|
"""Quantization options for the VADD command."""
|
|
|
|
NOQUANT = "NOQUANT"
|
|
BIN = "BIN"
|
|
Q8 = "Q8"
|
|
|
|
|
|
class CallbacksOptions(Enum):
|
|
"""Options that can be set for the commands callbacks"""
|
|
|
|
RAW = "RAW"
|
|
WITHSCORES = "WITHSCORES"
|
|
WITHATTRIBS = "WITHATTRIBS"
|
|
ALLOW_DECODING = "ALLOW_DECODING"
|
|
RESP3 = "RESP3"
|
|
|
|
|
|
class VectorSetCommands(CommandsProtocol):
|
|
"""Redis VectorSet commands"""
|
|
|
|
def vadd(
|
|
self,
|
|
key: KeyT,
|
|
vector: Union[List[float], bytes],
|
|
element: str,
|
|
reduce_dim: Optional[int] = None,
|
|
cas: Optional[bool] = False,
|
|
quantization: Optional[QuantizationOptions] = None,
|
|
ef: Optional[Number] = None,
|
|
attributes: Optional[Union[dict, str]] = None,
|
|
numlinks: Optional[int] = None,
|
|
) -> Union[Awaitable[int], int]:
|
|
"""
|
|
Add vector ``vector`` for element ``element`` to a vector set ``key``.
|
|
|
|
``reduce_dim`` sets the dimensions to reduce the vector to.
|
|
If not provided, the vector is not reduced.
|
|
|
|
``cas`` is a boolean flag that indicates whether to use CAS (check-and-set style)
|
|
when adding the vector. If not provided, CAS is not used.
|
|
|
|
``quantization`` sets the quantization type to use.
|
|
If not provided, int8 quantization is used.
|
|
The options are:
|
|
- NOQUANT: No quantization
|
|
- BIN: Binary quantization
|
|
- Q8: Signed 8-bit quantization
|
|
|
|
``ef`` sets the exploration factor to use.
|
|
If not provided, the default exploration factor is used.
|
|
|
|
``attributes`` is a dictionary or json string that contains the attributes to set for the vector.
|
|
If not provided, no attributes are set.
|
|
|
|
``numlinks`` sets the number of links to create for the vector.
|
|
If not provided, the default number of links is used.
|
|
|
|
For more information, see https://redis.io/commands/vadd.
|
|
"""
|
|
if not vector or not element:
|
|
raise DataError("Both vector and element must be provided")
|
|
|
|
pieces = []
|
|
if reduce_dim:
|
|
pieces.extend(["REDUCE", reduce_dim])
|
|
|
|
values_pieces = []
|
|
if isinstance(vector, bytes):
|
|
values_pieces.extend(["FP32", vector])
|
|
else:
|
|
values_pieces.extend(["VALUES", len(vector)])
|
|
values_pieces.extend(vector)
|
|
pieces.extend(values_pieces)
|
|
|
|
pieces.append(element)
|
|
|
|
if cas:
|
|
pieces.append("CAS")
|
|
|
|
if quantization:
|
|
pieces.append(quantization.value)
|
|
|
|
if ef:
|
|
pieces.extend(["EF", ef])
|
|
|
|
if attributes:
|
|
if isinstance(attributes, dict):
|
|
# transform attributes to json string
|
|
attributes_json = json.dumps(attributes)
|
|
else:
|
|
attributes_json = attributes
|
|
pieces.extend(["SETATTR", attributes_json])
|
|
|
|
if numlinks:
|
|
pieces.extend(["M", numlinks])
|
|
|
|
return self.execute_command(VADD_CMD, key, *pieces)
|
|
|
|
def vsim(
|
|
self,
|
|
key: KeyT,
|
|
input: Union[List[float], bytes, str],
|
|
with_scores: Optional[bool] = False,
|
|
with_attribs: Optional[bool] = False,
|
|
count: Optional[int] = None,
|
|
ef: Optional[Number] = None,
|
|
filter: Optional[str] = None,
|
|
filter_ef: Optional[str] = None,
|
|
truth: Optional[bool] = False,
|
|
no_thread: Optional[bool] = False,
|
|
epsilon: Optional[Number] = None,
|
|
) -> Union[Awaitable[VSimResult], VSimResult]:
|
|
"""
|
|
Compare a vector or element ``input`` with the other vectors in a vector set ``key``.
|
|
|
|
``with_scores`` sets if similarity scores should be returned for each element in the result.
|
|
|
|
``with_attribs`` ``with_attribs`` sets if the results should be returned with the
|
|
attributes of the elements in the result, or None when no attributes are present.
|
|
|
|
``count`` sets the number of results to return.
|
|
|
|
``ef`` sets the exploration factor.
|
|
|
|
``filter`` sets the filter that should be applied for the search.
|
|
|
|
``filter_ef`` sets the max filtering effort.
|
|
|
|
``truth`` when enabled, forces the command to perform a linear scan.
|
|
|
|
``no_thread`` when enabled forces the command to execute the search
|
|
on the data structure in the main thread.
|
|
|
|
``epsilon`` floating point between 0 and 1, if specified will return
|
|
only elements with distance no further than the specified one.
|
|
|
|
For more information, see https://redis.io/commands/vsim.
|
|
"""
|
|
|
|
if not input:
|
|
raise DataError("'input' should be provided")
|
|
|
|
pieces = []
|
|
options = {}
|
|
|
|
if isinstance(input, bytes):
|
|
pieces.extend(["FP32", input])
|
|
elif isinstance(input, list):
|
|
pieces.extend(["VALUES", len(input)])
|
|
pieces.extend(input)
|
|
else:
|
|
pieces.extend(["ELE", input])
|
|
|
|
if with_scores or with_attribs:
|
|
if get_protocol_version(self.client) in ["3", 3]:
|
|
options[CallbacksOptions.RESP3.value] = True
|
|
|
|
if with_scores:
|
|
pieces.append("WITHSCORES")
|
|
options[CallbacksOptions.WITHSCORES.value] = True
|
|
|
|
if with_attribs:
|
|
pieces.append("WITHATTRIBS")
|
|
options[CallbacksOptions.WITHATTRIBS.value] = True
|
|
|
|
if count:
|
|
pieces.extend(["COUNT", count])
|
|
|
|
if epsilon:
|
|
pieces.extend(["EPSILON", epsilon])
|
|
|
|
if ef:
|
|
pieces.extend(["EF", ef])
|
|
|
|
if filter:
|
|
pieces.extend(["FILTER", filter])
|
|
|
|
if filter_ef:
|
|
pieces.extend(["FILTER-EF", filter_ef])
|
|
|
|
if truth:
|
|
pieces.append("TRUTH")
|
|
|
|
if no_thread:
|
|
pieces.append("NOTHREAD")
|
|
|
|
return self.execute_command(VSIM_CMD, key, *pieces, **options)
|
|
|
|
def vdim(self, key: KeyT) -> Union[Awaitable[int], int]:
|
|
"""
|
|
Get the dimension of a vector set.
|
|
|
|
In the case of vectors that were populated using the `REDUCE`
|
|
option, for random projection, the vector set will report the size of
|
|
the projected (reduced) dimension.
|
|
|
|
Raises `redis.exceptions.ResponseError` if the vector set doesn't exist.
|
|
|
|
For more information, see https://redis.io/commands/vdim.
|
|
"""
|
|
return self.execute_command(VDIM_CMD, key)
|
|
|
|
def vcard(self, key: KeyT) -> Union[Awaitable[int], int]:
|
|
"""
|
|
Get the cardinality(the number of elements) of a vector set with key ``key``.
|
|
|
|
Raises `redis.exceptions.ResponseError` if the vector set doesn't exist.
|
|
|
|
For more information, see https://redis.io/commands/vcard.
|
|
"""
|
|
return self.execute_command(VCARD_CMD, key)
|
|
|
|
def vrem(self, key: KeyT, element: str) -> Union[Awaitable[int], int]:
|
|
"""
|
|
Remove an element from a vector set.
|
|
|
|
For more information, see https://redis.io/commands/vrem.
|
|
"""
|
|
return self.execute_command(VREM_CMD, key, element)
|
|
|
|
def vemb(
|
|
self, key: KeyT, element: str, raw: Optional[bool] = False
|
|
) -> Union[
|
|
Awaitable[Optional[Union[List[EncodableT], Dict[str, EncodableT]]]],
|
|
Optional[Union[List[EncodableT], Dict[str, EncodableT]]],
|
|
]:
|
|
"""
|
|
Get the approximated vector of an element ``element`` from vector set ``key``.
|
|
|
|
``raw`` is a boolean flag that indicates whether to return the
|
|
internal representation used by the vector.
|
|
|
|
|
|
For more information, see https://redis.io/commands/vemb.
|
|
"""
|
|
options = {}
|
|
pieces = []
|
|
pieces.extend([key, element])
|
|
|
|
if get_protocol_version(self.client) in ["3", 3]:
|
|
options[CallbacksOptions.RESP3.value] = True
|
|
|
|
if raw:
|
|
pieces.append("RAW")
|
|
|
|
options[NEVER_DECODE] = True
|
|
if (
|
|
hasattr(self.client, "connection_pool")
|
|
and self.client.connection_pool.connection_kwargs["decode_responses"]
|
|
) or (
|
|
hasattr(self.client, "nodes_manager")
|
|
and self.client.nodes_manager.connection_kwargs["decode_responses"]
|
|
):
|
|
# allow decoding in the postprocessing callback
|
|
# if the user set decode_responses=True
|
|
# in the connection pool
|
|
options[CallbacksOptions.ALLOW_DECODING.value] = True
|
|
|
|
options[CallbacksOptions.RAW.value] = True
|
|
|
|
return self.execute_command(VEMB_CMD, *pieces, **options)
|
|
|
|
def vlinks(
|
|
self, key: KeyT, element: str, with_scores: Optional[bool] = False
|
|
) -> Union[
|
|
Awaitable[
|
|
Optional[
|
|
List[Union[List[Union[str, bytes]], Dict[Union[str, bytes], Number]]]
|
|
]
|
|
],
|
|
Optional[List[Union[List[Union[str, bytes]], Dict[Union[str, bytes], Number]]]],
|
|
]:
|
|
"""
|
|
Returns the neighbors for each level the element ``element`` exists in the vector set ``key``.
|
|
|
|
The result is a list of lists, where each list contains the neighbors for one level.
|
|
If the element does not exist, or if the vector set does not exist, None is returned.
|
|
|
|
If the ``WITHSCORES`` option is provided, the result is a list of dicts,
|
|
where each dict contains the neighbors for one level, with the scores as values.
|
|
|
|
For more information, see https://redis.io/commands/vlinks
|
|
"""
|
|
options = {}
|
|
pieces = []
|
|
pieces.extend([key, element])
|
|
|
|
if with_scores:
|
|
pieces.append("WITHSCORES")
|
|
options[CallbacksOptions.WITHSCORES.value] = True
|
|
|
|
return self.execute_command(VLINKS_CMD, *pieces, **options)
|
|
|
|
def vinfo(self, key: KeyT) -> Union[Awaitable[dict], dict]:
|
|
"""
|
|
Get information about a vector set.
|
|
|
|
For more information, see https://redis.io/commands/vinfo.
|
|
"""
|
|
return self.execute_command(VINFO_CMD, key)
|
|
|
|
def vsetattr(
|
|
self, key: KeyT, element: str, attributes: Optional[Union[dict, str]] = None
|
|
) -> Union[Awaitable[int], int]:
|
|
"""
|
|
Associate or remove JSON attributes ``attributes`` of element ``element``
|
|
for vector set ``key``.
|
|
|
|
For more information, see https://redis.io/commands/vsetattr
|
|
"""
|
|
if attributes is None:
|
|
attributes_json = "{}"
|
|
elif isinstance(attributes, dict):
|
|
# transform attributes to json string
|
|
attributes_json = json.dumps(attributes)
|
|
else:
|
|
attributes_json = attributes
|
|
|
|
return self.execute_command(VSETATTR_CMD, key, element, attributes_json)
|
|
|
|
def vgetattr(
|
|
self, key: KeyT, element: str
|
|
) -> Union[Optional[Awaitable[dict]], Optional[dict]]:
|
|
"""
|
|
Retrieve the JSON attributes of an element ``element `` for vector set ``key``.
|
|
|
|
If the element does not exist, or if the vector set does not exist, None is
|
|
returned.
|
|
|
|
For more information, see https://redis.io/commands/vgetattr.
|
|
"""
|
|
return self.execute_command(VGETATTR_CMD, key, element)
|
|
|
|
def vrandmember(
|
|
self, key: KeyT, count: Optional[int] = None
|
|
) -> Union[
|
|
Awaitable[Optional[Union[List[str], str]]], Optional[Union[List[str], str]]
|
|
]:
|
|
"""
|
|
Returns random elements from a vector set ``key``.
|
|
|
|
``count`` is the number of elements to return.
|
|
If ``count`` is not provided, a single element is returned as a single string.
|
|
If ``count`` is positive(smaller than the number of elements
|
|
in the vector set), the command returns a list with up to ``count``
|
|
distinct elements from the vector set
|
|
If ``count`` is negative, the command returns a list with ``count`` random elements,
|
|
potentially with duplicates.
|
|
If ``count`` is greater than the number of elements in the vector set,
|
|
only the entire set is returned as a list.
|
|
|
|
If the vector set does not exist, ``None`` is returned.
|
|
|
|
For more information, see https://redis.io/commands/vrandmember.
|
|
"""
|
|
pieces = []
|
|
pieces.append(key)
|
|
if count is not None:
|
|
pieces.append(count)
|
|
return self.execute_command(VRANDMEMBER_CMD, *pieces)
|