AIAB核心方法

1.通用聊天逻辑方法processChatRequest()

描述

processChatRequest()方法用于接收到用户请求之后对请求进行第一步处理，对请求进行验证和相关处理

参数

下表为processChatRequest()方法的参数

参数名	类型	示例
request	ChatCompletionRequest	请求体
sdkKey	String	1000563_ebf080bb426e7555
flagKey	String	buy
userId	String	Admin123

示例

此示例演示了以下每个概念的用法：

1. 当为非流式请求时，直接返回预设回答

2. 当为流式请求时，调用分桶与转发方法获取真实响应返回

private ResponseEntity<?> processChatRequest(ChatCompletionRequest request,
                                                 String flagKey,
                                                 String sdkKey,
                                                 String userId) {                                
    ......

    // 获取stream是否为流式（和AB测试平台对接都为true）
    boolean stream = request.getStream() != null ? request.getStream() : true;

    ......

    if (stream) {
        try {
            ResponseEntity<String> response = proxyService.chatCompletion(request, flagKey, sdkKey, userId);
            String bodyData = response.getBody();

            return ResponseEntity.ok()
                    .contentType(MediaType.TEXT_EVENT_STREAM)
                    .header(HttpHeaders.CACHE_CONTROL, "no-cache")
                    .body(bodyData);
        }catch (Exception e) {
            return ResponseEntity.badRequest().body(Collections.singletonMap("error", e.getMessage()));
        }
    }

    ChatResponse response = new ChatResponse(
            "chatcmpl-" + System.currentTimeMillis(),
            "chat.completion",
            System.currentTimeMillis() / 1000,
            "custom-model",
            Arrays.asList(new Choice(0, new ChatMessage("assistant", fullResponse), "stop")),
            new Usage(promptTokens, completionTokens, promptTokens + completionTokens)
    );

    return ResponseEntity.ok(response);

}

2.处理真实大模型真实调用chatCompletion()

描述

chatCompletion()用于对大模型进行实际分桶、实际大模型API调用以及发送请求事件

参数

下表为chatCompletion()方法的各项参数

参数名	类型	示例
request	ChatCompletionRequest	请求体
sdkKey	String	1000563_ebf080bb426e7555
flagKey	String	buy
userId	String	Admin123

示例

此示例演示了以下每个概念的用法：

1. 根据参数对用户进行分桶，获取抽取到的大模型

2. 从配置中获取对应大模型的配置信息

3. 真实调用大模型接口

public ResponseEntity<String> chatCompletion(ChatCompletionRequest request,
                                                 String flagKey,
                                                 String sdkKey,
                                                 String userId) {

    // 分桶逻辑,获取抽取到的模型
    Map<String, String> variableMap = proxyDecisionService.userDecide(flagKey, sdkKey, userId);
    String name = variableMap.get("model");

    .....

    // 获取大模型配置信息
    String apiKey = (String) configStore.get(name).get("apiKey");
    String providerUrl = (String) configStore.get(name).get("url");

    //构造真实请求体
    HttpHeaders headers = new HttpHeaders();
    headers.setContentType(MediaType.APPLICATION_JSON);
    headers.set("Authorization", "Bearer " + apiKey);

    // 重置model值为分桶结果值
    ChatCompletionRequest modifiedRequest = new ChatCompletionRequest();
    org.springframework.beans.BeanUtils.copyProperties(request,modifiedRequest);
    modifiedRequest.setModel((String) configStore.get(name).get("model"));
    modifiedRequest.setMaxTokens((int) Double.parseDouble(variableMap.get("max_token")));
    modifiedRequest.setTemperature(Double.parseDouble(variableMap.get("temperature")));
    modifiedRequest.setTopP(Double.parseDouble(variableMap.get("top_p")));
    modifiedRequest.setPresencePenalty(Double.parseDouble(variableMap.get("presence_penalty")));
    modifiedRequest.setFrequencyPenalty(Double.parseDouble(variableMap.get("frequency_penalty")));
    if(modifiedRequest.getStream()) {
        ChatCompletionRequest.StreamOptions streamOptions = new ChatCompletionRequest.StreamOptions(true);
        modifiedRequest.setStreamOptions(streamOptions);
    }

    HttpEntity<ChatCompletionRequest> entity = new HttpEntity<>(modifiedRequest, headers);

    ......

    //发起真实大模型请求
    try {
        ResponseEntity<String> response = restTemplate.exchange(
                    providerUrl + "/chat/completions",
                    HttpMethod.POST,
                    entity,
                    String.class
            );

        //发送请求事件
        ......
    }catch(Exception e) {
        ......
    }
}